From jcw@equi4.com Thu Mar 29 04:12:18 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA04181 for ; Thu, 29 Mar 2001 04:12:18 -0600 From: Jean-Claude Wippler To: metakit Date: Thu, 29 Mar 2001 02:07:58 -0800 Message-Id: <20010329100758.23554@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Welcome! Sender: metakit-admin@www.equi4.com Errors-To: metakit-admin@www.equi4.com X-BeenThere: metakit@www.equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Welcome to the new mailing list. There are still some quirks in this setup but it ought to work. I will soon move over all users of the old metakit-core and metakit-scripting mailing lists - currently hosted by SourceForge. Once that works out, the metakit- announce list (which holds several hundred subscribers) will also be relocated and announced. Reasons for these changes were: - my SF setup has not been fixed after *5* months of pleading - the split between -core and -scripting turns out to be confusing - the new setup lets me to manage good archives of all messages Regards, Jean-Claude From jcw@equi4.com Thu Mar 29 13:03:27 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA23881 for ; Thu, 29 Mar 2001 13:03:24 -0600 From: Jean-Claude Wippler To: metakit Date: Thu, 29 Mar 2001 10:59:12 -0800 Message-Id: <20010329185912.6172@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] mailing lists moved Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The metakit-core@equi4.com and metakit-scripting@equi4.com mailing lists have been merged into one and moved away from SourceForge to the equi4.com server. You are receiving this email because you were subscribed to either of the above lists. This change has several unfortunate side-effects: - you need to re-enable daily digests again, if you want them - you need to alter your password if you don't want the new one - all mailing archives are lost (but I'll try to restore them) As before, all list administration can be done through the web, because the new list is again based on Mailman. Just go to: http://www.equi4.com/mailman/listinfo/metakit See also the new "announce" mailing list, which replaces the previous "metakit-announce" list. It too is now located on the equi4.com server: http://www.equi4.com/mailman/listinfo/announce Please let me know if the relocated list does not work for you. Allow me to add that because of this move, we will at last have properly working archives for all new messages, and that problems with mail not being sent to everyone should now be fully resolved: these were the two main reasons for this change. Regards, Jean-Claude From chris@pinebush.com Thu Mar 29 16:40:48 2001 Received: from pinebush.com (host-64-65-203-122.choiceone.net [64.65.203.122]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA31350 for ; Thu, 29 Mar 2001 16:40:48 -0600 Received: from pinebush.com (winntsrv1.pinebush.com [10.0.0.17]) by pinebush.com (8.9.3/8.9.3) with ESMTP id RAA14855 for ; Thu, 29 Mar 2001 17:36:27 -0500 (EST) Message-ID: <3AC3B8EB.54620796@pinebush.com> Date: Thu, 29 Mar 2001 17:36:27 -0500 From: Chris Nelson Organization: Pinebush Technologies, Inc. (http://www.pinebush.com) X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Problem with open files on 'Windows Me Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I realize this is vague and wide open but it's all I've got right now and I'm hoping it rings a bell with someone. We have a Tcl-based 'Windows application that uses MetaKit v2.0. On NT, 95, and 98, all is well but on Me, the application appears to leave it's MetaKit file open such that after the application is exited, the file can't be deleted because the system thinks it's in use. Same binary, same code paths, etc. The only variable seems to be the OS. Any insight would be appreciated. Chris -- Rens-se-LEER is a county. RENS-se-ler is a city. R-P-I is a school! From jcw@equi4.com Thu Mar 29 18:49:43 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA02278; Thu, 29 Mar 2001 18:49:37 -0600 From: Jean-Claude Wippler To: , Date: Thu, 29 Mar 2001 16:45:25 -0800 Message-Id: <20010330004525.17056@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.3.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There's a new 2.3.4 release of MetaKit. It consolidates and fixes a few more issues since 2.3.3 was released many months ago. This release now includes the code needed to build TclKit, an installation-free runtime combining MetaKit with Tcl and Tk. The new code passes all tests on Win32, Linux, and Solaris (except for one difference in test result "s33c", which can be ignored for now). The next release will be called "final" (really!), as soon as some more platform builds have been verified. As before, a detailed change log is online at: http://www.equi4.com/metakit/CHANGES The source download (including a WHATSNEW summary) is at: http://www.equi4.com/pub/mk/metakit-2.3.4-29.tar.gz Binary releases will be added to the web site shortly. Two other changes need to be mentioned here: - As you can see from this email, MK's mailing lists have changed. Please post to metakit@equi4.com from now on. - The CVS repository has been relocated to my "equi4.com" server. For details, see http://www.equi4.com/metakit/ Regards, Jean-Claude From akao347@hotmail.com Thu Mar 29 20:33:25 2001 Received: from hotmail.com (oe44.law4.hotmail.com [216.33.148.81]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA03842 for ; Thu, 29 Mar 2001 20:33:24 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 29 Mar 2001 18:28:33 -0800 X-Originating-IP: [18.245.5.30] From: "A K" To: References: <20010314062842.C2EE124889@mail.digital-smarties.com> <20010327075252.3689@smtp.telus.net> Date: Thu, 29 Mar 2001 21:26:28 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 30 Mar 2001 02:28:33.0805 (UTC) FILETIME=[1E0273D0:01C0B8C1] Subject: [Metakit] Re: [MK-Core] Demo crashes alternatively Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, This problem still occurs in the 2.3.4 release. It only happen on debug builds. It doesn't matter if you try to use STD or MFC, they all have the same behavior. It is something related to committing after retrieving views differently from how it is already stored. I tried to reconstruct the exact situation to cause the crash, but I was unable to. There seems to be some sort of bad karma around mkdemo. Kao. ----- Original Message ----- From: "Jean-Claude Wippler" To: Sent: Tuesday, March 27, 2001 2:52 AM Subject: Re: [MK-Core] Demo crashes alternatively > Geoffrey Zhu wrote: > > >I just downloaded the metakit database. It is really a great product. But > >here is a problem: the mkdemo included in the sample project will crash the > >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, > >especially when I am tracing the code line by line in debug mode. Everything > >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I run > >the compiled program outside the IDE. > > > >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows NT > >4.0 with SP6. > > > >Anybody has any clue? > > Could you please fetch the latest release from the new CVS repository on > equi4.com and try things there? The CVS commands (on Unix) are: > cvs -d:pserver:anonymous@equi4.com:/home/cvs login > > cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit > > I seem to recall something like this happening before, but AFAIK it has > been resolved. Or were you perhaps using MK 2.0? > > -jcw > > > _______________________________________________ > metakit-core@equi4.com - http://www.equi4.com/metakit/ > http://lists.sourceforge.net/lists/listinfo/metakit-core > From readonly@getsoft.com Thu Mar 29 20:46:29 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA04098 for ; Thu, 29 Mar 2001 20:46:28 -0600 Received: from NEVILLE550 (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id TAA15706; Thu, 29 Mar 2001 19:42:04 -0700 (MST) Message-ID: <200103301242030770.00CE00C2@getsoft.com> In-Reply-To: References: <20010314062842.C2EE124889@mail.digital-smarties.com> <20010327075252.3689@smtp.telus.net> X-Mailer: Calypso Version 3.10.03.02 (3) Date: Fri, 30 Mar 2001 12:42:03 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "A K" , metakit-core@equi4.com Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id UAA04098 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Kao, FYI I've had memory corruptions and other exceptions using 2.3.3 with VC++ / MFC. *********** REPLY SEPARATOR *********** On 29 Mar 2001 at 21:26 A K wrote: >Hi, > >This problem still occurs in the 2.3.4 release. It only happen on debug >builds. It doesn't matter if you try to use STD or MFC, they all have the >same behavior. It is something related to committing after retrieving views >differently from how it is already stored. > >I tried to reconstruct the exact situation to cause the crash, but I was >unable to. There seems to be some sort of bad karma around mkdemo. > >Kao. > > >----- Original Message ----- >From: "Jean-Claude Wippler" >To: >Sent: Tuesday, March 27, 2001 2:52 AM >Subject: Re: [MK-Core] Demo crashes alternatively > > >> Geoffrey Zhu wrote: >> >> >I just downloaded the metakit database. It is really a great product. But >> >here is a problem: the mkdemo included in the sample project will crash >the >> >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, >> >especially when I am tracing the code line by line in debug mode. >Everything >> >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I >run >> >the compiled program outside the IDE. >> > >> >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows >NT >> >4.0 with SP6. >> > >> >Anybody has any clue? >> >> Could you please fetch the latest release from the new CVS repository on >> equi4.com and try things there? The CVS commands (on Unix) are: >> cvs -d:pserver:anonymous@equi4.com:/home/cvs login >> >> cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit >> >> I seem to recall something like this happening before, but AFAIK it has >> been resolved. Or were you perhaps using MK 2.0? >> >> -jcw >> >> >> _______________________________________________ >> metakit-core@equi4.com - http://www.equi4.com/metakit/ >> http://lists.sourceforge.net/lists/listinfo/metakit-core >> >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find lots of information on ED and while you're there you can click on 'Stay Informed' to join our mailing list and 'Future Plans' to read about Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From gzhu@peak6.com Thu Mar 29 20:54:05 2001 Received: from p6chifs1.peak6.com (node-cffbbe42.powerinter.net [207.251.190.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id UAA04273 for ; Thu, 29 Mar 2001 20:54:05 -0600 Received: by P6CHIFS1 with Internet Mail Service (5.5.2653.19) id ; Thu, 29 Mar 2001 20:49:39 -0600 Message-ID: From: Geoffrey Zhu To: "'metakit-core@equi4.com'" Subject: RE: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 20:49:37 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From the fact that it crashes every other time, I figured that it may be the problem of the data file. I deleted the .DAT file that comes with the package and mkdemo seems to be fine. If I replace it with the old .DAT file, mkdemo will crash every other time. -----Original Message----- From: A K [mailto:akao347@hotmail.com] Sent: Thursday, March 29, 2001 8:26 PM To: metakit-core@equi4.com Subject: [Metakit] Re: [MK-Core] Demo crashes alternatively Hi, This problem still occurs in the 2.3.4 release. It only happen on debug builds. It doesn't matter if you try to use STD or MFC, they all have the same behavior. It is something related to committing after retrieving views differently from how it is already stored. I tried to reconstruct the exact situation to cause the crash, but I was unable to. There seems to be some sort of bad karma around mkdemo. Kao. ----- Original Message ----- From: "Jean-Claude Wippler" To: Sent: Tuesday, March 27, 2001 2:52 AM Subject: Re: [MK-Core] Demo crashes alternatively > Geoffrey Zhu wrote: > > >I just downloaded the metakit database. It is really a great product. But > >here is a problem: the mkdemo included in the sample project will crash the > >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, > >especially when I am tracing the code line by line in debug mode. Everything > >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I run > >the compiled program outside the IDE. > > > >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows NT > >4.0 with SP6. > > > >Anybody has any clue? > > Could you please fetch the latest release from the new CVS repository on > equi4.com and try things there? The CVS commands (on Unix) are: > cvs -d:pserver:anonymous@equi4.com:/home/cvs login > > cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit > > I seem to recall something like this happening before, but AFAIK it has > been resolved. Or were you perhaps using MK 2.0? > > -jcw > > > _______________________________________________ > metakit-core@equi4.com - http://www.equi4.com/metakit/ > http://lists.sourceforge.net/lists/listinfo/metakit-core > _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From akao347@hotmail.com Thu Mar 29 21:16:39 2001 Received: from hotmail.com (oe69.law4.hotmail.com [216.33.148.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA04558 for ; Thu, 29 Mar 2001 21:16:39 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 29 Mar 2001 19:11:45 -0800 X-Originating-IP: [18.245.5.30] From: "A K" To: References: Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 22:09:49 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 30 Mar 2001 03:11:45.0955 (UTC) FILETIME=[270D2730:01C0B8C7] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Yes, it is related to the .dat file. Let me try to explain a bit more. So the first time you run the program, it creates the database with the following structure: address[name:S,country:S] then it adds a third field, so the structure becomes: address[name:S,country:S,age:I] The second time you run the program, it tries to conver the structure from address[name:S,country:S,age:I] to address[name:S,country:S], which causes a crash on commit. It appears to succed in commit, but it still crashes. So the next time you run the program, the database has the structure address[name:S,country:S], which is exactly what the program initially wants. I don't seem to be very good at explaining. Brendan. ----- Original Message ----- From: "Geoffrey Zhu" To: Sent: Thursday, March 29, 2001 9:49 PM Subject: RE: [Metakit] Re: [MK-Core] Demo crashes alternatively > From the fact that it crashes every other time, I figured that it may be the > problem of the data file. I deleted the .DAT file that comes with the > package and mkdemo seems to be fine. If I replace it with the old .DAT file, > mkdemo will crash every other time. > > > > > > -----Original Message----- > From: A K [mailto:akao347@hotmail.com] > Sent: Thursday, March 29, 2001 8:26 PM > To: metakit-core@equi4.com > Subject: [Metakit] Re: [MK-Core] Demo crashes alternatively > > > Hi, > > This problem still occurs in the 2.3.4 release. It only happen on debug > builds. It doesn't matter if you try to use STD or MFC, they all have the > same behavior. It is something related to committing after retrieving views > differently from how it is already stored. > > I tried to reconstruct the exact situation to cause the crash, but I was > unable to. There seems to be some sort of bad karma around mkdemo. > > Kao. > > > ----- Original Message ----- > From: "Jean-Claude Wippler" > To: > Sent: Tuesday, March 27, 2001 2:52 AM > Subject: Re: [MK-Core] Demo crashes alternatively > > > > Geoffrey Zhu wrote: > > > > >I just downloaded the metakit database. It is really a great product. But > > >here is a problem: the mkdemo included in the sample project will crash > the > > >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, > > >especially when I am tracing the code line by line in debug mode. > Everything > > >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I > run > > >the compiled program outside the IDE. > > > > > >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows > NT > > >4.0 with SP6. > > > > > >Anybody has any clue? > > > > Could you please fetch the latest release from the new CVS repository on > > equi4.com and try things there? The CVS commands (on Unix) are: > > cvs -d:pserver:anonymous@equi4.com:/home/cvs login > > > > cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit > > > > I seem to recall something like this happening before, but AFAIK it has > > been resolved. Or were you perhaps using MK 2.0? > > > > -jcw > > > > > > _______________________________________________ > > metakit-core@equi4.com - http://www.equi4.com/metakit/ > > http://lists.sourceforge.net/lists/listinfo/metakit-core > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From akao347@hotmail.com Thu Mar 29 21:17:44 2001 Received: from hotmail.com (oe25.law4.hotmail.com [216.33.148.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA04581 for ; Thu, 29 Mar 2001 21:17:43 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 29 Mar 2001 19:12:52 -0800 X-Originating-IP: [18.245.5.30] From: "A K" To: References: <20010314062842.C2EE124889@mail.digital-smarties.com> <20010327075252.3689@smtp.telus.net> <200103301242030770.00CE00C2@getsoft.com> Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 22:10:54 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 30 Mar 2001 03:12:52.0784 (UTC) FILETIME=[4EE27300:01C0B8C7] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Can you give some examples? I'm not using the MFC, but I am worried about any possible memory corruptions / exceptions. Kao. ----- Original Message ----- From: "Neville Franks" To: "A K" ; Sent: Thursday, March 29, 2001 9:42 PM Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Kao, FYI I've had memory corruptions and other exceptions using 2.3.3 with VC++ / MFC. *********** REPLY SEPARATOR *********** On 29 Mar 2001 at 21:26 A K wrote: >Hi, > >This problem still occurs in the 2.3.4 release. It only happen on debug >builds. It doesn't matter if you try to use STD or MFC, they all have the >same behavior. It is something related to committing after retrieving views >differently from how it is already stored. > >I tried to reconstruct the exact situation to cause the crash, but I was >unable to. There seems to be some sort of bad karma around mkdemo. > >Kao. > > >----- Original Message ----- >From: "Jean-Claude Wippler" >To: >Sent: Tuesday, March 27, 2001 2:52 AM >Subject: Re: [MK-Core] Demo crashes alternatively > > >> Geoffrey Zhu wrote: >> >> >I just downloaded the metakit database. It is really a great product. But >> >here is a problem: the mkdemo included in the sample project will crash >the >> >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, >> >especially when I am tracing the code line by line in debug mode. >Everything >> >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I >run >> >the compiled program outside the IDE. >> > >> >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows >NT >> >4.0 with SP6. >> > >> >Anybody has any clue? >> >> Could you please fetch the latest release from the new CVS repository on >> equi4.com and try things there? The CVS commands (on Unix) are: >> cvs -d:pserver:anonymous@equi4.com:/home/cvs login >> >> cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit >> >> I seem to recall something like this happening before, but AFAIK it has >> been resolved. Or were you perhaps using MK 2.0? >> >> -jcw >> >> >> _______________________________________________ >> metakit-core@equi4.com - http://www.equi4.com/metakit/ >> http://lists.sourceforge.net/lists/listinfo/metakit-core >> >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find lots of information on ED and while you're there you can click on 'Stay Informed' to join our mailing list and 'Future Plans' to read about Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From jcw@equi4.com Thu Mar 29 21:55:06 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA05017 for ; Thu, 29 Mar 2001 21:55:01 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 19:50:06 -0800 Message-Id: <20010330035006.20918@smtp.telus.net> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A K wrote: >Can you give some examples? I'm not using the MFC, but I am worried about >any possible memory corruptions / exceptions. I've been able to reproduce the problem. It does not seem related to MFC. It crashes when run from MSVC6, but *not* when run from the command line. Every other time, as reported. Strange. I will investigate. Luckily it's a debug build, so that does make things easier! -jcw P.S. Please post to metakit@equi4.com instead of metakit-core@equi4.com / metakit-scripting@equi4.com from now on, those latter two mailing lists are about to be shut down (all subscriptions have been transferred). From akao347@hotmail.com Thu Mar 29 22:38:30 2001 Received: from hotmail.com (oe11.law4.hotmail.com [216.33.148.115]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA05492 for ; Thu, 29 Mar 2001 22:38:30 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 29 Mar 2001 20:33:38 -0800 X-Originating-IP: [18.245.5.30] From: "A K" To: References: <20010330035006.20918@smtp.telus.net> Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 23:31:41 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 30 Mar 2001 04:33:38.0871 (UTC) FILETIME=[9760A470:01C0B8D2] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Really? The program still crashes when I run from the commandline? Are you running mkdemo.exe or mkdemo_d.exe (using your build scripts). Kao. ----- Original Message ----- From: "Jean-Claude Wippler" To: Sent: Thursday, March 29, 2001 10:50 PM Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively > A K wrote: > > >Can you give some examples? I'm not using the MFC, but I am worried about > >any possible memory corruptions / exceptions. > > I've been able to reproduce the problem. It does not seem related to > MFC. It crashes when run from MSVC6, but *not* when run from the command > line. Every other time, as reported. > > Strange. I will investigate. Luckily it's a debug build, so that does > make things easier! > > -jcw > > P.S. Please post to metakit@equi4.com instead of metakit-core@equi4.com > / metakit-scripting@equi4.com from now on, those latter two mailing lists > are about to be shut down (all subscriptions have been transferred). > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From klisa@online.fr Fri Mar 30 01:18:57 2001 Received: from mark.labs.trema.com (mark.labs.trema.com [194.103.215.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA06804 for ; Fri, 30 Mar 2001 01:18:56 -0600 Received: from dellvladimir (pclabs189.labs.trema.com [194.103.215.189]) by mark.labs.trema.com (8.9.3/8.9.3) with SMTP id JAA10733 for ; Fri, 30 Mar 2001 09:14:08 +0200 (MET DST) Message-ID: <003a01c0b8e9$0f679030$bdd767c2@labs.trema.com> From: "Vladimir Klisic" To: References: <20010323163724.7AEAC25558@mail.digital-smarties.com> <20010327075023.4571@smtp.telus.net> Date: Fri, 30 Mar 2001 09:14:21 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Re: [MK-Core] Metakit performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Even if you do not know the number of rows in advance, pre-allocating > say chunks of 1000 rows whenever you run out, and cleaning up at > the end, can make all the difference. It makes all the difference in the world, indeed. Thanks. What I'm getting right now is almost constant row preparation and insertion time. Insertion time is now roughly between 1 and 2 seconds for 1000 rows in debug build. It will likely be much faster in release build, with or without Metakit being inlined (unfortunately, I cannot test it right now). > The difference between C++ and Python surprises me. My only > explanation is that one was compiled with q4_INLINE and > the other without. It could be true. Python test program uses Metakit DLL which was downloaded from equi4 web site, and I was building Metakit without q4_INLINE defined. > How many ints/floats/strings do you have *each* per row? It's hard to say. It depends on each particular stored procedure that is used as primary data source. On average, string columns should comprise 1/3 of all columns. > The speed of MK is directly proportional to the number of fields. Also, > strings are variable-sized and a lot slower to deal with than ints/floats. Some basic profiling showed that - 45% of the time was spent in RtlAllocateHeap (just another name for HeapAlloc on Win32). Called roughly 14 million times for 40000 rows. - 22% of the time was spent RtlFreeHeap. Called roughly 14 million times. - 5% of the time was spent in RtlReallocateHeap. Called 10 million times. Next five functions on this list are - c4_DoubleRef::operator= (4.5%). Called 2 million times. - c4_View::Add (3.3%). Called 40000 times :). - c4_IntRef::operator= (3%). Called 1.4 million times. - c4_StringRef::operator= (2%). Called 720000 times. - c4_Row::~c4_Row (2%). Same as c4_View, called 40000. Results above are obtained before I used row-preallocation code. I expected to see more direct way to move data from c4_Row to physical file on the disk (without that much memory management calls), but I guess it's the trade-off between simplicity and flexibility. And it still works quite fast if row-preallocation technique is used. Since there is significant number of heap allocations, it may be a good idea to use separate heaps for some or all Metakit types? Having only objects of the same size on the heap could help heap manager perform better (not to mention less memory fragmentation). > what performance are you after? and what type of operations > are dominant in your app? It goes like this: 1. Data is fetched from the stored procedure. Basically, it's a loop in which rows get stored one-by-one in Metakit file. No additional actions are taken in between, just inserts. Needs to be as fast as possible (or even faster). 2. Data is processed. Main loop goes through all the rows, creating several _new_ columns (existing columns are not modified). For each column, its value gets calculated out of values from other columns. (This is by far the most problematic part, and I did not even come to it because I was stuck with insertion performance.) What happens here is that a) Type of new column is not known in advance. It is determined once the value is calculated for the very first time. b) Once the very first row is processed, types of all additional columns are known and Metakit structure needs to be updated on the fly. Content of those additional columns is then inserted to the same Metakit file from step 1, repeating this for every row from then on. 3. Data is retrieved by some other application. Now the loop goes through all the rows, just reading the content. Needs to be as fast as possible. As you can see, looks like I'm using only core Metakit functionality. It's highly unlikely that I will ever need to add something more to it, so the question is: "what can I do to speed up everything from step 1 to step 3?" Thanks, Vladimir From jcw@equi4.com Fri Mar 30 01:49:30 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA07438 for ; Fri, 30 Mar 2001 01:49:28 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Re: [MK-Core] Metakit performance Date: Thu, 29 Mar 2001 23:45:20 -0800 Message-Id: <20010330074520.26913@smtp.telus.net> In-Reply-To: <003a01c0b8e9$0f679030$bdd767c2@labs.trema.com> References: <003a01c0b8e9$0f679030$bdd767c2@labs.trema.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Vladimir Klisic wrote: [...] >Some basic profiling showed that >- 45% of the time was spent in RtlAllocateHeap (just another name for >HeapAlloc on Win32). Called roughly 14 million times for 40000 rows. I.e. 35 times per row (1/3rd of the 100 props are strings, you say?). >- 22% of the time was spent RtlFreeHeap. Called roughly 14 million times. >- 5% of the time was spent in RtlReallocateHeap. Called 10 million times. Maybe it's possible to avoid some string allocations altogether. Can you determine whether most mallocs take places inside MK? Are you using c4_String? MK is relatively careful to not do too many allocs (c4_Bytes are used internally, which avoid mallocs for strings under 16 bytes). If we can determine where the mallocs are, that would help a lot. Can you interrupt a few random times and get stack traces? It's important to find out what *calls* the RtlAllocateHeap code. >question is: "what can I do to speed up everything from step 1 to step 3?" I think tracing the cause of the mallocs is a useful exercise. Only as last resort would I consider making the mallocs faster. Based on yout stats, even instant mallocs can "only" give you a 72% speed increase (i.e. less than an order of magnitude). -jcw From vladimir.klisic@trema.com Fri Mar 30 03:14:22 2001 Received: from mark.labs.trema.com (mark.labs.trema.com [194.103.215.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA09238 for ; Fri, 30 Mar 2001 03:14:21 -0600 Received: from dellvladimir (pclabs189.labs.trema.com [194.103.215.189]) by mark.labs.trema.com (8.9.3/8.9.3) with SMTP id LAA10914 for ; Fri, 30 Mar 2001 11:09:38 +0200 (MET DST) Message-ID: <00ba01c0b8f9$31966040$bdd767c2@labs.trema.com> From: "Vladimir Klisic" To: Subject: Re: [Metakit] Re: [MK-Core] Metakit performance Date: Fri, 30 Mar 2001 11:09:54 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00B5_01C0B909.F28D3AC0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_00B5_01C0B909.F28D3AC0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable > how do you do profiling on win32? Performance tuning feature in VC++ is availabe only in MSVC Enterprise editions. Assuming that you have one, step-by-step explanation about how = to do it can be found in MSDN Library. Just follow Visual C++ documentation = -> Using Visual C++ -> Visual C++ Programmer's Guide -> Performance Tuning. = You may also wish to check MS online Knowledge Base. There is also another way to do it: using NuMega TrueTime profiler, part = of excellent suite called NuMega DevPartner Studio (www.numega.com). = TrueTime integrates into MSVC development environment, making profiling as easy = as 'Build with TrueTime', 'Run with TrueTime'. Last time I was checking it, they were offering 15 or 30-day demo at their site (it was not recently, though). Highly recommended (just as their BoundsChecker, btw). In my case, due to some particularities of build process I wasn't able = to profile the application I'm working on directly from MSVC IDE. Instead, = I've extracted all Metakit-related code out of it and created small test application (with exactly the same code constructions) which was easier = to profile. In your case it should be much easier. ------=_NextPart_000_00B5_01C0B909.F28D3AC0 Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
> how do you do profiling on=20 win32?

Performance tuning feature in VC++ is availabe only in = MSVC=20 Enterprise
editions. Assuming that you have one, step-by-step = explanation=20 about how to
do it can be found in MSDN Library. Just follow Visual = C++=20 documentation ->
Using Visual C++ -> Visual C++ Programmer's = Guide=20 -> Performance Tuning. You
may also wish to check MS online = Knowledge=20 Base.

There is also another way to do it: using NuMega TrueTime = profiler,=20 part of
excellent suite called NuMega DevPartner Studio (www.numega.com). TrueTime
integrates into MSVC development environment, = making=20 profiling as easy as
'Build with TrueTime', 'Run with TrueTime'. Last = time I=20 was checking it,
they were offering 15 or 30-day demo at their site = (it was=20 not recently,
though). Highly recommended (just as their = BoundsChecker,=20 btw).

In my case, due to some particularities of build process I = wasn't=20 able to
profile the application I'm working on directly from MSVC = IDE.=20 Instead, I've
extracted all Metakit-related code out of it and = created small=20 test
application (with exactly the same code constructions) which was = easier=20 to
profile. In your case it should be much=20 easier.

------=_NextPart_000_00B5_01C0B909.F28D3AC0-- From readonly@getsoft.com Fri Mar 30 05:32:22 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA15137 for ; Fri, 30 Mar 2001 05:32:21 -0600 Received: from NEVILLE550 (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id EAA21412; Fri, 30 Mar 2001 04:27:51 -0700 (MST) Message-ID: <200103302127510650.00998EBC@getsoft.com> In-Reply-To: <00ba01c0b8f9$31966040$bdd767c2@labs.trema.com> References: <00ba01c0b8f9$31966040$bdd767c2@labs.trema.com> X-Mailer: Calypso Version 3.10.03.02 (3) Date: Fri, 30 Mar 2001 21:27:51 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "Vladimir Klisic" , metakit@equi4.com Subject: Re: [Metakit] Re: [MK-Core] Metakit performance Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=====_9859516716334=_" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=====_9859516716334=_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable An very good Windows Profiler can be found at http://www.glowcode.com/ *********** REPLY SEPARATOR *********** On 30 Mar 2001 at 11:09 Vladimir Klisic wrote: > how do you do profiling on win32? Performance tuning feature in VC++ is availabe only in MSVC Enterprise editions. Assuming that you have one, step-by-step explanation about how to do it can be found in MSDN Library. Just follow Visual C++ documentation -> Using Visual C++ -> Visual C++ Programmer's Guide -> Performance Tuning.= You may also wish to check MS online Knowledge Base. There is also another way to do it: using NuMega TrueTime profiler, part of excellent suite called NuMega DevPartner Studio (www.numega.com). TrueTime integrates into MSVC development environment, making profiling as easy as 'Build with TrueTime', 'Run with TrueTime'. Last time I was checking it, they were offering 15 or 30-day demo at their site (it was not recently, though). Highly recommended (just as their BoundsChecker, btw). In my case, due to some particularities of build process I wasn't able to profile the application I'm working on directly from MSVC IDE. Instead,= I've extracted all Metakit-related code out of it and created small test application (with exactly the same code constructions) which was easier to profile. In your case it should be much easier. Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find= lots of information on ED and while you're there you can click on 'Stay= Informed' to join our mailing list and 'Future Plans' to read about= Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- --=====_9859516716334=_ Content-Type: text/html; charset="us-ascii"
An very good Windows Profiler can be found at http://www.glowcode.com/
 

*********** REPLY SEPARATOR ***********

On 30 Mar 2001 at 11:09 Vladimir Klisic wrote:
> how do you do profiling on win32?

Performance tuning feature in VC++ is availabe only in MSVC Enterprise
editions. Assuming that you have one, step-by-step explanation about how to
do it can be found in MSDN Library. Just follow Visual C++ documentation ->
Using Visual C++ -> Visual C++ Programmer's Guide -> Performance Tuning. You
may also wish to check MS online Knowledge Base.

There is also another way to do it: using NuMega TrueTime profiler, part of
excellent suite called NuMega DevPartner Studio (www.numega.com). TrueTime
integrates into MSVC development environment, making profiling as easy as
'Build with TrueTime', 'Run with TrueTime'. Last time I was checking it,
they were offering 15 or 30-day demo at their site (it was not recently,
though). Highly recommended (just as their BoundsChecker, btw).

In my case, due to some particularities of build process I wasn't able to
profile the application I'm working on directly from MSVC IDE. Instead, I've
extracted all Metakit-related code out of it and created small test
application (with exactly the same code constructions) which was easier to
profile. In your case it should be much easier.

Best Regards,
    Neville Franks (Author of ED for Windows)

PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find=
 lots of information on ED and while you're there you can click on 'Stay=
 Informed' to join our mailing list and 'Future Plans' to read about=
 Version 4.

--------------------------------------------------------------
Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia
            Developers and publishers of ED for Windows
Ph  +61 3 9885 4445          WWW  http://www.getsoft.com
Fax +61 3 9885 4444         Email sales@getsoft.com
---------------------------------------------------------------
--=====_9859516716334=_-- From lvirden@cas.org Fri Mar 30 08:24:07 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA19201 for ; Fri, 30 Mar 2001 08:24:06 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id JAA16734 for ; Fri, 30 Mar 2001 09:19:11 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.9.3+Sun/m4_8.9.3/CAS_CLIENT-1.17) id JAA05529; Fri, 30 Mar 2001 09:19:09 -0500 (EST) Date: Fri, 30 Mar 2001 09:19:09 -0500 (EST) From: "Larry W. Virden" Message-Id: <0103300919.AA5475@cas.org> To: Subject: [Metakit] Forwarded: Returned mail: User unknown, Wikit change needed - preventing wikit from trying to create pages with titles that cause problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I think that the wikit should modify titles that contain characters or strings that cause problems. For instance, pages that end in .tcl only get their title boldfaced and resized. The rest of the text doesn't get processed. The wikit should just not create files by that name (at least by default) - instead, perhaps the user should be asked 'are you sure'? Also, if the user included punctuation that would cause problems, then it should be removed before the title is created... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From bernardl@acumenassociates.com Fri Mar 30 12:36:40 2001 Received: from amadeus.acumenassociates.com ([205.219.35.70]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA27894 for ; Fri, 30 Mar 2001 12:36:39 -0600 Received: from c1524207a ([65.12.238.16]) by amadeus.acumenassociates.com (Post.Office MTA v3.5.3 release 223 ID# 0-59667U100L2S100V35) with SMTP id com for ; Fri, 30 Mar 2001 10:43:19 -0800 Message-ID: <00ab01c0b947$7d12bb10$6b01a8c0@bllvu1.wa.home.com> From: "Bernard Littau" To: Date: Fri, 30 Mar 2001 10:30:25 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Subject: [Metakit] More optimization Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been following the discussion on optimizing loading, and have used some of the suggestions to good effect. My database, however, has a view within a row. For example: c4_View db = rage.GetAs( "db[a:I,b:I,c:S,d:I,e:I," "foo[x:I,y:S,z:I]," "bar[x:I,y:S,z:I]," "baz[x:I,y:S]]" ); If I run the insert with the foo, bar and baz inserts commented out, the inserts are very fast. I am preallocating rows, and using a RowRef to do the data inserts. If I add in even just foo, and foo is just one row, the process starts to slow as more rows get added to db. What strategies can be used to optimize the load given one has views within a row? Thanks, Bernard Littau From jcw@equi4.com Fri Mar 30 12:45:30 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA28342 for ; Fri, 30 Mar 2001 12:45:29 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] More optimization Date: Fri, 30 Mar 2001 10:41:30 -0800 Message-Id: <20010330184130.26531@smtp.telus.net> In-Reply-To: <00ab01c0b947$7d12bb10$6b01a8c0@bllvu1.wa.home.com> References: <00ab01c0b947$7d12bb10$6b01a8c0@bllvu1.wa.home.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bernard Littau wrote: > c4_View db = >rage.GetAs( > "db[a:I,b:I,c:S,d:I,e:I," > "foo[x:I,y:S,z:I]," > "bar[x:I,y:S,z:I]," > "baz[x:I,y:S]]" ); > >If I run the insert with the foo, bar and baz inserts commented out, the >inserts are very fast. I am preallocating rows, and using a RowRef to do >the data inserts. > >If I add in even just foo, and foo is just one row, the process starts to >slow as more rows get added to db. > >What strategies can be used to optimize the load given one has views >within a row? How are you setting subviews? Constructing rows is slow (you are correct in using rowref's as much as possible). Could you show some code on how you create and set foo? MK might be copying subviews too often, with code examples I can suggest things to try. Hm. Or maybe you could load column-wise? Define the view without subviews, and load the a..e columns. Then restructure to include foo, and in a loop set all foo values to what you need, then bar, etc (or maybe all of them at once). -jcw From jcw@equi4.com Fri Mar 30 14:45:54 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA00320 for ; Fri, 30 Mar 2001 14:45:53 -0600 From: Jean-Claude Wippler To: Date: Fri, 30 Mar 2001 12:41:38 -0800 Message-Id: <20010330204138.18631@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] How to unsubscribe Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: With the renewed traffic on the relocated metakit@equi4.com list, I've received a few requests from people who wish to unsubscribe, so here's the info: - go to http://www.equi4.com/mailman/listinfo/metakit - go to your option page as follows: enter your email address at the *bottom*, press "Edit Options" - all passwords were changed, so you need to find out yours press "Email My Password To Me" - when you have it, enter that password and press "Unsubscribe" Alternately, you could choose to batch messages into daily digests: - Choose "On" in the "Set Digest Mode" field - Enter your password at the bottom of the page - Press "Submit My Changes" All of the above use the Mailman web interface. The other way to do things (but I've never tried it), is to send a message to "metakit- request@equi4.com" with "help" in the body of your email msg. Note that a reminder with your (new) password is emailed to you once a month, the first one should happen within in a day or so. My apologies for the password changes and for losing all "send digest" preferences, I saw no way to maintain these in the new list setup. -jcw From bernardl@acumenassociates.com Fri Mar 30 16:09:32 2001 Received: from amadeus.acumenassociates.com ([205.219.35.70]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA02825 for ; Fri, 30 Mar 2001 16:09:32 -0600 Received: from c1524207a ([65.12.238.16]) by amadeus.acumenassociates.com (Post.Office MTA v3.5.3 release 223 ID# 0-59667U100L2S100V35) with SMTP id com for ; Fri, 30 Mar 2001 14:16:23 -0800 Message-ID: <001d01c0b965$40944c30$6b01a8c0@bllvu1.wa.home.com> From: "Bernard Littau" To: Date: Fri, 30 Mar 2001 14:03:28 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Subject: [Metakit] hash use... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am a bit confused as to how to add a hash to a view. From some of the MetaKit documentation: Example of use: c4_View data = storage.GetAs("people[name:S,age:I]"); c4_View datah = storage.GetAs("people_H1[name:S,age:I]"); c4_View hash = raw.Hash(datah, 1); ... hash.GetSize() ... hash.Add(...) Can I assume that "raw" in the above should be "data"? It looks to me that the hash view is not persistent, but that datah and data are. Is this correct? When data and datah are loaded from storage (new program execution) does one always need to execute the third line above that returns the hash view? I take it additions are made to the hash view, and they propagate to data and datah? In the same documentation: The map_ view must be empty the first time this hash view is used, so that MetaKit can fill it based on whatever rows are already present in the underlying view. After that, neither the underlying view nor the map view may be modified other than through this hash mapping layer. The defined structure of the map view must be "_H:I,_R:I". What is meant by the last line? datah above does not have this structure. If I just wanted to hash based on name, would datah become: c4_View datah = storage.GetAs("people_H1[name:S]"); How do I actually do a lookup using the hash? Do I Find on the hash view? Thanks, Bernard From jcw@equi4.com Fri Mar 30 16:25:29 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA03322 for ; Fri, 30 Mar 2001 16:25:28 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] hash use... Date: Fri, 30 Mar 2001 14:20:57 -0800 Message-Id: <20010330222057.2448@smtp.telus.net> In-Reply-To: <001d01c0b965$40944c30$6b01a8c0@bllvu1.wa.home.com> References: <001d01c0b965$40944c30$6b01a8c0@bllvu1.wa.home.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bernard Littau wrote: > >I am a bit confused as to how to add a hash to a view. > >>From some of the MetaKit documentation: > >Example of use: > > c4_View data = storage.GetAs("people[name:S,age:I]"); > c4_View datah = storage.GetAs("people_H1[name:S,age:I]"); > c4_View hash = raw.Hash(datah, 1); > ... hash.GetSize() ... > hash.Add(...) That example is wrong. The "datah" view must have the form: "people_H1[_H:I,_R:I]" >Can I assume that "raw" in the above should be "data"? Yes, another error: c4_View hash = data.Hash(datah, 1); >It looks to me that the hash view is not persistent, but that datah and data >are. Is this correct? Yes. So on open, you have to reconstruct hash again: c4_View hash = data.Hash(datah, 1); That is instant. >When data and datah are loaded from storage (new program execution) does one >always need to execute the third line above that returns the hash view? Yes. >I take it additions are made to the hash view, and they propagate to data >and datah? It's slightly different: the "hash" view is a custom viewer which intercepts changes. It then "turns around" and applies the appropriate changes to data and datah. >In the same documentation: >The map_ view must be empty the first time this hash view is used, so that >MetaKit can fill it based on whatever rows are already present in the >underlying view. After that, neither the underlying view nor the map view >may be modified other than through this hash mapping layer. The defined >structure of the map view must be "_H:I,_R:I". > >What is meant by the last line? datah above does not have this structure. I hope the above corrections clear this up... >If I just wanted to hash based on name, would datah become: > c4_View datah = storage.GetAs("people_H1[name:S]"); No. The "1" arg to Hash says how many of the first props comprise the key. The datah view does not hold data, it's just hash codes and mapping stuff, all handled internally. Always use the following, regardless of what data is: c4_View datah = storage.GetAs("people_H1[_H:I,_R:I]"); >How do I actually do a lookup using the hash? Do I Find on the hash view? Yes. Give Find a row with just a "name" prop. It'll see that the request matches the first field of the hash and will switch to hashing. If Tcl doesn't confuse you too much, see examples/mapped.tcl - likewise, examples/find.py has useful sample code (plus timing results). -jcw From bernard@blama.com Fri Mar 30 16:44:05 2001 Received: from bandit.telemarker.com ([205.219.35.71]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA03953 for ; Fri, 30 Mar 2001 16:44:05 -0600 Received: from c1524207a ([65.12.238.16]) by bandit.telemarker.com (Post.Office MTA v3.5.3 release 223 ID# 0-69692U200L2S100V35) with SMTP id com for ; Fri, 30 Mar 2001 14:23:32 -0800 Message-ID: <006101c0b96a$142d5100$6b01a8c0@bllvu1.wa.home.com> From: bernard@blama.com (Bernard Littau) To: References: <001d01c0b965$40944c30$6b01a8c0@bllvu1.wa.home.com> <20010330222057.2448@smtp.telus.net> Subject: Re: [Metakit] hash use... Date: Fri, 30 Mar 2001 14:38:01 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thank You, that helps a bunch! > >Example of use: > > > > c4_View data = storage.GetAs("people[name:S,age:I]"); > > c4_View datah = storage.GetAs("people_H1[name:S,age:I]"); > > c4_View hash = raw.Hash(datah, 1); > > ... hash.GetSize() ... > > hash.Add(...) > > That example is wrong. The "datah" view must have the form: > "people_H1[_H:I,_R:I]" > > >Can I assume that "raw" in the above should be "data"? > > Yes, another error: c4_View hash = data.Hash(datah, 1); > [snip] From bernard@blama.com Fri Mar 30 17:22:14 2001 Received: from bandit.telemarker.com ([205.219.35.71]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA05091; Fri, 30 Mar 2001 17:22:13 -0600 Received: from c1524207a ([65.12.238.16]) by bandit.telemarker.com (Post.Office MTA v3.5.3 release 223 ID# 0-69692U200L2S100V35) with SMTP id com; Fri, 30 Mar 2001 15:01:40 -0800 Message-ID: <006901c0b96f$687dfed0$6b01a8c0@bllvu1.wa.home.com> From: bernard@blama.com (Bernard Littau) To: "Jean-Claude Wippler" Cc: References: <00e701c0b94b$4ef070c0$6b01a8c0@bllvu1.wa.home.com> <20010330221207.10060@smtp.telus.net> Subject: Re: [Metakit] More optimization Date: Fri, 30 Mar 2001 15:16:10 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Please don't expect me to fully go into your specific app, I'm just > glancing through to see how I can help. Your concern is to fill a DB > quickly. You have many rows in the main view, and a couple of small-or- > empty subviews in each row, right? Correct, many rows at the top level, each with 5 subviews, and each of those subviews having none to eight rows. > > Are you filling rows incrementally, including the subviews? Or do > subviews get extended in a random order? Creating empty subviews should > not take much time. Th efirst row added will have the overhead of > setting up the subview. After that you get O(N) performance, but you may > not have to preallocate subviews if it's mostly about a few rows. Then > again, albums have a dozen or so songs, right? You could preallocate > subviews to size 10, say. And track all sizes in use. Then, at the end > go through all rows and truncate all subviews to the right size. I figured since the number of rows in each subview was small, that preallocation would not gain much. The problem here is curious, as the time to insert just one row into one subview starts to creep up as more and more rows are added to the top level view. I did a test with 1000 top level rows, each with many columns and one subview. If I do not fill the subview at all, each row is added in zero ms by GetTickCount(), basically, the debug output showing the timing for each row scrolls too fast to read. If I then poulate the one subview with one row, the behavior changes. At first top level rows are added fast, 0 ms using GetTickCount(). By the time a few hundred top level rows are added, the time to insert one such top level row starts to climb, 10 ms, 20 ms and so on. And that is with just one subview with just one row in it. The full blown insert gets glacial, in computer terms :-) Here is my quick table of insert speed against number of rows added at the top level (this is for the full blown dataset), using the first set of code below for each of five subviews. 1 <10 ms 500 130 ms 1000 260 ms 1500 390 ms 2000 490 ms 2500 610 ms The problem is not one of insert speed per se, as the insert speed is quite acceptable, initially. It's just that adding more and more top level rows with a subview row causes the insert speed to slow down. This was unexpected. The original, slow, code to insert rows into a subview looked like this: Data *pM = metaData.pM; while (pM) { pId( m_row ) = atol(pM->id); pDegree( m_row ) = pM->degree; if ( pM->name ) pName( m_row ) = pM->name; m_view.Add( m_row ); pM = pM->pNextM; } pMView( s_row ) = m_view.SortOn( pId ); m_view.RemoveAll(); The code that does not slow down significantly as more and more rows are added looks like this: Data *pM = metaData.pM; c4_View m_view = pMView( s_row ); // do one pass through the data to get a count of rows needed int m_count = 0; while( pM ) { m_count++; pM = pM->pNextM; } // preallocate the rows mood_view.SetSize( m_count ); // reset back to the beginning of the data, and go though again adding to the subview pM = metaData.pM; int m_iter = 0; while (pM) { c4_RowRef m_row = m_view[ m_iter++ ]; pId( m_row ) = atol(pM->id); pDegree( m_row ) = pM->degree; if ( pM->name ) pName( m_row ) = pM->name; // m_view.Add( m_row ); pM = pM->pNextM; } // pMView( s_row ) = m_view.SortOn( pId ); // m_view.RemoveAll(); I am not sure why this helps, but it sure does. The second code example allows adds of top level rows at 10 ms or less for thousands of top level rows. I'd be curious as to why this behavior happens, but for now I am happy to have the data being inserted at an acceptable rate. I hope this helps someone else, too. Best, Bernard Littau From jcw@equi4.com Sat Mar 31 16:14:58 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA20363 for ; Sat, 31 Mar 2001 16:14:57 -0600 From: Jean-Claude Wippler To: Date: Sat, 31 Mar 2001 14:10:58 -0800 Message-Id: <20010331221058.22623@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit bug fix and binaries Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Just days after the 2.3.4 release, an important bug has been resolved (new in the 2.3.x code). This caused the crash on every 2nd call of mkdemo (debug build, Windows) - and may have been the cause of other intermittent crashes during commit and exit. The source code fix has been checked into CVS at equi4.com, updated binaries for Linux and Windows (also of Mk4tcl and Mk4py) are at: http://www.equi4.com/pub/mk/ (in the *-2.3.4-29-* dirs) Also, TclKit has been rebuilt, with binaries for Windows and Linux: http://www.equi4.com/pub/tk/ (in the *-8.4-21-* dirs) For details abotu this one-line change, see: http://www.equi4.com/previews/thebug.txt If you try things out, please let me know if there are any problems! Builds / tweaks for other platforms are most welcome. -jcw From resnikb@EUnet.yu Sun Apr 1 12:45:33 2001 Received: from smtp.EUnet.yu (smtp.EUnet.yu [194.247.192.50]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA27614 for ; Sun, 1 Apr 2001 12:45:32 -0500 Received: from P-1.114.EUnet.yu (P-1.114.EUnet.yu [213.240.1.114]) by smtp.EUnet.yu (8.11.2/8.11.2) with ESMTP id f31HeuH14781 for ; Sun, 1 Apr 2001 19:40:56 +0200 (MEST) Date: Sun, 1 Apr 2001 19:37:24 +0200 From: Bojan Resnik X-Mailer: The Bat! (v1.51) Personal Reply-To: Bojan Resnik X-Priority: 3 (Normal) Message-ID: <1444246228.20010401193724@eunet.yu> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Saving changes to a file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I created a simple application, just as a starting point in learning MetaKit API, but it doesn't quite work as it should. I'm using the MetaKit 2.3.4.29 with the latest bug fixed, compiled under VC++ 6.0 on Windows 98. This is just a simple test program: //------------ Begin -------------------- c4_StringProp p_name("name"); c4_IntProp p_age("age"); c4_Storage data_file( "MatPod.mk", true ); c4_View v_people = data_file.GetAs("person[name:S,age:I]"); v_people.Add( p_name["Bojan"] + p_age[22] ); v_people.Add( p_name["Tamara"] + p_age[28] ); data_file.Commit(); const int v_size = v_people.GetSize(); std::ostringstream s; for ( int p_index = 0; p_index < v_size; ++p_index ) { c4_RowRef r = v_people[p_index]; s << "Name: " << (const char *)(p_name(r)) << '\n' << "Age: " << t4_i32(p_age(r)) << '\n' << std::endl; } //------------ End -------------------- When this executes, the result displayed is Name: Age: 0 Name: Age: 0 And nothing is stored in the file. If I change the argument to c4_Storage constructor to false, disallowing changes to the file, it magically displays the correct data, but again nothing is stored in the output file. I suppose this is a bug in MetaKit, but I may very well be doing something wrong. Regards, Bojan Resnik From jcw@equi4.com Tue Apr 10 14:24:50 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA14160; Tue, 10 Apr 2001 14:24:49 -0500 From: Jean-Claude Wippler To: Cc: Date: Tue, 10 Apr 2001 10:36:58 -0700 Message-Id: <20010410173658.5022@smtp.telus.net> In-Reply-To: <00256A29.005BEF37.00@gblutmg01.gmio.com> References: <00256A29.005BEF37.00@gblutmg01.gmio.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Re: Metakit pkgindex.tcl Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Binayak, >I tried to download the latest windows version of mk4tcl. I got a bunch of >dlls, but no pkgindex.tcl. >The examples appear to assume the existence of one, as they start with a >'package require'. I also >looked in the source code tcl directory, but couldn't find it there >either. Am >I missing something? No - you are right, it's not properly set up. I always use TclKit nowadays, so I don't use Mk4tcl as a separate shared library anymore: http://www.equi4.com/tclkit/ But you can create the file "pkgIndex.tcl" as follows: package ifneeded Mk4tcl 2.3.4 \ [list load [file join [set dir] mk4tcl[info sharedlibextension]]] Place both of them together in a directory called "mk4tcl-2.3.4", and place that directory in a place where Tcl looks for extensions (or keep it in "." and insert "lappend auto_path ." at the top of your script). -jcw From bernard@blama.com Wed Apr 11 17:24:18 2001 Received: from bandit.telemarker.com ([205.219.35.71]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA26091 for ; Wed, 11 Apr 2001 17:24:18 -0500 Received: from c1524207a ([205.219.35.36]) by bandit.telemarker.com (Post.Office MTA v3.5.3 release 223 ID# 0-69692U200L2S100V35) with SMTP id com for ; Wed, 11 Apr 2001 15:02:21 -0700 Message-ID: <007501c0c2d5$5e277280$2423dbcd@bllvu1.wa.home.com> From: bernard@blama.com (Bernard Littau) To: Date: Wed, 11 Apr 2001 15:18:42 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Subject: [Metakit] Clean up issues... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I am seeing a fair bit of unreleased memory from Metakit 2.3.3 when my Windows application is exited. I have a pointer to the Storage and that returns a View when I do the GetAs pMetadb = new c4_Storage( "metadb", true ); MetaView = pMetadb->GetAs( ... ); pId = new c4_IntProp("a_id"); pTitle = new c4_StringProp( "a_title" ); I tried doing delete pMetadb, but that crashes from time to time. No idea how to clean up the view. What is the correct way to clean up the Metakit objects? Thanks, Bernard Littau Woodinville, WA From jcw@equi4.com Wed Apr 11 17:51:36 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA26613 for ; Wed, 11 Apr 2001 17:51:36 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Clean up issues... Date: Wed, 11 Apr 2001 15:46:08 -0700 Message-Id: <20010411224608.69@smtp.telus.net> In-Reply-To: <007501c0c2d5$5e277280$2423dbcd@bllvu1.wa.home.com> References: <007501c0c2d5$5e277280$2423dbcd@bllvu1.wa.home.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bernard Littau wrote: >I am seeing a fair bit of unreleased memory from Metakit 2.3.3 when my >Windows application is exited. That's bad. The last tests I did with 2.3.4 seemed ok, but if I missed any leaks, I definitely would like to plug them. >I have a pointer to the Storage and that returns a View when I do the GetAs > >pMetadb = new c4_Storage( "metadb", true ); > MetaView = pMetadb->GetAs( ... ); > > pId = new c4_IntProp("a_id"); > pTitle = new c4_StringProp( "a_title" ); > >I tried doing delete pMetadb, but that crashes from time to time. No idea >how to clean up the view. You can get rid of a view with: MetaView = c4_View (); (Well, no more data, still an empty view object, until MetaView goes out of scope) >What is the correct way to clean up the Metakit objects? As usual: delete if allocated with new on the heap, and leaving scope if allocated on the stack. My hunch is that 2.3.3 is the problem. FYI, just after 2.3.4 a bug was fixed which may well explain your crashes, see: http://www.equi4.com/metakit/CHANGES and http://www.equi4.com/previews/thebug.txt The best version is in CVS, see: http://www.equi4.com/metakit/download.html -jcw From steven@haryan.to Sun Apr 15 07:48:02 2001 Received: from bdg.centrin.net.id (IDENT:root@bdg.centrin.net.id [202.146.253.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA00314 for ; Sun, 15 Apr 2001 07:47:59 -0500 Received: from yola.haryan.to ([202.146.244.232]) by bdg.centrin.net.id (8.10.2/8.10.2) with ESMTP id f3FCgEt14316 for ; Sun, 15 Apr 2001 19:42:16 +0700 Message-Id: <4.3.2.7.2.20010415170420.01b651b8@202.53.255.51> X-Sender: steven@202.53.255.51 (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Sun, 15 Apr 2001 19:42:10 +0700 To: metakit@equi4.com From: Steven Haryanto Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] failed test report (metakit-2.3.4-29) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Metakit version: ---------------- metakit-2.3.4-29 Relevant lines: --------------- diff ../unix/../tests/ok tests diff ../unix/../tests/ok/s33c.txt tests/s33c.txt 1,6c1 < VIEW 1 rows = a:V < 0: subview 'a' < VIEW 3 rows = p1:B < 0: (2b) < 1: (4b) < 2: (5b) --- > VIEW 1 rows = make: *** [test] Error 1 Also, since I am completely ignorant of libtool, this is a warning I get when I make the 'install' and 'install-python' targets: libtool: install: warning: remember to run `libtool --finish /usr/local/lib' 'install-python' did not copy Mk4py.so & metakit.py to my python site-packages directory. Not a big deal, though. My platform: ------------ [root@server builds]# uname -a Linux server 2.4.2 #4 SMP Mon Apr 9 19:38:48 JAVT 2001 i586 unknown [root@server builds]# gcc --version egcs-2.91.66 [root@server builds]# python Python 1.5.2 (#1, Feb 1 2000, 16:32:16) [GCC egcs-2.91.66 19990314/Linux (egcs- on linux-i386 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> Cheers, -- sh From steven@haryan.to Sun Apr 15 07:48:03 2001 Received: from bdg.centrin.net.id (IDENT:root@bdg.centrin.net.id [202.146.253.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA00318 for ; Sun, 15 Apr 2001 07:48:01 -0500 Received: from yola.haryan.to ([202.146.244.232]) by bdg.centrin.net.id (8.10.2/8.10.2) with ESMTP id f3FCgIt14321 for ; Sun, 15 Apr 2001 19:42:19 +0700 Message-Id: <4.3.2.7.2.20010415194240.01cf9658@202.53.255.51> X-Sender: steven@202.53.255.51 (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Sun, 15 Apr 2001 19:46:31 +0700 To: metakit@equi4.com From: Steven Haryanto Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] sortrev() syntax Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have started playing with Metakit for Python again after a while. vw.sortrev() syntax seems a little awkward to me. Is there a reason why it is so? Regards, Steve From jyl@best.com Fri Apr 20 12:19:03 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA18353 for ; Fri, 20 Apr 2001 12:19:03 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id KAA04341; Fri, 20 Apr 2001 10:12:21 -0700 (PDT) Message-ID: <3AE06F62.B7B4C2D5@best.com> Date: Fri, 20 Apr 2001 10:18:27 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com References: <1444246228.20010401193724@eunet.yu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] [ANNOUNCE] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Home page: http://e4graph.sourceforge.net e4Graph is a C++ library based on Metakit that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. e4Graph support input of arbitrary data graphs from XML using the expat parser, and generates XML output from stored data graphs. Changes: Improved C++ binding; improved efficiency and reliability; improved Tcl binding; eliminated many memory leaks; new Java binding; new support for MacOS X. From konrad@minus1.de Sun Apr 22 07:08:14 2001 Received: from sun2.ruf.uni-freiburg.de (sun2.ruf.uni-freiburg.de [132.230.1.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA04123 for ; Sun, 22 Apr 2001 07:08:13 -0500 Received: from fohnhab.minus1.de (gw.sw-stusie.uni-freiburg.de [132.230.131.225]) by sun2.ruf.uni-freiburg.de (8.9.3+Sun/8.9.1) with ESMTP id OAA00235 for ; Sun, 22 Apr 2001 14:01:57 +0200 (MET DST) Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 14rIc1-0001M6-00 for metakit@equi4.com; Sun, 22 Apr 2001 14:05:09 +0200 Date: Sun, 22 Apr 2001 14:05:09 +0200 From: Konrad Anton To: metakit@equi4.com Message-ID: <20010422140509.A5124@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] (Python) problem with commit-aside for read-write opened files Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all... The commit-aside feature looks very promising, but I cannot make it work the way I think it is supposed to work... I open a storage 'sto' with opening mode 1 (read-write), open another read-write storage, attach the second to the first with sto.aside(sto2). If I make changes to views in sto and call sto.commit(1) (and, for general safety, sto2.commit()), I expect the main storage file to contain the newest changes (even when I reopen it in read-only without commit-aside buffer), at least that's what I conclude from python.html. (See code below) But when I reopen it read-only, my view looks like it did before I opened it in commit-aside mode. Is this correct? ---SNIP SNIP--------------- #!/usr/bin/python import metakit def printView(vw): for row in vw: print row.x, print import os try: os.remove("TEMP.mk") os.remove("TEMP.mka") except OSError: pass print "opening RW, adding 111..." sto = metakit.storage("TEMP.mk", 1) vw = sto.getas("wuff[x:I]") vw.append(x=111) sto.commit() del sto, vw print "reopening commit-aside-RW" sto = metakit.storage("TEMP.mk", 1) sto2 = metakit.storage("TEMP.mka", 1) sto.aside(sto2) vw = sto.view("wuff") print "contents: " printView(vw) print "removing 111, appending 222, full commit" del vw[0] vw.append(x=222) sto.commit(1) # commit as commit can sto2.commit(1) print "contents:" printView(vw) del sto, sto2, vw sto = metakit.storage("TEMP.mk", 0) vw = sto.view("wuff") print "reopening base file, contents should be 222:" printView(vw) ---/SNIP SNIP-------------- output is: opening RW, adding 111... reopening commit-aside-RW contents: 111 removing 111, appending 222, full commit contents: 222 reopening base file, contents should be 222: 111 Any help would be appreciated. Konrad. -- Konrad Anton gpg-keyid: 0x22954D8A gunkl|dadadodo sagt: Es dramaturgisch macht aber sich mir wird und er hat sich her? From piet@cs.uu.nl Sun Apr 22 10:59:07 2001 Received: from pietoost (stol-104-182.uu.studentennet.nl [145.98.104.182]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA05393 for ; Sun, 22 Apr 2001 10:59:06 -0500 Received: from [127.0.0.1] by pietoost (ArGoSoft Mail Server, Version 1.61 (1.6.1.2)); Sun, 22 Apr 2001 17:52:54 +0200 X-Mailer: emacs 20.7.1 (via feedmail 8 I); VM 6.75 under Emacs 20.7.1 Message-ID: <15074.65104.500000.897747@gargle.gargle.HOWL> Date: Sun, 22 Apr 2001 17:52:48 +0200 (W. Europe Daylight Time) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: metakit@equi4.com From: piet@cs.uu.nl (Piet van Oostrum) Subject: [Metakit] exclusive access to metakit database Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I need exclusive access to a metakit file for a short period (to do an update from a web page). It is for a reservation system and I don't want multiple reservations to be made for the same slot. I am using it from Python and I thought if I use db = metakit.storage(dbfile, 1) it would open it for exclusive access (at least no other process should be able to open it in the same way). However I tried this on Windows 95 and Windows 2000 and I can do the open in two seperate processes without error. I can then make two updates and of course the first one gets lost. As far as I can see using commit-extend mode will not solve the problem as it also would cause lost updates. How can this be solved? -- Piet van Oostrum Dillenburg 6 3738 VN Maartensdijk From guenther.fischer@hrz.tu-chemnitz.de Tue Apr 24 22:15:03 2001 Received: from obelix.hrz.tu-chemnitz.de (obelix.hrz.tu-chemnitz.de [134.109.132.55]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA13001 for ; Tue, 24 Apr 2001 22:15:02 -0500 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by obelix.hrz.tu-chemnitz.de with esmtp (Exim 3.22 #4) id 14sLHp-0002A5-00 for metakit@equi4.com; Wed, 25 Apr 2001 11:08:37 +0200 Date: Wed, 25 Apr 2001 11:08:36 +0200 (MEST) From: Guenther Fischer To: In-Reply-To: <15074.65104.500000.897747@gargle.gargle.HOWL> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] tkhtml Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I use the tkhtml.[so|dll] from the mk4doc and the hv.tcl as a start script to include a helping system to my application. It works fine, but for special help, I want to jump into a html-file like filename.htm#about If I use the and the reference it works - if I start with the URL/Filename filename.htm#about it shows only the begin of the document ... Any idea? -- ~Guenther Fischer From sbornuz@yahoo.com Sat Apr 28 14:48:14 2001 Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA28180 for ; Sat, 28 Apr 2001 14:48:13 -0500 Received: from 11dial29.xnet.ro (HELO jumpy) (217.10.213.29) by smtp.mail.vip.sc5.yahoo.com with SMTP; 28 Apr 2001 19:49:15 -0000 X-Apparently-From: From: "Stefan-Mircea BORNUZ" To: "metakit4 list" Date: Sat, 28 Apr 2001 22:49:21 +0300 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Building kviewer with BCBuilder4 Standard Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I would like to use mk4 with BCBuilder4 Standard. While compiling the sample kviewer CBuilder project I get [C++ Error] kview.cpp(33): E2034 Cannot convert 'c4_Storage' to 'c4_View'. [C++ Warning] kview.cpp(45): W8004 'root' is assigned a value that is never used. [C++ Error] kview.cpp(90): E2285 Could not find a match for 'c4_View::operator =(c4_Storage)'. in //------------------------------------------------------------------------ void __fastcall TMainForm::FormActivate(TObject *Sender) { c4_View root = *_storage; <----------------------------HERE !!! TTreeNode* top = StructTree->TopItem; _path.SetSize(0); const char* desc = _storage->Description(); SetupTree(top, desc); StructTree->FullExpand(); StructTree->Selected = StructTree->Items->Item[0]; SetupData(); } //------------------------------------------------------------------------- Any ideas ? Many thanks, Stefan _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From sbornuz@yahoo.com Sun Apr 29 04:10:37 2001 Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id EAA00585 for ; Sun, 29 Apr 2001 04:10:37 -0500 Received: from 76dial200.xnet.ro (HELO jumpy) (213.233.76.200) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Apr 2001 09:11:49 -0000 X-Apparently-From: From: "Stefan-Mircea BORNUZ" To: "metakit4 list" Subject: RE: [Metakit] Building kviewer with BCBuilder4 Standard Date: Sun, 29 Apr 2001 12:12:08 +0300 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ooops! Sorry! I have messed v2.01 headers with v2.3.4-29 libs! S. > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of Stefan-Mircea BORNUZ > Sent: 28 aprilie 2001 22:49 > To: metakit4 list > Subject: [Metakit] Building kviewer with BCBuilder4 Standard > > > Hello, > > I would like to use mk4 with BCBuilder4 Standard. > > While compiling the sample kviewer CBuilder project I get > > [C++ Error] kview.cpp(33): E2034 Cannot convert 'c4_Storage' to 'c4_View'. > [C++ Warning] kview.cpp(45): W8004 'root' is assigned a value > that is never > used. > [C++ Error] kview.cpp(90): E2285 Could not find a match for > 'c4_View::operator =(c4_Storage)'. > > in > > //------------------------------------------------------------------------ > void __fastcall TMainForm::FormActivate(TObject *Sender) > { > c4_View root = *_storage; <----------------------------HERE !!! > TTreeNode* top = StructTree->TopItem; > > _path.SetSize(0); > > const char* desc = _storage->Description(); > SetupTree(top, desc); > > StructTree->FullExpand(); > StructTree->Selected = StructTree->Items->Item[0]; > > SetupData(); > } > //---------------------------------------------------------------- > --------- > > > Any ideas ? > > Many thanks, > > Stefan > > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From fgeiger@datec.at Tue May 1 02:37:00 2001 Received: from datec.at (datec.at [213.208.128.21]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA28380 for ; Tue, 1 May 2001 02:36:59 -0500 Received: from nb0204 [194.183.151.149] by datec.at (SMTPD32-6.04) id A7035220272; Tue, 01 May 2001 09:34:27 +0200 Reply-To: From: "F. GEIGER" To: Date: Tue, 1 May 2001 09:35:56 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] New to Mk4py - Where are the samples? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi everybody, encountered Mk4py and find it kind of pretty cool. What really is a pitty is the lack of samples. The page www.equi4.com/metakit/python.html is too less - at least for me. Does anybody have some samples? In concrete: I wanted to fake a SQL-SELECT-statement using LIKE and tried to do it by using view.map() and function of mine. This returns a RowRef array and that's where I got stuck already. Another one: The page mentioned above tells me to import metakit. All I could import was Mk4py. Whta's about metakit? Where is it? Platform: ActivePython 2.0 on Windows NT4. Best regards Franz GEIGER From gmcm@hypernet.com Tue May 1 18:12:40 2001 Received: from hypernet.com ([204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA23903 for ; Tue, 1 May 2001 18:12:39 -0500 Received: from me (204.176.40.129) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Tue, 1 May 2001 19:12:24 -0400 From: "Gordon McMillan" To: , Date: Tue, 1 May 2001 19:12:23 -0400 Subject: Re: [Metakit] New to Mk4py - Where are the samples? Reply-to: gmcm@hypernet.com Message-ID: <3AEF0A97.22657.7C8A1DFA@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Franz GEIGER wrote: > > encountered Mk4py and find it kind of pretty cool. What really is > a pitty is the lack of samples. The page > www.equi4.com/metakit/python.html is too less - at least for me. > Does anybody have some samples? > > In concrete: I wanted to fake a SQL-SELECT-statement using LIKE > and tried to do it by using view.map() and function of mine. This > returns a RowRef array and that's where I got stuck already. I think you mean filter, not map? (map is for mutating the rows in the view, and returns None, at least in my version.) filter returns a new view. The view has a single column, named _index. It's a number - the index in the original view of a row satisfying your filter function. So it might go something like this: view = db.view('customers') myfilter = lambda(row: row.lname == 'Smi') indices = view.filter(myfilter) for indexrec in indices: # do something with view[indexrec._index] You could also sort on the column you're interested in, then select using lo and hi: copy = view.sort(view.lname) v2 = copy.select({'lname':'Smi'}, {'lname':'Smi\377'}) In this case you won't be able to mutate anything. > Another one: The page mentioned above tells me to import metakit. > All I could import was Mk4py. Whta's about metakit? Where is it? Mk4py is all you need (it's statically linked to the C++ metakit library). > Platform: ActivePython 2.0 on Windows NT4. > > Best regards > Franz GEIGER > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit - Gordon From gmcm@hypernet.com Tue May 1 22:03:16 2001 Received: from hypernet.com ([204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA26322 for ; Tue, 1 May 2001 22:03:16 -0500 Received: from me (204.176.40.74) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Tue, 1 May 2001 23:02:59 -0400 From: "Gordon McMillan" To: , Date: Tue, 1 May 2001 23:02:59 -0400 Subject: Re: [Metakit] New to Mk4py - Where are the samples? Reply-to: gmcm@hypernet.com Message-ID: <3AEF40A3.14240.7D5D3B8A@localhost> Priority: normal In-reply-to: <3AEF0A97.22657.7C8A1DFA@localhost> References: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Oops, hopefully obviously I meant: myfilter = lambda(row: row.lname[:3] == 'Smi') - Gordon > Franz GEIGER wrote: > > > > encountered Mk4py and find it kind of pretty cool. What really > > is a pitty is the lack of samples. The page > > www.equi4.com/metakit/python.html is too less - at least for > > me. Does anybody have some samples? > > > > In concrete: I wanted to fake a SQL-SELECT-statement using LIKE > > and tried to do it by using view.map() and function of mine. > > This returns a RowRef array and that's where I got stuck > > already. > > I think you mean filter, not map? (map is for mutating the rows > in the view, and returns None, at least in my version.) > > filter returns a new view. The view has a single column, named > _index. It's a number - the index in the original view of a row > satisfying your filter function. So it might go something like > this: > > view = db.view('customers') > myfilter = lambda(row: row.lname == 'Smi') > indices = view.filter(myfilter) > for indexrec in indices: > # do something with view[indexrec._index] > > You could also sort on the column you're interested in, then > select using lo and hi: copy = view.sort(view.lname) v2 = > copy.select({'lname':'Smi'}, {'lname':'Smi\377'}) > > In this case you won't be able to mutate anything. > > > > Another one: The page mentioned above tells me to import > > metakit. All I could import was Mk4py. Whta's about metakit? > > Where is it? > > Mk4py is all you need (it's statically linked to the C++ metakit > library). > > > Platform: ActivePython 2.0 on Windows NT4. > > > > Best regards > > Franz GEIGER > > > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > > - Gordon > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From cer@transelec.cl Wed May 2 09:14:59 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA12133 for ; Wed, 2 May 2001 09:14:57 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id KAA22324; Wed, 2 May 2001 10:13:14 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Wed, 2 May 2001 10:19:03 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C03F2E330@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: gmcm@hypernet.com Cc: metakit@equi4.com Subject: RE: [Metakit] New to Mk4py - Where are the samples? Date: Wed, 2 May 2001 10:19:01 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi About the last release, I got Mk4py, but I can't find MkSQL Anybody knows where is it? Cristian Echeverria From cer@transelec.cl Thu May 3 14:33:58 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA28636 for ; Thu, 3 May 2001 14:33:57 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id PAA12475; Thu, 3 May 2001 15:32:03 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Thu, 3 May 2001 15:37:57 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C03FEB6EB@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: gmcm@hypernet.com Cc: metakit@equi4.com Subject: FW: [Metakit] New to Mk4py - Where are the samples? Date: Thu, 3 May 2001 15:37:54 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id OAA28636 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Please, I need an answer Where can I find MkSQL? Is this working with Mk4py 2.3.4-29? > -----Original Message----- > From: Echeverría, Cristian On Behalf Of Echeverria Rabi, Cristian > Rodrigo, TRANSELEC S.A. > Sent: Wednesday, May 02, 2001 10:11 AM > To: 'gmcm@hypernet.com' > Cc: 'metakit@equi4.com' > Subject: RE: [Metakit] New to Mk4py - Where are the samples? > Importance: High > > Hi > About the last release, I got Mk4py, but I can't find MkSQL > Anybody knows where is it? > > Cristian Echeverria From jcw@equi4.com Thu May 3 14:41:45 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA28897 for ; Thu, 3 May 2001 14:41:45 -0500 From: Jean-Claude Wippler To: Subject: Re: FW: [Metakit] New to Mk4py - Where are the samples? Date: Thu, 3 May 2001 12:41:57 -0700 Message-Id: <20010503194157.17172@smtp.telus.net> In-Reply-To: <16D7A2FE3592D41184A200508BDE826C03FEB6EB@SYN201> References: <16D7A2FE3592D41184A200508BDE826C03FEB6EB@SYN201> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. wrote: >Please, I need an answer >Where can I find MkSQL? I'm afraid that the answer may not help you much furter: MkSQL is not finished, nor supported. I've added some comments about this on the MetaKit Wiki: http://www.equi4.com/metakit/wiki.cgi/126.html -jcw From jcw@equi4.com Thu May 3 15:10:40 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29901 for ; Thu, 3 May 2001 15:10:29 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] tkhtml Date: Thu, 3 May 2001 13:04:21 -0700 Message-Id: <20010503200421.29161@smtp.telus.net> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: >I use the tkhtml.[so|dll] from the mk4doc and the hv.tcl as a start script >to include a helping system to my application. > >It works fine, but for special help, I want to jump into a html-file >like filename.htm#about > >If I use the and the reference it works >- if I start with the URL/Filename filename.htm#about it shows only the >begin of the document ... > >Any idea? Yes, I've come across the same issue before. This seems to be an issue which the current hv.tcl or tkhtml.tcl do not address - I suggest contacting the author of tkhtml about this, Richard Hipp . -jcw From jcw@equi4.com Thu May 3 15:10:47 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29909 for ; Thu, 3 May 2001 15:10:40 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] (Python) problem with commit-aside for read-write opened files Date: Thu, 3 May 2001 13:06:50 -0700 Message-Id: <20010503200650.20101@smtp.telus.net> In-Reply-To: <20010422140509.A5124@minus1.de> References: <20010422140509.A5124@minus1.de> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Konrad Anton wrote: >The commit-aside feature looks very promising, but I cannot make it >work the way I think it is supposed to work... Have you checked the examples/*.py files in the source distribution? There should be a commit-aside example in there. Notre that the new commit modes are not fully tested. I do not expect to have them working 100% in time for the next release - there are some tricky issues which are as yet unresolved. -jcw From jcw@equi4.com Thu May 3 15:10:52 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29912 for ; Thu, 3 May 2001 15:10:47 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] exclusive access to metakit database Date: Thu, 3 May 2001 13:10:29 -0700 Message-Id: <20010503201029.32271@smtp.telus.net> In-Reply-To: <15074.65104.500000.897747@gargle.gargle.HOWL> References: <15074.65104.500000.897747@gargle.gargle.HOWL> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Piet van Oostrum wrote: >I need exclusive access to a metakit file for a short period (to do an >update from a web page). It is for a reservation system and I don't >want multiple reservations to be made for the same slot. > >I am using it from Python and I thought if I use > db = metakit.storage(dbfile, 1) >it would open it for exclusive access (at least no other process >should be able to open it in the same way). However I tried this on >Windows 95 and Windows 2000 and I can do the open in two seperate >processes without error. I can then make two updates and of course the >first one gets lost. > >As far as I can see using commit-extend mode will not solve the >problem as it also would cause lost updates. > >How can this be solved? The MK 2.01 release used to include locking, but because it turned out to be too hard (for me anyway, but all help is welcome) to support this reliably across all patforms (and especially with NFS), I decided to drop this instead of leaving an unreliable solution. My suggestion would be to implement a locking scheme *around* MK from Python, using the flock etc primitives it provides. Not locking will lead to trouble. The moment a commit is done, *all* other open versions of the data file are at risk. Well, technically speaking, read-only versions would only fail on a second commit - but it's easier to just consider a write as invalidating all open files. -jcw From jcw@equi4.com Thu May 3 15:14:03 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA30070 for ; Thu, 3 May 2001 15:14:03 -0500 From: Jean-Claude Wippler To: metakit Date: Thu, 3 May 2001 13:14:04 -0700 Message-Id: <20010503201404.16814@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Mailing list - reply to? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For some reason, after a recent HD replacement and full system re- install, Mailman 2.0.3 seems to want to set the reply address of every post to this list to "metakit-admin@equi4.com" instead of leaving the reply to sender. I have no idea why. If anyone has a suggestion, please let me know. Until it is resolved, please manually adjust the reply-to destination - metakit-admin@equi4.com is definitely not right! -jcw From nathan@vividworks.com Wed May 9 13:01:32 2001 Received: from mail.frontlogic.com (h139-142-46-3.gtcust.grouptelecom.net [139.142.46.3]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id NAA02652 for ; Wed, 9 May 2001 13:01:30 -0500 Message-Id: <200105091801.NAA02652@trixie.triqs.com> Received: (qmail 18204 invoked from network); 9 May 2001 17:51:49 -0000 Received: from unknown (HELO localhost) (139.142.46.131) by mail.frontlogic.com with SMTP; 9 May 2001 17:51:49 -0000 Date: Wed, 9 May 2001 11:59:58 -0500 Content-Type: text/plain; format=flowed; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v388) From: Nathan Heagy To: metakit@equi4.com X-Mailer: Apple Mail (2.388) Content-Transfer-Encoding: 7bit Subject: [Metakit] How to compile Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I can't figure out how to compile Mk4py. According to the very vague and complex build instructions for MetaKit it would seem that the python module gets built automatically: * Linux / gcc 2.95.2 Builds with gcc 2.95.2 work out of the box. The Python extension ends up as Mk4py.so (ignore libmk4py.*), the Tcl extension is called Mk4tcl.so (ignore libmk4tcl.*). Other than this there is *no* mention of how to build the python module. In addition to updating the documentation to be clear, can anyone tell me how to do this? thanks, nathan From lvirden@cas.org Thu May 17 07:11:58 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA21371 for ; Thu, 17 May 2001 07:11:57 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id IAA26742 for ; Thu, 17 May 2001 08:11:16 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f4HCBEI03901; Thu, 17 May 2001 08:11:14 -0400 (EDT) Date: Thu, 17 May 2001 08:11:14 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010517081114.AAB3879@cas.org> To: Subject: [Metakit] Help sought getting tclkit/metakit/wiki to work as CGI Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Platform: SPARC Solaris Web server: Netscape Enterprise I have the wikit.cgi file set up to point to the tclkit and the wikit.tkd. I can run the cgi script from the command line and it works. When I try to run it as a CGI , the server reports this error: [17/May/2001:07:57:20] failure (13910): for host 134.243.55.72 trying to GET /~lwv26/cgi-bin/wikit.cgi, cgieng_scan_headers reports: the CGI program /home/lwv26/castest_html/cgi-bin/wikit.cgi did not produce a valid header (name without value: got line "file open failed") Anyone have any ideas? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From lvirden@cas.org Thu May 17 07:34:02 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA21899 for ; Thu, 17 May 2001 07:34:02 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id IAA03794 for ; Thu, 17 May 2001 08:33:18 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f4HCXIm04211; Thu, 17 May 2001 08:33:18 -0400 (EDT) Date: Thu, 17 May 2001 08:33:18 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010517083318.AAB4202@cas.org> Subject: Re: [Metakit] Help sought getting tclkit/metakit/wiki to work as CGI In-Reply-To: <20010517081114.AAB3879@cas.org> of Thu, 17 May 2001 08:11:14 -0400 (EDT) To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The good news is that I found what my problem was - there was a permission problem with tclkit causing that error msg. Sorry for the bother. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From rmenegon@bigpond.net.au Thu May 17 03:55:47 2001 Received: from mailin2.email.bigpond.com (juicer14.bigpond.com [139.134.6.23]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA13655 for ; Thu, 17 May 2001 03:55:46 -0500 Received: from unspecified.host ([144.135.24.78]) by mailin2.email.bigpond.com (Netscape Messaging Server 4.15) with SMTP id GDH2DA00.D8E for ; Thu, 17 May 2001 19:00:46 +1000 Received: from CPE-144-132-187-24.nsw.bigpond.net.au ([144.132.187.24]) by bwmam04.mailsvc.email.bigpond.com(MailRouter V2.9d 8329/575000); 17 May 2001 18:59:13 Received: from 192.168.11.13 ([192.168.11.13]) by 192.168.11.1 (WinRoute Pro 4.1) with SMTP; Thu, 17 May 2001 18:55:52 +1000 Received: by localhost with Microsoft MAPI; Thu, 17 May 2001 19:00:06 +1000 Message-ID: <01C0DF03.964034A0.rmenegon@bigpond.net.au> From: Rob Menegon Reply-To: "rmenegon@bigpond.net.au" To: "'metakit@equi4.com'" Date: Thu, 17 May 2001 19:00:05 +1000 Organization: MenMac X-Mailer: Microsoft Internet E-mail/MAPI - 8.0.0.4211 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] Getting started Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Some background I have recently embarked on a project to store and analyse statistics associated with particulat sport that my kids are involved in. The basic structure goes something like this: Meeting Code / unique identifier Name Event Code / unique identifier Name Competitor Code / unique identifier Name and so on Competitor Code / unique identifier Name and so on Event Code / unique identifier Name Competitor Code / unique identifier Name and so on Competitor Code / unique identifier Name and so on In summary there can be multiple Meetings each containing some local information as well as many events which in turn have local information and contain many competitor information bits ... much like a directory listing in structure. Help I have been beating myself up on how I should store this so that I can retrieve the information easily and quickly. In my investigations I came across your product. I have described my structure outlined above by the following: c4_View vMeeting = storage.GetAs("meeting[mcode:S,mname:S," "event[ecode:S,rname:S," "competitor[ccode:S,cname:S]]]"); My question is relating to how I update (access) the event and competitor information and retrieve this information later using this structure. pMeetingCode (row) = "AAAA"; pMeetingName (row) = "FirstMeet"; OR perhaps I am way of the mark and need to structure it differently i.e. 3 views, Meetings, Events, Competitors Is someone able to get me started? Rob _______________________________________ Rob Menegon 349 Somerville Road Hornsby Heights NSW 2077 Sydney, Australia (Mobile) +61 (0)416 271 348 ( Voice) +61 (0)2 9477 1348 ( Fax) +61 (0)2 8570 2031 rmenegon@bigpond.net.au _______________________________________ This email contains information intended for the addressee only. This information is confidential and may be privileged. If you are not the intended recipient, you must not read, use, copy or distribute this email or its attachments. If you have received this in error, please notify me immediately by return email or telephone, and delete and destroy this email. Thank you. _______________________________________ From g.lancaster@mbf.com.au Thu May 17 18:19:36 2001 Received: from mbfdns ([203.101.28.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA03371 for ; Thu, 17 May 2001 18:19:34 -0500 Received: from MBF-INT1-Message_Server by mbfdns with Novell_GroupWise; Fri, 18 May 2001 09:05:41 +1000 Message-Id: X-Mailer: Novell GroupWise 5.5.2 Date: Fri, 18 May 2001 09:16:25 +1000 From: "Garth Lancaster" To: , Subject: Re: [Metakit] Getting started Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline X-Guinevere: 1.0.13 ; Medical Benefits Fun Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id SAA03371 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Rob, I think you would be better off using a recursive 'graph-node' structure to do this (ie represent your data) - it makes traversal etc easier - Jacob Levy has doen this nicely in a package called e4graph, at http://e4graph.sourceforge.net/ - this sits on top of MetaKit, and says good things about the extensibility/usefulness of MetaKit itself (I have no commercial interest in anything I describe here, in case you're wondering - I just come across a lot of usefull pieces in my 'travels', and like to share knowledge) If you wanted to make this 'transportable', you could :- traverse your graph structure and build an XML data-set, or unravel the graph to a flat file, or maybe read the MetaKit representation of the Graph directly ... I can send you the URL for a 'lightweight' XML package (not free, but cheap), that I found, if you like, if you want/need to do the first option (of course, there are other ways to skin a cat, maybe someone else out there has better ideas) have fun, regards, Garth Garth Lancaster MBF InfoTech (Infrastructure) Integration Engineer g.lancaster@mbf.com.au Ph : +61 2 9323 9534 Fax : +61 2 9267 7359 From Slick@battleop.net Thu May 17 23:29:37 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA05497 for ; Thu, 17 May 2001 23:29:36 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Thu, 17 May 2001 21:35:42 -0700 Message-ID: <004901c0df53$7b6b8ee0$0200a8c0@pacbell.net> From: "Slick" To: Date: Thu, 17 May 2001 21:32:00 -0700 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0046_01C0DF18.CEEB2520" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Getting started with the latest revision v2.3.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0046_01C0DF18.CEEB2520 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I have been reading the web site with some interest. However, some = links go to older versions for example v2.0. Is there 1 zip file for Windows that contains all needed files? I need = instructions on how to set it up and where to get the latest header = files etc so that I can experiment with it. Any links/instructions are appreciated. I saw the directory with the = latest *.lib and *.dll files but where are the related header files? Thanks (I'll continue to dig around the web site also) ------=_NextPart_000_0046_01C0DF18.CEEB2520 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I have been reading the web site with = some=20 interest.  However, some links go to older versions for example=20 v2.0.
 
Is there 1 zip file for Windows that = contains all=20 needed files?  I need instructions on how to set it up and where to = get the=20 latest header files etc so that I can experiment with it.
 
Any links/instructions are = appreciated.  I saw=20 the directory with the latest *.lib and *.dll files but where are the = related=20 header files?
 
Thanks (I'll continue to dig around the = web site=20 also)
------=_NextPart_000_0046_01C0DF18.CEEB2520-- From jcw@equi4.com Fri May 18 00:37:01 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id AAA05944 for ; Fri, 18 May 2001 00:37:00 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Getting started with the latest revision v2.3.4 Date: Thu, 17 May 2001 22:36:50 -0700 Message-Id: <20010518053650.27829@smtp.telus.net> In-Reply-To: <004901c0df53$7b6b8ee0$0200a8c0@pacbell.net> References: <004901c0df53$7b6b8ee0$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: >Is there 1 zip file for Windows that contains all needed files? I need >instructions on how to set it up and where to get the latest header files >etc so that I can experiment with it. > >Any links/instructions are appreciated. I saw the directory with the >latest *.lib and *.dll files but where are the related header files? The sources and build files (also for MSVC6, Windows) are in: http://www.equi4.com/pub/mk/metakit.tar.gz This currently points to: http://www.equi4.com/pub/mk/metakit-2.3.4-29.tar.gz The public C++ headers are all in the include/ subdirectory. For Python or Tcl, you need no headers - just mk4py.dll/mk4tcl.dll ... -jcw From Slick@battleop.net Fri May 18 20:20:46 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id UAA06208 for ; Fri, 18 May 2001 20:20:44 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Fri, 18 May 2001 18:26:46 -0700 Message-ID: <001301c0e002$43e34c60$0200a8c0@pacbell.net> From: "Slick" To: Date: Fri, 18 May 2001 18:23:09 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I put just this one line of code in: c4_StringProp pName ("name"); I have the following includes: #include "mk4.h" #include "mk4str.h" I get the following errors: error LNK2001: unresolved external symbol "public: __thiscall c4_Property::c4_Property(char,char const *)" (??0c4_Property@@QAE@DPBD@Z) error LNK2001: unresolved external symbol "public: __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) I built all the metakit builds that I could (static, debug etc). I have the relevant paths set up to the metakit directories. What could be happening here? Thanks From jcw@equi4.com Fri May 18 20:36:50 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA06337; Fri, 18 May 2001 20:36:47 -0500 From: Jean-Claude Wippler To: , Subject: Re: [Metakit] Linking error Date: Fri, 18 May 2001 18:36:40 -0700 Message-Id: <20010519013640.17413@smtp.telus.net> In-Reply-To: <001301c0e002$43e34c60$0200a8c0@pacbell.net> References: <001301c0e002$43e34c60$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: >I put just this one line of code in: > >c4_StringProp pName ("name"); > >I have the following includes: > >#include "mk4.h" >#include "mk4str.h" > >I get the following errors: > >error LNK2001: unresolved external symbol "public: __thiscall >c4_Property::c4_Property(char,char const *)" (??0c4_Property@@QAE@DPBD@Z) >error LNK2001: unresolved external symbol "public: __thiscall >c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) > >I built all the metakit builds that I could (static, debug etc). > >I have the relevant paths set up to the metakit directories. > >What could be happening here? Try adding the define "q4_INLINE" (either in the compiler, or as "#define a4_INLINE 1" *before* the mk4 includes). Does this fix things? If so, I'll mention it on the Metakit wiki. -jcw From jerry@usd1.com Fri May 18 20:59:24 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA06499 for ; Fri, 18 May 2001 20:59:23 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f4J1wji24114 for ; Fri, 18 May 2001 18:58:46 -0700 (PDT) Date: Fri, 18 May 2001 18:40:17 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae <1jerry@usd1.com> Organization: usd1 X-Priority: 3 (Normal) Message-ID: <6917523888.20010518184017@usd1.com> To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Python 2.1 & Mk4py.dll Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello metakit, Just installed Python 2.1. The version of Mk4py.dll I have been using is dated 11/23/00 and is 221k. Python didn't like it loading python20.dll, so I downloaded the newest Mk4py today. (Oddly, it is MUCH smaller at ~70k!). It still does not work. Gives the following message: >>> import Mk4py Fatal Python error: Interpreter not initialized (version mismatch?) abnormal program termination I have tried it with the Mk4py.dll in the same directory, just to make sure I'm using the correct one. Any help? -- Best regards, 1Jerry From jcw@equi4.com Fri May 18 21:19:56 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA06644 for ; Fri, 18 May 2001 21:19:55 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Python 2.1 & Mk4py.dll Date: Fri, 18 May 2001 19:19:43 -0700 Message-Id: <20010519021943.6085@smtp.telus.net> In-Reply-To: <6917523888.20010518184017@usd1.com> References: <6917523888.20010518184017@usd1.com> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry McRae wrote: >Just installed Python 2.1. The version of Mk4py.dll I have been using >is dated 11/23/00 and is 221k. Python didn't like it loading >python20.dll, so I downloaded the newest Mk4py today. (Oddly, it is >MUCH smaller at ~70k!). It still does not work. Gives the following >message: > >>>> import Mk4py >Fatal Python error: Interpreter not initialized (version mismatch?) > >abnormal program termination > >I have tried it with the Mk4py.dll in the same directory, just to make >sure I'm using the correct one. The "newest" was also linked against 2.0 AFAIK. If anyone has newer releases, I'd be happy to accept 2.1 builds for Windows, Linux, etc - and put them next to the others on the web. (Tcl has "stubs", it never needs a recompile, mumble, mumble...) The 70 Kb size is due to UPX compression, see http://wildsau.idv.uni-linz.ac.at/mfx/upx.html -jcw From jyl@best.com Fri May 18 23:00:15 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA07171 for ; Fri, 18 May 2001 23:00:15 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id UAA11380; Fri, 18 May 2001 20:58:10 -0700 (PDT) Message-ID: <3B05F0D1.5B3D35B0@best.com> Date: Fri, 18 May 2001 21:04:33 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: "rmenegon@bigpond.net.au" CC: "'metakit@equi4.com'" Subject: Re: [Metakit] Getting started References: <01C0DF03.964034A0.rmenegon@bigpond.net.au> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This looks like an ideal application for e4Graph. Check out http://e4graph.sourceforge.net/ The e4Graph software uses Metakit as a storage mechanism. --JYL Rob Menegon wrote: > Some background > > I have recently embarked on a project to store and analyse statistics > associated with particulat sport that my kids are involved in. The basic > structure goes something like this: > > Meeting > Code / unique identifier > Name > Event > Code / unique identifier > Name > Competitor > Code / unique identifier > Name > and so on > Competitor > Code / unique identifier > Name > and so on > Event > Code / unique identifier > Name > Competitor > Code / unique identifier > Name > and so on > Competitor > Code / unique identifier > Name > and so on > > In summary there can be multiple Meetings each containing some local > information as well as many events which in turn have local information and > contain many competitor information bits ... much like a directory listing > in structure. > > Help > > I have been beating myself up on how I should store this so that I can > retrieve the information easily and quickly. In my investigations I came > across your product. I have described my structure outlined above by the > following: > > c4_View vMeeting = storage.GetAs("meeting[mcode:S,mname:S," > "event[ecode:S,rname:S," > "competitor[ccode:S,cname:S]]]"); > > My question is relating to how I update (access) the event and competitor > information and retrieve this information later using this structure. > > pMeetingCode (row) = "AAAA"; > pMeetingName (row) = "FirstMeet"; > > OR perhaps I am way of the mark and need to structure it differently i.e. 3 > views, Meetings, Events, Competitors > > Is someone able to get me started? > > Rob > _______________________________________ > Rob Menegon > > 349 Somerville Road > Hornsby Heights > NSW 2077 > Sydney, Australia > (Mobile) +61 (0)416 271 348 > ( Voice) +61 (0)2 9477 1348 > ( Fax) +61 (0)2 8570 2031 > rmenegon@bigpond.net.au > _______________________________________ > This email contains information intended for the addressee only. This > information is confidential and may be privileged. If you are not the > intended recipient, you must not read, use, copy or distribute this email > or > its attachments. If you have received this in error, please notify me > immediately by return email or telephone, and delete and destroy this > email. > Thank you. > _______________________________________ > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jyl@best.com Fri May 18 23:02:23 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA07210 for ; Fri, 18 May 2001 23:02:20 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id VAA16422; Fri, 18 May 2001 21:00:27 -0700 (PDT) Message-ID: <3B05F160.D51B1427@best.com> Date: Fri, 18 May 2001 21:06:56 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Garth Lancaster CC: rmenegon@bigpond.net.au, metakit@equi4.com Subject: Re: [Metakit] Getting started References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks Garth for the mention! I should add that e4Graph already supports XML input/output so your problem is nearly completely solved. You just have to design the tree structure you want and serialize/deserialize it and you should be close to done. --JYL Garth Lancaster wrote: > Rob, I think you would be better off using a recursive 'graph-node' structure to do this (ie represent your data) - it makes traversal etc easier - Jacob Levy has doen this nicely in a package called e4graph, at http://e4graph.sourceforge.net/ - this sits on top of MetaKit, and says good things about the extensibility/usefulness of MetaKit itself > > (I have no commercial interest in anything I describe here, in case you're wondering - I just come across a lot of usefull pieces in my 'travels', and like to share knowledge) > > If you wanted to make this 'transportable', you could :- traverse your graph structure and build an XML data-set, or unravel the graph to a flat file, or maybe read the MetaKit representation of the Graph directly ... I can send you the URL for a 'lightweight' XML package (not free, but cheap), that I found, if you like, if you want/need to do the first option > > (of course, there are other ways to skin a cat, maybe someone else out there has better ideas) > > have fun, regards, Garth > > Garth Lancaster > MBF InfoTech (Infrastructure) Integration Engineer > g.lancaster@mbf.com.au > Ph : +61 2 9323 9534 > Fax : +61 2 9267 7359 > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From Slick@battleop.net Sat May 19 01:04:59 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA08687 for ; Sat, 19 May 2001 01:04:57 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Fri, 18 May 2001 23:10:56 -0700 Message-ID: <000b01c0e029$f5f18120$0200a8c0@pacbell.net> From: "Slick" To: Date: Fri, 18 May 2001 23:07:18 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I tried adding the following immediately before the includes in my .cpp file: #define q4_inline 1 also tried #define a4_inline 1 since you mentioned both (not sure which is correct) When I choose build|compile I don't get any errors. It is when they are linked. Normally I just choose build (F7) when I'm working since no reason for 2 steps. I am not familiar with the #define q4_inline 1 command. Is it possible that I don't have all of the necessary metakit .dll or .libs built? I don't know what the minimum config is under VC6 in a MFC app. Unfortunate that I can't get it to work after only 1 line of code :) In the directory config I have the following settings: executable files \builds include files \include library files \builds source files \src From jcw@equi4.com Sat May 19 01:48:35 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA09604 for ; Sat, 19 May 2001 01:48:35 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Linking error Date: Fri, 18 May 2001 23:48:23 -0700 Message-Id: <20010519064823.9621@smtp.telus.net> In-Reply-To: <000b01c0e029$f5f18120$0200a8c0@pacbell.net> References: <000b01c0e029$f5f18120$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: >I tried adding the following immediately before the includes in my .cpp >file: > >#define q4_inline 1 #define q4_INLINE 1 (case matters in C++) >#define a4_inline 1 >since you mentioned both (not sure which is correct) Oops, sorry. It's "q". -jcw From Slick@battleop.net Sat May 19 12:16:29 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA22952 for ; Sat, 19 May 2001 12:16:27 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Sat, 19 May 2001 10:22:27 -0700 Message-ID: <001901c0e087$c5c36e60$0200a8c0@pacbell.net> From: "Slick" To: Date: Sat, 19 May 2001 10:18:50 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Still does not work after doing the following: #define q4_INLINE 1 //mine #include "mk4.h" //mine #include "mk4str.h" //mine I do not know what the inline means and was supposed to accomplish. My guess is that I am missing a file - could it be that I get these errors because on of the dll's or lib's didn't build when I set up metakit? fyi I have built mk_test_d.exe and it appears to work by running through all of the tests ok. They scroll quickly but it looks like they all work. From jyl@best.com Sat May 19 12:27:33 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA23054 for ; Sat, 19 May 2001 12:27:33 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id KAA13638; Sat, 19 May 2001 10:25:25 -0700 (PDT) Message-ID: <3B06AE0A.37F39BC0@best.com> Date: Sat, 19 May 2001 10:31:54 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Slick CC: metakit@equi4.com Subject: Re: [Metakit] Linking error References: <001901c0e087$c5c36e60$0200a8c0@pacbell.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I could take a look to try and reproduce your problem if you could send all needed files. Zip format preferred. --JYL Slick wrote: > Still does not work after doing the following: > > #define q4_INLINE 1 //mine > #include "mk4.h" //mine > #include "mk4str.h" //mine > > I do not know what the inline means and was supposed to accomplish. > > My guess is that I am missing a file - could it be that I get these errors > because on of the dll's or lib's didn't build when I set up metakit? > > fyi I have built mk_test_d.exe and it appears to work by running through all > of the tests ok. They scroll quickly but it looks like they all work. > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From Slick@battleop.net Mon May 21 21:37:54 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA15384 for ; Mon, 21 May 2001 21:37:52 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Mon, 21 May 2001 19:43:40 -0700 Message-ID: <000b01c0e268$84704400$0200a8c0@pacbell.net> From: "Slick" To: Date: Mon, 21 May 2001 19:40:08 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I still can't get metakit to work. The same error even in a new MFC app: Linking... Demo1Doc.obj : error LNK2001: unresolved external symbol "public: __thiscall c4_Property::c4_Property(char,char const *)" (??0c4_Property@@QAE@DPBD@Z) Demo1Doc.obj : error LNK2001: unresolved external symbol "public: __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) Debug/Demo1.exe : fatal error LNK1120: 2 unresolved externals Error executing link.exe. Demo1.exe - 3 error(s), 0 warning(s) Any further ideas? What can I check for? From j.p.fletcher@aston.ac.uk Tue May 22 09:37:12 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA01147; Tue, 22 May 2001 09:37:10 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.16 #1) id 152DHY-0001Gh-00; Tue, 22 May 2001 15:37:08 +0100 From: "John Fletcher" To: Date: Tue, 22 May 2001 15:34:57 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT CC: Jean-Claude Wippler Priority: normal In-reply-to: <19341204142430.7498@216.110.36.111> References: <20010109183103.7EBCA24D57@mail> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Subject: [Metakit] Re: [MK-Core] Metakit for Mingw32 (Dev-C++) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler To: metakit-core Subject: Re: [MK-Core] Metakit for Mingw32 (Dev-C++) Date sent: Tue, 9 Jan 2001 12:52:46 -0800 > >From: Darel Finkbeiner > [...] > >I've compiled the library with the Mingw32 compiler that comes with Dev-C++ > >(gcc-2.95.2 on windows). I used what was in the 2.3.3-28 and it seems to > >work ok so far. > > > >How should I make this available for people to use? Or who should I send > >it to to let the project put it up on their site? > > > >I also can include written instructions for integrated with the Dev-C++ IDE. > > > >Darel Finkbeiner > > Hello Darel, > > I'll be happy to add such a download to the http://www.equi4.com/mk/ > tree, binary as well as source+instructions if you like. Alternately, > you can put it somewhere else and I could list it on the http:// > www.equi4.com/metakit/ homepage. Whichever is most convenient for you. > > Thanks for letting us know... > > -jcw > > _______________________________________________ > metakit-core@equi4.com - http://www.equi4.com/metakit/ > http://lists.sourceforge.net/mailman/listinfo/metakit-core Hello Did the mingw32 version get loaded up somewhere? If so, where please. John Fletcher > ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From cer@transelec.cl Tue May 22 20:12:40 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA17852; Tue, 22 May 2001 20:12:39 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id VAA14865; Tue, 22 May 2001 21:08:02 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Tue, 22 May 2001 21:15:19 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: Jean-Claude Wippler , metakit@equi4.com Subject: RE: [Metakit] Python 2.1 & Mk4py.dll Date: Tue, 22 May 2001 08:53:32 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi >The 70 Kb size is due to UPX compression, see > http://wildsau.idv.uni-linz.ac.at/mfx/upx.html I'am Mk4py user and I think metakit is great, but I have a problem with exe or dll compressors like UPX. Jordan Russell, the author of Inno Setup Installer, write the next about exe compressors, and I like to know if you agree or not: (http://www.jrsoftware.org/striprlc.htm) Why not use an EXE compressor? Some have asked why I made StripReloc when there are EXE compression programs such as ASPack and UPX that will trim more bytes off of executables than StripReloc ever could. The reason is there are downsides to using EXE compressors. Most notably: * Upon startup of a compressed EXE/DLL, all of the code is decompressed from the disk image into memory in one pass, which can cause disk thrashing if the system is low on memory and is forced to access the swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates memory for code pages on demand (i.e. when they are executed). * Multiple instances of a compressed EXE/DLL create multiple instances of the code in memory. (So if your EXE is 1 MB and the user starts 5 instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the code is only stored in memory once and shared between instances. * Some older virus scanners flag compressed EXE/DLLs as being virus-infected. (This is this reason I stopped compressing Inno Setup's EXEs.) From Slick@battleop.net Wed May 23 00:23:25 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id AAA19415 for ; Wed, 23 May 2001 00:23:24 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Tue, 22 May 2001 22:29:11 -0700 Message-ID: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> From: "Slick" To: Date: Tue, 22 May 2001 22:25:40 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Building DLL and LIB files Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm still trying to work out why I can't get Metakit to work. In some of the documentation I read : _______ The MS Visual C++ 6.0 project is "win/msvc60/mksrc.dsw", with subprojects for the C++ demo (mkdemo), building dll's (mkdll), static libs (mklib), regression tests (mktest), as well as Tcl (mktcl) and Python (mkpython). It has been set up to place all intermediate files and final results in the "builds/" subdirectory, even though you'll launch it from "win/". To build with STL containers and strings under MSVC, define "q4_STD". To build with MFC containers and strings under MSVC, define "q4_MFC". _______ If I am using MFC where do I put the define command before building the dll and libs? I tried in the config.h file at the top with various combinations such as : #define q4_MFC but i get errors like this: C:\Metakit\metakit-2.3.4-29\src\header.h(45) : fatal error C1017: invalid integer constant expression custom.cpp C:\Metakit\metakit-2.3.4-29\src\header.h(45) : fatal error C1017: invalid integer constant expression derived.cpp Does anyone have v2.3.4 working well and could simply explain how they got the source to build dll's and lib's that actually work? Thanks for the help. I may have to abandon trying to get it to work since I am not a guru and can't work this out. From Slick@battleop.net Wed May 23 01:48:52 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA20017 for ; Wed, 23 May 2001 01:48:51 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Tue, 22 May 2001 23:54:38 -0700 Message-ID: <004301c0e354$bdea97c0$0200a8c0@pacbell.net> From: "Slick" To: Date: Tue, 22 May 2001 23:51:06 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] For others potentially having problems v2.3.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I don't know if I am the only one who has had problems getting Metakit up and running. Jean_Claude I can build and run both mkdemo and mktest. Mkdemo says the country of paco pena is spain and mktest runs through a whole series of tests that appear to work with the exception of e05 it says failed a(s == "hi!hello"). Most of them scroll too quickly but i think they worked. Then if I create a new MFC sdi app to keep it simple and add : c4_StringProp pNew ("Hey"); plus #include "mk4.h" //mine I get errors: --------------------Configuration: mktest2 - Win32 Debug-------------------- Compiling resources... Compiling... StdAfx.cpp Compiling... mktest2.cpp MainFrm.cpp mktest2Doc.cpp mktest2View.cpp Generating Code... Linking... mktest2View.obj : error LNK2001: unresolved external symbol "public: __thiscall c4_Property::c4_Property(char,char const *)" (??0c4_Property@@QAE@DPBD@Z) mktest2View.obj : error LNK2001: unresolved external symbol "public: __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) Debug/mktest2.exe : fatal error LNK1120: 2 unresolved externals Error executing link.exe. mktest2.exe - 3 error(s), 0 warning(s) Oh well... From spahievi@vega.bg Wed May 23 02:17:10 2001 Received: from moria.vega.bg (moria.vega.bg [217.197.128.1]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA20430 for ; Wed, 23 May 2001 02:17:08 -0500 Received: from b1-dial-pool172.vega.bg (b1-dial-pool172.vega.bg [217.197.129.172]) by moria.vega.bg (8.9.1a/8.9.1) with ESMTP id JAA01459 for ; Wed, 23 May 2001 09:17:24 +0300 Date: Wed, 23 May 2001 09:45:57 +0300 From: Niki Spahiev X-Mailer: The Bat! (v1.52) UNREG / FE6CDE745D9DC872 Reply-To: spahievi X-Priority: 3 (Normal) Message-ID: <482270334.20010523094557@vega.bg> To: metakit@equi4.com Subject: Re: [Metakit] Building DLL and LIB files In-Reply-To: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> References: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Slick, Wednesday, May 23, 2001, 8:25:40 AM, you wrote: [...] S> #define q4_MFC #define q4_MFC 1 -- regards, Niki Spahiev From spahievi@vega.bg Wed May 23 02:17:13 2001 Received: from moria.vega.bg (moria.vega.bg [217.197.128.1]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA20434 for ; Wed, 23 May 2001 02:17:11 -0500 Received: from b1-dial-pool172.vega.bg (b1-dial-pool172.vega.bg [217.197.129.172]) by moria.vega.bg (8.9.1a/8.9.1) with ESMTP id JAA01486 for ; Wed, 23 May 2001 09:17:29 +0300 Date: Wed, 23 May 2001 09:46:14 +0300 From: Niki Spahiev X-Mailer: The Bat! (v1.52) UNREG / FE6CDE745D9DC872 Reply-To: spahievi X-Priority: 3 (Normal) Message-ID: <772287551.20010523094614@vega.bg> To: metakit@equi4.com Subject: Re[2]: [Metakit] Python 2.1 & Mk4py.dll In-Reply-To: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> References: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Echeverria, Tuesday, May 22, 2001, 3:53:32 PM, you wrote: ERCRTSA> Hi >>The 70 Kb size is due to UPX compression, see >> http://wildsau.idv.uni-linz.ac.at/mfx/upx.html ERCRTSA> I'am Mk4py user and I think metakit is great, but I have a problem with exe ERCRTSA> or dll compressors like UPX. ERCRTSA> Jordan Russell, the author of Inno Setup Installer, write the next about exe ERCRTSA> compressors, and I like to know if you agree or not: ERCRTSA> (http://www.jrsoftware.org/striprlc.htm) ERCRTSA> Why not use an EXE compressor? ERCRTSA> Some have asked why I made StripReloc when there are EXE compression ERCRTSA> programs such as ASPack and UPX that will trim more bytes off of executables ERCRTSA> than StripReloc ever could. ERCRTSA> The reason is there are downsides to using EXE compressors. Most notably: ERCRTSA> * Upon startup of a compressed EXE/DLL, all of the code is ERCRTSA> decompressed from the disk image into memory in one pass, which can cause ERCRTSA> disk thrashing if the system is low on memory and is forced to access the ERCRTSA> swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates memory ERCRTSA> for code pages on demand (i.e. when they are executed). ERCRTSA> * Multiple instances of a compressed EXE/DLL create multiple instances ERCRTSA> of the code in memory. (So if your EXE is 1 MB and the user starts 5 ERCRTSA> instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the ERCRTSA> code is only stored in memory once and shared between instances. ERCRTSA> * Some older virus scanners flag compressed EXE/DLLs as being ERCRTSA> virus-infected. (This is this reason I stopped compressing Inno Setup's ERCRTSA> EXEs.) A agree with this. Also it's impossible to extract dependency info from compressed DLLs. -- regards, Niki Spahiev From cer@transelec.cl Wed May 23 07:55:51 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA27756 for ; Wed, 23 May 2001 07:55:50 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id IAA26237; Wed, 23 May 2001 08:50:29 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Wed, 23 May 2001 08:57:47 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C045C9A11@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: spahievi Cc: metakit@equi4.com Subject: RE: Re[2]: [Metakit] Python 2.1 & Mk4py.dll Date: Wed, 23 May 2001 08:57:25 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It must be said that the same UPX compressor can be used to uncompress exes and dlls compressed with it. (This is the main reason I have it on my PC) Cristian Echeverria > -----Original Message----- > From: Niki Spahiev [SMTP:spahievi@vega.bg] > Sent: Wednesday, May 23, 2001 2:46 AM > To: metakit@equi4.com > Subject: Re[2]: [Metakit] Python 2.1 & Mk4py.dll > > Hello Echeverria, > > Tuesday, May 22, 2001, 3:53:32 PM, you wrote: > > ERCRTSA> Hi > > >>The 70 Kb size is due to UPX compression, see > >> http://wildsau.idv.uni-linz.ac.at/mfx/upx.html > > ERCRTSA> I'am Mk4py user and I think metakit is great, but I have a > problem with exe > ERCRTSA> or dll compressors like UPX. > ERCRTSA> Jordan Russell, the author of Inno Setup Installer, write the > next about exe > ERCRTSA> compressors, and I like to know if you agree or not: > ERCRTSA> (http://www.jrsoftware.org/striprlc.htm) > > > ERCRTSA> Why not use an EXE compressor? > ERCRTSA> Some have asked why I made StripReloc when there are EXE > compression > ERCRTSA> programs such as ASPack and UPX that will trim more bytes off of > executables > ERCRTSA> than StripReloc ever could. > ERCRTSA> The reason is there are downsides to using EXE compressors. Most > notably: > ERCRTSA> * Upon startup of a compressed EXE/DLL, all of the code is > ERCRTSA> decompressed from the disk image into memory in one pass, which > can cause > ERCRTSA> disk thrashing if the system is low on memory and is forced to > access the > ERCRTSA> swap file. In contrast, with uncompressed EXE/DLLs, the OS > allocates memory > ERCRTSA> for code pages on demand (i.e. when they are executed). > ERCRTSA> * Multiple instances of a compressed EXE/DLL create > multiple instances > ERCRTSA> of the code in memory. (So if your EXE is 1 MB and the user > starts 5 > ERCRTSA> instances of it, that adds up to 5+ MB.) With uncompressed > EXE/DLLs, the > ERCRTSA> code is only stored in memory once and shared between instances. > ERCRTSA> * Some older virus scanners flag compressed EXE/DLLs as > being > ERCRTSA> virus-infected. (This is this reason I stopped compressing Inno > Setup's > ERCRTSA> EXEs.) > > A agree with this. Also it's impossible to extract dependency info from > compressed DLLs. > > -- > regards, > Niki Spahiev From msackett@yin.interaccess.com Wed May 23 09:39:24 2001 Received: from postal.interaccess.com (from.interaccess.com [207.208.131.20]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA31296 for ; Wed, 23 May 2001 09:39:24 -0500 Received: from yin.interaccess.com (yin.interaccess.com [207.208.45.69]) by postal.interaccess.com (8.10.2/8.10.2) with SMTP id f4NEcFm13404 for ; Wed, 23 May 2001 09:38:15 -0500 (CDT) Received: (qmail 11193 invoked by uid 28828); 23 May 2001 14:38:03 -0000 Date: Wed, 23 May 2001 09:38:03 -0500 (CDT) From: Mike Sackett To: Slick cc: metakit@equi4.com Subject: Re: [Metakit] For others potentially having problems v2.3.4 In-Reply-To: <004301c0e354$bdea97c0$0200a8c0@pacbell.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I don't know enough about Metakit to be able to comment on the specific issue you're running into, but I do have a general suggestion: Since you appear to be running Visual C++, you can use the dumpbin utility to get a dump of the symbols defined by a particular module or library. In the case of the error you describe, it's apparent that mktest2View.obj is attempting to import variations of the c4_Property constructor which are not being provided by any module available to the linker. This often means, as I'm sure you're aware, that the build options supplied when the library was built differ in some way from those used to build the referencing module. At this point, it's often helpful to use your compiler's command line options to get a listing of the preprocessed source. For example, if you use the /E or /P options when compiling mktest2View, you can see how *it* is interpreting Metakit's headers. You can then look at the headers themselves and try to figure out the variations and what affects them. It's painful and slow, but in the end you have all the tools you need on your machine to figure out what's going on. I think one of the lessons most people learn after a while is to be very careful about build flags, preprocessor flags, etc., when combining libraries. I tend to look at this kind of debugging as being somewhat of a fixed cost that I have to pay every time I add a new, complex library to an existing build. ..Mike On Tue, 22 May 2001, Slick wrote: > I don't know if I am the only one who has had problems getting Metakit up > and running. > > Jean_Claude I can build and run both mkdemo and mktest. Mkdemo says the > country of paco pena is spain and mktest runs through a whole series of > tests that appear to work with the exception of e05 it says failed a(s == > "hi!hello"). Most of them scroll too quickly but i think they worked. > > Then if I create a new MFC sdi app to keep it simple and add : > > c4_StringProp pNew ("Hey"); > > plus > > #include "mk4.h" //mine > > I get errors: > > --------------------Configuration: mktest2 - Win32 Debug-------------------- > Compiling resources... > Compiling... > StdAfx.cpp > Compiling... > mktest2.cpp > MainFrm.cpp > mktest2Doc.cpp > mktest2View.cpp > Generating Code... > Linking... > mktest2View.obj : error LNK2001: unresolved external symbol "public: > __thiscall c4_Property::c4_Property(char,char const *)" > (??0c4_Property@@QAE@DPBD@Z) > mktest2View.obj : error LNK2001: unresolved external symbol "public: > __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) > Debug/mktest2.exe : fatal error LNK1120: 2 unresolved externals > Error executing link.exe. > > mktest2.exe - 3 error(s), 0 warning(s) > > Oh well... > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jcw@equi4.com Wed May 23 17:11:33 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA14700 for ; Wed, 23 May 2001 17:11:33 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Building DLL and LIB files Date: Wed, 23 May 2001 15:01:15 -0700 Message-Id: <20010523220115.18616@smtp.telus.net> In-Reply-To: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> References: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: >I'm still trying to work out why I can't get Metakit to work. I'm sorry that this is causing you so much trouble. MSVC can be quite fussy about things w.r.t. mixing pre-compiled code. I've set up a sample project with a complete snapshot and information on what was needed to make it build and run properly. It's at: http://www.equi4.com/pub/mk/metakit-2.3.4-29-win/ -jcw From jcw@equi4.com Wed May 23 17:12:51 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA14778 for ; Wed, 23 May 2001 17:12:50 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Python 2.1 & Mk4py.dll Date: Wed, 23 May 2001 15:12:41 -0700 Message-Id: <20010523221241.1770@smtp.telus.net> In-Reply-To: <6917523888.20010518184017@usd1.com> References: <6917523888.20010518184017@usd1.com> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry McRae wrote: >Just installed Python 2.1. The version of Mk4py.dll I have been using >is dated 11/23/00 and is 221k. Python didn't like it loading >python20.dll, so I downloaded the newest Mk4py today. I've added a build for use with Python 2.1, see: http://www.equi4.com/pub/mk/metakit-2.3.4-29-win/py21/ -jcw From jcw@equi4.com Wed May 23 17:16:53 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA14957 for ; Wed, 23 May 2001 17:16:52 -0500 From: Jean-Claude Wippler To: Subject: RE: [Metakit] Python 2.1 & Mk4py.dll Date: Wed, 23 May 2001 15:16:41 -0700 Message-Id: <20010523221641.21016@smtp.telus.net> In-Reply-To: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> References: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. wrote: >>The 70 Kb size is due to UPX compression, see >> http://wildsau.idv.uni-linz.ac.at/mfx/upx.html First off, please allow me to (respectfully) disagree on every issue... >* Upon startup of a compressed EXE/DLL, all of the code is >decompressed from the disk image into memory in one pass, which can cause >disk thrashing if the system is low on memory and is forced to access the >swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates memory >for code pages on demand (i.e. when they are executed). That should not matter with a tiny package such as MetaKit. >* Multiple instances of a compressed EXE/DLL create multiple instances >of the code in memory. (So if your EXE is 1 MB and the user starts 5 >instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the >code is only stored in memory once and shared between instances. I'm no expert on this, but I'd assume that this not the case if the exe is shared as well (which is standard on Win32, AFAIK). >* Some older virus scanners flag compressed EXE/DLLs as being >virus-infected. (This is this reason I stopped compressing Inno Setup's >EXEs.) By now, I would hope that these older scanners have been updated? Anyway, as you noted, the compression is reversible. And in rare burst of trying to be maximally helpful, I've rebuilt and replaced "mk4vc60.dll" :) -jcw From cer@transelec.cl Wed May 23 17:22:29 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA15221 for ; Wed, 23 May 2001 17:22:28 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id SAA08433; Wed, 23 May 2001 18:17:13 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Wed, 23 May 2001 18:24:33 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C04679B11@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: spahievi@vega.bg Cc: metakit@equi4.com Subject: FW: [Metakit] Python 2.1 & Mk4py.dll Date: Wed, 23 May 2001 18:24:30 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I agree with you on point 1 and 3, this is not relevant, but point 2 is really important and Niki Spahiev has pointed other problems. Niki can you help with this? > -----Original Message----- > From: Jean-Claude Wippler [SMTP:jcw@equi4.com] > Sent: Wednesday, May 23, 2001 6:10 PM > To: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. > Subject: RE: [Metakit] Python 2.1 & Mk4py.dll > > Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. > wrote: > > >>The 70 Kb size is due to UPX compression, see > >> http://wildsau.idv.uni-linz.ac.at/mfx/upx.html > > First off, please allow me to (respectfully) disagree on every issue... > > >* Upon startup of a compressed EXE/DLL, all of the code is > >decompressed from the disk image into memory in one pass, which can cause > >disk thrashing if the system is low on memory and is forced to access the > >swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates > memory > >for code pages on demand (i.e. when they are executed). > > That should not matter with a tiny package such as MetaKit. > > >* Multiple instances of a compressed EXE/DLL create multiple instances > >of the code in memory. (So if your EXE is 1 MB and the user starts 5 > >instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the > >code is only stored in memory once and shared between instances. > > I'm no expert on this, but I'd assume that this not the case if the exe > is shared as well (which is standard on Win32, AFAIK). > > >* Some older virus scanners flag compressed EXE/DLLs as being > >virus-infected. (This is this reason I stopped compressing Inno Setup's > >EXEs.) > > By now, I would hope that these older scanners have been updated? > > Anyway, as you noted, the compression is reversible. And in rare burst > of trying to be maximally helpful, I've rebuilt and replaced "mk4vc60.dll" > :) > > -jcw From Slick@battleop.net Wed May 23 21:26:21 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA19181 for ; Wed, 23 May 2001 21:26:20 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Wed, 23 May 2001 19:32:04 -0700 Message-ID: <002701c0e3f9$3cf624e0$0200a8c0@pacbell.net> From: "Slick" To: Date: Wed, 23 May 2001 19:28:37 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking errors Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I downloaded mksample and I can get it to work. That is the good news. However, most of the time I use debug builds to check things out. I tried also copying the mk4vc60_d.dll and lib files that I built from the 2.3.4 release but I get the same linking errors that I was getting before. Do you not post the debug libs and dlls to your download site? From spahievi@vega.bg Thu May 24 03:56:28 2001 Received: from moria.vega.bg (moria.vega.bg [217.197.128.1]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA21503 for ; Thu, 24 May 2001 03:56:26 -0500 Received: from b1-dial-pool27.vega.bg (b1-dial-pool27.vega.bg [217.197.129.27]) by moria.vega.bg (8.9.1a/8.9.1) with ESMTP id KAA02171 for ; Thu, 24 May 2001 10:56:37 +0300 Date: Thu, 24 May 2001 11:11:02 +0300 From: Niki Spahiev X-Mailer: The Bat! (v1.52) UNREG / FE6CDE745D9DC872 Reply-To: spahievi X-Priority: 3 (Normal) Message-ID: <23276972.20010524111102@vega.bg> To: metakit@equi4.com Subject: Re[2]: [Metakit] Python 2.1 & Mk4py.dll In-Reply-To: <20010523221641.21016@smtp.telus.net> References: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> <20010523221641.21016@smtp.telus.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >>* Multiple instances of a compressed EXE/DLL create multiple instances >>of the code in memory. (So if your EXE is 1 MB and the user starts 5 >>instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the >>code is only stored in memory once and shared between instances. JCW> I'm no expert on this, but I'd assume that this not the case if the exe JCW> is shared as well (which is standard on Win32, AFAIK). Sharing uses memory mapped files which is impossible with packed DLLs. -- regards, Niki Spahiev From lvirden@cas.org Thu May 24 14:20:07 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA07624 for ; Thu, 24 May 2001 14:20:07 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id PAA13512 for ; Thu, 24 May 2001 15:18:50 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f4OJIn318598; Thu, 24 May 2001 15:18:49 -0400 (EDT) Date: Thu, 24 May 2001 15:18:49 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010524151849.AAB18578@cas.org> To: metakit@equi4.com Subject: [Metakit] White papers/technical papers/etc. on Tclkit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Besides the 'ancient' seminal paper JC presented at the last Tcl Conference, and an upcoming conference paper by Steve Landers, I am unaware of other papers written concerning TclKit. I know that one esteemed journalist has written a column or two about it; however I was wondering if anyone knew of other papers detailing their work with the idea. If there is a better mailing list on which to ask this question, please let me know. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From rmenegon@bigpond.net.au Fri May 25 05:31:55 2001 Received: from mailin1.bigpond.com (juicer13.bigpond.com [139.134.6.21]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA22045 for ; Fri, 25 May 2001 05:31:54 -0500 Received: from unspecified.host ([139.134.4.58]) by mailin1.bigpond.com (Netscape Messaging Server 4.15) with SMTP id GDW04S00.IM1 for ; Fri, 25 May 2001 20:36:28 +1000 Received: from CPE-144-132-187-24.nsw.bigpond.net.au ([144.132.187.24]) by mail1.bigpond.com (Claudes-Perennial-MailRouter V2.9c 1/8606054); 25 May 2001 20:30:50 Received: from 192.168.11.13 ([192.168.11.13]) by 192.168.11.1 (WinRoute Pro 4.1) with SMTP; Fri, 25 May 2001 20:32:19 +1000 Received: by localhost with Microsoft MAPI; Fri, 25 May 2001 20:31:26 +1000 Message-ID: <01C0E559.ABFF76F0.rmenegon@bigpond.net.au> From: Rob Menegon Reply-To: "rmenegon@bigpond.net.au" To: "'metakit@equi4.com'" Date: Fri, 25 May 2001 20:31:25 +1000 Organization: MenMac X-Mailer: Microsoft Internet E-mail/MAPI - 8.0.0.4211 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] Synchronisation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Does anyone have any advice on file sharing mechanisms? I have a scenario in which multiple sources need to update the same metakit database. At present I am leaning towards all the process posting the updates to a queue and giving another process the responsibility of clearing the queue and updating the metakit database file. Any thought on the matter wold be greatly received. Rob _______________________________________ Rob Menegon 349 Somerville Road Hornsby Heights NSW 2077 Sydney, Australia (Mobile) +61 (0)416 271 348 ( Voice) +61 (0)2 9477 1348 ( Fax) +61 (0)2 8570 2031 rmenegon@bigpond.net.au _______________________________________ This email contains information intended for the addressee only. This information is confidential and may be privileged. If you are not the intended recipient, you must not read, use, copy or distribute this email or its attachments. If you have received this in error, please notify me immediately by return email or telephone, and delete and destroy this email. Thank you. _______________________________________ From claird@starbase.neosoft.com Fri May 25 07:42:24 2001 Received: from mail3.iadfw.net (mail3.iadfw.net [209.196.123.3]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id HAA24427 for ; Fri, 25 May 2001 07:42:24 -0500 Received: from starbase.neosoft.com from [206.109.1.32] by mail3.iadfw.net (/\##/\ Smail3.1.30.16 #30.25) with esmtp for sender: id ; Fri, 25 May 2001 07:42:52 -0500 (CDT) Received: (from claird@localhost) by starbase.neosoft.com (8.9.3/8.9.3) id HAA20822; Fri, 25 May 2001 07:54:24 -0500 (CDT) (envelope-from claird) Date: Fri, 25 May 2001 07:54:24 -0500 (CDT) From: Cameron Laird Message-Id: <200105251254.HAA20822@starbase.neosoft.com> To: metakit@equi4.com, rmenegon@bigpond.net.au Subject: Re: [Metakit] Synchronisation In-Reply-To: <01C0E559.ABFF76F0.rmenegon@bigpond.net.au> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >From metakit-admin@equi4.com Fri May 25 05:48:13 2001 > . > . > . >Does anyone have any advice on file sharing mechanisms? I have a scenario >in which multiple sources need to update the same metakit database. At >present I am leaning towards all the process posting the updates to a queue >and giving another process the responsibility of clearing the queue and >updating the metakit database file. > . > . > . Tequila might fit your needs perfectly. Tequila is a TclKit "add-on" Jean-Claude wrote last year. From g.lancaster@mbf.com.au Fri May 25 18:18:59 2001 Received: from fs1-smtp (fs1-smtp.mbf.com.au [203.101.28.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA08806 for ; Fri, 25 May 2001 18:18:56 -0500 Received: from MBF-INT1-Message_Server by fs1-smtp with Novell_GroupWise; Sat, 26 May 2001 09:04:08 +1000 Message-Id: X-Mailer: Novell GroupWise 5.5.2 Date: Sat, 26 May 2001 09:14:47 +1000 From: "Garth Lancaster" To: , Subject: Re: [Metakit] Synchronisation Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline X-Guinevere: 1.0.13 ; Medical Benefits Fun Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id SAA08806 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I agree with the theory of what Rob said, "all the process posting the updates to a queue and giving another process the responsibility of clearing the queue and updating the metakit database file." and Tequila might be ok Cameron, except Im not a Tcl kinda guy and I dont know if Rob is either... (but hey, thanks, and each to his own poison :-) ) When I asked about this a while back , JCW suggested that I need a central Metakit Server handling all the requests, locking etc, from possibly multiple clients, which sounded pretty plausible/do-able.. so its likely that such a server would be queue based - he also pointed me to the 'FAQ'/Discussion pages at the MetaKit Wiki :- http://www.equi4.com/metakit/wiki.cgi/ which had some notes on locking etc ... Cameron - you know more about Tequila than I do - is it possible to 'simulate' its 'mode of operation' from C++ do you think, or a different approach required ? (I'll start looking at it in my free time - thats usually between 10pm and 5am !! ) If anyone (Rob included) actually has implementation ideas, why dont we discuss them on the Metakit Wiki, so we can share them ? Garth Garth Lancaster MBF InfoTech (Infrastructure) Integration Engineer g.lancaster@mbf.com.au Ph : +61 2 9323 9534 Fax : +61 2 9267 7359 From g.lancaster@mbf.com.au Fri May 25 18:23:20 2001 Received: from fs1-smtp (fs1-smtp.mbf.com.au [203.101.28.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA08954 for ; Fri, 25 May 2001 18:23:18 -0500 Received: from MBF-INT1-Message_Server by fs1-smtp with Novell_GroupWise; Sat, 26 May 2001 09:08:31 +1000 Message-Id: X-Mailer: Novell GroupWise 5.5.2 Date: Sat, 26 May 2001 09:19:12 +1000 From: "Garth Lancaster" To: , , Subject: Re: [Metakit] Synchronisation Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline X-Guinevere: 1.0.13 ; Medical Benefits Fun Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id SAA08954 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The link to Tequila on the Metakit Wiki is broken - but I see at http://www.equi4.com/tequila/ "Tequila is now part of the MetaKit open source distribution." 'G' From rvcoulman@acm.org Fri May 25 23:51:41 2001 Received: from mail.dlcwest.com (backup.dlcwest.com [204.83.37.9]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA13315 for ; Fri, 25 May 2001 23:51:40 -0500 Received: (qmail 20669 invoked by uid 0); 26 May 2001 04:20:02 -0000 Received: from unknown (HELO acm.org) (204.83.37.45) by backup.dlcwest.com with SMTP; 26 May 2001 04:20:02 -0000 Message-ID: <3B0F3733.A8058F5F@acm.org> Date: Fri, 25 May 2001 22:55:15 -0600 From: Randy & Val Coulman X-Mailer: Mozilla 4.7 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Slick CC: metakit@equi4.com Subject: Re: [Metakit] For others potentially having problems v2.3.4 References: <004301c0e354$bdea97c0$0200a8c0@pacbell.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: > > I don't know if I am the only one who has had problems getting Metakit up > and running. > > > --------------------Configuration: mktest2 - Win32 Debug-------------------- > Compiling resources... > Compiling... > StdAfx.cpp > Compiling... > mktest2.cpp > MainFrm.cpp > mktest2Doc.cpp > mktest2View.cpp > Generating Code... > Linking... > mktest2View.obj : error LNK2001: unresolved external symbol "public: > __thiscall c4_Property::c4_Property(char,char const *)" > (??0c4_Property@@QAE@DPBD@Z) > mktest2View.obj : error LNK2001: unresolved external symbol "public: > __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) > Debug/mktest2.exe : fatal error LNK1120: 2 unresolved externals > Error executing link.exe. > > mktest2.exe - 3 error(s), 0 warning(s) > This suggestion is so simple and obvious that I can't believe no one has suggested it yet, but are you sure you're linking your application with the Metakit library? You say you've built a whole bunch of versions of the library, and that you have all of your paths set up, but are you actually telling the linker to use it? If not, this would be your problem. Just go on the Link tab in the Project Settings and add the Metakit library you want to the Object/library modules edit box. Randy -- Randy and Val Coulman rvcoulman@acm.org From Slick@battleop.net Sat May 26 01:19:35 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA14691 for ; Sat, 26 May 2001 01:19:34 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Fri, 25 May 2001 23:26:09 -0700 Message-ID: <001001c0e5ac$212e6c40$0200a8c0@pacbell.net> From: "Slick" To: Date: Fri, 25 May 2001 23:21:41 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking errors Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thank you all for your help. I now have it working. I am somewhat of a newbie in some areas and better in others. I had previously added another 3rd party class just by specifying the lib path under the tools | options menu. I assumed that metakit would work the same way. Probably a newbie mistake but it wouldn't have hurt to specify you have to type the lib name in also in another area. Maybe I've learned something and that is how you are supposed to do it. Anyway, I have it working (including different builds :)). My next problem is good examples and docs. For example I need a code snippet to show me how to find a row in a view based upon a prop value. I am trying things like: int x = vServer.Find(pServer "Two"); but I can't work it out. Are there any samples other than what is posted on the metakit site? I have achieved loading, saving and deleting rows. Also, out of curiosity when I get to the point of concurrent access is it easiest to do a batch to write out to the database? I assume you can't manipulate the database in different threads at the same time (my ultimate goal but a way off yet). From cristi_clot@hotmail.com Sat May 26 09:06:36 2001 Received: from hotmail.com (f49.law8.hotmail.com [216.33.241.49]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA22069 for ; Sat, 26 May 2001 09:06:26 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sat, 26 May 2001 06:52:48 -0700 Received: from 217.10.194.6 by lw8fd.law8.hotmail.msn.com with HTTP; Sat, 26 May 2001 13:52:48 GMT X-Originating-IP: [217.10.194.6] From: "Clot Cristian" To: metakit@equi4.com Date: Sat, 26 May 2001 13:52:48 -0000 Mime-Version: 1.0 Content-Type: text/html Message-ID: X-OriginalArrivalTime: 26 May 2001 13:52:48.0441 (UTC) FILETIME=[26068290:01C0E5EB] Subject: [Metakit] Newcomer at metakit. Hello to all of you! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive:
Hi, I'm a new member of your community, and I found MetaKit a very very good engine.
I am from Bucharest, Romania.
I'm using MetaKit for 2 months and I have 2 questions for you, if you are so kind to
answer me.
 
1. I tried to import some .dbf IV files of about 2 - 3 milion records in MetaKit, but (depending on size of the table, number of fields, etc.), the engine crashes when it reach  aprox. the 1.200.000'th record. I also use Jean-Claude's program for importing dbf files, but the same thing happens. Why ? How can I get over it ?
2. When I make a new sorted view on a Character field, and I go "Search" command, the found record is not the real one, but 3 or 4 records bellow or above. When I create the view, I  create it dinamically and access it through a pointer, and when I destroy it, the engine crashes. Why ? When I try to read records from this view, I also get a crush. This happens only on Character fields. can you tell me how can I make it work, please ?
 
Thank you very much,
Cristi CLOT
 
 
P.S. Jean-Claude, you're the best and I really, really apreciate what you're doing for internet and programmers community. Thanks.

Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

From cristi_clot@hotmail.com Sat May 26 09:12:34 2001 Received: from hotmail.com (f153.law8.hotmail.com [216.33.241.153]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA22132 for ; Sat, 26 May 2001 09:12:34 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sat, 26 May 2001 06:59:25 -0700 Received: from 217.10.194.6 by lw8fd.law8.hotmail.msn.com with HTTP; Sat, 26 May 2001 13:59:25 GMT X-Originating-IP: [217.10.194.6] From: "Clot Cristian" To: metakit@equi4.com Date: Sat, 26 May 2001 13:59:25 -0000 Mime-Version: 1.0 Content-Type: text/html Message-ID: X-OriginalArrivalTime: 26 May 2001 13:59:25.0886 (UTC) FILETIME=[12EBCDE0:01C0E5EC] Subject: [Metakit] Re about file sharing mechanism to Rob MENEGON Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive:
Did you try a client-server architecture ? I use oneright now and it works very good.

Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

From nicolasb@maich.gr Mon May 28 03:53:14 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id DAA15977 for ; Mon, 28 May 2001 03:53:12 -0500 Received: FROM maich.gr BY mail-server ; Mon May 28 11:58:48 2001 +0300 Message-ID: <3B1214D2.4DEF5E41@maich.gr> Date: Mon, 28 May 2001 12:05:22 +0300 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: Clot Cristian CC: metakit@equi4.com Subject: Re: [Metakit] Re about file sharing mechanism to Rob MENEGON References: Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, What exactly are you doing? Nicolas Boretos Clot Cristian wrote: > > Did you try a client-server architecture ? I use oneright now and it > works very good. > > ---------------------------------------------------------------------- > Get Your Private, Free E-mail from MSN Hotmail at > http://www.hotmail.com. > > _______________________________________________ metakit mailing list - > metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From rmenegon@bigpond.net.au Mon May 28 13:48:09 2001 Received: from mailin2.email.bigpond.com (juicer14.bigpond.com [139.134.6.23]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA08570 for ; Mon, 28 May 2001 13:48:07 -0500 Received: from unspecified.host ([144.135.24.75]) by mailin2.email.bigpond.com (Netscape Messaging Server 4.15) with SMTP id GE2L0900.G9L; Tue, 29 May 2001 09:52:57 +1000 Received: from CPE-144-132-187-24.nsw.bigpond.net.au ([144.132.187.24]) by bwmam03.mailsvc.email.bigpond.com(MailRouter V2.9d 8323/935193); 29 May 2001 09:52:56 Received: from 192.168.11.202 ([192.168.11.202]) by 192.168.11.1 (WinRoute Pro 4.1) with SMTP; Tue, 29 May 2001 09:48:22 +1000 Received: by localhost with Microsoft MAPI; Tue, 29 May 2001 09:48:15 +1000 Message-ID: <01C0E824.7B842B60.rmenegon@bigpond.net.au> From: Rob Menegon Reply-To: "rmenegon@bigpond.net.au" To: "'nicolasb@maich.gr'" , "'cristi_clot@hotmail.com'" Cc: "'metakit@equi4.com'" Date: Tue, 29 May 2001 09:48:14 +1000 Organization: MenMac X-Mailer: Microsoft Internet E-mail/MAPI - 8.0.0.4211 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] about file sharing mechanism to Rob MENEGON (Nicolas Boretos) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas, I am reading data from various sources ... last count this was 6 serial lines but that the sources could be disk files as well. While the data is related, no particular source has the entire picture (it has its own unique piece of the puzzle), however it has enough information to uniquely identify its destination (there is come commonality) ... I need to aggregate the data in order to build the picture. The data is streaming in continuously over an 8 hour period. I am using MetaKit to provide the data aggregation and my original model was one that involved involved each of the readers (1 to 6 in this case) updating the database. This posed the access / sharing issue that I raised earlier. I have sinced modified my stance and introduced a writer process. The readers are responsible for reading and posting data to a queue and the writer is responsible for reading from the queue and updating the database. It give me other real estate and configuration flexibility advantages. By the way the environment is NT and C++ While I have not tested the theory I think that I could have also used the mutex kernel object for mutual exclusion of serveral threads. I am making the assumption that the only time MetaKit does any writing to disk is on the Commit call. This would allow me to permit a thread at a time into this section of code. That is my thinking at least. Rob _______________________________________ Rob Menegon 349 Somerville Road Hornsby Heights NSW 2077 Sydney, Australia (Mobile) +61 (0)416 271 348 ( Voice) +61 (0)2 9477 1348 ( Fax) +61 (0)2 8570 2031 rmenegon@bigpond.net.au _______________________________________ This email contains information intended for the addressee only. This information is confidential and may be privileged. If you are not the intended recipient, you must not read, use, copy or distribute this email or its attachments. If you have received this in error, please notify me immediately by return email or telephone, and delete and destroy this email. Thank you. _______________________________________ From ahmad@u.washington.edu Wed May 30 17:57:20 2001 Received: from mxout2.cac.washington.edu (mxout2.cac.washington.edu [140.142.33.4]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA10210 for ; Wed, 30 May 2001 17:57:20 -0500 Received: from mailhost2.u.washington.edu (mailhost2.u.washington.edu [140.142.33.2]) by mxout2.cac.washington.edu (8.11.2+UW01.01/8.11.2+UW01.04) with ESMTP id f4V3v3o17893 for ; Wed, 30 May 2001 20:57:03 -0700 Received: from cae0 (g-turkiyyah-2.ce.washington.edu [128.95.204.230]) by mailhost2.u.washington.edu (8.11.2+UW01.01/8.11.2+UW01.04) with SMTP id f4V3v2C30341 for ; Wed, 30 May 2001 20:57:02 -0700 Message-ID: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> From: "Ahmad El Husseini" To: Subject: [Metakit] Eficiency problem Date: Wed, 30 May 2001 21:01:31 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello,... I am working on a Structural Modeling software that has Tcl as its main interface and below Tcl there are bunch of solvers that do Dynamic Structural analysis (for earthquakes). Response quantities of objects (displacements, forces, etc) are generated at the end of every timestep. I am using Metakit as my database system where I write data from the C++ classes and extract data from the Tcl Interface. A typical project contains 4000 object and every object has an average of 4 double properties. analysis is done around 1000 times. I tried the following data format alternatives: * every object is a view and at the end of every timestep a row is added to the view. This was EXTREMELY slow. Object(property1, property2, property3, ...) 4000 such views with 1000 row each * I grouped objects into two categories and every category is a view with every object as a subview (formatted the data into columns) and at the end of every timestep a row is added. Category1(c1-Obj1, c1-Obj2,....) c1-Obj1(prop1, prop2, ...) ... Category2(c2-Obj1, c2-Obj2,....) c2-Obj1(prop1, prop2, ...) ... The second alternative proved to be way more efficient than the first one but still extremely slow. I am wondering: first, is Metakit the right tool to use for such a project? second, What is slowing things this much? (it takes minutes to write 0.5 Mb of data, I never exceeded 50 timesteps with only 1000 objects, it simply takes forever) Note: The solvers are very fast (relative to similar analysis programs) I expect to write data out to disk once or twice a second. Any help is greatly appreciated thanks, Ahmad From jcw@equi4.com Wed May 30 18:39:57 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA10505 for ; Wed, 30 May 2001 18:39:55 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Eficiency problem Date: Wed, 30 May 2001 21:39:40 -0700 Message-Id: <20010531043940.32689@smtp.telus.net> In-Reply-To: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> References: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ahmad El Husseini wrote: [...] >* every object is a view and at the end of every timestep a row is added to >the view. This was EXTREMELY slow. >Object(property1, property2, property3, ...) >4000 such views with 1000 row each It will be much faster to preallocate rows (and perhaps truncate when done). Do not append rows in a loop. In itself, 4000 views of 1000 rows of 4 doubles, i.e. 64 Mb, should be fine. But you're probably better off storing things the other way around first: one view per analysis run, with 4000 rows - one per object. You could then later re-open the file and re-organize the other way around. The time-consuming factor is touching all of the 4000 views on each cycle. Metakit is column-wise, so adding even just one row means the entire view gets written on commit. >first, is Metakit the right tool to use for such a project? If you are willing to rethink into columns and try not to commit every single time, yes - I think so. >second, What is slowing things this much? (it takes minutes to write 0.5 Mb >of data, I never exceeded 50 timesteps with only 1000 objects, it simply >takes forever) You're basically rewriting the entire db on each commit, because all views have been altered. >Note: The solvers are very fast (relative to similar analysis programs) I >expect to write data out to disk once or twice a second. -jcw From jyl@best.com Wed May 30 19:20:07 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA10836 for ; Wed, 30 May 2001 19:20:06 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id WAA16879; Wed, 30 May 2001 22:18:35 -0700 (PDT) Message-ID: <3B15D5B9.68FD70DF@best.com> Date: Wed, 30 May 2001 22:25:13 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Ahmad El Husseini CC: metakit@equi4.com Subject: Re: [Metakit] Eficiency problem References: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ahmad Another option is to store things in e4Graph which is built on top of Metakit. Each object could be a node with four "double" vertices. Check out http://e4graph.sourceforge.net. --JYL Ahmad El Husseini wrote: > Hello,... > > I am working on a Structural Modeling software that has Tcl as its main > interface and below Tcl there are bunch of solvers that do Dynamic > Structural analysis (for earthquakes). Response quantities of objects > (displacements, forces, etc) are generated at the end of every timestep. I > am using Metakit as my database system where I write data from the C++ > classes and extract data from the Tcl Interface. > > A typical project contains 4000 object and every object has an average of 4 > double properties. analysis is done around 1000 times. > > I tried the following data format alternatives: > > * every object is a view and at the end of every timestep a row is added to > the view. This was EXTREMELY slow. > Object(property1, property2, property3, ...) > 4000 such views with 1000 row each > > * I grouped objects into two categories and every category is a view with > every object as a subview (formatted the data into columns) and at the end > of every timestep a row is added. > Category1(c1-Obj1, c1-Obj2,....) > c1-Obj1(prop1, prop2, ...) > ... > Category2(c2-Obj1, c2-Obj2,....) > c2-Obj1(prop1, prop2, ...) > ... > The second alternative proved to be way more efficient than the first one > but still extremely slow. > > I am wondering: > first, is Metakit the right tool to use for such a project? > second, What is slowing things this much? (it takes minutes to write 0.5 Mb > of data, I never exceeded 50 timesteps with only 1000 objects, it simply > takes forever) > Note: The solvers are very fast (relative to similar analysis programs) I > expect to write data out to disk once or twice a second. > > Any help is greatly appreciated > thanks, > Ahmad > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From ahmad@u.washington.edu Thu May 31 10:09:12 2001 Received: from mxout1.cac.washington.edu (mxout1.cac.washington.edu [140.142.32.5]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA03464 for ; Thu, 31 May 2001 10:09:10 -0500 Received: from mailhost2.u.washington.edu (mailhost2.u.washington.edu [140.142.33.2]) by mxout1.cac.washington.edu (8.11.2+UW01.01/8.11.2+UW01.04) with ESMTP id f4VK8n115503 for ; Thu, 31 May 2001 13:08:49 -0700 Received: from cae0 (g-turkiyyah-2.ce.washington.edu [128.95.204.230]) by mailhost2.u.washington.edu (8.11.2+UW01.01/8.11.2+UW01.04) with SMTP id f4VK8nC32375 for ; Thu, 31 May 2001 13:08:49 -0700 Message-ID: <00b101c0ea0e$3a6e6910$e6cc5f80@ce.washington.edu> From: "Ahmad El Husseini" To: References: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> <20010531043940.32689@smtp.telus.net> Subject: Re: [Metakit] Eficiency problem Date: Thu, 31 May 2001 13:13:59 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Jean-Claude Wippler" [...] > But you're probably better off storing things the other way around first: > one view per analysis run, with 4000 rows - one per object. Is it more efficient to write 4000 columns, one per object, every object will be a nested view (with the whole view containing one row only)? because as you mentioned before adding a row causes metakit to write the whole view again. From: "Jacob Levy" > Another option is to store things in e4Graph ... Does e4Graph optimize the way data is formatted? Thanks very much for your time, Ahmad From claird@starbase.neosoft.com Thu May 31 10:24:41 2001 Received: from mail3.iadfw.net (mail3.iadfw.net [209.196.123.3]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA03909 for ; Thu, 31 May 2001 10:24:41 -0500 Received: from starbase.neosoft.com from [206.109.1.32] by mail3.iadfw.net (/\##/\ Smail3.1.30.16 #30.25) with esmtp for sender: id ; Thu, 31 May 2001 15:25:39 -0500 (CDT) Received: (from claird@localhost) by starbase.neosoft.com (8.9.3/8.9.3) id PAA44525; Thu, 31 May 2001 15:38:36 -0500 (CDT) (envelope-from claird) Date: Thu, 31 May 2001 15:38:36 -0500 (CDT) From: Cameron Laird Message-Id: <200105312038.PAA44525@starbase.neosoft.com> To: g.lancaster@mbf.com.au, metakit@equi4.com, rmenegon@bigpond.net.au Subject: Re: [Metakit] Synchronisation In-Reply-To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >From metakit-admin@equi4.com Fri May 25 18:33:17 2001 > . > . > . >When I asked about this a while back , JCW suggested that I need a central Metakit Server handling all the requests, locking etc, from possibly multiple clients, which sounded pretty plausible/do-able.. so its likely that such a server would be queue based - he also pointed me to the 'FAQ'/Discussion pages at the MetaKit Wiki :- >http://www.equi4.com/metakit/wiki.cgi/ >which had some notes on locking etc ... >Cameron - you know more about Tequila than I do - is it possible to 'simulate' its 'mode of operation' from C++ do you think, or a different approach required ? (I'll start looking at it in my free time - thats usually between 10pm and 5am !! ) > . > . > . Yes. But if you don't have a background in Tcl, I think it's probably best just to start from Jean-Claude's suggestion and comments, as you refer to them above. From cer@transelec.cl Thu May 31 10:56:56 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA04867 for ; Thu, 31 May 2001 10:56:46 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id QAA27983 for ; Thu, 31 May 2001 16:51:53 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Thu, 31 May 2001 16:59:48 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C048B55B0@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: metakit@equi4.com Date: Thu, 31 May 2001 16:59:46 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Subject: [Metakit] Mk4py: Closing a metakit file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Python 2.0, Mk4py 2.3.4-29 Wich is the right way to close a metakit file? I have an aplication that compare a metakit database into the aplication directory with a posible new database in other directory (a public directory on a network). Then the aplication copy the new file inside the aplication directory replacing the old one. Then the aplication continue using the updated database The only way I could do the copy step, without craching my aplication, was deleting the python objects asociated to the metakit databases. I could't find any way to close the metakit file than this one. Is this the right way to do this, or there is a close() method somewhere? My python code is something like this: ############################### file_one = mk.storage(CURREN_DB,0) #read only file_two = mk.storage(NEW_DB,0) #read only ..... compare(file_one, file_two) ..... del file_one # <---- If I delete this lines del file_two # <---- the aplications crach shutil.copyfile(NEW_DB, CURRENT_DB) final_file = mk.storage(CURREN_DB,0) .... work_with_the_file() .... ################################# Thanks Cristian Echeverria From rcohen@dial.oleane.com Thu May 31 23:53:34 2001 Received: from caroubier.wanadoo.fr (smtp-rt-6.wanadoo.fr [193.252.19.160]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA18226 for ; Thu, 31 May 2001 23:53:32 -0500 Received: from mahonia.wanadoo.fr (193.252.19.58) by caroubier.wanadoo.fr; 1 Jun 2001 11:52:38 +0200 Received: from dial.oleane.com (217.128.52.70) by mahonia.wanadoo.fr; 1 Jun 2001 11:52:11 +0200 Message-ID: <3B1773CD.82744536@dial.oleane.com> Date: Fri, 01 Jun 2001 11:51:57 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] join Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I have just began to use metakit, and I cannot do a join ... c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); c4_View table2=database.GetAs("table2[index2:I,val2:D]"); // fill tables c4_View join; c4_IntProp index("index"); c4_IntProp index2("index2"); join.AddProperty(index2); join.AddProperty(index); c4_View result=table2.Join(join,table1); the result view has no data ! Thanks for any help -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From Slick@battleop.net Fri Jun 1 15:29:28 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id PAA11605 for ; Fri, 1 Jun 2001 15:29:27 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Fri, 1 Jun 2001 18:36:35 -0700 Message-ID: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> From: "Slick" To: Date: Fri, 1 Jun 2001 18:32:27 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Avoiding using global variables Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have a dialog class that accesses a metakit database. I have it working well. However, I had the variables declared globally. I decided that I wanted to have the variables just local to the class because: (1) I don't want a lot of global variables, and (2) I am worried that the database remains open using resources even after the dialog is closed. This may not be true but I can't find much actual documentation on metakit usage ( i like the small discussion on getting started on the metakit site but beyond this there isn't much). I tried simply adding member variables but when building they are misinterpreted as function declarations because of the (). For example public: c4_StringProp pServer ("server"); gives error C2660: 'pServer' : function does not take 1 parameters The only way I have been able to do it is to declare the variables as static within the class. I don't think this is a good solution. All comments etc appreciated. From readonly@getsoft.com Fri Jun 1 15:45:25 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA11752 for ; Fri, 1 Jun 2001 15:45:24 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id TAA18543; Fri, 1 Jun 2001 19:44:57 -0600 (MDT) Message-ID: <200106021145070534.0A8383A4@getsoft.com> In-Reply-To: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> References: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Sat, 02 Jun 2001 11:45:07 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "Slick" , metakit@equi4.com Subject: Re: [Metakit] Avoiding using global variables Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id PAA11752 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick, This has nothing to do with MK and is basic C++ 101 stuff. You can't assign a value to a class member in the declaration. This should most likely be done in the class constructor. It has been a while since I've used MK but you most likely need to use a pointer to c4_StringProp. eg. c4_StringProp* m_pServer; In the constructor: m_pServer = new c4_StringProp("server"); and don't forget to delete it in the destructor. And yes, statics and globals are BAD. *********** REPLY SEPARATOR *********** On 1 Jun 2001 at 18:32 Slick wrote: >I have a dialog class that accesses a metakit database. I have it working >well. However, I had the variables declared globally. I decided that I >wanted to have the variables just local to the class because: > >(1) I don't want a lot of global variables, and >(2) I am worried that the database remains open using resources even after >the dialog is closed. This may not be true but I can't find much actual >documentation on metakit usage ( i like the small discussion on getting >started on the metakit site but beyond this there isn't much). > >I tried simply adding member variables but when building they are >misinterpreted as function declarations because of the (). For example > >public: > c4_StringProp pServer ("server"); >gives >error C2660: 'pServer' : function does not take 1 parameters > >The only way I have been able to do it is to declare the variables as static >within the class. I don't think this is a good solution. > >All comments etc appreciated. > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find lots of information on ED and while you're there you can click on 'Stay Informed' to join our mailing list and 'Future Plans' to read about Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From jcw@equi4.com Fri Jun 1 23:08:57 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19092 for ; Fri, 1 Jun 2001 23:08:56 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Avoiding using global variables Date: Sat, 2 Jun 2001 00:47:35 -0700 Message-Id: <20010602074735.29040@smtp.telus.net> In-Reply-To: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> References: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: [...] >public: > c4_StringProp pServer ("server"); >gives >error C2660: 'pServer' : function does not take 1 parameters That's not proper C++. Constructors of members are specified in the constructor of the encapsulating class, not in the declaration of the member: class Classy { public: Classy () : pServer ("server") { } c4_StringProp pServer; }; -jcw From jcw@equi4.com Fri Jun 1 23:08:58 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19095 for ; Fri, 1 Jun 2001 23:08:58 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Mk4py: Closing a metakit file Date: Sat, 2 Jun 2001 01:16:20 -0700 Message-Id: <20010602081620.6923@smtp.telus.net> In-Reply-To: <16D7A2FE3592D41184A200508BDE826C048B55B0@SYN201> References: <16D7A2FE3592D41184A200508BDE826C048B55B0@SYN201> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. wrote: >Python 2.0, Mk4py 2.3.4-29 > >Wich is the right way to close a metakit file? There is no other way to close than to delete all references to the storage object, I'm afraid. This, as you discovered, works: db = metakit.storage(...) ... del db You could also use: db = None The key is that every reference to the storage object must be gone. Once the internal reference count drops to zero, the datafile will be closed. -jcw From jcw@equi4.com Fri Jun 1 23:09:03 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19102 for ; Fri, 1 Jun 2001 23:09:02 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] join Date: Sat, 2 Jun 2001 02:09:03 -0700 Message-Id: <20010602090903.15357@smtp.telus.net> In-Reply-To: <3B1773CD.82744536@dial.oleane.com> References: <3B1773CD.82744536@dial.oleane.com> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > c4_View join; > c4_IntProp index("index"); > c4_IntProp index2("index2"); > join.AddProperty(index2); > join.AddProperty(index); > c4_View result=table2.Join(join,table1); Which can also be written as: c4_View result=table2.Join((index2,index),table1); >the result view has no data ! Without knowing what data table1 and table2 contain, it's hard to tell whether it should. Try a simpler join first: c4_View result=table2.Join(index2,table1); Look at tests/tcusto2.cpp, test "c17" for a simple example. If you cannot solve the problem, please send me a simple test case so I can investigate furher. -jcw From jcw@equi4.com Fri Jun 1 23:09:02 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19098; Fri, 1 Jun 2001 23:08:58 -0500 From: Jean-Claude Wippler To: , , Subject: Re: [Metakit] Synchronisation Date: Sat, 2 Jun 2001 01:58:00 -0700 Message-Id: <20010602085800.16605@smtp.telus.net> In-Reply-To: <200105312038.PAA44525@starbase.neosoft.com> References: <200105312038.PAA44525@starbase.neosoft.com> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Garth Lancaster wrote: >> [... Tequila ...] is it possible to >'simulate' its 'mode of operation' from C++ do you think, or a different >approach required ? Tequila is a tiny network server written in Tcl. It doesn't do that much, just accept requests and propagate change requests to all attached clients - Tcl makes such things trivial. Note that the current Tequila design "talks" Tcl over the wire. You'd be having more work to construct and decode Tcl-style commands and lists in C++, than is probably worth it. Back to the original issue... Rob Menegon wrote: >Does anyone have any advice on file sharing mechanisms? I have a scenario >in which multiple sources need to update the same metakit database. At >present I am leaning towards all the process posting the updates to a queue >and giving another process the responsibility of clearing the queue and >updating the metakit database file. > >Any thought on the matter wold be greatly received. You'll have to coordinate reading as well, MK does a considerable amount of caching, so a change (even if it's a single writer), will not make readers aware of the changes. One surprising part will be that readers will appear to work properly at first, because a single commit does not invalidate the previous one. So readers will simply continue to see the previous state. The trouble is that the *next* commit can clobber that previous state, at which point readers can crash and burn any time, i.e.: reader opens -> write commits twice -> reader is in trouble. To put it differently: as long as there are readers, the system must avoid altering state underneath them. A second issue is that at some point, you'll probably want the readers to see the new state, not the original. This is the reason why I introduced esoteric things such as "commit- extend" and "commit-aside" (both still experimental). Commit-extend just keeps appending, thus guaranteeing that readers continue to see the original state at all times (but files will grow very very quickly). Commit-aside stores differences into a separate file, again letting readers continue to see a consistent (old) state. One approach which seems worth pursueing, is the combination of both: - file A holds a dataset - file A' stores changes, in a growing manner That means readers can open A, for the original state, or even A + A' for the latest state. A single writer keeps on appending differences to A'. Then, at some point everyone has to close their datasets, the changes from A' are folded back into A, and A' is deleted. There are several trouble-spots with this today: - commit-extend/commit-aside have not been tested enough - the combination has not yet been tried at all - commit-aside is still dumb, it stores all data, not differences Not of these are show stoppers, but they have to be fixed/finished first. It may all seem a bit strange to have to resort to such unconventional tactics, but you have to keep in mind that MK stores its data column- wise. This means that normally, a change to one value in a view of 100,000 rows, causes an entire *column* of values to be saved on commit. Commit-aside was designed to avoid this, and store a byte-oriented "difference set". Commit-extend solves the issue of a writer never modifying things underneath readers. Both can work with locking, and hence without contention - i.e. at top speed. With a bit of extra machinery (semaphores, probably), one could then signal changes and make readers adjust themselves to new commits efficiently. I think. All in all, there's a lot of potential in this IMO, but it's not there today. If you need something soon, I continue to suggest writing a very simple server (or better still, adopting something that exists), and pick a protocol (perhaps a standard one), then make all I/O go through the server, i.e. both reading *and* writing. In a way, that turns clients into "dumb" participants, but it also gets around a lot of problems. Locking can be added at a higher level, i.e. through a few server primitives. I'd be happy to discuss this further (the wiki is a good idea), but have to add that I cannot spend time on this before August. To close the circle of this email: I have recently started to alter Tequila so it uses an even simpler, and less language-specific protocol over the wire. It might be effective to code the server in Tcl or Python, yet do it in such a way that it fits easily into a C/C++ client (or any other language). -jcw From Slick@battleop.net Sat Jun 2 06:37:08 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA23701 for ; Sat, 2 Jun 2001 06:37:07 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Sat, 2 Jun 2001 09:44:12 -0700 Message-ID: <008101c0eb82$ac9fbf00$0200a8c0@pacbell.net> From: "Slick" To: Date: Sat, 2 Jun 2001 09:40:03 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Avoiding using global variables Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: public: > c4_StringProp pServer ("server"); >gives >error C2660: 'pServer' : function does not take 1 parameters That's not proper C++. Constructors of members are specified in the constructor of the encapsulating class, not in the declaration of the member: class Classy { public: Classy () : pServer ("server") { } c4_StringProp pServer; }; -jcw ______________________________ All I am trying to do is define the variables within my dialog class so that I can use them in different subs within the dialog class. I just need them to show as member variables. I tried your example but I can't get it to work. Here is one of the tries: class SettingsDlg : public CDialog { // Construction public: SettingsDlg(CWnd* pParent = NULL); // standard constructor //c4_StringProp pServer ("server"); SettingsDlg(): pServer ("server") {} c4_StringProp pServer; CString m_curServerName; BOOL m_bNewitem; The dlg constructor is already set up. Putting in SettingsDlg(): doesn't make sense to me. I have about 8 metakit variables that I want to create as member variables. pServer is just one of them. Like I mentioned before I have it working with them as either global or static but there has to be a better way. This is probably my lack of skill. From jyl@best.com Sat Jun 2 07:14:12 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA24017 for ; Sat, 2 Jun 2001 07:14:12 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id KAA28241; Sat, 2 Jun 2001 10:13:12 -0700 (PDT) Message-ID: <3B192036.58CF8BA0@best.com> Date: Sat, 02 Jun 2001 10:19:50 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Ahmad El Husseini CC: metakit@equi4.com Subject: Re: [Metakit] Eficiency problem References: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> <20010531043940.32689@smtp.telus.net> <00b101c0ea0e$3a6e6910$e6cc5f80@ce.washington.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ahmad Apologies for not replying earlier -- I was away. The (*THE*) best way to understand how e4Graph stores its data is to read the definition of its storage layout in core/mkdriver/mkstorage.h at the head of the file. This is (on purpose) not documented so I can change it if there's a need. With that caveat, here is some explanation. e4Graph is designed to store arbitrary graph-like data in a table format in a relational or column-oriented database like Metakit. I intend to write (one day far in the future :) drivers for other database systems such as mySQL, GDBM, etc. What follows (and what you'll see in mkstorage.h) is Metakit specific. The details will be vastly different when you use any other database drivers, without impacting in any way the program you write to the e4Graph API. The Metakit database driver for e4Graph stores its date in one storage. That storage contains exactly these views: #define E4_GRAPHMARKERS "e4GraphMarkers[nameID:I,next:I,root:I,used:I]" #define E4_GRAPHNODES \ "e4GraphNodes[firstVertex:I,lastVertex:I,vertexCount:I,parentID:I," \ "used:I,refCount:I,nodeMarkers:I]" #define E4_GRAPHVERTICES \ "e4GraphVertices[nameID:I,nodeID:I,vertexType:I,row:I,next:I,prev:I,used:I]" #define E4_GRAPHFLOATS "e4GraphFloats[f:F,used:I]" #define E4_GRAPHSTRINGS "e4GraphStrings[s:S,used:I]" #define E4_GRAPHNAMES "e4GraphNames[n:S,used:I]" #define E4_GRAPHBINARY "e4GraphBinary[b:B,used:I]" #define E4_GRAPHUNUSED "e4GraphUnused[first:I]" #define E4_GRAPHPARENTS "e4GraphParents[nodeID:I,count:I,used:I]" So, after examining this you'll see that this is highly optimized to store graph-like structures with the ability to efficiently traverse, update, modify and commit. For further information about how to use e4Graph in your program, please check the web site at http://e4graph.sourceforge.net. Also, if you need more sample code, check out the e4XML directory, the tcl directory and the test directory in the source code distributions. Finally, if you have more questions, I'll be happy to answer them on this mailing list or in private, as you prefer. One final note: e4Graph is scheduled to be in Beta by mid-June and the final 1.0 release is in September or October 2001. It's based on Metakit 2.3.4-29, which I hope will be final (Metakit 2.4.0) by that time. --JYL Ahmad El Husseini wrote: > From: "Jean-Claude Wippler" > [...] > > But you're probably better off storing things the other way around first: > > one view per analysis run, with 4000 rows - one per object. > > Is it more efficient to write 4000 columns, one per object, every object > will be a nested view (with the whole view containing one row only)? because > as you mentioned before adding a row causes metakit to write the whole view > again. > > From: "Jacob Levy" > > Another option is to store things in e4Graph ... > > Does e4Graph optimize the way data is formatted? > > Thanks very much for your time, > Ahmad > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Sat Jun 2 07:50:02 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA24497 for ; Sat, 2 Jun 2001 07:50:01 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Avoiding using global variables Date: Sat, 2 Jun 2001 10:50:08 -0700 Message-Id: <20010602175008.6571@smtp.telus.net> In-Reply-To: <008101c0eb82$ac9fbf00$0200a8c0@pacbell.net> References: <008101c0eb82$ac9fbf00$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: [...] >class SettingsDlg : public CDialog >{ >// Construction > >public: > SettingsDlg(CWnd* pParent = NULL); // standard constructor > //c4_StringProp pServer ("server"); > SettingsDlg(): pServer ("server") {} > c4_StringProp pServer; > > CString m_curServerName; > BOOL m_bNewitem; > >The dlg constructor is already set up. Putting in SettingsDlg(): doesn't >make sense to me. I have about 8 metakit variables that I want to create as >member variables. pServer is just one of them. > >Like I mentioned before I have it working with them as either global or >static but there has to be a better way. This is probably my lack of skill. This is a C++ issue and has nothing to do with MK. In your code, you have: SettingsDlg::SettingsDlg(CWnd* pParent) { ... Change it to: SettingsDlg::SettingsDlg(CWnd* pParent) : pServer ("server") { ... That's as far as I can go with answering general C++ Q's on this list, I'm afraid. -jcw From jerry@usd1.com Tue Jun 12 14:47:38 2001 Received: from asteroid.pacifier.com (asteroid.pacifier.com [199.2.117.154]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA26838 for ; Tue, 12 Jun 2001 14:47:37 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id f5D0jtM23476 for ; Tue, 12 Jun 2001 17:45:55 -0700 (PDT) Date: Tue, 12 Jun 2001 17:45:19 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <12131613106.20010612174519@usd1.com> To: Metakit Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] [python] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I have been writing a batch processing system in Python based on MetaKit that will take a number of records (10K to 200k) from an outside source (flat, SQL, CSV), process it, then export selected fields to a different outside source. I have created dictionary driven classes for Records and DataSets. These classes view and modify the data in the tables directly, handling the multiple levels (sub-views). Question 1) I have read that it is more efficient to pre-allocate the row. How is that done with Python? Question 2) I have been struggling with creating a less direct method of creating / changing fields in Mk through Python. When creating or modifying a row from an import routine or a screen, I need to verify the integrity of the row before I 'commit' it to the table (view). Option A: Use .commit() & .rollback(). I rejected this immediately because of the disk I/O overhead when writing 100,000 records. Option B: copy the contents of a row into a Python structure (I wish there was a Mk function to do this -- it is a bit tedious); modify / verify the changes; copy back to row (append if new). Two MAJOR problems: 1) creating new... Using view 'test[id:I,name,sales[region,value:I]]'.. newrec = {'id':1,'name':'bob','sales':[{'region':'N','value':100}]} vw.append(newrec) works, appends 'bob', with id 1 and one row in 'sales', but all fields are empty. problem 2), updating existing rows. I found the following statement to be very bad for the health of the program: vw[n] = newrec Thanks for reading this far! I hope this makes sense. -- Thanks, Jerry McRae MetaKit 2.3.4, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From jerry@usd1.com Tue Jun 12 17:10:48 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA29464 for ; Tue, 12 Jun 2001 17:10:48 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f5D395i02922 for ; Tue, 12 Jun 2001 20:09:06 -0700 (PDT) Date: Tue, 12 Jun 2001 20:08:34 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <15540209510.20010612200834@usd1.com> To: metakit Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] [python] more tries Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, (addendum to last message) Spending concerted effort today to making a 'buffer' for data entry and importing, I have tried the following ideas. C) reserving row zero for the I/O buffer, since it has exactly the same properties, then copy or append if it verifies. Downside: I can figure out no easy way of clearing the row's contents before using. Something like vw.__dict__ = {} would be real nice. D) add a new row to the end to use as the buffer, then delete it. Downside: the .delete() function crashes for me. using interactive prompt: Python 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> import mk >>> vw=mk.storage('com.mk',1).view('com') >>> len(vw) 1000 >>> vw.delete(999) >>> len(vw) SystemError: null argument to internal routine >>> vw >>> vw[0].fname # is a valid property Traceback (most recent call last): File "", line 1, in ? AttributeError: fname >>> vw >>> vw.structure() [] #-------------------------------- using PyWin: >>> vw.append(vw[1000]) #copy last row 1001 >>> vw.delete(1001) #delete last row Firing event 'ProcessEnter' failed. Traceback (most recent call last): File "f:\python21\pythonwin\pywin\scintilla\bindings.py", line 141, in fire rc = apply(binding.handler, args) File "f:\python21\pythonwin\pywin\framework\interact.py", line 476, in ProcessEnterEvent self.history.history_store(source) File "f:\python21\pythonwin\pywin\idle\IdleHistory.py", line 75, in history_store self.history.remove(source) SystemError: null argument to internal routine >>> len(vw) # it did work, though 999 #------------------------------- It is odd that id does not say anything about MetaKit in the errors. E) create a separate view for a buffer. I think this will work. I.E. vwb = vw[0:0] recbuffer = vwb.append() ...set all the fields -- (which still involves multiple loops for complex structures) vw.append(vwb[0]) -or- vw[index] = vwb[0] One idea I had was that it would be nice to have a mini-rollback option... to un-do the append, or the changes just to the last record. [Another thing I think would be nice is the equivalent of the .filter() that actually returns a new view. This would save several steps!] Other options? How do others work with rows and balance the commits/rollbacks and the direct row access? -- Thanks again!, Jerry McRae MetaKit 2.3.4, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From barnard@aragorn.stat.harvard.edu Wed Jun 13 03:34:44 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA09083 for ; Wed, 13 Jun 2001 03:34:43 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id JAA07550; Wed, 13 Jun 2001 09:43:42 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.28162.951393.336937@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 09:43:30 -0400 (EDT) To: Jerry McRae Cc: Metakit Subject: [Metakit] [python] In-Reply-To: <12131613106.20010612174519@usd1.com> References: <12131613106.20010612174519@usd1.com> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, Jerry McRae writes: > Hi, > > I have been writing a batch processing system in Python based on > MetaKit that will take a number of records (10K to 200k) from an > outside source (flat, SQL, CSV), process it, then export selected > fields to a different outside source. I have created dictionary driven > classes for Records and DataSets. These classes view and modify the > data in the tables directly, handling the multiple levels (sub-views). > > Question 1) I have read that it is more efficient to pre-allocate the > row. How is that done with Python? You cannot do this with Mk4py, however, it's easy to add the necessary code to add the functionality to the Python interface: (1) Add the following to PyView.cpp: static char *setsize__doc = "setsize(nrows)\n"; static PyObject *PyView_setsize(PyView *o, PyObject *_args) { try { PWOSequence args(_args); PWONumber nrows = PWONumber(args[0]); o->SetSize((int) nrows); return nrows.disOwn(); } catch (PWException e) { return e.toPython(); } } (2) Add the following to the ViewMethods array in PyView.cpp: {"setsize", (PyCFunction)PyView_setsize, METH_VARARGS, setsize__doc}, Of course you'll need to rebuild the Mk4py library. > Question 2) I have been struggling with creating a less direct method > of creating / changing fields in Mk through Python. When creating or > modifying a row from an import routine or a screen, I need to verify > the integrity of the row before I 'commit' it to the table (view). > > Option A: Use .commit() & .rollback(). I rejected this immediately > because of the disk I/O overhead when writing 100,000 records. > > Option B: copy the contents of a row into a Python structure (I wish > there was a Mk function to do this -- it is a bit tedious); modify > / verify the changes; copy back to row (append if new). Two MAJOR > problems: 1) creating new... > Using view 'test[id:I,name,sales[region,value:I]]'.. > newrec = {'id':1,'name':'bob','sales':[{'region':'N','value':100}]} > vw.append(newrec) > > works, appends 'bob', with id 1 and one row in 'sales', but all fields > are empty. > > problem 2), updating existing rows. I found the following statement > to be very bad for the health of the program: > vw[n] = newrec > > Thanks for reading this far! I hope this makes sense. I cannot offer any advice on this. I've made a Python wrapper class around PyView so that I can completely customize setting, getting, etc. of my views. It's been a lot of work, but I'm now extremely happy with the efforts. Best, John Barnard -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From barnard@aragorn.stat.harvard.edu Wed Jun 13 04:24:30 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA10806 for ; Wed, 13 Jun 2001 04:24:30 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id KAA07636; Wed, 13 Jun 2001 10:33:27 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.31156.462747.59373@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 10:33:24 -0400 (EDT) To: Jerry McRae Cc: metakit Subject: [Metakit] [python] more tries In-Reply-To: <15540209510.20010612200834@usd1.com> References: <15540209510.20010612200834@usd1.com> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry McRae writes: > D) add a new row to the end to use as the buffer, then delete it. > Downside: the .delete() function crashes for me. > > using interactive prompt: > Python 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32 > Type "copyright", "credits" or "license" for more information. > >>> import mk > >>> vw=mk.storage('com.mk',1).view('com') > >>> len(vw) > 1000 > >>> vw.delete(999) > >>> len(vw) > SystemError: null argument to internal routine I also get this message after deleting a row regardless of the command given after the deletion. I'm running Python 2.1, Metakit 2.3.4, and Windows 2000 SP1. If I make a change in the code for delete in PyView.cpp from static PyObject* PyView_delete(PyView *o, PyObject* _args) { try { PWOSequence args(_args); int ndx = PWONumber (args[0]); //o->RemoveAt(ndx); PWOSequence seq; o->setSlice(ndx, ndx+1, seq); Py_INCREF(Py_None); return Py_None; } catch(PWException e) { return e.toPython(); } } to static PyObject* PyView_delete(PyView *o, PyObject* _args) { try { PWOSequence args(_args); int ndx = PWONumber (args[0]); o->RemoveAt(ndx); // PWOSequence seq; // o->setSlice(ndx, ndx+1, seq); Py_INCREF(Py_None); return Py_None; } catch(PWException e) { return e.toPython(); } } i.e., uncomment the RemoveAt line and comment out the slice lines, I no longer get the error message. I haven't had the time to look closely at setSlice, which eventually calls RemoveAt and does some range checking. The raw RemoveAt might be dangerous without some range checking, but at least it appears to work. -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From barnard@aragorn.stat.harvard.edu Wed Jun 13 05:12:12 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA12303 for ; Wed, 13 Jun 2001 05:12:12 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id LAA07728; Wed, 13 Jun 2001 11:21:02 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.34014.171893.297213@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 11:21:02 -0400 (EDT) To: metakit@equi4.com X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Subject: [Metakit] Deleting rows and Obtaining Views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've observed, as demonstrated in the example below, that under Mk4py after deleting a row from a view, deleting the view, and reloading the view from storage that the reloaded view still has the row deletion, i.e., isn't the original view as I expected. Am I misunderstanding how to retrieve the original view? If I delete the storage object containing the view, then recreate the storage and view I get the original view. Why do I need to delete and recreate the storage? Python 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> import metakit as mk >>> test2 = mk.storage("junk", 1) >>> v2 = test2.view("v2") >>> mk.dump(v2) var1 var2 ---- ---- 0.0 0.0 0.0 1.0 0.0 2.0 0.0 3.0 0.0 4.0 ---- ---- Total: 5 rows >>> v2.delete(3) >>> mk.dump(v2) var1 var2 ---- ---- 0.0 0.0 0.0 1.0 0.0 2.0 0.0 4.0 ---- ---- Total: 4 rows >>> del v2 >>> v2 = test2.view("v2") >>> mk.dump(v2) var1 var2 ---- ---- 0.0 0.0 0.0 1.0 0.0 2.0 0.0 4.0 ---- ---- Total: 4 rows -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From gmcm@hypernet.com Wed Jun 13 05:54:54 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA13463 for ; Wed, 13 Jun 2001 05:54:53 -0500 Received: from me (204.176.40.67) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Wed, 13 Jun 2001 11:52:18 -0400 From: "Gordon McMillan" To: Jerry McRae , Metakit Date: Wed, 13 Jun 2001 11:52:01 -0400 Subject: Re: [Metakit] [python] Reply-to: gmcm@hypernet.com Message-ID: <3B2753E1.3600.3E49835B@localhost> Priority: normal In-reply-to: <12131613106.20010612174519@usd1.com> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, I tend to use MK for persistence, not batch, so I tend to commit on every little change. But I can help with some of these questions. > I have been writing a batch processing system in Python based > on MetaKit that will take a number of records (10K to 200k) > from an outside source (flat, SQL, CSV), process it, then > export selected fields to a different outside source. I have > created dictionary driven classes for Records and DataSets. > These classes view and modify the data in the tables directly, > handling the multiple levels (sub-views). > > Question 1) I have read that it is more efficient to > pre-allocate the row. How is that done with Python? Not as efficient as the underlying C++, but you can build a large view fairly efficiently by using repeated concatenation for i in range(10): v = v + v > Question 2) I have been struggling with creating a less direct > method of creating / changing fields in Mk through Python. > When creating or modifying a row from an import routine or a > screen, I need to verify the integrity of the row before I > 'commit' it to the table (view). > > Option A: Use .commit() & .rollback(). I rejected this > immediately because of the disk I/O overhead when writing > 100,000 records. > > Option B: copy the contents of a row into a Python structure (I > wish there was a Mk function to do this -- it is a bit > tedious); modify / verify the changes; copy back to row (append > if new). Two MAJOR problems: 1) creating new... Using view > 'test[id:I,name,sales[region,value:I]]'.. newrec = > {'id':1,'name':'bob','sales':[{'region':'N','value':100}]} > vw.append(newrec) > > works, appends 'bob', with id 1 and one row in 'sales', but all > fields are empty. I'm afraid there's no way around the fact that subviews are created empty. > problem 2), updating existing rows. I found the following > statement to be very bad for the health of the program: vw[n] = > newrec You might do something like this for a start: class RowProxy: def __init__(self, row): self._row = row def __getattr__(self, nm): return getattr(self._row, nm) def commit(self): for k,v in self.__dict__.items(): if hasattr(self._row, k): setattr(self._row, k, v) With this proxy, changing an attribute will change it in the proxy's __dict__. row.shoesize will always return the current value of shoesize (which may be the original value - you get the original value with row._row.shoesize). row.commit will update the underlying row in the view. From your second message, I have no idea what's going wrong with view.delete. It certainly works for me, but I haven't tried MK with Py 2.1 yet (I wonder if it has something to do with gc). OTOH, I almost always use a isdel attribute on rows. That's because: r = v[5] r.name = 'fred' v.delete(0) r.name # no longer 'fred', because he's moved to v[4] I'm not sure what you're getting at with trying to create a buffer space in MK. If I have to intercept the changes to a row, I use a proxy like that above. For inserting new rows, I either build values in a dict, use an instance of a dummy class (basically sytactic sugar around the instance __dict__), or append an empty row and modify it. Note that you can proxy a view to return proxied rows with: class Wrapper: def __init__(self, view, klass): self.__obj = view self.__klass = klass def __getattr__(self, nm): return getattr(self.__obj, nm) def __getitem__(self, i): return self.__klass(self.__obj[i]) def __len__(self): return len(self.__obj) The reference semantics of MK rows can be hard to get used to, but once you do, it can be very powerful. - Gordon From barnard@aragorn.stat.harvard.edu Wed Jun 13 07:34:31 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA16490 for ; Wed, 13 Jun 2001 07:34:31 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id NAA07969; Wed, 13 Jun 2001 13:43:31 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.42562.868686.728229@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 13:43:30 -0400 (EDT) To: metakit@equi4.com Subject: RE: [Metakit] Deleting rows and Obtaining Views X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. writes: > you need to do a commit() to save the changes until deleting the storage > object > in you case: > > test2.commit() That makes the row deletion permanent. I was trying to retrieve the original view (without the row deletion) without destroying the storage object and then recreating it. -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From akao347@hotmail.com Wed Jun 13 08:45:41 2001 Received: from hotmail.com (oe76.law4.hotmail.com [216.33.148.172]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA19974 for ; Wed, 13 Jun 2001 08:45:40 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 13 Jun 2001 11:43:48 -0700 X-Originating-IP: [151.203.34.140] From: "A K" To: References: <15143.42562.868686.728229@aragorn.stat.harvard.edu> Subject: Re: [Metakit] Deleting rows and Obtaining Views Date: Wed, 13 Jun 2001 14:44:43 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 13 Jun 2001 18:43:48.0945 (UTC) FILETIME=[C8BB7C10:01C0F438] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, This might not be the answer you want, but try calling Rollback. It makes sense that when you destroy the view, that the data in the Storage is still modified since you are suppose to detach your views before destroyng the Storage. Brendan. ----- Original Message ----- From: "John Barnard" To: Sent: Wednesday, June 13, 2001 1:43 PM Subject: RE: [Metakit] Deleting rows and Obtaining Views > Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. writes: > > you need to do a commit() to save the changes until deleting the storage > > object > > in you case: > > > > test2.commit() > > That makes the row deletion permanent. I was trying to retrieve the > original view (without the row deletion) without destroying the > storage object and then recreating it. > > -- > John Barnard > Senior Research Statistician, deCODE Genetics > 124 Mount Auburn Str., Suite 200N > Cambridge, MA 02138 > Phone: (617) 520-6612 > Fax: (617) 576-5874 > Email: barnard@stat.harvard.edu > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From barnard@aragorn.stat.harvard.edu Wed Jun 13 09:02:09 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA20901 for ; Wed, 13 Jun 2001 09:02:09 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id PAA08092; Wed, 13 Jun 2001 15:10:59 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.47811.702754.104680@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 15:10:59 -0400 (EDT) To: "A K" Cc: Subject: Re: [Metakit] Deleting rows and Obtaining Views In-Reply-To: References: <15143.42562.868686.728229@aragorn.stat.harvard.edu> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brendan, A K writes: > Hi, > > This might not be the answer you want, but try calling Rollback. > > It makes sense that when you destroy the view, that the data in the Storage > is still modified since you are suppose to detach your views before > destroyng the Storage. Thanks. That's what I wanted. Not sure how I overlooked it! John From akao347@hotmail.com Wed Jun 13 10:39:31 2001 Received: from hotmail.com (oe41.law4.hotmail.com [216.33.148.78]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA25470 for ; Wed, 13 Jun 2001 10:39:31 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 13 Jun 2001 13:37:40 -0700 X-Originating-IP: [151.203.38.197] From: "A K" To: "Metakit" References: <3B2753E1.3600.3E49835B@localhost> Date: Wed, 13 Jun 2001 16:38:47 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 13 Jun 2001 20:37:40.0458 (UTC) FILETIME=[B0A1A4A0:01C0F448] Subject: [Metakit] CustomViewer versus DerivedSequence Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am in the process of writing an application where it is necessary to notify changes to the "derived" views. It seems that there are two primary back-ends to a derived view. One is based off the CustomViewer and the other is based off DerivedSequence. It also seems that derived views based off a DerivedSequence will receive notification changes, whereas CustomViewer will not. Are these statements true? What are the reasons for doing one or the other? Would it be good if I ported the CustomViewers to DerivedSequences? Thank you, Brendan Kao. From akao347@hotmail.com Wed Jun 13 10:57:54 2001 Received: from hotmail.com (oe4.law4.hotmail.com [216.33.148.108]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA26477 for ; Wed, 13 Jun 2001 10:57:54 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 13 Jun 2001 13:56:03 -0700 X-Originating-IP: [151.203.38.197] From: "A K" To: "Metakit" References: <3B2753E1.3600.3E49835B@localhost> Subject: Re: [Metakit] CustomViewer versus DerivedSequence Date: Wed, 13 Jun 2001 16:57:10 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 13 Jun 2001 20:56:03.0453 (UTC) FILETIME=[421152D0:01C0F44B] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Oops, Correction to what I said. I mean that derived views based off DerivedSequence derived views will receive notification messages, where as if it were based off a CustomViewer derived view, it would not. Brendan ----- Original Message ----- From: "A K" To: "Metakit" Sent: Wednesday, June 13, 2001 4:38 PM Subject: [Metakit] CustomViewer versus DerivedSequence > Hi, > > I am in the process of writing an application where it is necessary to > notify changes to the "derived" views. It seems that there are two primary > back-ends to a derived view. One is based off the CustomViewer and the > other is based off DerivedSequence. It also seems that derived views based > off a DerivedSequence will receive notification changes, whereas > CustomViewer will not. Are these statements true? > > What are the reasons for doing one or the other? Would it be good if I > ported the CustomViewers to DerivedSequences? > > Thank you, > Brendan Kao. > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jerry@usd1.com Wed Jun 13 11:27:47 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA27817 for ; Wed, 13 Jun 2001 11:27:46 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f5DLPui18591; Wed, 13 Jun 2001 14:25:57 -0700 (PDT) Date: Wed, 13 Jun 2001 14:25:25 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <11719078322.20010613142525@usd1.com> To: "Gordon McMillan" CC: Metakit Subject: Re[2]: [Metakit] [python] In-reply-To: <3B2753E1.3600.3E49835B@localhost> References: <3B2753E1.3600.3E49835B@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Gordon, Wednesday, June 13, 2001, 8:52:01 AM, you wrote: GM> Jerry, GM> I tend to use MK for persistence, not batch, so I tend to GM> commit on every little change. But I can help with some of GM> these questions. >> I have been writing a batch processing system in Python based >> on MetaKit that will take a number of records (10K to 200k) >> from an outside source (flat, SQL, CSV), process it, then >> export selected fields to a different outside source. I have >> created dictionary driven classes for Records and DataSets. >> These classes view and modify the data in the tables directly, >> handling the multiple levels (sub-views). >> >> Question 1) I have read that it is more efficient to >> pre-allocate the row. How is that done with Python? GM> Not as efficient as the underlying C++, but you can build a GM> large view fairly efficiently by using repeated concatenation GM> for i in range(10): GM> v = v + v This looks like a real easy way to do this. Unfortunately, while it worked very fast when I tried it, the new view does not have new rows, but has 'pseudo rows', that all point to the same row: ## Example ---------->> >>> vw=dbt.getas('test[name,id,val:I]') >>> len(vw) 1 >>> vw=vw+vw >>> vw[1] >>> vw[1].name='bob' >>> dbt.commit() >>> len(vw) 2 >>> vw=vw+vw >>> vw.structure() [, ... >>> mk.dump(vw) name id val ---- -- --- bob 0 bob 0 bob 0 bob 0 ---- -- --- Total: 4 rows ### hmmm - I did not assign [0] to bob!? >>> vw[2].name='fred' >>> mk.dump(vw) name id val ---- -- --- fred 0 fred 0 fred 0 fred 0 ---- -- --- Total: 4 rows #-------------<< Likewise, if I try vw = vw * 8, I get eight rows, but without any properties. The del vw[x] command works great. Didn't think to try that. Thanks, Niki Thanks John for the cpp code. I don't have a C++ compiler nor the experience to use one. I use something like RowProxy and Wrapper, except I am doing it a bit differently. I might be able to modify it to use the commit() logic. The challenge I have is with nested rows and other complex interfaces. I will go into a little more detail. Keeping track of adding a new row to a nested view so it appears transparent, then doing a commit() to commit the 'deepcopy' of the structure would be more of a challenge. I keep a separate record layout structure that allows me to use specific classes on each and every column. This allows a very complex, dictionary driven interface. One of my goals to to allow the user to modify and/or create fields as he/she wishes using scripting. r=dfile[1] # both r and dfile are wrapper classes print r.earn # prints 10050 directly from the underlying PyRow print r._earn # prints $100.50, using a row class wrapper r.earn += 1 # sets the internal value directly, now 10051 r._earn += 1 # sets the value to 10151. The class knows scale of 2 print r._totalEarn # would print a total based on a virtual field. Basically just an eval of a string I store in the dictionary. Another dictionary type is a 'code' type, where I assign strings to numbers, but only store the number. The PyProperty will be an "I". Example r.service # 2 r._service # 'good' r._service = 'excellent' r.service # 4 r._service > 'good' # true, compares internal values r._service +=1 r._service # 'outstanding' Now the more complicated dictionary interfaces -- handling nested views. Using the getas string fragment 'pay[code,val:I]' I allow one dictionary entry to handle this. (Actually codeF('pay',1) in the dictionary layout handles it, the '1' says the value portion will be a money class). So now we can do... r.pay #PyView object at .... len(r.pay) # 0 r._pay # r._pay['E']=100 # adds new row, code='E', val=100 r._pay._NW=2 # adds new row, code="NW", val=200 r._pay._NW+=1 # adds 1 to row with code="NW", now val = 300 r._pay('NW').val # 300 r._pay('NW')._val # r._pay.sum() # 400 - with class interfaces I can get the sum, first, last, next (for iterating), keys, values, items, and clearit to clear whole nested structure. r._pay.keys() # ['E','NW'] Since the fields and sub-fields can be just about anything, storing all the changed values somewhere else until committed is another big challenge. [snip snip] GM> - Gordon Q1) in the above example, on PyWin the properties list as: [, ... at the interactive prompt, the same command ( vw.structure() ) I get [Property('I', 'iid'), Property( ... Does anyone know why? Q2) the primary reason for choosing MetaKet was the nested views. When I am adding 100,000 records, it seems from the mailing list that it will be faster to create a view without the nested views, add to that, then re-structure. Will this save me some time? Wish 1) method to import / export Python structure direct to/from a PyRow. Maybe by exposing a __dict__ interface, including the dictionary methods. Wish 3) have vw = vw * 1000 create 1000 new rows in current view -- Best regards, Jerry From gmcm@hypernet.com Wed Jun 13 14:15:43 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA00560 for ; Wed, 13 Jun 2001 14:15:42 -0500 Received: from me (204.176.40.132) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Wed, 13 Jun 2001 20:13:07 -0400 From: "Gordon McMillan" To: Jerry McRae Date: Wed, 13 Jun 2001 20:13:03 -0400 Subject: Re: Re[2]: [Metakit] [python] Reply-to: gmcm@hypernet.com CC: Metakit Message-ID: <3B27C94F.23905.13D31FF@localhost> Priority: normal In-reply-to: <11719078322.20010613142525@usd1.com> References: <3B2753E1.3600.3E49835B@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, > GM> Not as efficient as the underlying C++, but you can build a > GM> large view fairly efficiently by using repeated concatenation > GM> for i in range(10): GM> v = v + v > > This looks like a real easy way to do this. Unfortunately, while > it worked very fast when I tried it, the new view does not have > new rows, but has 'pseudo rows', that all point to the same row: Not exactly. They're all copies of the same row. >>> v = s.getas('test[name:S,shoesize:I]') >>> v.append({}) 0 >>> for i in range(3): v= v + v ... >>> len(v) 8 >>> v[3].name='fred' >>> v[7].name '' > Likewise, if I try vw = vw * 8, I get eight rows, but without > any properties. Yeah. I don't know what's wrong there. I guess I'm to blame, because that looks like my original code. My excuse is that I hadn't used the C++ interface to MetaKit when I wrote the wrapper, so I really didn't know what the calls did when I wrapped them. > I use something like RowProxy and Wrapper, except I am doing it > a bit differently. I might be able to modify it to use the > commit() logic. The challenge I have is with nested rows and > other complex interfaces. I struggled for a time with trying to make it so you could pass a Python object with an attribute that was a list, and have the list populate the corresponding subview. But the C++ API doesn't work that way, and I gave up. In using Mk4py (extensively), I really haven't missed it much. There are a lot of habits from RDB (and ODB) that don't work with MK. Fairly often I have to poke around before I figure out what will work with MK. OTOH, the lack of clutter (and the speed, and the compactness) make it all worthwhile. I know Jean-Claude is working (off & on) on a revised engine that should allow writing a cleaner and more efficient Python interface. ... > GM> - Gordon > > Q1) in the above example, on PyWin the properties list as: > [, ... > at the interactive prompt, the same command ( vw.structure() > ) I get > [Property('I', 'iid'), Property( ... > Does anyone know why? I think PythonWin uses repr where the interpreter calls the print slot (or maybe it's vice versa). > Q2) the primary reason for choosing MetaKet was the nested views. > When > I am adding 100,000 records, it seems from the mailing list > that it will be faster to create a view without the nested > views, add to that, then re-structure. Will this save me > some time? Sorry, I can't answer that. The only MK app I have that has counts approaching that doesn't use subviews. > Wish 1) method to import / export Python structure direct to/from > a PyRow. > Maybe by exposing a __dict__ interface, including the > dictionary methods. I don't know why you're asking for that. A row has attributes, gotten through the standard getattr mechanism. Setting a row from a Python object will see if the object is a dictionary, then it will try list (not recommended, since it's not very safe), and then will try sucking attributes from whatever it is. The only thing I see here that's non-Python is that you have to ask the view what the properties of the row are. > Wish 3) have vw = vw * 1000 create 1000 new rows in current view Unfortunately, I still don't understand the C++ API well enough to know why the properties get lost. - Gordon From jerry@usd1.com Wed Jun 13 18:52:33 2001 Received: from asteroid.pacifier.com (asteroid.pacifier.com [199.2.117.154]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA04146 for ; Wed, 13 Jun 2001 18:52:32 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id f5E4ohM17297; Wed, 13 Jun 2001 21:50:43 -0700 (PDT) Date: Wed, 13 Jun 2001 21:50:09 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <445766549.20010613215009@usd1.com> To: "Gordon McMillan" CC: Metakit Subject: Re[4]: [Metakit] [python] In-reply-To: <3B27C94F.23905.13D31FF@localhost> References: <3B2753E1.3600.3E49835B@localhost> <3B27C94F.23905.13D31FF@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Gordon, Thanks for the reply. The append with __add__ still doesn't work for me. See below. Wednesday, June 13, 2001, 5:13:03 PM, you wrote: GM> Jerry, >> GM> Not as efficient as the underlying C++, but you can build a >> GM> large view fairly efficiently by using repeated concatenation >> GM> for i in range(10): GM> v = v + v >> >> This looks like a real easy way to do this. Unfortunately, while >> it worked very fast when I tried it, the new view does not have >> new rows, but has 'pseudo rows', that all point to the same row: GM> Not exactly. They're all copies of the same row. >>>> v = s.getas('test[name:S,shoesize:I]') >>>> v.append({}) GM> 0 >>>> for i in range(3): v= v + v GM> ... >>>> len(v) GM> 8 >>>> v[3].name='fred' >>>> v[7].name GM> '' *** I tried this exact code both on PyWin and Python (version 2.1 (#15 April 16 2001)) and I get the results I stated earlier, any change to any row changes them all. I tried with the v.append({}) and the way I usually do it, v.append(). >> Likewise, if I try vw = vw * 8, I get eight rows, but without >> any properties. GM> Yeah. I don't know what's wrong there. I guess I'm to blame, GM> because that looks like my original code. My excuse is that I GM> hadn't used the C++ interface to MetaKit when I wrote the GM> wrapper, so I really didn't know what the calls did when I GM> wrapped them. >> I use something like RowProxy and Wrapper, except I am doing it >> a bit differently. I might be able to modify it to use the >> commit() logic. The challenge I have is with nested rows and >> other complex interfaces. GM> I struggled for a time with trying to make it so you could pass GM> a Python object with an attribute that was a list, and have the GM> list populate the corresponding subview. But the C++ API GM> doesn't work that way, and I gave up. In using Mk4py GM> (extensively), I really haven't missed it much. GM> GM> There are a lot of habits from RDB (and ODB) that don't work GM> with MK. Fairly often I have to poke around before I figure out GM> what will work with MK. OTOH, the lack of clutter (and the GM> speed, and the compactness) make it all worthwhile. GM> GM> I know Jean-Claude is working (off & on) on a revised engine GM> that should allow writing a cleaner and more efficient Python GM> interface. GM> ... >> GM> - Gordon >> >> Q1) in the above example, on PyWin the properties list as: >> [, ... >> at the interactive prompt, the same command ( vw.structure() >> ) I get >> [Property('I', 'iid'), Property( ... >> Does anyone know why? GM> I think PythonWin uses repr where the interpreter calls the GM> print slot (or maybe it's vice versa). *** 'tis an odd an curious thing. Both interpreters operate as expected on most classes and builtins. In PyWin (repr or str) it quotes "pyProperty object at", and Python command line (repr or str) it always displays the more useful "Property('I',...". Things that make ya go hmmmm. >> Q2) the primary reason for choosing MetaKet was the nested views. >> When >> I am adding 100,000 records, it seems from the mailing list >> that it will be faster to create a view without the nested >> views, add to that, then re-structure. Will this save me >> some time? GM> Sorry, I can't answer that. The only MK app I have that has GM> counts approaching that doesn't use subviews. >> Wish 1) method to import / export Python structure direct to/from >> a PyRow. >> Maybe by exposing a __dict__ interface, including the >> dictionary methods. GM> I don't know why you're asking for that. A row has attributes, GM> gotten through the standard getattr mechanism. Setting a row GM> from a Python object will see if the object is a dictionary, then GM> it will try list (not recommended, since it's not very safe), and GM> then will try sucking attributes from whatever it is. *** access to dictionary methods keys, items, values, has_attr would be useful. Right now if I pas a PyRowRef to anything, I ALWAYS have to pass more information, because it loses all context. There is no way, AFAIK, to know what attribute names are available, valid, or even how many of them there are. Nor do I know what view it is a member of or the index. I end up always passing the view and a dictionary to all my wrapper classes so I can verify the field (row) names, since the whole point to my project is that the user can create all but a small subset of rows on their own. GM> The only thing I see here that's non-Python is that you have to GM> ask the view what the properties of the row are. >> Wish 3) have vw = vw * 1000 create 1000 new rows in current view GM> Unfortunately, I still don't understand the C++ API well enough GM> to know why the properties get lost. GM> - Gordon -- Best regards, Jerry From pvision@planet-d.net Wed Jun 13 22:39:01 2001 Received: from relais-int8.globalintranet.net (mailgate.globalintranet.net [194.206.181.244]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA06458 for ; Wed, 13 Jun 2001 22:39:00 -0500 Received: from STARGATE ([10.255.7.37]) by relais-int8.globalintranet.net (Netscape Messaging Server 4.15) with ESMTP id GEWVXK00.V2Q for ; Thu, 14 Jun 2001 10:36:56 +0200 Received: from m2881 ([10.228.8.190]) by STARGATE (Lotus Domino Release 5.07a) with SMTP id 2001061410375650:1027 ; Thu, 14 Jun 2001 10:37:56 +0200 Message-ID: <000801c0f4ae$1a328820$be08e40a@m2881> From: "Olivier Brunet" To: Date: Thu, 14 Jun 2001 10:43:27 +0200 MIME-Version: 1.0 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-MIMETrack: Itemize by SMTP Server on stargate/DMZ(Release 5.07a |May 14, 2001) at 14/06/2001 10:37:56, Serialize by Router on stargate/DMZ(Release 5.07a |May 14, 2001) at 14/06/2001 10:37:57, Serialize complete at 14/06/2001 10:37:57 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_01C0F4BE.D849C250" Subject: [Metakit] Threading Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: C'est un message de format MIME en plusieurs parties. ------=_NextPart_000_0005_01C0F4BE.D849C250 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" =20 After reading the Wiki, I'm still not clear about some threading issues. = Using Metakit C++ 2.3.4 on Windows (and after on Linux), I have N = threads who needs to access and update 2 views in one storage. Is this = supported right out of the box ? Or if not what should be added ? And if = yes how ? Currently I have tried 2 ways: 15 threads, opening N times same storage (N<=3D15), getting views and = updating and commiting, seemed to work (no crashes) but seemed to = "forget" some updates on view 15 threads, all sharing the same storage objects, views (global = variables), and doing update and commits =3D> random crashes. Enlighgt me please. ------=_NextPart_000_0005_01C0F4BE.D849C250 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="iso-8859-1"
   
After reading the Wiki, I'm still not = clear about=20 some threading issues. Using Metakit C++ 2.3.4 on Windows (and after on = Linux),=20 I have N threads who needs to access and update 2 views in one storage. = Is this=20 supported right out of the box ? Or if not what should be added ? And if = yes how=20 ?
 
Currently I have tried 2 = ways:
 
15 threads, opening N times same = storage=20 (N<=3D15), getting views and updating and commiting, seemed to work = (no=20 crashes) but seemed to "forget" some updates on view
 
15 threads, all sharing the same = storage objects,=20 views (global variables), and doing update and commits =3D> random=20 crashes.
 
Enlighgt me = please.
------=_NextPart_000_0005_01C0F4BE.D849C250-- From akao347@hotmail.com Thu Jun 14 02:54:24 2001 Received: from hotmail.com (oe35.law4.hotmail.com [216.33.148.28]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA10811 for ; Thu, 14 Jun 2001 02:54:24 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 14 Jun 2001 05:52:31 -0700 X-Originating-IP: [151.203.38.197] From: "A K" To: References: <000801c0f4ae$1a328820$be08e40a@m2881> Subject: Re: [Metakit] Threading Date: Thu, 14 Jun 2001 08:53:35 -0400 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_005B_01C0F4AF.7F399C30" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 14 Jun 2001 12:52:31.0078 (UTC) FILETIME=[DFC1F860:01C0F4D0] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_005B_01C0F4AF.7F399C30 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, Well all the emails this week and the wiki web site seem to say that the = proper way to do this is to give only one thread access to the database, = and have all the threads post to this one thread for data. Brendan Kao ----- Original Message -----=20 From: Olivier Brunet=20 To: metakit@equi4.com=20 Sent: Thursday, June 14, 2001 4:43 AM Subject: [Metakit] Threading =20 After reading the Wiki, I'm still not clear about some threading = issues. Using Metakit C++ 2.3.4 on Windows (and after on Linux), I have = N threads who needs to access and update 2 views in one storage. Is this = supported right out of the box ? Or if not what should be added ? And if = yes how ? Currently I have tried 2 ways: =20 15 threads, opening N times same storage (N<=3D15), getting views and = updating and commiting, seemed to work (no crashes) but seemed to = "forget" some updates on view 15 threads, all sharing the same storage objects, views (global = variables), and doing update and commits =3D> random crashes. Enlighgt me please. ------=_NextPart_000_005B_01C0F4AF.7F399C30 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi,
 
Well all the emails this week and the = wiki web site=20 seem to say that the proper way to do this is to give only one thread = access to=20 the database, and have all the threads post to this one thread for=20 data.
 
 
Brendan Kao
----- Original Message -----
From:=20 Olivier=20 Brunet
Sent: Thursday, June 14, 2001 = 4:43=20 AM
Subject: [Metakit] = Threading

   
After reading the Wiki, I'm still not = clear about=20 some threading issues. Using Metakit C++ 2.3.4 on Windows (and after = on=20 Linux), I have N threads who needs to access and update 2 views in one = storage. Is this supported right out of the box ? Or if not what = should be=20 added ? And if yes how ?
 
Currently I have tried 2 = ways:
 
15 threads, opening N times same = storage=20 (N<=3D15), getting views and updating and commiting, seemed to work = (no=20 crashes) but seemed to "forget" some updates on view
 
15 threads, all sharing the same = storage objects,=20 views (global variables), and doing update and commits =3D> random=20 crashes.
 
Enlighgt me=20 please.
------=_NextPart_000_005B_01C0F4AF.7F399C30-- From gmcm@hypernet.com Thu Jun 14 05:14:06 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA14054 for ; Thu, 14 Jun 2001 05:14:05 -0500 Received: from me (204.176.40.87) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 14 Jun 2001 11:11:26 -0400 From: "Gordon McMillan" To: Jerry McRae Date: Thu, 14 Jun 2001 11:11:23 -0400 Subject: Re: Re[4]: [Metakit] [python] Reply-to: gmcm@hypernet.com CC: Metakit Message-ID: <3B289BDB.6377.473A746@localhost> Priority: normal In-reply-to: <445766549.20010613215009@usd1.com> References: <3B27C94F.23905.13D31FF@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, > Thanks for the reply. The append with __add__ still doesn't work > for me. See below. Then it looks like a bug crept in. I'll try to have a look at CVS and see if I can track it down. [snip] > >> Wish 1) method to import / export Python structure direct > >> to/from a PyRow. > >> Maybe by exposing a __dict__ interface, including the > >> dictionary methods. There are a bunch of issues here, some of them MetaKit's, some Python's. The current row.attr syntax is both very nice and very efficient. It's efficient because it's lazy. In my MK usage, I do a lot of RDB-ish things (searching etc.), normally involving only a few columns. MK shines in that respect, because it's column oriented. Mk4py would be dog slow if it didn't maintain that laziness. So a real Python dict is out. There's also a namespace issue: in a Python type object, methods and attributes live in the same namespace. If a PyRowRef had methods, they'd conflict with column names (actually column names would win). Conceivably, a PyRowRef could implement either the sequence interface (so row[i] would access the ith property) or the mapping interface (so row[i] would access the property named "i"). Python makes doing both interfaces (together) very, very difficult. And the mapping interface is not complete (you need to implement certain methods to make the object really look like a dict - the slot only contains the equivalent of __len__, __getitem__ and __setitem__). Both "items" and "values" are certainly not outlandish names for columns (I've used "values" for a subview), so I think the mapping interface is out. The seq interface is more self contained, but not that good a match. What is row[3]? The value of the 3rd property? The 3rd property (as in PyProperty object)? A tuple (property, value)? You can only assign to the value so only the first choice makes sense. Many seq methods make no sense (slicing? concat? repeat?). So even if implemented, it wouldn't help you. That leaves (unless I've missed something) a (lazily created) special attribute named "__dict__" that does implement the mapping protocol and methods. Maybe I'm being lazy, but that sounds like a lot of work :-). Even the issue of what view does a row belong to (and what's it's index) is not simple. Consider: v = db.view('customers') v2 = v.select(shoesize=48) # European sizes, of course r = v2[0] Now r is a ref to v2[0], but v2 is not a "real" view. It's a wrapper around v, where r has some comletely different index. If you're just reading, you probably care about v2, but if you're trying to update, you're interested in r's position in v. MetaKit is opaque on these issues, but even if it weren't, I don't know how to do a sane API that handles these issues (this one is one-deep; but you can chain this stuff up to arbitrary depths). Jean-Claude has a proof-of-concept of a new engine, which should let Mk4py reach a little deeper into the workings. He's too busy right now to reply even in monosyllabic grunts, (and typically, when he's not busy, I am), so don't take that as anything more than a glimmer of hope :-). - Gordon From barnard@aragorn.stat.harvard.edu Thu Jun 14 06:37:04 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA16896 for ; Thu, 14 Jun 2001 06:37:04 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id MAA12609; Thu, 14 Jun 2001 12:46:02 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15144.59973.685666.665537@aragorn.stat.harvard.edu> Date: Thu, 14 Jun 2001 12:45:57 -0400 (EDT) To: gmcm@hypernet.com Cc: Jerry McRae , Metakit Subject: Re: Re[4]: [Metakit] [python] In-Reply-To: <3B289BDB.6377.473A746@localhost> References: <3B27C94F.23905.13D31FF@localhost> <3B289BDB.6377.473A746@localhost> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gordon McMillan writes: > > >> Wish 1) method to import / export Python structure direct > > >> to/from a PyRow. > > >> Maybe by exposing a __dict__ interface, including the > > >> dictionary methods. > > There are a bunch of issues here, some of them MetaKit's, > some Python's. The current row.attr syntax is both > very nice and very efficient. > It's efficient because it's lazy. In my MK usage, I do a lot of > RDB-ish things (searching etc.), normally involving only a few > columns. MK shines in that respect, because it's column > oriented. Mk4py would be dog slow if it didn't maintain that > laziness. So a real Python dict is out. Agreed. > There's also a namespace issue: in a Python type object, > methods and attributes live in the same namespace. If a > PyRowRef had methods, they'd conflict with column names > (actually column names would win). > > Conceivably, a PyRowRef could implement either the > sequence interface (so row[i] would access the ith property) or > the mapping interface (so row[i] would access the property > named "i"). Python makes doing both interfaces (together) > very, very difficult. And the mapping interface is not complete > (you need to implement certain methods to make the object > really look like a dict - the slot only contains the equivalent of > __len__, __getitem__ and __setitem__). I'm missing something. It would be relatively easy to provide a C function for getitem in the mapping protocol that retrieves the ith element of a row when i is an integer, the element corresponding to the property name i when i is of type string, and the element corresponding to the property i when i is of type PyProperty. I'll provide such a function tomorrow. You're essentially implementing the get/set part of the mapping protocol with special behavior depending on the type of the key -- the sequence protocol only allows you to pass integers to get/set. You could still use the row.propname way to get and set values in addition to the new ways. > Both "items" and "values" are certainly not outlandish names for > columns (I've used "values" for a subview), so I think the mapping > interface is out. Again, I'm missing something. If I add a __getitem__ and __setitem__ in C, I'm not using any names like "items" or "values". You can name the properties anything you want. > The seq interface is more self contained, but not that good a > match. What is row[3]? The value of the 3rd property? The 3rd > property (as in PyProperty object)? A tuple (property, value)? > You can only assign to the value so only the first choice > makes sense. Many seq methods make no sense (slicing? > concat? repeat?). So even if implemented, it wouldn't help you. Slicing makes sense -- row[2:4] gives me a row with elements 2 and 3. Granted concat and repeat probably don't, but the beauty of Python is you don't have to implement them. If all we want is to be able to do row[i] with i a number, then we can skip the rest of the sequence protocol. > That leaves (unless I've missed something) a (lazily created) > special attribute named "__dict__" that does implement the > mapping protocol and methods. Maybe I'm being lazy, but that > sounds like a lot of work :-). > > Even the issue of what view does a row belong to (and what's > it's index) is not simple. Consider: > > v = db.view('customers') > v2 = v.select(shoesize=48) # European sizes, of course > r = v2[0] > > Now r is a ref to v2[0], but v2 is not a "real" view. It's a wrapper > around v, where r has some comletely different index. If you're > just reading, you probably care about v2, but if you're trying to > update, you're interested in r's position in v. MetaKit is opaque > on these issues, but even if it weren't, I don't know how to do a > sane API that handles these issues (this one is one-deep; but > you can chain this stuff up to arbitrary depths). Agreed, however, this linkage to arbitrary depths is very powerful. I've been trying to handle some these issues by writing Python class wrappers around PyView and PyRowRef so that I can manage the lifetimes of "child" (derived/linked) views. For example, if I insert a new row into a "master" view, all of its children views are invalidated. It took me awhile to get the wrapper classes the way I wanted, but that's because I kept changing the design! John -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From gmcm@hypernet.com Thu Jun 14 08:49:53 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA20350 for ; Thu, 14 Jun 2001 08:49:52 -0500 Received: from me (204.176.40.91) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 14 Jun 2001 14:47:11 -0400 From: "Gordon McMillan" To: John Barnard Date: Thu, 14 Jun 2001 14:47:09 -0400 Subject: Re: Re[4]: [Metakit] [python] Reply-to: gmcm@hypernet.com CC: Jerry McRae , Metakit Message-ID: <3B28CE6D.7885.5392E5C@localhost> Priority: normal In-reply-to: <15144.59973.685666.665537@aragorn.stat.harvard.edu> References: <3B289BDB.6377.473A746@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John, [me (Gordon)] > > There's also a namespace issue: in a Python type object, > > methods and attributes live in the same namespace. If a > > PyRowRef had methods, they'd conflict with column names > > (actually column names would win). > > Conceivably, a PyRowRef > could implement either the > sequence interface (so row[i] would > access the ith property) or > the mapping interface (so row[i] > would access the property > named "i"). Python makes doing both > interfaces (together) > very, very difficult. And the mapping > interface is not complete > (you need to implement certain > methods to make the object > really look like a dict - the slot > only contains the equivalent of > __len__, __getitem__ and > __setitem__). > > I'm missing something. It would be relatively easy to provide a C > function for getitem in the mapping protocol that retrieves the > ith element of a row when i is an integer, the element > corresponding to the property name i when i is of type string, > and the element corresponding to the property i when i is of type > PyProperty. I'll provide such a function tomorrow. You're > essentially implementing the get/set part of the mapping protocol > with special behavior depending on the type of the key -- the > sequence protocol only allows you to pass integers to get/set. > You could still use the row.propname way to get and set values in > addition to the new ways. Hmm. I think you can get row[3] to work, but you can't get for prop in row: to work (that requires the seq protocol). > > Both "items" and "values" are certainly not outlandish names > for > columns (I've used "values" for a subview), so I think the > mapping > interface is out. > > Again, I'm missing something. If I add a __getitem__ and > __setitem__ in C, I'm not using any names like "items" or > "values". You can name the properties anything you want. But what Jerry *wants* is row.items(), or row.keys(). Those are methods, not slots. He wants to be able to get to the property names through the row instead of the view. Aside from the messy implementation details, the properties belong to the view, not the row, so there are semantic issues as well. > > The seq interface is more self contained, but not that good a > > match. What is row[3]? The value of the 3rd property? The 3rd > > property (as in PyProperty object)? A tuple (property, value)? > > You can only assign to the value so only the first choice > > makes sense. Many seq methods make no sense (slicing? > concat? > repeat?). So even if implemented, it wouldn't help you. > > Slicing makes sense -- row[2:4] gives me a row with elements 2 > and 3. Or maybe it returns a list [prop2, prop3]. If it returned a row, you'd really be getting a rowref to the only row in a temporary view (with data copied into it). So maybe a list [value2, value3] makes more sense. At times like these, I'm glad Guido is the language desinger, not me :-). > Granted concat and repeat probably don't, but the beauty > of Python is you don't have to implement them. If all we want is > to be able to do row[i] with i a number, then we can skip the > rest of the sequence protocol. > > > That leaves (unless I've missed something) a (lazily created) > > special attribute named "__dict__" that does implement the > > mapping protocol and methods. Maybe I'm being lazy, but that > > sounds like a lot of work :-). > > Even the issue of what view > does a row belong to (and what's > it's index) is not simple. > Consider: > > v = db.view('customers') > v2 = > v.select(shoesize=48) # European sizes, of course > r = v2[0] > > > Now r is a ref to v2[0], but v2 is not a "real" view. It's a > wrapper > around v, where r has some comletely different index. > If you're > just reading, you probably care about v2, but if > you're trying to > update, you're interested in r's position in > v. MetaKit is opaque > on these issues, but even if it weren't, > I don't know how to do a > sane API that handles these issues > (this one is one-deep; but > you can chain this stuff up to > arbitrary depths). > > Agreed, however, this linkage to arbitrary depths is very > powerful. I've been trying to handle some these issues by writing > Python class wrappers around PyView and PyRowRef so that I can > manage the lifetimes of "child" (derived/linked) views. For > example, if I insert a new row into a "master" view, all of its > children views are invalidated. It took me awhile to get the > wrapper classes the way I wanted, but that's because I kept > changing the design! If those wrappers point the way to a sane API, I'd love to see them! - Gordon From barnard@aragorn.stat.harvard.edu Fri Jun 15 09:36:51 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA15552 for ; Fri, 15 Jun 2001 09:36:51 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id PAA17654; Fri, 15 Jun 2001 15:45:51 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="mMALseFBLG" Content-Transfer-Encoding: 7bit Message-ID: <15146.26068.846358.125463@aragorn.stat.harvard.edu> Date: Fri, 15 Jun 2001 15:45:24 -0400 (EDT) To: gmcm@hypernet.com Cc: John Barnard , Jerry McRae , Metakit Subject: Re: Re[4]: [Metakit] [python] In-Reply-To: <3B28CE6D.7885.5392E5C@localhost> References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --mMALseFBLG Content-Type: text/plain; charset=us-ascii Content-Description: message body text Content-Transfer-Encoding: 7bit Inspired by the recent discussions, I've made several additions to PyRowRef.cpp. The modified version is attached below. With this modification you now do the following (row is of type PyRowRef): row["var1"] # equivalent to row.var1 row[i] # returns ith element of row, where i is an integer row[i:j] # returns a list containing the elements i through j, where i # and j are integers row[:] # returns all elements of row in a list # can use the usual slice notation, e.g., -4:-2, etc to get # corresponding elements in a list row[i:j:k] # returns elements of the row corresponding to the slice # i:j:k, where k is the step size (extended slice notation) # in a list (i or j or k can be missing) Using same indices as above but with ,1 appended returns the corresponding properties instead of the elements. E.g., row[:,1] returns a list of all of the properties "for" the row, and row["var1",1] returns the property for var1. In fact, it doesn't have to ,1 it can be ,anything (e.g. row[:,10.2]) -- I use the presence of a second index as a flag to return PyProperties instead of elements. It's not as elegant as row.keys(), but it avoids any name conflicts. You can also set elements of rows using the indices: row["var1"] = 10 row[2:4] = ['a', 'b'] # must be a sequence with length >= 2 row[2:6:2] = (1,4) # must be a sequence with length >= 2 row[:] = x # x is a sequence with length >= length of row Note that row[:,1] = x raises as error since :,1 is a passed to the code as a tuple and a tuple index is not supported for setting. If people find this useful, I suggest it be added to the next release of metakit. Best, John --mMALseFBLG Content-Type: text/plain Content-Description: PyRowRef.cpp Content-Disposition: inline; filename="PyRowRef.cpp" Content-Transfer-Encoding: base64 Ly8gIENvcHlyaWdodCAxOTk5IE1jTWlsbGFuIEVudGVycHJpc2VzLCBJbmMuIC0tIHd3dy5t Y21pbGxhbi1pbmMuY29tDQ0KLy8gIENvcHlyaWdodCAoQykgMTk5OS0yMDAwIEplYW4tQ2xh dWRlIFdpcHBsZXIgPGpjd0BlcXVpNC5jb20+DQ0KLy8NDQovLyAgUm93UmVmIGNsYXNzIGlt cGxlbWVudGF0aW9uDQ0KLy8gIE1vZGlmaWVkIGJ5IEouIEJhcm5hcmQgDQ0KDQ0KI2luY2x1 ZGUgPFBXT1NlcXVlbmNlLmg+DQ0KI2luY2x1ZGUgPFBXT051bWJlci5oPg0NCiNpbmNsdWRl ICJQeVJvd1JlZi5oIg0NCiNpbmNsdWRlICJQeVZpZXcuaCINDQojaW5jbHVkZSAiUHlQcm9w ZXJ0eS5oIg0NCg0NCnN0YXRpYyBQeU1ldGhvZERlZiBSb3dSZWZNZXRob2RzW10gPSB7DQ0K ICAgIHswLCAwLCAwLCAwfQ0NCn07DQ0KDQ0Kc3RhdGljIHZvaWQgUHlSb3dSZWZfZGVhbGxv YyhQeVJvd1JlZiAqbykgew0NCiAgICAvL28tPn5QeVJvd1JlZigpOw0NCiAgICBkZWxldGUg bzsNDQp9DQ0KDQ0Kc3RhdGljIGludCBQeVJvd1JlZl9wcmludChQeVJvd1JlZiAqbywgRklM RSAqZiwgaW50KSB7DQ0KICAgIGZwcmludGYoZiwgIjxQeVJvd1JlZiBvYmplY3QgYXQgJXg+ IiwgKGludClvKTsNDQogICAgcmV0dXJuIDA7DQ0KfQ0NCg0NCnN0YXRpYyBQeU9iamVjdCog UHlSb3dSZWZfZ2V0YXR0cihQeVJvd1JlZiAqbywgY2hhciAqbm0pIHsNDQogICAgdHJ5IHsN DQogICAgICAgIFB5T2JqZWN0KiBhdHRyID0gby0+Z2V0UHJvcGVydHlWYWx1ZShubSk7DQ0K ICAgICAgICBpZiAoYXR0cikNDQogICAgICAgICAgICByZXR1cm4gYXR0cjsNDQogICAgfQ0N CiAgICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgICAgICByZXR1cm4gZS50b1B5dGhv bigpOw0NCiAgICB9DQ0KICAgIHJldHVybiBQeV9GaW5kTWV0aG9kKFJvd1JlZk1ldGhvZHMs IChQeU9iamVjdCogKW8sIG5tKTsNDQp9DQ0KDQ0Kc3RhdGljIGludCBQeVJvd1JlZl9zZXRh dHRyKFB5Um93UmVmICpvLCBjaGFyICpubSwgUHlPYmplY3QqIHYpIHsNDQogICAgdHJ5IHsN DQogICAgICAgIFB5UHJvcGVydHkgKnAgPSBvLT5nZXRQcm9wZXJ0eShubSk7DQ0KICAgICAg ICBpZiAocCkgew0NCiAgICAgICAgICAgIGlmICh2KQ0NCiAgICAgICAgICAgICAgICBQeVJv d1JlZjo6c2V0RnJvbVB5dGhvbigqbywgKnAsIHYpOw0NCiAgICAgICAgICAgIGVsc2UgDQ0K ICAgICAgICAgICAgICAgIFB5Um93UmVmOjpzZXREZWZhdWx0KCpvLCAqcCk7DQ0KICAgICAg ICAgICAgUHlfREVDUkVGKHApOw0NCiAgICAgICAgICAgIHJldHVybiAwOw0NCiAgICAgICAg fQ0NCiAgICAgICAgUHlFcnJfU2V0U3RyaW5nKFB5RXhjX0F0dHJpYnV0ZUVycm9yLCAiZGVs ZXRlIG9mIG5vbi1leGlzdGluZyBhdHRyaWJ1dGUiKTsNDQogICAgICAgIHJldHVybiAtMTsN DQogICAgfQ0NCiAgICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgICAgICBlLnRvUHl0 aG9uKCk7DQ0KICAgICAgICByZXR1cm4gLTE7DQ0KICAgIH0NDQp9DQ0KDQ0Kc3RhdGljIGlu dCBQeVJvd19sZW5ndGgoUHlSb3dSZWYgKm8pIHsNDQogIHJldHVybiBvLT5Db250YWluZXIo KS5OdW1Qcm9wZXJ0aWVzKCk7DQ0KfQ0NCg0NCnN0YXRpYyBQeU9iamVjdCogUHlSb3dfZ2V0 aXRlbShQeVJvd1JlZiAqbywgaW50IG4pIHsNDQogICAgdHJ5IHsNDQogICAgICBpZiAobj49 MCAmJiBuIDwgby0+Q29udGFpbmVyKCkuTnVtUHJvcGVydGllcygpKSB7DQ0KCXJldHVybiBv LT5hc1B5dGhvbihvLT5Db250YWluZXIoKS5OdGhQcm9wZXJ0eShuKSk7DQ0KICAgICAgfQ0N CiAgICAgIHRocm93IFBXRXhjZXB0aW9uKFB5RXhjX0luZGV4RXJyb3IsICJJbmRleCBvdXQg b2YgcmFuZ2UiKTsNDQogICAgfQ0NCiAgICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAg ICAgICByZXR1cm4gZS50b1B5dGhvbigpOw0NCiAgICB9DQ0KfQ0NCg0NCnN0YXRpYyBpbnQg UHlSb3dfc2V0aXRlbShQeVJvd1JlZiAqbywgaW50IG4sIFB5T2JqZWN0KiB2KSB7DQ0KICAg IHRyeSB7DQ0KICAgICAgaWYgKG4+PTAgJiYgbiA8IG8tPkNvbnRhaW5lcigpLk51bVByb3Bl cnRpZXMoKSkgew0NCgljb25zdCBjNF9Qcm9wZXJ0eSBwID0gby0+Q29udGFpbmVyKCkuTnRo UHJvcGVydHkobik7DQ0KCWlmICh2KQ0NCgkgIFB5Um93UmVmOjpzZXRGcm9tUHl0aG9uKCpv LCBwLCB2KTsNDQoJZWxzZSANDQoJICBQeVJvd1JlZjo6c2V0RGVmYXVsdCgqbywgcCk7DQ0K CXJldHVybiAwOw0NCiAgICAgIH0NDQogICAgICBQeUVycl9TZXRTdHJpbmcoUHlFeGNfSW5k ZXhFcnJvciwgIkluZGV4IG91dCBvZiByYW5nZSIpOw0NCiAgICAgIHJldHVybiAtMTsNDQog ICAgfQ0NCiAgICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgICAgICBlLnRvUHl0aG9u KCk7DQ0KICAgICAgICByZXR1cm4gLTE7DQ0KICAgIH0NDQp9DQ0KDQ0Kc3RhdGljIFB5T2Jq ZWN0KiBQeVJvd19nZXRzbGljZShQeVJvd1JlZiAqbywgaW50IGlsb3csIGludCBpaGlnaCkg ew0NCiAgdHJ5IHsNDQogICAgaW50IGwgPSBvLT5Db250YWluZXIoKS5OdW1Qcm9wZXJ0aWVz KCk7DQ0KICAgIGlmIChpaGlnaCA8IDApIGloaWdoICs9IGw7DQ0KICAgIGlmIChpbG93ICA8 IDApIGlsb3cgICs9IGw7DQ0KICAgIGlmIChpbG93IDwgMCkgaWxvdyA9IDA7DQ0KICAgIGVs c2UgaWYgKGlsb3cgPiBsKSBpbG93ID0gbDsNDQogICAgaWYgKGloaWdoIDwgMCkgaWhpZ2gg PSAwOw0NCiAgICBlbHNlIGlmIChpaGlnaCA+IGwpIGloaWdoID0gbDsNDQogICAgaWYgKGlo aWdoIDwgaWxvdykgaWhpZ2ggPSBpbG93Ow0NCiAgICANDQogICAgUHlPYmplY3QgKm91dCA9 IFB5TGlzdF9OZXcoaWhpZ2gtaWxvdyk7DQ0KICAgIGludCBpLCByaTsNDQogICAgZm9yKHJp PWlsb3csaT0wOyByaSA8IGloaWdoOyByaSsrLGkrKykgew0NCiAgICAgIFB5TGlzdF9TZXRJ dGVtKG91dCwgaSwgUHlSb3dfZ2V0aXRlbShvLCByaSkpOw0NCiAgICB9DQ0KICAgIHJldHVy biBvdXQ7DQ0KICB9DQ0KICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgIHJldHVybiBl LnRvUHl0aG9uKCk7DQ0KICB9DQ0KfQ0NCg0NCnN0YXRpYyBpbnQgUHlSb3dfc2V0c2xpY2Uo UHlSb3dSZWYgKm8sIGludCBpbG93LCBpbnQgaWhpZ2gsIFB5T2JqZWN0ICp2YWx1ZSkgew0N CiAgdHJ5IHsNDQogICAgaW50IGwgPSBvLT5Db250YWluZXIoKS5OdW1Qcm9wZXJ0aWVzKCk7 DQ0KICAgIGlmIChpaGlnaCA8IDApIGloaWdoICs9IGw7DQ0KICAgIGlmIChpbG93ICA8IDAp IGlsb3cgICs9IGw7DQ0KICAgIGlmIChpbG93IDwgMCkgaWxvdyA9IDA7DQ0KICAgIGVsc2Ug aWYgKGlsb3cgPiBsKSBpbG93ID0gbDsNDQogICAgaWYgKGloaWdoIDwgMCkgaWhpZ2ggPSAw Ow0NCiAgICBlbHNlIGlmIChpaGlnaCA+IGwpIGloaWdoID0gbDsNDQogICAgaWYgKGloaWdo IDwgaWxvdykgaWhpZ2ggPSBpbG93Ow0NCiAgICBpbnQgc2xlbiA9IGloaWdoLWlsb3c7DQ0K DQ0KICAgIGlmIChQeVNlcXVlbmNlX0NoZWNrKHZhbHVlKSAmJiAoUHlPYmplY3RfTGVuZ3Ro KHZhbHVlKSA+PSBzbGVuKSkgew0NCiAgICAgIGludCBpLCByaTsNDQogICAgICBmb3Iocmk9 aWxvdyxpPTA7IHJpIDwgaWhpZ2g7IHJpKyssaSsrKSB7DQ0KCVB5Um93X3NldGl0ZW0obywg cmksIFB5U2VxdWVuY2VfR2V0SXRlbSh2YWx1ZSwgaSkpOw0NCiAgICAgIH0NDQogICAgICBy ZXR1cm4gMDsNDQogICAgfQ0NCiAgICB0aHJvdyBQV0V4Y2VwdGlvbihQeUV4Y19UeXBlRXJy b3IsICJWYWx1ZSBtdXN0IGJlIGEgc2VxdWVuY2Ugd2l0aCBsZW5ndGggPj0gbGVuZ3RoIG9m IHNsaWNlZCByb3ciKTsNDQogIH0NDQogIGNhdGNoKFBXRXhjZXB0aW9uIGUpIHsNDQogICAg ZS50b1B5dGhvbigpOw0NCiAgICByZXR1cm4gLTE7DQ0KICB9DQ0KfQ0NCg0NCnN0YXRpYyBp bnQgc2xpY2VfR2V0SW5kaWNlcyhQeVNsaWNlT2JqZWN0ICpyLCBpbnQgbGVuZ3RoLCANDQog ICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50ICpzdGFydCwgaW50ICpzdG9wLCBpbnQg KnN0ZXApDQ0Kew0NCiAgICBpZiAoci0+c3RlcCA9PSBQeV9Ob25lKSB7DQ0KCSpzdGVwID0g MTsNDQogICAgfSBlbHNlIHsNDQoJaWYgKCFQeUludF9DaGVjayhyLT5zdGVwKSkgcmV0dXJu IC0xOw0NCgkqc3RlcCA9IFB5SW50X0FzTG9uZyhyLT5zdGVwKTsNDQogICAgfQ0NCiAgICBp ZiAoci0+c3RhcnQgPT0gUHlfTm9uZSkgew0NCgkqc3RhcnQgPSAqc3RlcCA8IDAgPyBsZW5n dGgtMSA6IDA7DQ0KICAgIH0gZWxzZSB7DQ0KCWlmICghUHlJbnRfQ2hlY2soci0+c3RhcnQp KSByZXR1cm4gLTE7DQ0KCSpzdGFydCA9IFB5SW50X0FzTG9uZyhyLT5zdGFydCk7DQ0KCWlm ICgqc3RhcnQgPCAwKSAqc3RhcnQgKz0gbGVuZ3RoOw0NCiAgICB9DQ0KICAgIGlmIChyLT5z dG9wID09IFB5X05vbmUpIHsNDQoJKnN0b3AgPSAqc3RlcCA8IDAgPyAtMSA6IGxlbmd0aDsN DQogICAgfSBlbHNlIHsNDQoJaWYgKCFQeUludF9DaGVjayhyLT5zdG9wKSkgcmV0dXJuIC0x Ow0NCgkqc3RvcCA9IFB5SW50X0FzTG9uZyhyLT5zdG9wKTsNDQoJaWYgKCpzdG9wIDwgMCkg KnN0b3AgKz0gbGVuZ3RoOw0NCiAgICB9DQ0KDQ0KICAgIGlmICgqc3RhcnQgPiAobGVuZ3Ro LTEpKSAqc3RhcnQgPSBsZW5ndGg7DQ0KICAgIGlmICgqc3RhcnQgPCAwKSAqc3RhcnQgPSAw Ow0NCiAgICBpZiAoKnN0b3AgPCAtMSkgKnN0b3AgPSAtMTsNDQogICAgZWxzZSBpZiAoKnN0 b3AgPiBsZW5ndGgpICpzdG9wID0gbGVuZ3RoOw0NCiAgICByZXR1cm4gMDsNDQp9DQ0KDQ0K DQ0Kc3RhdGljIGludCBnZXRfc2xpY2UoUHlPYmplY3QgKm9wLCBpbnQgbWF4LCBpbnQgKm5w LCBpbnQgKnNwKSB7DQ0KICAgIGludCBzdGFydCwgc3RvcCwgc3RlcDsNDQoJDQ0KICAgIGlm IChQeVNsaWNlX0NoZWNrKG9wKSkgew0NCglpZiAoc2xpY2VfR2V0SW5kaWNlcygoUHlTbGlj ZU9iamVjdCAqKW9wLCBtYXgsIA0NCgkJCSAgICAgJnN0YXJ0LCAmc3RvcCwgJnN0ZXApID09 IC0xKSByZXR1cm4gLTE7DQ0KCQkNDQoJaWYgKHN0ZXAgIT0gMCkgew0NCgkgICAgaWYgKHN0 ZXAgPCAwKSAqbnAgPSAoc3RvcC1zdGFydCsxK3N0ZXApL3N0ZXA7DQ0KCSAgICBlbHNlICpu cCA9IChzdG9wLXN0YXJ0LTErc3RlcCkvc3RlcDsNDQoJfSBlbHNlIHsNDQoJICAgIGlmIChz dG9wID09IHN0YXJ0KSB7DQ0KCQkqbnAgPSAwOyBzdGVwID0gMTsNDQoJICAgIH0NDQoJICAg IGVsc2UgcmV0dXJuIC0xOw0NCgl9DQ0KCWlmICgqbnAgPCAwKSAqbnAgPSAwOw0NCgkqc3Ag PSBzdGVwOw0NCglyZXR1cm4gc3RhcnQ7DQ0KICAgIH0gIA0NCiAgICByZXR1cm4gLTE7DQ0K fQ0NCg0NCnN0YXRpYyBQeU9iamVjdCogUHlSb3dfZ2V0cHlwcm9waXRlbShQeVJvd1JlZiAq c2VsZiwgUHlPYmplY3QgKm9wKSB7DQ0KICBpbnQgcHJvcGluZGV4Ow0NCiAgdHJ5IHsNDQog ICAgaWYgKFB5SW50X0NoZWNrKG9wKSkgew0NCiAgICAgIGludCBuID0gUHlJbnRfQXNMb25n KG9wKTsNDQogICAgICBpZiAobiA+PSAwICYmIG4gPCBzZWxmLT5Db250YWluZXIoKS5OdW1Q cm9wZXJ0aWVzKCkpIHsNDQoJcmV0dXJuIG5ldyBQeVByb3BlcnR5KHNlbGYtPkNvbnRhaW5l cigpLk50aFByb3BlcnR5KG4pKTsNDQogICAgICB9DQ0KICAgICAgdGhyb3cgUFdFeGNlcHRp b24oUHlFeGNfSW5kZXhFcnJvciwgIkluZGV4IG91dCBvZiByYW5nZSIpOw0NCiAgICB9DQ0K ICAgIGVsc2UgaWYgKFB5U3RyaW5nX0NoZWNrKG9wKSkgew0NCiAgICAgIHByb3BpbmRleCA9 IHNlbGYtPkNvbnRhaW5lcigpLkZpbmRQcm9wSW5kZXhCeU5hbWUoUHlTdHJpbmdfQXNTdHJp bmcob3ApKTsNDQogICAgICBpZiAocHJvcGluZGV4ICE9IC0xKSB7DQ0KCXJldHVybiBuZXcg UHlQcm9wZXJ0eShzZWxmLT5Db250YWluZXIoKS5OdGhQcm9wZXJ0eShwcm9waW5kZXgpKTsN DQogICAgICB9DQ0KICAgICAgdGhyb3cgUFdFeGNlcHRpb24oUHlFeGNfQXR0cmlidXRlRXJy b3IsICJObyBwcm9wZXJ0eSBjb3VsZCBiZSBmb3VuZCB3aXRoIHRoYXQgbmFtZSIpOw0NCiAg ICB9DQ0KICAgIGVsc2UgaWYgKFB5UHJvcGVydHlfQ2hlY2sob3ApKSB7DQ0KICAgICAgcHJv cGluZGV4ID0gc2VsZi0+Q29udGFpbmVyKCkuRmluZFByb3BlcnR5KCgoUHlQcm9wZXJ0eSop IG9wKS0+R2V0SWQoKSk7DQ0KICAgICAgaWYgKHByb3BpbmRleCAhPSAtMSkgew0NCglyZXR1 cm4gbmV3IFB5UHJvcGVydHkoc2VsZi0+Q29udGFpbmVyKCkuTnRoUHJvcGVydHkocHJvcGlu ZGV4KSk7DQ0KICAgICAgfQ0NCiAgICAgIHRocm93IFBXRXhjZXB0aW9uKFB5RXhjX0F0dHJp YnV0ZUVycm9yLCAiR2l2ZW4gcHJvcGVydHkgZG9lcyBub3QgZXhpc3QgZm9yIHRoaXMgcm93 Iik7DQ0KICAgIH0NDQogICAgZWxzZSBpZiAoUHlTbGljZV9DaGVjayhvcCkpIHsNDQogICAg ICBpbnQgaSwgcmksIG5fc3RlcHMsIHN0ZXBfc2l6ZTsNDQogICAgICBpZiAoKHJpID0gZ2V0 X3NsaWNlKG9wLCBzZWxmLT5Db250YWluZXIoKS5OdW1Qcm9wZXJ0aWVzKCksIA0NCgkJCSAg Jm5fc3RlcHMsICZzdGVwX3NpemUpKSA+PSAwKSB7DQ0KCVB5T2JqZWN0ICpvdXQgPSBQeUxp c3RfTmV3KG5fc3RlcHMpOw0NCglmb3IoaT0wOyBpIDwgbl9zdGVwczsgaSsrLCByaSArPSBz dGVwX3NpemUpIHsNDQoJICANDQoJICBQeUxpc3RfU2V0SXRlbShvdXQsIGksIA0NCgkJCSBu ZXcgUHlQcm9wZXJ0eShzZWxmLT5Db250YWluZXIoKS5OdGhQcm9wZXJ0eShyaSkpKTsNDQoJ fQ0NCglyZXR1cm4gb3V0Ow0NCiAgICAgIH0NDQogICAgICB0aHJvdyBQV0V4Y2VwdGlvbihQ eUV4Y19JbmRleEVycm9yLCAiaW52YWxpZCBzbGljZSIpOw0NCiAgICB9DQ0KICB0aHJvdyBQ V0V4Y2VwdGlvbihQeUV4Y19UeXBlRXJyb3IsICJUeXBlIG9mIGluZGV4IG5vdCBzdXBwb3J0 ZWQ6IG11c3QgYmUgb2YgdHlwZSBpbnRlZ2VyLCBzdHJpbmcsIFB5UHJvcGVydHksIG9yIHNs aWNlIik7DQ0KICB9DQ0KICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgIHJldHVybiBl LnRvUHl0aG9uKCk7DQ0KICB9DQ0KfQ0NCg0NCnN0YXRpYyBQeU9iamVjdCogUHlSb3dfZ2V0 cHlpdGVtKFB5Um93UmVmICpzZWxmLCBQeU9iamVjdCAqb3ApIHsNDQogIHRyeSB7DQ0KICAg IGlmIFB5VHVwbGVfQ2hlY2sob3ApIHsNDQogICAgICBpZiAoUHlUdXBsZV9TaXplKG9wKSA+ IDEpIHsNDQoJcmV0dXJuIFB5Um93X2dldHB5cHJvcGl0ZW0oc2VsZiwgUHlUdXBsZV9HZXRJ dGVtKG9wLCAwKSk7DQ0KICAgICAgfQ0NCiAgICB9DQ0KICAgIGVsc2UgaWYgKFB5SW50X0No ZWNrKG9wKSkgew0NCiAgICAgIGludCBuID0gUHlJbnRfQXNMb25nKG9wKTsNDQogICAgICBp ZiAobj49MCAmIG4gPCBzZWxmLT5Db250YWluZXIoKS5OdW1Qcm9wZXJ0aWVzKCkpIHsNDQoJ cmV0dXJuIHNlbGYtPmFzUHl0aG9uKHNlbGYtPkNvbnRhaW5lcigpLk50aFByb3BlcnR5KG4p KTsNDQogICAgICB9DQ0KICAgICAgdGhyb3cgUFdFeGNlcHRpb24oUHlFeGNfSW5kZXhFcnJv ciwgIkluZGV4IG91dCBvZiByYW5nZSIpOw0NCiAgICB9DQ0KICAgIGVsc2UgaWYgKFB5U3Ry aW5nX0NoZWNrKG9wKSkgew0NCiAgICAgIHJldHVybiBQeVJvd1JlZl9nZXRhdHRyKHNlbGYs IFB5U3RyaW5nX0FzU3RyaW5nKG9wKSk7DQ0KICAgIH0NDQogICAgZWxzZSBpZiAoUHlQcm9w ZXJ0eV9DaGVjayhvcCkpIHsNDQogICAgICByZXR1cm4gUHlSb3dSZWZfZ2V0YXR0cihzZWxm LCAoY2hhciopICgoKFB5UHJvcGVydHkqKW9wKS0+TmFtZSgpKSk7DQ0KICAgIH0NDQogICAg ZWxzZSBpZiAoUHlTbGljZV9DaGVjayhvcCkpIHsNDQogICAgICBpbnQgaSwgcmksIG5fc3Rl cHMsIHN0ZXBfc2l6ZTsNDQogICAgICBpZiAoKHJpID0gZ2V0X3NsaWNlKG9wLCBzZWxmLT5D b250YWluZXIoKS5OdW1Qcm9wZXJ0aWVzKCksIA0NCgkJCSAgJm5fc3RlcHMsICZzdGVwX3Np emUpKSA+PSAwKSB7DQ0KCVB5T2JqZWN0ICpvdXQgPSBQeUxpc3RfTmV3KG5fc3RlcHMpOw0N Cglmb3IoaT0wOyBpIDwgbl9zdGVwczsgaSsrLCByaSArPSBzdGVwX3NpemUpIHsNDQoJICBQ eUxpc3RfU2V0SXRlbShvdXQsIGksIFB5Um93X2dldGl0ZW0oc2VsZiwgcmkpKTsNDQoJfQ0N CglyZXR1cm4gb3V0Ow0NCiAgICAgIH0NDQogICAgICB0aHJvdyBQV0V4Y2VwdGlvbihQeUV4 Y19JbmRleEVycm9yLCAiaW52YWxpZCBzbGljZSIpOw0NCiAgICB9DQ0KICB0aHJvdyBQV0V4 Y2VwdGlvbihQeUV4Y19UeXBlRXJyb3IsICJUeXBlIG9mIGluZGV4IG5vdCBzdXBwb3J0ZWQ6 IG11c3QgYmUgb2YgdHlwZSBpbnRlZ2VyLCBzdHJpbmcsIFB5UHJvcGVydHksIG9yIHNsaWNl Iik7DQ0KICB9DQ0KICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgIHJldHVybiBlLnRv UHl0aG9uKCk7DQ0KICB9DQ0KfQ0NCg0NCg0NCnN0YXRpYyBpbnQgUHlSb3dfc2V0cHlpdGVt KFB5Um93UmVmICpzZWxmLCBQeU9iamVjdCAqaW5kZXgsIFB5T2JqZWN0ICp2YWx1ZSkgew0N CiAgdHJ5IHsNDQogICAgaWYgKFB5SW50X0NoZWNrKGluZGV4KSkgew0NCiAgICAgIHJldHVy biBQeVJvd19zZXRpdGVtKHNlbGYsIFB5SW50X0FzTG9uZyhpbmRleCksIHZhbHVlKTsNDQog ICAgfQ0NCiAgICBlbHNlIGlmIChQeVN0cmluZ19DaGVjayhpbmRleCkpIHsNDQogICAgICBy ZXR1cm4gUHlSb3dSZWZfc2V0YXR0cihzZWxmLCBQeVN0cmluZ19Bc1N0cmluZyhpbmRleCks IHZhbHVlKTsNDQogICAgfQ0NCiAgICBlbHNlIGlmIChQeVByb3BlcnR5X0NoZWNrKGluZGV4 KSkgew0NCiAgICAgIHJldHVybiBQeVJvd1JlZl9zZXRhdHRyKHNlbGYsIChjaGFyKikgKCgo UHlQcm9wZXJ0eSopaW5kZXgpLT5OYW1lKCkpLA0NCgkJCSAgICAgIHZhbHVlKTsNDQogICAg fQ0NCiAgICBlbHNlIGlmIChQeVNsaWNlX0NoZWNrKGluZGV4KSkgew0NCiAgICAgIGludCBp LCByaSwgbl9zdGVwcywgc3RlcF9zaXplOw0NCiAgICAgIGlmICgocmkgPSBnZXRfc2xpY2Uo aW5kZXgsIHNlbGYtPkNvbnRhaW5lcigpLk51bVByb3BlcnRpZXMoKSwgDQ0KCQkJICAmbl9z dGVwcywgJnN0ZXBfc2l6ZSkpID49IDApIHsNDQoJaWYgKFB5U2VxdWVuY2VfQ2hlY2sodmFs dWUpICYmIChQeU9iamVjdF9MZW5ndGgodmFsdWUpID49IG5fc3RlcHMpKSB7DQ0KCSAgZm9y KGk9MDsgaSA8IG5fc3RlcHM7IGkrKywgcmkgKz0gc3RlcF9zaXplKSB7DQ0KCSAgICBQeVJv d19zZXRpdGVtKHNlbGYsIHJpLCBQeVNlcXVlbmNlX0dldEl0ZW0odmFsdWUsIGkpKTsNDQoJ ICB9DQ0KCSAgcmV0dXJuIDA7DQ0KCX0NDQoJdGhyb3cgUFdFeGNlcHRpb24oUHlFeGNfVHlw ZUVycm9yLCAiVmFsdWUgbXVzdCBiZSBhIHNlcXVlbmNlIHdpdGggbGVuZ3RoID49IGxlbmd0 aCBvZiBzbGljZWQgcm93Iik7DQ0KICAgICAgfQ0NCiAgICAgIHRocm93IFBXRXhjZXB0aW9u KFB5RXhjX0luZGV4RXJyb3IsICJpbnZhbGlkIHNsaWNlIik7CSAgICANDQogICAgfQ0NCiAg ICB0aHJvdyBQV0V4Y2VwdGlvbihQeUV4Y19UeXBlRXJyb3IsICJUeXBlIG9mIGluZGV4IG5v dCBzdXBwb3J0ZWQ6IG11c3QgYmUgb2YgdHlwZSBpbnRlZ2VyLCBzdHJpbmcsIFB5UHJvcGVy dHksIG9yIHNsaWNlIik7DQ0KICB9DQ0KICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAg IGUudG9QeXRob24oKTsNDQogICAgcmV0dXJuIC0xOw0NCiAgfQ0NCn0NDQoNDQpzdGF0aWMg UHlNYXBwaW5nTWV0aG9kcyBSb3dBc01hcHBpbmcgPSB7DQ0KICAgIChpbnF1aXJ5KVB5Um93 X2xlbmd0aCwJCS8qbXBfbGVuZ3RoKi8NDQogICAgKGJpbmFyeWZ1bmMpUHlSb3dfZ2V0cHlp dGVtLCAgIC8qbXBfc3Vic2NyaXB0Ki8NDQogICAgKG9iam9iamFyZ3Byb2MpUHlSb3dfc2V0 cHlpdGVtLAkvKm1wX2Fzc19zdWJzY3JpcHQqLw0NCn07DQ0KDQ0Kc3RhdGljIFB5U2VxdWVu Y2VNZXRob2RzIFJvd0FzU2VxID0gew0NCiAgICAoaW5xdWlyeSlQeVJvd19sZW5ndGgsIC8v c3FfbGVuZ3RoDQ0KICAgIDAsIC8vc3FfY29uY2F0DQ0KICAgIDAsIC8vc3FfcmVwZWF0DQ0K ICAgIChpbnRhcmdmdW5jKVB5Um93X2dldGl0ZW0sIC8vc3FfaXRlbQ0NCiAgICAoaW50aW50 YXJnZnVuYylQeVJvd19nZXRzbGljZSwgLy9zcV9zbGljZQ0NCiAgICAoaW50b2JqYXJncHJv YylQeVJvd19zZXRpdGVtLCAvL3NxX2Fzc19pdGVtDQ0KICAgIChpbnRpbnRvYmphcmdwcm9j KVB5Um93X3NldHNsaWNlLCAvL3NxX2Fzc19zbGljZQ0NCn07DQ0KDQ0KDQ0KLy8gICAgDQ0K DQ0KUHlUeXBlT2JqZWN0IFB5Um93UmVmdHlwZSA9IHsNDQogICAgUHlPYmplY3RfSEVBRF9J TklUKCZQeVR5cGVfVHlwZSkNDQogICAgMCwNDQogICAgIlB5Um93UmVmIiwNDQogICAgc2l6 ZW9mKFB5Um93UmVmKSwNDQogICAgMCwNDQogICAgKGRlc3RydWN0b3IpUHlSb3dSZWZfZGVh bGxvYywgLyp0cF9kZWFsbG9jKi8NDQogICAgKHByaW50ZnVuYylQeVJvd1JlZl9wcmludCwg Lyp0cF9wcmludCovDQ0KICAgIChnZXRhdHRyZnVuYylQeVJvd1JlZl9nZXRhdHRyLCAvKnRw X2dldGF0dHIqLw0NCiAgICAoc2V0YXR0cmZ1bmMpUHlSb3dSZWZfc2V0YXR0ciwgICAgICAv KnRwX3NldGF0dHIqLw0NCiAgICAoY21wZnVuYykwLCAvKnRwX2NvbXBhcmUqLw0NCiAgICAo cmVwcmZ1bmMpMCwgLyp0cF9yZXByKi8NDQogICAgMCwgICAgICAvKnRwX2FzX251bWJlciov DQ0KICAgICZSb3dBc1NlcSwgIC8qdHBfYXNfc2VxdWVuY2UqLw0NCiAgICAmUm93QXNNYXBw aW5nLCAgICAgIC8qdHBfYXNfbWFwcGluZyovDQ0KfTsNDQoNDQpQeVJvd1JlZjo6UHlSb3dS ZWYoY29uc3QgYzRfUm93UmVmJiBvLCBQeVZpZXcgKm93bmVyKQ0NCiAgOiBQeUhlYWQgKFB5 Um93UmVmdHlwZSksIGM0X1Jvd1JlZihvKSB7DQ0KfQ0NCg0NCiAgICAvLyB3aXRoIHRoYW5r cyB0byBOaWtpIFNwYWhpZXYgZm9yIGltcHJvdmluZyBjb252ZXJzaW9ucyBhbmQgZXJyb3Ig Y2hlY2tzDQ0Kdm9pZCBQeVJvd1JlZjo6c2V0RnJvbVB5dGhvbihjb25zdCBjNF9Sb3dSZWYm IHJvdywgY29uc3QgYzRfUHJvcGVydHkmIHByb3AsIFB5T2JqZWN0KiBhdHRyKSB7DQ0KICAg IHN3aXRjaCAocHJvcC5UeXBlKCkpIHsNDQogICAgICAgIGNhc2UgJ0knOiAgIA0NCiAgICAg ICAgICAgIGlmIChQeUludF9DaGVjayhhdHRyKSkNDQogICAgICAgICAgICAgICAgKChjb25z dCBjNF9JbnRQcm9wJikgcHJvcCkgKHJvdykgPSBQeUludF9BU19MT05HKGF0dHIpOw0NCgkg ICAgZWxzZSBpZiAoYXR0ciAhPSBQeV9Ob25lKQ0NCgkgICAgew0NCgkJUFdPTnVtYmVyIG51 bWJlciAoYXR0cik7DQ0KICAgICAgICAgICAgICAgICgoY29uc3QgYzRfSW50UHJvcCYpIHBy b3ApIChyb3cpID0gKGxvbmcpIG51bWJlcjsNDQoJICAgIH0NDQogICAgICAgICAgICBicmVh azsNDQogICAgICAgIGNhc2UgJ0wnOiAgIA0NCgkgIGlmIChQeUludF9DaGVjayhhdHRyKSkN DQoJICAgICgoY29uc3QgYzRfTG9uZ1Byb3AmKSBwcm9wKSAocm93KSA9IFB5SW50X0FTX0xP TkcoYXR0cik7DQ0KCSAgZWxzZSBpZiAoUHlMb25nX0NoZWNrKGF0dHIpKQ0NCgkgICAgKChj b25zdCBjNF9Mb25nUHJvcCYpIHByb3ApIChyb3cpID0gUHlMb25nX0FzTG9uZyhhdHRyKTsN DQoJICBlbHNlIGlmIChhdHRyICE9IFB5X05vbmUpDQ0KCSAgICB7DQ0KCSAgICAgIFBXT051 bWJlciBudW1iZXIgKGF0dHIpOw0NCgkgICAgICAoKGNvbnN0IGM0X0xvbmdQcm9wJikgcHJv cCkgKHJvdykgPSAobG9uZykgbnVtYmVyOw0NCgkgICAgfQ0NCgkgIGJyZWFrOw0NCiAgICAg ICAgY2FzZSAnRic6ICAgDQ0KICAgICAgICAgICAgaWYgKFB5RmxvYXRfQ2hlY2soYXR0cikp DQ0KICAgICAgICAgICAgICAgICgoY29uc3QgYzRfRmxvYXRQcm9wJikgcHJvcCkgKHJvdykg PSBQeUZsb2F0X0FTX0RPVUJMRShhdHRyKTsNDQoJICAgIGVsc2UgaWYgKGF0dHIgIT0gUHlf Tm9uZSkNDQoJICAgIHsNDQoJCVBXT051bWJlciBudW1iZXIgKGF0dHIpOw0NCiAgICAgICAg ICAgICAgICAoKGNvbnN0IGM0X0Zsb2F0UHJvcCYpIHByb3ApIChyb3cpID0gKGRvdWJsZSkg bnVtYmVyOw0NCgkgICAgfQ0NCiAgICAgICAgICAgIGJyZWFrOw0NCiAgICAgICAgY2FzZSAn RCc6ICAgDQ0KICAgICAgICAgICAgaWYgKFB5RmxvYXRfQ2hlY2soYXR0cikpDQ0KICAgICAg ICAgICAgICAgICgoY29uc3QgYzRfRG91YmxlUHJvcCYpIHByb3ApIChyb3cpID0gUHlGbG9h dF9BU19ET1VCTEUoYXR0cik7DQ0KCSAgICBlbHNlIGlmIChhdHRyICE9IFB5X05vbmUpDQ0K CSAgICB7DQ0KCQlQV09OdW1iZXIgbnVtYmVyIChhdHRyKTsNDQogICAgICAgICAgICAgICAg KChjb25zdCBjNF9Eb3VibGVQcm9wJikgcHJvcCkgKHJvdykgPSAoZG91YmxlKSBudW1iZXI7 DQ0KCSAgICB9DQ0KICAgICAgICAgICAgYnJlYWs7DQ0KICAgICAgICBjYXNlICdTJzogDQ0K ICAgICAgICAgICAgaWYgKFB5U3RyaW5nX0NoZWNrKGF0dHIpKSB7DQ0KICAgICAgICAgICAg ICAgIGM0X0J5dGVzIHRlbXAgKFB5U3RyaW5nX0FTX1NUUklORyhhdHRyKSwNDQoJCQkJUHlT dHJpbmdfR0VUX1NJWkUoYXR0cikgKyAxLCBmYWxzZSk7DQ0KICAgICAgICAgICAgICAgIHBy b3AgKHJvdykuU2V0RGF0YSh0ZW1wKTsNDQogICAgICAgICAgICB9DQ0KCSAgICBlbHNlIGlm IChhdHRyICE9IFB5X05vbmUpDQ0KCQl0aHJvdyBQV0V4Y2VwdGlvbihQeUV4Y19UeXBlRXJy b3IsICJ3cm9uZyB0eXBlIGZvciBTdHJpbmdQcm9wIik7DQ0KICAgICAgICAgICAgYnJlYWs7 DQ0KICAgICAgICBjYXNlICdWJzogICANDQogICAgICAgICAgICBpZiAoUHlWaWV3X0NoZWNr KGF0dHIpKSB7DQ0KICAgICAgICAgICAgICAgIFB5VmlldyAqb2JqID0gKFB5VmlldyopYXR0 cjsNDQogICAgICAgICAgICAgICAgKChjb25zdCBjNF9WaWV3UHJvcCYpIHByb3ApKHJvdykg PSAqb2JqOw0NCiAgICAgICAgICAgIH0NDQogICAgICAgICAgICBlbHNlIHsgIA0NCiAgICAg ICAgICAgICAgICAoKGNvbnN0IGM0X1ZpZXdQcm9wJikgcHJvcCkgKHJvdykgPSBjNF9WaWV3 ICgpOw0NCiAgICAgICAgICAgICAgICBQeVZpZXcgdG1wKCgoY29uc3QgYzRfVmlld1Byb3Am KSBwcm9wKShyb3cpKTsNDQogICAgICAgICAgICAgICAgUFdPU2VxdWVuY2UgbHN0KGF0dHIp Ow0NCiAgICAgICAgICAgICAgICB0bXAuU2V0U2l6ZShsc3QubGVuKCkpOw0NCiAgICAgICAg ICAgICAgICBmb3IgKGludCBpPTA7IGk8bHN0LmxlbigpOyBpKyspDQ0KICAgICAgICAgICAg ICAgIHsNDQogICAgICAgICAgICAgICAgICAgIFB5T2JqZWN0KiBlbnRyeSA9IGxzdFtpXTsN DQogICAgICAgICAgICAgICAgICAgIHRtcC5zZXRJdGVtKGksIGVudHJ5KTsNDQogICAgICAg ICAgICAgICAgICAgIFB5X0RFQ1JFRihlbnRyeSk7DQ0KICAgICAgICAgICAgICAgIH0NDQog ICAgICAgICAgICB9DQ0KICAgICAgICAgICAgYnJlYWs7DQ0KICAgICAgICBjYXNlICdCJzoN DQogICAgICAgIGNhc2UgJ00nOiAgIA0NCiAgICAgICAgICAgIGlmIChQeVN0cmluZ19DaGVj ayhhdHRyKSkgew0NCiAgICAgICAgICAgICAgICBjNF9CeXRlcyB0ZW1wIChQeVN0cmluZ19B U19TVFJJTkcoYXR0ciksDQ0KCQkJCVB5U3RyaW5nX0dFVF9TSVpFKGF0dHIpLCBmYWxzZSk7 DQ0KICAgICAgICAgICAgICAgIHByb3AgKHJvdykuU2V0RGF0YSh0ZW1wKTsNDQogICAgICAg ICAgICB9DQ0KCSAgICBlbHNlIGlmIChhdHRyICE9IFB5X05vbmUpDQ0KCQl0aHJvdyBQV0V4 Y2VwdGlvbihQeUV4Y19UeXBlRXJyb3IsICJ3cm9uZyB0eXBlIGZvciBCeXRlUHJvcCIpOw0N CiAgICB9DQ0KfQ0NCg0NCnZvaWQgUHlSb3dSZWY6OnNldERlZmF1bHQoY29uc3QgYzRfUm93 UmVmJiByb3csIGNvbnN0IGM0X1Byb3BlcnR5JiBwcm9wKSB7DQ0KICAgIHN3aXRjaCAocHJv cC5UeXBlKCkpIHsNDQogICAgICAgIGNhc2UgJ0knOg0NCiAgICAgICAgICAgICgoY29uc3Qg YzRfSW50UHJvcCYpIHByb3ApIChyb3cpID0gMDsNDQogICAgICAgICAgICBicmVhazsNDQog ICAgICAgIGNhc2UgJ0wnOg0NCiAJICAgICgoY29uc3QgYzRfTG9uZ1Byb3AmKSBwcm9wKSAo cm93KSA9IDA7DQ0KCSAgICBicmVhazsNDQogICAgICAgIGNhc2UgJ0YnOiANDQogICAgICAg ICAgICAoKGNvbnN0IGM0X0Zsb2F0UHJvcCYpIHByb3ApIChyb3cpID0gMC4wOw0NCiAgICAg ICAgICAgIGJyZWFrOw0NCiAgICAgICAgIGNhc2UgJ0QnOg0NCiAgICAgICAgICAgICAoKGNv bnN0IGM0X0RvdWJsZVByb3AmKSBwcm9wKSAocm93KSA9IDAuMDsNDQogICAgICAgICAgICAg YnJlYWs7DQ0KICAgICAgICBjYXNlICdTJzoNDQogICAgICAgICAgICAoKGNvbnN0IGM0X1N0 cmluZ1Byb3AmKSBwcm9wKSAocm93KSA9ICIiOw0NCiAgICAgICAgICAgIGJyZWFrOw0NCiAg ICAgICAgY2FzZSAnVic6ICAgDQ0KICAgICAgICAgICAgKChjb25zdCBjNF9WaWV3UHJvcCYp IHByb3ApIChyb3cpID0gYzRfVmlldyAoKTsNDQogICAgICAgICAgICBicmVhazsNDQogICAg ICAgIGNhc2UgJ0InOg0NCiAgICAgICAgY2FzZSAnTSc6DQ0KICAgICAgICAgICAgew0NCiAg ICAgICAgICAgICAgICBjNF9CeXRlcyB0ZW1wOw0NCiAgICAgICAgICAgICAgICBwcm9wIChy b3cpLlNldERhdGEodGVtcCk7DQ0KICAgICAgICAgICAgfQ0NCiAgICB9DQ0KfQ0NCg0NClB5 T2JqZWN0KiBQeVJvd1JlZjo6YXNQeXRob24oY29uc3QgYzRfUHJvcGVydHkmIHByb3ApIHsN DQogICAgc3dpdGNoIChwcm9wLlR5cGUoKSkgew0NCiAgICBjYXNlICdJJzogew0NCiAgICAg ICAgUFdPTnVtYmVyIHJzbHQoKChjb25zdCBjNF9JbnRQcm9wJilwcm9wKSgqdGhpcykpOw0N CiAgICAgICAgcmV0dXJuIHJzbHQuZGlzT3duKCk7DQ0KICAgICAgICAgICAgICB9DQ0KICAg IGNhc2UgJ0wnOiB7DQ0KICAgICAgcmV0dXJuIFB5TG9uZ19Gcm9tTG9uZygoKGNvbnN0IGM0 X0xvbmdQcm9wJilwcm9wKSgqdGhpcykpOw0NCiAgICAgICAgICAgICAgfQ0NCiAgICBjYXNl ICdGJzogew0NCiAgICAgICAgUFdPTnVtYmVyIHJzbHQoKChjb25zdCBjNF9GbG9hdFByb3Am KXByb3ApKCp0aGlzKSk7DQ0KICAgICAgICByZXR1cm4gcnNsdC5kaXNPd24oKTsNDQogICAg ICAgICAgICAgIH0NDQogICAgY2FzZSAnRCc6IHsNDQogICAgICAgIFBXT051bWJlciByc2x0 KCgoY29uc3QgYzRfRG91YmxlUHJvcCYpcHJvcCkoKnRoaXMpKTsNDQogICAgICAgIHJldHVy biByc2x0LmRpc093bigpOw0NCiAgICAgICAgICAgICAgfQ0NCiAgICBjYXNlICdTJzogew0N CiAgICAgICAgLy9jNF9TdHJpbmcgdG1wID0gKChjNF9TdHJpbmdQcm9wJilwcm9wKS5HZXQo KnRoaXMpOw0NCiAgICAgICAgUFdPU3RyaW5nIHJzbHQoKChjNF9TdHJpbmdQcm9wJilwcm9w KS5HZXQoKnRoaXMpKTsNDQogICAgICAgIHJldHVybiByc2x0LmRpc093bigpOw0NCiAgICAg ICAgICAgICAgfQ0NCiAgICBjYXNlICdWJzogew0NCiAgICAgICAgcmV0dXJuIG5ldyBQeVZp ZXcoKChjb25zdCBjNF9WaWV3UHJvcCYpcHJvcCkoKnRoaXMpKTsNDQogICAgICAgICAgICAg IH0NDQogICAgY2FzZSAnQic6DQ0KICAgIGNhc2UgJ00nOiB7DQ0KICAgICAgICBjNF9CeXRl cyB0ZW1wOw0NCiAgICAgICAgcHJvcCAoKnRoaXMpLkdldERhdGEodGVtcCk7DQ0KICAgICAg ICBQV09TdHJpbmcgcnNsdCgoY29uc3QgY2hhciopdGVtcC5Db250ZW50cygpLCB0ZW1wLlNp emUoKSk7DQ0KICAgICAgICByZXR1cm4gcnNsdC5kaXNPd24oKTsNDQogICAgICAgICAgICAg IH0NDQogICAgfQ0NCiAgICByZXR1cm4gMDsNDQp9DQ== --mMALseFBLG-- From jerry@usd1.com Fri Jun 15 10:21:54 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA16986 for ; Fri, 15 Jun 2001 10:21:53 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f5FKJui00232; Fri, 15 Jun 2001 13:19:57 -0700 (PDT) Date: Fri, 15 Jun 2001 13:19:24 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <11218023581.20010615131924@usd1.com> To: John Barnard CC: gmcm@hypernet.com, Metakit Subject: Re[6]: [Metakit] [python] In-reply-To: <15146.26068.846358.125463@aragorn.stat.harvard.edu> References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi John, This sounds very useful. General Question: >MyList = row[:] >NewList = DoSomethingWith(MyList) >row[:]= NewList As long as NewList has the same surface level element types and number as MyList, this will work? Even with nested views? Specific Question: Can someone make an updated Mk4Py.dll for me, for handle snakes but I do not C. -- Thanks!, Jerry Friday, June 15, 2001, 12:45:24 PM, you wrote: JB> Inspired by the recent discussions, I've made several additions to JB> PyRowRef.cpp. The modified version is attached below. With this JB> modification you now do the following (row is of type PyRowRef): JB> row["var1"] # equivalent to row.var1 JB> row[i] # returns ith element of row, where i is an integer JB> row[i:j] # returns a list containing the elements i through j, where i JB> # and j are integers JB> row[:] # returns all elements of row in a list JB> # can use the usual slice notation, e.g., -4:-2, etc to get JB> # corresponding elements in a list JB> row[i:j:k] # returns elements of the row corresponding to the slice JB> # i:j:k, where k is the step size (extended slice notation) JB> # in a list (i or j or k can be missing) JB> Using same indices as above but with ,1 appended returns the JB> corresponding properties instead of the elements. E.g., row[:,1] JB> returns a list of all of the properties "for" the row, and JB> row["var1",1] returns the property for var1. In fact, it doesn't have JB> to ,1 it can be ,anything (e.g. row[:,10.2]) -- I use the presence of JB> a second index as a flag to return PyProperties instead of JB> elements. It's not as elegant as row.keys(), but it avoids any name JB> conflicts. JB> You can also set elements of rows using the indices: JB> row["var1"] = 10 JB> row[2:4] = ['a', 'b'] # must be a sequence with length >= 2 JB> row[2:6:2] = (1,4) # must be a sequence with length >= 2 JB> row[:] = x # x is a sequence with length >= length of row JB> Note that row[:,1] = x raises as error since :,1 is a passed to the JB> code as a tuple and a tuple index is not supported for setting. JB> If people find this useful, I suggest it be added to the next release JB> of metakit. JB> Best, JB> John From barnard@aragorn.stat.harvard.edu Fri Jun 15 10:29:51 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA17247 for ; Fri, 15 Jun 2001 10:29:50 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id QAA17731; Fri, 15 Jun 2001 16:38:50 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15146.29273.140550.390713@aragorn.stat.harvard.edu> Date: Fri, 15 Jun 2001 16:38:49 -0400 (EDT) To: Jerry McRae Cc: John Barnard , gmcm@hypernet.com, Metakit Subject: Re[6]: [Metakit] [python] In-Reply-To: <11218023581.20010615131924@usd1.com> References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> <11218023581.20010615131924@usd1.com> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, Jerry McRae writes: > This sounds very useful. Great. > General Question: > >MyList = row[:] > >NewList = DoSomethingWith(MyList) > >row[:]= NewList > > As long as NewList has the same surface level element types and number > as MyList, this will work? Even with nested views? Yes, I believe it will, although I haven't yet tested it with nested views. E.g., this works as expected row[:] = [x**2 for x in row[:]] i.e., replaces the elements of the row with their squared values. As long as the values you're assigning to a row position are accepted types for that property type, it should work (including subviews). > Specific Question: Can someone make an updated Mk4Py.dll for me, for > handle snakes but I do not C. What version of Python are you using. I have one for Python 2.1. John From barnard@aragorn.stat.harvard.edu Fri Jun 15 10:40:20 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA17653 for ; Fri, 15 Jun 2001 10:40:20 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id QAA17747; Fri, 15 Jun 2001 16:49:18 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15146.29902.477687.839249@aragorn.stat.harvard.edu> Date: Fri, 15 Jun 2001 16:49:18 -0400 (EDT) To: John Barnard Cc: Jerry McRae , gmcm@hypernet.com, Metakit Subject: Re[6]: [Metakit] [python] In-Reply-To: <15146.29273.140550.390713@aragorn.stat.harvard.edu> References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> <11218023581.20010615131924@usd1.com> <15146.29273.140550.390713@aragorn.stat.harvard.edu> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John Barnard writes: > Jerry, > Yes, I believe it will, although I haven't yet tested it with nested > views. E.g., this works as expected > > row[:] = [x**2 for x in row[:]] A simpler and more efficient (saves on creating a list of the row elements) but equivalent form is row[:] = [x**2 for x in row] John From rcohen@dial.oleane.com Fri Jun 15 10:54:40 2001 Received: from apicra.wanadoo.fr (smtp-rt-3.wanadoo.fr [193.252.19.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA18258 for ; Fri, 15 Jun 2001 10:54:40 -0500 Received: from andira.wanadoo.fr (193.252.19.152) by apicra.wanadoo.fr; 15 Jun 2001 22:52:40 +0200 Received: from dial.oleane.com (217.128.52.6) by andira.wanadoo.fr; 15 Jun 2001 22:52:31 +0200 Message-ID: <3B2A8387.33367DEB@dial.oleane.com> Date: Fri, 15 Jun 2001 22:52:07 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> <11218023581.20010615131924@usd1.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] MacOs crash Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I made a little program that create 10000 rows of very simple data and store it in the file with metakit. (see at end of mail) It works perfectly on Windows and Solaris. But on MacOs 8 it crashes. If I reduce MAX to 9000 it runs ok. Even with 50Mb allocated to the app, it still crashes for 10000 insertions and not for 9000. After about 8500 times view.Add() is called, a huge bloc of memory is requested to the system (I saw that on the system panel "about your computer" where memory is displayed). then another, and the app that was 2 or 3 megas is now about 30 Mb. (at 9000) I beleive at 10000 it goes to more than 50Mb ! I dont understand why it happens. I tried to call Commit() every 1000 Add, but it did not change anything Is there any way to solve that ? I would like it to work on MacOs 8 and then not use the mapped memory that would be available on MacOs 9.1 (anyway I would not know how to set it). Thanks for any Idea. I cannot use metakit if a 10000 rows insert is crashing with no warn...gulp -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 ====================================================================== #define MAX 10000 long idx; c4_StringProp id ("id"); c4_IntProp index("index"); c4_DoubleProp val("val"); c4_BytesProp blob("blob"); c4_Storage database("test1.vxb", true); c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); c4_Row row; for (idx=0;idx; Sun, 17 Jun 2001 03:33:01 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sun, 17 Jun 2001 06:18:25 -0700 X-Originating-IP: [151.203.35.48] From: "A K" To: "Mailing list Metakit" References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> <11218023581.20010615131924@usd1.com> <3B2A8387.33367DEB@dial.oleane.com> Subject: Re: [Metakit] MacOs crash Date: Sun, 17 Jun 2001 09:19:28 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 17 Jun 2001 13:18:25.0382 (UTC) FILETIME=[FD6F3060:01C0F72F] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, or does it also work on the old Mac OS'es as well? Thank you, Brendan Kao ----- Original Message ----- From: "Riccardo Cohen" To: "Mailing list Metakit" Sent: Friday, June 15, 2001 5:52 PM Subject: [Metakit] MacOs crash > Hello, > > I made a little program that create 10000 rows of very simple data and store it in the file with metakit. > (see at end of mail) > > It works perfectly on Windows and Solaris. But on MacOs 8 it crashes. If I reduce MAX to 9000 it runs ok. > > Even with 50Mb allocated to the app, it still crashes for 10000 insertions and not for 9000. > > After about 8500 times view.Add() is called, a huge bloc of memory is requested to the system (I > saw that on the system panel "about your computer" where memory is displayed). then another, and > the app that was 2 or 3 megas is now about 30 Mb. (at 9000) > I beleive at 10000 it goes to more than 50Mb ! > > I dont understand why it happens. > > I tried to call Commit() every 1000 Add, but it did not change anything > > > Is there any way to solve that ? I would like it to work on MacOs 8 and then not use the mapped > memory that would be available on MacOs 9.1 (anyway I would not know how to set it). > > > Thanks for any Idea. I cannot use metakit if a 10000 rows insert is crashing with no warn...gulp > -- > Riccardo Cohen > > Articque > Les Roches > 37230 Fondettes > France > email = rcohen@dial.oleane.com > web = http://www.articque.com > tel: +33 02 47 49 90 49 > fax: +33 02 47 49 91 49 > > ====================================================================== > #define MAX 10000 > long idx; > c4_StringProp id ("id"); > c4_IntProp index("index"); > c4_DoubleProp val("val"); > c4_BytesProp blob("blob"); > c4_Storage database("test1.vxb", true); > c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); > c4_Row row; > for (idx=0;idx { > sprintf(tmp,"id%ld",idx); > id(row)=tmp; > index(row)=idx; > val(row)=idx*195.0465; > blob(row)=c4_Bytes(tmp,strlen(tmp)+1); > table1.Add(row); > } > database.Commit(); > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From berdahl@intelligentparadigm.com Mon Jun 18 08:41:49 2001 Received: from intelligentparadigm.com (sdsl-64-139-36-165.dsl.sjc.megapath.net [64.139.36.165]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA09710 for ; Mon, 18 Jun 2001 08:41:47 -0500 Received: from [10.0.1.17] ([10.0.1.17] verified) by intelligentparadigm.com (Stalker SMTP Server 1.8b8) with ESMTP id S.0000214425 for ; Mon, 18 Jun 2001 11:39:18 -0700 From: Eric Berdahl To: Mailing list Metakit Subject: Re: [Metakit] MacOs crash Date: Mon, 18 Jun 2001 11:38:30 -0700 Message-Id: <20010618183830.27984@mail.intelligentparadigm.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A K wrote: >I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, >or does it also work on the old Mac OS'es as well? I don't know if MetaKit runs on MacOS X, but I would expect to work fairly well there. I'm using it right now on MacOS 9.x in a couple applications. Works first rate. Hope this helps, Eric ----------------------------------------------------------------------- Eric Berdahl "Humor is like a frog -- you can find out Man Behind the Curtain how it works by dissection, but the frog Intelligent Paradigm tends to die in the process." berdahl@serendipity.org - Unknown berdahl@intelligentparadigm.com From rcohen@dial.oleane.com Tue Jun 19 00:48:46 2001 Received: from magnolia.wanadoo.fr (smtp-rt-11.wanadoo.fr [193.252.19.62]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id AAA25111 for ; Tue, 19 Jun 2001 00:48:45 -0500 Received: from citronier.wanadoo.fr (193.252.19.222) by magnolia.wanadoo.fr; 19 Jun 2001 12:46:28 +0200 Received: from dial.oleane.com (193.252.49.172) by citronier.wanadoo.fr; 19 Jun 2001 12:46:28 +0200 Message-ID: <3B2F3B78.39C9B918@dial.oleane.com> Date: Tue, 19 Jun 2001 12:46:00 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Subject: Re: [Metakit] MacOs crash References: <20010618183830.27984@mail.intelligentparadigm.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Eric Berdahl wrote: > > A K wrote: > > >I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, > >or does it also work on the old Mac OS'es as well? > > I don't know if MetaKit runs on MacOS X, but I would expect to work > fairly well there. > > I'm using it right now on MacOS 9.x in a couple applications. Works first > rate. > > Hope this helps, > Eric > Did you have any crash like mine ? -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From berdahl@intelligentparadigm.com Tue Jun 19 06:16:11 2001 Received: from intelligentparadigm.com (sdsl-64-139-36-165.dsl.sjc.megapath.net [64.139.36.165]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA02127 for ; Tue, 19 Jun 2001 06:16:11 -0500 Received: from [216.15.61.23] (HELO [10.0.1.28]) by intelligentparadigm.com (Stalker SMTP Server 1.8b8) with ESMTP id S.0000214685 for ; Tue, 19 Jun 2001 09:13:35 -0700 From: Eric Berdahl To: Mailing list Metakit Subject: Re: [Metakit] MacOs crash Date: Tue, 19 Jun 2001 09:12:15 -0700 Message-Id: <20010619161215.26238@mail.intelligentparadigm.com> In-Reply-To: <3B2F3B78.39C9B918@dial.oleane.com> References: <3B2F3B78.39C9B918@dial.oleane.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: >Eric Berdahl wrote: >> >> I'm using it right now on MacOS 9.x in a couple applications. Works first >> rate. >> >> Hope this helps, >> Eric >> > >Did you have any crash like mine ? Nope. Works first rate for me. Hope this helps, Eric ----------------------------------------------------------------------- Eric Berdahl "Macintosh -- we might not get everything Man Behind the Curtain right, but at least we knew the century Intelligent Paradigm was going to end." berdahl@serendipity.org - Douglas Adams on the Y2K problem berdahl@intelligentparadigm.com From brunobl@ifrance.com Tue Jun 19 08:22:30 2001 Received: from postfix2-2.free.fr (postfix@postfix2-2.free.fr [213.228.0.140]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA05994; Tue, 19 Jun 2001 08:22:29 -0500 Received: from [212.27.37.198] (dijon-37-198.dial.proxad.net [212.27.37.198]) by postfix2-2.free.fr (Postfix) with ESMTP id 61B696B801; Tue, 19 Jun 2001 20:20:40 +0200 (CEST) User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Tue, 19 Jun 2001 20:20:13 +0200 Subject: Re: [Metakit] MacOs crash From: Bruno Blondeau To: , Message-ID: In-Reply-To: <200106171701.MAA22639@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, > or does it also work on the old Mac OS'es as well? It's running fine under the old MacOS as well. I tested it successfuly under MacOS 8, 9 and X. Bruno -- Bruno Blondeau http://www.brunoblondeau.com You can praise them, disagree with them, quote them, disbelieve them, glorify or vilify them. About the only thing you can't do is ignore them. From bryan@tapestrysolutions.com Fri Jun 22 10:24:59 2001 Received: from tapestrysolutions.com (tapestry-2.customer.nethere.net [209.132.102.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA18633 for ; Fri, 22 Jun 2001 10:24:58 -0500 Received: from tapestrysolutions.com (toscana [204.94.74.81]) by tapestrysolutions.com (8.8.7/8.8.7) with ESMTP id OAA12673 for ; Fri, 22 Jun 2001 14:06:13 -0700 Message-ID: <3B33A8EE.D1260786@tapestrysolutions.com> Date: Fri, 22 Jun 2001 13:22:06 -0700 From: Me X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] MacOs crash Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > > I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, > > or does it also work on the old Mac OS'es as well? > > It's running fine under the old MacOS as well. I tested it successfuly under > MacOS 8, 9 and X. > > > Bruno Does anyone have a version of Mk4py.slb that will work with MacPython 2.1? From jimdesu@pacbell.net Fri Jun 22 12:40:40 2001 Received: from JMITCHE27100D (unk-17-26-112.peoplesoft.com [209.17.26.112]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA27035 for ; Fri, 22 Jun 2001 12:40:40 -0500 Received: from [127.0.0.1] by JMITCHE27100D (ArGoSoft Mail Server Plus, Version 1.6 (1.6.0.0)); Fri, 22 Jun 2001 15:38:09 -0700 Message-Id: <5.1.0.14.0.20010622153639.00a72120@postoffice.pacbell.net> X-Sender: jimdesu@postoffice.pacbell.net X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 22 Jun 2001 15:38:07 -0700 To: metakit@equi4.com From: James Mitchell Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] newbie Q: sizing? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I've recently read the explanation & sales-blurb on the metakit web page and am impressed. It mentions that it's not for 1M+ records at the moment, but the blurb is specified to be out of date. Can anyone say what the currently supported volumes are? Hoping to use it, James From readonly@getsoft.com Fri Jun 22 13:32:50 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA27781 for ; Fri, 22 Jun 2001 13:32:49 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id RAA04929; Fri, 22 Jun 2001 17:30:38 -0600 (MDT) Message-ID: <200106230930380193.0422B297@getsoft.com> In-Reply-To: <5.1.0.14.0.20010622153639.00a72120@postoffice.pacbell.net> References: <5.1.0.14.0.20010622153639.00a72120@postoffice.pacbell.net> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Sat, 23 Jun 2001 09:30:38 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "James Mitchell" , metakit@equi4.com Subject: Re: [Metakit] newbie Q: sizing? Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id NAA27781 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: James, In my experience I didn't have much joy using > 10,000 rows. When it got to 100K - 300K I had to give up. There were basic things like sorting, inserting, deleting which would take too long. You can optimize the adds by padding with blank dummy rows, but the code gets convaluted. Further you can't apply this trick to deletes. I also had regular unexplained crashes with large'ish databases. And found problems with sorting, where the sort order wasn't always 100% right. If you need indexed lookup (BTrees) you wan't get any help in MK. This observations are from 2.3.3.28 If the database isn't too dynamic or performance isn't a big issue it may be workable for you. I'm sure other peoples mileage will vary on this. *********** REPLY SEPARATOR *********** On 22 Jun 2001 at 15:38 James Mitchell wrote: >Hello, > >I've recently read the explanation & sales-blurb on the metakit web page >and am impressed. It mentions that it's not for 1M+ records at the moment, >but the blurb is specified to be out of date. Can anyone say what the >currently supported volumes are? > >Hoping to use it, > >James > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find lots of information on ED and while you're there you can click on 'Stay Informed' to join our mailing list and 'Future Plans' to read about Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From jcw@equi4.com Mon Jun 25 16:09:48 2001 Received: from lizzie (bjgl45sdy2xh.bc.hsia.telus.net [216.232.138.208]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA08274 for ; Mon, 25 Jun 2001 16:09:47 -0500 From: Jean-Claude Wippler To: metakit@equi4.com Date: Mon, 25 Jun 2001 19:10:41 -0700 X-Mailer: KMail [version 1.1.99] Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Message-Id: <0106251910410B.14952@lizzie> Content-Transfer-Encoding: 8bit Subject: [Metakit] Upcoming release 2.4.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It's been a while coming... the *final* 2.4.0 release of MetaKit is imminent. The latest changes have just been checked into CVS, info on fetching it is at: http://www.equi4.com/metakit/download.html The change log list all recent details (not really that many since 2.3.4): http://www.equi4.com/metakit/CHANGES Please let me know if I botched anything, forgot to incorporate a patch/fix, or in any way need to make adjustments before freezing 2.4.0 for real. There are still a few days to make minor mods. Please try and exercise it - but do note that a switch from < 2.3.x causes an irreversible file format conversion. Regards, Jean-Claude From rcohen@dial.oleane.com Wed Jun 27 12:00:51 2001 Received: from areca.wanadoo.fr (smtp-rt-4.wanadoo.fr [193.252.19.156]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA07803 for ; Wed, 27 Jun 2001 12:00:50 -0500 Received: from mahonia.wanadoo.fr (193.252.19.58) by areca.wanadoo.fr; 27 Jun 2001 23:57:54 +0200 Received: from dial.oleane.com (217.128.52.54) by mahonia.wanadoo.fr; 27 Jun 2001 23:57:43 +0200 Message-ID: <3B3A64C2.744FEA00@dial.oleane.com> Date: Wed, 27 Jun 2001 23:57:06 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] multiple or single file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I need an information before designing the database : If I have 10 or 20 tables, and some of them are about 10k rows. Will it be faster to store them in separate files or is it all right in the same file ? (I dont need to join, information is independant) Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jerry@usd1.com Thu Jun 28 11:49:10 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA08790 for ; Thu, 28 Jun 2001 11:49:10 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f5SLkCi16883 for ; Thu, 28 Jun 2001 14:46:12 -0700 (PDT) Date: Thu, 28 Jun 2001 14:17:42 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <19516621092.20010628141742@usd1.com> To: metakit Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] python: compare to type of PyProperty Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Can I compare to Mk types without creating or accessing existing objects? From generic modules I can compare a passed object to a string with "obj == type('')" or "import types; obj == types.StringType". The only way I know how to compare to Mk types is to either point to a known variable of that object: "obj == type(vw[0].name)", which is way to specific for the module to know. Or (2) I import mk4py and create dummy copies of all my objects to compare against. Or (3) specific hacks for each type, like all PyProperty items will have an attribute of 'name', so I can USUALLY get a match with: try: getattr(obj,'name') except AttributeError: # not a PyProperty Any other ideas? -- Thanks, Jerry McRae MetaKit 2.3.4, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From jcw@equi4.com Thu Jun 28 23:56:08 2001 Received: from lizzie (bjgl45sdy2xh.bc.hsia.telus.net [216.232.138.208]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA18036; Thu, 28 Jun 2001 23:56:07 -0500 From: Jean-Claude Wippler To: announce@equi4.com, metakit@equi4.com Date: Fri, 29 Jun 2001 02:57:09 -0700 X-Mailer: KMail [version 1.1.99] Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Message-Id: <01062902570906.00686@lizzie> Content-Transfer-Encoding: 8bit Subject: [Metakit] [ANN] MetaKit 2.4.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The latest MetaKit 2.4.0 code has been released. This version is intended for use as stable production release, and now supercedes all previous ones. - This release, as well the previous 2.3.4, has been in production use in several projects in the past months. I advise using 2.4.0 for every new project, and converting all older projects - there have been some changes in the API (such as dropping c4_Storage::Store - use GetAs from now on). - Changes since 2.3.4 are relatively minor, see http://www.equi4.com/metakit/CHANGES - There's an older but still valid summary of changes since 2.01 at: http://www.equi4.com/previews/WHATSNEW - The source archive of this new release is at: http://www.equi4.com/pub/mk/metakit-2.4.0-30.tar.gz - So far, there are binaries for Windows (MSVC 6) and for Linux (gcc 3.0): http://www.equi4.com/pub/mk/ - Be aware that 2.4.0 converts older datafiles during open, and saves the file in a new format on commit. The file size overhead has not really increased, but this conversion may lead to unused gaps (reused later). - Future major releases of MetaKit will only support the new file format, so if you have old datafiles (in use or archived), be sure to use this release whenever you want to convert them. - MetaKit docs continue to be weak and lacking (as well as lagging). - TclKit build 8.4-25 (based on MetaKit 2.4.0) is available from these areas: http://www.equi4.com/pub/tk/tclkit-8.4-25-linux/ http://www.equi4.com/pub/tk/tclkit-8.4-25-win/ The sources for building TclKit are in the MetaKit archive (see tcl/kit/). - The homepages of MetaKit and TclKit are as before, respectively: http://www.equi4.com/metakit/ and http://www.equi4.com/tclkit/ Regards, Jean-Claude P.S. As of tomorrow, I will be off-line until August. Have a nice summer, (or for the growing group of Australian users down-under: winter!) From dherity@iolfree.ie Sat Jun 30 04:12:12 2001 Received: from fargo.iolfree.ie (fargo.iolfree.ie [194.145.128.32]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA25126 for ; Sat, 30 Jun 2001 04:12:12 -0500 Received: from dervish (b-airlock236.esatclear.ie [194.145.131.236]) by fargo.iolfree.ie (8.9.3/8.9.3) with SMTP id PAA18396; Sat, 30 Jun 2001 15:09:02 +0100 Message-ID: <03f001c1016c$a1fa3670$d9a2a5c2@s3group.com> From: "Dominic Herity" To: Cc: Date: Sat, 30 Jun 2001 14:57:41 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_03EB_01C10175.03105D10" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Subject: [Metakit] Metakit Application - Reality Check Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_03EB_01C10175.03105D10 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, I wonder if Metakit users could comment on this possible application. Platform: Windows 98/NT4 or later Compiler: C++ Builder 5 or 6 Database Size: 100K Records, 50MB Medium: CD-ROM. The database and software are both on a CDROM. The software runs off the CD without need for installation. The database is encrypted. (Seems like a use for a Strategy class.) The startup time must be short. My first look at Metakit suggests that its suitable for the purpose, but = I wonder if people familiar with it will see a problem (like scalability, for = example). Comments appreciated. Best Regards Dominic Herity ------=_NextPart_000_03EB_01C10175.03105D10 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi,
 
I wonder if Metakit users could comment = on this=20 possible application.
 
Platform: Windows 98/NT4 or = later
Compiler: C++ Builder 5 or = 6
Database Size: 100K Records, = 50MB
Medium: CD-ROM.
 
The database and software are both on a = CDROM.
The software runs off the CD = without need for=20 installation.
The database is encrypted. (Seems = like a use=20 for a Strategy class.)
The startup time must be = short.
 
My first look at Metakit suggests that = its suitable=20 for the purpose, but I wonder if
people familiar with it will see a = problem (like=20 scalability, for example).
 
Comments appreciated.
 
Best Regards
Dominic Herity
 
------=_NextPart_000_03EB_01C10175.03105D10-- From nrogers@softhome.net Wed Jul 4 08:28:55 2001 Received: from tango.SoftHome.net (tango.SoftHome.net [204.144.231.49]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA10697 for ; Wed, 4 Jul 2001 08:28:55 -0500 Received: (qmail 12145 invoked by uid 417); 4 Jul 2001 18:30:29 -0000 Received: from cx292990-b.lbbck1.tx.home.com (HELO cx292990-b) (24.15.152.161) by i-softhome-tango with SMTP; 4 Jul 2001 18:30:29 -0000 Date: Wed, 4 Jul 2001 13:25:58 -0500 From: Nathan Rogers To: metakit@equi4.com Message-Id: <20010704132558.66b31426.nrogers@softhome.net> X-Mailer: Sylpheed version 0.4.64 (GTK+ 1.2.10; i386-debian-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] hashing bug Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm running MetaKit 2.4.0 on Linux. Most of the time it works great, but I ran into a case where Find() on a hashed view goes into an infinite loop. Here is a simplified test case that hangs every time I run it. #include #include void main() { c4_Storage db; c4_View test_data, test_hash, test; c4_StringProp pKey ("key"); test_data = db.GetAs("test_data[key:S]"); test_hash = db.GetAs("test_hash[_H:I,_R:I]"); test = test_data.Hash(test_hash, 1); test.Add(pKey["b93655249726e5ef4c68e45033c2e0850570e1e07"]); test.Add(pKey["2ab03fba463d214f854a71ab5c951cea096887adf"]); test.Add(pKey["2e196eecb91b02c16c23360d8e1b205f0b3e3fa3d"]); test.Find(pKey["7c0734c9187133f34588517fb5b39294076f22ba3"]); } From ASeifert@XSystemsInc.com Thu Jul 5 07:50:25 2001 Received: from dom01.xsystemsinc.com (hidden-user@xsystems5.vtcrc.net [12.5.16.43] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA15315 for ; Thu, 5 Jul 2001 07:50:25 -0500 Received: from demo2 ([64.204.192.3]) by dom01.xsystemsinc.com (Lotus Domino Release 5.0.7) with SMTP id 2001070513472314:879 ; Thu, 5 Jul 2001 13:47:23 -0400 Message-ID: <008001c1057a$0c9c6690$0300a8c0@xsystemsinc.com> Reply-To: "Allen Seifert" From: "Allen Seifert" To: Date: Thu, 5 Jul 2001 13:43:49 -0400 Organization: X.Systems, Inc. MIME-Version: 1.0 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 X-MIMETrack: Itemize by SMTP Server on dom01/xsystems(Release 5.0.7 |March 21, 2001) at 07/05/2001 01:47:23 PM, Serialize by Router on dom01/xsystems(Release 5.0.7 |March 21, 2001) at 07/05/2001 01:47:24 PM, Serialize complete at 07/05/2001 01:47:24 PM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] where is the latest Mk4Tcl download Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I would like to get the latest Mk4Tcl binary download for Windows - presumably 2.4. All I can find on the Equi4 web site is the 2.0 distribution. Can someone point me to the correct URL for this binary? Thanks, Allen From lvirden@cas.org Thu Jul 5 08:19:36 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA16642 for ; Thu, 5 Jul 2001 08:19:33 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id OAA13412 for ; Thu, 5 Jul 2001 14:15:57 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f65IFuM08480; Thu, 5 Jul 2001 14:15:56 -0400 (EDT) Date: Thu, 5 Jul 2001 14:15:56 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010705141556.AAB8457@cas.org> Subject: Re: [Metakit] where is the latest Mk4Tcl download In-Reply-To: Your message of Thu, 5 Jul 2001 13:43:49 -0400 To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It looks to me as if http://www.equi4.com/pub/mk/metakit-2.4.0-30-win/ is what you want - but I'm confused because it was pretty simple to find it, so I'm uncertain what you might have found. It is the directory you find looking at http://www.equi4.com/metakit/ which is the Metakit web page... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From remy@shockfish.com Tue Jul 10 08:29:03 2001 Received: from berau.travel.ch (postfix@www.abp.ch [193.135.59.43] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA15163 for ; Tue, 10 Jul 2001 08:28:56 -0500 Received: from pc2.shockfish.com (shockfishpc1.epfl.ch [128.179.48.2]) by berau.travel.ch (Postfix) with ESMTP id 94DD512F015 for ; Tue, 10 Jul 2001 20:23:06 +0200 (MEST) Message-Id: <5.0.2.1.0.20010710201542.00a80898@mail1.travel.ch> X-Sender: remymail@mail1.travel.ch X-Mailer: QUALCOMM Windows Eudora Version 5.0.2 Date: Tue, 10 Jul 2001 20:31:48 +0200 To: From: Remy Blank - Shockfish Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Strange implementation of c4_FormatS::DoCompare Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, While studying the implementation of the various format handlers within Metakit, I came across a strange implementation detail in c4_FormatS. This is the function: int c4_FormatS::DoCompare(const c4_Bytes& b1_, const c4_Bytes& b2_) { c4_String v1 ((const char*) b1_.Contents(), b1_.Size()); c4_String v2 ((const char*) b2_.Contents(), b2_.Size()); return v1.CompareNoCase(v2); } While this is totally correct, the construction of v1 and v2 from the contents of b1_ and b2_ means one memory allocation and one full copy each, even *before* they can be compared, and then the release of that memory at the end of the function. c4_String::CompareNoCase() is quite simple, too: int c4_String::CompareNoCase(const char* str) const { return Data() == str ? 0 : stricmp(Data(), str); } Wouldn't it be much more efficient to just inline CompareNoCase() "manually" into c4_FormatS::DoCompare(), working directly on the pointers? This would improve row comparisons, for example when sorting views. I imagine something like: int c4_FormatS::DoCompare(const c4_Bytes& b1_, const c4_Bytes& b2_) { char const* s1 = (const char*) b1_.Contents(); char const* s2 = (const char*) b2_.Contents(); return s1 == s2 ? 0 : stricmp(s1, s2); } Anyway, I'll run a few tests and see what pops out. --Remy From lvirden@cas.org Tue Jul 10 16:29:34 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA22278 for ; Tue, 10 Jul 2001 16:29:33 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id WAA04721 for ; Tue, 10 Jul 2001 22:25:37 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f6B2PYs05947; Tue, 10 Jul 2001 22:25:34 -0400 (EDT) Date: Tue, 10 Jul 2001 22:25:34 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010710222534.AAB5921@cas.org> To: metakit@equi4.com Subject: [Metakit] Wikit help sought Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What are the commands that I have available when interacting with a wikit so that I can look at the history of wikit pages. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From weyhan@my-deja.com Fri Jul 13 05:12:53 2001 Received: from mail22.bigmailbox.com (mail22.bigmailbox.com [209.132.220.199] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA01669 for ; Fri, 13 Jul 2001 05:12:53 -0500 Received: by mail22.bigmailbox.com (8.8.7/8.8.7) id IAA30506; Fri, 13 Jul 2001 08:08:44 -0700 Date: Fri, 13 Jul 2001 08:08:44 -0700 Message-Id: <200107131508.IAA30506@mail22.bigmailbox.com> Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: binary X-Mailer: MIME-tools 4.104 (Entity 4.116) Mime-Version: 1.0 X-Originating-Ip: [203.126.136.170] From: "Wey-Han Ng" To: metakit@equi4.com Subject: [Metakit] s33c: test fail Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I have downloaded and compile Metakit 2.4.0 on Solaris 2.5. When I try to run "make test" I got the following error: ... l07 - Huge description Done. diff ../unix/../tests/ok tests diff ../unix/../tests/ok/s33c.txt tests/s33c.txt 1,6c1 < VIEW 1 rows = a:V < 0: subview 'a' < VIEW 3 rows = p1:B < 0: (2b) < 1: (4b) < 2: (5b) --- > VIEW 1 rows = make: *** [test] Error 1 Is there a problem with my build? ------------------------------------------------------------ --== Sent via Deja.com ==-- http://www.deja.com/ From bryan@tapestrysolutions.com Tue Jul 17 15:33:03 2001 Received: from tapestrysolutions.com (tapestry-2.customer.nethere.net [209.132.102.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA07401 for ; Tue, 17 Jul 2001 15:33:02 -0500 Received: from tapestrysolutions.com (toscana [204.94.74.81]) by tapestrysolutions.com (8.8.7/8.8.7) with ESMTP id TAA13470 for ; Tue, 17 Jul 2001 19:14:16 -0700 Message-ID: <3B54E5E1.21CB677D@tapestrysolutions.com> Date: Tue, 17 Jul 2001 18:26:57 -0700 From: Me X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com References: <200107051701.MAA24376@trixie.triqs.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Where is the latest Mk4py download Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Allen Seifert wrote: > It looks to me as if http://www.equi4.com/pub/mk/metakit-2.4.0-30-win/ is what you > want Does anyone know where I can get the latest Mk4py for the Mac. The only version I can find (http://www.equi4.com/pub/mk/metakit-2.3.3-29-mac/) doesn't work with MacPython 2.1. From guenther.fischer@hrz.tu-chemnitz.de Thu Jun 21 20:42:11 2001 Received: from asterix.hrz.tu-chemnitz.de (asterix.hrz.tu-chemnitz.de [134.109.132.84]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA27209 for ; Thu, 21 Jun 2001 20:42:11 -0500 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by asterix.hrz.tu-chemnitz.de with esmtp (Exim 3.22 #4) id 15N7Gf-0002Ok-00 for metakit@equi4.com; Thu, 19 Jul 2001 08:26:37 +0200 Date: Thu, 19 Jul 2001 08:26:36 +0200 (MEST) From: Guenther Fischer To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] python and mk4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I will done a side step from tcl to python. My first try works wrong: - in my current dir is Mk4py.so and metakit py Type: python Python 1.5.2 (#1, Dec 21 2000, 15:29:08) [GCC egcs-2.91.66 19990314/Linux (egcs- on linux-i386 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import metakit >>> db = metakit.storage("wein.dat",r) Traceback (innermost last): File "", line 1, in ? AttributeError: storage >>> the same error with your Millions.py example. Shurly a silly error, but please any help... -- ~Guenther Fischer From guenther.fischer@hrz.tu-chemnitz.de Mon Jul 23 07:53:35 2001 Received: from obelix.hrz.tu-chemnitz.de (obelix.hrz.tu-chemnitz.de [134.109.132.55]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA17569 for ; Mon, 23 Jul 2001 07:53:34 -0500 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by obelix.hrz.tu-chemnitz.de with esmtp (Exim 3.22 #4) id 15OfE4-0005Pp-00 for metakit@equi4.com; Mon, 23 Jul 2001 14:54:20 +0200 Date: Mon, 23 Jul 2001 14:54:20 +0200 (MEST) From: Guenther Fischer To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Unicode und Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I will read my mk4-database written with Mk4tcl with Python/mk4py - all works fine, but the german umlaut and other european character will fail. As I remember such characters are coded in unicode - I think this happens automatically with mk4tcl, but with mk4py not. Is this true - what can I do? -- ~Guenther Fischer From niki@vintech.bg Mon Jul 23 08:24:23 2001 Received: from www.vintech.bg (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA18439 for ; Mon, 23 Jul 2001 08:24:19 -0500 Received: from vintech.bg (sierra [10.0.0.7]) by www.vintech.bg (8.11.2/8.8.7) with ESMTP id f6NDRZO11871; Mon, 23 Jul 2001 16:27:35 +0300 Message-ID: <3B5C2628.80508@vintech.bg> Date: Mon, 23 Jul 2001 16:27:04 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.2) Gecko/20010628 X-Accept-Language: bg, en MIME-Version: 1.0 To: Guenther Fischer CC: metakit@equi4.com Subject: Re: [Metakit] Unicode und Mk4py References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: > Hi, > I will read my mk4-database written with Mk4tcl with Python/mk4py - > all works fine, but the german umlaut and other european character > will fail. As I remember such characters are coded in unicode - > I think this happens automatically with mk4tcl, but with mk4py not. > > Is this true - what can I do? > > AFAIK metakit has no unicode support. If you use Python >= 2.0 try utf-1 encoding or decoding with string from Mk4py. HTH Niki Spahiev From jw2@scriptthing.com Tue Jul 24 22:51:06 2001 Received: from compuvar04.compuvar.com (HERMES.COMPUVAR.COM [209.101.164.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA17046 for ; Tue, 24 Jul 2001 22:51:04 -0500 Received: from [208.57.85.201] by compuvar04.compuvar.com (NTMail 5.06.0016/NT0699.00.e2308d62) with ESMTP id nznubbaa for metakit@equi4.com; Tue, 24 Jul 2001 20:51:12 -0700 Mime-Version: 1.0 X-Sender: jw2@mail.scriptthing.com Message-Id: Date: Tue, 24 Jul 2001 20:51:29 -0700 To: metakit@equi4.com From: "James W. Walker" Content-Type: text/plain; charset="us-ascii" ; format="flowed" Subject: [Metakit] MK 2.4.0 crashing Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Today I tried updating from MK 2.0.1 to 2.4.0, and it crashes when trying to open files created by the older MK. I don't have a huge amount of time for debugging this before I have to ship a product, so I'll probably have to revert to 2.0.1. But in case anyone has any ideas or relevant experience, here's what I discovered so far: Soon after c4_Persist::LoadAll starts, c4_Strategy::EndOfData sets _rootPos to 0 and _rootLen to -1. Comments in the code say that a _rootLen value of -1 flags an "old-style header", so I guess that's correct, but I don't know what _rootPos is supposed to be. The consequence, though, is that LoadAll sets _oldSeek to 0. Then c4_Persist::FetchOldValue reads the very start of the file, and c4_Column::PullValue tries to interpret the initial 'LJ' etc. as a number. -- James W. Walker, ScriptPerfection Enterprises, Inc. From titoffen@pacbell.net Sat Jul 28 23:18:49 2001 Received: from snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19192 for ; Sat, 28 Jul 2001 23:18:48 -0500 Message-Id: <200107290418.XAA19192@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta5.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GH7003EXVZZHT@mta5.snfc21.pbi.net> for metakit@equi4.com; Sat, 28 Jul 2001 21:19:12 -0700 (PDT) Date: Sat, 28 Jul 2001 21:20:03 -0700 From: Tito Ciuro To: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-3-893334421 Subject: [Metakit] Search() and Locate() are broken! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-3-893334421 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hello, To my surprise, I found out that both, Search() and Locate() are broken. If I'm not mistaken, v2.3.4r29 and v2.4.0 contain the same code for both functions. Assumptions: ------------ The data file contains a specific record with ID 604 at index 601. The file does not contain any negative numbers. There are 640 rows in total. Testing Search(): -------------- index = dataFileView.Search(pUID[604]); ---> index = 601 OK! index = dataFileView.Search(pUID[999]); ---> index = 640 (last + 1) OK! index = dataFileView.Search(pUID[-1]); ---> index = 0 WRONG! Comments on Search(): -------------------- Searching ID 604 works fine. Searching ID 999 works fine, but perhaps it would be better to obtain index = -1 (such as in Find(), for example.) Searching ID -1, which doesn't exist, returns index 0, which is wrong (index 0 contains ID 1 in my case). Testing Search(): -------------- count = dataFileView.Locate(pUID[604], &index); ---> count = 0; index = 601 WRONG! count = dataFileView.Locate(pUID[999], &index); ---> count = 0; index = 640 (last + 1) OK! count = dataFileView.Locate(pUID[-1], &index); ---> count = 0; index = 0 WRONG! Comments on Locate(): -------------------- Locating ID 604 doesn't work. It should be count = 1; index = 601. Locating ID 999 works fine, but perhaps it would be better to obtain index = -1 (such as in Find(), for example.) Locating ID -1, which doesn't exist, returns index 0, which is wrong (index 0 contains ID 1 in my case). Final comment: -------------- I can solve this issue by using Find(), but its linear search isn't as optimized as Search() or Locate(). It really shows when profiling the code... :( Has anyone seen this before? Thanks, -- Tito --Apple-Mail-3-893334421 Content-Transfer-Encoding: quoted-printable Content-Type: text/enriched; charset=us-ascii CourierHello, To my surprise, I found out that both, Search() and Locate() are broken. If I'm not mistaken, v2.3.4r29 and v2.4.0 contain the same code for both functions. Assumptions: ------------ The data file contains a specific record with ID 604 at index 601. The file does not contain any negative numbers. There are 640 rows in total. Testing Search(): -------------- index =3D dataFileView.Search(pUID[604]); ---> index =3D 601 OK! index =3D dataFileView.Search(pUID[999]); ---> index =3D 640 (last + 1) = OK! index =3D dataFileView.Search(pUID[-1]); ---> index =3D 0 WRONG! Comments on Search(): -------------------- Searching ID 604 works fine. Searching ID 999 works fine, but perhaps it would be better to obtain index =3D -1 (such as in Find(), for example.) Searching ID -1, which doesn't exist, returns index 0, which is wrong (index 0 contains ID 1 in my case). Testing Search(): -------------- count =3D dataFileView.Locate(pUID[604], &index); ---> count =3D 0; = index =3D 601 WRONG! count =3D dataFileView.Locate(pUID[999], &index); ---> count =3D 0; = index =3D 640 (last + 1) OK! count =3D dataFileView.Locate(pUID[-1], &index); ---> count =3D 0; index = =3D 0 WRONG! Comments on Locate(): -------------------- Locating ID 604 doesn't work. It should be count =3D 1; index =3D 601. Locating ID 999 works fine, but perhaps it would be better to obtain index =3D -1 (such as in Find(), for example.) Locating ID -1, which doesn't exist, returns index 0, which is wrong (index 0 contains ID 1 in my case). Final comment: -------------- I can solve this issue by using Find(), but its linear search isn't as optimized as Search() or Locate(). It really shows when profiling the code... :( Has anyone seen this before? Thanks, -- Tito= --Apple-Mail-3-893334421-- From lvirden@cas.org Sun Jul 29 04:49:03 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA29121 for ; Sun, 29 Jul 2001 04:49:03 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id f6T9mrA02589 for ; Sun, 29 Jul 2001 05:48:53 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f6T9mre25023; Sun, 29 Jul 2001 05:48:53 -0400 (EDT) Date: Sun, 29 Jul 2001 05:48:53 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010729054853.AAB25010@cas.org> To: metakit@equi4.com Subject: [Metakit] Wikit question - anyone looked into the caching code? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've an idea for what seems like a simple improvement to the wikit's caching code and wondered if anyone was familar with this part of the code? I'd suggest that as the wikit display code fetchs in the cache, that it check the state of any link marked within the cached'd page as uncreated, and if that state is no longer true, that it update the page and the cache appropriately. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From readonly@getsoft.com Sun Jul 29 04:56:57 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA29470 for ; Sun, 29 Jul 2001 04:56:57 -0500 Received: from 192.168.101.11 (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id DAA23624; Sun, 29 Jul 2001 03:57:05 -0600 (MDT) Date: Sun, 29 Jul 2001 19:56:43 +1000 From: Neville Franks X-Mailer: The Bat! (v1.53d) UNREG / CD5BF9353B3B7091 Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <143105308669.20010729195643@getsoft.com> To: Tito Ciuro CC: metakit@equi4.com Subject: Re: [Metakit] Search() and Locate() are broken! In-Reply-To: <200107290418.XAA19192@trixie.triqs.com> References: <200107290418.XAA19192@trixie.triqs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Tito, Sunday, July 29, 2001, 2:20:03 PM, you wrote: TC> Hello, TC> To my surprise, I found out that both, Search() and Locate() are broken. TC> If I'm not mistaken, v2.3.4r29 and v2.4.0 contain the same code for both TC> functions. ..... TC> Has anyone seen this before? I haven't used Metakit for some time, however I did find the sort code was buggy. You should find an earlier post from me about this. I'm assuming Search() and Locate() work on a sorted view so if this is right then the sort may be the cause of the problem. -- Best regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From gmcm@hypernet.com Sun Jul 29 06:30:25 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA00802 for ; Sun, 29 Jul 2001 06:30:24 -0500 Received: from me (204.176.40.43) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sun, 29 Jul 2001 07:24:48 -0400 From: "Gordon McMillan" To: Tito Ciuro , metakit@equi4.com Date: Sun, 29 Jul 2001 07:29:19 -0400 Subject: Re: [Metakit] Search() and Locate() are broken! Reply-to: gmcm@hypernet.com Message-ID: <3B63BB4F.5013.70E6B97@localhost> Priority: normal In-reply-to: <200107290418.XAA19192@trixie.triqs.com> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, > index = dataFileView.Search(pUID[604]); ---> index = 601 OK! > index = dataFileView.Search(pUID[999]); ---> index = 640 (last + > 1) OK! index = dataFileView.Search(pUID[-1]); ---> index = 0 > WRONG! No, these are correct. Search tells you where the value is or where it belongs (where it should be inserted to preserve order). It does not tell if it exists. > count = dataFileView.Locate(pUID[604], &index); ---> count = 0; > index = 601 WRONG! count = dataFileView.Locate(pUID[999], > &index); ---> count = 0; index = 640 (last + 1) OK! count = > dataFileView.Locate(pUID[-1], &index); ---> count = 0; index = 0 > WRONG! I'm not familiar with Locate, but my guess is that the last test is OK (like Search, telling you where to insert). Looks like you did find a bug with the first test. - Gordon From titoffen@pacbell.net Sun Jul 29 11:54:45 2001 Received: from snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA03888 for ; Sun, 29 Jul 2001 11:54:44 -0500 Message-Id: <200107291654.LAA03888@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta5.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GH80052JUZU7B@mta5.snfc21.pbi.net> for metakit@equi4.com; Sun, 29 Jul 2001 09:55:07 -0700 (PDT) Date: Sun, 29 Jul 2001 09:55:58 -0700 From: Tito Ciuro Subject: Re: [Metakit] Search() and Locate() are broken! In-reply-to: <3B63BB4F.5013.70E6B97@localhost> To: gmcm@hypernet.com Cc: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-2-938689125 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2-938689125 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hi Gordon, On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: > Tito, > >> index = dataFileView.Search(pUID[604]); ---> index = 601 OK! >> index = dataFileView.Search(pUID[999]); ---> index = 640 (last + >> 1) OK! >> index = dataFileView.Search(pUID[-1]); ---> index = 0 >> WRONG! > > No, these are correct. Search tells you where the value is or > where it belongs (where it should be inserted to preserve > order). It does not tell if it exists. Huh?? I would expect something like PositionWhenInserted() to do that, but certainly not Search(). Shouldn't Search() tell me where the match (that is, the index) exists? (either -1 or an out-of-bounds value if it doesn't would be great) In my first post above, how good is Search() if it can't distinguish between 0 being a "this is where it should go" kind of index and 0 being a "it doesn't exist" kind of index? In other words, if Search() returns an index, how do we know if it found (or didn't find) a match? >> count = dataFileView.Locate(pUID[604], &index); ---> count = 0; >> index = 601 WRONG! >> count = dataFileView.Locate(pUID[999], >> &index); ---> count = 0; index = 640 (last + 1) OK! >> count = dataFileView.Locate(pUID[-1], &index); ---> count = 0; >> index = 0 >> WRONG! > > I'm not familiar with Locate, but my guess is that the last test > is OK (like Search, telling you where to insert). Looks like you > did find a bug with the first test. Locate() is defined in c4_View in the following manner: > /// Return number of matching keys, and pos of first one as arg > int c4_View::Locate(const c4_RowRef& crit_, int* pos_) const It seems a pretty straightforward call... Thanks for replying Gordon! -- Tito --Apple-Mail-2-938689125 Content-Transfer-Encoding: quoted-printable Content-Type: text/enriched; charset=us-ascii Hi Gordon, On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: Tito, index =3D dataFileView.Search(pUID[604]); ---> index =3D 601 = OK! index =3D dataFileView.Search(pUID[999]); ---> index =3D 640 (last + 1) OK! index =3D dataFileView.Search(pUID[-1]); ---> index =3D 0 WRONG! No, these are correct. Search tells you where the value is or=20 where it belongs (where it should be inserted to preserve=20 order). It does not tell if it exists. Huh?? I would expect something like PositionWhenInserted() to do that, but certainly not Search(). Shouldn't Search() tell me where the match (that is, the index) exists? (either -1 or an out-of-bounds value if it doesn't would be great) In my first post above, how good is Search() if it can't distinguish between 0 being a "this is where it should go" kind of index and 0 being a "it doesn't exist" kind of index? In other words, if Search() returns an index, how do we know if it found (or didn't find) a match?=20 0000,0000,DEB7 count =3D dataFileView.Locate(pUID[604], &index); ---> count =3D 0; index =3D 601 WRONG! count =3D dataFileView.Locate(pUID[999], &index); ---> count =3D 0; index =3D 640 (last + 1) = OK! count =3D dataFileView.Locate(pUID[-1], &index); ---> count =3D 0; index =3D 0 WRONG! I'm not familiar with Locate, but my guess is that the last test=20 is OK (like Search, telling you where to insert). Looks like you=20 did find a bug with the first test. Locate() is defined in c4_View in the following manner: /// Return number of matching keys, and pos of first one as arg int c4_View::Locate(const c4_RowRef& crit_, int* pos_) = const0000,6363,1212 It seems a pretty straightforward call... Thanks for replying Gordon! -- Tito= --Apple-Mail-2-938689125-- From gmcm@hypernet.com Sun Jul 29 12:49:06 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA04407 for ; Sun, 29 Jul 2001 12:49:06 -0500 Received: from me (204.176.40.129) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sun, 29 Jul 2001 13:43:28 -0400 From: "Gordon McMillan" To: Tito Ciuro Date: Sun, 29 Jul 2001 13:47:59 -0400 Subject: Re: [Metakit] Search() and Locate() are broken! Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3B64140F.24405.869181A@localhost> Priority: normal References: <3B63BB4F.5013.70E6B97@localhost> In-reply-to: <1215697061-57448175@hypernet.com> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, I'm not defending the API, just pointing out how it works. It looks like Locate was an attempt to overcome the awkward API of Search, but I haven't really adjusted to newer releases of MetaKit yet. All of my (old) code tests the index for out-of- bounds, and then the value of the property after the Search. You should probably post the full text of your Locate test so Jean-Claude can look at it when he gets home next week. (And I'll upgrade my code to use Locate, since it does what I want!) - Gordon > On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: > > > Tito, > > > >> index = dataFileView.Search(pUID[604]); ---> index = 601 OK! > >> index = dataFileView.Search(pUID[999]); ---> index = 640 (last > >> + 1) OK! index = dataFileView.Search(pUID[-1]); ---> index = 0 > >> WRONG! > > > > No, these are correct. Search tells you where the value is or > > where it belongs (where it should be inserted to preserve > > order). It does not tell if it exists. > > Huh?? I would expect something like PositionWhenInserted() to do > that, but certainly not Search(). Shouldn't Search() tell me > where the match (that is, the index) exists? (either -1 or an > out-of-bounds value if it doesn't would be great) > > In my first post above, how good is Search() if it can't > distinguish between 0 being a "this is where it should go" kind > of index and 0 being a "it doesn't exist" kind of index? > > In other words, if Search() returns an index, how do we know if > it found (or didn't find) a match? > > >> count = dataFileView.Locate(pUID[604], &index); ---> count = > >> 0; index = 601 WRONG! count = dataFileView.Locate(pUID[999], > >> &index); ---> count = 0; index = 640 (last + 1) OK! count = > >> dataFileView.Locate(pUID[-1], &index); ---> count = 0; index = > >> 0 WRONG! > > > > I'm not familiar with Locate, but my guess is that the last > > test is OK (like Search, telling you where to insert). Looks > > like you did find a bug with the first test. > > Locate() is defined in c4_View in the following manner: > > > /// Return number of matching keys, and pos of first one as arg > > int c4_View::Locate(const c4_RowRef& crit_, int* pos_) const > > It seems a pretty straightforward call... > > Thanks for replying Gordon! > > -- Tito > From titoffen@pacbell.net Sun Jul 29 18:20:39 2001 Received: from pltn13.pbi.net (mta7.pltn13.pbi.net [64.164.98.8]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA08709 for ; Sun, 29 Jul 2001 18:20:38 -0500 Message-Id: <200107292320.SAA08709@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta7.pltn13.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GH90009JCUZPY@mta7.pltn13.pbi.net> for metakit@equi4.com; Sun, 29 Jul 2001 16:20:59 -0700 (PDT) Date: Sun, 29 Jul 2001 16:21:34 -0700 From: Tito Ciuro Subject: Re: [Metakit] Search() and Locate() are broken! In-reply-to: <3B64140F.24405.869181A@localhost> To: gmcm@hypernet.com Cc: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-1-961825626 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-961825626 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hello Gordon, On Sunday, July 29, 2001, at 10:47 AM, Gordon McMillan wrote: > Tito, > > I'm not defending the API, just pointing out how it works. I'm sorry I gave you the impression I was upset. I have been working all night long trying to optimize my code, only to find out that the *most* critical part of the code is broken (or so it seems!) If Search() or Locate() worked (remember, I need to find out whether negative values exist), the app would run zippy. Instead, when the file is somewhat large (2 MB/700 rows), the time it takes to perform a Find() is just too long. An binary search would kick some serious butt. Given that with both, Search() and Locate(), there isn't a way to know whether it found a match or not, I'm stuck with the less optimized Find(). I have nothing against the API though!... So, I want to extend my thanks to you and Neville Franks for taking the time to answer my questions.concerns. :-) > It looks like Locate was an attempt to overcome the awkward > API of Search, but I haven't really adjusted to newer releases > of MetaKit yet. All of my (old) code tests the index for out-of- > bounds, and then the value of the property after the Search. If you look for a negative number that doesn't exist in your data file, what index do you get back? > You should probably post the full text of your Locate test so > Jean-Claude can look at it when he gets home next week. > (And I'll upgrade my code to use Locate, since it does what I > want!) > - Gordon What do you mean by the full text of the Locate() test? The test is very easy: try to execute a query as stated above. Thank you very much, -- Tito >> On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: >> >>> Tito, >>> >>>> index = dataFileView.Search(pUID[604]); ---> index = 601 OK! >>>> index = dataFileView.Search(pUID[999]); ---> index = 640 (last >>>> + 1) OK! index = dataFileView.Search(pUID[-1]); ---> index = 0 >>>> WRONG! >>> >>> No, these are correct. Search tells you where the value is or >>> where it belongs (where it should be inserted to preserve >>> order). It does not tell if it exists. >> >> Huh?? I would expect something like PositionWhenInserted() to do >> that, but certainly not Search(). Shouldn't Search() tell me >> where the match (that is, the index) exists? (either -1 or an >> out-of-bounds value if it doesn't would be great) >> >> In my first post above, how good is Search() if it can't >> distinguish between 0 being a "this is where it should go" kind >> of index and 0 being a "it doesn't exist" kind of index? >> >> In other words, if Search() returns an index, how do we know if >> it found (or didn't find) a match? >> >>>> count = dataFileView.Locate(pUID[604], &index); ---> count = >>>> 0; index = 601 WRONG! count = dataFileView.Locate(pUID[999], >>>> &index); ---> count = 0; index = 640 (last + 1) OK! count = >>>> dataFileView.Locate(pUID[-1], &index); ---> count = 0; index = >>>> 0 WRONG! >>> >>> I'm not familiar with Locate, but my guess is that the last >>> test is OK (like Search, telling you where to insert). Looks >>> like you did find a bug with the first test. >> >> Locate() is defined in c4_View in the following manner: >> >>> /// Return number of matching keys, and pos of first one as arg >>> int c4_View::Locate(const c4_RowRef& crit_, int* pos_) const >> >> It seems a pretty straightforward call... >> >> Thanks for replying Gordon! >> >> -- Tito >> > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > ____________________________________________ "The pen is mightier than the sword, and considerably easier to write with." Marty Feldman --Apple-Mail-1-961825626 Content-Transfer-Encoding: quoted-printable Content-Type: text/enriched; charset=us-ascii Hello Gordon, On Sunday, July 29, 2001, at 10:47 AM, Gordon McMillan wrote: Tito, I'm not defending the API, just pointing out how it works. I'm sorry I gave you the impression I was upset. I have been working all night long trying to optimize my code, only to find out that the *most* critical part of the code is broken (or so it seems!) If Search() or Locate() worked (remember, I need to find out whether negative values exist), the app would run zippy. Instead, when the file is somewhat large (2 MB/700 rows), the time it takes to perform a Find() is just too long. An binary search would kick some serious butt. Given that with both, Search() and Locate(), there isn't a way to know whether it found a match or not, I'm stuck with the less optimized Find(). I have nothing against the API though!... So, I want to extend my thanks to you and Neville Franks for taking the time to answer my questions.concerns. :-) It looks like Locate was an attempt to overcome the awkward=20 API of Search, but I haven't really adjusted to newer releases=20 of MetaKit yet. All of my (old) code tests the index for out-of- bounds, and then the value of the property after the Search. If you look for a negative number that doesn't exist in your data file, what index do you get back? 0000,0000,DEB7 You should probably post the full text of your Locate test so=20 Jean-Claude can look at it when he gets home next week.=20 (And I'll upgrade my code to use Locate, since it does what I=20 want!)0000,0000,DEB7 0000,0000,DEDE- Gordon What do you mean by the full text of the Locate() test? The test is very easy: try to execute a query as stated above. Thank you very much, -- Tito 0000,0000,DEB7 On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: Tito, index =3D dataFileView.Search(pUID[604]); ---> index =3D 601 = OK! index =3D dataFileView.Search(pUID[999]); ---> index =3D 640 (last + 1) OK! index =3D dataFileView.Search(pUID[-1]); ---> index =3D 0 WRONG! No, these are correct. Search tells you where the value is or where it belongs (where it should be inserted to preserve order). It does not tell if it exists. Huh?? I would expect something like PositionWhenInserted() to do that, but certainly not Search(). Shouldn't Search() tell me where the match (that is, the index) exists? (either -1 or an out-of-bounds value if it doesn't would be great) In my first post above, how good is Search() if it can't distinguish between 0 being a "this is where it should go" kind of index and 0 being a "it doesn't exist" kind of index? In other words, if Search() returns an index, how do we know if it found (or didn't find) a match? count =3D dataFileView.Locate(pUID[604], &index); ---> count =3D 0; index =3D 601 WRONG! count =3D dataFileView.Locate(pUID[999], &index); ---> count =3D 0; index =3D 640 (last + 1) OK! count =3D dataFileView.Locate(pUID[-1], &index); ---> count =3D 0; index =3D 0 WRONG! I'm not familiar with Locate, but my guess is that the last test is OK (like Search, telling you where to insert). Looks like you did find a bug with the first test. Locate() is defined in c4_View in the following manner: /// Return number of matching keys, and pos of first one as arg int c4_View::Locate(const c4_RowRef& crit_, int* pos_) const It seems a pretty straightforward call... Thanks for replying Gordon! -- Tito _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit = Verdana________________________________= ____________ 0000,0000,FFFF"The pen is mightier than the sword, and considerably easier to write with." 0000,6666,3333 Marty Feldman= --Apple-Mail-1-961825626-- From titoffen@pacbell.net Sun Jul 29 18:30:23 2001 Received: from snfc21.pbi.net (mta6.snfc21.pbi.net [206.13.28.240]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA08805 for ; Sun, 29 Jul 2001 18:30:23 -0500 Message-Id: <200107292330.SAA08805@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta6.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GH900G4ZDB706@mta6.snfc21.pbi.net> for metakit@equi4.com; Sun, 29 Jul 2001 16:30:43 -0700 (PDT) Date: Sun, 29 Jul 2001 16:31:35 -0700 From: Tito Ciuro Subject: Re: [Metakit] Search() and Locate() are broken! In-reply-to: <200107300916450047.1A29DC88@getsoft.com> To: readonly@getsoft.com Cc: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-2-962427212 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2-962427212 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hello Neville! On Sunday, July 29, 2001, at 04:16 PM, Neville Franks wrote: > If the column you want to search on is not in a sorted order then you > can create a sorted (derived) view to Search on. Because the sort > didn't always sort correctly, search would fail or return the wrong > thing. Humm... interesting. Let's see: the first column in my view contains the UIDs. These are all sorted physically in the file (executing Dump verifies that). Could it be that when I obtain a View via GetAs() the sort is not honored? Should I obtain a sorted view instead? I see some sort-related functions... how could I specify a sorted view based on a specific column? Another question: I wrote a getRowForUID() function, which basically returns a c4_RowRef. I use this function a lot, and this is where Search() would save *a lot* of time. So wouldn't be too costly to get a sorted view each time I needed to search for a particular UID? > I gave up on trying to use MK because of problems like this and others, > which was a pity as it is nice in many ways. That's too bad. I have looked a several other solutions and I must say that I *really like* MK. It sure rocks! :-) > Hope that helps. Thanks so much! -- Tito --Apple-Mail-2-962427212 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=us-ascii Hello Neville! On Sunday, July 29, 2001, at 04:16 PM, Neville Franks wrote: 0000,0000,DEDEIf the column you want to search on is not in a sorted order then you can create a sorted (derived) view to Search on. Because the sort didn't always sort correctly, search would fail or return the wrong thing. Humm... interesting. Let's see: the first column in my view contains the UIDs. These are all sorted physically in the file (executing Dump verifies that). Could it be that when I obtain a View via GetAs() the sort is not honored? Should I obtain a sorted view instead? I see some sort-related functions... how could I specify a sorted view based on a specific column? Another question: I wrote a getRowForUID() function, which basically returns a c4_RowRef. I use this function a lot, and this is where Search() would save *a lot* of time. So wouldn't be too costly to get a sorted view each time I needed to search for a particular UID? 0000,0000,DEDE I gave up on trying to use MK because of problems like this and others, which was a pity as it is nice in many ways. That's too bad. I have looked a several other solutions and I must say that I *really like* MK. It sure rocks! :-) 0000,0000,DEDE Hope that helps. Thanks so much! -- Tito --Apple-Mail-2-962427212-- From readonly@getsoft.com Sun Jul 29 19:43:27 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA09497 for ; Sun, 29 Jul 2001 19:43:27 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id SAA12410; Sun, 29 Jul 2001 18:43:23 -0600 (MDT) Message-ID: <200107301042480221.1A78A52E@getsoft.com> In-Reply-To: <200107292330.SAA08805@trixie.triqs.com> References: <200107292330.SAA08805@trixie.triqs.com> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Mon, 30 Jul 2001 10:42:48 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "Tito Ciuro" Cc: metakit@equi4.com Subject: Re: [Metakit] Search() and Locate() are broken! Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id TAA09497 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Tito, On 29 Jul 2001 at 16:31 Tito Ciuro wrote: >Hello Neville! > >On Sunday, July 29, 2001, at 04:16 PM, Neville Franks wrote: > >> If the column you want to search on is not in a sorted order then you >> can create a sorted (derived) view to Search on. Because the sort >> didn't always sort correctly, search would fail or return the wrong >> thing. > >Humm... interesting. Let's see: the first column in my view contains the >UIDs. These are all sorted physically in the file (executing Dump >verifies that). Rows are either added to the end of a view or inserted, depending on what you are doing. So the result of these operations is the natural order. >Could it be that when I obtain a View via GetAs() the sort is not >honored? Should I obtain a sorted view instead? I see some sort-related >functions... how could I specify a sorted view based on a specific >column? GetAs() doesn't do any sorting. >Another question: I wrote a getRowForUID() function, which basically >returns a c4_RowRef. I use this function a lot, and this is where >Search() would save *a lot* of time. So wouldn't be too costly to get a >sorted view each time I needed to search for a particular UID? A sorted view is a map of the underlying real view. When items are added etc. to the real view they appear in the sorted view so you don't need to create new sorted views. >> I gave up on trying to use MK because of problems like this and others, >> which was a pity as it is nice in many ways. > >That's too bad. I have looked a several other solutions and I must say >that I *really like* MK. It sure rocks! :-) You mentioned in your other post that you have 700 rows. I would have thought Find would be fast enough as this realy isn't many rows. I ran into problems when I got over 10,000 rows. If your DB is going to scale up to this sort of size I suggest you start working with it at these larger sizes now. Best Regards, Neville Franks (Author of ED for Windows) PS. Did you know ED4W Version V4 has been released? This is a major new release with lots of great new features and enhancements. See our What's New page http://www.getsoft.com/whats_new.html for details. And while you're there you can click on 'Stay Informed' to join our mailing list. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From gmcm@hypernet.com Sun Jul 29 20:23:49 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA09829 for ; Sun, 29 Jul 2001 20:23:45 -0500 Received: from me (204.176.40.53) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sun, 29 Jul 2001 21:17:59 -0400 From: "Gordon McMillan" To: Tito Ciuro Date: Sun, 29 Jul 2001 21:22:27 -0400 Subject: Re: [Metakit] Search() and Locate() are broken! Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3B647E93.21055.A092A47@localhost> Priority: normal In-reply-to: <200107292320.SAA08709@trixie.triqs.com> References: <3B64140F.24405.869181A@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, > ... I have been > working all night long trying to optimize my code, only to find > out that the *most* critical part of the code is broken (or so it > seems!) If Search() or Locate() worked (remember, I need to find > out whether negative values exist), the app would run zippy. > Instead, when the file is somewhat large (2 MB/700 rows), the > time it takes to perform a Find() is just too long. An binary > search would kick some serious butt. > > Given that with both, Search() and Locate(), there isn't a way to > know whether it found a match or not, I'm stuck with the less > optimized Find(). I have nothing against the API though!... Locate() *should* fill your needs. Search() does, it's just awkward (you need to check the row at the returned index to see if the UID matches what you Search()ed on). > If you look for a negative number that doesn't exist in your data > file, what index do you get back? I believe in your example, all UIDs were positive, so any negative UID would return 0. > What do you mean by the full text of the Locate() test? The test > is very easy: try to execute a query as stated above. Well, what I was actually getting at was "is the view sorted correctly", since Search() & Locate() both only work on views sorted on the property you're searching on. From your discussion with Neville: > Humm... interesting. Let's see: the first column in my view > contains the UIDs. These are all sorted physically in the > file (executing Dump verifies that). > Could it be that when I obtain a View via GetAs() the sort is > not honored? No. While that *could* affect the structure, I'm pretty sure it won't affect the order. > Should I obtain a sorted view instead? I see > some sort-related functions... how could I specify a sorted > view based on a specific column? To be safe, I'd sort it when first opened. For inserts, I'd use InsertAt(), using the index returned by Search() or Locate(). That should be enough (if you're sure that's the only way rows get added, you only need to sort once). If UID is your first property, then view->Sort() will do it. > Another question: I wrote a getRowForUID() function, which > basically returns a c4_RowRef. I use this function a lot, and > this is where Search() would save *a lot* of time. So wouldn't > be too costly to get sorted view each time I needed to > search for a particular UID? It would be costly (although it's only indexes getting shuffled, not data), but it shouldn't be necessary. I've never observed Neville's problems with sorted views, but I've never used MetaKit beyond a few 10Ks of rows. I have been surprised at MetaKit being case-insensitive, but that's never actually hurt me. - Gordon From readonly@getsoft.com Sun Jul 29 21:09:48 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA10286 for ; Sun, 29 Jul 2001 21:09:47 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id UAA21517; Sun, 29 Jul 2001 20:09:48 -0600 (MDT) Message-ID: <200107301209130247.1AC7C330@getsoft.com> In-Reply-To: <3B647E93.21055.A092A47@localhost> References: <3B64140F.24405.869181A@localhost> <3B647E93.21055.A092A47@localhost> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Mon, 30 Jul 2001 12:09:13 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: gmcm@hypernet.com, "Tito Ciuro" Cc: metakit@equi4.com Subject: Re: [Metakit] Search() and Locate() are broken! Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id VAA10286 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Gordon, >It would be costly (although it's only indexes getting shuffled, >not data), but it shouldn't be necessary. I've never observed >Neville's problems with sorted views, but I've never used >MetaKit beyond a few 10Ks of rows. If you scan the entire View and compare the row.sort clm value with the previous row using the sort comparision function you may find the sort order is wrong. I rracked this down to a problem with the MK Merge Sort code but wasn't able to fix it. >I have been surprised at MetaKit being case-insensitive, but >that's never actually hurt me. I sent an update to JC which let you use your own sort comparison function, but I don't know if it has ever been incorporated into MK. Best Regards, Neville Franks (Author of ED for Windows) PS. Did you know ED4W Version V4 has been released? This is a major new release with lots of great new features and enhancements. See our What's New page http://www.getsoft.com/whats_new.html for details. And while you're there you can click on 'Stay Informed' to join our mailing list. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From readonly@getsoft.com Mon Jul 30 01:46:34 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA13587 for ; Mon, 30 Jul 2001 01:46:34 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id AAA23347; Mon, 30 Jul 2001 00:46:43 -0600 (MDT) Message-ID: <200107301646090510.1BC54E77@getsoft.com> In-Reply-To: <200107300622.AAA19906@getsoft.com> References: <200107300622.AAA19906@getsoft.com> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Mon, 30 Jul 2001 16:46:09 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "Tito Ciuro" Cc: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id BAA13587 Subject: [Metakit] Re: Still not working... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 29 Jul 2001 at 23:22 Tito Ciuro wrote: >On Sunday, July 29, 2001, at 10:57 PM, Neville Franks wrote: > >> Hi Tito, >> I'm relying just on my memory here as I can't find my MK code but I >> thought you created you view as normal, and then created a sorted view >> mapped on top of the real view. For insert/add you work with the >> underlying real view and just use the sorted view for lookup purposes. >> >> Hopefully there is an example somewhere. > >Hello guys, > >I guess I didn't explain myself properly before. One of my questions was >whether the sort view would be always be kept in sync with the real view. > >Obviously, If I start the app my mapping the sort view on top of the >real view, both are considered to hold the same information; they're in >sync. > >However, as I add/edit/delete information from the real view, would the >sort view "notice" this changes? Or do I have to call Sort() prior to >performing a query? (my guess is that this methodology would be very >expensive) > >If on the other hand, both views are kept in sync automatically, then >great! Then there's little more to add to the code to make it work. My understanding is that the sorted view is kept in sync with the real view. The sort view is in fact just pointers into the real view. The only cost is the time to do the sort in the first place. Also because the sorted view doesn't persist you have to do this each time you open the database. For a large database this can be an expensive operation. This is an area where persistant indexing, such as a btree would be very usefull. >I'll have to continue tomorrow. It's 11:20 PM and I'm exhausted! :-) > >Have a great Monday and again, a million more thanks, > >-- Tito Best Regards, Neville Franks (Author of ED for Windows) PS. Did you know ED4W Version V4 has been released? This is a major new release with lots of great new features and enhancements. See our What's New page http://www.getsoft.com/whats_new.html for details. And while you're there you can click on 'Stay Informed' to join our mailing list. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From guenther.fischer@hrz.tu-chemnitz.de Tue Jul 31 05:04:01 2001 Received: from obelix.hrz.tu-chemnitz.de (obelix.hrz.tu-chemnitz.de [134.109.132.55]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA31431 for ; Tue, 31 Jul 2001 05:04:00 -0500 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by obelix.hrz.tu-chemnitz.de with esmtp (Exim 3.22 #4) id 15RWNq-0001ti-00 for metakit@equi4.com; Tue, 31 Jul 2001 12:04:14 +0200 Date: Tue, 31 Jul 2001 12:04:14 +0200 (MEST) From: Guenther Fischer To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The Mk4py-dok is a little bit small ... I have a database application with Mk4. I can now read it in python and show all the rows, but how to update? def SaveEntry(f, id): ix = vw.find(ID=id) # vw is the view as global if f.NAME.IsModified(): # was the presented entry modified? vw.insert(ix,NAME=f.NAME.GetValue()) # NAME is a property name but nothing will happens - no error too -- ~Guenther Fischer From gmcm@hypernet.com Tue Jul 31 09:05:54 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA08374 for ; Tue, 31 Jul 2001 09:05:53 -0500 Received: from me (204.176.40.60) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Tue, 31 Jul 2001 10:00:07 -0400 From: "Gordon McMillan" To: Guenther Fischer , Date: Tue, 31 Jul 2001 10:04:34 -0400 Subject: Re: [Metakit] Mk4py Reply-to: gmcm@hypernet.com Message-ID: <3B6682B2.17552.11E94412@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: > > I have a database application with Mk4. I can now read it in > python and show all the rows, but how to update? > > def SaveEntry(f, id): > ix = vw.find(ID=id) # vw is the view as global if > f.NAME.IsModified(): # was the presented entry > modified? > vw.insert(ix,NAME=f.NAME.GetValue()) # NAME is a property > name > > but nothing will happens - no error too First make sure the storage is open for update. You'll also need a commit at some point to make changes permanent. What your code is doing is adding a second row with only one property filled in. You probably want something more like: if f.NAME.IsModified(): ix = vw.find(ID=id) if ix > -1: vw[ix].NAME=f.NAME.GetValue() db.commit() # can be done later else: # error - ID not found - Gordon From acarter@covad.net Tue Jul 31 19:43:03 2001 Received: from smtp6.covad.net (psmtp6.array3.laserlink.net [63.65.123.56]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA27683 for ; Tue, 31 Jul 2001 19:43:02 -0500 Received: from acarter1 (tide79.microsoft.com [131.107.3.79]) by smtp6.covad.net (8.9.3/8.9.3) with ESMTP id UAA11581 for ; Tue, 31 Jul 2001 20:43:14 -0400 (EDT) From: "Andrew Carter" To: Date: Tue, 31 Jul 2001 17:43:13 -0700 Message-ID: <87C80E823E824644BCB711D6348262B50181A5FB@red-msg-05.redmond.corp.microsoft.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0008_01C119E8.46C95030" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2526.0000 Importance: Normal Subject: [Metakit] Primary key generation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0008_01C119E8.46C95030 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit I'm just getting started with MetaKit. I like what I've seen so far. I will probably want to maintain mostly relational-style data in my data files. As such, I would like to generate primary key values for rows of data. Is there an easy way to do this? One way would be to use a MAX() type function - I couldn't find this built-into the tool set. Also, is there any way to enforce constraints like unique properties? If not, this would be a useful future feature. Thanks, Andrew ------=_NextPart_000_0008_01C119E8.46C95030 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I’m just getting started with MetaKit.  I = like what I’ve seen so far.  I will probably want to maintain mostly = relational-style data in my data files.  As such, I would like to generate primary key = values for rows of data.  Is there an easy way to do this?  One way would be = to use a MAX() type function – I couldn’t find this built-into the tool = set.  Also, is there any way to enforce constraints like unique properties?  If = not, this would be a useful future feature.


Thanks,

Andrew

------=_NextPart_000_0008_01C119E8.46C95030-- From titoffen@pacbell.net Fri Aug 3 03:02:25 2001 Received: from snfc21.pbi.net (mta6.snfc21.pbi.net [206.13.28.240]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA17740 for ; Fri, 3 Aug 2001 03:02:25 -0500 Message-Id: <200108030802.DAA17740@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta6.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GHH00KUAFNXYC@mta6.snfc21.pbi.net> for metakit@equi4.com; Fri, 03 Aug 2001 01:02:22 -0700 (PDT) Date: Fri, 03 Aug 2001 01:03:20 -0700 From: Tito Ciuro To: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-2--808751601 Subject: [Metakit] MK Transaction Modes Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2--808751601 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hello! After testing and watching MK closely, I realized what is slowing down my queries. A little background: I wrote an API to access the data from multiple processes (Unix). The file is open in "commit-extend" mode, which makes the file grow enormously when saving changes. Just to give you a quick example, saving a single string (17 chars) increases the file size by about 16KB. Yikes! The documentation is very clear stating that saving in commit-extend mode makes the file grow very quickly. Browsing MK home page, I just came across the following link: http://www.equi4.com/metakit/wiki.cgi/82.html In particular, I believe that a model that will work better for me is the following one (I have selected the important bits in bold): > Combined modes > > The features described above open a range of options for dealing with > multi-user / multi-tasking scenario's, even though they do not yet > handle all possible cases. > > First of all, commit-extend offers extreme performance in the case when > lots of readers require access to a consistent state, but only few > changes are made. Readers will see the state at time of opening the > datafile (they can close and re-open any time to see new committed > changes). This mode is efficient because readers never wait, not even > while the extender is committing changes. > > The drawback of commti extend is the potentially large disk space > consumption, and hence the need to reclaim and clean up periodically. > This requires exclusive access. > > Commit-aside can be very effective for single-task situations with lots > of commits, since commit performance is relatively high, and disk space > consuption is limited. But again, at some point, changes will need to > be consolidated, and again enclusive access mode is needed to fold all > changes back into the main datafile and to clear the aside file. > > Finally, and interesting option is to use commit aside with a read-only > main datafile, and commit-extend for the secondary aside file. Since > only changes are saved, the aside file will not grow that quickly, and > since both files support concurrency, multiple readers can get at the > latest consistent state without delay. The only contention is that if > there are multiple writers, these will have to find a way to take turns > in making changes. > > Yet more scenarios will become available later, once the main datafile > can also be used as the aside file. This is currently not allowed. Has anyone tried this "commit aside" option. I've tried, but without success: 1) Open a c4_Storage in ~/Library/Test/mainFile. I set the mode to read only (I tried both, "read-only, original contents" and "read-only, most recently committed contents") 2) Open the second c4_Storage in ~/Library/Test/mainFile_aside. I set the mode to commit-extend. 3) Called mainFile->storage->SetAside(mainFile_aside). 4) Perform all operations via mainFile. Questions: 1) assuming the code works with one c4_Storage/commit-extend mode, why is this commit-extend-aside mode not working? 2) should I deal with the aside file directly, or is MK handling everything behind the scenes? 3) With multiple readers, I assume I have to open *both files* in read only, right? Then on save, reopen in exclusive write and perform the save. The doubt I have is whether I have to reopen only the aside file or also the main file? It's a little confusing because the doc doesn't mention much. Also, the FAQ doesn't contain an entry which deals with this issue. Last, but not least, I was wondering if there's a simple way to compact the data (in this commit-extend-aside model). Thanks a lot in advance, -- Tito --Apple-Mail-2--808751601 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=us-ascii Hello! After testing and watching MK closely, I realized what is slowing down my queries. A little background: I wrote an API to access the data from multiple processes (Unix). The file is open in "commit-extend" mode, which makes the file grow enormously when saving changes. Just to give you a quick example, saving a single string (17 chars) increases the file size by about 16KB. Yikes! The documentation is very clear stating that saving in commit-extend mode makes the file grow very quickly. Browsing MK home page, I just came across the following link: http://www.equi4.com/metakit/wiki.cgi/82.html In particular, I believe that a model that will work better for me is the following one (I have selected the important bits in bold): Combined modes The features described above open a range of options for dealing with multi-user / multi-tasking scenario's, even though they do not yet handle all possible cases. First of all, commit-extend offers extreme performance in the case when lots of readers require access to a consistent state, but only few changes are made. Readers will see the state at time of opening the datafile (they can close and re-open any time to see new committed changes). This mode is efficient because readers never wait, not even while the extender is committing changes. The drawback of commti extend is the potentially large disk space consumption, and hence the need to reclaim and clean up periodically. This requires exclusive access. Commit-aside can be very effective for single-task situations with lots of commits, since commit performance is relatively high, and disk space consuption is limited. But again, at some point, changes will need to be consolidated, and again enclusive access mode is needed to fold all changes back into the main datafile and to clear the aside file. Finally, and interesting option is to use commit aside with a read-only main datafile, and commit-extend for the secondary aside file. Since only changes are saved, the aside file will not grow that quickly, and since both files support concurrency, multiple readers can get at the latest consistent state without delay. The only contention is that if there are multiple writers, these will have to find a way to take turns in making changes. Yet more scenarios will become available later, once the main datafile can also be used as the aside file. This is currently not allowed. Has anyone tried this "commit aside" option. I've tried, but without success: 1) Open a c4_Storage in ~/Library/Test/mainFile. I set the mode to read only (I tried both, "read-only, original contents" and "read-only, most recently committed contents") 2) Open the second c4_Storage in ~/Library/Test/mainFile_aside. I set the mode to commit-extend. 3) Called mainFile->storage->SetAside(mainFile_aside). 4) Perform all operations via mainFile. Questions: 1) assuming the code works with one c4_Storage/commit-extend mode, why is this commit-extend-aside mode not working? 2) should I deal with the aside file directly, or is MK handling everything behind the scenes? 3) With multiple readers, I assume I have to open *both files* in read only, right? Then on save, reopen in exclusive write and perform the save. The doubt I have is whether I have to reopen only the aside file or also the main file? It's a little confusing because the doc doesn't mention much. Also, the FAQ doesn't contain an entry which deals with this issue. Last, but not least, I was wondering if there's a simple way to compact the data (in this commit-extend-aside model). Thanks a lot in advance, -- Tito --Apple-Mail-2--808751601-- From tciuro@yahoo.com Fri Aug 10 18:10:36 2001 Received: from web4002.mail.yahoo.com (web4002.mail.yahoo.com [216.115.104.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA01481 for ; Fri, 10 Aug 2001 18:10:36 -0500 Message-ID: <20010810231009.20315.qmail@web4002.mail.yahoo.com> Received: from [17.202.22.175] by web4002.mail.yahoo.com via HTTP; Fri, 10 Aug 2001 16:10:09 PDT Date: Fri, 10 Aug 2001 16:10:09 -0700 (PDT) From: Tito Ciuro To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] Compacting a MetaKit file? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I was trying to compact the MK file by doing this: #include "mk4.h" #include "mk4str.h" int main() { // Create the property I'll use c4_StringProp pName ("Name"); // Open the storage file in exclusive read/write c4_Storage storage ("test.dat", 1); // Obtain the view c4_View vAddress = storage.GetAs("address[Name:S]"); // Edit a property pName(vAddress[0]) = "John Doe"; // Save the changes storage.Commit(); return 0; } The page in the following link: http://www.equi4.com/metakit/wiki.cgi/82.html states: -----> begin quote [...] The way to reclaim free space, is to re-open the datafile in exclusive mode (1), and then commit normally. There are a number of ways to actully bring the file back to its minimal size (this won't happen right away, since space will only become re-usable after the commit completes succesfully). <----- end quote So it seems that I'm doing the right thing, no? The last sentence puzzles me... When will the file size be reduced? After calling commit(), I would expect the file size to be reduced significantly, but this isn't the case. By the way... the edit goes through and gets saved properly. Regarding the file size, I've seen that 1000 records added and commited once (at the end), results in 96KB. If I commit after each save (commit-extend mode), the file size if 45.5MB. With this file, searching and editing takes forever, so I would like to compact the file, hopefully getting close to the 96KB mark. Has anyone accomplished this? Thanks, -- Tito __________________________________________________ Do You Yahoo!? Send instant messages & get email alerts with Yahoo! Messenger. http://im.yahoo.com/ From tciuro@yahoo.com Fri Aug 10 22:38:07 2001 Received: from web4001.mail.yahoo.com (web4001.mail.yahoo.com [216.115.104.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id WAA06079 for ; Fri, 10 Aug 2001 22:38:06 -0500 Message-ID: <20010811033738.3188.qmail@web4001.mail.yahoo.com> Received: from [17.202.22.175] by web4001.mail.yahoo.com via HTTP; Fri, 10 Aug 2001 20:37:38 PDT Date: Fri, 10 Aug 2001 20:37:38 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Compacting a MetaKit file? To: Tito Ciuro , metakit@equi4.com In-Reply-To: <20010810231009.20315.qmail@web4002.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi guys! OK... I solved the puzzle, so here's the code that does it: FILE *myOldFile = fopen (kBigDataFileName, "r"); c4_FileStream myOldStream (myOldFile, true); FILE *myNewFile = fopen (kSmallDataFileName, "w"); c4_FileStream myNewStream (myNewFile, true); storage.LoadFrom( myOldStream ); storage.SaveTo( myNewStream ); fclose (myOldFile); fclose (myNewFile); If you think there's a better way, by all means, let me know! :) I created 2 data files by opening in Commit-Extend mode, then call commit() after each save. This has created huge files (as expected) which is great to test compacting times. Care about the results? Here we go then (sorry if it looks out of whack): Results: ---------- Number of records 1,000 5,000 original size 45.5MB 1.1GB compacted size 96KB 464KB time to compact (local) 110 ms 530 ms time to compact (NFS) 580 ms 2970 ms As you can see, most users will use data files locally, so compacting can almost be done on each save! :) One thing I'd love to see if some way to calculate what the wasted space ratio is for a given data file. (Jean-Claude?) That would give us a pointer as to how often we should trigger a custom Save & Compact method. I hope this helps! -- Tito --- Tito Ciuro wrote: > Hello, > > I was trying to compact the MK file by doing this: > > #include "mk4.h" > #include "mk4str.h" > > int main() > { > // Create the property I'll use > c4_StringProp pName ("Name"); > > // Open the storage file in exclusive read/write > c4_Storage storage ("test.dat", 1); > > // Obtain the view > c4_View vAddress = > storage.GetAs("address[Name:S]"); > > // Edit a property > pName(vAddress[0]) = "John Doe"; > > // Save the changes > storage.Commit(); > > return 0; > } > > The page in the following link: > > http://www.equi4.com/metakit/wiki.cgi/82.html > > states: > > -----> begin quote > [...] The way to reclaim free space, is to re-open > the > datafile in exclusive mode (1), and then commit > normally. There are a number of ways to actully > bring > the file back to its minimal size (this won't happen > right away, since space will only become re-usable > after the commit completes succesfully). > <----- end quote > > So it seems that I'm doing the right thing, no? The > last sentence puzzles me... When will the file size > be > reduced? After calling commit(), I would expect the > file size to be reduced significantly, but this > isn't > the case. By the way... the edit goes through and > gets > saved properly. > > Regarding the file size, I've seen that 1000 records > added and commited once (at the end), results in > 96KB. > If I commit after each save (commit-extend mode), > the > file size if 45.5MB. With this file, searching and > editing takes forever, so I would like to compact > the > file, hopefully getting close to the 96KB mark. > > Has anyone accomplished this? > > Thanks, > > -- Tito > > __________________________________________________ > Do You Yahoo!? > Send instant messages & get email alerts with Yahoo! > Messenger. > http://im.yahoo.com/ > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Send instant messages & get email alerts with Yahoo! Messenger. http://im.yahoo.com/ From jyl@best.com Sat Aug 11 16:23:07 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA13374; Sat, 11 Aug 2001 16:23:07 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id OAA26406; Sat, 11 Aug 2001 14:21:49 -0700 (PDT) Message-ID: <3B75A430.E6D9E858@best.com> Date: Sat, 11 Aug 2001 14:31:28 -0700 From: Jacob Levy X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com, jcw@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] How to "rewrite" a view? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm trying to change the format of a view from #define OLDFORMAT "e4GraphMarkers[nameID:I,next:I,root:I,used:I,userData:I]" to: #define NEWFORMAT "e4GraphMarkers[nameID:I,next:I,root:I,flags:I,userData:I]" I believe I've nearly got it, with the following code: c4_IntProp pUsed("used"); c4_IntProp pFlags("flags"); c4_Storage s = .... c4_View oldview = s.GetAs(OLDFORMAT); c4_View newview = oldview.Rename(pUsed, pFlags); s.Store("e4GraphMarkers", newview); Ooops. It seems that MK 2.4.0 doesn't support Store anymore -- what should I use instead? --JYL From titoffen@pacbell.net Sat Aug 11 17:13:59 2001 Received: from pltn13.pbi.net (mta7.pltn13.pbi.net [64.164.98.8]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA13679 for ; Sat, 11 Aug 2001 17:13:58 -0500 Received: from monkey2000 ([209.233.25.139]) by mta7.pltn13.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with SMTP id <0GHX0001JCEFDV@mta7.pltn13.pbi.net> for metakit@equi4.com; Sat, 11 Aug 2001 15:13:28 -0700 (PDT) Date: Sat, 11 Aug 2001 15:14:29 -0700 From: Tito Ciuro Subject: Re: [Metakit] How to "rewrite" a view? To: Jacob Levy , metakit@equi4.com Message-id: <000c01c122b2$fd6f2fd0$6601a8c0@monkey2000> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Mailer: Microsoft Outlook Express 5.50.4522.1200 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7bit X-Priority: 3 X-MSMail-priority: Normal References: <3B75A430.E6D9E858@best.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jacob, Given that MK can change views on the fly, why not simply calling s.Commit()? My understanding is that if you do the following: > c4_Storage s = .... > > c4_View oldview = s.GetAs(NEWFORMAT); > s.Commit(); Then the column "used" gets dropped and "flags" gets added. Of course, the values held by "used" are lost. However, it seems to me that if you perform the steps you've specified and call s.Commit() instead of Store(), the column "used" will be renamed to "flags" while keeping its values. Let me know what happens, OK? Cheers, -- Tito > #define OLDFORMAT > "e4GraphMarkers[nameID:I,next:I,root:I,used:I,userData:I]" > > to: > > #define NEWFORMAT > "e4GraphMarkers[nameID:I,next:I,root:I,flags:I,userData:I]" > > I believe I've nearly got it, with the following code: > > c4_IntProp pUsed("used"); > c4_IntProp pFlags("flags"); > > c4_Storage s = .... > > c4_View oldview = s.GetAs(OLDFORMAT); > c4_View newview = oldview.Rename(pUsed, pFlags); > > s.Store("e4GraphMarkers", newview); > > Ooops. It seems that MK 2.4.0 doesn't support Store anymore -- what > should I use instead? > > --JYL > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jyl@best.com Sat Aug 11 17:45:15 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA13889; Sat, 11 Aug 2001 17:45:14 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id PAA17577; Sat, 11 Aug 2001 15:42:54 -0700 (PDT) Message-ID: <3B75B73E.97BE388D@best.com> Date: Sat, 11 Aug 2001 15:52:46 -0700 From: Jacob Levy X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Tito Ciuro CC: metakit@equi4.com, jcw@equi4.com Subject: Re: [Metakit] How to "rewrite" a view? References: <3B75A430.E6D9E858@best.com> <000c01c122b2$fd6f2fd0$6601a8c0@monkey2000> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: OK I didnt give as much information as I should have. I want to keep the old values of "used" and just rename the column to "flags". In the meantime I've redone the whole thing as follows: c4_View oldview = s.GetAs(OLDFORMAT); oldview.AddProperty(pFlags); for (i = 0, l = oldview.GetSize(); i < l; i++) { if (pUsed(oldview[i]) == 1) { pFlags(oldview[i]) = E4_INUSE; } } /* * Now get the old view in the new format (which should drop the "used" property): */ oldview = s.GetAs(NEWFORMAT); /* * We're done, save everything. */ s.Commit(); } This works except I'm not sure that the "used" column really is gone. Oh well.. Good enough. --JYL Tito Ciuro wrote: > Hi Jacob, > > Given that MK can change views on the fly, why not simply calling > s.Commit()? My understanding is that if you do the following: > > > c4_Storage s = .... > > > > c4_View oldview = s.GetAs(NEWFORMAT); > > s.Commit(); > > Then the column "used" gets dropped and "flags" gets added. Of course, the > values held by "used" are lost. > > However, it seems to me that if you perform the steps you've specified and > call s.Commit() instead of Store(), the column "used" will be renamed to > "flags" while keeping its values. > > Let me know what happens, OK? > > Cheers, > > -- Tito > > > #define OLDFORMAT > > "e4GraphMarkers[nameID:I,next:I,root:I,used:I,userData:I]" > > > > to: > > > > #define NEWFORMAT > > "e4GraphMarkers[nameID:I,next:I,root:I,flags:I,userData:I]" > > > > I believe I've nearly got it, with the following code: > > > > c4_IntProp pUsed("used"); > > c4_IntProp pFlags("flags"); > > > > c4_Storage s = .... > > > > c4_View oldview = s.GetAs(OLDFORMAT); > > c4_View newview = oldview.Rename(pUsed, pFlags); > > > > s.Store("e4GraphMarkers", newview); > > > > Ooops. It seems that MK 2.4.0 doesn't support Store anymore -- what > > should I use instead? > > > > --JYL > > > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > From rcohen@dial.oleane.com Mon Aug 13 02:25:21 2001 Received: from sun2.articque.com (sun2.articque.com [195.25.225.73]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA29134 for ; Mon, 13 Aug 2001 02:25:20 -0500 Received: from dial.oleane.com ([195.25.225.86]) by sun2.articque.com (8.10.1/8.10.0) with ESMTP id f7D8Jr323973 for ; Mon, 13 Aug 2001 09:19:54 +0100 (GMT+0100) Message-ID: <3B778E73.3F5A9F91@dial.oleane.com> Date: Mon, 13 Aug 2001 09:23:15 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Subject: Re: [Metakit] Compacting a MetaKit file? References: <20010811033738.3188.qmail@web4001.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > I created 2 data files by opening in Commit-Extend > mode, then call commit() after each save. This has > created huge files (as expected) which is great to > test compacting times. Care about the results? Here we > go then (sorry if it looks out of whack): I was quite interested in this information because I could not understand in which case a 96kb data could be stored in a 45Mb file ? Could you tell me what to do and why it does it ? Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From tciuro@yahoo.com Mon Aug 13 03:02:50 2001 Received: from web4002.mail.yahoo.com (web4002.mail.yahoo.com [216.115.104.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id DAA29434 for ; Mon, 13 Aug 2001 03:02:49 -0500 Message-ID: <20010813080210.7521.qmail@web4002.mail.yahoo.com> Received: from [209.233.25.139] by web4002.mail.yahoo.com via HTTP; Mon, 13 Aug 2001 01:02:10 PDT Date: Mon, 13 Aug 2001 01:02:10 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Compacting a MetaKit file? To: Riccardo Cohen , Mailing list Metakit In-Reply-To: <3B778E73.3F5A9F91@dial.oleane.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Riccardo, During Commit-Extend mode, each save forces MK to rewrite the *entire* column. This is why data grows so quickly and slows down queries. Therefore, once in a while, it's necessary to compact the file. I sent an email with the code that will allow you to compact the file. Just i case, here it is: FILE *myOldFile = fopen ("myOldFile.dat", "r"); c4_FileStream myOldStream (myOldFile, true); FILE *myNewFile = fopen ("myNewFile.dat", "w"); c4_FileStream myNewStream (myNewFile, true); storage.LoadFrom( myOldStream ); storage.SaveTo( myNewStream ); fclose (myOldFile); fclose (myNewFile); I hope this helps, -- Tito --- Riccardo Cohen wrote: > > I created 2 data files by opening in Commit-Extend > > mode, then call commit() after each save. This has > > created huge files (as expected) which is great to > > test compacting times. Care about the results? > Here we > > go then (sorry if it looks out of whack): > > I was quite interested in this information because I > could not understand in which case a 96kb data could > be stored in a 45Mb file ? Could you tell me what to > do and why it does it ? > > Thanks > > -- > Riccardo Cohen > > Articque > Les Roches > 37230 Fondettes > France > email = rcohen@dial.oleane.com > web = http://www.articque.com > tel: +33 02 47 49 90 49 > fax: +33 02 47 49 91 49 > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Send instant messages & get email alerts with Yahoo! Messenger. http://im.yahoo.com/ From rcohen@dial.oleane.com Mon Aug 13 06:54:44 2001 Received: from sun2.articque.com (sun2.articque.com [195.25.225.73]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA32005 for ; Mon, 13 Aug 2001 06:54:43 -0500 Received: from dial.oleane.com ([195.25.225.86]) by sun2.articque.com (8.10.1/8.10.0) with ESMTP id f7DCn5324397; Mon, 13 Aug 2001 13:49:09 +0100 (GMT+0100) Message-ID: <3B77CD8D.BFF8964@dial.oleane.com> Date: Mon, 13 Aug 2001 13:52:29 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Galaxy , Mailing list Metakit Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] strstr Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Does anyone know where I could find the source code of standard strstr() function ? (I would like to know how they made it so quick !!) Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From vv@iware.com Mon Aug 13 07:12:08 2001 Received: from sdrc.com (heimdall.sdrc.com [146.122.132.195]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA32452 for ; Mon, 13 Aug 2001 07:12:08 -0500 Received: from tyr.sdrc.com (mailhub-cvg.sdrc.com [146.122.142.31]) by sdrc.com (8.9.1/8.9.1) with ESMTP id IAA03605; Mon, 13 Aug 2001 08:11:29 -0400 (EDT) Received: from boojum.iware.com (boojum [209.57.178.2]) by tyr.sdrc.com (8.8.6 (PHNE_17190)/8.8.5) with ESMTP id IAA15498; Mon, 13 Aug 2001 08:11:27 -0400 (EDT) Received: from bane.iware.com (bane.iware.com [209.57.178.105]) by boojum.iware.com (8.9.3/8.9.0) with ESMTP id IAA21453; Mon, 13 Aug 2001 08:11:28 -0400 Message-Id: <4.3.2.7.2.20010813081053.03b28ee8@mailhost.iware.com> X-Sender: vv@mailhost.iware.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Mon, 13 Aug 2001 08:11:27 -0400 To: gum@yahoogroups.com, Mailing list Galaxy , Mailing list Metakit From: "Victor R. Volkman" In-Reply-To: <3B77CD8D.BFF8964@dial.oleane.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Re: [gum] strstr Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Assuming you have VC++ and you bothered to install the C Run Time (CRT) source code it would be staring you in the face: Directory of C:\VS\VC98\CRT\SRC 06/17/1998 12:00a 1,382 STRSTR.C 1 File(s) 1,382 bytes At 01:52 PM 8/13/2001 +0100, Riccardo Cohen wrote: >Hi, >Does anyone know where I could find the source code of standard strstr() >function ? >(I would like to know how they made it so quick !!) > >Thanks > >-- >Riccardo Cohen > >Articque >Les Roches >37230 Fondettes >France >email = rcohen@dial.oleane.com >web = http://www.articque.com >tel: +33 02 47 49 90 49 >fax: +33 02 47 49 91 49 > > > >Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ From art_stine@mindspring.com Mon Aug 13 14:22:54 2001 Received: from smtp5-cm.mail.eni.net (smtp5-cm.mail.eni.net [216.133.226.138]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA14658 for ; Mon, 13 Aug 2001 14:22:54 -0500 Received: from DUALIE.mindspring.com (node-d8e9522.powerinter.net [216.233.82.2]) by smtp5-cm.mail.eni.net (8.9.3/8.9.3) with ESMTP id MAA02091; Mon, 13 Aug 2001 12:22:15 -0700 Message-Id: <5.1.0.14.0.20010813121957.0d331a88@pop3.norton.antivirus> X-Sender: art_stine/pop.mindspring.com@pop3.norton.antivirus X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 13 Aug 2001 12:22:06 -0700 To: gum@yahoogroups.com, Mailing list Galaxy , Mailing list Metakit From: Art Stine In-Reply-To: <3B77CD8D.BFF8964@dial.oleane.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Re: [gum] strstr Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: At 01:52 PM 8/13/2001 +0100, Riccardo Cohen wrote: >Hi, >Does anyone know where I could find the source code of standard strstr() >function ? >(I would like to know how they made it so quick !!) > >Thanks Go to www.freebsd.org and look thru the sources on line in their cvsweb interface (http://www.FreeBSD.org/cgi/cvsweb.cgi/) ... for i386, strstr() is not asm itself, but strncmp() is (which is where the bulk of the work is done). -art From nobody@www.prosapia.com Mon Aug 13 18:57:48 2001 Received: from www.prosapia.com ([64.40.109.174]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA23898 for ; Mon, 13 Aug 2001 18:57:48 -0500 Date: Mon, 13 Aug 2001 16:57:08 -0700 Message-Id: <200108132357.QAA10239@www.prosapia.com> To: metakit@equi4.com From: pr@grokster.com Subject: [Metakit] Grokster: Next Generation File Sharing Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sirs! We would like you to consider using/reviewing the Next Generation File Sharing tool. It is called Grokster and can be seen at www.Grokster.com Grokster accesses over 400,000+ online users through the FastTrack P2P Stack. Grokster is the latest generation of file sharing software. The Grokster generation of file sharing presents these exciting features: * Peer-to-Peer: True person to person file searches and transfers. * Fast Searches: Unlike Gnutella type programs, Grokster's SuperNode technology provides for very fast searches. * Detailed Searches: The Grokster technology tracks detailed file info and allows detailed searches on it. * No Incomplete/Failed Downloads: Grokster automatically removes incomplete downloads. * Auto Resume: Grokster will automatically seek out the same file from multiple sources and continue downloading until complete. * Fastest Downloads: Grokster will download the file from multiple sources with the fastest connections as fast as you can download, automatically. * File Preview: Preview files as you start downloading them. * All File Types: Share any type of file. * Full Bandwidth Controls: Grokster allows for full contol uploads/downloads and total bandwidth. This is a one time email. You will receive no other emails from us. Thank you for your time. Cheers relations@grokster.com http://www.grokster.com Grokster: Next Generation File Sharing From emailsebek@home.com Mon Aug 20 17:14:14 2001 Received: from femail25.sdc1.sfba.home.com (femail25.sdc1.sfba.home.com [24.254.60.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA27774 for ; Mon, 20 Aug 2001 17:14:04 -0500 Received: from home.com ([24.20.193.155]) by femail25.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20010820221242.WPGO21066.femail25.sdc1.sfba.home.com@home.com> for ; Mon, 20 Aug 2001 15:12:42 -0700 Message-ID: <3B818C51.89E04DEF@home.com> Date: Mon, 20 Aug 2001 15:16:49 -0700 From: Daniel Tracy X-Mailer: Mozilla 4.51 [en] (Win95; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Metakit: Multi-language storage access Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm developing an application in which I need to have access to a single storage file in two different languages: Python and C++. I clearly can't open it in each language using the .storage member function because I'd have two in memory instances. What I need is a simple way of sharing a storage with the two languages, but I haven't found one. Do I have to initiate the storage in Python and access an underlying C++ pointer, passing it to an initializing function in C++? I'd rather load the file in C++ given my program design. It would be most ideal if Mk4py could hook up to an existing, open C++ metakit class... Daniel From andreas.muegge@gmx.de Tue Aug 21 06:39:46 2001 Received: from mx0.gmx.net (mx0.gmx.net [213.165.64.100]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA07811 for ; Tue, 21 Aug 2001 06:39:45 -0500 Received: (qmail 18459 invoked by uid 0); 21 Aug 2001 11:38:05 -0000 Date: Tue, 21 Aug 2001 13:38:05 +0200 (MEST) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: metakit@equi4.com Cc: Riccardo Cohen MIME-Version: 1.0 References: <3B77CD8D.BFF8964@dial.oleane.com> Subject: Re: [Metakit] strstr X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [62.180.31.3] Message-ID: <32630.998393885@www36.gmx.net> X-Mailer: WWW-Mail 1.5 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Does anyone know where I could find the source code of standard strstr() > function ? > (I would like to know how they made it so quick !!) I found out that writing your own function based on the Boyer-Moore Sunday algorithm gives even better results than asm optimized but stupid strstr(). If someone is interested in the code he can send me an email - I have written a small class which uses the algo. Ciao, Andreas -- GMX - Die Kommunikationsplattform im Internet. http://www.gmx.net From j.p.fletcher@aston.ac.uk Tue Aug 21 07:04:11 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA08552 for ; Tue, 21 Aug 2001 07:04:06 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 15ZAGM-0000GV-00 for metakit@equi4.com; Tue, 21 Aug 2001 13:04:06 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Tue, 21 Aug 2001 13:01:05 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Priority: normal X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Subject: [Metakit] Re: Metakit: Multi-language storage access Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sorry, I meant to send this to the list. John ------- Forwarded Message Follows ------- From: Self To: Daniel Tracy Subject: Re: [Metakit] Metakit: Multi-language storage access Date sent: Tue, 21 Aug 2001 10:05:25 +0100 From: Daniel Tracy To: metakit@equi4.com Subject: [Metakit] Metakit: Multi-language storage access Date sent: Mon, 20 Aug 2001 15:16:49 -0700 > I'm developing an application in which I need to have access to a single storage file in > two different languages: Python and C++. I clearly can't open it in each language using > the .storage member function because I'd have two in memory instances. What I need is a > simple way of sharing a storage with the two languages, but I haven't found one. Do I > have to initiate the storage in Python and access an underlying C++ pointer, passing it to > an initializing function in C++? I'd rather load the file in C++ given my program > design. It would be most ideal if Mk4py could hook up to an existing, open C++ metakit > class... > > Daniel > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > Have you looked at SWIG, http://www.swig.org/ which would enable you to access your C++ code from python? I have only used SWIG for TCL and C++ myself. John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From jyl@best.com Sat Aug 25 14:07:58 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA00699; Sat, 25 Aug 2001 14:07:52 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id MAA06454; Sat, 25 Aug 2001 12:05:49 -0700 (PDT) Message-ID: <3B87F99D.68031A2C@best.com> Date: Sat, 25 Aug 2001 12:16:45 -0700 From: Jacob Levy X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com, jcw@equi4.com CC: jyl@best.com Subject: [Metakit] e4Graph 1.0a4 released References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: [JCW: Could you please post this on Metakit-announce? Thanks, --JYL] I'm pleased to announce the release of e4Graph 1.0a4. Get it from here: http://sourceforge.net/projects/e4graph/ WHAT IS E4GRAPH: e4Graph is a C++ library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. The e4Graph library is built on top of Metakit 2.4.0. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. Here are the CHANGE NOTES: Release: 1.0a4, 08/24/2001 This is the fourth release, the fourth Alpha BUG FIXES: * Re-imlemented deletion mechanism completely from scratch to fix severe performance issues for deletion of cyclical graphs (JYL). * Fixed a memory leak in the e4Graph Metakit storage driver (JYL). * Fixed several memory leaks in the Tcl binding of e4Graph (JYL). * Fixed e4_Node::SetVertex and e4_Node::SetVertexByRank; the bug was that when the previous value of the vertex was a node, the vertex would disappear instead of being set to a new value (DVM). * Fixed bug with XML parsing of character data sections; the bug was that a contiguous character data section could result in a series of "__data__" vertices instead of one "__data__" vertex containing the entire text of the section (JPF). NEW FUNCTIONALITY: * Implemented a framework for automatically updating the format of storages written by an older release to the format used by the new release. This should be transparent to users. Storage files written by e4Graph 1.0a3 should be readable by e4Graph 1.0a4 and onwards. Also implemented a mechanism to detect when a storage written by a newer release is read in an older release. In the future this will detect e.g. when a storage written by e4Graph 1.0b1 is read in a client using e4Graph 1.0a4. * New method e4_Node::DeleteFirstVertexWithNode to delete the first vertex of this node that has the argument node as its value (JYL, DVM). * XML parsing and generation of XML comments, XML CDATA sections, XML processing instructions, XML declarations, and document type declarations (JYL, JPF). STORAGE FORMAT CHANGE: The storage format used by e4Graph 1.0a4 is different from that used by e4Graph 1.0a3. However, e4Graph 1.0a4 is able to read storages written by e4Graph 1.0a3 and update their format on the fly to that used by e4Graph 1.0a4. e4Graph 1.0a3 can not read storages written by e4Graph 1.0a4 and later. E4GRAPH DIRECTORY STRUCTURE REORGANIZED: The 'core' directory has been renamed to 'main' (DVM). E4GRAPH BUILD STRUCTURE REORGANIZED: All build output goes to builds/* by default (depending on the configuration options given on Unix systems; this is the default on win32) (JYL, DVM). The win32 builds leave their product in builds\win32 instead of in individual sub-target directories (JYL, DVM). E4GRAPH NOW SUPPORTS STATIC LIBRARY BUILDS ON WIN32: * Static libraries and static linking of executables with e4Graph lead to faster execution at runtime. For example, teste4lib, the statically linked API tester, is about 14% faster than teste4dll, the dynamically linked API tester (DVM, JYL). * E4_STATIC must be defined for all those who want to compile from the e4graph headers and then link to the static library. I wish this were not the case, but there was no way around it. If it is not defined, it will only link with the dll library (DVM). * The 'core' project has been removed, and replaced by two new ones: - e4dll which is the dll-generating version - e4lib which is a statically-linked version In order to track this, the 'testcore' project has also been removed, and replaced by two new ones: - teste4dll which is the dll-requiring test program - teste4lib which is a statically-linked stand-alone program Note that running teste4dll requires that the e4graph and metakit dynamic libraries be installed at runtime in order to work. Teste4lib, however, has statically linked in the e4graph and metakit libraries, so that it does not depend on metakit or e4graph libraries to be installed at runtime (DVM). GENERAL BUILD CHANGES: * This release depends on Metakit 2.4.0 instead of on Metakit 2.3.4-29 (JYL). * This release depends on Expat 1.95.2 instead of on Expat 1.95.1 (JYL). * The current project settings and Makefiles require that the Metakit source code root directory be located as a peer to e4graph's root directory, and that it be called 'metakit-2.4.0'. This may be easily accomplished by downloading the zip or tar version of that release and unpacking it in the parent of where the e4graph code is rooted. Alternatively, the CVS version of Metakit may be used if its root directory name is changed from 'mk' to 'metakit-2.4.0'. If this is followed, the project files and Makefiles in e4graph may be used without alteration (JYL, DVM). BUILD CHANGES ON WIN32: * All compile output (including .pch, .obj, etc.) now goes to the 'builds\win32' directory. This means that you can delete everything there, and be sure of a clean build, but also that there is NO mess left behind in the other directories. Note that it now uses the post-link step ONLY to copy the header files (DVM). a. all .lib and .dll files are now in 'builds\win32\lib' b. all .exe files are now in 'builds\win32\bin' c. all .h files are still in 'builds\win32\include' d. all .obj files are now in 'builds\msvc\\Debug' or 'builds\msvc\\Release' * Since the debug/release and static/dynamic versions of the files are now in the same directories, they have different names to distinguish them. The naming convention is stolen from metakit. The following is representative of the convention, covering the dynamic/static and debug/release combinations for the e4graph and teste4graph code. The others follow similar conventions (DVM). a. Static Release in 'builds\win32\lib\e4graphs.lib' b. Static Debug in 'builds\win32\lib\e4graphs_d.lib' c. Dynamic Release in 'builds\win32\lib\e4graph.lib/dll' d. Dynamic Debug in 'builds\win32\lib\e4graph_d.lib/dll' e. Dynamic Test Release in 'builds\win32\bin\teste4dll.exe' f. Dynamic Test Debug in 'builds\win32\bin\teste4dll_d.exe' g. Static Test Release in 'builds\win32\bin\teste4lib.exe' h. Static Test Debug in 'builds\win32\bin\teste4lib_d.exe' --JYL From jerry@usd1.com Mon Aug 27 15:03:58 2001 Received: from asteroid.pacifier.com ([199.2.117.154]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA07244 for ; Mon, 27 Aug 2001 15:03:54 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id f7RK1gM25804 for ; Mon, 27 Aug 2001 13:01:43 -0700 (PDT) Date: Mon, 27 Aug 2001 13:01:07 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.53d) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <6515374064.20010827130107@usd1.com> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Python & row copies on new view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am having a problem the FIRST time a row is copied in a view. Any string fields that are set get converted to garbage the first time. I make a lot of use of copying rows, and I would like not to have to put special logic in to check if it is the first copy done for each view! The second and subsequent times it works perfectly. Example: >>> db=mk.storage() >>> t2 = db.getas('t2[name]') >>> t2.append() >>> t2.append() >>> t2[1].name = 'fred' >>> t2[0].name '' >>> t2[0] = t2[1] >>> t2[0].name '\xd0\xb26\x01\x18\xff8\x01' >>> t2[0] = t2[1] >>> t2[0].name 'fred' I get the same results if there are multiple string columns, if multiple rows were added, or if I copy to a row other than zero. Anyone have solutions or suggestions? -- Thanks, Jerry McRae MetaKit 2.4.0, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From lupoja@qwest.net Fri Aug 31 20:49:28 2001 Received: from dnvrpop3.dnvr.uswest.net (dnvrpop3.dnvr.uswest.net [206.196.128.5]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id UAA01154 for ; Fri, 31 Aug 2001 20:49:27 -0500 Received: (qmail 455 invoked by uid 0); 1 Sep 2001 01:47:34 -0000 Received: from ndialup193.dnvr.uswest.net (209.180.244.193) by dnvrpop3.dnvr.uswest.net with SMTP; 1 Sep 2001 01:47:34 -0000 Received: from egor.linux.home (localhost [127.0.0.1]) by localhost (8.12.0.Beta19/8.12.0.Beta19/Debian 8.12.0.Beta19) with ESMTP id f811lWIG006796 for ; Fri, 31 Aug 2001 19:47:32 -0600 Received: (from lupoja@localhost) by egor.linux.home (8.12.0.Beta19/8.12.0.Beta19/Debian 8.12.0.Beta19) id f811kbLL006721; Fri, 31 Aug 2001 19:46:37 -0600 Date: Fri, 31 Aug 2001 19:46:37 -0600 Message-Id: <200109010146.f811kbLL006721@egor.linux.home> From: "James A. Lupo" To: metakit@equi4.com Subject: [Metakit] I/O Error on Commit After Load Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm using Mk4tcl-2.4.0 on Linux and MS Windows-9x. Any time I attempt an "mk::file commit" after doing an "mk::file load" I get an I/O error. The data appears to have been set correctly following the load, but there is no way to save the data to a file. Here's a toy script that demonstrates the problem, stored in a file name "t.tcl" and sourced in tclsh (make sure t.dat does not exist before trying script): package require Mk4tcl mk::file open db t.dat mk::view layout db.dummy {f1 f2} mk::row append db.dummy f1 "field one" f2 "field two" mk::file commit db array set result [mk::get db.dummy!0] puts "Original, saved settings" puts "f1 = $result(f1)" puts "f2 = $result(f2)" set f [open t.sav w] mk::file save db $f close $f mk::set db.dummy!0 f1 "one" f2 "two" mk::file commit db array set result [mk::get db.dummy!0] puts "Modified settings" puts "f1 = $result(f1)" puts "f2 = $result(f2)" set f [open t.sav r] mk::file load db $f close $f array set result [mk::get db.dummy!0] puts "Reloaded settings." puts "f1 = $result(f1)" puts "f2 = $result(f2)" mk::file commit db mk::file close db Here is the results of running the script: % source t.tcl Original, saved settings f1 = field one f2 = field two Modified settings f1 = one f2 = two Reloaded settings. f1 = field one f2 = field two I/O error during commit This sort of thing use to work before. -- Jim ------------------- Optional Methods ----------------------- Dr James A. Lupo (303) 423-2652 9667 Independence Dr. lupoja@qwest.net Westminster CO 80021-6845 jalupo@regis.edu From jcw@equi4.com Wed Sep 5 12:32:46 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA18241 for ; Wed, 5 Sep 2001 12:32:45 -0500 Received: from takkie.equi4.nl ([195.121.235.162]) by smtp04.wxs.nl (Netscape Messaging Server 4.05) with ESMTP id GJ79YO00.VNC for ; Wed, 5 Sep 2001 19:30:24 +0200 Received: from [10.0.1.4] (airsie [195.108.246.51]) by takkie.equi4.nl (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f85HTdf15812 for ; Wed, 5 Sep 2001 19:29:39 +0200 From: Jean-Claude Wippler To: Date: Wed, 5 Sep 2001 19:30:26 +0200 Message-Id: <20010905173026.32078@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Fixed several bugs Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: After a (too) long absence, I've just fixed a number of bugs and checked them into CVS (see http://www.equi4.com/metakit/download.html for details): Hashing bug fixed: failed to terminate when looking for a missing key after a hash collision. The fill count was not being tracked properly to enforce that there is always at least one unused slot. Fixed bug when setting a string from a higher row: the data gets trashed, because too little copying was done while creating a gap. This bug may have led to some other cases of "damaged" datafiles. Solved by copying input data in c4_FormatB::SetOne(). Fixed long-standing bug with commit-after-load, by now doing a full view/subview copy. This is inefficient, but it avoids all sorts of mapping/strategy problems. This also fixes test s33 (at last!). Added regression tests b27, c21, m01, and s36 to check above fixes. Many many thanks to James Lupo, Jerry McRae, and Nathan Rogers for reporting these problems along with such great code samples - these have each been a big help in finding and fixing the problems. If any further problems remain, please let me know. Regards, Jean-Claude From j.p.fletcher@aston.ac.uk Fri Sep 7 04:58:09 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA10359 for ; Fri, 7 Sep 2001 04:58:09 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 15fIMY-0005T1-00 for metakit@equi4.com; Fri, 07 Sep 2001 10:55:50 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Fri, 7 Sep 2001 10:54:03 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Priority: normal X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Subject: [Metakit] Metakit 2.4.0 with mingw32 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am compiling the metakit 2.4.0 distribution with gcc 2.95.2 under mingw32. This combination defines __GCC__ and _WIN32 and this requires a patch to HEADER.H as follows: Comparing files header.h and f:header.h ****** header.h #define q4_DOS 1 #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || defined(_AIX) ****** f:header.h #define q4_DOS 1 #elif defined (_WIN32) && defined (__GNUC__) #define d4_OS_H "win.h" #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || defined(_AIX) ****** A further patch is needed in FILEIO.CPP Comparing files fileio.cpp and f:fileio.cpp ****** fileio.cpp void c4_FileStrategy::ResetFileMapping() { #if q4_WIN32 if (_mapStart != 0) ****** f:fileio.cpp void c4_FileStrategy::ResetFileMapping() { #if q4_WIN32 && !defined(__GNUC__) if (_mapStart != 0) ****** although this leaves the function ResetFileMapping() empty. Further testing is in progress. John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From j.p.fletcher@aston.ac.uk Fri Sep 7 05:51:44 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA11458 for ; Fri, 7 Sep 2001 05:51:43 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 15fJCO-0002tp-00 for metakit@equi4.com; Fri, 07 Sep 2001 11:49:24 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Fri, 7 Sep 2001 11:47:37 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] Metakit 2.4.0 with mingw32 - correction Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > I am compiling the metakit 2.4.0 distribution with gcc 2.95.2 under > mingw32. This combination defines __GCC__ and _WIN32 and __GCC__ should read __GNUC__ - sorry, John > this requires a patch to HEADER.H as follows: > > Comparing files header.h and f:header.h > ****** header.h > #define q4_DOS 1 > #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || > defined(_AIX) > ****** f:header.h > #define q4_DOS 1 > #elif defined (_WIN32) && defined (__GNUC__) > #define d4_OS_H "win.h" > #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || > defined(_AIX) > ****** > > A further patch is needed in FILEIO.CPP > > Comparing files fileio.cpp and f:fileio.cpp > > > > ****** fileio.cpp > void c4_FileStrategy::ResetFileMapping() > { > #if q4_WIN32 > if (_mapStart != 0) > ****** f:fileio.cpp > void c4_FileStrategy::ResetFileMapping() > { > #if q4_WIN32 && !defined(__GNUC__) > if (_mapStart != 0) > ****** > > although this leaves the function ResetFileMapping() empty. > > Further testing is in progress. > > John Fletcher > ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From pdominioni@rsr.it Mon Sep 10 10:35:28 2001 Received: from mono.mmservice.com ([217.56.215.2]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA01105 for ; Mon, 10 Sep 2001 10:35:23 -0500 Received: (qmail 24573 invoked from network); 10 Sep 2001 21:37:08 -0000 Received: from ppp-144-1.24-151.libero.it (HELO server) (151.24.1.144) by 0 with SMTP; 10 Sep 2001 21:37:08 -0000 Message-ID: <008701c13a0e$5d8eb030$7e7e7e7e@server> From: "RSR - Piero Dominioni" To: Date: Mon, 10 Sep 2001 17:28:35 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Subject: [Metakit] How to cross-compile for TQM823L? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi everybody. I downloaded Metakit-2.4.0.tar.gz and followed the instructions to build it under Unix (I've a Slckware 7.2 Linux-box on my Pentium II PC). Everything seem to go fine: I'm not familiar yet with Metakit, but the command: ./demo gives this result: The country of Paco Pena is: Spain So I guess the demo works and Metakit works too... My first need, however, is to embed metakit on a PowerPC board, TQM823L, with Linux installed on it. I have got the cross-compilation tools (powerpc-linux-gcc, powerpc-linux-ld, etc.), I've looked in the mailing list archives (without appreciable results) and I've tryed to hand- modify the templates for configure and Makefile. My attempt lead to errors, because I really don't know which are the correct settings to put in these files. Does anyone tell me how to modify the auto-configuration files or how to write a suitable Makefile by hand in order to cross-compile for my Motorola PowerPC 823 platform? Is there some specific documentation about this matter? Thanks in advance for any kind of help. Best regards. Piero Dominioni R.S.R. srl P.S.: If some information is missing to give a complete answer, please, let me know! From remy@shockfish.com Tue Sep 11 01:57:25 2001 Received: from berau.travel.ch (postfix@www.abp.ch [193.135.59.43] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA06067 for ; Tue, 11 Sep 2001 01:57:16 -0500 Received: from pc2.shockfish.com (shockfishpc1.epfl.ch [128.179.67.16]) by berau.travel.ch (Postfix) with ESMTP id 4F0EC12F015; Tue, 11 Sep 2001 08:50:38 +0200 (MEST) Message-Id: <5.0.2.1.0.20010911082955.00a7fea8@mail1.travel.ch> X-Sender: remymail@mail1.travel.ch (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 5.0.2 Date: Tue, 11 Sep 2001 08:53:31 +0200 To: metakit@equi4.com From: Remy Blank - Shockfish Cc: pdominioni@rsr.it Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Re: How to cross-compile for TQM823L? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Piero, I cross-compiled Metakit for ARM almost without changes to the makefiles or configure scripts. That's how I do it: - Go to "build" directory - Set environment variables for all build tools: AR=arm-linux-ar AS=arm-linux-as CC=arm-linux-gcc CXX=arm-linux-gcc NM=arm-linux-nm OBJCOPY=arm-linux-objcopy OBJDUMP=arm-linux-objdump RANLIB=arm-linux-ranlib STRIP=arm-linux-strip - Execute "../unix/configure --host=i386-pc-linux-gnu" (With other configuration option like "--prefix" if you need them) - Execute "make core" This builds the library. - Finally "make install-mk" This installs the library (I don't know if you need this step. You'll probably just want to copy the files in ".libs" to your target system.) Obviously, you'll want to substitute "arm-linux" with your particular target. IIRC, I had to make a few minor changes to "unix/configure" to make it work. There was one error in a test case where the script tries to compile a program with a main() without return value specification, and this stopped the compiler with an error. I can't remember the other changes. All this was with metakit-2.3.3. I don't know if it's still necessary with the newest version. Good luck! --Remy Blank PS: For the configure script, try this patch: --- configure 2001/02/15 08:47:16 1.1.1.1 +++ configure 2001/02/15 08:50:29 1.2 @@ -1925,7 +1925,7 @@ int main() { /* Ultrix mips cc rejects this. */ -typedef int charset[2]; const charset x; +//typedef int charset[2]; const charset x; /* SunOS 4.1.1 cc rejects this. */ char const *const *ccp; char **p; @@ -2000,7 +2000,7 @@ #include "confdefs.h" int main() { -} $ac_kw foo() { +} $ac_kw int foo() { ; return 0; } EOF if { (eval echo configure:2007: \"$ac_compile\") 1>&5; (eval $ac_compile) 2>&5; }; then From lvirden@cas.org Thu Sep 13 06:11:28 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA14307 for ; Thu, 13 Sep 2001 06:11:27 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id f8DB8F620279 for ; Thu, 13 Sep 2001 07:08:15 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f8DB8Dv24063; Thu, 13 Sep 2001 07:08:13 -0400 (EDT) Date: Thu, 13 Sep 2001 07:08:13 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010913070813.AAB24044@cas.org> To: metakit@equi4.com Subject: [Metakit] Wikit 'database' questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have a wikit that I would like to learn to manipulate. 1. How can I determine the titles of each of the pages in the wikit? 2. How can I extract the latest versions of all the pages in the wikit? 3. How can I take these extracted pages and insert them into a new, empty, wikit (say one that has been built with updated code, etc.) 4. How can I access the history of changes to any specific page on the wikit? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From horst@freedict.de Sun Sep 16 11:26:42 2001 Received: from mail.netbeat.de (mail.netbeat.de [212.6.214.38]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id LAA20097 for ; Sun, 16 Sep 2001 11:26:35 -0500 From: horst@freedict.de Received: (qmail 19883 invoked from network); 16 Sep 2001 16:36:20 -0000 Received: from dialin-145-254-116-141.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.116.141) by mail.netbeat.de with SMTP; 16 Sep 2001 16:36:20 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id f8GFrke01995 for ; Sun, 16 Sep 2001 17:53:47 +0200 Message-Id: <200109161553.f8GFrke01995@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Sun, 16 Sep 2001 17:53:43 +0200 (CEST) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] metakit & python - how to hash and sort Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I am using metakit in a python application, and want to (efficiently) access the elements (dictionary entries) in a sorted manner - that is the n'th element should be the n'th position when printed in a dictionary. what funktions would I use to do this? - I tried hash and ordered, but could not use both of them together Thanks, Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From klas@flum.net Tue Sep 18 12:00:20 2001 Received: from naver.egenweb.net (naver.egenweb.net [66.33.25.87]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA17573 for ; Tue, 18 Sep 2001 12:00:20 -0500 Received: from localhost (flum@localhost) by naver.egenweb.net (8.9.3/8.9.3) with ESMTP id TAA24825 for ; Tue, 18 Sep 2001 19:10:33 +0200 Date: Tue, 18 Sep 2001 19:10:32 +0200 (CEST) From: Klas Axelsson X-Sender: flum@naver.egenweb.net To: metakit@equi4.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Corrupt file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Have been using MetaKit for a short time and just had a problem with a corrupt database. I'm afraid that I haven't examined this very much myself yet, I intend to do a few tests and such, but I'll give a short description right away in case I never get around to it. I'm using Mk4py.so version 2.4.0 for linux. I find it very likely that the problem was caused by a full disk, the file could be rescued by removing the end of the file. If anyone would like to see the corrupt data file (about 9k) send me a private email. Regards Klas From tciuro@yahoo.com Wed Sep 19 14:33:03 2001 Received: from web20607.mail.yahoo.com (web20607.mail.yahoo.com [216.136.226.165]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA02563 for ; Wed, 19 Sep 2001 14:33:03 -0500 Message-ID: <20010919192956.16826.qmail@web20607.mail.yahoo.com> Received: from [17.202.22.175] by web20607.mail.yahoo.com via HTTP; Wed, 19 Sep 2001 12:29:56 PDT Date: Wed, 19 Sep 2001 12:29:56 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Corrupt file To: Klas Axelsson , metakit@equi4.com In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I think we have a real corruption bug in our hands. Klas sent me an example of a MetaKit data file which had been corrupted. He believes that it was caused by a lack of hard disk space. I ran the MetaKit's dump utility to check the contents and here's the output: board.backup: 0 properties VIEW 1 rows = Opening the file with a text editor reveals that the schema has been destroyed. However, the contents of the file are there. I have seen *exactly* the same problem, where 1MB, 5.3MB, and 10MB files have been corrupted. However, I disagree with Klas in one respect: hard disk space has nothing to do with it. I know that because I have 20GB of free disk space. I've seen this a few times, and at first I thought that it could be that I sent a corrupted schema to MetaKit, and the confusion caused MK to corrupt the data file. I still have no evidence that this is the case, but seeing Klas having the same problem, I wonder whether we have a serious bug in MetaKit. The version I use is 2.3.4r29/C++. Does anyone have more info on this issue? Jean-Claude, would you happen to know what could cause this? If you need a tangible proof, I can send you and example data file that has been corrupted. So, heads up everybody. Take care, -- Tito --- Klas Axelsson wrote: > Have been using MetaKit for a short time and just > had a problem with a > corrupt database. > > I'm afraid that I haven't examined this very much > myself yet, I intend to > do a few tests and such, but I'll give a short > description right away in > case I never get around to it. > > I'm using Mk4py.so version 2.4.0 for linux. > > I find it very likely that the problem was caused by > a full disk, the file > could be rescued by removing the end of the file. > > If anyone would like to see the corrupt data file > (about 9k) send me a > private email. > > Regards > Klas > > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Terrorist Attacks on U.S. - How can you help? Donate cash, emergency relief information http://dailynews.yahoo.com/fc/US/Emergency_Information/ From gmcm@hypernet.com Wed Sep 19 15:05:04 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA03990 for ; Wed, 19 Sep 2001 15:05:01 -0500 Received: from me (204.176.40.99) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Wed, 19 Sep 2001 16:00:42 -0400 From: "Gordon McMillan" To: metakit@equi4.com Date: Wed, 19 Sep 2001 16:00:40 -0400 Subject: Re: [Metakit] Corrupt file Reply-to: gmcm@hypernet.com Message-ID: <3BA8C128.7977.166E73DE@localhost> Priority: normal In-reply-to: <20010919192956.16826.qmail@web20607.mail.yahoo.com> References: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito wrote: > > I think we have a real corruption bug in our hands. I've had more than one corrupted MetaKit file, but in every case, it turned out that *I* had corrupted it. It's easy to do with threads (or multiple processes). You can do it single-threaded, too, by not thinking through callbacks / event driven code, or GUI-driven idle-tasks. In one app, it was an event-driven bug that slipped past months of QA testing and showed up in the field. One of QA's favorite tests was running out of disk space. I never saw my db get corrupted from that. (OTOH, the app did commits frequently, and slung files around like mad, so had lots of out- of-disk-space checks to protect the integrity of the files.) I was never able to learn squat from a corrupted DB. It was only in figuring out how to re-create the problem that it became obvious what I was doing. - Gordon ----rest of message-------------- > Klas sent me an example of a MetaKit data file which > had been corrupted. He believes that it was caused by > a lack of hard disk space. I ran the MetaKit's dump > utility to check the contents and here's the output: > > board.backup: 0 properties > > > VIEW 1 rows = > > Opening the file with a text editor reveals that the > schema has been destroyed. However, the contents of > the file are there. > > I have seen *exactly* the same problem, where 1MB, > 5.3MB, and 10MB files have been corrupted. However, I > disagree with Klas in one respect: hard disk space has > nothing to do with it. I know that because I have 20GB > of free disk space. > > I've seen this a few times, and at first I thought > that it could be that I sent a corrupted schema to > MetaKit, and the confusion caused MK to corrupt the > data file. I still have no evidence that this is the > case, but seeing Klas having the same problem, I > wonder whether we have a serious bug in MetaKit. > > The version I use is 2.3.4r29/C++. > > Does anyone have more info on this issue? Jean-Claude, > would you happen to know what could cause this? If you > need a tangible proof, I can send you and example data > file that has been corrupted. > > So, heads up everybody. > > Take care, > > -- Tito > > > --- Klas Axelsson wrote: > > Have been using MetaKit for a short time and just > > had a problem with a > > corrupt database. > > > > I'm afraid that I haven't examined this very much > > myself yet, I intend to > > do a few tests and such, but I'll give a short > > description right away in > > case I never get around to it. > > > > I'm using Mk4py.so version 2.4.0 for linux. > > > > I find it very likely that the problem was caused by > > a full disk, the file > > could be rescued by removing the end of the file. > > > > If anyone would like to see the corrupt data file > > (about 9k) send me a > > private email. > > > > Regards > > Klas > > > > > > > > > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > __________________________________________________ > Terrorist Attacks on U.S. - How can you help? > Donate cash, emergency relief information > http://dailynews.yahoo.com/fc/US/Emergency_Information/ > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Wed Sep 19 15:38:54 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA05441 for ; Wed, 19 Sep 2001 15:38:53 -0500 Received: from takkie.equi4.local ([195.121.237.245]) by smtp05.wxs.nl (Netscape Messaging Server 4.05) with ESMTP id GJXFVL00.Z92; Wed, 19 Sep 2001 22:35:45 +0200 Received: from [10.0.1.41] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f8JKZ3v15326; Wed, 19 Sep 2001 22:35:08 +0200 From: Jean-Claude Wippler To: Klas Axelsson , metakit Subject: Re: [Metakit] Corrupt file Date: Wed, 19 Sep 2001 22:36:20 +0200 Message-Id: <20010919203620.10776@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Klas, >I'm using Mk4py.so version 2.4.0 for linux. > >I find it very likely that the problem was caused by a full disk, the file >could be rescued by removing the end of the file. > >If anyone would like to see the corrupt data file (about 9k) send me a >private email. Please do send me a copy. This sounds ominous. I have another report of damage - using MK > 2.01, i.e. with the new file format implementation. The fact that removing the end "fixes" the file is a sign that you caught the problem very early on: MK commits by calculating the new file size, putting a dummy marker there, saving changes without altering the last- committed data (but not necessarily writing just at the end: it may write in unused space), and then as very last step changing the end marker to a "commit-ok" one. The last changes (Sep 5) fixed bugs which can corrupt files. I am currently chasing another problem, which seems partly just in Mk4tcl, but has also uncovered a deeper one (though it does not seem to corrupt - just fail). As soon as that has been resolved, I'll put new binaries on the web. As well as all changes in CVS of course. The key question now is: can the problem be repeated? It will be especially important once there is a new build - to make sure everything is fixed. -jcw From PSchlegel@moodlogic.com Wed Sep 26 14:52:55 2001 Received: from hqmx02.hq.moodlogic.com (nobody@dsl081-060-070.sfo1.dsl.speakeasy.net [64.81.60.70]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA23928 for ; Wed, 26 Sep 2001 14:52:38 -0500 Received: by hqmx02.hq.moodlogic.com with Internet Mail Service (5.5.2653.19) id ; Wed, 26 Sep 2001 12:05:18 -0700 Message-ID: <04BE866D1B25C348B9C40E043B216D4836A9B9@hqmx02.hq.moodlogic.com> From: Paul Schlegel To: "'metakit@equi4.com'" Date: Wed, 26 Sep 2001 12:05:17 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Threading problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I'm working on a Win32 project that uses multiple MetaKit databases accessed in different threads. I've seen several notes that MetaKit isn't thread safe, but for some time it seemed that I was getting away with using thread locks around code that accesses these databases until reports of corrupted databases started coming in. After some investigation, a partner of mine noticed that in the file "fileio.cpp" around line 248 there is a block a Win32 code that uses a static buffer (which would be shared by all threads) for writing files. This strongly appears to be a culprit in the corruption that we are seeing. So, my questions are... 1) What are the repercussions of making this static buffer a variable in the c4_FileStrategy class, or perhaps even a stack variable? 2) My project is intended for Win98 and above. There is a comment in the block of code in question that says it is a hack for network drives on Win95. Could I possible remove this block of code (and the static buffer) altogether? 3) Are there other parts of the code that make threaded use of MetaKit impossible, i.e. use of global or static buffers? What parts of the code are they? For clarification, I'm using one thread lock object per database (as opposed to one for all MetaKit accesses). I suppose that if only one lock was used for all database interactions, then any contention with this static buffer would be avoided. Thanks in advance. From horst@freedict.de Thu Sep 27 23:49:08 2001 Received: from mail.netbeat.de ([62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA11566 for ; Thu, 27 Sep 2001 23:49:07 -0500 From: horst@freedict.de Received: (qmail 22678 invoked from network); 28 Sep 2001 04:29:37 -0000 Received: from dialin-145-254-113-076.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.113.76) by mail.netbeat.de with SMTP; 28 Sep 2001 04:29:37 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id f8RL9Is00856 for ; Thu, 27 Sep 2001 23:09:19 +0200 Message-Id: <200109272109.f8RL9Is00856@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Thu, 27 Sep 2001 23:09:15 +0200 (CEST) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] MetaKit usage for Dictionary application Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I asked the question already on this newsgroup, but as I did not receive an answer, I ask again: I am writing a dictionary application in (wx)Python, which stores the data in utf-8. Actually the translation is stored in XML, but this is not of importance, as each translation is stored in a separate database row. I expect to have 100k .. 1M Words (definitions) in the dictionary (for each language), and the translations are about 200bytes summing up to 20MBytes of translation text. The database also should store extra information, as the Author and Date last changed. Two things appear important at the moment: a) the application (obviously) needs to access the translation for a given word quite fast. - there is no need for case sensitivity, or similar string operations I am not sure which of the views I should apply (assuming that doing nothing special will not give efficient access) b) For a wordlist, I want to display the words in a dictionary sorted way, according to the language beeing used (buzzword collate). - MetaKit uses utf-8 to store data, but it does not provide a way of sorting in the required way, does it? Luckily Python does provide a function to generate a index which can be used in a "normal" sort (locale.strxfrm). Would it be a good idea to store the index in a "B" field, and sort it? How would I connect the two requirenments? v1 = view("B:collate,S:word,S:author,S:date") v2 = v1.sort() v3 = v2. ... I use v1 to insert data, v2 to display the wordlist and v3 to access the translation from the "Word" Thank you for your help Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From horst@freedict.de Sat Sep 29 02:40:38 2001 Received: from mail.netbeat.de (mail.netbeat.de [212.6.214.38]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id CAA22518 for ; Sat, 29 Sep 2001 02:40:37 -0500 From: horst@freedict.de Received: (qmail 11461 invoked from network); 29 Sep 2001 07:51:23 -0000 Received: from dialin-145-254-118-200.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.118.200) by mail.netbeat.de with SMTP; 29 Sep 2001 07:51:23 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id f8T7DJI01418 for ; Sat, 29 Sep 2001 09:13:20 +0200 Message-Id: <200109290713.f8T7DJI01418@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Sat, 29 Sep 2001 09:13:17 +0200 (CEST) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] metakit documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I think some of the documentation from "the MetaKit database format" actually should go into the MetaKit documentation. - Otherwise it is difficult to understand some of the mecanics behind MetaKit Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From horst@freedict.de Sat Sep 29 02:40:40 2001 Received: from mail.netbeat.de (mail.netbeat.de [212.6.214.38]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id CAA22523 for ; Sat, 29 Sep 2001 02:40:39 -0500 From: horst@freedict.de Received: (qmail 11482 invoked from network); 29 Sep 2001 07:51:26 -0000 Received: from dialin-145-254-118-200.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.118.200) by mail.netbeat.de with SMTP; 29 Sep 2001 07:51:26 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id f8T7INI01449 for ; Sat, 29 Sep 2001 09:18:24 +0200 Message-Id: <200109290718.f8T7INI01449@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Sat, 29 Sep 2001 09:18:20 +0200 (CEST) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] Compressed memo fields? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, yet another idea: Reading through "the MetaKit data format", I wondered, if (combined with the blocked view) a compressed memo (or whatever) would be feasible. I do have data, which compresses very well (a factor of approx. 10). So compression propably would be a good idea. -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From jrichards@starband.net Sun Sep 30 11:38:31 2001 Received: from c0mailgw08.prontomail.com (mailgw.prontomail.com [216.163.180.10]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA13614 for ; Sun, 30 Sep 2001 11:38:31 -0500 Received: by c0mailgw08.prontomail.com (NPlex 5.5.029) id 3B9FF675004F29F4 for metakit@equi4.com; Sun, 30 Sep 2001 09:27:26 -0700 Received: from 148.75.36.102 by SmtpServer for ; Sun, 30 Sep 2001 16:27:23 +0000 X-Sender: 14195.starband.net From: "Jim Richards" To: Date: Sun, 30 Sep 2001 12:24:29 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Subject: [Metakit] Groupby and Counts Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am looking for some guidance on using groupby and counts. I am new to python and don not fully understand the documention supplied. I am hoping to use these two functions to further analyze my firewall logs. Thanks Jim Richards From mlh@idi.ntnu.no Sun Sep 30 12:53:22 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA14562 for ; Sun, 30 Sep 2001 12:53:21 -0500 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id TAA27503 for ; Sun, 30 Sep 2001 19:49:29 +0200 (MEST) Message-ID: <011401c149d8$4118c330$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: Date: Sun, 30 Sep 2001 19:49:29 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Time series? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've been thinking about using metakit to store/process time series data, but I'm not quite sure how to store it... Simply put, I have a number of sequences, where each sequence element has a timestamp. The value of the element might be a number, or a vector, or anything, really. (But I can live with only having a number, I guess.) How is this best stored? One possibility is certainly to just stuff each sequence in one binary field... Another is to have two columns per sequence, but when I have hundreds or thousands of sequences, something tells me this isn't a good idea... :) A simple solution would be to use timestamp, sequence ID, and value as three fields in one table... But if I need to access the elements of a sequence sequentially, can this be done efficiently somehow? (I.e. can I be certain that they are stored that way or something?) Perhaps MetaKit isn't the right solution for this, I don't know. -- Magnus Lie Hetland http://www.hetland.org "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick From gmcm@hypernet.com Sun Sep 30 12:57:39 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA14650 for ; Sun, 30 Sep 2001 12:57:39 -0500 Received: from me (204.176.40.88) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sun, 30 Sep 2001 13:47:53 -0400 From: "Gordon McMillan" To: "Jim Richards" , Date: Sun, 30 Sep 2001 13:53:17 -0400 Subject: Re: [Metakit] Groupby and Counts Reply-to: gmcm@hypernet.com Message-ID: <3BB723CD.1059.4E9FC9FA@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jim Richards wrote: > I am looking for some guidance on using groupby and counts. I am > new to python and don not fully understand the documention > supplied. I am hoping to use these two functions to further > analyze my firewall logs. That's an open-ended question. Let's assume you've shoved your logs into a view log[ipfrom:S,ipto:S,tmstmp:D,devfrom:S,devto:S...] Given: logv = db.view('log') You'd use: v2 = logv.groupby(logv.ipfrom, 'details') or v3 = logv.counts(logv.ipfrom, 'detailcounts') (or any other column(s) in place of ipfrom...) and note that len(v2[i].details) == v3[i].detailcounts Also note that v2[i].details is a view, so can be iterated, groupby-ed, counts-ed or anything else. Does that get you started? - Gordon From jcw@equi4.com Mon Oct 1 04:35:02 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA27687 for ; Mon, 1 Oct 2001 04:35:01 -0500 Received: from takkie.equi4.local ([195.121.232.180]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKIT3U00.NZ7 for ; Mon, 1 Oct 2001 11:31:06 +0200 Received: from [10.0.1.19] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f919TuF16688 for ; Mon, 1 Oct 2001 11:29:57 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Threading problems Date: Mon, 1 Oct 2001 11:31:53 +0200 Message-Id: <20011001093153.23954@mail.planet.nl> In-Reply-To: <04BE866D1B25C348B9C40E043B216D4836A9B9@hqmx02.hq.moodlogic.com> References: <04BE866D1B25C348B9C40E043B216D4836A9B9@hqmx02.hq.moodlogic.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Paul Schlegel wrote: [...] >After some investigation, a partner of mine noticed that in the file >"fileio.cpp" around line 248 there is a block a Win32 code that uses a >static buffer (which would be shared by all threads) for writing files. >This strongly appears to be a culprit in the corruption that we are seeing. > >So, my questions are... > >1) What are the repercussions of making this static buffer a variable in the >c4_FileStrategy class, or perhaps even a stack variable? Either way sounds like a good idea. >2) My project is intended for Win98 and above. There is a comment in the >block of code in question that says it is a hack for network drives on >Win95. Could I possible remove this block of code (and the static buffer) >altogether? No, I doubt it. Memory mapped files in Win9x seem to be somewhat "hacked-on" to the DOS core. The problem is that one cannot write to file without affecting the seek pointer which the memory-mapped read operation requires. Then again, it's quite easy to test: use a network drive and try it. Make local changes in a sufficiently large view, then commit. A few attempts will probably trigger the problem. >3) Are there other parts of the code that make threaded use of MetaKit >impossible, i.e. use of global or static buffers? What parts of the code >are they? No, this is AFAIK the only issue. MK used to be ok, until this particular case had to be worked around. >For clarification, I'm using one thread lock object per database (as opposed >to one for all MetaKit accesses). I suppose that if only one lock was used >for all database interactions, then any contention with this static buffer >would be avoided. Yes, but again, if you fix the above buffer contention, you should be fine as long as each datafile is accessed only within the thread thatr opens it. -jcw From jcw@equi4.com Mon Oct 1 04:35:18 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA27696 for ; Mon, 1 Oct 2001 04:35:18 -0500 Received: from takkie.equi4.local ([195.121.232.180]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKIT4801.8WE for ; Mon, 1 Oct 2001 11:31:20 +0200 Received: from [10.0.1.19] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f919UAF16720 for ; Mon, 1 Oct 2001 11:30:10 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] MetaKit usage for Dictionary application Date: Mon, 1 Oct 2001 11:32:07 +0200 Message-Id: <20011001093207.2361@mail.planet.nl> In-Reply-To: <200109272109.f8RL9Is00856@eaglesnest.mceggman> References: <200109272109.f8RL9Is00856@eaglesnest.mceggman> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: [dictionary app] >Two things appear important at the moment: >a) the application (obviously) needs to access the translation for a >given word quite fast. - there is no need for case sensitivity, or >similar string operations > >I am not sure which of the views I should apply (assuming that doing >nothing special will not give efficient access) I'd look at hashing, and once things get larger also blocking. A hashed mapping provides hashed access as add-on to an existing view. A blocked mapping maintains the illusion of a single view, but internally breaks things up into subviews, which is likely to be more efficient for say >100,000 rows. >b) For a wordlist, I want to display the words in a dictionary sorted >way, according to the language beeing used (buzzword collate). - MetaKit >uses utf-8 to store data, but it does not provide a way of sorting in >the required way, does it? Not directly, but if you create a view with a single int as contents, referring to the index in the big data view, then you can "remap" them, i.e. apply one as a permutation to the other. The result is a view which has been reshuffled: import metakit db=metakit.storage() v1=db.getas('v1[text:S]') v2=db.getas('v2[index:I]') for s in "hello there": v1.append(s) for i in range(len(v1),0,-1): v2.append(i-1) v3=v1.remapwith(v2) metakit.dump(v1,'v1') metakit.dump(v2,'v2') metakit.dump(v3,'v3') The result is that v3 has the contents of v1, but with all rows reversed. It's all virtual, no copying is done. -jcw From jcw@equi4.com Mon Oct 1 04:35:56 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA27716 for ; Mon, 1 Oct 2001 04:35:56 -0500 Received: from takkie.equi4.local ([195.121.232.180]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKIT4J01.61B for ; Mon, 1 Oct 2001 11:31:31 +0200 Received: from [10.0.1.19] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f919ULF16724 for ; Mon, 1 Oct 2001 11:30:21 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Compressed memo fields? Date: Mon, 1 Oct 2001 11:32:18 +0200 Message-Id: <20011001093218.13651@mail.planet.nl> In-Reply-To: <200109290718.f8T7INI01449@eaglesnest.mceggman> References: <200109290718.f8T7INI01449@eaglesnest.mceggman> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: >Reading through "the MetaKit data format", I wondered, if (combined with >the blocked view) a compressed memo (or whatever) would be feasible. > >I do have data, which compresses very well (a factor of approx. 10). So >compression propably would be a good idea. Good point, thanks for bringing up the issue again. There are two aspects to this: compression on an item-by-item basis, and compression over entire columns. The first is easy to add on top of MK, and is in fact what TclKit does by default in the Virtual File System it implements. The second is more involved. There are ways to implement them, similarly to what the "myio" does with encryption, but it means writing a new derived "c4_Strategy" class. I've refrained from implementing this as part of MK, because it would need some sort of new interface to be able to specify what compression engine to use, and because ideally compression should not be done data-file wide but only on specific columns. OTOH, it might be simpler to stick to compress-everything and then simply split things into two datafiles after all. -jcw From jcw@equi4.com Mon Oct 1 04:36:06 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA27735 for ; Mon, 1 Oct 2001 04:36:06 -0500 Received: from takkie.equi4.local ([195.121.232.180]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKIT4T00.H9N for ; Mon, 1 Oct 2001 11:31:41 +0200 Received: from [10.0.1.19] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f919UVF16728 for ; Mon, 1 Oct 2001 11:30:32 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Time series? Date: Mon, 1 Oct 2001 11:32:28 +0200 Message-Id: <20011001093228.28634@mail.planet.nl> In-Reply-To: <011401c149d8$4118c330$156ff181@idi.ntnu.no> References: <011401c149d8$4118c330$156ff181@idi.ntnu.no> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Magnus Lie Hetland wrote: >I've been thinking about using metakit to store/process >time series data, but I'm not quite sure how to store >it... > >Simply put, I have a number of sequences, where each >sequence element has a timestamp. The value of the >element might be a number, or a vector, or anything, >really. (But I can live with only having a number, >I guess.) > >How is this best stored? One possibility is certainly >to just stuff each sequence in one binary field... >Another is to have two columns per sequence, but when >I have hundreds or thousands of sequences, something >tells me this isn't a good idea... :) > >A simple solution would be to use timestamp, sequence >ID, and value as three fields in one table... But if >I need to access the elements of a sequence sequentially, >can this be done efficiently somehow? (I.e. can I be >certain that they are stored that way or something?) > >Perhaps MetaKit isn't the right solution for this, I >don't know. It's hard to tell. The main aspect where you may expect good performance as well as compact representation when using MK, is when the data type is homogenous in "some" direction. MK's data storage is columnar - you need to keep this in mind to get to grips with the implications for type. If sequences are typed, but different sequences can have different types, then you might want to consider a data structure of the form: seq[id:I,type:S,index:I] ints[vec[i:I]] strings[vec[s:S]] doubles[vec[d:D]] What this does is create a sequence of sequence desriptions, each description has an id (well, whatever you need to identify it), a type (being "ints", "strings", "doubles", i.e. the name of the view where the data actually resides. Then, each of the remaining views is a strongly- typed collection of sequences. If a sequence has several fields, then you could specify them instead as: seq[id:I,fields[name:S,type:S,index:I]] The datastructures created this way are essentially vectors: a sequence is one (or more) vectors, accessed by indexing as a subview in the set which has the proper datatype. Hm, maybe I've lost you while describing this... perhaps a brief sample, or a description of a few operations you need to implement, would make it easier to make suggestions or mention trade-offs. For an elaborate system which uses similar design tricks to store generalized graphs in MK, see also Jacob Levy's "e4graph", it's at: http://sf.net/projects/e4graph -jcw From mlh@idi.ntnu.no Mon Oct 1 18:39:33 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA28614 for ; Mon, 1 Oct 2001 18:39:32 -0500 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id BAA28234 for ; Tue, 2 Oct 2001 01:35:35 +0200 (MEST) Message-ID: <036501c14ad1$c507d650$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: Date: Tue, 2 Oct 2001 01:35:35 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] A Question about Blocked and Hashed Views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: (Sorry if this is a very obvious question...) In metakit for Python, it seems that view.hash() and view.blocked() can be called without any arguments, which seems reasonable... But the documentation gives the impression that you must supply a mapview and a blockview, respectively. (And, it seems to be able to specify numkeys for hash you have to supply a mapview). My question is basically, what would be the point of supplying such a view, if its structure is set, and you can't edit it? (Or is that only the case for hash? I haven't really understood what "blockview" is...) And... If these are optional arguments (as they seem to be), perhaps that should be indicated in the docs? Just a thought. -- Magnus Lie Hetland http://www.hetland.org "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick From mlh@idi.ntnu.no Mon Oct 1 18:58:07 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA28914 for ; Mon, 1 Oct 2001 18:58:06 -0500 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id BAA29063 for ; Tue, 2 Oct 2001 01:54:10 +0200 (MEST) Message-ID: <036b01c14ad4$5d631020$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: References: <011401c149d8$4118c330$156ff181@idi.ntnu.no> <20011001093228.28634@mail.planet.nl> Subject: Re: [Metakit] Time series? Date: Tue, 2 Oct 2001 01:54:10 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Jean-Claude Wippler" > > Magnus Lie Hetland wrote: [snip] > It's hard to tell. The main aspect where you may expect good performance > as well as compact representation when using MK, is when the data type is > homogenous in "some" direction. MK's data storage is columnar - you need > to keep this in mind to get to grips with the implications for type. Yes, I've understood that. > If sequences are typed, but different sequences can have different types, > then you might want to consider a data structure of the form: > seq[id:I,type:S,index:I] > ints[vec[i:I]] > strings[vec[s:S]] > doubles[vec[d:D]] Ah... That's clever :) [snip] > If a sequence has several fields, then you could specify them instead as: > seq[id:I,fields[name:S,type:S,index:I]] Even more clever :)) (I'm gradually getting used to the mk way of doing things -- and I like it a lot.) > The datastructures created this way are essentially vectors: a sequence > is one (or more) vectors, accessed by indexing as a subview in the set > which has the proper datatype. Right. > Hm, maybe I've lost you while describing this... perhaps a brief sample, > or a description of a few operations you need to implement, would make it > easier to make suggestions or mention trade-offs. Ok... Small example (based on your suggestions). I'm working on (among other things) comparing time sequences. Let's say you want to compare melodies. The sequence would have two fields: Time and pitch (for each note -- we'll use only the "note on" timestamp). Then I could use something like this? tune[notes[time:D,pitch:D]] (Assuming that the tune id is simply its index.) I guess one of the things that would have to be efficient is to iterate over the notes of a given tune, which would be quite efficient, right? I think the main reason why I couldn't see how to do this sort of thing is that I didn't think of using subviews. (Stuck in a simpler table terminology, I guess ;) > For an elaborate system which uses similar design tricks to store > generalized graphs in MK, see also Jacob Levy's "e4graph", it's at: > http://sf.net/projects/e4graph I've seen it, although I haven't looked at the details. Perhaps I should. > -jcw - Magnus, who is very happy about using MetaKit in Python :)) -- Magnus Lie Hetland http://www.hetland.org "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick From lvirden@cas.org Tue Oct 2 09:50:37 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA17839 for ; Tue, 2 Oct 2001 09:50:37 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id f92Ek8709866 for ; Tue, 2 Oct 2001 10:46:08 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f92Ek7A22432; Tue, 2 Oct 2001 10:46:07 -0400 (EDT) Date: Tue, 2 Oct 2001 10:46:07 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20011002104607.AAB22397@cas.org> To: Subject: [Metakit] Help sought for dealing with the Tcler's Wiki scripted document Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What are the commands needed to interact with the scripts and data that comprise the Tcler's Wiki as it exists today? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Wed Oct 3 07:03:34 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA06235 for ; Wed, 3 Oct 2001 07:03:32 -0500 Received: from takkie.equi4.local ([195.121.234.226]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKMPAB00.209 for ; Wed, 3 Oct 2001 13:58:59 +0200 Received: from [10.0.1.21] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f93Bw6F00880 for ; Wed, 3 Oct 2001 13:58:06 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Help sought for dealing with the Tcler's Wiki scripted document Date: Wed, 3 Oct 2001 13:59:24 +0200 Message-Id: <20011003115924.11317@mail.planet.nl> In-Reply-To: <20011002104607.AAB22397@cas.org> References: <20011002104607.AAB22397@cas.org> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >What are the commands needed to interact with the scripts and data that >comprise the Tcler's Wiki as it exists today? As with every scripted document: it's all in the scripts for you to inspect and adjust as you like. The complicating factor with the current version of the Tcl'ers Wiki is that it uses pre-VFS style scripted documents. The only way to access *was* through a somewhat tricky "command line interface". But there is good news... here is a script which lets you list and extract the contents of all pages stored in the wiki, it's at: http://www.equi4.com/wikit/wikitool Do a chmod +x and run it without args to get info about usage. This self-contained script should run with any recent Tcl installation, because it includes a brand-new Metakit datafile reader coded entirely in Tcl. Do you wish to extract the wikit's own scripting code as well? -jcw From lvirden@cas.org Wed Oct 3 07:09:16 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA06612 for ; Wed, 3 Oct 2001 07:09:16 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id f93C4i720404 for ; Wed, 3 Oct 2001 08:04:44 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f93C4gd04554; Wed, 3 Oct 2001 08:04:42 -0400 (EDT) Date: Wed, 3 Oct 2001 08:04:42 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20011003080442.AAB4543@cas.org> Subject: Re: [Metakit] Help sought for dealing with the Tcler's Wiki In-Reply-To: <20011003115924.11317@mail.planet.nl> of Wed, 3 Oct 2001 13:59:24 +0200 To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler > Do you wish to extract the wikit's own scripting code as well? Yes, I sometimes have a need/desire to do this as well. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From nicolasb@maich.gr Wed Oct 3 09:00:26 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA11401 for ; Wed, 3 Oct 2001 09:00:18 -0500 Received: FROM maich.gr BY mail-server ; Wed Oct 03 17:08:17 2001 +0300 Message-ID: <3BBB1CD8.17CFA553@maich.gr> Date: Wed, 03 Oct 2001 17:12:40 +0300 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Subject: [Metakit] Copying/Saving Views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I wonder how one could simply copy a view or a slice of a view from a main data file, and save it to file in Tclkit. I want basically to keep track of record(s) deleted from a view and stick them in a deleted.dat file, which maybe opened for inspection in the future... set main_view [mk::view open main.view] set deleted_view [mk::view open deleted.view] Now what? Currently looping through records and it's not much fun, esp. when a datafile contains~20 views... Would appreciate any help... tia, nicolas boretos From nrogers@softhome.net Sat Oct 6 12:04:14 2001 Received: from softhome.net (jive.SoftHome.net [66.54.152.27]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA16111 for ; Sat, 6 Oct 2001 12:04:14 -0500 Received: from cx292990-b ([24.254.252.167]) by softhome.net with esmtp; Sat, 06 Oct 2001 10:56:57 -0600 Date: Sat, 6 Oct 2001 12:00:21 -0500 From: Nathan Rogers To: metakit@equi4.com Message-Id: <20011006120021.290da3de.nrogers@softhome.net> X-Mailer: Sylpheed version 0.6.2 (GTK+ 1.2.10; i386-debian-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] cvs hash performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've been trying out the CVS version of Metakit lately, and I noticed that the time it takes to append a row to a hashed view has increased dramatically since 2.4. With a few thousand rows, inserts take about 100 times longer than they used to. Adding blocking to the data and/or map views doesn't seem to help. Any ideas on what might be causing this or what can be done about it? Also, I'm having trouble getting Ordered views to work in C++. I've been trying code like this. c4_Storage db; c4_IntProp pKey ("key"); c4_View vw = db.GetAs("test[key:I]"); c4_View ov = vw.Ordered(); ov.Add(pKey[3]); But it always seems to crash. The same thing seems to work fine in python and tcl. Nathan Rogers From tciuro@mac.com Sat Oct 6 13:39:30 2001 Received: from smtp-out.mac.com ([204.179.120.86]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA16798 for ; Sat, 6 Oct 2001 13:39:30 -0500 Received: from smtp-relay01.mac.com (smtp-relay01-qfe3 [10.13.10.224]) by smtp-out.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f96IXOa10691 for ; Sat, 6 Oct 2001 11:33:25 -0700 (PDT) Received: from asmtp01.mac.com ([10.13.10.65]) by smtp-relay01.mac.com (Netscape Messaging Server 4.15 relay01 Jun 21 2001 23:53:48) with ESMTP id GKSRMM00.3SA for ; Sat, 6 Oct 2001 11:35:10 -0700 Received: from localhost ([209.233.25.139]) by asmtp01.mac.com (Netscape Messaging Server 4.15 asmtp01 Jun 21 2001 23:53:48) with ESMTP id GKSRML00.VD0 for ; Sat, 6 Oct 2001 11:35:09 -0700 Date: Sat, 6 Oct 2001 11:35:10 -0700 Mime-Version: 1.0 (Apple Message framework v500) Content-Type: multipart/alternative; boundary=Apple-Mail-1-463790074 From: Tito Ciuro To: metakit@equi4.com Message-Id: X-Mailer: Apple Mail (2.500) Subject: [Metakit] Why is View.Search() not working? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-463790074 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hi, I posted a similar email a while ago and nobody came up with an answer. Let's try again! :) I'm trying to use the Search() method because it's supposed to be faster than Find(). The code I use to try it out is this: > void QuickTest(c4_View *cacheView) > { > c4_View aView = cacheView->Sort(); > int i, num, value; > > { > c4_IntProp pUID ("UID"); > num = aView.GetSize(); > > printf("Number of records: %d\n", num); > for (i = 0; i < num; i++) { > value = pUID(aView[i]); > printf(" ID %d is in row %d\n", value, i); > } > > printf(" Search for -1: is in row %d\n", > aView.Search(pUID[-1])); > printf(" Search for 1: is in row %d\n", > aView.Search(pUID[1])); > printf(" Search for 3: is in row %d\n", > aView.Search(pUID[3])); > } > } This the output I get: > Number of records: 4 > ID 1 is in row 0 > ID 2 is in row 1 > ID 3 is in row 2 > ID 4 is in row 3 > Search for -1: is in row 0 > Search for 1: is in row 0 > Search for 3: is in row 2 How come that searching for a non-existing value returns row zero? (row zero is occupied by UID #1) If Search() returned -1 for non-found values, it'll be a lot better! Is this a known bug? Thanks, -- Tito --Apple-Mail-1-463790074 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hi, I posted a similar email a while ago and nobody came up with an answer. Let's try again! :) I'm trying to use the Search() method because it's supposed to be faster than Find(). The code I use to try it out is this: void QuickTest(c4_View *cacheView) { c4_View aView = cacheView->Sort(); int i, num, value; { c4_IntProp pUID ("UID"); num = aView.GetSize(); printf("Number of records: %d\n", num); for (i = 0; i << num; i++) { value = pUID(aView[i]); printf(" ID %d is in row %d\n", value, i); } printf(" Search for -1: is in row %d\n", aView.Search(pUID[-1])); printf(" Search for 1: is in row %d\n", aView.Search(pUID[1])); printf(" Search for 3: is in row %d\n", aView.Search(pUID[3])); } } This the output I get: Number of records: 4 ID 1 is in row 0 ID 2 is in row 1 ID 3 is in row 2 ID 4 is in row 3 Search for -1: is in row 0 Search for 1: is in row 0 Search for 3: is in row 2 How come that searching for a non-existing value returns row zero? (row zero is occupied by UID #1) If Search() returned -1 for non-found values, it'll be a lot better! Is this a known bug? Thanks, -- Tito --Apple-Mail-1-463790074-- From nrogers@softhome.net Sat Oct 6 14:39:05 2001 Received: from softhome.net (jive.SoftHome.net [66.54.152.27]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA17204 for ; Sat, 6 Oct 2001 14:39:05 -0500 Received: from cx292990-b ([24.254.252.167]) by softhome.net with esmtp; Sat, 06 Oct 2001 13:31:50 -0600 Date: Sat, 6 Oct 2001 14:35:18 -0500 From: Nathan Rogers To: metakit@equi4.com Subject: Re: [Metakit] Why is View.Search() not working? Message-Id: <20011006143518.4445d80e.nrogers@softhome.net> In-Reply-To: References: X-Mailer: Sylpheed version 0.6.2 (GTK+ 1.2.10; i386-debian-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It may not be terribly intuitive, but Search is doing the right thing. Also, the behavior you noticed is documented. Suppose you want to insert a row if the key doesn't exist. If Search returned -1 you would have to binary search again to know where the row should be inserted. So having Search defined the way it is speeds up this common situation. Before you can use the value returned by Search, you need to check the row to see if it really matches or not. I suppose it would be nice for Search to check this for you. Maybe it could take an optional bool* to return this information in. Something like this should work. int c4_View::Search(const c4_RowRef& crit_, bool *found_=0) const { int l = -1, u = GetSize(); while (l + 1 != u) { const int m = (l + u) >> 1; if (_seq->Compare(m, &crit_) < 0) l = m; else u = m; } if(found_ != NULL) { if(u < GetSize() && _seq->Compare(u, &crit_) == 0) *found_ = true; else *found_ = false; } } return u; } Nathan Rogers From tciuro@yahoo.com Sat Oct 6 16:09:10 2001 Received: from web20602.mail.yahoo.com (web20602.mail.yahoo.com [216.136.226.158]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA18117 for ; Sat, 6 Oct 2001 16:09:10 -0500 Message-ID: <20011006210454.95512.qmail@web20602.mail.yahoo.com> Received: from [63.202.175.206] by web20602.mail.yahoo.com via HTTP; Sat, 06 Oct 2001 14:04:54 PDT Date: Sat, 6 Oct 2001 14:04:54 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Why is View.Search() not working? To: Nathan Rogers , metakit@equi4.com In-Reply-To: <20011006143518.4445d80e.nrogers@softhome.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Now it makes more sense... and it's indeed logical the way it works. But like you suggested, perhaps it would be more complete if Search() also returned whether a match was found. Thanks a lot! Best regards, -- Tito __________________________________________________ Do You Yahoo!? NEW from Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. http://geocities.yahoo.com/ps/info1 From jrichards@starband.net Wed Oct 10 19:11:20 2001 Received: from c0mailgw13.prontomail.com (mailgw.prontomail.com [216.163.180.10]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA05301 for ; Wed, 10 Oct 2001 19:11:20 -0500 Received: from C6SERVICE11.prontomail.com (216.163.178.10) by c0mailgw13.prontomail.com (NPlex 5.5.029) id 3BAF80D5003B1A99 for metakit@equi4.com; Wed, 10 Oct 2001 16:59:28 -0700 Received: from gateway (148.75.36.102) by C6SERVICE11.prontomail.com (NPlex 5.5.029) id 3BC2C77E000098A3 for metakit@equi4.com; Wed, 10 Oct 2001 16:59:28 -0700 From: "Jim Richards" To: Date: Wed, 10 Oct 2001 19:56:18 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Subject: [Metakit] Thank You Gordon McMillan!!! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been working with MetaKit for Python for approx. three weeks now. It was suggested that I use it for analyzing my firewall data. Well today I hit a milestone and felt it appropriate to let you and others know that MetaKit is real. I imported 238Mb of data which consisted of 2,315,000 packets or rows of data. The import took approx 45 minutes and the analysis which consists of various counts and groupings took about 15 minutes. I am impressed with MetaKits ease of use and would suggest that more Python programmers use it for similar tasks. Thank you for your effort and I hope to use MetaKit for other projects as well. It truly is a solid product. Jim Richards Unix / NT Administrator Opticom, Inc. 200 Brickstone Square Andover, MA 01810 From gmcm@hypernet.com Wed Oct 10 21:18:03 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA08545 for ; Wed, 10 Oct 2001 21:18:03 -0500 Received: from me (204.176.40.47) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Wed, 10 Oct 2001 22:07:38 -0400 From: "Gordon McMillan" To: "Jim Richards" Date: Wed, 10 Oct 2001 22:13:01 -0400 Subject: Re: [Metakit] Thank You Gordon McMillan!!! Reply-to: gmcm@hypernet.com CC: Message-ID: <3BC4C7ED.26669.83E8E773@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jim, If you find the Python bindings for MetaKit to be pleasant to work with, I am very, very grateful. However, most of your praise should be directed to Jean-Claude Wippler who is the sole author (and genius) behind MetaKit. (Nearly every project I do these days involves MetaKit somehow - a whole bunch of stuff I used to think of as drudgery is now fun.) - Gordon > I have been working with MetaKit for Python for approx. three > weeks now. It was suggested that I use it for analyzing my > firewall data. Well today I hit a milestone and felt it > appropriate to let you and others know that MetaKit is real. I > imported 238Mb of data which consisted of 2,315,000 packets or > rows of data. The import took approx 45 minutes and the analysis > which consists of various counts and groupings took about 15 > minutes. I am impressed with MetaKits ease of use and would > suggest that more Python programmers use it for similar tasks. > > Thank you for your effort and I hope to use MetaKit for other > projects as well. It truly is a solid product. > > Jim Richards > Unix / NT Administrator > Opticom, Inc. > 200 Brickstone Square > Andover, MA 01810 > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit - Gordon From jcw@equi4.com Fri Oct 12 09:21:45 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA05482 for ; Fri, 12 Oct 2001 09:21:45 -0500 Received: from takkie.equi4.local ([195.121.232.221]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL3IEW00.0IG for ; Fri, 12 Oct 2001 15:49:44 +0200 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9CDlrl32485 for ; Fri, 12 Oct 2001 15:48:10 +0200 From: Jean-Claude Wippler To: metakit Date: Fri, 12 Oct 2001 15:49:13 +0200 Message-Id: <20011012134913.28579@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Custom-extended TclKit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There's a README and tar preview on the web, containing adjusted versions of sdx (called nsdx for now), and tclkit (Windows and Linux) - more or less the same as the 8.4-25 distribution with upgraded Tcl/Tk/MK, and a small patch: http://www.equi4.com/previews/nsdx.README http://www.equi4.com/previews/nsdx.tar.gz What these provide, is the option to create a single-file application based on TclKit, and including scripts, libraries, and extensions. I'm making this available for those who need this *now* or want to have some advance info on what is coming. It's rough, you may have to dive in and try out things to make it fly - but it should work. Comments welcome. Patches and improvements too! -jcw From jcw@equi4.com Sun Oct 14 08:54:21 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA03327 for ; Sun, 14 Oct 2001 08:54:21 -0500 Received: from takkie.equi4.local ([195.121.233.233]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL77QD00.OO2 for ; Sun, 14 Oct 2001 15:49:25 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EDlFB01009 for ; Sun, 14 Oct 2001 15:47:15 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Copying/Saving Views Date: Sun, 14 Oct 2001 15:47:54 +0200 Message-Id: <20011014134754.30060@mail.planet.nl> In-Reply-To: <3BBB1CD8.17CFA553@maich.gr> References: <3BBB1CD8.17CFA553@maich.gr> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas Boretos wrote: >I wonder how one could simply copy a view or a slice of a view from a >main data file, and save it to file in Tclkit. I want basically to keep >track of record(s) deleted from a view and stick them in a deleted.dat >file, which maybe opened for inspection in the future... > >set main_view [mk::view open main.view] > >set deleted_view [mk::view open deleted.view] > >Now what? The Tcl objectified interface is not quite as complete as the C++/Python ones. My suggestion would be to try one of the following instead: - Open read-only or nocommit, then delete what you don't want, and "serialize" the remainder, using something like: set fd [open saved.mk w] mk::file save db $fd close $fd This produces a MK datafile with just what is there at the time of saving, and by not committing these changes back in the original datafile, you are essentially create a "filtered" backup. - Open a fresh db, define the structure properly, and then copy views from the original into the new datafile. This is not possible with the base Mk4tcl interface, but with the objectified one, you might get away with it: just copy the rows you want to save. IIRC, the subviews will automatically get copied along with it *if* the strcuture of the new db has the proper structure. >Currently looping through records and it's not much fun, esp. when a >datafile contains~20 views... I don't understand... it's code running *for* you, i.e. the computer is doing the work, right? Is it slow? What am I missing? -jcw From jcw@equi4.com Sun Oct 14 08:54:42 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA03332; Sun, 14 Oct 2001 08:54:41 -0500 Received: from takkie.equi4.local ([195.121.233.233]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL77QC00.QFH; Sun, 14 Oct 2001 15:49:24 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EDlEB01006; Sun, 14 Oct 2001 15:47:14 +0200 From: Jean-Claude Wippler To: announce , metakit Date: Sun, 14 Oct 2001 15:36:29 +0200 Message-Id: <20011014133629.18740@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.1 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There are new distributions of MetaKit and TclKit on the website. These are "maintenance releases" - wrapping up recent fixes and minor changes: http://www.equi4.com/metakit/ http://www.equi4.com/tclkit/ Builds for Linux and Windows have also been uploaded, see: http://www.equi4.com/pub/mk/ http://www.equi4.com/pub/tk/ These include static libs, and the extensions Mk4py and Mk4tcl. The CHANGELOG can be found at http://www.equi4.com/metakit/CHANGES Btw, I am looking for people to see if we can come up with an API and wrapper for Perl - and would appreciate any suggestions or references! -jcw From jcw@equi4.com Sun Oct 14 08:54:43 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA03335 for ; Sun, 14 Oct 2001 08:54:42 -0500 Received: from takkie.equi4.local ([195.121.233.233]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL77QD00.JI7 for ; Sun, 14 Oct 2001 15:49:25 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EDlGB01012 for ; Sun, 14 Oct 2001 15:47:16 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] cvs hash performance Date: Sun, 14 Oct 2001 15:49:56 +0200 Message-Id: <20011014134956.3678@mail.planet.nl> In-Reply-To: <20011006120021.290da3de.nrogers@softhome.net> References: <20011006120021.290da3de.nrogers@softhome.net> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nathan Rogers wrote: >I've been trying out the CVS version of Metakit lately, and I noticed that >the time it takes to append a row to a hashed view has increased >dramatically since 2.4. With a few thousand rows, inserts take about 100 >times longer than they used to. Adding blocking to the data and/or map >views doesn't seem to help. Any ideas on what might be causing this or >what can be done about it? Hm, it loks like I've not really solved the issue right. The hashing logic is probably wrong, causing the code to loop looking for a free slot perhaps. >Also, I'm having trouble getting Ordered views to work in C++. I've been >trying code like this. > >c4_Storage db; >c4_IntProp pKey ("key"); >c4_View vw = db.GetAs("test[key:I]"); >c4_View ov = vw.Ordered(); >ov.Add(pKey[3]); > >But it always seems to crash. The same thing seems to work fine in python >and tcl. I will look into this (debug mode does not provide any clues, I presume?). Thanks for the sample code. -jcw From mlh@idi.ntnu.no Sun Oct 14 09:59:54 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA04358; Sun, 14 Oct 2001 09:59:53 -0500 Received: from vier.idi.ntnu.no (IDENT:22955@vier.idi.ntnu.no [129.241.103.4]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with ESMTP id QAA01533; Sun, 14 Oct 2001 16:55:06 +0200 (MEST) Date: Sun, 14 Oct 2001 16:55:06 +0200 (MEST) From: Magnus Lie Hetland To: Jean-Claude Wippler cc: metakit Subject: Re: [Metakit] MetaKit 2.4.1 In-Reply-To: <20011014133629.18740@mail.planet.nl> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Sun, 14 Oct 2001, Jean-Claude Wippler wrote: > Btw, I am looking for people to see if we can come up with an API and > wrapper for Perl - and would appreciate any suggestions or references! A thought: How about jni-wrapper for Java? I know I would use it... Not sure if I could write it, though :) > -jcw -- Magnus Lie Hetland http://www.hetland.org "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick From jcw@equi4.com Sun Oct 14 14:22:27 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA08149 for ; Sun, 14 Oct 2001 14:22:27 -0500 Received: from takkie.equi4.local ([195.121.232.127]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL7MWG01.1UJ for ; Sun, 14 Oct 2001 21:17:04 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EJEtB02633 for ; Sun, 14 Oct 2001 21:14:55 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] MetaKit 2.4.1 Date: Sun, 14 Oct 2001 21:11:51 +0200 Message-Id: <20011014191151.1131@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Magnus Lie Hetland wrote: >A thought: How about jni-wrapper for Java? I know I would use it... >Not sure if I could write it, though :) Yes, I would be interested in that too (and there have been requests in the past, though less recent). I would need help with this as well, to find out how to set things up (development-wise), and to have some feedback on usability and testing. In itself, MK's C++ design ought to map relatively straight to Java, I would hope. -jcw From mpwilson@earthlink.net Sun Oct 14 15:01:06 2001 Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA08812 for ; Sun, 14 Oct 2001 15:01:05 -0500 Received: from BRIGADOON (1Cust24.VR1.NYC4.broadband.uu.net [63.13.131.24]) by pintail.mail.pas.earthlink.net (EL-8_9_3_3/8.9.3) with ESMTP id MAA07085; Sun, 14 Oct 2001 12:56:12 -0700 (PDT) Date: Sun, 14 Oct 2001 16:05:20 -0400 From: "Michael P. Wilson" X-Mailer: The Bat! (v1.53bis) Personal Reply-To: "Michael P. Wilson" X-Priority: 3 (Normal) Message-ID: <121656456717.20011014160520@earthlink.net> To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] VLDB performance? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hya, I've been lurking around looking for a relatively low-level library for maintaining an extremely large (ok, to me anyway) database of fairly large objects (about 3k of large-ish text columns plus blobs of up to 5 meg.) The semantic needs are very thin. I only ever need append and read-only access, including fairly basic selection and filtering across the set. Are there any immediate "gotchas" in MetaKit that I should know about before giving it a shot? I don't need complex data types, advanced 'select' management, record updating, etc. However, all fields are variable length including the blob field. Oh, btw, when I say "extremely large" I mean something in excess of a 1.2Terabytes (and growing at about 100-200 meg a day.) SQL databases are just too heavyweight. I'd write it myself if I can't find something that readily serves the need. Any comments? (no, not from this crowd ;) Thanks - Mike P.S. Regardless of whether or not metakit fits my particular, fairly peculiar needs, I've got to say it seems like a serious butt kicker. From jcw@equi4.com Sun Oct 14 15:33:05 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA09286 for ; Sun, 14 Oct 2001 15:33:04 -0500 Received: from takkie.equi4.local ([195.121.232.88]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL7Q6A01.H8A for ; Sun, 14 Oct 2001 22:27:46 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EKPdB03064 for ; Sun, 14 Oct 2001 22:25:39 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] VLDB performance? Date: Sun, 14 Oct 2001 22:28:43 +0200 Message-Id: <20011014202843.17133@mail.planet.nl> In-Reply-To: <121656456717.20011014160520@earthlink.net> References: <121656456717.20011014160520@earthlink.net> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Michael P. Wilson wrote: >I've been lurking around looking for a relatively low-level library for >maintaining an extremely large (ok, to me anyway) database of fairly >large objects (about 3k of large-ish text columns plus blobs of up to 5 >meg.) > >The semantic needs are very thin. I only ever need append and >read-only access, including fairly basic selection and filtering across >the set. > >Are there any immediate "gotchas" in MetaKit that I should know about >before giving it a shot? I don't need complex data types, advanced >'select' management, record updating, etc. However, all fields are >variable length including the blob field. > >Oh, btw, when I say "extremely large" I mean something in excess of a >1.2Terabytes (and growing at about 100-200 meg a day.) Most things you say are within what I'd consider tough but still feasible. Large objects do not in themselves cause much trouble, because they are stored individually. Access to a row when the large object field is not used, is as fast as if the large object had not been present, due to the column-wise data organization. But the 1.2 Tb kills things. MK is based on memory mapped files and (currently) requires that the entire file be mapped (R/O). On a 32-bit machine, don't even *think* about datasets over 2 Gb. There is only one way to make this fly: use a 64-bit architecture which supports mapping the entire DBMS. I'd be interested to see where bottlenecks turn up, if any, and then try to resolve it. There are numerous tricks to apply, and the view mechanism on top of MK allows for a wide range of "special-purpose cleverness". Read-only data in itself makes the issue feasible IMO. The "append-only" mode you need is trickier, because - again due to the column-wise design - appends, even of a single item, normally have the effect of copying an entire column. One solution is to segment the data, and then recombine it virtually - which the various custom views can IMO be made to do just fine. But let's not forget the reality that 1 Tb is way, way, over what MK has been used for today - and ever considered for AFAIK. It will take deep knowledge of the access/use/modification patterns and of the MK design to have any chance at success with this. Just a thought: why not save the data serially to file(s), and have MK *manage* it, i.e. store offsets and some key navigational data, for example? Transaction safety can surely be obtained, and the amount of data residing in MK could go down dramatically. Again, keep in mind that custom views can hide this physical design choice. -jcw From ctrueman@dial.pipex.com Mon Oct 15 02:07:55 2001 Received: from gadolinium.btinternet.com (gadolinium.btinternet.com [194.73.73.111]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA17813 for ; Mon, 15 Oct 2001 02:07:55 -0500 Received: from [213.123.154.112] (helo=iclassfactory) by gadolinium.btinternet.com with smtp (Exim 3.22 #6) id 15t1mE-0004dC-00 for metakit@equi4.com; Mon, 15 Oct 2001 08:03:06 +0100 Reply-To: From: "Chris Trueman" To: Date: Mon, 15 Oct 2001 08:00:55 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Subject: [Metakit] Multi-process access to MetaKit repository Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've only glanced at the introduction and some of the sample code and my question is: Can MetaKit be used as a central repository where multiple programs read and write data from/to it? I guess my concerns are about the concurrency features of the engine. Any help would be appreciated. Thanks. Chris. From rcohen@dial.oleane.com Mon Oct 15 02:15:33 2001 Received: from apicra.wanadoo.fr (smtp-rt-3.wanadoo.fr [193.252.19.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA17966 for ; Mon, 15 Oct 2001 02:15:32 -0500 Received: from mel-rta8.wanadoo.fr (193.252.19.79) by apicra.wanadoo.fr; 15 Oct 2001 09:10:01 +0200 Received: from dial.oleane.com (80.11.89.224) by mel-rta8.wanadoo.fr; 15 Oct 2001 09:09:56 +0200 Message-ID: <3BCA99F0.80A92AA0@dial.oleane.com> Date: Mon, 15 Oct 2001 09:10:24 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: en-US,fr,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Subject: Re: [Metakit] MetaKit 2.4.1 References: <20011014191151.1131@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been wrapping several little apis from C to Java. It it really easy with C functions but not so easy with C++ classes (never tried!). I am not a java specialist, but if there is nobody else, I can help. I will not have much time to write or test for the moment but can give examples and information. -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From ctrueman@dial.pipex.com Mon Oct 15 06:30:29 2001 Received: from gadolinium.btinternet.com (gadolinium.btinternet.com [194.73.73.111]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA27543 for ; Mon, 15 Oct 2001 06:30:28 -0500 Received: from [213.123.154.112] (helo=iclassfactory) by gadolinium.btinternet.com with smtp (Exim 3.22 #6) id 15t5sI-0001xU-00 for metakit@equi4.com; Mon, 15 Oct 2001 12:25:38 +0100 Reply-To: From: "Chris Trueman" To: Date: Mon, 15 Oct 2001 12:23:28 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Subject: [Metakit] Performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I was trialling the performance of the storage engine using code similiar to: // Declare a bunch of string and int properties c4_StringProp name("Name"); c4_IntProp age("Age"); c4_View test = db.GetAt("test[Name:S, Age:I]"); for(int i = 0; i < 1000000; ++i) { c4_Row row; name(row) = age(row) = rnd(65); test.Add(row); db.Commit(); } With a view that contains more properties that what I gave abouve (around 22), the performance is not that great. It writes about 3 or 4 rows per second. I assume that it my lack of knowledge of programming against MetaKit that is causing this poor rate of insertion. Any ideas? Chris. From jcw@equi4.com Mon Oct 15 08:16:41 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA00831 for ; Mon, 15 Oct 2001 08:16:41 -0500 Received: from takkie.equi4.local ([195.121.234.230]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL90NN00.DUE for ; Mon, 15 Oct 2001 15:11:47 +0200 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9FD9jB08082 for ; Mon, 15 Oct 2001 15:09:45 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Performance Date: Mon, 15 Oct 2001 15:11:37 +0200 Message-Id: <20011015131137.17738@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Chris Trueman wrote: >for(int i = 0; i < 1000000; ++i) >{ > c4_Row row; ^^^^^^^^^^ This one will slow you down a tad, move it up - but you won't see the effect until you first fix the big bottleneck below. Row constructors/ destructors are not top-speed, it's best to reuse them if you can. > name(row) = Strings are considerably slower than ints/floats, since they are variable-size. Clearly there is not much you can do, if you need strings... > age(row) = rnd(65); > > test.Add(row); This is slow, preallocate before entering the loop, then use "test[i] = row;". Or at least resize in chunks, instead of growing the view on each iteration (just as you would with C arrays in general). > db.Commit(); ^^^^^^^^^^^^ That kills performance, it's inside the loop. You need to move it down. Or at least call it every N cycles. >} Have a look at the MetaKit wiki FAQs http://www.equi4.com/metakit/ wiki.cgi for more info. It's not abundant, but it's there. See also: http://www.equi4.com/metakit/api/tips.html#faster Now that you have a baseline, please feel free to report how you progress :) -jcw From claird@starbase.neosoft.com Mon Oct 15 15:19:30 2001 Received: from mx1.airmail.net (mx1.airmail.net [209.196.77.98]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29153 for ; Mon, 15 Oct 2001 15:19:30 -0500 Received: from mail3.iadfw.net ([209.196.123.3]) by mx1.airmail.net with smtp (Exim 3.16 #10) id 15tE8E-0004uB-00; Mon, 15 Oct 2001 15:14:39 -0500 Received: from starbase.neosoft.com from [206.109.1.32] by mail3.iadfw.net (/\##/\ Smail3.1.30.16 #30.61) with esmtp for sender: id ; Mon, 15 Oct 2001 15:17:19 -0500 (CDT) Received: (from claird@localhost) by starbase.neosoft.com (8.9.3/8.9.3) id PAA37455; Mon, 15 Oct 2001 15:19:31 -0500 (CDT) (envelope-from claird) Date: Mon, 15 Oct 2001 15:19:31 -0500 (CDT) From: Cameron Laird Message-Id: <200110152019.PAA37455@starbase.neosoft.com> To: ctrueman@dial.pipex.com, metakit@equi4.com Subject: Re: [Metakit] Multi-process access to MetaKit repository In-Reply-To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > From metakit-admin@equi4.com Mon Oct 15 02:12:21 2001 > Reply-To: > From: "Chris Trueman" > To: > . > . > . > I've only glanced at the introduction and some of the sample code and my > question is: > Can MetaKit be used as a central repository where multiple programs read and > write data from/to it? > I guess my concerns are about the concurrency features of the engine. > . > . > . Yes and no. Officially, MetaKit "does not offer true concurrent access". That's what the home page says. HOWEVER, plenty of people use MetaKit in multi-user applications. A lovely mechanism for this is Tequila, part of the MetaKit family [http://www.equi4.com/metakit/wiki.cgi/19.html]. From tciuro@mac.com Mon Oct 15 16:55:16 2001 Received: from smtp-out.mac.com ([204.179.120.85]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA02440 for ; Mon, 15 Oct 2001 16:55:16 -0500 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtp-out.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9FLaWJ20454 for ; Mon, 15 Oct 2001 14:36:32 -0700 (PDT) Received: from asmtp01.mac.com ([10.13.10.65]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GL9OKY00.TZR for ; Mon, 15 Oct 2001 14:48:34 -0700 Received: from ciurti ([17.202.22.175]) by asmtp01.mac.com (Netscape Messaging Server 4.15 asmtp01 Jun 21 2001 23:53:48) with ESMTP id GL9OKX00.TIT for ; Mon, 15 Oct 2001 14:48:33 -0700 Date: Mon, 15 Oct 2001 14:48:33 -0700 Mime-Version: 1.0 (Apple Message framework v500) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Tito Ciuro To: metakit@equi4.com Content-Transfer-Encoding: 7bit Message-Id: <612F38BD-C1B6-11D5-8F25-00039310A0E8@mac.com> X-Mailer: Apple Mail (2.500) Subject: [Metakit] How Can I Add/Remove Views in MetaKit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, It seems that the documentation is a bit out of date. I see a "Store" method in c4_Storage, which "[...]Attach a view using specified name in this storage object.", but the the mk4.h file states that it has been dropped. So the question is: is there a way to add and remove views on demand? BTW, I'm using MetaKit/C++. Thanks x 1000, -- Tito From ctrueman@dial.pipex.com Mon Oct 15 17:03:48 2001 Received: from carbon.btinternet.com (carbon.btinternet.com [194.73.73.92]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA02897 for ; Mon, 15 Oct 2001 17:03:47 -0500 Received: from [213.123.154.10] (helo=iclassfactory) by carbon.btinternet.com with smtp (Exim 3.22 #6) id 15tFl8-0006qy-00; Mon, 15 Oct 2001 22:58:55 +0100 Reply-To: From: "Chris Trueman" To: "Tito Ciuro" , Subject: RE: [Metakit] How Can I Add/Remove Views in MetaKit? Date: Mon, 15 Oct 2001 22:56:43 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-Reply-To: <612F38BD-C1B6-11D5-8F25-00039310A0E8@mac.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: GetAs instead? Chris. > It seems that the documentation is a bit out of date. I see a "Store" > method in c4_Storage, which "[...]Attach a view using specified name in > this storage object.", but the the mk4.h file states that it has been > dropped. > > So the question is: is there a way to add and remove views on demand? From tciuro@mac.com Mon Oct 15 17:08:15 2001 Received: from smtp-out.mac.com ([204.179.120.85]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA03166 for ; Mon, 15 Oct 2001 17:08:15 -0500 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtp-out.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9FLnuJ22279 for ; Mon, 15 Oct 2001 14:49:56 -0700 (PDT) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GL9P7B00.90K for ; Mon, 15 Oct 2001 15:01:59 -0700 Received: from ciurti ([17.202.22.175]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GL9P7A00.18J; Mon, 15 Oct 2001 15:01:58 -0700 Date: Mon, 15 Oct 2001 15:01:58 -0700 Subject: Re: [Metakit] How Can I Add/Remove Views in MetaKit? Content-Type: multipart/alternative; boundary=Apple-Mail-4--893685258 Mime-Version: 1.0 (Apple Message framework v500) Cc: To: From: Tito Ciuro In-Reply-To: Message-Id: <40F217A6-C1B8-11D5-8F25-00039310A0E8@mac.com> X-Mailer: Apple Mail (2.500) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-4--893685258 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hi Chris, If I understand correctly, GetAs() creates a "virtual" view which becomes "physical" after Commit(), correct? Once the view has been stored in the data file, how do I remove it? Thanks, -- Tito On Monday, October 15, 2001, at 02:56 PM, Chris Trueman wrote: > GetAs instead? > > > Chris. > >> It seems that the documentation is a bit out of date. I see a "Store" >> method in c4_Storage, which "[...]Attach a view using specified name in >> this storage object.", but the the mk4.h file states that it has been >> dropped. >> >> So the question is: is there a way to add and remove views on demand? > > ____________________________________________ "I'm desperately trying to figure out why kamikaze pilots wore helmets." Dave Edison --Apple-Mail-4--893685258 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hi Chris, If I understand correctly, GetAs() creates a "virtual" view which becomes "physical" after Commit(), correct? Once the view has been stored in the data file, how do I remove it? Thanks, -- Tito On Monday, October 15, 2001, at 02:56 PM, Chris Trueman wrote: GetAs instead? Chris. It seems that the documentation is a bit out of date. I see a "Store" method in c4_Storage, which "[...]Attach a view using specified name in this storage object.", but the the mk4.h file states that it has been dropped. So the question is: is there a way to add and remove views on demand? Lucida Grande ____________________________________________Lucida Grande 0000,0000,FFFF"I'm desperately trying to figure out why kamikaze pilots wore helmets." 0000,6666,3333 Dave Edison --Apple-Mail-4--893685258-- From ctrueman@dial.pipex.com Mon Oct 15 17:15:29 2001 Received: from rhenium (rhenium.btinternet.com [194.73.73.93]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA03637 for ; Mon, 15 Oct 2001 17:15:24 -0500 Received: from [213.123.154.10] (helo=iclassfactory) by rhenium with smtp (Exim 3.22 #6) id 15tFwI-0000Ok-00; Mon, 15 Oct 2001 23:10:26 +0100 Reply-To: From: "Chris Trueman" To: "Tito Ciuro" Cc: Subject: RE: [Metakit] How Can I Add/Remove Views in MetaKit? Date: Mon, 15 Oct 2001 23:08:13 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-Reply-To: <40F217A6-C1B8-11D5-8F25-00039310A0E8@mac.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What about setting the view size to 0 and committing the results? Chris. If I understand correctly, GetAs() creates a "virtual" view which becomes "physical" after Commit(), correct? Once the view has been stored in the data file, how do I remove it? From tciuro@mac.com Mon Oct 15 17:27:09 2001 Received: from smtp-out.mac.com ([204.179.120.85]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA04310 for ; Mon, 15 Oct 2001 17:27:08 -0500 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtp-out.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9FM97J25126 for ; Mon, 15 Oct 2001 15:09:07 -0700 (PDT) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GL9Q3G00.C43 for ; Mon, 15 Oct 2001 15:21:16 -0700 Received: from ciurti ([17.202.22.175]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GL9Q3G00.TA7; Mon, 15 Oct 2001 15:21:16 -0700 Date: Mon, 15 Oct 2001 15:21:15 -0700 Subject: Re: [Metakit] How Can I Add/Remove Views in MetaKit? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v500) Cc: To: From: Tito Ciuro In-Reply-To: Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.500) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The doc states: "[...]void SetSize (int newSize_, int growBy_ =-1); Since views act like dynamic arrays, you can quickly change their size. Increasing the size will append rows with zero/empty values, while decreasing it will delete the last rows. The growBy_ parameter is currently unused." So it seems that setting the size of a view to zero will remove all rows, leaving the view there. I'd like to remove the view + rows, the whole thing. Thanks, -- Tito On Monday, October 15, 2001, at 03:08 PM, Chris Trueman wrote: > What about setting the view size to 0 and committing the results? > > > Chris. > > If I understand correctly, GetAs() creates a "virtual" view which > becomes > "physical" after Commit(), correct? Once the view has been stored in the > data file, how do I remove it? From jcw@equi4.com Thu Oct 18 05:43:12 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA07956 for ; Thu, 18 Oct 2001 05:43:11 -0500 Received: from takkie.equi4.local ([195.121.233.208]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLEDJK00.EI8 for ; Thu, 18 Oct 2001 12:38:08 +0200 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9IAaTB28821 for ; Thu, 18 Oct 2001 12:36:30 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] cvs hash performance Date: Thu, 18 Oct 2001 12:39:05 +0200 Message-Id: <20011018103905.25823@mail.planet.nl> In-Reply-To: <20011006120021.290da3de.nrogers@softhome.net> References: <20011006120021.290da3de.nrogers@softhome.net> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nathan Rogers wrote: >I've been trying out the CVS version of Metakit lately, and I noticed that >the time it takes to append a row to a hashed view has increased >dramatically since 2.4. With a few thousand rows, inserts take about 100 >times longer than they used to. Adding blocking to the data and/or map >views doesn't seem to help. Any ideas on what might be causing this or >what can be done about it? This was a glaring bug, introduced recently and causing the hash map to be reconstructed on every insertion... >Also, I'm having trouble getting Ordered views to work in C++. I've been >trying code like this. > >c4_Storage db; >c4_IntProp pKey ("key"); >c4_View vw = db.GetAs("test[key:I]"); >c4_View ov = vw.Ordered(); >ov.Add(pKey[3]); > >But it always seems to crash. The same thing seems to work fine in python >and tcl. This was a problem in interaction between inserts (which are positional) and ordered views (which ignore position, and implicitly derive one). The Add implementation was changed to use InsertAt instead of SetAtGrow (which does not work with ordered views). The code in CVS should be ok now for both issues. Thanks for reporting both problems, and for the sample code which helped figure out this second problem. -jcw From jcw@equi4.com Thu Oct 18 07:15:14 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA13210 for ; Thu, 18 Oct 2001 07:15:13 -0500 Received: from takkie.equi4.local ([195.121.234.220]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLEHSV01.0MB for ; Thu, 18 Oct 2001 14:10:07 +0200 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9IC8TB29245 for ; Thu, 18 Oct 2001 14:08:29 +0200 From: Jean-Claude Wippler To: metakit Date: Thu, 18 Oct 2001 14:03:16 +0200 Message-Id: <20011018120316.2001@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Mapped view performance overview Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There's now a page on the MetaKit wiki, exploring the various aspects of performance w.r.t. adding, commits, and searching. This is the output of a Tcl script which is in the MK 2.4.x distributions - annotated to explain some of the behavior: http://www.equi4.com/metakit/wiki.cgi/129.html As with all performance issues - please add salt to suit your taste... :) -jcw From jcw@equi4.com Thu Oct 18 07:50:29 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA15539 for ; Thu, 18 Oct 2001 07:50:28 -0500 Received: from takkie.equi4.local ([195.121.234.111]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLEJER00.GCC for ; Thu, 18 Oct 2001 14:44:51 +0200 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9ICh0B29438 for ; Thu, 18 Oct 2001 14:43:02 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] How Can I Add/Remove Views in MetaKit? Date: Thu, 18 Oct 2001 14:45:21 +0200 Message-Id: <20011018124521.7856@mail.planet.nl> In-Reply-To: <612F38BD-C1B6-11D5-8F25-00039310A0E8@mac.com> References: <612F38BD-C1B6-11D5-8F25-00039310A0E8@mac.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito Ciuro wrote: >So the question is: is there a way to add and remove views on demand? Yes, use GetAs with a view name but no structure, i.e. to define v1: c4_View v = storage.GetAs("v1[...]"); To remove it (just ignore the result value from this call): storage.GetAs("v1"); You may still see the view as temporary property, but after commit it's gone. The name "GetAs" is a bit misleading, now that "Store" is gone. One reason for getting rid of Store, was to encourage you to set up the view with GetAs *before* filling it. You can still do it after the fact, but it's less efficient: c4_View vdata; <... fill vdata here ...> c4_View v = storage.GetAs("v1[...]"); v.InsertAt(0, vdata); This will do a full copy (and consume twice as much memory). -jcw From rjn103s@mail.mgr3.k12.mo.us Fri Oct 19 14:07:31 2001 Received: from mail.mgr3.k12.mo.us (mail.mgr3.k12.mo.us [204.184.227.130]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA15167 for ; Fri, 19 Oct 2001 14:07:30 -0500 Received: (from Unknown UID 65534@localhost) by mail.mgr3.k12.mo.us (8.11.6/8.11.4) id f9JIw4R05141; Fri, 19 Oct 2001 13:58:04 -0500 (CDT) (envelope-from rjn103s@mail.mgr3.k12.mo.us) Date: Fri, 19 Oct 2001 13:58:04 -0500 (CDT) Message-Id: <200110191858.f9JIw4R05141@mail.mgr3.k12.mo.us> X-Authentication-Warning: mail.mgr3.k12.mo.us: Unknown UID 65534 set sender to rjn103s@mail.mgr3.k12.mo.us using -f From: "Richard Nelson" To: metakit@equi4.com Reply-To: rjn103s@mail.mgr3.k12.mo.us X-Mailer: NeoMail 1.25 X-IPAddress: 172.16.0.5 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] FreeBSD4.4 complie help needed for MetaKit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, using Tcl-8.3.3 / Tk-8.3 ============Make tcl output======================== make tcl /bin/sh ./libtool --mode=compile c++ -c -O2 -fomit-frame-pointer -I../unix/../include -I../unix/../../include -I/usr/local/lib/tcl8.3/../generic -DUSE_TCL_STUBS ../unix/../tcl/mk4tcl.cpp rm -f .libs/mk4tcl.lo c++ -c -O2 -fomit-frame-pointer -I../unix/../include -I../unix/../../include -I/usr/local/lib/tcl8.3/../generic -DUSE_TCL_STUBS ../unix/../tcl/mk4tcl.cpp -fPIC -DPIC -o .libs/mk4tcl.lo In file included from ../unix/../tcl/mk4tcl.cpp:5: ../unix/../tcl/mk4tcl.h:8: tcl.h: No such file or directory *** Error code 1 Any ideas? Many thanks! -- Panther - Webmail Nelson MG From stug@pacbell.net Sun Oct 21 15:56:55 2001 Received: from snfc21.pbi.net (mta6.snfc21.pbi.net [206.13.28.240]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29080 for ; Sun, 21 Oct 2001 15:56:55 -0500 Received: from slk ([207.105.204.47]) by mta6.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with SMTP id <0GLK00GMVPY381@mta6.snfc21.pbi.net> for metakit@equi4.com; Sun, 21 Oct 2001 13:51:39 -0700 (PDT) Date: Sun, 21 Oct 2001 13:57:54 -0700 From: Stu To: metakit@equi4.com Message-id: <000901c15a73$0e525460$0200a8c0@pacbell.net> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-Mailer: Microsoft Outlook Express 6.00.2600.0000 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7bit X-Priority: 3 X-MSMail-priority: Normal Subject: [Metakit] How do I do a static build with Metakit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I know this isn't Metakit specific but I can't seem to get a build that doesn't require the metakit dll. I am also linking statically to mfc. Thanks From OOO@vmts.ru Sun Oct 21 23:29:23 2001 Received: from mail2.vmts.ru (mail.vmts.ru [212.122.1.133]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA05869 for ; Sun, 21 Oct 2001 23:29:21 -0500 Received: by mail2.vmts with Internet Mail Service (5.5.2653.19) id ; Mon, 22 Oct 2001 15:19:12 +1100 Message-ID: <09DE0F07AB09D4119D390050DA394B84A7CE78@mail2.vmts> From: "Oleg O. Orlov" To: metakit@equi4.com Date: Mon, 22 Oct 2001 15:19:12 +1100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C15AB0.B3D095E0" Subject: [Metakit] more than 3 hash keys Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C15AB0.B3D095E0 Content-Type: text/plain; charset="koi8-r" what's wrong in following sources ???: create some properties: c4_StringProp pSourceIp ("source_ip"); c4_IntProp pSourcePort ("source_port"); c4_StringProp pDestIp ("dest_ip"); c4_IntProp pDestPort ("dest_port"); c4_IntProp pPkts ("pkts"); link it with storage c4_Storage storage ("connections.dat", true); create base view c4_View vNote = storage.GetAs( "vNote[source_ip:S,source_port:I,dest_ip:S,dest_port:I,pkts:I"); create view for hash c4_View hNote = storage.GetAs("hNote[_H:I,_R:I]"); create hash with 4 keys stored as special view c4_View hash = vNote.Hash(hNote,4); _____________________________________________________________ All works perfect up to 3 first key of base view An 4-th key causes infinite loop in program ... _________________ Oleg Orlov ------_=_NextPart_001_01C15AB0.B3D095E0 Content-Type: text/html; charset="koi8-r"
what's wrong in following sources ???:
 
    create some properties:
 
c4_StringProp pSourceIp ("source_ip");
c4_IntProp pSourcePort ("source_port");
c4_StringProp pDestIp ("dest_ip");    
c4_IntProp pDestPort ("dest_port");   
c4_IntProp pPkts ("pkts");            
 
    link it with storage
 
c4_Storage storage ("connections.dat", true);
 
    create base view
 
c4_View vNote = storage.GetAs(
    "vNote[source_ip:S,source_port:I,dest_ip:S,dest_port:I,pkts:I");
 
    create view for hash
 
c4_View hNote = storage.GetAs("hNote[_H:I,_R:I]");
 
    create hash with 4 keys stored as special view
 
c4_View hash = vNote.Hash(hNote,4);
 
_____________________________________________________________
 
 
All works perfect up to 3 first key of base view
An 4-th key causes infinite loop in program
 
 
...
 
 
_________________
Oleg Orlov
------_=_NextPart_001_01C15AB0.B3D095E0-- From rjn103s@mail.mgr3.k12.mo.us Tue Oct 23 09:19:51 2001 Received: from mail.mgr3.k12.mo.us (mail.mgr3.k12.mo.us [204.184.227.130]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA24331 for ; Tue, 23 Oct 2001 09:19:50 -0500 Received: (from Unknown UID 65534@localhost) by mail.mgr3.k12.mo.us (8.11.6/8.11.4) id f9NE9gD82886; Tue, 23 Oct 2001 09:09:42 -0500 (CDT) (envelope-from rjn103s@mail.mgr3.k12.mo.us) Date: Tue, 23 Oct 2001 09:09:42 -0500 (CDT) Message-Id: <200110231409.f9NE9gD82886@mail.mgr3.k12.mo.us> X-Authentication-Warning: mail.mgr3.k12.mo.us: Unknown UID 65534 set sender to rjn103s@mail.mgr3.k12.mo.us using -f From: "Richard Nelson" To: metakit@equi4.com Reply-To: rjn103s@mail.mgr3.k12.mo.us X-Mailer: NeoMail 1.25 X-IPAddress: 172.16.0.1 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] FreeBSD now compiles but fails test-tcl (assistance needed) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, first let me thanks Dirk Meyer for his work in providing me a FreeBSD port for metakit-2.4.1 that compiles. I am testing so it can be added to the FreeBSD ports tree. The problem is that the port will compile but fails the test-tcl option. Below is, I hope, the critical output from this make option to provide insight: c++ -c -I./../unix/../include ./../unix/../tests/tbasics.cpp -o tbasics.o c++ -c -I./../unix/../include ./../unix/../tests/tcusto1.cpp -o tcusto1.o c++ -c -I./../unix/../include ./../unix/../tests/tcusto2.cpp -o tcusto2.o c++ -c -I./../unix/../include ./../unix/../tests/tdiffer.cpp -o tdiffer.o c++ -c -I./../unix/../include ./../unix/../tests/textend.cpp -o textend.o c++ -c -I./../unix/../include ./../unix/../tests/tformat.cpp -o tformat.o c++ -c -I./../unix/../include ./../unix/../tests/tlimits.cpp -o tlimits.o c++ -c -I./../unix/../include ./../unix/../tests/tmapped.cpp -o tmapped.o c++ -c -I./../unix/../include ./../unix/../tests/tnotify.cpp -o tnotify.o c++ -c -I./../unix/../include ./../unix/../tests/tresize.cpp -o tresize.o c++ -c -I./../unix/../include ./../unix/../tests/tstore1.cpp -o tstore1.o c++ -c -I./../unix/../include ./../unix/../tests/tstore2.cpp -o tstore2.o c++ -c -I./../unix/../include ./../unix/../tests/tstore3.cpp -o tstore3.o c++ -c -I./../unix/../include ./../unix/../tests/tstore4.cpp -o tstore4.o /bin/sh ./libtool --mode=link c++ -static -o regress regress.o tbasics.o tcusto1.o tcusto2.o tdiffer.o textend.o tformat.o tlimits.o tmapped.o tnotify.o tresize.o tstore1.o tstore2.o tstore3.o tstore4.o libmk4.a c++ -o regress regress.o tbasics.o tcusto1.o tcusto2.o tdiffer.o textend.o tformat.o tlimits.o tmapped.o tnotify.o tresize.o tstore1.o tstore2.o tstore3.o tstore4.o libmk4.a regress.o: In function `ViewDisplay(c4_View const &, __sFILE *, int)': regress.o(.text+0x201): undefined reference to `c4_String::c4_String(char, int)' regress.o: In function `c4_String::c4_String(void)': regress.o(.gnu.linkonce.t.__9c4_String+0x13): undefined reference to `c4_String::Init(void const *, int)' regress.o: In function `c4_String::operator+=(c4_String const &)': regress.o(.gnu.linkonce.t.__apl__9c4_StringRC9c4_String+0x26): undefined reference to `operator+(c4_String const &, c4_String const &)' regress.o(.gnu.linkonce.t.__apl__9c4_StringRC9c4_String+0x55): undefined reference to `c4_String::operator=(c4_String const &)' regress.o: In function `c4_String::GetLength(void) const': regress.o(.gnu.linkonce.t.GetLength__C9c4_String+0x21): undefined reference to `c4_String::FullLength(void) const' tbasics.o: In function `TestBasics(void)': tbasics.o(.text+0x5cc): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0xfce): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0x1212): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0x13be): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0xa9a0): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0xaa89): more undefined references to `c4_String::c4_String(char const *)' follow tbasics.o: In function `operator+(c4_String const &, char const *)': tbasics.o(.gnu.linkonce.t.__pl__FRC9c4_StringPCc+0x5e): undefined reference to `operator+(c4_String const &, c4_String const &)' tbasics.o: In function `operator==(char const *, c4_String const &)': tbasics.o(.gnu.linkonce.t.__eq__FPCcRC9c4_String+0x14): undefined reference to `c4_String::Compare(char const *) const' tbasics.o: In function `operator==(c4_String const &, char const *)': tbasics.o(.gnu.linkonce.t.__eq__FRC9c4_StringPCc+0x14): undefined reference to `c4_String::Compare(char const *) const' tcusto1.o: In function `TestCustom1(void)': tcusto1.o(.text+0x47e0): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x48cc): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x49b8): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x4aa4): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x5f09): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x5ff1): more undefined references to `c4_String::c4_String(char const *)' follow textend.o: In function `c4_String::c4_String(void const *, int)': textend.o(.gnu.linkonce.t.__9c4_StringPCvi+0x19): undefined reference to `c4_String::Init(void const *, int)' tlimits.o: In function `TestLimits(void)': tlimits.o(.text+0x16c): undefined reference to `c4_String::operator=(c4_String const &)' tlimits.o(.text+0xac1): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0xc1c): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0xd81): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0x231c): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0x24f0): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0x2614): more undefined references to `c4_String::c4_String(char const *)' follow tlimits.o: In function `TestLimits(void)': tlimits.o(.text+0x307f): undefined reference to `c4_String::operator=(c4_String const &)' tlimits.o: In function `c4_String::operator+=(char const *)': tlimits.o(.gnu.linkonce.t.__apl__9c4_StringPCc+0x25): undefined reference to `c4_String::c4_String(char const *)' tlimits.o: In function `operator+(char const *, c4_String const &)': tlimits.o(.gnu.linkonce.t.__pl__FPCcRC9c4_String+0x29): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.gnu.linkonce.t.__pl__FPCcRC9c4_String+0x5e): undefined reference to `operator+(c4_String const &, c4_String const &)' tnotify.o: In function `TestNotify(void)': tnotify.o(.text+0x84dc): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x85d0): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x86c4): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x87b8): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x88ac): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x89a0): more undefined references to `c4_String::c4_String(char const *)' follow tresize.o: In function `TestResize(void)': tresize.o(.text+0x1a1e): undefined reference to `c4_String::c4_String(char, int)' tresize.o(.text+0x1b02): undefined reference to `c4_String::c4_String(char, int)' tresize.o(.text+0x1bee): undefined reference to `c4_String::c4_String(char, int)' tresize.o(.text+0x1cc2): undefined reference to `c4_String::c4_String(char, int)' tstore3.o: In function `TestStores3(void)': tstore3.o(.text+0x6b4): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0x7a4): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0x894): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0x984): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0xab4): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0xba4): more undefined references to `c4_String::c4_String(char const *)' follow *** Error code 1 Stop in /usr/home/rjent/work2/metakit/work/metakit-2.4.1/builds. *** Error code 1 Stop in /usr/home/rjent/work2/metakit. *** Error code 1 Stop in /usr/home/rjent/work2/metakit. *** Error code 1 Stop in /usr/home/rjent/work2/metakit. Any assistance on correcting this matter will be greatly appreciated! Again thanks to Dirk Meyer and a special thanks to JCW for all his work! -- Panther - Webmail Nelson MG From jcw@equi4.com Tue Oct 23 11:43:33 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA01445 for ; Tue, 23 Oct 2001 11:43:33 -0500 Received: from takkie.equi4.local ([195.121.235.177]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLO3IR01.8AS for ; Tue, 23 Oct 2001 18:37:39 +0200 Received: from [10.0.1.14] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9NGahB24663 for ; Tue, 23 Oct 2001 18:36:43 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] FreeBSD now compiles but fails test-tcl (assistance needed) Date: Tue, 23 Oct 2001 18:37:59 +0200 Message-Id: <20011023163759.3257@mail.planet.nl> In-Reply-To: <200110231409.f9NE9gD82886@mail.mgr3.k12.mo.us> References: <200110231409.f9NE9gD82886@mail.mgr3.k12.mo.us> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Richard Nelson wrote: >first let me thanks Dirk Meyer for his work in providing me a FreeBSD >port for metakit-2.4.1 that compiles. I am testing so it can be added >to the FreeBSD ports tree. The problem is that the port will compile but >fails the test-tcl option. Below is, I hope, the critical output from >this make option to provide insight: > >c++ -c -I./../unix/../include ./../unix/../tests/tbasics.cpp -o tbasics.o [...] >c++ -o regress regress.o tbasics.o tcusto1.o tcusto2.o tdiffer.o >textend.o tformat.o tlimits.o tmapped.o tnotify.o tresize.o tstore1.o >tstore2.o tstore3.o tstore4.o libmk4.a >regress.o: In function `ViewDisplay(c4_View const &, __sFILE *, int)': >regress.o(.text+0x201): undefined reference to >`c4_String::c4_String(char, int)' >regress.o: In function `c4_String::c4_String(void)': >regress.o(.gnu.linkonce.t.__9c4_String+0x13): undefined reference to >`c4_String::Init(void const *, int)' [...] This definition appears to be in mk4str.inl. If the main code was compiled with option "-Dq4_STD" then the same setting should be given to compile regress.cpp etc. Either that, or there is a mixup with inline enabled/disabled. Try adding "-Dq4_INLINE" to the compilation of regress.cpp etc. Would it be possible to send me info on how to improve the build process for FreeBSD? Or alternately, could you add a note on the MetaKit Wiki on how what needs to be adjusted to build MK 2.4.1 on FreeBSD? The page to adjust is: http://www.equi4.com/metakit/wiki.cgi/35.html As for Tcl: be sure to specify the "--with-tcl=" option to configure, with being the path of the unix/ dir of a compiled (recent) Tcl distribution. -jcw From jcw@equi4.com Tue Oct 23 12:01:10 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA02536 for ; Tue, 23 Oct 2001 12:01:10 -0500 Received: from takkie.equi4.local ([195.121.235.177]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLO3VG00.JU4 for ; Tue, 23 Oct 2001 18:45:16 +0200 Received: from [10.0.1.14] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9NGiEB24685 for ; Tue, 23 Oct 2001 18:44:14 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] How do I do a static build with Metakit? Date: Tue, 23 Oct 2001 18:45:06 +0200 Message-Id: <20011023164506.4861@mail.planet.nl> In-Reply-To: <000901c15a73$0e525460$0200a8c0@pacbell.net> References: <000901c15a73$0e525460$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stu wrote: >I know this isn't Metakit specific but I can't seem to get a build that >doesn't require the metakit dll. I am also linking statically to mfc. This sort of stuff can be quite tricky to figure and get right. If you are using MSVC, then you should build the "mklib" target, not "mkdll". The MSVC project is in "win/msvc60/mksrc.dsw". BTW, there is a static build of MK 2.4.1 for Windows over here: http://www.equi4.com/pub/mk/metakit-2.4.1-31-win/mk4vc60s.lib -jcw From rjn103s@mail.mgr3.k12.mo.us Tue Oct 23 13:23:39 2001 Received: from mail.mgr3.k12.mo.us (mail.mgr3.k12.mo.us [204.184.227.130]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA07897 for ; Tue, 23 Oct 2001 13:23:39 -0500 Received: (from Unknown UID 65534@localhost) by mail.mgr3.k12.mo.us (8.11.6/8.11.4) id f9NIDSD02232; Tue, 23 Oct 2001 13:13:28 -0500 (CDT) (envelope-from rjn103s@mail.mgr3.k12.mo.us) Date: Tue, 23 Oct 2001 13:13:28 -0500 (CDT) Message-Id: <200110231813.f9NIDSD02232@mail.mgr3.k12.mo.us> X-Authentication-Warning: mail.mgr3.k12.mo.us: Unknown UID 65534 set sender to rjn103s@mail.mgr3.k12.mo.us using -f From: "Richard Nelson" To: metakit@equi4.com Reply-To: rjn103s@mail.mgr3.k12.mo.us Subject: Re: [Metakit] FreeBSD now compiles but fails test-tcl (Success!) X-Mailer: NeoMail 1.25 X-IPAddress: 172.16.0.1 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, Greetings, > Either that, or there is a mixup with inline enabled/disabled. Try > adding "-Dq4_INLINE" to the compilation of regress.cpp etc. > yes this seemed to do the trick!! All runs well and so does the test! > Would it be possible to send me info on how to improve the build process > for FreeBSD? Or alternately, could you add a note on the MetaKit Wiki on > how what needs to be adjusted to build MK 2.4.1 on FreeBSD? The page to > adjust is: > http://www.equi4.com/metakit/wiki.cgi/35.html First all credit must go to Dirk Meyer who provided me with a port for metakit. I believe that the plan is to put this in the ports tree and keep the port up to date. If a FreeBSD user wants MetaKit they will only have to cd /usr/ports/databases/metakit and do a make. There will be some settings that they need to adjust in their /etc/make.conf for specific building of metakit and these can be documented after the final port is established and submitted to the ports tree;) I will try to assist with this:) I have forwarded you messge to Dirk and he may be able to provide ways to improve the compile under FreeBSD. As for now a freshly rolled port seems to address much:) Perhaps after the new port is submitted a make option for the TclKit could be added;) > > As for Tcl: be sure to specify the "--with-tcl=" option to > configure, with being the path of the unix/ dir of a compiled > (recent) Tcl distribution. I think Dirk did magic in the port Makefile to address the dir setting for FreeBSD4.4. > > -jcw > Thanks so much for your reply and I will attempt to do some documentation on the wikit after final port is rolled and submitted. > -- Panther - Webmail Nelson MG From rjn103s@mail.mgr3.k12.mo.us Wed Oct 24 09:51:05 2001 Received: from mail.mgr3.k12.mo.us (mail.mgr3.k12.mo.us [204.184.227.130]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA04052 for ; Wed, 24 Oct 2001 09:51:05 -0500 Received: (from Unknown UID 65534@localhost) by mail.mgr3.k12.mo.us (8.11.6/8.11.4) id f9OEejl87684; Wed, 24 Oct 2001 09:40:45 -0500 (CDT) (envelope-from rjn103s@mail.mgr3.k12.mo.us) Date: Wed, 24 Oct 2001 09:40:45 -0500 (CDT) Message-Id: <200110241440.f9OEejl87684@mail.mgr3.k12.mo.us> X-Authentication-Warning: mail.mgr3.k12.mo.us: Unknown UID 65534 set sender to rjn103s@mail.mgr3.k12.mo.us using -f From: "Richard Nelson" To: metakit@equi4.com Reply-To: rjn103s@mail.mgr3.k12.mo.us X-Mailer: NeoMail 1.25 X-IPAddress: 172.16.0.5 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] FreeBSD4.4 status (some errors during make testing) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, Dirk has kindly provided a new port to me that seems close to production ready, however, now that testing has been integrated into the port some feedback from the tests suggest that there may still be some issues. Below is a snippet from where the testing gives feedback of problems: diff ./../unix/../tests/ok tests cd ./../unix/../tcl/test && tclsh8.3 all.tcl Processing 9 scripts...  mk1basic    FAILED: mk1basic.6 - open same file again  mk2chan  mk3struct  mk4commit  mk5object    FAILED: mk5object.7 - blocked hash view  mk6fixed  mk7limit  mk8fail  mk9crash Failed 2 of 32 tests: *** mk1basic.6 - open same file again *** C db2 {file open failed} *** mk5object.7 - blocked hash view *** C 17 9 Thoughts welcome! -- Panther - Webmail Nelson MG From john_matthews27@hotmail.com Fri Oct 26 02:52:35 2001 Received: from hotmail.com (f119.law14.hotmail.com [64.4.21.119]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA05209 for ; Fri, 26 Oct 2001 02:52:25 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Fri, 26 Oct 2001 00:46:22 -0700 Received: from 193.120.90.34 by lw14fd.law14.hotmail.msn.com with HTTP; Fri, 26 Oct 2001 07:46:22 GMT X-Originating-IP: [193.120.90.34] From: "John Matthews" To: metakit@equi4.com Date: Fri, 26 Oct 2001 07:46:22 +0000 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 26 Oct 2001 07:46:22.0615 (UTC) FILETIME=[4EA77270:01C15DF2] Subject: [Metakit] Metakit Memory Consmption Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm evaluating Metakit 2.4.0 as the engine for a Windows-based product involving browsing a CDROM database. I'm using Borland C++ Builder 5. The database is encrypted using a Strategy class. It consists of a single table with 76K rows and 28 columns. Most columns are strings and about three have strings of highly variable length (mostly short, but a few exceeding 1KB). The total database size is 39MB. Tests browsing the database show very high memory consumption at 36MB (up from 5MB initially), even though only one row was accessed at a time (one c4_View, one c4-Row, ...). Could the memory consumption be caused by Metakit? If so, is there anything I can do to mitigate it? Thanks in Advance JM _________________________________________________________________ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp From jcw@equi4.com Fri Oct 26 04:57:25 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA10543 for ; Fri, 26 Oct 2001 04:57:24 -0500 Received: from takkie.equi4.local ([195.121.232.87]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLT4Q401.1S8 for ; Fri, 26 Oct 2001 11:51:40 +0200 Received: from [10.0.1.15] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9Q9p5B06912 for ; Fri, 26 Oct 2001 11:51:06 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Metakit Memory Consmption Date: Fri, 26 Oct 2001 11:47:14 +0200 Message-Id: <20011026094714.16044@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John Matthews wrote: >[...] The database is encrypted using a Strategy class. [...] >Tests browsing the database show very high memory consumption at 36MB (up >from 5MB initially), even though only one row was accessed at a time (one >c4_View, one c4-Row, ...). > >Could the memory consumption be caused by Metakit? >If so, is there anything I can do to mitigate it? Yes, the strategy class disables the use of memory mapped files, or more precisely: it creates a copy in memory of all data. You can get rid of all buffered data by closing and re-opening, or by calling c4_Storage::Rollback, which is more convenient. In both cases, you have to "re-attach" the view: your view object becomes empty, so you have to call c4_Storage::View or c4_Storage::GetAs again. This should be relatively quick in MK 2.4, because the time to open a datafile is no longer proportional to the complexity of the stored data structure, but more or less constant. Then again, calculating file offsets for a column with 76K strings does take a few cycles. You can reduce the latency on first access (also after rollback), by considering a "Blocked" view, i.e. one which segments a view into a bunch of smaller subviews. This requires a change in your data structure. I couldn't find a C++ example to illustrate, not even a Python one, but there is one in Tcl, see examples/mapped.tcl, line 23. The idea that instead of a view "v[a:S,b:S,c:S]", you define the view as "_B[v[a:S,b:S,c:S]]". Then, instead of c4_View v = storage.View("v"); use: c4_View v = storage.View("_B"); v = v.Blocked(); The rest of your code does not need to change. It is essential that you do the same while filling the view with data. What this does is to transparently create an extra layer of indirection, storing the 76K or so rows in subviews with each some 8.5K rows. Access for a full traversal will be slower, but access to a single item will avoid the (one-time) 76K string offset calculations needed when a column is accessed. You'll need to compare to find out whether such a change is an improvement. Feel free to post comparisons and timing/memory results to this mailing list :) Regards, Jean-Claude From jcw@equi4.com Fri Oct 26 04:57:46 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA10559 for ; Fri, 26 Oct 2001 04:57:45 -0500 Received: from takkie.equi4.local ([195.121.232.87]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLT4Q500.N5A for ; Fri, 26 Oct 2001 11:51:41 +0200 Received: from [10.0.1.15] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9Q9p6B06915 for ; Fri, 26 Oct 2001 11:51:07 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] FreeBSD4.4 status (some errors during make testing) Date: Fri, 26 Oct 2001 11:51:57 +0200 Message-Id: <20011026095157.28662@mail.planet.nl> In-Reply-To: <200110241440.f9OEejl87684@mail.mgr3.k12.mo.us> References: <200110241440.f9OEejl87684@mail.mgr3.k12.mo.us> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id EAA10559 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Richard Nelson wrote: >  mk1basic >    FAILED: mk1basic.6 - open same file again This is a bug in the test suite, you can ignore it. It is a remnant from the time when MK did locking on the file (which is a hairy topic, cross- platform). I'll fix it - thanks! >  mk5object >    FAILED: mk5object.7 - blocked hash view This is probably fixed in the latest revision in CVS, release 2.4.1 had a bug in the hashing logic, see the change log at: http://www.equi4.com/metakit/CHANGES -jcw From mlh@idi.ntnu.no Sat Oct 27 13:31:06 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA22242 for ; Sat, 27 Oct 2001 13:31:05 -0500 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id UAA07196 for ; Sat, 27 Oct 2001 20:25:25 +0200 (MEST) Message-ID: <006b01c15f14$bf889af0$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: Date: Sat, 27 Oct 2001 20:25:25 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Another possible binding... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've come to think of another binding that would be great to have for MetaKit: PHP. Whenever I use PHP I keep using flat text files (comma-separated or the like), because I usually don't want the hassle of SQL databases, and dbm-style bases aren't flexible enough... MetaKit would be just wonderful to have there... (Perhaps it could even be included in the PHP distribution? They don't seem adverse to include lots of stuff... ;) Just a thought. I have no idea of how to implement such bindings for PHP myself. -- Magnus Lie Hetland http://hetland.org From nathan.thompson-amato@dnamerican.com Mon Oct 29 15:01:32 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id PAA01143 for ; Mon, 29 Oct 2001 15:01:31 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id qa054382 for ; Mon, 29 Oct 2001 16:06:52 -0500 Message-ID: <3BDDC23E.9040304@dnamerican.com> Date: Mon, 29 Oct 2001 15:55:26 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com References: <006b01c15f14$bf889af0$156ff181@idi.ntnu.no> Content-Type: multipart/mixed; boundary="------------050302070103040506000906" Subject: [Metakit] Hashed lookups in C++ Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------050302070103040506000906 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, all. I'm new to MetaKit, and I'm having a bit of trouble using hashed lookups on views combined with Commit() calls; as far as I can tell, my Commit()s aren't actually writing anything to the data file except the key for each record. If anybody here can point me at a snippet of working code that uses hashed lookups, I'd appreciate it. Also, I've attached a short segment of code that demonstrates the problem I'm having; the first time it's run, it works as expected, but once the data.dat file is created, "age" is always zero. Thanks, Nathan --------------050302070103040506000906 Content-Type: text/plain; name="mktest.cc" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="mktest.cc" #include #include int main(void) { c4_StringProp name ("name"); c4_IntProp age ("age"); c4_Storage storage ("data.dat", 1); c4_View view = storage.GetAs("data[name:S,age:I]"); // Create a mapping view for fast lookups by name c4_View view_h = storage.GetAs("data_H1[_H:I,_R:I]"); // Build the hash table c4_View hash_view = view.Hash(view_h, 1); c4_Row row; name (row) = "Bob"; age (row) = 20; hash_view.Add(row); storage.Commit(); // Look for "Bob" int index = hash_view.Find(name ["Bob"]); cout << "Bob is at index " << index << endl; row = hash_view[index]; cout << "Info for Bob:" << (const char *)(name(row)) << ";" << (t4_i32)(age(row)) << endl; return 0; } --------------050302070103040506000906-- From tciuro@mac.com Mon Oct 29 20:20:38 2001 Received: from smtpout.mac.com ([204.179.120.85]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA14324 for ; Mon, 29 Oct 2001 20:20:38 -0600 Received: from smtp-relay01.mac.com (smtp-relay01-qfe3 [10.13.10.224]) by smtpout.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9U2Eda29509 for ; Mon, 29 Oct 2001 18:14:39 -0800 (PST) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay01.mac.com (Netscape Messaging Server 4.15 relay01 Jun 21 2001 23:53:48) with ESMTP id GLZY8L00.162 for ; Mon, 29 Oct 2001 18:14:45 -0800 Received: from ciurti.apple.com ([17.202.22.175]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GLZY8K00.86I for ; Mon, 29 Oct 2001 18:14:44 -0800 Date: Mon, 29 Oct 2001 18:14:39 -0800 Mime-Version: 1.0 (Apple Message framework v501) Content-Type: multipart/alternative; boundary=Apple-Mail-1-331075676 From: Tito Ciuro To: metakit@equi4.com Message-Id: X-Mailer: Apple Mail (2.501) Subject: [Metakit] Problems with View.Search()? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-331075676 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hello, I have the following method: SInt32 quickFindRow (DataFileInfo* info, SInt32 uid) { if (info == NULL || info->mainViewStructure == NULL) return - kRecordNotFound; c4_View aView = *info->viewCache; if (aView.GetSize() == 0) return kRecordNotFound; // Find where the UID is (or should be) c4_IntProp pUID (k_AF_UIDProperty); SInt32 rowNumber = aView.Search(pUID[uid]); // Get the UID of the selected row and see if it matches // with what we're looking for SInt32 uidInRow = pUID(aView[rowNumber]); if (uidInRow == uid) // Yes, we found the match return rowNumber; else // We have to do this because sometimes MK seems to go one row above the match (???) if (rowNumber > 0) { rowNumber--; uidInRow = pUID(aView[rowNumber]); if (uidInRow == uid) // Yes, we found the match return rowNumber; } // No, the UID doesn't exist return kRecordNotFound; } There is a problem with Search(): I'm looking for ID 284 in the datafile, which I know it's located in row #266 (dump confirms that) but row 267 is returned instead (ID 285). Why is that? This is why I decrement the row number in the above snippet, then compare the row at that position and return the proper result (the row number if found; kRecordNotFound (-1) otherwise). This is a hack, of course!... Why is Search() behaving like that? Is Search() returning the next row where the ID should go? Is there something I'm missing? Some notes/assumptions: - c4_View aView = *info->viewCache is a view that I keep open while the datafile is open. I do that because calling GetAs() turns out to be an expensive operation. All ops (adding, editing, removing) are performed in this view. I also keep it sorted (I think!) by calling: _info->viewCache = sortView(_info->viewCache); c4_View* sortView (c4_View* aView) { c4_View sortedView = *aView; sortedView.Sort(); return aView; } - I tried to use Locate() instead: int rowNumber = kRecordNotFound; SInt32 numMatches = aView.Locate(pUID[uid], &rowNumber); It returns zero matches and row #267. I have no idea why... Questions: 1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I sorting the view properly? 2) Why is Search() and Locate() failing? Thanks so much, -- Tito --Apple-Mail-1-331075676 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hello, I have the following method: 0000,0000,FFFFSInt32 quickFindRow (DataFileInfo* info, SInt32 uid) { if (info == NULL || info->mainViewStructure == NULL) return - kRecordNotFound; c4_View aView = *info->viewCache; if (aView.GetSize() == 0) return kRecordNotFound; // Find where the UID is (or should be) c4_IntProp pUID (k_AF_UIDProperty); SInt32 rowNumber = aView.Search(pUID[uid]); // Get the UID of the selected row and see if it matches // with what we're looking for SInt32 uidInRow = pUID(aView[rowNumber]); if (uidInRow == uid) // Yes, we found the match return rowNumber; else // We have to do this because sometimes MK seems to go one row above the match (???) if (rowNumber > 0) { rowNumber--; uidInRow = pUID(aView[rowNumber]); if (uidInRow == uid) // Yes, we found the match return rowNumber; } // No, the UID doesn't exist return kRecordNotFound; } There is a problem with Search(): I'm looking for ID 284 in the datafile, which I know it's located in row #266 (dump confirms that) but row 267 is returned instead (ID 285). Why is that? This is why I decrement the row number in the above snippet, then compare the row at that position and return the proper result (the row number if found; kRecordNotFound (-1) otherwise). This is a hack, of course!... Why is Search() behaving like that? Is Search() returning the next row where the ID should go? Is there something I'm missing? Some notes/assumptions: - c4_View aView = *info->viewCache is a view that I keep open while the datafile is open. I do that because calling GetAs() turns out to be an expensive operation. All ops (adding, editing, removing) are performed in this view. I also keep it sorted (I think!) by calling: 0000,0000,FFFF_info->viewCache = sortView(_info->viewCache); c4_View* sortView (c4_View* aView) { c4_View sortedView = *aView; sortedView.Sort(); return aView; } - I tried to use Locate() instead: 0000,0000,FFFFint rowNumber = kRecordNotFound; SInt32 numMatches = aView.Locate(pUID[uid], &rowNumber); It returns zero matches and row #267. I have no idea why... Questions: 1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I sorting the view properly? 2) Why is Search() and Locate() failing? Thanks so much, -- Tito --Apple-Mail-1-331075676-- From jcw@equi4.com Tue Oct 30 00:57:51 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id AAA18725 for ; Tue, 30 Oct 2001 00:57:51 -0600 Received: from takkie.equi4.local ([195.121.234.25]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM0B2M00.NA3 for ; Tue, 30 Oct 2001 07:51:58 +0100 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9U6nYI08063 for ; Tue, 30 Oct 2001 07:49:44 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Problems with View.Search()? Date: Tue, 30 Oct 2001 07:49:01 +0100 Message-Id: <20011030064901.1028@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito Ciuro wrote: >Why is Search() behaving like that? Is Search() returning the next row >where the ID should go? Is there something I'm missing? Search returns the position of the row where it is. If absent, it returns the position where it should be inserted to maintain sort order. Search only works if the view is sorted, and uses binary search. >_info->viewCache = sortView(_info->viewCache); > >c4_View* sortView (c4_View* aView) >{ > c4_View sortedView = *aView; > sortedView.Sort(); > return aView; >} This does little more than eat CPU cycles. You're throwing away the sorted result. This call is a no-op. >1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I >sorting the view properly? Nope, I'm afraid not. >2) Why is Search() and Locate() failing? I'm pretty sure that if you were to check the view, you'd find it not to have all rows in sorted order. But you have to be careful how you fix this. Doing "v = v.Sort()" will produce a sorted view, but it also creates (and then hides) an extra level of indirection at each call. The result of a sort is a virtual view, which consists of a pointer to the original plus a permutation vector. Access will become slower each time you stack a sort on top of the previous one, and modification will very quickly become unusable (wrong in fact, due to the current limitations on stacked sort view updates). There are a few ways out: - keep the view sorted at all times and keep it stored that way - or sort once on open, then (copy it and) keep the volatile copy sorted - do not sort, but hash instead, if this used for exact-key access - write your own sort/ordering logic, optionally hide it in a custom view - stick with brute force, i.e. linear scan It could be helpful to write a little checking routine which traverses the entire view and verifies that the ID of each successive row is larger than its predecessor. It would no doubt have caught the unsorted status of your view. -jcw From jcw@equi4.com Tue Oct 30 01:22:50 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA19181 for ; Tue, 30 Oct 2001 01:22:49 -0600 Received: from takkie.equi4.local ([195.121.232.92]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM0C8800.MLD for ; Tue, 30 Oct 2001 08:16:56 +0100 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9U7EgI08276 for ; Tue, 30 Oct 2001 08:14:42 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Hashed lookups in C++ Date: Tue, 30 Oct 2001 08:18:00 +0100 Message-Id: <20011030071800.23880@mail.planet.nl> In-Reply-To: <3BDDC23E.9040304@dnamerican.com> References: <3BDDC23E.9040304@dnamerican.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nathan Thompson-Amato wrote: >Hi, all. I'm new to MetaKit, and I'm having a bit of trouble using hashed >lookups on views combined with Commit() calls; as far as I can tell, my >Commit()s aren't actually writing anything to the data file except the key >for each record. > >If anybody here can point me at a snippet of working code that uses hashed >lookups, I'd appreciate it. Also, I've attached a short segment of code >that demonstrates the problem I'm having; the first time it's run, it works >as expected, but once the data.dat file is created, "age" is always zero. [...] >int main(void) >{ [...] > name (row) = "Bob"; > age (row) = 20; > hash_view.Add(row); > > storage.Commit(); [...] >} This works for me - in the sense that the view will not grow and keep on returning the same info (age 20), because hashing assumes keys are unique, so "adding" in this case becomes a replace internally. I suspect that you are not using the very latest revision. A bug in hashing was fixed *after* the 2.4.1 release, see: http://www.equi4.com/metakit/CHANGES You're best off getting the latest from CVS, see: http://www.equi4.com/metakit/download.html -jcw From nathan.thompson-amato@dnamerican.com Tue Oct 30 08:20:15 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA04097 for ; Tue, 30 Oct 2001 08:20:14 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ha054425 for ; Tue, 30 Oct 2001 09:24:39 -0500 Message-ID: <3BDEB577.40107@dnamerican.com> Date: Tue, 30 Oct 2001 09:13:11 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Hashed lookups in C++ References: <3BDDC23E.9040304@dnamerican.com> <20011030071800.23880@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > >> name (row) = "Bob"; >> age (row) = 20; >> hash_view.Add(row); >> >> storage.Commit(); >> > > This works for me - in the sense that the view will not grow and keep on > returning the same info (age 20), because hashing assumes keys are > unique, so "adding" in this case becomes a replace internally. Using MetaKit 2.4.1, this code returned age 20 the first time and age 0 every subsequent time. (I should've mentioned that in my first post.) I've just tried the same code with the latest CVS version, and it seems to work the way I expect (always returning 20). Thanks very much for your help, Nathan From j.p.fletcher@aston.ac.uk Tue Oct 30 08:25:33 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA04486 for ; Tue, 30 Oct 2001 08:25:32 -0600 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 15yZk3-0007Ub-00 for metakit@equi4.com; Tue, 30 Oct 2001 14:19:47 +0000 From: "John Fletcher" To: Date: Tue, 30 Oct 2001 14:17:43 -0000 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] Hashed lookups in C++ Priority: normal In-reply-to: <20011030071800.23880@mail.planet.nl> References: <3BDDC23E.9040304@dnamerican.com> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler To: Subject: Re: [Metakit] Hashed lookups in C++ Date sent: Tue, 30 Oct 2001 08:18:00 +0100 > Nathan Thompson-Amato wrote: > > >Hi, all. I'm new to MetaKit, and I'm having a bit of trouble using hashed > >lookups on views combined with Commit() calls; as far as I can tell, my > >Commit()s aren't actually writing anything to the data file except the key > >for each record. > > > >If anybody here can point me at a snippet of working code that uses hashed > >lookups, I'd appreciate it. Also, I've attached a short segment of code > >that demonstrates the problem I'm having; the first time it's run, it works > >as expected, but once the data.dat file is created, "age" is always zero. > [...] > >int main(void) > >{ > [...] > > name (row) = "Bob"; > > age (row) = 20; > > hash_view.Add(row); > > > > storage.Commit(); > [...] > >} > > This works for me - in the sense that the view will not grow and keep on > returning the same info (age 20), because hashing assumes keys are > unique, so "adding" in this case becomes a replace internally. > > I suspect that you are not using the very latest revision. A bug in > hashing was fixed *after* the 2.4.1 release, see: > http://www.equi4.com/metakit/CHANGES > You're best off getting the latest from CVS, see: > http://www.equi4.com/metakit/download.html > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > I have just encountered the same sort of problem with hashed tables with duplicate keys. Is it possible to get access to the patched code other than via cvs, which I do not have on my PC based system. Thanks John ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From jcw@equi4.com Tue Oct 30 09:15:56 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA07731 for ; Tue, 30 Oct 2001 09:15:55 -0600 Received: from takkie.equi4.local ([195.121.235.100]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM0Y4S01.MH1 for ; Tue, 30 Oct 2001 16:10:04 +0100 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9UF7rI11048 for ; Tue, 30 Oct 2001 16:07:53 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Hashed lookups in C++ Date: Tue, 30 Oct 2001 16:09:39 +0100 Message-Id: <20011030150939.390@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Tue, Oct 30, 2001, John Fletcher wrote: [...] >Is it possible to get access to the patched code other than via cvs, >which I do not have on my PC based system. Isn't there wincvs? Anyway, I've placed a current CVS snapshot over here: http://www.equi4.com/previews/metakit.tar.gz -jcw From tciuro@mac.com Tue Oct 30 09:34:31 2001 Received: from smtpout.mac.com (smtpout.mac.com [204.179.120.89]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA09056 for ; Tue, 30 Oct 2001 09:34:30 -0600 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtpout.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9UFSX221767 for ; Tue, 30 Oct 2001 07:28:33 -0800 (PST) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GM0YZN00.E8J for ; Tue, 30 Oct 2001 07:28:35 -0800 Received: from localhost ([209.233.25.139]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GM0YZM00.DGW for ; Tue, 30 Oct 2001 07:28:34 -0800 Date: Tue, 30 Oct 2001 07:28:34 -0800 Subject: Re: [Metakit] Problems with View.Search()? Content-Type: multipart/alternative; boundary=Apple-Mail-1-378710705 Mime-Version: 1.0 (Apple Message framework v501) From: Tito Ciuro To: metakit In-Reply-To: <20011030064901.1028@mail.planet.nl> Message-Id: X-Mailer: Apple Mail (2.501) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-378710705 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hello, Thanks Jean-Claude for your prompt answer. > - do not sort, but hash instead, if this used for exact-key access I liked your suggestion, so I implemented it and seems to work fine (searching/editing/adding/removing). I based my implementation on the code used in the regression test as supplied with MetaKit: > c4_View v1 = s1.GetAs("v1[p1:S]"); > c4_View v2 = s1.GetAs("v2[_H:I,_R:I]"); > c4_View v3 = v1.Hash(v2, 1); What I noticed though is that v2 is also saved on file. Is v2 needed on disk? If possible, how can avoid having MetaKit save the hashed view on disk each time I call Commit()? Thanks in advance, -- Tito On Monday, October 29, 2001, at 10:49 PM, Jean-Claude Wippler wrote: > Tito Ciuro wrote: > >> Why is Search() behaving like that? Is Search() returning the next row >> where the ID should go? Is there something I'm missing? > > Search returns the position of the row where it is. If absent, it > returns the position where it should be inserted to maintain sort order. > Search only works if the view is sorted, and uses binary search. > >> _info->viewCache = sortView(_info->viewCache); >> >> c4_View* sortView (c4_View* aView) >> { >> c4_View sortedView = *aView; >> sortedView.Sort(); >> return aView; >> } > > This does little more than eat CPU cycles. You're throwing away the > sorted result. This call is a no-op. > >> 1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I >> sorting the view properly? > > Nope, I'm afraid not. > >> 2) Why is Search() and Locate() failing? > > I'm pretty sure that if you were to check the view, you'd find it not to > have all rows in sorted order. > > But you have to be careful how you fix this. Doing "v = v.Sort()" will > produce a sorted view, but it also creates (and then hides) an extra > level of indirection at each call. The result of a sort is a virtual > view, which consists of a pointer to the original plus a permutation > vector. Access will become slower each time you stack a sort on top of > the previous one, and modification will very quickly become unusable > (wrong in fact, due to the current limitations on stacked sort view > updates). > > There are a few ways out: > - keep the view sorted at all times and keep it stored that way > - or sort once on open, then (copy it and) keep the volatile copy > sorted > - do not sort, but hash instead, if this used for exact-key access > - write your own sort/ordering logic, optionally hide it in a custom > view > - stick with brute force, i.e. linear scan > > It could be helpful to write a little checking routine which traverses > the entire view and verifies that the ID of each successive row is > larger > than its predecessor. It would no doubt have caught the unsorted status > of your view. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > --Apple-Mail-1-378710705 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hello, Thanks Jean-Claude for your prompt answer. 0000,0000,DEDE - do not sort, but hash instead, if this used for exact-key access 0000,0000,DEDE 0000,0000,0000I liked your suggestion, so I implemented it and seems to work fine (searching/editing/adding/removing). I based my implementation on the code used in the regression test as supplied with MetaKit: c4_View v1 = s1.GetAs("v1[p1:S]"); c4_View v2 = s1.GetAs("v2[_H:I,_R:I]"); c4_View v3 = v1.Hash(v2, 1); What I noticed though is that v2 is also saved on file. Is v2 needed on disk? If possible, how can avoid having MetaKit save the hashed view on disk each time I call Commit()? Thanks in advance, -- Tito 0000,0000,DEDE On Monday, October 29, 2001, at 10:49 PM, Jean-Claude Wippler wrote: Tito Ciuro < wrote: Why is Search() behaving like that? Is Search() returning the next row where the ID should go? Is there something I'm missing? Search returns the position of the row where it is. If absent, it returns the position where it should be inserted to maintain sort order. Search only works if the view is sorted, and uses binary search. _info->viewCache = sortView(_info->viewCache); c4_View* sortView (c4_View* aView) { c4_View sortedView = *aView; sortedView.Sort(); return aView; } This does little more than eat CPU cycles. You're throwing away the sorted result. This call is a no-op. 1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I sorting the view properly? Nope, I'm afraid not. 2) Why is Search() and Locate() failing? I'm pretty sure that if you were to check the view, you'd find it not to have all rows in sorted order. But you have to be careful how you fix this. Doing "v = v.Sort()" will produce a sorted view, but it also creates (and then hides) an extra level of indirection at each call. The result of a sort is a virtual view, which consists of a pointer to the original plus a permutation vector. Access will become slower each time you stack a sort on top of the previous one, and modification will very quickly become unusable (wrong in fact, due to the current limitations on stacked sort view updates). There are a few ways out: - keep the view sorted at all times and keep it stored that way - or sort once on open, then (copy it and) keep the volatile copy sorted - do not sort, but hash instead, if this used for exact-key access - write your own sort/ordering logic, optionally hide it in a custom view - stick with brute force, i.e. linear scan It could be helpful to write a little checking routine which traverses the entire view and verifies that the ID of each successive row is larger than its predecessor. It would no doubt have caught the unsorted status of your view. -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit --Apple-Mail-1-378710705-- From jcw@equi4.com Tue Oct 30 09:40:41 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA09504 for ; Tue, 30 Oct 2001 09:40:40 -0600 Received: from takkie.equi4.local ([195.121.235.236]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM0Z9Y00.UYP for ; Tue, 30 Oct 2001 16:34:46 +0100 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9UFWZI11359 for ; Tue, 30 Oct 2001 16:32:35 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Problems with View.Search()? Date: Tue, 30 Oct 2001 16:34:20 +0100 Message-Id: <20011030153420.3530@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito Ciuro wrote: >> - do not sort, but hash instead, if this used for exact-key access > >I liked your suggestion, so I implemented it and seems to work fine >(searching/editing/adding/removing). I based my implementation on the >code used in the regression test as supplied with MetaKit: > >> c4_View v1 = s1.GetAs("v1[p1:S]"); >> c4_View v2 = s1.GetAs("v2[_H:I,_R:I]"); >> c4_View v3 = v1.Hash(v2, 1); > >What I noticed though is that v2 is also saved on file. Is v2 needed on >disk? If possible, how can avoid having MetaKit save the hashed view on >disk each time I call Commit()? Doesn't the following work as expected? c4_View v1 = s1.GetAs("v1[p1:S]"); c4_View v2; c4_View v3 = v1.Hash(v2, 1); -jcw From tciuro@mac.com Tue Oct 30 10:03:15 2001 Received: from smtpout.mac.com (smtpout.mac.com [204.179.120.89]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA11397; Tue, 30 Oct 2001 10:03:15 -0600 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtpout.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9UFvD201176; Tue, 30 Oct 2001 07:57:13 -0800 (PST) Received: from asmtp01.mac.com ([10.13.10.65]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GM10BI00.GH6; Tue, 30 Oct 2001 07:57:18 -0800 Received: from localhost ([209.233.25.139]) by asmtp01.mac.com (Netscape Messaging Server 4.15 asmtp01 Jun 21 2001 23:53:48) with ESMTP id GM10BH00.K0L; Tue, 30 Oct 2001 07:57:17 -0800 Date: Tue, 30 Oct 2001 07:57:17 -0800 Subject: Re: [Metakit] Hashed lookups in C++ Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v501) Cc: metakit@equi4.com To: Jean-Claude Wippler From: Tito Ciuro In-Reply-To: <20011030071800.23880@mail.planet.nl> Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.501) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, > Doesn't the following work as expected? > > c4_View v1 = s1.GetAs("v1[p1:S]"); > c4_View v2; > c4_View v3 = v1.Hash(v2, 1); Thanks Jean-Claude once again for your help. It works as expected. I guess the confusion was in this comment found in the view.cpp file: > * The defined structure of the map view must be "_H:I,_R:I". Also, the code in the regression test (file tmapped.cpp) seemed to confirmed that: > c4_StringProp p1 ("p1"); > > c4_Storage s1; > c4_View v1 = s1.GetAs("v1[p1:S]"); > c4_View v2 = s1.GetAs("v2[_H:I,_R:I]"); > c4_View v3 = v1.Hash(v2, 1); So this is why I thought it was required to have this type of view. In any case, thank you! :) -- Tito On Monday, October 29, 2001, at 11:18 PM, Jean-Claude Wippler wrote: > Nathan Thompson-Amato wrote: > >> Hi, all. I'm new to MetaKit, and I'm having a bit of trouble using >> hashed >> lookups on views combined with Commit() calls; as far as I can tell, my >> Commit()s aren't actually writing anything to the data file except the >> key >> for each record. >> >> If anybody here can point me at a snippet of working code that uses >> hashed >> lookups, I'd appreciate it. Also, I've attached a short segment of >> code >> that demonstrates the problem I'm having; the first time it's run, it >> works >> as expected, but once the data.dat file is created, "age" is always >> zero. > [...] >> int main(void) >> { > [...] >> name (row) = "Bob"; >> age (row) = 20; >> hash_view.Add(row); >> >> storage.Commit(); > [...] >> } > > This works for me - in the sense that the view will not grow and keep on > returning the same info (age 20), because hashing assumes keys are > unique, so "adding" in this case becomes a replace internally. > > I suspect that you are not using the very latest revision. A bug in > hashing was fixed *after* the 2.4.1 release, see: > http://www.equi4.com/metakit/CHANGES > You're best off getting the latest from CVS, see: > http://www.equi4.com/metakit/download.html > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From steve@improvision.com Tue Oct 30 18:38:20 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA19753 for ; Tue, 30 Oct 2001 18:38:19 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id AAA08064 for ; Wed, 31 Oct 2001 00:30:37 GMT Received: from [10.0.1.2] ([213.105.144.33]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971 for ; Wed, 31 Oct 2001 00:32:25 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Wed, 31 Oct 2001 00:32:22 +0000 From: Steve Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Problem with partial access of byte properties in a hashed view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, New to Metakit and not sure whether I am doing something wrong or have found a bug! I have a view that has a bytes (B) property and is hashed on the first column (a unique integer key to the view). I find that I cannot partially access the bytes field using Access() or Modify() through the hash view - the c4_Bytes object returned is empty. I can reproduce this by changing test b26 as follows: B(b26, Partial memo field access, 0) { c4_BytesProp p1 ("p1"); c4_Storage s1; c4_View raw = s1.GetAs("raw[p1:B]"); c4_View hash = s1.GetAs("hash[_H:I,_R:I]"); c4_View v1 = raw.Hash(hash, 1); v1.Add(p1 [c4_Bytes ("12345", 5)]); A(v1.GetSize() == 1); c4_Bytes buf = p1 (v1[0]); A(buf.Size() == 5); A(buf == c4_Bytes ("12345", 5)); buf = p1(v1[0]).Access(1,3); A(buf == c4_Bytes ("234", 3)); p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); buf = p1 (v1[0]); A(buf == c4_Bytes ("12ab5", 5)); p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); buf = p1 (v1[0]); A(buf == c4_Bytes ("1ABCab5", 7)); p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); buf = p1 (v1[0]); A(buf == c4_Bytes ("1Axyz", 5)); p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); buf = p1 (v1[0]); A(buf == c4_Bytes ("1Axy3456", 8)); } E; Anyone know if I am using the hash view in the wrong way, or is this a bug? Cheers, Steve. P.S. Very, very impressed with Metakit - on only 2 days I have totally rewritten the database engine for one of our products, it's just so neat! From jcw@equi4.com Wed Oct 31 02:07:04 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA00593 for ; Wed, 31 Oct 2001 02:07:03 -0600 Received: from takkie.equi4.local ([195.121.233.32]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM28X200.RZU for ; Wed, 31 Oct 2001 09:00:38 +0100 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9V7wXI15574 for ; Wed, 31 Oct 2001 08:58:33 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Problem with partial access of byte properties in a hashed view Date: Wed, 31 Oct 2001 09:01:46 +0100 Message-Id: <20011031080146.16916@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Steve Baxter wrote: >New to Metakit and not sure whether I am doing something wrong or have found >a bug! You have found a bug. Thanks for the example, which made understanding this one easy. Unfortunately, the problem is a fundamental one which I cannot easily fix (more precisely: I could fix Access, but I can't fix Modify without losing its performance advantage for partial access). > B(b26, Partial memo field access, 0) > { > c4_BytesProp p1 ("p1"); > c4_Storage s1; > c4_View raw = s1.GetAs("raw[p1:B]"); > c4_View hash = s1.GetAs("hash[_H:I,_R:I]"); > c4_View v1 = raw.Hash(hash, 1); > > v1.Add(p1 [c4_Bytes ("12345", 5)]); > A(v1.GetSize() == 1); > c4_Bytes buf = p1 (v1[0]); > A(buf.Size() == 5); > A(buf == c4_Bytes ("12345", 5)); > buf = p1(v1[0]).Access(1,3); > A(buf == c4_Bytes ("234", 3)); > p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); > buf = p1 (v1[0]); > A(buf == c4_Bytes ("12ab5", 5)); > p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1ABCab5", 7)); > p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1Axyz", 5)); > p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1Axy3456", 8)); > } E; The above will work properly, if you replace all Access/Modify calls to operate on the underlying "raw" view instead of "v1". The situation can be summarized as: the new mapped views (hash, ordered, blocked, and indexed) are fine for normal get/set access, but the illusion they present breaks down with Access/Modify for byte properties. In all cases *except* blocked, this will work properly when you use the underlying "raw" view. This is due to the fact that Access/Modify do not insert/delete/move rows, but care must be taken so a key is never modified. Thanks for reporting this. I've added a page on the MetaKit to document this issue: http://www.equi4.com/metakit/wiki.cgi/132.html -jcw From steve@improvision.com Wed Oct 31 04:37:18 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA05170; Wed, 31 Oct 2001 04:37:17 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id KAA07465; Wed, 31 Oct 2001 10:29:32 GMT Received: from [192.168.1.78] ([192.168.1.78]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Wed, 31 Oct 2001 10:31:20 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Wed, 31 Oct 2001 10:31:20 +0000 Subject: Re: [Metakit] Problem with partial access of byte properties in a hashed view From: Stephen Baxter To: Jean-Claude Wippler , Message-ID: In-Reply-To: <20011031080146.16916@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Cool - I'm not going mad after all. One more question - I am going to use the hash search to find the record to update or read, then update or read it using Modify() and Access(). Can I assume that records in raw and v1 correspond? What I mean is can I assume that row 5 in v1 is the same record as row 5 in raw? Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Wed, 31 Oct 2001 09:01:46 +0100 > To: > Subject: Re: [Metakit] Problem with partial access of byte properties in a > hashed view > > Steve Baxter wrote: > >> New to Metakit and not sure whether I am doing something wrong or have found >> a bug! > > You have found a bug. Thanks for the example, which made understanding > this one easy. Unfortunately, the problem is a fundamental one which I > cannot easily fix (more precisely: I could fix Access, but I can't fix > Modify without losing its performance advantage for partial access). > >> B(b26, Partial memo field access, 0) >> { >> c4_BytesProp p1 ("p1"); >> c4_Storage s1; >> c4_View raw = s1.GetAs("raw[p1:B]"); >> c4_View hash = s1.GetAs("hash[_H:I,_R:I]"); >> c4_View v1 = raw.Hash(hash, 1); >> >> v1.Add(p1 [c4_Bytes ("12345", 5)]); >> A(v1.GetSize() == 1); >> c4_Bytes buf = p1 (v1[0]); >> A(buf.Size() == 5); >> A(buf == c4_Bytes ("12345", 5)); >> buf = p1(v1[0]).Access(1,3); >> A(buf == c4_Bytes ("234", 3)); >> p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); >> buf = p1 (v1[0]); >> A(buf == c4_Bytes ("12ab5", 5)); >> p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); >> buf = p1 (v1[0]); >> A(buf == c4_Bytes ("1ABCab5", 7)); >> p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); >> buf = p1 (v1[0]); >> A(buf == c4_Bytes ("1Axyz", 5)); >> p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); >> buf = p1 (v1[0]); >> A(buf == c4_Bytes ("1Axy3456", 8)); >> } E; > > The above will work properly, if you replace all Access/Modify calls to > operate on the underlying "raw" view instead of "v1". > > The situation can be summarized as: the new mapped views (hash, ordered, > blocked, and indexed) are fine for normal get/set access, but the > illusion they present breaks down with Access/Modify for byte properties. > In all cases *except* blocked, this will work properly when you use the > underlying "raw" view. This is due to the fact that Access/Modify do not > insert/delete/move rows, but care must be taken so a key is never modified. > > Thanks for reporting this. I've added a page on the MetaKit to document > this issue: http://www.equi4.com/metakit/wiki.cgi/132.html > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From terabaap@users.sourceforge.net Wed Oct 31 05:14:59 2001 Received: from femail40.sdc1.sfba.home.com (femail40.sdc1.sfba.home.com [24.254.60.34]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA06540 for ; Wed, 31 Oct 2001 05:14:59 -0600 Received: from galadriel ([24.183.20.14]) by femail40.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011031110900.QSBS2143.femail40.sdc1.sfba.home.com@galadriel> for ; Wed, 31 Oct 2001 03:09:00 -0800 Date: Wed, 31 Oct 2001 05:09:35 -0600 From: Yumpee X-Mailer: The Bat! (v1.49) UNREG / CD5BF9353B3B7091 Reply-To: Yumpee X-Priority: 3 (Normal) Message-ID: <55213009958.20011031050935@users.sourceforge.net> To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Mk4py: Internal Python error with simple test program Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I've been playing around with Mk4py and encountered some strange behavior with metakit-2.4.1 built from source on Win98 with MSVC6 and Python-2.0: Consider the following program which creates a view, adds two records and then tries to dump the view: -------------------------------- import metakit db = metakit.storage() v = db.getas('t[f:S]') i1 = v.append(f="FOO") i2 = v.append(f="BAR") metakit.dump(v, "orig") # -- dump is a view pretty-printer v.delete(i1) # -- defined in metakit.py metakit.dump(v,"after delete") -------------------------------- Running this gives me this output and traceback: =================================================== orig f --- FOO BAR --- Total: 2 rows Traceback (most recent call last): File "test.py", line 10, in ? metakit.dump(v,"after delete") File "metakit.py", line 27, in dump if prop.type in ('I','F','D','V'): SystemError: null argument to internal routine ==================================================== I tried putting in some prints in the dump() routine defined in metakit.py and there seems to be nothing wrong with "prop" or "prop.type" but Python barfs when it comes to testing prop.type in that tuple of characters. Now comes the weird part. I put in a print statement in the dump() routine just before the "if prop.type check": ---------------------------------------- print type(prop.type) ---------------------------------------- And now the test program runs just fine!! Usually if this happens in a C program (printf statement preventing core dump), it means that memory is getting corrupted somewhere and the printf is preventing the corruption. Wondering if that is the case with the Mk4py extension also ... Y. From steve@improvision.com Wed Oct 31 06:04:46 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA08546 for ; Wed, 31 Oct 2001 06:04:44 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id LAA22809 for ; Wed, 31 Oct 2001 11:56:58 GMT Received: from [10.0.1.10] ([192.168.1.2]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971 for ; Wed, 31 Oct 2001 11:58:45 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Wed, 31 Oct 2001 11:58:44 +0000 From: Steve Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Another problem with byte properties Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I think I may have found a second problem with storing byte properties. I commit my storage after every "transaction" in my database. I am finding though that changes to byte properties are not correctly committed. I can reproduce this by changing test b26 again to have file-based storage and to commit after each change to the string. After a couple of changes, the test fails. Changes to other properties seem to be stored correctly, it's just the byte properties that are corrupted. Here is the modified test: B(b26, Partial memo field access, 0) { c4_BytesProp p1 ("p1"); c4_Storage s1( "test.dat", true ); c4_View v1 = s1.GetAs("v1[key:I,p1:B]"); v1.Add(p1 [c4_Bytes ("12345", 5)]); A(v1.GetSize() == 1); s1.Commit(); c4_Bytes buf = p1 (v1[0]); A(buf.Size() == 5); A(buf == c4_Bytes ("12345", 5)); buf = p1(v1[0]).Access(1,3); A(buf == c4_Bytes ("234", 3)); p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); s1.Commit(); buf = p1 (v1[0]); A(buf == c4_Bytes ("12ab5", 5)); p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); s1.Commit(); buf = p1 (v1[0]); A(buf == c4_Bytes ("1ABCab5", 7)); p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); s1.Commit(); buf = p1 (v1[0]); A(buf == c4_Bytes ("1Axyz", 5)); p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); s1.Commit(); buf = p1 (v1[0]); A(buf == c4_Bytes ("1Axy3456", 8)); } E; You need to remember to delete test.dat before running the test each time or A(v1.GetSize() == 1) will fail. Again, have I misunderstood the use of Commit() or is this a bug? Cheers, Steve. From jcw@equi4.com Wed Oct 31 06:20:57 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA09400 for ; Wed, 31 Oct 2001 06:20:56 -0600 Received: from takkie.equi4.local ([195.121.233.224]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM2KO700.T3V for ; Wed, 31 Oct 2001 13:14:31 +0100 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9VCCRI17373 for ; Wed, 31 Oct 2001 13:12:27 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Problem with partial access of byte properties in a hashed view Date: Wed, 31 Oct 2001 12:50:35 +0100 Message-Id: <20011031115035.14692@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: [hash] >Can I assume that records in raw and v1 correspond? What I mean is can I >assume that row 5 in v1 is the same record as row 5 in raw? Yes. -jcw From gmcm@hypernet.com Wed Oct 31 07:42:17 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA12676 for ; Wed, 31 Oct 2001 07:42:15 -0600 Received: from me (204.176.40.87) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Wed, 31 Oct 2001 08:30:19 -0500 From: "Gordon McMillan" To: Yumpee , metakit@equi4.com Date: Wed, 31 Oct 2001 08:35:49 -0500 Subject: Re: [Metakit] Mk4py: Internal Python error with simple test program Reply-to: gmcm@hypernet.com Message-ID: <3BDFB7E5.11460.42FBEB2D@localhost> Priority: normal In-reply-to: <55213009958.20011031050935@users.sourceforge.net> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Seems to be an error leftover from the delete that hasn't been cleared. So, for example "v2 = v.sort()" also gets the error, but succeeds on a 2nd try. Need a PyErr_Clear() someplace in delete. > I've been playing around with Mk4py > and encountered some strange behavior > with metakit-2.4.1 built from source > on Win98 with MSVC6 and Python-2.0: > > Consider the following program > which creates a view, adds two > records and then tries to dump > the view: > -------------------------------- > import metakit > > db = metakit.storage() > v = db.getas('t[f:S]') > i1 = v.append(f="FOO") > i2 = v.append(f="BAR") > metakit.dump(v, "orig") # -- dump is a view pretty-printer > v.delete(i1) # -- defined in metakit.py > metakit.dump(v,"after delete") -------------------------------- > > Running this gives me this > output and traceback: > =================================================== > orig > f > --- > FOO > BAR > --- > Total: 2 rows > Traceback (most recent call last): > File "test.py", line 10, in ? > metakit.dump(v,"after delete") > File "metakit.py", line 27, in dump > if prop.type in ('I','F','D','V'): > SystemError: null argument to internal routine > ==================================================== > > I tried putting in some prints in the > dump() routine defined in metakit.py > and there seems to be nothing wrong > with "prop" or "prop.type" but Python > barfs when it comes to testing prop.type > in that tuple of characters. > > Now comes the weird part. I put in > a print statement in the dump() > routine just before the "if prop.type > check": > ---------------------------------------- > print type(prop.type) > ---------------------------------------- > > And now the test program runs just fine!! > > Usually if this happens in a C program > (printf statement preventing core dump), > it means that memory is getting corrupted > somewhere and the printf is preventing > the corruption. Wondering if that is > the case with the Mk4py extension also ... > > Y. > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit - Gordon From jcw@equi4.com Wed Oct 31 17:32:11 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA07733 for ; Wed, 31 Oct 2001 17:32:11 -0600 Received: from takkie.equi4.local ([195.121.235.139]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM3FRL00.CEB for ; Thu, 1 Nov 2001 00:26:09 +0100 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9VNO8I21565 for ; Thu, 1 Nov 2001 00:24:09 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Mk4py: Internal Python error with simple test program Date: Thu, 1 Nov 2001 00:27:29 +0100 Message-Id: <20011031232729.15988@mail.planet.nl> In-Reply-To: <3BDFB7E5.11460.42FBEB2D@localhost> References: <3BDFB7E5.11460.42FBEB2D@localhost> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: >Seems to be an error leftover from the delete that hasn't been >cleared. So, for example "v2 = v.sort()" also gets the error, but >succeeds on a 2nd try. Need a PyErr_Clear() someplace in delete. Bingo, but a pretty tricky little bugger, this one. The solution: diff -r1.4 PyView.cpp 86c86 < PWOSequence seq; --- > PWOTuple seq; 903c903 < PWOSequence seq; --- > PWOTuple seq; [...] >> v.delete(i1) # -- defined in metakit.py >> metakit.dump(v,"after delete") -------------------------------- [...] >> Traceback (most recent call last): >> File "test.py", line 10, in ? >> metakit.dump(v,"after delete") >> File "metakit.py", line 27, in dump >> if prop.type in ('I','F','D','V'): >> SystemError: null argument to internal routine I've updated CVS, and http://www.equi4.com/metakit/CHANGES Thanks, Yumpee , for reporting this. -jcw From jcw@equi4.com Thu Nov 1 04:25:36 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA20677; Thu, 1 Nov 2001 04:25:35 -0600 Received: from takkie.equi4.local ([195.121.234.119]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM49ZV00.NCX; Thu, 1 Nov 2001 11:19:07 +0100 Received: from [10.0.1.7] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA1AH0I23933; Thu, 1 Nov 2001 11:17:10 +0100 From: Jean-Claude Wippler To: announce , metakit Date: Thu, 1 Nov 2001 11:20:16 +0100 Message-Id: <20011101102016.27063@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MkSQL Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a brief announcement to draw your attention to a (very slick, IMO) Python project called "MkSQL" which implements an amazingly complete SQL engine on top of the MetaKit database. Full details can be found here: http://www.mcmillan-inc.com/mksqlintro.html This second incarnation of MkSQL includes a Python DBI v2 interface, and adds support for referential integrity constraints. Like MetaKit itself, MkSQL is not multi-user, but it could no doubt be turned into such a solution by setting it up as a server (and addressing the issues of concurrent commits and isolation, if multi-user modification is required). MkSQL is the brainchild of Gordon McMillan , who also created the Mk4py binding which makes the C++ Metakit core fit so well to the Python world (see http://www.equi4.com/metakit/ for further info). Regards, Jean-Claude From steve@improvision.com Thu Nov 1 12:13:44 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA06250 for ; Thu, 1 Nov 2001 12:13:44 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA21573 for ; Thu, 1 Nov 2001 18:05:55 GMT Received: from [192.168.1.78] ([192.168.1.78]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Thu, 01 Nov 2001 18:07:43 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Thu, 01 Nov 2001 18:07:43 +0000 Subject: Re: [Metakit] Another problem with byte properties From: Stephen Baxter To: Steve Baxter , Message-ID: In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm still completely stuck on this. If I leave my commit until the file is closed, I get no problems. If I commit at any other time, after a few commits I get corrupted data written to the file. Are there limitations on when c4_Storage::Commit() can be called? Cheers, Steve. > From: Steve Baxter > Date: Wed, 31 Oct 2001 11:58:44 +0000 > To: > Subject: [Metakit] Another problem with byte properties > > I think I may have found a second problem with storing byte properties. > > I commit my storage after every "transaction" in my database. I am finding > though that changes to byte properties are not correctly committed. > > I can reproduce this by changing test b26 again to have file-based storage > and to commit after each change to the string. After a couple of changes, > the test fails. Changes to other properties seem to be stored correctly, > it's just the byte properties that are corrupted. > > Here is the modified test: > > B(b26, Partial memo field access, 0) > { > c4_BytesProp p1 ("p1"); > c4_Storage s1( "test.dat", true ); > c4_View v1 = s1.GetAs("v1[key:I,p1:B]"); > > v1.Add(p1 [c4_Bytes ("12345", 5)]); > A(v1.GetSize() == 1); > s1.Commit(); > > c4_Bytes buf = p1 (v1[0]); > A(buf.Size() == 5); > A(buf == c4_Bytes ("12345", 5)); > buf = p1(v1[0]).Access(1,3); > A(buf == c4_Bytes ("234", 3)); > p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); > s1.Commit(); > > buf = p1 (v1[0]); > A(buf == c4_Bytes ("12ab5", 5)); > p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); > s1.Commit(); > > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1ABCab5", 7)); > p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); > s1.Commit(); > > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1Axyz", 5)); > p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); > s1.Commit(); > > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1Axy3456", 8)); > } E; > > You need to remember to delete test.dat before running the test each time or > A(v1.GetSize() == 1) will fail. > > Again, have I misunderstood the use of Commit() or is this a bug? > > Cheers, > > Steve. > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Fri Nov 2 09:45:23 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA29844 for ; Fri, 2 Nov 2001 09:45:22 -0600 Received: from takkie.equi4.local ([195.121.233.129]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM6JGP01.4RY for ; Fri, 2 Nov 2001 16:38:49 +0100 Received: from [10.0.1.8] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA2Fb3I01713 for ; Fri, 2 Nov 2001 16:37:03 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Another problem with byte properties Date: Fri, 2 Nov 2001 16:38:53 +0100 Message-Id: <20011102153853.23501@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >I'm still completely stuck on this. If I leave my commit until the file is >closed, I get no problems. If I commit at any other time, after a few >commits I get corrupted data written to the file. > >Are there limitations on when c4_Storage::Commit() can be called? No. This was a bug in the new 2.3/2.4 storage format code, which only shows when you change a small item using partial access, i.e. Modify. The commit works, but it leaves a bad pointer so subsequent use in the same program will dereference a stale pointer. I've commited the one-line fix to CVS. Thanks for your patience, Steve! -jcw From nathan.thompson-amato@dnamerican.com Fri Nov 2 14:49:22 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA16063 for ; Fri, 2 Nov 2001 14:49:22 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ca054758 for ; Fri, 2 Nov 2001 15:54:35 -0500 Message-ID: <3BE3054D.4000201@dnamerican.com> Date: Fri, 02 Nov 2001 15:42:53 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit References: <20011030064901.1028@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Statically linking libmk4.a fails; library is stripped Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, I can't seem to statically link against Metakit (either 2.4.1 or the CVS version). Apparently, libmk4.a is being stripped by the 'make install' rule, and the resulting archive exports no symbols (according to 'nm'). When I statically link against the unstripped library, the link succeeds and my program runs. Perhaps the 'make install' rule needs changing? Or am I missing something? Thanks, Nathan From jcw@equi4.com Fri Nov 2 15:26:32 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA18366 for ; Fri, 2 Nov 2001 15:26:31 -0600 Received: from takkie.equi4.local ([195.121.233.211]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM6ZA000.7FY for ; Fri, 2 Nov 2001 22:20:24 +0100 Received: from [10.0.1.8] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA2LIdI04295 for ; Fri, 2 Nov 2001 22:18:39 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Statically linking libmk4.a fails; library is stripped Date: Fri, 2 Nov 2001 22:21:03 +0100 Message-Id: <20011102212103.2499@mail.planet.nl> In-Reply-To: <3BE3054D.4000201@dnamerican.com> References: <3BE3054D.4000201@dnamerican.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nathan Thompson-Amato wrote: >Perhaps the 'make install' rule needs changing? Or am I missing something? Oops. Fixed. In CVS. Blush. Thanks. -jcw From terabaap@users.sourceforge.net Fri Nov 2 21:03:15 2001 Received: from femail24.sdc1.sfba.home.com (femail24.sdc1.sfba.home.com [24.0.95.149]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA30404 for ; Fri, 2 Nov 2001 21:03:15 -0600 Received: from galadriel ([24.183.20.14]) by femail24.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011103025704.XOAV18561.femail24.sdc1.sfba.home.com@galadriel> for ; Fri, 2 Nov 2001 18:57:04 -0800 Date: Fri, 2 Nov 2001 20:57:06 -0600 From: Manoj Plakal X-Mailer: The Bat! (v1.49) UNREG / CD5BF9353B3B7091 Reply-To: Manoj Plakal X-Priority: 3 (Normal) Message-ID: <129442696434.20011102205706@users.sourceforge.net> To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Derived join views don't stay updated? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is there any documentation on when derived views stay updated and when they don't? I was testing deriving views by joining two views and it looked like the derived view wouldn't get updated if the join column values were changed in the underlying views. Sample Python code below. Manoj ---------------------------------------------- import metakit db = metakit.storage() v = db.getas('t[f:S,f2:S]') v.append(f="FOO",f2="U2") v.append(f="BAR",f2="Prodigy") v.append(f="BAZ",f2="NIN") metakit.dump(v, "orig") v2 = db.getas('t2[f:S]') v2.append(f="BAZ") v2.append(f="FOO") metakit.dump(v2, "to join") v3 = v.join(v2, metakit.property( 'S', 'f' ) ) metakit.dump(v3, "joined" ) v[0].f2="Dylan" # -- Changes U2 to Dylan in v3 metakit.dump(v3, "updated join" ) v2[1].f="BAR" # -- Does not change v3, shouldn't it? metakit.dump(v3, "updated join" ) ---------------------------------------------- From steve@improvision.com Mon Nov 5 03:44:20 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA01736; Mon, 5 Nov 2001 03:44:19 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id JAA02076; Mon, 5 Nov 2001 09:36:15 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 05 Nov 2001 09:38:01 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 09:38:06 +0000 Subject: Re: [Metakit] Another problem with byte properties From: Stephen Baxter To: Jean-Claude Wippler , Message-ID: In-Reply-To: <20011102153853.23501@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: That fixes the problem - thanks once again! Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Fri, 2 Nov 2001 16:38:53 +0100 > To: > Subject: Re: [Metakit] Another problem with byte properties > > Stephen Baxter wrote: > >> I'm still completely stuck on this. If I leave my commit until the file is >> closed, I get no problems. If I commit at any other time, after a few >> commits I get corrupted data written to the file. >> >> Are there limitations on when c4_Storage::Commit() can be called? > > No. > > This was a bug in the new 2.3/2.4 storage format code, which only shows > when you change a small item using partial access, i.e. Modify. The > commit works, but it leaves a bad pointer so subsequent use in the same > program will dereference a stale pointer. > > I've commited the one-line fix to CVS. Thanks for your patience, Steve! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From steve@improvision.com Mon Nov 5 06:24:16 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA08286 for ; Mon, 5 Nov 2001 06:24:15 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id MAA19826 for ; Mon, 5 Nov 2001 12:16:09 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971 for ; Mon, 05 Nov 2001 12:18:02 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 12:18:01 +0000 From: Stephen Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] c4_BytesRef::Access() reads all data Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Another problem with accessing binary data I'm afraid. I'm using a Mac (therefore no memory mapped files). I'm finding that if I read from a byte property using Access(), *all* the data in the byte property is read. Since my byte properties can each be many MB in size this takes forever! It is critical to my application that I can read partial streams. The problem seems to be that c4_Column::SetupSegments() runs through and reads the entire set of segments even if only one segment is needed. I would guess that this problem does not occur with memory-mapped files as the data is not actually read. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ From jcw@equi4.com Mon Nov 5 08:37:07 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA14868 for ; Mon, 5 Nov 2001 08:37:07 -0600 Received: from takkie.equi4.local ([195.121.234.204]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMC0AL01.D1M for ; Mon, 5 Nov 2001 15:30:21 +0100 Received: from [10.0.1.12] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA5ESmI21870 for ; Mon, 5 Nov 2001 15:28:58 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] c4_BytesRef::Access() reads all data Date: Mon, 5 Nov 2001 15:07:02 +0100 Message-Id: <20011105140702.4152@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >I'm using a Mac (therefore no memory mapped files). I'm finding that if I >read from a byte property using Access(), *all* the data in the byte >property is read. Since my byte properties can each be many MB in size this >takes forever! It is critical to my application that I can read partial >streams. > >The problem seems to be that c4_Column::SetupSegments() runs through and >reads the entire set of segments even if only one segment is needed. I >would guess that this problem does not occur with memory-mapped files as the >data is not actually read. Correct. In Mac OS 9., support was added for memory mapped files. Unfortunately, the docs say that mapping may only be used if the file is not open for any other use (if I remember correctly). That rules out the current mechanism MK uses: a R/O mapping, with R/W file access. Commits write to file (accessing the R/O map when copying parts which have not changed, but never writing to parts which are accessed through the R/O map so there are never problems with VM page coherence, no matter how limited mmap support is). Technically speaking it *could* be fixed for this case - by copying necessary data from the mapping, then unmapping, then opening the file for write, then writing from in-mem data only, then closing, then re- mapping R/O. But that would be a limited solution - for anything < OS 9, this would still not work. On the Mac, MK has always had high memory use (up to having all data in mem). On Mac OS X, normal memory-mapped file mechanism work just fine, btw. Better still, on Mac OS X 10.1, I'm told MK is included - used by AddressServices :) I'm afraid that there is no quick way out. You could slice up your data into smaller chunks (and perhaps use blocked views to slice up the resulting columns, though I'd try simple chunking first). Or store this particular part of your data in a separate datafile and have MK *manage* such "blobs" instead of actually storing them. I.e. store offset/length values in MK, and use a seek/read for actual access (it'll take more work for writing, i.e. managing free space, but it's definitely feasible). -jcw From steve@improvision.com Mon Nov 5 08:58:02 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA16081; Mon, 5 Nov 2001 08:58:02 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id OAA24068; Mon, 5 Nov 2001 14:49:56 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 05 Nov 2001 14:51:48 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 14:51:47 +0000 Subject: Re: [Metakit] c4_BytesRef::Access() reads all data From: Stephen Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011105140702.4152@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hmm, nasty - I see what you mean. I did just try implementing lazy reading within Metakit. Basically, I changed c4_Column so that the _segments array can be filled sparsely. When it actually needs the data it goes to fetch it from disk. The problem here seems to be that it slows down non-bytes columns. Do you think there is any use in me pursuing this? I have actually already implemented the "metakit-managing-many-files" approach - we have to do this because our databases can be 10s of GB in size (lots of data though not necessarily lots of records). I would also like a "single file" solution for users who have smaller data requirements, but if this is not possible then so be it. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Mon, 5 Nov 2001 15:07:02 +0100 > To: metakit > Subject: Re: [Metakit] c4_BytesRef::Access() reads all data > > Stephen Baxter wrote: > >> I'm using a Mac (therefore no memory mapped files). I'm finding that if I >> read from a byte property using Access(), *all* the data in the byte >> property is read. Since my byte properties can each be many MB in size this >> takes forever! It is critical to my application that I can read partial >> streams. >> >> The problem seems to be that c4_Column::SetupSegments() runs through and >> reads the entire set of segments even if only one segment is needed. I >> would guess that this problem does not occur with memory-mapped files as the >> data is not actually read. > > Correct. > > In Mac OS 9., support was added for memory mapped files. > > Unfortunately, the docs say that mapping may only be used if the file is > not open for any other use (if I remember correctly). That rules out the > current mechanism MK uses: a R/O mapping, with R/W file access. Commits > write to file (accessing the R/O map when copying parts which have not > changed, but never writing to parts which are accessed through the R/O > map so there are never problems with VM page coherence, no matter how > limited mmap support is). > > Technically speaking it *could* be fixed for this case - by copying > necessary data from the mapping, then unmapping, then opening the file > for write, then writing from in-mem data only, then closing, then re- > mapping R/O. But that would be a limited solution - for anything < OS 9, > this would still not work. > > On the Mac, MK has always had high memory use (up to having all data in > mem). On Mac OS X, normal memory-mapped file mechanism work just fine, > btw. Better still, on Mac OS X 10.1, I'm told MK is included - used by > AddressServices :) > > I'm afraid that there is no quick way out. You could slice up your data > into smaller chunks (and perhaps use blocked views to slice up the > resulting columns, though I'd try simple chunking first). Or store this > particular part of your data in a separate datafile and have MK *manage* > such "blobs" instead of actually storing them. I.e. store offset/length > values in MK, and use a seek/read for actual access (it'll take more work > for writing, i.e. managing free space, but it's definitely feasible). > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From cherry314159@yahoo.com Mon Nov 5 10:35:26 2001 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA22320 for ; Mon, 5 Nov 2001 10:35:26 -0600 Received: from pool-138-89-71-221.mad.east.verizon.net (HELO localhost.localdomain) (138.89.71.221) by smtp.mail.vip.sc5.yahoo.com with SMTP; 5 Nov 2001 16:29:12 -0000 X-Apparently-From: From: Jose Vasconcellos To: metakit@equi4.com Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/0.15 (Preview Release) Date: 05 Nov 2001 11:20:24 -0500 Message-Id: <1004977225.23523.15.camel@localhost.localdomain> Mime-Version: 1.0 Subject: [Metakit] Catfish file format Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I've written a Python version of catfish so I could browse the database from Linux. It's a bit alpha at the moment but it does work. I've also added an option to create or update a database. My problem is that I'm not clear on the definition of the 'info' view. Can anyone describe the 8 items that make up this view? Jose Vasconcellos _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From nathan.thompson-amato@dnamerican.com Mon Nov 5 14:12:10 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA04230 for ; Mon, 5 Nov 2001 14:11:55 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ea054838 for ; Mon, 5 Nov 2001 15:17:02 -0500 Message-ID: <3BE6F0FA.90700@dnamerican.com> Date: Mon, 05 Nov 2001 15:05:14 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: multipart/mixed; boundary="------------060102060208090303040307" Subject: [Metakit] Forgotten change after Find() on hash view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------060102060208090303040307 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit I'm having a bit of trouble updating existing rows when using a hash table view: after updating a row and doing a Find() on the hash view, the new values for the row are forgotten. Perhaps somebody can tell me where I'm going wrong... I've attached a code snippet to illustrate. Thanks, Nathan --------------060102060208090303040307 Content-Type: text/plain; name="mktest.cc" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="mktest.cc" #include #include int main(void) { c4_StringProp name ("name"); c4_IntProp age ("age"); c4_Storage storage ("data.dat", 1); c4_View view = storage.GetAs("data[name:S,age:I]"); // Create a mapping view for fast lookups by name c4_View view_h = storage.GetAs("data_H1[_H:I,_R:I]"); // Build the hash table c4_View hash_view = view.Hash(view_h, 1); c4_Row row; // Add Bob name (row) = "Bob"; age (row) = 20; hash_view.Add(row); // Look for Bob int index = hash_view.Find(name ["Bob"]); cout << "Bob is at index " << index << endl; row = hash_view[index]; cout << "Info for Bob: " << (const char *)(name(row)) << ";" << (t4_i32)(age(row)) << endl; // Change Bob age(row) = 22; cout << "Changed info for Bob: " << (const char *)(name(row)) << ";" << (t4_i32)(age(row)) << endl; // Find Bob again index = hash_view.Find(name ["Bob"]); cout << "Bob is at index " << index << endl; row = hash_view[index]; // Bob's age is 20 again! cout << "Changed info for Bob after a new Find(): " << (const char *)(name(row)) << ";" << (t4_i32)(age(row)) << endl; return 0; } --------------060102060208090303040307-- From steve@improvision.com Mon Nov 5 15:29:59 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA09487 for ; Mon, 5 Nov 2001 15:29:58 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id VAA21580 for ; Mon, 5 Nov 2001 21:21:52 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 05 Nov 2001 21:23:43 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 21:23:42 +0000 Subject: Re: [Metakit] Forgotten change after Find() on hash view From: Stephen Baxter To: Nathan Thompson-Amato , Message-ID: In-Reply-To: <3BE6F0FA.90700@dnamerican.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is it because a c4_Row is being used when you really wanted a c4_RowRef? I would guess that the row is a copy of the data from the view, not the real data. When you update the row you are updating the copy, not the real row. Try using a c4_Row to add the data, but a c4_RowRef when you get it in; row = hash_view[index]; Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Nathan Thompson-Amato > Organization: D.N. American > Date: Mon, 05 Nov 2001 15:05:14 -0500 > To: metakit@equi4.com > Subject: [Metakit] Forgotten change after Find() on hash view > > I'm having a bit of trouble updating existing rows when using a hash table > view: after updating a row and doing a Find() on the hash view, the new > values for the row are forgotten. Perhaps somebody can tell me where I'm > going wrong... > > I've attached a code snippet to illustrate. > > Thanks, > > Nathan > > > #include > #include > > int main(void) > { > c4_StringProp name ("name"); > c4_IntProp age ("age"); > > c4_Storage storage ("data.dat", 1); > > c4_View view = storage.GetAs("data[name:S,age:I]"); > > // Create a mapping view for fast lookups by name > c4_View view_h = storage.GetAs("data_H1[_H:I,_R:I]"); > > // Build the hash table > c4_View hash_view = view.Hash(view_h, 1); > > c4_Row row; > > // Add Bob > name (row) = "Bob"; > age (row) = 20; > hash_view.Add(row); > > // Look for Bob > int index = hash_view.Find(name ["Bob"]); > > cout << "Bob is at index " << index << endl; > > row = hash_view[index]; > > cout << "Info for Bob: " > << (const char *)(name(row)) << ";" > << (t4_i32)(age(row)) << endl; > > // Change Bob > age(row) = 22; > > cout << "Changed info for Bob: " > << (const char *)(name(row)) << ";" > << (t4_i32)(age(row)) << endl; > > // Find Bob again > index = hash_view.Find(name ["Bob"]); > > cout << "Bob is at index " << index << endl; > > row = hash_view[index]; > > // Bob's age is 20 again! > cout << "Changed info for Bob after a new Find(): " > << (const char *)(name(row)) << ";" > << (t4_i32)(age(row)) << endl; > > return 0; > } > From nathan.thompson-amato@dnamerican.com Mon Nov 5 16:17:06 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA13020 for ; Mon, 5 Nov 2001 16:16:55 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ta054853 for ; Mon, 5 Nov 2001 17:21:56 -0500 Message-ID: <3BE70E3F.3050307@dnamerican.com> Date: Mon, 05 Nov 2001 17:10:07 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Forgotten change after Find() on hash view References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: > Is it because a c4_Row is being used when you really wanted a c4_RowRef? I > would guess that the row is a copy of the data from the view, not the real > data. When you update the row you are updating the copy, not the real row. > > Try using a c4_Row to add the data, but a c4_RowRef when you get it in; > > row = hash_view[index]; > Good call! At least at first glance, this seems to work perfectly. I'd better stare at the docs a bit and make sure I'm understanding just what Refs are supposed to be for. Many thanks, Nathan From steve@improvision.com Mon Nov 5 16:28:52 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA14008 for ; Mon, 5 Nov 2001 16:28:52 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id WAA23334 for ; Mon, 5 Nov 2001 22:20:45 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 05 Nov 2001 22:22:37 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 22:22:34 +0000 Subject: Re: [Metakit] Forgotten change after Find() on hash view From: Stephen Baxter To: Nathan Thompson-Amato , Message-ID: In-Reply-To: <3BE70E3F.3050307@dnamerican.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: If I've got this right (and I've only been using Metakit for a week), a c4_Row is an independent view with one row, whereas a c4_RowRef is like a cursor within a view. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Nathan Thompson-Amato > Organization: D.N. American > Date: Mon, 05 Nov 2001 17:10:07 -0500 > To: metakit@equi4.com > Subject: Re: [Metakit] Forgotten change after Find() on hash view > > Stephen Baxter wrote: > >> Is it because a c4_Row is being used when you really wanted a c4_RowRef? I >> would guess that the row is a copy of the data from the view, not the real >> data. When you update the row you are updating the copy, not the real row. >> >> Try using a c4_Row to add the data, but a c4_RowRef when you get it in; >> >> row = hash_view[index]; >> > > > Good call! At least at first glance, this seems to work perfectly. I'd > better stare at the docs a bit and make sure I'm understanding just what > Refs are supposed to be for. > > Many thanks, > > Nathan > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Mon Nov 5 17:36:38 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA19371 for ; Mon, 5 Nov 2001 17:36:38 -0600 Received: from takkie.equi4.local ([195.121.236.56]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMCPAE01.52A; Tue, 6 Nov 2001 00:30:14 +0100 Received: from [10.0.1.12] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA5NSoI25925; Tue, 6 Nov 2001 00:28:51 +0100 From: Jean-Claude Wippler To: Cc: Subject: Re: [Metakit] Forgotten change after Find() on hash view Date: Tue, 6 Nov 2001 00:31:09 +0100 Message-Id: <20011105233109.25893@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >If I've got this right (and I've only been using Metakit for a week), a >c4_Row is an independent view with one row, whereas a c4_RowRef is like a >cursor within a view. Correct. In C++ terms, it's like "int a = b" versus "int& a = b". I just checked in a small change to allow "const c4_RowRef&" as arg in two places where a "c4_Cursor" was being used (while leaving the old API intact). It looks like the c4_Cursor is not really needed, one can do just about the same with a "const c4_RowRef&". It may be a bit confusing though: that's a reference to a C++ object which is represents a *reference* to a MK row... Longer term, I would like to further simplify the API. There's not terribly much left to simplify, but while trying to use "SWIG" to wrap MK, it became apparent that a number of classes can be left out of the API without limiting the functionality. Speaking of SWIG: I've done a very brief test with 1.3.9, and it *looks* like SWIG might now be up to the task of wrapping MK. There's still the issue of operator overloading (most of these are already redundant), and of function overloading (which is also not excessively frequent in the MK API). Well... a bit of heavy hacking led to (ta-dah!): >>> a basic test working in Java, Perl, Ruby, Python, and Tcl Just proof of concept for now, but if anyone wants to have a go at this, you can unpack the following in "/swig/", and try it (Makefile is for Linux): http://www.equi4.com/previews/test1.tar.gz I'm not sure if there is a SWIG interface for PHP. There is one for Lua, IIRC. The task of binding MK to new languages will take more effort than such a quick trial, and requires people with experience in those languages IMO to come up with bindings which work as naturally as the Tcl and Python ones do. Please let me know if you can help with any of this, or know others who might be interested. -jcw From beazley@cs.uchicago.edu Mon Nov 5 17:52:48 2001 Received: from laime.cs.uchicago.edu (laime.cs.uchicago.edu [128.135.11.244]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA20538; Mon, 5 Nov 2001 17:52:48 -0600 Received: from gargoyle.cs.uchicago.edu (gargoyle.cs.uchicago.edu [128.135.11.238]) by laime.cs.uchicago.edu (8.10.2/8.9.3) with ESMTP id fA5NkVs07111; Mon, 5 Nov 2001 17:46:32 -0600 (CST) Received: by gargoyle.cs.uchicago.edu (Postfix, from userid 32587) id 94F1731D55; Mon, 5 Nov 2001 17:46:30 -0600 (CST) From: David Beazley MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15335.9430.350668.215048@gargoyle.cs.uchicago.edu> Date: Mon, 5 Nov 2001 17:46:30 -0600 (CST) To: Jean-Claude Wippler Cc: , Subject: Re: [Metakit] Forgotten change after Find() on hash view In-Reply-To: <20011105233109.25893@mail.planet.nl> References: <20011105233109.25893@mail.planet.nl> X-Mailer: VM 6.71 under 21.1 "20 Minutes to Nikko" XEmacs Lucid (patch 2) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler writes: > > Speaking of SWIG: I've done a very brief test with 1.3.9, and it *looks* > like SWIG might now be up to the task of wrapping MK. There's still the > issue of operator overloading (most of these are already redundant), and > of function overloading (which is also not excessively frequent in the MK > API). Well... a bit of heavy hacking led to (ta-dah!): FYI, SWIG-1.3.10 (in development) allows overloaded operators to be wrapped (with the understanding that certain operators might not map cleanly onto the target language). If there are other C++ features that need to be addressed, I'd certainly be interested in that. Cheers, Dave From jcw@equi4.com Mon Nov 5 19:17:14 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA26122 for ; Mon, 5 Nov 2001 19:17:08 -0600 Received: from takkie.equi4.local ([195.121.234.186]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMCTXZ01.M72 for ; Tue, 6 Nov 2001 02:10:47 +0100 Received: from [10.0.1.12] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA619II26951 for ; Tue, 6 Nov 2001 02:09:28 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Catfish file format Date: Tue, 6 Nov 2001 02:11:24 +0100 Message-Id: <20011106011124.9829@mail.planet.nl> In-Reply-To: <1004977225.23523.15.camel@localhost.localdomain> References: <1004977225.23523.15.camel@localhost.localdomain> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jose Vasconcellos wrote: >I've written a Python version of catfish so I could browse >the database from Linux. It's a bit alpha at the moment but >it does work. I've also added an option to create or update >a database. My problem is that I'm not clear on the definition >of the 'info' view. Can anyone describe the 8 items that >make up this view? The source code is in the MK source distribution (in win/catfish/), does that help? It's been a while, I'd need to dig up the details. Essentially, these fields store some info about the disk (cluster size, capacity, etc), so that CF can report usage, type of medium, and such things. Not everything is used in the current CF utility, btw. -jcw From jcw@equi4.com Tue Nov 6 04:03:27 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA16083 for ; Tue, 6 Nov 2001 04:03:20 -0600 Received: from takkie.equi4.local ([195.121.234.96]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMDIA200.W0R; Tue, 6 Nov 2001 10:56:26 +0100 Received: from [10.0.1.12] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA69sxI28852; Tue, 6 Nov 2001 10:55:10 +0100 From: Jean-Claude Wippler To: metakit Cc: David Beazley Date: Tue, 6 Nov 2001 10:57:42 +0100 Message-Id: <20011106095742.6558@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Correction, SWIG Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: To follow up on the last post... SWIG 1.3.9 is definitely *much* closer to getting MK wrapped for use in Java, Perl, and Ruby. Contrary to my previous comments, which were partly based on the version that comes standard with SuSE Linux 7.1 (1.3a5, IIRC), very few changes are need to get a basic test working. I've made a diff showing the changes at http://www.equi4.com/previews/ test2.diff - in summary: - anonymous union (private, commented out) - operator casting to a type ("operator int()" etc) - "long long" cause compiles to choke (at least the Java wrap) - a few other changes to exclude some MK defs from wrapping Operators are properly skipped, default args appear to be ok, overloaded member appear to be skipped ok, and friends no longer cause errors. This second test has regressed w.r.t. the Ruby wrapper, which no longer compiles, I've not checked this further for now. Finally, two fixes had to be made (they were also in test1): - I'm inserting a call to Tcl_InitStubs in the Tcl wrapper, so that this thing compiles with stub support (-DUSE_TCL_STUBS), for things like binary Tcl version independence. The sed script applied to the generated C++ wrapper is: /interp == 0/i \ if (Tcl_InitStubs(interp, "8.1", 0) == NULL) return TCL_ERROR; - In Ruby 1.6.5, VALUE() needs to have the "..." elipsis removed, or it won't compile. The sed script for this is the highly sophisticated (2 changes): /\.\.\./s///g -jcw From terabaap@yumpee.org Tue Nov 6 04:53:39 2001 Received: from femail23.sdc1.sfba.home.com (femail23.sdc1.sfba.home.com [24.0.95.148]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA18732 for ; Tue, 6 Nov 2001 04:53:38 -0600 Received: from yumpee.org ([24.183.20.14]) by femail23.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011106104714.VFSW23889.femail23.sdc1.sfba.home.com@yumpee.org> for ; Tue, 6 Nov 2001 02:47:14 -0800 Message-ID: <3BE7BFEC.2080800@yumpee.org> Date: Tue, 06 Nov 2001 04:48:12 -0600 From: Manoj Plakal User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.5) Gecko/20011011 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Derived join views don't stay updated\ References: <675614530.20011106044250@users.sourceforge.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Any comments on whether this is a bug or expected behavior? Manoj Manoj Plakal wrote: > Is there any documentation on when derived views > stay updated and when they don't? > > I was testing deriving views by joining two views > and it looked like the derived view wouldn't > get updated if the join column values were changed > in the underlying views. > > Sample Python code below. > > Manoj > > ---------------------------------------------- > import metakit > > db = metakit.storage() > > v = db.getas('t[f:S,f2:S]') > v.append(f="FOO",f2="U2") > v.append(f="BAR",f2="Prodigy") > v.append(f="BAZ",f2="NIN") > metakit.dump(v, "orig") > > v2 = db.getas('t2[f:S]') > v2.append(f="BAZ") > v2.append(f="FOO") > metakit.dump(v2, "to join") > > v3 = v.join(v2, metakit.property( 'S', 'f' ) ) > metakit.dump(v3, "joined" ) > > v[0].f2="Dylan" # -- Changes U2 to Dylan in v3 > metakit.dump(v3, "updated join" ) > > v2[1].f="BAR" # -- Does not change v3, shouldn't it? > metakit.dump(v3, "updated join" ) > ---------------------------------------------- From mlh@idi.ntnu.no Tue Nov 6 05:36:03 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA21531 for ; Tue, 6 Nov 2001 05:36:02 -0600 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id MAA25192 for ; Tue, 6 Nov 2001 12:29:44 +0100 (MET) Message-ID: <01f301c166b6$556c5a90$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: References: <20011105233109.25893@mail.planet.nl> Subject: Re: [Metakit] Forgotten change after Find() on hash view Date: Tue, 6 Nov 2001 12:29:44 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Jean-Claude Wippler" [...] > Speaking of SWIG: I've done a very brief test with 1.3.9, and it *looks* > like SWIG might now be up to the task of wrapping MK. There's still the > issue of operator overloading (most of these are already redundant), and > of function overloading (which is also not excessively frequent in the MK > API). Well... a bit of heavy hacking led to (ta-dah!): > > >>> a basic test working in Java, Perl, Ruby, Python, and Tcl Wow! That's great :) > I'm not sure if there is a SWIG interface for PHP. It doesn't seem like it (although I can't get up swig.org right now). Pity. > There is one for Lua, IIRC. Another nice language... :) > -jcw -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.sf.net From jcw@equi4.com Wed Nov 7 17:07:12 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA02973 for ; Wed, 7 Nov 2001 17:07:12 -0600 Received: from takkie.equi4.local ([195.121.238.84]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMGD8H01.4OM; Thu, 8 Nov 2001 00:00:17 +0100 Received: from [10.0.1.13] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA7MxDI07896; Wed, 7 Nov 2001 23:59:13 +0100 From: Jean-Claude Wippler To: metakit Cc: Date: Thu, 8 Nov 2001 00:00:52 +0100 Message-Id: <20011107230052.3585@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] XML Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: If anyone is interested, here's a new trial to load XML data into MetaKit: http://www.equi4.com/previews/xml2.README http://www.equi4.com/previews/xml2.tar.gz The "mk4xml" tool in there is not optimized, and should not be taken as an indication of how efficient/inefficient things can be made. My first goal is to get more experience with having such data in MK and finding ways to effectively deal with it. Things like DOM would be nice (but take a much larger effort). Transferring data XML <-> MK <-> XML is a secondary issue for now, and IMO feasible efficiently in several ways. -jcw From baspey@yahoo.com Wed Nov 7 18:14:46 2001 Received: from web11505.mail.yahoo.com (web11505.mail.yahoo.com [216.136.172.37]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA06341 for ; Wed, 7 Nov 2001 18:14:45 -0600 Message-ID: <20011108000823.25007.qmail@web11505.mail.yahoo.com> Received: from [212.27.49.251] by web11505.mail.yahoo.com via HTTP; Wed, 07 Nov 2001 16:08:23 PST Date: Wed, 7 Nov 2001 16:08:23 -0800 (PST) From: popeye sailorman To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1502978244-1005178103=:23354" Subject: [Metakit] 3 bytes memory leak when creating c4_Storage on the heap Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1502978244-1005178103=:23354 Content-Type: text/plain; charset=us-ascii Hi, the following code is a simplification of what JCW wrote on the mailing list subject "Using a storage as member in an object": c4_Storage * MyStorage = new c4_Storage ("test.dat", true); delete MyStorage; Whenever I place this code anywhere in my program, I get a 3 bytes memory leak when execution is finished. I am using Visual C++, with the wxWindows framework, under Windows ME. When executing in Debug mode, wxWindows offers the facility of reporting memory leaks. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1502978244-1005178103=:23354 Content-Type: text/html; charset=us-ascii

Hi,

the following code is a simplification of what JCW wrote on the mailing list subject "Using a storage as member in an object":

c4_Storage * MyStorage = new c4_Storage ("test.dat", true);

delete MyStorage;

Whenever I place this code anywhere in my program, I get a 3 bytes memory leak when execution is finished.

I am using Visual C++, with the wxWindows framework, under Windows ME.

When executing in Debug mode, wxWindows offers the facility of reporting memory leaks.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1502978244-1005178103=:23354-- From jyl@best.com Wed Nov 7 23:43:50 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA16612; Wed, 7 Nov 2001 23:43:50 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id VAA26746; Wed, 7 Nov 2001 21:36:08 -0800 (PST) Message-ID: <3BEA27D3.9050304@best.com> Date: Wed, 07 Nov 2001 22:36:03 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit , Steve.Ball@zveno.com Subject: Re: [Metakit] XML References: <20011107230052.3585@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: That's very interesting. Of course it's also possible to represent the structure directly as a tree with the e4Graph package, which provides the capability to input XML directly and also to produce XML output. JYL Jean-Claude Wippler wrote: >If anyone is interested, here's a new trial to load XML data into MetaKit: > http://www.equi4.com/previews/xml2.README > http://www.equi4.com/previews/xml2.tar.gz > >The "mk4xml" tool in there is not optimized, and should not be taken as >an indication of how efficient/inefficient things can be made. My first >goal is to get more experience with having such data in MK and finding >ways to effectively deal with it. Things like DOM would be nice (but >take a much larger effort). Transferring data XML <-> MK <-> XML is a >secondary issue for now, and IMO feasible efficiently in several ways. > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From Steve.Ball@zveno.com Thu Nov 8 00:35:55 2001 Received: from waycool.zveno.com (waycool.zveno.com [210.8.44.221]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id AAA17873; Thu, 8 Nov 2001 00:35:50 -0600 From: Steve.Ball@zveno.com Received: from zveno.com (wakool.zveno.office [192.168.1.2]) by waycool.zveno.com (8.9.3/8.9.3) with ESMTP id RAA20814; Thu, 8 Nov 2001 17:46:24 +1100 Message-ID: <3BEA2A3F.E9BBD4AB@zveno.com> Date: Thu, 08 Nov 2001 17:46:23 +1100 Reply-To: Steve.Ball@zveno.com Organization: Zveno Pty Ltd X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-22mdk i686) X-Accept-Language: en MIME-Version: 1.0 To: Jacob Levy CC: Jean-Claude Wippler , metakit Subject: Re: [Metakit] XML References: <20011107230052.3585@mail.planet.nl> <3BEA27D3.9050304@best.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jacob, You wrote: > > That's very interesting. Of course it's also possible to represent the > structure directly as a tree with the e4Graph package, which provides > the capability to input XML directly and also to produce XML output. Oops... I should have mentioned your work to Jean-Claude earlier in our conversation. I had a look at e4graph a while ago. It's good stuff. My only thought was to suggest that a (Tcl)DOM interface be created for it, rather than an API of its own. Jacob: your thoughts? Cheers, Steve Ball > Jean-Claude Wippler wrote: > > >If anyone is interested, here's a new trial to load XML data into MetaKit: > > http://www.equi4.com/previews/xml2.README > > http://www.equi4.com/previews/xml2.tar.gz > > > >The "mk4xml" tool in there is not optimized, and should not be taken as > >an indication of how efficient/inefficient things can be made. My first > >goal is to get more experience with having such data in MK and finding > >ways to effectively deal with it. Things like DOM would be nice (but > >take a much larger effort). Transferring data XML <-> MK <-> XML is a > >secondary issue for now, and IMO feasible efficiently in several ways. > > > >-jcw > > > >_______________________________________________ > >metakit mailing list - metakit@equi4.com > >http://www.equi4.com/mailman/listinfo/metakit > > -- Steve Ball | XSLT Standard Library | Training & Seminars Zveno Pty Ltd | Web Tcl Complete | XML XSL Schemas http://www.zveno.com/ | TclXML TclDOM | Tcl, Web Development Steve.Ball@zveno.com +---------------------------+--------------------- Ph. +61 2 6242 4099 | Mobile (0413) 594 462 | Fax +61 2 6242 4099 From baspey@yahoo.com Thu Nov 8 01:06:32 2001 Received: from web11505.mail.yahoo.com (web11505.mail.yahoo.com [216.136.172.37]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA18629 for ; Thu, 8 Nov 2001 01:06:31 -0600 Message-ID: <20011108070007.79049.qmail@web11505.mail.yahoo.com> Received: from [212.27.48.59] by web11505.mail.yahoo.com via HTTP; Wed, 07 Nov 2001 23:00:07 PST Date: Wed, 7 Nov 2001 23:00:07 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1447379016-1005202807=:79040" Subject: [Metakit] 3 bytes memory leak when creating c4_Storage on the heap Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1447379016-1005202807=:79040 Content-Type: text/plain; charset=us-ascii I apologize for the "popeye sailorman" nickname Let's say my mail account was misconfigured No offense intended ;-) --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1447379016-1005202807=:79040 Content-Type: text/html; charset=us-ascii

I apologize for the "popeye sailorman" nickname

Let's say my mail account was misconfigured

No offense intended   ;-)


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1447379016-1005202807=:79040-- From jcw@equi4.com Thu Nov 8 03:22:24 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA22526 for ; Thu, 8 Nov 2001 03:22:24 -0600 Received: from takkie.equi4.local ([195.121.233.134]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMH5PS00.NR0; Thu, 8 Nov 2001 10:15:28 +0100 Received: from [10.0.1.14] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA89ENI10063; Thu, 8 Nov 2001 10:14:24 +0100 From: Jean-Claude Wippler To: metakit Cc: Subject: Re: [Metakit] XML Date: Thu, 8 Nov 2001 10:08:27 +0100 Message-Id: <20011108090827.26763@mail.planet.nl> In-Reply-To: <3BEA27D3.9050304@best.com> References: <3BEA27D3.9050304@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >That's very interesting. Of course it's also possible to represent the >structure directly as a tree with the e4Graph package, which provides >the capability to input XML directly and also to produce XML output. Yes, I mentioned e4Graph in the README, but let me do better than that: http://sf.net/projects/e4graph My apologies - I should also have mentioned it in the post to this list. -jcw From jcw@equi4.com Thu Nov 8 03:22:53 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA22542; Thu, 8 Nov 2001 03:22:52 -0600 Received: from takkie.equi4.local ([195.121.233.134]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMH5QK00.BR8; Thu, 8 Nov 2001 10:15:56 +0100 Received: from [10.0.1.14] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA89ETI10067; Thu, 8 Nov 2001 10:14:29 +0100 From: Jean-Claude Wippler To: , Subject: Re: [Metakit] 3 bytes memory leak when creating c4_Storage on the heap Date: Thu, 8 Nov 2001 10:16:30 +0100 Message-Id: <20011108091630.13175@mail.planet.nl> In-Reply-To: <20011108000823.25007.qmail@web11505.mail.yahoo.com> References: <20011108000823.25007.qmail@web11505.mail.yahoo.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Wed, Nov 7, 2001, popeye sailorman , also known as Pascal Baspeyras wrote: >c4_Storage * MyStorage = new c4_Storage ("test.dat", true); >delete MyStorage; > >Whenever I place this code anywhere in my program, I get a 3 bytes memory >leak when execution is finished. > >I am using Visual C++, with the wxWindows framework, under Windows ME. > >When executing in Debug mode, wxWindows offers the facility of reporting >memory leaks. Yes, I use MSVC6/MFC/Debug once in a while to do the same. The "leak" you report is, ehm... intentional. It is caused by lines 173 and 178 in src/string.cpp - where a static copy of an empty string is kept for efficiency (sharing it and bumping refcounts instead of allocating a new one). The bad news is that this will not be fixed. The good news, is that the 3-byte leak does not ever increase or repeat: no matter how much you use and allocate things in MK, there will be at most *one* such 3-byte object leak in the whole application. -jcw From j.p.fletcher@aston.ac.uk Thu Nov 8 06:21:57 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA28810 for ; Thu, 8 Nov 2001 06:21:56 -0600 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 161o5v-00067x-00 for metakit@equi4.com; Thu, 08 Nov 2001 12:15:43 +0000 From: "John Fletcher" To: metakit Date: Thu, 8 Nov 2001 12:13:34 -0000 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] XML Priority: normal In-reply-to: <20011108090827.26763@mail.planet.nl> References: <3BEA27D3.9050304@best.com> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler To: metakit Copies to: Subject: Re: [Metakit] XML Date sent: Thu, 8 Nov 2001 10:08:27 +0100 > Jacob Levy wrote: > > >That's very interesting. Of course it's also possible to represent the > >structure directly as a tree with the e4Graph package, which provides > >the capability to input XML directly and also to produce XML output. > > Yes, I mentioned e4Graph in the README, but let me do better than that: > http://sf.net/projects/e4graph > My apologies - I should also have mentioned it in the post to this list. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > I have been experimenting with the E4Graph XML output. The published version includes in the output atributes which enable the XML to be reread by E4Graph to reconstruct the graph even if there are back refences. This means that if the tree was originally built from XML the output will not be the same as the input. I want to be able to store and retrieve MathML so that I need to turn off the extra attributes. I have been using a modified version of the E4Graph XML for this, with an extra parameter to control the output. I was not entirely happy with this because of the possibility of back references. It may well be that the direct Metakit approach will give me a different way of achieving the same results.# John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From baspey@yahoo.com Thu Nov 8 08:42:13 2001 Received: from web11508.mail.yahoo.com (web11508.mail.yahoo.com [216.136.172.40]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA01678 for ; Thu, 8 Nov 2001 08:42:13 -0600 Message-ID: <20011108143547.27962.qmail@web11508.mail.yahoo.com> Received: from [213.228.40.235] by web11508.mail.yahoo.com via HTTP; Thu, 08 Nov 2001 06:35:47 PST Date: Thu, 8 Nov 2001 06:35:47 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1099233020-1005230147=:27065" Subject: [Metakit] 3 bytes memory leak when creating c4_Storage on the heap Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1099233020-1005230147=:27065 Content-Type: text/plain; charset=us-ascii Jean-Claude Wippler wrote: >The bad news is that this will not be fixed. The good news, is that the >3-byte leak does not ever increase or repeat: no matter how much you use >and allocate things in MK, there will be at most *one* such 3-byte object >leak in the whole application. I had indeed noticed that the memory leak was always of 3 bytes, whatever the number of strorage objects allocated on the heap. Let's hope the user won't close and relaunch the application one billion times ;-) You hack too much, Jean-Claude. You hack too much, but your database is great. I definitively adopt it. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1099233020-1005230147=:27065 Content-Type: text/html; charset=us-ascii

Jean-Claude Wippler wrote:

>The bad news is that this will not be fixed.  The good news, is that the
>3-byte leak does not ever increase or repeat: no matter how much you use
>and allocate things in MK, there will be at most *one* such 3-byte object
>leak in the whole application.

I had indeed noticed that the memory leak was always of 3 bytes, whatever the number

of strorage objects allocated on the heap.

Let's hope the user won't close and relaunch the application one billion times  ;-)

You hack too much, Jean-Claude.

You hack too much, but your database is great.

I definitively adopt it.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1099233020-1005230147=:27065-- From jyl@best.com Thu Nov 8 09:26:51 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA03749; Thu, 8 Nov 2001 09:26:49 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id HAA22847; Thu, 8 Nov 2001 07:20:09 -0800 (PST) Message-ID: <3BEAB0B3.1050709@best.com> Date: Thu, 08 Nov 2001 08:20:03 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit , Steve.Ball@zveno.com Subject: Re: [Metakit] XML References: <3BEA27D3.9050304@best.com> <20011108090827.26763@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks JC I'm curious as to the motivation for mkXML and what you can do with it that's hard or impossible with e4Graph. JYL Jean-Claude Wippler wrote: >Jacob Levy wrote: > >>That's very interesting. Of course it's also possible to represent the >>structure directly as a tree with the e4Graph package, which provides >>the capability to input XML directly and also to produce XML output. >> > >Yes, I mentioned e4Graph in the README, but let me do better than that: > http://sf.net/projects/e4graph >My apologies - I should also have mentioned it in the post to this list. > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From jcw@equi4.com Fri Nov 9 12:39:36 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA09250 for ; Fri, 9 Nov 2001 12:39:35 -0600 Received: from takkie.equi4.local ([195.121.234.244]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMJQ6A00.LHP; Fri, 9 Nov 2001 19:32:34 +0100 Received: from [10.0.1.15] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA9IVJI19752; Fri, 9 Nov 2001 19:31:40 +0100 From: Jean-Claude Wippler Cc: metakit , Subject: Re: [Metakit] XML Date: Fri, 9 Nov 2001 19:31:41 +0100 Message-Id: <20011109183141.5051@mail.planet.nl> In-Reply-To: <3BEAB0B3.1050709@best.com> References: <3BEAB0B3.1050709@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >I'm curious as to the motivation for mkXML and what you can do with it >that's hard or impossible with e4Graph. That was not my focus for doing this. I wanted to explore the issues when the data is "relatively tabular", yet encoded as XML. Things which have structure, although substantial parts are really tables (and possibly subtables). By trying to force XML trees into plain MK views, instead of a generalized node structure, one has the ability to gradually re-structure data further (manually, or perhaps even automatically). It all depends of the context, clearly. There are many ways to use MK, I guess what I'm trying to find out here is to what extent one can reconstruct tabular (and therefore very efficient columnar) formats. Note that this is just exploration - no fancy coding at all, I just dusted off the 3-year old mk4xml utility I had, and threw a few new tricks at that dog... There are *still* more ways to do things, such as the "^" recursive structure definition which was added in 2.3/2.4, i.e. things like "a[b:S,c:I,d:[^]]" if memory serves me well. The big drawback of that, as in hierarchical file systems, is that nodes can only be identified by a variable-length "path" (the path could be an efficient short vector of ints, but it's variable-sized). -jcw From jyl@best.com Fri Nov 9 17:14:27 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA23789; Fri, 9 Nov 2001 17:14:27 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id PAA29378; Fri, 9 Nov 2001 15:06:58 -0800 (PST) Message-ID: <3BEC6F7F.2040300@best.com> Date: Fri, 09 Nov 2001 16:06:23 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit , Steve.Ball@zveno.com Subject: Re: [Metakit] XML References: <3BEAB0B3.1050709@best.com> <20011109183141.5051@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ummm.. Can you explain a bit about the ^ notation? What does it do? --JYL Jean-Claude Wippler wrote: >Jacob Levy wrote: > >>I'm curious as to the motivation for mkXML and what you can do with it >>that's hard or impossible with e4Graph. >> > >That was not my focus for doing this. I wanted to explore the issues >when the data is "relatively tabular", yet encoded as XML. Things which >have structure, although substantial parts are really tables (and >possibly subtables). > >By trying to force XML trees into plain MK views, instead of a >generalized node structure, one has the ability to gradually re-structure >data further (manually, or perhaps even automatically). It all depends >of the context, clearly. > >There are many ways to use MK, I guess what I'm trying to find out here >is to what extent one can reconstruct tabular (and therefore very >efficient columnar) formats. > >Note that this is just exploration - no fancy coding at all, I just >dusted off the 3-year old mk4xml utility I had, and threw a few new >tricks at that dog... > >There are *still* more ways to do things, such as the "^" recursive >structure definition which was added in 2.3/2.4, i.e. things like >"a[b:S,c:I,d:[^]]" if memory serves me well. The big drawback of that, >as in hierarchical file systems, is that nodes can only be identified by >a variable-length "path" (the path could be an efficient short vector of >ints, but it's variable-sized). > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From tito@apple.com Sun Nov 11 19:11:35 2001 Received: from mail-out1.apple.com (mail-out1.apple.com [17.254.0.52]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA20165 for ; Sun, 11 Nov 2001 19:11:35 -0600 Received: from mailgate2.apple.com (A17-129-100-225.apple.com [17.129.100.225]) by mail-out1.apple.com (8.11.3/8.11.3) with ESMTP id fAC14uu08197 for ; Sun, 11 Nov 2001 17:04:56 -0800 (PST) Received: from scv2.apple.com (scv2.apple.com) by mailgate2.apple.com (Content Technologies SMTPRS 4.2.1) with ESMTP id for ; Sun, 11 Nov 2001 17:04:55 -0800 Received: from localhost ([17.254.160.6]) by scv2.apple.com (8.11.3/8.11.3) with ESMTP id fAC14sH17360 for ; Sun, 11 Nov 2001 17:04:54 -0800 (PST) Date: Sun, 11 Nov 2001 17:04:54 -0800 Mime-Version: 1.0 (Apple Message framework v501) Content-Type: multipart/alternative; boundary=Apple-Mail-2--697392676 From: Tito Ciuro To: metakit@equi4.com Message-Id: <48711456-D709-11D5-9F53-00039310A0E8@apple.com> X-Mailer: Apple Mail (2.501) Subject: [Metakit] Performance questions about MetaKit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2--697392676 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hello, I have a few quick questions regarding MetaKit: 1) Is there a limit of columns per view? 2) Is there a performance hit after any specific number of columns? 3) Is there a performance hit after any specific number of rows? 4) What is the biggest int MetaKit can encode? (long long?) Thanks! -- Tito ____________________________________________ "By the time they had diminished from 50 to 8, the other dwarves began to suspect 'Hungry' ..." -- Gary Larson, "The Far Side" --Apple-Mail-2--697392676 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hello, I have a few quick questions regarding MetaKit: 1) Is there a limit of columns per view? 2) Is there a performance hit after any specific number of columns? 3) Is there a performance hit after any specific number of rows? 4) What is the biggest int MetaKit can encode? (long long?) Thanks! -- Tito ____________________________________________ "By the time they had diminished from 50 to 8, the other dwarves began to suspect 'Hungry' ..." -- Gary Larson, "The Far Side" --Apple-Mail-2--697392676-- From baspey@yahoo.com Sun Nov 11 22:35:15 2001 Received: from web11502.mail.yahoo.com (web11502.mail.yahoo.com [216.136.172.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id WAA28522 for ; Sun, 11 Nov 2001 22:35:15 -0600 Message-ID: <20011112042836.96786.qmail@web11502.mail.yahoo.com> Received: from [212.27.48.193] by web11502.mail.yahoo.com via HTTP; Sun, 11 Nov 2001 20:28:36 PST Date: Sun, 11 Nov 2001 20:28:36 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-641453618-1005539316=:96710" Subject: [Metakit] Strange behaviours using hash views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-641453618-1005539316=:96710 Content-Type: text/plain; charset=us-ascii Hello Jean-Claude, I observe quite strange (and annoying) behaviours when I try to use a hash view. I'm using the latest cvs under Windows. For the test, I parse a text file containing 7000 entries. I build my database quite conventionaly (I hope): c4_Storage MyDatabase = ("MyDatabase.dat", true); c4_View vData = MyStorage .GetAs("Data[Field1:S,Field2:S]"); c4_View vMap = MyStorage .GetAs("Map[_H:I,_R:I]"); c4_View vHash = vData.Hash(vMap, 2); c4_StringProp pField1("Field1"); c4_StringProp pField2("Field2"); c4_Row Row; For each entry read from the text file, I do: pField1(Row) = "found text"; pField2(Row) = "next found text"; vHash.Add(Row); And when finished, I commit. THE FIRST WEIRD BEHAVIOR: I run the program once to construct my database file. OK But then, every time I rerun the program, the file gets bigger (quite bigger), getting fat from the bottom, though the number of rows stays the same (normal), as I can see using KitViewer. Every two times I rerun the program, the map view doubles the number of its rows, which increases of course the size of the database file, but the size of the file also increases when the map view does not double its rows... SECOND WEIRD BEHAVIOUR: I read (just read) my text file, and for each entry, search (just search) for the row position of the corresponding item in the database: int RowIndex = vHash.Find(pField1["blablabla"]); And the big O cost I observe is nothing like 1. Strangely, each search takes more time than the preceding, and it takes several seconds to find the last 500 items... (I have a Celeron 800) THIRD WEIRD BEHAVIOUR: If I compare the time it takes to build the database in the first place, with the time it takes to search each item (see above), well... the building is about 10 times faster than the searching (may be normal due to disk accesses, but is it really ?) Thanks for paying attention to my distress. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-641453618-1005539316=:96710 Content-Type: text/html; charset=us-ascii

Hello Jean-Claude,

I observe quite strange (and annoying) behaviours when I try to use a hash view.

I'm using the latest cvs under Windows.

For the test, I parse a text file containing 7000 entries.

I build my database quite conventionaly (I hope):

c4_Storage MyDatabase = ("MyDatabase.dat", true);

c4_View vData = MyStorage .GetAs("Data[Field1:S,Field2:S]");

c4_View vMap = MyStorage .GetAs("Map[_H:I,_R:I]");

c4_View vHash = vData.Hash(vMap, 2);

c4_StringProp pField1("Field1");

c4_StringProp pField2("Field2");

c4_Row Row;

 

For each entry read from the text file, I do:

pField1(Row) = "found text";

pField2(Row) = "next found text";

vHash.Add(Row);

And when finished, I commit.

 

THE FIRST WEIRD BEHAVIOR:

I run the program once to construct my database file.

OK

But then, every time I rerun the program, the file gets bigger (quite bigger), getting fat from the bottom, though the number of rows stays the same (normal), as I can see using KitViewer.

Every two times I rerun the program, the map view doubles the number of its rows, which increases of course the size of the database file, but the size of the file also increases when the map view does not double its rows...

SECOND WEIRD BEHAVIOUR:

I read (just read) my text file, and for each entry, search (just search) for the row position of the corresponding item in the database:

int RowIndex = vHash.Find(pField1["blablabla"]);

And the big O cost I observe is nothing like 1.

Strangely, each search takes more time than the preceding, and it takes several seconds to find the last 500 items... (I have a Celeron 800)

THIRD WEIRD BEHAVIOUR:

If I compare the time it takes to build the database in the first place, with the time it takes to search each item (see above), well... the building is about 10 times faster than the searching (may be normal due to disk accesses, but is it really ?)

 

Thanks for paying attention to my distress.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-641453618-1005539316=:96710-- From baspey@yahoo.com Sun Nov 11 23:15:13 2001 Received: from web11504.mail.yahoo.com (web11504.mail.yahoo.com [216.136.172.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA31038 for ; Sun, 11 Nov 2001 23:15:13 -0600 Message-ID: <20011112050833.54799.qmail@web11504.mail.yahoo.com> Received: from [212.27.48.102] by web11504.mail.yahoo.com via HTTP; Sun, 11 Nov 2001 21:08:33 PST Date: Sun, 11 Nov 2001 21:08:33 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1496710524-1005541713=:54305" Subject: [Metakit] Strange behaviours using hash views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1496710524-1005541713=:54305 Content-Type: text/plain; charset=us-ascii Oooops... In my precedent posting, please don't pay attention to the third strange behaviour I describe and which simply is a consequence of the increasing search time described in strange behaviour n° 2. And I also apologize for the too long lines. Please have mercy on a poor fellow like me. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1496710524-1005541713=:54305 Content-Type: text/html; charset=us-ascii

Oooops...

In my precedent posting, please don't pay attention to the third

strange behaviour I describe and which simply is a consequence

of the increasing search time described in strange behaviour n° 2.

And I also apologize for the too long lines.

Please have mercy on a poor fellow like me.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1496710524-1005541713=:54305-- From baspey@yahoo.com Mon Nov 12 09:07:48 2001 Received: from web11507.mail.yahoo.com (web11507.mail.yahoo.com [216.136.172.39]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA22164 for ; Mon, 12 Nov 2001 09:07:48 -0600 Message-ID: <20011112150107.87023.qmail@web11507.mail.yahoo.com> Received: from [212.27.48.216] by web11507.mail.yahoo.com via HTTP; Mon, 12 Nov 2001 07:01:07 PST Date: Mon, 12 Nov 2001 07:01:07 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-282460218-1005577267=:86790" Subject: [Metakit] Strange behaviours using hash views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-282460218-1005577267=:86790 Content-Type: text/plain; charset=us-ascii Inspecting Find() with the debugger confirmed that it uses a linear search (from first row to the last one if necessary) in spite of the fact that the view is a hash view. Nonetheless I suppose using Find() is the only possibility, because Search() and Locate() are intended for binary search on sorted views (unless I'm wrong). --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-282460218-1005577267=:86790 Content-Type: text/html; charset=us-ascii

Inspecting Find() with the debugger confirmed that it uses a linear search (from first row to the last one if necessary)

in spite of the fact that the view is a hash view.

Nonetheless I suppose using Find() is the only possibility, because Search() and Locate() are intended for binary

search on sorted views (unless I'm wrong).


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-282460218-1005577267=:86790-- From jcw@equi4.com Mon Nov 12 09:53:12 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA25044 for ; Mon, 12 Nov 2001 09:53:11 -0600 Received: from takkie.equi4.local ([195.121.235.2]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMP2GN01.0VU for ; Mon, 12 Nov 2001 16:45:59 +0100 Received: from [10.0.1.21] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fACFjXI06704 for ; Mon, 12 Nov 2001 16:45:33 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Strange behaviours using hash views Date: Mon, 12 Nov 2001 16:47:26 +0100 Message-Id: <20011112154726.23523@mail.planet.nl> In-Reply-To: <20011112042836.96786.qmail@web11502.mail.yahoo.com> References: <20011112042836.96786.qmail@web11502.mail.yahoo.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Pascal Baspeyras wrote: >c4_View vData = MyStorage .GetAs("Data[Field1:S,Field2:S]"); >c4_View vMap = MyStorage .GetAs("Map[_H:I,_R:I]"); >c4_View vHash = vData.Hash(vMap, 2); [...] >But then, every time I rerun the program, the file gets bigger (quite >bigger), getting fat from the bottom, though the number of rows stays the >same (normal), as I can see using KitViewer. This one I can't quite figure out - still thinking... >int RowIndex = vHash.Find(pField1["blablabla"]); You specified a 2-key hash (the "2" above). MK cannot locate items with hashing when given one field, hence it resorts to linear search. -jcw From baspey@yahoo.com Mon Nov 12 11:44:35 2001 Received: from web11507.mail.yahoo.com (web11507.mail.yahoo.com [216.136.172.39]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id LAA00640 for ; Mon, 12 Nov 2001 11:44:34 -0600 Message-ID: <20011112173753.17702.qmail@web11507.mail.yahoo.com> Received: from [213.228.42.101] by web11507.mail.yahoo.com via HTTP; Mon, 12 Nov 2001 09:37:53 PST Date: Mon, 12 Nov 2001 09:37:53 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1134683454-1005586673=:17251" Subject: [Metakit] Strange behaviours using hash views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1134683454-1005586673=:17251 Content-Type: text/plain; charset=us-ascii Hello Jean-Claude, Please stop thinking because that's no use... I made a trip into twilight zone: the behaviour I described I can't reproduce anymore. Maybe the mere fact that I shut down my computer for the night. I wanted to join to my mail the 2 incriminated files, but now the second rerun still builds a bigger file, but because each row is present 2 times (I didn't know it was possible with a hash view, thought that if an entry already existed, it replaced the precedent one). As for the find(), you are off course right. Thanks a lot for your help, and who knows, if I can reproduce the pathological behaviour, I'll tell you about. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1134683454-1005586673=:17251 Content-Type: text/html; charset=us-ascii

Hello Jean-Claude,

Please stop thinking because that's no use...

I made a trip into twilight zone: the behaviour I described I can't

reproduce anymore.

Maybe the mere fact that I shut down my computer for the night.

I wanted to join to my mail the 2 incriminated files, but now the second

rerun still builds a bigger file, but because each row is present 2 times

(I didn't know it was possible with a hash view, thought that if an entry

already existed, it replaced the precedent one).

As for the find(), you are off course right.

Thanks a lot for your help, and who knows, if I can reproduce the

pathological behaviour, I'll tell you about.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1134683454-1005586673=:17251-- From jcw@equi4.com Wed Nov 14 17:16:02 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA20844; Wed, 14 Nov 2001 17:16:01 -0600 Received: from takkie.equi4.local ([195.121.235.10]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMTCAF01.G7U; Thu, 15 Nov 2001 00:08:39 +0100 Received: from [10.0.1.23] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAEN60004372; Thu, 15 Nov 2001 00:06:10 +0100 From: Jean-Claude Wippler To: , Subject: Re: [Metakit] Performance questions about MetaKit Date: Thu, 15 Nov 2001 00:08:43 +0100 Message-Id: <20011114230843.7262@mail.planet.nl> In-Reply-To: <48711456-D709-11D5-9F53-00039310A0E8@apple.com> References: <48711456-D709-11D5-9F53-00039310A0E8@apple.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, >1) Is there a limit of columns per view? Not really. Thousands have been used. Have you tried? >2) Is there a performance hit after any specific number of columns? >3) Is there a performance hit after any specific number of rows? Not really. There may be slow-downs, quadratic slow-downs even. But there's probably no way to answer this in general. It depends. MK can soar and crawl. I'm not trying to give a cheeky answer... it really depends. There are all sorts of surprises, both in what bogs down, and in what one can do to fix it. >4) What is the biggest int MetaKit can encode? (long long?) Yes, see "mk4.h". Note that 64-bit ints are a separate (non-autosizing) type. -jcw From jcw@equi4.com Wed Nov 14 17:35:18 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA21933 for ; Wed, 14 Nov 2001 17:35:18 -0600 Received: from takkie.equi4.local ([195.121.235.194]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMTD6K01.7L1; Thu, 15 Nov 2001 00:27:56 +0100 Received: from [10.0.1.23] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAENPG004742; Thu, 15 Nov 2001 00:25:22 +0100 From: Jean-Claude Wippler Cc: metakit , Subject: Re: [Metakit] XML Date: Thu, 15 Nov 2001 00:27:55 +0100 Message-Id: <20011114232755.19435@mail.planet.nl> In-Reply-To: <3BEC6F7F.2040300@best.com> References: <3BEC6F7F.2040300@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >>There are *still* more ways to do things, such as the "^" recursive >>structure definition which was added in 2.3/2.4, i.e. things like >>"a[b:S,c:I,d:[^]]" if memory serves me well. [...] >Ummm.. Can you explain a bit about the ^ notation? What does it do? It recurses, i.e. re-uses the definition of its parent view: "a[b:S,c:I,d:[^]]" "a[b:S,c:I,d:[b:S,c:I,d:[^]]]" "a[b:S,c:I,d:[b:S,c:I,d:[b:S,c:I,d:[^]]]]" etc... In other words, this defines a tree of same-typed nodes, each with a variable number of children. It's not going to be particularly efficient in things like binary trees, because columns/subcolumns end up being short and numerous. The big surprise (for me, anyway) was that on-the-fly restructuring continued to work without coding anything special for it. See the examples/selfref.py demo. -jcw From steven.leblanc@mayahtt.com Fri Nov 16 15:42:17 2001 Received: from qmail.mayahtt.ca (mayahtt.com [199.84.144.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA21059 for ; Fri, 16 Nov 2001 15:42:17 -0600 Received: from _HOSTNAME_ (indigo23.mayahtt.ca [192.9.201.194]) by qmail.mayahtt.ca (8.11.2/8.11.2) with SMTP id fAGLZ4X08078 for ; Fri, 16 Nov 2001 16:35:04 -0500 Received: by _HOSTNAME_ (sSMTP sendmail emulation); Fri, 16 Nov 2001 16:35:04 -0500 Date: Fri, 16 Nov 2001 16:35:04 -0500 From: Steve Leblanc To: metakit@equi4.com Message-ID: <20011116163504.B267130@mayahtt.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Organization: Maya Heat Transfer Technologies Subject: [Metakit] Problem on SGI and HP Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi I'm really interested in using Metakit for some projects at work, but it's got to work on all our platforms. So, the first thing I did was run the regression test on everything we have. Windows, HP-UX, and Solaris worked but there is a problem on IRIX and AIX. On both platforms TestMapped goes into an infinite loop. This seems to happen during the third call to v3.Add() when it gets down to HashViewer::LookDict. It gets stuck in the for() loop and the value of i just cycles between 4 different values. The loop never meets the termination conditions. I really don't have much time to investigate this, but I'd love to use MK so I'll play around with it a bit. I'll admit that I don't really know how MK works so it's going to be tough to debug. I saw a comment about an earlier infinite loop problem with the hashing stuff, so maybe some- one can give me a hint about where to start. I'd hate to have to trash MK because of this. Thanks -- Steve Leblanc Maya Heat Transfer Technologies Ltd. Phone: (514) 369-5706 Fax: (514) 369-4200 From jcw@equi4.com Fri Nov 16 16:59:16 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA25826; Fri, 16 Nov 2001 16:59:15 -0600 Received: from takkie.equi4.local ([195.121.236.94]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMX0V300.V03; Fri, 16 Nov 2001 23:52:15 +0100 Received: from [10.0.1.26] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAGMnp019669; Fri, 16 Nov 2001 23:50:02 +0100 From: Jean-Claude Wippler To: , metakit Subject: Re: [Metakit] Problem on SGI and HP Date: Fri, 16 Nov 2001 23:51:36 +0100 Message-Id: <20011116225136.13230@mail.planet.nl> In-Reply-To: <20011116163504.B267130@mayahtt.com> References: <20011116163504.B267130@mayahtt.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Steve Leblanc wrote: >I'm really interested in using Metakit for some projects at work, but >it's got to work on all our platforms. So, the first thing I did was >run the regression test on everything we have. Windows, HP-UX, and >Solaris worked but there is a problem on IRIX and AIX. On both platforms >TestMapped goes into an infinite loop. This seems to happen during the >third call to v3.Add() when it gets down to HashViewer::LookDict. >It gets stuck in the for() loop and the value of i just cycles between >4 different values. The loop never meets the termination conditions. Please make sure you use the latest code in CVS. If you comment out TestMapped, does the rest run properly? if so, we could start narrowing it down to things like word sizes, signed/unsigned, and such... Note that the hash view is relatively young. If all other tests run ok, then you can safely assume that MK is in pretty good shape. I'll walk through the code and re-check things, if you could please tell me a bit about architecture details, especially how they differ from the Win32/ Linux systems (x86), since I know those best. Quick thought: could this be as mundane as "just" an endian-ness issue? Btw, HP-UX is good news. It hasn't always been like that - what system/ os/machine revision would that be? -jcw From baspey@yahoo.com Sun Nov 18 21:22:57 2001 Received: from web11504.mail.yahoo.com (web11504.mail.yahoo.com [216.136.172.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA01747 for ; Sun, 18 Nov 2001 21:22:57 -0600 Message-ID: <20011119031550.51194.qmail@web11504.mail.yahoo.com> Received: from [62.147.36.130] by web11504.mail.yahoo.com via HTTP; Sun, 18 Nov 2001 19:15:50 PST Date: Sun, 18 Nov 2001 19:15:50 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-563452070-1006139750=:49291" Subject: [Metakit] Using hash value as a reference (??) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-563452070-1006139750=:49291 Content-Type: text/plain; charset=us-ascii Hi everybody, In my database, the keys in my first view are movie titles. The second view is a dictionary build from the words contained in these titles. For each word in the dictionary, there is a subview listing all the titles containing this word. Of course, for saving space, the dictionary view only records the index of the movie title. But here is my problem: for efficiency reasons, the main view (containing the movie titles) has to be ordered. Thus the index of a movie is very likely to change after inserting other movies. The solution I have thought of is to construct the view containing the movie titles as a blocked/hash/ordered view. All right. And then, in the dictionary view, the movies would be listed not with their index position, but with their hash value instead. Does this sound crazy or totally stupid to you ? In order to do so, it seems rather easy to find what the hash value of a movie is, but what might be more difficult is to reach a movie by directly giving its hash value. Thank you for giving your opinion. --------------------------------- Do You Yahoo!? Find the one for you at Yahoo! Personals. --0-563452070-1006139750=:49291 Content-Type: text/html; charset=us-ascii

Hi everybody,

In my database, the keys in my first view are movie titles.

The second view is a dictionary build from the words contained in these titles.

For each word in the dictionary, there is a subview listing all the titles containing

this word.

Of course, for saving space, the dictionary view only records the index of the movie title.

But here is my problem: for efficiency reasons, the main view (containing the movie titles)

has to be ordered.

Thus the index of a movie is very likely to change after inserting other movies.

The solution I have thought of is to construct the view containing the movie titles as a

blocked/hash/ordered view. All right. And then, in the dictionary view, the movies would

be listed not with their index position, but with their hash value instead.

Does this sound crazy or totally stupid to you ?

In order to do so, it seems rather easy to find what the hash value of a movie is, but

what might be more difficult is to reach a movie by directly giving its hash value.

Thank you for giving your opinion.


Do You Yahoo!?
Find the one for you at Yahoo! Personals. --0-563452070-1006139750=:49291-- From mitch.chapman@bioreason.com Mon Nov 19 13:10:13 2001 Received: from gateway.bioreason.com (www.bioreason.com [207.108.245.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA14396 for ; Mon, 19 Nov 2001 13:10:13 -0600 Received: by gateway.bioreason.com (Postfix, from userid 200) id B455C2237; Mon, 19 Nov 2001 12:01:28 +0000 (/etc/localtime) Received: by bioreason.com via smwrap Version 1.0 id smwrapVdMpRw; Mon Nov 19 12:01:09 2001 by mailhost.bioreason.com (Postfix) with ESMTP id 5CD9F3577 for ; Mon, 19 Nov 2001 12:01:42 +0000 (/etc/localtime) Message-ID: <3BF958AB.F9AEDFE5@bioreason.com> Date: Mon, 19 Nov 2001 12:08:27 -0700 From: Mitch Chapman Organization: Bioreason, Inc. MIME-Version: 1.0 To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Why does this fail? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm new to Metakit, and am having problems with apparent memory corruption when using Metakit 2.4.1 with either Python 2.1 or Python 2.2b2. The script on which I'm working sometimes segfaults with a gdb stack trace which starts in the Python 2.x garbage collector. I tried to produce a simple problem-demonstration script to include with this message, but succeeded only in getting a different runtime error -- a SystemError. It must be Monday :) Anyway, the demonstration script appears below, together with its output under both Python2.1 and Python2.2b2 (Metakit 2.4.1 having been compiled and installed separately for each). Anybody have any ideas what's going on? Feel free to ask for more information about configurations, versions, etc., than is provided in the script output. Thanks for help. -- Mitch Chapman Mitch.Chapman@bioreason.com ----------------------------------------------------------------------- #!/usr/bin/env python """Demonstrate some odd SystemErrors with Metakit and Python.""" import gc, metakit import sys, os # Show the current platform: osVersion = open("/etc/mandrake-release").read().strip() print "Operating System:", osVersion print " ".join(os.uname()) print print "Python:", sys.version print print "Metakit:", metakit.version print def MKRecord(**kw): return kw db = metakit.storage() version = db.getas("version[ident:S]") # Uncomment this to trigger a SystemError further downstream: del version[:] version.setsize(1) # This causes a 'SystemError: null argument to internal routine' # exception. value = MKRecord(ident="Version 1.0") version[0] = value db.commit() gc.collect() ----------------------------------------------------------------------- Operating System: Linux Mandrake release 7.2 (Odyssey) for i586 Linux calabash.bioreason.com 2.2.17-21mdk #1 Thu Oct 5 13:16:08 CEST 2000 i686 Python: 2.1 (#1, Oct 13 2001, 13:52:22) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Traceback (most recent call last): File "CrashMe.py", line 29, in ? value = MKRecord(ident="Version 1.0") SystemError: null argument to internal routine ----------------------------------------------------------------------- Operating System: Linux Mandrake release 7.2 (Odyssey) for i586 Linux calabash.bioreason.com 2.2.17-21mdk #1 Thu Oct 5 13:16:08 CEST 2000 i686 Python: 2.2b2 (#1, Nov 19 2001, 09:55:10) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Traceback (most recent call last): File "CrashMe.py", line 29, in ? value = MKRecord(ident="Version 1.0") SystemError: null argument to internal routine From jcw@equi4.com Mon Nov 19 13:22:29 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA15367 for ; Mon, 19 Nov 2001 13:22:28 -0600 Received: from takkie.equi4.local ([195.121.237.142]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GN2ASO01.4V2 for ; Mon, 19 Nov 2001 20:14:48 +0100 Received: from [10.0.1.29] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAJJCw006223 for ; Mon, 19 Nov 2001 20:12:58 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Why does this fail? Date: Mon, 19 Nov 2001 20:14:54 +0100 Message-Id: <20011119191454.14624@mail.planet.nl> In-Reply-To: <3BF958AB.F9AEDFE5@bioreason.com> References: <3BF958AB.F9AEDFE5@bioreason.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mitch Chapman wrote: >The script on which I'm working sometimes segfaults with a gdb >stack trace which starts in the Python 2.x garbage collector. The segfault is serious, I hope you can reproduce it - somehow. ># Uncomment this to trigger a SystemError further downstream: >del version[:] This error was solved and reported in the change log: http://www.equi4.com/metakit/CHANGES 2001-10-31 Fixed Mk4py error flag clear on delete When deleting rows, a slice was constructed from a PWOSequence, which generates an otherwise harmless error when its length is checked. The flag was not cleared, causing errors in subsequent Python statements. Changed to a PWOTuple in PyView.cpp (2x). This change is more recent than the 2.4.1 release, see the download page for info on how to obtain the latest source code from CVS: http://www.equi4.com/metakit/download.html -jcw From mitch.chapman@bioreason.com Mon Nov 19 13:38:15 2001 Received: from gateway.bioreason.com (www.bioreason.com [207.108.245.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA16525; Mon, 19 Nov 2001 13:38:14 -0600 Received: by gateway.bioreason.com (Postfix, from userid 200) id 114B82237; Mon, 19 Nov 2001 12:29:30 +0000 (/etc/localtime) Received: by bioreason.com via smwrap Version 1.0 id smwrapJaby99; Mon Nov 19 12:28:41 2001 by mailhost.bioreason.com (Postfix) with ESMTP id DE5403577; Mon, 19 Nov 2001 12:29:14 +0000 (/etc/localtime) Message-ID: <3BF95F20.FBC23F86@bioreason.com> Date: Mon, 19 Nov 2001 12:36:00 -0700 From: Mitch Chapman Organization: Bioreason, Inc. MIME-Version: 1.0 To: Jean-Claude Wippler Cc: metakit Subject: Re: [Metakit] Why does this fail? References: <3BF958AB.F9AEDFE5@bioreason.com> <20011119191454.14624@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > > Mitch Chapman wrote: > ># Uncomment this to trigger a SystemError further downstream: > >del version[:] > > This error was solved and reported in the change log: > http://www.equi4.com/metakit/CHANGES > > 2001-10-31 Fixed Mk4py error flag clear on delete Thanks! > This change is more recent than the 2.4.1 release, see the download page > for info on how to obtain the latest source code from CVS: > http://www.equi4.com/metakit/download.html Is there another way to get at the changes? I seem to be unable to make CVS connections from here... -- Mitch Chapman Mitch.Chapman@bioreason.com From steven.leblanc@mayahtt.com Mon Nov 19 14:15:02 2001 Received: from qmail.mayahtt.ca (mayahtt.com [199.84.144.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA19364 for ; Mon, 19 Nov 2001 14:15:02 -0600 Received: from _HOSTNAME_ (indigo23.mayahtt.ca [192.9.201.194]) by qmail.mayahtt.ca (8.11.2/8.11.2) with SMTP id fAJK7oX22248 for ; Mon, 19 Nov 2001 15:07:52 -0500 Received: by _HOSTNAME_ (sSMTP sendmail emulation); Mon, 19 Nov 2001 15:07:51 -0500 Date: Mon, 19 Nov 2001 15:07:51 -0500 From: Steve Leblanc To: metakit@equi4.com Message-ID: <20011119150751.A281197@mayahtt.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Organization: Maya Heat Transfer Technologies Subject: [Metakit] Problem on SGI and AIX Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi [snip: re infinite loop in TestMapped on SGI and AIX] I tried using the latest CVS code and it gave me the same problem. The good news is that everything works if I comment out the call to TestMapped and I don't need the hash view for my application. Here's what I know about the achitecture of the two platforms on which this test hangs: SGI --- OS: IRIX 6.5 Compiler: MIPS C++ compiler, vers 7.2.1 32 bit big-endian AIX --- OS: AIX 3.4 Compiler: IBM C++ Compiler, vers 3.6.4 32 bit, big-endian (I think) HP-UX worked fine with this configuration: HP-UX B.10.20 using HP ANSI C++ B3910B A.01.21 I don't think it's purely an endianness issue because Solaris is also big-endian and the test runs fine. However, it may well be some low-level thing like that, since the loop in question does a lot of bit-twiddling. P.S. Sorry for breaking the thread. I wasn't subscribed to the mailing list when I sent my first mail, so I can't thread this one to the back of yours. -- Steve Leblanc Maya Heat Transfer Technologies Ltd. Phone: (514) 369-5706 Fax: (514) 369-4200 From mitch.chapman@bioreason.com Mon Nov 19 16:17:22 2001 Received: from gateway.bioreason.com (www.bioreason.com [207.108.245.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA27926 for ; Mon, 19 Nov 2001 16:17:21 -0600 Received: by gateway.bioreason.com (Postfix, from userid 200) id C9D61223B; Mon, 19 Nov 2001 15:08:36 +0000 (/etc/localtime) Received: by bioreason.com via smwrap Version 1.0 id smwrapTYSg3d; Mon Nov 19 15:08:17 2001 by mailhost.bioreason.com (Postfix) with ESMTP id 1CEE23577 for ; Mon, 19 Nov 2001 15:08:50 +0000 (/etc/localtime) Message-ID: <3BF98487.5C434126@bioreason.com> Date: Mon, 19 Nov 2001 15:15:35 -0700 From: Mitch Chapman Organization: Bioreason, Inc. MIME-Version: 1.0 To: Metakit Mailing List Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Python, Metakit, Linux, SegFaults -- more info Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Below are three Python scripts, with output, which trigger segfaults on Mandrake Linux 7.2. The problems may already have been discovered and fixed,* in which case please pardon the waste of bandwidth. All of these scripts use Metakit incorrectly. They also show usage errors of the sort to which a[t least one] new user is prone, so may be useful for documentation purposes. ("Are you new to Metakit? Well, don't do this.") -- Mitch Chapman Mitch.Chapman@bioreason.com *(I did check the ChangeLog but found no clearly related entries.) ======================================================================== #!/usr/bin/env python """Crash through indexing error -- same cause as SystemError?""" import metakit, sys, os # Show the current platform: osVersion = open("/etc/mandrake-release").read().strip() print "OS:", osVersion print "Python:", sys.version print "Metakit:", metakit.version print db = metakit.storage() schema = db.getas("version[ident:S]") schema[:] = [] # This will crash because it's indexing beyond the end of schema. # Should it produce an IndexError instead? schema[0] = {'ident': 'Kilroy'} ----------------------------------------------------------------------- OS: Linux Mandrake release 7.2 (Odyssey) for i586 Python: 2.2b2 (#1, Nov 19 2001, 09:55:10) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Segmentation fault ======================================================================== #!/usr/bin/env python """Crash by saving an in-memory store?""" import metakit, sys, os # Show the current platform: osVersion = open("/etc/mandrake-release").read().strip() print "OS:", osVersion print "Python:", sys.version print "Metakit:", metakit.version print db = metakit.storage() schema = db.getas("version[ident:S]") schema[:] = [] schema.append({'ident': 'Kilroy'}) # This will crash because storage has no associated file? db.commit() ----------------------------------------------------------------------- OS: Linux Mandrake release 7.2 (Odyssey) for i586 Python: 2.2b2 (#1, Nov 19 2001, 09:55:10) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Segmentation fault ======================================================================== #!/usr/bin/env python """Crash through bad subview manipulation.""" import sys, metakit, os # Show the current platform: osVersion = open("/etc/mandrake-release").read().strip() print "OS:", osVersion print "Python:", sys.version print "Metakit:", metakit.version print db = metakit.storage("crash.dat", 1) records = db.getas("record[ident:I,contents[cid:I]]") numRecords = 500 records[:] = [] records.setsize(numRecords) # Why does this crash instead of complaining about the value of # contents? contents = range(10) for i in range(numRecords): records[i] = {'ident': i, 'contents': contents} print "Saved", i sys.stdout.flush() db.commit() ----------------------------------------------------------------------- OS: Linux Mandrake release 7.2 (Odyssey) for i586 Python: 2.2b2 (#1, Nov 19 2001, 09:55:10) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Saved 0 Saved 1 Saved 2 Saved 3 Saved 4 Saved 5 Saved 6 Saved 7 Segmentation fault From mlh@idi.ntnu.no Mon Nov 19 18:02:53 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA01522 for ; Mon, 19 Nov 2001 18:02:52 -0600 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id AAA03069 for ; Tue, 20 Nov 2001 00:55:37 +0100 (MET) Message-ID: <045301c17155$af6f7280$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: References: <20011119031550.51194.qmail@web11504.mail.yahoo.com> Subject: Re: [Metakit] Using hash value as a reference (??) Date: Tue, 20 Nov 2001 00:55:37 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have another suggestion for you: If the sorting/hashing is only for lookup, and you don't delete titles from your system, what about having a separate view which only contains the titles (not sorted)? Then the dictionary could contain the index of the title in this view, you could look up the title, and use the sorting/hashing to look up that title in the main view. You might want to store a reverse version of this too (name -> index) but that could be done by simply adding an index field to your sorted/hashed view. It's simple, at least. -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.sf.net From baspey@yahoo.com Tue Nov 20 02:18:44 2001 Received: from web11503.mail.yahoo.com (web11503.mail.yahoo.com [216.136.172.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id CAA17418 for ; Tue, 20 Nov 2001 02:18:44 -0600 Message-ID: <20011120081127.51629.qmail@web11503.mail.yahoo.com> Received: from [62.147.34.109] by web11503.mail.yahoo.com via HTTP; Tue, 20 Nov 2001 00:11:27 PST Date: Tue, 20 Nov 2001 00:11:27 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1246303962-1006243887=:50488" Subject: [Metakit] Using hash value as a reference (??) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1246303962-1006243887=:50488 Content-Type: text/plain; charset=us-ascii Thanks a lot Magnus, your solution is quite interesting. I wanted my first view to be sorted in order to be able to list all the movies in alphabetical order, the most instantly possible, even with an important number of movies. Of course, having an already ordered view would have been the most efficient. Furthermore, since my last posting, I have implemented a third solution: I have added a field im my first view corresponding to the index of the next movie in the alphabetical order, thus creating same sort of linked list. Everything is quite normal as the rest of the database is concerned: I reference movies in the dictionary thanks to their index, the fact that they are unordered having no incidence. But when I want to do this special thing which is to list all movies orderly, I begin by the first row and then follow down the path given by this additional field. Without entering into all details, adding a movie goes like this: I append the movie at the end of the view; if this movie is alphabetically before the first row, I permute the two rows; otherwise, I randomly pick up 50 movie titles and keep the closest above my new movie title; from this movie I go down the track until I find the place where my new movie should be inserted; I simply make my new movie point toward the movie pointed by the preceding movie , and then make the preceding movie point toward my new movie. The row pointed by the last movie is -1. Although this may seem complicated, it proves itself more efficient to manage than an ordered view. --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1246303962-1006243887=:50488 Content-Type: text/html; charset=us-ascii

Thanks a lot Magnus,

your solution is quite interesting.

I wanted my first view to be sorted in order to be able to list all the movies in alphabetical order,

the most instantly possible, even with an important number of movies.

Of course, having an already ordered view would have been the most efficient.

Furthermore, since my last posting, I have implemented a third solution: I have added a field

im my first view corresponding to the index of the next movie in the alphabetical order, thus creating

same sort of linked list.

Everything is quite normal as the rest of the database is concerned: I reference movies in the dictionary

thanks to their index, the fact that they are unordered having no incidence.

But when I want to do this special thing which is to list all movies orderly, I begin by the first row and

then follow down the path given by this additional field.

Without entering into all details, adding a movie goes like this: I append the movie at the end of the view;

if this movie is alphabetically before the first row, I permute the two rows; otherwise, I randomly pick up 50 movie titles and keep

the closest above my new movie title; from this movie I go down the track until I find the place where my new movie should be

inserted; I simply make my new movie point toward the movie pointed by the preceding movie , and then make the preceding

movie point toward my new movie. The row pointed by the last movie is -1.

Although this may seem complicated, it proves itself more efficient to manage than an ordered view.


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1246303962-1006243887=:50488-- From baspey@yahoo.com Tue Nov 20 18:30:22 2001 Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA17144 for ; Tue, 20 Nov 2001 18:30:21 -0600 Message-ID: <20011121002307.47859.qmail@web11501.mail.yahoo.com> Received: from [62.147.34.236] by web11501.mail.yahoo.com via HTTP; Tue, 20 Nov 2001 16:23:07 PST Date: Tue, 20 Nov 2001 16:23:07 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1608752103-1006302187=:43744" Subject: [Metakit] Using hash value as a reference (??) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1608752103-1006302187=:43744 Content-Type: text/plain; charset=us-ascii Before someone tells me: I had missed the function RemapWith(), and I guess that it will make my solution neater. --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1608752103-1006302187=:43744 Content-Type: text/html; charset=us-ascii

Before someone tells me:

I had missed the function RemapWith(), and I guess that it will make my solution neater.


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1608752103-1006302187=:43744-- From baspey@yahoo.com Wed Nov 21 01:09:43 2001 Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA02064 for ; Wed, 21 Nov 2001 01:09:43 -0600 Message-ID: <20011121070228.5511.qmail@web11501.mail.yahoo.com> Received: from [62.147.39.8] by web11501.mail.yahoo.com via HTTP; Tue, 20 Nov 2001 23:02:28 PST Date: Tue, 20 Nov 2001 23:02:28 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-2061439712-1006326148=:5058" Subject: [Metakit] Using hash value as a reference (??) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-2061439712-1006326148=:5058 Content-Type: text/plain; charset=us-ascii I most certainly soliloquize, but in the dubious case somebody followed my history with interest, here are the latest news (otherwise, sorry for the pollution): RemapWith() will certainly be very helpful when it comes to dictionaries but, in the end, I won't use it to order movie titles, and will stick with my method, because it is faster when inserting new movies (single permutation instead of moving down a bunch of rows). --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-2061439712-1006326148=:5058 Content-Type: text/html; charset=us-ascii

I most certainly soliloquize, but in the dubious case somebody followed my history

with interest, here are the latest news (otherwise, sorry for the pollution):

RemapWith() will certainly be very helpful when it comes to dictionaries but, in the

end, I won't use it to order movie titles, and will stick with my method, because it is faster

when inserting new movies (single permutation instead of moving down a bunch of rows).


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-2061439712-1006326148=:5058-- From baspey@yahoo.com Thu Nov 22 01:04:31 2001 Received: from web11505.mail.yahoo.com (web11505.mail.yahoo.com [216.136.172.37]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA24587 for ; Thu, 22 Nov 2001 01:04:30 -0600 Message-ID: <20011122065709.35661.qmail@web11505.mail.yahoo.com> Received: from [62.147.37.252] by web11505.mail.yahoo.com via HTTP; Wed, 21 Nov 2001 22:57:09 PST Date: Wed, 21 Nov 2001 22:57:09 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1063177582-1006412229=:33046" Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1063177582-1006412229=:33046 Content-Type: text/plain; charset=us-ascii I am very disappointed by the performance I observe (hope I'm doing something wrong). I use Visual C++ under Windows ME, with the latest CVS depository. I have a blocked/hash view, in which each row consists of 5 string properties, 1 int property and 1 view property. Rows are always added at the end of the view, after having used Find() to check that they're not already present. Finally, I commit every 1000 rows added. And here is what I observe: For the first thousand rows => one Find() takes about 1200 CPU cycles Commit() takes about 170,000 CPU cycles this is satisfying ... except that it degrades steadily ! For the 30th thousand rows => one Find() takes about 1900 CPU cycles Commit() takes about 1,400,000 CPU cycles Since I have close to 300,000 rows to add, that will take ages (days in fact). 30,000 rows only means a database of 1,8 Mo, so I guess that 300,000 rows would mean a 20 Mo database. An unmanageable database of only 20 Mo, this can't be. I have checked that find() was not performing a linear search, but really used the hash value, inducing a theorical O(1) cost.: c4_View::Find() calls c4_CustomSeq::RestrictSearch(), which calls c4_HashViewer::LoopUp(), which calls c4_HashViewer::CalcHash() and c4_HashViewer::LookDict(). Pleeeaase someone tell me what I'm doing wrooong ! --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1063177582-1006412229=:33046 Content-Type: text/html; charset=us-ascii

I am very disappointed by the performance I observe (hope I'm doing something

wrong).

I use Visual C++ under Windows ME, with the latest CVS depository.

I have a blocked/hash view, in which each row consists of 5 string properties,

1 int property and 1 view property.

Rows are always added at the end of the view, after having used Find() to check

that they're not already present.

Finally, I commit every 1000 rows added.

And here is what I observe:

For the first thousand rows => one Find() takes about 1200 CPU cycles

                                             Commit() takes about 170,000 CPU cycles

this is satisfying ... except that it degrades steadily !

For the 30th thousand rows => one Find() takes about 1900 CPU cycles

                                             Commit() takes about 1,400,000 CPU cycles

Since I have close to 300,000 rows to add, that will take ages (days in fact).

30,000 rows only means a database of 1,8 Mo, so I guess that 300,000 rows

would mean a 20 Mo database. An unmanageable database of only 20 Mo,

this can't be.

I have checked that find() was not performing a linear search, but really used

the hash value, inducing a theorical O(1) cost.:

c4_View::Find() calls c4_CustomSeq::RestrictSearch(),

which calls c4_HashViewer::LoopUp(),

which calls c4_HashViewer::CalcHash() and c4_HashViewer::LookDict().

Pleeeaase someone tell me what I'm doing wrooong !




Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1063177582-1006412229=:33046-- From jcw@equi4.com Thu Nov 22 04:17:24 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA32533 for ; Thu, 22 Nov 2001 04:17:24 -0600 Received: from takkie.equi4.local ([195.121.232.20]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GN750C00.J4Z for ; Thu, 22 Nov 2001 10:57:48 +0100 Received: from [10.0.1.32] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAM9uH023438 for ; Thu, 22 Nov 2001 10:56:18 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Performance can't be that bad Date: Thu, 22 Nov 2001 10:40:07 +0100 Message-Id: <20011122094007.15865@mail.planet.nl> In-Reply-To: <20011122065709.35661.qmail@web11505.mail.yahoo.com> References: <20011122065709.35661.qmail@web11505.mail.yahoo.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Pascal Baspeyras wrote: (Is there a way to alter your email settings? Your text comes out double-spaced all the time - it's quite tedious to read...) >I have a blocked/hash view, in which each row consists of 5 string properties, >1 int property and 1 view property. >Rows are always added at the end of the view, after having used Find() to >check that they're not already present. >Finally, I commit every 1000 rows added. >Since I have close to 300,000 rows to add, that will take ages (days in fact). >30,000 rows only means a database of 1,8 Mo, so I guess that 300,000 rows >would mean a 20 Mo database. An unmanageable database of only 20 Mo, >this can't be. That's 60 bytes of data per row, each 5 strings, an int and a subview. The subview may be killing performance. How's speed when you leave it out of the definition? -jcw From baspey@yahoo.com Thu Nov 22 10:37:42 2001 Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA17485 for ; Thu, 22 Nov 2001 10:37:42 -0600 Message-ID: <20011122163021.73054.qmail@web11501.mail.yahoo.com> Received: from [62.147.35.252] by web11501.mail.yahoo.com via HTTP; Thu, 22 Nov 2001 08:30:21 PST Date: Thu, 22 Nov 2001 08:30:21 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-328133064-1006446621=:68541" Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-328133064-1006446621=:68541 Content-Type: text/plain; charset=us-ascii Thanks a lot Jean-Claude for your reply. >(Is there a way to alter your email settings? Your text comes out >double-spaced all the time - it's quite tedious to read...) I will try, but I don't control much of anything... >That's 60 bytes of data per row, each 5 strings, an int and a subview. Yes indeed. For this test, I only provide 2 strings and 1 int as data.The other fields are left untouched, the subview included. >The subview may be killing performance. How's speed when you leave it >out of the definition? I suppressed the subview from the structure but still got the same result, whichis good news in a sense.I ran the program on another computer, but still had the same result, only slowerbecause it's a slower computer (convert days into weeks ;-) )I used MyView = MyStorage.GetAs() instead of MyView = MyStorage.View() andstill got the same.I tried stopping the program, launch it again and resume the additions, but performancekept on where it left. I run out of ideas... Thanks for your attention. --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-328133064-1006446621=:68541 Content-Type: text/html; charset=us-ascii
Thanks a lot Jean-Claude for your reply.
 
>(Is there a way to alter your email settings?  Your text comes out
>double-spaced all the time - it's quite tedious to read...)
I will try, but I don't control much of anything...
 
>That's 60 bytes of data per row, each 5 strings, an int and a subview.
 
Yes indeed. For this test, I only provide 2 strings and 1 int as data.
The other fields are left untouched, the subview included.
 
>The subview may be killing performance.  How's speed when you leave it
>out of the definition?
 
I suppressed the subview from the structure but still got the same result, which
is good news in a sense.
I ran the program on another computer, but still had the same result, only slower
because it's a slower computer (convert days into weeks ;-)  )
I used MyView = MyStorage.GetAs() instead of MyView = MyStorage.View() and
still got the same.
I tried stopping the program, launch it again and resume the additions, but performance
kept on where it left.
 
I run out of ideas...
 
Thanks for your attention.


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-328133064-1006446621=:68541-- From steve@improvision.com Thu Nov 22 10:56:13 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA18557 for ; Thu, 22 Nov 2001 10:56:12 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id QAA14879 for ; Thu, 22 Nov 2001 16:46:52 GMT Received: from [192.168.1.24] ([192.168.1.24]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Thu, 22 Nov 2001 16:48:51 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Thu, 22 Nov 2001 16:48:49 +0000 Subject: Re: [Metakit] Performance can't be that bad From: Stephen Baxter To: Pascal Baspeyras , Message-ID: In-Reply-To: <20011122163021.73054.qmail@web11501.mail.yahoo.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: How about using a blocked view - might that stop Commit() slowing down as you add more and more rows? -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Pascal Baspeyras > Date: Thu, 22 Nov 2001 08:30:21 -0800 (PST) > To: metakit@equi4.com > Subject: [Metakit] Performance can't be that bad > > Thanks a lot Jean-Claude for your reply. >(Is there a way to alter your email > settings? Your text comes out >> double-spaced all the time - it's quite tedious to read...) > I will try, but I don't control much of anything... >That's 60 bytes of data > per row, each 5 strings, an int and a subview. Yes indeed. For this test, I > only provide 2 strings and 1 int as data.The other fields are left untouched, > the subview included. >The subview may be killing performance. How's speed > when you leave it >> out of the definition? I suppressed the subview from the structure but still >> got the same result, whichis good news in a sense.I ran the program on >> another computer, but still had the same result, only slowerbecause it's a >> slower computer (convert days into weeks ;-) )I used MyView = >> MyStorage.GetAs() instead of MyView = MyStorage.View() andstill got the >> same.I tried stopping the program, launch it again and resume the additions, >> but performancekept on where it left. I run out of ideas... Thanks for your >> attention. > > > --------------------------------- > Do You Yahoo!? > Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. From jcw@equi4.com Thu Nov 22 11:40:13 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA21228 for ; Thu, 22 Nov 2001 11:40:12 -0600 Received: from takkie.equi4.local ([195.121.235.215]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GN7Q1X00.M2G for ; Thu, 22 Nov 2001 18:32:21 +0100 Received: from [10.0.1.32] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAMHUf025628 for ; Thu, 22 Nov 2001 18:30:55 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Performance can't be that bad Date: Thu, 22 Nov 2001 18:32:31 +0100 Message-Id: <20011122173231.18514@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >How about using a blocked view - might that stop Commit() slowing down as >you add more and more rows? Indeed. I believe he mentioned blocked. But I hope it's blocked in the main view, not the hash map (which is going to be just a bunch of fast int vectors). If the hash map persists, that is (this is optional). The other thing that might be happening is that the commit causes a full hash map reconstruction. This could be tested by removing the hash before a commit and timing to see whether it makes a difference until commit returns. -jcw From Murat_Berk@bmc.com Thu Nov 22 12:32:44 2001 Received: from creeper.bmc.com (firebird.bmc.com [198.207.223.228]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA24269 for ; Thu, 22 Nov 2001 12:32:44 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2/8.10.2) with ESMTP id fAMIOWG25557; Thu, 22 Nov 2001 12:24:32 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 22 Nov 2001 12:24:45 -0600 Message-ID: From: "Berk, Murat" To: "'Stephen Baxter'" , Pascal Baspeyras , metakit@equi4.com Subject: RE: [Metakit] Performance can't be that bad Date: Thu, 22 Nov 2001 12:24:43 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We had similar problems with insertion speed and find/search performace. We find ways around those issues. (We are using 2.01 release) 1) We don't use the Find and Search stuff in the Metakit for string stuff. We wrote custom Find and Search functions which deals with these issues. We figured out that c4_StringRef is very fast and we are using it to access data w/o extracting it out. bool Mk_Utils::Utf8StringIsEqualC4Property(const char* utf8_ptr, size_t len, const c4_StringProp& p, const c4_RowRef& r) { c4_Bytes result; c4_StringRef strRef = p(r); strRef.GetData(result); if ( (result.Size()-1) != (int)len ) return false; const char* ptr1 = result.Size() > 0 ? (const char*) result.Contents() : ""; for ( size_t j = 0; j < len; j++ ) if ( utf8_ptr[j] != ptr1[j] ) return false; return true; } Then we copied Search algorithm from the metakit sources and together they perform much faster overall. I do not remember exact Quantify #'s but most of the time was passing constructing these c4_String objects and in the == operator. 2) Preallocate rows before you insert anything. We allocate 512 rows at a time. But initially it was very slow to add strings into these views. The reason is the internal offset array is reallocated everytime and grows 1 at a time. What we did was set the last row of the preallocated bunch to a dummy value. What this does is allocate all the offset stuff for all 512 rows and also leaves preallocated memory called slack in the data. When you insert new data, it just shifts slack pointer around and works very fast. 3) We really did not have problem w/ the speed of the commits. The only problem we are having is commit uses to much of c4_String which slows it down. 4) We have some memory leaks for the memory allocated in c4_String for nullVector. Everywhere do we use locks around the metakit (1 per open database and we have a lot of databases open at any given time). This works great except the reference counting for the c4_String objects which uses internal nullVector. Since that is shared by all the databases and not protected across threads, we do see reference counting problems and it leaks. I was looking removing all that stuff and using a special value for refcount (like (char)-1) to indicate we do not need to reference count, but it is not much a problem right now. -----Original Message----- From: Stephen Baxter [mailto:steve@improvision.com] Sent: Thursday, November 22, 2001 10:49 AM To: Pascal Baspeyras; metakit@equi4.com Subject: Re: [Metakit] Performance can't be that bad How about using a blocked view - might that stop Commit() slowing down as you add more and more rows? -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Pascal Baspeyras > Date: Thu, 22 Nov 2001 08:30:21 -0800 (PST) > To: metakit@equi4.com > Subject: [Metakit] Performance can't be that bad > > Thanks a lot Jean-Claude for your reply. >(Is there a way to alter your email > settings? Your text comes out >> double-spaced all the time - it's quite tedious to read...) > I will try, but I don't control much of anything... >That's 60 bytes of data > per row, each 5 strings, an int and a subview. Yes indeed. For this test, I > only provide 2 strings and 1 int as data.The other fields are left untouched, > the subview included. >The subview may be killing performance. How's speed > when you leave it >> out of the definition? I suppressed the subview from the structure but still >> got the same result, whichis good news in a sense.I ran the program on >> another computer, but still had the same result, only slowerbecause it's a >> slower computer (convert days into weeks ;-) )I used MyView = >> MyStorage.GetAs() instead of MyView = MyStorage.View() andstill got the >> same.I tried stopping the program, launch it again and resume the additions, >> but performancekept on where it left. I run out of ideas... Thanks for your >> attention. > > > --------------------------------- > Do You Yahoo!? > Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From baspey@yahoo.com Thu Nov 22 12:35:51 2001 Received: from web11504.mail.yahoo.com (web11504.mail.yahoo.com [216.136.172.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA24451 for ; Thu, 22 Nov 2001 12:35:50 -0600 Message-ID: <20011122182829.43424.qmail@web11504.mail.yahoo.com> Received: from [62.147.35.96] by web11504.mail.yahoo.com via HTTP; Thu, 22 Nov 2001 10:28:29 PST Date: Thu, 22 Nov 2001 10:28:29 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-2014375866-1006453709=:39863" Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-2014375866-1006453709=:39863 Content-Type: text/plain; charset=us-ascii >I believe he mentioned blocked. But I hope it's blocked in the main view, not >the hash map>If the hash map persists, that is. The hash map persists.I first build an empty structure making: c4_View View = Storage.GetAs("Movies[_B[Title:S,Year:S]]"); c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]"); And when it comes to the part of the program where I add rows, I do: c4_View View = Storage.View("Movies"); c4_View ViewBlocked = View.Blocked(); c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]"); c4_View ViewBlockedHash = ViewBlocked.Hash(Map, 2); c4_StringProp pTitle("Title"); c4_StringProp pYear("Year"); And from this point, I make all operations on ViewBlockedHash , i.e: ViewBlockedHash.Add(pTitle["Les bronzés font du ski"] + pYear["1979"]); or ViewBlockedHash.Find(pTitle["Les bronzés font du ski"] + pYear["1979"]); It warms my heart to see you all trying to help. P.S: I missed it completely for the mail format. All my excuses. --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-2014375866-1006453709=:39863 Content-Type: text/html; charset=us-ascii
>I believe he mentioned blocked.  But I hope it's blocked in the main view, not
>the hash map
>If the hash map persists, that is.
 
The hash map persists.
I first build an empty structure making:
    c4_View View = Storage.GetAs("Movies[_B[Title:S,Year:S]]");
    c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]");
 
And when it comes to the part of the program where I add rows, I do:
    c4_View View = Storage.View("Movies");
    c4_View ViewBlocked = View.Blocked();
    c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]");
    c4_View ViewBlockedHash = ViewBlocked.Hash(Map, 2);
 
    c4_StringProp pTitle("Title");
    c4_StringProp pYear("Year");
 
 
 
And from this point, I make all operations on ViewBlockedHash , i.e:
    ViewBlockedHash.Add(pTitle["Les bronzés font du ski"] + pYear["1979"]);
    or ViewBlockedHash.Find(pTitle["Les bronzés font du ski"] + pYear["1979"]);
 
It warms my heart to see you all trying to help.
 
P.S: I missed it completely for the mail format. All my excuses.


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-2014375866-1006453709=:39863-- From steve@improvision.com Thu Nov 22 12:40:07 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA24688 for ; Thu, 22 Nov 2001 12:40:06 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA23330 for ; Thu, 22 Nov 2001 18:30:47 GMT Received: from [192.168.1.24] ([192.168.1.24]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Thu, 22 Nov 2001 18:32:45 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Thu, 22 Nov 2001 18:32:45 +0000 Subject: Re: [Metakit] Performance can't be that bad From: Stephen Baxter To: "Berk, Murat" , Pascal Baspeyras , Message-ID: In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It would be *really* nice if Metakit did this internally, allocating rows say 512 at a time. This is what std::vector does, it makes a big difference to speed when adding items to an array. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: "Berk, Murat" > Date: Thu, 22 Nov 2001 12:24:43 -0600 > To: "'Stephen Baxter'" , Pascal Baspeyras > , metakit@equi4.com > Subject: RE: [Metakit] Performance can't be that bad > > 2) Preallocate rows before you insert anything. We allocate 512 rows at a > time. But initially it was very slow to add strings into these views. The > reason is the internal offset array is reallocated everytime and grows 1 at > a time. What we did was set the last row of the preallocated bunch to a > dummy value. What this does is allocate all the offset stuff for all 512 > rows and also leaves preallocated memory called slack in the data. When you > insert new data, it just shifts slack pointer around and works very fast. From baspey@yahoo.com Thu Nov 22 13:46:51 2001 Received: from web11502.mail.yahoo.com (web11502.mail.yahoo.com [216.136.172.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id NAA27895 for ; Thu, 22 Nov 2001 13:46:51 -0600 Message-ID: <20011122193930.82490.qmail@web11502.mail.yahoo.com> Received: from [62.147.38.85] by web11502.mail.yahoo.com via HTTP; Thu, 22 Nov 2001 11:39:30 PST Date: Thu, 22 Nov 2001 11:39:30 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: My mails are getting worsest. It's beyond my will, I swear. What Berk Murat says about c4_String object seems quite interesting, even though I haven't digested everything yet. But I think he missed one issue: I don't use Find() for brute-force linear search. I use it in conjonction with a hash view. What worries me the most about Find() performance is that it quickly requires 2 more time to perform (and then 3, 4 more times) when it is supposed to perform at O(1) cost. __________________________________________________ Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. http://geocities.yahoo.com/ps/info1 From baspey@yahoo.com Thu Nov 22 17:06:33 2001 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id RAA05428 for ; Thu, 22 Nov 2001 17:06:33 -0600 Received: from toulouse-1-a7-32-1.dial.proxad.net (HELO desktop) (62.147.32.1) by smtp.mail.vip.sc5.yahoo.com with SMTP; 22 Nov 2001 22:59:10 -0000 Message-ID: <000f01be166b$8b8036e0$0120933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sun, 22 Nov 1998 23:57:55 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_000A_01BE1673.EC0EDE00" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_000A_01BE1673.EC0EDE00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I repost 2 of my previous messages which where completely unreadable. Hope that will work this time... Message1: >That's 60 bytes of data per row, each 5 strings, an int and a subview. Yes indeed. For this test, I only provide 2 strings and 1 int as data. The other fields are left untouched, the subview included. >The subview may be killing performance. How's speed when you leave it >out of the definition? I suppressed the subview from the structure but still got the same = result, which is good news in a sense. I ran the program on another computer, but still had the same result, = only slower because it's a slower computer (convert days into weeks ;-) ) I used MyView =3D MyStorage.GetAs() instead of MyView =3D = MyStorage.View() and still got the same. I tried stopping the program, launch it again and resume the additions, = but performance kept on where it left. Message2: >I believe he mentioned blocked. But I hope it's blocked in the main = view, not=20 >the hash map >If the hash map persists, that is.=20 The hash map persists.I first build an empty structure making: c4_View View =3D Storage.GetAs("Movies[_B[Title:S,Year:S]]"); c4_View Map =3D Storage.GetAs("MoviesMap[_H:I,_R:I]"); And when it comes to the part of the program where I add rows, I do: c4_View View =3D Storage.View("Movies"); c4_View ViewBlocked =3D View.Blocked(); c4_View Map =3D Storage.GetAs("MoviesMap[_H:I,_R:I]"); c4_View ViewBlockedHash =3D ViewBlocked.Hash(Map, 2); c4_StringProp pTitle("Title"); c4_StringProp pYear("Year"); And from this point, I make all operations on ViewBlockedHash , i.e: ViewBlockedHash.Add(pTitle["Les bronz=E9s font du ski"] + = pYear["1979"]); or ViewBlockedHash.Find(pTitle["Les bronz=E9s font du ski"] + = pYear["1979"]); =20 ------=_NextPart_000_000A_01BE1673.EC0EDE00 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I repost 2 of my previous messages = which where=20 completely unreadable.
Hope that will work this time...
 
Message1:
 
>That's 60 bytes of data per row, each 5 strings, an int and a=20 subview.
 
Yes indeed. For this test, I only = provide 2 strings=20 and 1 int as data.
The other fields are left untouched, = the subview=20 included.
 
>The subview may be killing performance.  How's speed when = you=20 leave it
>out of the definition?
 
I suppressed the subview from the = structure but=20 still got the same result, which
is good news in a sense.
I ran the program on another computer, = but still=20 had the same result, only slower
because it's a slower computer (convert = days into=20 weeks ;-)  )
I used MyView =3D MyStorage.GetAs() = instead of MyView=20 =3D MyStorage.View() and
still got the same.
I tried stopping the program, launch it = again and resume the additions, but performance
kept on where it left.
 
 
Message2:
 
>I believe he mentioned blocked.  But I hope it's blocked = in the=20 main view, not
>the hash map
>If the hash map persists, that is.
 
The hash map persists.I first build an empty structure = making:
    c4_View View =3D=20 Storage.GetAs("Movies[_B[Title:S,Year:S]]");
    c4_View Map =3D=20 Storage.GetAs("MoviesMap[_H:I,_R:I]");
 
 And when it comes to the part of the program where I add = rows, I=20 do:
    c4_View View =3D Storage.View("Movies");
    c4_View ViewBlocked =3D View.Blocked();
    c4_View Map =3D=20 Storage.GetAs("MoviesMap[_H:I,_R:I]");
    c4_View ViewBlockedHash =3D = ViewBlocked.Hash(Map,=20 2);
     c4_StringProp pTitle("Title");
    c4_StringProp pYear("Year");
 
And from this point, I make all operations on ViewBlockedHash , = i.e:
    ViewBlockedHash.Add(pTitle["Les bronz=E9s font = du ski"] +=20 pYear["1979"]);
    or ViewBlockedHash.Find(pTitle["Les bronz=E9s = font du=20 ski"] + pYear["1979"]);

 
------=_NextPart_000_000A_01BE1673.EC0EDE00-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Thu Nov 22 18:06:15 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA08283 for ; Thu, 22 Nov 2001 18:06:14 -0600 Received: from toulouse-2-a7-37-162.dial.proxad.net (HELO desktop) (62.147.37.162) by smtp.mail.vip.sc5.yahoo.com with SMTP; 22 Nov 2001 23:58:51 -0000 Message-ID: <001501be1673$e2179900$a225933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Mon, 23 Nov 1998 00:57:35 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0012_01BE167C.4239E9C0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0012_01BE167C.4239E9C0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable =20 I repost 2 of my previous messages which where completely unreadable. Hope that will work this time... Message1: >That's 60 bytes of data per row, each 5 strings, an int and a subview. Yes indeed. For this test, I only provide 2 strings and 1 int as data. The other fields are left untouched, the subview included. >The subview may be killing performance. How's speed when you leave it >out of the definition? I suppressed the subview from the structure but still got the same = result, which is good news in a sense. I ran the program on another computer, but still had the same result, = only slower because it's a slower computer (convert days into weeks ;-) ) I used MyView =3D MyStorage.GetAs() instead of MyView =3D = MyStorage.View() and still got the same. I tried stopping the program, launch it again and resume the additions, = but performance kept on where it left. Message2: >I believe he mentioned blocked. But I hope it's blocked in the main = view, not=20 >the hash map >If the hash map persists, that is.=20 The hash map persists.I first build an empty structure making: c4_View View =3D Storage.GetAs("Movies[_B[Title:S,Year:S]]"); c4_View Map =3D Storage.GetAs("MoviesMap[_H:I,_R:I]"); And when it comes to the part of the program where I add rows, I do: c4_View View =3D Storage.View("Movies"); c4_View ViewBlocked =3D View.Blocked(); c4_View Map =3D Storage.GetAs("MoviesMap[_H:I,_R:I]"); c4_View ViewBlockedHash =3D ViewBlocked.Hash(Map, 2); c4_StringProp pTitle("Title"); c4_StringProp pYear("Year"); And from this point, I make all operations on ViewBlockedHash , i.e: ViewBlockedHash.Add(pTitle["Les bronz=E9s font du ski"] + = pYear["1979"]); or ViewBlockedHash.Find(pTitle["Les bronz=E9s font du ski"] + = pYear["1979"]); ------=_NextPart_000_0012_01BE167C.4239E9C0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
 
I repost 2 of my previous messages = which where=20 completely unreadable.
Hope that will work this time...
 
Message1:
 
>That's 60 bytes of data per row, each 5 strings, an int and a=20 subview.
 
Yes indeed. For this test, I only = provide 2 strings=20 and 1 int as data.
The other fields are left untouched, = the subview=20 included.
 
>The subview may be killing performance.  How's speed when = you=20 leave it
>out of the definition?
 
I suppressed the subview from the = structure but=20 still got the same result, which
is good news in a sense.
I ran the program on another computer, = but still=20 had the same result, only slower
because it's a slower computer (convert = days into=20 weeks ;-)  )
I used MyView =3D MyStorage.GetAs() = instead of MyView=20 =3D MyStorage.View() and
still got the same.
I tried stopping the program, launch it = again and resume the additions, but performance
kept on where it left.
 
 
Message2:
 
>I believe he mentioned blocked.  But I hope it's blocked = in the=20 main view, not
>the hash map
>If the hash map persists, that is.
 
The hash map persists.I first build an empty structure = making:
    c4_View View =3D=20 Storage.GetAs("Movies[_B[Title:S,Year:S]]");
    c4_View Map =3D=20 Storage.GetAs("MoviesMap[_H:I,_R:I]");
 
 And when it comes to the part of the program where I add = rows, I=20 do:
    c4_View View =3D Storage.View("Movies");
    c4_View ViewBlocked =3D View.Blocked();
    c4_View Map =3D=20 Storage.GetAs("MoviesMap[_H:I,_R:I]");
    c4_View ViewBlockedHash =3D = ViewBlocked.Hash(Map,=20 2);
     c4_StringProp pTitle("Title");
    c4_StringProp pYear("Year");
 
And from this point, I make all operations on ViewBlockedHash , = i.e:
    ViewBlockedHash.Add(pTitle["Les bronz=E9s font = du ski"] +=20 pYear["1979"]);
    or ViewBlockedHash.Find(pTitle["Les bronz=E9s = font du=20 ski"] + pYear["1979"]);
------=_NextPart_000_0012_01BE167C.4239E9C0-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jcw@equi4.com Fri Nov 23 04:50:10 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA28139 for ; Fri, 23 Nov 2001 04:50:09 -0600 Received: from takkie.equi4.local ([195.121.234.170]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GN91NU00.H8O for ; Fri, 23 Nov 2001 11:40:42 +0100 Received: from [10.0.1.33] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fANAdC030062 for ; Fri, 23 Nov 2001 11:39:21 +0100 From: Jean-Claude Wippler To: Metakit Subject: Re: [Metakit] Performance can't be that bad Date: Fri, 23 Nov 2001 11:20:35 +0100 Message-Id: <20011123102035.3201@mail.planet.nl> In-Reply-To: <001501be1673$e2179900$a225933e@desktop> References: <001501be1673$e2179900$a225933e@desktop> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id EAA28139 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Pascal, >I repost 2 of my previous messages which where completely unreadable. And this one was dated 1998, meaning I had to chase it in my mail box :) >The hash map persists.I first build an empty structure making: > c4_View View = Storage.GetAs("Movies[_B[Title:S,Year:S]]"); > c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]"); > > And when it comes to the part of the program where I add rows, I do: > c4_View View = Storage.View("Movies"); > c4_View ViewBlocked = View.Blocked(); > c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]"); > c4_View ViewBlockedHash = ViewBlocked.Hash(Map, 2); > c4_StringProp pTitle("Title"); > c4_StringProp pYear("Year"); Looks ok. You're using the latest code from CVS, right? (Not 2.4.1, it has a bad hashing bug!). Why are you saving Year as string? Int would be much faster. What happens if you do not do commit at all, how's performance as things grow? >And from this point, I make all operations on ViewBlockedHash , i.e: > ViewBlockedHash.Add(pTitle["Les bronzés font du ski"] + pYear["1979"]); > or ViewBlockedHash.Find(pTitle["Les bronzés font du ski"] + >pYear["1979"]); (C'est des livres, ou de la musique, ta librarie?) -jcw From baspey@yahoo.com Fri Nov 23 10:26:18 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA11344 for ; Fri, 23 Nov 2001 10:26:18 -0600 Received: from toulouse-2-a7-38-109.dial.proxad.net (HELO desktop) (62.147.38.109) by smtp.mail.vip.sc5.yahoo.com with SMTP; 23 Nov 2001 16:18:52 -0000 Message-ID: <001501c1743a$581ceb60$6d26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 23 Nov 2001 17:16:18 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0010_01C17442.90546C80" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0010_01C17442.90546C80 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I do hope my nightmarish experience with mails has come to an end ! JCW kindly wrote: >You're using the latest code from CVS, right? Right. >Why are you saving Year as string? Int would be much faster. I'll of course do that later, but for the sake of the test, it should = not matter. >What happens if you do not do commit at all, how's performance as = things grow? As things grow, performance is bad. The program very rapidly slows down = because Find() is called for each new value entered, and Find() takes fastly = increasing time. Commit() is one big issue, and Find() is another one. I could understand Commit() takes increasing time (though I don't like = it), but Find() should not slow down at such a noticeable pace. It's a hash lookup ! (Voyons Jean-Claude... Les bronz=E9s font du ski, c'est pas de la = litt=E9rature) ------=_NextPart_000_0010_01C17442.90546C80 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I do hope my nightmarish experience = with mails has=20 come to an end !
 
JCW kindly wrote:
>You're using the latest code from = CVS,=20 right?
 
Right.
 
>Why are you saving Year as string?  Int would be much=20 faster.
 
I'll of course do that later, but for the sake of the test, it = should not=20 matter.
 
>What = happens if you=20 do not do commit at all, how's performance as things=20 grow?
As things grow, performance is = bad. The=20 program very rapidly slows down because
Find() is called for each new = value entered,=20 and Find() takes fastly increasing time.
Commit() is one big issue, and = Find() is=20 another one.
I could understand Commit() takes = increasing time=20 (though I don't like it), but Find()
should not slow down at such a = noticeable pace.=20 It's a hash lookup !
 
 
(Voyons Jean-Claude... Les = bronz=E9s font du=20 ski, c'est pas de la litt=E9rature)
------=_NextPart_000_0010_01C17442.90546C80-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Fri Nov 23 12:09:47 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA15787 for ; Fri, 23 Nov 2001 12:09:47 -0600 Received: from toulouse-1-a7-35-123.dial.proxad.net (HELO desktop) (62.147.35.123) by smtp.mail.vip.sc5.yahoo.com with SMTP; 23 Nov 2001 18:02:21 -0000 Message-ID: <003901c17448$ccf59b40$c626933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 23 Nov 2001 18:59:53 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0026_01C17451.093DC2A0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Sending mails the right way Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0026_01C17451.093DC2A0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello, I send my mails to this group using Outlook Express 5 with my mail account on Yahoo! Mail. As I guess you can see, the result is ugly. If I send a mail to myself, the result is perfect, but if I send to this = group, that's a disaster... Could you please tell me how, yourself, you post to this group. ------=_NextPart_000_0026_01C17451.093DC2A0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hello,
 
I send my mails to this group using = Outlook Express=20 5 with my mail
account on Yahoo! Mail.
As I guess you can see, the result is=20 ugly.
If I send a mail to myself, the result = is perfect,=20 but if I send to this group,
that's a disaster...
 
Could you please tell me how, yourself, =  you=20 post to this group.
------=_NextPart_000_0026_01C17451.093DC2A0-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From tciuro@mac.com Fri Nov 23 12:20:40 2001 Received: from smtpout.mac.com (smtpout.mac.com [204.179.120.89]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA16218 for ; Fri, 23 Nov 2001 12:20:39 -0600 Received: from smtp-relay02.mac.com (server-source-si02 [10.13.10.6]) by smtpout.mac.com (8.12.1/8.10.2/1.0) with ESMTP id fANHSrsD018529 for ; Fri, 23 Nov 2001 09:28:53 -0800 (PST) Received: from asmtp01.mac.com ([10.13.10.65]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GN9MM100.P1Q for ; Fri, 23 Nov 2001 10:13:13 -0800 Received: from localhost ([63.202.175.181]) by asmtp01.mac.com (Netscape Messaging Server 4.15 asmtp01 Jun 21 2001 23:53:48) with ESMTP id GN9MM000.40R; Fri, 23 Nov 2001 10:13:12 -0800 Date: Fri, 23 Nov 2001 10:13:12 -0800 Subject: Re: [Metakit] Performance can't be that bad Content-Type: multipart/alternative; boundary=Apple-Mail-1-314704782 Mime-Version: 1.0 (Apple Message framework v501) Cc: Metakit To: Pascal Baspeyras From: Tito Ciuro In-Reply-To: <001501c1743a$581ceb60$6d26933e@desktop> Message-Id: X-Mailer: Apple Mail (2.501) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-314704782 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Salut Pascal, I'd like to share with you a few things I've noticed regarding MetaKit's=20= performance. 1) If your data file grows very quickly (as in commit-extend mode):=20 compact it upon save. With this mode, data grows quickly because a change in a column requires=20= the entire column to be rewritten. In my data file, I know that a=20 particular record is, at the most, 400 bytes. So, when I save, I=20 calculate roughly the size of the record: (file size/number of records).=20= If the resulting record size is greater than, say, 600 bytes, I trigger=20= the compacting method. This has proven to reduce the file size to its=20 minimum size and improve search speeds quite a bit. 2) GetAs() is expensive: cache the view. It's way better to create one and leave it open, then perform all ops=20 with it. Sampling thereafter shows quite a different picture. 3) A hashed view may be your best friend: using it whenever possible. I use my own call rowWithUID() quite a bit. The first column is a unique=20= ID which I use extensively to obtain its row. Using hashed view + Find()=20= has proven to be an *enormous* gain. 4) Cache the hashed view: powerful combo. Apply (1) and (2) above to speed up things a lot. I hope this helps you a bit, -- Tito On Friday, November 23, 2001, at 08:16 AM, Pascal Baspeyras wrote: > I do hope my nightmarish experience with mails has come to an end ! > =A0 > JCW kindly wrote: > >You're using the latest code from CVS, right? > =A0 > Right. > =A0 > >Why are you saving Year as string?=A0 Int would be much faster. > =A0 > I'll of course do that later, but for the sake of the test, it should=20= > not matter. > =A0 > >What happens if you do not do commit at all, how's performance as=20 > things grow? > As things grow, performance is bad. The program very rapidly slows = down=20 > because > Find() is called for each new value entered, and Find() takes fastly=20= > increasing time. > Commit() is one big issue, and Find() is another one. > I could understand Commit() takes increasing time (though I don't like=20= > it), but Find() > should not slow down at such a noticeable pace. It's a hash lookup ! > =A0 > =A0 > (Voyons Jean-Claude... Les bronz=E9s font du ski, c'est pas de la=20 > litt=E9rature) > ____________________________________________ "Have you ever noticed? Anybody going slower than you is an idiot, and anyone going faster than you is a maniac." George Carlin --Apple-Mail-1-314704782 Content-Transfer-Encoding: quoted-printable Content-Type: text/enriched; charset=ISO-8859-1 Salut Pascal, I'd like to share with you a few things I've noticed regarding MetaKit's performance. 1) If your data file grows very quickly (as in commit-extend mode): compact it upon save. With this mode, data grows quickly because a change in a column requires the entire column to be rewritten. In my data file, I know that a particular record is, at the most, 400 bytes. So, when I save, I calculate roughly the size of the record: (file size/number of records). If the resulting record size is greater than, say, 600 bytes, I trigger the compacting method. This has proven to reduce the file size to its minimum size and improve search speeds quite a bit. 2) GetAs() is expensive: cache the view. It's way better to create one and leave it open, then perform all ops with it. Sampling thereafter shows quite a different picture. 3) A hashed view may be your best friend: using it whenever possible. I use my own call rowWithUID() quite a bit. The first column is a unique ID which I use extensively to obtain its row. Using hashed view + Find() has proven to be an *enormous* gain. 4) Cache the hashed view: powerful combo. Apply (1) and (2) above to speed up things a lot. I hope this helps you a bit, -- Tito On Friday, November 23, 2001, at 08:16 AM, Pascal Baspeyras wrote: ArialI do hope my nightmarish experience with mails has come to an end = ! =A0 ArialJCW kindly = wrote: Arial>You're using the latest code from CVS, right? =A0 ArialRight. =A0 >Why are you saving Year as string?=A0 Int would be much faster. =A0 I'll of course do that later, but for the sake of the test, it should not matter. =A0 Arial>What happens if you do not do commit at all, how's performance as things grow? As things grow, performance is bad. The program very rapidly slows down because ArialFind() is called for each new value entered, and Find() takes fastly increasing = time. ArialCommit() is one big issue, and Find() is another one. ArialI could understand Commit() takes increasing time (though I don't like it), but = Find() Arialshould not slow down at such a noticeable pace. It's a hash lookup ! =A0 =A0 Arial(Voyons Jean-Claude... Les bronz=E9s font du ski, c'est pas de la = litt=E9rature) Lucida Grande = ____________________________________________Lucida Grande 0000,0000,FFFF"Have you ever noticed? Anybody going slower than you is an idiot, and anyone going faster than you is a maniac." = 0000,6666,3333 George Carlin= --Apple-Mail-1-314704782-- From baspey@yahoo.com Fri Nov 23 15:04:42 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id PAA23191 for ; Fri, 23 Nov 2001 15:04:41 -0600 Received: from toulouse-2-a7-36-250.dial.proxad.net (HELO desktop) (62.147.36.250) by smtp.mail.vip.sc5.yahoo.com with SMTP; 23 Nov 2001 20:57:14 -0000 Message-ID: <000e01c17461$3a3e5da0$fa24933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 23 Nov 2001 21:54:26 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bonjour amis francophiles, Many thanks to Tito and Berk (I don't like much Jean-Louis Murat either) for giving out their precious optimization secrets. A special thank to Tito, whose mail looked almost as ugly as mines. I finished a program intended to extensively test Metakit performance: I randomly generate string and add them to a blocked/hash view, and I do nothing else, hence no risk of perturbation. With only 1 field, the Commit() is still as bad, but surprisingly, the Find() is correct. Now, I will try with 2 hashed fields. And after that, I will see if changing the commit mode improves things. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From readonly@getsoft.com Fri Nov 23 15:42:57 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA24946 for ; Fri, 23 Nov 2001 15:42:57 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=OMNIBOOK) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 167Nxd-0003kv-00; Fri, 23 Nov 2001 14:34:13 -0700 Date: Sat, 24 Nov 2001 08:35:50 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <77163008358.20011124083550@getsoft.com> To: "Pascal Baspeyras" CC: "Metakit" Subject: Re: [Metakit] Performance can't be that bad In-Reply-To: <000e01c17461$3a3e5da0$fa24933e@desktop> References: <000e01c17461$3a3e5da0$fa24933e@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It seems that many questions posted on the Metakit list revolve around performance issues, and the various hacks people finish up using to to try and get the performance they need for the specific applications. Jean-Claude states that Metakits performance varies dramatically depending on how you use it, which seems to be related to its column oriented design. From my personal experience I spent a solid 6+ weeks trying to use Metakit in an application. I was able to dramatically improve performance in some areas, but got absolutely nowhere in others. For example sorting a view with a large number of rows took far too long. This was earlier in the year, so things may have improved by now. The bottom line for me was that I wasn't able to get either acceptable performance, nor reliable use out of MK. MK is an elegant and interesting DB system, but in my experience it proved too difficult to use, was too unpredictable in its performance and required trying lots of hacks to get it to perform for each different DB design you through at it. I put this all down to a learning experience and moved on, writing a DB of my own with which exhibits very good performance without me worrying about how big it gets. The DB I'm referring to is the Source Database engine at the heart of the Source Browser/Class View in my programmers editor, ED for Windows (www.getsoft.com). I applaud the work Jean-Claude has done, but at the same time people need to determine whether MK is the appropriate solution for them, and consider how much time they are willing to invest in order to make this decision. Saturday, November 24, 2001, 7:54:26 AM, you wrote: PB> Bonjour amis francophiles, PB> Many thanks to Tito and Berk (I don't like much Jean-Louis Murat either) for PB> giving out their precious optimization secrets. PB> A special thank to Tito, whose mail looked almost as ugly as mines. PB> I finished a program intended to extensively test Metakit performance: PB> I randomly generate string and add them to a blocked/hash view, and I do PB> nothing PB> else, hence no risk of perturbation. PB> With only 1 field, the Commit() is still as bad, but surprisingly, the PB> Find() is correct. PB> Now, I will try with 2 hashed fields. PB> And after that, I will see if changing the commit mode improves things. PB> _________________________________________________________ PB> Do You Yahoo!? PB> Get your free @yahoo.com address at http://mail.yahoo.com PB> _______________________________________________ PB> metakit mailing list - metakit@equi4.com PB> http://www.equi4.com/mailman/listinfo/metakit -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From baspey@yahoo.com Fri Nov 23 16:58:46 2001 Received: from smtp010.mail.yahoo.com (smtp010.mail.yahoo.com [216.136.173.30]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA28742 for ; Fri, 23 Nov 2001 16:58:45 -0600 Received: from toulouse-1-a7-34-199.dial.proxad.net (HELO desktop) (62.147.34.199) by smtp.mail.vip.sc5.yahoo.com with SMTP; 23 Nov 2001 22:51:18 -0000 Message-ID: <000701c17471$2a224520$c722933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 23 Nov 2001 23:48:49 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >From my personal experience I spent a solid 6+ weeks trying to use >Metakit in an application. I stick to Metakit mostly because I also spent a lot of time trying to master it. Indeed, I find documentation awful. Information is often somehere, but the question is: where ? In the end, the source code is often the best source of information (what about a real synthesis, not just "wiki.cgi\4_2a.htm" and the prehistorical "intro.htm"). Now that I have decided for a database design according to my needs, I'm terribly disappointed by the performance. By posting to that group, I was hoping someone tells me what blunt mistake I was making. >I put this all down to a learning experience and moved on, writing a >DB of my own with which exhibits very good performance without me >worrying about how big it gets. The DB I'm referring to is the Source >Database engine at the heart of the Source Browser/Class View in my >programmers editor, ED for Windows (www.getsoft.com). I went to your site, and: 1/ ED isn't free nor open source 2/ ED for Windows does it work on Macintosh and Linux ? 3/ ED database format can it go from one platform to another one transparently ? I may indeed give up Metakit and move on to Berkeley DB, but I will first hand my evaluation program to Jean-Claude for him to give his own interpretation. My guess is that Jean-Claude can't make thorough tests of Metakit because users use it in very various ways, and that there are still imperfections in the recent mapping views. But to tell the truth, I still hope that the malfunction is all my fault. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From readonly@getsoft.com Fri Nov 23 17:23:35 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA30325 for ; Fri, 23 Nov 2001 17:23:35 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 167PYB-0007uT-00; Fri, 23 Nov 2001 16:16:03 -0700 Date: Sat, 24 Nov 2001 10:18:17 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <93286188377.20011124101817@getsoft.com> To: "Pascal Baspeyras" CC: "Metakit" Subject: Re: [Metakit] Performance can't be that bad In-Reply-To: <000701c17471$2a224520$c722933e@desktop> References: <000701c17471$2a224520$c722933e@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: PB> Neville Franks wrote: >>From my personal experience I spent a solid 6+ weeks trying to use >>Metakit in an application. PB> I stick to Metakit mostly because I also spent a lot of time trying to PB> master it. PB> Indeed, I find documentation awful. Information is often somehere, but PB> the question is: where ? In the end, the source code is often the best PB> source PB> of information (what about a real synthesis, not just "wiki.cgi\4_2a.htm" PB> and PB> the prehistorical "intro.htm"). Yes I agree the documentation is awful. In my experience this is a common trait of "Free Open Source Software". I have largely lost all interest in Open Source because of issues like this. There are some very nice and interesting projects around, but the cost to try and use them is far, far, far too high. And again in my experience you invest a lot of time only to find out they are unusable for whatever reason. As a wise person said once, "there is no such thing as a free lunch." PB> Now that I have decided for a database design according to my needs, I'm PB> terribly disappointed by the performance. By posting to that group, I was PB> hoping someone tells me what blunt mistake I was making. >>I put this all down to a learning experience and moved on, writing a >>DB of my own with which exhibits very good performance without me >>worrying about how big it gets. The DB I'm referring to is the Source >>Database engine at the heart of the Source Browser/Class View in my >>programmers editor, ED for Windows (www.getsoft.com). PB> I went to your site, and: PB> 1/ ED isn't free nor open source Correct, I'm afraid I can't put food on my table by giving away my work. This is my livelihood, not something I do as a plaything. PB> 2/ ED for Windows does it work on Macintosh and Linux ? The Mac market is too small, and Linux people expect everything to be available for free, and come with source. PB> 3/ ED database format can it go from one platform to another one PB> transparently ? Like ED itself the Database is not meant to be cross platform. At present it is designed specifically meet EDs requirements - ie. fast, large databases. This includes very fast updates, and lookups and full multithreaded support. As you edit your code the database gets updated in real time, without any impact on your editing. PB> I may indeed give up Metakit and move on to Berkeley DB, but I will first PB> hand PB> my evaluation program to Jean-Claude for him to give his own interpretation. I'd suggest also looking at MySQL. I'm using that on my Web server and have been very impressed with its capabilities. That said it is a totally different product to MK. PB> My guess is that Jean-Claude can't make thorough tests of Metakit because PB> users PB> use it in very various ways, and that there are still imperfections in the PB> recent mapping views. PB> But to tell the truth, I still hope that the malfunction is all my fault. That wasn't my experience, as I stated earlier. PB> _________________________________________________________ PB> Do You Yahoo!? PB> Get your free @yahoo.com address at http://mail.yahoo.com PB> _______________________________________________ PB> metakit mailing list - metakit@equi4.com PB> http://www.equi4.com/mailman/listinfo/metakit -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From baspey@yahoo.com Fri Nov 23 20:08:23 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id UAA11643 for ; Fri, 23 Nov 2001 20:08:21 -0600 Received: from toulouse-2-a7-36-94.dial.proxad.net (HELO desktop) (62.147.36.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 02:00:51 -0000 Message-ID: <000e01c1748b$a43e3160$5e24933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 01:22:17 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >Yes I agree the documentation is awful. What I hate the most is that you read things which might have been stated 2 years ago, and never know if a limitation still applies or not. >In my experience this is a common trait of >"Free Open Source Software". "Free Open Source Software" can be really great. What about Linux itself ? I also use wxWindows which I find marvellous. Speaking of wxWindows, I have conversations directly with the authors, which is always an incredible moment, just like when JCW personally answer my mails. Free software writers are passionated, talented, and they generally do a great job. Quality isn't proportional to the price, far from it. >The Mac market is too small. I have chosen wxWindows and Metakit for my program to be available on a maximum of platforms. It's an important requirement. By the way, I hate the word "Market". >I'd suggest also looking at MySQL. MySQL uses its own database system, but can also integrate the Berkeley Database, which is a sign of quality. Berkeley DB has great portability, small footprint and source availability like another database I know (Metakit). Free software writers deserve the greatest indulgence and the greatest gratitude for giving away their free time like this. They are the "Mother Theresa" of the computer world. In fact, I foretell that they will rule it in the end (bye bye M$...) _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jyl@best.com Fri Nov 23 20:13:12 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA12030 for ; Fri, 23 Nov 2001 20:13:02 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id SAA05285; Fri, 23 Nov 2001 18:04:13 -0800 (PST) Message-ID: <3BFF0E75.7020006@best.com> Date: Fri, 23 Nov 2001 19:05:25 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Neville Franks CC: Pascal Baspeyras , Metakit Subject: Re: [Metakit] Performance can't be that bad References: <000e01c17461$3a3e5da0$fa24933e@desktop> <77163008358.20011124083550@getsoft.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Three comments: * I browsed your web site and couldn't find the DB available as a separate product. Is that intentional? * Since I couldn't examine your DB I cannot comment on the performance and features. I dont know whether the DB in your editor is as rich as MK and in fact whether it needs to be as rich as MK, for the uses you intend. * I've used MK extensively and quickly learned to use it efficiently. My use is the simplest possible. I dont use Find(), hashed views, and so forth. I found all derived views to be too slow to use for my purposes, but learned to do without. You can see how I use MK by examining http://e4graph.sourceforge.net, the source code is all available. All MK related functionality is in the subdirectory mkdriver in the source tree. Regards, --JYL >It seems that many questions posted on the Metakit list revolve around >performance issues, and the various hacks people finish up using to to >try and get the performance they need for the specific applications. > >Jean-Claude states that Metakits performance varies dramatically >depending on how you use it, which seems to be related to its column >oriented design. > >>From my personal experience I spent a solid 6+ weeks trying to use >Metakit in an application. I was able to dramatically improve >performance in some areas, but got absolutely nowhere in others. For >example sorting a view with a large number of rows took far too long. >This was earlier in the year, so things may have improved by now. > >The bottom line for me was that I wasn't able to get either acceptable >performance, nor reliable use out of MK. MK is an elegant and >interesting DB system, but in my experience it proved too difficult to >use, was too unpredictable in its performance and required trying lots >of hacks to get it to perform for each different DB design you >through at it. > >I put this all down to a learning experience and moved on, writing a >DB of my own with which exhibits very good performance without me >worrying about how big it gets. The DB I'm referring to is the Source >Database engine at the heart of the Source Browser/Class View in my >programmers editor, ED for Windows (www.getsoft.com). > >I applaud the work Jean-Claude has done, but at the same time people >need to determine whether MK is the appropriate solution for them, and >consider how much time they are willing to invest in order to make this >decision. > > >Saturday, November 24, 2001, 7:54:26 AM, you wrote: > >PB> Bonjour amis francophiles, > >PB> Many thanks to Tito and Berk (I don't like much Jean-Louis Murat either) for >PB> giving out their precious optimization secrets. >PB> A special thank to Tito, whose mail looked almost as ugly as mines. > >PB> I finished a program intended to extensively test Metakit performance: >PB> I randomly generate string and add them to a blocked/hash view, and I do >PB> nothing >PB> else, hence no risk of perturbation. > >PB> With only 1 field, the Commit() is still as bad, but surprisingly, the >PB> Find() is correct. >PB> Now, I will try with 2 hashed fields. >PB> And after that, I will see if changing the commit mode improves things. > >PB> _________________________________________________________ >PB> Do You Yahoo!? >PB> Get your free @yahoo.com address at http://mail.yahoo.com > >PB> _______________________________________________ >PB> metakit mailing list - metakit@equi4.com >PB> http://www.equi4.com/mailman/listinfo/metakit > > From jcw@equi4.com Fri Nov 23 21:10:32 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA16866 for ; Fri, 23 Nov 2001 21:10:32 -0600 Received: from takkie.equi4.local ([195.121.234.41]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNAB4800.MFY for ; Sat, 24 Nov 2001 04:02:32 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAO317002326 for ; Sat, 24 Nov 2001 04:01:17 +0100 From: Jean-Claude Wippler To: Metakit Subject: Re: [Metakit] Performance can't be that bad Date: Sat, 24 Nov 2001 04:03:54 +0100 Message-Id: <20011124030354.7219@mail.planet.nl> In-Reply-To: <000701c17471$2a224520$c722933e@desktop> References: <000701c17471$2a224520$c722933e@desktop> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tsk, tsk, tsk: "it's a bad worker who blames his tools" ... Let's get this right, folks: MK run circles around Berkeley DB and MySQL, under the right circumstances. Feel free to find out for yourself. Others have. And yes, I keep saying that at times MK's performance can go down a *lot*. With so much finger pointing going on, I simply had to go check things. Below are some results, using the Tcl scripting language on a PIII/650 under SuSE Linux 7.3, with 256 MB RAM. C++ ought to perform better. The results - as I see them, and I apologize if I made mistakes: - the whole test, creating 6 files of 500,000 rows, ran in 11 minutes (it consists of adding unique strings, constructed from /usr/dist/words) - I'm only doing a single commit at the end, and have no subviews in here - times below are in millisecs, f1/f2/f5 are flat, b1/b2/b5 are blocked, f1 is 1 key + 1 val, f2 is 2 keys, f5 is 2 keys + 4 vals (ditto for b*) - in the worst case, it takes under 7s to add 10,000 rows (see "b5" below) (blocked, 2-key hash, 6 string props, at 500,000 rows, file size 52 Mb) - in the unblocked case, performance is linear (.75 sec/10k-rows for f2), in the blocked case it degrades by a factor 2x from start to finish - finds take 54 .. 104 microseconds, commits between 1.5 and 25.7 seconds I do not see a significant slowdown for 1-key vs. 2-key hashes. I do not see O(N) behavior anywhere, other than in the number or properties / file size. I'll ask again, Pascal: are you sure that you have the latest code from CVS? The linear slowdown you have been reporting sounds similar to a recent bug, which was resolved resolved (check the change logs). Now can we please get on with it? I can't really be expected to help everyone debug their software, nor even help them learn it, can I? When problems are reported, and that is what this list is for, then there are two equally effective ways to progress: try to figure out things yourself (and let us all know, please!), or construct a small example that illustrates what is happening in a way others can easily reproduce. I'm happy to say that several people have done this in the past, and it has been a great way to fix/improve MK. And for cases where the "problems" are your own: be glad that many will be answered by subscribers on this list, simply because they can spot some mistakes easily. My software is free, and I'm doing just fine commercially thank you, but my time is limited. The deal is very simple: if we all join forces to improve the software and the docs, then I'll do my best to fix issues where I can, and to integrate improvements others offer. And now that there is a rare opportunity to say this: my warmest thanks to all the people who have helped (and continue to help) find/fix problems. You share the credit for making MK what it is today, and for what it will be tomorrow. -jcw P.S. Here is the complete test output, followed by the complete Tcl script: ===================================== ========================================== % ntclkit slow.tcl w = contributory Sat Nov 24 00:37:31 CET 2001 f1: .................................................. 500000 rows, 22890442 b 0: k1 k1_Chapman0 v v_Chapman0 1: k1 k1_beckoning0 v v_beckoning0 2: k1 k1_belittled0 v v_belittled0 f2: .................................................. 500000 rows, 23390449 b 0: k1 k1_Chapman0 k2 k2_Chapman0 v {} 1: k1 k1_beckoning0 k2 k2_beckoning0 v {} 2: k1 k1_belittled0 k2 k2_belittled0 v {} f5: .................................................. 500000 rows, 52893926 b 0: k1 k1_Chapman0 k2 k2_Chapman0 v1 v1_Chapman0 v2 v2_Chapman0 v3 v3_Chapman0 v v_Chapman0 1: k1 k1_beckoning0 k2 k2_beckoning0 v1 v1_beckoning0 v2 v2_beckoning0 v3 v3_beckoning0 v v_beckoning0 2: k1 k1_belittled0 k2 k2_belittled0 v1 v1_belittled0 v2 v2_belittled0 v3 v3_belittled0 v v_belittled0 b1: .................................................. 500000 rows, 22918775 b 0: k1 k1_Chapman0 v v_Chapman0 1: k1 k1_beckoning0 v v_beckoning0 2: k1 k1_belittled0 v v_belittled0 b2: .................................................. 500000 rows, 23420800 b 0: k1 k1_Chapman0 k2 k2_Chapman0 v {} 1: k1 k1_beckoning0 k2 k2_beckoning0 v {} 2: k1 k1_belittled0 k2 k2_belittled0 v {} b5: .................................................. 500000 rows, 52974159 b 0: k1 k1_Chapman0 k2 k2_Chapman0 v1 v1_Chapman0 v2 v2_Chapman0 v3 v3_Chapman0 v v_Chapman0 1: k1 k1_beckoning0 k2 k2_beckoning0 v1 v1_beckoning0 v2 v2_beckoning0 v3 v3_beckoning0 v v_beckoning0 2: k1 k1_belittled0 k2 k2_belittled0 v1 v1_belittled0 v2 v2_belittled0 v3 v3_belittled0 v v_belittled0 Sat Nov 24 00:48:28 CET 2001 b1 b2 b5 f1 f2 f5 cols( 10000) = 1242.50 1304.00 2680.61 777.08 795.82 1479.10 cols( 20000) = 1335.40 1433.63 2929.18 770.77 842.36 1480.55 cols( 30000) = 1448.52 1549.82 3107.09 837.14 907.41 1414.22 cols( 40000) = 1344.77 1441.45 3042.39 741.56 757.63 1464.62 cols( 50000) = 1683.81 1806.10 3453.16 1013.09 1148.84 1566.98 cols( 60000) = 1358.38 1472.10 3180.01 716.69 889.23 1342.18 cols( 70000) = 1394.52 1527.15 3295.67 724.70 783.99 1406.92 cols( 80000) = 1440.27 1556.47 3344.69 735.12 804.01 1379.41 cols( 90000) = 2264.91 2396.75 4200.34 1190.70 1644.88 1940.82 cols(100000) = 1470.20 1610.09 3472.78 713.38 929.40 1319.38 cols(110000) = 1503.52 1637.00 3518.44 723.18 741.28 1318.49 cols(120000) = 1522.37 1667.66 3631.21 718.91 789.59 1322.77 cols(130000) = 1541.93 1702.36 3726.43 724.64 735.77 1325.02 cols(140000) = 1577.06 1741.61 3728.57 725.03 752.13 1336.44 cols(150000) = 1600.60 1768.46 3836.94 732.58 767.03 1349.27 cols(160000) = 1628.90 1807.21 3864.10 738.50 774.70 1351.62 cols(170000) = 1651.83 1853.33 3944.51 755.37 787.41 1357.57 cols(180000) = 3694.33 3905.01 6139.05 1654.60 1680.87 2274.20 cols(190000) = 1660.46 1857.05 3954.51 715.48 751.46 1326.79 cols(200000) = 1670.83 1878.95 4097.30 708.23 751.60 1329.17 cols(210000) = 1702.75 1906.41 4129.13 716.49 745.58 1354.67 cols(220000) = 1719.62 1926.42 4242.19 719.36 753.08 1334.60 cols(230000) = 1741.48 1968.90 4346.46 720.20 755.74 1313.73 cols(240000) = 1758.04 1992.53 4384.92 714.93 755.37 1322.02 cols(250000) = 1785.96 2018.08 4446.50 724.12 757.57 1320.53 cols(260000) = 1791.03 2016.54 4448.53 719.30 767.97 1344.32 cols(270000) = 1831.66 2072.68 4582.49 724.47 765.00 1354.10 cols(280000) = 1854.82 2093.85 4685.77 734.55 768.16 1348.03 cols(290000) = 1871.86 2131.60 4739.70 725.76 771.90 1373.31 cols(300000) = 1900.85 2159.78 4783.72 740.63 770.87 1357.07 cols(310000) = 1917.00 2179.84 4891.21 736.46 781.40 1364.52 cols(320000) = 1950.14 2204.38 4961.11 740.58 778.29 1379.75 cols(330000) = 1969.40 2233.66 5056.38 750.75 790.07 1371.60 cols(340000) = 1980.81 2270.14 5156.39 748.22 783.81 1381.50 cols(350000) = 7900.49 8180.1911189.29 2625.26 2637.88 3265.47 cols(360000) = 2011.92 2275.95 5314.29 719.10 756.82 1334.90 cols(370000) = 2036.50 2305.38 5391.69 720.18 752.38 1355.60 cols(380000) = 2044.61 2329.94 5495.08 716.65 751.96 1347.62 cols(390000) = 2064.97 2388.83 5586.73 718.49 762.35 1354.92 cols(400000) = 2088.34 2410.12 5722.40 712.94 751.62 1349.71 cols(410000) = 2100.25 2432.17 5781.90 714.60 767.75 1377.17 cols(420000) = 2120.84 2458.69 5895.63 720.20 749.74 1374.17 cols(430000) = 2146.21 2479.88 5981.36 719.72 763.38 1365.93 cols(440000) = 2159.94 2505.02 6129.47 726.59 757.63 1373.81 cols(450000) = 2172.09 2528.55 6223.94 721.32 762.63 1346.21 cols(460000) = 2203.20 2559.08 6342.73 727.50 759.12 1351.56 cols(470000) = 2209.74 2591.13 6494.58 724.56 762.50 1374.38 cols(480000) = 2246.69 2597.51 6622.03 720.50 760.39 1365.53 cols(490000) = 2249.65 2632.48 6716.73 730.03 770.98 1362.94 cols(500000) = 2269.11 2643.57 6849.40 724.77 765.88 1390.11 stats(b1-commit) = 1609.204 stats(b1-find) = 710.0 stats(b2-commit) = 1888.466 stats(b2-find) = 990.0 stats(b5-commit) = 25706.905 stats(b5-find) = 1040.0 stats(f1-commit) = 1575.921 stats(f1-find) = 540.0 stats(f2-commit) = 1851.443 stats(f2-find) = 780.0 stats(f5-commit) = 13972.162 stats(f5-find) = 780.0 Sat Nov 24 00:48:28 CET 2001 % ===================================== ========================================== # Trying to find out how hash performance degrades as size increases if [catch {package require Mk4tcl}] { catch {load ./Mk4tcl.so mk4tcl} catch {load ./Mk4tcl_d.dll mk4tcl} } proc loadwords {step} { global warray set fd [open words] for {set i 0} {$i < $step && [gets $fd line] >= 0} {incr i} { set warray($line) $i } close $fd return $line } proc timedRun {tag count args} { set usec [lindex [time $args $count] 0] lappend ::stats($tag) [expr {$count*$usec/1000.0}] } proc setup {keys type} { file delete _large.mk mk::file open db _large.mk -nocommit set layout "$keys v" if {$type == "b"} { set layout "{_B {$layout}}" } mk::view layout db.words $layout if {$type == "b"} { mk::view open db.words rawdata rename [rawdata view blocked] data } else { mk::view open db.words data } mk::view layout db.words_map {_H:I _R:I} mk::view open db.words_map map rename [data view hash map 1] words } proc teardown {} { catch { rename words "" } catch { rename data "" } catch { rename map "" } catch { rename rawdata "" } mk::file close db } proc fill1 {seq} { global warray foreach {k v} [array get warray] { set x $k$seq words insert end k1 k1_$x v v_$x } } proc fill2 {seq} { global warray foreach {k v} [array get warray] { set x $k$seq words insert end k1 k1_$x k2 k2_$x } } proc fill5 {seq} { global warray foreach {k v} [array get warray] { set x $k$seq words insert end k1 k1_$x k2 k2_$x v1 v1_$x v2 v2_$x v3 v3_$x v v_$x } } proc find1 {w} { words find k1 k1_${w}5 } proc find2 {w} { words find k1 k1_${w}5 k2 k2_${w}5 } proc find5 {w} { words find k1 k1_${w}5 k2 k2_${w}5 } set step 10000 #set step 100 set mult 50 set w [loadwords $step] puts "w = $w" puts [clock format [clock seconds]] foreach type {f b} { foreach keys {{k1} {k1 k2} {k1 k2 v1 v2 v3}} { set nkeys [llength $keys] set mode "$type$nkeys" puts -nonewline stderr "$mode: " setup $keys $type for {set i 0} {$i < $mult} {incr i} { timedRun $mode-fill 1 fill$nkeys $i puts -nonewline stderr . } timedRun $mode-find 10000 find$nkeys $w timedRun $mode-commit 1 mk::file commit db puts stderr " [words size] rows, [file size _large.mk] b" for {set i 0} {$i < 3} {incr i} { puts " $i: [words get $i]" } teardown } } puts [clock format [clock seconds]] puts -nonewline "\n " foreach x [lsort [array names stats *-fill]] { puts -nonewline [format %8s [lindex [split $x -] 0]] set i 0 foreach y $stats($x) { incr i $step append cols([format %6d $i]) [format {%8.2f} $y] } unset stats($x) } puts \n parray cols puts "" parray stats puts "" puts [clock format [clock seconds]] ===================================== ========================================== From baspey@yahoo.com Fri Nov 23 21:31:49 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA17528 for ; Fri, 23 Nov 2001 21:31:49 -0600 Received: from toulouse-2-a7-36-80.dial.proxad.net (HELO desktop) (62.147.36.80) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 03:24:22 -0000 Message-ID: <001901c17497$4f4cc700$5024933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 04:20:37 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >I've used MK extensively and quickly learned to use it efficiently. My >use is the simplest possible. I dont use Find(), hashed views, and so >forth. I found all derived views to be too slow to use for my purposes, >but learned to do without. In my idea, Metakit was simple, rustic. No fancy features like security, SQL, elaborated relational concepts. Furthermore, there are smart ideas like colum-wise data structure and memory mapped files (also on-the-fly restructuring, but this I consider rather marginal). How could this be so ill-programmed that it performs much worse than other databases which perform much complicated tasks ?? I'd like to know what is the secret of these other databases. Because a CPU can perform a certain number of operations per second, and no more. I do hope there isn't a profound flaw in Metakit. Has someone ever achieved ambitious realisation with Metakit ? I mean big database files, or use of advanced functionality like hashing. Not just serialization of initialisation file. I will now repost a message I saw, dated of 14th october 2001, and which sounds like science-fiction (the subject was "VLDB performance ?"). I will try to get in touch with Michael P. Wilson to know how things have turned out. Here is the answer JCW gave him: Michael P. Wilson wrote: >I've been lurking around looking for a relatively low-level library for >maintaining an extremely large (ok, to me anyway) database of fairly >large objects (about 3k of large-ish text columns plus blobs of up to 5 >meg.) > >The semantic needs are very thin. I only ever need append and >read-only access, including fairly basic selection and filtering across >the set. > >Are there any immediate "gotchas" in MetaKit that I should know about >before giving it a shot? I don't need complex data types, advanced >'select' management, record updating, etc. However, all fields are >variable length including the blob field. > >Oh, btw, when I say "extremely large" I mean something in excess of a >1.2Terabytes (and growing at about 100-200 meg a day.) Most things you say are within what I'd consider tough but still feasible. Large objects do not in themselves cause much trouble, because they are stored individually. Access to a row when the large object field is not used, is as fast as if the large object had not been present, due to the column-wise data organization. But the 1.2 Tb kills things. MK is based on memory mapped files and (currently) requires that the entire file be mapped (R/O). On a 32-bit machine, don't even *think* about datasets over 2 Gb. There is only one way to make this fly: use a 64-bit architecture which supports mapping the entire DBMS. I'd be interested to see where bottlenecks turn up, if any, and then try to resolve it. There are numerous tricks to apply, and the view mechanism on top of MK allows for a wide range of "special-purpose cleverness". Read-only data in itself makes the issue feasible IMO. The "append-only" mode you need is trickier, because - again due to the column-wise design - appends, even of a single item, normally have the effect of copying an entire column. One solution is to segment the data, and then recombine it virtually - which the various custom views can IMO be made to do just fine. But let's not forget the reality that 1 Tb is way, way, over what MK has been used for today - and ever considered for AFAIK. It will take deep knowledge of the access/use/modification patterns and of the MK design to have any chance at success with this. Just a thought: why not save the data serially to file(s), and have MK *manage* it, i.e. store offsets and some key navigational data, for example? Transaction safety can surely be obtained, and the amount of data residing in MK could go down dramatically. Again, keep in mind that custom views can hide this physical design choice. -jcw _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Fri Nov 23 21:59:03 2001 Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA18118 for ; Fri, 23 Nov 2001 21:59:03 -0600 Received: from toulouse-2-a7-37-194.dial.proxad.net (HELO desktop) (62.147.37.194) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 03:51:36 -0000 Message-ID: <005101c1749b$1d18c500$5024933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 04:49:11 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >"it's a bad worker who blames his tools" ... I've never been happier to discover I'm a bad worker ! >MK run circles around Berkeley DB and MySQL, >under the right circumstances. I believe you, and that's great news because it would have been terrible to have wasted my time and be compelled to move on to another solution. >I'll ask again, Pascal: are you sure that you have the latest code from >CVS? The linear slowdown you have been reporting sounds similar to a >recent bug, which was resolved resolved (check the change logs). I'm gonna reload and recompile everything. I would be so relieved to discover I've mixed-up somewhere. >Now can we please get on with it? I feel really sorry to have been some source of discordance. But for a while, I was wondering if Metakit wasn't fake... But I'm now convinced it's not. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Fri Nov 23 22:21:46 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id WAA18578 for ; Fri, 23 Nov 2001 22:21:45 -0600 Received: from toulouse-2-a7-38-136.dial.proxad.net (HELO desktop) (62.147.38.136) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 04:14:18 -0000 Message-ID: <000701c1749e$491a4e00$8826933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 05:11:54 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Is JCW rich ? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >My software is free, and I'm doing just fine commercially thank you. Does this mean you get some income from Metakit. That would be good news, but I can't figure out how. And I'm curious... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From readonly@getsoft.com Fri Nov 23 23:24:45 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19808 for ; Fri, 23 Nov 2001 23:24:45 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 167V9k-0002mw-00; Fri, 23 Nov 2001 22:15:12 -0700 Date: Sat, 24 Nov 2001 16:17:26 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <21307737703.20011124161726@getsoft.com> To: Jacob Levy CC: Pascal Baspeyras , Metakit Subject: Re[2]: [Metakit] Performance can't be that bad In-Reply-To: <3BFF0E75.7020006@best.com> References: <000e01c17461$3a3e5da0$fa24933e@desktop> <77163008358.20011124083550@getsoft.com> <3BFF0E75.7020006@best.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jacob, Saturday, November 24, 2001, 2:05:25 PM, you wrote: JL> Neville JL> Three comments: JL> * I browsed your web site and couldn't find the DB available as a JL> separate product. Is that intentional? The DB is something I've developed purely for use within ED. I have no intention or interest in having it available as a separate product. It is used to track every function, procedure, class, struct etc, within a project, and is updated in real time as you edit the code. It works very effectively with large projects with many thousands of files and 100's of thousands of rows. This is what I was trying to use MK for, but was unable to. JL> * Since I couldn't examine your DB I cannot comment on the performance JL> and features. I dont know whether the DB in your editor is as rich as MK JL> and in fact whether it needs to be as rich as MK, for the uses you intend. It is not in any way intended to be compared or compete with MK. It is specific to my requirements and certainly isn't as feature rich as MK, nor does it need to be. It does have very fast B+Tree access, which is something that was lacking in MK when I tried to use it, and something I very much need. JL> * I've used MK extensively and quickly learned to use it efficiently. My JL> use is the simplest possible. I dont use Find(), hashed views, and so JL> forth. I found all derived views to be too slow to use for my purposes, JL> but learned to do without. I think this is one of the big problems. ie. You need to find some way to get MK to work for you, and this seems to different for each application. If after much effort you still can't find a efficient path what do you do? I had a situation where a sort on 100,000 rows in MK would take 5-10 minutes, which made it impossible for me to use in this specific application. If MK had had indexed access then it would have been a very different picture. JL> You can see how I use MK by examining JL> http://e4graph.sourceforge.net, the source code is all available. All MK JL> related functionality is in the subdirectory mkdriver in the source tree. Thanks. JL> Regards, --JYL >>It seems that many questions posted on the Metakit list revolve around >>performance issues, and the various hacks people finish up using to to >>try and get the performance they need for the specific applications. >> >>Jean-Claude states that Metakits performance varies dramatically >>depending on how you use it, which seems to be related to its column >>oriented design. >> >>>From my personal experience I spent a solid 6+ weeks trying to use >>Metakit in an application. I was able to dramatically improve >>performance in some areas, but got absolutely nowhere in others. For >>example sorting a view with a large number of rows took far too long. >>This was earlier in the year, so things may have improved by now. >> >>The bottom line for me was that I wasn't able to get either acceptable >>performance, nor reliable use out of MK. MK is an elegant and >>interesting DB system, but in my experience it proved too difficult to >>use, was too unpredictable in its performance and required trying lots >>of hacks to get it to perform for each different DB design you >>through at it. >> >>I put this all down to a learning experience and moved on, writing a >>DB of my own with which exhibits very good performance without me >>worrying about how big it gets. The DB I'm referring to is the Source >>Database engine at the heart of the Source Browser/Class View in my >>programmers editor, ED for Windows (www.getsoft.com). >> >>I applaud the work Jean-Claude has done, but at the same time people >>need to determine whether MK is the appropriate solution for them, and >>consider how much time they are willing to invest in order to make this >>decision. >> >> >>Saturday, November 24, 2001, 7:54:26 AM, you wrote: >> >>PB> Bonjour amis francophiles, >> >>PB> Many thanks to Tito and Berk (I don't like much Jean-Louis Murat either) for >>PB> giving out their precious optimization secrets. >>PB> A special thank to Tito, whose mail looked almost as ugly as mines. >> >>PB> I finished a program intended to extensively test Metakit performance: >>PB> I randomly generate string and add them to a blocked/hash view, and I do >>PB> nothing >>PB> else, hence no risk of perturbation. >> >>PB> With only 1 field, the Commit() is still as bad, but surprisingly, the >>PB> Find() is correct. >>PB> Now, I will try with 2 hashed fields. >>PB> And after that, I will see if changing the commit mode improves things. >> >>PB> _________________________________________________________ >>PB> Do You Yahoo!? >>PB> Get your free @yahoo.com address at http://mail.yahoo.com >> >>PB> _______________________________________________ >>PB> metakit mailing list - metakit@equi4.com >>PB> http://www.equi4.com/mailman/listinfo/metakit >> >> -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From jcw@equi4.com Sat Nov 24 05:19:44 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA26504 for ; Sat, 24 Nov 2001 05:19:43 -0600 Received: from takkie.equi4.local ([195.121.238.150]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNAXRE01.LPK for ; Sat, 24 Nov 2001 12:11:38 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAOBAF004339 for ; Sat, 24 Nov 2001 12:10:26 +0100 From: Jean-Claude Wippler Cc: Metakit Subject: [Metakit] Performance can't be that bad Date: Sat, 24 Nov 2001 12:13:08 +0100 Message-Id: <20011124111308.18336@mail.planet.nl> In-Reply-To: <21307737703.20011124161726@getsoft.com> References: <21307737703.20011124161726@getsoft.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >The DB is something I've developed purely for use within ED. Well, it's kind of tricky to compare two things, when one is unavailable. >JL> * I've used MK extensively and quickly learned to use it efficiently. My >JL> use is the simplest possible. I dont use Find(), hashed views, and so >JL> forth. I found all derived views to be too slow to use for my purposes, >JL> but learned to do without. That was pre 2.0 or even earlier. Hashes and blocked views did not exist at the time. So again, it's a bit hard to comment about such a design today. >I think this is one of the big problems. ie. You need to find some way >to get MK to work for you, and this seems to different for each >application. If after much effort you still can't find a efficient >path what do you do? I had a situation where a sort on 100,000 rows >in MK would take 5-10 minutes, which made it impossible for me to use >in this specific application. If MK had had indexed access then it >would have been a very different picture. Gee, 5..10 minutes to sort 100,000 rows, how do you do that? Indexed access is easy. Create a custom sort, have it store the order in a second view, and use RemapWith to make that order persist. This is not 2.4 specific, it could have been done the same way years ago. Neville, could you do me a favor? Can you at least level the playing field by giving me a chance to reproduce and explain this? Perhaps by sending code or a description of code which explains the structure and dataset you used? As for how hard it is to make MK fly in everyone's particular case: if you learn to ride a bike, and then expect to be able to use that experience to drive a car, then boy you're in for a rough ride. Is it a drawback of MK that it is not a bike (i.e. not row-wise, but column-wise) in its design? My advice to everyone is: if you don't like MK, or don't have the ability, time, interest, or whatever to think through the concepts and take advantage of MK, then please... go ride a nice bike, and let the rest of us on this mailing list enjoy and enhance MK. -jcw From jcw@equi4.com Sat Nov 24 06:17:52 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA27644 for ; Sat, 24 Nov 2001 06:17:46 -0600 Received: from takkie.equi4.local ([195.121.235.183]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNB0GC02.13O for ; Sat, 24 Nov 2001 13:09:48 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAOC8a004849 for ; Sat, 24 Nov 2001 13:08:36 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Is JCW rich ? Date: Sat, 24 Nov 2001 12:55:29 +0100 Message-Id: <20011124115529.12245@mail.planet.nl> In-Reply-To: <000701c1749e$491a4e00$8826933e@desktop> References: <000701c1749e$491a4e00$8826933e@desktop> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Dear Pascal / popeye, >Jean-Claude Wippler wrote: >>My software is free, and I'm doing just fine commercially thank you. > >Does this mean you get some income from Metakit. >That would be good news, but I can't figure out how. >And I'm curious... Sure, but does that mean you have to repeatedly waste the time of hundreds of people subscribed to this mailing list with this sort of chit-chat? (FYI, yes I feel rich, in the sense that I make more every year from consulting than I spend. Thanks to MK, which is going a lot more places these days.) Please allow me to adjust the advice I gave to you before. First, I apologize for the time you had to waste on MetaKit. It is a silly design and will most likely never meet your requirements. You will be far better of by using Berkeley DB, MySQL, or Postgres. They are all free, extremely well documented, easy to learn and use, and with lots and lots of people to help. Second, please go play outside and socialize with real people a bit more often. It helps. Now, for the last time... can we get on with it, please? -jcw From jcw@equi4.com Sat Nov 24 06:17:54 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA27645 for ; Sat, 24 Nov 2001 06:17:49 -0600 Received: from takkie.equi4.local ([195.121.235.183]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNB0GD01.XY4 for ; Sat, 24 Nov 2001 13:09:49 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAOC8b004852 for ; Sat, 24 Nov 2001 13:08:37 +0100 From: Jean-Claude Wippler To: Subject: RE: [Metakit] Performance can't be that bad Date: Sat, 24 Nov 2001 13:10:46 +0100 Message-Id: <20011124121046.27655@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Berk, Murat wrote: >We had similar problems with insertion speed and find/search performace. >We find ways around those issues. (We are using 2.01 release) > >1) We don't use the Find and Search stuff in the Metakit for string stuff. You will find that hash/ordered views in 2.3/2.4 now automatically cause Find to work optimally. >We wrote custom Find and Search functions which deals with these issues. We >figured out that c4_StringRef is very fast and we are using it to access >data w/o extracting it out. > >bool Mk_Utils::Utf8StringIsEqualC4Property(const char* utf8_ptr, size_t len, >const c4_StringProp& p, const c4_RowRef& r) Ah, UTF8. Yes, it seems like altering the comparisons to it is long overdue. I look into how much work it would be to make these functions customizable (without introducing globals). >2) Preallocate rows before you insert anything. We allocate 512 rows at a >time. But initially it was very slow to add strings into these views. The >reason is the internal offset array is reallocated everytime and grows 1 at >a time. What we did was set the last row of the preallocated bunch to a >dummy value. What this does is allocate all the offset stuff for all 512 >rows and also leaves preallocated memory called slack in the data. When you >insert new data, it just shifts slack pointer around and works very fast. Thanks. This issue needs to be revisited as well. The grow-by-one behavior has changed, because more data now uses the efficient internal "roving-gap" technique. Let me get this straight: is this about the very simple situation of adding a row with string to a view, and doing so as views grow very large? I'll be happy to time and refine the logic - but can only do so when the exact issue is clear to me. >3) We really did not have problem w/ the speed of the commits. The only >problem we are having is commit uses to much of c4_String which slows it >down. The c4_String class, and that includes build variants which derive from MFC's "CString" and STL'd "string", has been a huge bottleneck in the past, which is why the current MK no longer has them in the public API, and internally greatly reduces their use. Most internal data management now relies on c4_Bytes. Again, show me a case, preferably in code, and I can dive in. >4) We have some memory leaks for the memory allocated in c4_String for >nullVector. Everywhere do we use locks around the metakit (1 per open >database and we have a lot of databases open at any given time). This works >great except the reference counting for the c4_String objects which uses >internal nullVector. Since that is shared by all the databases and not >protected across threads, we do see reference counting problems and it >leaks. >I was looking removing all that stuff and using a special value for refcount >(like (char)-1) to indicate we do not need to reference count, but it is not >much a problem right now. Perhaps simply removing the special case for empty strings would do it? If the optimization to share empty strings is undone, then data sharing would go away. You can't use "(char) -1", it's already used IIRC. What you could do, is alter the decref code to never delete the special empty string, and then simply fully ignore inc/decrefs on that single object. Performance-wise, that will probably not have much impact at all. I'll be happy to look into it - feel free to follow up by email, to give more details (and to refresh my memory on why you haven't yet switched to 2.4.1 :) -jcw From jcw@equi4.com Sat Nov 24 08:56:10 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA30544 for ; Sat, 24 Nov 2001 08:56:09 -0600 Received: from takkie.equi4.local ([195.121.235.232]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNB7S501.O58 for ; Sat, 24 Nov 2001 15:48:05 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAOEks006019 for ; Sat, 24 Nov 2001 15:46:54 +0100 From: Jean-Claude Wippler Cc: Metakit Date: Sat, 24 Nov 2001 15:49:37 +0100 Message-Id: <20011124144937.22689@mail.planet.nl> In-Reply-To: <93286188377.20011124101817@getsoft.com> References: <93286188377.20011124101817@getsoft.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Conclusions about speed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: (My apologies for boring everyone with what seems to have become a mud- slinging contest - but when false accusations start flying around, I have to respond) [...] >Yes I agree the documentation is awful. In my experience this is a >common trait of "Free Open Source Software". I have largely lost all >interest in Open Source because of issues like this. It's called a self-fulfilling prophecy. Being someone who sells software (as opposed to services), it would seem to me that you have an interest in seeing free software flounder. It's quite intriguing that you choose to remain on the MK mailing list, even though you do not seem to have an interest in it. From another post: >I think this is one of the big problems. ie. You need to find some way >to get MK to work for you, and this seems to different for each >application. If after much effort you still can't find a efficient >path what do you do? I had a situation where a sort on 100,000 rows >in MK would take 5-10 minutes, which made it impossible for me to use >in this specific application. Yeah, right. The code below sorts 280,000 strings of average length 10 in 28 seconds. Oh, well ... let's just say that Tcl is 20x faster than C++ :) -jcw ===================================== ========================================== % ntclkit sort.tcl Sat Nov 24 15:04:13 CET 2001 ...... 280000 rows, 3090569 b 0: text realest0 1: text Fujitsu0 2: text denotational0 Sat Nov 24 15:06:07 CET 2001 stats(commit) = 359.32 stats(count) = 40000 80000 120000 160000 200000 240000 280000 stats(fill) = 1456.11 884.91 885.72 893.49 884.72 894.40 898.60 stats(sort) = 3229.93 6943.24 10 903.93 14960.48 19230.05 23377.88 27659.37 Sat Nov 24 15:06:07 CET 2001 % ===================================== ========================================== # Timing of the view sort operation package require Mk4tcl proc timedRun {tag count args} { set usec [lindex [time $args $count] 0] append ::stats($tag) [format {%9.2f} [expr {$count*$usec/1000.0}]] } proc fill {seq} { global warray foreach {k v} [array get warray] { mk::row append db.words text $k$seq } } set step 40000 set mult 7 set fd [open /usr/share/dict/words] for {set i 0} {$i < $step && [gets $fd line] >= 0} {incr i} { set warray($line) $i } close $fd puts [clock format [clock seconds]] file delete _large.mk mk::file open db _large.mk -nocommit mk::view layout db.words text for {set i 0} {$i < $mult} {incr i} { append stats(count) [format {%9d} [expr {($i+1)*$step}]] timedRun fill 1 fill $i timedRun sort 1 mk::select db.words -sort text puts -nonewline stderr . } timedRun commit 1 mk::file commit db puts stderr " [mk::view size db.words] rows, [file size _large.mk] b" for {set i 0} {$i < 3} {incr i} { puts " $i: [mk::get db.words!$i]" } mk::file close db puts [clock format [clock seconds]] puts "" parray stats puts "" puts [clock format [clock seconds]] ===================================== ========================================== From jyl@best.com Sat Nov 24 09:29:58 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA31220; Sat, 24 Nov 2001 09:29:57 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id HAA26270; Sat, 24 Nov 2001 07:21:40 -0800 (PST) Message-ID: <3BFFC957.2020706@best.com> Date: Sat, 24 Nov 2001 08:22:47 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Metakit CC: Jean-Claude Wippler Subject: Re: [Metakit] Performance can't be that bad References: <21307737703.20011124161726@getsoft.com> <20011124111308.18336@mail.planet.nl> Content-Type: multipart/alternative; boundary="------------030709000209010804020201" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------030709000209010804020201 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Just want to confirm one thing that JCW said, that I tried to use derived views pre-2.0 and they were not fast enough for me. I am now using the latest MK with much better performance even for my simple use, and I haven't gone back to try the derived views again. I definitely will, to provide more advanced functionality like searching over an entire DB. All in all I'm very happy with what MK provides. Since its free I feel like the value per feature is infinite :) By the way, I had serious problems with deleting values from my DB, using a schema I developed to represent directed graphs inside MK. The end of it was that it was my fault, and JCW helped me patiently throughout the resolution. Thanks JCW :) I'm not yet pushing performance to its limits, but I too would like Commit() to be faster. If I recall, this definitely is something JCW has been working on continuously, and he's promised improvements. So I'm happy -- the issue is out in the open, and the author acknowledges it and promised to do something about it. That's good enough for me. --JYL Jean-Claude Wippler wrote: >Neville Franks wrote: > >>The DB is something I've developed purely for use within ED. >> > >Well, it's kind of tricky to compare two things, when one is unavailable. > >>JL> * I've used MK extensively and quickly learned to use it efficiently. My >>JL> use is the simplest possible. I dont use Find(), hashed views, and so >>JL> forth. I found all derived views to be too slow to use for my purposes, >>JL> but learned to do without. >> > >That was pre 2.0 or even earlier. Hashes and blocked views did not exist >at the time. So again, it's a bit hard to comment about such a design today. > >>I think this is one of the big problems. ie. You need to find some way >>to get MK to work for you, and this seems to different for each >>application. If after much effort you still can't find a efficient >>path what do you do? I had a situation where a sort on 100,000 rows >>in MK would take 5-10 minutes, which made it impossible for me to use >>in this specific application. If MK had had indexed access then it >>would have been a very different picture. >> > >Gee, 5..10 minutes to sort 100,000 rows, how do you do that? > >Indexed access is easy. Create a custom sort, have it store the order in >a second view, and use RemapWith to make that order persist. This is not >2.4 specific, it could have been done the same way years ago. > >Neville, could you do me a favor? Can you at least level the playing >field by giving me a chance to reproduce and explain this? Perhaps by >sending code or a description of code which explains the structure and >dataset you used? > >As for how hard it is to make MK fly in everyone's particular case: if >you learn to ride a bike, and then expect to be able to use that >experience to drive a car, then boy you're in for a rough ride. Is it a >drawback of MK that it is not a bike (i.e. not row-wise, but column-wise) >in its design? > >My advice to everyone is: if you don't like MK, or don't have the >ability, time, interest, or whatever to think through the concepts and >take advantage of MK, then please... go ride a nice bike, and let the >rest of us on this mailing list enjoy and enhance MK. > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > --------------030709000209010804020201 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Just want to confirm one thing that JCW said, that I tried to use derived views pre-2.0 and they were not fast  enough for me. I  am now using the latest MK with much better performance even for my simple use, and I haven't gone back to try the derived views again. I definitely will, to provide more advanced functionality like searching over an entire DB.

All in all I'm very happy with what MK provides. Since its free  I feel like the value per feature is infinite :)

By the way, I had serious problems with deleting values from my DB, using a schema I developed to represent directed graphs inside MK. The end of it was that it was my fault, and JCW helped me patiently throughout the resolution. Thanks JCW :)

I'm not yet pushing performance to its limits, but I too would like Commit() to be faster. If I recall, this definitely is something JCW has been working on continuously, and he's promised improvements. So I'm happy -- the issue is out in the open, and the author acknowledges it and promised to do something about it. That's good enough for me.

--JYL

Jean-Claude Wippler wrote:
Neville Franks <readonly@getsoft.com> wrote:

The DB is something I've developed purely for use within ED.

Well, it's kind of tricky to compare two things, when one is unavailable.

JL> * I've used MK extensively and quickly learned to use it efficiently. My 
JL> use is the simplest possible. I dont use Find(), hashed views, and so 
JL> forth. I found all derived views to be too slow to use for my purposes, 
JL> but learned to do without.

That was pre 2.0 or even earlier.  Hashes and blocked views did not exist
at the time.  So again, it's a bit hard to comment about such a design today.

I think this is one of the big problems. ie. You need to find some way
to get MK to work for you, and this seems to different for each
application. If after much effort you still can't find a efficient
path what do you do?  I had a situation where a sort on 100,000 rows
in MK would take 5-10 minutes, which made it impossible for me to use
in this specific application. If MK had had indexed access then it
would have been a very different picture.

Gee, 5..10 minutes to sort 100,000 rows, how do you do that?

Indexed access is easy.  Create a custom sort, have it store the order in
a second view, and use RemapWith to make that order persist.  This is not
2.4 specific, it could have been done the same way years ago.

Neville, could you do me a favor?  Can you at least level the playing
field by giving me a chance to reproduce and explain this?  Perhaps by
sending code or a description of code which explains the structure and
dataset you used?

As for how hard it is to make MK fly in everyone's particular case: if
you learn to ride a bike, and then expect to be able to use that
experience to drive a car, then boy you're in for a rough ride.  Is it a
drawback of MK that it is not a bike (i.e. not row-wise, but column-wise)
in its design?

My advice to everyone is: if you don't like MK, or don't have the
ability, time, interest, or whatev
er to think through the concepts and
take advantage of MK, then please... go ride a nice bike, and let the
rest of us on this mailing list enjoy and enhance MK.

-jcw

_______________________________________________
metakit mailing list  -  metakit@equi4.com
http://www.equi4.com/mailman/listinfo/metakit


--------------030709000209010804020201-- From jyl@best.com Sat Nov 24 09:33:56 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA31350; Sat, 24 Nov 2001 09:33:55 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id HAA26633; Sat, 24 Nov 2001 07:25:11 -0800 (PST) Message-ID: <3BFFCA2B.4060905@best.com> Date: Sat, 24 Nov 2001 08:26:19 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Subject: Re: [Metakit] Is JCW rich ? References: <000701c1749e$491a4e00$8826933e@desktop> <20011124115529.12245@mail.planet.nl> Content-Type: multipart/alternative; boundary="------------070504090403020806050906" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------070504090403020806050906 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit JC One *important* point that was raised here that shouldn't get lost: the documentation provided for MK could use some enhancing. I agree that the line of discussion about whether you're rich and so forth is obscene and out of place. :( --JYL Jean-Claude Wippler wrote: >Dear Pascal / popeye, > >>Jean-Claude Wippler wrote: >> >>>My software is free, and I'm doing just fine commercially thank you. >>> >>Does this mean you get some income from Metakit. >>That would be good news, but I can't figure out how. >>And I'm curious... >> > >Sure, but does that mean you have to repeatedly waste the time of >hundreds of people subscribed to this mailing list with this sort of >chit-chat? > >(FYI, yes I feel rich, in the sense that I make more every year from >consulting than I spend. Thanks to MK, which is going a lot more places >these days.) > >Please allow me to adjust the advice I gave to you before. > >First, I apologize for the time you had to waste on MetaKit. It is a >silly design and will most likely never meet your requirements. You will >be far better of by using Berkeley DB, MySQL, or Postgres. They are all >free, extremely well documented, easy to learn and use, and with lots and >lots of people to help. > >Second, please go play outside and socialize with real people a bit more >often. It helps. > >Now, for the last time... can we get on with it, please? > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > --------------070504090403020806050906 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit JC

One *important* point that was raised here that shouldn't get lost: the documentation provided for MK could use some enhancing.

I agree that the line of discussion about whether you're rich and so forth is obscene and out of place. :(

--JYL

Jean-Claude Wippler wrote:
Dear Pascal / popeye,

Jean-Claude Wippler wrote:
My software is free, and I'm doing just fine commercially thank you.
Does this mean you get some income from Metakit.
That would be good news, but I can't figure out how.
And I'm curious...

Sure, but does that mean you have to repeatedly waste the time of
hundreds of people subscribed to this mailing list with this sort of
chit-chat?

(FYI, yes I feel rich, in the sense that I make more every year from
consulting than I spend.  Thanks to MK, which is going a lot more places
these days.)

Please allow me to adjust the advice I gave to you before.  

First, I apologize for the time you had to waste on MetaKit.  It is a
silly design and will most likely never meet your requirements.  You will
be far better of by using Berkeley DB, MySQL, or Postgres.  They are all
free, extremely well documented, easy to learn and use, and with lots and
lots of people to help.

Second, please go play outside and socialize with real people a bit more
often.  It helps.

Now, for the last time... can we get on with it, please?

-jcw

_______________________________________________
me
takit mailing list  -  metakit@equi4.com
http://www.equi4.com/mailman/listinfo/metakit


--------------070504090403020806050906-- From baspey@yahoo.com Sat Nov 24 10:29:15 2001 Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA32399 for ; Sat, 24 Nov 2001 10:29:15 -0600 Received: from toulouse-2-a7-39-28.dial.proxad.net (HELO desktop) (62.147.39.28) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 16:21:46 -0000 Message-ID: <001501c17503$e891b760$1c27933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 17:19:57 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >please go play outside and socialize with real people a bit more >often. It helps. You're a mean person Jean-Claude, you know that ! If you read again all my postings (sorry if there's a lot), you will realize that I'm not a bad person myself. I was simply in the position of someone who thought he had made a blunt mistake but couldn't find which one (the title I chose was "performance *can't be* that bad"). And the only answers I got were that if you don't use clever tricks, performance is indeed that bad. I little went outside because I had launched the subject and thought I had to answer other person's mails, out of politeness and respect. >First, I apologize for the time you had to waste on MetaKit. It is a >silly design and will most likely never meet your requirements. You will >be far better of by using Berkeley DB, MySQL, or Postgres. They are all >free, extremely well documented, easy to learn and use, and with lots and >lots of people to help. I have indeed spent a lot of time on Metakit, and a better documentation would have helped, that's no doubt. Now that I am convinced it is worth it, I will spend the extra time necessary to reach my goal. Sorry, I don't give up Metakit. The only thing I'll give up is this mailing list. I will still listen to your clever conversations but myself, poor peasant, will never post again, by fear that other subscribers may unsubscribe... Jacob Levy wrote (about my subject "is JCW rich ?"): >I agree that the line of discussion about whether you're rich and so >forth is obscene and out of place. :( I suffer, really ! My title was of course only a joke ! But I was really interested in knowing if Jean-Claude got income by providing Metakit's services. Because I never paid a dollar for Metakit and Jean-Claude spends a lot of time on it, I would have been happy to know that Metakit can be a source of income for him. So long. You'll never know if I'd survived all this humiliation or put a bullet beetween my ears. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From konrad@minus1.de Sat Nov 24 13:12:38 2001 Received: from moutvdom01.kundenserver.de (moutvdom01.kundenserver.de [195.20.224.200]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA03190 for ; Sat, 24 Nov 2001 13:12:37 -0600 Received: from [195.20.224.219] (helo=mrvdom03.schlund.de) by moutvdom01.kundenserver.de with esmtp (Exim 2.12 #2) id 167i6v-0003ac-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:05:09 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom03.schlund.de with esmtp (Exim 2.12 #2) id 167i6v-0006KB-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:05:09 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 167i9Q-00019t-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:07:44 +0100 Date: Sat, 24 Nov 2001 20:07:43 +0100 From: Konrad Anton To: metakit@equi4.com Message-ID: <20011124200743.A4397@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.2.5i Subject: [Metakit] accessing row attributes in Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello everybody. I'm using Mk4py 2.4.1 on Python 2.1.0 on Linux. Suppose I've got a metakit view >>> s = metakit.storage("TMPwuff.mk",1) >>> v = s.getas("wuff[bla:S,blub:I]") is there any way to generically get the values of all columns in a row in v, like in: aRow=v[44] for colname in ["bla", "blub"]: print aRow[colname], I can work around it by using print eval("aRow."+colname) instead, but this looks like a dirty hack to me. I've also tried aRow.get(key) and aRow.__getattr__(key), but no success. Ideas, anyone? TIA Konrad PS. shoulder-pat shoulder-pat to JCW for Metakit. It works for me. Some time in the future, I'll announce a little transaction processing system for it, but it is not quite ready for public dégustation. -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: They couldn't have somebody to day. From konrad@minus1.de Sat Nov 24 13:42:33 2001 Received: from moutvdom01.kundenserver.de (moutvdom01.kundenserver.de [195.20.224.200]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA03813 for ; Sat, 24 Nov 2001 13:42:33 -0600 Received: from [195.20.224.208] (helo=mrvdom01.schlund.de) by moutvdom01.kundenserver.de with esmtp (Exim 2.12 #2) id 167iZs-0006cO-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:35:04 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom01.schlund.de with esmtp (Exim 2.12 #2) id 167iZs-0000s1-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:35:04 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 167iGh-0001Ab-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:15:15 +0100 Date: Sat, 24 Nov 2001 20:15:15 +0100 From: Konrad Anton To: metakit@equi4.com Message-ID: <20011124201515.A4488@minus1.de> References: <20011124200743.A4397@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20011124200743.A4397@minus1.de>; from konrad on Sat, Nov 24, 2001 at 08:07:43PM +0100 Subject: [Metakit] Re: accessing row attributes in Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Konrad. Long time no see. > instead, but this looks like a dirty hack to me. I've also tried > aRow.get(key) and aRow.__getattr__(key), but no success. use getattr(aRow, key). > TIA > Konrad You're welcome. Konrad. -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: They couldn't have somebody to day. From gmcm@hypernet.com Sat Nov 24 13:42:35 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA03817 for ; Sat, 24 Nov 2001 13:42:35 -0600 From: gmcm@hypernet.com Received: from PARANOIA (204.176.40.83) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Sat, 24 Nov 2001 14:28:58 -0500 To: metakit@equi4.com Date: Sat, 24 Nov 2001 14:35:44 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] accessing row attributes in Python Message-ID: <3BFFB040.22925.BFE04001@localhost> Priority: normal In-reply-to: <20011124200743.A4397@minus1.de> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 24 Nov 2001 at 20:07, Konrad Anton wrote: [snip] > is there any way to generically get the values of all columns in a row > in v, like in: > > aRow=v[44] > for colname in ["bla", "blub"]: > print aRow[colname], > > I can work around it by using > > print eval("aRow."+colname) > > instead, but this looks like a dirty hack to me. I've also tried > aRow.get(key) and aRow.__getattr__(key), but no success. Almost. getattr(row, "colname") (You're dealing with an old-fashioned type object which doesn't have a method "__getattr__", but it does have a getattr slot.) From tciuro@mac.com Sat Nov 24 14:24:43 2001 Received: from smtpout.mac.com (smtpout.mac.com [204.179.120.89]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA04723; Sat, 24 Nov 2001 14:24:43 -0600 Received: from smtp-relay02.mac.com (server-source-si02 [10.13.10.6]) by smtpout.mac.com (8.12.1/8.10.2/1.0) with ESMTP id fAOJTRsD017032; Sat, 24 Nov 2001 11:29:27 -0800 (PST) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GNBN0Q00.E11; Sat, 24 Nov 2001 12:17:14 -0800 Received: from localhost ([63.202.173.4]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GNBN0P00.CIS; Sat, 24 Nov 2001 12:17:13 -0800 Date: Sat, 24 Nov 2001 12:17:13 -0800 Subject: Re: [Metakit] Is JCW rich ? Content-Type: multipart/alternative; boundary=Apple-Mail-1-408545894 Mime-Version: 1.0 (Apple Message framework v501) Cc: Jean-Claude Wippler , metakit To: Jacob Levy From: Tito Ciuro In-Reply-To: <3BFFCA2B.4060905@best.com> Message-Id: <3F32B6E0-E118-11D5-BB10-0003934510E2@mac.com> X-Mailer: Apple Mail (2.501) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-408545894 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hey, hey, hey! Hold your horses! Let's give Jean-Claude a break, or better yet, two. Without his dedication and hard work, I, for one, wouldn't have had the slightest chance to finish my project. Not only has MK proven to be a workhorse, but The Man himself has gone great lengths making sure that "we get it right." In one of my first posts, sometime in March, I wrote an email complaining the state of the somewhat aged documentation. Yes, it's true. Yes, it's too bad. But guess what? You can read the list's comments. You can read the FAQ. You can read the headers. If in the end you're still bothered by it, why don't you contribute? How come I hear complaints without giving anything back? What the heck is this? C'mon! Let's show some manners and initiative, shall we? Thanks a million Jean-Claude for your immense help and dedication. I owe you a big one; you know what I'm talking about... -- Tito On Saturday, November 24, 2001, at 08:26 AM, Jacob Levy wrote: > JC > > One *important* point that was raised here that shouldn't get lost: the > documentation provided for MK could use some enhancing. > > I agree that the line of discussion about whether you're rich and so > forth is obscene and out of place. :( > > --JYL > ____________________________________________ "I told my psychiatrist that everyone hates me. He said I was being ridiculous - everyone hasn't met me yet." Rodney Dangerfield --Apple-Mail-1-408545894 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hey, hey, hey! Hold your horses! Let's give Jean-Claude a break, or better yet, two. Without his dedication and hard work, I, for one, wouldn't have had the slightest chance to finish my project. Not only has MK proven to be a workhorse, but The Man himself has gone great lengths making sure that "we get it right." In one of my first posts, sometime in March, I wrote an email complaining the state of the somewhat aged documentation. Yes, it's true. Yes, it's too bad. But guess what? You can read the list's comments. You can read the FAQ. You can read the headers. If in the end you're still bothered by it, why don't you contribute? How come I hear complaints without giving anything back? What the heck is this? C'mon! Let's show some manners and initiative, shall we? Thanks a million Jean-Claude for your immense help and dedication. I owe you a big one; you know what I'm talking about... -- Tito On Saturday, November 24, 2001, at 08:26 AM, Jacob Levy wrote: JC One *important* point that was raised here that shouldn't get lost: the documentation provided for MK could use some enhancing. I agree that the line of discussion about whether you're rich and so forth is obscene and out of place. :( --JYL Lucida Grande ____________________________________________Lucida Grande 0000,0000,FFFF"I told my psychiatrist that everyone hates me. He said I was being ridiculous - everyone hasn't met me yet." 0000,6666,3333 Rodney Dangerfield --Apple-Mail-1-408545894-- From readonly@getsoft.com Sat Nov 24 17:34:56 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA09407; Sat, 24 Nov 2001 17:34:55 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 167mCi-0002CZ-00; Sat, 24 Nov 2001 16:27:25 -0700 Date: Sun, 25 Nov 2001 10:29:42 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <167373274049.20011125102942@getsoft.com> To: Jean-Claude Wippler CC: Metakit Subject: Re: [Metakit] Performance can't be that bad In-Reply-To: <20011124111308.18336@mail.planet.nl> References: <21307737703.20011124161726@getsoft.com> <20011124111308.18336@mail.planet.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Saturday, November 24, 2001, 10:13:08 PM, you wrote: JCW> Neville Franks wrote: >>The DB is something I've developed purely for use within ED. JCW> Well, it's kind of tricky to compare two things, when one is unavailable. As I stated earlier I was never trying to compare what I've done with MK. I simply wanted to state my experience with MK. Furthermore from the outset I mentioned that I applauded the work you have done with MK. It was unfortunate for me that I wasn't able to get it to meet my needs. >>JL> * I've used MK extensively and quickly learned to use it efficiently. My >>JL> use is the simplest possible. I dont use Find(), hashed views, and so >>JL> forth. I found all derived views to be too slow to use for my purposes, >>JL> but learned to do without. JCW> That was pre 2.0 or even earlier. Hashes and blocked views did not exist JCW> at the time. So again, it's a bit hard to comment about such a design today. >>I think this is one of the big problems. ie. You need to find some way >>to get MK to work for you, and this seems to different for each >>application. If after much effort you still can't find a efficient >>path what do you do? I had a situation where a sort on 100,000 rows >>in MK would take 5-10 minutes, which made it impossible for me to use >>in this specific application. If MK had had indexed access then it >>would have been a very different picture. JCW> Gee, 5..10 minutes to sort 100,000 rows, how do you do that? Using c4_View.Sort() There was approx. 7 columns in the view and the sort was on a string. This was with several hundred thousand rows on a P3 550Mhz PC. JCW> Indexed access is easy. Create a custom sort, have it store the order in JCW> a second view, and use RemapWith to make that order persist. This is not JCW> 2.4 specific, it could have been done the same way years ago. I recall discussing my need to persist the Sort on this list, but can't recall any solution being suggested. Having to wait several minutes or even 30 seconds each time the app starts while the Sort is performed is I'm afraid unacceptable to my users. JCW> Neville, could you do me a favor? Can you at least level the playing JCW> field by giving me a chance to reproduce and explain this? Perhaps by JCW> sending code or a description of code which explains the structure and JCW> dataset you used? Jean-Claude, I think this would be a waste of your valuable time and mine. I was simply using c4_View.Sort()as mentioned above. Sorting was one issue, real time updates with many adds and deletes was another performance problem for me. When you are writing code in an editor any delays to your typing would make the editor unusable. With ED each and every edit will trigger database lookups and potential updates. This needs to have minimal perceived impact to the user. In other words no delays are acceptable. JCW> As for how hard it is to make MK fly in everyone's particular case: if JCW> you learn to ride a bike, and then expect to be able to use that JCW> experience to drive a car, then boy you're in for a rough ride. Is it a JCW> drawback of MK that it is not a bike (i.e. not row-wise, but column-wise) JCW> in its design? JCW> My advice to everyone is: if you don't like MK, or don't have the JCW> ability, time, interest, or whatever to think through the concepts and JCW> take advantage of MK, then please... go ride a nice bike, and let the JCW> rest of us on this mailing list enjoy and enhance MK. I invested considerable time and effort in MK and in the end found that I could not get it to meet my specific needs. This came at a high cost to me, but I do not regret it. Furthermore I'm interested in seeing how MK evolves, which is one of the reasons I've stayed on this list. I have also assisted some MK users in the past, and provided some code to enable custom sort sequences etc. If you would prefer not to have me on this list then please say so and I'll remove myself. I have no interest in seeing free software flounder, as you stated. The lesson I've learnt over the past year is that "free software" has cost me dearly. I estimate I spent approx. $16K of my time on MK. Obviously my expectations were wrong and I know that now. You can't possibly expect to get the level of documentation, support etc.etc. from free software that you can from commercial software. Even many commercial packages fall short, but at least you can rightfully demand a certain level of assistance. Over time commercial products that don't meet their customers needs will fail and disappear from the marketplace. Unfortunately this isn't the case with "free software" which seems to live on forever, potentially costing people dearly. I see this along with a lack of adequate peer review a fundamental problem with "free software". Please note that I am not referring to MK here. It is very unfortunate that this thread has degenerated into a mud slinging match. This was certainly never my intention. I simply wanted to state my observations and personal experience with MK, thinking that they may be of some interest to others. -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From jyl@best.com Sun Nov 25 12:03:34 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA32692; Sun, 25 Nov 2001 12:03:31 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id JAA09438; Sun, 25 Nov 2001 09:55:54 -0800 (PST) Message-ID: <3C013F0B.1020209@best.com> Date: Sun, 25 Nov 2001 10:57:15 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Efficiency question Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude Is there a big difference in efficiency if I create a temporary row object and do my manipulations on that, and afterwards write the row to the view, or if I manipulate individual properties one at a time? For example, the MK driver in e4Graph is full of idioms like this: /* * Initialize the marker. */ pFlags(markers[idx]) = E4_INUSE | E4_DETACHED; pUserData(markers[idx]) = 0; pNameID(markers[idx]) = nameID; Is each expression of the form markers[idx] creating a temporary row object that is immediately discarded, or is it a bit more efficient than that to do the above? Is the above significanly less efficient than: c4_Row row = markers[idx]; pFlags(row) = E4_INUSE | E4_DETACHED; pUserData(row) = 0; pNameID(row) = nameID; markers[idx] = row; Thanks, --JYL From jcw@equi4.com Sun Nov 25 13:19:23 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA01588 for ; Sun, 25 Nov 2001 13:19:22 -0600 Received: from takkie.equi4.local ([195.121.233.70]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNDEMU01.02H for ; Sun, 25 Nov 2001 20:11:18 +0100 Received: from [10.0.1.35] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAPJAH015265 for ; Sun, 25 Nov 2001 20:10:17 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Efficiency question Date: Sun, 25 Nov 2001 20:12:31 +0100 Message-Id: <20011125191231.21914@mail.planet.nl> In-Reply-To: <3C013F0B.1020209@best.com> References: <3C013F0B.1020209@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >Is there a big difference in efficiency if I create a temporary row >object and do my manipulations on that, and afterwards write the row to >the view, or if I manipulate individual properties one at a time? MK stores data column-wise. If you have 5 columns, then fetching them all and copying it into a temp row, then storing it back is 10 data movements. If you need to alter one property, for example, then temp rows would be the worst thing you can do. > pFlags(markers[idx]) = E4_INUSE | E4_DETACHED; > pUserData(markers[idx]) = 0; > pNameID(markers[idx]) = nameID; > >Is each expression of the form markers[idx] creating a temporary row >object that is immediately discarded, or is it a bit more efficient than >that to do the above? > >Is the above significanly less efficient than: > > c4_Row row = markers[idx]; > pFlags(row) = E4_INUSE | E4_DETACHED; > pUserData(row) = 0; > pNameID(row) = nameID; > markers[idx] = row; On the contrary, your first example is *more* efficient. Did you time it? Follow the "Tips and Tricks" link on the API page: http://www.equi4.com/metakit/api/ That leads to: http://www.equi4.com/metakit/api/tips.html#faster The point of column-wise data, is that many loops tend to not deal with all properties. With columns, such unused data never gets loaded. That is why, for example, sorting speed is not related to the number of properties in a view. For a different reason (CPU caches, both instruction- and data), but again due to the columnar structure underneath MK, when you loop over large amounts of data, don't do: for each row: for each column: blah But do, when possible: for each column: for each row: blah I cannot repeat it enough, it seems: the concept of storing data in columns yet presenting a traditional row-wise API, as MK does, is one which has profound implications. The difference between getting it right and getting it wrong, is often several orders of magnitude, especially when aiming for high-end performance. I wish I knew how to make the concept clearer, I really do. -jcw From jcw@equi4.com Sun Nov 25 13:30:13 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA01812; Sun, 25 Nov 2001 13:30:13 -0600 Received: from takkie.equi4.local ([195.121.237.165]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNDF4W02.R33; Sun, 25 Nov 2001 20:22:08 +0100 Received: from [10.0.1.35] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAPJKu015355; Sun, 25 Nov 2001 20:21:06 +0100 From: Jean-Claude Wippler To: Cc: metakit Subject: Re: [Metakit] Efficiency question #2 Date: Sun, 25 Nov 2001 20:23:31 +0100 Message-Id: <20011125192331.26641@mail.planet.nl> In-Reply-To: <3C013F0B.1020209@best.com> References: <3C013F0B.1020209@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > pNameID(markers[idx]) = nameID; > >Is each expression of the form markers[idx] creating a temporary row >object that is immediately discarded, [...] Aha, sorry for missing this on first reading: "markers[idx]" is *not* a temp row. It is totally unrelated to writing things like "pNameID [123]", which indeed is a temp row that gets to hold one property value, briefly. The confusion is most unfortunate, and cause by overloading the same [] operator, simply because C++ offers few alternatives. No, "views[index]" is *the* most efficient way to refer to any (conceptual) row in a view. It is, internall, a c4_RowRef object - which in turn is a very very lightweight object (a "flyweight", in gang-of-four parlance). On the other hand, "property[value]" creates a view, sizes it to hold one row, and allocates one column to hold the value. It's extremely convenient, but needs to be avoided inside tight loops. See the "tips & tricks" page. Thanks for helping clarify this key issue. -jcw From jcw@equi4.com Mon Nov 26 06:10:02 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA09618 for ; Mon, 26 Nov 2001 06:10:01 -0600 Received: from takkie.equi4.local ([195.121.232.195]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNEPF600.T3L for ; Mon, 26 Nov 2001 13:01:54 +0100 Received: from [10.0.1.35] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAQC0w019861 for ; Mon, 26 Nov 2001 13:00:59 +0100 From: Jean-Claude Wippler To: Metakit Mailing List Subject: Re: [Metakit] Python, Metakit, Linux, SegFaults -- more info Date: Mon, 26 Nov 2001 13:02:58 +0100 Message-Id: <20011126120258.9976@mail.planet.nl> In-Reply-To: <3BF98487.5C434126@bioreason.com> References: <3BF98487.5C434126@bioreason.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mitch Chapman wrote: >Below are three Python scripts, with output, which trigger segfaults Ok, I've commited some changes to CVS and updated the snapshot at: http://www.equi4.com/previews/metakit.tar.gz (For this time, this snapshot won't be updated all the time!) >schema[:] = [] ># This will crash because it's indexing beyond the end of schema. ># Should it produce an IndexError instead? >schema[0] = {'ident': 'Kilroy'} Fixed, a check was moved to catch this. It made another problem with exception handling in Mk4py show up, which has been worked around for the time being. >db = metakit.storage() [...] ># This will crash because storage has no associated file? >db.commit() The same exception problem. Commit to such a temp storage generated the proper Python error, but its handling went south. >contents = range(10) >for i in range(numRecords): > records[i] = {'ident': i, 'contents': contents} Fixed. Added code to reject a sequence of scalars in this context. -jcw From nicolasb@maich.gr Mon Nov 26 06:44:36 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA11444 for ; Mon, 26 Nov 2001 06:44:31 -0600 Received: FROM maich.gr BY mail-server ; Mon Nov 26 14:52:14 2001 +0200 Message-ID: <3C023B96.CFF4FEA9@maich.gr> Date: Mon, 26 Nov 2001 14:54:46 +0200 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, Too much un-related noise on this list... FWIW, I feel a need to chime in here, and hopefully put some things in perspective... IMO, Metakit ia a developers tool, with a rich api and the ability to "try it this way if it didn't work as expected the first way..." Developers in general develop. If not let's all stick to .mdb files and a VB front end and resultant 30 MB application files, and install nightmares (maybe that;s another story)... Metakit is free and Open Source...(Used to be 90USD for a full developers version, libraries and source, correct me if I am wrong JC..) Metakit is available on numerous platforms Metakit's file are transportable accross paltforms Metakit has AFAIK, 3 interfaces... Metakit has real support. My problems/bugs are addressed/fixed in a day or less. Usually less than an hour.. Metakit is constantly being improved, and functionality added...(v1.8---v2.4) Core documentation is there. I heve learned to read the sources/lists/faqs for the rest. Yes a bound book would be nice Yes, other options exist, some open, some not, some free some not (Is Faircom still charging ~16K for their cross platform C libs?) Taken as whole, this is a tall order... So where do we stand? Some recommendations? JC, maybe you should re-instate you old Mk pricing along with your open source approach. Then you probably wont get questions on the state of your financial affairs... It would also make it easier for people to show their appreciation for your work and support and such.... Documentation; Someone that must have it; write a book and I'll print it and bind it.... Maybe clearly/simply summarize Mk major developments and current directions and potential limitations (read TeraBytes) on a single html page with big fonts so people can see it if they dont feel like reading changes.txt or the Metakit intros...... Finally a note to Mr. Pascal. Please take a breath, re-look at your project and see if Metakit is appropriate, and let us know. It's how I learn at least... And for JC, as in Tito's last post, thanx a bunch for all you have done for me...It's been too long since we met, I must dream up another project, Is July 2002 open:-? Sincerely Nicolas Boretos Metakit user since V1.8/Mk4tcl 0.5 From baspey@yahoo.com Mon Nov 26 09:44:40 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA22265 for ; Mon, 26 Nov 2001 09:44:39 -0600 Received: from toulouse-2-a7-38-223.dial.proxad.net (HELO desktop) (62.147.38.223) by smtp.mail.vip.sc5.yahoo.com with SMTP; 26 Nov 2001 15:37:03 -0000 Message-ID: <000901c1768f$fb069cc0$df26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Mon, 26 Nov 2001 16:34:32 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, here are my public apologizes to everyone, and most especially to JCW. You may accept them or not... I feel like shit, but claim not having had bad intentions at any time. I have been informed that my style deserved me, and believe it or not, I did not realize at that time that I sounded aggressive. Metakit fits my expectations in all points: embeddable, small footprint, cross-platform, open source, free. After having impregnated myself with Metakit concepts, I have worked out a design for my database according to Metakit's particular architecture (especially subviews). Then came the moment I tried the result and ooops, that's really not good... I did not assault this forum but instead made experiences for 2 days. Convinced I was making some silly mistake but unable to find which one, I posted a mail on this list. And then bad luck fell down on me since I received answers mostly from people telling me that Metakit is pourrave, and on another side, could not find the source of my problem even with the help of the dedicated Jean-Claude. You are people who know that Metakit is great, but not much of you intervened to testify... But those are public apologizes so I'll end up with these words: Thanks a lot Jean-Claude. I apologize to everyone. P.S: please don't call me mister anymore. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From mitch.chapman@bioreason.com Mon Nov 26 10:11:56 2001 Received: from gateway.bioreason.com (www.bioreason.com [207.108.245.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA24276; Mon, 26 Nov 2001 10:11:55 -0600 Received: by gateway.bioreason.com (Postfix, from userid 200) id A6B272201; Mon, 26 Nov 2001 09:04:04 +0000 (/etc/localtime) Received: by bioreason.com via smwrap Version 1.0 id smwrappwbGca; Mon Nov 26 09:03:05 2001 by mailhost.bioreason.com (Postfix) with ESMTP id F265630DC; Mon, 26 Nov 2001 09:02:43 +0000 (/etc/localtime) Message-ID: <3C02692F.979A0BDF@bioreason.com> Date: Mon, 26 Nov 2001 09:09:19 -0700 From: Mitch Chapman Organization: Bioreason, Inc. MIME-Version: 1.0 To: Jean-Claude Wippler Cc: Metakit Mailing List Subject: Re: [Metakit] Python, Metakit, Linux, SegFaults -- more info References: <3BF98487.5C434126@bioreason.com> <20011126120258.9976@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > > On Mitch Chapman wrote: > > >Below are three Python scripts, with output, which trigger segfaults > > Ok, I've commited some changes to CVS and updated the snapshot at: > http://www.equi4.com/previews/metakit.tar.gz > (For this time, this snapshot won't be updated all the time!) Thanks! That was a quick turnaround. Thanks also for making these changes available via HTTP. I think our firewall is to blame for our inability to access remote CVS repositories from work, but in any case you've saved me alot of time (go home, do a CVS update, email the results to work) and hassle. -- Off to get the snapshot, Mitch From steve@improvision.com Mon Nov 26 10:36:02 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA25895 for ; Mon, 26 Nov 2001 10:36:01 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id QAA05222 for ; Mon, 26 Nov 2001 16:26:24 GMT Received: from [192.168.1.25] ([192.168.1.25]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971 for ; Mon, 26 Nov 2001 16:28:21 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 26 Nov 2001 16:28:25 +0000 From: Stephen Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Possible bug (and fix) in remap.cpp Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I have found what might be a small bug in remap.cpp Line 647 reads: bool c4_BlockedViewer::RemoveRows(int pos_, int count_) { d4_assert(count_ > 0); d4_assert(pos_ + count_ < GetSize()); . . . I suspect the assert should read: d4_assert(pos_ + count_ <= GetSize()); As it asserts if pos_ is the last row and count_ is 1 (i.e. delete the last row). Is this correct? Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ From mlh@idi.ntnu.no Mon Nov 26 11:09:54 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA28810 for ; Mon, 26 Nov 2001 11:09:53 -0600 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id SAA16554; Mon, 26 Nov 2001 18:02:13 +0100 (MET) Message-ID: <028901c1769c$18130540$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: "Pascal Baspeyras" Cc: References: <000901c1768f$fb069cc0$df26933e@desktop> Subject: Re: [Metakit] Retro-spection Date: Mon, 26 Nov 2001 18:02:13 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Pascal Baspeyras" > Hello, > > here are my public apologizes to everyone, and most especially > to JCW. > You may accept them or not... > > I feel like shit, but claim not having had bad intentions at > any time. Just for the record, I found the hostile reactions of this list towards you excessive. It seemed to me that intentions were wrongly read into your postings. -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.sf.net From horst@freedict.de Mon Nov 26 11:33:42 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id LAA30647 for ; Mon, 26 Nov 2001 11:33:41 -0600 From: horst@freedict.de Received: (qmail 29958 invoked from network); 26 Nov 2001 17:48:57 -0000 Received: from dialin-145-254-111-247.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.111.247) by mail.netbeat.de with SMTP; 26 Nov 2001 17:48:57 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fAQGeai04683; Mon, 26 Nov 2001 17:40:37 +0100 Message-Id: <200111261640.fAQGeai04683@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Mon, 26 Nov 2001 17:40:33 +0100 (CET) Reply-To: horst@freedict.de To: metakit@equi4.com In-Reply-To: <20011125192331.26641@mail.planet.nl> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] I guess, I did not understand metakits hashed view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I am not sure, I understood correct how the hashed view is meant to work. Attached is the a test file, which demonstrates where I do have difficulties. When I replace vwh.append with view.append, things become slow with a number of entries, but the result is as expected: with each append, the nuber of items increases. Thanks, Horst import metakit import locale import time Auth = "hey" Date = time.time() Quality = 1 Status = 0 dict = "test.mk" db = metakit.storage(dict, 1) db.autocommit() # word: The Heardowrd for the definition # definition: The definition in TEI format # auth: Who last changed the definition # date: when was it last changed # the quality of the entry # 0: no quality # 10: computer generated # 20: novice, non linguist # ... # 100: absolut perfect entry (perhaps can never be achieved) # negative numbers are used when the entry is not conformant # (to the DTD, or ...) - The value still represents who # generted it. - So -10 is invalid computer generated # status # 0: word is new # 1: word is "work in progress" # 2: word is eidted (at least once) view = db.getas("dict[dictionary:S,word:S,collate:B,definition:S,auth:S,date:D,quality:I,status:I]") map = db.getas('map[_H:I,_R:I]') vwh = view.hash(map,1 ) sorted = vwh.sort(vwh.collate) for Word in ("a", "b", "c", "d"): "Add a definition to the database" Def = Word + Word Collate = locale.strxfrm(Word) Word = Word.encode('utf-8') Def = Def.encode('utf-8') vwh.append(word=Word, collate=Collate, definition=Def, auth=Auth, date=Date, quality=Quality, status = Status); for r in vwh: print r.word for r in vwh: print r.word -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From jcw@equi4.com Mon Nov 26 13:09:41 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA06169 for ; Mon, 26 Nov 2001 13:09:40 -0600 Received: from takkie.equi4.local ([195.121.238.184]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNF8UK02.DM1 for ; Mon, 26 Nov 2001 20:01:32 +0100 Received: from [10.0.1.36] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAQJ0d023765 for ; Mon, 26 Nov 2001 20:00:39 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] I guess, I did not understand metakits hashed view Date: Mon, 26 Nov 2001 20:02:35 +0100 Message-Id: <20011126190235.2360@mail.planet.nl> In-Reply-To: <200111261640.fAQGeai04683@eaglesnest.mceggman> References: <200111261640.fAQGeai04683@eaglesnest.mceggman> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: >view = db.getas("dict[dictionary:S,wo >rd:S,collate:B,definition:S,auth:S,date:D,quality:I,status:I]") >map = db.getas('map[_H:I,_R:I]') >vwh = view.hash(map,1 ) >sorted = vwh.sort(vwh.collate) > >for Word in ("a", "b", "c", "d"): [...] > vwh.append(word=Word, collate=Collate, definition=Def, auth=Auth, >date=Date, quality=Quality, status = Status); You're defining a view with "dictionary" as first property. Then you hash, specifying that its key is the first field. Then you add items which all have the same value for "dictionary" (unset, i.e. the empty string). The result is a view which contains the last entry ("append" is confusing in the context of hashes, which work with unique keys). -jcw From baspey@yahoo.com Mon Nov 26 23:33:55 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA07000 for ; Mon, 26 Nov 2001 23:33:55 -0600 Received: from toulouse-2-a7-36-254.dial.proxad.net (HELO desktop) (62.147.36.254) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 05:26:16 -0000 Message-ID: <007201c17703$d1bb3a00$fe24933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 06:24:06 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Be warned: this is an un-aggressive mail ! Magnus Lie Hetland wrote: >Just for the record, I found the hostile reactions of this list >towards you excessive. It seemed to me that intentions were >wrongly read into your postings. I'm not sure you will like it Magnus, but I LOVE YOU. If we lived in California, I would marry you ! I totally agree with what you say, and I totally agree with what I've said. Nobody can't deny that I made my possible to defend Metakit and to say "I must be doing something wrong". I have already received (a few) supportive mails on my private E-mail address. Thanks a lot guys. And all readers are welcome to send me theirs. I will quote one of them: >Too often I have found in my life people who, because of some misunderstanding >- sometimes cultural, sometimes because of translation problems, sometimes >because of personal problems or issues - end up being 'flamed' in public >and, because of the embarassment, give up on some particular mailing >list or product. >What I have tried to do when I find _myself_ in such situations is this: >1. examine what I have said and determine whether I owe someone an >apology - either public or private >2. examine what benefits I get from the product and decide if a public >apology should be offered - whether I owe it or not. >3. most of all, determine when I am willing to give up having the last word >and just letting things go, hoping that after some decent period of time >I can expect things to 'resume normalcy'. I'll follow this wise advice. I'll follow it right now ! Before someone puts it on the table again, let's talk about my "is JCW rich ?" subject. Title was of course a prank. Richness doesn't always mean money. But on the other hand, subject was serious: I'd really like my application to be available for free. I have never heard of JCW charging for his services, so I would have liked to know what kind of partnership or sponsorship it is possible to contract. This may certainly not be the right forum to discuss such subject, but if someone could tell me about his experience or direct me toward the right forum... I'm French and in France we less easily talk about money than the Americans do. So where could the fellow who called me obcene come from ? End of the digression. I have practiced flagellation in public. But nobody seems interested in knowing what solution I have found to my problem. The answer is simple: at that time, none. You all remember that I've cause much trouble and that a lot of persons hate me and call me Mr Pascal. But what did I complain about ? All right, I'm gonna help you. ;-) I had performance problems with hash lookup and commit. Now I will ask of you an act of solidarity: let's leave JCW apart from this. I have been much of an annoyance to him those days. Let's give him a break, or better, two. Shoulder-pat Jean-Claude. Shoulder-pat. I'm sincere. Hello sir Borretos ! Now that noble defensors of Metakit have arrived on their white horses, we may try to solve this among us, following the ancient code (love this one). My mail is getting real long, so I'll split it in two. To be continued... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Mon Nov 26 23:36:17 2001 Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA07103 for ; Mon, 26 Nov 2001 23:36:17 -0600 Received: from toulouse-2-a7-36-254.dial.proxad.net (HELO desktop) (62.147.36.254) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 05:28:36 -0000 Message-ID: <008501c17704$25690a60$fe24933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 06:27:01 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Part II This mail is the continuation of another. So if you didn't read the first mail, I leave it to your cleverness what you have to do. JCW wrote: >With so much finger pointing going on, I simply had to go check things. >Below are some results, using the Tcl scripting language on a PIII/650 >under SuSE Linux 7.3, with 256 MB RAM. C++ ought to perform better. This was followed by a long series of numbers. But what did those numbers say in fact. They only showed that Add() takes constant time, that's all. They didn't tell us anything interesting about Find() and Commit(), so let's forget them. On the contrary, JCW mailed me other results, which tell a lot more. Those are the results of a test JCW wrote especially for me in C++. I shall never thank him enough for the time he has dedicated to me. I don't include the source for brevity, but here are those results for1 view with 1 column: /* Timing results on Linux Suse 7.3, gcc 2.95.3, PII/650, 256 Mb RAM: NORMAL VIEW ROW ADD FIND COMMIT SIZE 25000 7 uS 81921 uS 30691 uS 1998304 bytes 50000 8 uS 164343 uS 55400 uS 3552352 bytes 75000 7 uS 247122 uS 169282 uS 6408400 bytes 100000 7 uS 327152 uS 162672 uS 6408400 bytes 125000 8 uS 411907 uS 135025 uS 8571424 bytes 150000 8 uS 495606 uS 156729 uS 11469448 bytes 175000 8 uS 580956 uS 181985 uS 11469448 bytes 200000 8 uS 660399 uS 218114 uS 15396472 bytes 225000 7 uS 743788 uS 247174 uS 15396472 bytes 250000 7 uS 824172 uS 259353 uS 20499496 bytes BLOCKED VIEW ROW ADD FIND COMMIT SIZE 25000 11 uS 153098 uS 24847 uS 663167 bytes 50000 13 uS 347281 uS 48650 uS 1275917 bytes 75000 14 uS 579764 uS 72652 uS 1872917 bytes 100000 15 uS 856977 uS 96022 uS 2433167 bytes 125000 17 uS 1160784 uS 119760 uS 2998667 bytes 150000 18 uS 1513262 uS 143609 uS 3609065 bytes 175000 20 uS 1898778 uS 168134 uS 4134665 bytes 200000 22 uS 2339882 uS 195486 uS 4727417 bytes 225000 29 uS 2799415 uS 219007 uS 5329385 bytes 250000 41 uS 3287381 uS 243094 uS 5940569 bytes HASH VIEW ROW ADD FIND COMMIT SIZE 25000 23 uS 69 uS 33690 uS 2037185 bytes 50000 34 uS 73 uS 63885 uS 4713013 bytes 75000 24 uS 83 uS 90765 uS 6285961 bytes 100000 19 uS 83 uS 134531 uS 9706905 bytes 125000 31 uS 88 uS 153336 uS 10755509 bytes 150000 27 uS 89 uS 188538 uS 11804113 bytes 175000 20 uS 83 uS 232263 uS 16506741 bytes 200000 25 uS 84 uS 251398 uS 20701101 bytes 225000 24 uS 86 uS 289060 uS 22798281 bytes 250000 35 uS 96 uS 381825 uS 22798281 bytes BLOCKED HASH VIEW ROW ADD FIND COMMIT SIZE 25000 42 uS 72 uS 28302 uS 1582622 bytes 50000 44 uS 88 uS 60463 uS 3942086 bytes 75000 37 uS 85 uS 83439 uS 4466402 bytes 100000 35 uS 116 uS 120280 uS 7087926 bytes 125000 51 uS 104 uS 146270 uS 9185134 bytes 150000 50 uS 125 uS 164883 uS 9185134 bytes 175000 57 uS 109 uS 236153 uS 12330918 bytes 200000 62 uS 112 uS 240761 uS 14428098 bytes 225000 56 uS 130 uS 267224 uS 14428098 bytes 250000 84 uS 106 uS 281014 uS 20719638 bytes So what do we learn about Commit(). Whatever the kind of view, committing 250,000 rows takes roughly 10 more times that committing 25,000 rows. The relation is in fact linear, as we best see with the hash view (close to 3, 6, 9, 12, 15, 18, etc...). 25,000 x 10 = 250,000 Committing the last 25,000 rows takes as long as committing the 250,000 rows of the full database. We can generalize: whatever the number of new rows you commit, that takes as long as committing the entire database, i.e. if you add 10 rows in a 300,000 rows database, that will take as long as if you commited 300,000 rows. If we enter inside details, that's even worse than that. Let's stick to the hash view example: In 33690 uS, we add 25,000 rows and commit them. In 10 times 33690us, we should be able to add 250,000 rows and commit them. But in fact, in 381825 uS, all we do is adding 25,000 rows and commit them. I hope you see the picture though it's hard to describe. So here is Commit's performance. That seems terribly disabling to me but everyone seems satisfied with it. The conclusion is simple: I'm too demanding. Let's talk about Berkeley DB (excuse me for the swearword, let's call it Bedebeu). I am in no way trying to sell Bedebeu to you. I have never tryed it so I don't know how its Commit performs. But Bedebeu prouds itself to scale up to 256 terabytes. If Bedebeu's Commit performs as Metakit's Commit, programmers certainly think it twice before adding a row. ;-) Sorry Sir Borretos, I've heard you ! "Why don't you use Bedebeu and leave us alone ?" you're saying. Because on paper Metakit satisfies me much more than Bedebeu. Because I've spent quite some time learning Metakit's concepts (by the way, I really don't understand this need to change all commonly used database terms into others). Because I've even designed a structure which suits my needs taking advantage of Metakit's architecture (especially subviews). And because I'm in love with Jean-Claude (I write to him day and night). Let's move on to my second problem: Find() performance. As you can see on JCW's test, there's no problem with Find() at all. It performs smoothly wether there's 25,000 rows or 250,000. And that's where I beg for your assistance, noble assembly. Because I have written an equivalent test program, and with this program I see Find() execution time double, even triple when reaching the 250,000th row. JCW cheched my code and confirmed it was OK. So for a long time, we thought, JCW and I, that it was because my computer did not have 256 Mo of RAM, but 64Mo or 128 Mo according to the computer I used. Que nenni ! My program performed as badly when run by JCW's 256 Mo computer ! Here I am now: I'm pretty sure it's a compilation problem. I'm certain of using the latest cvs sources, so the problem is not there. When I compile \win\msvc60\mksrc.dsp, I get mkvc60.lib, mkvc60_d.lib, mkvc60s.lib, mkvc60s_d.lib, mkvc60s_mfc_d.lib and mkvc60s_std_d.lib. mkvc60s.lib seems to be the right static library to use. But I get with it the bad results I descrive above. And I also get those bad results when using the dll. By the way, mkvc60s.lib is 542 ko. Metakit is supposed to be less than 200 ko... I don't really understand. I would really appreciate if someone confirmed me that I am using Metakit the right way. Or if someone thought of a tricky pitfall I may have fallen into. If you went through all this titanic mail (3h30), you've earned my eternal consideration. ;-) _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From niki@vintech.bg Tue Nov 27 02:38:31 2001 Received: from www.vintech.bg (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA11207 for ; Tue, 27 Nov 2001 02:38:26 -0600 Received: from vintech.bg (sierra [10.0.0.7]) by www.vintech.bg (8.11.2/8.8.7) with ESMTP id fAR8cbs07386 for ; Tue, 27 Nov 2001 10:38:37 +0200 Message-ID: <3C035061.2020608@vintech.bg> Date: Tue, 27 Nov 2001 10:35:45 +0200 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.6) Gecko/20011120 X-Accept-Language: bg, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Retro-spection References: <008501c17704$25690a60$fe24933e@desktop> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Here I am now: I'm pretty sure it's a compilation problem. > I'm certain of using the latest cvs sources, so the problem is not there. > When I compile \win\msvc60\mksrc.dsp, I get mkvc60.lib, mkvc60_d.lib, > mkvc60s.lib, mkvc60s_d.lib, mkvc60s_mfc_d.lib and mkvc60s_std_d.lib. > mkvc60s.lib seems to be the right static library to use. > But I get with it the bad results I descrive above. > And I also get those bad results when using the dll. Well you compare Linux with Windows. We all know which is better. > By the way, mkvc60s.lib is 542 ko. Metakit is supposed to be less than > 200 ko... I don't really understand. Lib files are not 100% code. Even DLL are not 100% code. HTH, Niki Spahiev From steve@improvision.com Tue Nov 27 04:43:01 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA16065 for ; Tue, 27 Nov 2001 04:43:00 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id KAA16821 for ; Tue, 27 Nov 2001 10:33:20 GMT Received: from [192.168.1.25] ([192.168.1.25]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Tue, 27 Nov 2001 10:35:16 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Tue, 27 Nov 2001 10:35:19 +0000 Subject: Re: [Metakit] Retro-spection From: Stephen Baxter To: Pascal Baspeyras , Metakit Message-ID: In-Reply-To: <008501c17704$25690a60$fe24933e@desktop> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is the one that is odd. I thought the whole point of blocked views was that we did *not* have to commit the entire view, just the "block" that has changed. JCW - any thoughts? I did a performance test with a large database last night and commits do rapidly become very slow with a few thousand rows. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: "Pascal Baspeyras" > Date: Tue, 27 Nov 2001 06:27:01 +0100 > To: "Metakit" > Subject: [Metakit] Retro-spection > > BLOCKED VIEW > > ROW ADD FIND COMMIT SIZE > 25000 11 uS 153098 uS 24847 uS 663167 bytes > 50000 13 uS 347281 uS 48650 uS 1275917 bytes > 75000 14 uS 579764 uS 72652 uS 1872917 bytes > 100000 15 uS 856977 uS 96022 uS 2433167 bytes > 125000 17 uS 1160784 uS 119760 uS 2998667 bytes > 150000 18 uS 1513262 uS 143609 uS 3609065 bytes > 175000 20 uS 1898778 uS 168134 uS 4134665 bytes > 200000 22 uS 2339882 uS 195486 uS 4727417 bytes > 225000 29 uS 2799415 uS 219007 uS 5329385 bytes > 250000 41 uS 3287381 uS 243094 uS 5940569 bytes > From lvirden@cas.org Tue Nov 27 06:08:35 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA20639 for ; Tue, 27 Nov 2001 06:08:34 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id fARC0Oj06607 for ; Tue, 27 Nov 2001 07:00:24 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fARC0M020182; Tue, 27 Nov 2001 07:00:22 -0500 (EST) Date: Tue, 27 Nov 2001 07:00:22 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011127070022.AAB19982@cas.org> Subject: Re: [Metakit] Retro-spection In-Reply-To: Your message of Tue, 27 Nov 2001 06:24:06 +0100 To: "Metakit" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Pascal Baspeyras" > But on the other hand, subject was serious: I'd really like my application > to be available for free. I have never heard of JCW charging for his > services, > so I would have liked to know what kind of partnership or sponsorship it is > possible to contract. This may certainly not be the right forum to discuss > such subject, but if someone could tell me about his experience or direct me > toward the right forum... Interesting topic - probably not on-topic here. JCW, I'd be happy to redirect elsewhere - do you want us to move over to usenet somewhere? The topic is "how does a programmer make money from open source" and JCW and I have discussed this in private email for years . An interesting experience I recently had was seeing someone who actually had been awarded software patents announcing on comp.lang.tcl some open source software! Here are some ideas - I am not suggesting that any or all of these relate to JCW's situation. Just a variety of thoughts that are available. o custom programming for hire - writing specific applications for a client (which might include some portion of code returnable to the open source community) o support for hire - the priority resolution of particular software in specific applications. While many problems are of a nature that their resolution can come over time, some problems need resolved NOW - and some are willing to pay for that timely service (in this case, it is very important that fixes roll back into the general source tree) o training - the one on one approach to training (for question answering, hand holding, etc.) is sometimes preferred over the reading of man pages (in this case, it would be helpful if improvement to docs, etc. rolled back into the open source community) o extending software/documentation for hire - some people appreciate the open source nature of a product so much they are willing to 'hire' a developer to code or write particular functionality or reference material for the general community. (in this case, the very nature of the work is for improvements to be rolled back into the open source community) > I had performance problems with hash lookup and commit. I don't recall in this painful thread whether the specific code was provided. However, I generally ask people, when reporting problems like "the library is too slow" or "the program doesn't work" to provide small coherent examples of the problem, including the error output, and a detailed description of the environment being used (including version numbers, etc.). I also encourage software writers to develop tools that gather this info to make such problem reports easier to submit... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Tue Nov 27 07:37:13 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA26086 for ; Tue, 27 Nov 2001 07:37:12 -0600 Received: from takkie.equi4.local ([195.121.236.245]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNGO4D00.SZK for ; Tue, 27 Nov 2001 14:29:01 +0100 Received: from [10.0.1.37] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fARDSE028834 for ; Tue, 27 Nov 2001 14:28:14 +0100 From: Jean-Claude Wippler To: Metakit Date: Tue, 27 Nov 2001 14:28:55 +0100 Message-Id: <20011127132855.27220@mail.planet.nl> In-Reply-To: <20011127070022.AAB19982@cas.org> References: <20011127070022.AAB19982@cas.org> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] A plea Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I would like to request a time-out from everyone on this mailing list. There is right and there is wrong. There are facts and there is emotion. There is justice and there is injustice. I am embarrassed to say that I have overlooked something important. Please allow me the time it takes to fully understand and resolve it. When I do, you will all be the first to know. Now that so much energy has been used up, let's try to make the very best use of what still remains. Regards, Jean-Claude From nicolasb@maich.gr Tue Nov 27 07:44:06 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id HAA26530 for ; Tue, 27 Nov 2001 07:44:01 -0600 Received: FROM maich.gr BY mail-server ; Tue Nov 27 15:51:41 2001 +0200 Message-ID: <3C039B04.4A9472B4@maich.gr> Date: Tue, 27 Nov 2001 15:54:12 +0200 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: metakit@equi4.com References: <001801c17708$57df9a00$bc24933e@desktop> Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Subject: [Metakit] Cultural Differences? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Pascal Baspeyras wrote: > > Be warned: this is an un-aggressive mail ! I am sorry Pascal (you asked not to be called mister), to me at this seems like an aggressive mail, or at least mockery to this list.. > > > >Too often I have found in my life people who, because of some > misunderstanding > >- sometimes cultural, sometimes because of translation problems, sometimes > >because of personal problems or issues - end up being 'flamed' in public > >and, because of the embarassment, give up on some particular mailing > >list or product. In the Greek culture, particularly when we do not know someone, out of respect, we refer to someone as Mr. ... Additionally, we address them in the plural but in English there is neither a written nor a spoken distinction between "you guy", and "you guys"; it's just "you". In Greek, "you" singular, trans-literated is "esei", while "you" plural is "eseis" with an "s" appended. I am sorry that you mis-read my intentions. > >What I have tried to do when I find _myself_ in such situations is this: > >1. examine what I have said and determine whether I owe someone an > >apology - either public or private > >2. examine what benefits I get from the product and decide if a public > >apology should be offered - whether I owe it or not. > >3. most of all, determine when I am willing to give up having the last word > >and just letting things go, hoping that after some decent period of time > >I can expect things to 'resume normalcy'. I was really hoping that the last word was your apology yesterday, but alas:-( > > Hello sir Borretos ! Now that noble defensors of Metakit have arrived on > their white horses, we may try to solve this among us, following the ancient > code (love this one). For someone who gets easily irked with monikers, you obviously dont have a problem using them... Please evaluate your sense of humor sir! (no pun intended) and please dont waste my time anymore with your rants poorly disguised as diatribes... > > My mail is getting real long, so I'll split it in two. > > To be continued... Must you??? nicolas boretos (spelled with one r) > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Tue Nov 27 08:20:15 2001 Received: from smtp013.mail.yahoo.com (smtp013.mail.yahoo.com [216.136.173.57]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA28847 for ; Tue, 27 Nov 2001 08:20:15 -0600 Received: from toulouse-2-a7-38-254.dial.proxad.net (HELO desktop) (62.147.38.254) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 14:12:34 -0000 Message-ID: <005b01c1774d$56f3f420$fe26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 15:09:54 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] A plea Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Jean-Claude, You write: >I am embarrassed to say that I have overlooked something important. >Please allow me the time it takes to fully understand and resolve it. >When I do, you will all be the first to know. I suppose you're talking about the Commit() issue which degrades Metakit's performance a lot. It looks like transparent on-the-fly restructuring is a very costly feature which is scarcely used. Wouldn't it be great if we could enable or disable it at will ? Just an idea. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From mlh@idi.ntnu.no Tue Nov 27 09:23:23 2001 Received: from zevs.idi.ntnu.no (IDENT:0@[129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA32559 for ; Tue, 27 Nov 2001 09:23:23 -0600 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id QAA06288 for ; Tue, 27 Nov 2001 16:15:42 +0100 (MET) Message-ID: <00c601c17756$614af640$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: Date: Tue, 27 Nov 2001 16:15:40 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Recommended article Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Useful guidelines for electronic communication: http://www.rand.org/publications/MR/R3283/ -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.sf.net From baspey@yahoo.com Tue Nov 27 09:50:31 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA01774 for ; Tue, 27 Nov 2001 09:50:30 -0600 Received: from toulouse-2-a7-36-173.dial.proxad.net (HELO desktop) (62.147.36.173) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 15:42:49 -0000 Message-ID: <005c01c17759$f28b61a0$fe26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 16:36:29 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Niki Spahiev wrote: >Lib files are not 100% code. Even DLL are not 100% code. Thanks a lot for this info. I didn't know it could be in such a ratio. Niki Spahiev wrote >Well you compare Linux with Windows. We all know which is better. You're absolutely right. I had thought about that but forgot to mention. It is indeed a possibility that Find() performs right under Linux and wrong under Windows. Here is the C++ program JCW mailed me: #include #include #include #include #include #include long ticks() { struct timeval tv; struct timezone tz; gettimeofday(&tv, &tz); return tv.tv_sec * 1000000 + tv.tv_usec; } int main(int argc, char **argv) { c4_StringProp pKey ("key"); c4_Storage storage ("Database.yumm", true); c4_View data = storage.GetAs("data[key:S]"); if (argc > 1) { if (strchr(argv[1], 'b')) { data = storage.GetAs("data[_B[key:S]]"); data = data.Blocked(); } if (strchr(argv[1], 'h')) { c4_View map = storage.GetAs("map[_H:I,_R:I]"); data = data.Hash(map, 1); } } char buf [25]; c4_Row row; struct stat sb; long t; puts(" ROW ADD FIND COMMIT SIZE"); for (int i = 1; i <= 250000; ++i) { sprintf(buf, "%10d%10d", 100000000 + i, 200000000 + i); pKey (row) = buf; t = ticks(); data.Add(row); long at = ticks() - t; if (i % 1000 == 0) { t = ticks(); int n = data.Find(row); long ft = ticks() - t; if (n < 0) { puts(buf); return 1; } t = ticks(); storage.Commit(); long ct = ticks() - t; if (i % 25000 == 0) { stat("Database.yumm", &sb); printf("%9d %9d uS %9d uS %9d uS %9d bytes \n", i, at, ft, ct, sb.st_size); } } } return 0; } He then launches the program with this command line: % g++ -Dq4_INLINE mkhash.cpp -lmk4 % for i in "" b h bh; do rm -f Database.yumm; a.out $i; done Unfortunatley, Windows API doesn't have gettimeofday() nor stat(). Here are the modifications I propose to run an equivalent program under Windows: int main(int argc, char **argv) { // will be used for performance measuring LARGE_INTEGER Freq, Time1, Time2; QueryPerformanceFrequency(&Freq); double AddTook = 0; double FindTook = 0; double CommitTook = 0; c4_StringProp pKey ("key"); c4_Storage storage ("Database.yumm", true); c4_View data = storage.GetAs("data[key:S]"); /* if (argc > 1) { if (strchr(argv[1], 'b')) { data = storage.GetAs("data[_B[key:S]]"); data = data.Blocked(); } if (strchr(argv[1], 'h')) { c4_View map = storage.GetAs("map[_H:I,_R:I]"); data = data.Hash(map, 1); } } */ // let's do only the blocked/hash test data = storage.GetAs("data[_B[key:S]]"); data = data.Blocked(); c4_View map = storage.GetAs("map[_H:I,_R:I]"); data = data.Hash(map, 1); wxString msg; char buf [25]; c4_Row row; puts(" ROW ADD FIND COMMIT SIZE"); for (int i = 1; i <= 250000; ++i) { sprintf(buf, "%10d%10d", 100000000 + i, 200000000 + i); pKey (row) = buf; QueryPerformanceCounter(&Time1); data.Add(row); QueryPerformanceCounter(&Time2); AddTook = ((double)Time2.QuadPart - (double)Time1.QuadPart) / (double)Freq.QuadPart; if (i % 1000 == 0) { QueryPerformanceCounter(&Time1); int n = data.Find(row); QueryPerformanceCounter(&Time2); FindTook = ((double)Time2.QuadPart - (double)Time1.QuadPart) / (double)Freq.QuadPart; /* if (n < 0) { puts(buf); return; } */ QueryPerformanceCounter(&Time1); storage.Commit(); QueryPerformanceCounter(&Time2); CommitTook = ((double)Time2.QuadPart - (double)Time1.QuadPart) / (double)Freq.QuadPart; if (i % 25000 == 0) { // stat("Database.yumm", &sb); // printf("%9d %9d uS %9d uS %9d uS %9d bytes \n", // i, at, ft, ct, sb.st_size); Printf(_("%d Add() => %f average Find() => %f Commit() => %f"), i, AddTook, FindTook, CommitTook); } } } return 0; } When JCW got almost constant Find times under Linux, here is what I get using the above code under Windows on my Celeron 800 with 64 Mo of RAM: 25,000 rows => 73 us 50,000 rows => 86 us 75,000 rows => 132 us 100,000 rows => 152 us 125,000 rows => 163 us 150,000 rows => 185 us 175,000 rows => 125 us 200,000 rows => 216 us 225,000 rows => 215 us 250,000 rows => 147 us N.B: it's even more worrying if I go beyond 250,000 rows or if I sample more often than every 25,000 rows (code is easy to change). Feel free to try this code on your own computer with your own compiling methods. That may help a bunch. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Tue Nov 27 09:55:50 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA02121 for ; Tue, 27 Nov 2001 09:55:50 -0600 Received: from toulouse-1-a7-33-231.dial.proxad.net (HELO desktop) (62.147.33.231) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 15:48:10 -0000 Message-ID: <006801c1775a$b1678cc0$fe26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 16:45:03 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Cultural Differences? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas Boretos wrote: >> >> My mail is getting real long, so I'll split it in two. >> >> To be continued... >Must you??? That's the second part which is interesting. The first part was just for relocating things. My apologizes to Magnus Lie Hetland for this first part. ;-) _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Tue Nov 27 10:36:55 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA04470 for ; Tue, 27 Nov 2001 10:36:54 -0600 Received: from toulouse-2-a7-37-171.dial.proxad.net (HELO desktop) (62.147.37.171) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 16:29:14 -0000 Message-ID: <000701c17760$6e23f2e0$ab25933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 17:25:52 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks a lot to Larry W. Virden for rehabilitating my "Is JCW subject". The solutions Larry talk about apply very well to a library like Metakit, and are no real surprise to me. The application I'm working on is rather of that kind: a sympathetical utility which may interest a real lot of users because it's useful and easy to use. Most of this kind of applications are free, though there often is a paying pro version. I think of apps like WinZip, WinAmp, Acrobat Reader, Netscape Navigator, etc... All of them have an important development's cost, but the user don't pays anything. Their fundings are mystery to me. I know this is not the right place, but I'd really like to continue this discussion somewhere. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From horst@freedict.de Tue Nov 27 23:40:37 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA11919 for ; Tue, 27 Nov 2001 23:40:37 -0600 From: horst@freedict.de Received: (qmail 29169 invoked from network); 28 Nov 2001 05:56:01 -0000 Received: from dialin-145-254-112-096.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.112.96) by mail.netbeat.de with SMTP; 28 Nov 2001 05:56:01 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fAS5WBi15086; Wed, 28 Nov 2001 06:32:12 +0100 Message-Id: <200111280532.fAS5WBi15086@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Wed, 28 Nov 2001 06:32:08 +0100 (CET) Reply-To: horst@freedict.de Subject: Re: [Metakit] Retro-spection To: baspey@yahoo.com cc: metakit@equi4.com In-Reply-To: <008501c17704$25690a60$fe24933e@desktop> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, the only remarkable news in this mail I find is: On 27 Nov, Pascal Baspeyras wrote: > Part II > > NORMAL VIEW > > ROW ADD FIND COMMIT SIZE > 250000 7 uS 824172 uS 259353 uS 20499496 bytes > > BLOCKED VIEW > > ROW ADD FIND COMMIT SIZE > 250000 41 uS 3287381 uS 243094 uS 5940569 bytes > > HASH VIEW > > ROW ADD FIND COMMIT SIZE > 250000 35 uS 96 uS 381825 uS 22798281 bytes > > BLOCKED HASH VIEW > > ROW ADD FIND COMMIT SIZE > 250000 84 uS 106 uS 281014 uS 20719638 bytes > and now go back and compare the sizes - I find the blocked view very interesting indeed (not the find performance so) - any hint's? Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From horst@freedict.de Tue Nov 27 23:40:41 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA11923 for ; Tue, 27 Nov 2001 23:40:40 -0600 From: horst@freedict.de Received: (qmail 29193 invoked from network); 28 Nov 2001 05:56:04 -0000 Received: from dialin-145-254-112-096.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.112.96) by mail.netbeat.de with SMTP; 28 Nov 2001 05:56:04 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fAS5Tui15059; Wed, 28 Nov 2001 06:29:57 +0100 Message-Id: <200111280529.fAS5Tui15059@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Wed, 28 Nov 2001 06:29:53 +0100 (CET) Reply-To: horst@freedict.de Subject: Re: [Metakit] Retro-spection To: baspey@yahoo.com cc: metakit@equi4.com In-Reply-To: <008501c17704$25690a60$fe24933e@desktop> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 27 Nov, Pascal Baspeyras wrote: > Part II .... noise .... > > If we enter inside details, that's even worse than that. > Let's stick to the hash view example: > In 33690 uS, we add 25,000 rows and commit them. > In 10 times 33690us, we should be able to add 250,000 rows and > commit them. > But in fact, in 381825 uS, all we do is adding 25,000 rows and > commit them. > I hope you see the picture though it's hard to describe. did you work it out? if you commit 25000 newly added rows at a time, it works out to be 10us per new row, but did you work out how metakit works? it's organised in columns, it's organised in columns, it's organised in columns. The example tells you this: - if things are time critical avoid commits, add as many rows as possible, then commit all new additions - metakit is open source, so as an experienced programmer you are welcome to take the source, have a look at it, and modify it locally to your needs. - When you find them usefull, send patches back to the original author, and he can consider to implement them into the product. - make sure you are fully aware, that you understand that metakit works differently to most other approaches (you mentioned: .. not mentioned ... > > Let's talk about Berkeley DB (excuse me for the swearword, let's call > it Bedebeu). > I am in no way trying to sell Bedebeu to you. > I have never tryed it so I don't know how its Commit performs. > But Bedebeu prouds itself to scale up to 256 terabytes. > If Bedebeu's Commit performs as Metakit's Commit, programmers > certainly think it twice before adding a row. ;-) But adding a row is very fast in MetaKit, it's the commit which appears to bother you. - couldn't you for example just commit on program exit? ... more noise ... -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From baspey@yahoo.com Wed Nov 28 01:30:50 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA15015 for ; Wed, 28 Nov 2001 01:30:49 -0600 Received: from toulouse-2-a7-37-21.dial.proxad.net (HELO desktop) (62.147.37.21) by smtp.mail.vip.sc5.yahoo.com with SMTP; 28 Nov 2001 07:23:05 -0000 Message-ID: <000f01c177dd$4c305c00$1525933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Wed, 28 Nov 2001 08:20:04 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tsk, tsk, tsk, Horst... I prefer to forget the aggressive parts of your mails where you say nothing. Let's talk about the argumentative part. If I understand well, Commit() suits your needs perfectly well, and almost everyone must be in your case since nobody complained before me. Maybe Commit's poor performance is only recent, but I couldn't say since I use hash view and thus have to use the latest cvs sources. I am no maniac whose only preoccupation is to measure things. If I took time to measure Find() and Commit() performances, it's because I was looking for a bottleneck I could observe with my bare eyes: the operation of adding 1000 new rows should have (in my idea at least) performed in constant time since I used hash view, but instead it took ever increasing time, very rapidly increasing time. >did you work it out? if you commit 25000 newly added rows at a time, it >works out to be 10us per new row, but did you work out how metakit >works? it's organised in columns, it's organised in columns, it's >organised in columns. Column-wise organisation ? Yeah, heard about something like that. By the way, this test program is only 1 column. So I guess I didn't really work out how Metakit works because I don't understand why column-wise organisation should justify such results against row-wise organisation. I may be wrong but I think that 25,000 newly added rows means 10 us per row only if the database was blank, but it will become 100 us if the database already had 225,000 rows. I come to believe that you really missed my point... >if things are time critical avoid commits, add as many rows as >possible, then commit all new additions. Do you know of many database applications where you have all data from the start and where you just read it after that ? In my case, only the more addicted user will reach 300,000 rows. But as an average user, he won't have any idea of my program's underlying structure, as if I'm using Metakit or not. He will be placed in the case where he only made few additions and wait minutes for program to close (guess he will blame windows as always, unless he uses the Linux or Mac version of my program). 300,000 rows should be the biggest view. But there will 2 such views, plus a dictionary for fast word lookup. On this mailing list, someone once asked JCW if it was better to split a database into many. JCW told him it has no impact and using a single database would be more practical. According to my observation, I must say that this is a false statement. I also disagree with the documentation which states that using a blocked view will allow your database to "scale up indefinitely". I can't see in the test any benefit due to blocked view. On the contrary, "due to the extra level of indirection, blocked views are slightly slower". >But adding a row is very fast in MetaKit, it's the commit which appears >to bother you. - couldn't you for example just commit on program exit? As I said, my program will commit only on closing, but most of the time the user will have only made tens or hundreds of additions, like always in real-life. And I insist: the fact that Add() is fast can be completely ruined by Commit() performance. Maybe we should make a poll to know if people consider Commit's behaviour to be pathological or not ? Let me remind you my case: I really couldn't believe that was normal behaviour ( hence the "Performance can't be that bad" title) and asked help to know what I was doing wrong (did you really read my mails or did you stop to the fact that it's good to hate me ?). JCW agreed with me and tried hard to find out were I was doing wrong. Until the point we are now, were we know that this really is Metakit's "normal" behaviour. >metakit is open source, so as an experienced programmer you are >welcome to take the source, have a look at it, and modify it locally >to your needs. - When you find them usefull, send patches back to the >original author, and he can consider to implement them into the >product. If nothing changes, it's a (very unlikely) possibility that I modify sources for my "special" needs. That don't inspire me much since it will then be tedious to follow Metakit's evolution. Here is my position now: I made a bug report and am waiting for JCW conclusions. I think he understands the situation better than you do, and that's a chance. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jcw@equi4.com Wed Nov 28 08:28:46 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA02023 for ; Wed, 28 Nov 2001 08:28:46 -0600 Received: from takkie.equi4.local ([195.121.234.97]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNIL6602.BNJ for ; Wed, 28 Nov 2001 15:20:30 +0100 Received: from [10.0.1.38] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fASEJp006457 for ; Wed, 28 Nov 2001 15:19:52 +0100 From: Jean-Claude Wippler To: metakit Date: Wed, 28 Nov 2001 15:21:48 +0100 Message-Id: <20011128142148.5023@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] commit performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Here are the results of my timing experiments and changes. I'll try to keep it short, but need to throw figures around to explain things. The results below come from examples/mkhash.cpp, now in CVS (and as tar in the previews/ area). All tests were performed on the same Linux (SuSE 7.3) and PIII/650 + 256 Mb RAM notebook as before. First, building up a view of 250,000 rows, committing every 998, then 2 rows (that last commit is timed, i.e. a 2-row change). This is a hash view, with the data stored flat and the map not stored: dhs ROW ADD FIND COMMIT SIZE 25000 21 uS 154 uS 15077 uS 1936098 bytes 50000 36 uS 114 uS 32699 uS 3427104 bytes 75000 21 uS 112 uS 49081 uS 6094110 bytes 100000 22 uS 115 uS 61174 uS 8131092 bytes 125000 28 uS 130 uS 77383 uS 8131092 bytes 150000 29 uS 131 uS 94857 uS 10819116 bytes 175000 16 uS 121 uS 111775 uS 14389140 bytes 200000 26 uS 133 uS 119887 uS 14389140 bytes 225000 21 uS 129 uS 149678 uS 14389140 bytes 250000 39 uS 137 uS 166089 uS 19135164 bytes Adds and finds are O(1). Commit is O(N). File size is roughly O(N). Now the same, using a blocked view for the data: Dhs ROW ADD FIND COMMIT SIZE 25000 25 uS 116 uS 2177 uS 687868 bytes 50000 36 uS 104 uS 3989 uS 1436068 bytes 75000 26 uS 102 uS 5950 uS 2158018 bytes 100000 22 uS 99 uS 7642 uS 2853718 bytes 125000 49 uS 110 uS 9170 uS 3523168 bytes 150000 32 uS 110 uS 11171 uS 4166368 bytes 175000 24 uS 108 uS 12746 uS 4783318 bytes 200000 30 uS 113 uS 14435 uS 5374018 bytes 225000 29 uS 114 uS 16264 uS 5956570 bytes 250000 41 uS 123 uS 17759 uS 6494770 bytes Adds and finds are not affected much. Commit time is lower, because only changes to the modified (subview-) block need to be committed. File size is lower, because many small columns now reuse free space. File space usage is not much different, but the first case leads to a lot of free space in the file (re-used, but present nevertheless). The above figures are what happens after I fixed a major performance bug, if you compare with what has been posted before, there is a huge difference. The bug affects all string/bytes properties, when there are many views/subviews. Strings are stored as 2..3 columns, one with all data, the second with all string sizes, and the third with info about oversize (previously called "memo") entries. The bug was that sizes and oversize info was *always* written, even for columns in which nothing at all had changed in the string data. That means that for blocked views, sizes etc (but not the data itself) was being written to disk on commit for *all* blocks, including untouched ones. The current code no longer does an increasing number of writes in the 10 commits from 25k..250k rows (this has been verified with strace). But there still is an O(N) behavior in here. My current explanation for it is the fact that MK does full free-space re-determination on each commit, and this is proportional to the number of columns that are present in the entire datafile. That calculation could be tuned further (still O(N)), and there are provisions in the file format to cluster subtrees of data so the entire tree space is accounted for in one step. I have not yet explored that, and have to add that I may not get to it for some time to come (this is one reason for offering an Enterprise License: to reorder priorities and make things happen). In short: now that this particular I/O bottleneck has been removed, another CPU-bound - and O(N) - one moves to the surface. I think. I have tweaked a bit of the inlines, which showed up as bottlenecks in some profiling tests, but with mixed results (the German term for it is "verschlimmbesserung", i.e. 2 steps forwards, 2..3 steps back). I suspect that these are CPU cache and cache alignment effects. My priority was to better understand the issues, and subsequently to make sure that the change would continue to produce correct results. There are no leaks AFAIK, all regression tests pass, and so do the Tcl tests (they did bring up one bug in the changes, now resolved). There are still several improvements to make: the blocked view uses a fixed 500..1000 row block size, it could be adaptive based on the total number of rows, to lead to more scalable O(log N) performance. The free-space calculations could be done differently, in ways which will alter the O(x) effects and in simply streamlining it further. So where does this leave MK? Again, I think it illustrates that the performance of column-wise storage continues to be a highly complex tradeoff. Commits have always been the Achilles' heel, which is not so surprising, considering what commit does (even with one change). Blocked views are (now!) a step in the right direction, IMO. The untapped option for now, is commit-aside, which was specifically designed to bring differential commits into the picture. But it is not ready for prime time (in that it is not even differential today). At a higher level, another option is to create a difference custom viewer, i.e. one which caches changes, but makes them appear "real". An where does that leave me? Well, I must say that I was premature in rejecting the possibility that there could be a fault in Metakit. Inside every criticism there is a grain of truth - next time I will spend more effort looking for that. If I have offended anyone on this list with my defensiveness I offer my sincere apologies. But it pleases me that there's a winner in this, after all. MetaKit has evolved further, and thanks to everyone, performs significantly better than before in certain circumstances. Of course, whether that is sufficient for each of your needs remains up to you. Regards, Jean-Claude From lvirden@cas.org Wed Nov 28 08:40:13 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA02779 for ; Wed, 28 Nov 2001 08:40:13 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id fASEVwj28424 for ; Wed, 28 Nov 2001 09:31:58 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fASEVuk14756; Wed, 28 Nov 2001 09:31:56 -0500 (EST) Date: Wed, 28 Nov 2001 09:31:56 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011128093155.AAB14672@cas.org> Subject: Re: [Metakit] commit performance In-Reply-To: <20011128142148.5023@mail.planet.nl> of Wed, 28 Nov 2001 15:21:48 +0100 To: metakit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler > But it pleases me that there's a winner in this, after all. MetaKit > has evolved further, and thanks to everyone, performs significantly > better than before in certain circumstances. Of course, whether > that is sufficient for each of your needs remains up to you. Despite many criticisms people have of open source code, it is experiences like this that continue to keep me cheering free software on. I have had experiences in the past where I have provided detailed bug reports to vendors about their closed source application behaviors. In one case, I even was able to provide a code fix to a script that they were shipping which had incorrect behavior. The attitude was "well, we don't wish to be bothered fixing that problem - it is too low a priority". In my case, I just continued using my version of the script, and felt bad for all the people also using the product who either were unaware of the problem or not able to create the fix. In other cases, serious data damaging bugs were simply documented as "don't give sort lines longer than 1k as sort will break those lines" and so on. This too seems to me to be a bad approach - to know of a serious bug, and to simply document it as a limitation rather than spending the time to fix things. But, one might argue, the vendor has to allocate their resources carefully, picking the highest priority bugs to fix. And I would not disagree. I would, however, counter with the idea that with open source, one has the option of dealing with one's personal bugs directly, rather than waiting for the problem to rise on someone else's priority list... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From baspey@yahoo.com Wed Nov 28 09:31:44 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA05929 for ; Wed, 28 Nov 2001 09:31:44 -0600 Received: from toulouse-2-a7-39-154.dial.proxad.net (HELO desktop) (62.147.39.154) by smtp.mail.vip.sc5.yahoo.com with SMTP; 28 Nov 2001 15:23:47 -0000 Message-ID: <001001c17820$726e4380$9a27933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Wed, 28 Nov 2001 16:21:07 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] commit performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >If I have offended anyone on this list with my defensiveness I >offer my sincere apologies. I take them with pleasure. ;-) Larry W. Virden wrote: >Despite many criticisms people have of open source code, it is experiences >like this that continue to keep me cheering free software on. Me too. I always had had good experiences before. And now, let's go take a look at this cvs ! _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jyl@best.com Wed Nov 28 09:56:32 2001 Received: from sj1-3-4-18.iserver.com (sj1-3-4-18.iserver.com [192.220.127.211]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA07549 for ; Wed, 28 Nov 2001 09:56:32 -0600 Received: (qmail 14087 invoked by uid 19667); 28 Nov 2001 15:48:47 -0000 Received: from unknown (HELO best.com) ([unknown]) (envelope-sender ) by unknown (qmail-ldap-1.03) with SMTP for ; 28 Nov 2001 15:48:47 -0000 Message-ID: <3C050994.77F0298D@best.com> Date: Wed, 28 Nov 2001 07:58:12 -0800 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Subject: Re: [Metakit] commit performance References: <20011128142148.5023@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks, JC! Very good. That explains why even when I changed the data type in the program you wrote and which Pascal posted to I from S, I still saw (much faster! :) O(N) commit behavior. I modified the program to store the index I in the Ith row. I also verified by changing the program to commit only every 25K that that delaying the commit does not make a huge difference, neither in size nor in time, which means that your free space reclamation is a waste of time for each commit -- it seems OK to do it only once in a while. Finally, because I was running on Solaris, I thought to test the effects of NFS and remoteness of the file system on how Commit behaves. I could post the results when I get into work, later. The salient bit of info is that Commit time seems dominated by I/O overhead and that NFS can make Commit take upto 4X more time on a remote mounted FS than on a local one... Sheesh :( (not MK's fault). One question: as I wrote before, I am addressing rows by index and really need to know what row# a row occupied. Does this rule out use of Hashed or Blocked views? I could of course modify my schema to include my own index in each row, but it seems (MUCH) less efficient to find a row by searching with Find than byjust saying view[i]. Any insights appreciated! --JYL Jean-Claude Wippler wrote: > Here are the results of my timing experiments and changes. I'll try > to keep it short, but need to throw figures around to explain things. > The results below come from examples/mkhash.cpp, now in CVS (and as > tar in the previews/ area). All tests were performed on the same > Linux (SuSE 7.3) and PIII/650 + 256 Mb RAM notebook as before. > > First, building up a view of 250,000 rows, committing every 998, then > 2 rows (that last commit is timed, i.e. a 2-row change). This is a > hash view, with the data stored flat and the map not stored: > > dhs > ROW ADD FIND COMMIT SIZE > 25000 21 uS 154 uS 15077 uS 1936098 bytes > 50000 36 uS 114 uS 32699 uS 3427104 bytes > 75000 21 uS 112 uS 49081 uS 6094110 bytes > 100000 22 uS 115 uS 61174 uS 8131092 bytes > 125000 28 uS 130 uS 77383 uS 8131092 bytes > 150000 29 uS 131 uS 94857 uS 10819116 bytes > 175000 16 uS 121 uS 111775 uS 14389140 bytes > 200000 26 uS 133 uS 119887 uS 14389140 bytes > 225000 21 uS 129 uS 149678 uS 14389140 bytes > 250000 39 uS 137 uS 166089 uS 19135164 bytes > > Adds and finds are O(1). Commit is O(N). File size is roughly O(N). > > Now the same, using a blocked view for the data: > > Dhs > ROW ADD FIND COMMIT SIZE > 25000 25 uS 116 uS 2177 uS 687868 bytes > 50000 36 uS 104 uS 3989 uS 1436068 bytes > 75000 26 uS 102 uS 5950 uS 2158018 bytes > 100000 22 uS 99 uS 7642 uS 2853718 bytes > 125000 49 uS 110 uS 9170 uS 3523168 bytes > 150000 32 uS 110 uS 11171 uS 4166368 bytes > 175000 24 uS 108 uS 12746 uS 4783318 bytes > 200000 30 uS 113 uS 14435 uS 5374018 bytes > 225000 29 uS 114 uS 16264 uS 5956570 bytes > 250000 41 uS 123 uS 17759 uS 6494770 bytes > > Adds and finds are not affected much. Commit time is lower, because > only changes to the modified (subview-) block need to be committed. > File size is lower, because many small columns now reuse free space. > File space usage is not much different, but the first case leads to > a lot of free space in the file (re-used, but present nevertheless). > > The above figures are what happens after I fixed a major performance > bug, if you compare with what has been posted before, there is a huge > difference. The bug affects all string/bytes properties, when there > are many views/subviews. Strings are stored as 2..3 columns, one > with all data, the second with all string sizes, and the third with > info about oversize (previously called "memo") entries. The bug was > that sizes and oversize info was *always* written, even for columns > in which nothing at all had changed in the string data. That means > that for blocked views, sizes etc (but not the data itself) was being > written to disk on commit for *all* blocks, including untouched ones. > > The current code no longer does an increasing number of writes in the > 10 commits from 25k..250k rows (this has been verified with strace). > > But there still is an O(N) behavior in here. My current explanation > for it is the fact that MK does full free-space re-determination on > each commit, and this is proportional to the number of columns that > are present in the entire datafile. That calculation could be tuned > further (still O(N)), and there are provisions in the file format to > cluster subtrees of data so the entire tree space is accounted for in > one step. I have not yet explored that, and have to add that I may > not get to it for some time to come (this is one reason for offering > an Enterprise License: to reorder priorities and make things happen). > > In short: now that this particular I/O bottleneck has been removed, > another CPU-bound - and O(N) - one moves to the surface. I think. > > I have tweaked a bit of the inlines, which showed up as bottlenecks > in some profiling tests, but with mixed results (the German term for > it is "verschlimmbesserung", i.e. 2 steps forwards, 2..3 steps back). > I suspect that these are CPU cache and cache alignment effects. > > My priority was to better understand the issues, and subsequently to > make sure that the change would continue to produce correct results. > There are no leaks AFAIK, all regression tests pass, and so do the > Tcl tests (they did bring up one bug in the changes, now resolved). > > There are still several improvements to make: the blocked view uses > a fixed 500..1000 row block size, it could be adaptive based on the > total number of rows, to lead to more scalable O(log N) performance. > The free-space calculations could be done differently, in ways which > will alter the O(x) effects and in simply streamlining it further. > > So where does this leave MK? Again, I think it illustrates that the > performance of column-wise storage continues to be a highly complex > tradeoff. Commits have always been the Achilles' heel, which is not > so surprising, considering what commit does (even with one change). > Blocked views are (now!) a step in the right direction, IMO. > > The untapped option for now, is commit-aside, which was specifically > designed to bring differential commits into the picture. But it is > not ready for prime time (in that it is not even differential today). > At a higher level, another option is to create a difference custom > viewer, i.e. one which caches changes, but makes them appear "real". > > An where does that leave me? Well, I must say that I was premature > in rejecting the possibility that there could be a fault in Metakit. > Inside every criticism there is a grain of truth - next time I will > spend more effort looking for that. If I have offended anyone on > this list with my defensiveness I offer my sincere apologies. > > But it pleases me that there's a winner in this, after all. MetaKit > has evolved further, and thanks to everyone, performs significantly > better than before in certain circumstances. Of course, whether > that is sufficient for each of your needs remains up to you. > > Regards, > Jean-Claude > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jyl@best.com Wed Nov 28 09:56:43 2001 Received: from sj1-3-4-18.iserver.com (sj1-3-4-18.iserver.com [192.220.127.211]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA07575 for ; Wed, 28 Nov 2001 09:56:43 -0600 Received: (qmail 14100 invoked by uid 19667); 28 Nov 2001 15:48:59 -0000 Received: from unknown (HELO best.com) ([unknown]) (envelope-sender ) by unknown (qmail-ldap-1.03) with SMTP for ; 28 Nov 2001 15:48:59 -0000 Message-ID: <3C0509A0.3602CC8D@best.com> Date: Wed, 28 Nov 2001 07:58:24 -0800 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Subject: Re: [Metakit] commit performance References: <20011128142148.5023@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks, JC! Very good. That explains why even when I changed the data type in the program you wrote and which Pascal posted to I from S, I still saw (much faster! :) O(N) commit behavior. I modified the program to store the index I in the Ith row. I also verified by changing the program to commit only every 25K that that delaying the commit does not make a huge difference, neither in size nor in time, which means that your free space reclamation is a waste of time for each commit -- it seems OK to do it only once in a while. Finally, because I was running on Solaris, I thought to test the effects of NFS and remoteness of the file system on how Commit behaves. I could post the results when I get into work, later. The salient bit of info is that Commit time seems dominated by I/O overhead and that NFS can make Commit take upto 4X more time on a remote mounted FS than on a local one... Sheesh :( (not MK's fault). One question: as I wrote before, I am addressing rows by index and really need to know what row# a row occupied. Does this rule out use of Hashed or Blocked views? I could of course modify my schema to include my own index in each row, but it seems (MUCH) less efficient to find a row by searching with Find than byjust saying view[i]. Any insights appreciated! --JYL Jean-Claude Wippler wrote: > Here are the results of my timing experiments and changes. I'll try > to keep it short, but need to throw figures around to explain things. > The results below come from examples/mkhash.cpp, now in CVS (and as > tar in the previews/ area). All tests were performed on the same > Linux (SuSE 7.3) and PIII/650 + 256 Mb RAM notebook as before. > > First, building up a view of 250,000 rows, committing every 998, then > 2 rows (that last commit is timed, i.e. a 2-row change). This is a > hash view, with the data stored flat and the map not stored: > > dhs > ROW ADD FIND COMMIT SIZE > 25000 21 uS 154 uS 15077 uS 1936098 bytes > 50000 36 uS 114 uS 32699 uS 3427104 bytes > 75000 21 uS 112 uS 49081 uS 6094110 bytes > 100000 22 uS 115 uS 61174 uS 8131092 bytes > 125000 28 uS 130 uS 77383 uS 8131092 bytes > 150000 29 uS 131 uS 94857 uS 10819116 bytes > 175000 16 uS 121 uS 111775 uS 14389140 bytes > 200000 26 uS 133 uS 119887 uS 14389140 bytes > 225000 21 uS 129 uS 149678 uS 14389140 bytes > 250000 39 uS 137 uS 166089 uS 19135164 bytes > > Adds and finds are O(1). Commit is O(N). File size is roughly O(N). > > Now the same, using a blocked view for the data: > > Dhs > ROW ADD FIND COMMIT SIZE > 25000 25 uS 116 uS 2177 uS 687868 bytes > 50000 36 uS 104 uS 3989 uS 1436068 bytes > 75000 26 uS 102 uS 5950 uS 2158018 bytes > 100000 22 uS 99 uS 7642 uS 2853718 bytes > 125000 49 uS 110 uS 9170 uS 3523168 bytes > 150000 32 uS 110 uS 11171 uS 4166368 bytes > 175000 24 uS 108 uS 12746 uS 4783318 bytes > 200000 30 uS 113 uS 14435 uS 5374018 bytes > 225000 29 uS 114 uS 16264 uS 5956570 bytes > 250000 41 uS 123 uS 17759 uS 6494770 bytes > > Adds and finds are not affected much. Commit time is lower, because > only changes to the modified (subview-) block need to be committed. > File size is lower, because many small columns now reuse free space. > File space usage is not much different, but the first case leads to > a lot of free space in the file (re-used, but present nevertheless). > > The above figures are what happens after I fixed a major performance > bug, if you compare with what has been posted before, there is a huge > difference. The bug affects all string/bytes properties, when there > are many views/subviews. Strings are stored as 2..3 columns, one > with all data, the second with all string sizes, and the third with > info about oversize (previously called "memo") entries. The bug was > that sizes and oversize info was *always* written, even for columns > in which nothing at all had changed in the string data. That means > that for blocked views, sizes etc (but not the data itself) was being > written to disk on commit for *all* blocks, including untouched ones. > > The current code no longer does an increasing number of writes in the > 10 commits from 25k..250k rows (this has been verified with strace). > > But there still is an O(N) behavior in here. My current explanation > for it is the fact that MK does full free-space re-determination on > each commit, and this is proportional to the number of columns that > are present in the entire datafile. That calculation could be tuned > further (still O(N)), and there are provisions in the file format to > cluster subtrees of data so the entire tree space is accounted for in > one step. I have not yet explored that, and have to add that I may > not get to it for some time to come (this is one reason for offering > an Enterprise License: to reorder priorities and make things happen). > > In short: now that this particular I/O bottleneck has been removed, > another CPU-bound - and O(N) - one moves to the surface. I think. > > I have tweaked a bit of the inlines, which showed up as bottlenecks > in some profiling tests, but with mixed results (the German term for > it is "verschlimmbesserung", i.e. 2 steps forwards, 2..3 steps back). > I suspect that these are CPU cache and cache alignment effects. > > My priority was to better understand the issues, and subsequently to > make sure that the change would continue to produce correct results. > There are no leaks AFAIK, all regression tests pass, and so do the > Tcl tests (they did bring up one bug in the changes, now resolved). > > There are still several improvements to make: the blocked view uses > a fixed 500..1000 row block size, it could be adaptive based on the > total number of rows, to lead to more scalable O(log N) performance. > The free-space calculations could be done differently, in ways which > will alter the O(x) effects and in simply streamlining it further. > > So where does this leave MK? Again, I think it illustrates that the > performance of column-wise storage continues to be a highly complex > tradeoff. Commits have always been the Achilles' heel, which is not > so surprising, considering what commit does (even with one change). > Blocked views are (now!) a step in the right direction, IMO. > > The untapped option for now, is commit-aside, which was specifically > designed to bring differential commits into the picture. But it is > not ready for prime time (in that it is not even differential today). > At a higher level, another option is to create a difference custom > viewer, i.e. one which caches changes, but makes them appear "real". > > An where does that leave me? Well, I must say that I was premature > in rejecting the possibility that there could be a fault in Metakit. > Inside every criticism there is a grain of truth - next time I will > spend more effort looking for that. If I have offended anyone on > this list with my defensiveness I offer my sincere apologies. > > But it pleases me that there's a winner in this, after all. MetaKit > has evolved further, and thanks to everyone, performs significantly > better than before in certain circumstances. Of course, whether > that is sufficient for each of your needs remains up to you. > > Regards, > Jean-Claude > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Thu Nov 29 03:36:29 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA03431 for ; Thu, 29 Nov 2001 03:36:28 -0600 Received: from [10.0.1.39] ([195.121.234.104]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNK2AY00.Q2R for ; Thu, 29 Nov 2001 10:28:10 +0100 From: Jean-Claude Wippler Cc: metakit Subject: Re: [Metakit] commit performance Date: Thu, 29 Nov 2001 10:29:34 +0100 Message-Id: <20011129092934.629@mail.planet.nl> In-Reply-To: <3C0509A0.3602CC8D@best.com> References: <3C0509A0.3602CC8D@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > [...] I also verified by changing the program to commit >only every 25K that that delaying the commit does not make a huge >difference, neither in size nor in time, which means that your free space >reclamation is a waste of time for each commit -- it seems OK to do it >only once in a while. That's no so easy to change. I need to carry more info across a commit, right now commit cleans up a lot, and lets the next one re-determine things. Note that free space use is not stored on disk - it is implicit in the tree - so storing anything at all requires knowing the layout of the entire data structure on disk. But it's a good idea, now that we're this far such an optimization might be quite worthwhile. [...] >One question: as I wrote before, I am addressing rows by index and really >need to know what row# a row occupied. Does this rule out use of Hashed >or Blocked views? I could of course modify my schema to include my own >index in each row, but it seems (MUCH) less efficient to find a row by >searching with Find than byjust saying view[i]. Any insights appreciated! Blocked views will behave like any other view, you can in principle replace anything by blocked views. The trade-off is that access to row #i takes a bit more time (there's now a tiny scan loop, whereas flat indexing is immediate). Find can search fast, but only under certain circumstances: - if the underlying view is a hashed one, and if the find key includes all of the ones on which the has works (usually simply the first prop in the view), then it takes advantage of hashing, as the examples/mkhash.cpp code shows - if the underlying view is ordered, and as above, the key is appropriate, then again Find switches to binary search internally (as doing Search yourself) - in other cases, and for remaining properties, Find resorts to linear scan You say: >I am addressing rows by index and really >need to know what row# a row occupied. I don't understand. By index in which view? Row# in which view? Aren't those two the same thing? -jcw From baspey@yahoo.com Thu Nov 29 05:21:11 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id FAA07662 for ; Thu, 29 Nov 2001 05:21:11 -0600 Received: from toulouse-1-a7-32-121.dial.proxad.net (HELO desktop) (62.147.32.121) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 11:13:21 -0000 Message-ID: <003701c178c6$9f85fa60$7920933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 12:11:38 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello everybody, I guess you will say that I'm the kind of guy who's never satisfied, but I still have problems with Metakit. The fact is that JCW performed all his tests under Linux, while my application performed under Windows. And when I ran my application, it was really obvious, even before measuring, that something was wrong with Commit() and Find(). At last, I have come to understand why so many persons talked about "my special needs": the figures given by JWC were good, and are even better since the latest cvs changes. Indeed, even before the recent bug fix, Commit() never took more than 1/3 second, which would have been quite satisfying for my needs. Alas, I was far from having those good results under Windows... And if JCW found a bug, that's a pure coincidence if I spotted it. If I did not have had my Windows' problems, he would not have sent me his Commit() performance at all. Let me remind you the results JCW gets using examples\mkhash.cpp on his PIII/650 + 256 Mb RAM notebook: dhs ROW ADD FIND COMMIT SIZE 25000 21 uS 154 uS 15077 uS 1936098 bytes 50000 36 uS 114 uS 32699 uS 3427104 bytes 75000 21 uS 112 uS 49081 uS 6094110 bytes 100000 22 uS 115 uS 61174 uS 8131092 bytes 125000 28 uS 130 uS 77383 uS 8131092 bytes 150000 29 uS 131 uS 94857 uS 10819116 bytes 175000 16 uS 121 uS 111775 uS 14389140 bytes 200000 26 uS 133 uS 119887 uS 14389140 bytes 225000 21 uS 129 uS 149678 uS 14389140 bytes 250000 39 uS 137 uS 166089 uS 19135164 bytes Dhs ROW ADD FIND COMMIT SIZE 25000 25 uS 116 uS 2177 uS 687868 bytes 50000 36 uS 104 uS 3989 uS 1436068 bytes 75000 26 uS 102 uS 5950 uS 2158018 bytes 100000 22 uS 99 uS 7642 uS 2853718 bytes 125000 49 uS 110 uS 9170 uS 3523168 bytes 150000 32 uS 110 uS 11171 uS 4166368 bytes 175000 24 uS 108 uS 12746 uS 4783318 bytes 200000 30 uS 113 uS 14435 uS 5374018 bytes 225000 29 uS 114 uS 16264 uS 5956570 bytes 250000 41 uS 123 uS 17759 uS 6494770 bytes And here is what I get under Windows with my Celeron 800 + 64 Mb RAM notebook: dhs ROW ADD FIND COMMIT 25000 18 uS 368 uS 274497 uS 50000 36 uS 416 uS 518412 uS 75000 20 uS 450 uS 740287 uS 100000 27 uS 452 uS 970711 uS 125000 25 uS 217 uS 1096427 uS 150000 33 uS 505 uS 1432881 uS 175000 17 uS 603 uS 4129216 uS 200000 33 uS 593 uS 6728838 uS 225000 22 uS 299 uS 8930381 uS 250000 41 uS 336 uS 7629655 uS Dhs ROW ADD FIND COMMIT 25000 20 uS 223 uS 69844 uS 50000 29 uS 247 uS 52325 uS 75000 22 uS 246 uS 56790 uS 100000 20 uS 269 uS 73695 uS 125000 72 uS 244 uS 69494 uS 150000 28 uS 256 uS 52919 uS 175000 21 uS 218 uS 69990 uS 200000 29 uS 423 uS 44764 uS 225000 26 uS 554 uS 83888 uS 250000 38 uS 510 uS 61994 uS (I don't think this big difference may be due to the fact that my computer has "only" 64 Mb RAM. Indeed, I had already sent to JCW a test program which performed poorly with his 256 Mb computer) I have to precise that Dhs performs good thanks to the recent bug fix, but performed bad before. But since I am under Windows, I couldn't run examples\mkhash.cpp as it, because Windows' API does not have such things as gettimeofday() or stat(). I had to modify things, and there's always a possibility that I made wrong somewhere. For your information, here is how I made to execute examples\mkhash.cpp under Windows: I used \win\msvc60\mhtest.dsp as a fundation. I replaced the main procedure (found in file \test\regress.cpp) with this one: #include #include #include #include #include #include int main() { char buf [25]; c4_Row row; //long t; LARGE_INTEGER Freq, t1, t2; QueryPerformanceFrequency(&Freq); double at, ct, ft, ht; int nkeys = 1; setvbuf(stdout, NULL, _IOLBF, BUFSIZ); c4_Storage storage ("test.dat", true); c4_StringProp pKey ("key"), pKey2 ("key2"); c4_View data = pKey; c4_IntProp pH ("_H"), pR ("_R"); c4_View map = (pH, pR); //const char *s = argc > 1 ? argv[1] : "-"; char *s = ""; gets(s); if (strchr(s, '2')) { pKey2 (row) = "abcdefghijklmnopqrstuvwxyz"; nkeys = 2; if (strchr(s, 'd')) data = storage.GetAs("data[key:S,key2:S]"); if (strchr(s, 'D')) { data = storage.GetAs("data[_B[key:S,key2:S]]"); data = data.Blocked(); } } else { if (strchr(s, 'd')) data = storage.GetAs("data[key:S]"); if (strchr(s, 'D')) { data = storage.GetAs("data[_B[key:S]]"); data = data.Blocked(); } } if (strchr(s, 'm')) map = storage.GetAs("map[_H:I,_R:I]"); if (strchr(s, 'M')) { map = storage.GetAs("map[_B[_H:I,_R:I]]"); map = map.Blocked(); } if (strchr(s, 'h')) data = data.Hash(map, nkeys); if (strchr(s, 'o')) data = data.Ordered(); int limit = 250000; /* if (argc > 2) limit = atoi(argv[2]); puts(s); */ puts(" ROW ADD FIND COMMIT"); for (int i = 1; i <= limit; ++i) { sprintf(buf, "%10d%10d", 100000000 + i, 200000000 + i); pKey (row) = buf; // t = ticks(); QueryPerformanceCounter(&t1); data.Add(row); // long at = ticks() - t; QueryPerformanceCounter(&t2); at = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; // if ((i+2) % 1000 == 0 && strchr(s, 's')) if ((i+2) % 1000 == 0) storage.Commit(); if (i % 1000 == 0) { // t = ticks(); QueryPerformanceCounter(&t1); storage.Commit(); // long ct = ticks() - t; QueryPerformanceCounter(&t2); ct = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; if (i % (limit/10) == 0) { // t = ticks(); QueryPerformanceCounter(&t1); int n = data.Find(row); // long ft = ticks() - t; QueryPerformanceCounter(&t2); ft = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; if (n < 0) { puts(buf); return 1; } /* struct stat sb; stat("test.dat", &sb); printf("%9d %9d uS %9d uS %9d uS %9d bytes \n", i, at, ft, ct, sb.st_size); */ printf("%9d %f uS %f uS %f uS\n", i, at*1000000, ft*1000000, ct*1000000); } } } if (strchr(s, 'H')) { // t = ticks(); QueryPerformanceCounter(&t1); data = data.Hash(map, nkeys); // long ht = ticks() - t; QueryPerformanceCounter(&t2); ht = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; // t = ticks(); QueryPerformanceCounter(&t1); storage.Commit(); // long ct = ticks() - t; QueryPerformanceCounter(&t2); ct = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; printf("construct hash: %d uS, then commit: %d uS\n", ht, ct); } return 0; } Then, I selected the menu item "Build" - "Set Active Configuration" and chose "mktest - Win32Release". And to finish, I selected the menu item "Project" - "Settings", then the "Link" tab, and in the "Object/library modules" textbox, I wrote "..\..\builds\mk4vc60s.lib". In my idea, Metakit has a bug under Windows, which is the source of all my troubles from the beginning... Does someone object ? _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Thu Nov 29 06:00:18 2001 Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA09194 for ; Thu, 29 Nov 2001 06:00:17 -0600 Received: from toulouse-1-a7-32-121.dial.proxad.net (HELO desktop) (62.147.32.121) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 11:52:26 -0000 Message-ID: <001e01c178cc$15376d20$7920933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 12:50:43 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'd just like to add that I now understand perfectly the attitude of JCW (and others) toward me. This attitude was really fully justified, and that's even incredible JCW granted me so much attention ! JCW, YOU'RE REALLY A GREAT MAN. That's really a shame we did not localize the source of the problem sooner... At that time, I was talking about my mails formatting, when I said to JCW: >when something can go wrong, it always goes wrong with me. >I'm that kind of person. >Just have a look at my mails... This time, it went wrong from beginning till end. You get the picture of what my life can be... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From lvirden@cas.org Thu Nov 29 06:16:45 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA09924 for ; Thu, 29 Nov 2001 06:16:45 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id fATC8Qv07563 for ; Thu, 29 Nov 2001 07:08:27 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fATC8PH25321; Thu, 29 Nov 2001 07:08:25 -0500 (EST) Date: Thu, 29 Nov 2001 07:08:25 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011129070825.AAB25282@cas.org> Subject: Re: [Metakit] Metakit and Windows In-Reply-To: Your message of Thu, 29 Nov 2001 12:11:38 +0100 To: "Metakit" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Can I try to get a summary of what you see as the problem? Is it that on Windows (which windows was it again?) Metakit is taking as much as 18 times longer to perform the comit than it does on Linux - even when using a comperable program? Could someone work with Pascal and his version of the modified benchmark to make certain that it is being compiled comperably (I'm not a C++ programmer, so I'm not aware of what optimization flags , etc. one needs to include/avoid on various platforms)? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Thu Nov 29 06:21:31 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA10159 for ; Thu, 29 Nov 2001 06:21:30 -0600 Received: from [10.0.1.39] ([195.121.232.210]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNK9Y101.PA9 for ; Thu, 29 Nov 2001 13:13:13 +0100 From: Jean-Claude Wippler To: Metakit Subject: Re: [Metakit] Metakit and Windows Date: Thu, 29 Nov 2001 13:14:42 +0100 Message-Id: <20011129121442.27964@mail.planet.nl> In-Reply-To: <20011129070825.AAB25282@cas.org> References: <20011129070825.AAB25282@cas.org> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >Can I try to get a summary of what you see as the problem? >Is it that on Windows (which windows was it again?) Metakit is taking >as much as 18 times longer to perform the comit than it does on Linux - >even when using a comperable program? There more... it's also O(1), which is most surprising. >Could someone work with Pascal and his version of the modified benchmark >to make certain that it is being compiled comperably (I'm not a C++ >programmer, so I'm not aware of what optimization flags , etc. one needs >to include/avoid on various platforms)? I'm on the case, sir :) -jcw From baspey@yahoo.com Thu Nov 29 06:34:34 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA10828 for ; Thu, 29 Nov 2001 06:34:33 -0600 Received: from toulouse-1-a7-32-121.dial.proxad.net (HELO desktop) (62.147.32.121) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 12:26:46 -0000 Message-ID: <000801c178d0$e0dd1340$7920933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 13:24:46 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >Can I try to get a summary of what you see as the problem? >Is it that on Windows (which windows was it again?) Metakit is taking >as much as 18 times longer to perform the comit than it does on Linux - >even when using a comperable program? I'm using Windows Millenium Edition, and Visual C++ SP5. When I see the results of MkHash.cpp, only Commit() looks worrying. But when I was running my application, I also had blatant performance problem with Find(). I have no explanation. Thanks a lot for your attention. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Thu Nov 29 07:46:28 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id HAA14594 for ; Thu, 29 Nov 2001 07:46:28 -0600 Received: from toulouse-2-a7-36-195.dial.proxad.net (HELO desktop) (62.147.36.195) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 13:38:40 -0000 Message-ID: <001b01c178da$eba8dac0$c324933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 14:36:57 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, There's at least one thing I could specify about the blatant Find() performance problem in my app: before adding any row, I use Find() to check that the entry doesn't already exist. And then I use Add(), which induces another Find(), a useless one I don't know how to get rid of, but that's another problem ;-) In one word: my app uses lots of Find(), which made the problem blatant. P.S: someone has reminded me of the poor performance of Windows OS compared to Linux OS. Hope that's not the only explanation. And there's still the possibility I did wrong somewhere. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From lvirden@cas.org Thu Nov 29 08:03:46 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA15541 for ; Thu, 29 Nov 2001 08:03:45 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id fATDtHv02428 for ; Thu, 29 Nov 2001 08:55:17 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fATDtGQ27499; Thu, 29 Nov 2001 08:55:16 -0500 (EST) Date: Thu, 29 Nov 2001 08:55:16 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011129085516.AAB27405@cas.org> Subject: Re: [Metakit] Metakit and Windows In-Reply-To: Your message of Thu, 29 Nov 2001 14:36:57 +0100 To: "Metakit" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Certainly I would not be surprised to see an overall difference in performance between Linux and Windows/ME. I would however be somewhat surprised if one particular aspect of Metakit were greatly slower than all others. HOWEVER, I would not be willing to rule out the possibility that some limitation in any particular OS could have negative impact on some specific function of the library. I would hope, however, that in that case, some OS specific work around might be possible. Pascal, I know that in general this is a frustrating time. I would make one suggestion - while your application has its own characteristic behaviors which you are attempting to resolve, it would be MOST productive if you could come up with a relatively small, coherent example which demonstrates the problems you are seeing with Find, independant of all the various processing that is going on. I would be the first to recognize that this is not always possible. However, it is the preferable place to begin tracking down a problem such as you are encountering with Find. If this is not possible, then I would recommend begining the effort to create some sort of minimal bundle containing all the pieces necessary to recreate the problem - this would include application, data, etc. Finally, and I know this may sound silly, but have you tried using the latest metakit and your app, but with a database newly created with the new metakit? Perhaps something inside the database got mangled and that is the source of your problem? I've seen cases with other apps where use of an underlying dll or library resulted in a slightly incorrect database, resulting in various misbehaviors that were unexplainable. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jyl@best.com Thu Nov 29 09:06:45 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA19587 for ; Thu, 29 Nov 2001 09:06:45 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id GAA11686; Thu, 29 Nov 2001 06:58:26 -0800 (PST) Message-ID: <3C065B53.8030905@best.com> Date: Thu, 29 Nov 2001 07:59:15 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: "Larry W. Virden" CC: Metakit , Jacob At Sun Subject: Re: [Metakit] Metakit and Windows References: <20011129070825.AAB25282@cas.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A large portion of the overhead of Commit is due to file system issues and I/O costs. I'll post some numbers later today (against the unfixed Metakit unfortunately) which demonstrate up to four times slower commit on an NFS mounted file system than on a local UFS mounted file system. Comparing file system performance accross OSes is very dicey, one cannot draw any conclusions from performance of MK Commit on one system to predict performance on another, because the way the OS does I/O has a large effect. JYL Larry W. Virden wrote: >Can I try to get a summary of what you see as the problem? >Is it that on Windows (which windows was it again?) Metakit is taking >as much as 18 times longer to perform the comit than it does on Linux - >even when using a comperable program? > >Could someone work with Pascal and his version of the modified benchmark >to make certain that it is being compiled comperably (I'm not a C++ >programmer, so I'm not aware of what optimization flags , etc. one needs >to include/avoid on various platforms)? > From baspey@yahoo.com Thu Nov 29 09:52:04 2001 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA22873 for ; Thu, 29 Nov 2001 09:52:04 -0600 Received: from toulouse-2-a7-37-160.dial.proxad.net (HELO desktop) (62.147.37.160) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 15:44:16 -0000 Message-ID: <002801c178ec$772ae780$a025933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 16:41:39 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >HOWEVER, I would not be willing to rule out the possibility that some limitation >in any particular OS could have negative impact on some specific function of >the library. I would hope, however, that in that case, some OS specific >work around might be possible. That's incredible! Either you read in my mind, either that's an obvious point ! I had thought of writing the same phrase as your last one but did not. I also used the word "work around" and wanted to check it inside the dictionary! Larry also wrote: >it would be MOST productive if you could come up with a relatively small, >coherent example which demonstrates the problems you are seeing with Find, >independant of all the various processing that is going on. As I said elsewhere, I have already sent an executable + source to JCW. The timing was expressed in CPU cycles instead of uS, but Jean-Claude saw it clearly that it performed poorly. This example was built with the pre-bug sources and was in fact a stripped version of my full app and randomly generated a string, used find(), then Add(), and finally committed every 1000 additions. It measured the average Find(), that is it took into account the value of all 1000 Find(). This sample have most certainly been trashed, but I may mail it again on demand. Regards. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From readonly@getsoft.com Thu Nov 29 11:27:09 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA30391 for ; Thu, 29 Nov 2001 11:27:08 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 169UqJ-00017F-00; Thu, 29 Nov 2001 10:19:23 -0700 Date: Fri, 30 Nov 2001 04:21:25 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <152783213601.20011130042125@getsoft.com> To: "Pascal Baspeyras" CC: "Metakit" Subject: Re: [Metakit] Metakit and Windows In-Reply-To: <002801c178ec$772ae780$a025933e@desktop> References: <002801c178ec$772ae780$a025933e@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Might I humbly suggest that Pascal try increasing the amount of RAM in his computer. 64M is not really adequate in todays world. Windows performance can improve considerably with more RAM. When doing your tests you also need to take into account what other applications are running on the PC. I'd suggest closing everything possible, including VC++. You might also want to Defragment your drive and perform any other functions which may help to improve your PCs performance, such as increasing the size of your Windows swap file, or make sure Windows has the optimum settings for it. You should also be using FAT32 and not FAT. From memory you have a Celeron which may also have some overall impact. I have heard about various problems with Win ME and would not recommend anyone using it. W2K is by far a better development platform. Of course if the potential users of your product have machines with ME and 64M of RAM then you must consider that. SiSoft Sandra is a very good program for doing benchmarks on a PC and seeing where performance may be able to be improved. VC5 is obviously quite old now, so updating to VC6 is another consideration. And there are issues such as Release Build vs. Debug Build and compiler optimization settings. Debug Builds will be inherently slower than Release Builds. I hope this has been of some help. Friday, November 30, 2001, 2:41:39 AM, you wrote: PB> Larry W. Virden wrote: >>HOWEVER, I would not be willing to rule out the possibility that some PB> limitation >>in any particular OS could have negative impact on some specific function PB> of >>the library. I would hope, however, that in that case, some OS specific >>work around might be possible. PB> That's incredible! Either you read in my mind, either that's an obvious PB> point ! PB> I had thought of writing the same phrase as your last one but did not. PB> I also used the word "work around" and wanted to check it inside the PB> dictionary! PB> Larry also wrote: >>it would be MOST productive if you could come up with a relatively small, >>coherent example which demonstrates the problems you are seeing with Find, >>independant of all the various processing that is going on. PB> As I said elsewhere, I have already sent an executable + source to JCW. PB> The timing was expressed in CPU cycles instead of uS, but Jean-Claude saw PB> it clearly that it performed poorly. PB> This example was built with the pre-bug sources and was in fact a stripped PB> version of my full app and randomly generated a string, used find(), then PB> Add(), and finally committed every 1000 additions. PB> It measured the average Find(), that is it took into account the value of PB> all PB> 1000 Find(). PB> This sample have most certainly been trashed, but I may mail it again on PB> demand. PB> Regards. -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From baspey@yahoo.com Thu Nov 29 12:09:15 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA01407 for ; Thu, 29 Nov 2001 12:09:14 -0600 Received: from toulouse-2-a7-39-199.dial.proxad.net (HELO desktop) (62.147.39.199) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 18:01:18 -0000 Message-ID: <001701c178ff$9c2a5c60$c727933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 18:58:01 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >Might I humbly suggest that Pascal try increasing the amount of RAM in >his computer. 64M is not really adequate in todays world. Windows >performance can improve considerably with more RAM. I may be wrong, but I really think we can discard the memory problem. I have same problems with my desktop 128 Mb computer, and JCW had problems when running my test program with his 256 Mb notebook. I may once again be wrong, but I think we may discard the compilation problem too. To run MkHash.cpp, I simply changed MkTest.dsp, that is a project configured by JCW himself. As always, all suggestions are welcome and appreciated. Neville also wrote: >VC5 is obviously quite old now, so updating to VC6 is another >consideration. I made a typo: I meant Visual Studio 6 Service Pack 5. Thanks Neville for giving me the opportunity to precise those important points. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From steve@improvision.com Thu Nov 29 12:10:37 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA01559 for ; Thu, 29 Nov 2001 12:10:36 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA14524 for ; Thu, 29 Nov 2001 18:00:47 GMT Received: from [192.168.1.25] ([192.168.1.25]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Thu, 29 Nov 2001 18:02:48 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Thu, 29 Nov 2001 18:02:48 +0000 Subject: Re: [Metakit] Metakit and Windows From: Stephen Baxter To: Neville Franks , Pascal Baspeyras CC: Metakit Message-ID: In-Reply-To: <152783213601.20011130042125@getsoft.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Just to second this - debug builds under Win can be 100s of times slower than release because they use the debugging versions of MFC, memory management etc. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Neville Franks > Organization: Soft As It Gets Pty Ltd > Reply-To: Neville Franks > Date: Fri, 30 Nov 2001 04:21:25 +1100 > To: "Pascal Baspeyras" > Cc: "Metakit" > Subject: Re: [Metakit] Metakit and Windows > > Might I humbly suggest that Pascal try increasing the amount of RAM in > his computer. 64M is not really adequate in todays world. Windows > performance can improve considerably with more RAM. > > When doing your tests you also need to take into account what other > applications are running on the PC. I'd suggest closing everything > possible, including VC++. > > You might also want to Defragment your drive and perform any other > functions which may help to improve your PCs performance, such as > increasing the size of your Windows swap file, or make sure Windows > has the optimum settings for it. You should also be using FAT32 and > not FAT. From memory you have a Celeron which may also have some > overall impact. > > I have heard about various problems with Win ME and would not > recommend anyone using it. W2K is by far a better development > platform. Of course if the potential users of your product have > machines with ME and 64M of RAM then you must consider that. > > SiSoft Sandra is a very good program for doing benchmarks on a PC and > seeing where performance may be able to be improved. > > VC5 is obviously quite old now, so updating to VC6 is another > consideration. And there are issues such as Release Build vs. Debug > Build and compiler optimization settings. Debug Builds will be > inherently slower than Release Builds. > > I hope this has been of some help. > > > Friday, November 30, 2001, 2:41:39 AM, you wrote: > > PB> Larry W. Virden wrote: > >>> HOWEVER, I would not be willing to rule out the possibility that some > PB> limitation >>> in any particular OS could have negative impact on some specific function > PB> of >>> the library. I would hope, however, that in that case, some OS specific >>> work around might be possible. > > PB> That's incredible! Either you read in my mind, either that's an obvious > PB> point ! > PB> I had thought of writing the same phrase as your last one but did not. > PB> I also used the word "work around" and wanted to check it inside the > PB> dictionary! > > PB> Larry also wrote: >>> it would be MOST productive if you could come up with a relatively small, >>> coherent example which demonstrates the problems you are seeing with Find, >>> independant of all the various processing that is going on. > > PB> As I said elsewhere, I have already sent an executable + source to JCW. > PB> The timing was expressed in CPU cycles instead of uS, but Jean-Claude saw > PB> it clearly that it performed poorly. > PB> This example was built with the pre-bug sources and was in fact a stripped > PB> version of my full app and randomly generated a string, used find(), then > PB> Add(), and finally committed every 1000 additions. > PB> It measured the average Find(), that is it took into account the value of > PB> all > PB> 1000 Find(). > > PB> This sample have most certainly been trashed, but I may mail it again on > PB> demand. > > PB> Regards. > > > -- > Best Regards, > Neville Franks, Author of ED for Windows - the programmers editor, an > indispensable tool for great software development. > http://www.getsoft.com Version 4.01 just released, with a C++ Class View that > actually works. > mailto:readonly@getsoft.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From nicolasb@maich.gr Thu Nov 29 12:11:58 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA01687 for ; Thu, 29 Nov 2001 12:11:57 -0600 Received: FROM maich.gr BY mail-server ; Thu Nov 29 20:19:35 2001 +0200 Message-ID: <3C067CCF.B76B9E87@maich.gr> Date: Thu, 29 Nov 2001 20:22:07 +0200 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: Neville Franks CC: Pascal Baspeyras , Metakit Subject: Re: [Metakit] Metakit and Windows References: <002801c178ec$772ae780$a025933e@desktop> <152783213601.20011130042125@getsoft.com> Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: > > Might I humbly suggest that Pascal try increasing the amount of RAM in > his computer. 64M is not really adequate in todays world. Windows > performance can improve considerably with more RAM. Hi, While I tend to think that win and memory is not (at least not) the main problem, a seat of the pants experience that I have had is the following; Notebook, 750Mhz, 256Mb, Win2k on one partition and Redhat6.5 on the other. Following sw on both. Oracle 8.1.7 ACS3.4 AOL3.2 Response times from a seperate browser pc. Linux averaged ~1-2 seconds for most page requests Win2K averaged at least ~40 seconds and many times longer for many others; essentially rendering the system useless. Defragged win, similar swaps etc... You figure... nicolas boretos P.S. This is why I love tclhttpd/metakit for most things I do..... From baspey@yahoo.com Thu Nov 29 13:23:33 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id NAA07749 for ; Thu, 29 Nov 2001 13:23:32 -0600 Received: from toulouse-2-a7-37-237.dial.proxad.net (HELO desktop) (62.147.37.237) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 19:15:02 -0000 Message-ID: <001901c17909$e8fdd3a0$ed25933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 20:12:21 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >Just to second this - debug builds under Win can be 100s of times slower >than release because they use the debugging versions of MFC, memory >management etc. I agree that debug version is much slower than release version. But let this be clear, we're talking about RELEASE VERSION here. I know that Stephen understood right, but I wouldn't like people to get confused... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From horst@freedict.de Fri Nov 30 13:44:58 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id NAA07928 for ; Fri, 30 Nov 2001 13:44:58 -0600 From: horst@freedict.de Received: (qmail 23962 invoked from network); 30 Nov 2001 20:00:33 -0000 Received: from dialin-145-254-112-143.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.112.143) by mail.netbeat.de with SMTP; 30 Nov 2001 20:00:33 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fATLEjl02046; Thu, 29 Nov 2001 22:14:46 +0100 Message-Id: <200111292114.fATLEjl02046@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Thu, 29 Nov 2001 22:14:42 +0100 (CET) Reply-To: horst@freedict.de Subject: Re: [Metakit] Metakit and Windows To: baspey@yahoo.com cc: metakit@equi4.com In-Reply-To: <001901c17909$e8fdd3a0$ed25933e@desktop> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, do you know the cygwin tools? - With them you can compile (GNU) unix programs and run them under windows. This might help to establish where problems are. - I guess people would be able to help you more with options and other things, when you use these tools. What is the other's opinion? Horst On 29 Nov, Pascal Baspeyras wrote: > Stephen Baxter wrote: >>Just to second this - debug builds under Win can be 100s of times slower >>than release because they use the debugging versions of MFC, memory >>management etc. > > I agree that debug version is much slower than release version. > > But let this be clear, we're talking about RELEASE VERSION here. > > I know that Stephen understood right, but I wouldn't like people to get > confused... > > -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From baspey@yahoo.com Fri Nov 30 14:54:30 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA11768 for ; Fri, 30 Nov 2001 14:54:30 -0600 Received: from toulouse-2-a7-37-57.dial.proxad.net (HELO desktop) (62.147.37.57) by smtp.mail.vip.sc5.yahoo.com with SMTP; 30 Nov 2001 20:46:32 -0000 Message-ID: <000b01c179df$db8cf300$3925933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 30 Nov 2001 21:44:33 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Meatkit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Horst wrote: >do you know the cygwin tools? - With them you can compile (GNU) unix >programs and run them under windows. >This might help to establish where problems are. - I guess people would >be able to help you more with options and other things, when you use >these tools. JCW last wrote, 32 hours ago: >>Could someone work with Pascal and his version of the modified benchmark >>to make certain that it is being compiled comperably (I'm not a C++ >>programmer, so I'm not aware of what optimization flags , etc. one needs >>to include/avoid on various platforms)? >I'm on the case, sir :) I suppose that JCW has observed the poor Windows' performance I was talking about. From this point, I guess he'll have to determine if that's a bug or an OS limitation, if it can or not be improved. It's most certainly a tricky problem which takes time to solve, and JCW certainly also has other important things to do. He may manifest himself if I'm wrong. Otherwise, let's give him all the time he needs. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jcw@equi4.com Fri Nov 30 15:21:18 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA13056 for ; Fri, 30 Nov 2001 15:21:17 -0600 Received: from [10.0.1.55] ([195.121.233.47]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNMTLH00.AKE for ; Fri, 30 Nov 2001 22:12:53 +0100 From: Jean-Claude Wippler To: metakit Date: Fri, 30 Nov 2001 22:14:33 +0100 Message-Id: <20011130211433.20709@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Commit timing Unix/Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: My recent changes introduced a small bug. Now fixed, with thanks to Steve Baxter, who narrowed it down with a small test case. The (further reduced) code is now in the regression test suite (m02). The timing program examples/mkhash.cpp has been upgraded with more options, and now also works under Windows. For further notes on changes made, please review the change log. Everything has been checked into CVS. As Jacob Levy pointed out, cross-platform timing is hairy stuff. I've been able to finally build more or less equivalent versions of the mkhash timing program, which now shows more or less identical behavior (one key issue is that inlining must be turned on, in the library and in the app using it: -Dq4_INLINE). The results for Linux: % a.out Dhs ROW ADD FIND COMMIT SIZE 25000 26 uS 179 uS 2402 uS 687868 bytes 50000 37 uS 104 uS 4137 uS 1436068 bytes 75000 27 uS 100 uS 5575 uS 2158018 bytes 100000 22 uS 109 uS 7432 uS 2853718 bytes 125000 40 uS 109 uS 8733 uS 3523168 bytes 150000 31 uS 109 uS 10256 uS 4166368 bytes 175000 23 uS 187 uS 12276 uS 4783318 bytes 200000 30 uS 109 uS 13499 uS 5374018 bytes 225000 29 uS 112 uS 15244 uS 5956570 bytes 250000 41 uS 121 uS 16814 uS 6494770 bytes For Win NT4: C:\> mkhash Dhs ROW ADD FIND COMMIT SIZE 25000 73 uS 270 uS 8864 uS 687868 bytes 50000 50 uS 462 uS 13426 uS 1436068 bytes 75000 79 uS 399 uS 15424 uS 2158018 bytes 100000 35 uS 409 uS 19489 uS 2853718 bytes 125000 84 uS 435 uS 18298 uS 3523168 bytes 150000 65 uS 446 uS 21482 uS 4166368 bytes 175000 51 uS 307 uS 26864 uS 4783318 bytes 200000 66 uS 389 uS 45472 uS 5374018 bytes 225000 65 uS 327 uS 36261 uS 5956570 bytes 250000 93 uS 404 uS 35041 uS 6494770 bytes The Linux build uses gcc 2.95.3, Windows build is MSVC6 (which I have seen to optimize and malloc substantially better than VC5 at times). There's considerably more disk activity under Windows, indicating that either the "fflush" is causing actual sync-to-disk, or that its general sync setting are very different from Linux (I use ReiserFS, a modern journalling FS). One conclusion seems to be: your mileage *WILL* vary! -jcw From steve@improvision.com Sat Dec 1 12:24:36 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA11480; Sat, 1 Dec 2001 12:24:35 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA18333; Sat, 1 Dec 2001 18:14:36 GMT Received: from [10.0.1.4] ([213.105.144.17]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Sat, 01 Dec 2001 18:16:37 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Sat, 01 Dec 2001 18:16:35 +0000 Subject: Re: [Metakit] Commit timing Unix/Windows From: Steve Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011130211433.20709@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Excellent - thanks for the quick work JC! I'll try it first thing Monday. I'll also try and get some timing info for my G4 Mac to add into this mix. Cheers, Steve. > From: Jean-Claude Wippler > Date: Fri, 30 Nov 2001 22:14:33 +0100 > To: metakit > Subject: [Metakit] Commit timing Unix/Windows > > My recent changes introduced a small bug. Now fixed, with thanks to > Steve Baxter, who narrowed it down with a small test case. The (further > reduced) code is now in the regression test suite (m02). > > The timing program examples/mkhash.cpp has been upgraded with more > options, and now also works under Windows. > > For further notes on changes made, please review the change log. > Everything has been checked into CVS. > > As Jacob Levy pointed out, cross-platform timing is hairy stuff. I've > been able to finally build more or less equivalent versions of the mkhash > timing program, which now shows more or less identical behavior (one key > issue is that inlining must be turned on, in the library and in the app > using it: -Dq4_INLINE). > > The results for Linux: > % a.out > Dhs > ROW ADD FIND COMMIT SIZE > 25000 26 uS 179 uS 2402 uS 687868 bytes > 50000 37 uS 104 uS 4137 uS 1436068 bytes > 75000 27 uS 100 uS 5575 uS 2158018 bytes > 100000 22 uS 109 uS 7432 uS 2853718 bytes > 125000 40 uS 109 uS 8733 uS 3523168 bytes > 150000 31 uS 109 uS 10256 uS 4166368 bytes > 175000 23 uS 187 uS 12276 uS 4783318 bytes > 200000 30 uS 109 uS 13499 uS 5374018 bytes > 225000 29 uS 112 uS 15244 uS 5956570 bytes > 250000 41 uS 121 uS 16814 uS 6494770 bytes > > For Win NT4: > C:\> mkhash > Dhs > ROW ADD FIND COMMIT SIZE > 25000 73 uS 270 uS 8864 uS 687868 bytes > 50000 50 uS 462 uS 13426 uS 1436068 bytes > 75000 79 uS 399 uS 15424 uS 2158018 bytes > 100000 35 uS 409 uS 19489 uS 2853718 bytes > 125000 84 uS 435 uS 18298 uS 3523168 bytes > 150000 65 uS 446 uS 21482 uS 4166368 bytes > 175000 51 uS 307 uS 26864 uS 4783318 bytes > 200000 66 uS 389 uS 45472 uS 5374018 bytes > 225000 65 uS 327 uS 36261 uS 5956570 bytes > 250000 93 uS 404 uS 35041 uS 6494770 bytes > > The Linux build uses gcc 2.95.3, Windows build is MSVC6 (which I have > seen to optimize and malloc substantially better than VC5 at times). > > There's considerably more disk activity under Windows, indicating that > either the "fflush" is causing actual sync-to-disk, or that its general > sync setting are very different from Linux (I use ReiserFS, a modern > journalling FS). > > One conclusion seems to be: your mileage *WILL* vary! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From konrad@minus1.de Sun Dec 2 15:17:47 2001 Received: from moutvdom00.kundenserver.de (moutvdom00.kundenserver.de [195.20.224.149]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA12164 for ; Sun, 2 Dec 2001 15:17:46 -0600 Received: from [195.20.224.219] (helo=mrvdom03.schlund.de) by moutvdom00.kundenserver.de with esmtp (Exim 2.12 #2) id 16Adrq-0004C3-00 for metakit@equi4.com; Sun, 2 Dec 2001 22:09:42 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom03.schlund.de with esmtp (Exim 2.12 #2) id 16Adrq-00077h-00 for metakit@equi4.com; Sun, 2 Dec 2001 22:09:42 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16AduN-0000Xw-00 for metakit@equi4.com; Sun, 02 Dec 2001 22:12:19 +0100 Date: Sun, 2 Dec 2001 22:12:19 +0100 From: Konrad Anton To: Metakit list Message-ID: <20011202221219.A2000@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] Behaviour in disk-full condition Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all... While playing with Mk4py, trying to invent nasty tests for my application, I get the following behaviour: The following Python programme is supposed to append rows to a view until disk space runs out. Running it under Linux (Debian potato, kernel 2.2.20, Metakit 2.4.1 for Python 2.1.0), it keeps running until the filesystem is full, but then it segfaults, which is a little impolite for a Python script. #--------------------------------------------------------------- import metakit s=metakit.storage("/mnt/misc/bla.mk", 1) v=s.getas("wuff[a:S,b:S]") for i in range(10000000): v.append(a=str(i)+"bla di bla",b=str(i)+"bla blub") if (i+1)%1000 == 0: print "row",i s.commit() #--------------------------------------------------------------- As you see, it keeps creating unique rows consisting of string attributes and committing every 1000 rows. I tested it on a 300k loop mounted filesystem (both minix and ext2) and a 1M loop mounted ext2 filesystem. Additionally, top claims that "python" is 157 MB big in the "SIZE" and "RSS" categories. In all of these cases, the test programme finally segfaults. Is this a bug, or am I just supposed to not let disk space run short (which is a bit tricky in multitasking environments)? yes-I-know,-disk-space-is-cheap-ly y'rs, Konrad -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: Alice, had no notion poor man. From steve@improvision.com Mon Dec 3 05:10:39 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA27774; Mon, 3 Dec 2001 05:10:38 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id LAA23753; Mon, 3 Dec 2001 11:00:32 GMT Received: from [192.168.1.11] ([192.168.1.11]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 03 Dec 2001 11:02:31 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 03 Dec 2001 11:02:33 +0000 Subject: Re: [Metakit] Commit timing Unix/Windows From: Stephen Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011130211433.20709@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Some more results from a dual G4/500 Mac on both MacOS X and MacOS 9.2.1: MacOS X ======= Dhs ROW ADD FIND COMMIT SIZE 25000 24 uS 807 uS 3431 uS 687868 bytes 50000 35 uS 975 uS 5438 uS 1436068 bytes 75000 31 uS 768 uS 6263 uS 2158018 bytes 100000 21 uS 660 uS 8174 uS 2853718 bytes 125000 36 uS 621 uS 8938 uS 3523168 bytes 150000 30 uS 490 uS 11184 uS 4166368 bytes 175000 23 uS 633 uS 13882 uS 4783318 bytes 200000 30 uS 578 uS 14659 uS 5374018 bytes 225000 30 uS 505 uS 17378 uS 5956570 bytes 250000 43 uS 518 uS 18537 uS 6494770 bytes MacOS 9.2.1 =========== Dhs ROW ADD FIND COMMIT SIZE 25000 23 uS 569 uS 2837 uS 687868 bytes 50000 38 uS 520 uS 4481 uS 1436068 bytes 75000 27 uS 503 uS 5720 uS 2158018 bytes 100000 23 uS 496 uS 7472 uS 2853718 bytes 125000 42 uS 471 uS 9300 uS 3523168 bytes 150000 34 uS 475 uS 11427 uS 4166368 bytes 175000 30 uS 464 uS 13151 uS 4783318 bytes 200000 38 uS 466 uS 15227 uS 5374018 bytes 225000 37 uS 430 uS 17286 uS 5956570 bytes 250000 52 uS 427 uS 19304 uS 6494770 bytes These are with the latest changes from CVS. Quite interesting that the Mac seems to be faster than Windows - is this on an NTFS partition? Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Fri, 30 Nov 2001 22:14:33 +0100 > To: metakit > Subject: [Metakit] Commit timing Unix/Windows > > My recent changes introduced a small bug. Now fixed, with thanks to > Steve Baxter, who narrowed it down with a small test case. The (further > reduced) code is now in the regression test suite (m02). > > The timing program examples/mkhash.cpp has been upgraded with more > options, and now also works under Windows. > > For further notes on changes made, please review the change log. > Everything has been checked into CVS. > > As Jacob Levy pointed out, cross-platform timing is hairy stuff. I've > been able to finally build more or less equivalent versions of the mkhash > timing program, which now shows more or less identical behavior (one key > issue is that inlining must be turned on, in the library and in the app > using it: -Dq4_INLINE). > > The results for Linux: > % a.out > Dhs > ROW ADD FIND COMMIT SIZE > 25000 26 uS 179 uS 2402 uS 687868 bytes > 50000 37 uS 104 uS 4137 uS 1436068 bytes > 75000 27 uS 100 uS 5575 uS 2158018 bytes > 100000 22 uS 109 uS 7432 uS 2853718 bytes > 125000 40 uS 109 uS 8733 uS 3523168 bytes > 150000 31 uS 109 uS 10256 uS 4166368 bytes > 175000 23 uS 187 uS 12276 uS 4783318 bytes > 200000 30 uS 109 uS 13499 uS 5374018 bytes > 225000 29 uS 112 uS 15244 uS 5956570 bytes > 250000 41 uS 121 uS 16814 uS 6494770 bytes > > For Win NT4: > C:\> mkhash > Dhs > ROW ADD FIND COMMIT SIZE > 25000 73 uS 270 uS 8864 uS 687868 bytes > 50000 50 uS 462 uS 13426 uS 1436068 bytes > 75000 79 uS 399 uS 15424 uS 2158018 bytes > 100000 35 uS 409 uS 19489 uS 2853718 bytes > 125000 84 uS 435 uS 18298 uS 3523168 bytes > 150000 65 uS 446 uS 21482 uS 4166368 bytes > 175000 51 uS 307 uS 26864 uS 4783318 bytes > 200000 66 uS 389 uS 45472 uS 5374018 bytes > 225000 65 uS 327 uS 36261 uS 5956570 bytes > 250000 93 uS 404 uS 35041 uS 6494770 bytes > > The Linux build uses gcc 2.95.3, Windows build is MSVC6 (which I have > seen to optimize and malloc substantially better than VC5 at times). > > There's considerably more disk activity under Windows, indicating that > either the "fflush" is causing actual sync-to-disk, or that its general > sync setting are very different from Linux (I use ReiserFS, a modern > journalling FS). > > One conclusion seems to be: your mileage *WILL* vary! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From karl@magicnotes.com Mon Dec 3 05:40:56 2001 Received: from mta05ps.bigpond.com (mta05ps.bigpond.com [144.135.25.137]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA29220 for ; Mon, 3 Dec 2001 05:40:55 -0600 Received: from SuperBoy ([144.135.25.84]) by mta05ps.bigpond.com (Netscape Messaging Server 4.15) with SMTP id GNRN1X00.6U3 for ; Mon, 3 Dec 2001 21:39:33 +1000 Received: from CPE-203-45-10-217.vic.bigpond.net.au ([203.45.10.217]) by psmam06.mailsvc.email.bigpond.com(MailRouter V2.9k 8419/28092434); 03 Dec 2001 21:32:47 Date: Mon, 3 Dec 2001 22:32:33 +1100 From: Karl Edwall X-Mailer: The Bat! (v1.53t) Personal Reply-To: Karl Edwall Organization: Eskil Software X-Priority: 3 (Normal) Message-ID: <1107103003.20011203223233@magicnotes.com> To: metakit Subject: Re[2]: [Metakit] Commit timing Unix/Windows In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sorry if I'm stating the obvious but has anybody run these using some profiling tools like Numega TrueTime or or similar? If not I'd be happy to do so. I've always found that gives you a better idea of obvious bottlenecks. Regards, Karl Edwall Stephen Baxter wrote; >> Some more results from a dual G4/500 Mac on both MacOS X and MacOS 9.2.1: >> >> MacOS X >> ======= >> >>Dhs >> ROW ADD FIND COMMIT SIZE >> 25000 24 uS 807 uS 3431 uS 687868 bytes From jcw@equi4.com Mon Dec 3 05:55:58 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA30018 for ; Mon, 3 Dec 2001 05:55:58 -0600 Received: from [10.0.1.55] ([195.121.233.114]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNRNEZ02.40R for ; Mon, 3 Dec 2001 12:47:23 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Commit timing Unix/Windows Date: Mon, 3 Dec 2001 12:48:17 +0100 Message-Id: <20011203114817.13241@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen, >Some more results from a dual G4/500 Mac on both MacOS X and MacOS 9.2.1: [...] >These are with the latest changes from CVS. Quite interesting that the Mac >seems to be faster than Windows - is this on an NTFS partition? Yes. Eh, no. Eh, hm. I've been running off a Samba network drive and off the C: drive (NTFS), not sure which ones my last results were from. I think C:\. One explanation for the performance diff of PowerPC G4 w.r.t. Pentium PIII could be that PowerPC is more "super-scalar/pipelined" (and RISC vs. CISC). It's been plaguing Apple for ages now that a "measly 300 MHz" machine can be as fast as a Wintel machine at much higher clock rates. Imagine buying a machine, same price, same features, but one 300 and the other 650 MHz - which do you pick? I'm not so surprised that tight loops and well-optimized code end up making a RISC machine go in overdrive. That's exactly what their design is about. Hence my last remark: >> One conclusion seems to be: your mileage *WILL* vary! With G4's "AltiVec" vector instructions, one might push MK yet further, but that would require custom coding (and a deep understanding of that side of things). FWIW, one goal for the next G5's appears to be to go way up in MHz, just to get over this perception (see http://www.mosr.com/ for rumors about such things). I'd like to add that this wasn't about comparing the various CPU's or OS'es (not for me, anyway). Though it's always interesting to compare figures. (and pushing me another nudge to using Mac OS X + G4 for development) -jcw From jcw@equi4.com Mon Dec 3 06:06:25 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA30607 for ; Mon, 3 Dec 2001 06:06:25 -0600 Received: from [10.0.1.55] ([195.121.233.253]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNRNWF01.BHS for ; Mon, 3 Dec 2001 12:57:51 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: Re[2]: [Metakit] Commit timing Unix/Windows Date: Mon, 3 Dec 2001 12:59:01 +0100 Message-Id: <20011203115901.31868@mail.planet.nl> In-Reply-To: <1107103003.20011203223233@magicnotes.com> References: <1107103003.20011203223233@magicnotes.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Karl Edwall wrote: >Sorry if I'm stating the obvious but has anybody run these using some >profiling tools like Numega TrueTime or or similar? If not I'd be happy to do >so. I've always found that gives you a better idea of obvious >bottlenecks. Yes, I did (a while back using MSVC's profiling), last week or so using gcc/gprof on Linux. Helped me tweak a little, but it was only a cursory check. There are still several things happening at commit time which could be done differently - but for now this ought to be sufficient. -jcw From jcw@equi4.com Mon Dec 3 06:21:12 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA31411 for ; Mon, 3 Dec 2001 06:21:12 -0600 Received: from [10.0.1.55] ([195.121.235.154]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNROL100.2HB for ; Mon, 3 Dec 2001 13:12:37 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Behaviour in disk-full condition Date: Mon, 3 Dec 2001 13:13:08 +0100 Message-Id: <20011203121308.3990@mail.planet.nl> In-Reply-To: <20011202221219.A2000@minus1.de> References: <20011202221219.A2000@minus1.de> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Konrad Anton wrote: [...] >import metakit > >s=metakit.storage("/mnt/misc/bla.mk", 1) >v=s.getas("wuff[a:S,b:S]") > >for i in range(10000000): > v.append(a=str(i)+"bla di bla",b=str(i)+"bla blub") > if (i+1)%1000 == 0: > print "row",i > s.commit() >#--------------------------------------------------------------- > >As you see, it keeps creating unique rows consisting of string >attributes and committing every 1000 rows. Good you mention said it, email indentation messed it up. >I tested it on a 300k loop mounted filesystem (both minix and ext2) >and a 1M loop mounted ext2 filesystem. Additionally, top claims that >"python" is 157 MB big in the "SIZE" and "RSS" categories. Yes, you are likely to get huge memory use without "blocking views". It's them "column-wise storage" again. Replace this: v=s.getas("wuff[a:S,b:S]") By this: v=s.getas("_B[wuff[a:S,b:S]]").blocked() Should scale far better, in mem-use and in commit speed. >In all of these cases, the test programme finally segfaults. This could be one of the three problems reported by Mitch Chapman (20- 11), which were fixed on 26-11 (in CVS). The commit fails, but passing back the error to Python was hosed. >Is this a bug, or am I just supposed to not let disk space run short >(which is a bit tricky in multitasking environments)? It is (was, I hope) a bug. MK should detect failure to commit properly, regardless what the reason was, and report this. With so many changes and fixes in the recent past, I'm planning to have a new 2.4.2 release - within at most a week or two. Those who have CVS access, and an interest in getting everything top notch, please *do* continue to update and give the code a very heavy workout (last CVS check-in was a few minutes ago, a silly oversight in my last changes). -jcw From steve@improvision.com Mon Dec 3 06:24:54 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA31647; Mon, 3 Dec 2001 06:24:53 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id MAA06913; Mon, 3 Dec 2001 12:14:49 GMT Received: from [192.168.1.11] ([192.168.1.11]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 03 Dec 2001 12:16:45 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 03 Dec 2001 12:16:48 +0000 Subject: Re: Re[2]: [Metakit] Commit timing Unix/Windows From: Stephen Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011203115901.31868@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I will do this on the Mac now everything is working again. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Mon, 3 Dec 2001 12:59:01 +0100 > To: metakit > Subject: Re: Re[2]: [Metakit] Commit timing Unix/Windows > > Karl Edwall wrote: > >> Sorry if I'm stating the obvious but has anybody run these using some >> profiling tools like Numega TrueTime or or similar? If not I'd be happy to do >> so. I've always found that gives you a better idea of obvious >> bottlenecks. > > Yes, I did (a while back using MSVC's profiling), last week or so using > gcc/gprof on Linux. Helped me tweak a little, but it was only a cursory > check. There are still several things happening at commit time which > could be done differently - but for now this ought to be sufficient. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From steve@improvision.com Mon Dec 3 06:33:42 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA32164; Mon, 3 Dec 2001 06:33:41 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id MAA11891; Mon, 3 Dec 2001 12:23:36 GMT Received: from [192.168.1.11] ([192.168.1.11]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 03 Dec 2001 12:25:35 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 03 Dec 2001 12:25:38 +0000 Subject: Re: [Metakit] Commit timing Unix/Windows From: Stephen Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011203114817.13241@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Stephen, > >> Some more results from a dual G4/500 Mac on both MacOS X and MacOS 9.2.1: > [...] >> These are with the latest changes from CVS. Quite interesting that the Mac >> seems to be faster than Windows - is this on an NTFS partition? > > Yes. Eh, no. Eh, hm. I've been running off a Samba network drive and > off the C: drive (NTFS), not sure which ones my last results were from. > I think C:\. > > One explanation for the performance diff of PowerPC G4 w.r.t. Pentium > PIII could be that PowerPC is more "super-scalar/pipelined" (and RISC vs. > CISC). It's been plaguing Apple for ages now that a "measly 300 MHz" > machine can be as fast as a Wintel machine at much higher clock rates. > Imagine buying a machine, same price, same features, but one 300 and the > other 650 MHz - which do you pick? > > I'm not so surprised that tight loops and well-optimized code end up > making a RISC machine go in overdrive. That's exactly what their design > is about. > > Hence my last remark: >>> One conclusion seems to be: your mileage *WILL* vary! Yes indeed - it's useful to know what to expect on different platforms though - it at least tells me everything is being compiled and inlined in the right way. Lots of other things come into play as well. We have found that non-inlined calls can be very slow on the Mac (much worse than Intel platforms), maybe because of all the registers that need to be saved in each stack frame. Inlining a tight-loop method can increase performance by as much as 10x quite instantly. Also the Mac filesystem does not seem to be as fast as Windows in general. > With G4's "AltiVec" vector instructions, one might push MK yet further, > but that would require custom coding (and a deep understanding of that > side of things). Altivec is great for crunching through truly vast amounts of data (we use it for accelerated 3D volume rendering). Writing good Altivec code is *hard* though (every time I try the Altivec code ends up slower than scalar code - fortunately I have one guy in my team who is gifted at this though). > FWIW, one goal for the next G5's appears to be to go way up in MHz, just > to get over this perception (see http://www.mosr.com/ for rumors about > such things). If you believe MacOS Rumors and The Register, it looks like the 1GHz G5 will outperform a 2GHz P4 quite easily. God knows how fast the 2.8GHz G5 will be (the fastest G5s are testing at this speed though it is going to be quite a while before we see them anywhere). I'll let you know as soon as they come out and I persuade my boss to buy me one :). > I'd like to add that this wasn't about comparing the various CPU's or > OS'es (not for me, anyway). Though it's always interesting to compare > figures. > > (and pushing me another nudge to using Mac OS X + G4 for development) Personally I feel that MacOS X is not yet ready for primetime, at least when compared to 9. If you're coming from Linux or Windows your view might be different! It *is* making progress and the rough edges are gradually being rounded off though... Cheers, Steve. From konrad@minus1.de Mon Dec 3 11:02:00 2001 Received: from mout02.kundenserver.de (mout02.kundenserver.de [195.20.224.133]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA16536 for ; Mon, 3 Dec 2001 11:01:59 -0600 Received: from [195.20.224.220] (helo=mrvdom04.kundenserver.de) by mout02.kundenserver.de with esmtp (Exim 2.12 #2) id 16AwLq-0007j1-00 for metakit@equi4.com; Mon, 3 Dec 2001 17:53:54 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom04.kundenserver.de with esmtp (Exim 2.12 #2) id 16AwLp-0000Id-00 for metakit@equi4.com; Mon, 3 Dec 2001 17:53:53 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16AwNa-0000Dj-00 for metakit@equi4.com; Mon, 03 Dec 2001 17:55:42 +0100 Date: Mon, 3 Dec 2001 17:55:42 +0100 From: Konrad Anton To: Metakit list Message-ID: <20011203175542.A790@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] Segfault when committing a read-only storage Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all. The following Python program segfaults (Python 2.1.0, Metakit release 2.4.1, Debian GNU/Linux potato). What it does is * make sure there is a nonempty storage * open it with opening mode 0 (readonly) * write and commit to it Here is the source. #------------------------------------------------------------------ import metakit print "creating storage bla.mk" s = metakit.storage("bla.mk", 1) v = s.getas("wuff[a:S,b:S]") v.append(a="Eins",b="Zwei") s.commit() del s print "open readonly, then write" s = metakit.storage("bla.mk", 0) v = s.view("wuff") print "append..." v.append(a="drei",b="vier") print "commit..." s.commit() print "success" #------------------------------------------------------------------- The segfault occurs after "commit..." is printed. While I understand that I have no right to commit changes to a read-only storage, I wish it just told me so with a catchable exception. I haven't verified if this problem is among those fixed in the latest CVS version (is there a public CVS somewhere? I haven't seen any announced anywhere), but I report it anyway, since the bug report yesterday was so much fun ;) Have a nice whatever-time-of-day, Konrad. -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The end dance? From jcw@equi4.com Mon Dec 3 11:11:52 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA17249 for ; Mon, 3 Dec 2001 11:11:51 -0600 Received: from [10.0.1.55] ([195.121.236.174]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNS21H02.BGC for ; Mon, 3 Dec 2001 18:03:17 +0100 From: Jean-Claude Wippler To: Metakit list Subject: Re: [Metakit] Segfault when committing a read-only storage Date: Mon, 3 Dec 2001 18:05:05 +0100 Message-Id: <20011203170505.15774@mail.planet.nl> In-Reply-To: <20011203175542.A790@minus1.de> References: <20011203175542.A790@minus1.de> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Konrad Anton wrote: >s = metakit.storage("bla.mk", 0) >v = s.view("wuff") >print "append..." >v.append(a="drei",b="vier") >print "commit..." >s.commit() [...] >The segfault occurs after "commit..." is printed. While I understand >that I have no right to commit changes to a read-only storage, I wish >it just told me so with a catchable exception. > >I haven't verified if this problem is among those fixed in the latest >CVS version (is there a public CVS somewhere? I haven't seen any >announced anywhere), but I report it anyway, since the bug report >yesterday was so much fun ;) Well, I don't want to spoil the fun for you - but yes, this is exactly what Mitch Chapman reported on this list... The MK CVS repository is on www.equi4.com, its access is documented on the MK home page: http://www.equi4.com/metakit/ - please let me know if that isn't clear or accurate. There's a snapshot of MK in the previews/ area, which very likely solves the above, but I'm not guaranteeing that it gets updated each time. If you want bleeding edge, then you need to set up CVS. And as always, the change log is at: http://www.equi4.com/metakit/CHANGES/ I try to keep it up to date with the latest in CVS - please make sure you check the top entries before submitting a problem which may well have been resolved. -jcw From jcw@equi4.com Mon Dec 3 11:15:36 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA17494 for ; Mon, 3 Dec 2001 11:15:35 -0600 Received: from [10.0.1.55] ([195.121.236.174]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNS27K02.7FG for ; Mon, 3 Dec 2001 18:06:56 +0100 From: Jean-Claude Wippler To: Metakit list Date: Mon, 3 Dec 2001 18:08:47 +0100 Message-Id: <20011203170847.22260@mail.planet.nl> In-Reply-To: <20011203170505.15774@mail.planet.nl> References: <20011203170505.15774@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Correction Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >And as always, the change log is at: > http://www.equi4.com/metakit/CHANGES/ Whoops, it is and always has been at: http://www.equi4.com/metakit/CHANGES -jcw From konrad@minus1.de Mon Dec 3 11:56:06 2001 Received: from mout02.kundenserver.de (mout02.kundenserver.de [195.20.224.133]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA19572 for ; Mon, 3 Dec 2001 11:56:05 -0600 Received: from [195.20.224.208] (helo=mrvdom01.schlund.de) by mout02.kundenserver.de with esmtp (Exim 2.12 #2) id 16AxCE-0000aL-00 for metakit@equi4.com; Mon, 3 Dec 2001 18:48:02 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom01.schlund.de with esmtp (Exim 2.12 #2) id 16AwnF-0001DO-00 for metakit@equi4.com; Mon, 3 Dec 2001 18:22:13 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16Awpl-0001KF-00 for metakit@equi4.com; Mon, 03 Dec 2001 18:24:49 +0100 Date: Mon, 3 Dec 2001 18:24:49 +0100 From: Konrad Anton To: Metakit list Message-ID: <20011203182449.A976@minus1.de> References: <20011203175542.A790@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20011203175542.A790@minus1.de>; from konrad@minus1.de on Mon, Dec 03, 2001 at 05:55:42PM +0100 Subject: [Metakit] problem solved in CVS (was: Segfault when committing a read-only storage) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, Dec 03, 2001 at 05:55:42PM +0100, Konrad Anton wrote: > The following Python program segfaults (Python 2.1.0, Metakit release > 2.4.1, Debian GNU/Linux potato). > (...) > I haven't verified if this problem is among those fixed in the latest > CVS version (is there a public CVS somewhere? I haven't seen any > announced anywhere), but I report it anyway, since the bug report > yesterday was so much fun ;) Haha. Well, the CVS wasn't that *hard* to find once I took off the sunglasses -- sorry. Results with the CVS version of Metakit of just a couple of minutes ago show that both former segfault conditions now raise proper IOError exceptions. So the time-to-bugfix-in-CVS-version is actually negative. That's just great! Sorry for the wasted time, have a nice remaining-of-day, Konrad. -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The end dance? From Christophe.Muller@research.gemplus.com Wed Dec 5 11:43:34 2001 Received: from difool.gemplus.com (difool.gemplus.com [195.25.133.180]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA02816 for ; Wed, 5 Dec 2001 11:43:33 -0600 Received: from research.gemplus.com (unknown [172.28.50.32]) by difool.gemplus.com (Postfix) with ESMTP id 8DF0DC1FF; Wed, 5 Dec 2001 18:33:43 +0100 (MET) Received: from aphrodite (aphrodite [172.28.50.32]) by research.gemplus.com (8.8.8+Sun/8.8.8) with SMTP id SAA22303; Wed, 5 Dec 2001 18:30:17 +0100 (MET) Message-ID: <3C0E59A7.7324@research.gemplus.com> Date: Wed, 05 Dec 2001 18:30:15 +0100 From: Christophe Muller Organization: GEMPLUS X-Mailer: Mozilla 3.0Gold (X11; I; SunOS 5.5.1 sun4u) MIME-Version: 1.0 To: metakit@equi4.com Cc: lvirden@cas.org Content-Type: multipart/mixed; boundary="------------42852FDF6E5D" Subject: [Metakit] WiKit DB corruption and a "wikit-dump" script Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------42852FDF6E5D Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hello, [Note: this is a message about WiKit, more than about MetaKit, although I'm not sure the corruption I got comes from the former or the latter..?? If there is a more appropriate place to talk about Wikit, please let me know and sorry about the disturbance.] [Note to JC: I was the 'CM' on you wiki.. :-)] Larry W. Virden wrote (in September): > I have a wikit that I would like to learn to manipulate. > 1. How can I determine the titles of each of the pages in the > wikit? > 2. How can I extract the latest versions of all the pages in the > wikit? > 3. How can I take these extracted pages and insert them into a > new, empty, wikit (say one that has been built with updated code, > etc.) > 4. How can I access the history of changes to any specific page > on the wikit? Like Larry, I'm trying to set-up a wikit internally. I have just grabbed a copy of WiKit at: http://www.equi4.com/pub/tk/examples/wikit.bin I first started to compile everything by myself (with the latest sources taken from /pub/tars/) and ran it under Apache but I got a very badly corrupted database yesterday evening and spent hours to get back data from an hexa dump sdx could gave me... I couldn't figure out how it happened. First the dates got corrupted (i.e., dozens of fancy entries in [Changes], from 1930 to 2067, you get the idea), then after a while all the page titles went wrong and I could see a mixed of text in each page.. I don't really know how to reproduce that. It seems that it happened because I was using at the same time the Tk GUI (./wikit) and the CGI way... I seem to recall to have read somewhere that they cannot mix yet.. If there is a more stable version of Wikit/Tclkit than the one I'm using, I'd be glad to know.. (I'd rather stick to the stable version when editing content). Anyway, today (is a new day) and I got a fresh compiled copy of tclkit from: http://www.equi4.com/pub/tk/tclkit-8.4-26-linux/ have put everything back in place, have used the local mode *only at the beginning* and then I have used only the CGI mode. Also, like Larry, I was so concerned about saving the content I typed today that I have looked inside the tcl scripts (by unpacking them with sdx) and started developing a small script of my own that basically (I think) addresses points 1 and 2 of his request. However, I still have no clue on how to use that to re-create a wikit from scratch in reverse. Any help would be appreciated. Also any doc on wikit design, how to modify and repack custom versions, how to easily extract and reload the database data, etc. would be welcomed. It would be nice if we could see a file in the .vfs tree that contains the data (e.g., pages.db), and if we could pretty-print and parse it back from ASCII form.. I have the idea of saving RCS/CVS modifs at each "Save".. This would definitely reassure me :-). I have attached my script draft. At least with it, my content is not lost in case of corruption. Any help on how to "build" a wikit database programmatically would be appreciated. Also it seems to me that when two persons are editing the same page, the last "Save" will erase the first one: Am I right? Isn't it possible to use MetaKit commit features? Thanks for your help. If I improve the script, I'll post the results. Cheers, Christophe. = Most of the VAX instructions are in microcode, = = but HALT and NO-OP are in hardware for efficiency. = -- ------------------------------------------------------------- Christophe.Muller@research.gemplus.com - Gemplus Research Lab Phone: +33 4-42-36-57-83 | Disclaimer: I don't speak for Gemplus Gemplus doesn't speak for me... it is better that way! ------------------------------------------------------------- --------------42852FDF6E5D Content-Type: text/plain; charset=us-ascii; name="wikit-dump" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="wikit-dump" #!/usr/local/bin/tclkit # Dumps a wikit that has been filled with pages in "source" form # (i.e., with the wikit page markup) # # 2001-12-05, cm set header {FILE AUTOMATICALLY SAVED BY wikit-dump. PLEASE DO NOT EDIT} if {[llength $argv] != 2} { puts stderr "Usage: $argv0 infile outfile" exit 1 } mk::file open db [lindex $argv 0] -readonly set all [mk::select db.pages] puts "The following pages will be saved: $all" puts -nonewline "Saving..." set out [open [lindex $argv 1] w] fconfigure $out -translation binary -encoding binary puts $out $header foreach i $all { puts -nonewline "." puts $out "__________________________________________________" puts $out "Page: $i in wikit source form." puts $out "Name: [mk::get db.pages!$i name]" puts $out "Date: [mk::get db.pages!$i date]" puts $out "" puts $out [mk::get db.pages!$i page] } close $out puts "Done." exit --------------42852FDF6E5D-- From lvirden@cas.org Wed Dec 5 12:24:21 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA04386 for ; Wed, 5 Dec 2001 12:24:21 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id fB5IFa214809 for ; Wed, 5 Dec 2001 13:15:36 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fB5IFY625463; Wed, 5 Dec 2001 13:15:34 -0500 (EST) Date: Wed, 5 Dec 2001 13:15:34 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011205131534.AAB25421@cas.org> Subject: Re: [Metakit] WiKit DB corruption and a "wikit-dump" script In-Reply-To: Your message of Wed, 05 Dec 2001 18:30:15 +0100 To: metakit@equi4.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: In the past, this has been 'the place' to discuss wikits. I don't know what JCW's thoughts are on this topic. I do know, however, that JCW's been hard at work at the wikit at , fixing problems that sound amazinging like what you describe, during the past few weeks. Hopefully he will have more to add when he gets to his mail. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Thu Dec 6 06:57:43 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA04188 for ; Thu, 6 Dec 2001 06:57:42 -0600 Received: from [10.0.1.55] ([195.121.232.191]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNXA9T01.39L for ; Thu, 6 Dec 2001 13:49:05 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] WiKit DB corruption and a "wikit-dump" script Date: Thu, 6 Dec 2001 13:50:18 +0100 Message-Id: <20011206125018.32530@mail.planet.nl> In-Reply-To: <3C0E59A7.7324@research.gemplus.com> References: <3C0E59A7.7324@research.gemplus.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Christophe Muller wrote: >[Note: this is a message about WiKit, more than about MetaKit, >although I'm not sure the corruption I got comes from the former >or the latter..?? If there is a more appropriate place to talk >about Wikit, please let me know and sorry about the disturbance.] No, there is no better place right now - though this topic is likely to interest only a few people on this list. I'll answer here, perhaps we could continue the discussion on a wiki page on the Tcl'ers Wiki? >Like Larry, I'm trying to set-up a wikit internally. > >I have just grabbed a copy of WiKit at: >http://www.equi4.com/pub/tk/examples/wikit.bin > >I first started to compile everything by myself (with the latest >sources taken from /pub/tars/) and ran it under Apache but I got >a very badly corrupted database yesterday evening and spent hours >to get back data from an hexa dump sdx could gave me... I couldn't >figure out how it happened. First the dates got corrupted (i.e., >dozens of fancy entries in [Changes], from 1930 to 2067, you get >the idea), then after a while all the page titles went wrong and >I could see a mixed of text in each page.. > >I don't really know how to reproduce that. It seems that it >happened because I was using at the same time the Tk GUI (./wikit) >and the CGI way... I seem to recall to have read somewhere that >they cannot mix yet.. Your problem appears to be caused by two processes accessing the MK datafile at the same time (and modifying it). Column-wise storage means that damage, however small, may end up looking very ugly and bad all over the datafile. So although the effect is indeed "corruption", this really is caused by multi-user access to a design which does not deal with it. The wikit does do its own manual locking, so I'm a bit puzzled how this could have happened. When you do "tclkit wikit.tkd dump", do you see a "lock.tcl" file listed? if so, then locking should really have been active at every launch (you weren't doing this as root, I assume?). >Anyway, today (is a new day) and I got a fresh compiled copy of >tclkit from: http://www.equi4.com/pub/tk/tclkit-8.4-26-linux/ >have put everything back in place, have used the local mode >*only at the beginning* and then I have used only the CGI mode. That sounds like a good idea. >Also, like Larry, I was so concerned about saving the content >I typed today that I have looked inside the tcl scripts (by >unpacking them with sdx) and started developing a small script [...] The wikit scripts are in an intermediate state. Though they use VFS and you can indeed use SDX to get/set them, keep in mind that the wiki pages are still stored in "raw" MetaKit format, i.e. as a (non-VFS) view. >I have attached my script draft. Did you look at the (quite recent) "wikitool" mentioned over here? http://www.equi4.com/wikit/ >Also it seems to me that when two persons are editing the >same page, the last "Save" will erase the first one: Am I >right? Isn't it possible to use MetaKit commit features? Yes. That is not so easy to change: the main use is over the web, and HTTP is stateless... it is not obvious whether someone is "currently editing" a page, especially not because all requests are handled by separate CGI launches. Your script looks ok to me, btw. You're doing the essentials for looping and fetching data. It ought to be possible to do the reverse as well, but there are issues such as cerating new pages (and page id's) which are not trivial. Then again, if it's merely for restoring existing content, then it should be feasible. FWIW, I'm working on and off on a very much simpler wiki, and taking advantage of the latest VFS to get rid of all the issues you are running into. But it's hard to say when it will be ready / or at least useful enough for others to work with. -jcw From jcw@equi4.com Wed Dec 12 09:32:03 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA07231; Wed, 12 Dec 2001 09:32:02 -0600 Received: from [10.0.1.55] ([195.121.236.29]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GO8LE401.QUJ; Wed, 12 Dec 2001 16:22:52 +0100 From: Jean-Claude Wippler To: announce , metakit Date: Wed, 12 Dec 2001 16:19:47 +0100 Message-Id: <20011212151947.15191@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Exactly two years (to this very day) after MetaKit 2.0 was released as open source software, I am proud to announce a new MetaKit 2.4.2 release. This release passes all my tests, and contains several changes since 2.4.1: - the hashing view has been improved and now works well with blocking - commit speed improved with many subviews containing strings - the tcl/kit/ directory has been removed (TclKit is being split off) - optional builds of Mk4py and Mk4tcl have been streamlined - more sample/timing code added to "examples/" in the src distribution - various bug fixes, warning msgs cleaned up, and minor tweaks The detailed change log is in the source distribution and on the web: http://www.equi4.com/metakit/CHANGES Downloads for the source, as well as builds for Windows, Linux, MacOS 9+X, FreeBSD, and more (Solaris coming soon) are all at: http://www.equi4.com/pub/mk/ MetaKit's home page is, as before, at http://www.equi4.com/metakit/ I've started to take some notes w.r.t. documentation on the MetaKit wiki: http://www.equi4.com/metakit/wiki.cgi Feel free to add/amend, or even add your own pages and examples. If you create a build for another platform and it sails cleanly through "make test", then I'd be most happy (eager!) to include it in the download area. If the build is not successful, please post info to metakit@equi4.com so readers can try to help. Thanks to everyone for reporting bugs, providing help on the mailing list, and more generally for helping clarify trouble spots and broaden the use of MetaKit. May it serve you well, also in 2002! Jean-Claude From mmesser@masari.com Wed Dec 12 13:20:24 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA19330 for ; Wed, 12 Dec 2001 13:20:14 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000017; 12 Dec 01 09:57:52 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 12 Dec 01 09:57:15 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Wed, 12 Dec 2001 09:56:56 -0800 MIME-Version: 1.0 Message-ID: <3C1729E8.25321.75FCE80@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] Mk4py: problem with view.hash() ?? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: # This code crashes the interpreter. I tried 2.4.1 ad 2.4.2. # Am I doing something wrong here? import metakit db = metakit.storage("datafile.mk",1) vw = db.getas("people[first:S,last:S,shoesize:I]") vw.append(first='John',last='Lennon',shoesize=44) vw.append(first='Flash',last='Gordon',shoesize=42) db.commit() for r in vw: print r.first, r.last, r.shoesize tbl = {} newvw = vw.hash(tbl, numkeys=1) #for item in tbl.items(): print item #for r in newvw: print r.first, r.last, r.shoesize From jcw@equi4.com Wed Dec 12 13:59:16 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA21382 for ; Wed, 12 Dec 2001 13:59:15 -0600 Received: from [10.0.1.55] ([195.121.234.63]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GO8XRH00.YJP for ; Wed, 12 Dec 2001 20:50:05 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Mk4py: problem with view.hash() ?? Date: Wed, 12 Dec 2001 20:40:58 +0100 Message-Id: <20011212194058.5245@mail.planet.nl> In-Reply-To: <3C1729E8.25321.75FCE80@localhost> References: <3C1729E8.25321.75FCE80@localhost> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Messer wrote: >This code crashes the interpreter. Am I doing something wrong? [...] >tbl = {} >newvw = vw.hash(tbl, numkeys=1) Well... tbl needs to me a MK view, not a Python dict. For example: tbl = metakit.view() Or look at the *.py files in the examples/ dir (in the MK source dist). Hm, MK shouldn't crash... that is definitely a weakness in MK. -jcw From mmesser@masari.com Wed Dec 12 23:16:40 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA10145 for ; Wed, 12 Dec 2001 23:16:40 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000020; 12 Dec 01 19:54:04 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 12 Dec 01 19:53:49 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Wed, 12 Dec 2001 19:53:34 -0800 MIME-Version: 1.0 Message-ID: <3C17B5BE.11018.9820919@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] mk4py: outer joins Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This code gives the following error: Traceback (most recent call last): File "test.py", line 20, in ? for r in vw.join(vw2, vw.shoesize, outer=1): print r.first, r.last, r.shoesize, r.desc SystemError: error return without exception set It works fine when I remove outer=1 from the join. It still gives and error when outer=0 is specified. __________________________________________________________________ import metakit import sys db = metakit.storage() #Create a view (this is the MetaKit term for "table"): vw = db.getas("people[first:S,last:S,shoesize:I]") vw2 = db.getas("sizes[shoesize:I,desc:S]") #Add two rows (this is the MetaKit term for "record"): vw.append(first='John',last='Lennon',shoesize=44) vw.append(first='Flash',last='Gordon',shoesize=45) vw.append(first='John',last='Gordon',shoesize=46) vw.append(first='Flash',last='Lennon',shoesize=47) vw2.append(shoesize=44, desc='Small') vw2.append(shoesize=45, desc='Medium') for r in vw.join(vw2, vw.shoesize, outer=1): print r.first, r.last, r.shoesize, r.desc print From mmesser@masari.com Wed Dec 12 23:21:28 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA10272 for ; Wed, 12 Dec 2001 23:21:28 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000022; 12 Dec 01 19:58:52 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 12 Dec 01 19:58:46 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Wed, 12 Dec 2001 19:58:25 -0800 MIME-Version: 1.0 Message-ID: <3C17B6E1.10816.9867AAB@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] mk4py: infinite loop when passing dictionary criteria to view.find() Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The invocation of the "bat" function results in an infinite loop. Isn't it legal to pass criteria in dictionary form to view.find() ? _______________________________________________________________-- import metakit db = metakit.storage() #Create a view (this is the MetaKit term for "table"): vw = db.getas("people[first:S,last:S,shoesize:I]") vw.append(first='John',last='Lennon',shoesize=44) vw.append(first='Flash',last='Gordon',shoesize=42) def foo(v, criteria): for r in v.select(criteria): print r.first, r.last, r.shoesize def bar(v): i = v.find(last='Lennon', start=0) while(i >= 0): print v[i].first, v[i].last, v[i].shoesize i = v.find(last='Lennon', start=i+1) def bat(v, criteria): i = v.find(criteria, start=0) while(i >= 0): print v[i].first, v[i].last, v[i].shoesize i = v.find(criteria, start=i+1) criteria = {'last':'Lennon'} foo(vw, criteria) print bar(vw) print bat(vw, criteria) print From j.p.fletcher@aston.ac.uk Thu Dec 13 07:25:19 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA25357 for ; Thu, 13 Dec 2001 07:25:18 -0600 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 16EVjQ-0005Lk-00 for metakit@equi4.com; Thu, 13 Dec 2001 13:17:00 +0000 From: "John Fletcher" To: metakit Date: Thu, 13 Dec 2001 13:14:31 -0000 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] MetaKit 2.4.2 Priority: normal In-reply-to: <20011212151947.15191@mail.planet.nl> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler To: announce , metakit Subject: [Metakit] MetaKit 2.4.2 Date sent: Wed, 12 Dec 2001 16:19:47 +0100 > Exactly two years (to this very day) after MetaKit 2.0 was released as > open source software, I am proud to announce a new MetaKit 2.4.2 release. > Thanks for this and for all the work. Are there any implications for the public interface? In particular, can I substitute this release in builds for E4Graph? Thanks John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From gmcm@hypernet.com Thu Dec 13 07:39:34 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA25998 for ; Thu, 13 Dec 2001 07:39:33 -0600 Received: from PARANOIA (204.176.40.64) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 13 Dec 2001 08:24:31 -0500 From: "Gordon McMillan" To: "Mark Messer" , metakit@equi4.com Date: Thu, 13 Dec 2001 08:30:36 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] mk4py: outer joins Reply-to: gmcm@hypernet.com Message-ID: <3C18672C.8079.5E37B26@localhost> Priority: normal In-reply-to: <3C17B5BE.11018.9820919@localhost> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 12 Dec 2001 at 19:53, Mark Messer wrote: > > This code gives the following error: > > Traceback (most recent call last): > File "test.py", line 20, in ? > for r in vw.join(vw2, vw.shoesize, outer=1): print r.first, > r.last, r.shoesize, r.desc > SystemError: error return without exception set join is not a keyword-accepting method. vw.jon(vw2, vw.shoesize, 1) will give you an outer join, while vw.join(vw2, vw.shoesize, 0) is equivalent to vw.join(vw2, vw.shoesize) -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Thu Dec 13 07:39:35 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA26001 for ; Thu, 13 Dec 2001 07:39:34 -0600 Received: from PARANOIA (204.176.40.64) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 13 Dec 2001 08:24:33 -0500 From: "Gordon McMillan" To: "Mark Messer" , metakit@equi4.com Date: Thu, 13 Dec 2001 08:30:36 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] mk4py: infinite loop when passing dictionary criteria to view.find() Reply-to: gmcm@hypernet.com Message-ID: <3C18672C.25988.5E37B80@localhost> Priority: normal In-reply-to: <3C17B6E1.10816.9867AAB@localhost> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 12 Dec 2001 at 19:58, Mark Messer wrote: > The invocation of the "bat" function results in an infinite loop. > > Isn't it legal to pass criteria in dictionary form to view.find() ? Yes, but it's not legal to use a criteria dictionary AND a keyword arg at the same time. Instead of v.find(criteria, start=i+1) you could: criteria['start'] = i+1 v.find(criteria) > _______________________________________________________________-- > > > import metakit > > db = metakit.storage() > > #Create a view (this is the MetaKit term for "table"): > vw = db.getas("people[first:S,last:S,shoesize:I]") > vw.append(first='John',last='Lennon',shoesize=44) > vw.append(first='Flash',last='Gordon',shoesize=42) > > def foo(v, criteria): > for r in v.select(criteria): print r.first, r.last, r.shoesize > > def bar(v): > i = v.find(last='Lennon', start=0) > while(i >= 0): > print v[i].first, v[i].last, v[i].shoesize > i = v.find(last='Lennon', start=i+1) > > def bat(v, criteria): > i = v.find(criteria, start=0) > while(i >= 0): > print v[i].first, v[i].last, v[i].shoesize > i = v.find(criteria, start=i+1) > > criteria = {'last':'Lennon'} > foo(vw, criteria) > print > bar(vw) > print > bat(vw, criteria) > print > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Thu Dec 13 07:52:58 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA26644 for ; Thu, 13 Dec 2001 07:52:58 -0600 Received: from PARANOIA (204.176.40.64) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 13 Dec 2001 08:38:24 -0500 From: "Gordon McMillan" To: "Mark Messer" , metakit@equi4.com Date: Thu, 13 Dec 2001 08:44:29 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] mk4py: infinite loop when passing dictionary criteria to view.find() Reply-to: gmcm@hypernet.com Message-ID: <3C186A6D.4510.5F03116@localhost> Priority: normal In-reply-to: <3C18672C.25988.5E37B80@localhost> References: <3C17B6E1.10816.9867AAB@localhost> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 13 Dec 2001 at 8:30, Gordon McMillan wrote: > Instead of > v.find(criteria, start=i+1) > you could: > criteria['start'] = i+1 > v.find(criteria) Oops. If you want to use start, you can't use a dictionary for criteria. That can probably be improved. BTW, if you expect a number of hits on criteria, you're often better off using select. -- Gordon http://www.mcmillan-inc.com/ From mmesser@masari.com Thu Dec 13 15:28:47 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA19526 for ; Thu, 13 Dec 2001 15:28:46 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000033; 13 Dec 01 12:06:05 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 13 Dec 01 12:05:51 -0800 Received: from abusimbel (63.94.91.2) by masari.com (Mercury/32 v3.21c) with ESMTP ID MG000032; 13 Dec 01 12:05:46 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Thu, 13 Dec 2001 13:19:45 -0800 MIME-Version: 1.0 Message-ID: <3C18AAF1.19696.F9246C1@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] mk2py: more weird stuff with vs.hash() and vs.order() Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: import metakit db = metakit.storage() #Create a view (this is the MetaKit term for "table"): vw = db.getas("people[last:S,first:S,shoesize:I]") vw.append(first='John',last='Lennon',shoesize=44) vw.append(first='Flash',last='Gordon',shoesize=42) mapvw = db.getas("map[_H:I,_R:I]") # THIS CAUSES A CRASH!! # hashvw1a = vw.hash(mapvw, numkeys=1) hashvw1b = vw.hash(mapvw, 1) ordvw1a = vw.ordered(numkeys=1) ordvw1b = vw.ordered(1) ordvw2a = vw.ordered(numkeys=2) ordvw2b = vw.ordered(2) def foo(v, criteria): for r in v.select(criteria): print 'foo:', r.first, r.last, r.shoesize print 'foo: done.\n' def bar(v): i = v.find(last='Gordon', start=0) while(i >= 0): print 'bar:', v[i].first, v[i].last, v[i].shoesize i = v.find(last='Gordon', start=i+1) print 'bar: done.\n' criteria = {'last':'Gordon'} foo(vw, criteria) foo(hashvw1b, criteria) foo(ordvw1a, criteria) foo(ordvw1b, criteria) foo(ordvw2a, criteria) foo(ordvw2b, criteria) bar(vw) bar(hashvw1b) bar(ordvw1a) # no results - isn't "ordered" an identity view???? bar(ordvw1b) # no results bar(ordvw2a) # no results bar(ordvw2b) From mmesser@masari.com Thu Dec 13 15:38:07 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA20045 for ; Thu, 13 Dec 2001 15:38:07 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000035; 13 Dec 01 12:15:25 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 13 Dec 01 12:15:23 -0800 Received: from abusimbel (63.94.91.2) by masari.com (Mercury/32 v3.21c) with ESMTP ID MG000034; 13 Dec 01 12:15:14 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Thu, 13 Dec 2001 13:29:13 -0800 MIME-Version: 1.0 Message-ID: <3C18AD29.19923.F9AEFD0@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] mk2py: more weird stuff with vs.hash() and vs.order() (FOLLOW UP) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: By the way, if you change the "getas" call to this: vw = db.getas("people[test:I,last:S,first:S,shoesize:I]") it works for all cases. From horst@freedict.de Sun Dec 16 03:15:29 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id DAA21603 for ; Sun, 16 Dec 2001 03:15:13 -0600 From: horst@freedict.de Received: (qmail 6932 invoked from network); 16 Dec 2001 09:31:56 -0000 Received: from dialin-145-254-117-023.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.117.23) by mail.netbeat.de with SMTP; 16 Dec 2001 09:31:56 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fBG8nrM23586; Sun, 16 Dec 2001 09:49:54 +0100 Message-Id: <200112160849.fBG8nrM23586@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Sun, 16 Dec 2001 09:49:50 +0100 (CET) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] Metakit for Python and File locking Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I did not find information on metakit for Python and file locking on the net. Is there a prefered way of locking metakit databases, and / or are there things to keep in mind, when implementing file locking. Note: I am implementing file locking, for a email based system, primarily running on linux, but platform independence certainly is desirable. Thanks for hints, Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From jcw@equi4.com Sun Dec 16 04:06:38 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA22079 for ; Sun, 16 Dec 2001 04:06:38 -0600 Received: from [10.0.1.55] ([195.121.234.16]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GOFKZD02.HOK for ; Sun, 16 Dec 2001 10:57:13 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Metakit for Python and File locking Date: Sun, 16 Dec 2001 10:53:58 +0100 Message-Id: <20011216095358.2032@mail.planet.nl> In-Reply-To: <200112160849.fBG8nrM23586@eaglesnest.mceggman> References: <200112160849.fBG8nrM23586@eaglesnest.mceggman> X-Mailer: CTM PowerMail 3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: >I did not find information on metakit for Python and file locking on the >net. Is there a prefered way of locking metakit databases, and / or are >there things to keep in mind, when implementing file locking. File locking was at one point part of Mk4py and Mk4tcl, but I took it out again, after struggling hard to make it work reliably on all platforms. I was looking for a solution which did not require an extra lock file, and which did not need cleaning up if an application was aborted prematurely. It's tricky stuff, across Unix variants and Windows to get this right. And a sort-of-usually-working solution which gives a false sense of working properly but doesn't is worse than none... hence the decision to take it out again. Nowadays, I tend to use lock files. Write the pid in th elock file, and when another process wants to acquire the lock: check that the locking process is still alive - if not, break the lock by removing the lock file. On Unix, there is the /proc// to check things, else do a "ps". Even then however, truly reliable locking on NFS file systems, in all their (sometimes broken) variations, seems to be next to impossible. Even with lockfiles, apparently. There's a discussion (Tcl-centric) about this on the Tcl'ers wiki: Python might be in a somewhat better position, since it implements lockf (in the Posix layer, AFAIK). Another source of information (perhaps dated by now) for me, was "procmail" which seems to goo out of its way to try to solve this. No URL, you'll have to chase it on Google to find sources and look there. I'm still very interested in solving this. Even a partial, but very clearly specified, solution would be useful to have in MK. -jcw From jcw@equi4.com Mon Dec 17 14:10:52 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA28170 for ; Mon, 17 Dec 2001 14:10:52 -0600 Received: from [10.0.1.55] ([195.121.233.224]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GOI7N401.956 for ; Mon, 17 Dec 2001 21:01:52 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] mk2py: more weird stuff with vs.hash() and vs.order() Date: Mon, 17 Dec 2001 20:55:19 +0100 Message-Id: <20011217195519.16006@mail.planet.nl> In-Reply-To: <3C18AAF1.19696.F9246C1@localhost> References: <3C18AAF1.19696.F9246C1@localhost> X-Mailer: CTM PowerMail 3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Messer wrote: (Sorry for the late response, I was tied up with some other work) >import metakit > >db = metakit.storage() > >#Create a view (this is the MetaKit term for "table"): >vw = db.getas("people[last:S,first:S,shoesize:I]") >vw.append(first='John',last='Lennon',shoesize=44) >vw.append(first='Flash',last='Gordon',shoesize=42) > >mapvw = db.getas("map[_H:I,_R:I]") > ># THIS CAUSES A CRASH!! ># hashvw1a = vw.hash(mapvw, numkeys=1) Resolved in Gordon's recent email: drop the "numkeys=". >hashvw1b = vw.hash(mapvw, 1) >ordvw1a = vw.ordered(numkeys=1) [drop the above] >ordvw1b = vw.ordered(1) >ordvw2a = vw.ordered(numkeys=2) [drop the above] >ordvw2b = vw.ordered(2) > >def foo(v, criteria): > for r in v.select(criteria): print 'foo:', r.first, r.last, r.shoesize > print 'foo: done.\n' > >def bar(v): > i = v.find(last='Gordon', start=0) > while(i >= 0): > print 'bar:', v[i].first, v[i].last, v[i].shoesize > i = v.find(last='Gordon', start=i+1) > print 'bar: done.\n' > >criteria = {'last':'Gordon'} > >foo(vw, criteria) >foo(hashvw1b, criteria) >foo(ordvw1a, criteria) >foo(ordvw1b, criteria) >foo(ordvw2a, criteria) >foo(ordvw2b, criteria) > >bar(vw) >bar(hashvw1b) >bar(ordvw1a) # no results - isn't "ordered" an identity view???? [ignore, see above] >bar(ordvw1b) # no results >bar(ordvw2a) # no results [ignore, see above] >bar(ordvw2b) The *one* key issue in this email is: bar(ordvw1b) # no results This can be explained by the fact that "ordered" is a mapping view which maintains the inderlying view in sort order. What needs to be documented better, though, is that it also requires the view as given to it at definition to already be in the proper sort order. In other words: bar(ordvw1b) # no results <-- you got bitten While: bar(ordvw2b) <-- you got lucky The last case just "happened" to find the key you were looking for. Thanks for reporting this. There is a clear distinction of which I was not aware between the hashed and the ordered view: - The hashed view uses a secondary view to maintain a hashing order, and builds it on-the-fly when defined on a view which already contains data. Whereas: - The ordered view does not use a secondary view. It expects the view to be sorted properly, and then maintains the sort order when insertions and deletions are applied. Things get more complicated, though. There is also an "indexed" mapping view, which is similar to "hash" in that it maintains a secondary sort index (a set of row numbers, basically). It too will create the index if it is found to be empty at definition time. And to make a long story even longer: there are restrictions in what these views do when keys are modified, i.e. properies which participate in the hash/sort order. The only proper solution for now is to *never* alter keys. If you have to change a key property, use a row deletion + insertion instead. Conclusion: there is still a lot of work left to do in MK to get all these special cases resolved properly, and fully automatically... -jcw From hans.yandle@cingular.com Thu Dec 20 10:37:18 2001 Received: from bscc-gateway.bscc.bls.com (smtp.cingular.com [170.35.240.204]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA24430 for ; Thu, 20 Dec 2001 10:37:18 -0600 From: hans.yandle@cingular.com Received: from laho-gateway.bscc.bls.com (mailhub-laho [30.146.28.2]) by bscc-gateway.bscc.bls.com (8.9.3/8.9.3) with ESMTP id LAA06831 for ; Thu, 20 Dec 2001 11:27:39 -0500 (EST) Received: from zeus.houstoncellular.com (apollo.houstoncell.com [172.1.9.3]) by laho-gateway.bscc.bls.com (8.9.3/8.9.3) with ESMTP id LAA12712 for ; Thu, 20 Dec 2001 11:27:38 -0500 (EST) Received: by scorpexc4.houstoncell.com with Internet Mail Service (5.5.2653.19) id ; Thu, 20 Dec 2001 10:27:37 -0600 Message-ID: <5B572F4E690AD411ADC200508BCF39920324D26B@scorpexc2.houstoncell.com> To: metakit@equi4.com Date: Thu, 20 Dec 2001 10:27:24 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C18973.3512BF70" Subject: [Metakit] SPARC/Solaris 8 Installation and Compile Instructions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C18973.3512BF70 Content-Type: text/plain; charset="iso-8859-1" I am interested in using your product with Python 2.1.1 on a Sun Solaris 8 Sparc server. At present, all my attempts to compile and install have failed. Can anyone help on this effort? Hints, comments, suggestions, etc are all welcome. ------_=_NextPart_001_01C18973.3512BF70 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable SPARC/Solaris 8 Installation and Compile Instructions

I am interested in using your product = with Python 2.1.1 on a Sun Solaris 8 Sparc server. At present, all my = attempts to compile and install have failed. Can anyone help on this = effort? Hints, comments, suggestions, etc are all welcome.

------_=_NextPart_001_01C18973.3512BF70-- From mlh@idi.ntnu.no Thu Dec 20 15:44:41 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA05497 for ; Thu, 20 Dec 2001 15:44:41 -0600 Received: from vier.idi.ntnu.no (IDENT:22955@vier.idi.ntnu.no [129.241.103.4]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with ESMTP id WAA16563; Thu, 20 Dec 2001 22:35:30 +0100 (MET) Received: (from mlh@localhost) by vier.idi.ntnu.no (8.9.3/8.9.3) id WAA04535; Thu, 20 Dec 2001 22:35:30 +0100 (MET) From: Magnus Lie Hetland Date: Thu, 20 Dec 2001 22:35:30 +0100 To: hans.yandle@cingular.com Cc: metakit@equi4.com Subject: Re: [Metakit] SPARC/Solaris 8 Installation and Compile Instructions Message-ID: <20011220223530.F3753@idi.ntnu.no> References: <5B572F4E690AD411ADC200508BCF39920324D26B@scorpexc2.houstoncell.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <5B572F4E690AD411ADC200508BCF39920324D26B@scorpexc2.houstoncell.com>; from hans.yandle@cingular.com on Thu, Dec 20, 2001 at 10:27:24AM -0600 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Thu, Dec 20, 2001 at 10:27:24AM -0600, hans.yandle@cingular.com wrote: > I am interested in using your product with Python 2.1.1 on a Sun Solaris 8 > Sparc server. At present, all my attempts to compile and install have > failed. Can anyone help on this effort? Hints, comments, suggestions, etc > are all welcome. I'm sure this won't be very helpful, but I've used MetaKit with Python 2.1 (and 2.2) on Solaris 8 without any problems at all. Since I had no problems, I also have no real hints, but perhaps the fact that it can be done could serve as an encouragement? :) -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From lvirden@cas.org Thu Dec 20 21:56:42 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA16098 for ; Thu, 20 Dec 2001 21:56:42 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id fBL3l2223656 for ; Thu, 20 Dec 2001 22:47:02 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fBL3l0V17077; Thu, 20 Dec 2001 22:47:00 -0500 (EST) Date: Thu, 20 Dec 2001 22:47:00 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011220224659.AAB17061@cas.org> Subject: Re: [Metakit] SPARC/Solaris 8 Installation and Compile Instructions In-Reply-To: Your message of Thu, 20 Dec 2001 10:27:24 -0600 To: metakit@equi4.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I suggest that you describe what compiler you are using, what version of python, metakit, etc. you are using, what configure options you are using, etc. as well as showing how it fails. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jyl@best.com Thu Dec 20 23:38:45 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA17618 for ; Thu, 20 Dec 2001 23:38:45 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id VAA17777; Thu, 20 Dec 2001 21:29:07 -0800 (PST) Message-ID: <3C22D765.4000202@best.com> Date: Thu, 20 Dec 2001 22:32:05 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Magnus Lie Hetland CC: hans.yandle@cingular.com, metakit@equi4.com Subject: Re: [Metakit] SPARC/Solaris 8 Installation and Compile Instructions References: <5B572F4E690AD411ADC200508BCF39920324D26B@scorpexc2.houstoncell.com> <20011220223530.F3753@idi.ntnu.no> Content-Type: multipart/alternative; boundary="------------010908050002090903010707" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------010908050002090903010707 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit I've just finished porting my e4Graph software (http://e4graph.sourceforge.net/) to Metakit 2.4.2 on both Solaris 7 and on Windows 98, with absolutely no problems. All tests of both Metakit and e4Graph pass flawlessly, and I think Metakit 2.4.2 is the best release so far. Upgrade if you can, is my advice. JYL Magnus Lie Hetland wrote: >On Thu, Dec 20, 2001 at 10:27:24AM -0600, hans.yandle@cingular.com wrote: > >>I am interested in using your product with Python 2.1.1 on a Sun Solaris 8 >>Sparc server. At present, all my attempts to compile and install have >>failed. Can anyone help on this effort? Hints, comments, suggestions, etc >>are all welcome. >> > >I'm sure this won't be very helpful, but I've used MetaKit with >Python 2.1 (and 2.2) on Solaris 8 without any problems at all. >Since I had no problems, I also have no real hints, but perhaps >the fact that it can be done could serve as an encouragement? > >:) > --------------010908050002090903010707 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit I've just finished porting my e4Graph software (http://e4graph.sourceforge.net/) to Metakit 2.4.2 on both Solaris 7 and on Windows 98, with absolutely no problems. All tests of both Metakit and e4Graph pass flawlessly, and I think Metakit 2.4.2 is the best release so far. Upgrade if you can, is my advice.

JYL

Magnus Lie Hetland wrote:
On Thu, Dec 20, 2001 at 10:27:24AM -0600, hans.yandle@cingular.com wrote:
I am interested in using your product with Python 2.1.1 on a Sun Solaris 8
Sparc server. At present, all my attempts to compile and install have
failed. Can anyone help on this effort? Hints, comments, suggestions, etc
are all welcome.

I'm sure this won't be very helpful, but I've used MetaKit with
Python 2.1 (and 2.2) on Solaris 8 without any problems at all.
Since I had no problems, I also have no real hints, but perhaps
the fact that it can be done could serve as an encouragement?

:)


--------------010908050002090903010707-- From jcw@equi4.com Fri Dec 21 07:26:04 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA24964 for ; Fri, 21 Dec 2001 07:26:04 -0600 Received: from [10.0.1.55] ([195.121.236.58]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GOP3J801.9Z8 for ; Fri, 21 Dec 2001 14:16:20 +0100 From: Jean-Claude Wippler To: metakit Date: Fri, 21 Dec 2001 14:13:08 +0100 Message-Id: <20011221131308.15747@mail.planet.nl> In-Reply-To: <3C22D765.4000202@best.com> References: <3C22D765.4000202@best.com> X-Mailer: CTM PowerMail 3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Breaking with the past Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >I've just finished porting my e4Graph software >(http://e4graph.sourceforge.net/) to Metakit 2.4.2 on both Solaris 7 and >on Windows 98, with absolutely no problems. All tests of both Metakit >and e4Graph pass flawlessly, and I think Metakit 2.4.2 is the best >release so far. Upgrade if you can, is my advice. Thanks. The state of 2.4.2, or rather HEAD-of-CVS (fixes continue to be made), is such that by now I urge everyone who hasn't migrated before to reconsider and start planning for it. The stability of the original 2.01 release is almost becoming a *disadvantage* now, in terms of version management. Let me try to summarize the main issues which affect a move towards 2.4.2: - File format: moving from 2.01 to 2.4 introduces a file format change, which is final after the first commit. This process is automatic, and there are no problems I know of (other than a very old 1.8.6 conversion issue marked in the source code), but it is important to note that after commit, 2.01 code can no longer use the file. - File open times have improved dramatically for complex datasets (many views) in that theuy are now essentially constant-time, but some of the time gained does come back as being slower at commit time. The reason is that free space determination still takes a full traversal of the "column-tree", even though data is not being loaded. - The M(emo) datatype is gone and has been merged with B(ytes). Memo- type storage is now automatic and adaptive, and works for B(ytes) as well as S(tring). This means that the choice between storing data as B(ytes) or S(tring) can now be made in terms of whether you want null-bytes, and the convenience of passing char* pointers in and out of MK calls. Comparisons on S(tring) continue to be vase-insensitive ASCII (with UTF-8 planned), while B(ytes) sort order is based on memcmp. - The c4_Storage::Store() call is gone. Restructuring must be done with GetAs. - Serialization no longer generates datafiles which *only* can be serialized back in. The new format allows normal memory-mapped and random-access use and is more suitable for streaming because the header includes the bytecount of what follows. Serialization is now the proper way to efficiently generate a new optimally-packed copy of a MK datafile. - Storage objects are now a special type of view (and derived from c4_View). One benefit is that you can init a storage as empty, then assign a new storage, so the need to allocate storage objects on the heap is gone, even when used as members in other objects. - There are plenty new features to help tempt you to make the switch, most notably the blocked and hash mapping views, which can deal far more efficiently with large datasets. Commit-extend and commit-aside need mentioning too, though these have not yet been optimized or as thoroughly tested as the rest. I would like to urge everyone who is still using a pre-2.4.2 release to make plans for a transition. We're in a chicken-and-egg type of situation: unless MK gets beaten up massively, I can never hope to resolve any remaining issues, and if I don't you may think that switching is still too risky. For those who use a recent release: please do report anything you find that does not work as expected, but also feel free to mention successful uses, so that everyone can get a feel for the amount of real use and correct operation. If confidentiality is an issue, feel free to email me and I'll post things without mentioning the source of the information. Let me finish off with all remaining issues in MK 2.4.2 that I'm aware of: - a crash on exit on Windows which *might* be a crash-on-commit in MK (long-time bug, can't debug because a debug build works) - just reported: Description() fails when the storage has no views at all (will resolve this asap) - hangs on LinuxPPC with m01 and m03, not yet investigated (gcc 2.95.3) And here's the list of 2.4.2 (or CVS) builds which works 100%, AFAIK: Windows/Linux/FreeBSD, Solaris (Sparc+x86), Mac (68k+PPC), MacOS X Also, as a to the past: a recent 2.4.x build on Win 3.1x (16-bit). Regards, Jean-Claude From jyl@best.com Fri Dec 21 09:02:57 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA28348 for ; Fri, 21 Dec 2001 09:02:55 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id GAA14895 for ; Fri, 21 Dec 2001 06:52:50 -0800 (PST) Message-ID: <3C235B7F.5080304@best.com> Date: Fri, 21 Dec 2001 07:55:43 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Stupid questions: how to get rid of a column in a view,how to discard a view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: System: Win98 Metakit 2.4.2. Due to format changes, I want to convert an old storage to the new format. In this conversion I'll lose several columns. Is it the case that when you GetAs in the old format, do other transformations, then GetAs in the new format, then commit, the old unneeded columns are discarded? That was my plan, however I'm not sure it's correct. Concrete example (showing stylized changes for only one view): Old format: nodes[firstMarker:I,firstVertex:I,lastVertex:I,vertexCount:I,...] New format: nodes[firstVerex:I,lastVertex:I,vertexCount:I,...] In other words the firstMarker:I should be discarded going from the old format to the new format. Second question: one of the views in the storage's old format should not be present when the storage is saved in the new format. Does setting its length to zero before committing make it not be saved during commit? Thanks! JYL From jcw@equi4.com Fri Dec 21 11:23:31 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA00582 for ; Fri, 21 Dec 2001 11:23:30 -0600 Received: from [10.0.1.55] ([195.121.236.72]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GOPEJR00.Q02 for ; Fri, 21 Dec 2001 18:14:15 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Breaking with the past Date: Fri, 21 Dec 2001 18:10:43 +0100 Message-Id: <20011221171043.28161@mail.planet.nl> In-Reply-To: <20011221131308.15747@mail.planet.nl> References: <20011221131308.15747@mail.planet.nl> X-Mailer: CTM PowerMail 3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: To follow up on my own post: > - just reported: Description() fails when the storage has no views at all > (will resolve this asap) It looks like this was ok, but the catch is that this case returns NULL. I've just checked an optimization for GetAs into CVS, which makes it an order of magnitude faster than before when no restructuring is needed and when the storage contains many (or very complex) views. -jcw From baspey@yahoo.com Mon Dec 24 12:23:56 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA11371 for ; Mon, 24 Dec 2001 12:23:55 -0600 Received: from toulouse-1-a7-35-155.dial.proxad.net (HELO desktop) (62.147.35.155) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Dec 2001 18:14:30 -0000 Message-ID: <000e01c18ca6$6f860b40$9b23933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Mon, 24 Dec 2001 19:11:16 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Memory leak when scope of c4_View is global Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello and merry Christmas to everybody, I use C++ and Metakit 2.4.2 (hope lots of people will follow JCW's advice and break with the past like me). I have a view which is used by many different objects. One solution would be to declare this view as a globlal variable. I know that using globals is not elegant programming, but that would simplify my code alot. Another solution would be to use a pointer pointing toward this view. But I don't know the equivalent to MyView[RowIndex] when MyView is a pointer instead of a real c4_View. So I choose the global solution, but I get important memory leaks. Adding "c4_View MyView;" somewhere globally, generates a 48-bytes memory leak. And if I use this view and forget to do "MyView = c4_View();" before the program stops, there are then lots lots of memory leaks. Any comment appreciated. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Tue Jan 1 05:58:39 2002 Received: from smtp013.mail.yahoo.com (smtp013.mail.yahoo.com [216.136.173.57]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id FAA07771 for ; Tue, 1 Jan 2002 05:58:39 -0600 Received: from toulouse-1-a7-34-34.dial.proxad.net (HELO desktop) (62.147.34.34) by smtp.mail.vip.sc5.yahoo.com with SMTP; 1 Jan 2002 11:48:44 -0000 Message-ID: <000b01c192b9$d0ed4420$2222933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 1 Jan 2002 12:43:30 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Memory leak when scope of c4_View is global Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello and Happy New Year to everyone, the memory leak I reported before has nothing to do with Metakit. It's all my fault as I could see when I tried to declare as global another variable, and also got memory leaks. Globals are tricky, and I now understand better why they should be avoided as much as possible. I won't go any further on the subject since it's only a C++ related question. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From Mark.Oakden@ubsw.com Thu Jan 10 05:18:32 2002 Received: from gate2.ldn.ubswarburg.com (gate2.ldn.ubswarburg.com [139.149.1.38]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA29271 for ; Thu, 10 Jan 2002 05:18:31 -0600 From: Mark.Oakden@ubsw.com Received: (from smap@localhost) by gate2.ldn.ubswarburg.com (8.8.8/8.8.8) id LAA29808 for ; Thu, 10 Jan 2002 11:08:01 GMT Received: from (nine.ubswarburg.com [192.168.0.4]) by gate2 via smap (V2.0) id xma029719; Thu, 10 Jan 2002 11:07:57 GMT Received: from ln4p1013pos.ldn.swissbank.com (virscan2 [192.168.0.4]) by virscan2.swissbank.com (8.8.8/8.8.8) with ESMTP id LAA11561 for ; Thu, 10 Jan 2002 11:10:01 GMT Received: from ln4p2350.ldn.swissbank.com (ln4p2350.ldn.swissbank.com [172.16.232.73]) by ln4p1013pos.ldn.swissbank.com (8.8.8/8.8.8) with ESMTP id LAA27128 for ; Thu, 10 Jan 2002 11:07:54 GMT Received: from localhost (root@localhost) by ln4p2350.ldn.swissbank.com (8.8.6 (PHNE_14041)/8.8.6/WDR gamma evision: 1.4 $) with ESMTP id LAA07978 for ; Thu, 10 Jan 2002 11:07:54 GMT X-OpenMail-Hops: 1 Date: Thu, 10 Jan 2002 11:07:53 +0000 Message-Id: MIME-Version: 1.0 TO: metakit@equi4.com Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline ;Creation-Date="Thu, 10 Jan 2002 11:07:52 +0000" Content-Transfer-Encoding: 7bit X-WDR-Disclaimer: Version $Revision: 1.15 $ Subject: [Metakit] Wikit corruption Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Sorry if this is the wrong forum to ask this question. I suspect that the cause of my problem is Wikit specific, but that the recovery (if it is possible) may require Metakit knowledge. [I have also posted this question to comp.lang.tcl] I have been setting up an internal Wiki using Wikit. I got it working but yesterday everything got corrupted somehow. I suspect that it happened when two people were editing via CGI at the same time, but can't prove that. The main page has now got a block of binary looking junk on the top and all the links on it to pages we had created have reverted to the "[New Link]" appearance (with linked "[" brackets to edit the "new" page). Has anybody seen this before? I assume it can be prevented with appropriate script/tclkit versions. Can I retrieve my pages from the corrupt Wikit file? Cheers, Mark -- Mark Oakden O:+44 20 75686190 M:+44 7703 531701 Platform Architecture and Design www.perotsystems.com perotsystems @ UBS Private Banking Europe www.ubs.com ---> any legalese/disclaimer below this line was added in transit <--- Visit our website at http://www.ubswarburg.com This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. From stephen@myseraph.org Thu Jan 10 05:19:37 2002 Received: from c007.snv.cp.net (c007-h011.c007.snv.cp.net [209.228.33.217]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id FAA29318 for ; Thu, 10 Jan 2002 05:19:35 -0600 Received: (cpmta 15160 invoked from network); 10 Jan 2002 03:09:00 -0800 Received: from 65.184.159.237 (HELO JEREMY) by smtp.directvinternet.com (209.228.33.217) with SMTP; 10 Jan 2002 03:09:00 -0800 X-Sent: 10 Jan 2002 11:09:00 GMT Message-ID: <001101c199c7$3a0dd050$0100a8c0@JEREMY> From: "Stephen Hansen" To: Date: Thu, 10 Jan 2002 03:09:09 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Is Metakit an answer to this problem? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, This has a few different sub-questions all lumped together, i'm sorry :) In a project i'm working on, my database needs will conform to a structure of approximately this shape: Objects[ ID:I, Owner:I, Attributes[ Name:S, Value:S], Body:S] That is, a database of items, each with a unique integer id and integer owner, and then a series of arbitrary Name:Value pairs and the actual data of the object. Its really not a complicated database, but it may start getting large in theory-- although certainly not into the GB range of things. In theory, thousands of items, each with anywhere between ten to twenty name:value pairs. Now, speed is absolutely critical. What I'll be doing with the above structure is a periodic search to answer such questions as, "for owner#, find all items who have attribute 'Flagged' set to 'Yes'." These searches may be frequent, and are not necessarily cacheable, and response time is very important. It is potentially possible that I can store a list of the most common attributes on the db record itself, making the description essentially: Objects[ ID:I, Owner:I, Flagged:S, Shown:S, etc:S, Attributes[ Name:S, Value:S], Body:S] There are approximately 10-15 attributes which are most common, and which I can thus hardwire into the db there-- and if you think it would bring a significant performance boost, I will. There will still be the need for an arbitrary name-value section though, even though it would then be significantly less used. Due to Metakit's column-wise setup, it seems to me that it is perhaps the right tool for the job. What do you think? For instance, I imagine something of the effect of: usersItems = items.select(id=UserId) flaggedItems = userItems.select(...) The above is in Python, which is what will be used for everything until it becomes necessary to recode the bottlenecks in C/C++. I'm not precisely sure how the second line should work, due to my current inexperience with Metakit's syntax; i'm almost tempted to just use a 'for' loop and go over the usersItems, and doing a find() on the Attributes subview of each item then compare it. There may or may not be a better way :) Anyways. I'm somewhat groggy, sorry for any horrendous grammar or logic errors in the above. :) Thanks in advance. --Stephen From gmcm@hypernet.com Thu Jan 10 08:19:36 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA03963 for ; Thu, 10 Jan 2002 08:19:35 -0600 Received: from PARANOIA (204.176.40.121) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Thu, 10 Jan 2002 09:09:12 -0500 From: "Gordon McMillan" To: Date: Thu, 10 Jan 2002 09:03:46 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Is Metakit an answer to this problem? Reply-to: gmcm@hypernet.com Message-ID: <3C3D58F2.4853.3E43A294@localhost> Priority: normal In-reply-to: <001101c199c7$3a0dd050$0100a8c0@JEREMY> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 10 Jan 2002 at 3:09, Stephen Hansen wrote: > In a project i'm working on, my database needs will conform > to a structure of approximately this shape: > > Objects[ > ID:I, > Owner:I, > Attributes[ > Name:S, > Value:S], > Body:S] > > .... In theory, thousands of items, each with anywhere > between ten to twenty name:value pairs. Sounds very reasonable. > Now, speed is absolutely critical. What I'll be doing > with the above structure is a periodic search to answer such > questions as, "for owner#, find all items who have > attribute 'Flagged' set to 'Yes'." These searches may be > frequent, and are not necessarily cacheable, and response > time is very important. It is potentially possible that I > can store a list of the most common attributes on > the db record itself [snip] > which I can thus hardwire into the db there-- and if you > think it would bring a significant performance boost, I > will. I certainly wouldn't start out that way! I kind of doubt you'll need it. > There will still be the need for an arbitrary > name-value section though, even though it would then be > significantly less used. [snip] > For instance, I imagine something of the effect of: > > usersItems = items.select(id=UserId) > flaggedItems = userItems.select(...) > userrecs = items.select(id=Userid) useritems = userrecs.flatten(userrecs.Attributes) Now you have a flat view, and your search becomes flagged = useritems.select(useritems.Name="Flagged", useritems.Value="Yes") If you just interested in existence, then a find would be faster in the last step. -- Gordon http://www.mcmillan-inc.com/ From erlavigne@wanadoo.fr Fri Jan 11 14:10:29 2002 Received: from mel-rto1.wanadoo.fr (smtp-out-1.wanadoo.fr [193.252.19.188]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA09796 for ; Fri, 11 Jan 2002 14:10:23 -0600 Received: from mel-rta5.wanadoo.fr (193.252.19.122) by mel-rto1.wanadoo.fr; 11 Jan 2002 20:59:16 +0100 Received: from [192.168.0.2] (193.252.31.240) by mel-rta5.wanadoo.fr; 11 Jan 2002 20:58:56 +0100 User-Agent: Microsoft-Entourage/9.0.2509 Date: Fri, 11 Jan 2002 20:58:55 +0100 From: Eric To: Message-ID: In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Metakit memory usage issue Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add data into a nested view. Addind data into a nested view seeems to use a lot of memory (and on macOS memory is a limited resource). Adding about 200 entries (with just one entry in the nested view for each) requires more than 1 Mb. This memory is not lost because when the storage is closed this memory is released. But importing a few hundred entries requires a few megabytes of available memory, and that's not very convenient. May be I'm doing something wrong in my code? You will find below a sample source code (for MacOS) reproducing this problem. Any help (or workaround) will be appreciated. Thanks in advance. -- Eric //======================================================================== #include "mk4.h" static long sFreeMem, sStartFreeMem; static void DumpFreeMem() { long theFreeMem = ::FreeMem(); printf("\tMemory change: %ld Kb %ld b\n", ((theFreeMem - sFreeMem) / 1024L), ((theFreeMem - sFreeMem) % 1024L)); sFreeMem = theFreeMem; } #define kTest_Desc "value[text:S,names[name:S,nickname:S]]" static void TestMetaKit() { sFreeMem = sStartFreeMem = ::FreeMem(); c4_Storage theStorage("test", true); c4_StringProp pText("text"); c4_ViewProp pNames("names"); c4_StringProp pNickName("nickname"); c4_StringProp pName("name"); c4_View theView = theStorage.GetAs(kTest_Desc); c4_Row theRow; theView.SetSize(0); for (size_t i = 0 ; i < 10 ; i++) { pText(theRow) = "test"; int n = theView.Add(theRow); for (size_t j = 0 ; j < 1 ; j++) { c4_View theNamesView = pNames(theView[n]); theNamesView.Add(pName["John"] + pNickName["johnny"]); } DumpFreeMem(); } long theFreeMem = ::FreeMem(); printf("\tTotal change: %ld Kb %ld b\n", ((theFreeMem - sStartFreeMem) / 1024L), ((theFreeMem - sStartFreeMem) % 1024L)); } From Murat_Berk@bmc.com Fri Jan 11 14:25:14 2002 Received: from creeper.bmc.com (camaro.bmc.com [198.207.223.231]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA10274 for ; Fri, 11 Jan 2002 14:25:14 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2/8.10.2) with ESMTP id g0BKDW710422; Fri, 11 Jan 2002 14:13:32 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Fri, 11 Jan 2002 14:13:50 -0600 Message-ID: From: "Berk, Murat" To: "'Eric'" , metakit@equi4.com Subject: RE: [Metakit] Metakit memory usage issue Date: Fri, 11 Jan 2002 14:13:48 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We had exact some problem w/ 2.0.1 This happens if you create nested views. Each nested view's column allocates 4KB memory. So even if you have a nested view w/ only 2-3 columns, (name, value), you use around 16K min (since string fields require 2 columns, one of them index or someting like that) -----Original Message----- From: Eric [mailto:erlavigne@wanadoo.fr] Sent: Friday, January 11, 2002 1:59 PM To: metakit@equi4.com Subject: [Metakit] Metakit memory usage issue Hello, I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add data into a nested view. Addind data into a nested view seeems to use a lot of memory (and on macOS memory is a limited resource). Adding about 200 entries (with just one entry in the nested view for each) requires more than 1 Mb. This memory is not lost because when the storage is closed this memory is released. But importing a few hundred entries requires a few megabytes of available memory, and that's not very convenient. May be I'm doing something wrong in my code? You will find below a sample source code (for MacOS) reproducing this problem. Any help (or workaround) will be appreciated. Thanks in advance. -- Eric //======================================================================== #include "mk4.h" static long sFreeMem, sStartFreeMem; static void DumpFreeMem() { long theFreeMem = ::FreeMem(); printf("\tMemory change: %ld Kb %ld b\n", ((theFreeMem - sFreeMem) / 1024L), ((theFreeMem - sFreeMem) % 1024L)); sFreeMem = theFreeMem; } #define kTest_Desc "value[text:S,names[name:S,nickname:S]]" static void TestMetaKit() { sFreeMem = sStartFreeMem = ::FreeMem(); c4_Storage theStorage("test", true); c4_StringProp pText("text"); c4_ViewProp pNames("names"); c4_StringProp pNickName("nickname"); c4_StringProp pName("name"); c4_View theView = theStorage.GetAs(kTest_Desc); c4_Row theRow; theView.SetSize(0); for (size_t i = 0 ; i < 10 ; i++) { pText(theRow) = "test"; int n = theView.Add(theRow); for (size_t j = 0 ; j < 1 ; j++) { c4_View theNamesView = pNames(theView[n]); theNamesView.Add(pName["John"] + pNickName["johnny"]); } DumpFreeMem(); } long theFreeMem = ::FreeMem(); printf("\tTotal change: %ld Kb %ld b\n", ((theFreeMem - sStartFreeMem) / 1024L), ((theFreeMem - sStartFreeMem) % 1024L)); } _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Fri Jan 11 15:40:34 2002 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA12382 for ; Fri, 11 Jan 2002 15:40:33 -0600 Received: from [10.0.1.55] ([195.121.233.99]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GPSMDX02.9KF for ; Fri, 11 Jan 2002 22:29:57 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Metakit memory usage issue Date: Fri, 11 Jan 2002 22:27:11 +0100 Message-Id: <20020111212711.17344@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Eric, >I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add >data into a nested view. >Addind data into a nested view seeems to use a lot of memory (and on macOS >memory is a limited resource). > >Adding about 200 entries (with just one entry in the nested view for each) >requires more than 1 Mb. This memory is not lost because when the storage is >closed this memory is released. But importing a few hundred entries requires >a few megabytes of available memory, and that's not very convenient. Yes, at some point, I changed MK to allocate columns in 4 Kb increments, and with subviews, you'll end up with at least one of those each as soon as there is some data. The trouble is that on the Mac (OS 8/9), things are particularly tricky because of the lack of memory-mapped files. It'll take some work to improve this. Basically, what needs to be done is to track the "over-allocated" columns, and then truncate the ones which haven't been modified for a while - so as to limit the number of columns which waste so much space. I'm quite hesitant to do this, since it is Mac-specific (more accurately: an issue only of platforms without mmap support). Dropping all the slack is not a real option, because that could severely reduce modify-performance. If anyone reading this has an idea for a simpler fix, please share your wisdom! -jcw From jcw@equi4.com Fri Jan 11 15:41:03 2002 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA12409 for ; Fri, 11 Jan 2002 15:41:02 -0600 Received: from [10.0.1.55] ([195.121.233.99]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GPSMDW00.IGZ for ; Fri, 11 Jan 2002 22:29:56 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Wikit corruption Date: Fri, 11 Jan 2002 22:16:12 +0100 Message-Id: <20020111211612.32007@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark.Oakden@ubsw.com wrote: >I have been setting up an internal Wiki using Wikit. I got it working >but yesterday everything got corrupted somehow. > >I suspect that it happened when two people were editing via CGI at the >same time, but can't prove that. Yes, this seems plausible. I've responded in more detail to your post on the Tcl'ers Wiki mailing list at . -jcw From Murat_Berk@bmc.com Fri Jan 11 15:53:51 2002 Received: from babbler.bmc.com (firebird.bmc.com [198.207.223.228]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA12815; Fri, 11 Jan 2002 15:53:51 -0600 Received: from ec02-hou.bmc.com (localhost [127.0.0.1]) by babbler.bmc.com (8.10.2/8.10.2) with ESMTP id g0BLhN114442; Fri, 11 Jan 2002 15:43:23 -0600 (CST) Received: by EC02-HOU.bmc.com with Internet Mail Service (5.5.2653.19) id ; Fri, 11 Jan 2002 15:40:38 -0600 Message-ID: From: "Berk, Murat" To: "'Jean-Claude Wippler'" , metakit Subject: RE: [Metakit] Metakit memory usage issue Date: Fri, 11 Jan 2002 15:40:38 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: When we save them, it is saved in a compact form. Because size of database is not huge. Does this means that when it is loaded, we do not use memory mapped area, but copy to the memory? Murat -----Original Message----- From: Jean-Claude Wippler [mailto:jcw@equi4.com] Sent: Friday, January 11, 2002 3:27 PM To: metakit Subject: Re: [Metakit] Metakit memory usage issue Eric, >I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add >data into a nested view. >Addind data into a nested view seeems to use a lot of memory (and on macOS >memory is a limited resource). > >Adding about 200 entries (with just one entry in the nested view for each) >requires more than 1 Mb. This memory is not lost because when the storage is >closed this memory is released. But importing a few hundred entries requires >a few megabytes of available memory, and that's not very convenient. Yes, at some point, I changed MK to allocate columns in 4 Kb increments, and with subviews, you'll end up with at least one of those each as soon as there is some data. The trouble is that on the Mac (OS 8/9), things are particularly tricky because of the lack of memory-mapped files. It'll take some work to improve this. Basically, what needs to be done is to track the "over-allocated" columns, and then truncate the ones which haven't been modified for a while - so as to limit the number of columns which waste so much space. I'm quite hesitant to do this, since it is Mac-specific (more accurately: an issue only of platforms without mmap support). Dropping all the slack is not a real option, because that could severely reduce modify-performance. If anyone reading this has an idea for a simpler fix, please share your wisdom! -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Fri Jan 11 16:08:27 2002 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA13285 for ; Fri, 11 Jan 2002 16:08:27 -0600 Received: from [10.0.1.55] ([195.121.233.181]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GPSNO601.3NB for ; Fri, 11 Jan 2002 22:57:42 +0100 From: Jean-Claude Wippler To: metakit Subject: RE: [Metakit] Metakit memory usage issue Date: Fri, 11 Jan 2002 22:55:04 +0100 Message-Id: <20020111215504.19817@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Berk, Murat wrote: >When we save them, it is saved in a compact form. Because size of database >is not huge. Does this means that when it is loaded, we do not use memory >mapped area, but copy to the memory? If I understand you right: no, the allocation happens when buffers get allocated for columns. When reading data, that never happens. It's upon modifying it. So... yes, perhaps in this case a workaround would be to commit somewhat more frequently on the Mac (this will slow it down, i.e. trade speed for memory use). -jcw From erlavigne@wanadoo.fr Fri Jan 11 16:15:15 2002 Received: from mel-rto7.wanadoo.fr (smtp-out-7.wanadoo.fr [193.252.19.26]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA13496 for ; Fri, 11 Jan 2002 16:15:14 -0600 Received: from mel-rta2.wanadoo.fr (193.252.19.152) by mel-rto7.wanadoo.fr; 11 Jan 2002 23:04:10 +0100 Received: from [192.168.0.2] (193.252.31.240) by mel-rta2.wanadoo.fr; 11 Jan 2002 23:03:53 +0100 User-Agent: Microsoft-Entourage/9.0.2509 Date: Fri, 11 Jan 2002 23:03:52 +0100 Subject: Re: [Metakit] Metakit memory usage issue From: Eric To: metakit Message-ID: In-Reply-To: <20020111212711.17344@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude, Why not adding a 'Compact' method to c4_View class that will do this work. It will be the developer responsibility to call this method when memory will get low (or on a regular basis whenn adding a lot of entries). This should not impact performance in all other cases and will offer a clean way to deal with this special case. I don't know if it's possible (and how complex it is) to implement this 'Compact' algorithm but personnally I will be very happy with it. -- Eric > Eric, > >> I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add >> data into a nested view. >> Addind data into a nested view seeems to use a lot of memory (and on macOS >> memory is a limited resource). >> >> Adding about 200 entries (with just one entry in the nested view for each) >> requires more than 1 Mb. This memory is not lost because when the storage is >> closed this memory is released. But importing a few hundred entries requires >> a few megabytes of available memory, and that's not very convenient. > > Yes, at some point, I changed MK to allocate columns in 4 Kb increments, > and with subviews, you'll end up with at least one of those each as soon > as there is some data. > > The trouble is that on the Mac (OS 8/9), things are particularly tricky > because of the lack of memory-mapped files. > > It'll take some work to improve this. Basically, what needs to be done > is to track the "over-allocated" columns, and then truncate the ones > which haven't been modified for a while - so as to limit the number of > columns which waste so much space. > > I'm quite hesitant to do this, since it is Mac-specific (more accurately: > an issue only of platforms without mmap support). Dropping all the slack > is not a real option, because that could severely reduce modify-performance. > > If anyone reading this has an idea for a simpler fix, please share your > wisdom! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jvasco@bellatlantic.net Sun Jan 13 20:51:18 2002 Received: from out001pub.verizon.net (out001pub.verizon.net [206.46.170.101]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA04386 for ; Sun, 13 Jan 2002 20:51:17 -0600 Received: from localhost.localdomain (pool-138-89-75-20.mad.east.verizon.net [138.89.75.20]) by out001pub.verizon.net with ESMTP for ; id g0E2e4OB006892 Sun, 13 Jan 2002 20:40:04 -0600 (CST) From: Jose Vasconcellos To: metakit@equi4.com Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/1.0 (Preview Release) Date: 13 Jan 2002 21:39:15 -0500 Message-Id: <1010975956.11969.0.camel@localhost.localdomain> Mime-Version: 1.0 Subject: [Metakit] Building Mk4py with python2.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For the adventurous that want to try using python2.2 be warned! The release has a declaration error such that Mk4py won't compile. The fix is to change the declaration in the python include file abstract.h. There is a define PyMapping_DelItemString that should be set to PyDict_DelItemString. It is erroneously set to PyObject_DelItemString which doesn't exist. Regards, Jose Vasconcellos From stephen@myseraph.org Sun Jan 13 23:16:09 2002 Received: from c007.snv.cp.net (c007-h014.c007.snv.cp.net [209.228.33.221]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA05163 for ; Sun, 13 Jan 2002 23:16:06 -0600 Received: (cpmta 9519 invoked from network); 13 Jan 2002 21:04:52 -0800 Received: from 65.184.159.237 (HELO JEREMY) by smtp.directvinternet.com (209.228.33.221) with SMTP; 13 Jan 2002 21:04:52 -0800 X-Sent: 14 Jan 2002 05:04:52 GMT Message-ID: <000501c19cb9$118eab50$0100a8c0@JEREMY> From: "Stephen Hansen" To: References: <1010975956.11969.0.camel@localhost.localdomain> Subject: Re: [Metakit] Building Mk4py with python2.2 Date: Sun, 13 Jan 2002 21:05:21 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Actually, the "real" fix is to use CVS to grab the release22-maint branch, where this bug is fixed. It is actually suspsoed to be PyObject_DelItemString, which was simply ommitted from abstract.c before the release. :) Go to, http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Object s/abstract.c, and you'll see on the top of the list is the fix we're discussing here. --S > For the adventurous that want to try using python2.2 be > warned! The release has a declaration error such that > Mk4py won't compile. The fix is to change the declaration > in the python include file abstract.h. There is a define > PyMapping_DelItemString that should be set to PyDict_DelItemString. > It is erroneously set to PyObject_DelItemString which doesn't > exist. > > Regards, > > Jose Vasconcellos > > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From ia@innovatech.dk Mon Jan 14 09:32:29 2002 Received: from mail.web-solutions.dk (ns2.web-solutions.dk [212.130.48.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA19362 for ; Mon, 14 Jan 2002 09:32:29 -0600 Message-Id: <200201141532.JAA19362@trixie.triqs.com> Received: from there ([80.62.45.76]) by mail.web-solutions.dk (Merak 4.10.050) with SMTP id JGB36881 for ; Mon, 14 Jan 2002 16:21:10 +0100 Content-Type: text/plain; charset="iso-8859-1" From: Israel Alvarez Reply-To: ia@innovatech.dk Organization: InnovaTech To: metakit@equi4.com Date: Mon, 14 Jan 2002 16:19:38 -0500 X-Mailer: KMail [version 1.3.1] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Metakit] Installation problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all!, I'm trying to install Metakit 2.4.2 in my Red Hat Linux System 7.2, with gcc 2.96. The problem I have is that when I run the ./configure i get: .... > checking for ANSI C header files... no > checking for working const... no > checking for inline... no > checking size of long... 0 > ./configure: line 5886: syntax error near unexpected token `AC_CHECK_TYPES(long' > ./configure: line 5886: `AC_CHECK_TYPES(long long)' and it fails. If I try to run ./autoconf I get: > autoconf: Undefined macros: > configure.in:68:AC_CHECK_TYPES(long long) and it fails. It seems like I don't have autoconf properly running, but I've never worked with it so I don't know what to do. In the other way, Is is possible to use your libmk4.so and libmk4.a libraries downloaded from your site and linking them to my C++ application? If I could do this, I think I would not need to install the sources. Is it OK? Thanks in advance -- Israel Alvarez ____________________________ InnovaTech Dortheavej 10 DK-2400 København NV ____________________________ Phone +45 70 27 28 11 Cell. Phone +45 20 93 00 53 ____________________________ http://www.InnovaTech.dk From gmcm@hypernet.com Mon Jan 14 11:40:03 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA24018 for ; Mon, 14 Jan 2002 11:40:02 -0600 Received: from PARANOIA (204.176.40.69) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Mon, 14 Jan 2002 12:29:31 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Mon, 14 Jan 2002 12:28:53 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Installation problems Reply-to: gmcm@hypernet.com Message-ID: <3C42CF05.27878.1082B02@localhost> Priority: normal In-reply-to: <200201141532.JAA19362@trixie.triqs.com> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=ISO-8859-1 Content-description: Mail message body Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from Quoted-printable to 8bit by trixie.triqs.com id LAA24018 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Israel, Don't know why that happens, but start over by running configure yourself, and it goes away. -- Gordon http://www.mcmillan-inc.com/ On 14 Jan 2002 at 16:19, Israel Alvarez wrote: > Hi all!, > I'm trying to install Metakit 2.4.2 in my Red Hat Linux System > 7.2, with gcc 2.96. The problem I have is that when I run the > ./configure i get: > > .... > > checking for ANSI C header files... no > > checking for working const... no > > checking for inline... no > > checking size of long... 0 > > ./configure: line 5886: syntax error near unexpected token > `AC_CHECK_TYPES(long' > > ./configure: line 5886: `AC_CHECK_TYPES(long long)' > > and it fails. > > If I try to run ./autoconf I get: > > > autoconf: Undefined macros: > > configure.in:68:AC_CHECK_TYPES(long long) > > and it fails. > > It seems like I don't have autoconf properly running, but I've > never worked with it so I don't know what to do. > > In the other way, Is is possible to use your libmk4.so and > libmk4.a libraries downloaded from your site and linking them to > my C++ application? If I could do this, I think I would not need > to install the sources. Is it OK? > > Thanks in advance > > -- > > Israel Alvarez > ____________________________ > > InnovaTech > Dortheavej 10 > DK-2400 København NV > ____________________________ > > Phone +45 70 27 28 11 > Cell. Phone +45 20 93 00 53 > ____________________________ > > http://www.InnovaTech.dk > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From mlh@idi.ntnu.no Tue Jan 15 15:41:55 2002 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA15736 for ; Tue, 15 Jan 2002 15:41:54 -0600 Received: from vier.idi.ntnu.no (IDENT:22955@vier.idi.ntnu.no [129.241.103.4]) by zevs.idi.ntnu.no (8.10.2+Sun/8.9.3) with ESMTP id g0FLV3p06255 for ; Tue, 15 Jan 2002 22:31:03 +0100 (MET) Received: (from mlh@localhost) by vier.idi.ntnu.no (8.9.3/8.9.3) id WAA07881 for metakit@equi4.com; Tue, 15 Jan 2002 22:31:03 +0100 (MET) Date: Tue, 15 Jan 2002 22:31:03 +0100 From: Magnus Lie Hetland To: Metakit Mailing List Message-ID: <20020115223103.A7437@idi.ntnu.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] Swig for PHP... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: By chance I looked at the Swig pages and it seems there is an experimental PHP module now... Perhaps this means that we can get PHP bindings for Metakit? - Magnus (who doesn't feel kompetent to do the binding himself...) -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From horst@freedict.de Tue Jan 15 23:05:23 2002 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA25218 for ; Tue, 15 Jan 2002 23:05:22 -0600 From: horst@freedict.de Received: (qmail 12933 invoked from network); 16 Jan 2002 05:24:40 -0000 Received: from dialin-145-254-115-239.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.115.239) by mail.netbeat.de with SMTP; 16 Jan 2002 05:24:40 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id g0FLJOk14855; Tue, 15 Jan 2002 22:19:25 +0100 Message-Id: <200201152119.g0FLJOk14855@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Tue, 15 Jan 2002 22:19:20 +0100 (CET) Reply-To: horst@freedict.de To: metakit MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] how to use hash with two keys? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I try to use a hashed view with to key fields. But the performance I get is poor. To see what I do, apply the patch to find.py in the examples dir 80c80 < r = vw.find(p1=k) --- > r = vw.find(p1=k, p2=k+k) 96c96 < vwh = vw.hash(map) --- > vwh = vw.hash(map,2) 102c102 < vwh = vw.hash(map) --- > vwh = vw.hash(map,2) 114,115c114,115 < hash_key(n/2, 10000) < hash_key(n+1, 10000) --- > hash_key(n/2, 1000000/n) > hash_key(n+1, 1000000/n) What should I have done differently? Thanks, Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From peter@tbots.com Wed Jan 16 11:41:08 2002 Received: from tbots.com (pop.cloud-nine.co.uk [62.172.32.16]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA16228 for ; Wed, 16 Jan 2002 11:41:07 -0600 Date: Wed, 16 Jan 2002 17:29:28 +0000 Message-Id: <200201161729.AA263717176@tbots.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii From: "peter" Reply-To: To: X-Mailer: Subject: [Metakit] metakit questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: hi Jean Claude There have been a few new releases of metakit since we last emailed each other. I hope you are well!! I have a couple of questions regarding metakit which I hope you can answer. The questions are primarily related to larger database files where I would require very fast operations. I wonder if metakit is the right choice given all the performance enhancements that have been made over the last 2 years. 1) when calling 'commit' - is the entire datafile rewritten, or just the part that has changed? 2) since metakit does not index its tables/columns , and I need best possible performance, is there any way to pick between binary search or some form of hashing? how does metakit compare to say a 'b-tree on disk' type of storage form? I know I could do an exhaustive performance test on this :-), but I'm a few weeks from a major product release, and I just wondered about the theoretical aspects of this. I havent found any notes on this in the documentation. 3) Does metakit have the equivalent of the SQL ...WHERE IN (1,2,3) , and if so, how does metakit go about the looking up the records? (I guess this relates to question 2 4) Does metakit now have equivalent of DELETE WHERE .... I remember having to reverse iterate through views do delete rows. If these questions are already answered in available literature somewhere, I would be grateful for a pointer. Best Regards, Peter Ritter ******************************************************** Peter Ritter Eurasia Software 0049 (0)172 695 2846 0049 (0)69 68600700 ******************************************************** From guenther.fischer@hrz.tu-chemnitz.de Thu Jan 17 01:24:48 2002 Received: from obelix.hrz.tu-chemnitz.de (obelix.hrz.tu-chemnitz.de [134.109.132.55]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA10364 for ; Thu, 17 Jan 2002 01:24:48 -0600 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by obelix.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #1) id 16R6kC-0005vp-00 for metakit@equi4.com; Thu, 17 Jan 2002 08:13:52 +0100 Date: Thu, 17 Jan 2002 08:13:51 +0100 (MET) From: Guenther Fischer To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from QUOTED-PRINTABLE to 8bit by trixie.triqs.com id BAA10364 Subject: [Metakit] 2 questions about tclkit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, 1. I use tclkit for linux/windows for a wine-database for german users (all texts in german language). With the latest version of tclkit I got the button names not shown correct (only those with german umlaut like ÄÖÜ and so on). Where there some changes in encoding? 2. I use tkhtml with tclkit to show help and other views. Now I also want to use images inside the html pages. gif will work but not jpeg. So I insert Img1.2 with libimg1.2.so and package require Img and get no a undefined Tk_PhotoExpand. Is there any help? thanks -- ~Guenther Fischer From jtate@mi-corporation.com Thu Jan 17 12:27:03 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA00567 for ; Thu, 17 Jan 2002 12:27:03 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0HIG0f12985 for ; Thu, 17 Jan 2002 13:16:00 -0500 From: "Joseph Tate" To: "MetaKit" Date: Thu, 17 Jan 2002 13:16:55 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Porting issues Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm trying to port Metakit to Windows CE so that our application (which runs under wince and hopefully windows 9x/NT) can have a common database backend. I've run into a few problems in the port and have a few questions. First of all, am I reinventing the wheel? Has someone else done this port? Second, is this a good idea given the limited memory and space on the PocketPC platform? If not, is there another similar solution that would be better suited? Third, the biggest issue in moving from Windows to Windows CE is the move from single byte to double byte characters. I've #def'd char->short, and replaced all the str* functions with their wide char equivalents, fixed memcopy issues in the string classes, etc, but now have issues with Views. Is there somewhere else in the Metakit code that relies on char being a single byte? (Not having a console or access to stdout/stderr is another issue, but I think I've got that handled.). Thanks in advance for your help. Joseph From guenther.fischer@hrz.tu-chemnitz.de Fri Jan 18 02:15:23 2002 Received: from asterix.hrz.tu-chemnitz.de (asterix.hrz.tu-chemnitz.de [134.109.132.84]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA20545 for ; Fri, 18 Jan 2002 02:15:22 -0600 Received: from pat.hrz.tu-chemnitz.de ([134.109.132.143] ident=mail) by asterix.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #1) id 16RU0c-0006wf-00 for metakit@equi4.com; Fri, 18 Jan 2002 09:04:22 +0100 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=root) by pat.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #2) id 16RU0b-0007kC-00 for metakit@equi4.com; Fri, 18 Jan 2002 09:04:21 +0100 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.9.3/8.9.3) with ESMTP id JAA24185 for ; Fri, 18 Jan 2002 09:04:21 +0100 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Fri, 18 Jan 2002 09:04:21 +0100 (MET) From: Guenther Fischer To: metakit@equi4.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] tclkit and iso8859-1 ? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi again, I try to verify the differences in encoding betwen the 2 tclkit's I use. (the latest and some releases back). I try to change the system encoding (I know I should do that in real ...): tclkit % encoding system identity % encoding system iso8859-1 % encoding system iso8859-1 and with the latest: ~/tclkit % encoding system identity % encoding system iso8859-1 unknown encoding "iso8859-1" and with an actual tclsh: /usr/local/ActiveTcl/bin/tclsh % encoding system iso8859-1 (the last is the best :-) What can I do? -- ~Guenther Fischer From genec@goldsmithsinc.com Wed Jan 23 09:06:43 2002 Received: from mail.goldsmithsinc.com ([64.66.89.67]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA13995 for ; Wed, 23 Jan 2002 09:06:43 -0600 Received: from genec (genec [10.1.1.100]) by mail.goldsmithsinc.com (Switch-2.0.0/Switch-2.0.0) with SMTP id g0NEjeV16602 for ; Wed, 23 Jan 2002 08:45:40 -0600 Message-ID: <003501c1a41d$7f8fc560$6401010a@genec> From: "Gene Christian" To: Date: Wed, 23 Jan 2002 08:51:54 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-Mimeole: Produced By Microsoft MimeOLE V5.50.4807.1700 Subject: [Metakit] Python questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been working with the 2.4.2 version of metakit for a few days, and I have run into some problems with the python interface. First, I can't get the wrap function to work properly. The wrap.py file that comes in the distribution shows correct results only for the tuple (position based) wrapping. The class wrapping shows three rows, but the string data is blank, and the integer data is all zeros. The dictionary wrapping gets the integers right, but the strings show junk. Here is the output I get from the wrap.py under Windows 2000: class objects: a b c - - - 0 0 0 - - - Total: 3 rows dictionary elements: a b c --- --- - x?y x?y 1 x?y x?y 2 x?y x?y 3 --- --- - Total: 3 rows tuples (by position): a b c ----- ----- - one un 1 two deux 2 three trois 3 ----- ----- - Total: 3 rows I have tested this on a Windows 2000 PC using ActiveState's 2.1.1 distribution and the new 2.1.2 distribution from python.org. I was using the Mk4py.dll binary. I also tested the wrap.py function on a Debian 3.0 (Woody) system using the debian python 2.1.1 and the metakit binary. In all three cases, the result was the same. Also, I have been exploring the derived views and sub-views and I don't think that the results I am getting are correct. I first tried to use the view.select() function to create a derived view, and I expected to be able to append a record to the derived view and have the main view updated as well. I couldn't get this to work. I have included below a copy of the commands I used to test this: >>> import metakit >>> mk = metakit >>> v = mk.view() >>> p1 = mk.property('S','group') >>> p2 = mk.property('S','data') >>> v.addproperty(p1) 0 >>> v.addproperty(p2) 1 >>> v.structure() [Property('S', 'group'), Property('S', 'data')] >>> v.append(group="a",data="one") 0 >>> v.append(group="a",data="two") 1 >>> v.append(group="b",data="X") 2 >>> v.append(group="b",data="Y") 3 >>> mk.dump(v) group data ----- ---- a one a two b X b Y ----- ---- Total: 4 rows >>> v1 = v.select(group='a') >>> mk.dump(v1) group data ----- ---- a one a two ----- ---- Total: 2 rows >>> v1.append(group='a',data='three') 2 >>> mk.dump(v1) group data ----- ---- a one a two ----- ---- Total: 2 rows >>> mk.dump(v) group data ----- ---- a one a two b X b Y ----- ---- Total: 4 rows >>> v.append(group='a',data='three') 4 >>> mk.dump(v) group data ----- ----- a one a two b X b Y a three ----- ----- Total: 5 rows >>> mk.dump(v1) group data ----- ----- a one a two a three ----- ----- Total: 3 rows >>> I have had similar results with the groupby function. Am I using the wrong version of python (is it compiled for 2.0)? I could compile metakit on Linux, but my target platform is Windows and I don't have a windows compiler. Is this a bug or am I using it wrong? Thanks. Gene Christian From gmcm@hypernet.com Wed Jan 23 10:20:27 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA16506 for ; Wed, 23 Jan 2002 10:20:27 -0600 Received: from PARANOIA (204.176.40.51) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Wed, 23 Jan 2002 11:09:15 -0500 From: "Gordon McMillan" To: Date: Wed, 23 Jan 2002 11:09:00 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Python questions Reply-to: gmcm@hypernet.com Message-ID: <3C4E99CC.19083.2F184439@localhost> Priority: normal In-reply-to: <003501c1a41d$7f8fc560$6401010a@genec> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 23 Jan 2002 at 8:51, Gene Christian wrote: > I have been working with the 2.4.2 version of metakit for a few > days, and I have run into some problems with the python > interface. > > First, I can't get the wrap function to work properly. > The wrap.py file that comes in the distribution shows > correct results only for the tuple (position based) > wrapping. The class wrapping shows three rows, but the > string data is blank, and the integer data is all zeros. > The dictionary wrapping gets the integers right, but the > strings show junk. You're right. I'll look into it. I've never used wrap with anything but tuples, so I don't know how long that's been broken. > Also, I have been exploring the derived views and > sub-views and I don't think that the results I am getting > are correct. I first tried to use the view.select() > function to create a derived view, and I expected to be > able to append a record to the derived view and have the > main view updated as well. Nope. You can alter a row in the selected view, and the changes will be reflected in the base view, but you can't change the (derived) container. If you append to the base view, a selected or sorted view will see the changes. You've stumbled into a confusing area. If you pull Mk4py from equi4.com's cvs, I've straightened it out so that it's much clearer what you can & cannot do (no more silent errors). There are now Views, Viewers, ROViewers. Also RORowRefs as well as RowRefs. Base views, view copies and the mapping views are all Views (for a mapping view, don't touch the base view). You can do anything. Viewers let you modify rows, but not the container (sort, select return Viewers). Basically, anything else gives you a ROViewer. Of course, sorting / selecting a ROViewer leaves it a ROViewer. > I have had similar results with the groupby function. Which is RO. > Am I using the wrong version of python (is it compiled for > 2.0)? I could compile metakit on Linux, but my target > platform is Windows and I don't have a windows compiler. > Is this a bug or am I using it wrong? I could send you a Mk4py built from CVS for 2.1 if you like. It's still "experimental", though I'm using it fulltime. -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Thu Jan 24 09:06:37 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA19332 for ; Thu, 24 Jan 2002 09:06:35 -0600 Received: from PARANOIA (204.176.40.101) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 24 Jan 2002 09:55:18 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Thu, 24 Jan 2002 09:55:01 -0500 MIME-Version: 1.0 Reply-to: gmcm@hypernet.com CC: David Ascher Message-ID: <3C4FD9F5.16568.33FAE306@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] Windows DLLs Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For the benefit of Windows users without compilers, I have placed builds of the CVS Mk4py for Python21 and Python22 at http://www.mcmillan-inc.com/dnld/Mk4py2.1-cvs.zip http://www.mcmillan-inc.com/dnld/Mk4py2.2-cvs.zip In addition to the fancy type stuff discussed yesterday, these have the fixes for wrap (which used to work only for tuples). Warning: this used to work: wrap([0,1], [property('I','x')]) although it shouldn't have. That now needs to be: wrap([(0,), (1,)], [property('I', 'x')], 1) -- Gordon http://www.mcmillan-inc.com/ From mh@pixar.com Thu Jan 24 12:50:44 2002 Received: from pixar.pixar.com (pixar.pixar.com [138.72.10.20]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA29045 for ; Thu, 24 Jan 2002 12:50:44 -0600 Received: from postal.pixar.com (postal.pixar.com [138.72.126.101]) by pixar.pixar.com (8.9.3/8.9.3) with ESMTP id KAA26612 for ; Thu, 24 Jan 2002 10:38:48 -0800 (PST) Received: from pixar.com (dreadnok [138.72.16.110]) by postal.pixar.com (8.11.1/8.11.1) with ESMTP id g0OIceJ14495 for ; Thu, 24 Jan 2002 10:38:40 -0800 (PST) Message-ID: <3C5054B7.5050107@pixar.com> Date: Thu, 24 Jan 2002 10:38:47 -0800 From: Mark Harrison User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.7) Gecko/20011221 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] cannot run more than one instance of wikit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is on linux, apache. If two instances of wikit are executed, one of them returns an error 500 with this in the log file: [error] (26)Text file busy: exec of /mh/htdocs/t1/wikit.cgi failed [error] [client 138.72.16.110] Premature end of script headers: /mh/htdocs/t1/wikit.cgi For now I'm working around it with setting the apache parameters to > MinSpareServers 1 > MaxSpareServers 1 > StartServers 1 > MaxClients 1 > MaxRequestsPerChild 100 Any ideas? Thanks, Mark --- Mark Harrison -- mh@pixar.com Pixar Animation Studios From nathan.thompson-amato@dnamerican.com Thu Jan 24 16:23:34 2002 Received: from montecarlo.DNAMERICAN.COM ([66.109.177.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA05660 for ; Thu, 24 Jan 2002 16:23:24 -0600 Received: from [66.109.177.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ma067976 for ; Thu, 24 Jan 2002 17:26:03 -0500 Message-ID: <3C50867C.9020906@dnamerican.com> Date: Thu, 24 Jan 2002 17:11:08 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.6) Gecko/20011120 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Multiple databases in one file: possible? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, I've been using Metakit for a single database (one basic view with a hash table on top of it) for a while now, and it works beautifully. However, I'd like to add an entirely independent database without using a separate storage object. I've tried creating a new view with something like c4_View *otherView = new c4_View(storage->GetAs("otherData[name:I]")); but attempts to access this database cause crashes. Should I expect this to work? If not, is there some other technique I should be using? Thanks, Nathan From jcw@equi4.com Mon Jan 28 10:38:19 2002 Received: from [10.0.1.55] (ipc379e8df.dial.wxs.nl [195.121.232.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGcI830203 for ; Mon, 28 Jan 2002 10:38:18 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] metakit questions Date: Mon, 28 Jan 2002 17:34:30 +0100 Message-Id: <20020128163430.14182@triqs.com> In-Reply-To: <200201161729.AA263717176@tbots.com> References: <200201161729.AA263717176@tbots.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: peter wrote: >I have a couple of questions regarding metakit which I hope you >can answer. The questions are primarily related to larger database >files where I would require very fast operations. I wonder if >metakit is the right choice given all the performance enhancements >that have been made over the last 2 years. > >1) when calling 'commit' - is the entire datafile rewritten, or >just the part that has changed? Just the part that has changed, but this has to be qualified further: "part" being columns. With blocked views, columns are broken down in smaller chunks, which is why blocked views greatly improve commit speed in large views. >2) since metakit does not index its tables/columns , and I need >best possible performance, is there any way to pick between binary >search or some form of hashing? how does metakit compare to say >a 'b-tree on disk' type of storage form? I know I could do an >exhaustive performance test on this :-), but I'm a few weeks from >a major product release, and I just wondered about the theoretical >aspects of this. I havent found any notes on this in the >documentation. The hashing performance of MK has been, eh, hashed over quite a bit lately. I think that you will find that hashed views provide excellent access-by key performance. By layering an ordered view over a blocked view, you get something which is very similar to a B-tree (but fixed at two levels). There are still some optimizations left to do to take full advantage of locality-of- reference for non-leaf blocks (speaking in B-tree terms, not MK), but it's a start. And it has the same semantics: an ordered view, which can be efficiently traversed. Note also that c4_View::Locate() is an improved API for (binary) Search, which lets you find out how many rows match. The logic of this all is automatic: if you do a Find on the first property in a hashed view, MK will take advantage of hashing. On other Finds, it reverts to the linear approach. Same for locate, ordered, etc. >3) Does metakit have the equivalent of the SQL ...WHERE IN (1,2,3) >, and if so, how does metakit go about the looking up the records? >(I guess this relates to question 2 No. Do a sequential scan. Keep in mind that sequential scans continue to be blazingly fast (even if O(N)). >4) Does metakit now have equivalent of DELETE WHERE .... >I remember having to reverse iterate through views do delete rows. This too has not changed. Issue 3) and 4) are basically limited by the lack of an API in C++ to specify these things (how to pass a "where expression" in C++ for example, without having to implement a complete parser). I admit that I haven't given it much thought recently. -jcw From jcw@equi4.com Mon Jan 28 10:40:49 2002 Received: from [10.0.1.55] (ipc379e8df.dial.wxs.nl [195.121.232.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGem830343 for ; Mon, 28 Jan 2002 10:40:48 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] tclkit and iso8859-1 ? Date: Mon, 28 Jan 2002 17:35:05 +0100 Message-Id: <20020128163505.25706@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: >Hi again, >I try to verify the differences in encoding betwen the 2 tclkit's I use. >(the latest and some releases back). [1] >% encoding system >iso8859-1 [2] >% encoding system iso8859-1 >unknown encoding "iso8859-1" The Unicode encoding logic has been causing me a headache for ages. The good news is that in the new TclKit builds, things are starting to look better. The version which is most likely to work for you is at http://www.equi4.com/previews/tclkit.* It behaves as [1], as far as I can tell. You are right in saying that the default encoding should have been set up right away, but there seems to be a nasty chicken-and-egg issue: TclKit needs to open the MK database before it has access to its runtime files, such as encodings. For now, the above will let you set the encoding, AFAIK. On Windows, things are better - and properly set up. I am looking into Linux/Unix, to see why the standard Tcl startup logic is not getting things right. -jcw From jcw@equi4.com Mon Jan 28 10:40:51 2002 Received: from [10.0.1.55] (ipc379e8df.dial.wxs.nl [195.121.232.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGeo830347 for ; Mon, 28 Jan 2002 10:40:50 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] cannot run more than one instance of wikit? Date: Mon, 28 Jan 2002 17:36:17 +0100 Message-Id: <20020128163617.29498@triqs.com> In-Reply-To: <3C5054B7.5050107@pixar.com> References: <3C5054B7.5050107@pixar.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Harrison wrote: >This is on linux, apache. > >If two instances of wikit are executed, one of them returns >an error 500 with this in the log file: > >[error] (26)Text file busy: exec of /mh/htdocs/t1/wikit.cgi failed >[error] [client 138.72.16.110] Premature end of script headers: > /mh/htdocs/t1/wikit.cgi > >For now I'm working around it with setting the apache parameters to [...] This looks like a Unix issue. TclKit is an executable with data et the end. That data is in fact a MetaKit database, which gets opened read- only as memory-mapped file. It seems that on some Unix systems, that extra open or mmap prevents an exec on it from running a second instance. Try running things through a shell script, my CGI's usually look like: #!/bin/sh cd /myhome/data exec /myhome/bin/tclkit wikit.tkd -jcw From jcw@equi4.com Mon Jan 28 10:40:52 2002 Received: from [10.0.1.55] (ipc379e8df.dial.wxs.nl [195.121.232.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGep830351 for ; Mon, 28 Jan 2002 10:40:52 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] how to use hash with two keys? Date: Mon, 28 Jan 2002 17:36:18 +0100 Message-Id: <20020128163618.22596@triqs.com> In-Reply-To: <20020128154353.30922@triqs.com> References: <20020128154353.30922@triqs.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: >I try to use a hashed view with to key fields. >But the performance I get is poor. > >To see what I do, apply the patch to find.py in the examples dir > >80c80 >< r = vw.find(p1=k) >--- >> r = vw.find(p1=k, p2=k+k) >96c96 >< vwh = vw.hash(map) >--- >> vwh = vw.hash(map,2) >102c102 >< vwh = vw.hash(map) >--- >> vwh = vw.hash(map,2) >114,115c114,115 >< hash_key(n/2, 10000) >< hash_key(n+1, 10000) >--- >> hash_key(n/2, 1000000/n) >> hash_key(n+1, 1000000/n) > >What should I have done differently? The line "r = vw.find(p1=k, p2=k+k)" passes p1 and p2 as Python keyword arguments to Mk4py (which internally is a dict). That, unfortunately means that the order is not preserved. My hunch is that Mk4py then builds a MK row from it in the opposite order (p2,p1). The problem with multi-key optimized finds in MK, is that the key given must be in the exact same order as declared in the view. In this case, the Python hashing exposes this weakness. I think you should try using a list (or a tuple) instead: r = vw.find([k, k+k]) The order is now implicit from the view, and hence always right. -jcw From jtate@mi-corporation.com Mon Jan 28 10:47:38 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGlc830686 for ; Mon, 28 Jan 2002 10:47:38 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0SGlWf07528 for ; Mon, 28 Jan 2002 11:47:32 -0500 From: "Joseph Tate" To: "MetaKit" Date: Mon, 28 Jan 2002 11:48:44 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Metakit porting issue. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've ported Metakit to Windows CE replacing all char-centric code with two-byte characters. I've also ported most of the test suite. I am now to the hash mapping functions and have found an infinite loop. The loop happens in c4_HashViewer::LookDict when //tmapped.cpp line 18 v3.Add(p1 [_T("b93655249726e5ef4c68e45033c2e0850570e1e07")]); is called, but arises from the creation of the view in lines above. The problem is that the empty placeholder row is created with the value -1, which is fine, but when IsUnused is finally called within LookDict the value has been morphed to 255 (if changed in the debug editor, it works fine). It's taking the "empty" value -1, stored in 8 bits, and reading it as if it was stored in 16 bits. The value -1 is copied multiple times between the creation of the view and the addition of the first element, but I can't seem to find where the loss of bitlength occurs. Can anyone point me in the right direction? Thanks, Joseph From jtate@mi-corporation.com Mon Jan 28 12:34:49 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SIYn802511 for ; Mon, 28 Jan 2002 12:34:49 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0SIYgf08032 for ; Mon, 28 Jan 2002 13:34:42 -0500 From: "Joseph Tate" To: "MetaKit" Subject: RE: [Metakit] Metakit porting issue. Date: Mon, 28 Jan 2002 13:35:55 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Figured out that issue. Problem was I was using an unsigned entity in Get_8i, and unsigned -1 is 255 when you're only using one byte. Joseph -----Original Message----- From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf Of Joseph Tate Sent: Monday, January 28, 2002 11:49 AM To: MetaKit Subject: [Metakit] Metakit porting issue. I've ported Metakit to Windows CE replacing all char-centric code with two-byte characters. I've also ported most of the test suite. I am now to the hash mapping functions and have found an infinite loop. The loop happens in c4_HashViewer::LookDict when //tmapped.cpp line 18 v3.Add(p1 [_T("b93655249726e5ef4c68e45033c2e0850570e1e07")]); is called, but arises from the creation of the view in lines above. The problem is that the empty placeholder row is created with the value -1, which is fine, but when IsUnused is finally called within LookDict the value has been morphed to 255 (if changed in the debug editor, it works fine). It's taking the "empty" value -1, stored in 8 bits, and reading it as if it was stored in 16 bits. The value -1 is copied multiple times between the creation of the view and the addition of the first element, but I can't seem to find where the loss of bitlength occurs. Can anyone point me in the right direction? Thanks, Joseph _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From guenther.fischer@hrz.tu-chemnitz.de Mon Jan 28 13:26:37 2002 Received: from asterix.hrz.tu-chemnitz.de (asterix.hrz.tu-chemnitz.de [134.109.132.84]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SJQa804762 for ; Mon, 28 Jan 2002 13:26:37 -0600 Received: from pat.hrz.tu-chemnitz.de ([134.109.132.143] ident=mail) by asterix.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #1) id 16VHQJ-0002gh-00 for metakit@equi4.com; Mon, 28 Jan 2002 20:26:35 +0100 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=root) by pat.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #2) id 16VHQJ-0003JS-00 for metakit@equi4.com; Mon, 28 Jan 2002 20:26:35 +0100 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.9.3/8.9.3) with ESMTP id UAA15699 for ; Mon, 28 Jan 2002 20:26:35 +0100 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Mon, 28 Jan 2002 20:26:35 +0100 (MET) From: Guenther Fischer To: metakit Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Jpeg images with tclkit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I tried to use the Img to add jpeg to view in tclkit. I found no way to add this package always some errors. I also build a tclkit for linux from the source an the img library with the same source. I didn't get it run - is there anybody who has done this. Could you add the Img to the new tclkit or to kitten Jean Cloude? -- ~Guenther Fischer From tciuro@yahoo.com Mon Jan 28 13:49:22 2002 Received: from web20603.mail.yahoo.com (web20603.mail.yahoo.com [216.136.226.161]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g0SJnM805680 for ; Mon, 28 Jan 2002 13:49:22 -0600 Message-ID: <20020128194914.71226.qmail@web20603.mail.yahoo.com> Received: from [80.32.136.3] by web20603.mail.yahoo.com via HTTP; Mon, 28 Jan 2002 11:49:14 PST Date: Mon, 28 Jan 2002 11:49:14 -0800 (PST) From: Tito Ciuro To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] MetaKit: Little vs big endian Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all, What is the status of MetaKit regarding little vs big endian? Is this something that I can count on in the current release? (2.4.2) Has anyone tried to access the same MetaKit data file from two machines with different endieness at the same time? Issues anyone? TIA, -- Tito __________________________________________________ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com From jtate@mi-corporation.com Mon Jan 28 14:08:45 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SK8i806393; Mon, 28 Jan 2002 14:08:44 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0SK8bf08635; Mon, 28 Jan 2002 15:08:37 -0500 From: "Joseph Tate" To: "MetaKit" , "Jean-Claude Wippler" Date: Mon, 28 Jan 2002 15:09:51 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Metakit Ported to Windows CE Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have ported Metakit verison 2.4.2 to Windows CE. Please let me know if you would like sources/projects/binary builds. I'm cleaning it up right now: making sure that with all my #ifdef statements that it still compiles under windows, etc. Joseph From jyl@best.com Mon Jan 28 23:57:05 2002 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0T5v5819660 for ; Mon, 28 Jan 2002 23:57:05 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id VAA04615; Mon, 28 Jan 2002 21:56:28 -0800 (PST) Message-ID: <3C564862.4080907@best.com> Date: Mon, 28 Jan 2002 22:59:46 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Tito Ciuro CC: metakit@equi4.com Subject: Re: [Metakit] MetaKit: Little vs big endian References: <20020128194914.71226.qmail@web20603.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Tito I regularly transport Metakit storages between Sparc and Intel, no issues so far. I'd be surprised if there were any, Metakit storages are supposed to be portable and architecture neutral. JYL Tito Ciuro wrote: >Hello all, > >What is the status of MetaKit regarding little vs big >endian? Is this something that I can count on in the >current release? (2.4.2) > >Has anyone tried to access the same MetaKit data file >from two machines with different endieness at the same >time? Issues anyone? > >TIA, > >-- Tito > >__________________________________________________ >Do You Yahoo!? >Great stuff seeking new owners in Yahoo! Auctions! >http://auctions.yahoo.com >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From steve@improvision.com Tue Jan 29 04:19:38 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TAJc824227 for ; Tue, 29 Jan 2002 04:19:38 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id KAA21880 for ; Tue, 29 Jan 2002 10:17:10 GMT Received: from [192.168.1.91] ([192.168.1.91]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Tue, 29 Jan 2002 10:19:34 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Tue, 29 Jan 2002 10:19:37 +0000 Subject: Re: [Metakit] MetaKit: Little vs big endian From: Stephen Baxter To: Jacob Levy , Tito Ciuro CC: Message-ID: In-Reply-To: <3C564862.4080907@best.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Works for me - we're using it between Mac and PC with no problems. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jacob Levy > Date: Mon, 28 Jan 2002 22:59:46 -0800 > To: Tito Ciuro > Cc: metakit@equi4.com > Subject: Re: [Metakit] MetaKit: Little vs big endian > > Hi Tito > > I regularly transport Metakit storages between Sparc and Intel, no > issues so far. I'd be surprised if there were any, Metakit storages are > supposed to be portable and architecture neutral. > > JYL > > Tito Ciuro wrote: > >> Hello all, >> >> What is the status of MetaKit regarding little vs big >> endian? Is this something that I can count on in the >> current release? (2.4.2) >> >> Has anyone tried to access the same MetaKit data file >> from two machines with different endieness at the same >> time? Issues anyone? >> >> TIA, >> >> -- Tito >> >> __________________________________________________ >> Do You Yahoo!? >> Great stuff seeking new owners in Yahoo! Auctions! >> http://auctions.yahoo.com >> _______________________________________________ >> metakit mailing list - metakit@equi4.com >> http://www.equi4.com/mailman/listinfo/metakit >> > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Tue Jan 29 04:33:20 2002 Received: from [10.0.1.55] (ipc379e9a0.dial.wxs.nl [195.121.233.160]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TAXJ824821 for ; Tue, 29 Jan 2002 04:33:19 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] MetaKit: Little vs big endian Date: Tue, 29 Jan 2002 11:23:39 +0100 Message-Id: <20020129102339.19092@triqs.com> In-Reply-To: <20020128194914.71226.qmail@web20603.mail.yahoo.com> References: <20020128194914.71226.qmail@web20603.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito Ciuro wrote: >What is the status of MetaKit regarding little vs big >endian? Is this something that I can count on in the >current release? (2.4.2) Yes, it works fine. With one exception, which I am chasing right now: there is an issue with load/save (i.e. seraalizing the datafile out), when the datafile was created on a machine with the opposite endianness. But for normal opens/commit, it should not cause problems anywhere. -jcw From jcw@equi4.com Tue Jan 29 04:33:22 2002 Received: from [10.0.1.55] (ipc379e9a0.dial.wxs.nl [195.121.233.160]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TAXL824825 for ; Tue, 29 Jan 2002 04:33:21 -0600 From: Jean-Claude Wippler To: MetaKit Subject: Re: [Metakit] Metakit Ported to Windows CE Date: Tue, 29 Jan 2002 11:30:20 +0100 Message-Id: <20020129103020.19246@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joseph Tate wrote: >I have ported Metakit verison 2.4.2 to Windows CE. Please let me know if >you would like sources/projects/binary builds. I'm cleaning it up right >now: making sure that with all my #ifdef statements that it still compiles >under windows, etc. Joseph, thank you very much for doing this - I apologize for not responding sooner (I had started to several times, but each time thought of some detail for this port which needed for thought). I'm impressed that you got this far and would very much like to provide your build on the website for others to download, if you permit. Also, I'd like to check a few things out w.r.t. the changes you needed to make o get things working. Ultimately, it would be great if we can merge the source code changes so WinCE support becomes part of the standard distribution. SInce I have a WinCE 2.11 PDA lying around (Cassiopeia E11 - i.e. MIPS), I could check things and try them out even. But the big Q is how to build for it: last time I looked into this, there was a free D/L from Microsoft for the MSVC (5? 6? I have both) environment. Is that the way to go - or is there a gcc option nowadays. -jcw From jcw@equi4.com Tue Jan 29 07:03:32 2002 Received: from [10.0.1.55] (ipc379e836.dial.wxs.nl [195.121.232.54]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TD3V831813 for ; Tue, 29 Jan 2002 07:03:32 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Jpeg images with tclkit Date: Tue, 29 Jan 2002 13:57:20 +0100 Message-Id: <20020129125720.21189@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: >I tried to use the Img to add jpeg to view in tclkit. I found no way >to add this package always some errors. (For non-Tcl'ers: img is an extension which lets Tk understand more graphic file formats) Here's how I got it to work: - took the img1.2.4/ copy from browsex (since I had it lying around) - did "./configure" - PATCHED the Makefile to also link in X11 (see below) (maybe it even works as is, *if* you "package require Tk" first?) (hm, no, in Linux it won't work - X11 is linked in statically) - did "make all" ("all" is needed to make jpg, png, etc) - then launched tclkit: % pwd /home/jcw/img1.2.4 % load ./libimg1.2.so % ls libjpeg/*.jpg libjpeg/: testimg.jpg testimgp.jpg testorig.jpg testprog.jpg % image create photo i -file libjpeg/testimg.jpg i % pack [button .b -image i] % And bingo... I'm seeing a red rose. The above tells me that the build and stubs logic are ok. But you may run into problems trying to embed this stuff into a scripted document. The reason is that "libimg1.2.so" depends on a few other .so's (jpeg, png, etc). The trick is a nasty but common idiom in Tcl: before the "load blah/libimg1.2.so" you must insert lines such as: catch {load blah/libjpeg.so} That will fail, since there is no tcl init setup in there - but it will nevertheless (!) have loaded the shared library (be sure to place those shared libs next to libimg1.2.so). A kludge, but effective... -jcw P.S. The necessary Makefile change is to add X libs to the last line of the snippet shown here: # Directories in which the X11 includes and libraries can be found TK_XINCLUDES = TK_XLIBSW = -L/usr/X11R6/lib -lX11 # Libraries to be included with libimg$(VERSION).so TCL_SHARED_LIBS = -L/usr/lib -ltkstub8.3 -L/usr/lib \ -ltclstub8.3${TCL_DBGX} ${TK_XLIBSW} P.P.S. Yes, adding "img" to Kitten would be a good idea - I'll keep it in mind in the next update round for it. From guenther.fischer@hrz.tu-chemnitz.de Tue Jan 29 08:11:22 2002 Received: from asterix.hrz.tu-chemnitz.de (asterix.hrz.tu-chemnitz.de [134.109.132.84]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TEBL801016 for ; Tue, 29 Jan 2002 08:11:21 -0600 Received: from pat.hrz.tu-chemnitz.de ([134.109.132.143] ident=mail) by asterix.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #1) id 16VYym-0000Rf-00 for metakit@equi4.com; Tue, 29 Jan 2002 15:11:20 +0100 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=root) by pat.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #2) id 16VYym-0008Cn-00 for metakit@equi4.com; Tue, 29 Jan 2002 15:11:20 +0100 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.9.3/8.9.3) with ESMTP id PAA31346 for ; Tue, 29 Jan 2002 15:11:20 +0100 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Tue, 29 Jan 2002 15:11:19 +0100 (MET) From: Guenther Fischer To: metakit@equi4.com Subject: Re: [Metakit] Jpeg images with tclkit In-Reply-To: <20020129125720.21189@triqs.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude, thank you very much for your answer. I hope for the future to see Img in kitten because I need Img for linux and windows. Thank you too for the answer around the iso-8859-1. The tclkit.bin and the tclkit.exe does the needed help. BTW - I had compiled tclkit, metakit etc. from your sources and the characterset problem was fixed for linux. But for now I will take the two tclkit's from the previews area. I tried your linux solution with tclkit and img1.2.4 as described and got: couldn't load file "libimg1.2.so": /afs/tu-chemnitz.de/home/urz/f/fischer/img1.2.4/libimg1.2.so: undefined symbol: log If you see the problem let me know, if not I will wait because the linux solution is only the half I need ... It is a good idea to collect the extensions for tclkit in the kitten library. Perhaps somebody can help to fill the kitten library ... ??? On Tue, 29 Jan 2002, Jean-Claude Wippler wrote: > Guenther Fischer wrote: > > >I tried to use the Img to add jpeg to view in tclkit. I found no way > >to add this package always some errors. > > (For non-Tcl'ers: img is an extension which lets Tk understand more > graphic file formats) > > Here's how I got it to work: > - took the img1.2.4/ copy from browsex (since I had it lying around) > - did "./configure" > - PATCHED the Makefile to also link in X11 (see below) > (maybe it even works as is, *if* you "package require Tk" first?) > (hm, no, in Linux it won't work - X11 is linked in statically) > - did "make all" ("all" is needed to make jpg, png, etc) > - then launched tclkit: > % pwd > /home/jcw/img1.2.4 > % load ./libimg1.2.so > % ls libjpeg/*.jpg > libjpeg/: > testimg.jpg testimgp.jpg testorig.jpg testprog.jpg > % image create photo i -file libjpeg/testimg.jpg > i > % pack [button .b -image i] > % > > And bingo... I'm seeing a red rose. > > The above tells me that the build and stubs logic are ok. But you may > run into problems trying to embed this stuff into a scripted document. > The reason is that "libimg1.2.so" depends on a few other .so's (jpeg, > png, etc). The trick is a nasty but common idiom in Tcl: before the > "load blah/libimg1.2.so" you must insert lines such as: > catch {load blah/libjpeg.so} > That will fail, since there is no tcl init setup in there - but it will > nevertheless (!) have loaded the shared library (be sure to place those > shared libs next to libimg1.2.so). A kludge, but effective... > > -jcw > > P.S. The necessary Makefile change is to add X libs to the last line of > the snippet shown here: > > # Directories in which the X11 includes and libraries can be found > TK_XINCLUDES = > TK_XLIBSW = -L/usr/X11R6/lib -lX11 > > # Libraries to be included with libimg$(VERSION).so > TCL_SHARED_LIBS = -L/usr/lib -ltkstub8.3 -L/usr/lib \ > -ltclstub8.3${TCL_DBGX} ${TK_XLIBSW} > > P.P.S. Yes, adding "img" to Kitten would be a good idea - I'll keep it > in mind in the next update round for it. > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > -- ~Guenther Fischer From tciuro@yahoo.com Tue Jan 29 09:41:32 2002 Received: from web20609.mail.yahoo.com (web20609.mail.yahoo.com [216.136.226.167]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g0TFfW803816 for ; Tue, 29 Jan 2002 09:41:32 -0600 Message-ID: <20020129154127.76964.qmail@web20609.mail.yahoo.com> Received: from [80.26.55.39] by web20609.mail.yahoo.com via HTTP; Tue, 29 Jan 2002 07:41:27 PST Date: Tue, 29 Jan 2002 07:41:27 -0800 (PST) From: Tito Ciuro Subject: Re: [Metakit] MetaKit: Little vs big endian To: Jean-Claude Wippler , metakit@equi4.com In-Reply-To: <20020129102339.19092@triqs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Jean-Claude, This is great news. Serializing is something that I use in order to compact MK data files (since I'm opening the file in commit-extend mode.) Just being curious... is this open issue relatively easy to solve? Thank you, -- Tito --- Jean-Claude Wippler wrote: > Tito Ciuro wrote: > > >What is the status of MetaKit regarding little vs > big > >endian? Is this something that I can count on in > the > >current release? (2.4.2) > > Yes, it works fine. > > With one exception, which I am chasing right now: > there is an issue with > load/save (i.e. seraalizing the datafile out), when > the datafile was > created on a machine with the opposite endianness. > > But for normal opens/commit, it should not cause > problems anywhere. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com From tciuro@yahoo.com Tue Jan 29 09:43:38 2002 Received: from web20606.mail.yahoo.com (web20606.mail.yahoo.com [216.136.226.164]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g0TFhc803951 for ; Tue, 29 Jan 2002 09:43:38 -0600 Message-ID: <20020129154337.93182.qmail@web20606.mail.yahoo.com> Received: from [80.26.55.39] by web20606.mail.yahoo.com via HTTP; Tue, 29 Jan 2002 07:43:37 PST Date: Tue, 29 Jan 2002 07:43:37 -0800 (PST) From: Tito Ciuro Subject: Re: [Metakit] MetaKit: Little vs big endian To: Jacob Levy Cc: metakit@equi4.com In-Reply-To: <3C564862.4080907@best.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jacob, I'm aware that MK is multi-platform, but sometimes there are some issues that may complicate things a bit... :) It's great to hear that you haven't had major issues. Thanks for your reply, -- Tito --- Jacob Levy wrote: > Hi Tito > > I regularly transport Metakit storages between Sparc > and Intel, no > issues so far. I'd be surprised if there were any, > Metakit storages are > supposed to be portable and architecture neutral. > > JYL > > Tito Ciuro wrote: > > >Hello all, > > > >What is the status of MetaKit regarding little vs > big > >endian? Is this something that I can count on in > the > >current release? (2.4.2) > > > >Has anyone tried to access the same MetaKit data > file > >from two machines with different endieness at the > same > >time? Issues anyone? > > > >TIA, > > > >-- Tito > > > >__________________________________________________ > >Do You Yahoo!? > >Great stuff seeking new owners in Yahoo! Auctions! > >http://auctions.yahoo.com > >_______________________________________________ > >metakit mailing list - metakit@equi4.com > >http://www.equi4.com/mailman/listinfo/metakit > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com From jtate@mi-corporation.com Tue Jan 29 10:49:47 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TGnl807316; Tue, 29 Jan 2002 10:49:47 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0TGnff13805; Tue, 29 Jan 2002 11:49:41 -0500 From: "Joseph Tate" To: "Jean-Claude Wippler" , "MetaKit" Subject: RE: [Metakit] Metakit Ported to Windows CE Date: Tue, 29 Jan 2002 11:50:55 -0500 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0010_01C1A8BB.3585B3B0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <20020129103020.19246@triqs.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0010_01C1A8BB.3585B3B0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit > Joseph, thank you very much for doing this - I apologize for not > responding sooner (I had started to several times, but each time thought > of some detail for this port which needed for thought). Not a problem. I feel like I have a pretty good feel now for the inner workings of Metakit. > I'm impressed that you got this far and would very much like to provide > your build on the website for others to download, if you permit. I'll package it up and send it to you. > Also, I'd like to check a few things out w.r.t. the changes you needed to > make o get things working. Ultimately, it would be great if we can merge > the source code changes so WinCE support becomes part of the standard > distribution. Well, I've ported it with that in mind. My metakit tree still compiles and passes the test suite under win2k. It should patch ok. I decided to use Metakit so that I could use the same underlying embedded database structure in an application that I'm porting from Pocket PC to Windows. The Pocket PC architecture provides a limited functionality database system, but it's ram based and subject to corruption if the device crashes or whatever. Plus the database is not available on Windows, so we either needed to have two different database backends, or port Metakit to CE. We opted for the added functionality provided by porting Metakit to CE. The biggest thing with portability (and this may require a change to file format if the datafiles are to be moved accross platforms) is the treatment of strings as two-byte character arrays instead of single byte chars. What I did was created a wince.h header file that defined char as a unsigned short. Then elsewhere in the code where an array of single bytes was needed I used t4_byte and a new type, t4_sbyte, for unsigned and signed single bytes (respectively). So in the code anywhere that had a variable definition of char * mmm that wasn't referring to strings was changed to either t4_byte* or t4_sbyte*. I also #def'd a lot of the str maninip functions to their wide char counterparts. wince.h is attached so that you can see what I've done with it. I'll also include header.h so you can see what changes I made there. Eventually I'd like to see all of the string classes changed so that they're based on CHAR and not char. That way a char remains as defined in the C and C++ specs as a single signed byte. I'm sure it'd be pretty simple to change. First priority was getting it to work, then making it pretty. The other thing I did was surround almost every "string" with the _T("string") construct which casts it to a wide char array. The only place where this was not done was with memo fields. Now, about the string problem in the file format. I'm not familiar with the file format at all. I do know that I had to expressly change the file length when I got to the textend.cpp test: #include "regress.h" #ifndef q4_WCE const int kSize1 = 41; const int kSize2 = 85; #else const int kSize1 = 48; const int kSize2 = 99; #endif An extra 7 bytes per entry. I've not delved into why that is, but I'm pretty sure it has to do with the the multi-byte nature of strings on CE. How can I make this play well? I'm worried that a file created on a non-CE system will be read in with two bytes per character, munging the data. And vice versa, a file created on CE will be read in with null characters between each and every character on non-CE systems. Is there some flag we can set in the data file header? Some kind of bytes per character indicator? > SInce I have a WinCE 2.11 PDA lying around (Cassiopeia E11 - i.e. MIPS), > I could check things and try them out even. But the big Q is how to > build for it: last time I looked into this, there was a free D/L from > Microsoft for the MSVC (5? 6? I have both) environment. Is that the way > to go - or is there a gcc option nowadays. There is a free download from Microsoft called embedded Visual C++. It's available for download from http://www.microsoft.com/mobile/downloads/emvt30.asp . Take note of the CDKey on the page. It uses a unique project workspace format, so you can't open it with Vis Studio. I've not tested the port on anything except the x86 emulator, so your help in testing/debugging would be much appreciated. Well, that's enough for one e-mail. I'll package up my metakit tree and send it in another e-mail. Joseph ------=_NextPart_000_0010_01C1A8BB.3585B3B0 Content-Type: application/octet-stream; name="header.h" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="header.h" // header.h -- // $Id: header.h,v 1.6 2001/12/11 01:59:51 wcvs Exp $ // This is part of MetaKit, the homepage is = http://www.equi4.com/metakit/ /** @file * The internal header included in all source files */ #ifndef __HEADER_H__ #define __HEADER_H__ /////////////////////////////////////////////////////////////////////////= //// #include "config.h" /////////////////////////////////////////////////////////////////////////= //// // A number of preprocessor options are used in the source code // // q4_DOS MS-DOS real-mode OS // q4_MAC Apple Macintosh OS // q4_UNIX Unix, any flavor // q4_VMS DEC OpenVMS OS // q4_WIN Microsoft Windows OS, any flavor // q4_WIN32 Microsoft Windows OS, 32-bit // // q4_MFC Microsoft MFC framework // q4_STD Standard STL version // q4_UNIV Universal version // // q4_BOOL compiler supports bool datatype // q4_CHECK enable assertion checks // q4_FIX manual header fix (see above) // q4_INLINE enable inline expansion // q4_KITDLL compile as DLL (shared library) // q4_MULTI compile for multi-threading // q4_NOLIB do not add automatic lib linkage (MSVC5) // q4_NO_NS don't use namespaces for STL // q4_OK assume all software is perfect // q4_STRICT do not disable any compiler warnings // q4_TINY small version, no floating point // /////////////////////////////////////////////////////////////////////////= //// #define __K4CONF_H__ // skip section in "mk4.h", since we use = "header.h" // if neither MFC nor STD are specified, default to Universal = version #if !q4_MFC && !q4_STD && !defined (q4_UNIV) #define q4_UNIV 1 #endif #if defined (_WIN32_WCE) // Microsoft Windows CE #include "wince.h" #else //#define _T(x) x //Conversion to wchar_t* disabled. #endif /////////////////////////////////////////////////////////////////////////= //// // You can either use '#define q4_xxx 1' to flag the choice of an OS, or // use a '#define d4_OS_H "file.h"' to force inclusion of a header = later. #if defined (__MINGW32__) #define d4_OS_H "win.h" #elif defined (MSDOS) && defined (__GNUC__) #define q4_DOS 1 #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || = defined(_AIX) #define q4_UNIX 1 #elif defined (__VMS) #define q4_VMS 1 #elif defined (macintosh) #define q4_MAC 1 #elif !defined (d4_OS_H) #define d4_OS_H "win.h" #endif /////////////////////////////////////////////////////////////////////////= //// // Use '#define q4_xxx 1' to flag the choice of a CPU. #if defined (_M_I86) || defined (_M_IX86) || defined (i386) #define q4_I86 1 #if defined (_M_I86SM) #define q4_TINY 1 #endif #elif defined (__powerc) #define q4_PPC 1 #elif defined (__alpha) #define q4_AXP 1 #define q4_LONG64 1 #elif defined (__VMS) #define q4_VAX 1 #else #define q4_M68K 1 #endif /////////////////////////////////////////////////////////////////////////= //// // Use '#define q4_xxx 1' to flag the choice of an IDE, and optionally = also // add '#include "file.h"' to force inclusion of a header file right = here. #if defined (__BORLANDC__) // Borland C++ #include "borc.h" #elif defined (__DECCXX) // DEC C++ #define q4_DECC 1 #elif defined (__GNUC__) // GNU C++ #include "gnuc.h" #elif defined (__MWERKS__) // Metrowerks CodeWarrior = C++ #include "mwcw.h" #elif defined (_MSC_VER) // Microsoft Visual C++ #include "msvc.h" #elif defined (__SC__) // Symantec C++ #define q4_SYMC 1 #elif defined (__WATCOMC__) // Watcom C++ #define q4_WATC 1 #endif /////////////////////////////////////////////////////////////////////////= //// // Some of the options take precedence over others #if !q4_BOOL && !q4_STD // define a bool datatype #define false 0 #define true 1 #define bool int #endif #if !q4_CHECK // disable assertions #undef d4_assert #define d4_dbgdef(x) #define d4_assert(x) #endif #if q4_NO_NS // don't use namespaces #define d4_std #else #define d4_std std #endif #if HAVE_MEMMOVE #define d4_memmove(d,s,n) memmove(d,s,n) #elif HAVE_BCOPY #define d4_memmove(d,s,n) bcopy(s,d,n) #else #define d4_memmove f4_memmove extern void f4_memmove(void* d, const void* s, int n); #endif #ifndef q4_WCE typedef unsigned char t4_byte; // create typedefs for t4_byte, etc. typedef char t4_sbyte; // Need both unsigned and signed bytes #endif //defined in wince.h #if SIZEOF_LONG =3D=3D 8 typedef int t4_i32; // longs are 64b, so int must be 32b #else typedef long t4_i32; // longs aren't 64b, so they are 32b #endif /////////////////////////////////////////////////////////////////////////= //// // Include header files which contain additional os/cpu/ide/fw specifics #ifdef d4_OS_H // operating system dependencies #include d4_OS_H #endif /////////////////////////////////////////////////////////////////////////= //// // Several defines should always be set #ifndef d4_assert // assertion macro #include #define d4_assert assert #endif #ifndef d4_dbgdef // conditionally compiled #ifdef NDEBUG #define d4_dbgdef(x) #else #define d4_dbgdef(x) x #endif #endif #ifndef d4_new // heap allocator #define d4_new new #endif #ifndef d4_reentrant // thread-local storage #define d4_reentrant #endif /////////////////////////////////////////////////////////////////////////= //// // Public definitions, plus a few more framework-specific ones #include "mk4.h" #if q4_MFC #include "mfc.h" #elif q4_STD #include "std.h" #elif q4_UNIV #include "univ.h" #endif #ifdef _MSC_VER #pragma warning(disable: 4100 4127 4135 4244 4511 4512 4514) #endif #include /////////////////////////////////////////////////////////////////////////= //// // Report unexpected combinations of settings #if !q4_FIX #if (q4_DOS+q4_MAC+q4_UNIX+q4_VMS+q4_WIN) !=3D 1 #error Exactly one operating system should have been defined #endif #if (q4_MFC+q4_STD+q4_UNIV) !=3D 1 #error Exactly one container library should have been defined #endif #endif =20 /////////////////////////////////////////////////////////////////////////= //// #endif ------=_NextPart_000_0010_01C1A8BB.3585B3B0 Content-Type: application/octet-stream; name="wince.h" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="wince.h" #ifndef __WINCE_H__INCLUDED__ #define __WINCE_H__INCLUDED__ //Moved from header.h typedef unsigned char t4_byte; // create typedefs for t4_byte, etc. typedef char t4_sbyte; // Need both unsigned and signed bytes //Definitions needed elsewhere #include "tchar.h" #include #define q4_CHECK 1 #define d4_assert(x) ASSERT(x) #define d4_dbgdef(x) x //Set up the char handling #defines #define _CHAR_DEFINED 1 #define char unsigned short #define sprintf swprintf #define fprintf fwprintf #define printf wprintf #define strcspn wcscspn #define strchr wcschr #define strlen wcslen #define strcmp wcscmp #define stricmp _wcsicmp #define strrchr wcsrchr #define strpbrk wcspbrk #define strstr wcsstr #define strspn wcsspn #define _strdup _wcsdup #define strcpy wcscpy #define strtol wcstol //file handling routines #define fopen _wfopen #define puts _putws #define fputs fputws #endif // __WINCE_H__INCLUDED__ ------=_NextPart_000_0010_01C1A8BB.3585B3B0-- From jrichards@starband.net Thu Jan 31 08:22:19 2002 Received: from C9Mailgw07.amadis.com ([216.163.188.203]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0VEMJ829661 for ; Thu, 31 Jan 2002 08:22:19 -0600 Received: from c9service11.amadis.com (10.9.0.1) by C9Mailgw07.amadis.com (NPlex 5.5.029) id 3C508FA10005940A for metakit@equi4.com; Thu, 31 Jan 2002 06:18:55 -0800 Received: from localhost (148.63.243.140) by c9service11.amadis.com (NPlex 5.5.042) id 3C4292B200079AFD for metakit@equi4.com; Thu, 31 Jan 2002 06:18:55 -0800 Date: Thu, 31 Jan 2002 09:12:46 -0500 Mime-Version: 1.0 (Apple Message framework v472) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jim Richards To: metakit@equi4.com Content-Transfer-Encoding: 7bit Message-Id: <9983A778-1654-11D6-A822-000393085C6C@starband.net> X-Mailer: Apple Mail (2.472) Subject: [Metakit] Install MetaKit on OS X Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All; I am looking for some guidance on installing MetaKit on OS X 10.1. I have downloaded the binary files libmk4.a.gz and libmk4.dylib.gz as well as metakit.py. I decompressed both lib files, moved them to $PYTHONHOME/lib and renamed them Mk4py.a and Mk4py.dylib. I can import metakit but if I try print metakit.version all I get is an error as follows: AttributeError: 'module' object has no attribute 'version' I am using Python 2.2 Thanks Jim Richards From lvirden@cas.org Thu Jan 31 08:50:29 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0VEoT830835 for ; Thu, 31 Jan 2002 08:50:29 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g0VEoM224359 for ; Thu, 31 Jan 2002 09:50:22 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g0VEoJk21037; Thu, 31 Jan 2002 09:50:19 -0500 (EST) Date: Thu, 31 Jan 2002 09:50:19 -0500 (EST) From: "Larry W. Virden" Message-Id: <20020131095019.AAB21013@cas.org> Subject: Re: [Metakit] Jpeg images with tclkit In-Reply-To: Your message of Tue, 29 Jan 2002 15:11:19 +0100 (MET) To: metakit@equi4.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Guenther Fischer > I tried your linux solution with tclkit and img1.2.4 as described and > got: > couldn't load file "libimg1.2.so": > /afs/tu-chemnitz.de/home/urz/f/fischer/img1.2.4/libimg1.2.so: undefined > symbol: log > > If you see the problem let me know, if not I will wait because the linux > solution is only the half I need ... Add a -lm as the last of the series of libraries used when creating libimg... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Fri Feb 1 10:50:21 2002 Received: from [10.0.1.55] (ipc379eb91.dial.wxs.nl [195.121.235.145]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11GoK809824; Fri, 1 Feb 2002 10:50:20 -0600 From: Jean-Claude Wippler To: metakit , announce Date: Fri, 1 Feb 2002 17:47:51 +0100 Message-Id: <20020201164751.4817@texas.triqs.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There is a new release of MetaKit, "the database that fits in the palm of your hand". Some details about this 2.4.3 release: - sources plus builds for Windows, Linux, and Mac are on the web - various bug fixes, see http://www.equi4.com/metakit/CHANGES - a cross-platform byte-order serialization bug has been fixed - the Python builds are for Python 2.2 (several improvements by GM) - the Tcl builds should run on any Tcl installation >= 8.1 - www.equi4.com now runs on a new server, it appears to be stable The following problems remain, as far as I know: - conversion from 2.0 to 2.4 has been reported to fail, workaround is to serialize the old file to a new one, and then use that - one report of excessive file growth (after doing lots of commits?) A note on where MK stands and where it's going: - I will only build for the above 3 platforms from now on (but of course as many platforms as possible remain supported) - release 2.4.3 is solid, future 2.4.x releases are for bug fixes - one day soon, I'll go to 2.5.x for adding new features to MK See the home page for all further details and downloads: http://www.equi4.com/metakit/ Happy programming :) Regards, Jean-Claude From jgodfrey@optinest.com Fri Feb 1 15:32:53 2002 Received: from linux2.optinest.com (node-20.65.210.128.in-addr.arpa [65.210.128.20] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11LWq001493 for ; Fri, 1 Feb 2002 15:32:52 -0600 Received: from nt40ld (nt4-0ld [65.210.128.93]) by linux2.optinest.com (8.11.3/8.11.3) with SMTP id g11KZVi08878 for ; Fri, 1 Feb 2002 14:35:49 -0600 From: "Jeff Godfrey" To: Date: Fri, 1 Feb 2002 14:46:48 -0600 Message-ID: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_005F_01C1AB2F.46E4B1D0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] metakit and mk4tcl newbie Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_005F_01C1AB2F.46E4B1D0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Hi, I have just started using "metakit" through "mk4tcl". My question is about the "mk::file commit" command and when it should be used. I am writing a simple client server app where the clients will request database activities (retrieve, update, store data) and the server will perform the tasks by accessing a metakit database. I assume that in the event of a problem (hardware failure, server crash, bug, etc) any data that has not been stored using "mk::file commit" will be lost? If this is true, is there any reason I would not want to commit every change immediately after making it (bad form, performance, ??)? Basically I am thinking about doing the following in the server code: - Start server - Open metakit database file - make change - commit change - make change - commit change - make change - commit change - Close metakit database file upon shutting down the server Does this seem logical, or is there a problem with this simple design? Thanks for any input. Jeff Godfrey Optimation USA (800) 523-4744 jgodfrey@optinest.com ------=_NextPart_000_005F_01C1AB2F.46E4B1D0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi,

I have just started using "metakit"=20 through "mk4tcl".  My question is about the "mk::file = commit"=20 command and when it should be used.  I am writing a simple client = server=20 app where the clients will request database activities (retrieve, = update, store=20 data) and the server will perform the tasks by accessing a metakit=20 database.  I assume that in the event of a problem (hardware = failure,=20 server crash, bug, etc) any data that has not been stored using = "mk::file=20 commit" will be lost?  If this is true, is there any reason I=20 would not want to commit every change immediately after making it = (bad=20 form, performance, ??)?
 
Basically I am=20 thinking about doing the following in the server=20 code:
 
- Start=20 server
- Open metakit=20 database file
 
- make=20 change
- commit=20 change
 
- make=20 change
- commit=20 change
 
- make=20 change
- commit=20 change
 
- Close=20 metakit database file upon shutting down the = server
 
Does this seem=20 logical, or is there a problem with this simple design? =20
 
Thanks for any=20 input.
 
Jeff Godfrey
Optimation USA
(800)=20 523-4744
jgodfrey@optinest.com =

 

 
------=_NextPart_000_005F_01C1AB2F.46E4B1D0-- From jcw@equi4.com Fri Feb 1 15:55:28 2002 Received: from [10.0.1.55] (ipc379ed7a.dial.wxs.nl [195.121.237.122]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11LtR002433 for ; Fri, 1 Feb 2002 15:55:27 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] metakit and mk4tcl newbie Date: Fri, 1 Feb 2002 22:53:08 +0100 Message-Id: <20020201215308.26827@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: >I assume that in the event of a problem >(hardware failure, server crash, bug, etc) any data that has not been stored >using "mk::file commit" will be lost? Correct. >If this is true, is there any reason >I would not want to commit every change immediately after making it (bad >form, performance, ??)? Performance. But don't worry about it until you need to. You might find that the performance you need is there no matter how often you commit. >Basically I am thinking about doing the following in the server code: > >- Start server >- Open metakit database file > >- make change >- commit change > >- make change >- commit change > >- make change >- commit change > >- Close metakit database file upon shutting down the server > >Does this seem logical, or is there a problem with this simple design? Looks fine. There can only be one process opening the file. -jcw From genec@goldsmithsinc.com Fri Feb 1 16:01:47 2002 Received: from mail.goldsmithsinc.com ([64.66.89.67]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11M1j002725 for ; Fri, 1 Feb 2002 16:01:46 -0600 Received: from genec (genec [10.1.1.100]) by mail.goldsmithsinc.com (Switch-2.0.0/Switch-2.0.0) with SMTP id g11Lphq14386 for ; Fri, 1 Feb 2002 15:51:43 -0600 Message-ID: <00df01c1ab6b$82bbca40$6401010a@genec> From: "Gene Christian" To: "metakit" Date: Fri, 1 Feb 2002 15:57:58 -0600 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00DC_01C1AB39.38152540" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Subject: [Metakit] 2.4.3 and python 2.1? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_00DC_01C1AB39.38152540 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Is it still possible to build version 2.4.3 for python 2.1 or are some = of the changes specific to the 2.2 version? If it is possible, has = anybody done it? On a related note, would the MS Visual C++ 6.0 standard edition be the = right choice for compiling python extensions? I feel bad asking about = Windows binaries, and would rather be able to produce them myself. The = standard edition isn't that expensive (under $100.00), but if it takes = the professional edition then I'll have to skip it. Thanks! Gene Christian ------=_NextPart_000_00DC_01C1AB39.38152540 Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Is it still possible to build version 2.4.3 for = python 2.1 or=20 are some of the changes specific to the 2.2 version?  If it is possible, has anybody done it?
 
On a related note, would the MS Visual C++ 6.0 = standard=20 edition be the right choice for compiling python extensions?  I = feel bad=20 asking about Windows binaries, and would rather be able to produce them=20 myself.  The standard edition isn't that expensive (under $100.00), = but if=20 it takes the professional edition then I'll have to skip = it.
 
Thanks!
 
Gene Christian
------=_NextPart_000_00DC_01C1AB39.38152540-- From jcw@equi4.com Fri Feb 1 16:10:53 2002 Received: from [10.0.1.55] (ipc379e99a.dial.wxs.nl [195.121.233.154]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11MAq003110 for ; Fri, 1 Feb 2002 16:10:53 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] 2.4.3 and python 2.1? Date: Fri, 1 Feb 2002 23:08:30 +0100 Message-Id: <20020201220830.30136@triqs.com> In-Reply-To: <00df01c1ab6b$82bbca40$6401010a@genec> References: <00df01c1ab6b$82bbca40$6401010a@genec> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gene Christian wrote: >Is it still possible to build version 2.4.3 for python 2.1 or are some of >the changes specific to the 2.2 version? If it is possible, has anybody >done it? No problem at all. I don't have a build for you, though :( The easy way out is to stick to 2.4.2 ... >On a related note, would the MS Visual C++ 6.0 standard edition be the >right choice for compiling python extensions? I feel bad asking about >Windows binaries, and would rather be able to produce them myself. The >standard edition isn't that expensive (under $100.00), but if it takes >the professional edition then I'll have to skip it. I'm not sure how "crippled" the std ed is. I build with 6 all the time. having said that: with 2.4.2, I also built with MinGW. Even cross platform, i.e. from Linux generating Win binaries. One suggestion could be to give MinGW a shot, it's good stuff. -jcw From juneaftn@hananet.net Sat Feb 2 01:52:40 2002 Received: from mo02.hananet.net ([211.202.13.165]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g127qd011353 for ; Sat, 2 Feb 2002 01:52:39 -0600 Received: from [211.201.213.181] by (Terrace internet messaging server 3.0 (for Hananet)) with ESMTP id 2002020216:52:32:301091.303.253 for ; Sat, 02 Feb 2002 16:52:32 +0900 (KST) Message-ID: <008001c1abbe$74bd8d40$b5d5c9d3@hananet.net> From: "Changjune Kim" To: Date: Sat, 2 Feb 2002 16:51:42 +0900 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] MetaKit 2.4.3 Storage Locking Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, First of all, thank you for this wonderful software. but, I have some problem here. As I read in the documentation, it says MetaKit has a simple locking model for "Storage." When opening(or creating) a file you can choose read-only, or read-write mode. It doesn't seem to work on win32 (esp. Win 98SE). Isn't it suppposed to raise an error or block when you try to open the same file with read-write mode again in a different process? Thanks in advance. June From dgporter@erols.com Sat Feb 2 19:09:39 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1319c017614 for ; Sat, 2 Feb 2002 19:09:38 -0600 Received: from 66-44-7-242.s2020.apx1.lnh.md.dialup.rcn.com ([66.44.7.242] helo=there) by smtp01.mrf.mail.rcn.net with smtp (Exim 3.33 #10) id 16XB9w-0000n6-00 for metakit@equi4.com; Sat, 02 Feb 2002 20:09:33 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Don Porter Reply-To: dgporter@erols.com To: metakit@equi4.com Date: Sat, 2 Feb 2002 20:07:58 -0500 X-Mailer: KMail [version 1.3.1] References: <20020201164751.4817@texas.triqs.com> In-Reply-To: <20020201164751.4817@texas.triqs.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: Subject: [Metakit] Mk4Tcl concurrency Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, The 2.4.3 release seems like a good time to give Metakit another try. First let me contribute something. If others are using the very latest HEAD version of Tcl from CVS like I am, they should add the flag -DUSE_NON_CONST to the CXX_SWITCHES_TCL definition in the Metakit Makefile in order to get Mk4tcl to compile. I apologize in advance if I'm asking newbie questions, but it appears to me that most of the documentation and FAQs I find are for version 1.8.5 of Metakit, so I don't trust them to have any relevance to the current release. I'll be using Metakit through its Tcl binding. The documentation suggests that commands like [mk::select] offer only a synchronous, blocking interface. I can imagine that this command might take a perceptible length of time to complete, while blocking screen redraws of Tk, or service of network connections. Am I correct that the only solution available is to use multiple threads? Assuming so, it seems it would be a straightforward matter to write a Tcl package that wraps around Mk4Tcl, and makes use of the Tcl Thread package to create asynchronous callback forms of the Metakit commands. I think that would be preferred to pushing thread management issues onto all users of Mk4Tcl. Has this already been done? If not, are others interested in this idea, or does Mk4tcl offer other solutions for this problem that I don't know about. Also, which Mk4Tcl commands are the ones where blocking time is a problem? -- | Don Porter dgporter@erols.com | | "Some days you just can't get rid of a bomb!" | | -- Adam West as BATMAN | |______________________________________________________________________| From gmcm@hypernet.com Sat Feb 2 20:13:45 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g132Dj018060 for ; Sat, 2 Feb 2002 20:13:45 -0600 Received: from PARANOIA (204.176.40.97) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Sat, 2 Feb 2002 21:13:46 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Sat, 2 Feb 2002 21:13:55 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Mk4Tcl concurrency Reply-to: gmcm@hypernet.com Message-ID: <3C5C5693.32590.64C1AB2A@localhost> Priority: normal References: <20020201164751.4817@texas.triqs.com> In-reply-to: X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 2 Feb 2002 at 20:07, Don Porter wrote: > I'll be using Metakit through its Tcl binding. The > documentation suggests that commands like > [mk::select] offer only a synchronous, blocking > interface. I can imagine that this command might > take a perceptible length of time to complete, while > blocking screen redraws of Tk, or service of network > connections. Am I correct that the only solution > available is to use multiple threads? Can't speak to Mk4tcl, but as an Mk4py user, I would say that for a typical GUI app: - you don't have to think about if your db is in the small Ks of rows - it require some thought in the <100K range - after that, you might have to consider your algorithms seriously. YMMV, but I've never had to dedicate a thread to Mk yet. -- Gordon http://www.mcmillan-inc.com/ From mark@markroseman.com Sun Feb 3 16:03:25 2002 Received: from admin.cgocable.net (admin.cgocable.net [24.226.1.21]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g13M3Pa01410 for ; Sun, 3 Feb 2002 16:03:25 -0600 Received: from eyak-can4.markroseman.com (d141-164-68.home.cgocable.net [24.141.164.68]) by admin.cgocable.net (8.10.0/8.9.3) with ESMTP id g13IKmn17166 for ; Sun, 3 Feb 2002 13:22:15 -0500 (EST) Message-Id: <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> X-Sender: X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Sun, 03 Feb 2002 13:15:25 -0500 To: metakit@equi4.com From: Mark Roseman Subject: Re: [Metakit] Mk4Tcl concurrency In-Reply-To: References: <20020201164751.4817@texas.triqs.com> <20020201164751.4817@texas.triqs.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: As Gordon suggested, for most small-ish datasets, the time to complete most any operation in Metakit is very quick. For the odd time when you are doing operations that will cause issues, yes, running in a thread or another process may be useful. However, I'd strongly suggest not dealing with it until the point you know its an issue (whether through practice, or experimentation ahead of time). As an example, our web-based app runs under AOLserver, and we keep a lot of central data held in Metakit databases. The multiple AOLserver threads make socket connections to a separate single-threaded process running the database, so in effect requests have to wait their turn to be processed. Despite some fairly substantial queries and reasonable numbers of simultaneous users, we've never had issues with hugely perceptible delays on the client side. Mark Roseman http://www.markroseman.com From jyl@best.com Sun Feb 3 16:57:47 2002 Received: from sj1-3-4-18.securesites.net (sj1-3-4-18.securesites.net [192.220.127.211]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g13Mvla02086 for ; Sun, 3 Feb 2002 16:57:47 -0600 Received: (qmail 19362 invoked by uid 19667); 3 Feb 2002 17:31:02 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 3 Feb 2002 17:31:02 -0000 Message-ID: <3C5D8360.3020506@best.com> Date: Sun, 03 Feb 2002 10:37:20 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] [ANNOUNCE] e4Graph 1.0a5 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: WHERE TO GET: ============= File releases: http://sourceforge.net/projects/e4graph/ Documentation: http://e4graph.sourceforge.net/ Change log: http://e4graph.sourceforge.net/changes.txt WHAT IS IT: =========== e4Graph is a C++ and Tcl library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. CHANGES: ======== For all changes, see: http://e4graph.sourceforge.net/changes.txt Major changes: Markers are no longer supported as a storage entity; instead, each storage has a distinguished root node. Nodes and vertices are no longer explicitly deleted from a storage; instead, they are detached from their containing entity or entities, and when the user program no longer has a reference to the detached node or vertex, the resources associated with this entity are automatically reclaimed. DEPENDENCIES: ============= This release requires Metakit 2.4.3, Tcl 8.3 and optionally Expat 1.95.2. From dgporter@erols.com Sun Feb 3 17:45:28 2002 Received: from smtp03.mrf.mail.rcn.net (smtp03.mrf.mail.rcn.net [207.172.4.62]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g13NjRa02675 for ; Sun, 3 Feb 2002 17:45:27 -0600 Received: from 66-44-18-88.s2120.apx2.lnh.md.dialup.rcn.com ([66.44.18.88] helo=there) by smtp03.mrf.mail.rcn.net with smtp (Exim 3.33 #10) id 16XS4o-0000LE-00 for metakit@equi4.com; Sun, 03 Feb 2002 14:13:23 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Don Porter Reply-To: dgporter@erols.com To: metakit@equi4.com Date: Sun, 3 Feb 2002 14:11:51 -0500 X-Mailer: KMail [version 1.3.1] References: <20020201164751.4817@texas.triqs.com> <3C5C5693.32590.64C1AB2A@localhost> In-Reply-To: <3C5C5693.32590.64C1AB2A@localhost> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: Subject: [Metakit] Patches? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is this the right place to offer patches for MetaKit? I have a patch that corrects Mk4tcl's failure to export its public commands. Where do I send it? -- | Don Porter dgporter@erols.com | | "Some days you just can't get rid of a bomb!" | | -- Adam West as BATMAN | |______________________________________________________________________| From nicolasb@maich.gr Mon Feb 4 03:01:04 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g14913a06957 for ; Mon, 4 Feb 2002 03:01:03 -0600 Received: FROM maich.gr BY mail-server ; Mon Feb 04 11:19:21 2002 +0200 Message-ID: <3C5E5268.C21291E3@maich.gr> Date: Mon, 04 Feb 2002 11:20:40 +0200 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: Mark Roseman CC: metakit@equi4.com Subject: Re: [Metakit] Mk4Tcl concurrency References: <20020201164751.4817@texas.triqs.com> <20020201164751.4817@texas.triqs.com> <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Is your metakit db driver for aolserver (and api?) something that could be made available to the public. We currently use tclhttpd, but would like to migrate some stuff to aolserver... tia, nicolas boretos Mark Roseman wrote: > > As Gordon suggested, for most small-ish datasets, the time to > complete most any operation in Metakit is very quick. For the > odd time when you are doing operations that will cause issues, > yes, running in a thread or another process may be useful. > However, I'd strongly suggest not dealing with it until the > point you know its an issue (whether through practice, or > experimentation ahead of time). > > As an example, our web-based app runs under AOLserver, and we > keep a lot of central data held in Metakit databases. The > multiple AOLserver threads make socket connections to a separate > single-threaded process running the database, so in effect > requests have to wait their turn to be processed. Despite some > fairly substantial queries and reasonable numbers of simultaneous > users, we've never had issues with hugely perceptible delays on > the client side. > > Mark Roseman > http://www.markroseman.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From mark@markroseman.com Mon Feb 4 12:54:58 2002 Received: from admin.cgocable.net (admin.cgocable.net [24.226.1.21]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g14Iswa29157 for ; Mon, 4 Feb 2002 12:54:58 -0600 Received: from eyak-can4.markroseman.com (d141-164-68.home.cgocable.net [24.141.164.68]) by admin.cgocable.net (8.10.0/8.9.3) with ESMTP id g14IvBn28443 for ; Mon, 4 Feb 2002 13:57:14 -0500 (EST) Message-Id: <5.1.0.14.0.20020204135036.00ac7768@roseman.myservers.org> X-Sender: X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 04 Feb 2002 13:51:42 -0500 To: metakit@equi4.com From: Mark Roseman Subject: Re: [Metakit] Mk4Tcl concurrency In-Reply-To: <3C5E5268.C21291E3@maich.gr> References: <20020201164751.4817@texas.triqs.com> <20020201164751.4817@texas.triqs.com> <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We didn't do a generic database driver for Metakit, but instead did something thats fairly application specific. So no, its not available. Is your metakit db driver for aolserver (and api?) something that could >be made available to the public. >We currently use tclhttpd, but would like to migrate some stuff to >aolserver... Mark Roseman http://www.markroseman.com From genec@goldsmithsinc.com Mon Feb 4 13:01:41 2002 Received: from mail.goldsmithsinc.com ([64.66.89.67]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g14J1ea29504 for ; Mon, 4 Feb 2002 13:01:40 -0600 Received: from genec (genec [10.1.1.100]) by mail.goldsmithsinc.com (Switch-2.0.0/Switch-2.0.0) with SMTP id g14IpVq29320 for ; Mon, 4 Feb 2002 12:51:31 -0600 Message-ID: <015201c1adad$d6814cb0$6401010a@genec> From: "Gene Christian" To: "metakit" References: <00df01c1ab6b$82bbca40$6401010a@genec> <20020201220830.30136@triqs.com> Subject: Re: [Metakit] 2.4.3 and python 2.1? Date: Mon, 4 Feb 2002 12:57:48 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > I'm not sure how "crippled" the std ed is. I build with 6 all the time. > having said that: with 2.4.2, I also built with MinGW. Even cross > platform, i.e. from Linux generating Win binaries. > > One suggestion could be to give MinGW a shot, it's good stuff. Thanks for the tip. It took a little while to get set up, but after a little experimentation MinGW worked just fine. Gene. From wlb@multisysid.com Tue Feb 5 17:26:01 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g15NQ1a24348 for ; Tue, 5 Feb 2002 17:26:01 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id TAA19501 for ; Tue, 5 Feb 2002 19:24:45 -0500 Message-ID: <000d01c1ae9d$a9e41480$6101a8c0@WB02> From: "wlb" To: Date: Tue, 5 Feb 2002 19:34:32 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Porting to Windows CE Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I spent the day porting to Windows CE before reading the archives to see that someone else has made some good progress. daaahhhh. But it looks like it was not a waste of time at all. The previous port to WCE looks to be H/PC compatible. My port focuses on PocketPC, which is mostly a subset of H/PC. I had to deal with the problem of no _osfhandle() to convert from a FILE * to a HANDLE, so I had to replace the FILE *'s with HANDLE's, fopen()'s with CreateFile()'s, etc. If you have a solution to this problem, let me know. These changes are very compatible not only with H/PC, but all versions of Windows. I created a new config variable, q4_WCE. I did not convert text to Unicode. I'm still not 100% sure that that is what I want to do, but then again, it does indeed sound like a good idea. If you can merge the for CE port soon, I would love to add my diffs to that version / release. So far, here are my regression results. It still has a few problems, but close indeed. Extend fails for me also. That's high on my list to understand better. wlb b00 - Should fail *** Failed: A(false) *** <<< done. b01 - Should succeed <<< done. b02 - Int property <<< done. b03 - Float property <<< done. b04 - String property <<< done. b05 - View property <<< done. b06 - View construction <<< done. b07 - Row manipulation <<< done. b08 - Row expressions <<< done. b09 - View manipulation <<< done. b10 - View sorting <<< done. b11 - View selection <<< done. b12 - Add after remove <<< done. b13 - Clear view entry <<< done. b14 - Empty view outlives temp storage <<< done. b15 - View outlives temp storage <<< done. b16 - View outlives cleared temp storage <<< done. b17 - Double property <<< done. b18 - SetAtGrow usage <<< done. b19 - Bytes property <<< done. b20 - Search sorted view <<< done. b21 - Memo property <<< done. b22 - Stored view references <<< done. b23 - Sort comparison fix <<< done. b24 - Custom view comparisons <<< done. b25 - Copy row from derived <<< done. b26 - Partial memo field access <<< done. b27 - Copy value to another row <<< done. n01 - Add to selection <<< done. n02 - Remove from selection <<< done. n03 - Modify into selection <<< done. n04 - Modify out of selection <<< done. n05 - Add to sorted <<< done. n06 - Remove from sorted <<< done. n07 - New property through sort <<< done. n08 - Nested project and select <<< done. n09 - Multiple dependencies <<< done. n10 - Modify sorted duplicates <<< done. n11 - Resize compound derived view <<< done. n12 - Alter multiply derived view <<< done. n13 - Project without <<< done. c01 - Slice forward <<< done. c02 - Slice backward <<< done. c03 - Slice reverse <<< done. c04 - Cartesian product <<< done. c05 - Remapping <<< done. c06 - Pairwise combination <<< done. c07 - Concatenate views <<< done. c08 - Rename property <<< done. c09 - GroupBy operation *** Failed: A(false) *** <<< done. c10 - Counts operation *** Failed: A(false) *** <<< done. c11 - Unique operation <<< done. c12 - Union operation <<< done. c13 - Intersect operation <<< done. c14 - Different operation <<< done. c15 - Minus operation <<< done. c16 - View comparisons <<< done. c17 - Join operation *** Failed: A(false) *** <<< done. c18 - Groupby sort fix <<< done. c19 - JoinProp operation <<< done. c20 - Wide cartesian product <<< done. c21 - Join on compound key <<< done. r00 - Simple insert <<< done. r01 - Simple removes <<< done. r02 - Large inserts and removes <<< done. r03 - Binary property insertions <<< done. r04 - Scripted string property tests <<< done. s00 - Simple storage <<< done. s01 - Integer storage <<< done. s02 - Float storage <<< done. s03 - String storage <<< done. s04 - View storage <<< done. s05 - Store and reload <<< done. s06 - Commit twice <<< done. s07 - Commit modified <<< done. s08 - View after storage <<< done. s09 - Copy storage <<< done. s10 - Stream storage *** Failed: A(v1.GetSize() == 3) *** *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(v3.GetSize() == 3) *** *** Failed: A(v4.GetSize() == 2) *** *** Failed: A(v1.GetSize() == 3) *** *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(v3.GetSize() == 3) *** *** Failed: A(v4.GetSize() == 2) *** *** Failed: A(v1.GetSize() == 3) *** *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(v3.GetSize() == 3) *** *** Failed: A(v4.GetSize() == 2) *** *** Failed: A(v1.GetSize() == 4) *** *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(v3.GetSize() == 3) *** *** Failed: A(v4.GetSize() == 2) *** <<< done. s11 - Commit and rollback <<< done. s12 - Remove subview <<< done. s13 - Remove middle subview <<< done. s14 - Replace attached subview <<< done. s15 - Add after removed subviews <<< done. s16 - Add after removed ints <<< done. s17 - Add after removed strings <<< done. s18 - Empty storage <<< done. s19 - Empty view outlives storage <<< done. s20 - View outlives storage <<< done. s21 - Test demo scenario <<< done. s22 - Double storage <<< done. s23 - Find absent record <<< done. s24 - Bitwise storage <<< done. s25 - Bytes storage <<< done. s26 - Bitwise autosizing <<< done. s27 - Bytes restructuring <<< done. s28 - Doubles added later <<< done. s29 - Delete bytes property <<< done. d01 - Commit aside <<< done. e01 - Extend new file <<< done. e02 - Extend committing twice <<< done. e03 - Read during extend *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(p1 (v2[0]) == 123) *** *** Failed: A(v3.GetSize() == 2) *** *** Failed: A(p1 (v3[0]) == 123) *** *** Failed: A(p1 (v3[1]) == 456) *** <<< done. e04 - Extend during read *** Failed: A(false) *** <<< done. e05 - Test memory mapping <<< done. e06 - Rollback during extend *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(p1 (v2[0]) == 123) *** *** Failed: A(v2a.GetSize() == 1) *** *** Failed: A(p1 (v2a[0]) == 123) *** <<< done. f01 - Add view to format <<< done. f02 - Remove view from format <<< done. f03 - Rollback format change <<< done. f04 - Rearrange format <<< done. f05 - Nested reformat <<< done. f06 - Flip foreign data <<< done. f07 - Automatic structure info (obsolete) <<< done. f08 - Automatic storage format <<< done. f09 - Partial restructuring <<< done. f10 - Committed restructuring <<< done. f11 - Delete missing view <<< done. m01 - Hash mapping <<< done. m02 - Blocked view bug <<< done. m03 - Hash adds <<< done. l00 - Lots of properties *** Failed: A(false) *** <<< done. l01 - Over 32 Kb of integers <<< done. l02 - Over 64 Kb of strings <<< done. l03 - Force sections in storage *** Failed: A(false) *** <<< done. l04 - Modify sections in storage <<< done. l05 - Delete from 32 Kb of strings <<< done. l06 - Bit field manipulations <<< done. l07 - Huge description *** Failed: A(false) *** <<< done. From wlb@multisysid.com Wed Feb 6 08:04:19 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16E4Ia12753 for ; Wed, 6 Feb 2002 08:04:18 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id KAA21981 for ; Wed, 6 Feb 2002 10:03:04 -0500 Message-ID: <000501c1af18$5e3c71a0$6101a8c0@WB02> From: "wlb" To: Date: Wed, 6 Feb 2002 10:12:52 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] WCE port update Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: My WCE port now passes all regression tests....well, I didn't actually run TestStore4 since I still need to #if the fopen()'s and replace with CreateFile()'s for q4_WCE. Previous tests failed because I was being too stingy with the FILE_SHARE_READ and FILE_SHARE_WRITE flags. Together with a couple of errors in GENERIC_READ and GENERIC_WRITE flags in the regression tests. When files cannot be opened with the requested permissions catastrophic failures result in the most unexpected places. I'll look at the code some more and see if I can't detect those problems before something explodes. Mine port is still ASCII, with no testing using MFC or Unicode. Question to the the previous WCE porter: What syntax did you use for Unicode constants? _T("text"), _("text"), L"text", MYMACRO("text") ? I would like to be consistant with you. wlb From jtate@mi-corporation.com Wed Feb 6 09:28:22 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16FSKa15933 for ; Wed, 6 Feb 2002 09:28:22 -0600 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g16FSAf24227; Wed, 6 Feb 2002 10:28:10 -0500 From: "Joseph Tate" To: "wlb" , Subject: RE: [Metakit] WCE port update Date: Wed, 6 Feb 2002 10:29:37 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <000501c1af18$5e3c71a0$6101a8c0@WB02> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I used _T(""). I'll send you my project files so that you can take a look at them. I couldn't post them to the list because they were too big. Joseph > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of wlb > Sent: Wednesday, February 06, 2002 9:13 AM > To: metakit@equi4.com > Subject: [Metakit] WCE port update > > > > My WCE port now passes all regression tests....well, I > didn't actually run TestStore4 since I still need to #if > the fopen()'s and replace with CreateFile()'s for q4_WCE. > > Previous tests failed because I was being too stingy with the > FILE_SHARE_READ and FILE_SHARE_WRITE flags. > Together with a couple of errors in GENERIC_READ and > GENERIC_WRITE flags in the regression tests. > > When files cannot be opened with the requested permissions > catastrophic failures result in the most unexpected places. > I'll look at the code some more and see if I can't detect those > problems before something explodes. > > Mine port is still ASCII, with no testing using MFC or Unicode. > > Question to the the previous WCE porter: What syntax > did you use for Unicode constants? _T("text"), _("text"), L"text", > MYMACRO("text") ? > > I would like to be consistant with you. > > wlb > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jtate@mi-corporation.com Wed Feb 6 09:57:04 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16Fv4a17362 for ; Wed, 6 Feb 2002 09:57:04 -0600 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g16Fuvf24415; Wed, 6 Feb 2002 10:56:57 -0500 From: "Joseph Tate" To: "wlb" , Subject: RE: [Metakit] Porting to Windows CE Date: Wed, 6 Feb 2002 10:58:24 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <000d01c1ae9d$a9e41480$6101a8c0@WB02> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: If you take a look at the header files that I posted to the list on Jan 29 you can see how I handled the file creation/handle problem. I didn't have to replace all FILE* items with HANDLEs. See the following snippet from fileio.cpp: #ifndef q4_WCE FlushFileBuffers((HANDLE) _get_osfhandle(_fileno(_file))); HANDLE h = ::CreateFileMapping((HANDLE) _get_osfhandle(_fileno(_file)), 0, PAGE_READONLY, 0, len, 0); #else FlushFileBuffers((HANDLE) _fileno(_file)); HANDLE h = ::CreateFileMapping((HANDLE) _fileno(_file), 0, PAGE_READONLY, 0, len, 0); #endif //q4_WCE It turns out that _fileno returns a handle. You just have to cast it. I couldn't find an acceptable replacement for freopen, but that was only used for creating the output files during testing. It passes all the regression tests, even teststore4. The issue that my port has is that everything is w_char rather than char, which is a serious issue because if a file is created on CE it may not open under !CE. It definitely needs more work, but as of right now, my resources have been redirected elsewhere. I'll be getting back to work on this in a week or two. Good luck. Joseph > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of wlb > Sent: Tuesday, February 05, 2002 6:35 PM > To: metakit@equi4.com > Subject: [Metakit] Porting to Windows CE > > > I spent the day porting to Windows CE before reading the > archives to see that someone else has made some good > progress. > > daaahhhh. > > But it looks like it was not a waste of time at all. The previous > port to WCE looks to be H/PC compatible. My port focuses > on PocketPC, which is mostly a subset of H/PC. I had to > deal with the problem of no _osfhandle() to convert from a > FILE * to a HANDLE, so I had to replace the FILE *'s > with HANDLE's, fopen()'s with CreateFile()'s, etc. If you > have a solution to this problem, let me know. > > These changes are very compatible not only with H/PC, but > all versions of Windows. > > I created a new config variable, q4_WCE. > > I did not convert text to Unicode. I'm still not 100% sure that > that is what I want to do, but then again, it does indeed sound > like a good idea. > > If you can merge the for CE port soon, I would love to add my > diffs to that version / release. > > So far, here are my regression results. It still has a few problems, > but close indeed. > > Extend fails for me also. That's high on my list to understand better. > > wlb > > b00 - Should fail *** Failed: A(false) *** > <<< done. > b01 - Should succeed <<< done. > b02 - Int property <<< done. > b03 - Float property <<< done. > b04 - String property <<< done. > b05 - View property <<< done. > b06 - View construction <<< done. > b07 - Row manipulation <<< done. > b08 - Row expressions <<< done. > b09 - View manipulation <<< done. > b10 - View sorting <<< done. > b11 - View selection <<< done. > b12 - Add after remove <<< done. > b13 - Clear view entry <<< done. > b14 - Empty view outlives temp storage <<< done. > b15 - View outlives temp storage <<< done. > b16 - View outlives cleared temp storage <<< done. > b17 - Double property <<< done. > b18 - SetAtGrow usage <<< done. > b19 - Bytes property <<< done. > b20 - Search sorted view <<< done. > b21 - Memo property <<< done. > b22 - Stored view references <<< done. > b23 - Sort comparison fix <<< done. > b24 - Custom view comparisons <<< done. > b25 - Copy row from derived <<< done. > b26 - Partial memo field access <<< done. > b27 - Copy value to another row <<< done. > n01 - Add to selection <<< done. > n02 - Remove from selection <<< done. > n03 - Modify into selection <<< done. > n04 - Modify out of selection <<< done. > n05 - Add to sorted <<< done. > n06 - Remove from sorted <<< done. > n07 - New property through sort <<< done. > n08 - Nested project and select <<< done. > n09 - Multiple dependencies <<< done. > n10 - Modify sorted duplicates <<< done. > n11 - Resize compound derived view <<< done. > n12 - Alter multiply derived view <<< done. > n13 - Project without <<< done. > c01 - Slice forward <<< done. > c02 - Slice backward <<< done. > c03 - Slice reverse <<< done. > c04 - Cartesian product <<< done. > c05 - Remapping <<< done. > c06 - Pairwise combination <<< done. > c07 - Concatenate views <<< done. > c08 - Rename property <<< done. > c09 - GroupBy operation *** Failed: A(false) *** > <<< done. > c10 - Counts operation *** Failed: A(false) *** > <<< done. > c11 - Unique operation <<< done. > c12 - Union operation <<< done. > c13 - Intersect operation <<< done. > c14 - Different operation <<< done. > c15 - Minus operation <<< done. > c16 - View comparisons <<< done. > c17 - Join operation *** Failed: A(false) *** > <<< done. > c18 - Groupby sort fix <<< done. > c19 - JoinProp operation <<< done. > c20 - Wide cartesian product <<< done. > c21 - Join on compound key <<< done. > r00 - Simple insert <<< done. > r01 - Simple removes <<< done. > r02 - Large inserts and removes <<< done. > r03 - Binary property insertions <<< done. > r04 - Scripted string property tests <<< done. > s00 - Simple storage <<< done. > s01 - Integer storage <<< done. > s02 - Float storage <<< done. > s03 - String storage <<< done. > s04 - View storage <<< done. > s05 - Store and reload <<< done. > s06 - Commit twice <<< done. > s07 - Commit modified <<< done. > s08 - View after storage <<< done. > s09 - Copy storage <<< done. > s10 - Stream storage *** Failed: A(v1.GetSize() == 3) *** > *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(v3.GetSize() == 3) *** > *** Failed: A(v4.GetSize() == 2) *** > *** Failed: A(v1.GetSize() == 3) *** > *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(v3.GetSize() == 3) *** > *** Failed: A(v4.GetSize() == 2) *** > *** Failed: A(v1.GetSize() == 3) *** > *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(v3.GetSize() == 3) *** > *** Failed: A(v4.GetSize() == 2) *** > *** Failed: A(v1.GetSize() == 4) *** > *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(v3.GetSize() == 3) *** > *** Failed: A(v4.GetSize() == 2) *** > <<< done. > s11 - Commit and rollback <<< done. > s12 - Remove subview <<< done. > s13 - Remove middle subview <<< done. > s14 - Replace attached subview <<< done. > s15 - Add after removed subviews <<< done. > s16 - Add after removed ints <<< done. > s17 - Add after removed strings <<< done. > s18 - Empty storage <<< done. > s19 - Empty view outlives storage <<< done. > s20 - View outlives storage <<< done. > s21 - Test demo scenario <<< done. > s22 - Double storage <<< done. > s23 - Find absent record <<< done. > s24 - Bitwise storage <<< done. > s25 - Bytes storage <<< done. > s26 - Bitwise autosizing <<< done. > s27 - Bytes restructuring <<< done. > s28 - Doubles added later <<< done. > s29 - Delete bytes property <<< done. > d01 - Commit aside <<< done. > e01 - Extend new file <<< done. > e02 - Extend committing twice <<< done. > e03 - Read during extend *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(p1 (v2[0]) == 123) *** > *** Failed: A(v3.GetSize() == 2) *** > *** Failed: A(p1 (v3[0]) == 123) *** > *** Failed: A(p1 (v3[1]) == 456) *** > <<< done. > e04 - Extend during read *** Failed: A(false) *** > <<< done. > e05 - Test memory mapping <<< done. > e06 - Rollback during extend *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(p1 (v2[0]) == 123) *** > *** Failed: A(v2a.GetSize() == 1) *** > *** Failed: A(p1 (v2a[0]) == 123) *** > <<< done. > f01 - Add view to format <<< done. > f02 - Remove view from format <<< done. > f03 - Rollback format change <<< done. > f04 - Rearrange format <<< done. > f05 - Nested reformat <<< done. > f06 - Flip foreign data <<< done. > f07 - Automatic structure info (obsolete) <<< done. > f08 - Automatic storage format <<< done. > f09 - Partial restructuring <<< done. > f10 - Committed restructuring <<< done. > f11 - Delete missing view <<< done. > m01 - Hash mapping <<< done. > m02 - Blocked view bug <<< done. > m03 - Hash adds <<< done. > l00 - Lots of properties *** Failed: A(false) *** > <<< done. > l01 - Over 32 Kb of integers <<< done. > l02 - Over 64 Kb of strings <<< done. > l03 - Force sections in storage *** Failed: A(false) *** > <<< done. > l04 - Modify sections in storage <<< done. > l05 - Delete from 32 Kb of strings <<< done. > l06 - Bit field manipulations <<< done. > l07 - Huge description *** Failed: A(false) *** > <<< done. > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From wlb@multisysid.com Wed Feb 6 10:37:10 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16Gb9a19952 for ; Wed, 6 Feb 2002 10:37:09 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id MAA23341 for ; Wed, 6 Feb 2002 12:35:55 -0500 Message-ID: <001b01c1af2d$b92df330$6101a8c0@WB02> From: "wlb" To: Date: Wed, 6 Feb 2002 12:45:45 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Porting to Windows CE Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > It turns out that _fileno returns a handle. You just have to cast it. Well I'll be. Hats off to you. It also appears you have spent more time on this than I. At least for now, I do have time and energy to dedicate to this port. I understand the problem of portability of data files to non-Unicode platforms. I would think that portability would require a library which simultaneously supported unicode and non-unicode methods, as well as an indicator in the data file header. I guess that last statement is not 100% correct: you could have two DLL's, one with Unicode and the other with ASCII. I prefer one DLL. One DLL which handles both cases would preclude the use of (the MicroSoft standard) _tcscmp and their other _t??? functions. (Thinking aloud.) Other than the c4_String class, there are few uses of char *'s. Most of the occurrances are in naming properties, descriptions, and the storage format string used in GetAs( ). c4_String. Huh. I have noticed that CE's MFC-CString can be assigned and ASCII value directly without first converting to Unicode. I haven't checked to see if he stores it ASCII or silently converts to Unicode. If I'm not mistaken, there is something in the data file header which indicates big-endian / little-endian. Maybe there are a few more flags or values in that area that could be useful. I'll dig some more unless jcw shines some light on it. I'm very interested in knowning what jcw thinks would be the correct method to integrate Unicode. I'm also interested in knowing what you (Mr. Tate) think would be the most interesting / needed areas of work. I'm looking forward to reviewing the code. wlb From jtate@mi-corporation.com Wed Feb 6 13:14:32 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16JEVa28441 for ; Wed, 6 Feb 2002 13:14:31 -0600 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g16JENf25328; Wed, 6 Feb 2002 14:14:23 -0500 From: "Joseph Tate" To: "wlb" , Subject: RE: [Metakit] Porting to Windows CE Date: Wed, 6 Feb 2002 14:15:52 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <001b01c1af2d$b92df330$6101a8c0@WB02> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Well I'll be. Hats off to you. It also appears you have spent more > time on this than I. Thanks. I spent about a week and a half on it. > Other than the c4_String class, there are few uses of char *'s. Most > of the occurrances are in naming properties, descriptions, and the > storage format string used in GetAs( ). c4_String. Huh. I have > noticed that CE's MFC-CString can be assigned and ASCII value > directly without first converting to Unicode. I haven't checked to > see if he stores it ASCII or silently converts to Unicode. The CE CString class is strictly ASCII. You can convert CString's to unicode, but only through the MultiByteToWideChar function and then through WideCharToMultiByte to get unicode back to ASCII. This doesn't help us at all. CStrings are evil anyway. If you want to know why, ask me off list. As you'll see (see wince.h), what I did to make my life easier was to #define char to short, and then enclose everything that needed to be a short* in _T(""). > If I'm not mistaken, there is something in the data file header which > indicates big-endian / little-endian. Maybe there are a few more flags > or values in that area that could be useful. I'll dig some more unless > jcw shines some light on it. I would support this addition to the header. > I'm very interested in knowning what jcw thinks would be the correct > method to integrate Unicode. I think it's something that definitely needs to be looked at as metakit is used in more and more world wide applications. We could do it in a way that would make it totally transparent to the user, especially if we used something like UTF-8 unicode, where a file that would normally be pure ascii remains pure ascii. If done correctly, no-one needs to know of the change, and we can even use the same file format. People who want unicode support just use different parameters or function calls, or compile options. > I'm also interested in knowing what you (Mr. Tate) think would be > the most interesting / needed areas of work. The next step I believe would be to replace all remaining instances of char in the code with CHAR, i.e. something that can be typedef'd according to the platform that it was compiled on (t4_char perhaps?). Either that or duplicate every function that takes a char* parameter so that it's overloaded to take a short* as well (we may wish to do this anyway so that we don't have to have _T("") everywhere in the metakit code). There are still places in the code (mostly in the integer handling) that need a signed single byte entity, and here it would be nice to still have char available, though having t4_byte and t4_sbyte (s for signed) functions well. Some more thought needs to go into this though. Perhaps the best option is to keep the t4_string class with a unicode backend (short*), but provide operators and conversion functions to make it easy to insert and extract ascii if needs be. I don't know. Another thing to look at is textend.cpp in the test suite, figuring out why there's an extra seven bytes written per entry. This would be a good place to start to get us closer to files that work across all platforms. > I'm looking forward to reviewing the code. Let me know if you need any help with clarification or understanding. It's nice to not have to think about this on my own anymore. Joseph From wlb@multisysid.com Wed Feb 6 15:15:48 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16LFla02682 for ; Wed, 6 Feb 2002 15:15:47 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id RAA25809 for ; Wed, 6 Feb 2002 17:14:32 -0500 Message-ID: <001301c1af54$a609fa70$6101a8c0@WB02> From: "wlb" To: References: Subject: Re: [Metakit] Porting to Windows CE Date: Wed, 6 Feb 2002 17:24:21 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >> Other than the c4_String class, there are few uses of char *'s. Most >> of the occurrances are in naming properties, descriptions, and the >> storage format string used in GetAs( ). c4_String. Huh. I have >> noticed that CE's MFC-CString can be assigned and ASCII value >> directly without first converting to Unicode. I haven't checked to >> see if he stores it ASCII or silently converts to Unicode. > > The CE CString class is strictly ASCII. You can convert CString's to > unicode, but only through the MultiByteToWideChar function and then through > WideCharToMultiByte to get unicode back to ASCII. This doesn't help us at > all. CStrings are evil anyway. If you want to know why, ask me off list. > As you'll see (see wince.h), what I did to make my life easier was to > #define char to short, and then enclose everything that needed to be a > short* in _T(""). There are a couple of shortcut functions that don't take nearly as many parameters: mbstowcs() and wcstombs(). I have always used CString in CE as Unicode... CString csText = _T("my text"); LPCTSTR szText = csText, and it has always given me the expected results...so I thought it was Unicode. Even more evil are people who write functions like: CString FindText( CString key ) { ... } without understanding what is a copy constructor. I have stories (and code) to make you laugh. Ask me off list. >> If I'm not mistaken, there is something in the data file header which >> indicates big-endian / little-endian. Maybe there are a few more flags >> or values in that area that could be useful. I'll dig some more unless >> jcw shines some light on it. > >I would support this addition to the header. I looked through the code. There is no real big/little endian checking. There was an earlier header format which was little ending and the code has a comment to that effect. Officially, strings are stored UTF-8. The "Overview of Metadata Format" document on the web site states: { This string, as well as all strings stored as properties of rows, is stored in UTF-8 format, which allow storing Unicode strings while maintaining compatibility of datafiles across all platforms. } I'll dig through the file format and textend.cpp. Worst case scenario, I can create a file ASCII and one UTF-16 (your port) and hexdump them to see the differences. > I think it's something that definitely needs to be looked at as metakit is > used in more and more world wide applications. We could do it in a way that > would make it totally transparent to the user, especially if we used > something like UTF-8 unicode, where a file that would normally be pure ascii > remains pure ascii. If done correctly, no-one needs to know of the change, > and we can even use the same file format. People who want unicode support > just use different parameters or function calls, or compile options. UTF-8 is nice. For CE, we would still have to watch the UTF-8 to UTF-16 conversions. Vice versa, for metakit we need to watch the UTF-16 to UTF-8 conversions. The only languages which I am handling (English/Spanish/Portugese/ French) are happy with mbstowcs() and wcwtombs() functions, then treat as ASCII or UTF-16. Not theoretically correct, but it works for me :) I've been programming CE for a long time, but metakit for only two days. It will take a few more days for me to get up to speed on the internals of metakit, read all the docs, and read all the code. wlb From mbloore@yahoo.com Wed Feb 6 21:30:56 2002 Received: from web9102.mail.yahoo.com (web9102.mail.yahoo.com [216.136.128.239]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g173Uua17360 for ; Wed, 6 Feb 2002 21:30:56 -0600 Message-ID: <20020207033055.72906.qmail@web9102.mail.yahoo.com> Received: from [65.93.22.92] by web9102.mail.yahoo.com via HTTP; Wed, 06 Feb 2002 19:30:55 PST Date: Wed, 6 Feb 2002 19:30:55 -0800 (PST) From: mARK bLOORE To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] empty data file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: running the following program, under RH linux 7.1, compiled with gcc 2.96: #include int main() { cout << endl << endl << "Hello World, this is MetaKit!" << endl; c4_StringProp pName ("name"); c4_StringProp pCountry ("country"); c4_Storage vAddress ("myfile.dat", 1); c4_Row row; cout << "rows: " << vAddress.GetSize() << endl; pName (row) = "John Williams"; pCountry (row) = "UK"; vAddress.Add(row); vAddress.Add(pName ["Julien Coco"] + pCountry ["Netherlands"]); cout << "name[0]: " << pName (vAddress[0]) << endl; cout << "name[1]: " << pName (vAddress[1]) << endl; bool committed = vAddress.Commit(); cout << "committed: " << (committed ? "yes" : "no") << endl; c4_Strategy& strat = vAddress.Strategy(); bool valid = strat.IsValid(); cout << "strategy valid: " << (valid ? "yes" : "no") << endl; cout << "rows: " << vAddress.GetSize() << endl; cout << "Goodbye World" << endl; return 0; } prints the following, on the first and subsequent runs: Hello World, this is MetaKit! rows: 1 name[0]: name[1]: John Williams committed: yes strategy valid: yes rows: 3 Goodbye World myfile.dat is created on the first run, but is empty, and stays empty on later runs. debugging, i see that during Commit() 'limit' in c4_SaveContext::SaveIt() is set to 11, which causes the routine to exit rather than write an empty file. i am not clear on how its value was decided, except that it came from a c4_Allocator. what am i missing? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com From jcw@equi4.com Thu Feb 7 04:53:54 2002 Received: from [10.0.1.55] (ipc379e97b.dial.wxs.nl [195.121.233.123]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g17Arra22776 for ; Thu, 7 Feb 2002 04:53:53 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] empty data file Date: Thu, 7 Feb 2002 11:51:36 +0100 Message-Id: <20020207105137.23502@triqs.com> In-Reply-To: <20020207033055.72906.qmail@web9102.mail.yahoo.com> References: <20020207033055.72906.qmail@web9102.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: > c4_Storage vAddress ("myfile.dat", 1); [...] > vAddress.Add(row); [...] >myfile.dat is created on the first run, but is empty, and stays empty >on later runs. Ouch - this use should not have been possible. It's a side effect of the generalizations made not so long ago. The way to do it is: c4_Storage storage ("myfile.dat", 1); c4_View vAddress = storage.GetAs("address[name:S,country:S]"); [etc] A storage object holds one or more named views. It may act like a view itself, but please just consider that a design flaw for now. See sample code in examples/, demos/, and tests/ in the source distribution. -jcw From mbloore@yahoo.com Mon Feb 11 17:49:08 2002 Received: from web14301.mail.yahoo.com (web14301.mail.yahoo.com [216.136.173.77]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1BNn8a01219 for ; Mon, 11 Feb 2002 17:49:08 -0600 Message-ID: <20020211234908.86930.qmail@web14301.mail.yahoo.com> Received: from [65.93.23.229] by web14301.mail.yahoo.com via HTTP; Mon, 11 Feb 2002 15:49:08 PST Date: Mon, 11 Feb 2002 15:49:08 -0800 (PST) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] storing large numbers of strings Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i am working on an application that will store data about large numbers of files, perhaps 100,000 in a directory. i am using essentilly the CatFish structure, and finding that saving the file names is very slow. (storing my extra data will only make it worse.) i have tried putting the names into memos, but that does not help (not really a surprise). i tried setting the growth granularity to 100100 without effect (but i don't suppose that could help, since strings are not of fixed size). storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, and 100,000 takes almost 700 sec. without the names (just date and size) i get 100,000 in about 15 sec. can anyone suggest anything i might try to improve this? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com From Murat_Berk@bmc.com Mon Feb 11 22:43:31 2002 Received: from babbler.bmc.com (camaro.bmc.com [198.207.223.231]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1C4hVa07976 for ; Mon, 11 Feb 2002 22:43:31 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by babbler.bmc.com (8.10.2/8.10.2) with ESMTP id g1C4gcm25327; Mon, 11 Feb 2002 22:42:38 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Mon, 11 Feb 2002 22:39:34 -0600 Message-ID: From: "Berk, Murat" To: "'mARK bLOORE'" , metakit Subject: RE: [Metakit] storing large numbers of strings Date: Mon, 11 Feb 2002 22:39:33 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The growth granularity is ignored. Set the size of view to 100,000. Set the lost row for name column to a dummy value (which forces all internal structures to be created to track string offsets. Then set it using for ( i = 1...100000 ) { c4_RowRef r = view[i]; pName(r) = string value } -----Original Message----- From: mARK bLOORE [mailto:mbloore@yahoo.com] Sent: Monday, February 11, 2002 5:49 PM To: metakit Subject: [Metakit] storing large numbers of strings i am working on an application that will store data about large numbers of files, perhaps 100,000 in a directory. i am using essentilly the CatFish structure, and finding that saving the file names is very slow. (storing my extra data will only make it worse.) i have tried putting the names into memos, but that does not help (not really a surprise). i tried setting the growth granularity to 100100 without effect (but i don't suppose that could help, since strings are not of fixed size). storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, and 100,000 takes almost 700 sec. without the names (just date and size) i get 100,000 in about 15 sec. can anyone suggest anything i might try to improve this? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From wlb@multisysid.com Tue Feb 12 09:40:15 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1CFeFa26931 for ; Tue, 12 Feb 2002 09:40:15 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id LAA22334 for ; Tue, 12 Feb 2002 11:39:16 -0500 Message-ID: <002401c1b3dc$cd1ab0a0$6101a8c0@WB02> From: "wlb" To: Date: Tue, 12 Feb 2002 11:49:05 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Unicode recommendation and WCE Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Metakit is capable of UTF-8. However, it would be more appropriate to say that Metakit handles single-byte null terminated strings as a fundamental data type. Metakit neither validates, enforces, nor converts UTF-8 to/from ASCII. A lot of the data which we save as strings is not UTF-8 encoded. ASCII and UTF-8 only overlap on 7-bit characters. Accented characters which we commenly know and love in most non-English languages are usually 8-bit ASCII, but require a two-byte representation in UTF-8. Indeed. Even in English, we are accustomed to borrowing words from other languages with these accents. Cafe' is the only word which immediately comes to mind. Personal names with accents should also conserve those accents in English. So, if you want to save your data UTF-8, it will work fine (null terminated 8-bit strings), or ASCII. But if you save your data UTF-8, expect to read it UTF-8, and vice versa. Metakit is agnostic. Only the application knows the true encoding. In Unicode speak, there are two commonly used double-byte (or wide character) representations. UCS-2 and UTF-16. For our purposes, I will consider them interchangable. The rules of manipulation are simple and fixed such as our standard null terminated strings, except that each 'code' is two bytes. I suggest creating a database column type. Call it '2', call it 'U', call it anything you want. It would deal with wchar_t *'s as the standard 'S'tring type deals with char *'s. In addition, I would recommend overloading the 12 or so functions which take variable names or structure information to provide wchar_t *'s as well as char *'s. Internal structures should be either one or the other, but on-the-fly conversions here should not suffer any measurable performance penalty. Structure and property name lookups really should not be inside a loop. Some operators would also need to be overloaded. This approach would have the following advantages: - a database can be copied from one machine to another - no penalty on the native machine to convert to/from ASCII or UTF8 if machine natively handles UCS-2 or UTF-16. - Any build of metakit can handle simultaneously wchar_t *'s or char *'s (where the host compiler handles wchar_t's, which is ANSI.) You might also want to consider the following: - There is a standard method documented in the Unicode literature for attaching a two-byte prefix to a string to determine byte order. The Unicode documentation is online at www.unicode.org. It is longish and committee like but very thorough. Please excuse the long-ish explanation. Let me know what you think. wlb From mbloore@yahoo.com Tue Feb 12 19:53:58 2002 Received: from web14308.mail.yahoo.com (web14308.mail.yahoo.com [216.136.173.156]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1D1rwa25792 for ; Tue, 12 Feb 2002 19:53:58 -0600 Message-ID: <20020213015349.56824.qmail@web14308.mail.yahoo.com> Received: from [65.93.23.229] by web14308.mail.yahoo.com via HTTP; Tue, 12 Feb 2002 17:53:49 PST Date: Tue, 12 Feb 2002 17:53:49 -0800 (PST) From: mARK bLOORE Subject: RE: [Metakit] storing large numbers of strings To: metakit In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thank you for your reply. Unfortunatly, preallocating the view causes things to get much slower! I am at a loss to explain this. Preallocating 10,000 rows and listing 10,000 files is pretty quick, though still slower than not preallocating. Preallocating 100,000 rows and listing the same 10,000 files is about twenty times slower (60 sec vs. 3). The slowdown happens in the for ( i = 1...10000 ) { c4_RowRef r = view[i]; pName(r) = string value } step. The step which copies that view into the directory's row takes an additional 3 seconds either way. --- "Berk, Murat" wrote: > The growth granularity is ignored. > > Set the size of view to 100,000. > Set the lost row for name column to a dummy value (which forces all > internal > structures to be created to track string offsets. > Then set it using > for ( i = 1...100000 ) > { > c4_RowRef r = view[i]; > pName(r) = string value > } > > -----Original Message----- > From: mARK bLOORE [mailto:mbloore@yahoo.com] > Sent: Monday, February 11, 2002 5:49 PM > To: metakit > Subject: [Metakit] storing large numbers of strings > > > i am working on an application that will store data about large > numbers > of files, perhaps 100,000 in a directory. i am using essentilly the > CatFish structure, and finding that saving the file names is very > slow. > (storing my extra data will only make it worse.) i have tried > putting > the names into memos, but that does not help (not really a surprise). > i tried setting the growth granularity to 100100 without effect (but > i > don't suppose that could help, since strings are not of fixed size). > > storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, > and > 100,000 takes almost 700 sec. without the names (just date and size) > i > get 100,000 in about 15 sec. > > can anyone suggest anything i might try to improve this? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com From Murat_Berk@bmc.com Tue Feb 12 21:05:21 2002 Received: from babbler.bmc.com (firebird.bmc.com [198.207.223.228]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1D35La27411 for ; Tue, 12 Feb 2002 21:05:21 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by babbler.bmc.com (8.10.2/8.10.2) with ESMTP id g1D38BG14337; Tue, 12 Feb 2002 21:08:11 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Tue, 12 Feb 2002 21:05:07 -0600 Message-ID: From: "Berk, Murat" To: "'mARK bLOORE'" , metakit Subject: RE: [Metakit] storing large numbers of strings Date: Tue, 12 Feb 2002 21:05:03 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Did you do pName(_view[10000-1]) = dummy string before entering the row. That is the key not preallocating. Otherwise you are right it gets slower. Murat -----Original Message----- From: mARK bLOORE [mailto:mbloore@yahoo.com] Sent: Tuesday, February 12, 2002 7:54 PM To: metakit Subject: RE: [Metakit] storing large numbers of strings Thank you for your reply. Unfortunatly, preallocating the view causes things to get much slower! I am at a loss to explain this. Preallocating 10,000 rows and listing 10,000 files is pretty quick, though still slower than not preallocating. Preallocating 100,000 rows and listing the same 10,000 files is about twenty times slower (60 sec vs. 3). The slowdown happens in the for ( i = 1...10000 ) { c4_RowRef r = view[i]; pName(r) = string value } step. The step which copies that view into the directory's row takes an additional 3 seconds either way. --- "Berk, Murat" wrote: > The growth granularity is ignored. > > Set the size of view to 100,000. > Set the lost row for name column to a dummy value (which forces all > internal > structures to be created to track string offsets. > Then set it using > for ( i = 1...100000 ) > { > c4_RowRef r = view[i]; > pName(r) = string value > } > > -----Original Message----- > From: mARK bLOORE [mailto:mbloore@yahoo.com] > Sent: Monday, February 11, 2002 5:49 PM > To: metakit > Subject: [Metakit] storing large numbers of strings > > > i am working on an application that will store data about large > numbers > of files, perhaps 100,000 in a directory. i am using essentilly the > CatFish structure, and finding that saving the file names is very > slow. > (storing my extra data will only make it worse.) i have tried > putting > the names into memos, but that does not help (not really a surprise). > i tried setting the growth granularity to 100100 without effect (but > i > don't suppose that could help, since strings are not of fixed size). > > storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, > and > 100,000 takes almost 700 sec. without the names (just date and size) > i > get 100,000 in about 15 sec. > > can anyone suggest anything i might try to improve this? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From mbloore@yahoo.com Tue Feb 12 21:33:30 2002 Received: from web14303.mail.yahoo.com (web14303.mail.yahoo.com [216.136.173.79]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1D3XUa28124 for ; Tue, 12 Feb 2002 21:33:30 -0600 Message-ID: <20020213033325.65183.qmail@web14303.mail.yahoo.com> Received: from [65.93.23.229] by web14303.mail.yahoo.com via HTTP; Tue, 12 Feb 2002 19:33:25 PST Date: Tue, 12 Feb 2002 19:33:25 -0800 (PST) From: mARK bLOORE Subject: RE: [Metakit] storing large numbers of strings To: metakit In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I did. I later tried removing that, but it made no difference. I also tried allocating 100,000 but only setting view[9999]. That was just as slow as setting view[99999], when only creating 10,000 rows. The important thing, I think, is that preallocating 10x too much makes for a huge slowdown. I find that if i don't store the file names (just two longs, for date and size), then things are fast. Also, storing the names speeds up towards the end. I think that putting names into the string column must be doing memory moves that are slowing it down, even though nothing has been stored in the rows that are moved. --- "Berk, Murat" wrote: > Did you do > pName(_view[10000-1]) = dummy string > before entering the row. That is the key not preallocating. > Otherwise you are right it gets slower. > > Murat > > -----Original Message----- > From: mARK bLOORE [mailto:mbloore@yahoo.com] > Sent: Tuesday, February 12, 2002 7:54 PM > To: metakit > Subject: RE: [Metakit] storing large numbers of strings > > > Thank you for your reply. > > Unfortunatly, preallocating the view causes things to get much > slower! > I am at a loss to explain this. > > Preallocating 10,000 rows and listing 10,000 files is pretty quick, > though still slower than not preallocating. Preallocating 100,000 > rows > and listing the same 10,000 files is about twenty times slower (60 > sec > vs. 3). > The slowdown happens in the > for ( i = 1...10000 ) > { > c4_RowRef r = view[i]; > pName(r) = string value > } > step. The step which copies that view into the directory's row takes > an additional 3 seconds either way. > > > --- "Berk, Murat" wrote: > > The growth granularity is ignored. > > > > Set the size of view to 100,000. > > Set the lost row for name column to a dummy value (which forces all > > internal > > structures to be created to track string offsets. > > Then set it using > > for ( i = 1...100000 ) > > { > > c4_RowRef r = view[i]; > > pName(r) = string value > > } > > > > -----Original Message----- > > From: mARK bLOORE [mailto:mbloore@yahoo.com] > > Sent: Monday, February 11, 2002 5:49 PM > > To: metakit > > Subject: [Metakit] storing large numbers of strings > > > > > > i am working on an application that will store data about large > > numbers > > of files, perhaps 100,000 in a directory. i am using essentilly > the > > CatFish structure, and finding that saving the file names is very > > slow. > > (storing my extra data will only make it worse.) i have tried > > putting > > the names into memos, but that does not help (not really a > surprise). > > i tried setting the growth granularity to 100100 without effect > (but > > i > > don't suppose that could help, since strings are not of fixed > size). > > > > storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, > > and > > 100,000 takes almost 700 sec. without the names (just date and > size) > > i > > get 100,000 in about 15 sec. > > > > can anyone suggest anything i might try to improve this? > > > __________________________________________________ > Do You Yahoo!? > Send FREE Valentine eCards with Yahoo! Greetings! > http://greetings.yahoo.com > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com From gerard@xsystemsinc.com Wed Feb 13 12:34:01 2002 Received: from dom01.xsystemsinc.com (hidden-user@dom01.xsystemsinc.com [12.5.16.43]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1DIY1a25133 for ; Wed, 13 Feb 2002 12:34:01 -0600 Received: from GERARD ([64.204.192.3]) by dom01.xsystemsinc.com (Lotus Domino Release 5.0.8) with SMTP id 2002021313335799:3043 ; Wed, 13 Feb 2002 13:33:57 -0500 Message-ID: <007d01c1b4bd$0d374870$0c00a8c0@GERARD> From: "Gerard Duquette" To: Date: Wed, 13 Feb 2002 13:34:19 -0500 MIME-Version: 1.0 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-MIMETrack: Itemize by SMTP Server on dom01/xsystems(Release 5.0.8 |June 18, 2001) at 02/13/2002 01:33:58 PM, Serialize by Router on dom01/xsystems(Release 5.0.8 |June 18, 2001) at 02/13/2002 01:34:01 PM, Serialize complete at 02/13/2002 01:34:01 PM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] AIX metakit shared object Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am in the process of installing metakit for one of our customers. So far thats not a problem, the problem is that they are on IBM AIX servers. I have tried to find the mk4tcl.so shared object for AIX with no success. The best I can do is get the source code and compile it. Now it gets complicated, because they are not really experienced when it comes to compiling. I have tried to setup a cross compiler on one of our servers and quickly found out this would take way to much time. This is where the world comes in, specifically the world of metakit users. Has anyone succesfully compiled metakit for IBM AIX RS600, and would you be willing to share the fruits of your success, if not would you help in getting it compiled? I admit I am new to this list and this is a bit much to ask the first time, but I dont have much options left. This would be greatly appreciated. Thank you Gerard Duquette X.Systems, Inc (703) 330-1645 ext111 gerard@xsystemsinc.com From cloudshadow@21cn.com Thu Feb 14 07:26:23 2002 Received: from 21cn.com ([61.140.60.248]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1EDQMa29450 for ; Thu, 14 Feb 2002 07:26:22 -0600 Message-Id: <200202141326.g1EDQMa29450@trixie.triqs.com> Received: from shadow([61.142.221.12]) by 21cn.com(AIMC 2.9.5.2) with SMTP id jm1c3c6c2e7f; Thr, 14 Feb 2002 21:27:30 +0800 Date: Thu, 14 Feb 2002 21:26:11 +0800 From: cloudshadow To: "metakit@equi4.com" X-mailer: FoxMail 4.0 Alpha 1 [cn] Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=====002_Dragon680087657523_=====" Subject: [Metakit] how can i change the DBF files to the metakit files type? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --=====002_Dragon680087657523_===== Content-Type: text/plain; charset="GB2312" Content-Transfer-Encoding: base64 --=====002_Dragon680087657523_===== Content-Type: text/html; charset="GB2312" Content-Transfer-Encoding: base64 PEhUTUw+DQo8SEVBRD4NCjxtZXRhIGh0dHAtZXF1aXZlPSJDb250ZW50LVR5cGUiIGNvbnRlbnQ9 InRleHQvaHRtbDsgY2hhcnNldD1HQjIzMTIiPg0KPG1ldGEgaHR0cC1lcXVpdmU9IkNvbnRlbnQt VHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PUdCMjMxMiI+DQo8TUVUQSBOQU1FPSJH RU5FUkFUT1IiIENvbnRlbnQ9Ik1pY3Jvc29mdCBESFRNTCBFZGl0aW5nIENvbnRyb2wiPg0KPFRJ VExFPjwvVElUTEU+DQo8L0hFQUQ+DQo8Qk9EWT4NCjxESVY+Jm5ic3A7DQo8L0RJVj4NCjwvQk9E WT4NCjwvSFRNTD4NCg== --=====002_Dragon680087657523_=====-- From horst@freedict.de Thu Feb 14 12:15:11 2002 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1EIFAa09189 for ; Thu, 14 Feb 2002 12:15:11 -0600 Received: (qmail 3098 invoked from network); 14 Feb 2002 18:49:26 -0000 Received: from dialin-145-254-115-124.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.115.124) by mail.netbeat.de with SMTP; 14 Feb 2002 18:49:26 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id g1EI7Bm00968; Thu, 14 Feb 2002 19:07:12 +0100 Message-Id: <200202141807.g1EI7Bm00968@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Thu, 14 Feb 2002 19:07:08 +0100 (CET) From: horst@freedict.de Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] python metakit / blocked / hashed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I failed to get a blocked (and hashed) view working with python metakit. - Are there examples available? By the way, some python examples include metakit, others Mk4py. Which one really should be used, or are there no differences? And yet another remark: the python documentation does not mention how to install subviews yet. It would be nice to include that at some point in the future. (I managed to install subviews (looking at tcl), but it would have helped). Thanks, Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From gmcm@hypernet.com Fri Feb 15 09:02:46 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1FF2ha07210 for ; Fri, 15 Feb 2002 09:02:44 -0600 Received: from PARANOIA (204.176.40.107) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Fri, 15 Feb 2002 10:02:40 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Fri, 15 Feb 2002 10:02:06 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] python metakit / blocked / hashed Reply-to: gmcm@hypernet.com Message-ID: <3C6CDC9E.2807.42FAE64@localhost> Priority: normal In-reply-to: <200202141807.g1EI7Bm00968@eaglesnest.mceggman> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 14 Feb 2002 at 19:07, horst@freedict.de wrote: > I failed to get a blocked (and hashed) view working > with python metakit. - Are there examples available? Probably not. Meaningful examples are harder to write than you might think. Make sure your base view is in the right order (or empty). Create the block / hash from the base view. Now do everything through the block / hash view. > By the way, some python examples include metakit, > others Mk4py. Which one really should be used, or > are there no differences? The only thing metakit adds is dump(). > And yet another remark: the python documentation > does not mention how to install subviews yet. It > would be nice to include that at some point in the > future. (I managed to install subviews (looking at > tcl), but it would have helped). Contributions gratefully accepted. -- Gordon http://www.mcmillan-inc.com/ From mh@pixar.com Fri Feb 15 09:59:51 2002 Received: from pixar.pixar.com (pixar.pixar.com [138.72.10.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1FFxoa09241 for ; Fri, 15 Feb 2002 09:59:50 -0600 Received: from postal.pixar.com (postal.pixar.com [138.72.126.101]) by pixar.pixar.com (8.9.3/8.9.3) with ESMTP id HAA13479 for ; Fri, 15 Feb 2002 07:59:44 -0800 (PST) Received: from pixar.com (dreadnok [138.72.16.110]) by postal.pixar.com (8.11.1/8.11.1) with ESMTP id g1FFxaF11735 for ; Fri, 15 Feb 2002 07:59:37 -0800 (PST) Message-ID: <3C6D306F.7090708@pixar.com> Date: Fri, 15 Feb 2002 07:59:43 -0800 From: Mark Harrison User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.7) Gecko/20011221 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com References: <3C6CDC9E.2807.42FAE64@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] newest wikit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is this the right place for wikit questions? Here are a couple... 1. How can I tell what version of the wiki software I'm running? 2. I think I have the latest, but it does not seem to support the inline display of images... I can make it work on the tclers wiki but not on my local copy. 3. I downloaded the full tclwiki, but cannot get it to run. error messages below. 4. Is the wiki src available? I want to add a "bug" tag that will refer to our internal bug database page, and an "isbn" tag for book references. Many thanks, Mark ~/bin/linux/tclkit bigwiki.tkd don't know how to start 'main.tcl' while executing "error "don't know how to start '$argMain.tcl'"" invoked from within "if {$failed && $errMsg != ""} { error $errMsg $savedInfo puts $errMsg exit 2 }" ("eval" body line 527) invoked from within "eval $script" (file "bigwiki.tkd" line 8) From horst@freedict.de Fri Feb 15 14:59:11 2002 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1FKxAa22627 for ; Fri, 15 Feb 2002 14:59:10 -0600 Received: (qmail 19294 invoked from network); 15 Feb 2002 21:33:33 -0000 Received: from dialin-145-254-119-253.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.119.253) by mail.netbeat.de with SMTP; 15 Feb 2002 21:33:33 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id g1FHjkQ17142; Fri, 15 Feb 2002 18:45:47 +0100 Message-Id: <200202151745.g1FHjkQ17142@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Fri, 15 Feb 2002 18:45:44 +0100 (CET) From: horst@freedict.de Reply-To: horst@freedict.de Subject: Re: [Metakit] python metakit / blocked / hashed To: gmcm@hypernet.com cc: metakit@equi4.com In-Reply-To: <3C6CDC9E.2807.42FAE64@localhost> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 15 Feb, Gordon McMillan wrote: > On 14 Feb 2002 at 19:07, horst@freedict.de wrote: > >> I failed to get a blocked (and hashed) view working >> with python metakit. - Are there examples available? > > Probably not. Meaningful examples are harder > to write than you might think. to bad - So I prepared some example code, which failed for my setup. Any hits? #################################################### import sys; sys.path.append('../builds'); import Mk4py; mk = Mk4py print sys.argv[0], '-', 'Mk4py', mk.version, '-', sys.platform from time import time db = mk.storage() vw = db.getas('_B[vw[p1:I,p2:I]]').blocked() map = db.getas('_B[map[_H:I,_R:I]]').blocked() def filldb(n=1000): del vw[:] t0 = time(); for i in xrange(n): vw.append(p1=i, p2=i+i) print 'filldb %d rows: %g sec' % (n, time() - t0) # this find will do a linear scan def find_raw(k,n=1000): t0 = time(); for i in xrange(n): r = vw.find(p1=k) print ' find_raw %d times, key %d -> %d: %g sec' % (n, k, r, time() - t0) for n in (100, 1000, 10000, 100000): filldb(n) find_raw(n/2, 1000000/n) find_raw(n+1, 1000000/n) find_ord(n/2, 1000000/n) #################################################### The output actually is: find3.py - Mk4py 2.4.3 - linux2 filldb 100 rows: 0.00252903 sec Traceback (most recent call last): File "find3.py", line 28, in ? find_raw(n/2, 1000000/n) File "find3.py", line 22, in find_raw r = vw.find(p1=k) ValueError: Object has no usable attributes > > Make sure your base view is in the right order (or > empty). Create the block / hash from the base view. > Now do everything through the block / hash view. > >> By the way, some python examples include metakit, >> others Mk4py. Which one really should be used, or >> are there no differences? > > The only thing metakit adds is dump(). I guess, what I really asked was: which one (Mk4py or metakit) should be used for beeing save in > >> And yet another remark: the python documentation >> does not mention how to install subviews yet. It >> would be nice to include that at some point in the >> future. (I managed to install subviews (looking at >> tcl), but it would have helped). > > Contributions gratefully accepted. Structure definitions consist of a list of properties. Subviews are specified as a sublist of two entries: the name and the list of properties in that subview. Note that subviews add two levels of nesting (see phones in the phonebook example below). w = db.getas("name:S,Address:S,phones:[category:S,phone:S]]") or simply: subviews can be created as in the given example. (see above) > > -- Gordon > http://www.mcmillan-inc.com/ > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From vaclav.slavik@matfyz.cz Fri Feb 15 13:48:03 2002 Received: from smtp3.vol.cz (smtp3.vol.cz [195.250.128.83]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1FJm1a19459 for ; Fri, 15 Feb 2002 13:48:02 -0600 Received: from there (prahad-3-78.dialup.vol.cz [212.27.197.246]) by smtp3.vol.cz (8.11.6/8.11.3) with SMTP id g1FJltO21509 for ; Fri, 15 Feb 2002 20:47:56 +0100 (CET) (envelope-from vaclav.slavik@matfyz.cz) Message-Id: <200202151947.g1FJltO21509@smtp3.vol.cz> Content-Type: text/plain; charset="iso-8859-1" From: Vaclav Slavik Date: Fri, 15 Feb 2002 20:47:43 +0100 X-Mailer: KMail [version 1.3] MIME-Version: 1.0 To: metakit@equi4.com Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g1FJm1a19459 Subject: [Metakit] hashed views never stored in file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm trying to use hashes in MetaKit 2.4.3-33 (which I found a week ago and which is a perfect library for my needs, thanks a lot!) and there's a problem I don't know how to solve: it appears that data stored via hash view are never stored in the storage object! It can be shown by this piece of code: c4_IntProp p1("p1"); c4_StringProp p2("p2"); c4_Storage storage("myfile.dat", true); c4_View data = storage.GetAs("data[key:S,val1:S,val2:I]"); c4_View data_h = storage.GetAs("data_H[_H:I,_R:I]"); c4_View hashed = data.Hash(data_h,1); hashed.Add(p1[1] + p2["one"]); hashed.Add(p1[2] + p2["two"]); storage.Commit(); // myfile.dat has 0 bytes size! It can also be seen in examples/mkhash.cpp when run as "./mkhash h". Same thing will happen, empty file... Am I doing something wrong? Thanks! Vaclav From jcw@equi4.com Sun Feb 17 17:23:29 2002 Received: from [10.0.1.55] (ipc379e83a.dial.wxs.nl [195.121.232.58]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1HNNSa22909 for ; Sun, 17 Feb 2002 17:23:29 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] hashed views never stored in file Date: Mon, 18 Feb 2002 00:13:19 +0100 Message-Id: <20020217231319.29933@triqs.com> In-Reply-To: <200202151947.g1FJltO21509@smtp3.vol.cz> References: <200202151947.g1FJltO21509@smtp3.vol.cz> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Vaclav Slavik wrote: > c4_IntProp p1("p1"); > c4_StringProp p2("p2"); You've just defined properties "p1" and "p2". > c4_Storage storage("myfile.dat", true); > c4_View data = storage.GetAs("data[key:S,val1:S,val2:I]"); So you better use those names in the stored view as well. Replace the above line with something compatible, e.g.: c4_View data = storage.GetAs("data[p1:I,p2:S]"); > c4_View data_h = storage.GetAs("data_H[_H:I,_R:I]"); > c4_View hashed = data.Hash(data_h,1); > hashed.Add(p1[1] + p2["one"]); > hashed.Add(p1[2] + p2["two"]); > storage.Commit(); // myfile.dat has 0 bytes size! -jcw From jcw@equi4.com Sun Feb 17 17:23:31 2002 Received: from [10.0.1.55] (ipc379e83a.dial.wxs.nl [195.121.232.58]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1HNNUa22912 for ; Sun, 17 Feb 2002 17:23:30 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] newest wikit? Date: Mon, 18 Feb 2002 00:21:51 +0100 Message-Id: <20020217232151.32392@triqs.com> In-Reply-To: <3C6D306F.7090708@pixar.com> References: <3C6D306F.7090708@pixar.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Harrison wrote: >Is this the right place for wikit questions? Yes, though probably only few readers will be interested/involved. Might also try the TclersWiki mailing list on yahoo. >Here are a couple... > >1. How can I tell what version of the wiki software I'm running? There's no versioning, other than checking script file dates. >2. I think I have the latest, but it does not seem to support the > inline display of images... I can make it work on the tclers wiki > but not on my local copy. The Tcl'ers Wiki was customized a few times. I haven't gotten around to consolidating changes so far. >3. I downloaded the full tclwiki, but cannot get it to run. error > messages below. Same answer as 2, but even nastier: recent changes to the Tcl'ers Wiki broke the ability to use it in a general context, because it now stores data differently. This is in anticipation of more changes, and a planned move to a new "kiwi" based re-implementation of the wikit code (www.equi4.com/kiwi/). >4. Is the wiki src available? I want to add a "bug" tag that will > refer to our internal bug database page, and an "isbn" tag > for book references. Wikit is coded in Tcl, all sources are stored inside as a "scripted document". There's some documentation on that concept, and on how to use such beasts. The one complicating factor is that wikit precedes the current VFS-based design of scripted documents, so scripts are being stored in ways which differ from one revision to the next. For example: >~/bin/linux/tclkit bigwiki.tkd >don't know how to start 'main.tcl' > while executing >"error "don't know how to start '$argMain.tcl'"" This is a remnant of the old system. Name your file "wikit.something" and try again. The startup script looks at the way it was started - and that has to contain the prefix "wikit" to work. -jcw From jcw@equi4.com Sun Feb 17 17:26:42 2002 Received: from [10.0.1.55] (ipc379e83a.dial.wxs.nl [195.121.232.58]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1HNQfa22980 for ; Sun, 17 Feb 2002 17:26:41 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] how can i change the DBF files to the metakit files type? Date: Mon, 18 Feb 2002 00:25:04 +0100 Message-Id: <20020217232504.20279@triqs.com> In-Reply-To: <200202141326.g1EDQMa29450@trixie.triqs.com> References: <200202141326.g1EDQMa29450@trixie.triqs.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: cloudshadow wrote: >how can i change the DBF files to the metakit files type? There's an old example of a little C++ program which can read DBase files and present the content as a MK view. It's at: http://www.equi4.com/pub/mk/metakit-1.99/win-examples.zip You could use that to copy the records to an MK datafile. Or, if you use Python or Tcl, you could look for packages which read DBF files, and write a little script which copies the records. -jcw From mark@markroseman.com Sun Feb 17 17:39:49 2002 Received: from admin.cgocable.net (admin.cgocable.net [24.226.1.21]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1HNdna23213 for ; Sun, 17 Feb 2002 17:39:49 -0600 Received: from eyak-can4.markroseman.com (d141-160-232.home.cgocable.net [24.141.160.232]) by admin.cgocable.net (8.10.0/8.9.3) with ESMTP id g1HNg8n05135 for ; Sun, 17 Feb 2002 18:42:08 -0500 (EST) Message-Id: <5.1.0.14.0.20020217183342.03a96ff0@demo.teamwave.com> X-Sender: X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Sun, 17 Feb 2002 18:38:02 -0500 To: metakit@equi4.com From: Mark Roseman Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] metakit for tcl tutorial documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i've put together some new, more tutorial-style documentation intended to introduce metakit to tcl developers. i hope this will serve as a good complement to jean-claude's existing documentation. you can find a first draft at: http://www.markroseman.com/tcl/ i would greatly appreciate any comments or suggestions you might have on this draft. thanks mark From wlb@multisysid.com Mon Feb 18 12:32:58 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1IIWva13553 for ; Mon, 18 Feb 2002 12:32:57 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id OAA24064 for ; Mon, 18 Feb 2002 14:32:14 -0500 Message-ID: <000501c1b8ab$fbe323d0$6101a8c0@WB02> From: "wlb" To: Date: Mon, 18 Feb 2002 14:42:12 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Indexes as of 2.4.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I see support for indexes in the header files, and apparently in the bindings for tcl and python also. Alas, no regression tests demonstrate their usage. Do they work? Any docs/tests/samples? I will even read the code if someone could confirm that they are supposed to work. wlb From Murat_Berk@bmc.com Mon Feb 18 16:51:33 2002 Received: from creeper.bmc.com (camaro.bmc.com [198.207.223.231]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1IMpXa29551; Mon, 18 Feb 2002 16:51:33 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2/8.10.2) with ESMTP id g1IMp9c07428; Mon, 18 Feb 2002 16:51:09 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Mon, 18 Feb 2002 16:51:26 -0600 Message-ID: From: "Berk, Murat" To: "'Jean-Claude Wippler'" , metakit Date: Mon, 18 Feb 2002 16:51:23 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Commit problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude, We are experiencing commit problems w/ this simple test case using latest CVS version. When we commit database, the view is not committed. i.e If I move the addition of item before Commit() call, everything works. But if we commit after GetAs() w/o inserting any row, none of the rows get inserted. (It behaves as if it is detached.) Murat #include "mk4.h" #include "mk4str.h" #include int main() { c4_StringProp pName ("name"); c4_StringProp pCountry ("country"); c4_Storage storage ("myfile.dat", true); c4_View vAddress = storage.GetAs("address[name:S,country:S]"); storage.Commit(); c4_Row row; pName (row) = "John Williams"; pCountry (row) = "UK"; vAddress.Add(row); storage.Commit(); // Should have 1 row, have 0 rows return 0; } From wlb@multisysid.com Mon Feb 18 17:07:41 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1IN7ea30466 for ; Mon, 18 Feb 2002 17:07:40 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id TAA25824 for ; Mon, 18 Feb 2002 19:06:57 -0500 Message-ID: <000801c1b8d2$5d3e6e20$6101a8c0@WB02> From: "wlb" To: Date: Mon, 18 Feb 2002 19:16:56 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Indexes as of 2.4.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ...thinking aloud. In remap.cpp line around line 930, it says: bool c4_IndexedViewer::SetItem(int row_, int col_, const c4_Bytes& buf_) { ... if (keyMod) { // TODO adjust index } Which means, indexes may work, but don't update a key value, I believe. I really don't see when the map is stored. It is stored, isn't it? Or is this just a large index in memory for now? wlb From slok00@yahoo.com Tue Feb 19 09:32:48 2002 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1JFWma21432 for ; Tue, 19 Feb 2002 09:32:48 -0600 Received: from mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 19 Feb 2002 15:32:44 -0000 Message-Id: <5.1.0.14.0.20020219231806.02d76e78@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Tue, 19 Feb 2002 23:30:23 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-7F3E5D62; boundary="=======72EC1BC4=======" Subject: [Metakit] Mk4tcl.so installation question.. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======72EC1BC4======= Content-Type: text/plain; x-avg-checked=avg-ok-7F3E5D62; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit I downloaded Mk4tcl.so.gz and uncompress it to a directory "/home/user/test" I created a simple script mytest.tcl and put it in the same directory as Mk4tcl.so in "/home/user/test" When I run "wish mytest.tcl", I got the following messages ==== Error in startup script: can't find Mk4tcl while executing "package require Mk4tcl" {file "mytest.tcl" line 1} ===== How or where should I put Mk4tcl.so? Thanks. YekSoon --=======72EC1BC4======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-7F3E5D62 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.323 / Virus Database: 180 - Release Date: 2/8/2002 --=======72EC1BC4=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jcw@equi4.com Tue Feb 19 10:44:02 2002 Received: from [10.0.1.55] (ipc379eb0a.dial.wxs.nl [195.121.235.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1JGi1a24465 for ; Tue, 19 Feb 2002 10:44:01 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Indexes as of 2.4.3 Date: Tue, 19 Feb 2002 17:20:09 +0100 Message-Id: <20020219162009.8468@triqs.com> In-Reply-To: <000801c1b8d2$5d3e6e20$6101a8c0@WB02> References: <000801c1b8d2$5d3e6e20$6101a8c0@WB02> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: wlb wrote: >...thinking aloud. In remap.cpp line around line 930, it says: > >bool c4_IndexedViewer::SetItem(int row_, int col_, const c4_Bytes& buf_) >{ > >... > > if (keyMod) > { > // TODO adjust index > } > > >Which means, indexes may work, but don't update a key value, I believe. Exactly. This is the reason I did not take this further - the indexed mapping view is able to deal with the case of adding/deleting rows, and is ok as long as the key is not changed. Not sure just how far I got with inserting a second row with duplicate keys, but I think that was ok. But given that it is not yet solving the problem fully, >I really don't see when the map is stored. It is stored, isn't it? Or is >this just a large index in memory for now? Whether it persist or not depends on whether the index view is setup to persist prior to being used in the indexed mapping viewer. So yes, you can make it persist - by creating an appropriate view beforehand - or you can choose to use it only in a transient way. -jcw From jcw@equi4.com Tue Feb 19 10:44:04 2002 Received: from [10.0.1.55] (ipc379eb0a.dial.wxs.nl [195.121.235.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1JGi3a24471 for ; Tue, 19 Feb 2002 10:44:03 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Mk4tcl.so installation question.. Date: Tue, 19 Feb 2002 17:23:18 +0100 Message-Id: <20020219162318.13425@triqs.com> In-Reply-To: <5.1.0.14.0.20020219231806.02d76e78@pop.mail.yahoo.com> References: <5.1.0.14.0.20020219231806.02d76e78@pop.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Lok Yek Soon wrote: >I downloaded Mk4tcl.so.gz and uncompress it to a directory "/home/user/test" > >I created a simple script mytest.tcl and put it in the same directory as >Mk4tcl.so in "/home/user/test" >When I run "wish mytest.tcl", I got the following messages > >==== >Error in startup script: can't find Mk4tcl > while executing >"package require Mk4tcl" > {file "mytest.tcl" line 1} >===== > > >How or where should I put Mk4tcl.so? As with any shared library for Tcl, you could put it in the current dir and use "load ./Mk4tcl.so", or you could use "mk_pkgIndex" to create the required "pkgIndex.tcl" for using MK as a Tcl package. -jcw From teamwork@andinet.com Tue Feb 19 11:26:16 2002 Received: from kepler.andinet.com (kepler.andinet.com [208.221.129.18]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1JHQFa27120 for ; Tue, 19 Feb 2002 11:26:16 -0600 Received: by kepler.andinet.com (5.1.056) id 3C7289EF000000B6 for metakit@equi4.com; Tue, 19 Feb 2002 12:23:38 -0500 Message-ID: <3C727D8D00000035@kepler.andinet.com> Date: Tue, 19 Feb 2002 12:23:38 -0500 From: teamwork@andinet.com To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g1JHQFa27120 Subject: [Metakit] database sizes Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi: metakit last version one question? append records this way db = storage('demo',2) f = db.getas('frequents[drinker,bar,perweek:I]') for i in range(1000): f.append(drinker='lola', bar='lolas', perweek=6) db.commit() database size (demo) is 13kb but!!!!!!!!!!!!!!! append records this way db = storage('demo',2) f = db.getas('frequents[drinker,bar,perweek:I]') for i in range(1000): f.append(drinker='lola', bar='lolas', perweek=6) db.commit() database size (demo) is 6207kb help me please!!! libraries and programs: PythonWin 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32. MK 2.4.3 metakit.py Juan Carlos Rodriguez O. Team Work Ltda. Bogota Colombia. quirogaco@yahoo.es teamwork@andinet.com From jcw@equi4.com Tue Feb 19 11:33:28 2002 Received: from [10.0.1.55] (ipc379e8de.dial.wxs.nl [195.121.232.222]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1JHXRa27522 for ; Tue, 19 Feb 2002 11:33:27 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] database sizes Date: Tue, 19 Feb 2002 18:31:23 +0100 Message-Id: <20020219173123.9969@triqs.com> In-Reply-To: <3C727D8D00000035@kepler.andinet.com> References: <3C727D8D00000035@kepler.andinet.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: teamwork@andinet.com wrote: > for i in range(1000): > f.append(drinker='lola', bar='lolas', perweek=6) > db.commit() > >database size (demo) is 13kb You're opening in mode 2 (commit extend), which *only* writes changes at the end of the datafile. > for i in range(1000): > f.append(drinker='lola', bar='lolas', perweek=6) > db.commit() A thousand copies of the entire database. Opening in mode 1 will lead to a (much) smaller file. -jcw From quirogaco@yahoo.es Wed Feb 20 13:54:06 2002 Received: from web20107.mail.yahoo.com (web20107.mail.yahoo.com [216.136.226.44]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1KJs6a20497 for ; Wed, 20 Feb 2002 13:54:06 -0600 Message-ID: <20020220195405.73745.qmail@web20107.mail.yahoo.com> Received: from [200.13.218.176] by web20107.mail.yahoo.com via HTTP; Wed, 20 Feb 2002 20:54:05 CET Date: Wed, 20 Feb 2002 20:54:05 +0100 (CET) From: =?iso-8859-1?q?Juan=20carlos=20Rodriguez?= To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] dynamic - find Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi: JCW thank's, quick response before question. other question: situation one: n = 'FERRARI PAOLO' vw.find(name = n) work fine. how make dynamic find?, example: a = 'name' n = 'FERRARI PAOLO' vw.find(a = n) don't work exist other way? diferent of : eval("vw.find(name = 'FERRARI PAOLO')") thank's Juan Carlos Rodriguez O. Bogota, Colombia quirogaco@yahoo.es teamwork@andinet.com _______________________________________________________________ Do You Yahoo!? Yahoo! Messenger Comunicación instantánea gratis con tu gente. http://messenger.yahoo.es From jerry@usd1.com Wed Feb 20 15:37:45 2002 Received: from bfish.hurrah.com (bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1KLbja25873 for ; Wed, 20 Feb 2002 15:37:45 -0600 Received: from 12-225-195-8.client.attbi.com (12-225-195-8.client.attbi.com [12.225.195.8]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id g1KLbgx25102 for ; Wed, 20 Feb 2002 13:37:42 -0800 Date: Wed, 20 Feb 2002 13:36:01 -0800 From: Jerry McRae X-Mailer: The Bat! (v1.53d) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <921316543.20020220133601@usd1.com> To: Metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] .so files Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I tried to install Mk4py.so on the Linus (v 2.2.18) box for my website. Just copied in the latest. Date of 2/1/02, Linux version. I try import Mk4py (from Python 2.1) and get the message: ImportError: libstdc++-libc6.2-2.so.3: cannot open shared object file: No such file or directory I don't know Linus or C++, so I have no clue. I can import other librarys. This just happens to be used at the core of my product, so I cannot go any farther. I thought all I would have to do is copy the current .so file instead of my .dll on my system. Mk4py.so permissions: -rw-r--r Ideas? -- Thanks, Jerry McRae MetaKit 2.4.3, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From lvirden@cas.org Fri Feb 22 05:57:02 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1MBv2a29179 for ; Fri, 22 Feb 2002 05:57:02 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g1MBusb07910 for ; Fri, 22 Feb 2002 06:56:54 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g1MBuqn22429; Fri, 22 Feb 2002 06:56:52 -0500 (EST) Date: Fri, 22 Feb 2002 06:56:52 -0500 (EST) From: "Larry W. Virden" Message-Id: <20020222065652.AAB22420@cas.org> To: Metakit Subject: [Metakit] Call For Papers: 9th Annual Tcl/Tk Conference, Sept 16-20, 2002 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Call For Papers 9th Annual Tcl/Tk Conference September 16-20, 2002 Vancouver, BC, Canada We are pleased to announce the 9th Annual Tcl/Tk conference, to be held September 16-20, 2002, in Vancouver, BC, Canada. This conference is a forum to: * bring together Tcl/Tk researchers and practitioners * publish and present current work involving Tcl/Tk * learn about the latest developments in Tcl/Tk * plan for future Tcl/Tk related developments The conference program will include paper presentations, tutorials, Birds of a Feather (BOF) sessions and invited key-note talks. This call invites you to submit extended abstracts to be considered for the conference program. The conference schedule will consist of 2 days of tutorials (Monday - Tuesday) and 3 days for the main conference (Wednesday - Friday). Submission of Extended Abstracts The conference provides an opportunity to report on original Tcl/Tk research. The audience is practitioners and researchers who are intermediate or experienced users of Tcl/Tk. For this reason, reports on experiences and applications must draw out lessons for other Tcl/Tk developers. Topics include, but are not limited to: * System extensions * Novel Tcl/Tk-based applications * Experience reports on building applications in Tcl/Tk * Comparative evaluations of Tcl/Tk and other languages or toolkits for building applications * Use of different programming paradigms in Tcl/Tk and proposals for new directions. * New areas of exploration for the Tcl/Tk language Extended Abstracts should be written in English and 600 to 1000 words long (1-2 pages). Omit extraneous or redundant information. Length is not a direct factor in judging the quality of the submission. Send submissions as plain text to no later than April 28, 2002. Authors of accepted abstracts will have until August 20, 2002 to submit their final paper for the inclusion in the conference proceedings. The proceedings will be made available on CD-ROM, so extra materials like code samples are welcome. The authors will have 20-25 minutes to present the paper at the conference. The program committee will review and evaluate papers according to the following criteria: * Quantity and quality of novel content * Relevance and interest to the Tcl/Tk community * Suitability of content for presentation at the conference Proposals may report on commercial or non-commercial systems, but those with only blatant marketing content will not be accepted. Application and experience papers need to strike a balance between background on the application domain and the relevance of Tcl/Tk to the application. Application and experience papers should clearly explain how the application or experience illustrates a novel use of Tcl/Tk, and what lessons the Tcl/Tk community can derive from the application or experience to apply to their own development efforts. Papers accompanied by non-disclosure agreement forms will be returned to the author(s) unread. All submissions are held in the highest confidentiality prior to publication in the Proceedings, both as a matter of policy and in accord with the U. S. Copyright Act of 1976. Other Forms of Participation The program committee also welcomes proposals for panel discussions of up to 90 minutes. Proposals should include a list of confirmed panelists, a title and format, and a panel description with position statements from each panelist. Panels should have no more than four speakers, including the panel moderator, and should allow time for substantial interaction with attendees. Panels are not presentations of related research papers. Slots for Works-in-Progress (WIP) presentations and Birds-of-a-Feather sessions (BOFs) are available on a first-come, first-served basis starting in August 2002. Specific instructions for reserving WIP and BOF time slots will be provided in the registration information available in June 2002. Some WIP and BOF time slots will be held open for on-site reservation, so we encourage all attendees with interesting work in progress to consider presenting that work at the conference. Registration Information More information on the conference will be available in Spring 2002 at the conference web site (http://www.tcl.tk/community/tcl2002/) and published on various Tcl/Tk-related information channels. To keep in touch with conference announcements and Tcl events in general, subscribe to the tcl-announce list at: http://listserv.activestate.com/mailman/mysubs?show=announce by entering your email and selecting Tcl-announce. Conference Committee Jeff Hobbs ActiveState Corp General Chair Gerald Lester Aspen Technology Program Co-chair Kevin Kenny GE Global Research Center Program Co-chair Andreas Kupries ActiveState Corp Andrej Vckovski NetCetera Clif Flynt Noumena Corp Ken Jones Avia Training Larry Virden Chemical Abstracts Service (CAS) Mac Cody Raytheon Company Mark Roseman Sonexis, Inc. Steve Landers Digital Smarties Contact Information tcl2002@-SPAM-.tcl.tk http://www.tcl.tk/community/tcl2002/ -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Fri Feb 22 07:25:47 2002 Received: from [10.0.1.55] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1MDPka31920 for ; Fri, 22 Feb 2002 07:25:46 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] .so files Date: Fri, 22 Feb 2002 05:15:09 -0800 Message-Id: <20020222131509.13028@triqs.com> In-Reply-To: <921316543.20020220133601@usd1.com> References: <921316543.20020220133601@usd1.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry McRae wrote: >I tried to install Mk4py.so on the Linus (v 2.2.18) box for my website. >Just copied in the latest. Date of 2/1/02, Linux version. I try import >Mk4py (from Python 2.1) and get the message: > >ImportError: libstdc++-libc6.2-2.so.3: cannot open shared object file: No >such file or directory > >I don't know Linus or C++, so I have no clue. > >I can import other librarys. This just happens to be used at the core of my >product, so I cannot go any farther. I thought all I would have to do >is copy the current .so file instead of my .dll on my system. On Linux, there are several versions of the C/C++ runtime library (glibc 2.0/2.1/2.2) - it looks like you are loading a build of MK which does not match your environment. You can use "ldd Mk4py.so" to find out. You will have to either rebuild MK on your machine (it's not too hard, something like "cd builds; ../unix/configure --enable-python; make" is likely to work). Or you can upgrade your Linux system. Or you can try to find someone who has a similar setup and has a build ready or is willing to do it for you. Last option is to look for an slightly older build of MK on the www.equi4.com website, which does work with your runtime - you'll need to download and try. Regards, Jean-Claude From andreas.muegge@gmx.de Mon Feb 25 06:54:22 2002 Received: from mx0.gmx.net (mx0.gmx.net [213.165.64.100]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1PCsLa09395 for ; Mon, 25 Feb 2002 06:54:21 -0600 Received: (qmail 12626 invoked by uid 0); 25 Feb 2002 12:54:10 -0000 Date: Mon, 25 Feb 2002 13:54:10 +0100 (MET) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: Metakit MIME-Version: 1.0 X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [217.194.32.8] Message-ID: <3022.1014641650@www53.gmx.net> X-Mailer: WWW-Mail 1.5 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] Quick way to do a substring search on a property Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, in my application the metakit database looks like this: recipe[index:I,name:S,cat1:S,cat2:S,cat3:S,cat4:S,cat5:S,count:S,descr:S] The user should be able to find a "name" by entering parts of it - a typical substring search where searching for "potatoe" will find all recipes with "potatoe" somewhere in the name. My first approach was like this: c4_StringProp pName("name"); for (int i=0; i < nRecipesCount; i++) { const char* szFound = pName ( myView[i]); // ... some tolower() and strstr() stuff } Unfortunately this turns out to be rather slow, on my P3 I can only parse 2000 recipes per second - 25 sec just for retrieving 50'000 names from the database is not acceptable. Is there a better way to do it? Ciao, Andreas From slok00@yahoo.com Tue Feb 26 04:18:25 2002 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1QAIPa32445 for ; Tue, 26 Feb 2002 04:18:25 -0600 Received: from slok00 (AUTH LOGIN) at mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 26 Feb 2002 10:18:20 -0000 Message-Id: <5.1.0.14.0.20020226180255.026bd560@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Tue, 26 Feb 2002 18:03:54 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-68004E7; boundary="=======40956BAE=======" Subject: [Metakit] Metakit support for images? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======40956BAE======= Content-Type: text/plain; x-avg-checked=avg-ok-68004E7; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit Hi, Does MetaKit supports images? If so, how can I do that? Thanks YekSoon --=======40956BAE======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-68004E7 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 --=======40956BAE=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From slok00@yahoo.com Wed Feb 27 08:39:12 2002 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1REdCa13800 for ; Wed, 27 Feb 2002 08:39:12 -0600 Received: from slok00 (AUTH LOGIN) at mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Feb 2002 14:37:30 -0000 Message-Id: <5.1.0.14.0.20020227223043.02db0320@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Wed, 27 Feb 2002 22:35:00 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-3851DF4; boundary="=======B3A1155=======" Subject: [Metakit] tclKit and tclHttpd problem Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======B3A1155======= Content-Type: text/plain; x-avg-checked=avg-ok-3851DF4; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit My apologies if this is not the right mailing list to post with regards to TclKit. But, I am having trouble finding it. I am trying to turn tclHttpd into a single scripted document. I downloaded tclhttpd33.zip from http://tcl.activestate.com:8002/resource/software/tools/www/ and the latest tclkit.exe and follows the instructions at http://equi4.com/pub/tk/examples/tclhttpd.README to turn tclHttpd into a scripted document. However, after all the steps, when I tried running the scripted document (tclhttpd.bat) that I create , I got the following dialog error: == Fatal Error Don't know how to run tclhttpd for tclhttpd === The only thing that I didn't do is download tcllib as I believe it is already included in tclhttpd33.zip btw, I''m trying this on win2k without any wish or Tcl installed. Any ideas what could be wrong? Thanks YekSoon --=======B3A1155======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-3851DF4 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 --=======B3A1155=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From nicolasb@mail.maich.gr Wed Feb 27 09:33:28 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1RFXRa15978 for ; Wed, 27 Feb 2002 09:33:27 -0600 Received: FROM mail.maich.gr BY mail-server ; Wed Feb 27 17:53:17 2002 +0200 Message-ID: <3C7D00FA.6000407@mail.maich.gr> Date: Wed, 27 Feb 2002 17:53:30 +0200 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Lok Yek Soon CC: metakit@equi4.com Subject: Re: [Metakit] tclKit and tclHttpd problem References: <5.1.0.14.0.20020227223043.02db0320@pop.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I just tried it and had no problem.. Here's what I did 1. Extract tclhttpd3.3 to c:\ 2. rename to tclhttpd3.3.vfs 3. Add a file in c:\tclhttpd.vfs\bin called main.tcl containing set env(TCL_HTTPD_LIBRARY) [file join $scripdoc::self lib] source [file join $scripdoc::self bin httpd.tcl] 4. Create a batch file, sdxcreate.bat containing @tclkit sdx %1 %2 %3 %4 %5 %6 %7 %8 %9 5. Copy sdx.bin into c:\ and renameto sdx 6. Create a file called tclhttpd.bat in c:\ with the following @tclkit tclhttpd %1 %2 %3 %4 %5 %6 %7 %8 %9 7. in c:\ type in sdxcreate fs2sd tclhttpd after the Tk window comes up, you know it's done 8. Run tclhttpd.bat Note, tclkit.exe should be in c:\, or at least where system can findit or edit the batch file with fullpath. Note, the current tclhttpd3.3.1, looks for tcllib in the same level as tclhttpd. So either adjust as necessary, or use tcllib0.8 Hope this helps, Nicolas Lok Yek Soon wrote: > My apologies if this is not the right mailing list to post with > regards to TclKit. > But, I am having trouble finding it. > > I am trying to turn tclHttpd into a single scripted document. > I downloaded tclhttpd33.zip from > http://tcl.activestate.com:8002/resource/software/tools/www/ > and the latest tclkit.exe and follows the instructions at > http://equi4.com/pub/tk/examples/tclhttpd.README > to turn tclHttpd into a scripted document. > > However, after all the steps, when I tried running the scripted > document (tclhttpd.bat) that I create , > I got the following dialog error: > > == > Fatal Error > Don't know how to run tclhttpd for tclhttpd > === > > The only thing that I didn't do is download tcllib as I believe it is > already included in tclhttpd33.zip > > btw, I''m trying this on win2k without any wish or Tcl installed. > > Any ideas what could be wrong? > > Thanks > YekSoon > > >------------------------------------------------------------------------ > > >--- >Outgoing mail is certified Virus Free. >Checked by AVG anti-virus system (http://www.grisoft.com). >Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 > From slok00@yahoo.com Fri Mar 1 02:38:27 2002 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g218cRa05837 for ; Fri, 1 Mar 2002 02:38:27 -0600 Received: from slok00 (AUTH LOGIN) at mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 1 Mar 2002 08:31:57 -0000 Message-Id: <5.1.0.14.0.20020301161228.03372db0@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 01 Mar 2002 16:29:23 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-2435493D; boundary="=======7876096=======" Subject: [Metakit] tclhttpd's cgi-bin not working in scripted document. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======7876096======= Content-Type: text/plain; x-avg-checked=avg-ok-2435493D; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit The cgi-bin in the scripted doc "tclhttpd" that I created does not work. It return the following message when I request for the env.cgi that comes with tclhttpd. Erro Message on the browser as follows: = Got the error Bad Request while trying to obtain /cgi-bin/env.cgi/extra/path couldn't execute "/home/test/tclhttpd3.3-dist/tclhttpd/htdocs/cgi-bin/env.cgi": not a directory ===== These are the steps I do to create the scripted document tclhttpd 1. Extract tclhttpd3.3-dist.tar.gz to /home/test 2. cd tclhttpd3.3-dist 3. rename tclhttpd3.3 to tclhttpd.vfs 4. mv tcllib0.8 to tclhttpd.vfs/lib 5. add a file in c:\tclhttpd.vfs\bin called main.tcl containing set env(TCL_HTTPD_LIBRARY) [file join $scripdoc::self lib] source [file join $scripdoc::self bin httpd.tcl] 6.. copy tclkit and sdx.bin into tclhttpd3.3-dist 7. type "tclkit sdx.bin fs2sd tclhttpd" to create scripted doc. 8. start scripted document's webserver by typing ./tclhttpd 9. access the localsite http://localhost:8015/cgi-bin/env.cgi The cgi works when run from the webserver which is not in scripted doc. Is there any steps that needs to be done to make this works? Thanks YekSoon after the Tk window comes up, you know it's done 8. Run tclhttpd.bat --=======7876096======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-2435493D Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 --=======7876096=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From DavidA@ActiveState.com Sat Mar 2 00:05:01 2002 Received: from smtp1.ActiveState.com (gw.activestate.com [209.17.183.249]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g22650a04159 for ; Sat, 2 Mar 2002 00:05:01 -0600 Received: from smtp3.ActiveState.com (smtp3.ActiveState.com [192.168.3.19]) by smtp1.ActiveState.com (8.11.6/8.11.6) with ESMTP id g2264oV29799 for ; Fri, 1 Mar 2002 22:04:51 -0800 Received: from ActiveState.com (vpn6.ActiveState.com [192.168.3.236]) by smtp3.ActiveState.com (8.11.6/8.11.6) with ESMTP id g2264nL13910 for ; Fri, 1 Mar 2002 22:04:49 -0800 Message-ID: <3C806C94.FBF30FF6@ActiveState.com> Date: Fri, 01 Mar 2002 22:09:24 -0800 From: David Ascher X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: metakit Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Filtered-By: PerlMx makes it fast and easy. See http://www.ActiveState.com/Products/PerlMx/Header Subject: [Metakit] concurrent access layer for metakit/python? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm looking at using metakit in a web site (currently under Apache/Windows/"simple CGI", but I'd like the solution to work in a variety of web env'ts, such as mod_python). The site isn't high-traffic, so it's ok to serialize database accesses even relatively 'primitively'. My primary concern is to avoid data corruption due to the unpredictable timings of web hits, not performance. Are there any tricks that people use to make coding that kind of access as easy as possible? --david From jcw@equi4.com Sun Mar 3 03:27:10 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g239R9a25797 for ; Sun, 3 Mar 2002 03:27:09 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] concurrent access layer for metakit/python? Date: Sun, 3 Mar 2002 10:26:57 +0100 Message-Id: <20020303092657.21014@triqs.com> In-Reply-To: <3C806C94.FBF30FF6@ActiveState.com> References: <3C806C94.FBF30FF6@ActiveState.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: David Ascher wrote: >I'm looking at using metakit in a web site (currently under >Apache/Windows/"simple CGI", but I'd like the solution to work in a >variety of web env'ts, such as mod_python). > >The site isn't high-traffic, so it's ok to serialize database accesses >even relatively 'primitively'. My primary concern is to avoid data >corruption due to the unpredictable timings of web hits, not >performance. > >Are there any tricks that people use to make coding that kind of access >as easy as possible? Hi David, I can't think of a simple way to do it in the general case. For simple CGI (which is what I use in the Tcl'ers Wiki, plus page caching), I just lock/open-db/use/close/unlock. With a bit of logic to get rid of dangling locks (the database is safe): store PID and check /proc/PID to break locks. For more demanding uses, I'd use TclHttpd/AolServer (or a Python equivalent in your case) as server and run MK in it - that way all page accesses are properly serialized. There is a considerable amount of caching in MK, meaning that the lock really has to around the entire open/use/close group. Lastly, there is a "commit-extend mode" (open mode 2), which a single process can use while all others open read-only. That grows a datafile *very* quickly, but is a way as well and fast. It is badly documented, but solid nevertheless. The idea is that each process can do: open ro handle req if just reading if writing is needed: close lock, waiting open extend alter close release The only thing you need to do is occasionally compact the file: open ro serialize contents to a new file close replace original with newly created compact file The solution I have in mind for high-performance concurrent access is a combination of commit-extend and commit-aside (which saves diffs efficiently), but the implementation of that is not finished. This design will allow multiple-reader single writer with little overhead and no locking at all. For now, if I were you, I'd start off with the CGI model and full locks in Python. Changes to the locking model later on need IMO not affect much of the application coding. -jcw From jcw@equi4.com Sun Mar 3 10:17:22 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g23GHLa06207 for ; Sun, 3 Mar 2002 10:17:21 -0600 From: Jean-Claude Wippler To: metakit Date: Sun, 3 Mar 2002 17:17:15 +0100 Message-Id: <20020303161715.16068@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] TclKit status Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For those interested in TclKit (the single-file Tcl/Tk/MK runtime), here is a brief status report: - From now on, TclKit will have IncrTcl included (but not incrTk). - The latest source has been checked into the "tclkit" project (i.e. as a CVS project on equi4.com). It is no longer part of the MetaKit distribution - this seems more logical, long term. - TclKit now builds using the latest *unmodified* Tcl, Tk, IncrTcl, TclVFS, and MetaKit distributions. I intend to track these. - The 8.4-26 release that has been around for months in /pub/tk/ uses a pure-tcl VFS scheme. The new system uses the latest core Virtual File System as now present in the official Tcl 8.4a4. - One effect is that all file calls will be fully compatible with plain Tcl systems. Another effect is that Vince Darley's TclVFS is used for scripted documents and that his http:// logic works. See for some examples. Lastly, the performance is up (way up for large dirs, due to more caching). - There are new binaries in http://www.equi4.com/previews/tclkit.*, as of today. These builds for Linux and Windows really are very close to becoming the "official" TclKit release. Still to do: * tweak how shared libs are placed into temp and cleaned up * some more testing to verify good backward compatibility * a recently discovered readlink bug must be fixed (in VFS) - The current builds no longer have threading enabled, but this is temporary. TclKit fully supports Tcl's threading (including MK). I just need to figure out config/build issues again and test it. - Daniel Steffen recently built a TclKit for Mac, using Carbon for use with OS 8/9 and with X, but also "fat", i.e. PPC *and* 68K. I'm tracking MacTcl issues on , and intend to create a MacOS X version of TclKit in the near future. Both MetaKit and IncrTcl work fine, I just need to get to it. - In an attempt to keep (regain?) my sanity, I will only provide builds for Windows, Linux, and Macintosh from now on. Binaries for other platforms will be gratefully accepted and included in the equi4.com download area - as will all source code tweaks. My goal is still to have TclKit (and MetaKit) running on as many platforms as possible - but I have to depend on others for that. If you use TclKit under Windows or Linux, please give the latest builds a good workout. I will fix issues as soon as possible, and hope to announce solid revisions as soon as the latest Tcl/Tk revs are frozen (I *think* it's going to be called 8.4b1). -jcw From rjent@rjent.pair.com Sun Mar 3 20:39:23 2002 Received: from portal2.isz ([204.184.227.216]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g242dLa18355 for ; Sun, 3 Mar 2002 20:39:22 -0600 Received: (from nobody@localhost) by portal2.isz (8.11.6/8.11.6) id g242hOa06298; Sun, 3 Mar 2002 20:43:24 -0600 (CST) (envelope-from rjent@rjent.pair.com) Date: Sun, 3 Mar 2002 20:43:24 -0600 (CST) Message-Id: <200203040243.g242hOa06298@portal2.isz> X-Authentication-Warning: portal2.isz: nobody set sender to rjent@rjent.pair.com using -f From: "RJ Ent." To: metakit@equi4.com X-Mailer: NeoMail 1.25 with (Patch-2a by Nelson) X-IPAddress: 192.168.2.3 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] Multiuser version? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, I am curious about the potential for a multiuser version of metakit. Many thanks! From slok00@yahoo.com Tue Mar 5 01:22:08 2002 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g257M8a21639 for ; Tue, 5 Mar 2002 01:22:08 -0600 Received: from slok00 (AUTH login) at mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 5 Mar 2002 07:22:07 -0000 Message-Id: <5.1.0.14.0.20020305151625.02d527b0@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Tue, 05 Mar 2002 15:17:46 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-29D74A98; boundary="=======71D1585B=======" Subject: [Metakit] tclkit.exe and tclkitsh.exe Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======71D1585B======= Content-Type: text/plain; x-avg-checked=avg-ok-29D74A98; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit On windows, what's the difference between tclkit.exe and tclkitsh.exe? Under what circumstances do you use each? Thanks YekSoon --=======71D1585B======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-29D74A98 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 --=======71D1585B=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From nicole@bwh.harvard.edu Tue Mar 5 17:12:21 2002 Received: from clifford.bwh.harvard.edu (clifford.bwh.harvard.edu [134.174.9.41]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g25NCLa02579 for ; Tue, 5 Mar 2002 17:12:21 -0600 Received: from tango.bwh.harvard.edu (tango [134.174.9.35]) by clifford.bwh.harvard.edu (8.10.2+Sun/8.11.0) with ESMTP id g25NBOu12039 for ; Tue, 5 Mar 2002 18:11:25 -0500 (EST) Received: from localhost (nicole@localhost) by tango.bwh.harvard.edu (8.10.2+Sun/8.11.0) with ESMTP id g25NBOS10252 for ; Tue, 5 Mar 2002 18:11:24 -0500 (EST) Date: Tue, 5 Mar 2002 18:11:24 -0500 (EST) From: Nicole Aucoin To: metakit@equi4.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] MetaKit and cgi scripting Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, I'm fairly new to tcl and I'm almost positive that this is a question with a simple answer, but I've been unable to dig it up (web searches and a grep of the archives of this list) or figure it out on my own, so please bear with me. I have a fairly low access volume web script that I'd like to have use Mk4tcl. I can run the cgi script that calls the tcl script with no problems from my Solaris account, but when it's called from the web page with a user id of nobody, it can't load Mk4tcl.so. I catch the following error: couldn't load file "./Mk4tcl.so": ld.so.1: /local/os/bin/tclsh: fatal: libstdc++.so.2.10.0: open failed: No such file or directory I've tried setting env(LD_LIBRARY_PATH) and env(TCLLIBPATH) to the directory that this file lives in, but no success. I downloaded the latest Solaris Mk4tcl.so from the web site, will I need to compile the source and link it statically to get this to work? thanks, Nicole From reza@psych.utoronto.ca Wed Mar 6 11:13:25 2002 Received: from brodmann (CPE000102e3f968.cpe.net.cable.rogers.com [24.157.195.240]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g26HDLa13415 for ; Wed, 6 Mar 2002 11:13:25 -0600 Received: from [127.0.0.1] by brodmann (ArGoSoft Mail Server Freeware, Version 1.804a (1.8.0.4)); Wed, 6 Mar 2002 12:13:20 -0500 Date: Wed, 6 Mar 2002 12:13:18 -0500 From: Reza Habib X-Mailer: The Bat! (v1.53d) Personal Reply-To: Reza Habib Organization: Rotman Research Institute X-Priority: 3 (Normal) Message-ID: <17513781146.20020306121318@psych.utoronto.ca> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] possible bug? [MK 2.4.3, win2000, mingw 2.95.3-6] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi. I'm new to metakit. I think I have found a bug or maybe I'm doing something wrong (the subject shows the version and platform I'm using). Any help would be appreciated. I have a data structure that looks like this: #define FORMAT "Subject[Name:S,Number:I,Age:I,PresRate:I,ISI:I,Trials:I,List[Words_Section:I,Replace_Section:I],Random:I,Stimuli[Section:I,ListEntered:I,ListDeparted:I,Type:S,Word:S]]" I then get a view to this structure like this: c4_View vSubject = sDataFile.GetAs(FORMAT); and then a view to the subview 'Stimuli' like this: c4_ViewProp pStimuli = (c4_ViewProp&)vSubject.NthProperty(8); c4_View vWords = pStimuli(vSubject[1]); Then I add some rows to this view and use select to get two different views to this view like this: c4_View vSection = vWords.Select(pSection[i+1] + pType["Study"]); c4_View vReplace = vWords.Select(pSection[0] + pType["Replacement"]); I then want to change some of the values in these views. I do so like this: int r = rand() % vSection.GetSize(); //To get a random row c4_Row TempRow; //Get a row and set some properties pSection(TempRow) = i+1; pListEntered(TempRow) = CurrentTrial; pListDeparted(TempRow) = 0; pType(TempRow) = "Study"; pWord(TempRow) = pWord(vReplace[0]); vSection[r] = TempRow; //Assign row to view sDataFile.Commit(); //Save changes c4_Row TempRow2; //Same as above pSection(TempRow2) = 0; pListEntered(TempRow2) = 1;//pListEntered(vSection[r]); pListDeparted(TempRow2) = CurrentTrial; pType(TempRow2) = "Replaced"; pWord(TempRow2) = "Test";//pWord(vSection[r]); vReplace[0] = TempRow2; sDataFile.Commit(); Now, the first commit works as expected. That is the value in the correct view is changed, however, the second commit that is supposed to change the subview pointed to by vReplace ends up changing the subview pointed to by vSection. Here is a bit of the output from dump: This is what the file looks like before anything happens. 0: subview 'Stimuli' VIEW 196 rows = Section:I ListEntered:I ListDeparted:I Type:S Word:S 0: 1 1 0 'Study' 'purely' 1: 1 1 0 'Study' 'vacuum' 2: 1 1 0 'Study' 'warmth' 3: 1 1 0 'Study' 'affair' 4: 2 1 0 'Study' 'settle' 5: 2 1 0 'Study' 'worthy' .... 12: 0 0 0 'Replacement' 'phrase' 13: 0 0 0 'Replacement' 'uneasy' 14: 0 0 0 'Replacement' 'utopia' 15: 0 0 0 'Replacement' 'resist' Then, after the first commit, the first part (rows 1 - 5) should be changed but not the second part. This output shows that this happens: 0: 1 2 0 'Study' 'phrase' <------- this line changed 1: 1 1 0 'Study' 'vacuum' 2: 1 1 0 'Study' 'warmth' 3: 1 1 0 'Study' 'affair' 4: 2 1 0 'Study' 'settle' 5: 2 1 0 'Study' 'worthy' .... 12: 0 0 0 'Replacement' 'phrase' 13: 0 0 0 'Replacement' 'uneasy' 14: 0 0 0 'Replacement' 'utopia' 15: 0 0 0 'Replacement' 'resist' Now, after the second commit, the first row in the second part (row 12) should be changed. As you can see, this is not what happens, but rather the first row of the first part is changed: 0: 0 2 2 'Replaced' 'phrase' <----This was changed but shouldn't have been 1: 1 1 0 'Study' 'vacuum' 2: 1 1 0 'Study' 'warmth' 3: 1 1 0 'Study' 'affair' 4: 2 1 0 'Study' 'settle' 5: 2 1 0 'Study' 'worthy' .... 12: 0 0 0 'Replacement' 'phrase' <---- This should have been changed 13: 0 0 0 'Replacement' 'uneasy' 14: 0 0 0 'Replacement' 'utopia' 15: 0 0 0 'Replacement' 'resist' I did various tests to check to see whether I was doing something wrong. If I use the GetIndexOf() function to find the row number pointed to by vReplace[0], I get the correct answer either when I check within vReplace (i.e. vReplace.GetIndexOf(vReplace[0] = 0) or within the vWords view (i.e. vWords.GetIndexOf(vReplace[0]) = 12). Now, have I done something wrong in assigning to the row of this view or is this a bug? And if the latter, is there a workaround? Thanks (and sorry for the long email). Reza From squeezeboxguy@yahoo.co.uk Wed Mar 6 17:14:55 2002 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g26NEsa02765 for ; Wed, 6 Mar 2002 17:14:54 -0600 Received: from squeezeboxguy (AUTH login) at pc-80-195-147-244-du.blueyonder.co.uk (HELO ?192.168.57.2?) (squeezeboxguy@80.195.147.244) by smtp.mail.vip.sc5.yahoo.com with SMTP; 6 Mar 2002 22:08:39 -0000 From: Jeremy Cope To: metakit@equi4.com Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/1.0.1 Date: 06 Mar 2002 22:08:00 +0000 Message-Id: <1015452522.13955.9.camel@lancre> Mime-Version: 1.0 Subject: [Metakit] Mk4py.dll and Python 2.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is there available somewhere a build of Mk4py.dll for Python 2.2 on windows? I had some trouble with "Cannot find a required DLL" type errors, and was stumped until I had a look at the dll itself and discovered that it expects python 2.1 (ie python21.dll). I tried a quick hack with a hex editor, but this doesn't seem to have worked, so I need another solution. Please could someone help? Thanks in advance, Jez :) PS Thanks to everyone involved in development of metakit, it works just great on my Linux box and the interface is so easy to use! -- Fabricate diem, punc. Jeremy Cope, UK _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From genec@goldsmithsinc.com Thu Mar 7 07:44:49 2002 Received: from mail.goldsmithsinc.com ([64.66.89.67]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g27Dima30814 for ; Thu, 7 Mar 2002 07:44:48 -0600 Received: from genec (genec [10.1.1.100]) by mail.goldsmithsinc.com (Switch-2.0.0/Switch-2.0.0) with SMTP id g27DXQq29006; Thu, 7 Mar 2002 07:33:26 -0600 Message-ID: <003101c1c5dd$8dc2c240$6401010a@genec> From: "Gene Christian" To: "Jeremy Cope" , References: <1015452522.13955.9.camel@lancre> Subject: Re: [Metakit] Mk4py.dll and Python 2.2 Date: Thu, 7 Mar 2002 07:39:48 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I ran into the same thing, and somebody on this list recommended compiling it using mingw (which was something of an experience :-). I can email the dll to you if you would like (it runs 453K, so let me know if you me to send it). I have compiled a couple of python extensions using mingw, and it has worked very well so far. If you try it, you will want the msys (?) system that emulates a unix shell environment (so you can run the configure scripts that come with many packages). Gene ----- Original Message ----- From: "Jeremy Cope" To: Sent: Wednesday, March 06, 2002 4:08 PM Subject: [Metakit] Mk4py.dll and Python 2.2 > Is there available somewhere a build of Mk4py.dll for Python 2.2 on > windows? I had some trouble with "Cannot find a required DLL" type > errors, and was stumped until I had a look at the dll itself and > discovered that it expects python 2.1 (ie python21.dll). I tried a quick > hack with a hex editor, but this doesn't seem to have worked, so I need > another solution. Please could someone help? > > Thanks in advance, Jez :) > > PS Thanks to everyone involved in development of metakit, it works just > great on my Linux box and the interface is so easy to use! > -- > Fabricate diem, punc. > > Jeremy Cope, UK > > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From squeezeboxguy@yahoo.co.uk Thu Mar 7 08:00:04 2002 Received: from web14704.mail.yahoo.com (web14704.mail.yahoo.com [216.136.224.121]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g27E03a31602 for ; Thu, 7 Mar 2002 08:00:03 -0600 Message-ID: <20020307140002.63250.qmail@web14704.mail.yahoo.com> Received: from [212.219.131.2] by web14704.mail.yahoo.com via HTTP; Thu, 07 Mar 2002 06:00:02 PST Date: Thu, 7 Mar 2002 06:00:02 -0800 (PST) From: Jeremy Cope Subject: Re: [Metakit] Mk4py.dll and Python 2.2 To: Gene Christian , metakit@equi4.com In-Reply-To: <003101c1c5dd$8dc2c240$6401010a@genec> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Gene Christian wrote: > I ran into the same thing, and somebody on this list recommended > compiling > it using mingw (which was something of an experience :-). I can > email the > dll to you if you would like (it runs 453K, so let me know if you me > to send > it). Yes, please could you email it to me, this address will be fine. I would have given mingw a go myself, but I don't have enough free rein on the college computers, and I run Linux at home. One of these days I'll get round to setting up a cross compiler. But I'll shut up now before I go seriously offtopic :) Thanks very much, Jez ===== "Fabricate diem, punc" - Motto, Ankh-Morpork City Watch Jeremy Cope ICQ: 33490230; MSN: jezcope@yahoo.com; Jabber: jezcope@jabber.com __________________________________________________ Do You Yahoo!? Try FREE Yahoo! Mail - the world's greatest free email! http://mail.yahoo.com/ From jcw@equi4.com Sun Mar 10 17:23:41 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANNea03568 for ; Sun, 10 Mar 2002 17:23:41 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Mk4py.dll and Python 2.2 Date: Mon, 11 Mar 2002 00:23:18 +0100 Message-Id: <20020310232318.17175@triqs.com> In-Reply-To: <20020307140002.63250.qmail@web14704.mail.yahoo.com> References: <20020307140002.63250.qmail@web14704.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeremy Cope wrote: Gene Christian wrote: [we want Mk4py for Py2.2] The build I had was still erroneously linking against 2.1 - I've now removed 2.1 so my MSVC project only finds 2.2. New builds have been placed in Note that this is based on the latest CVS code, not 2.4.3 - but I don't want to move to a new release number just yet. -jcw P.S. Anyone using MK on Win32 with NTFS should check the CHANGE log - there has been a bug fix which addresses this particular combination. From jcw@equi4.com Sun Mar 10 17:46:34 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkXa03970 for ; Sun, 10 Mar 2002 17:46:33 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] MetaKit and cgi scripting Date: Mon, 11 Mar 2002 00:37:52 +0100 Message-Id: <20020310233753.1574@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicole Aucoin wrote: >I have a fairly low access volume web script that I'd like to have use >Mk4tcl. I can run the cgi script that calls the tcl script with no >problems from my Solaris account, but when it's called from the web page >with a user id of nobody, it can't load Mk4tcl.so. I catch the following >error: > >couldn't load file "./Mk4tcl.so": ld.so.1: /local/os/bin/tclsh: fatal: >libstdc++.so.2.10.0: open failed: No such file or directory > >I've tried setting env(LD_LIBRARY_PATH) and env(TCLLIBPATH) to the >directory that this file lives in, but no success. I downloaded the latest >Solaris Mk4tcl.so from the web site, will I need to compile the source and >link it statically to get this to work? I'm assuming it works one way and not the other, all on the same machine. Your webserver may be restricting access in many different ways. If it does a "chroot()", then that means you are running in a new area - you will need to check that it contains all the shared libraries. One way to find out, is to code a CGI script which returns the output of "ldd Mk4tcl.so", i.e. a list of shared library linkages and how they are being resolved. Or there is a UID permission issue. Neither of these areally related to MK, I'm afraid. It may well be that no other C++ shared libs have been run from the webserver, in which case this is a C++ / Webserver issue. An alternative is to have a look at TclKit, which combines Tcl, Tk, and MK in a single standalone executable. That build has no dynamic linkage (othe rthan std X11). There are builds for Solaris, see www.equi4.com/ pub/tk/. -jcw From jcw@equi4.com Sun Mar 10 17:46:36 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkZa03974 for ; Sun, 10 Mar 2002 17:46:35 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Metakit support for images? Date: Mon, 11 Mar 2002 00:38:08 +0100 Message-Id: <20020310233808.27229@triqs.com> In-Reply-To: <5.1.0.14.0.20020226180255.026bd560@pop.mail.yahoo.com> References: <5.1.0.14.0.20020226180255.026bd560@pop.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Lok Yek Soon wrote: >Does MetaKit supports images? If so, how can I do that? Images are binary data. Store them in fields of type "B", e.g. in Tcl: mk::view layout db.images {name data:B} -jcw From jcw@equi4.com Sun Mar 10 17:46:37 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkaa03977 for ; Sun, 10 Mar 2002 17:46:37 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Multiuser version? Date: Mon, 11 Mar 2002 00:38:17 +0100 Message-Id: <20020310233817.23131@triqs.com> In-Reply-To: <200203040243.g242hOa06298@portal2.isz> References: <200203040243.g242hOa06298@portal2.isz> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: RJ Ent. wrote: >I am curious about the potential for a multiuser version of metakit. The only multi-user scenario's supported by metakit are client/server (i.e. all access of MK datafiles in a single server process/thread), or the use of commit-extend (with suitable extra locking). See the MK wiki for more info: -jcw From jcw@equi4.com Sun Mar 10 17:46:39 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkca03980 for ; Sun, 10 Mar 2002 17:46:38 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] tclkit.exe and tclkitsh.exe Date: Mon, 11 Mar 2002 00:38:25 +0100 Message-Id: <20020310233825.19574@triqs.com> In-Reply-To: <5.1.0.14.0.20020305151625.02d527b0@pop.mail.yahoo.com> References: <5.1.0.14.0.20020305151625.02d527b0@pop.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Lok Yek Soon wrote: >On windows, what's the difference between tclkit.exe and tclkitsh.exe? >Under what circumstances do you use each? - tclkit includes Tk, and runs as GUI application - tclkitsh does not include Tk, and runs in console mode -jcw From jcw@equi4.com Sun Mar 10 17:46:41 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkea03984 for ; Sun, 10 Mar 2002 17:46:40 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] possible bug? [MK 2.4.3, win2000, mingw 2.95.3-6] Date: Mon, 11 Mar 2002 00:46:26 +0100 Message-Id: <20020310234626.6218@triqs.com> In-Reply-To: <17513781146.20020306121318@psych.utoronto.ca> References: <17513781146.20020306121318@psych.utoronto.ca> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Reza Habib wrote: >Hi. I'm new to metakit. I think I have found a bug or maybe I'm >doing something wrong (the subject shows the version and platform I'm >using). Any help would be appreciated. > >I have a data structure that looks like this: (Please keep your examples short - I can't deal with too much code...) >#define FORMAT "Subject[Name:S,Number >:I,Age:I,PresRate:I,ISI:I,Trials:I,Li >st[Words_Section:I,Replace_Section:I] >,Random:I,Stimuli[Section:I,ListEntered:I,ListDeparted:I,Type:S,Word:S]]" > >I then get a view to this structure like this: > > c4_View vSubject = sDataFile.GetAs(FORMAT); [...] > vSection[r] = TempRow; //Assign row to view > sDataFile.Commit(); //Save changes [...] > vReplace[0] = TempRow2; > sDataFile.Commit(); > >Now, the first commit works as expected. That is the value in the >correct view is changed, however, the second commit that is supposed >to change the subview pointed to by vReplace ends up changing the >subview pointed to by vSection. This may be the dreaded problem of MK losing track of "attached" vs. "unattached" views. I have two questions: - what happens if you comment out the first commit? - what happens if you do a commit after the GetAs? actually, to be really sure, replace it by the following: sDataFile.GetAs(FORMAT); sDataFile.Commit(); c4_View vSubject = sDataFile.View("Subject"); does that solve the commit weirdness? -jcw From jcw@equi4.com Sun Mar 10 18:39:21 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2B0dJa04805 for ; Sun, 10 Mar 2002 18:39:20 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Mk4Tcl concurrency Date: Mon, 11 Mar 2002 01:39:14 +0100 Message-Id: <20020311003914.30196@triqs.com> In-Reply-To: <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> References: <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Roseman wrote: >As an example, our web-based app runs under AOLserver, and we >keep a lot of central data held in Metakit databases. The >multiple AOLserver threads make socket connections to a separate >single-threaded process running the database, so in effect >requests have to wait their turn to be processed. Despite some >fairly substantial queries and reasonable numbers of simultaneous >users, we've never had issues with hugely perceptible delays on >the client side. Note that 2.4.x builds of Mk4tcl (can't remember when it was introduced) support safe access to it from a threaded Tcl environment. The contention is resolved through one big mutex preventing entry to all but one thread inside Mk4tcl. Simple, but effective (single process still, just for MT). This only has value with "mk::file open db file.dat -shared", which will expose "db" to all interpreters. In fact, this could be used as a way to transfer data between interps (note that file.dat can be omitted, thus turning this into an in-mem shared data structure). -jcw From jerry@usd1.com Mon Mar 11 13:41:54 2002 Received: from bfish.hurrah.com (bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2BJfsa07949 for ; Mon, 11 Mar 2002 13:41:54 -0600 Received: from 12-225-195-8.client.attbi.com (12-225-195-8.client.attbi.com [12.225.195.8]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id g2BJfoG15859 for ; Mon, 11 Mar 2002 11:41:50 -0800 Date: Mon, 11 Mar 2002 11:26:00 -0800 From: Jerry McRae X-Mailer: The Bat! (v1.53d) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <17100167637.20020311112600@usd1.com> To: Metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] subclassing MK types in py2.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I was experimenting with Python 2.2 and attempting to define a class inheriting RowRef and View types, and was not successful. I got this: >>> class vwtest(mk.RowRefType): ... pass ... Traceback (most recent call last): File "", line 1, in ? TypeError: type 'PyRowRef' is not an acceptable base type Is this possible? I would like to add a smart dictionary, but if it is not there, to use the original methods. -- Thanks, Jerry McRae MetaKit 2.4.3, Python 2.2, wxPython 2.2.5, Win98, 384M RAM From gmcm@hypernet.com Mon Mar 11 14:45:32 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2BKjWa12044 for ; Mon, 11 Mar 2002 14:45:32 -0600 Received: from PARANOIA (204.176.40.68) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Mon, 11 Mar 2002 15:45:39 -0500 From: "Gordon McMillan" To: Jerry McRae Date: Mon, 11 Mar 2002 15:45:51 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] subclassing MK types in py2.2 Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3C8CD12F.5277.810307A4@localhost> Priority: normal In-reply-to: <17100167637.20020311112600@usd1.com> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 11 Mar 2002 at 11:26, Jerry McRae wrote: > Hi, > > I was experimenting with Python 2.2 and attempting > to define a class inheriting RowRef and View types, > and was not successful. I got this: [snip] No. These are still old-fashioned types. Mk4py already plays some type object games. I don't know what it will take to make them new-style type objects. -- Gordon http://www.mcmillan-inc.com/ From marc.vollmer@handshake.de Mon Mar 11 15:21:49 2002 Received: from hs-gate.handshake.de (root@hs-gate.handshake.de [193.141.176.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2BLLna14024 for ; Mon, 11 Mar 2002 15:21:49 -0600 Received: from linux (hs3-200.handshake.de [193.141.176.200]) by hs-gate.handshake.de (8.9.3/8.9.3) with SMTP id WAA03392 for ; Mon, 11 Mar 2002 22:21:44 +0100 From: Marc Vollmer Reply-To: marc.vollmer@handshake.de To: metakit@equi4.com Date: Mon, 11 Mar 2002 22:04:24 +0100 X-Mailer: KMail [version 1.0.29.2] Content-Type: text/plain MIME-Version: 1.0 Message-Id: <02031122252200.04170@linux> Content-Transfer-Encoding: 8bit Subject: [Metakit] Newbie: How can I find a property? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I don't understand, how I can find a value of a proberty: My database is: Food.db: 1 properties element[bez:S,kal:F,fet:F,koh:F,eiw:F] VIEW 1 rows = element:V 0: subview 'element' VIEW 2 rows = bez:S kal:F fet:F koh:F eiw:F 0: 'Maultaschen schwäbisch' 180.1 11.6 8.9 10 1: 'Butter' 741.9 83.2 0.6 0.7 I search the Property "Butter", so I open the database and search: const char * tmp1; c4_Storage fooddb ("Food.db", false); c4_View v1 = fooddb.View("element"); c4_RowRef row = v1.ElementAt (0); c4_Property prop = v1.NthProperty(0); for (int i=0;i; Mon, 11 Mar 2002 15:40:00 -0600 Received: from squeezeboxguy (AUTH login) at pc-80-195-147-244-du.blueyonder.co.uk (HELO ?192.168.57.2?) (squeezeboxguy@80.195.147.244) by smtp.mail.vip.sc5.yahoo.com with SMTP; 11 Mar 2002 21:39:59 -0000 Subject: Re: [Metakit] Newbie: How can I find a property? From: Jeremy Cope To: marc.vollmer@handshake.de, metakit@equi4.com In-Reply-To: <02031122252200.04170@linux> References: <02031122252200.04170@linux> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/1.0.2 Date: 11 Mar 2002 21:39:28 +0000 Message-Id: <1015882810.8524.14.camel@lancre> Mime-Version: 1.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Phew! I'm impressed, I wouldn't have managed that! To start off with, you need to use the specifically type property classes, in this case c4_StringProp, rather than the base c4_Property. I don't think it's necessary to use NthProperty() at all unless you're writing something generic like the dump utility in the source distribution. I would go about it like this: c4_Storage fooddb("Food.db", false); c4_View v1 = fooddb.View("element"); c4_StringProp prop("bez"); c4_View v2 = v1.Select(prop["Butter"]); Then, you'll have a view v2 which contains all the rows where the "bez" property is "Butter", to do with what you will. Now I just hope the code above actually compiles and does what I think it does, in order to save myself from total embarressment. Try looking at the examples on the website (http://www.equi4.com/metakit/intro.html#codingExamples). The syntax isn't obvious at first, but it's got a rundown of how to do what. Hope that helps, Jez :) -- Fabricate diem, punc. Jeremy Cope, UK _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From marc.vollmer@handshake.de Tue Mar 12 15:13:08 2002 Received: from hs-lan.handshake.de (hs-lan.handshake.de [193.141.176.12]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2CLD7a15455 for ; Tue, 12 Mar 2002 15:13:08 -0600 Received: from linux (hs3-247.handshake.de [193.141.176.247]) by hs-lan.handshake.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with SMTP id WAA05594 for ; Tue, 12 Mar 2002 22:12:10 +0100 From: marc.vollmer@handshake.de To: metakit@equi4.com Subject: Re: [Metakit] Newbie: How can I find a property? Date: Tue, 12 Mar 2002 21:44:42 +0100 X-Mailer: KMail [version 1.0.29.2] Content-Type: text/plain References: <1015882810.8524.14.camel@lancre> In-Reply-To: <1015882810.8524.14.camel@lancre> MIME-Version: 1.0 Message-Id: <02031221524200.00796@linux> Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks. Now, I understand a litte more. Am Mon, 11 Mär 2002 schrieb Jeremy Cope: > c4_Storage fooddb("Food.db", false); > c4_View v1 = fooddb.View("element"); > c4_StringProp prop("bez"); > c4_View v2 = v1.Select(prop["Butter"]); It works fine, but I search the substring "Butter" or "butter". I added mydatabase: Food.db: 1 properties element[bez:S,kal:F,fet:F,koh:F,eiw:F] VIEW 1 rows = element:V 0: subview 'element' VIEW 3 rows = bez:S kal:F fet:F koh:F eiw:F 0: 'Maultaschen schwäbisch' 180.1 11.6 8.9 10 1: 'Butter' 741.9 83.2 0.6 0.7 2: 'Butterkäse' 299 23.5 0 21.7 By Select I find only item 1. And by Find: int count; count = v1.Find(prop1["Butter"]); count = v1.Find(prop1["Butter"]); The result is item 1 and item 1, not item 1 and item 2. :-( By Search the result is 0 and 0 :-(( How can I find the item 1 and 2? 1: 'Butter' 741.9 83.2 0.6 0.7 2: 'Butterkäse' 299 23.5 0 21.7 > Try looking at the examples on the website > (http://www.equi4.com/metakit/intro.html#codingExamples). The syntax > isn't obvious at first, but it's got a rundown of how to do what. This was a good tip, I have looked only in the HTML-Docs. From jcw@equi4.com Tue Mar 12 17:17:07 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2CNH6a24853 for ; Tue, 12 Mar 2002 17:17:06 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Newbie: How can I find a property? Date: Wed, 13 Mar 2002 00:16:54 +0100 Message-Id: <20020312231654.25608@triqs.com> In-Reply-To: <02031221524200.00796@linux> References: <02031221524200.00796@linux> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g2CNH6a24853 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: marc.vollmer@handshake.de wrote: [...] > VIEW 3 rows = bez:S kal:F fet:F koh:F eiw:F > 0: 'Maultaschen schwäbisch' 180.1 11.6 8.9 10 > 1: 'Butter' 741.9 83.2 0.6 0.7 > 2: 'Butterkäse' 299 23.5 0 21.7 > >By Select I find only item 1. And by Find: > > int count; > count = v1.Find(prop1["Butter"]); > count = v1.Find(prop1["Butter"]); > >The result is item 1 and item 1, not item 1 and item 2. :-( > >By Search the result is 0 and 0 :-(( > >How can I find the item 1 and 2? > > 1: 'Butter' 741.9 83.2 0.6 0.7 > 2: 'Butterkäse' 299 23.5 0 21.7 c4_StringProp pBez("bez"); for (int i = 0; i < v1.GetSize(); ++i) if (stristr(pBez (v1[i]), "butter") != NULL) puts("bingo"); (Assuming "stristr" is a case-insensitive version of strstr - I don't think it exists, but you can code it up, surely). This looks like brute force. It is. But as you will see, the time to perform such tight loops is usually surprisingly good (though O(N)). -jcw From dave@primco.org Thu Mar 14 03:15:14 2002 Received: from minerva.host4u.net (minerva.host4u.net [216.71.64.49]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2E9FDa32761 for ; Thu, 14 Mar 2002 03:15:13 -0600 Received: from redcloud (adsl-63-193-248-16.dsl.snfc21.pacbell.net [63.193.248.16]) by minerva.host4u.net (8.11.6/8.11.6) with ESMTP id g2E9EIC01775 for ; Thu, 14 Mar 2002 03:14:18 -0600 From: "David Primmer" To: Date: Thu, 14 Mar 2002 01:11:29 -0800 Message-ID: <000001c1cb38$3d93fb80$6401a8c0@redcloud> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] how to do substring selections Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm using python and I have a column called 'utc_datetime' with data like this: 2002-02-22T03:40:26-08:00 And I'd like to select rows based on a substring of utc_datetime but I don't know how to do this in metakit language. I've tried select(utc_datetime[0:10]=2002-02-22) but that didn't work of course. I'm generally very confused as to how to use the criteria. And what type of objects are they? I know that a view is can be used like a list but what about properties? I was disappointed to find out that I couldn't do python code inside the structure definition and it looks like I can't do any python inside the criteria. All I've seen is the overly simple name='John' examples. C'mon! I'm frequently doing selects on substrings in SQL. Have I just missed the examples in the docs or on this list? Should I switch to MetaSQL for this function? Starting to get really frustrated with metakit. Just doesn't seem like anyone's using it with python. Seems like there'd be better docs if there were. davep From slok00@yahoo.com Thu Mar 14 05:52:43 2002 Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g2EBqga08293 for ; Thu, 14 Mar 2002 05:52:42 -0600 Received: from slok00 (AUTH login) at mcns94.docsis216.scvmaxonline.com.sg (HELO hermes.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 14 Mar 2002 11:52:41 -0000 Message-Id: <5.1.0.14.0.20020314194846.00b0cb78@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Thu, 14 Mar 2002 19:51:23 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-6FA557B3; boundary="=======20103AF5=======" Subject: [Metakit] tclkit - scripted doc partial working Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======20103AF5======= Content-Type: text/plain; x-avg-checked=avg-ok-6FA557B3; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit on win2k, I create a scripted doc version of tclhttpd using tclkit.exe the scripted tclhttpd can serve static html files but requesting a standard cgi that comes with it gives me the following error: eg. If I request for env.cgi, which should dumps the environment variables gives me this error dialog: == Error in startup script Couldn't read file "c:/tclhttpd/htdocs/cgi-bin/env.cgi": no such file or directory" === What could be the problem? The main web server script "httpd.tcl" works fine, but a simple "env.cgi" doesn't. Both have "exec tclsh8.3" at the top of the script. YekSoon --=======20103AF5======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-6FA557B3 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.336 / Virus Database: 188 - Release Date: 3/11/2002 --=======20103AF5=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From gmcm@hypernet.com Thu Mar 14 07:01:07 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ED17a12017 for ; Thu, 14 Mar 2002 07:01:07 -0600 Received: from PARANOIA (204.176.40.75) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 14 Mar 2002 08:01:20 -0500 From: "Gordon McMillan" To: "David Primmer" , Date: Thu, 14 Mar 2002 08:01:31 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] how to do substring selections Reply-to: gmcm@hypernet.com Message-ID: <3C9058DB.9530.8ECCFF01@localhost> Priority: normal In-reply-to: <000001c1cb38$3d93fb80$6401a8c0@redcloud> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 14 Mar 2002 at 1:11, David Primmer wrote: > I'm using python and I have a column called > 'utc_datetime' with data like this: > > 2002-02-22T03:40:26-08:00 > > And I'd like to select rows based on a substring of > utc_datetime but I don't know how to do this in > metakit language. > > I've tried select(utc_datetime[0:10]=2002-02-22) but > that didn't work of course. Use the select range form: select({'utc_datetime':'2002-02-22'}, {'utc_datetime':'2002-02-22\377'}) > I'm generally very confused as to how to use the > criteria. And what type of objects are they? Dictionaries of the form {propname=value}, or keyword args propname=value. In sort (and others where value is not applicable), you pass in property objects. You can use view.propname, or construct a property object (2 properties are equal if they have the same name and type). > I know that a view is can be used like a list but > what about properties? What about them? > I was disappointed to find > out that I couldn't do python code inside the > structure definition and it looks like I can't do > any python inside the criteria. Metakit is a C++ library. Use view.filter if you want complex selection logic. You can then use remapwith to get a subset. > All I've seen is the > overly simple name='John' examples. C'mon! I'm > frequently doing selects on substrings in SQL. Have > I just missed the examples in the docs or on this > list? > > Should I switch to MetaSQL for this function? MkSQL doesn't do LIKE, either. > Starting to get really frustrated with metakit. Just > doesn't seem like anyone's using it with python. Not true. > Seems > like there'd be better docs if there were. As with most open source projects, complaints outnumber contributions. -- Gordon http://www.mcmillan-inc.com/ From dave@primco.org Thu Mar 14 07:07:22 2002 Received: from minerva.host4u.net (minerva.host4u.net [216.71.64.49]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ED7Ma12374 for ; Thu, 14 Mar 2002 07:07:22 -0600 Received: from redcloud (adsl-63-193-248-16.dsl.snfc21.pacbell.net [63.193.248.16]) by minerva.host4u.net (8.11.6/8.11.6) with ESMTP id g2ED6QX01817 for ; Thu, 14 Mar 2002 07:06:26 -0600 From: "David Primmer" To: Date: Thu, 14 Mar 2002 05:03:28 -0800 Message-ID: <000301c1cb58$a58aeb70$6401a8c0@redcloud> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-reply-to: Subject: [Metakit] RE: how to do substring selections - now sortrev Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Well I figured this one out for r in myview: if r.utc_datetime[0:10] == '2002-02-22': print r.utc_datetime now my question is, how to do a reverse sort? There's no detail on the arguments of sortrev and google can find no example of where it's used. In my case, I don't want some of the rows in natural order and some reversed, I want them all reversed. BTW, Why isn't there a simple view.sort(property,order) where order is descending or ascending? The docs for the C++ implementation state: This sorts descending on A: sorted = view.SortOnReverse(A, A) but using this argument syntax for python gives me sortedview = myview.sortrev(myview.utc_datetime,myview.utc_datetime) TypeError: Not a sequence Thanks for any help. > -----Original Message----- > From: David Primmer [mailto:dave@primco.org] > Sent: Thursday, March 14, 2002 1:11 AM > To: 'metakit@equi4.com' > Subject: how to do substring selections > > I'm using python and I have a column called 'utc_datetime' with data like > this: > > 2002-02-22T03:40:26-08:00 > > And I'd like to select rows based on a substring of utc_datetime but I > don't know how to do this in metakit language. > > I've tried select(utc_datetime[0:10]=2002-02-22) but that didn't work of > course. > > I'm generally very confused as to how to use the criteria. And what type > of objects are they? I know that a view is can be used like a list but > what about properties? I was disappointed to find out that I couldn't do > python code inside the structure definition and it looks like I can't do > any python inside the criteria. All I've seen is the overly simple > name='John' examples. C'mon! I'm frequently doing selects on substrings in > SQL. Have I just missed the examples in the docs or on this list? > > Should I switch to MetaSQL for this function? > > Starting to get really frustrated with metakit. Just doesn't seem like > anyone's using it with python. Seems like there'd be better docs if there > were. > > davep From squeezeboxguy@yahoo.co.uk Thu Mar 14 07:58:19 2002 Received: from web14708.mail.yahoo.com (web14708.mail.yahoo.com [216.136.224.125]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g2EDwJa15071 for ; Thu, 14 Mar 2002 07:58:19 -0600 Message-ID: <20020314135815.28470.qmail@web14708.mail.yahoo.com> Received: from [212.219.131.2] by web14708.mail.yahoo.com via HTTP; Thu, 14 Mar 2002 05:58:15 PST Date: Thu, 14 Mar 2002 05:58:15 -0800 (PST) From: Jeremy Cope Subject: Fwd: Re: [Metakit] RE: how to do substring selections - now sortrev To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > The docs for the C++ implementation state: > > This sorts descending on A: > sorted = view.SortOnReverse(A, A) > > but using this argument syntax for python gives me > > sortedview = > myview.sortrev(myview.utc_datetime,myview.utc_datetime) > TypeError: Not a sequence sortrev takes two tuples as arguments, the first taking, effectively, arguments for sort (in the usual order), the second taking arguments to sort in reverse order. So for the first argument in this case, you need an empty tuple for the first argument - an empty set of brackets (). For the second argument, you need to pass a tuple with just a single item, which by a quirk of Python syntax means enclosing the item in brackets followed by a comma. So this ought to work: sortedview = myview.sortrev( (), (myview.utc_datetime,) ) The apparently redundant trailing comma in the second argument is necessary. Hope this helps Jez :) ===== "Fabricate diem, punc" - Motto, Ankh-Morpork City Watch Jeremy Cope ICQ: 33490230; MSN: jezcope@yahoo.com; Jabber: jezcope@jabber.com __________________________________________________ Do You Yahoo!? Yahoo! Sports - live college hoops coverage http://sports.yahoo.com/ From dave@primco.org Thu Mar 14 10:32:31 2002 Received: from minerva.host4u.net (minerva.host4u.net [216.71.64.49]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2EGWVa24633 for ; Thu, 14 Mar 2002 10:32:31 -0600 Received: from redcloud (adsl-63-193-248-16.dsl.snfc21.pacbell.net [63.193.248.16]) by minerva.host4u.net (8.11.6/8.11.6) with ESMTP id g2EGVYf04978; Thu, 14 Mar 2002 10:31:34 -0600 From: "David Primmer" To: "'Jeremy Cope'" , Subject: RE: [Metakit] RE: how to do substring selections - now sortrev Date: Thu, 14 Mar 2002 08:28:33 -0800 Message-ID: <000001c1cb75$4d4145a0$6401a8c0@redcloud> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal In-Reply-To: <20020314135642.28007.qmail@web14708.mail.yahoo.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks Jeremy That was almost it. Not sure why but your solution didn't quite work. I had to modify it to sortedview = myview.sortrev((myview.utc_datetime,),(myview.utc_datetime,)) without specifying the first argument, my results would sort on the first property field in my view (which happens to be called "id"). Not sure if it would have "subsorted" by date after id but it works fine with the modification. Davep > > sortrev takes two tuples as arguments, the first taking, effectively, > arguments for sort (in the usual order), the second taking arguments to > sort in reverse order. So for the first argument in this case, you need > an empty tuple for the first argument - an empty set of brackets (). > For the second argument, you need to pass a tuple with just a single > item, which by a quirk of Python syntax means enclosing the item in > brackets followed by a comma. So this ought to work: > > sortedview = myview.sortrev( (), (myview.utc_datetime,) ) > > The apparently redundant trailing comma in the second argument is > necessary. From konrad@minus1.de Sun Mar 17 14:37:57 2002 Received: from moutvdomng1.kundenserver.de (moutvdomng1.kundenserver.de [212.227.126.181]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2HKbua01494 for ; Sun, 17 Mar 2002 14:37:56 -0600 Received: from [212.227.126.161] (helo=mrvdomng0.kundenserver.de) by moutvdomng1.kundenserver.de with esmtp (Exim 3.22 #2) id 16mhPf-0002jU-00 for metakit@equi4.com; Sun, 17 Mar 2002 21:37:55 +0100 Received: from [132.230.131.220] (helo=fohnhab.minus1.de) by mrvdomng0.kundenserver.de with esmtp (Exim 3.22 #2) id 16mhPf-00078T-00 for metakit@equi4.com; Sun, 17 Mar 2002 21:37:55 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16mh4m-00028M-00 for metakit@equi4.com; Sun, 17 Mar 2002 21:16:20 +0100 Date: Sun, 17 Mar 2002 21:16:20 +0100 From: Konrad Anton To: Metakit list Message-ID: <20020317211620.A8073@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] Commitaside-Readonly problems with Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all... I'm experimenting with the highly useful commit-aside modes and have run into a problem when the main file is opened read-only. My sample program creates a storage with a view "v" in it, then reopens the storage read-only and an empty commit-aside file read-write and puts them together. Then it appends a row, commits, appends another row and rolls back. ################################################### #prepare a storage s=metakit.storage("TMPtest.mk",1) v=s.getas("v[x:I]") v.append(x=0) s.commit() #open commit-aside with readonly main storage s=metakit.storage("TMPtest.mk",0) sa=metakit.storage("TMPtest.mka",1) s.aside(sa) v=s.view("v") v.append(x=1) print "appending 1", [r.x for r in v] sa.commit() # s.commit() would raise IOError print "committing", [r.x for r in v] v.append(x=2) print "appending 2", [r.x for r in v] sa.rollback() v=s.view("v") print "rollback aside storage", [r.x for r in v] s.rollback(0) #s.rollback() or s.rollback(1) give same result! v=s.view("v") print "rollback main storage", [r.x for r in v] ################################################### The output is: ################################################### appending 1 [0, 1] committing [0, 1] appending 2 [0, 1, 2] rollback aside storage [0, 1, 2] rollback main storage [0] ################################################### Is there a way I can rollback to the state after committing (second line in output)? Calling rollback() on the aside storage does nothing, whereas rollback(0) on the main storage does a full rollback, seemingly discarding the entire aside storage. I'm using Python 2.1.1 and Metakit 2.4.3-33 (but 2.4.2 gives same result). I can try the CVS version, too, but first I'd like to know if my expectations are right at all. TIA Konrad -- Konrad Anton ; Web: www.minus1.de ; gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: Who are you will you, have just the mushroom cat; remarked, till she remarked. From petitsoldat@rediffmail.com Mon Mar 18 06:28:47 2002 Received: from mailweb22.rediffmail.com ([203.199.83.146]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g2ICSja18373 for ; Mon, 18 Mar 2002 06:28:46 -0600 Received: (qmail 17729 invoked by uid 510); 18 Mar 2002 12:26:01 -0000 Date: 18 Mar 2002 12:26:01 -0000 Message-ID: <20020318122601.17728.qmail@mailweb22.rediffmail.com> Received: from unknown (203.196.128.8) by rediffmail.com via HTTP; 18 Mar 2002 12:26:01 -0000 MIME-Version: 1.0 From: "petitsoldat" Reply-To: "petitsoldat" To: metakit@equi4.com Content-type: text/plain; format=flowed Content-Disposition: inline Subject: [Metakit] Some help with using Metakit for large file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I need some help with using the Metakit C++ library in the best possible way. Here is a description of the problem. It only gives the context but you can skip to the next paragraph without reading it. I am trying to use Metakit for storing and reading a table from disk. The total number of records in the table will be around 1,000,000 while Metakit seems to work well for 100,000 rows per file. It implies that the table must be saved in multiple files (e.g. 10 files of 10,000 records each). Record consists of a string key and object pair with each key having multiple objects (10,000 object on average for each key, may be more) and the object size being 100 KB in some cases. Memory usage is a limiting factor on my computer so, ideally, I would like to limit the memory used. I am thinking of using multiple files and using a block_size parameter to limit the number of records in each file. I looked at the last few months archives and came across the term "blocked column" that if used can make commit() faster. What is a blocked column and how does it make commit faster? Another question I want to ask is on how Metakit does on-demand loading. What portion of the file is in memory at a given point of time, things like how many rows surrounding a row being read, entire rows or just the columns being used, page size, etc? Can someone throw light on these questions. Thanks Tarun From jcw@equi4.com Mon Mar 18 07:04:20 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ID4Ja20036 for ; Mon, 18 Mar 2002 07:04:19 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Some help with using Metakit for large file Date: Mon, 18 Mar 2002 14:00:45 +0100 Message-Id: <20020318130045.20280@triqs.com> In-Reply-To: <20020318122601.17728.qmail@mailweb22.rediffmail.com> References: <20020318122601.17728.qmail@mailweb22.rediffmail.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: petitsoldat wrote: >Here is a description of the problem. It only gives the context >but you can skip to the next paragraph without reading it. I am >trying to use Metakit for storing and reading a table from disk. >The total number of records in the table will be around 1,000,000 >while Metakit seems to work well for 100,000 rows per file. It >implies that the table must be saved in multiple files (e.g. 10 >files of 10,000 records each). Record consists of a string key and >object pair with each key having multiple objects (10,000 object >on average for each key, may be more) and the object size being >100 KB in some cases. Memory usage is a limiting factor on my >computer so, ideally, I would like to limit the memory used. I am >thinking of using multiple files and using a block_size parameter >to limit the number of records in each file. What is the average/expected total size of your data? 1,000,000 x 10,000 x 100 Kb would be terabytes... In general, you are better off storing things in a single file (but 2 Gb is a hard limit, if you don't hit other roadblocks first). >I looked at the last few months archives and came across the term >"blocked column" that if used can make commit() faster. What is a >blocked column and how does it make commit faster? It stores data as a view of subviews, but makes them appear as a single view. Commits are column-wise, so this segmented/blocked format reduces the size of columns. The story is rather more complex nowadays, though - large strings/bindata is stored as "memos", i.e. not columnwise at all. >Another question I want to ask is on how Metakit does on-demand >loading. What portion of the file is in memory at a given point of >time, things like how many rows surrounding a row being read, >entire rows or just the columns being used, page size, etc? MK works with memory mapped files. Data is not "read", it is "paged in" as it gets accessed (and paged out again as the OS juggles with VM). Pages tend to be 4 Kb on most hardware, so that's what gets transferred to/from disk. When you make changes, MK tends to read more of columns in. To keep memory use low, you need to find a balance for the frquency of commits (which flushes buffers and reduces memory use), and performance (which is higher when not committing too often). In read-only mode, memory use is essentially absent. Adding lots of data and only comitting at the end will require lots of (temporary) memory. -jcw From jcw@equi4.com Mon Mar 18 07:04:21 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ID4Ka20042 for ; Mon, 18 Mar 2002 07:04:21 -0600 From: Jean-Claude Wippler To: Metakit list Subject: Re: [Metakit] Commitaside-Readonly problems with Mk4py Date: Mon, 18 Mar 2002 14:03:17 +0100 Message-Id: <20020318130317.10621@triqs.com> In-Reply-To: <20020317211620.A8073@minus1.de> References: <20020317211620.A8073@minus1.de> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Konrad Anton wrote: (Thanks for making this example so simple to read) >#prepare a storage >s=metakit.storage("TMPtest.mk",1) >v=s.getas("v[x:I]") >v.append(x=0) >s.commit() > >#open commit-aside with readonly main storage >s=metakit.storage("TMPtest.mk",0) >sa=metakit.storage("TMPtest.mka",1) >s.aside(sa) >v=s.view("v") >v.append(x=1) >print "appending 1", [r.x for r in v] >sa.commit() ># s.commit() would raise IOError >print "committing", [r.x for r in v] >v.append(x=2) >print "appending 2", [r.x for r in v] >sa.rollback() >v=s.view("v") >print "rollback aside storage", [r.x for r in v] >s.rollback(0) >#s.rollback() or s.rollback(1) give same result! >v=s.view("v") >print "rollback main storage", [r.x for r in v] > >The output is: > >appending 1 [0, 1] >committing [0, 1] Correct, s now has extra changes, stored "aside" in sa. >appending 2 [0, 1, 2] >rollback aside storage [0, 1, 2] Appended, but not committed... hm, you're right: rollback should take you back to the last committed state. Try: reopen both and redo "s.aside(sa)". Should be [0, 1] - it it? >rollback main storage [0] > >Is there a way I can rollback to the state after committing (second >line in output)? Calling rollback() on the aside storage does nothing, >whereas rollback(0) on the main storage does a full rollback, seemingly >discarding the entire aside storage. That last part is correct. >I'm using Python 2.1.1 and Metakit 2.4.3-33 (but 2.4.2 gives same >result). I can try the CVS version, too, but first I'd like to know if >my expectations are right at all. Yes, I think they are. No need to go for CVS just yet, I have not made changes w.r.t. commit aside. Note also that commit-aside is lacking an important optimization (which is in the file format, but not yet coded): commit aside currently stores entire columns - it is intended to store byte-wise differences, making it *far* more space efficient than it is now. I will look into this - it looks like a commit-aside bug. Thanks. -jcw From konrad@minus1.de Mon Mar 18 14:08:00 2002 Received: from moutvdom00.kundenserver.de (moutvdom00.kundenserver.de [195.20.224.149]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2IK80a15993 for ; Mon, 18 Mar 2002 14:08:00 -0600 Received: from [172.19.20.63] (helo=mrvdomng2.kundenserver.de) by moutvdom00.kundenserver.de with esmtp (Exim 2.12 #2) id 16n3QD-00020B-00 for metakit@equi4.com; Mon, 18 Mar 2002 21:07:57 +0100 Received: from [132.230.131.220] (helo=fohnhab.minus1.de) by mrvdomng2.kundenserver.de with esmtp (Exim 3.22 #2) id 16n3QD-00078s-00 for metakit@equi4.com; Mon, 18 Mar 2002 21:07:57 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16n3Dn-0000lU-00 for metakit@equi4.com; Mon, 18 Mar 2002 20:55:07 +0100 Date: Mon, 18 Mar 2002 20:55:07 +0100 From: Konrad Anton To: Metakit list Subject: Re: [Metakit] Commitaside-Readonly problems with Mk4py Message-ID: <20020318205507.A2916@minus1.de> References: <20020317211620.A8073@minus1.de> <20020318130317.10621@triqs.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020318130317.10621@triqs.com>; from jcw@equi4.com on Mon, Mar 18, 2002 at 02:03:17PM +0100 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, Mar 18, 2002 at 02:03:17PM +0100, Jean-Claude Wippler wrote: > (Thanks for making this example so simple to read) How ironical was that? > Appended, but not committed... hm, you're right: rollback should take you > back to the last committed state. > > Try: reopen both and redo "s.aside(sa)". Should be [0, 1] - it it? New version of the script: ################################################################ import os try: os.unlink("TMPtest.mk") except: pass try: os.unlink("TMPtest.mka") except: pass import metakit #prepare a storage s=metakit.storage("TMPtest.mk",1) v=s.getas("v[x:I]") v.append(x=0) s.commit() #open commit-aside with readonly main storage s = metakit.storage("TMPtest.mk",0) sa= metakit.storage("TMPtest.mka",1) s.aside(sa) v=s.view("v") v.append(x=1) print "appending 1", [r.x for r in v] sa.commit() print "committing", [r.x for r in v] v.append(x=2) print "appending 2", [r.x for r in v] #trying JCW's idea s=metakit.storage("TMPtest.mk",0) sa=metakit.storage("TMPtest.mka",1) s.aside(sa) v=s.view("v") print "reopen", [r.x for r in v] ################################################################### With result: ################################################################### appending 1 [0, 1] committing [0, 1] appending 2 [0, 1, 2] reopen [0] ################################################################### Which is still not optimal. BTW: A commit-aside setup with read-write main storage does the same. I-always-thought-a-computer-could-produce-both-zeros-and-ones-ly y'rs Konrad -- Konrad Anton ; Web: www.minus1.de ; gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The blame on their own ears and mine the mushroom Cat and low and I suppose? From konrad@minus1.de Tue Mar 19 06:38:03 2002 Received: from moutvdomng1.kundenserver.de (moutvdomng1.kundenserver.de [212.227.126.181]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2JCc2a08392 for ; Tue, 19 Mar 2002 06:38:02 -0600 Received: from [212.227.126.161] (helo=mrvdomng0.kundenserver.de) by moutvdomng1.kundenserver.de with esmtp (Exim 3.22 #2) id 16nIsJ-00031q-00 for metakit@equi4.com; Tue, 19 Mar 2002 13:37:59 +0100 Received: from [132.230.131.220] (helo=fohnhab.minus1.de) by mrvdomng0.kundenserver.de with esmtp (Exim 3.22 #2) id 16nIsI-0007Ix-00 for metakit@equi4.com; Tue, 19 Mar 2002 13:37:58 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16nIde-0000NN-00 for metakit@equi4.com; Tue, 19 Mar 2002 13:22:50 +0100 Date: Tue, 19 Mar 2002 13:22:50 +0100 From: Konrad Anton To: Metakit list Subject: solved (was: Re: [Metakit] Commitaside-Readonly problems with Mk4py) Message-ID: <20020319132250.A1408@minus1.de> References: <20020317211620.A8073@minus1.de> <20020318130317.10621@triqs.com> <20020318205507.A2916@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020318205507.A2916@minus1.de>; from konrad@minus1.de on Mon, Mar 18, 2002 at 08:55:07PM +0100 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, Mar 18, 2002 at 08:55:07PM +0100, Konrad Anton wrote: > BTW: A commit-aside setup with read-write > main storage does the same. I've found the mistake. I was calling commit(0) on the aside storage for partial commits. If I call commit(0) on the main storage, it all works. The idea that I get IOErrors from committing the (read-only) main storage must have come from an experiment with a full commit. Great! Have a nice remaining day. Konrad -- Konrad Anton ; Web: www.minus1.de ; gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The blame on their own ears and see how Is But the three. From konrad@minus1.de Tue Mar 19 07:09:48 2002 Received: from moutng0.schlund.de (moutng0.kundenserver.de [212.227.126.170]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2JD9la11278 for ; Tue, 19 Mar 2002 07:09:47 -0600 Received: from [195.20.224.204] (helo=mrvdom00.kundenserver.de) by moutng0.schlund.de with esmtp (Exim 3.22 #2) id 16nJN4-0001Tc-00 for metakit@equi4.com; Tue, 19 Mar 2002 14:09:46 +0100 Received: from [132.230.131.220] (helo=fohnhab.minus1.de) by mrvdom00.kundenserver.de with esmtp (Exim 2.12 #2) id 16nJLL-0001pm-00 for metakit@equi4.com; Tue, 19 Mar 2002 14:07:59 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16nJDi-0000OK-00 for metakit@equi4.com; Tue, 19 Mar 2002 14:00:06 +0100 Date: Tue, 19 Mar 2002 14:00:06 +0100 From: Konrad Anton To: Metakit list Subject: Re: solved (was: Re: [Metakit] Commitaside-Readonly problems with Mk4py) Message-ID: <20020319140006.A1487@minus1.de> References: <20020317211620.A8073@minus1.de> <20020318130317.10621@triqs.com> <20020318205507.A2916@minus1.de> <20020319132250.A1408@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020319132250.A1408@minus1.de>; from konrad@minus1.de on Tue, Mar 19, 2002 at 01:22:50PM +0100 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Tue, Mar 19, 2002 at 01:22:50PM +0100, Konrad Anton wrote: > On Mon, Mar 18, 2002 at 08:55:07PM +0100, Konrad Anton wrote: > > BTW: A commit-aside setup with read-write > > main storage does the same. > > I've found the mistake. I was calling commit(0) on the aside storage > for partial commits. If I call commit(0) on the main storage, it all > works. The idea that I get IOErrors from committing the (read-only) > main storage must have come from an experiment with a full commit. No, it came from a fragment like sto=storage("somefile",0) # where somefile already contains views asideSto=storage("asidefile",1) sto.rollback(1) #(missing:) sto.aside(asideSto) va = sto.view("viewa") va.append(x=20) sto.commit(0) # raises IOError The IOError is raised iff I do not reassign the aside storage. That's something I can live with, but maybe it could be announced in the documentation? like (in python.html)... ,------------------ storage.rollback(full=0) Revert data and structure as was last committed to disk. In commit-aside mode, a "full" rollback reverts to the state of the original file and forgets about the aside file. (new:) After a rollback, your view objects are invalid (use the view or getas methods on your storage object to get them back). Furthermore, after a full rollback, the aside storage is detached from the main storage. Use the aside method on your main storage object to reattach it. If you do not reattach it, further commits will (try to) write to the main storage. `------------------ it's-getting-clearer-every-time-ly y'rs Konrad -- Konrad Anton ; Web: www.minus1.de ; gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The blame on their own ears and see how Is But the three. From gerard@xsystemsinc.com Tue Mar 19 09:10:40 2002 Received: from dom01.xsystemsinc.com (hidden-user@dom01.xsystemsinc.com [12.5.16.43]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2JFAea22271 for ; Tue, 19 Mar 2002 09:10:40 -0600 Received: from GERARD ([64.204.192.3]) by dom01.xsystemsinc.com (Lotus Domino Release 5.0.8) with SMTP id 2002031910103883:1929 ; Tue, 19 Mar 2002 10:10:38 -0500 Message-ID: <009f01c1cf58$43b14fd0$0c00a8c0@GERARD> From: "Gerard Duquette" To: Date: Tue, 19 Mar 2002 10:10:53 -0500 MIME-Version: 1.0 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-MIMETrack: Itemize by SMTP Server on dom01/xsystems(Release 5.0.8 |June 18, 2001) at 03/19/2002 10:10:39 AM, Serialize by Router on dom01/xsystems(Release 5.0.8 |June 18, 2001) at 03/19/2002 10:10:40 AM, Serialize complete at 03/19/2002 10:10:40 AM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Compiling Metakit for Unix Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been trying to compile metakit for a Unix box for some time now with no success. Originaly I wanted to compile it for AIX but that proved to be a big problem so now I am trying for HPUX10.2. Basically all I need is the shared object i.e. mk4tcl.so. I will be using metakit with tcl. I can not use the compiled HPUX object from the website because it was compiled for HPUX11 which is a 64bit OS and HPUX10.2 is a 32bit one. My question is simply has anybody tried to compile metakit, just the shared object, for either of these OS and would you be willing to share your experience. I dont need the shared object, just some pointers on how to get through this task without too much pain. I simply need to know what to look for, how to go about it, possibly a quick and dirty way to do it, anything would help. Thanks Gerard Duquette X.Systems, Inc (703) 330-1645 ext111 From Murat_Berk@bmc.com Tue Mar 19 09:25:23 2002 Received: from babbler.bmc.com (camaro.bmc.com [198.207.223.231]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2JFPNa23188 for ; Tue, 19 Mar 2002 09:25:23 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by babbler.bmc.com (8.10.2/8.10.2) with ESMTP id g2JFSfV07025; Tue, 19 Mar 2002 09:28:41 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Tue, 19 Mar 2002 09:25:15 -0600 Message-ID: From: "Berk, Murat" To: "'Gerard Duquette'" , metakit@equi4.com Subject: RE: [Metakit] Compiling Metakit for Unix Date: Tue, 19 Mar 2002 09:25:12 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We do compile it regularly for those and some other platforms both 32 and 64 bit mode. We do not use the configure script since it is integrated to our make system. On AIX, it will not compile out of box. I send fixes for AIX XLC 3.64 and above to Jean-Claude for 2.01 long time ago and same fixes applies to 2.4.3 Add these before c4_Cursor #if defined(os_aix) && defined(compiler_ibmcxx) && (compiler_ibmcxx > 500) bool operator== (const c4_RowRef& a_, const c4_RowRef& b_); bool operator!= (const c4_RowRef& a_, const c4_RowRef& b_); bool operator<= (const c4_RowRef& a_, const c4_RowRef& b_); bool operator>= (const c4_RowRef& a_, const c4_RowRef& b_); bool operator> (const c4_RowRef& a_, const c4_RowRef& b_); bool operator< (const c4_RowRef& a_, const c4_RowRef& b_); #endif Unless these are defined before the class definition, it does not compile. Other than that, it should compile pretty much on all platforms. And we do compile as shared object using our own make system. Murat -----Original Message----- From: Gerard Duquette [mailto:gerard@xsystemsinc.com] Sent: Tuesday, March 19, 2002 9:11 AM To: metakit@equi4.com Subject: [Metakit] Compiling Metakit for Unix I have been trying to compile metakit for a Unix box for some time now with no success. Originaly I wanted to compile it for AIX but that proved to be a big problem so now I am trying for HPUX10.2. Basically all I need is the shared object i.e. mk4tcl.so. I will be using metakit with tcl. I can not use the compiled HPUX object from the website because it was compiled for HPUX11 which is a 64bit OS and HPUX10.2 is a 32bit one. My question is simply has anybody tried to compile metakit, just the shared object, for either of these OS and would you be willing to share your experience. I dont need the shared object, just some pointers on how to get through this task without too much pain. I simply need to know what to look for, how to go about it, possibly a quick and dirty way to do it, anything would help. Thanks Gerard Duquette X.Systems, Inc (703) 330-1645 ext111 _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From petitsoldat@rediffmail.com Tue Mar 19 09:40:17 2002 Received: from mailweb9.rediffmail.com ([203.199.83.21]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g2JFeGa25451 for ; Tue, 19 Mar 2002 09:40:17 -0600 Received: (qmail 606 invoked by uid 510); 19 Mar 2002 15:39:26 -0000 Date: 19 Mar 2002 15:39:26 -0000 Message-ID: <20020319153926.605.qmail@mailweb9.rediffmail.com> Received: from unknown (203.196.128.8) by rediffmail.com via HTTP; 19 Mar 2002 15:39:26 -0000 MIME-Version: 1.0 From: "petitsoldat" Reply-To: "petitsoldat" To: metakit@equi4.com Subject: Re: Re: [Metakit] Some help with using Metakit for large file Content-type: text/plain; format=flowed Content-Disposition: inline Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, 18 Mar 2002 Jean-Claude Wippler wrote : >petitsoldat wrote: > > >What is the average/expected total size of your data? > 1,000,000 x 10,000 x 100 Kb would be terabytes... 100KB is the maximum size of an object (the average size is perhaps much smaller but is hard to predict). May be I should have said that there will be 2^7 keys at max with each key having O(10^4) objects on an average. I think this will keep the number of records in the table to around 1 million and total data size will be: 1,000,000 x 100 KB = 100 GB in the worst-case. Average case should bring this down by at least one order of magnitude. > >In general, you are better off storing things in a >single file (but 2 Gb is a hard limit, if you don't hit >other roadblocks first). > Thanks. Is 10,000 still the hard limit for views in the latest version of Metakit? > >I looked at the last few months archives and came > >across the term "blocked column" that if used can > >make commit() faster. What is a blocked column and > >how does it make commit faster? > >It stores data as a view of subviews, but makes them >appear as a single view. Commits are column-wise, so >this segmented/blocked format reduces the size of >columns. The story is rather more complex nowadays, >though - large strings/bindata is stored as "memos", >i.e. not columnwise at all. > It is interesting since this is just what I am trying to do by storing a table in several files. Let me know if there is any further explanation on blocked columns, somewhere on the wiki may be. And thanks for the quick reply. How are "memos" stored then? I apologise if it is mere confusion on my part, but isn't it true that memo fields are the same as c4_BytesProp in the newer releases. From jcw@equi4.com Fri Mar 22 17:08:16 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2MN8Fa22254 for ; Fri, 22 Mar 2002 17:08:15 -0600 From: Jean-Claude Wippler To: metakit Date: Sat, 23 Mar 2002 00:08:13 +0100 Message-Id: <20020322230813.19205@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] TclKit news Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is to let you know that there are new TclKit builds on the website. Many bug fixes, many new platforms, much streamlining, it's all at: http://www.equi4.com/tclkit/ Also of interest to Tcl'ers on this list, is a very young but growing collection of scripted documents, both applications and packages - an initiative by Steve Landers: http://mini.net/sdarchive/ Oh, wait, perhaps I should add what this is all about... "TclKit: is a single-file runtime for projects built with Tcl/Tk, IncrTcl, and MetaKit. It's all of these, but instantly deployable. See http://wiki.tcl.tk/TclKit "Scripted Documents" are a packaging scheme which can be used in combination with TclKit to create single-file multi-platform apps. See http://wiki.tcl.tk/Scripted It can all be summarized with the following three words: deployment, deployment, deployment... -jcw From steve@improvision.com Tue Mar 26 10:58:03 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2QGw2a30201 for ; Tue, 26 Mar 2002 10:58:02 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id QAA13119 for ; Tue, 26 Mar 2002 16:55:11 GMT Received: from [192.168.1.76] ([192.168.1.76]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971 for ; Tue, 26 Mar 2002 16:58:00 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Tue, 26 Mar 2002 16:57:59 +0000 From: Stephen Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Corrupted byte properties Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, We seem to be getting problems with corrupted byte properties being written to the database again. I can't find a pattern yet, but the corrupted items all seem to be small (<50 bytes). I'm using the latest sources out of CVS - has anyone seen any similar problems? Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ From kord@grub.org Wed Mar 27 10:12:15 2002 Received: from ant.grub.org (ant.grub.org [206.30.142.107]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2RGCFa24554 for ; Wed, 27 Mar 2002 10:12:15 -0600 Received: from localhost (kord@localhost) by ant.grub.org (8.11.2/8.11.2) with ESMTP id g2RGCC927174; Wed, 27 Mar 2002 10:12:12 -0600 X-Authentication-Warning: ant.grub.org: kord owned process doing -bs Date: Wed, 27 Mar 2002 10:12:12 -0600 (CST) From: Kord Campbell To: cc: In-Reply-To: <20020327072537.GC3819@asgard.pte.at> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Re: [Grub-client] successful compile and bugs (fwd) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Alfie, I'm not sure about compatability between Metakit 2.01 and 2.4.3. We never had any other version besides 2.4.3 installed, so we are (obviously) in the dark about the ramification of making people upgrade their Metakit library to run the client and whether or not this will break existing dependencys. It is my experience that it is more likely that it will break them than not! ;) I would imagine that a work around could be done by staticly linking the client with the metakit libary. That way, you could install the newer metakit in another directory, compile with it, and then still have the old shared linked library in use by flightgear. Question is, do Debian packages use source code, binaries, or both? BTW, I emailed Chris - I think you had the correct address in the cc! Kord On Wed, 27 Mar 2002 grub-client-admin@lists.sourceforge.net wrote: > * [2002-03-26 11:28]: > > ---------- Forwarded message ---------- > > Date: Mon, 25 Mar 2002 12:43:34 -0500 > > From: Christopher Thomas Nicodemus > > To: support@grub.org > > Subject: successful compile and bugs > > > > I successfully compiled grubclient 0.30 on Debian 3.0 > > uname -a > > Linux NN2kLinux 2.4.18 #13 Mon Mar 25 08:52:36 EST 2002 i686 unknown > > Kord, could you hand me Christopher's mail address? I can't compile > grub on Debian without an updated metakit package (silly metakit doesn't > install itself with a distinctive api-version so you can install > different versions of metakit side-by-side) so I'd like to know if he > used an updated metakit or what else trick he used to get it working? > > At least I have a pre-version at > for Debian that was working > already before this mail came in ,) It currently is just the question > if flightgear works with the newly build metakit library. If that > problem is solved (or if I found a way to install metakit-2.4.3 > side-by-side with metakit-2.01) I'll upload the package immediately. > > If you have any ideas about metakit that doesn't break flightgear it > would be nice to share them :) > > Kord, feel free to forward this message to Christopher, thanks. > Alfie > -- -------------------------------------------------------------- Kord Campbell Grub.Org Inc. President 6051 N. Brookline #118 Oklahoma City, OK 73112 kord@grub.org Voice: (405) 843-6336 http://www.grub.org Fax: (405) 848-5477 -------------------------------------------------------------- From jcw@equi4.com Wed Mar 27 10:39:42 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2RGdfa25958; Wed, 27 Mar 2002 10:39:42 -0600 From: Jean-Claude Wippler To: Cc: Subject: Re: [Metakit] Re: [Grub-client] successful compile and bugs (fwd) Date: Wed, 27 Mar 2002 17:38:29 +0100 Message-Id: <20020327163829.14101@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Kord Campbell wrote: [...] >I'm not sure about compatability between Metakit 2.01 and 2.4.3. >We never had any other version besides 2.4.3 installed, so we are >(obviously) in the dark about the ramification of making people >upgrade their Metakit library to run the client and whether or >not this will break existing dependencys. It is my experience >that it is more likely that it will break them than not! ;) > >I would imagine that a work around could be done by staticly linking >the client with the metakit libary. That way, you could install the >newer metakit in another directory, compile with it, and then still >have the old shared linked library in use by flightgear. MetaKit's file format changed from 2.0 to 2.3/2.4 - the current code is such that conversion from 2.01 to 2.4.3 should be automatic and effortless. This conversion does imply that older code will no longer recognize the dataformat. One of the design goals of MK, is that it always offers an upward migration path for datafiles. In the case of 2.01 -> 2.4.3, this conversion takes place on open (even with r/o files) and gets applied on commit. That sounds more complex than it really is, the file format has not changed all that much. My advice would be to go for a full update to 2.4.3 if possible (FWIW, I'll bring out a new release soon, because there are some valuable changes in CVS - so you might want to plan this but hold off for just a tad longer). There has been one reported case whereby the conversion caused problems, but I was not able to delve into that further - apart from that, I consider 2.4.3 to be just as stable as 2.01 (and preferable due to new features and higher performance). Please email me if you have concerns about upgrading, or run into any issues with the deployment side of it. -jcw From jcw@equi4.com Wed Mar 27 10:41:39 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2RGfca26134 for ; Wed, 27 Mar 2002 10:41:38 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: Re: [Metakit] Some help with using Metakit for large file Date: Wed, 27 Mar 2002 17:40:26 +0100 Message-Id: <20020327164026.9331@triqs.com> In-Reply-To: <20020319153926.605.qmail@mailweb9.rediffmail.com> References: <20020319153926.605.qmail@mailweb9.rediffmail.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: petitsoldat wrote: [...] > total data size will be: > >1,000,000 x 100 KB = 100 GB > >in the worst-case. Average case should bring this down by at least >one order of magnitude. This is well beyond what MK can deal with. -jcw From jcw@equi4.com Wed Mar 27 11:01:06 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2RH15a27397 for ; Wed, 27 Mar 2002 11:01:05 -0600 From: Jean-Claude Wippler To: Subject: RE: [Metakit] Compiling Metakit for Unix Date: Wed, 27 Mar 2002 18:01:01 +0100 Message-Id: <20020327170101.19707@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Berk, Murat wrote: [Gerard Duquette asks for info about building for AIX & HPUX 10.2] >We do compile it regularly for those and some other platforms both 32 and 64 >bit mode. We do not use the configure script since it is integrated to our >make system. > >On AIX, it will not compile out of box. I send fixes for AIX XLC 3.64 and >above to Jean-Claude for 2.01 long time ago [...] I've just committed your patch to CVS. [...] >Other than that, it should compile pretty much on all platforms. FWIW, I've started testing builds, and am trying to simplify the autoconf + libtool use of MK. One success, is that the ".0.0.0" suffix is gone, MK now generates and installs "normal" .so files. Successful builds within the past week or so include several Win/Mac setups, and Linux x86/alpha/ppc/sparc, FreeBSD, BSD/OS, Solaris, Tru64. The portability to things like AIX and HPUX is limited mostly by my lack of access, and/or working closely with people to get things right. I'm still struggling a bit with libtool in unix/Makefile.in - if anyone has tips on how to improve things: your comments will be very much appreciated. -jcw From dndposey@jps.net Wed Mar 27 18:58:31 2002 Received: from epic.mail.pas.earthlink.net (epic.mail.pas.earthlink.net [207.217.120.181]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2S0wSa20519 for ; Wed, 27 Mar 2002 18:58:29 -0600 Received: from 216-224-155-215.thegrid.net ([216.224.155.215] helo=merlot) by epic.mail.pas.earthlink.net with esmtp (Exim 3.33 #2) id 16qOF9-0000ow-00 for metakit@equi4.com; Wed, 27 Mar 2002 16:58:20 -0800 From: dndposey@jps.net To: metakit@equi4.com Date: Wed, 27 Mar 2002 17:02:32 -0800 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Message-ID: <3CA1FB28.596.2225F93@localhost> Priority: normal X-mailer: Pegasus Mail for Win32 (v3.12c) Subject: [Metakit] Latest TclKit binaries Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The latest Win32 binaries for on my Win98 system seem do to well *until* it exits. Then it crashes with an invalid page fault. I run Win98 on a Sony VAIO laptop w/128meg. Dale Posey ex of IBM (thank God!) From bobhicks@adelphia.net Sat Mar 30 12:18:19 2002 Received: from smtprelay9.dc2.adelphia.net (smtprelay9.dc2.adelphia.net [64.8.50.53]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2UIIIa10840 for ; Sat, 30 Mar 2002 12:18:18 -0600 Received: from homepcxuhigci7 ([24.49.32.46]) by smtprelay9.dc2.adelphia.net (Netscape Messaging Server 4.15) with SMTP id GTSTJT01.H3R for ; Sat, 30 Mar 2002 13:19:05 -0500 Message-ID: <000901c1d817$41dacbb0$6401a8c0@homepcxuhigci7> From: "Bob Hicks" To: Date: Sat, 30 Mar 2002 13:18:13 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Ruby bindings Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Would anyone be in the process of creating Ruby bindings for Metakit? Is anyone interested in such a beast? Bob From frank@canyon-medical.com Sat Mar 30 12:59:31 2002 Received: from mail3.centurytel.net (mail3.centurytel.net [209.142.136.99]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2UIxVa11188 for ; Sat, 30 Mar 2002 12:59:31 -0600 Received: from aSqueakSystem (pppoe0034.sm.centurytel.net [209.142.160.163]) by mail3.centurytel.net (8.12.2/8.12.2) with SMTP id g2UIxUu5016368 for ; Sat, 30 Mar 2002 12:59:31 -0600 (CST) Message-Id: <200203301859.g2UIxUu5016368@mail3.centurytel.net> X-Mailer: Celeste 2.0.4599 Date: Sat, 30 Mar 2002 12:58:02 -0600 Subject: Re: [Metakit] Ruby bindings To: From: Frank Sergeant Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: "Bob Hicks" wrote: > Would anyone be in the process of creating Ruby bindings for Metakit? Is > anyone interested in such a beast? I have an interest in creating Smalltalk (Dolphin or Squeak) bindings, but haven't gotten close to beginning. I seem to be getting further behind rather than catching up. -- Frank From jcw@equi4.com Sat Mar 30 14:46:49 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2UKkma12059 for ; Sat, 30 Mar 2002 14:46:48 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Latest TclKit binaries Date: Sat, 30 Mar 2002 21:46:25 +0100 Message-Id: <20020330204625.489@triqs.com> In-Reply-To: <3CA1FB28.596.2225F93@localhost> References: <3CA1FB28.596.2225F93@localhost> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: dndposey@jps.net wrote: >The latest Win32 binaries for on my Win98 system seem do to well >*until* it exits. >Then it crashes with an invalid page fault. >I run Win98 on a Sony VAIO laptop w/128meg. Yes, I've been hitting crash-on-exit in Windows for some time. The latest builds seem to be very consistent on Win98 - :( - but not unfortunately on NT4, where I have all the tools needed to chase this issue. Am currently working with Tcl'ers to try to get this resolved. There have been MetaKit-related crash-on-exit bugs in the past, but right now it looks more like something that is related to Tk 8.4a5 in some way. If you can research this some more, that would be a tremendous help: does it always crash, even just launch+exit? Is tclkitsh.exe ok? Please feel free to follow up by email - I really need to find out what the issue is and solve it. -jcw From jtaylor@shift-right.com Sat Mar 30 16:55:18 2002 Received: from mactufus.net2atlanta.com (ds2.net2atlanta.com [207.51.14.3]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2UMtIa12951 for ; Sat, 30 Mar 2002 16:55:18 -0600 Received: from lefty (208.147.135.177 [208.147.135.177]) by mactufus.net2atlanta.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id H91CMDG2; Sat, 30 Mar 2002 17:54:57 -0500 From: "John Taylor" To: Date: Sat, 30 Mar 2002 17:54:50 -0500 Message-ID: <000001c1d83d$e8ebd9a0$6401a8c0@lefty> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Multithreaded in C++ Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: FYI, If anyone one is interested. In the course of using Metakit in our products at work, we have created a Multi-threaded C++ wrapper for Metakit. It is a client/server model using ITC (inter-thread-communications) to isolate all database actions for a given a database to a single thread. The wrapper is part of my company's open source C++ library. The library can be downloaded at http://sourceforge.net/projects/openrepo. After downloading, the example project for the Metakit wrapper is: projects//metakit/examples/mt John T. Taylor Shift-Right Technologies, LLC. Tel: 678-344-3115 Fax: 770-736-9276 email: jtaylor@shift-right.com url: www.shift-right.com From jcw@equi4.com Sun Mar 31 14:43:41 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2VKhea26105; Sun, 31 Mar 2002 14:43:40 -0600 From: Jean-Claude Wippler To: announce , metakit Date: Sun, 31 Mar 2002 22:42:26 +0200 Message-Id: <20020331204226.29553@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is to announce a new stable release of MetaKit, the database that fits in the palm of your hand. There are several bug fixes and a few enhancements, see: http://www.equi4.com/metakit/CHANGES Most notable improvements occur with blocked view operations. Storing 100,000,000 ints, with one commit every 5,000,000: bigblock.tcl - Mk4tcl 2.4.4 - Linux filled commit #blocks filesize memuse total 216 s 575.2 ms 5001 19980575 b 23372 K 5000000 rows 362 s 674.9 ms 10001 40094082 b 25568 K 10000000 rows 749 s 543.6 ms 15001 60204629 b 26028 K 15000000 rows 285 s 576.9 ms 20001 80316241 b 27268 K 20000000 rows 289 s 609.1 ms 25001 100428917 b 28500 K 25000000 rows 296 s 649.2 ms 30001 120542657 b 29688 K 30000000 rows 291 s 696.6 ms 35001 140653465 b 31084 K 35000000 rows 281 s 719.9 ms 40001 160765337 b 32388 K 40000000 rows 287 s 757.3 ms 45001 180878273 b 33608 K 45000000 rows 281 s 790.1 ms 50001 200992273 b 34888 K 50000000 rows 292 s 834.0 ms 55001 221099345 b 36128 K 55000000 rows 282 s 1127.7 ms 60001 241211477 b 37284 K 60000000 rows 287 s 894.1 ms 65001 261324673 b 38508 K 65000000 rows 285 s 932.3 ms 70001 281437963 b 39736 K 70000000 rows 284 s 967.2 ms 75001 301557105 b 40960 K 75000000 rows 282 s 1002.8 ms 80001 321678315 b 41908 K 80000000 rows 285 s 1038.5 ms 85001 341801593 b 43132 K 85000000 rows 280 s 1071.2 ms 90001 361922943 b 44348 K 90000000 rows 281 s 1110.3 ms 95001 382046361 b 45580 K 95000000 rows 289 s 1404.0 ms 100001 402167851 b 46852 K 100000000 rows Notes: - memory use grows, but very very slowly, with 20 Mb being the hard baseline, since that represents 5 million 32-bit ints - filling performance remains constant (one unexplained peak), mostly because these are all appends, i.e. creating new blocks - this took over an hour, a lot of overhead is no doubt caused by using Tcl in such a tight loop, and by a known blocked view index access bottleneck (could be optimized for linear scans) - these performance figures are from a PIII/650, Linux SuSE 7.3 Warning: the build process has been changed to no longer create *.so.0.0.0 libraries, nor special Mk4py.so and Mk4tcl.so targets. Instead, please use libmk4py.so and libmk4tcl.so, or rename them to those old names if you prefer. As before, the MetaKit homepage and download area are at: http://www.equi4.com/metakit/ http://www.equi4.com/pub/mk/ Regards, Jean-Claude From joshm@taconic.net Sun Mar 31 18:45:45 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g310jja28067 for ; Sun, 31 Mar 2002 18:45:45 -0600 Received: from gatekeeper (ch-asc4-p5.taconic.net [205.231.28.149]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g310jfK06247 for ; Sun, 31 Mar 2002 19:45:42 -0500 (EST) Message-ID: <011801c1d916$65de7430$3396e7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" Date: Sun, 31 Mar 2002 19:44:35 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Newbie questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I love metakit (so far), but there are some things that I just don't seem able to grok: 1. I have a simple view into a storage object. I can add or replace fields in this view by simply setting them and calling commit on the storage object. But, if I create a new view based on the original view (for example, calling select to extract a revelent subset), and I make changes to this new view, the storage doesn't update, even if I call commit on it. What I've found that works is to use GetIndexOf to find the index into the original view, make the change in the original view, and commit. Why does modifying the original view change the underlying storage, but modifying the derived view does not? 2. I have a nested view as part of my main table. I can access this table by creating a view from the c4_ViewProp field in the main table. But then how do I add or remove rows from this nested view, and have the storage actually update? 3. One final question. If I want to search in the main table for rows where something is found *in the nested table*, is there any way to do this? For an example, suppose my main table is made up of "book" records, and each book contains a set of "chapter" records as a subview. Is there a logical way to search for all the books that have certain types of chapters? Is there useful documentation which describes how all of this works? The tutorial talks a little about nested views, but doesn't go into any real details about replacing data, deleting data, and so on. -- j From jcw@equi4.com Mon Apr 1 09:49:28 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31FnQa04674; Mon, 1 Apr 2002 09:49:26 -0600 From: Jean-Claude Wippler To: announce , metakit Date: Mon, 1 Apr 2002 17:48:08 +0200 Message-Id: <20020401154808.28447@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Tclkit 8.4-34 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: FYI, there is a new release of TclKit on the web, based on Metakit 2.4.4, Tcl/Tk 8.4a5, IncrTcl 3.2, and the latest TclVFS interface, see: http://www.equi4.com/tclkit/ Includes single-file builds for about a dozen platforms, which should work out of the box (i.e. after gunzip, "chmod +x", and a move/rename on Unix). Regards, Jean-Claude From jalupo@attbi.com Mon Apr 1 11:24:26 2002 Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31HOQa08913 for ; Mon, 1 Apr 2002 11:24:26 -0600 Received: from egor.linux.home ([12.253.55.237]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020401172420.VBDB1147.rwcrmhc52.attbi.com@egor.linux.home> for ; Mon, 1 Apr 2002 17:24:20 +0000 Received: from egor.linux.home (localhost [127.0.0.1]) by egor.linux.home (8.12.2/8.12.2/Debian -7) with ESMTP id g31HOFps019420; Mon, 1 Apr 2002 10:24:15 -0700 Received: (from jalupo@localhost) by egor.linux.home (8.12.2/8.12.2/Debian -7) id g31HO4fF019415; Mon, 1 Apr 2002 10:24:04 -0700 Date: Mon, 1 Apr 2002 10:24:04 -0700 From: "James A. Lupo" Message-Id: <200204011724.g31HO4fF019415@egor.linux.home> To: metakit@equi4.com Subject: [Metakit] Error Compiling metakit-2.4.4 on Linux Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I downloaded and attempted to compile the latest Metakit with these steps: cd builds ../unix/configure --with-tcl=/usr/include/tcl8.3 make The error I get is: /bin/sh ./libtool --tag=CXX g++ -c -O2 -I../unix/../include \ -I/usr/include/tcl8.3/generic -I/usr/include/tcl8.3 \ ../unix/../tcl/mk4tcl.cpp g++ -c -O2 -I../unix/../include -I/usr/include/tcl8.3/generic \ -I/usr/include/tcl8.3 ../unix/../tcl/mk4tcl.cpp -fPIC -DPIC \ -o .libs/mk4tcl.o ../unix/../tcl/mk4tcl.cpp: In method \ `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = "option")': ../unix/../tcl/mk4tcl.cpp:1483: argument passing to `char **' from `const char **' discards qualifiers make: *** [mk4tcl.o] Error 1 mk4tcl.cpp:1483 is: _error = Tcl_GetIndexFromObj(interp, obj_, table_, msg_, 0, &index); Tcl_GetIndexFromOBJ is defined in tclDecls.h as: /* 36 */ EXTERN int Tcl_GetIndexFromObj _ANSI_ARGS_((Tcl_Interp * interp, Tcl_Obj * objPtr, char ** tablePtr, char * msg, int flags, int * indexPtr)); I am tracking the Debian Unstable distribution. Kernel: 2.4.19 gcc: 2.95.4 g++: 2.95.4 tcl: 8.3.3-8 -- Jim ------------------- Optional Methods ----------------------- Dr James A. Lupo (303) 423-2652 9667 Independence Dr. jalupo@attbi.com Westminster CO 80021-6845 jalupo@regis.edu From jcw@equi4.com Mon Apr 1 11:33:25 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31HXNa09376; Mon, 1 Apr 2002 11:33:24 -0600 From: Jean-Claude Wippler To: "James A. Lupo" , Subject: Re: [Metakit] Error Compiling metakit-2.4.4 on Linux Date: Mon, 1 Apr 2002 19:31:49 +0200 Message-Id: <20020401173149.32091@triqs.com> In-Reply-To: <200204011724.g31HO4fF019415@egor.linux.home> References: <200204011724.g31HO4fF019415@egor.linux.home> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: James A. Lupo wrote: >I downloaded and attempted to compile the latest Metakit with these >steps: > > cd builds > ../unix/configure --with-tcl=/usr/include/tcl8.3 > make [...] > `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = >"option")': > ../unix/../tcl/mk4tcl.cpp:1483: argument passing to `char **' from > `const char **' discards qualifiers [...] Yes, this is the one instance where I have not been able to work around an incompatibility in the Tcl headers. Tcl 8.4a4 has recently been "constified", which in itself is a good thing. Unfortunately, no matter what cast I use in my code, the compile will break either in older tcl configs (as it does for you), or in the latest one. Solution: either change "table" to "(char**) table" on line 1483 of mk4tcl.cpp, or upgrade your tcl source tree to a recent 8.4-ish one. -jcw From dgporter@erols.com Mon Apr 1 12:58:19 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31IwIa13180; Mon, 1 Apr 2002 12:58:19 -0600 X-Info: This message was accepted for relay by smtp01.mrf.mail.rcn.net as the sender used SMTP authentication X-Trace: UmFuZG9tSVZzdqsEEpQLHVgM2SLuSTo6nKqjN48oy75wAPZU/9pirnvb25h0fu1H3NntaUJvYck= Received: from hamming.cam.nist.gov ([129.6.88.137] helo=erols.com) by smtp01.mrf.mail.rcn.net with asmtp (Exim 3.33 #10) id 16s70T-0005BQ-00; Mon, 01 Apr 2002 13:58:17 -0500 Message-ID: <3CA8ADC5.14C2D816@erols.com> Date: Mon, 01 Apr 2002 13:58:13 -0500 From: Donald G Porter Reply-To: dgporter@erols.com X-Mailer: Mozilla 4.79 [en] (X11; U; SunOS 5.8 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: "James A. Lupo" , metakit@equi4.com Subject: Re: [Metakit] Error Compiling metakit-2.4.4 on Linux References: <200204011724.g31HO4fF019415@egor.linux.home> <20020401173149.32091@triqs.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: James A. Lupo wrote: > > `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = > >"option")': > > ../unix/../tcl/mk4tcl.cpp:1483: argument passing to `char **' from > > `const char **' discards qualifiers Jean-Claude Wippler wrote: > Yes, this is the one instance where I have not been able to work around > an incompatibility in the Tcl headers. Tcl 8.4a4 has recently been > "constified", which in itself is a good thing. > > Unfortunately, no matter what cast I use in my code, the compile will > break either in older tcl configs (as it does for you), or in the latest one. Have you tried using the -DUSE_NON_CONST option in the Makefile? That should keep you 8.3 header-compatible until you're ready to drop 8.3 support. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@nist.gov Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From g.lancaster@mbf.com.au Mon Apr 1 16:15:31 2002 Received: from fs1-smtp.mbf.com.au ([61.88.251.20]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g31MFUa23902 for ; Mon, 1 Apr 2002 16:15:31 -0600 Received: from MBF-INT1-Message_Server by fs1-smtp.mbf.com.au with Novell_GroupWise; Tue, 02 Apr 2002 08:02:13 +1000 Message-Id: X-Mailer: Novell GroupWise 5.5.2 Date: Tue, 02 Apr 2002 08:13:09 +1000 From: "Garth Lancaster" To: , Subject: Re: [Metakit] Multithreaded in C++ Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline X-Guinevere: 1.0.13 ; Medical Benefits Fun Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g31MFUa23902 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hiyah John - this is one thing that has been a long time coming, a C++ MT 'wrapper' for Metakit - good stuff I followed your link to sourceforge, but couldnt see any files released by the project - and at the moment my ISP seems to have serious DNS problems, so I cant get to www.shift-right.com to have a look there... I'll keep trying .. congrats on putting something very useful within reach :-) regards, Garth Lancaster APATHY ERROR: Don't bother striking any key. Application has reported a "Not My Fault" in module KRNL.EXE in line 0200:103F Garth Lancaster MBF InfoTech (Infrastructure) Integration Engineer g.lancaster@mbf.com.au Ph : +61 2 9323 9534 Fax : +61 2 9267 7359 >>> "John Taylor" 03/31/02 08:54am >>> FYI, If anyone one is interested. In the course of using Metakit in our products at work, we have created a Multi-threaded C++ wrapper for Metakit. It is a client/server model using ITC (inter-thread-communications) to isolate all database actions for a given a database to a single thread. The wrapper is part of my company's open source C++ library. The library can be downloaded at http://sourceforge.net/projects/openrepo. After downloading, the example project for the Metakit wrapper is: projects//metakit/examples/mt John T. Taylor Shift-Right Technologies, LLC. Tel: 678-344-3115 Fax: 770-736-9276 email: jtaylor@shift-right.com url: www.shift-right.com _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit This email contains confidential information intended only for the addressee. If you received it in error, please inform us by reply email or by calling us on 131 137. Please also delete this email and destroy any hard copy. You must not disclose or use the information in this email. From jcw@equi4.com Mon Apr 1 16:45:42 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31Mjfa25457 for ; Mon, 1 Apr 2002 16:45:41 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Error Compiling metakit-2.4.4 on Linux Date: Tue, 2 Apr 2002 00:43:07 +0200 Message-Id: <20020401224307.17757@triqs.com> In-Reply-To: <3CA8ADC5.14C2D816@erols.com> References: <3CA8ADC5.14C2D816@erols.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Donald G Porter wrote: >James A. Lupo wrote: >> > `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = >> >"option")': >> > ../unix/../tcl/mk4tcl.cpp:1483: argument passing to `char **' from >> > `const char **' discards qualifiers [...] >Have you tried using the -DUSE_NON_CONST option in the Makefile? That >should keep you 8.3 header-compatible until you're ready to drop 8.3 >support. Ah, that and "CONST84" solved it. I've applied changes to mk4tcl.* so the code should now compile properly with previous Tcl headers - now in CVS. Thanks. -jcw P.S. Also reverted to an earlier libtool to undo the "--tag" trouble that was introduced in unix/Makefile.in - why oh why is is that whenever a release is made, then the *next* minute a useful change needs to be committed to CVS? From jcw@equi4.com Tue Apr 2 10:36:11 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g32Ga9a22986 for ; Tue, 2 Apr 2002 10:36:10 -0600 From: Jean-Claude Wippler To: metakit Date: Tue, 2 Apr 2002 18:34:53 +0200 Message-Id: <20020402163453.11193@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Tweak for 2.4.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There is a bad check in remap.cpp, line 531 - when compiled with assertion checking on. Just comment out the d4_assert call and things will be fine. Another more complete fix has been comitted to CVS. Thanks to several people for pointing this out. -jcw From luislavena@hotmail.com Wed Apr 3 09:25:30 2002 Received: from hotmail.com (oe41.law8.hotmail.com [216.33.240.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g33FPUa01333 for ; Wed, 3 Apr 2002 09:25:30 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 3 Apr 2002 07:25:24 -0800 X-Originating-IP: [200.43.142.2] From: "Luis Lavena" To: Date: Wed, 3 Apr 2002 12:25:15 -0300 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Message-ID: X-OriginalArrivalTime: 03 Apr 2002 15:25:24.0717 (UTC) FILETIME=[C6B4F1D0:01C1DB23] Subject: [Metakit] No 2.4.4 Mk4py for Python 2.1? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I check at equi4.com about the new release of 2.4.4 of Metakit. But after downloaded it (from http://www.equi4.com/pub/mk/metakit-2.4.4-34-win/) only get the Mk4py.dll wich aren't compatible with 2.1 (currently using 2.1.2). is the new only available to 2.2 pythonist? I can't do a build from cvs due lack of VC or any C/C++ compiler on Windows. Anyway, thanks. Regards. Luis From jcw@equi4.com Wed Apr 3 17:33:55 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g33NXsa23534 for ; Wed, 3 Apr 2002 17:33:54 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] No 2.4.4 Mk4py for Python 2.1? Date: Thu, 4 Apr 2002 01:33:14 +0200 Message-Id: <20020403233314.17009@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Luis Lavena wrote: >Hello, I check at equi4.com about the new release of 2.4.4 of Metakit. But >after downloaded it (from http://www.equi4.com/pub/mk/metakit-2.4.4-34-win/) >only get the Mk4py.dll wich aren't compatible with 2.1 (currently using >2.1.2). I'll be happy to put a copy on the server, if someone sends me one - but I don't have Python 2.1 installed anymore myself, unfortunately. You can use the 2.4.3 build, though, it's at: -jcw From joshm@taconic.net Thu Apr 4 13:36:45 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g34Jaia29091 for ; Thu, 4 Apr 2002 13:36:45 -0600 Received: from gatekeeper (ch-asc6-p21.taconic.net [205.231.148.21]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g34JahK12113 for ; Thu, 4 Apr 2002 14:36:43 -0500 (EST) Message-ID: <022b01c1dc0f$d9e26e80$9e1de7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: <011801c1d916$65de7430$3396e7cd@gatekeeper> Subject: Re: [Metakit] Newbie questions Date: Thu, 4 Apr 2002 14:35:16 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I wasn't sure if this made it across the mailing list or not, since I didn't get any replies. :-( Basically, I'm looking for documentation on how to add to/modify/replace/delete information stored in a subview (in C++). This is an extension of a problem I could not figure out, which is as follows: I create a view based on a storage object. In this view, I can easily add and modify fields in a table. This works well. However, when I search this view (create another view via select(), for example), while I can examine this view, if I make changes to it, the underlying storage isn't updated. When I manually follow the fields back to the original view (via GetIndexOf()), I'm able to update the original view and the underlying storage. Question 1: Why doesn't my modification of the view created via select() update the underlying storage? Question 2: If there isn't a way for it to update the underlying storage, how does one modify subviews, since they are only accessible indirectly? If my questions aren't clear, I can try to provide some source to help explain them, but at the moment, I don't have a simple test case. Any any all help will be greatly appreciated. -- j ----- Original Message ----- From: "Joshua Muskovitz" To: "MetaKit Mailing List" Sent: Sunday, March 31, 2002 7:44 PM Subject: [Metakit] Newbie questions > Hi, > > I love metakit (so far), but there are some things that I just don't seem > able to grok: > > 1. I have a simple view into a storage object. I can add or replace fields > in this view by simply setting them and calling commit on the storage > object. But, if I create a new view based on the original view (for > example, calling select to extract a revelent subset), and I make changes to > this new view, the storage doesn't update, even if I call commit on it. > What I've found that works is to use GetIndexOf to find the index into the > original view, make the change in the original view, and commit. Why does > modifying the original view change the underlying storage, but modifying the > derived view does not? > > 2. I have a nested view as part of my main table. I can access this table > by creating a view from the c4_ViewProp field in the main table. But then > how do I add or remove rows from this nested view, and have the storage > actually update? > > 3. One final question. If I want to search in the main table for rows > where something is found *in the nested table*, is there any way to do this? > For an example, suppose my main table is made up of "book" records, and each > book contains a set of "chapter" records as a subview. Is there a logical > way to search for all the books that have certain types of chapters? > > Is there useful documentation which describes how all of this works? The > tutorial talks a little about nested views, but doesn't go into any real > details about replacing data, deleting data, and so on. > > -- j > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From jcw@equi4.com Fri Apr 5 01:56:29 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g357uSa22853; Fri, 5 Apr 2002 01:56:28 -0600 From: Jean-Claude Wippler To: Joshua Muskovitz , metakit Subject: Re: [Metakit] Newbie questions Date: Fri, 5 Apr 2002 09:49:08 +0200 Message-Id: <20020405074908.9726@triqs.com> In-Reply-To: <022b01c1dc0f$d9e26e80$9e1de7cd@gatekeeper> References: <022b01c1dc0f$d9e26e80$9e1de7cd@gatekeeper> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joshua Muskovitz wrote: >Hi, I wasn't sure if this made it across the mailing list or not, since I >didn't get any replies. :-( Whoops... I totally missed this (it did get posted). >Question 1: Why doesn't my modification of the view created via select() >update the underlying storage? It's a limitation of the implementation - derived views are not as modifiable as I had hoped they would become (my only excuse is: it's tricky to get this right when derived views are stacked, i.e. a join of a select of a sort). The GetIndexOf call is the way to do it. Even then, you have to be careful because not all derived view combinations will see the changes (and some things become terribly slow, while propagating each change). A safer approach is to modify the originals, and tear-down/re-build the derived views before/after doing so, well at least in complex cases or when making massive changes. >Question 2: If there isn't a way for it to update the underlying storage, >how does one modify subviews, since they are only accessible indirectly? The logic is reversed: you create a row, then make a view *from* it: c4_Row row = ... int n = myview.Add(row); c4_View subn = pSubProp (myview [n]); ... now make changes to subn, which *is* in storage already ... The reason is that the reverse, creating a view full of data and then assigning it to some row in storage as being a subview, takes twice as much memory (first, it's a standalone view, then it gets copied into storage). It's still possible to say: pSubProp (myview [n]) = a_view_full_of_data; ... but it's causes a lot more overhead. >> Is there useful documentation which describes how all of this works? The >> tutorial talks a little about nested views, but doesn't go into any real >> details about replacing data, deleting data, and so on. Some sample code in examples/ (in the MetaKit source .tar.gz), lots of little snippets of C++ code, all independent in the regression test suite - tests/. Contributions are most welcome. You can edit a page on the MetaKit wiki (or ask me to set it up for you if the process is not clear), or send me text to add - I'm sure we can find a good spot for it, in the distribution, on the web, whatever. I'm feeling the (increasing) pain of not having good docs for each of the currently supported C++, Python, and Tcl languages (actually, Tcl is covered best now, due to Mark Roseman's recent contribution). Make a difference! Become part of the MK story! Share in the fame! See your name in virtual gold on the web! Share in the pride of doing the right thing! (Just having fun... but I *do* mean it, nevertheless!) -jcw From andreas.olsson@ctakt.com Fri Apr 5 09:14:22 2002 Received: from linus.ctakt.com (linus.ctakt.com [62.119.45.234]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g35FELa06544 for ; Fri, 5 Apr 2002 09:14:22 -0600 Received: from flint ([192.168.2.115]) by linus.ctakt.com (8.11.6/8.11.2) with SMTP id g35FEKp03034 for ; Fri, 5 Apr 2002 17:14:20 +0200 From: "Andreas Olsson" To: Date: Fri, 5 Apr 2002 16:14:19 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] loss of data at commit?! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I just downloaded and tried MetaKit 2.4.4-34. It works fine until I commit my changes to a view. When I print the content of my view before commit it contains three columns (key, name, value). After storage.Commit() I only get column key when I print. Any one having a solution? /Andreas From jcw@equi4.com Fri Apr 5 09:25:21 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g35FPKa07233 for ; Fri, 5 Apr 2002 09:25:20 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] loss of data at commit?! Date: Fri, 5 Apr 2002 17:25:13 +0200 Message-Id: <20020405152513.26497@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas, >I just downloaded and tried MetaKit 2.4.4-34. It works fine until I commit >my changes to a view. When I print the content of my view before commit it >contains three columns (key, name, value). After storage.Commit() I only get >column key when I print. Did you use storage.GetAs("...") to define the view with all three columns? If not, the commit will store the columns it knows about, and make the in-memory data match the definition, i.e. drop the other two columns. -jcw From andreas.olsson@ctakt.com Sat Apr 6 09:47:19 2002 Received: from linus.ctakt.com (linus.ctakt.com [62.119.45.234]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g36FlIa26424 for ; Sat, 6 Apr 2002 09:47:19 -0600 Received: from flint ([192.168.2.115]) by linus.ctakt.com (8.11.6/8.11.2) with SMTP id g36FlDp05062 for ; Sat, 6 Apr 2002 17:47:13 +0200 From: "Andreas Olsson" To: Date: Sat, 6 Apr 2002 17:47:12 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Problems with RemoveAt in 2.4.4-34 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! As I wrote in an earlier posting I'm a C++ newbie and also new to MetaKit. Therefore I apology for my simple questions. The problem is that I'm having a tight deadline for this project and I have no time to look up the solutions myself. Question: why doesn't view.RemoveAt(i) work? when I debug my application I end up in... void c4_FilterSeq::RemoveAt(int, int) { d4_assert(0); } where it of course fails. My code looks like this: void CTLocalDb::deleteRow(char *key) { c4_View deleteview; deleteview = view.Select(pKey [key]); int rowc = deleteview.GetSize(); deleteview.RemoveAt(0, rowc); // deleteview.SetSize(0); } Thanks / Andreas From jcw@equi4.com Tue Apr 9 10:37:20 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g39FbIa07345; Tue, 9 Apr 2002 10:37:18 -0500 From: Jean-Claude Wippler To: metakit Cc: will Date: Tue, 9 Apr 2002 17:36:47 +0200 Message-Id: <20020409153647.12221@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] tclkit crash-on-exit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There IS a crash-on-exit bug in the current TclKit windows builds. Reports vary as far as windows releases go - for some people it works on 2k but not 98, for others it works fine on 98. Several reports where launching and immediately stopping will crash on 98. I have one sequence of steps (involving alphatk) I run which crashes on all win platforms I tried, but *NOT* in the debug build. For some people things work fine all the time, even when trying to force the problem. In my most recent debugging rounds, crashes were never in Mk4tcl (though they have been in the past, perhaps there were multiple bugs). There may be an issue with Tk menu cleanup, but I'm not 100% sure, and even if accurate - knowing this does not solve things either, it merely would explain why the problem surfaces in different ways (even handling order, OS differences). Just uploaded which is a debug build made with MSVC6 (and quite large unpacked). So far, debug builds have never crashed on exit - which is one of the main reasons why I cannot resolve the issue: I can't make it fail in debug mode! If you run into crash-on-exit, I'd appreciate it is you could give the above version a try and tell me whether it works for you. If you get a crash and have MSVC6, please please go into debug mode and try to obtain a stack trace. Feel free to distribute the debug build, if it solves your problem for now. It is complete, but it's compiled with optimizations turned off. It's a very extreme measure, but it's all I can come up with... -jcw From smalllinux@netscape.net Tue Apr 9 15:40:42 2002 Received: from imo-r03.mx.aol.com (imo-r03.mx.aol.com [152.163.225.99]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g39Kega24535 for ; Tue, 9 Apr 2002 15:40:42 -0500 Received: from smalllinux@netscape.net by imo-r03.mx.aol.com (mail_out_v32.5.) id d.ed.383e25f (16238) for ; Tue, 9 Apr 2002 16:40:34 -0400 (EDT) Received: from netscape.com (mow-m11.webmail.aol.com [64.12.184.139]) by air-in03.mx.aol.com (v84.14) with ESMTP id MAILININ32-0409164034; Tue, 09 Apr 2002 16:40:34 -0500 Date: Tue, 09 Apr 2002 16:40:34 -0400 From: smalllinux@netscape.net To: metakit@equi4.com Message-ID: <5D30BF3B.5BBBD809.1080C5D1@netscape.net> X-Mailer: Atlas Mailer 2.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] tclkit crash on exit - win32 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ok, now I'm confused. Linudent crashes on exit on win98 using tclkit Jan 2002 build. On that same system, most tclkit applications exit fine, ml editor, wikit, etc. Linudent sources many different tcl files, uses kitten.bin, itcl and itk and iwidgets. Click on the window icon or exiting through the menu, both call a exit proc opening an iwidget. The last line of the dialog box would call the tcl exit builtin command. Using tclkit_d.exe does NOT crash on exit using the shutdown button choice. But it does crash on exit clicking the window icon. So I created a test script to show it. I don't get it. normal tclkit exit through shutdown = crash exit through windows = crash debug tclkit exit through shutdown = NO crash exit through windows = crash Steven G. http://linudent.sourceforge.net/ #!/usr/local/bin/tclkit package require scripdoc mk4vfs::mount kitten.bin ../libs/kitten.bin -readonly scripdoc::extendPath kitten.bin # main.tcl - application startup script package require Tk package require Itcl package require Itk package require Iwidgets # main.tcl - application startup script set MainGraphic drtux.gif set office_name testoffice set this_version "3.10" set version_date "April 9 2002" set beta_version "normal" set msglocale off proc dialog_position {dlg} { set width [winfo screenwidth .] set height [winfo screenwidth .] set x [expr {($width/2) - 260}] set y [expr {($height/2) - 300}] wm geometry $dlg +$x+$y } # copyright 2001 LinuDent # GPL version 2 #wm protocol . WM_DELETE_WINDOW shutdown #source exit_dialog.itk ;# provides proc shutdown # exit_application # we ask close before closing this is the parent of all modules proc exit_main_book {} { shutdown } wm title . Start_Book iwidgets::tabnotebook .notebook -tabpos n -equaltabs 0 -height 4i -width 6i\ -padx 3 -angle 30 -background cyan -backdrop grey65 set linudent [.notebook add -label [::msgcat::mc "linudent"] -command\ {destroy .notebook.canvas.notebook.button}] frame $linudent.frame -relief ridge -bd 3 -bg peachpuff label $linudent.frame.label -text [::msgcat::mc "$office_name"] pack $linudent.frame.label -side top -pady 2 pack $linudent.frame -fill both -expand 1 set status [.notebook add -label [::msgcat::mc "status"] -command\ {destroy .notebook.canvas.notebook.button}] label $status.label -text [::msgcat::mc "[::msgcat::mc "Status Screen\ - LinuDent"]"] pack $status.label -side top -pady 2 button $status.button -text [::msgcat::mc "[::msgcat::mc "click to reload Linudent config"]"] -command { load_config show_status puts "load config" } pack $status.button -side top -pady 2 proc show_status {} { destroy $::status.label1 label $::status.label1 -text [::msgcat::mc "[::msgcat::mc "You are running\ LinuDent version"] - $::this_version"] pack $::status.label1 destroy $::status.label2 label $::status.label2 -text [::msgcat::mc "[::msgcat::mc "This is the current\ version as of"] $::version_date "] pack $::status.label2 destroy $::status.label3 label $::status.label3 -text [::msgcat::mc "[::msgcat::mc "You are configured to\ use the"] $::beta_version version "] pack $::status.label3 destroy $::status.labelmsgcat label $::status.labelmsgcat -text [::msgcat::mc "[::msgcat::mc "Your locale is\ set to: "] [::msgcat::mclocale] "] pack $::status.labelmsgcat #frame $accounting.frame -bg green -container 1 -width 1400 -height 1200 #pack $accounting.frame -pady 37 } ;# end of proc show status show_status set modules [.notebook add -label [::msgcat::mc "modules"] -command\ {destroy .notebook.canvas.notebook.button}] label $modules.label -text [::msgcat::mc "modules Screen - LinuDent"] pack $modules.label -side top -pady 2 frame $modules.frame -height 500 -width 700 pack $modules.frame frame $modules.frame.buttonframe -height 500 -width 250 pack $modules.frame.buttonframe -side left -expand true -fill y frame $modules.frame.buttonframe2 -bg ivory -height 500 -width 700 pack $modules.frame.buttonframe2 -expand true -fill y set administration [.notebook add -label [::msgcat::mc "shutdown"]\ -command {destroy .notebook.canvas.notebook.button}] label $administration.label -text [::msgcat::mc "Shutdown Screen - LinuDent"] pack $administration.label -side top -pady 2 frame $administration.frame -height 500 -width 700 pack $administration.frame frame $administration.frame.buttonframe -height 100 -width 150 pack $administration.frame.buttonframe -side left -expand true -fill y frame $administration.frame.buttonframe2 -bg ivory -height 100 -width 700 pack $administration.frame.buttonframe2 -expand true -fill y button $administration.frame.buttonframe2.shutdown -text [::msgcat::mc\ "F10 Shut Down"] -bg red -fg yellow -command {shutdown} button $administration.frame.buttonframe.datashutdown -text [::msgcat::mc\ "shutdown database server"] -bg red -fg yellow -command { send databaseserver.tcl "exit" } button $administration.frame.buttonframe.demoshutdown -text [::msgcat::mc\ "close demo book"] -bg red -fg yellow -command { puts "hide patient book" } button $administration.frame.buttonframe2.schedshutdown -text [::msgcat::mc\ "close schedule book"] -bg red -fg yellow -command { puts "destroy scheduler" } button $administration.frame.buttonframe.reportshutdown -text [::msgcat::mc\ "close report book"] -bg red -fg yellow -command { puts "destroy reportbook" } button $administration.frame.buttonframe2.accountshutdown -text [::msgcat::mc\ "close account book"] -bg red -fg yellow -command { puts "destroy accountingbook" } pack $administration.frame.buttonframe.demoshutdown -expand true -fill x pack $administration.frame.buttonframe2.schedshutdown -expand true -fill x pack $administration.frame.buttonframe.datashutdown -expand true -fill x pack $administration.frame.buttonframe.reportshutdown -expand true -fill x pack $administration.frame.buttonframe2.accountshutdown -expand true -fill x pack $administration.frame.buttonframe2.shutdown -expand true -fill x frame .exit_frame -bd 0 button .exit_frame.exit -text [::msgcat::mc "Exit"] -command exit pack .exit_frame.exit pack .notebook -expand true -fill both .notebook select 1 update destroy .top update idletasks focus -force .notebook # ---------------------------------------------------------------------- # exit_dialog in [incr Widgets] # ---------------------------------------------------------------------- #\ # # Build a generic dialog # proc shutdown {} { if [winfo exist .exit_d] { .exit_d activate } else { iwidgets::dialog .exit_d .exit_d buttonconfigure OK -command { puts "pushed: OK" .exit_d deactivate 1 exit } .exit_d hide Apply .exit_d buttonconfigure Cancel -command { puts "pushed: Cancel" wm withdraw .exit_d .exit_d deactivate 0 } .exit_d hide Help # # Add something to the top of the dialog... # set win [.exit_d childsite] label $win.ex -text "Shuting down LinuDent\n(are you sure)" \ -background red \ -width 25 -height 5 pack $win.ex -expand yes -fill both -padx 4 -pady 4 .exit_d activate } focus -force .exit_d } __________________________________________________________________ Your favorite stores, helpful shopping tools and great gift ideas. Experience the convenience of buying online with Shop@Netscape! http://shopnow.netscape.com/ Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From guenther.fischer@hrz.tu-chemnitz.de Thu Apr 11 11:19:14 2002 Received: from meg.hrz.tu-chemnitz.de (mail@meg.hrz.tu-chemnitz.de [134.109.132.57]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3BGJDa08314 for ; Thu, 11 Apr 2002 11:19:13 -0500 Received: from pat.hrz.tu-chemnitz.de ([134.109.132.143] ident=mail) by meg.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #3) id 16vhI0-0005kg-00 for metakit@equi4.com; Thu, 11 Apr 2002 18:19:12 +0200 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=root) by pat.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #2) id 16vhI0-00014f-00 for metakit@equi4.com; Thu, 11 Apr 2002 18:19:12 +0200 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.9.3/8.9.3) with ESMTP id SAA16247 for ; Thu, 11 Apr 2002 18:19:11 +0200 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Thu, 11 Apr 2002 18:19:11 +0200 (MEST) From: Guenther Fischer To: metakit@equi4.com In-Reply-To: <20020409153647.12221@triqs.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] tclkit error on new redhad skipjack Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I do a redhat system update and tclkit doesn't work anymore ... tclkit-x86-linux.bin: dynamic-link.h:62: elf_get_dynamic_info: Assertion `! "bad dynamic tag"' failed. I got this error with all the last versions any idea? -- ~Guenther Fischer From jcw@equi4.com Thu Apr 11 11:31:02 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3BGV1a09243 for ; Thu, 11 Apr 2002 11:31:01 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] tclkit error on new redhad skipjack Date: Thu, 11 Apr 2002 18:29:06 +0200 Message-Id: <20020411162906.3668@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther, >I do a redhat system update and tclkit doesn't work anymore ... > >tclkit-x86-linux.bin: dynamic-link.h:62: elf_get_dynamic_info: Assertion >`! "bad dynamic tag"' failed. > >I got this error with all the last versions What is last versions, one of the following? What does the command "ldd tclkit-x86-linux.bin" return? And what does "file tclkit-x86-linux.bin" say? -jcw From jcw@equi4.com Fri Apr 12 08:29:59 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3CDTta25148; Fri, 12 Apr 2002 08:29:55 -0500 From: Jean-Claude Wippler To: Cc: Jeff Hobbs , Vince Darley , will Subject: Re: [Metakit] tclkit crash on exit - win32 Date: Fri, 12 Apr 2002 15:29:40 +0200 Message-Id: <20020412132940.31407@triqs.com> In-Reply-To: <5D30BF3B.5BBBD809.1080C5D1@netscape.net> References: <5D30BF3B.5BBBD809.1080C5D1@netscape.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The saga continues... I can now force a crash and get a stack trace with function names in it. You can help solve this issue, by downloading the new build of tclkit for Windows, which is a release build, but with symbols fully enabled: If you can make it crash *and* have MSVC6 installed, then you should be able to obtain a stack trace, by pressing "cancel" when the crash dialog appears. The stack trace is in the "call stack" windows, and can be copied and pasted to another application. It would be a great help if you could send those stack traces. I've got one on NT4, but before drawing conclusions it would be really helpful to see the issues on Windows 98 and 2k. The more there is to compare, the better. Please be sure to include info on windows version, and what you did to make the crash happen - in some cases a plain start + exit has been all it takes to trigger this problem. But even if you cannot reproduce or describe the exact steps taken before the crash - a stack trace would be most welcome. -jcw From jcw@equi4.com Sat Apr 13 06:13:04 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3DBD0a09390; Sat, 13 Apr 2002 06:13:00 -0500 From: Jean-Claude Wippler To: Cc: will , Jeff Hobbs Subject: Re: [Metakit] tclkit crash on exit - win32 Date: Sat, 13 Apr 2002 13:12:22 +0200 Message-Id: <20020413111222.22858@triqs.com> In-Reply-To: <20020412132940.31407@triqs.com> References: <20020412132940.31407@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There are new win32* builds for tclkit at: http://www.equi4.com/pub/tk/newer/ This fixes (to be accurate: works around) a, what I'd call "fully identified", problem in Tk menu cleanup. Please use these latest builds. Many thanks to Jeff Hobbs for helping figure out this tricky bug. Remaining problems (oh, there'll always be some) are not necessarily the same issue, so please do continue to let me know if bad things happen. -jcw From joshm@taconic.net Sun Apr 14 23:40:29 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3F4eTa07339 for ; Sun, 14 Apr 2002 23:40:29 -0500 Received: from gatekeeper (ch-asc5-p40.taconic.net [205.231.28.232]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g3F4eRB14068 for ; Mon, 15 Apr 2002 00:40:27 -0400 (EDT) Message-ID: <0e6f01c1e437$576e1190$9e1de7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" Date: Mon, 15 Apr 2002 00:38:06 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] How do I search for a substring? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Another novice question: I've got a column of names (say last names), and I want to do a lookup for some sort of typeahead completion in a UI. So I want to find all the names where the string in the column starts with some string. More specifically, the shortest one that matches (case INsensitive). Please tell me there is a better way then sorting the view and doing a binary search for the item. -- j Out The Door In 2K4! From nicolasb@maich.gr Mon Apr 15 08:09:03 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3FD92a21105 for ; Mon, 15 Apr 2002 08:09:02 -0500 Received: FROM maich.gr BY mail-server ; Mon Apr 15 16:30:22 2002 +0300 Message-ID: <3CBAD11B.5060406@maich.gr> Date: Mon, 15 Apr 2002 16:09:47 +0300 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com References: <20020412132940.31407@triqs.com> <20020413111222.22858@triqs.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Serializing Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I serialize a metakit file as below, (in tclkit) mk::file open db data.dat set fd [open out.dat w] mk::file save db $fd close $fd The file shrinks from ~5Mb to 1.5Mb When I re-open the new out.dat,I notice that some "large string records" are truncated... Is this how to correctly serialize the file..??? Sincerely, Nicolas Boretos From antonio@mac.ps100.net Mon Apr 15 09:10:20 2002 Received: from mac.ps100.net (h00045a2a5bb8.ne.client2.attbi.com [65.96.154.86]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FEAHa23442 for ; Mon, 15 Apr 2002 09:10:20 -0500 Received: from tintin ([127.0.0.1] helo=mac.ps100.net) by mac.ps100.net with smtp (Exim 3.12 #1 (Debian)) id 16x76h-0006lu-00 for ; Mon, 15 Apr 2002 10:05:25 -0400 Message-ID: <2814131.1018879481642.JavaMail.upgradewebapp@mac.ps100.net> From: Antonio Rodriguez To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit x-memora-stamp: 1018879480124 Date: Mon, 15 Apr 2002 10:05:25 -0400 Subject: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all, Trying to build Metakit on a portable Mac (I'm temporarily away from my linux machines) and I am running into a problem during the compile phase. I'm not used to building anything on OSX, so I'm not quite sure of how to go about fixing this: cd metakit/builds ../unix/configure --enable-python << lots of autoconf stuff here >> make << lots of stuff, then error: >> c++ -shared .libs/column.o .libs/custom.o .libs/derived.o .libs/fileio.o .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o .libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o .libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4.so /usr/bin/ld: Undefined symbols: _main ___builtin_delete ___builtin_vec_delete ___builtin_vec_new ___throw _terminate__Fv ___builtin_new ___pure_virtual ___rtti_si ___rtti_user make: *** [libmk4.la] Error 1 Thanks Antonio PS Please cc me on replies as I am not a regular list member yet. From jcw@equi4.com Mon Apr 15 09:46:00 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FEjxa24928 for ; Mon, 15 Apr 2002 09:45:59 -0500 From: Jean-Claude Wippler To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? Date: Mon, 15 Apr 2002 16:38:11 +0200 Message-Id: <20020415143811.9400@triqs.com> In-Reply-To: <0e6f01c1e437$576e1190$9e1de7cd@gatekeeper> References: <0e6f01c1e437$576e1190$9e1de7cd@gatekeeper> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joshua Muskovitz wrote: >I've got a column of names (say last names), and I want to do a lookup for >some sort of typeahead completion in a UI. So I want to find all the names >where the string in the column starts with some string. More specifically, >the shortest one that matches (case INsensitive). The "shortest" confuses me, if I search for "abx" in the following: aa abxdd abxe abxff ac Which one(s) do you want? >Please tell me there is a better way then sorting the view and doing a >binary search for the item. Can you elaborate? What search mechanism would you consider optimal? -jcw From jcw@equi4.com Mon Apr 15 09:46:04 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FEk3a24946; Mon, 15 Apr 2002 09:46:03 -0500 From: Jean-Claude Wippler To: Cc: Antonio Rodriguez Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 Date: Mon, 15 Apr 2002 16:44:48 +0200 Message-Id: <20020415144448.20573@triqs.com> In-Reply-To: <2814131.1018879481642.JavaMail.upgradewebapp@mac.ps100.net> References: <2814131.1018879481642.JavaMail.upgradewebapp@mac.ps100.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Antonio Rodriguez wrote: >Trying to build Metakit on a portable Mac (I'm temporarily away from my >linux machines) and I am running into a problem during the compile phase. > >I'm not used to building anything on OSX, so I'm not quite sure of how to >go about fixing this: > >cd metakit/builds >../unix/configure --enable-python ><< lots of autoconf stuff here >> >make ><< lots of stuff, then error: >> > >c++ -shared .libs/column.o .libs/custom.o .libs/derived.o .libs/ >fileio.o .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o >.libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4.so >/usr/bin/ld: Undefined symbols: >_main >___builtin_delete >___builtin_vec_delete >___builtin_vec_new >___throw >_terminate__Fv >___builtin_new >___pure_virtual >___rtti_si >___rtti_user >make: *** [libmk4.la] Error 1 Ah yes, I've been struggling to understand this myself. What I had to do was edit builds/libtool and replace the "-shared" with "-dynamiclib" to make it work - it seems that otherwise c++ is trying to build a self- contained "main" executable, not a shared lib for use by others. That is of course a kludge. If there is a Mac OSX expert on this list who can help with this, please do follow up if possible. Libtool remains as clear as mud to me, so I'm essentially just feeling my way around in considerable darkness... >PS Please cc me on replies as I am not a regular list member yet. Done! -jcw From jcw@equi4.com Mon Apr 15 09:46:02 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FEk1a24933 for ; Mon, 15 Apr 2002 09:46:01 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Serializing Date: Mon, 15 Apr 2002 16:39:55 +0200 Message-Id: <20020415143955.4301@triqs.com> In-Reply-To: <3CBAD11B.5060406@maich.gr> References: <3CBAD11B.5060406@maich.gr> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas Boretos wrote: >I serialize a metakit file as below, (in tclkit) > >mk::file open db data.dat >set fd [open out.dat w] >mk::file save db $fd >close $fd > >The file shrinks from ~5Mb to 1.5Mb > >When I re-open the new out.dat,I notice that some "large string records" >are truncated... > >Is this how to correctly serialize the file..??? Yes. Note that there have been recent updates w.r.t serialization, if you are using the latest builds of TclKit, then you may have found a bug. -jcw From joshm@taconic.net Mon Apr 15 11:32:18 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FGWIa29811 for ; Mon, 15 Apr 2002 11:32:18 -0500 Received: from gatekeeper (ch-as5396-2-p86.taconic.net [205.231.30.133]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g3FGW8f11360 for ; Mon, 15 Apr 2002 12:32:09 -0400 (EDT) Message-ID: <0ec901c1e49a$c1df54e0$9e1de7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: <0e6f01c1e437$576e1190$9e1de7cd@gatekeeper> <20020415143811.9400@triqs.com> Subject: Re: [Metakit] How do I search for a substring? Date: Mon, 15 Apr 2002 12:29:45 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > The "shortest" confuses me, if I search for "abx" in the following: > aa > abxdd > abxe > abxff > ac > Which one(s) do you want? Silly me. After all these years, you'd think I could write a more coherent question. Given a column with: aa aabx abxdd abxe abxff ac I have two search questions: 1. LefthandFind("abx") should find "abxe". This is the string which matches the input string from the left (ala CString::StartsWith()) *and* is the shortest (CString::GetLength()) of the possible results. If two strings match this criteria, then I'm flexible on which one to find. 2. FindAllContaining("abx") should find *all* rows containing "abx" as a substring. (CString::Find() != -1). This would find 4 rows. They could be returned as a view, or there could be some sort of iterator. -- j From steve@improvision.com Mon Apr 15 11:45:17 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FGjHa30557 for ; Mon, 15 Apr 2002 11:45:17 -0500 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id RAA03849; Mon, 15 Apr 2002 17:42:16 +0100 (BST) Received: from [192.168.1.82] ([192.168.1.82]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971; Mon, 15 Apr 2002 17:45:10 +0100 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 15 Apr 2002 17:45:10 +0100 Subject: Re: [Metakit] How do I search for a substring? From: Stephen Baxter To: Joshua Muskovitz , MetaKit Mailing List Message-ID: In-Reply-To: <0ec901c1e49a$c1df54e0$9e1de7cd@gatekeeper> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A brute force search is actually blazingly fast in Metakit (the latency for getting a string from the database is very low) - how many rows are you talking about? -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: "Joshua Muskovitz" > Date: Mon, 15 Apr 2002 12:29:45 -0400 > To: "MetaKit Mailing List" > Subject: Re: [Metakit] How do I search for a substring? > >> The "shortest" confuses me, if I search for "abx" in the following: >> aa >> abxdd >> abxe >> abxff >> ac >> Which one(s) do you want? > > Silly me. After all these years, you'd think I could write a more coherent > question. > > Given a column with: > > aa > aabx > abxdd > abxe > abxff > ac > > I have two search questions: > > 1. LefthandFind("abx") should find "abxe". This is the string which > matches the input string from the left (ala CString::StartsWith()) *and* is > the shortest (CString::GetLength()) of the possible results. If two strings > match this criteria, then I'm flexible on which one to find. > > 2. FindAllContaining("abx") should find *all* rows containing "abx" as a > substring. (CString::Find() != -1). This would find 4 rows. They could be > returned as a view, or there could be some sort of iterator. > > -- j > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From joshm@taconic.net Mon Apr 15 11:48:37 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FGmaa30768 for ; Mon, 15 Apr 2002 11:48:36 -0500 Received: from gatekeeper (ch-as5396-2-p86.taconic.net [205.231.30.133]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g3FGmYf16418 for ; Mon, 15 Apr 2002 12:48:35 -0400 (EDT) Message-ID: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: Subject: Re: [Metakit] How do I search for a substring? Date: Mon, 15 Apr 2002 12:46:08 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > A brute force search is actually blazingly fast in Metakit (the latency for > getting a string from the database is very low) - how many rows are you > talking about? I would say in the 10k - 100k row range. -- j From lvirden@cas.org Tue Apr 16 06:03:35 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3GB3Za13899 for ; Tue, 16 Apr 2002 06:03:35 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g3GB3SS11181 for ; Tue, 16 Apr 2002 07:03:28 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g3GB3RY16631; Tue, 16 Apr 2002 07:03:27 -0400 (EDT) Date: Tue, 16 Apr 2002 07:03:27 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020416070327.AAB16622@cas.org> To: "MetaKit Mailing List" Subject: [Metakit] Submit your papers! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It is time to submit your abstract for a paper or tutorial to be presented at the Tcl Conference! This is your chance to let others know what you are doing. Don't think "Oh, I just don't have anything to contribute to the conference" - letting people know what you are doing helps others as well as helps yourself! See the details at the conference web page! Join us in Sept. for Tcl'2002: http://www.tcl.tk/community/tcl2002/ -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From tciuro@yahoo.com Tue Apr 16 10:18:34 2002 Received: from web20606.mail.yahoo.com (web20606.mail.yahoo.com [216.136.226.164]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3GFIYa26326 for ; Tue, 16 Apr 2002 10:18:34 -0500 Message-ID: <20020416151829.89572.qmail@web20606.mail.yahoo.com> Received: from [63.202.173.197] by web20606.mail.yahoo.com via HTTP; Tue, 16 Apr 2002 08:18:29 PDT Date: Tue, 16 Apr 2002 08:18:29 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 To: Jean-Claude Wippler , metakit@equi4.com Cc: Antonio Rodriguez In-Reply-To: <20020415144448.20573@triqs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, On Mac OS X, I decided from day #1 to use a different route: use ProjectBuilder (it's included with Mac OS X) and package everything in a framework. Forget about those Libtool blues... It's dead easy and it works fine. -- Tito --- Jean-Claude Wippler wrote: > Antonio Rodriguez wrote: > > >Trying to build Metakit on a portable Mac (I'm > temporarily away from my > >linux machines) and I am running into a problem > during the compile phase. > > > >I'm not used to building anything on OSX, so I'm > not quite sure of how to > >go about fixing this: > > > >cd metakit/builds > >../unix/configure --enable-python > ><< lots of autoconf stuff here >> > >make > ><< lots of stuff, then error: >> > > > >c++ -shared .libs/column.o .libs/custom.o > .libs/derived.o .libs/ > >fileio.o .libs/field.o .libs/format.o > .libs/handler.o .libs/persist.o > >.libs/remap.o .libs/std.o .libs/store.o > .libs/string.o .libs/table.o > >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o > .libs/libmk4.so > >/usr/bin/ld: Undefined symbols: > >_main > >___builtin_delete > >___builtin_vec_delete > >___builtin_vec_new > >___throw > >_terminate__Fv > >___builtin_new > >___pure_virtual > >___rtti_si > >___rtti_user > >make: *** [libmk4.la] Error 1 > > Ah yes, I've been struggling to understand this > myself. What I had to do > was edit builds/libtool and replace the "-shared" > with "-dynamiclib" to > make it work - it seems that otherwise c++ is trying > to build a self- > contained "main" executable, not a shared lib for > use by others. That is > of course a kludge. > > If there is a Mac OSX expert on this list who can > help with this, please > do follow up if possible. Libtool remains as clear > as mud to me, so I'm > essentially just feeling my way around in > considerable darkness... > > >PS Please cc me on replies as I am not a regular > list member yet. > > Done! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From antonio@mac.ps100.net Tue Apr 16 10:46:46 2002 Received: from mac.ps100.net (h00045a2a5bb8.ne.client2.attbi.com [65.96.154.86]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3GFkka28272; Tue, 16 Apr 2002 10:46:46 -0500 Received: from tintin ([127.0.0.1] helo=mac.ps100.net) by mac.ps100.net with smtp (Exim 3.12 #1 (Debian)) id 16xV5C-0000Cm-00; Tue, 16 Apr 2002 11:41:26 -0400 Message-ID: <6667222.1018971656287.JavaMail.upgradewebapp@mac.ps100.net> From: Antonio Rodriguez To: jcw@equi4.com, metakit@equi4.com, tciuro@yahoo.com Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 Cc: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit x-memora-stamp: 1018971654816 Date: Tue, 16 Apr 2002 11:41:26 -0400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, Do you have your project builder project file? I'm not familiar with PB, but I'm assuming that it saves project options (compilation flags, target dirs, etc) to some sort of file(s). If so, maybe the could be included in an ext/ dir in the metakit dir. Since everyone with OSX has PB by default, this might be pretty handy. Thanks Antonio -|Tito Ciuro wrote on Tue Apr 16 11:14:12 EDT 2002: Hello, On Mac OS X, I decided from day #1 to use a different route: use ProjectBuilder (it's included with Mac OS X) and package everything in a framework. Forget about those Libtool blues... It's dead easy and it works fine. -- Tito --- Jean-Claude Wippler wrote: > Antonio Rodriguez wrote: > > >Trying to build Metakit on a portable Mac (I'm > temporarily away from my > >linux machines) and I am running into a problem > during the compile phase. > > > >I'm not used to building anything on OSX, so I'm > not quite sure of how to > >go about fixing this: > > > >cd metakit/builds > >../unix/configure --enable-python > ><< lots of autoconf stuff here >> > >make > ><< lots of stuff, then error: >> > > > >c++ -shared .libs/column.o .libs/custom.o > .libs/derived.o .libs/ > >fileio.o .libs/field.o .libs/format.o > .libs/handler.o .libs/persist.o > >.libs/remap.o .libs/std.o .libs/store.o > .libs/string.o .libs/table.o > >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o > .libs/libmk4.so > >/usr/bin/ld: Undefined symbols: > >_main > >___builtin_delete > >___builtin_vec_delete > >___builtin_vec_new > >___throw > >_terminate__Fv > >___builtin_new > >___pure_virtual > >___rtti_si > >___rtti_user > >make: *** [libmk4.la] Error 1 > > Ah yes, I've been struggling to understand this > myself. What I had to do > was edit builds/libtool and replace the "-shared" > with "-dynamiclib" to > make it work - it seems that otherwise c++ is trying > to build a self- > contained "main" executable, not a shared lib for > use by others. That is > of course a kludge. > > If there is a Mac OSX expert on this list who can > help with this, please > do follow up if possible. Libtool remains as clear > as mud to me, so I'm > essentially just feeling my way around in > considerable darkness... > > >PS Please cc me on replies as I am not a regular > list member yet. > > Done! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From tciuro@yahoo.com Tue Apr 16 19:41:55 2002 Received: from web20606.mail.yahoo.com (web20606.mail.yahoo.com [216.136.226.164]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3H0fsa03263 for ; Tue, 16 Apr 2002 19:41:54 -0500 Message-ID: <20020417004152.21010.qmail@web20606.mail.yahoo.com> Received: from [17.202.13.44] by web20606.mail.yahoo.com via HTTP; Tue, 16 Apr 2002 17:41:52 PDT Date: Tue, 16 Apr 2002 17:41:52 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 To: Antonio Rodriguez , jcw@equi4.com, metakit@equi4.com Cc: metakit@equi4.com In-Reply-To: <6667222.1018971656287.JavaMail.upgradewebapp@mac.ps100.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: No problem... Jean-Claude? Do you have any comments on this? Can I send you the pbproj file containing the settings? BTW... The pbproj has been created with the latest PB, which will be delivered in WWDC 2002 (May 6th). You should be able to open it with the the version delivered with 10.1.X though... Otherwise, let me know and I'll see what I can do... -- Tito --- Antonio Rodriguez wrote: > Tito, > > Do you have your project builder project file? I'm > not familiar with PB, but I'm assuming that it saves > project options (compilation flags, target dirs, > etc) to some sort of file(s). > > If so, maybe the could be included in an ext/ dir in > the metakit dir. Since everyone with OSX has PB by > default, this might be pretty handy. > > Thanks > > Antonio > > > -|Tito Ciuro wrote on Tue Apr 16 > 11:14:12 EDT 2002: > > Hello, > > On Mac OS X, I decided from day #1 to use a > different > route: use ProjectBuilder (it's included with Mac OS > X) and package everything in a framework. Forget > about > those Libtool blues... > > It's dead easy and it works fine. > > -- Tito > > --- Jean-Claude Wippler wrote: > > Antonio Rodriguez wrote: > > > > >Trying to build Metakit on a portable Mac (I'm > > temporarily away from my > > >linux machines) and I am running into a problem > > during the compile phase. > > > > > >I'm not used to building anything on OSX, so I'm > > not quite sure of how to > > >go about fixing this: > > > > > >cd metakit/builds > > >../unix/configure --enable-python > > ><< lots of autoconf stuff here >> > > >make > > ><< lots of stuff, then error: >> > > > > > >c++ -shared .libs/column.o .libs/custom.o > > .libs/derived.o .libs/ > > >fileio.o .libs/field.o .libs/format.o > > .libs/handler.o .libs/persist.o > > >.libs/remap.o .libs/std.o .libs/store.o > > .libs/string.o .libs/table.o > > >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o > > .libs/libmk4.so > > >/usr/bin/ld: Undefined symbols: > > >_main > > >___builtin_delete > > >___builtin_vec_delete > > >___builtin_vec_new > > >___throw > > >_terminate__Fv > > >___builtin_new > > >___pure_virtual > > >___rtti_si > > >___rtti_user > > >make: *** [libmk4.la] Error 1 > > > > Ah yes, I've been struggling to understand this > > myself. What I had to do > > was edit builds/libtool and replace the "-shared" > > with "-dynamiclib" to > > make it work - it seems that otherwise c++ is > trying > > to build a self- > > contained "main" executable, not a shared lib for > > use by others. That is > > of course a kludge. > > > > If there is a Mac OSX expert on this list who can > > help with this, please > > do follow up if possible. Libtool remains as > clear > > as mud to me, so I'm > > essentially just feeling my way around in > > considerable darkness... > > > > >PS Please cc me on replies as I am not a regular > > list member yet. > > > > Done! > > > > -jcw > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > __________________________________________________ > Do You Yahoo!? > Yahoo! Tax Center - online filing with TurboTax > http://taxes.yahoo.com/ > > __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From tciuro@yahoo.com Tue Apr 16 19:41:55 2002 Received: from web20606.mail.yahoo.com (web20606.mail.yahoo.com [216.136.226.164]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3H0fta03265 for ; Tue, 16 Apr 2002 19:41:55 -0500 Message-ID: <20020417004152.21010.qmail@web20606.mail.yahoo.com> Received: from [17.202.13.44] by web20606.mail.yahoo.com via HTTP; Tue, 16 Apr 2002 17:41:52 PDT Date: Tue, 16 Apr 2002 17:41:52 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 To: Antonio Rodriguez , jcw@equi4.com, metakit@equi4.com Cc: metakit@equi4.com In-Reply-To: <6667222.1018971656287.JavaMail.upgradewebapp@mac.ps100.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: No problem... Jean-Claude? Do you have any comments on this? Can I send you the pbproj file containing the settings? BTW... The pbproj has been created with the latest PB, which will be delivered in WWDC 2002 (May 6th). You should be able to open it with the the version delivered with 10.1.X though... Otherwise, let me know and I'll see what I can do... -- Tito --- Antonio Rodriguez wrote: > Tito, > > Do you have your project builder project file? I'm > not familiar with PB, but I'm assuming that it saves > project options (compilation flags, target dirs, > etc) to some sort of file(s). > > If so, maybe the could be included in an ext/ dir in > the metakit dir. Since everyone with OSX has PB by > default, this might be pretty handy. > > Thanks > > Antonio > > > -|Tito Ciuro wrote on Tue Apr 16 > 11:14:12 EDT 2002: > > Hello, > > On Mac OS X, I decided from day #1 to use a > different > route: use ProjectBuilder (it's included with Mac OS > X) and package everything in a framework. Forget > about > those Libtool blues... > > It's dead easy and it works fine. > > -- Tito > > --- Jean-Claude Wippler wrote: > > Antonio Rodriguez wrote: > > > > >Trying to build Metakit on a portable Mac (I'm > > temporarily away from my > > >linux machines) and I am running into a problem > > during the compile phase. > > > > > >I'm not used to building anything on OSX, so I'm > > not quite sure of how to > > >go about fixing this: > > > > > >cd metakit/builds > > >../unix/configure --enable-python > > ><< lots of autoconf stuff here >> > > >make > > ><< lots of stuff, then error: >> > > > > > >c++ -shared .libs/column.o .libs/custom.o > > .libs/derived.o .libs/ > > >fileio.o .libs/field.o .libs/format.o > > .libs/handler.o .libs/persist.o > > >.libs/remap.o .libs/std.o .libs/store.o > > .libs/string.o .libs/table.o > > >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o > > .libs/libmk4.so > > >/usr/bin/ld: Undefined symbols: > > >_main > > >___builtin_delete > > >___builtin_vec_delete > > >___builtin_vec_new > > >___throw > > >_terminate__Fv > > >___builtin_new > > >___pure_virtual > > >___rtti_si > > >___rtti_user > > >make: *** [libmk4.la] Error 1 > > > > Ah yes, I've been struggling to understand this > > myself. What I had to do > > was edit builds/libtool and replace the "-shared" > > with "-dynamiclib" to > > make it work - it seems that otherwise c++ is > trying > > to build a self- > > contained "main" executable, not a shared lib for > > use by others. That is > > of course a kludge. > > > > If there is a Mac OSX expert on this list who can > > help with this, please > > do follow up if possible. Libtool remains as > clear > > as mud to me, so I'm > > essentially just feeling my way around in > > considerable darkness... > > > > >PS Please cc me on replies as I am not a regular > > list member yet. > > > > Done! > > > > -jcw > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > __________________________________________________ > Do You Yahoo!? > Yahoo! Tax Center - online filing with TurboTax > http://taxes.yahoo.com/ > > __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From jcw@equi4.com Wed Apr 17 01:46:48 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3H6kla18924; Wed, 17 Apr 2002 01:46:47 -0500 From: Jean-Claude Wippler To: Antonio Rodriguez , Date: Wed, 17 Apr 2002 08:46:30 +0200 Message-Id: <20020417064630.1458@triqs.com> In-Reply-To: <20020417004152.21010.qmail@web20606.mail.yahoo.com> References: <20020417004152.21010.qmail@web20606.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Taking if further (Was: Building metakit 2.3.34 on Mac OSX 10.1.3) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, [macosx ProjectBuilder] >Jean-Claude? Do you have any comments on this? Can I >send you the pbproj file containing the settings? I'd be delighted to add a PB setup to the MK distribution. The deliverables that I think will matter most are - in decreasing order of importance: - MK static library - MK shared library - regression test (non-gui) application - debug build of the above, to help catch more issues As you know, i have PB set up (dec 2001 developer CD). I'll be more than happy to help tweak, test, extend it as needed. PB is still new to me, but I got surprising mileage out of it so far, it really is a great setup harnessing the gcc and gdb tools - so *YES* please! >BTW... The pbproj has been created with the latest PB, >which will be delivered in WWDC 2002 (May 6th). You >should be able to open it with the the version >delivered with 10.1.X though... Good point. That is indeed a concern - if this thing also works for people who do not have access to the latest and greatest PB, then so much the better. Let me re-iterate that I welcome all contributions to MetaKit: large and small, code and docs. MK might look like a one-man show to some, but it isn't by far anymore - and the more you help improve it in ways *you* find important, the better. MK is a foundation for almost everything I do, and I know a couple companies and people who pound & rely *very* heavily on it. You can take it forward too - trust me, MK's future looks extremely bright, for all of C++, Python, and Tcl. For reasons of confidentiality, I cannot always list all people involved, but I do always try to acknowledge all contributions as much as I can on the MK, Python, and Tcl web pages. Go for it, Tito & Antonio :) -jcw PS. No need to CC me, as I get all email from the list anyway. From jcw@equi4.com Wed Apr 17 02:08:54 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3H78ra20021 for ; Wed, 17 Apr 2002 02:08:53 -0500 From: Jean-Claude Wippler To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? Date: Wed, 17 Apr 2002 09:08:31 +0200 Message-Id: <20020417070832.22300@triqs.com> In-Reply-To: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joshua Muskovitz wrote: [Stephen Baxter] >> A brute force search is actually blazingly fast in Metakit >> [...] - how many rows are you talking about? > >I would say in the 10k - 100k row range. Ok, if you want instant GUI responsiveness per key hit, and given that you're doing this with strings, not numbers (which are one step faster still), you *might* need to add tricks. There are plenty to think of IMO. > 1. LefthandFind("abx") should find "abxe". Use c4_View's Search or Locate. They will not find a match, but they will find the position of the first entry >= your key (lexicographically, and ignoring case). This uses binary search. If the results are not as you want them, it'd be very easy to write a binary search yourself, and it'd be just as fast as these routines. > 2. FindAllContaining("abx") That is - without extra indexing preparations - not doable through any other means than brute force linear search. Use a loop from 0 to GetSize()-1 and do a strstr-like match yourself. The key to performance is to not create a copy of the string you get from MK. In 99% of the cases, you will be accessing a string which is in memory-mapped file, i.e. *straight* off the paged-in virtual memory mapped over the search column. To give an indication of speed: it would not surprise me if *even* brute force for both requirements leads to tenths-of-a-second access for 100k rows on today's desktop hardware. My suggestion would be to just try the brute force approach, as Stephen suggested. Avoid string copies, definitely avoid constructing a string object on each iteration, and you might just find out that "simplest" works fine. If you needs more speed in case #1, switch to the binary search. Of you need to optimize case #2, consider creating an extra int column, perhaps a bitmap for all the letters in the item - and then do a loop using "if ((bitmap & searchbits) == searchbits)". I've used similar tricks for full-text keyword searches at times, with very good results. -jcw From andreas.muegge@gmx.de Wed Apr 17 03:25:43 2002 Received: from mx0.gmx.net (mx0.gmx.de [213.165.64.100]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3H8Pha23624 for ; Wed, 17 Apr 2002 03:25:43 -0500 Received: (qmail 3783 invoked by uid 0); 17 Apr 2002 08:25:36 -0000 Date: Wed, 17 Apr 2002 10:25:35 +0200 (MEST) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: MetaKit Mailing List MIME-Version: 1.0 References: <20020417070832.22300@triqs.com> Subject: Re: [Metakit] How do I search for a substring? X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [217.194.32.16] Message-ID: <22644.1019031935@www47.gmx.net> X-Mailer: WWW-Mail 1.5 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude, [fulltext search] > If you needs more speed in case #1, switch to the binary search. Of you > need to optimize case #2, consider creating an extra int column, perhaps > a bitmap for all the letters in the item - and then do a loop using "if > ((bitmap & searchbits) == searchbits)". I've used similar tricks for > full-text keyword searches at times, with very good results. Can you show me a simple example for case 2? I much interested in speeding up my substring search. It's astonishing fast already but decreasing the response time for the user even further would be great. Andreas Muegge From ken@clres.com Wed Apr 17 10:03:12 2002 Received: from mail.his.com (root@mail.his.com [216.194.225.77]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HF3Ca14022; Wed, 17 Apr 2002 10:03:12 -0500 Received: from clres.com (max1k-10.his.com [216.194.203.10]) by mail.his.com (8.9.3/8.9.3) with ESMTP id LAA06723; Wed, 17 Apr 2002 11:03:06 -0400 (EDT) Message-ID: <3CBD8EA9.8080709@clres.com> Date: Wed, 17 Apr 2002 11:03:05 -0400 From: Ken Litkowski Organization: CL Research User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >>1. LefthandFind("abx") should find "abxe". > > Use c4_View's Search or Locate. They will not find a match, but they > will find the position of the first entry >= your key (lexicographically, > and ignoring case). This uses binary search. If the results are not as > you want them, it'd be very easy to write a binary search yourself, and > it'd be just as fast as these routines. > Are you saying that this is case-insensitive, so that using "Search" or "search" will return the same position when using Search or Locate? I have always gone through elaborate procedures to deal with case and it would certainly help to know that Search and Locate are case-insensitive. Ken -- Ken Litkowski TEL.: 301-482-0237 CL Research EMAIL: ken@clres.com 9208 Gue Road Damascus, MD 20872-1025 USA Home Page: http://www.clres.com From steve@improvision.com Wed Apr 17 10:16:32 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HFGVa15075; Wed, 17 Apr 2002 10:16:31 -0500 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id QAA00847; Wed, 17 Apr 2002 16:13:32 +0100 (BST) Received: from [192.168.1.82] ([192.168.1.82]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971; Wed, 17 Apr 2002 16:16:27 +0100 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Wed, 17 Apr 2002 16:16:28 +0100 Subject: Re: [Metakit] How do I search for a substring? From: Stephen Baxter To: Ken Litkowski , Jean-Claude Wippler CC: MetaKit Mailing List Message-ID: In-Reply-To: <3CBD8EA9.8080709@clres.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I found Search() and Find() could be slow for large datasets because of the number of virtual function calls involved. I ended up writing my own (simple) routine for doing case-insensitive string searches: /*************************************************************************** ************ FindString Find the string s in property prop in view. Returns the row if found or -1 otherwise. This basically replaces c4_View::Find() with a more efficient implementation. *************************************************************************** ************/ SInt32 XMetakitDB::FindString( const XString& s1, const c4_StringProp& prop, const c4_View& view, SInt32 start ) { // Do a direct comparison for speed register char c1; register char c2; char delta = 'A'-'a'; SInt32 count = view.GetSize(); SInt32 index = start; while( index < count ) { // Get the property const char *p1 = prop.Get(view[index]); // Do the compare const char *p2 = s1; while( 1 ) { c1 = *p1; c2 = *p2; p1++; p2++; // OK, we have two valid chars - make them both upper case if( (c1 >= 'a') && (c1 <= 'z') ) { c1 += delta; } if( (c2 >= 'a') && (c2 <= 'z') ) { c2 += delta; } if( c1 != c2 ) { // Strings are different break; } else if( !c1 ) { // We have reached the end of both of the strings // and they are identical return index; } } index++; } return -1; } -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Ken Litkowski > Organization: CL Research > Date: Wed, 17 Apr 2002 11:03:05 -0400 > To: Jean-Claude Wippler > Cc: MetaKit Mailing List > Subject: Re: [Metakit] How do I search for a substring? > > Jean-Claude Wippler wrote: > >>> 1. LefthandFind("abx") should find "abxe". >> >> Use c4_View's Search or Locate. They will not find a match, but they >> will find the position of the first entry >= your key (lexicographically, >> and ignoring case). This uses binary search. If the results are not as >> you want them, it'd be very easy to write a binary search yourself, and >> it'd be just as fast as these routines. >> > > > Are you saying that this is case-insensitive, so that using "Search" or > "search" will return the same position when using Search or Locate? I > have always gone through elaborate procedures to deal with case and it > would certainly help to know that Search and Locate are case-insensitive. > > Ken > -- > Ken Litkowski TEL.: 301-482-0237 > CL Research EMAIL: ken@clres.com > 9208 Gue Road > Damascus, MD 20872-1025 USA Home Page: http://www.clres.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From joshm@taconic.net Wed Apr 17 14:57:50 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HJvoa03243 for ; Wed, 17 Apr 2002 14:57:50 -0500 Received: from gatekeeper (ch-asc7-p6.taconic.net [205.231.148.54]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g3HJvmf13732 for ; Wed, 17 Apr 2002 15:57:48 -0400 (EDT) Message-ID: <00ab01c1e649$cb0a5e60$0f94e7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> <3CBD8EA9.8080709@clres.com> Subject: Re: [Metakit] How do I search for a substring? Date: Wed, 17 Apr 2002 15:55:14 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Are you saying that this is case-insensitive, so that using "Search" or > "search" will return the same position when using Search or Locate? I > have always gone through elaborate procedures to deal with case and it > would certainly help to know that Search and Locate are case-insensitive. We solved this problem by keeping an additional column of forced lower-case representations (not that much waste, given our particular design), and forcing the search key to lower case before doing the search. J-C, The idea of keeping a bitfield of used characters is ingenious, and it fits into an int! I'll have to contemplate whether this is useful in our particular application, but I'm sure I'll use it at some point in my life. Thanks for another cool algorithm to add to my heap. -- j From DavidA@ActiveState.com Wed Apr 17 15:09:15 2002 Received: from smtp1.ActiveState.com (gw.activestate.com [209.17.183.249]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HK9Fa04020 for ; Wed, 17 Apr 2002 15:09:15 -0500 Received: from smtp3.ActiveState.com (smtp3.ActiveState.com [192.168.3.19]) by smtp1.ActiveState.com (8.11.6/8.11.6) with ESMTP id g3HK8A228785 for ; Wed, 17 Apr 2002 13:08:10 -0700 Received: from ActiveState.com (loom.activestate.com [192.168.3.137]) by smtp3.ActiveState.com (8.11.6/8.11.6) with ESMTP id g3HK88e03614 for ; Wed, 17 Apr 2002 13:08:08 -0700 Message-ID: <3CBDD628.6090908@ActiveState.com> Date: Wed, 17 Apr 2002 13:08:08 -0700 From: David Ascher User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.9) Gecko/20020311 X-Accept-Language: en-us, en MIME-Version: 1.0 To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Filtered-By: PerlMx makes it fast and easy. See http://www.ActiveState.com/Products/PerlMx/Header Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The discussion re: how to do text searches in metakit makes me wonder if someone (jcw?) wouldn't want to use metakit to come up with a reusable full text indexer that was more portable and easier to integrate than any of the other open source ones I've seen. I've spent more hours than I care to admit researching the topic, and finding basically nothing which is both high-performing, under a non-restrictive license, portable, and easy to setup... There's a lot of blind alleys to go in when it comes to full text searching, but I think that JCW could come up with a framework that would solve the problem for 90% of the cases where it's likely to be used (inside apps for on-line help, inside simple web sites, etc.). Just a suggestion. --da From readonly@getsoft.com Wed Apr 17 15:54:34 2002 Received: from freedom.vosn.net ([157.238.46.77]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HKsYa06981 for ; Wed, 17 Apr 2002 15:54:34 -0500 Received: from c18520.eburwd2.vic.optusnet.com.au ([210.49.191.115] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.35 #1) id 16xwRU-0008B0-00; Wed, 17 Apr 2002 14:54:16 -0600 Date: Thu, 18 Apr 2002 06:54:16 +1000 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <16346665221.20020418065416@getsoft.com> To: David Ascher CC: MetaKit Mailing List Subject: Re[2]: [Metakit] How do I search for a substring? In-Reply-To: <3CBDD628.6090908@ActiveState.com> References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> <3CBDD628.6090908@ActiveState.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi David, I'm not exactly sure what you are looking for when you say "full text indexer" but you may find Mifluz http://www.gnu.org/software/mifluz/ of interest. From memory there are others like this as well. Thursday, April 18, 2002, 6:08:08 AM, you wrote: DA> The discussion re: how to do text searches in metakit makes me wonder if DA> someone (jcw?) wouldn't want to use metakit to come up with a reusable DA> full text indexer that was more portable and easier to integrate than DA> any of the other open source ones I've seen. I've spent more hours than DA> I care to admit researching the topic, and finding basically nothing DA> which is both high-performing, under a non-restrictive license, DA> portable, and easy to setup... DA> There's a lot of blind alleys to go in when it comes to full text DA> searching, but I think that JCW could come up with a framework that DA> would solve the problem for 90% of the cases where it's likely to be DA> used (inside apps for on-line help, inside simple web sites, etc.). DA> Just a suggestion. -- Best regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 now available. New: Projects, Source Database, Difference Analysis.... Soft As It Gets Pty Ltd, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Ph +61 3 9885 4445, Fax +61 3 9885 4444, mailto:readonly@getsoft.com From DavidA@ActiveState.com Wed Apr 17 16:28:26 2002 Received: from smtp1.ActiveState.com (gw.activestate.com [209.17.183.249]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HLSPa09457 for ; Wed, 17 Apr 2002 16:28:25 -0500 Received: from smtp3.ActiveState.com (smtp3.ActiveState.com [192.168.3.19]) by smtp1.ActiveState.com (8.11.6/8.11.6) with ESMTP id g3HLS0213145; Wed, 17 Apr 2002 14:28:00 -0700 Received: from ActiveState.com (loom.activestate.com [192.168.3.137]) by smtp3.ActiveState.com (8.11.6/8.11.6) with ESMTP id g3HLRxe12030; Wed, 17 Apr 2002 14:27:59 -0700 Message-ID: <3CBDE8DF.4090801@ActiveState.com> Date: Wed, 17 Apr 2002 14:27:59 -0700 From: David Ascher User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.9) Gecko/20020311 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Neville Franks CC: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> <3CBDD628.6090908@ActiveState.com> <16346665221.20020418065416@getsoft.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Filtered-By: PerlMx makes it fast and easy. See http://www.ActiveState.com/Products/PerlMx/Header Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >Hi David, >I'm not exactly sure what you are looking for when you say "full text >indexer" but you may find Mifluz http://www.gnu.org/software/mifluz/ >of interest. From memory there are others like this as well. > There are lots. Mifluz fails my particular test because it appears to have no users on Windows, and the GPL license is one I can't use in some projects. But thanks for the pointer, I hadn't heard of that particular one.-- there should be a list of all of the open source searching solutions out there somewhere... --david From antonio@mac.ps100.net Wed Apr 17 22:16:10 2002 Received: from mac.ps100.net (h00045a2a5bb8.ne.client2.attbi.com [65.96.154.86]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3I3G9a27196 for ; Wed, 17 Apr 2002 22:16:09 -0500 Received: from tintin ([127.0.0.1] helo=mac.ps100.net) by mac.ps100.net with smtp (Exim 3.12 #1 (Debian)) id 16y2K4-0003ak-00 for ; Wed, 17 Apr 2002 23:11:00 -0400 Message-ID: <2490106.1019099434764.JavaMail.upgradewebapp@mac.ps100.net> From: Antonio Rodriguez To: metakit@equi4.com Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit x-memora-stamp: 1019099434698 Date: Wed, 17 Apr 2002 23:11:00 -0400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hey all, So I made the changes you suggested to the libtool wrapper script J-C and I'm now getting what may be a more basic build error. I think that long-term Tito's approach is better (using native OSX dev. tools) but I am now being stubborn about getting this to work in the PB-less way. Below is my new error-- note that python headers for me is in /sw/include because I use fink: /bin/sh ./libtool --tag=CXX c++ -o libmk4py.la -O2 -I../unix/../include -I../unix/../src -I. -module -avoid-version \ -rpath /sw/lib PyProperty.lo PyRowRef.lo PyStorage.lo PyView.lo PWOImp.lo column.lo custom.lo derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo c++ -dynamiclib .libs/PyProperty.o .libs/PyRowRef.o .libs/PyStorage.o .libs/PyView.o .libs/PWOImp.o .libs/column.o .libs/custom.o .libs/derived.o .libs/fileio.o .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o .libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o .libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4py.so ld: Undefined symbols: _PyExc_IndexError _PyExc_TypeError _PyInt_FromLong _PyNumber_Check _PySequence_Check _PySequence_GetItem _PyString_AsString _PyString_FromString _PyString_FromStringAndSize _PyString_Type _PyType_IsSubtype _PyType_Type _Py_FindMethod _PyErr_Clear _PyErr_SetString _PyExc_AttributeError _PyFloat_FromDouble _PyFloat_Type _PyInt_Type _PyList_New _PyList_SetItem _PyLong_AsLong _PyLong_FromLong _PyLong_Type _PyNumber_Float _PyNumber_Int _PySequence_Size __Py_NoneStruct _PyDict_GetItemString _PyDict_SetItemString _PyDict_Type _PyExc_IOError _PyExc_ValueError _PyFile_AsFile _PyFile_Type _PyInstance_Type _PyObject_CallMethod _PyObject_GetAttrString _PyObject_SetAttrString _PyString_Size _Py_InitModule4 _PyCallable_Check _PyDict_DelItemString _PyDict_New _PyExc_KeyError _PyExc_RuntimeError _PyInt_AsLong _PyList_GetItem _PyList_Type _PyMapping_Check _PyMapping_GetItemString _PyMapping_HasKeyString _PyObject_IsTrue _PyObject_Size _PySequence_GetSlice _PyTuple_New _PyTuple_SetItem _PyTuple_Type __PyString_Resize _PyDict_SetItem _PyEval_CallObjectWithKeywords _PyList_AsTuple /usr/bin/libtool: internal link edit command failed make: *** [libmk4py.la] Error 1 Antonio -|Jean-Claude Wippler wrote on Mon Apr 15 10:42:06 EDT 2002: Antonio Rodriguez wrote: >Trying to build Metakit on a portable Mac (I'm temporarily away from my >linux machines) and I am running into a problem during the compile phase. > >I'm not used to building anything on OSX, so I'm not quite sure of how to >go about fixing this: > >cd metakit/builds >../unix/configure --enable-python ><< lots of autoconf stuff here >> >make ><< lots of stuff, then error: >> > >c++ -shared .libs/column.o .libs/custom.o .libs/derived.o .libs/ >fileio.o .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o >.libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4.so >/usr/bin/ld: Undefined symbols: >_main >___builtin_delete >___builtin_vec_delete >___builtin_vec_new >___throw >_terminate__Fv >___builtin_new >___pure_virtual >___rtti_si >___rtti_user >make: *** [libmk4.la] Error 1 Ah yes, I've been struggling to understand this myself. What I had to do was edit builds/libtool and replace the "-shared" with "-dynamiclib" to make it work - it seems that otherwise c++ is trying to build a self- contained "main" executable, not a shared lib for use by others. That is of course a kludge. If there is a Mac OSX expert on this list who can help with this, please do follow up if possible. Libtool remains as clear as mud to me, so I'm essentially just feeling my way around in considerable darkness... >PS Please cc me on replies as I am not a regular list member yet. Done! -jcw From jcw@equi4.com Fri Apr 19 04:33:34 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3J9XWa24809 for ; Fri, 19 Apr 2002 04:33:33 -0500 From: Jean-Claude Wippler To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? Date: Fri, 19 Apr 2002 11:32:52 +0200 Message-Id: <20020419093252.28655@triqs.com> In-Reply-To: <22644.1019031935@www47.gmx.net> References: <22644.1019031935@www47.gmx.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g3J9XWa24809 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas Mügge wrote: >[fulltext search] >> If you needs more speed in case #1, switch to the binary search. Of you >> need to optimize case #2, consider creating an extra int column, perhaps >> a bitmap for all the letters in the item - and then do a loop using "if >> ((bitmap & searchbits) == searchbits)". I've used similar tricks for >> full-text keyword searches at times, with very good results. > >Can you show me a simple example for case 2? I much interested in speeding >up my substring search. It's astonishing fast already but decreasing the >response time for the user even further would be great. Let me describe the logic: - add an int property to your view, say "pMap" - for each row, set the value as follows: - if key contains 'a' or 'A', set bit 0 - if key contains 'b' or 'B', set bit 1 - etc... - that leaves a few bits to spare, you could eve refine things by using bits for "has digits", "has punctuation", "is all uppercase", "1st upper + rest lower", and so on - when you want to do a "string contains" lookup, first calculate its mask, using the same rules as above Now the key is that you have a way to very quickly decide whether a row cannot possible match, using the expression shown above. On a miss, quickly move on, on a match, you'll need to check further using a normal string match. This idea of using a quick "filtering pre-check" is not new. In the case of MK, it makes brute force more effective. Int access is more efficient that string access which needs to deal with variable-size offest calculations. Here's another example of the same, which helps with full-text searching, assuming that one only wants to find matches on word-beginnings: - add 3 int properties, say: p1, p2, p3 - bit 0 in p1 means: there is at least one word in the text which starts with 'a' or 'A' (bit 1 for [bB], etc) - bit 0 in p2 means: there is at least one word in the text which has 'a' or 'A' in the second position - bit 0 in p2 means: there is at least one word in the text which has 'a' or 'A' in the third position - and so on, for a total overhead of 12 bytes per row Now, searching for "abe" means creating three searchbit ints, with respectively b0, b1, and b4 set. To find all matching rows, each of those three ints must match with p1/p2/p3, respectively. Of they do, then you will still find false positives, but a simple normal string search will weed those out. The result, pretty good full-text retrieval performance for moderate amounts of text. The idea can be refined to work with 1..N char searches (while still using 3 ints, these are probably often sufficient). Note that you can mix and match blocked viewers with normal views. Even if the text is stored in blocked form, you can choose to place these ints in normal flat (and huge) int vector form. That makes them far more efficient for brute force scanning, and the combination can be made through either the c4_View::Pair or the c4_View::RemapWith operators. There is a lot of hidden speed in such designs. It would be *grand* to add one or more new custom viewers to MK which support such search modes. I've been wanting to for ages, but haven't found the opportunity and time to work on it so far. If you'd like to have a go at this, and get at least the basics of such a mechanism in place, I'll gladly help tweak it and integrate it all into a future MetaKit release. As with so many things, simple tricks sometimes can go a long way. -jcw From andreas.muegge@gmx.de Fri Apr 19 07:19:12 2002 Received: from mx0.gmx.net (mx0.gmx.net [213.165.64.100]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3JCJBa01693 for ; Fri, 19 Apr 2002 07:19:11 -0500 Received: (qmail 1099 invoked by uid 0); 19 Apr 2002 12:19:00 -0000 Date: Fri, 19 Apr 2002 14:19:00 +0200 (MEST) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: MetaKit Mailing List MIME-Version: 1.0 References: <20020419093252.28655@triqs.com> Subject: Re: [Metakit] How do I search for a substring? X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [217.194.32.11] Message-ID: <15233.1019218740@www17.gmx.net> X-Mailer: WWW-Mail 1.5 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Jean-Claude, > Let me describe the logic: > - add an int property to your view, say "pMap" > - for each row, set the value as follows: > - if key contains 'a' or 'A', set bit 0 > - if key contains 'b' or 'B', set bit 1 > - etc... > ... Thank you! Now everything is clear, indeed a clever approach which I will try out in the next future. The only drawback is that it won't work when I must store longer strings (1k or more) but in that case I need some kind of fulltext indexing anyway. Btw, how could I store such an index in Metakit? Let's take for example something like this: , , , ..., For every word which occurs I create a new entry and then I must add a variable number of integers to this row (for each document/record where the word was found). How can I do this? With Subviews (and if yes how - this is a topic I haven't touched before)? Ciao, Andreas Muegge From brunobl@ifrance.com Thu Apr 25 19:08:17 2002 Received: from pop.univ-lyon1.fr (postfix@pop.univ-lyon1.fr [134.214.100.7]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3Q08Ha03826 for ; Thu, 25 Apr 2002 19:08:17 -0500 Received: by pop.univ-lyon1.fr (Postfix, from userid 3081) id F33C823EAC; Fri, 26 Apr 2002 02:08:15 +0200 (CEST) Received: from [134.214.164.67] (I435.resI.insa-lyon.fr [134.214.164.67]) by pop.univ-lyon1.fr (Postfix) with ESMTP id 288F9BF07 for ; Fri, 26 Apr 2002 02:08:15 +0200 (CEST) User-Agent: Microsoft-Entourage/10.0.0.1309 Date: Fri, 26 Apr 2002 02:08:14 +0200 From: Bruno Blondeau To: Message-ID: In-Reply-To: <200204191701.g3JH12a16229@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Subject: [Metakit] Unable to update from 2.4.1 to 2.4.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I recently updated my application, and also included a newer version of Metakit (2.4.4 instead of 2.4.1). I thought the data file would be converted automatically or the file format remain the same. However, it seems not to be the case. My application is buggy now. After spending several hours, I eventually reverted to 2.4.1, and everything works fine again... Do I have to do something to ask Metakit to convert my data file from the old format to the new one? Thanks, Bruno Blondeau From jcw@equi4.com Sun Apr 28 06:33:09 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SBX8a06555 for ; Sun, 28 Apr 2002 06:33:08 -0500 From: Jean-Claude Wippler To: metakit Date: Sun, 28 Apr 2002 13:32:19 +0200 Message-Id: <20020428113219.13489@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Bug fix, small ints Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Two bug fixes have just been checked into CVS - these were serious enough to point out, so here are the changelog entries: 2002-04-28 Fix small-int re-use view bug A nasty bug was reported by VPI which caused upper bytes to be truncated from int values. The problem appears when storing ints of 1..4 bits in a view, then clearing the view (so a gap is placed past the end of the column), then adding a row with an int of 2 or more bytes. This uncovered a bug in forgetting to truncate columns with sub-byte int storage. Now fixed, added regression test s43. This bug could also have affected string and byte storage, since these use int columns to store item sizes. Under very specific conditions, it may have lead to truncated or even mixed-up values. 2002-04-27 Fix nested mk4tcl loop bug Loops would exit prematurely when nested - due to objc/objv being overwritten in the inner loop. Affects mk4tcl.cpp and mk4too.cpp. FYI, there are now web interfaces to the MetaKit and the TclKit CVS repositories, based on ViewCVS - see the respective home pages. I'm still looking into a "clobbered string" issue, which appears to happen only when there are strings over 100b *and* there are at least several thousand rows. Unfortunately, I've not been able to reproduce the issue myself despite numerous ways of banging on MetaKit, so if you have run into such a problem, please let me know. -jcw From jcw@equi4.com Sun Apr 28 06:55:30 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SBtTa07068 for ; Sun, 28 Apr 2002 06:55:29 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 Date: Sun, 28 Apr 2002 13:54:37 +0200 Message-Id: <20020428115437.31702@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bruno Blondeau wrote: >I recently updated my application, and also included a newer version of >Metakit (2.4.4 instead of 2.4.1). > >I thought the data file would be converted automatically or the file format >remain the same. > >However, it seems not to be the case. > >My application is buggy now. After spending several hours, I eventually >reverted to 2.4.1, and everything works fine again... > >Do I have to do something to ask Metakit to convert my data file from the >old format to the new one? No, never. Either it has the same format (as it does from 2.3 to now), or it does conversion fully automatically (as it does for 2.01 and older). Maybe something else is the problem. I just checked in a bug fix, so I would definitely advise to either try the CVS latest source, or wait until there is a 2.4.5 release - which is forthcoming. Now that the 2.4.4 release is being used far far more intensively and by more people (well, better late than never), so it looks like we are finally banging hard enough on MK to make all remaining bugs show up. I'd appreciate it if you could give me some more information on what sort of problems you were seeing - especially with the latest source code. -jcw From jcw@equi4.com Sun Apr 28 07:10:06 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SCA5a07449 for ; Sun, 28 Apr 2002 07:10:05 -0500 From: Jean-Claude Wippler To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? Date: Sun, 28 Apr 2002 14:09:10 +0200 Message-Id: <20020428120910.32592@triqs.com> In-Reply-To: <15233.1019218740@www17.gmx.net> References: <15233.1019218740@www17.gmx.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g3SCA5a07449 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas Mügge wrote: [...] >store longer strings (1k or more) but in that case I need some kind of >fulltext indexing anyway. > >Btw, how could I store such an index in Metakit? Let's take for example >something like this: > >, , , ..., > >For every word which occurs I create a new entry and then I must add a >variable number of integers to this row (for each document/record where >the word was found). How can I do this? With Subviews (and if yes how - >this is a topic I haven't touched before)? Subviews are definitely an option, but you'll find that with 10k rows and more, each having a small subview, things do tend to slow down. You'll need to experiment and compare performance for your cases - it's hard to predict what works best for you. From what you describe, I can see two candidates worth trying: - create a hash, first prop is word, 2nd prop is subview of ints (and consider blocking, i.e. a hash on top of a blocked view) - store "flat", i.e. , ... - this would be wasteful since "word" is stored n times, but with an extra view containing only words, you can replace it with an index: words[text:S],index[wpos:I,docnum:I] (In this case, "words" would be a candidate for hashing) It'll depend a lot on expected sizes, but I wouldn't immediately worry if that last approach had a million rows in the "index" view (you can switch to a blocking structure later without affecting much of the rest of your code). It really depends a lot on access patterns, amount/type of change, how much performance you need to achieve, how much effort you are willing to spend on timing various approaches, and how much logic you are willing to add. For me, the ground rule is still: go for brute force until it becomes a bottleneck. -jcw From brunobl@ifrance.com Sun Apr 28 16:17:34 2002 Received: from pop.univ-lyon1.fr (postfix@pop.univ-lyon1.fr [134.214.100.7]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SLHXa19879 for ; Sun, 28 Apr 2002 16:17:33 -0500 Received: by pop.univ-lyon1.fr (Postfix, from userid 3081) id 0536D24264; Sun, 28 Apr 2002 23:17:31 +0200 (CEST) Received: from [134.214.164.67] (I435.resI.insa-lyon.fr [134.214.164.67]) by pop.univ-lyon1.fr (Postfix) with ESMTP id 30A57C1F8 for ; Sun, 28 Apr 2002 23:17:31 +0200 (CEST) User-Agent: Microsoft-Entourage/10.0.0.1309 Date: Sun, 28 Apr 2002 23:17:29 +0200 Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 From: Bruno Blondeau To: Message-ID: In-Reply-To: <200204281701.g3SH15a13834@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: on 28/04/02 19:01, metakit-request@equi4.com - metakit-request@equi4.com wrote: > Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 > Date: Sun, 28 Apr 2002 13:54:37 +0200 > > Bruno Blondeau wrote: > >> I recently updated my application, and also included a newer version of >> Metakit (2.4.4 instead of 2.4.1). >> >> I thought the data file would be converted automatically or the file format >> remain the same. >> >> However, it seems not to be the case. >> >> My application is buggy now. After spending several hours, I eventually >> reverted to 2.4.1, and everything works fine again... >> >> Do I have to do something to ask Metakit to convert my data file from the >> old format to the new one? > > No, never. Either it has the same format (as it does from 2.3 to now), > or it does conversion fully automatically (as it does for 2.01 and older). > > Maybe something else is the problem. I just checked in a bug fix, so I > would definitely advise to either try the CVS latest source, or wait > until there is a 2.4.5 release - which is forthcoming. > > Now that the 2.4.4 release is being used far far more intensively and by > more people (well, better late than never), so it looks like we are > finally banging hard enough on MK to make all remaining bugs show up. > > I'd appreciate it if you could give me some more information on what sort > of problems you were seeing - especially with the latest source code. The previous version of my application was using Metakit 2.4.1 The new version is using Metakit 2.4.4, and also adds some columns to the database. Maybe the problem is here. It also seems there is no problem using the new version using the new version if the database is created with this version (only people who updated to the new version have problem). The new version with the database change but still Metakit 2.4.1 doesn't have any problem reading the old files (but has problems to read database that were converted with Metakit 2.4.4, so I have a user base that is stuck with a corrupted database... ) Also, there is only a problem with one column. A binary one. Which already existed in the previous version. I'm using objective-C++, but it shouldn't matter. The problem occures when I'm doing this. c4_Bytes data = NOTE_CONTENT(NOTE_VIEW[index]); The data content seems to be wrong when I'm reading it (but the length is right) Reading the first bytes, I get something like: 616c0a5c 63663020 4c617572 656e7420 50696e65 When it should look like (well, the first half are always the same given the kind of data structure that is used): 72746664 00000000 03000000 02000000 01000000 Also, I'm working using MacOS X, so maybe there is an endianness issue? I'm not used to download files using a CVS, but I will try to do my best to see if it works. -- Bruno Blondeau http://www.brunoblondeau.com From brunobl@ifrance.com Sun Apr 28 17:10:41 2002 Received: from pop.univ-lyon1.fr (postfix@pop.univ-lyon1.fr [134.214.100.7]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SMAfa21122 for ; Sun, 28 Apr 2002 17:10:41 -0500 Received: by pop.univ-lyon1.fr (Postfix, from userid 3081) id 5B6F224267; Mon, 29 Apr 2002 00:10:40 +0200 (CEST) Received: from [134.214.164.67] (I435.resI.insa-lyon.fr [134.214.164.67]) by pop.univ-lyon1.fr (Postfix) with ESMTP id 98FAAC059 for ; Mon, 29 Apr 2002 00:10:39 +0200 (CEST) User-Agent: Microsoft-Entourage/10.0.0.1309 Date: Mon, 29 Apr 2002 00:10:37 +0200 Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 From: Bruno Blondeau To: Message-ID: In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I tried the latest version that is in the CVS, but results aren't better. I randomly get complete garbage from the database. For example, below, there wasn't any problem for rows 150 to 156, but garbage for the other ones. I'm really beginning to think there either was a problem in the previous version, or a new problem in the current version... (if you have any idea, please cc me, as I only receive a digest of this list, and I'm getting flooded by users with problems...) Thanks in advance, Bruno Blondeau 2002-04-28 23:49:13.746 iOrganizeX[2435] 150 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 0f2b0000 01000000 2002-04-28 23:49:14.832 iOrganizeX[2435] 151 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 0f2b0000 01000000 2002-04-28 23:49:20.489 iOrganizeX[2435] 152 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 dc030000 01000000 2002-04-28 23:49:21.011 iOrganizeX[2435] 153 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 15010000 01000000 2002-04-28 23:49:21.334 iOrganizeX[2435] 154 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 4b010000 01000000 2002-04-28 23:49:21.656 iOrganizeX[2435] 155 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 1d010000 01000000 2002-04-28 23:49:21.977 iOrganizeX[2435] 156 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 30010000 01000000 2002-04-28 23:49:22.318 iOrganizeX[2435] 157 : desc = <2f202020 28667261 6e5c2738 64616973 295c0a66 74703a2f 2f667470 2e726561 6c736f66 74776172 652e636f 2002-04-28 23:49:22.650 iOrganizeX[2435] 158 : desc = <35355c67 7265656e 3235355c 626c7565 3235353b 7d0a5c70 6172645c 74783536 305c7478 31313230 5c747831 2002-04-28 23:49:23.005 iOrganizeX[2435] 159 : desc = <6c0a5c63 66302049 20737563 63656564 65642077 69746820 73657474 696e6720 74686520 44726976 6572206e 2002-04-28 23:49:23.312 iOrganizeX[2435] 160 : desc = <305c7478 32323430 5c747832 3830305c 74783333 36305c74 78333932 305c7478 34343830 5c747835 3034305c 2002-04-28 23:49:23.614 iOrganizeX[2435] 161 : desc = <73752e69 6e666f2e 6170706c 652e636f 6d2f6865 6c706e65 78742f68 656c706e 6578742f 68656c70 68656c70 2002-04-28 23:49:23.917 iOrganizeX[2435] 162 : desc = <2e636974 65676c6f 62652e63 6f6d5c0a 68747470 3a2f2f77 77772e73 6b696470 65726665 63742e63 6f6d2f65 on 28/04/02 23:17, Bruno Blondeau - brunobl@ifrance.com wrote: > on 28/04/02 19:01, metakit-request@equi4.com - metakit-request@equi4.com > wrote: > >> Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 >> Date: Sun, 28 Apr 2002 13:54:37 +0200 >> >> Bruno Blondeau wrote: >> >>> I recently updated my application, and also included a newer version of >>> Metakit (2.4.4 instead of 2.4.1). >>> >>> I thought the data file would be converted automatically or the file format >>> remain the same. >>> >>> However, it seems not to be the case. >>> >>> My application is buggy now. After spending several hours, I eventually >>> reverted to 2.4.1, and everything works fine again... >>> >>> Do I have to do something to ask Metakit to convert my data file from the >>> old format to the new one? >> >> No, never. Either it has the same format (as it does from 2.3 to now), >> or it does conversion fully automatically (as it does for 2.01 and older). >> >> Maybe something else is the problem. I just checked in a bug fix, so I >> would definitely advise to either try the CVS latest source, or wait >> until there is a 2.4.5 release - which is forthcoming. >> >> Now that the 2.4.4 release is being used far far more intensively and by >> more people (well, better late than never), so it looks like we are >> finally banging hard enough on MK to make all remaining bugs show up. >> >> I'd appreciate it if you could give me some more information on what sort >> of problems you were seeing - especially with the latest source code. > > The previous version of my application was using Metakit 2.4.1 > > The new version is using Metakit 2.4.4, and also adds some columns to the > database. Maybe the problem is here. It also seems there is no problem using > the new version using the new version if the database is created with this > version (only people who updated to the new version have problem). > > The new version with the database change but still Metakit 2.4.1 doesn't have > any problem reading the old files (but has problems to read database that were > converted with Metakit 2.4.4, so I have a user base that is stuck with a > corrupted database... ) > > Also, there is only a problem with one column. A binary one. Which already > existed in the previous version. > > > I'm using objective-C++, but it shouldn't matter. The problem occures when I'm > doing this. > > c4_Bytes data = NOTE_CONTENT(NOTE_VIEW[index]); > > The data content seems to be wrong when I'm reading it (but the length is > right) > > > Reading the first bytes, I get something like: > > 616c0a5c 63663020 4c617572 656e7420 50696e65 > > When it should look like (well, the first half are always the same given the > kind of data structure that is used): > 72746664 00000000 03000000 02000000 01000000 > > > Also, I'm working using MacOS X, so maybe there is an endianness issue? > > > I'm not used to download files using a CVS, but I will try to do my best to > see if it works. -- Bruno Blondeau http://www.brunoblondeau.com From keks@instock.ru Mon Apr 29 06:13:55 2002 Received: from mx.instock.ru ([194.226.223.175]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3TBDsa08523 for ; Mon, 29 Apr 2002 06:13:54 -0500 Received: (qmail 4956 invoked by uid 404); 29 Apr 2002 11:14:57 -0000 Received: from keks@instock.ru by ns.instock.ru by uid 0 with qmail-scanner-1.10 (drweb. Clear:0. Processed in 0.061129 secs); 29 Apr 2002 11:14:57 -0000 Received: from c001-pautina-swamp.jacksattelecom.butovo.biz (HELO necropol-pc) (keks@instock.ru@62.117.74.178) by 0 with SMTP; 29 Apr 2002 11:14:57 -0000 Date: Mon, 29 Apr 2002 15:15:13 +0400 From: "Yuriy S. Polyakov" X-Mailer: The Bat! (v1.52f) Personal Reply-To: "Yuriy S. Polyakov" X-Priority: 3 (Normal) Message-ID: <140213179035.20020429151513@instock.ru> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] _indexed search Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello metakit, Whether allows MetaKit to make indexed search in value of a field? -- Yuris S. Polyakov From andreas.olsson@ctakt.com Mon Apr 29 06:35:48 2002 Received: from linus.ctakt.com (linus.ctakt.com [62.119.45.234]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3TBZla09469 for ; Mon, 29 Apr 2002 06:35:47 -0500 Received: from flint ([192.168.2.115]) by linus.ctakt.com (8.11.6/8.11.2) with SMTP id g3TBZkp30410 for ; Mon, 29 Apr 2002 13:35:46 +0200 From: "Andreas Olsson" To: Date: Mon, 29 Apr 2002 13:35:45 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] A bug in Locate? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I get different results when using Find and Locate. Find works fine but Locate returns 2 when there only is one matching row in the view. My call to Locate looks like this: int count = table.Locate( colName [pKey] , &pos ); I'd like to use Locate because I need both the count and pos value. /Andreas Olsson From jcw@equi4.com Mon Apr 29 06:54:53 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3TBspa10373 for ; Mon, 29 Apr 2002 06:54:52 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] A bug in Locate? Date: Mon, 29 Apr 2002 13:54:13 +0200 Message-Id: <20020429115413.20066@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas Olsson wrote: >I get different results when using Find and Locate. Find works fine but >Locate returns 2 when there only is one matching row in the view. Please verify that the view you are using locate on is indeed sorted. A quick test from 0..N comparing values will tell you if things are indeed sorted. -jcw From lvirden@cas.org Tue Apr 30 10:04:34 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3UF4Ya06110 for ; Tue, 30 Apr 2002 10:04:34 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g3UF4RS16885 for ; Tue, 30 Apr 2002 11:04:27 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g3UF4Oq22774; Tue, 30 Apr 2002 11:04:24 -0400 (EDT) Date: Tue, 30 Apr 2002 11:04:24 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020430110424.AAB22765@cas.org> To: Subject: [Metakit] Did anyone ever write Metakit bindings for perl? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A friend is looking for a simple database with which he could manage a personal comic book inventory database. His experience is with Oracle - but apparently there isn't a free version of Oracle server for Windows XP Home. I thought of Metakit, but then discovered to my horror that I could not find a Metakit binding for Perl, the language he codes in. Did anyone ever get around to writing a Tie module or something for Perl and Metakit? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Tue Apr 30 11:30:35 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3UGUYa10324; Tue, 30 Apr 2002 11:30:34 -0500 From: Jean-Claude Wippler To: announce , metakit Date: Tue, 30 Apr 2002 18:28:03 +0200 Message-Id: <20020430162803.2864@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] TclKit 8.4-35 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is to announce a new release of TclKit, the self-contained runtime with Tcl/Tk 8.4a4+, MetaKit 2.4.5, and incrTcl 3.2+. This release combines the latest revisions, "puts $tcl_platform(vfs)" will print the string "200205". There are now fourteen TclKit executables at: Platforms include Windows, Linux, Solaris, FreeBSD, MacOSX, S390. Last month's release files are still available at: Things I didn't have time for in this round were: - cleanup load shared lib (no idea why the core doesn't do it) - slight weirdness with encodings (though only on my own build) - custom-tclkit builds don't seem to always work properly These builds no longer have the "md5", "pink", "hexdump", "geteuid", and "nop" commands, nor do they try to fake Memchan or Trf (no package provides anymore). The md5 command can be obtained from critlib or trf. There is a "zlib" command built-in to do compression and crc calcs, and a "rechan" to build a memchan version if there is none. All of this is in preparation of making it possible to use/unpack scripted documents, for those who do not want to use the tclkit binaries nor build their own with genkit. For more details and download info, see the homepage: There's also a new ViewCVS interface to browse CVS from the web. Happy programming, -jcw From jcw@equi4.com Tue Apr 30 11:30:37 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3UGUaa10331; Tue, 30 Apr 2002 11:30:36 -0500 From: Jean-Claude Wippler To: announce , metakit Date: Tue, 30 Apr 2002 18:29:03 +0200 Message-Id: <20020430162903.16805@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.5-35 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is to announce a new release of MetaKit, the database that fits in the palm of your hand. This release fixes a number of bugs - no feature / file format changes: It has been used to build on over a dozen OS / platform combinations. There may be unresolved issues with corrupted string/byte properties (two reports, one macosx, the other win32 - perhaps the same bug), but I have *not* been able to reproduce it despite heavy banging and making 100,000's of random changes with verification of results (see examples/ random.tcl). For more details and download info, see the homepage: There's also a new ViewCVS interface to browse CVS from the web. Happy programming, -jcw From lvirden@cas.org Wed May 1 08:44:45 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g41Diia01953 for ; Wed, 1 May 2002 08:44:45 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g41DidS19018 for ; Wed, 1 May 2002 09:44:39 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g41Dibb00944; Wed, 1 May 2002 09:44:37 -0400 (EDT) Date: Wed, 1 May 2002 09:44:37 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020501094437.AAB898@cas.org> Subject: Compatibility issues [Was Re: [Metakit] TclKit 8.4-35 In-Reply-To: <20020430162803.2864@triqs.com> of Tue, 30 Apr 2002 18:28:03 +0200 To: metakit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I had a question that immediately came to mind when I read this: > These builds no longer have the "md5", "pink", "hexdump", "geteuid", and > "nop" commands, nor do they try to fake Memchan or Trf (no package > provides anymore). The md5 command can be obtained from critlib or trf. What are some of the strategies that scripted document writers on this list are using to assist in migrating data to handle this transition? One may in fact have no control over what tclkit gets used with their scripted documents, so one has to have some sort of strategy for moving data from the old document into a new one ... -- Support Internet Radio Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Wed May 1 10:02:31 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g41F2Ua05100 for ; Wed, 1 May 2002 10:02:31 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: Compatibility issues [Was Re: [Metakit] TclKit 8.4-35 Date: Wed, 1 May 2002 17:01:39 +0200 Message-Id: <20020501150139.20023@triqs.com> In-Reply-To: <20020501094437.AAB898@cas.org> References: <20020501094437.AAB898@cas.org> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >I had a question that immediately came to mind when I read this: > >> These builds no longer have the "md5", "pink", "hexdump", "geteuid", and >> "nop" commands, nor do they try to fake Memchan or Trf (no package >> provides anymore). The md5 command can be obtained from critlib or trf. > > >What are some of the strategies that scripted document writers on this list >are using to assist in migrating data to handle this transition? One >may in fact have no control over what tclkit gets used with their scripted >documents, so one has to have some sort of strategy for moving data from >the old document into a new one ... Good point. it is possible that dropping things like md5 will break some scripted documents which worked before. One strategy would be to create a separate scripted doc with the things that have been thrown out - and given that it's compiled code, it will need to contain builds for several platforms. For now, I was under the impression that very few people create scripted docs, and therefore the issue would not come up at all. By throwing out all "nice" add-ons which were in tclkit, keeping only those which are essential for its own operation (zlib, tclvfs, rechan), I'm hoping to establish a minimal common baseline. This is a matter of taste, partly, though - since the choice to always include Tk, Metakit, and IncrTcl obviously extends the core system substantially beyond "just the bare minimum to run a Tcl script". One idea to maintain compatibility with older tclkit's is to look for files next to the executable, say "tclkit-*.tkl" and mount them all through VFS, extending the Tcl search path to automatically find and use all packages inside them. That means one could simple place a special file next to a new tclkit, and older scripted docs would still be able to use older extensions and run without modification. This will become essential once Steve Landers' scripted document archive at http://mini.net/sdarchive/ is announced in public. -jcw From jtate@mi-corporation.com Wed May 1 13:26:14 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g41IQDa14235 for ; Wed, 1 May 2002 13:26:14 -0500 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g41IPtP18789 for ; Wed, 1 May 2002 14:25:55 -0400 From: "Joseph Tate" To: "MetaKit" Date: Wed, 1 May 2002 14:26:02 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Problem removing records Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have a problem with deleting records from a view. Ok, I've got a c4_Storage object that I add and delete records in. If I add a record and dump the database in a manner similar to the DumpFile method in the test app, the record shows up. When I delete the record and then dump the database, the record disappears, but when I go to add another new record, the old (deleted) record appears again. I am committing after each operation, but I can't figure out why the record shows up again. Is there something that I should look for? Perhaps a common Gotcha that I can make sure to avoid, etc. Does this sound familiar to anyone? Joseph From smalllinux@netscape.net Wed May 1 14:39:52 2002 Received: from imo-d09.mx.aol.com (imo-d09.mx.aol.com [205.188.157.41]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g41Jdoa18408 for ; Wed, 1 May 2002 14:39:51 -0500 Received: from smalllinux@netscape.net by imo-d09.mx.aol.com (mail_out_v32.5.) id d.3.3b57048 (16225) for ; Wed, 1 May 2002 15:39:40 -0400 (EDT) Received: from netscape.com (mow-d07.webmail.aol.com [205.188.138.71]) by air-in02.mx.aol.com (v84.16) with ESMTP id MAILININ21-0501153940; Wed, 01 May 2002 15:39:40 2000 Date: Wed, 01 May 2002 15:41:05 -0400 From: smalllinux@netscape.net To: metakit@equi4.com Message-ID: <6E743AEC.4B3AFB9D.1080C5D1@netscape.net> X-Mailer: Atlas Mailer 2.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] Scripted Documents on CD Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi JCW, I am selling a CD called Expert Tcl that boots into a live Linux system on any x386 machine with at least 16 meg of ram. The system should be able to startx to use X11 and then a demo of tcl/tk is available with over 100 tk apps and scripts. Tclkit is used on the CD to run the apps. I have been able to modify almost all pure tk applications to run on the CD, except I have not been able to get scripted apps from sdarchive to run. I assume it is a problem of temp unzipping directory? Do you have any suggestions that might help. Expert Tcl with no swap partition only has about 5 meg of free ram available for tmp space. Steven Gibson http://www.superant.com/ __________________________________________________________________ Your favorite stores, helpful shopping tools and great gift ideas. Experience the convenience of buying online with Shop@Netscape! http://shopnow.netscape.com/ Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From jcw@equi4.com Wed May 1 15:17:07 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g41KH6a20407 for ; Wed, 1 May 2002 15:17:06 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Scripted Documents on CD Date: Wed, 1 May 2002 22:15:13 +0200 Message-Id: <20020501201513.1210@triqs.com> In-Reply-To: <6E743AEC.4B3AFB9D.1080C5D1@netscape.net> References: <6E743AEC.4B3AFB9D.1080C5D1@netscape.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: smalllinux@netscape.net wrote: >I am selling a CD called Expert Tcl that boots into a live >Linux system on any x386 machine with at least 16 meg of ram. Yeah, saw that recently... most intriguing :) >The system should be able to startx to use X11 and then a demo >of tcl/tk is available with over 100 tk apps and scripts. Tclkit >is used on the CD to run the apps. > >I have been able to modify almost all pure tk applications to run >on the CD, except I have not been able to get scripted apps from >sdarchive to run. I assume it is a problem of temp unzipping directory? >Do you have any suggestions that might help. Expert Tcl with no swap >partition only has about 5 meg of free ram available for tmp space. Ah, yes. Scripted docs start of as scripts, and then re-open themselves as a r/w MetaKit database to mount as VFS. You'll need to get the latest sdx from /pub/tk/examples/, unpack each scripted doc, and then repack each by adding "-readonly" as final arg. That will re-create the scripted doc in a way which works in read-only mode (it's a change to the script header that starts the whole process of re-opening). There's no use of /tmp space, unless shared lib extensions are used. -jcw PS. Whee, ever considered making your system work on a dozen platforms? All you'd need to do nowadays, is add the latest builds to your CD... Hm, or maybe that's better reserved for a different CD distribution. From andreas.olsson@ctakt.com Thu May 2 09:10:32 2002 Received: from linus.ctakt.com (linus.ctakt.com [62.119.45.234]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g42EAVa03392 for ; Thu, 2 May 2002 09:10:31 -0500 Received: from flint ([192.168.2.115]) by linus.ctakt.com (8.11.6/8.11.2) with SMTP id g42EATp17724 for ; Thu, 2 May 2002 16:10:30 +0200 From: "Andreas Olsson" To: Date: Thu, 2 May 2002 16:10:27 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Using streams Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello! I have a small program that uses MetaKit. I now need to change it a bit so that it can use streams. I replaced c4_Storage *store = new c4_Storage("filename") with c4_Storage *store = new c4_Storage(); store->LoadFrom(mystream); It works fine to load the stream, but all changes are lost when I destroy the storage object (even if AutoCommit is true). Doesn't commit work on streams? --------- Andreas Olsson From jcw@equi4.com Sun May 5 15:43:42 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g45Khfa07237 for ; Sun, 5 May 2002 15:43:41 -0500 From: Jean-Claude Wippler To: metakit Date: Sun, 5 May 2002 22:42:34 +0200 Message-Id: <20020505204235.17792@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Please test Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There was a serious (datafile corrupting) bug in the latest release(s) of MetaKit, due to an incorrect over-agressive optimization. See the change log for details. The bug was introduced in either the 2.4.2 or the 2.4.3 release. There are new regression tests (s44 and s45) to bring out this issue. If you have had problems with corrupted string/bytes properties, then please try the latest version now in CVS. Unless someone reports problems with this change, I'll bring out a new update in the coming 2..3 days (this will become version 2.4.6). My apologies for not having solved this much sooner, but until today I had not been able to force the problem and figure it out. -jcw From jcw@equi4.com Mon May 6 17:30:24 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g46MUMa26292; Mon, 6 May 2002 17:30:22 -0500 From: Jean-Claude Wippler To: announce , metakit Date: Tue, 7 May 2002 00:30:00 +0200 Message-Id: <20020506223000.14008@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.6-36 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A new release of MetaKit is now available, due to an important bug fix. The problem, which may have been in releases as far back as 2.4.2, may have caused datafile corruption in some very specific scenario's: - string/bytes property items over 10,000 bytes - or items of at least 100b, if there are over 1,000..10,000 rows - at least two real commits (meaning something must have changed) - no change to the string/bytes property at all during these commits To phrase it differently: some items must have been stored as "memos", and at least two commits must have been done which only affect *other* columns. Source code and binaries for Linux, MacOS, and Win32 are now available, see the homepage at http://www.equi4.com/metakit/. This release supercedes all previous releases of MetaKit. Due to the impact of this specific bug fix, but also because release 2.4.6 is now considered to be extremely stable, I urge you to update to this latest codebase when possible. Older releases and older file formats will no longer be supported from now on. I do not intend to make any further "enhancements" in the 2.4.x release cycle, only surgical bug fixes - if deemed essential and 100% safe. -jcw PS. Note on datafile formats: All 2.4.x releases convert older MK file formats (2.01 and before) on the fly. Future major releases (2.5 and beyond) will most likely be able to stick with the current format, but will drop the conversion capability w.r.t. older formats. If you need to deal with very long-term file format compatibility, you will either have to stick to 2.4.x forever, or keep a 2.4.x-based conversion utility around once you move on (this is not going to be soon, 2.5 has not even been scheduled yet). If you have never used MK releases 2.01 (March 2000) or older, then there are no file format compatibility issues at all. The current format has a lot of leeway for future extensions. From fleck@isoc.de Mon May 6 19:03:07 2002 Received: from postfix.informatik.uni-bonn.de (postfix.informatik.uni-bonn.de [131.220.131.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g47036a00397 for ; Mon, 6 May 2002 19:03:06 -0500 Received: from huxley.graffe.de (p5082752E.dip.t-dialin.net [80.130.117.46]) by postfix.informatik.uni-bonn.de (Postfix) with ESMTP id E710670C56; Tue, 7 May 2002 02:03:04 +0200 (MEST) (envelope-from fleck@isoc.de) (envelope-to metakit@equi4.com) (2) (internal use: ta=1, tu=1, te=1, am=L, au=fleck) Received: by huxley.graffe.de (Postfix on SuSE Linux 7.2 (i386), from userid 500) id CDB666848B; Tue, 7 May 2002 02:07:14 +0200 (CEST) Date: Tue, 7 May 2002 02:07:14 +0200 From: Markus Fleck-Graffe To: metakit@equi4.com Message-ID: <20020507020714.A24214@postfix> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.16i Subject: [Metakit] Porting MK4 to SQLite VM? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude! In late 2001, the SQLite SQL92 engine changed to Public Domain licensing. SQLite is currently based on Berkeley DB-style storage, but (due to its abstracted SQL virtual machine) is supposed to be portable to other architectures. http://www.hwaci.com/sw/sqlite/ One major feature of SQLite is that it is extremely light-weight (unlike Metakit's current scripting-based SQL engines), which would fit very well with Metakit's own design goals. The SQLite virtual SQL machine appears to be moderately well-documented (albeit not 100% up-to-date). I'm quite sure that SQL capability is a popular feature request for Metakit (even though, of course, Metakit's data model is much more powerful than plain SQL92). Is anybody, by chance, already working on SQLite <-> Metakit bindings? If not, could you take a look at SQLite and assess how difficult (or easy) it would be to create basic Metakit bindings for SQLite? Yours, Markus. From ksong@accesslan.com Mon May 6 23:59:25 2002 Received: from mug.accesslan.com (mail.accesslan.com [216.91.53.165]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g474xOa15224; Mon, 6 May 2002 23:59:25 -0500 Received: from KSONG [172.16.140.127] by mug.accesslan.com (SMTPD32-6.06) id A0121538004C; Mon, 06 May 2002 22:03:14 -0700 Reply-To: From: "Keesang Song" To: Cc: Date: Mon, 6 May 2002 22:03:12 -0700 Message-ID: <004e01c1f584$7e1220d0$7f8c10ac@accesslan.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_004F_01C1F549.D1B348D0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2462.0000 Subject: [Metakit] [Q] Is there any old pre-compiled binary for Metakit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_004F_01C1F549.D1B348D0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit Hi, I found Metakit for Tcl through web searching. I downloaded latest one(2.4.6) from downloading place. Since I am using tcl 8.0(need to stick to it), when I tried to load Mk4tcl.dll, it caused error(required stubs support). Is there any old pre-compiled binary for Metakit? So that I can try Metakit with existing tcl 8.0 script. It(metakit) seems very easy to use. I want to try to integrate it with some flat data. Hopefully, I could get any old binary for this. I’ll appreciate any help… Thank you. Keesang 2040 Fortune Dr. Bldg. 102 San Jose, CA 95131 Bus. # 408.325.1720 Fax # 408.435.1381 Cell. # 408.930.5281 ------=_NextPart_000_004F_01C1F549.D1B348D0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi,

 

I found Metakit for Tcl through web = searching.

I downloaded latest one(2.4.6) from downloading = place.

Since I am using tcl 8.0(need to stick to it), when I tried to = load Mk4tcl.dll, it caused error(required stubs = support).

 

Is there any old pre-compiled binary for Metakit? So that I can = try Metakit with existing tcl 8.0 = script.

It(metakit) seems very easy to use. I want to try to integrate it = with some flat data.

 

Hopefully, I could get any old binary for = this.

 

I’ll appreciate any = help…

 

Thank you.

 

Keesang

 

2040 Fortune Dr. Bldg. 102

San = Jose, CA 95131

Bus. # 408.325.1720

Fax # 408.435.1381

Cell. # 408.930.5281

 <= /p>

------=_NextPart_000_004F_01C1F549.D1B348D0-- From ksong@accesslan.com Tue May 7 12:39:44 2002 Received: from mug.accesslan.com (mail.accesslan.com [216.91.53.165]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g47Hdia30553 for ; Tue, 7 May 2002 12:39:44 -0500 Received: from KSONG [172.16.140.127] by mug.accesslan.com (SMTPD32-6.06) id A247ABE0114; Tue, 07 May 2002 10:43:35 -0700 Reply-To: From: "Keesang Song" To: Date: Tue, 7 May 2002 10:43:34 -0700 Message-ID: <005d01c1f5ee$b62e1720$7f8c10ac@accesslan.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_005E_01C1F5B4.09CF3F20" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2462.0000 Subject: [Metakit] FW: [Q] Is there any old pre-compiled binary for Metakit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_005E_01C1F5B4.09CF3F20 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit Subject: [Q] Is there any old pre-compiled binary for Metakit? Hi, I found Metakit for Tcl through web searching. I downloaded latest one(2.4.6) from downloading place. Since I am using tcl 8.0(need to stick to it), when I tried to load Mk4tcl.dll, it caused error(required stubs support). Is there any old pre-compiled binary for Metakit? So that I can try Metakit with existing tcl 8.0 script. It(metakit) seems very easy to use. I want to try to integrate it with some flat data. Hopefully, I could get any old binary for this. I’ll appreciate any help… Thank you. Keesang 2040 Fortune Dr. Bldg. 102 San Jose, CA 95131 Bus. # 408.325.1720 Fax # 408.435.1381 Cell. # 408.930.5281 ------=_NextPart_000_005E_01C1F5B4.09CF3F20 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Subject: [Q] Is there any old pre-compiled binary for = Metakit?

 

Hi,

 

I found Metakit for Tcl through web = searching.

I downloaded latest one(2.4.6) from downloading = place.

Since I am using tcl 8.0(need to stick to it), when I tried to = load Mk4tcl.dll, it caused error(required stubs = support).

 

Is there any old pre-compiled binary for Metakit? So that I can = try Metakit with existing tcl 8.0 = script.

It(metakit) seems very easy to use. I want to try to integrate it = with some flat data.

 

Hopefully, I could get any old binary for = this.

 

I’ll appreciate any = help…

 

Thank you.

 

Keesang

 

2040 Fortune Dr. Bldg. 102

San = Jose, CA 95131

Bus. # 408.325.1720

Fax # 408.435.1381

Cell. # 408.930.5281

 <= /p>

------=_NextPart_000_005E_01C1F5B4.09CF3F20-- From jcw@equi4.com Tue May 7 13:42:53 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g47Igba03818 for ; Tue, 7 May 2002 13:42:48 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Is there any old pre-compiled binary for Metakit? Date: Tue, 7 May 2002 20:42:31 +0200 Message-Id: <20020507184231.27757@triqs.com> In-Reply-To: <005d01c1f5ee$b62e1720$7f8c10ac@accesslan.com> References: <005d01c1f5ee$b62e1720$7f8c10ac@accesslan.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Keesang Song wrote: >Is there any old pre-compiled binary for Metakit? So that I can try >Metakit with existing tcl 8.0 script. Ok, well, I don't want to start a trend, but here you go: Built against Tcl 8.0.5 - note that "mk::select -globnc" does not work (it is case sensitive because 8.0.5 has no support for it), and I had to make a few other minor source tweaks to get it to compile. Untested, but it ought to work... -jcw From jcw@equi4.com Tue May 7 14:19:58 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g47JJva06834 for ; Tue, 7 May 2002 14:19:57 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Porting MK4 to SQLite VM? Date: Tue, 7 May 2002 21:19:48 +0200 Message-Id: <20020507191948.15176@triqs.com> In-Reply-To: <20020507020714.A24214@postfix> References: <20020507020714.A24214@postfix> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Markus Fleck-Graffe wrote: [...] > http://www.hwaci.com/sw/sqlite/ > >One major feature of SQLite is that it is extremely light-weight (unlike >Metakit's current scripting-based SQL engines), which would fit very well >with Metakit's own design goals. The SQLite virtual SQL machine appears to >be moderately well-documented (albeit not 100% up-to-date). > >I'm quite sure that SQL capability is a popular feature request for Metakit >(even though, of course, Metakit's data model is much more powerful than >plain SQL92). I'm not sure. I would expect that there is a need for an embedded SQL engine, like sqlite, but I don't really get a lot of emails asking for SQL. One reason might be that MK has a reasonably decent set of relational algebra operators (which is why MkSQL can be built on top with relatively little scripting code). >Is anybody, by chance, already working on SQLite <-> Metakit bindings? Not that I know of - but I'd be delighted to be proven wrong . >If not, could you take a look at SQLite and assess how difficult (or easy) >it would be to create basic Metakit bindings for SQLite? I'm not sure this is easy to do. One can probably treat MK as an alternative to *dbm and bdb, but FYI that is not at all the way Gordon McMillan's MkSQL works. And it's not certain that one would get even decent performance that way (MK really soars when its column-wise design is taken advantage of). The truly exciting approach of MkSQL is that it takes SQL as a "recipe" to construct a tree structure of derived views, using MK's view operators. That means one can set up an SQL statement, and have it return a view without even accessing the disk. Only actual row (record) / property (attribute) accesses will actually trigger work to be done. This may seem inconsequential, but I suspect that there is quite a bit of performance lurking in this approach - as well as the option to optimize lots of things and take maximum advantage of subviews. Not to mention the fact that hashed and blocked viewers will continue to work *across* such an SQL design. The RaSQL project I worked on a while back went a similar route. Clearly, getting a nicely complete and conforming (whatever that is) implementation is a big project. But having said that - it remains to be seen whether one even needs all that machinery. I used to be an SQL affecionado in a previous life, but nowadays the simple logic of direct view operations (with all the tricks one can do with remapwith and other operators) suits me fine - and it's lighter even than sqlite. Don't get me wrong: I strongly support the relational data model. And if someone wants to create a binding for sqlite I'll support such a project - but until someone enlightens me on the virtue of switching from a scripting language VM to a special purpose SQL VM, I'm afraid I can only offer a blank stare . Is the fact that MkSQL was coded in Python a limitation? Would there be any value in taming/modularizing the Python system further, so it can be more easily embedded and deployed? Or even the Lua core used in RaSQL, for that matter? And most importantly perhaps: where would a "sqlite/mk" fit in, compared to all the other SQL solutions there are today? -jcw From jyl@best.com Wed May 8 00:38:40 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g485cda12763 for ; Wed, 8 May 2002 00:38:40 -0500 Received: (qmail 43649 invoked by uid 19667); 8 May 2002 05:38:39 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 8 May 2002 05:38:39 -0000 Message-ID: <3CD8BCF1.8090201@best.com> Date: Tue, 07 May 2002 22:51:45 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] FW: [Q] Is there any old pre-compiled binary for Metakit? References: <005d01c1f5ee$b62e1720$7f8c10ac@accesslan.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: [JCW: Could you please post this to Metakit-announce? Thanks, --JYL] WHAT ==== This is to announce the 1.0a6 release of e4Graph, the sixth and final Alpha release. WHERE TO GET: ============= File releases: http://sourceforge.net/projects/e4graph/ Documentation: http://e4graph.sourceforge.net/ Change log: http://e4graph.sourceforge.net/changes.txt WHAT IS IT: =========== e4Graph is a C++ and Tcl library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. CHANGES: ======== For all changes, see: http://e4graph.sourceforge.net/changes.txt DEPENDENCIES: ============= This release requires Metakit 2.4.6, Tcl 8.3 and optionally Expat 1.95.2. From FabienB@gilian.com Wed May 8 07:42:50 2002 Received: from gilianmail.gilian.com (gw-guilian.ser.netvision.net.il [199.203.171.166]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g48Cgma30338 for ; Wed, 8 May 2002 07:42:48 -0500 Received: by gilianmail with Internet Mail Service (5.5.2650.21) id <2Q0N0YC8>; Wed, 8 May 2002 15:42:55 +0300 Message-ID: <1797E7C8F378D411A1CE00D0B77E59CE572FA2@gilianmail> From: Fabien Benichou To: "'metakit@equi4.com'" Date: Wed, 8 May 2002 15:42:50 +0300 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="windows-1255" Subject: [Metakit] Bug in mk4dll.h Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I found that In Metakit version 2.4.5, the two following lines are missing in the file mk4dll.h: d4_DLLSPEC(bool) operator== (const c4_Reference& a_, const c4_Reference& b_); d4_DLLSPEC(bool) operator!= (const c4_Reference& a_, const c4_Reference& b_); Fabien. Fabien Benichou Software Engineer Gilian Technologies "The last line of defense against hacker sabotage!" www.gilian.com Tel: 972-9-956-0036 x233 FAX: 972-9-956-5668 From jyl@best.com Wed May 8 08:27:30 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g48DRUa32446 for ; Wed, 8 May 2002 08:27:30 -0500 Received: (qmail 5582 invoked by uid 19667); 8 May 2002 13:27:29 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 8 May 2002 13:27:29 -0000 Message-ID: <3CD92AD3.205@best.com> Date: Wed, 08 May 2002 06:40:35 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] [ANNOUNCE] e4Graph 1.0a6 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: WHAT ==== This is to announce the 1.0a6 release of e4Graph, the sixth and final Alpha release. WHERE TO GET: ============= File releases: http://sourceforge.net/projects/e4graph/ Documentation: http://e4graph.sourceforge.net/ Change log: http://e4graph.sourceforge.net/changes.txt WHAT IS IT: =========== e4Graph is a C++ and Tcl library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. CHANGES: ======== For all changes, see: http://e4graph.sourceforge.net/changes.txt DEPENDENCIES: ============= This release requires Metakit 2.4.6, Tcl 8.3 and optionally Expat 1.95.2. From j.p.fletcher@aston.ac.uk Wed May 8 09:06:09 2002 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g48E68a01984 for ; Wed, 8 May 2002 09:06:08 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 175S58-0004pS-00; Wed, 08 May 2002 15:06:14 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Wed, 8 May 2002 15:03:44 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] [ANNOUNCE] e4Graph 1.0a6 CC: Jacob Levy Priority: normal In-reply-to: <3CD92AD3.205@best.com> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jacob Levy To: metakit@equi4.com Subject: [Metakit] [ANNOUNCE] e4Graph 1.0a6 Date sent: Wed, 08 May 2002 06:40:35 -0700 > WHAT > ==== > This is to announce the 1.0a6 release of e4Graph, the sixth and final > Alpha release. > > WHERE TO GET: > ============= > > File releases: http://sourceforge.net/projects/e4graph/ > Documentation: http://e4graph.sourceforge.net/ > Change log: http://e4graph.sourceforge.net/changes.txt > > I have a problem downloading the files from sourceforge which instead of the file sends me an html to tell me to use mirrors but this does not work. John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From BobH@hslda.org Wed May 8 10:20:59 2002 Received: from dymwsm13.mailwatch.com (dymwsm13.mailwatch.com [204.253.83.37]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g48FKwa06203 for ; Wed, 8 May 2002 10:20:58 -0500 Received: from mwsc0210.mw4.mailwatch.com (mwsc0210.mw4.mailwatch.com [204.253.83.228]) by dymwsm13.mailwatch.com (8.11.0/8.11.0) with ESMTP id g48FKt420392 for ; Wed, 8 May 2002 11:20:55 -0400 Received: from mail pickup service by mwsc0210.mw4.mailwatch.com with Microsoft SMTPSVC; Wed, 8 May 2002 11:20:55 -0400 Received: from 204.253.83.71 ([204.253.83.71]) by MWSC0210 with SMTP id 0002000ab286545f-371e-46bd-ac27-0f4f728c3c67; Wed, 08 May 2002 11:20:54 -0500 Received: from proxy.hslda.org (proxy.hslda.org [65.168.81.4]) by dymwsm09.mailwatch.com (8.11.0/8.11.0) with ESMTP id g48FKr725768 for ; Wed, 8 May 2002 11:20:53 -0400 Received: from PHCDOM-MTA by proxy.hslda.org with Novell_GroupWise; Wed, 08 May 2002 11:20:53 -0400 Message-Id: X-Mailer: Novell GroupWise Internet Agent 6.0.1 Date: Wed, 08 May 2002 11:20:38 -0400 From: "Bob Hicks" To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline HOP-COUNT: 1 X-MAILWATCH-INSTANCEID: 0102000ab286545f-371e-46bd-ac27-0f4f728c3c67 X-OriginalArrivalTime: 08 May 2002 15:20:55.0235 (UTC) FILETIME=[F28A8130:01C1F6A3] Subject: [Metakit] Zope Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Has anyone tried to get Metakit working with Zope? Bob From jerry@usd1.com Wed May 8 11:34:43 2002 Received: from bfish.hurrah.com (root@bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g48GYha11775 for ; Wed, 8 May 2002 11:34:43 -0500 Received: from 12-225-195-8.client.attbi.com (12-225-195-8.client.attbi.com [12.225.195.8]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id g48GYh527342 for ; Wed, 8 May 2002 09:34:44 -0700 Date: Wed, 8 May 2002 09:34:14 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.53d) Personal Organization: usd1 X-Priority: 3 (Normal) Message-ID: <16177288089.20020508093414@usd1.com> To: Metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] re: Zope Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, > Has anyone tried to get Metakit working with Zope? > > Bob yes, but for read-only only. [deja vu] all MK instances have to be transient variables (_v_...) so it doesn't try to store them in ZODB. To list MK records easily with DTML, I had to wrap each row in a python class so it can act as a Zope namespace. The MK instances do not have the introspection necessary to act as a namespace. I have not taken the time handle the logic necessary to allow multiple connections to write to the database. my plans include using a long running process to manage writes, and use XML-RPC to talk to it. Unfortunately, like many others right now, I find my financially unsupported free time inadequate to support all of my interests and ideas at this point in time. not sure if that is any help. -- let's not fail to oppose unnecessary, repetitive, and redundant obfuscation Jerry McRae MetaKit 2.4.3, Python 2.2, wxPython 2.2.5, Win98, 384M RAM From jtate@mi-corporation.com Wed May 8 13:27:39 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g48IRba20602 for ; Wed, 8 May 2002 13:27:38 -0500 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g48IRSf24986 for ; Wed, 8 May 2002 14:27:29 -0400 From: "Joseph Tate" To: "MetaKit" Date: Wed, 8 May 2002 14:27:35 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Memory leak in view.cpp Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On line 1223, c4_Property::CleanupInternalData() is only called #ifdef q4_CHECK however on lines 1132-1142, the objects cleaned up by CleanupInternalData() are created whether or not q4_CHECK is defined. Therefore #ifndef q4_CHECK; sThreadLock, sPropNames, and s_PropCounts are never deconstructed and their memory never released. Joseph From chris@pinebush.com Thu May 9 09:16:16 2002 Received: from pinebush.com (host-64-65-203-122.choiceone.net [64.65.203.122]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g49EGGa26265 for ; Thu, 9 May 2002 09:16:16 -0500 Received: from pinebush.com (winntsrv1.pinebush.com [10.0.0.17]) by pinebush.com (8.9.3/8.9.3) with ESMTP id KAA28353 for ; Thu, 9 May 2002 10:16:14 -0400 (EDT) Message-ID: <3CDA84AE.DFF8AD30@pinebush.com> Date: Thu, 09 May 2002 10:16:14 -0400 From: Chris Nelson Organization: Pinebush Technologies, Inc. (http://www.pinebush.com) X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Problems with HP-UX Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have an application that uses MetaKit to store data in a structured file. It's built on HP-UX 10.20. When run on HP-UX 10.20, it fails and the file is corrupted. When run on HP-UX 11.00, it works fine and the file is intact. The application works fine on all versions of Solaris that I've tried (2.6 and 2.8, I think). I'm digging with debuggers and such but if this sounds familiar to anyone, I could use a clue. Chris -- Clarity comes from the [programmer], not the syntax. -- Roland Hough From jyl@best.com Thu May 9 10:15:31 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g49FFUa29398 for ; Thu, 9 May 2002 10:15:30 -0500 Received: (qmail 91341 invoked by uid 19667); 9 May 2002 15:15:30 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 9 May 2002 15:15:30 -0000 Message-ID: <3CDA95A2.2010805@best.com> Date: Thu, 09 May 2002 08:28:34 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Chris Nelson CC: metakit@equi4.com Subject: Re: [Metakit] Problems with HP-UX References: <3CDA84AE.DFF8AD30@pinebush.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What version of Metakit? How does the corruption manifest itself? Please give more details about what exactly does not work. Did you look in the Metakit change logs (http://www.equi4.com/metakit/CHANGES) to see if anything that looks similar to your problem was recently fixed? Thanks! --JYL Chris Nelson wrote: >I have an application that uses MetaKit to store data in a structured >file. It's built on HP-UX 10.20. When run on HP-UX 10.20, it fails and >the file is corrupted. When run on HP-UX 11.00, it works fine and the >file is intact. The application works fine on all versions of Solaris >that I've tried (2.6 and 2.8, I think). I'm digging with debuggers and >such but if this sounds familiar to anyone, I could use a clue. > > Chris > From chris@pinebush.com Thu May 9 10:48:14 2002 Received: from pinebush.com (host-64-65-203-122.choiceone.net [64.65.203.122]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g49FmEa31624 for ; Thu, 9 May 2002 10:48:14 -0500 Received: from pinebush.com (winntsrv1.pinebush.com [10.0.0.17]) by pinebush.com (8.9.3/8.9.3) with ESMTP id LAA29297; Thu, 9 May 2002 11:48:12 -0400 (EDT) Message-ID: <3CDA9A3C.52B1590E@pinebush.com> Date: Thu, 09 May 2002 11:48:12 -0400 From: Chris Nelson Organization: Pinebush Technologies, Inc. (http://www.pinebush.com) X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Jacob Levy CC: metakit@equi4.com Subject: Re: [Metakit] Problems with HP-UX References: <3CDA84AE.DFF8AD30@pinebush.com> <3CDA95A2.2010805@best.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > > What version of Metakit? 2.0 > How does the corruption manifest itself? Please give more details about > what exactly does not work. All of the keys are null strings. None of the values are correct. > Did you look in the Metakit change logs > (http://www.equi4.com/metakit/CHANGES) to see if anything that looks > similar to your problem was recently fixed? Not yet. Going there now... Chris -- They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin From jcw@equi4.com Thu May 9 11:02:10 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g49G29a32515; Thu, 9 May 2002 11:02:09 -0500 From: Jean-Claude Wippler To: Chris Nelson , Jacob Levy Cc: Subject: Re: [Metakit] Problems with HP-UX Date: Thu, 9 May 2002 17:57:25 +0200 Message-Id: <20020509155725.555@triqs.com> In-Reply-To: <3CDA9A3C.52B1590E@pinebush.com> References: <3CDA9A3C.52B1590E@pinebush.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Chris, >> What version of Metakit? > >2.0 Is that 2.01, rounded? >> How does the corruption manifest itself? Please give more details about >> what exactly does not work. > >All of the keys are null strings. None of the values are correct. If it is consistent, can you trim down an example? >> Did you look in the Metakit change logs >> (http://www.equi4.com/metakit/CHANGES) to see if anything that looks >> similar to your problem was recently fixed? > >Not yet. Going there now... I hate to play the upgrade game, but for several reasons, I'm going to anyway: have you considered upgrading to MK 2.4.6? There are some API changes. I know too little to comment on HP-UX details. There have been cases where compilers were not doing all the 100% right things in C++, such as destructing at th ewrong time (temp objects), which would affect MK. Are you compiling with all optimizations turned off everywhere? -jcw From chris@pinebush.com Thu May 9 11:19:27 2002 Received: from pinebush.com (host-64-65-203-122.choiceone.net [64.65.203.122]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g49GJRa00997; Thu, 9 May 2002 11:19:27 -0500 Received: from pinebush.com (winntsrv1.pinebush.com [10.0.0.17]) by pinebush.com (8.9.3/8.9.3) with ESMTP id MAA29466; Thu, 9 May 2002 12:19:26 -0400 (EDT) Message-ID: <3CDAA18D.2F23538C@pinebush.com> Date: Thu, 09 May 2002 12:19:25 -0400 From: Chris Nelson Organization: Pinebush Technologies, Inc. (http://www.pinebush.com) X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Jacob Levy , metakit@equi4.com Subject: Re: [Metakit] Problems with HP-UX References: <3CDA9A3C.52B1590E@pinebush.com> <20020509155725.555@triqs.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > > Chris, > > >> What version of Metakit? > > > >2.0 > > Is that 2.01, rounded? Nope. > >> How does the corruption manifest itself? Please give more details about > >> what exactly does not work. > > > >All of the keys are null strings. None of the values are correct. > > If it is consistent, can you trim down an example? Quite consistent. It would be very difficult to trim down an example from our large application. > >> Did you look in the Metakit change logs > >> (http://www.equi4.com/metakit/CHANGES) to see if anything that looks > >> similar to your problem was recently fixed? > > > >Not yet. Going there now... > > I hate to play the upgrade game, but for several reasons, I'm going to > anyway: have you considered upgrading to MK 2.4.6? There are some API > changes. Yeah, I know. But we _have_ considered and discarded the idea of upgrading. Our Solaris release it out and we're just trying to shake the bugs out of HP-UX. We don't want to change revs of a tool in the middle of a release. > I know too little to comment on HP-UX details. There have been cases > where compilers were not doing all the 100% right things in C++, such as > destructing at th ewrong time (temp objects), which would affect MK. Are > you compiling with all optimizations turned off everywhere? I expect quite the opposite is true: we're optimizing the heck out of it. Chris -- "Custody" and "visitation" are for prisoners. http://assembly.state.ny.us/leg/?bn=A01920&sh=t From chris@pinebush.com Thu May 9 15:03:35 2002 Received: from pinebush.com (host-64-65-203-122.choiceone.net [64.65.203.122]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g49K3Ya16716; Thu, 9 May 2002 15:03:34 -0500 Received: from pinebush.com (winntsrv1.pinebush.com [10.0.0.17]) by pinebush.com (8.9.3/8.9.3) with ESMTP id QAA01896; Thu, 9 May 2002 16:03:33 -0400 (EDT) Message-ID: <3CDAD615.C9EF2B5D@pinebush.com> Date: Thu, 09 May 2002 16:03:33 -0400 From: Chris Nelson Organization: Pinebush Technologies, Inc. (http://www.pinebush.com) X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Jacob Levy , metakit@equi4.com Subject: Re: [Metakit] Problems with HP-UX References: <3CDA9A3C.52B1590E@pinebush.com> <20020509155725.555@triqs.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Against all reason, this problem occurs ONLY when our application is installed on local disk of a particular workstation. Whatever fluke is causing it, we're giving up. Since our application will typically be installed on n/w disk, we're not going to worry about it. chris -- Clarity comes from the [programmer], not the syntax. -- Roland Hough From jyl@best.com Fri May 10 00:18:20 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g4A5IKa13373 for ; Fri, 10 May 2002 00:18:20 -0500 Received: (qmail 89685 invoked by uid 19667); 10 May 2002 05:18:19 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 10 May 2002 05:18:19 -0000 Message-ID: <3CDB5B29.5000900@best.com> Date: Thu, 09 May 2002 22:31:21 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Chris Nelson CC: Jean-Claude Wippler , metakit@equi4.com Subject: Re: [Metakit] Problems with HP-UX References: <3CDA9A3C.52B1590E@pinebush.com> <20020509155725.555@triqs.com> <3CDAA18D.2F23538C@pinebush.com> Content-Type: multipart/alternative; boundary="------------070000080408010000070108" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------070000080408010000070108 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit When you compile with lower optimization does it work on HP-UX? If so I'd suspect a bug in the compiler... --JYL Chris Nelson wrote: >Jean-Claude Wippler wrote: > >>Chris, >> >>>>What version of Metakit? >>>> >>>2.0 >>> >>Is that 2.01, rounded? >> > >Nope. > >>>>How does the corruption manifest itself? Please give more details about >>>>what exactly does not work. >>>> >>>All of the keys are null strings. None of the values are correct. >>> >>If it is consistent, can you trim down an example? >> > >Quite consistent. It would be very difficult to trim down an example >from our large application. > >>>>Did you look in the Metakit change logs >>>>(http://www.equi4.com/metakit/CHANGES) to see if anything that looks >>>>similar to your problem was recently fixed? >>>> >>>Not yet. Going there now... >>> >>I hate to play the upgrade game, but for several reasons, I'm going to >>anyway: have you considered upgrading to MK 2.4.6? There are some API >>changes. >> > >Yeah, I know. But we _have_ considered and discarded the idea of >upgrading. Our Solaris release it out and we're just trying to shake >the bugs out of HP-UX. We don't want to change revs of a tool in the >middle of a release. > >>I know too little to comment on HP-UX details. There have been cases >>where compilers were not doing all the 100% right things in C++, such as >>destructing at th ewrong time (temp objects), which would affect MK. Are >>you compiling with all optimizations turned off everywhere? >> > >I expect quite the opposite is true: we're optimizing the heck out of >it. > > Chris > --------------070000080408010000070108 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit When you compile with lower optimization does it work on HP-UX? If so I'd suspect a bug in the compiler...

--JYL

Chris Nelson wrote:
Jean-Claude Wippler wrote:
Chris,

What version of Metakit?
2.0
Is that 2.01, rounded?

Nope.

How does the corruption manifest itself? Please give more details about
what exactly does not work.
All of the keys are null strings.  None of the values are correct.
If it is consistent, can you trim down an example?

Quite consistent.  It would be very difficult to trim down an example
from our large application.

Did you look in the Metakit change logs
(http://www.equi4.com/metakit/CHANGES) to see if anything that looks
similar to your problem was recently fixed?
Not yet.  Going there now...
I hate to play the upgrade game, but for several reasons, I'm going to
anyway: have you considered upgrading to MK 2.4.6?  There are some API
changes.

Yeah, I know.  But we _have_ considered and discarded the idea of
upgrading.  Our Solaris release it out and we're just trying to shake
the bugs out of HP-UX.  We don't want to change revs of a tool in the
middle of a release.

I know too little to comment on HP-UX details.  There have been cases
where compilers were not doing all the 100% right things in C++, such as
destructing at th ewrong time (temp objects), which would affect MK.  Are
you compiling with all optimizations turned off everywhere?

I expect quite the opposite is true: we're optimizing the heck out of
it.

                                             Chris

--------------070000080408010000070108-- From chihung@singnet.com.sg Fri May 10 01:18:55 2002 Received: from smtp13.singnet.com.sg (smtp13.singnet.com.sg [165.21.6.33]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4A6Ira15527 for ; Fri, 10 May 2002 01:18:53 -0500 Received: from pike.singnet.com.sg (pike.singnet.com.sg [165.21.101.112]) by smtp13.singnet.com.sg (8.12.3/8.12.3) with ESMTP id g4A6IpDj005930; Fri, 10 May 2002 14:18:51 +0800 Received: (from cooluser@localhost) by pike.singnet.com.sg (8.9.3/8.9.3) id OAA11462; Fri, 10 May 2002 14:18:48 +0800 X-Authentication-Warning: pike.singnet.com.sg: cooluser set sender to chihung@singnet.com.sg using -f To: metakit@equi4.com Message-ID: <1021011528.3cdb664820c1b@pike.singnet.com.sg> Date: Fri, 10 May 2002 14:18:48 +0800 (SGT) From: Chan Chi Hung Cc: chihung@mbox2.singnet.com.sg Reply-To: chihung@singnet.com.sg MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: SingNet WebMail Subject: [Metakit] [Q] Cannot compile on Solaris 8 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am trying to compile Mk4tcl and encountered configuration problem. Is there any extra flag that I should turn on and what will be the correct value for my environment. I include my steps in configuring Mk4tcl. After some investigation, I tried to configure it with --host=sun4sol2. However, I got a compilation error. Appreciate your help. Many Thanks. --Chi Hung @ Singapore $ cd metakit-2.4.5/builds $ uname -a SunOS asp5 5.8 Generic_108528-09 sun4u sparc SUNW,Ultra-5_10 $ gcc --version 2.95.3 $ /usr/local2/bin/tclsh % set tcl_patchLevel 8.3.4 % exit $ ../unix/configure --with-tcl=/usr/local/include checking for Python configuration... not enabled checking for Tcl headers... found /usr/local2/include/tcl.h checking for g++... g++ checking for C++ compiler default output... a.out checking whether the C++ compiler works... configure: error: cannot run C++ compiled programs. If you meant to cross compile, use `--host'. $ ../unix/configure --with-tcl=/usr/local/include --host=sun4sol2 $ make /bin/sh ./libtool --mode=compile g++ -c -O2 -I../unix/../include -I/usr/local2/include/generic -I/usr/local2/include ../tcl/mk4tcl.cpp g++ -c -O2 -I../unix/../include -I/usr/local2/include/generic -I/usr/local2/include ../tcl/mk4tcl.cpp -fPIC -DPIC -o .libs/mk4tcl.o ../tcl/mk4tcl.cpp:1480: prototype for `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, char **, char *)' does not match any in class `Tcl' ../tcl/mk4tcl.h:351: candidate is: int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = "option") *** Error code 1 make: Fatal error: Command failed for target `mk4tcl.o' From umbright@taltrade.com Fri May 10 13:52:35 2002 Received: from talchiexch03.taltrade.com (talchiexch03.taltrade.com [207.48.80.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4AIqZa16468 for ; Fri, 10 May 2002 13:52:35 -0500 Received: from mail pickup service by talchiexch03.taltrade.com with Microsoft SMTPSVC; Fri, 10 May 2002 13:52:34 -0500 x-gfisavedcharset: iso-8859-1 Content-Type: text/plain; charset="iso-8859-1" Received: from talchiexch02.taltrade.com ([10.207.64.102]) by talchiexch03.taltrade.com with Microsoft SMTPSVC(5.0.2195.2966); Fri, 10 May 2002 13:52:32 -0500 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Date: Fri, 10 May 2002 13:52:30 -0500 Message-ID: <45CE1FBFCDAD2E4DA84B49F2F866FC4F18317A@talchiexch02.taltrade.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: change to a derived view of a derived view of a... Thread-Index: AcH4U9e9s1cEtegvSluOSp8bfM9D5Q== From: "Guy Umbright" To: X-OriginalArrivalTime: 10 May 2002 18:52:33.0179 (UTC) FILETIME=[D7EE7EB0:01C1F853] Subject: [Metakit] change to a derived view of a derived view of a... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have a series of views: base view1 (selected from base) view2 (selected from view1) view3 (selected from view2) Adding rows to base works fine, updating all others. But if try to update a value in view3, it doesn't seem to take. i.e. - propSomething(view3[0]) =3D 10; if I then retrieve the value is still says it is the original not 10. Is a selected view at that level read only or something? Or should it work? This electronic mail message and any attached files contain information = intended for the exclusive use of the individual or entity to whom it is = addressed and may contain information that is proprietary, privileged, = confidential and/or exempt from disclosure under applicable law. If you = are not the intended recipient, you are hereby notified that any = viewing, copying, disclosure or distribution of this information may be = subject to legal restriction or sanction. Please notify the sender, by = electronic mail or telephone, of any unintended recipients and delete = the original message without making any copies. From tonymann@racsa.co.cr Fri May 10 18:16:53 2002 Received: from hydra (saturno.racsa.co.cr [196.40.31.23] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4ANGqa30824 for ; Fri, 10 May 2002 18:16:53 -0500 Received: from tony ([196.40.52.152]) by hydra.racsa.co.cr (iPlanet Messaging Server 5.1 (built May 7 2001)) with SMTP id <0GVX0077C13YRM@hydra.racsa.co.cr> for metakit@equi4.com; Fri, 10 May 2002 17:04:20 -0500 (CDT) Date: Fri, 10 May 2002 17:10:45 -0600 From: Tony Mann Subject: Re: [Metakit] change to a derived view of a derived view of a... To: metakit@equi4.com Message-id: <004401c1f878$cef4d3c0$983428c4@tony> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-Mailer: Microsoft Outlook Express 6.00.2600.0000 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7BIT X-Priority: 3 X-MSMail-priority: Normal References: <45CE1FBFCDAD2E4DA84B49F2F866FC4F18317A@talchiexch02.taltrade.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For derived views, you need to make changes to the base view, and those changes will propogate to the derived views. Just use baseView.GetIndexOf to map a row ref from a derived view to an index in the base view: propSomething(base[base.GetIndexOf(view3[0])]) = 10; ..tony.. ----- Original Message ----- From: "Guy Umbright" To: Sent: Friday, May 10, 2002 12:52 PM Subject: [Metakit] change to a derived view of a derived view of a... I have a series of views: base view1 (selected from base) view2 (selected from view1) view3 (selected from view2) Adding rows to base works fine, updating all others. But if try to update a value in view3, it doesn't seem to take. i.e. - propSomething(view3[0]) = 10; if I then retrieve the value is still says it is the original not 10. Is a selected view at that level read only or something? Or should it work? From john@rkroll.com Fri May 10 20:44:35 2002 Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4B1iZa01726 for ; Fri, 10 May 2002 20:44:35 -0500 Received: from WATERFALL ([65.96.184.78]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with SMTP id <20020511014429.GDRI22408.rwcrmhc53.attbi.com@WATERFALL> for ; Sat, 11 May 2002 01:44:29 +0000 Message-ID: <001001c1f88b$d706e360$1202a8c0@WATERFALL> From: "john" To: Date: Fri, 10 May 2002 21:33:22 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] don't know how to run sdx for sdx?? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm having trouble using the tclkit on windows 2000. I have downloaded the latest files and cannot get the basic sdx document to run. tclkit.exe run file wo/arguments, popping up the tk and console windows. "tclkitsh sdx" will not run. It seems to be unable to find its initial script. I get: don't know how to run sdx for sdx I have tried this command from the wk2 command interpreter, cygwin shell and from inside the .bat file described on the web pages. A similar problem was reported for someone trying to build tclhttpd as a scripted doc a couple of months ago, but no clear answer to the problem was presented. The same sdx file runs fine on linux. Thanks, John From smalllinux@netscape.net Sun May 12 01:25:21 2002 Received: from imo-m09.mx.aol.com (imo-m09.mx.aol.com [64.12.136.164]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4C6PLa01337 for ; Sun, 12 May 2002 01:25:21 -0500 Received: from smalllinux@netscape.net by imo-m09.mx.aol.com (mail_out_v32.5.) id d.56.2a8723e (22683) for ; Sun, 12 May 2002 02:25:11 -0400 (EDT) Received: from netscape.com (mow-d05.webmail.aol.com [205.188.138.69]) by air-in04.mx.aol.com (v84.14) with ESMTP id MAILININ44-0512022510; Sun, 12 May 2002 02:25:10 -0500 Date: Sun, 12 May 2002 02:26:14 -0400 From: smalllinux@netscape.net To: metakit@equi4.com Message-ID: <78DABC35.5EDD2A7B.1080C5D1@netscape.net> X-Mailer: Atlas Mailer 2.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] Scripted Documents Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi JCW, Thanks for the advice on Scripted documents. When I change them to use m -readonly they work on the CD. Steven G. http://www.superant.com/ __________________________________________________________________ Your favorite stores, helpful shopping tools and great gift ideas. Experience the convenience of buying online with Shop@Netscape! http://shopnow.netscape.com/ Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From e2wugui@163.com Mon May 13 08:40:41 2002 Received: from sm4.163.com ([202.108.44.206]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4DDefa09397 for ; Mon, 13 May 2002 08:40:41 -0500 Received: from localhost (localhost [127.0.0.1]) by sm4.163.com (Postfix) with SMTP id 33C4A1C78416A for ; Mon, 13 May 2002 21:40:38 +0800 (CST) Received: from lch (unknown [61.144.142.9]) by 192.168.1.206 (Coremail) with SMTP id oFYAAFPC3zwAEI4J.1 for ; Mon, 13 May 2002 21:40:38 +0800 (CST) Message-ID: <000501c1fa83$3341bad0$9003a8c0@lch> From: "wugui" To: Date: Mon, 13 May 2002 21:36:32 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] problem ? Select and InsertAt Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i have a problem, mycode like: // version 2.4.2 c4_View vTableProp = storage.GetAs("names[name:S]"); c4_StringProp PName("TableName"); 1 c4_View vSelect = vTableProp.Select(PName["tom"]); 2 if (vSelect.GetSize() > 0) return; 3.1 // add. ok vTableProp.Add(PName["tom"]); 3.2 // insert. bug ??? vTableProp.InsertAt(0, PName["tom"]); // call stack c4_DWordArray::SetAt(int -33686018(XXX), long 0) c4_FilterSeq::FixupReverseMap() c4_FilterSeq::PostChange(c4_Notifier & {...}) c4_Notifier::~c4_Notifier() c4_Sequence::InsertAt(int 0, c4_Cursor {...}, int 1) c4_View::InsertAt(int 0, const c4_RowRef & {...}, int 1) // if no 1,2 then 3.1 3.2 are both ok ?? // version 2.4.6 CHANGES [ Copy 2001-10-18 Fixed recent hash bug, and add to ordered The recent "fix" to deal with hash misses introduced a huge bug, causing the mapping to be recalculated on each insertion. Adding to an ordered view did not always work, because the code was based on SetAtGrow, which is not suitable for an ordered view in which the row position is determined implicitly. Fixed by changing c4_View::Add to use c4_View::InsertAt instead. ] this will fixed this? i'm not good at english. i hope you can understand what i say. Thank you very much! e2wugui@163.com From umbright@taltrade.com Mon May 13 09:36:07 2002 Received: from talchiexch03.taltrade.com (talchiexch03.taltrade.com [207.48.80.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4DEa7a11713 for ; Mon, 13 May 2002 09:36:07 -0500 Received: from mail pickup service by talchiexch03.taltrade.com with Microsoft SMTPSVC; Mon, 13 May 2002 09:36:07 -0500 x-gfisavedcharset: iso-8859-1 Content-Type: text/plain; charset="iso-8859-1" Received: from talchiexch02.taltrade.com ([10.207.64.102]) by talchiexch03.taltrade.com with Microsoft SMTPSVC(5.0.2195.2966); Mon, 13 May 2002 09:36:05 -0500 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Subject: RE: [Metakit] change to a derived view of a derived view of a... Date: Mon, 13 May 2002 09:36:01 -0500 Message-ID: <45CE1FBFCDAD2E4DA84B49F2F866FC4F18317B@talchiexch02.taltrade.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Metakit] change to a derived view of a derived view of a... Thread-Index: AcH4eODmgE6HUMGTSOaQ2L7ci1orcACENWKw From: "Guy Umbright" To: X-OriginalArrivalTime: 13 May 2002 14:36:05.0806 (UTC) FILETIME=[8394F4E0:01C1FA8B] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I played with it a little more and figured out what you have supplied. At least I know it is definitely the right way to do it now. Thanks. > -----Original Message----- > From: Tony Mann [mailto:tonymann@racsa.co.cr] > Sent: Friday, May 10, 2002 6:11 PM > To: metakit@equi4.com > Subject: Re: [Metakit] change to a derived view of a derived view of > a... >=20 >=20 > For derived views, you need to make changes to the base view,=20 > and those > changes will propogate to the derived views. Just use=20 > baseView.GetIndexOf to > map a row ref from a derived view to an index in the base view: >=20 > propSomething(base[base.GetIndexOf(view3[0])]) =3D 10; >=20 > ..tony.. >=20 > ----- Original Message ----- > From: "Guy Umbright" > To: > Sent: Friday, May 10, 2002 12:52 PM > Subject: [Metakit] change to a derived view of a derived view of a... >=20 >=20 > I have a series of views: >=20 > base > view1 (selected from base) > view2 (selected from view1) > view3 (selected from view2) >=20 > Adding rows to base works fine, updating all others. But > if try to update a value in view3, it doesn't seem to take. >=20 > i.e. - propSomething(view3[0]) =3D 10; >=20 > if I then retrieve the value is still says it is the original > not 10. Is a selected view at that level read only or something? > Or should it work? >=20 > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit >=20 This electronic mail message and any attached files contain information = intended for the exclusive use of the individual or entity to whom it is = addressed and may contain information that is proprietary, privileged, = confidential and/or exempt from disclosure under applicable law. If you = are not the intended recipient, you are hereby notified that any = viewing, copying, disclosure or distribution of this information may be = subject to legal restriction or sanction. Please notify the sender, by = electronic mail or telephone, of any unintended recipients and delete = the original message without making any copies. From tonymann@racsa.co.cr Mon May 13 09:51:23 2002 Received: from che.kiza.com (IDENT:root@[198.144.196.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4DEpNa12408 for ; Mon, 13 May 2002 09:51:23 -0500 Received: from tony (puntarenas-a114.racsa.co.cr [196.40.52.243]) by che.kiza.com (8.11.2/8.11.2) with SMTP id g4DF2bH02379 for ; Mon, 13 May 2002 08:02:38 -0700 Message-ID: <003b01c1fa8d$b3365660$f33428c4@tony> From: "Tony Mann" To: References: <45CE1FBFCDAD2E4DA84B49F2F866FC4F18317B@talchiexch02.taltrade.com> Subject: Re: [Metakit] change to a derived view of a derived view of a... Date: Mon, 13 May 2002 08:51:37 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Also worth mentioning is that when using "mapping viewers" (like Indexed), the opposite is true. Changes made to a row in the mapped view will propogate upward to the base view. You can read all about this and other view oddities at: http://www.equi4.com/metakit/wiki.cgi/99.html ..tony.. > -----Original Message----- > From: Tony Mann [mailto:tonymann@racsa.co.cr] > Sent: Friday, May 10, 2002 6:11 PM > To: metakit@equi4.com > Subject: Re: [Metakit] change to a derived view of a derived view of > a... > > > For derived views, you need to make changes to the base view, > and those > changes will propogate to the derived views. Just use > baseView.GetIndexOf to > map a row ref from a derived view to an index in the base view: From umbright@taltrade.com Mon May 13 10:01:30 2002 Received: from talchiexch03.taltrade.com (talchiexch03.taltrade.com [207.48.80.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4DF1Ta12980 for ; Mon, 13 May 2002 10:01:29 -0500 Received: from mail pickup service by talchiexch03.taltrade.com with Microsoft SMTPSVC; Mon, 13 May 2002 10:01:28 -0500 x-gfisavedcharset: iso-8859-1 Content-Type: text/plain; charset="iso-8859-1" Received: from talchiexch02.taltrade.com ([10.207.64.102]) by talchiexch03.taltrade.com with Microsoft SMTPSVC(5.0.2195.2966); Mon, 13 May 2002 10:01:25 -0500 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Subject: RE: [Metakit] change to a derived view of a derived view of a... Date: Mon, 13 May 2002 10:01:20 -0500 Message-ID: <45CE1FBFCDAD2E4DA84B49F2F866FC4FF681F4@talchiexch02.taltrade.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Metakit] change to a derived view of a derived view of a... Thread-Index: AcH6jd5T8i0ldZWgSH+pyIfiellevAAARXyw From: "Guy Umbright" To: X-OriginalArrivalTime: 13 May 2002 15:01:25.0637 (UTC) FILETIME=[0D78C350:01C1FA8F] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks for the link! That is some great info. > -----Original Message----- > From: Tony Mann [mailto:tonymann@racsa.co.cr] > Sent: Monday, May 13, 2002 9:52 AM > To: metakit@equi4.com > Subject: Re: [Metakit] change to a derived view of a derived view of > a... >=20 >=20 > Also worth mentioning is that when using "mapping viewers"=20 > (like Indexed), > the opposite is true. Changes made to a row in the mapped view will > propogate upward to the base view. You can read all about=20 > this and other > view oddities at: >=20 > http://www.equi4.com/metakit/wiki.cgi/99.html >=20 > ..tony.. >=20 > > -----Original Message----- > > From: Tony Mann [mailto:tonymann@racsa.co.cr] > > Sent: Friday, May 10, 2002 6:11 PM > > To: metakit@equi4.com > > Subject: Re: [Metakit] change to a derived view of a derived view of > > a... > > > > > > For derived views, you need to make changes to the base view, > > and those > > changes will propogate to the derived views. Just use > > baseView.GetIndexOf to > > map a row ref from a derived view to an index in the base view: >=20 > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit >=20 This electronic mail message and any attached files contain information = intended for the exclusive use of the individual or entity to whom it is = addressed and may contain information that is proprietary, privileged, = confidential and/or exempt from disclosure under applicable law. If you = are not the intended recipient, you are hereby notified that any = viewing, copying, disclosure or distribution of this information may be = subject to legal restriction or sanction. Please notify the sender, by = electronic mail or telephone, of any unintended recipients and delete = the original message without making any copies. From jcw@equi4.com Tue May 14 07:36:22 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4ECaKa17263 for ; Tue, 14 May 2002 07:36:21 -0500 From: Jean-Claude Wippler To: metakit Date: Tue, 14 May 2002 14:35:01 +0200 Message-Id: <20020514123501.21014@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Adaptive int bug fix for 2.3/2.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: In the category: yet another quite important bug in MetaKit... Today, a new problem has been reported and resolved, affecting integers. For details, see the change log at http://www.equi4.com/metakit/CHANGES - the bug is triggered by a specific sequence of at least two inserts/ deletes and resizing of 1-byte ints to 2/4-byte ints. The effect it a single int being returned incorrectly (near the previously inserted/ deleted row), but *only* until the next commit. After commit, values are correct. I've checked a fix into CVS, along with test s46 to verify that it works. There will be a new release/distribution of MetaKit before the end of the month. -jcw From joshm@taconic.net Tue May 14 15:19:03 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4EKJ2a26836 for ; Tue, 14 May 2002 15:19:03 -0500 Received: from gatekeeper (mcha-ab087.taconic.net [205.231.148.182]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g4EKJ1428407 for ; Tue, 14 May 2002 16:19:01 -0400 (EDT) Message-ID: <095801c1fb84$16ff75e0$1b94e7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" Date: Tue, 14 May 2002 16:15:26 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Storing bitmaps -- should they go to a separate file? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hey all, A performance question. I have a db table which keeps track of information about some image files I have. This is simple stuff (location, filename, some fixed sized binary data, etc). I want to add in tiny thumbnail images to the data. These would be 1-10k in size (not sure of their exact memory requirements). The goal will be to store them as DIBs (windows bitmaps), and then access them directly from the db for maximum performance. Here's the question. This same db file has a lot of other semi-related data in it, in separate tables. Would it be better to store the DIBs in a separate db file (two columns -- the DIBs and a sortable key which corresponds to the table in the original file), or just stuff them in as yet another column in the original table? I expect that once the DIB is stored for a given image, it will very rarely change. But lots of other data in the file, especially in the other tables, will change frequently. Suggestions are much appreciated on this. -- j Out The Door In 2K4! From FabienB@gilian.com Wed May 15 04:06:23 2002 Received: from gilianmail.gilian.com (gw-guilian.ser.netvision.net.il [199.203.171.166]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4F96Ma14065 for ; Wed, 15 May 2002 04:06:23 -0500 Received: by gilianmail with Internet Mail Service (5.5.2650.21) id ; Wed, 15 May 2002 12:06:51 +0300 Message-ID: <1797E7C8F378D411A1CE00D0B77E59CE572FBD@gilianmail> From: Fabien Benichou To: "'metakit@equi4.com'" Date: Wed, 15 May 2002 12:06:50 +0300 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="windows-1255" Subject: [Metakit] Case sensitive Management Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I made a little change in Metakti 2.4.6 such that any method of c4_View can be case-sensitive or case-insensitive at any moment. For instance, the methods Sort, Select, Pair, GroupBy, Counts, Unique, Intersect, Union, Minus, Join cab be applied as case-sensitive or case-sensitive. I know that my changes can be improved and may imply some other problems but at least, we do not need to preprocess strings in order to use case-sensitive methods as suggested in metakit digest Vol 1, #198, Message 1. Here are the changes I made: In mk4.h: I add the static member in c4_View static bool _CaseSensitive; In view.cpp, line 969, I added: bool c4_View::_CaseSensitive = false; In format.cpp, the method code has been changed: c4_FormatS::DoCompare(const c4_Bytes& b1, const c4_Bytes& b2) { c4_String v1 ((const char *) b1_.Contents(), b1_.Size()); c4_String v2 ((const char *) b2_.Contents(), b2_.Size()); if ( c4_View::_CaseSensitive) return v1.Compare(v2); else return v1.CompareNoCase(v2); } Here is a sample code about how to use the case-sensitive management c4_View v1, v2, v3; c4_StringProp p1("p1"); v1.Add("Fab"); v1.Add("fab"); v2 = v1.Unique(); A(v2.GetSize() == 1) c4_View::_CaseSensitive = true; v3 = v1.Unique(); A(v3.GetSize() == 2) I must surely be improved and it may imply other problems. But instead of preprocessing strings Fabien Benichou Software Engineer Gilian Technologies "The last line of defense against hacker sabotage!" www.gilian.com Tel: 972-9-956-0036 x233 FAX: 972-9-956-5668 From tonymann@racsa.co.cr Wed May 15 10:58:26 2002 Received: from che.kiza.com (IDENT:root@[198.144.196.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4FFwPa22659 for ; Wed, 15 May 2002 10:58:25 -0500 Received: from tony (puntarenas-a81.racsa.co.cr [196.40.52.210]) by che.kiza.com (8.11.2/8.11.2) with SMTP id g4FGA3i01170 for ; Wed, 15 May 2002 09:10:04 -0700 Message-ID: <00ca01c1fc29$66a25d60$a43428c4@tony> From: "Tony Mann" To: References: <1797E7C8F378D411A1CE00D0B77E59CE572FBD@gilianmail> Date: Wed, 15 May 2002 09:56:59 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1255" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Duplicate and notifications Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Two questions, somewhat related: * What is the time/space cost of the c4_View::Duplicate() method? Is it reasonable to use this on a view with 10K records and 10 string columns? * I am using SortOn to create a derived view. However, I do not want this derived view to change as the underlying data is changed. That is, changing the sort key of an entry should not change its order in the derived view. Is there a way to turn off the notifications for a view? Or would this cause other problems? Right now I Duplicate() the derived view to disconnect the notifications. ..tony.. From jcw@equi4.com Thu May 16 02:23:47 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4G7Nka17316 for ; Thu, 16 May 2002 02:23:46 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Storing bitmaps -- should they go to a separate file? Date: Thu, 16 May 2002 09:13:33 +0200 Message-Id: <20020516071333.10053@triqs.com> In-Reply-To: <095801c1fb84$16ff75e0$1b94e7cd@gatekeeper> References: <095801c1fb84$16ff75e0$1b94e7cd@gatekeeper> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joshua Muskovitz wrote: >Here's the question. This same db file has a lot of other semi-related data >in it, in separate tables. Would it be better to store the DIBs in a >separate db file (two columns -- the DIBs and a sortable key which >corresponds to the table in the original file), or just stuff them in as yet >another column in the original table? > >I expect that once the DIB is stored for a given image, it will very rarely >change. But lots of other data in the file, especially in the other tables, >will change frequently. STick with one file until there are indications that it is not optimal. Column-wise storage means no I/O takes place for items which are not used. With one file, you get the benefit of consistency no matter what happens. The only overhead I can think of is free-space management, which affects commits mostly. -jcw From jcw@equi4.com Thu May 16 02:23:49 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4G7Nma17319 for ; Thu, 16 May 2002 02:23:48 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Case sensitive Management Date: Thu, 16 May 2002 09:13:51 +0200 Message-Id: <20020516071351.12818@triqs.com> In-Reply-To: <1797E7C8F378D411A1CE00D0B77E59CE572FBD@gilianmail> References: <1797E7C8F378D411A1CE00D0B77E59CE572FBD@gilianmail> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Fabien Benichou wrote: >I made a little change in Metakti 2.4.6 such that any method of c4_View can >be case-sensitive or case-insensitive at any moment. [...] >I add the static member in c4_View > static bool _CaseSensitive; > >In view.cpp, line 969, I added: > bool c4_View::_CaseSensitive = false; > >In format.cpp, the method code has been changed: > c4_FormatS::DoCompare(const c4_Bytes& b1, const c4_Bytes& b2) > { > c4_String v1 ((const char *) b1_.Contents(), b1_.Size()); > c4_String v2 ((const char *) b2_.Contents(), b2_.Size()); > if ( c4_View::_CaseSensitive) > return v1.Compare(v2); > else > return v1.CompareNoCase(v2); > } Yes, this looks like a valuable change, the current approach is too inflexible. There are a few comments I'd like to add: - Global state is convenient, except when used/altered all over the place (imagine writing a library which assumes certain settings, while apps using it differ in what they want - you end up doing save/set/use/restore all the time). - There may be an issue when this is used with threads, though basic use is probably fine (setting the mode more or less just once in the app, for example). - Why stop here: another very useful option would be UTF-8 aware comparisons. My own thinking was to move such a setting to the storage object (to the strategy object, actually). Since storages are thread-specific, this avoids the first two issues. By storing not a flag but an actual function pointer (or object with virtual member), the UTF-8 option would be feasible as well. Thanks for bringing this up. Your change is definitely an easy way to get the benefit of case sensitive vs. insensitive with little effort. I'll think a bit more on how to take this further so it gets resolved in the near future. -jcw From jalupo@attbi.com Thu May 16 13:19:57 2002 Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4GIJva04947 for ; Thu, 16 May 2002 13:19:57 -0500 Received: from egor.linux.home ([12.253.55.237]) by sccrmhc01.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020516181952.XUCJ8004.sccrmhc01.attbi.com@egor.linux.home> for ; Thu, 16 May 2002 18:19:52 +0000 Received: from egor.linux.home (localhost [127.0.0.1]) by egor.linux.home (8.12.3/8.12.3/Debian-9) with ESMTP id g4GIJon9006179; Thu, 16 May 2002 12:19:50 -0600 Received: (from jalupo@localhost) by egor.linux.home (8.12.3/8.12.3/Debian-9) id g4GIJlKL006175; Thu, 16 May 2002 12:19:48 -0600 Date: Thu, 16 May 2002 12:19:48 -0600 From: "James A. Lupo" Message-Id: <200205161819.g4GIJlKL006175@egor.linux.home> To: metakit@equi4.com Subject: [Metakit] Error Compiling metakit-2.4.6-36 on Linux Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I just downloaded and tried to compile metakit-2.4.6-36 on Linux. The steps I followed were: cd builds ../unix/configure --with-tcl=/usr/include/tcl8.3 make This failed with the error: /bin/sh ./libtool --mode=compile g++ -c -O2 -I../unix/../include \ -I/usr/include/tcl8.3/generic -I/usr/include/tcl8.3 \ ../unix/../tcl/mk4tcl.cpp g++ -c -O2 -I../unix/../include -I/usr/include/tcl8.3/generic \ -I/usr/include/tcl8.3 ../unix/../tcl/mk4tcl.cpp -fPIC -DPIC \ -o .libs/mk4tcl.o ../unix/../tcl/mk4tcl.cpp:1480: prototype for \ `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, char **, char *)' \ does not match any in class `Tcl' ../unix/../tcl/mk4tcl.h:351: candidate is: \ int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, \ char * = "option") make: *** [mk4tcl.o] Error 1 I'm using g++-2.95.4. This seems to be related to the problem I had with 2.4.4-34, but the fixes that were suggested then don't work here. -- Jim ------------------- Optional Methods ----------------------- Dr James A. Lupo (303) 423-2652 9667 Independence Dr. jalupo@attbi.com Westminster CO 80021-6845 jalupo@regis.edu From jcw@equi4.com Thu May 16 15:11:17 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4GKBFa15814; Thu, 16 May 2002 15:11:16 -0500 From: Jean-Claude Wippler To: "James A. Lupo" , Subject: Re: [Metakit] Error Compiling metakit-2.4.6-36 on Linux Date: Thu, 16 May 2002 22:08:29 +0200 Message-Id: <20020516200829.8184@triqs.com> In-Reply-To: <200205161819.g4GIJlKL006175@egor.linux.home> References: <200205161819.g4GIJlKL006175@egor.linux.home> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: James A. Lupo wrote: >I just downloaded and tried to compile metakit-2.4.6-36 on Linux. [...] > /bin/sh ./libtool --mode=compile g++ -c -O2 -I../unix/../include \ > -I/usr/include/tcl8.3/generic -I/usr/include/tcl8.3 \ > ../unix/../tcl/mk4tcl.cpp > g++ -c -O2 -I../unix/../include -I/usr/include/tcl8.3/generic \ > -I/usr/include/tcl8.3 ../unix/../tcl/mk4tcl.cpp -fPIC -DPIC \ > -o .libs/mk4tcl.o > ../unix/../tcl/mk4tcl.cpp:1480: prototype for \ > `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, char **, char *)' \ > does not match any in class `Tcl' This is a very unfortunate side-effect of the new const-ification that has been carried through in Tcl 8.4a4. The Mk4tcl codebase builds against the latest headers. You need to unpack and build (but not necessarily install) a recent Tcl 8.4a5 snapshot (i.e. http:// www.equi4.com/pub/tk/tars/tcl.tar.gz) and point MK's configure to it. The result will work on all Tcl/Tk releases from 8.1 on, it's just that the MK codebase has been adjusted to track the latest headers. I thought I had a workaround for this in there (grep for "CONST84"), but for some reason it's not kicking in... -jcw From lvirden@cas.org Mon May 20 06:54:07 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4KBs7a22312 for ; Mon, 20 May 2002 06:54:07 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g4KBs0S16855 for ; Mon, 20 May 2002 07:54:00 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g4KBrxm13280; Mon, 20 May 2002 07:53:59 -0400 (EDT) Date: Mon, 20 May 2002 07:53:59 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020520075359.AAB13264@cas.org> To: metakit@equi4.com Subject: [Metakit] Tclkit questions - what is the diff between upx vs exe and what are the vfs files? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Two questions I have deal with relatively recent additions to http://www.equi4.com/pub/tk/ - the win32 tclkit's now come in exe and upx.exe versions. What (other than minor size differences) are the differences? Secondly, what are the vfs-date.002-sh.bin and -tk.bin files? -- Support Internet Radio Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Tue May 21 15:50:18 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4LKoGa01787 for ; Tue, 21 May 2002 15:50:17 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Tclkit questions - what is the diff between upx vs exe and what are the vfs files? Date: Tue, 21 May 2002 22:42:31 +0200 Message-Id: <20020521204231.11797@triqs.com> In-Reply-To: <20020520075359.AAB13264@cas.org> References: <20020520075359.AAB13264@cas.org> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >Two questions I have deal with relatively recent additions to >http://www.equi4.com/pub/tk/ - the win32 tclkit's now come in exe and >upx.exe versions. What (other than minor size differences) are the >differences? UPX is a runtime compressor . Some people love it, others hate it, so the .exe.gz version is a plain exe, compressed for quicker downloading. One advantage of the .exe.gz versions is that they start up faster, which can be relevant for CGI use for example. >Secondly, what are the vfs-date.002-sh.bin and -tk.bin files? It's part of the new "genkit" design used to create tclkit distributions, see for details. VFS "runtimes" are versioned, this version number is available from inside tclkit as $tcl_platform(vfs). -jcw From lvirden@cas.org Wed May 22 06:50:05 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4MBo5a21741 for ; Wed, 22 May 2002 06:50:05 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g4MBnxS18370 for ; Wed, 22 May 2002 07:49:59 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g4MBnwx04154; Wed, 22 May 2002 07:49:58 -0400 (EDT) Date: Wed, 22 May 2002 07:49:58 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020522074958.AAB4130@cas.org> To: Subject: [Metakit] Determining latest code releases with respect to Tclkit/Kitten Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I was wondering this morning whether anyone had figured out the Tcl equivalent to http://www.versiontracker.com/ , etc. - some way of tracking what the latest 'versions' of programs and extensions. I notice that Kitten has the ability to update itself from a central repository - the next step would be a database on the side of the central respository that would allow 'editors' update a database with information regarding latest releases, new products, etc. Anyone besides me interested in discussing this topic? -- Support Internet Radio Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From lvirden@cas.org Wed May 22 08:25:46 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4MDPja27388 for ; Wed, 22 May 2002 08:25:45 -0500 Received: from srv13.cas.org (srv13 [134.243.65.81]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g4MDPbS19452 for ; Wed, 22 May 2002 09:25:37 -0400 (EDT) Received: (from lwv26@localhost) by srv13.cas.org (8.8.8+Sun/m4_8.8.8/CAS_CLIENT-1.16) id JAA10885; Wed, 22 May 2002 09:25:38 -0400 (EDT) Date: Wed, 22 May 2002 09:25:38 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020522092538.AAB10473@cas.org> Subject: Re: [Metakit] Tclkit questions - what is the diff between upx vs In-Reply-To: <20020521204231.11797@triqs.com> of Tue, 21 May 2002 22:42:31 +0200 To: metakit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: One idea for genkit - if it doesn't currently do this (I've not had time yet to check...) One thing I would love to see available either within tclkit or as an accompanying scripted document would be the documentation for all the code within tclkit. A similarly useful thing would be for kitten to include the documentation for all the extensions available within it. -- Support Internet Radio Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From umbright@taltrade.com Wed May 22 09:51:39 2002 Received: from talchiexch03.taltrade.com (talchiexch03.taltrade.com [207.48.80.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4MEpda32450 for ; Wed, 22 May 2002 09:51:39 -0500 Received: from mail pickup service by talchiexch03.taltrade.com with Microsoft SMTPSVC; Wed, 22 May 2002 09:51:34 -0500 x-gfisavedcharset: iso-8859-1 Content-Type: text/plain; charset="iso-8859-1" Received: from talchiexch02.taltrade.com ([10.207.64.102]) by talchiexch03.taltrade.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 22 May 2002 09:51:31 -0500 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Date: Wed, 22 May 2002 09:51:27 -0500 Message-ID: <45CE1FBFCDAD2E4DA84B49F2F866FC4F183180@talchiexch02.taltrade.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Mk4py on OS X Thread-Index: AcIBoCkppdHHdEspQzWUBf+YlKd9gA== From: "Guy Umbright" To: X-OriginalArrivalTime: 22 May 2002 14:51:31.0861 (UTC) FILETIME=[29457C50:01C201A0] Subject: [Metakit] Mk4py on OS X Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Some good things and some bad things (all on OS X 10.1.4)... Bad Thing: Under 2.4.6 "./configure --enable-python" does not=20 create a Mk4py target in the makefile. Good Thing: Under 2.4.3 I got Mk4py built and functional. One of the keys was adding "-bundle -bundle_loader = /usr/local/bin/python" to the Mk4py rule. Bad Thing: I have only managed to get it built on one of my machines. The other machine complains that python has already gone through the static linker, or some such. But I think I might have a tool version mismatch on the machines so hopefully I can find a resolution soon. When I figure out what is causing the problem between the machines and = can do a python and metakit build consistently, I will post further results. (apologies for the big stupid disclaimer) This electronic mail message and any attached files contain information = intended for the exclusive use of the individual or entity to whom it is = addressed and may contain information that is proprietary, privileged, = confidential and/or exempt from disclosure under applicable law. If you = are not the intended recipient, you are hereby notified that any = viewing, copying, disclosure or distribution of this information may be = subject to legal restriction or sanction. Please notify the sender, by = electronic mail or telephone, of any unintended recipients and delete = the original message without making any copies. From lvirden@cas.org Wed May 22 10:20:17 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4MFKGa02151 for ; Wed, 22 May 2002 10:20:16 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g4MFK8S00826 for ; Wed, 22 May 2002 11:20:08 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g4MFK8Q06793; Wed, 22 May 2002 11:20:08 -0400 (EDT) Date: Wed, 22 May 2002 11:20:08 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020522112008.AAB6573@cas.org> To: Subject: [Metakit] Future Kitten or sibling additions? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: In looking over the latest Kitten and Tclkit offerings, I noticed that a few of the extensions that we use (or at least that I'm investigating the use of) are not yet present. I don't know whether others would find these useful or not - if so, perhaps they could be added or a new offering made? BLT - great set of add on widgets and functionality Img - critical to dealing with most types of images snack - the premere tcl extension for sound manipulation cde - small script only extension to make integrading with CDE easier. effective tcl libraries - a series of potentially useful functions and applications memchan - another of Andreas Kupries useful extensions oratcl - one of several vendor specific database interfaces... too bad there isn't a more generic way to do databases tcldom - another of the tclxml suite of tools tclreadline - provides the user the ability to have interactive command line history within a tcl/wish interpreter tcltidy - tool to analyze html/xml , reporting errors or even fixing the code tclvfs and specific vfs filesystem extensions - I know there is some vfs code in tclkit - but where's the stuff like http, url, ftp, zip, webdav, etc. support? Is it in there somehwere but not found. tclx - a variety of useful and, in some cases, necessary, functions tclxslt - tcl binding for libxslt tkdnd - drag and drop support tklib - currently only one module here - cursor - but there's always hope for the future tls - ssl binding for tcl TrfCrypt - various encryption routines units - convert between units vu or vuwidgets - several useful widgets (pie and bar chart, etc.) hobbes widgets - a tcl only megawidget structure with various megawidgets built There are probably others : tkpiechart, scwoop, Agentk, critcl, Andreas Kupries set and listx commands, animated gif support, various useful procs from the wikit, SleepyCat's DB binding, Neil Madden's browser wrapper for TkHTML, canvas support for icons, edges, and graphs, the canvas visitor extension, Combat (CORBA broker interfacE), Iain's Containers extension (adds support for bags, queues, trees, priority queues, random queues, structs, stacks, hashes FIFO, LIFO, etc. some of George Staplin's code like the text widget overload that provides named highlighting, copy and paste support, etc. Bonnet's dictionary support, Wow - I had forgotten how much good stuff is out there on the net for Tcl and Tk! Obviously this stuff isn't all going into one Kitten file. I wonder if what we really need are a family of kittens (persian, calico, etc.) each one of which focuses on a particular domain of problem solution - extra widgets, tcl data structures, etc. Sigh - so much to do - so few brain cells to burn out trying to figure the best approach. all come to mind as things that I've thought about but haven't gotten to yet. -- Support Internet Radio Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From umbright@taltrade.com Thu May 23 09:37:52 2002 Received: from talchiexch03.taltrade.com (talchiexch03.taltrade.com [207.48.80.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4NEbpa24425 for ; Thu, 23 May 2002 09:37:52 -0500 Received: from mail pickup service by talchiexch03.taltrade.com with Microsoft SMTPSVC; Thu, 23 May 2002 09:37:51 -0500 x-gfisavedcharset: iso-8859-1 Content-Type: text/plain; charset="iso-8859-1" Received: from talchiexch02.taltrade.com ([10.207.64.102]) by talchiexch03.taltrade.com with Microsoft SMTPSVC(5.0.2195.2966); Thu, 23 May 2002 09:37:48 -0500 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Date: Thu, 23 May 2002 09:37:42 -0500 Message-ID: <45CE1FBFCDAD2E4DA84B49F2F866FC4F183183@talchiexch02.taltrade.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Metakit] Mk4py on OS X Thread-Index: AcIBscIPlsDNBA+ORGS0RpyRvT9U2gAtIUkQ From: "Guy Umbright" To: X-OriginalArrivalTime: 23 May 2002 14:37:48.0257 (UTC) FILETIME=[68C73D10:01C20267] Subject: [Metakit] Mk4py on OS X - Got it working! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I managed to get Mk4py 2.4.6 built and working on OS X 10.1.4 last = night. 1) I changed -shared to -dynamiclib (on line 6022) in libtool (libmk4 = would not link until I did this) 2) I added the 'bundle' parms to the libmk4py.la rule, but it does not = appear=20 to get passed through, so I do add a -v on the libtool for libmk4py, I = get: c++ -dynamiclib .libs/PyProperty.o .libs/PyRowRef.o .libs/PyStorage.o=20 .libs/PyView.o .libs/PWOImp.o .libs/column.o .libs/custom.o=20 .libs/derived.o .libs/fileio.o .libs/field.o .libs/format.o=20 .libs/handler.o .libs/persist.o .libs/remap.o .libs/std.o .libs/store.o=20 .libs/string.o .libs/table.o .libs/univ.o .libs/view.o .libs/viewx.o =20 -lc -o .libs/libmk4py.so which still fails (no surpise there). So I change it to remove=20 -dynamiclib (seemingly counterproductively as I caused it to be added by = #1 above)=20 & add the bundle stuff, so it looks like this: c++ -v -bundle -bundle_loader /usr/local/bin/python .libs/PyProperty.o = .libs/PyRowRef.o .libs/PyStorage.o .libs/PyView.o .libs/PWOImp.o=20 .libs/column.o .libs/custom.o .libs/derived.o .libs/fileio.o=20 .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o=20 .libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o=20 .libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4py.so which generates an ld of: /usr/bin/ld -arch ppc -bundle_loader /usr/local/bin/python -o=20 .libs/libmk4py.so -dynamic -bundle -lbundle1.o .libs/PyProperty.o=20 .libs/PyRowRef.o .libs/PyStorage.o .libs/PyView.o .libs/PWOImp.o=20 .libs/column.o .libs/custom.o .libs/derived.o .libs/fileio.o=20 .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o=20 .libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o=20 .libs/univ.o .libs/view.o .libs/viewx.o -lstdc++ -lc -lcc_dynamic=20 -lSystem which works! I rename copy libmk4py.so to Mk4py.so and copy it to site-packages=20 (along with the metakit.py file) and run demo.py and I am golden. Unconventional approach perhaps, but it works. And now, in theory, if=20 we know what those lower level invocations have to look like, someone=20 should be able to coerce it happen via the higher level tools. I will see if I can figure out how to do it (again a useful learning = experience) but as I now have a working version, it has to be lower priority for me = as I have some other issues pressing.=20 This electronic mail message and any attached files contain information = intended for the exclusive use of the individual or entity to whom it is = addressed and may contain information that is proprietary, privileged, = confidential and/or exempt from disclosure under applicable law. If you = are not the intended recipient, you are hereby notified that any = viewing, copying, disclosure or distribution of this information may be = subject to legal restriction or sanction. Please notify the sender, by = electronic mail or telephone, of any unintended recipients and delete = the original message without making any copies. From lvirden@cas.org Fri May 24 09:37:10 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4OEbAa31063 for ; Fri, 24 May 2002 09:37:10 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g4OEb4S17203 for ; Fri, 24 May 2002 10:37:04 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g4OEb0J10327; Fri, 24 May 2002 10:37:00 -0400 (EDT) Date: Fri, 24 May 2002 10:37:00 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020524103659.AAB9701@cas.org> To: Subject: [Metakit] Kitten question - how does one go about using Iwidgets? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: When I am using a normal wish, I can say: package require Iwidgets and that package loads. When I am using tclkit, I am getting an error. Here's what I see: Note that I have a current version of tclkit and kitten (I don't know how to report to you _what_ version though...). This is on SPARC Solaris. The two files are installed in /volws/lwv26/ldatae/bin . $ cat /home/lwv26/tst.kitten #! /volws/lwv26/ldatae/bin/tclkit package require scripdoc mk4vfs::mount /volws/lwv26/ldatae/bin/kitten.bin /volws/lwv26/ldatae/bin/kitten.bin -readonly scripdoc::extendPath /volws/lwv26/ldatae/bin/kitten.bin package require Iwidgets # ---------------------------------------------------------------------- option add *Calendar.buttonForeground black option add *Calendar.outline black option add *Calendar.weekdayBackground white option add *Calendar.weekendBackground mistyrose option add *Calendar.selectColor red . configure -background white iwidgets::calendar .cal pack .cal -expand yes -fill both iwidgets::optionmenu .days -labeltext "Start Day:" -command { set day [.days get] .cal configure -startday $day -days $caldays($day) } pack .days -padx 4 -pady 4 array set caldays { sunday {S M T W T F S} monday {Mo Tu We Th Fr Sa Su} } .days insert end sunday monday $ chmod 755 /home/lwv26/tst.kitten $ /home/lwv26/tst.kitten can't find package Itk 3.2 while executing "package require Itk 3.2" (file "/volws/lwv26/ldatae/bin/kitten.bin/lib/iwidgets3.0.1/iwidgets.tcl" line 19) invoked from within "source /volws/lwv26/ldatae/bin/kitten.bin/lib/iwidgets3.0.1/iwidgets.tcl" ("package ifneeded" script) invoked from within "package require Iwidgets" (file "/home/lwv26/tst.kitten" line 7) -- Support Internet Radio Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From mdvorak@ninell.cz Tue May 28 10:23:25 2002 Received: from mail.thundernet.cz (mail.thundernet.cz [62.77.87.114]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g4SFNPa09263 for ; Tue, 28 May 2002 10:23:25 -0500 Received: (qmail 10692 invoked from network); 28 May 2002 15:22:42 -0000 Received: from unknown (HELO smudla) (62.77.87.74) by mail.thundernet.cz with SMTP; 28 May 2002 15:22:42 -0000 From: "Martin Dvorak" To: Date: Tue, 28 May 2002 17:22:24 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-Mimeole: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] opening datafile Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, I am new to MetaKit so please forgive me if this question is anyhow silly. I wonder what is the correct way to open MetaKit datafiles. I need following behaviour: - if the datafile exists, attempt to open it and return result - if the datafile does not exist, attempt to create it and return result According to examples I though the correct way is to use CFileStrategy::DataOpen(filename, 1). But the behaviour of this method is somewhat weird: - try to open file and return true on success - if the file cannot be opened, try to create it and return false This behaviour seems incorrect to me, because: - if the file exists and there is some other reason why it cannot be opened, the method still tries to create the file - if the file does not exist but is created successfully, the return value is false Is this a bug or intentional behaviour? Martin From jcw@equi4.com Wed May 29 01:44:29 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4T6iSa31710 for ; Wed, 29 May 2002 01:44:29 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] opening datafile Date: Wed, 29 May 2002 08:39:33 +0200 Message-Id: <20020529063933.18991@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Martin Dvorak wrote: >MetaKit datafiles. I need following behaviour: >- if the datafile exists, attempt to open it > and return result >- if the datafile does not exist, attempt to > create it and return result > >According to examples I though the correct way >is to use CFileStrategy::DataOpen(filename, 1). Don't access strategy routines directly. Use the c4_Storage class to create objects to represent an open MK datafile. >But the behaviour of this method is somewhat >weird: >- try to open file and return true on success >- if the file cannot be opened, try to create > it and return false > >This behaviour seems incorrect to me, because: >- if the file exists and there is some other reason > why it cannot be opened, the method still tries > to create the file >- if the file does not exist but is created > successfully, the return value is false > >Is this a bug or intentional behaviour? Intentional, if not quite obvious, I admit. The way to check whether things are indeed in order, is to check "storage.Strategy().IsValid()". -jcw From jcw@equi4.com Wed May 29 01:47:48 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4T6lla31899 for ; Wed, 29 May 2002 01:47:47 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Kitten question - how does one go about using Iwidgets? Date: Wed, 29 May 2002 08:47:43 +0200 Message-Id: <20020529064743.11106@triqs.com> In-Reply-To: <20020524103659.AAB9701@cas.org> References: <20020524103659.AAB9701@cas.org> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >When I am using a normal wish, I can say: >package require Iwidgets >and that package loads. > >When I am using tclkit, I am getting an error. [...] >$ /home/lwv26/tst.kitten >can't find package Itk 3.2 The IncrTcl package recently went from 3.2 to 3.3 - it looks like there is a mixup, given than incrtcl 3.3 is in the latest tclkit builds. >Note that I have a current version of tclkit and kitten (I don't know >how to report to you _what_ version though...). Well... let me reiterate then: the way to unambiguously determine the TclKit release number, is puts $tcl_platform(vfs) The version of Kitten is the concatenation of all version numbers of packages in it (i.e. "kitten -l"). There is no additional version numbering :) -jcw From jcw@equi4.com Fri May 31 02:43:09 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4V7h8a28746; Fri, 31 May 2002 02:43:08 -0500 From: Jean-Claude Wippler To: announce , metakit Date: Fri, 31 May 2002 09:42:45 +0200 Message-Id: <20020531074245.3977@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.7 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is to announce availability of a new minor release. MetaKit 2.4.7 fixes some bugs, in the code and in the build process. There is a source code archive in the usual place - no binaries have been created. If you don't want to compile, you can stick with the 2.4.6 binaries available for Linux/MacOS/Win32. For details, see http://www.equi4.com/metakit/. -jcw From jcw@equi4.com Fri May 31 03:08:46 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4V88ja29873 for ; Fri, 31 May 2002 03:08:45 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Future Kitten or sibling additions? Date: Fri, 31 May 2002 09:55:12 +0200 Message-Id: <20020531075512.2290@triqs.com> In-Reply-To: <20020522112008.AAB6573@cas.org> References: <20020522112008.AAB6573@cas.org> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >In looking over the latest Kitten and Tclkit offerings, I noticed that >a few of the extensions that we use (or at least that I'm investigating >the use of) are not yet present. I don't know whether others would find >these useful or not - if so, perhaps they could be added or a new >offering made? [big list] >There are probably others : [another list] >Obviously this stuff isn't all going into one Kitten file. >I wonder if what we really need are a family of kittens (persian, calico, >etc.) each one of which focuses on a particular domain of problem solution >- extra widgets, tcl data structures, etc. > >Sigh - so much to do - so few brain cells to burn out trying to figure >the best approach. There is a substantial work and thinking going on behind the scenes to deal with this. Kitten is already bulging as a 2.5 Mb download, and indeed only scratching the surface (and personal itches) of what repositories could be. It's self-updating over the web, so the real issues are what to put in, how to deal with subsets (functional/ platform), what to do with docs, examples, revisions, and so on. I hope to be able to provide more info at the Euro Tcl conference end of next month in Munich. Kitten is just a way to get some stuff out now. For now Kitten evolution will be an "as time permits" thing, which is a sneaky way of saying: don't hold your breath just yet :) -jcw From jcw@equi4.com Fri May 31 03:08:48 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4V88ka29877 for ; Fri, 31 May 2002 03:08:47 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Error Compiling metakit-2.4.6-36 on Linux Date: Fri, 31 May 2002 09:56:20 +0200 Message-Id: <20020531075620.805@triqs.com> In-Reply-To: <200205161819.g4GIJlKL006175@egor.linux.home> References: <200205161819.g4GIJlKL006175@egor.linux.home> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: James A. Lupo wrote: >I just downloaded and tried to compile metakit-2.4.6-36 on Linux. >The steps I followed were: > > cd builds > ../unix/configure --with-tcl=/usr/include/tcl8.3 > make > >This failed with the error: > > /bin/sh ./libtool --mode=compile g++ -c -O2 -I../unix/../include \ > -I/usr/include/tcl8.3/generic -I/usr/include/tcl8.3 \ > ../unix/../tcl/mk4tcl.cpp > g++ -c -O2 -I../unix/../include -I/usr/include/tcl8.3/generic \ > -I/usr/include/tcl8.3 ../unix/../tcl/mk4tcl.cpp -fPIC -DPIC \ > -o .libs/mk4tcl.o > ../unix/../tcl/mk4tcl.cpp:1480: prototype for \ > `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, char **, char *)' \ > does not match any in class `Tcl' > ../unix/../tcl/mk4tcl.h:351: candidate is: \ > int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, \ > char * = "option") > > make: *** [mk4tcl.o] Error 1 > >I'm using g++-2.95.4. This seems to be related to the problem I had with >2.4.4-34, but the fixes that were suggested then don't work here. This has been fixed in 2.4.7 (you can also get it from CVS). -jcw From jcw@equi4.com Fri May 31 03:08:49 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4V88ma29882 for ; Fri, 31 May 2002 03:08:49 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Duplicate and notifications Date: Fri, 31 May 2002 09:58:45 +0200 Message-Id: <20020531075845.1913@triqs.com> In-Reply-To: <00ca01c1fc29$66a25d60$a43428c4@tony> References: <00ca01c1fc29$66a25d60$a43428c4@tony> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tony Mann wrote: >* What is the time/space cost of the c4_View::Duplicate() method? Is it >reasonable to use this on a view with 10K records and 10 string columns? I would think so. It's really impossible to answer, because it depends on *your* idea of "reasonable"... >* I am using SortOn to create a derived view. However, I do not want this >derived view to change as the underlying data is changed. That is, changing >the sort key of an entry should not change its order in the derived view. Is >there a way to turn off the notifications for a view? Or would this cause >other problems? Right now I Duplicate() the derived view to disconnect the >notifications. Duplicate is good. I think it does a "newview.InsertAt(0, view)" internally. -jcw From jcw@equi4.com Fri May 31 03:08:51 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4V88oa29886 for ; Fri, 31 May 2002 03:08:50 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] [Q] Cannot compile on Solaris 8 Date: Fri, 31 May 2002 10:00:12 +0200 Message-Id: <20020531080012.16339@triqs.com> In-Reply-To: <1021011528.3cdb664820c1b@pike.singnet.com.sg> References: <1021011528.3cdb664820c1b@pike.singnet.com.sg> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Chan Chi Hung wrote: >I am trying to compile Mk4tcl and encountered configuration >problem. Is there any extra flag that I should turn on and >what will be the correct value for my environment. I include >my steps in configuring Mk4tcl. [...] >$ make >/bin/sh ./libtool --mode=compile g++ -c -O2 -I../unix/../include -I/ >usr/local2/include/generic -I/usr/local2/include ../tcl/mk4tcl.cpp >g++ -c -O2 -I../unix/../include -I/usr/local2/include/generic -I/usr/ >local2/include ../tcl/mk4tcl.cpp -fPIC -DPIC -o .libs/mk4tcl.o >../tcl/mk4tcl.cpp:1480: prototype for `int >Tcl::tcl_GetIndexFromObj(Tcl_Obj *, char **, char *)' does not match any >in class `Tcl' >../tcl/mk4tcl.h:351: candidate is: int Tcl::tcl_GetIndexFromObj(Tcl_Obj >*, const char **, char * = "option") >*** Error code 1 >make: Fatal error: Command failed for target `mk4tcl.o' This has been fixed in 2.4.7 (you can also get it from CVS). -jcw From jcw@equi4.com Fri May 31 03:08:53 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4V88qa29890 for ; Fri, 31 May 2002 03:08:52 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Problems with HP-UX Date: Fri, 31 May 2002 10:03:01 +0200 Message-Id: <20020531080301.12654@triqs.com> In-Reply-To: <3CDA84AE.DFF8AD30@pinebush.com> References: <3CDA84AE.DFF8AD30@pinebush.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Chris Nelson wrote: >I have an application that uses MetaKit to store data in a structured >file. It's built on HP-UX 10.20. When run on HP-UX 10.20, it fails and >the file is corrupted. When run on HP-UX 11.00, it works fine and the >file is intact. The application works fine on all versions of Solaris >that I've tried (2.6 and 2.8, I think). I'm digging with debuggers and >such but if this sounds familiar to anyone, I could use a clue. Have you run the regression tests? ("make test" will do it) Please also make sure to use MK 2.4.6 or now 2.4.7, they fix some bugs. -jcw From jcw@equi4.com Fri May 31 03:08:55 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4V88sa29893 for ; Fri, 31 May 2002 03:08:54 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Bug in mk4dll.h Date: Fri, 31 May 2002 10:03:15 +0200 Message-Id: <20020531080315.23345@triqs.com> In-Reply-To: <1797E7C8F378D411A1CE00D0B77E59CE572FA2@gilianmail> References: <1797E7C8F378D411A1CE00D0B77E59CE572FA2@gilianmail> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Fabien Benichou wrote: >Hello, >I found that In Metakit version 2.4.5, the two following lines are missing >in the file mk4dll.h: > > d4_DLLSPEC(bool) operator== (const c4_Reference& a_, const c4_Reference& >b_); > d4_DLLSPEC(bool) operator!= (const c4_Reference& a_, const c4_Reference& >b_); Now in 2.4.7, thanks (with myapologies for taking so long). -jcw From jcw@equi4.com Fri May 31 03:08:56 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g4V88ta29899 for ; Fri, 31 May 2002 03:08:55 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Using streams Date: Fri, 31 May 2002 10:07:47 +0200 Message-Id: <20020531080747.29641@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas Olsson wrote: (Whoops, I never responded to this ... my apologies). >I have a small program that uses MetaKit. I now need to change it a bit so >that it can use streams. > >I replaced c4_Storage *store = new c4_Storage("filename") This storage object knows about "filename". >with >c4_Storage *store = new c4_Storage(); store->LoadFrom(mystream); But this one doesn't. >It works fine to load the stream, but all changes are lost when I destroy >the storage object (even if AutoCommit is true). > >Doesn't commit work on streams? You can't commit on a storage object which has no file associated with it. Either leave out the second "new", or use "saveto" to get the current storage contents to a specified file. -jcw From epsilonzero0@yahoo.com Sat Jun 1 01:09:05 2002 Received: from web20903.mail.yahoo.com (web20903.mail.yahoo.com [216.136.226.225]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g51695a18675 for ; Sat, 1 Jun 2002 01:09:05 -0500 Message-ID: <20020601060904.24141.qmail@web20903.mail.yahoo.com> Received: from [24.67.21.28] by web20903.mail.yahoo.com via HTTP; Fri, 31 May 2002 23:09:04 PDT Date: Fri, 31 May 2002 23:09:04 -0700 (PDT) From: Tom Johnes To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] view.select on subviews Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Good day. I am new to metakit. I am thinking of using it for a project. I am playing with the Python bindings and I am trying to figure out how to search a view with subviews. For example, import metakit db = metakit.storage() vw = db.getas("people[first:S,last:S,phones[number:S]]") vw.append(first='John',last='Wayne') vw[0].phones.append(number='111-555-1111') vw[0].phones.append(number='111-555-1212') vw.append(first='Bart',last='Gordon') vw[1].phones.append(number='111-555-1313') vw[1].phones.append(number='111-555-1414') for r in vw.select(phones.number='111-555-1212'): print r.first, r.last This does not work because of the "phones.number='111-555-1212'" syntax. I've also tried "phones[number='111-555-1212']". What is the right syntax to search a subview property in a view? __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com From gmcm@hypernet.com Sat Jun 1 09:27:25 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g51EROa31078 for ; Sat, 1 Jun 2002 09:27:25 -0500 Received: from PARANOIA (204.176.40.46) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sat, 1 Jun 2002 10:26:39 -0400 From: "Gordon McMillan" To: Tom Johnes , metakit@equi4.com Date: Sat, 1 Jun 2002 10:24:35 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] view.select on subviews Reply-to: gmcm@hypernet.com Message-ID: <3CF8A0E3.10196.4BDB4C34@localhost> Priority: normal In-reply-to: <20020601060904.24141.qmail@web20903.mail.yahoo.com> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 31 May 2002 at 23:09, Tom Johnes wrote: > Good day. I am new to metakit. I am thinking of > using it for a project. I am playing with the Python > bindings and I am trying to figure out how to search > a view with subviews. > > For example, > > import metakit > db = metakit.storage() > vw = > db.getas("people[first:S,last:S,phones[number:S]]") > vw.append(first='John',last='Wayne') > vw[0].phones.append(number='111-555-1111') > vw[0].phones.append(number='111-555-1212') > vw.append(first='Bart',last='Gordon') > vw[1].phones.append(number='111-555-1313') > vw[1].phones.append(number='111-555-1414') flat = vw.flatten(vw.phones) flat.select(number='111-555-1212') Note that flatten produces a viewer - no actual data is touched to produce it. -- Gordon http://www.mcmillan-inc.com/ From epsilonzero0@yahoo.com Sat Jun 1 10:50:37 2002 Received: from web20909.mail.yahoo.com (web20909.mail.yahoo.com [216.136.226.231]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g51Foba00915 for ; Sat, 1 Jun 2002 10:50:37 -0500 Message-ID: <20020601155036.5079.qmail@web20909.mail.yahoo.com> Received: from [24.67.21.28] by web20909.mail.yahoo.com via HTTP; Sat, 01 Jun 2002 08:50:36 PDT Date: Sat, 1 Jun 2002 08:50:36 -0700 (PDT) From: Tom Johnes Subject: Re: [Metakit] view.select on subviews To: gmcm@hypernet.com, metakit@equi4.com In-Reply-To: <3CF8A0E3.10196.4BDB4C34@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > flat = vw.flatten(vw.phones) > flat.select(number='111-555-1212') > > Note that flatten produces a viewer - no actual > data is touched to produce it. Gordon, While that works for the example I does not produce the result I had in mind. Let me explain the problem I am trying to tackle briefly. I'd like to create a database of recipies. I was thinking that each row could have a subview with a list of ingredients. Given a set of ingredients I would like to search the view for all recipies that exactly those ingredients. I was hoping I could select on a subview for the same property multiple times and that it would return any rows that had subviews that matched all ingredients. E.g.: recipies.select(ingredients='tomatos',ingredients='onions') but this does not work. If it did I could combine this with a check for the number of ingredients to do an exact match. A further problem is some ingredients have alternatives, thus the properties in the ingredients subview would be subviews themselves. I guess I would get around all this by using the filter function to walk down the view and pick out the right recipies but I wonder if this would be inneficient. I am wondering whether I could approach the problem with MkSQL and subselect. __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com From gmcm@hypernet.com Sat Jun 1 11:23:52 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g51GNqa01898 for ; Sat, 1 Jun 2002 11:23:52 -0500 Received: from PARANOIA (204.176.40.46) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sat, 1 Jun 2002 12:23:40 -0400 From: "Gordon McMillan" To: Tom Johnes Date: Sat, 1 Jun 2002 12:24:25 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] view.select on subviews Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3CF8BCF9.887.4C49018D@localhost> Priority: normal In-reply-to: <20020601155036.5079.qmail@web20909.mail.yahoo.com> References: <3CF8A0E3.10196.4BDB4C34@localhost> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 1 Jun 2002 at 8:50, Tom Johnes wrote: [snip] > I'd like to create a database of recipies. I was > thinking that each row could have a subview with a > list of ingredients. Given a set of ingredients I > would like to search the view for all recipies that > exactly those ingredients. [snip] > A further problem is some ingredients have > alternatives, thus the properties in the ingredients > subview would be subviews themselves. > > I guess I would get around all this by using the > filter function to walk down the view and pick out > the right recipies but I wonder if this would be > inneficient. The semantics of your search are such that I'd worry about correctness, first. What if in one recipe, green onions are an alternative for leeks, but not in the other? > I am wondering whether I could approach the problem > with MkSQL and subselect. MkSQL doesn't know what a subview is. If it did, it would almost certainly generate a filter (since that's what it generates for most queries). -- Gordon http://www.mcmillan-inc.com/ From epsilonzero0@yahoo.com Sun Jun 2 21:00:03 2002 Received: from web20910.mail.yahoo.com (web20910.mail.yahoo.com [216.136.226.232]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g53203a29541 for ; Sun, 2 Jun 2002 21:00:03 -0500 Message-ID: <20020603020002.63157.qmail@web20910.mail.yahoo.com> Received: from [24.67.21.28] by web20910.mail.yahoo.com via HTTP; Sun, 02 Jun 2002 19:00:02 PDT Date: Sun, 2 Jun 2002 19:00:02 -0700 (PDT) From: Tom Johnes To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] strange behaviour of view.minus under Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: While experimenting with python and metakit I wrote the following sample program: import metakit db = metakit.storage() recipes = db.getas("recipes[name:S,ingredients[ingredient:S]]") i = recipes.append(name='bloody mary') recipes[i].ingredients.append(ingredient='tabasco sauce') recipes[i].ingredients.append(ingredient='vodka') recipes[i].ingredients.append(ingredient='tomato juice') i = recipes.append(name='cosmopolitan') recipes[i].ingredients.append(ingredient='vodka') recipes[i].ingredients.append(ingredient='cramberry juice') recipes[i].ingredients.append(ingredient='lime juice') # search pl = [ metakit.property('S','ingredient') ] have = [ { 'ingredient' : 'vodka' }, { 'ingredient' : 'cramberry juice' }, { 'ingredient' : 'tabasco sauce' }, { 'ingredient' : 'lime juice' }, { 'ingredient' : 'tomato juice' } ] have_ingredients = metakit.wrap( have, pl ) for r in recipes: left = r.ingredients.minus( have_ingredients ); if len( r.ingredients.minus( have_ingredients ) ) == 0: print 'FOUND', r.name This program results in the following output: FOUND bloody mary FOUND cosmopolitan Yes, if I change the section of the program that creates the have_ingredients view so that the "tabasco sauce" entry is last, the program fails to find "bloody mary". have = [ { 'ingredient' : 'vodka' }, { 'ingredient' : 'cramberry juice' }, { 'ingredient' : 'lime juice' }, { 'ingredient' : 'tomato juice' }, { 'ingredient' : 'tabasco sauce' } ] results in: FOUND cosmopolitan For some reason the tabasco sauce entry is not being matches by the view.minus method. The returned view that a row with "tabasco sauce" while it should be empty. Any idea what is going on here? __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com From jcw@equi4.com Tue Jun 4 10:58:03 2002 Received: from [10.0.1.187] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g54Fvxa21389; Tue, 4 Jun 2002 10:58:00 -0500 From: Jean-Claude Wippler To: announce , metakit Cc: Tom Krehbiel , "Daniel A. Steffen" , Reinhard Max Date: Tue, 4 Jun 2002 17:57:55 +0200 Message-Id: <20020604155755.20604@triqs.com> X-Mailer: CTM PowerMail 3.1.2 carbon MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] TclKit 8.4-37 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The June release of TclKit is out, based on the latest CVS snapshots of all included components: Tcl/Tk 8.4a5 (solid, ignore the "a5"), Metakit 2.4.7, IncrTcl 3.3, and TclVfs 1.0. The goal for tclkit is to stabilize, there will be less and less changes to it from now on - other than tracking stable releases of its various subsystems. Recent changes listed at are: - fix: clean up shared libs from /tmp after load/use - removed old http 1.0 package (2.x has been around for ages) - tweaks to streamline builds a bit more There seems to be one regression in this latest build: the Unicode encoding system is not being set up properly on all Unix'es (win32 and macosx/darwin encodings are set ok, macosx is just a standard Unix nowadays, go figure...). The encoding issue can be verified as follows: if "encoding system" returns "identity", then it has not been initialized, it usually will be "iso8859-1". Any help with this would be deeply appreciated - I've been struggling with Unicode encoding setup in Tcl for *ages* :( Note that you might not see any encoding problems at all if you also have Tcl installed in some standard place on your system. For pointers to source and lots of binaries, see the TclKit home page: For a recent "classic" MacOS (7.5.5 - 9.2) build, see: It has no incrtcl yet, but apart from that it is complete and - being a "fat" binary app - it runs on 68K as well as PPC platforms. It's going to be hard for me to keep up with rebuilds on an ongoing basis, even though it's all more or less streamlined, based on a "genkit" scripts (see the homepage). If you have access to other platforms, or are willing to rebuild the latest for your system, you could help out a lot: do the "genkit shuffle", then send me your resulting "install/" and "out/" directories, and I will finish the job, build a distribution, and include it in current set on the web (basically it's a matter of appending the VFS runtime data). Enjoy, -jcw From steve@digital-smarties.com Tue Jun 4 20:22:11 2002 Received: from digital-smarties.com (IDENT:qmailr@gw.digital-smarties.com [203.59.226.186]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g551M8a30224 for ; Tue, 4 Jun 2002 20:22:09 -0500 Received: (qmail 19225 invoked by uid 8); 5 Jun 2002 01:22:05 -0000 Received: from ibook.digital-smarties.com (192.168.1.3, claiming to be "[0.0.0.0]") by gateway.digital-smarties.com with SMTP id smtpdq4afKI; Tue, 04 Jun 2002 21:21:57 EDT From: Steve Landers To: announce , metakit Subject: Re: [Metakit] TclKit 8.4-37 Date: Wed, 5 Jun 2002 09:21:55 +0800 Message-Id: <20020605012156.2597@mail.digital-smarties.com> In-Reply-To: <20020604155755.20604@triqs.com> References: <20020604155755.20604@triqs.com> X-Mailer: CTM PowerMail 3.1.2 carbon MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, Jean-Claude wrote ... >It's going to be hard for me to keep up with rebuilds on an ongoing >basis, even though it's all more or less streamlined, based on a "genkit" >scripts (see the homepage). If you have access to other platforms, or >are willing to rebuild the latest for your system, you could help out a >lot: do the "genkit shuffle", then send me your resulting "install/" and >"out/" directories, and I will finish the job, build a distribution, and >include it in current set on the web (basically it's a matter of >appending the VFS runtime data). Can I take this opportunity to encourage others to get involved. Now that the build process is streamlined (via genkit) it doesn't take a lot of time to do a couple of platform builds (I do the Solaris Sparc and x86) each month. And you don't even need to have the specific platform available - Sourceforge and others offer compile farms where you can compile Open Source software without charge (JCW uses this for some of the existing TclKit platforms). If you are interested in helping out by investing a few hours per month, can you contact either JCW or myself and we'll make sure that you have all the info you need. We'll all benefit from your help, having the comfort of knowing that TclKit is available on such a broad range of platforms. Thanks in anticipation Steve -- Steve Landers Scripting Design Studio Digital Smarties steve@digital-smarties.com Perth, Western Australia www.digital-smarties.com From imadjunk@hotmail.com Wed Jun 5 09:56:29 2002 Received: from hotmail.com (oe32.law10.hotmail.com [64.4.14.89]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g55EuTa08736 for ; Wed, 5 Jun 2002 09:56:29 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 5 Jun 2002 07:56:24 -0700 X-Originating-IP: [207.180.130.87] From: "Imad Jureidini" To: Date: Wed, 5 Jun 2002 10:56:22 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Message-ID: X-OriginalArrivalTime: 05 Jun 2002 14:56:24.0117 (UTC) FILETIME=[2940B650:01C20CA1] Subject: [Metakit] Metakit file doesn't shrink Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I've just started using Metakit, and so far I like it a lot. It's fast and powerful. I've encountered a small problem however. I have a metakit file in which I need to add and remove rows in a sorted view. This view is itself contained in another view. These operations work as expected, but the file doesn't shrink when I remove rows. I'm using Metakit 2.4.7-37 on Win32 using the MFC build. It's important that I limit the size of the metakit file, and that the file be able to shrink when data is removed because storage space will be limited. For this reason, I cannot dump the database into a new file in order to compact everything, since that would double the space requirement. Thanks for any suggestions, Imad Jureidini Note: This is unrelated, but I was running into a problem earlier which I resolved and might be useful as part as some sort of FAQ. When using the c4_View::Search() function to keep a view sorted, the properties used to sort the view must appear first in the view description. I was pulling my hair out for a while because of this. You veterans probably know this already, but hopefully it can save some time for a new user like me. From jcw@equi4.com Thu Jun 6 17:26:50 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g56MQna07729 for ; Thu, 6 Jun 2002 17:26:49 -0500 From: Jean-Claude Wippler To: metakit Date: Fri, 7 Jun 2002 00:26:14 +0200 Message-Id: <20020606222615.7361@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] ReadKit 1.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There's a new Tcl script called "readkit", which lets you list or extract the contents of any scripted document without requiring MetaKit or TclKit. The script contains a little MetaKit file format decoder in pure Tcl - though zlib support (either the Trf or Zlib extensions will do) is needed to be able to unpack compressed files. The aim of ReadKit is to offer a fallback option to get at data stored as scripted documents, even if MetaKit and Tclkit are not available. In fact, if you examine the script closely, you will see that it contains a pretty complete read-only emulation of the basic Mk4tcl commands, and that this same code could be used as basis for reading any data stored in MetaKit datafiles. The ReadKit script is listed on the scripted document home page: http://www.equi4.com/scripdoc/ The latest code has only been tested on Linux so far, but it should run on any platform with tclsh8.0 or later installed. This script has also been added to the Tclkit CVS repository. Comments, bug reports, fixes, patches - all feedback welcome. -jcw From jcw@equi4.com Thu Jun 6 17:50:57 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g56Mora08969 for ; Thu, 6 Jun 2002 17:50:55 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Metakit file doesn't shrink Date: Fri, 7 Jun 2002 00:50:11 +0200 Message-Id: <20020606225011.26217@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Imad Jureidini wrote: > I've just started using Metakit, and so far I like it a lot. It's fast >and powerful. I've encountered a small problem however. I have a metakit >file in which I need to add and remove rows in a sorted view. This view is >itself contained in another view. These operations work as expected, but >the file doesn't shrink when I remove rows. I'm using Metakit 2.4.7-37 on >Win32 using the MFC build. > It's important that I limit the size of the metakit file, and that the >file be able to shrink when data is removed because storage space will be >limited. For this reason, I cannot dump the database into a new file in >order to compact everything, since that would double the space requirement. Right now - there is no other option. Note that even though this takes space for holding the temp file while you create a compacted copy, the total space use of MetaKit remains relatively low - with strings less so, but with numeric data you will find that MK datafiles are surprisingly compact. There is a possibility to implement in-place compaction, but the worst case scenario would probably not be different from streaming to a second file and then replacing the original. The reason for thsi being that even during compaction, fail-safety must be guaranteed at all times, even when pulling the power plug. This requires a "stable storage" design, which requires potentially twice the amount of used storage (more in fact, due to fragmentation). Thanks for the note on search - yes, it's important to keep the implicit property order in mind. -jcw From jtate@mi-corporation.com Fri Jun 7 13:16:46 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g57IGja02493 for ; Fri, 7 Jun 2002 13:16:45 -0500 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g57IGcN12563 for ; Fri, 7 Jun 2002 14:16:38 -0400 From: "Joseph Tate" To: "MetaKit" Date: Fri, 7 Jun 2002 14:16:44 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] database dump utilities Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Anybody have a metakit database dump utility? I'm somehow getting a corrupt database that causes a crash, and I'd like to take a look at what's inside it. If nobody's got one, I'll just whip one up. Joseph From brett_schwarz@yahoo.com Fri Jun 7 13:41:40 2002 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g57Ifea03696 for ; Fri, 7 Jun 2002 13:41:40 -0500 Received: from 12-230-157-105.client.attbi.com (HELO thor.local) (brett?schwarz@12.230.157.105 with plain) by smtp.mail.vip.sc5.yahoo.com with SMTP; 7 Jun 2002 18:41:38 -0000 Subject: Re: [Metakit] database dump utilities From: Brett Schwarz To: Joseph Tate Cc: MetaKit In-Reply-To: References: Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.5 Date: 07 Jun 2002 11:40:39 -0700 Message-Id: <1023475240.2047.38.camel@thor> Mime-Version: 1.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What platform are you on? If you are on *nix, then you can use 'strings' to get a rough dump of the database file; i.e. strings mydb.mk (goes to stdout) HTH, --brett On Fri, 2002-06-07 at 11:16, Joseph Tate wrote: > Anybody have a metakit database dump utility? I'm somehow getting a corrupt > database that causes a crash, and I'd like to take a look at what's inside > it. If nobody's got one, I'll just whip one up. > > Joseph > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jtate@mi-corporation.com Tue Jun 11 12:27:31 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5BHRVa27660 for ; Tue, 11 Jun 2002 12:27:31 -0500 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g5BHRMN30972 for ; Tue, 11 Jun 2002 13:27:22 -0400 From: "Joseph Tate" To: "MetaKit" Date: Tue, 11 Jun 2002 13:27:30 -0400 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0021_01C2114B.BCC8C990" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Corrupt Data File Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0021_01C2114B.BCC8C990 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Ok, I'm out of my league here. Attached is a corrupted datafile. It asserts in mkdump. It was created on Windows CE, but non-corrupted datafiles can be dumped just fine, so the file formats are identical. It would seem that the database has column header data written in two places. I don't understand the Metakit file format well enough to figure out more than that though. What could possibly cause this kind of data corruption. Here's another clue, well possibly. The corruption only occurs in the release build, and cannot be reproduced in debug mode, making it nearly impossible to debug. The only thing I can think would be wrong is if the file writing process was happening in two places simultaneously, but we haven't been able to find any place in the code where that would happen. Joseph ------=_NextPart_000_0021_01C2114B.BCC8C990 Content-Type: application/octet-stream; name="Copy (3) of Mi-Forms_Reporter.Db" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="Copy (3) of Mi-Forms_Reporter.Db" JL=1A=00=00=00=04=D2=CB]z=10=974=E7=10=90}X=12>A=97=13=B3=C3=D4e2E=E2@=1F= z*p2E=E2@O=10'=932E=E2@O=05f=B12E=E2@=D5=08=00=00=00=00D=8F=95=17=FF=FF=CB= ]z=10=974=E7=10=90}X=12=B3=C3=D4e2E=E2@=1Fz*p2E=E2@O=10'=932E=E2@=07=00=00= =00D=8F=95ViewSessionAll=00ViewSessionOwn=00ViewSessionOwn=00=FF=0F=80=83= =8C=C4=98=D0=84=E8=80=02=E8=05=9D=83=EC=80=AD=EF=82=01=9C=80=80=80=80=C5r= ept[ID:I,Date:D,Type:I,Response:I,Message:S,User:S,ResponseString:S]=81=98= =01=9EObtain printed copies of form Comprehensive & you. version 2.2. =00Obtain printed copies of form Tester form (CalType) Win version 1.2. Obtain printed copies of form VHA Daily Rounding Form version 1.0. =00Obtain printed copies of form IRS-Roadside Log Report carry version = 1.1. Obtain printed copies of form IRS- BTR Reference Sample version 1.0. =00Authentication = failed.=00Ok=00=C0=A3=80=00=00=00=00=00=02=85=80=00=00K=00=00=02:JL=0A= =00=00=00=01=80Obtain printed copies of form Comprehensive & you. = version 2.2. =00Obtain printed copies of form Tester form (CalType) Win version 1.2. Obtain printed copies of form VHA Daily Rounding Form version 1.0. =00Obtain printed copies of form IRS-Roadside Log Report carry version = 1.1. Obtain printed copies of form IRS- BTR Reference Sample version 1.0. =00=80=00=00=00=00=00=04=05=80=00=00K=00=00=00=B6JL=0A= =00=00=00=00=BDViewSessionAll=00ViewSessionOwn=00ViewSessionOwn=00ViewSes= sionOwn=00=80=84=90=88=A0=98=81=B8=85=B9=02=FF=02=81=84=BE=80=BC=08=9D=82= =C2=80=83=05=80=81=05=83=80=80=C5rept[ID:I,Date:D,Type:I,Response:I,Messa= ge:S,User:S,ResponseString:S]=81=9E=08=D9=80=00=00=00=00=00=04=C2=80=00=00= K=00=00=04w ------=_NextPart_000_0021_01C2114B.BCC8C990-- From wlb@multisysid.com Tue Jun 11 13:02:58 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5BI2wa29846 for ; Tue, 11 Jun 2002 13:02:58 -0500 Received: from multisysid.com ([192.168.1.148]) by nutmeg.multisysid.com (8.9.3/8.8.7) with ESMTP id OAA03642; Tue, 11 Jun 2002 14:15:39 -0400 Message-ID: <3D063CA1.2040707@multisysid.com> Date: Tue, 11 Jun 2002 14:08:33 -0400 From: wlb User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0) Gecko/20020530 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Joseph Tate CC: MetaKit Subject: Re: [Metakit] Corrupt Data File References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joseph Tate wrote: >Ok, I'm out of my league here. Attached is a corrupted datafile. It >asserts in mkdump. It was created on Windows CE, but non-corrupted >datafiles can be dumped just fine, so the file formats are identical. It >would seem that the database has column header data written in two places. >I don't understand the Metakit file format well enough to figure out more >than that though. What could possibly cause this kind of data corruption. >Here's another clue, well possibly. The corruption only occurs in the >release build, and cannot be reproduced in debug mode, making it nearly >impossible to debug. The only thing I can think would be wrong is if the >file writing process was happening in two places simultaneously, but we >haven't been able to find any place in the code where that would happen. > >Joseph > > One thing which was a big surprise to me is that each message is launched in a different thread. Actually, I don't know that that is true of _all_ messages, but the message dispacher is very trigger-happy when it comes to creating threads. If someone double-clicks on a button, two threads will be created. If you write to a file based on such a user event, there is the possibility that the event will be fired twice -- simultaneously. I had a user that just loved to double-click on buttons and it took me a long time to figure out what was going on. Is thread safety enabled on WCE? I assume you are doing this in the PocketPC emulator. This emulator does have a few problems. The PocketPC-2002 emulator is a monster, but it is a complete CE image. (The PocketPC emulator is not a complete CE image. It remaps a lot of calls to DLL's in c:\winnt\system32.) If this was not in the emulator, then what processor? wlb From jtate@mi-corporation.com Tue Jun 11 13:33:37 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5BIXba31675 for ; Tue, 11 Jun 2002 13:33:37 -0500 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g5BIXRN31413; Tue, 11 Jun 2002 14:33:27 -0400 From: "Joseph Tate" To: "wlb" Cc: "MetaKit" Subject: RE: [Metakit] Corrupt Data File Date: Tue, 11 Jun 2002 14:33:36 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <3D063CA1.2040707@multisysid.com> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Arm. We can't reproduce it on the emulator debug or release, or on the device with a debug build. Setting up the PocketPC 2002 emulator is a pain, so I haven't done that yet. Don't know how much that would help. Not sure of the thread safety setting, but we've put semaphores around all the database calls anyway. The only thing I can think of is if the File handle is somehow getting shared; don't ask me how. Joseph > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of wlb > Sent: Tuesday, June 11, 2002 2:09 PM > To: Joseph Tate > Cc: MetaKit > Subject: Re: [Metakit] Corrupt Data File > > > Joseph Tate wrote: > > >Ok, I'm out of my league here. Attached is a corrupted datafile. It > >asserts in mkdump. It was created on Windows CE, but non-corrupted > >datafiles can be dumped just fine, so the file formats are identical. It > >would seem that the database has column header data written in > two places. > >I don't understand the Metakit file format well enough to figure out more > >than that though. What could possibly cause this kind of data > corruption. > >Here's another clue, well possibly. The corruption only occurs in the > >release build, and cannot be reproduced in debug mode, making it nearly > >impossible to debug. The only thing I can think would be wrong is if the > >file writing process was happening in two places simultaneously, but we > >haven't been able to find any place in the code where that would happen. > > > >Joseph > > > > > One thing which was a big surprise to me is that each message is > launched in a > different thread. Actually, I don't know that that is true of _all_ > messages, but > the message dispacher is very trigger-happy when it comes to creating > threads. > If someone double-clicks on a button, two threads will be created. If > you write > to a file based on such a user event, there is the possibility that the > event will be > fired twice -- simultaneously. I had a user that just loved to > double-click on > buttons and it took me a long time to figure out what was going on. > > Is thread safety enabled on WCE? > > I assume you are doing this in the PocketPC emulator. This emulator > does have > a few problems. The PocketPC-2002 emulator is a monster, but it is a > complete > CE image. (The PocketPC emulator is not a complete CE image. It remaps a > lot of calls to DLL's in c:\winnt\system32.) > > If this was not in the emulator, then what processor? > > wlb > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From wlb@multisysid.com Tue Jun 11 14:07:26 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5BJ7Qa01225 for ; Tue, 11 Jun 2002 14:07:26 -0500 Received: from multisysid.com ([192.168.1.148]) by nutmeg.multisysid.com (8.9.3/8.8.7) with ESMTP id PAA04549; Tue, 11 Jun 2002 15:20:08 -0400 Message-ID: <3D064BBD.8080705@multisysid.com> Date: Tue, 11 Jun 2002 15:13:01 -0400 From: wlb User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0) Gecko/20020530 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Joseph Tate CC: MetaKit Subject: Re: [Metakit] Corrupt Data File References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joseph Tate wrote: >Arm. We can't reproduce it on the emulator debug or release, or on the >device with a debug build. Setting up the PocketPC 2002 emulator is a pain, >so I haven't done that yet. Don't know how much that would help. Not sure >of the thread safety setting, but we've put semaphores around all the >database calls anyway. The only thing I can think of is if the File handle >is somehow getting shared; don't ask me how. > >Joseph > > The PPC2002 emulator is also big and very-very slow. It takes almost 30 secs to boot on my machine...and unlike the PPC emulator, it is very prone to loose its data, ie, it is too easy to cold boot. I haven't seen this problem in my app. But then again, I don't write to the database much at all. Once the database is created, I treat it pretty much read-only. There is a slim chance that you could be bitten by a bug in the flash memory driver if your data files are in flash. I have had problems with flash corruption on PPC devices from Symbol. Recovering from the flash corruption required reformatting the flash drive. The original IPAQ had no user-accessible flash, so no problem there :). I have never had problems with file corruption in RAM. Are you using a single semaphore to control all database writes, or do you have different semaphores in different parts of the program? wlb From jtate@mi-corporation.com Tue Jun 11 14:41:53 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5BJfqa03014 for ; Tue, 11 Jun 2002 14:41:52 -0500 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g5BJfgN31739; Tue, 11 Jun 2002 15:41:42 -0400 From: "Joseph Tate" To: "wlb" Cc: "MetaKit" Subject: RE: [Metakit] Corrupt Data File Date: Tue, 11 Jun 2002 15:41:51 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <3D064BBD.8080705@multisysid.com> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > The PPC2002 emulator is also big and very-very slow. It takes almost 30 > secs to boot > on my machine...and unlike the PPC emulator, it is very prone to loose > its data, ie, it is > too easy to cold boot. > > I haven't seen this problem in my app. But then again, I don't write to > the database > much at all. Once the database is created, I treat it pretty much > read-only. It doesn't happen for all of our databases (we have 6), just this one, and only when a combination of events happens. We've narrowed it down to one sequence that always reproduces the problem, but have gotten no further than that. > There is a slim chance that you could be bitten by a bug in the flash > memory driver if > your data files are in flash. I have had problems with flash corruption > on PPC devices > from Symbol. Recovering from the flash corruption required reformatting > the flash > drive. The original IPAQ had no user-accessible flash, so no problem > there :). I > have never had problems with file corruption in RAM. We're using ram, but it also ocurrs in Flash. > > Are you using a single semaphore to control all database writes, or do > you have > different semaphores in different parts of the program? Single semaphore. It's driving us nuts. From lancep@wapdomainz.com Thu Jun 13 00:46:53 2002 Received: from pax.host4u.net (pax.host4u.net [216.71.64.89]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5D5kqa20804 for ; Thu, 13 Jun 2002 00:46:52 -0500 Received: from snake (marinefoods.co.nz [203.167.190.46]) by pax.host4u.net (8.11.6/8.11.6) with SMTP id g5D5kde01455 for ; Thu, 13 Jun 2002 00:46:40 -0500 Message-ID: <012101c2129e$2d9c5cb0$4801a8c0@snake> Reply-To: "Lance Paine" From: "Lance Paine" To: Date: Thu, 13 Jun 2002 17:50:03 +1200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_011E_01C21302.BECE5210" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Advice? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_011E_01C21302.BECE5210 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi there,=20 I've just discovered MetaKit (thanks to a post on the wxWindows group), = and I'm very keen to start using it.=20 I have a few questions:=20 Is the VFS available for use on C/C++ platforms, or is that a tcl only = thing?=20 One of the things I'd like to use it for is the storage, client side of = a list of hierachical genres.=20 So things like=20 Rock&Roll/Pop/Mersey Beat Rock&Roll/Altern/Grunge Blues/Country/Acoustic Blues/Country/Dirty Blues/Chicago/Electric Blues/Chicago/Acoustic And need to be able to have an id associated with each one.=20 What's the best way of storing data like that in metakit?=20 Flatfile with a reference to the Parent:=20 1 | Rock&Roll | Null 2 | Pop | 1 etc.. or nested views?=20 Thanks in advance,=20 Lance.=20 ------=_NextPart_000_011E_01C21302.BECE5210 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi there,
I've just discovered MetaKit (thanks to = a post on=20 the wxWindows group), and I'm very keen to start using it.
 
I have a few questions:
Is the VFS available for use on C/C++ = platforms, or=20 is that a tcl only thing?
 
One of the things I'd like to use it = for is the=20 storage, client side of a list of hierachical genres.
So things like
 
Rock&Roll/Pop/Mersey = Beat
Rock&Roll/Altern/Grunge
Blues/Country/Acoustic
Blues/Country/Dirty
Blues/Chicago/Electric
Blues/Chicago/Acoustic
 
And need to be able to have an id associated with each one.
What's the best way of storing data like that in metakit?
 
Flatfile with a reference to the Parent:
1 | Rock&Roll | Null
2 | Pop     | 1
etc..
or nested views?
 
Thanks in advance,
 
Lance.
------=_NextPart_000_011E_01C21302.BECE5210-- From jyl@best.com Thu Jun 13 09:11:50 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g5DEBoa14648 for ; Thu, 13 Jun 2002 09:11:50 -0500 Received: (qmail 44000 invoked by uid 19667); 13 Jun 2002 14:11:49 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 13 Jun 2002 14:11:49 -0000 Message-ID: <3D08A86C.3060301@best.com> Date: Thu, 13 Jun 2002 07:13:00 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Lance Paine CC: metakit@equi4.com Subject: Re: [Metakit] Advice? References: <012101c2129e$2d9c5cb0$4801a8c0@snake> Content-Type: multipart/alternative; boundary="------------080505000605000007030004" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------080505000605000007030004 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Well, another chance to toot my own horn :) This sounds like an application tailor made for e4Graph. Check it out at http://e4graph.sourceforge.net/ e4Graph is built on top of Metakit, so all you learned about and did so far with Metakit still applies. JYL Lance Paine wrote: > Hi there, > > I've just discovered MetaKit (thanks to a post on the wxWindows > group), and I'm very keen to start using it. > > > > I have a few questions: > > Is the VFS available for use on C/C++ platforms, or is that a tcl only > thing? > > > > One of the things I'd like to use it for is the storage, client side > of a list of hierachical genres. > > So things like > > > > Rock&Roll/Pop/Mersey Beat > > Rock&Roll/Altern/Grunge > > Blues/Country/Acoustic > > Blues/Country/Dirty > > Blues/Chicago/Electric > > Blues/Chicago/Acoustic > > > > And need to be able to have an id associated with each one. > > What's the best way of storing data like that in metakit? > > > > Flatfile with a reference to the Parent: > > 1 | Rock&Roll | Null > > 2 | Pop | 1 > > etc.. > > or nested views? > > > > Thanks in advance, > > > > Lance. > --------------080505000605000007030004 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Well, another chance to toot my own horn :) This sounds like an application tailor made for e4Graph.

Check it out at http://e4graph.sourceforge.net/

e4Graph is built on top of Metakit, so all you learned about and did so far with Metakit still applies.

JYL

Lance Paine wrote:
Hi there,
I've just discovered MetaKit (thanks to a post on the wxWindows group), and I'm very keen to start using it.
 
I have a few questions:
Is the VFS available for use on C/C++ platforms, or is that a tcl only thing?
 
One of the things I'd like to use it for is the storage, client side of a list of hierachical genres.
So things like
 
Rock&Roll/Pop/Mersey Beat
Rock&Roll/Altern/Grunge
Blues/Country/Acoustic
Blues/Country/Dirty
Blues/Chicago/Electric
Blues/Chicago/Acoustic
 
And need to be able to have an id associated with each one.
What's the best way of storing data like that in metakit?
 
Flatfile with a reference to the Parent:
1 | Rock&Roll | Null
2 | Pop     | 1
etc..
or nested views?
 
Thanks in advance,
 
Lance.

--------------080505000605000007030004-- From jcw@equi4.com Fri Jun 14 17:26:06 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5EMQ5a07374 for ; Fri, 14 Jun 2002 17:26:05 -0500 From: Jean-Claude Wippler To: metakit Date: Sat, 15 Jun 2002 00:26:04 +0200 Message-Id: <20020614222604.31108@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] C++ docs updated Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: FYI, I've updated the C++ documentation of MetaKit to match the latest 2.4.7 release. No sweeping changes, just making sure the headers and the docs match up again - there were some embarrasingly old generated docs on the website. There's online docs and a standalone scripted document version (which works with Windows, Linux, and Solaris). Details on the MetaKit homepage, http://www.equi4.com/metakit/ -jcw From shmul@vself.com Mon Jun 17 06:26:31 2002 Received: from neo.vself.com ([192.117.144.254]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5HBQTa05374 for ; Mon, 17 Jun 2002 06:26:30 -0500 Received: from EXPERIENCE.vself.com (EXPERIENCE [10.0.0.80]) by neo.vself.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id MYWMRP6K; Mon, 17 Jun 2002 14:28:01 +0200 Message-Id: <5.1.0.14.0.20020617043526.029bcf18@10.0.0.3> X-Sender: shmul@10.0.0.3 X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 17 Jun 2002 05:28:14 -0700 To: metakit@equi4.com From: Shmulik Regev Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Metakit as a sophisticated in memory database Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm interested in opinions regarding the following, somewhat strange use of Metakit -- I'm in a need for a module that provides implementation for data structures with a convenient query language (or something similar). Full blown SQL is too much, but the ordinary "selects" with joins and nested queries is required. Persistency is NOT required, but efficiency is a must as somewhat heavy calculations are based on repeated queries to the data structures . The closest I got find are in-memory-databases. Once a user of Metakit (2.5 years ago) I recently gave it another look and was happy to discover the addition of custom views, which might (though I may be wrong here) supply some of the functionality I seek. My questions are: 1. In general, is Metakit suited for in-memory-database-like operation, in particular from the memory management point of view ? 2. Are the custom views implementations that exist powerful enough ? 3. (assuming 2 is yes) is the semantic of performing queries through custom views convenient ? 4. Can anyone recommend a different approach/tool ? Thanks, Shmulik Regev From jyl@best.com Mon Jun 17 08:40:51 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g5HDeoa11364 for ; Mon, 17 Jun 2002 08:40:50 -0500 Received: (qmail 39055 invoked by uid 19667); 17 Jun 2002 13:40:49 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 17 Jun 2002 13:40:49 -0000 Message-ID: <3D0DE739.4090905@best.com> Date: Mon, 17 Jun 2002 06:42:17 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Shmulik Regev CC: metakit@equi4.com Subject: Re: [Metakit] Metakit as a sophisticated in memory database References: <5.1.0.14.0.20020617043526.029bcf18@10.0.0.3> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Shmulik Good questions! The following are empirical personal observations, others may have conflicting answers: 1. Yes, Metakit is extremely efficient as an in memory database. Essentially all your data *is already* in memory once you opened the storage. I've found data access overheads to be uniform and low no matter where in the storage the data is located. 2. Depends on what you want to do, we don't know enough yet about your specific usage intentions, do we? :) However, I decided to implement my own operations rather than use derived views. I feel that if you know *exactly* what our needs are you may win by implementing your own tailored operations rather than using Metakit's derived views. The good news is that this is, while maybe more work, possible todo efficiently because Metakit is so efficient internally. 3, The semantics of derived views is very general. Thus it may be more complex than what you need. I think it's pretty much guaranteed that you'll be able to express what you need with their semantics, but you may be better off implementing your own. 4. Implement your own operations as needed, reuse what JC provided when the efficiency and convenience of the generalized operations satisfies your needs. Experiment! Hope this helps, JYL Shmulik Regev wrote: > I'm interested in opinions regarding the following, somewhat strange > use of Metakit -- > > I'm in a need for a module that provides implementation for data > structures with a convenient query language (or something similar). > Full blown SQL is too much, but the ordinary "selects" with joins and > nested queries is required. Persistency is NOT required, but > efficiency is a must as somewhat heavy calculations are based on > repeated queries to the data structures . The closest I got find are > in-memory-databases. Once a user of Metakit (2.5 years ago) I recently > gave it another look and was happy to discover the addition of custom > views, which might (though I may be wrong here) supply some of the > functionality I seek. My questions are: > > 1. In general, is Metakit suited for in-memory-database-like > operation, in particular from the memory management point of view ? > > 2. Are the custom views implementations that exist powerful enough ? > > 3. (assuming 2 is yes) is the semantic of performing queries through > custom views convenient ? > > 4. Can anyone recommend a different approach/tool ? > > Thanks, > Shmulik Regev > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From shmul@vself.com Mon Jun 17 14:40:46 2002 Received: from neo.vself.com ([192.117.144.254]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5HJeja28319 for ; Mon, 17 Jun 2002 14:40:45 -0500 Received: from EXPERIENCE.vself.com (diup-183-68.inter.net.il [213.8.183.68]) by neo.vself.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id MYWMRQAR; Mon, 17 Jun 2002 22:42:16 +0200 Message-Id: <5.1.0.14.0.20020617133008.02b83008@10.0.0.3> X-Sender: shmul@10.0.0.3 X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 17 Jun 2002 13:42:27 -0700 To: Jacob Levy From: Shmulik Regev Subject: Re: [Metakit] Metakit as a sophisticated in memory database Cc: metakit@equi4.com In-Reply-To: <3D0DE739.4090905@best.com> References: <5.1.0.14.0.20020617043526.029bcf18@10.0.0.3> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > >1. Yes, Metakit is extremely efficient as an in memory database. >Essentially all your data *is already* in memory once you opened the >storage. I've found data access overheads to be uniform and low no matter >where in the storage the data is located. This doesn't apply to my usage as I have no storage (everything is transient). >2. Depends on what you want to do, we don't know enough yet about your >specific usage intentions, do we? :) We basically deal with natural language processing and in our case it is more or less equivalent to the data kept during text indexing process. Specifically we require efficient mapping of words to their occurrences (including location) in the pages. > However, I decided to implement my own operations rather than use > derived views. I feel that if you know *exactly* what our needs are you > may win by implementing your own tailored operations rather than using > Metakit's derived views. The good news is that this is, while maybe more > work, possible todo efficiently because Metakit is so efficient internally. Can you elaborate ? What operations did you add and to what extent were you aware of Metakit's internal details ? I enjoying the code and design of it when I used it (as I said it was quite some time since then), but I recall that a lot of the implementation details were hidden (unless you rolled your sleeves and dug into the code). Is it open for extensions of the kind you refer to ? >3, The semantics of derived views is very general. Thus it may be more >complex than what you need. I think it's pretty much guaranteed that >you'll be able to express what you need with their semantics, but you may >be better off implementing your own. Understood. >4. Implement your own operations as needed, reuse what JC provided when >the efficiency and convenience of the generalized operations satisfies >your needs. Experiment! I wish I had the time to experiment :) Anyway, lets keep this discussion going, it provides excellent information for Metakit novices (such as myself). Cheers, Shmul From joshm@taconic.net Mon Jun 17 14:52:21 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5HJqLa28795 for ; Mon, 17 Jun 2002 14:52:21 -0500 Received: from gatekeeper (mcha-ad042.taconic.net [205.231.26.233]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g5HJqCG18699 for ; Mon, 17 Jun 2002 15:52:13 -0400 (EDT) Message-ID: <03cb01c21637$e9d94d60$4596e7cd@gatekeeper> From: "Joshua Muskovitz" To: References: <5.1.0.14.0.20020617043526.029bcf18@10.0.0.3> <5.1.0.14.0.20020617133008.02b83008@10.0.0.3> Subject: Re: [Metakit] Metakit as a sophisticated in memory database Date: Mon, 17 Jun 2002 15:48:06 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > This doesn't apply to my usage as I have no storage (everything is transient). I believe what he meant was that all metakit operations occur in memory, regardless of whether you create it with or without an underlying file (or stream). > We basically deal with natural language processing and in our case it is > more or less equivalent to the data kept during text indexing process. > Specifically we require efficient mapping of words to their occurrences > (including location) in the pages. This seems like a fine use for MK, and should reduce your coding time significantly. -- josh From nathan.thompson-amato@dnamerican.com Wed Jun 19 12:56:03 2002 Received: from montecarlo.DNAMERICAN.COM ([66.109.177.66]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g5JHu2a18745 for ; Wed, 19 Jun 2002 12:56:03 -0500 Received: from [66.109.177.113] by montecarlo.DNAMERICAN.COM (NTMail 3.03.0018/4c.adtc) with ESMTP id ka081650 for ; Wed, 19 Jun 2002 13:55:48 -0400 Message-ID: <3D10C56D.8050304@dnamerican.com> Date: Wed, 19 Jun 2002 13:54:53 -0400 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020605 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Per-record locking & Find()/GetAt() reliability Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm using Metakit in a multithreaded app, and I've been protecting Metakit with a single big POSIX-thread mutex. This has been working well, but I'd like to improve performance by locking at a finer granularity -- per record, if possible. To this end, I'd like to know whether the index returned by c4_View::Find() stays accurate if a record is added to the database. Similarly, does the c4_RowRef that c4_View::GetAt() returns remain accurate if a record is added? Thanks for your help, Nathan From targetemailextractor@btamail.net.cn Thu Jun 20 21:29:00 2002 Received: from localhost.com ([61.174.192.55]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g5L2Sua30649 for ; Thu, 20 Jun 2002 21:28:57 -0500 Message-Id: <200206210228.g5L2Sua30649@trixie.triqs.com> From: targetemailextractor@btamail.net.cn Reply-To: targetemailextractor@btamail.net.cn To: metakit@equi4.com Date: Fri, 21 Jun 2002 10:28:57 +0800 X-Mailer: QuickSender 1.05 MIME-Version: 1.0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: [Metakit] ADV: Direct email blaster, email addresses extractor, maillist verify, maillist manager........... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: =3CHTML=3E=3CHEAD=3E =3CSTYLE=3E=3C=2FSTYLE=3E =3C=2FHEAD=3E =3CBODY bgColor=3D#ffffff=3E =3CDIV=3E=3CFONT size=3D2=3E =3CDIV=3E=3CFONT face=3DArial=3E =3B=3C=2FFONT=3E =3C=2FDIV=3E =3CDIV=3E =3B=3C=2FDIV=3E =3CDIV=3E=3CSTRONG=3E=3CFONT color=3D#ff0080 face=3DArial size=3D4=3EDirect Email Blaster=3C=2FFONT=3E=3C=2FSTRONG=3E =3C=2FDIV=3E=3CFONT size=3D2=3E =3CP=3E=3CFONT face=3DArial=3E=3CB=3E=3CFONT color=3D#006600=3E=3CI=3EThe program will send mail at the rate of over 1=2C 000 e-mails per minute=2E =3B=3C=2FI=3E=3C=2FFONT=3E=3CBR=3ELegal and Fast sending bulk emails =3B=3CBR=3E=3CFONT color=3D#006600=3E=3CI=3EBuilt in SMTP server =3B=3C=2FI=3E=3C=2FFONT=3E=3CBR=3EHave Return Path =3B=3CBR=3ECan Check Mail Address =3B=3CBR=3E=3CFONT color=3D#006600=3E=3CI=3EMake Error Send Address List=28 Remove or Send Again=29 =3B=3C=2FI=3E=3C=2FFONT=3E=3CBR=3ESupport multi-threads=2E =3B=3CBR=3ESupport multi-smtp servers=2E =3B=3CBR=3EManages your opt-in E-Mail Lists =3B=3CBR=3EOffers an easy-to-use interface! =3B=3CBR=3EEasy to configure and use =3B=3C=2FB=3E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3E=3CA href=3D=22http=3A=2F=2Fwww=2Ewldinfo=2Ecom=2Fbj=5Fdownload=2Fedeb=5Fset=2Ezip=22=3E=3CSTRONG=3EDownload Now=3C=2FSTRONG=3E=3C=2FA=3E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CSTRONG=3E=3CFONT color=3D#ff0080 face=3DArial size=3D4=3EMaillist Verify=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3EMaillist Verify is intended for e-mail addresses and mail lists verifying=2E The main task is to determine which of addresses in the mail list are dead=2E The program is oriented=2C basically=2C on programmers which have their own mail lists to inform their users about new versions of their programs=2E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3EThe program works on the same algorithm as ISP mail systems do=2E Mail servers addresses for specified address are extracted from DNS=2E The program tries to connect with found SMTP-servers and simulates the sending of message=2E It does not come to the message =3CNOBR=3Esending ‿=3B=2FNOBR>=3B EMV disconnect as soon as mail server informs does this address exist or not=2E EMV can find=3C=2FNOBR=3E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3E=3CNOBR=3E =3Babout 90% of dead addresses ‿=3B=2FNOBR>=3B some mail systems receive all messages and only then see their =3B=3C=2FNOBR=3E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3E=3CNOBR=3Eaddresses and if the address is dead send the message back with remark about it=2E=3C=2FNOBR=3E=3C=2FFONT=3E=3C=2FP=3E=3CNOBR=3E =3CP=3E=3CA href=3D=22http=3A=2F=2Fwww=2Ewldinfo=2Ecom=2Fbj=5Fdownload=2Fbemv=5Fset=2Ezip=22=3E=3CSTRONG=3E=3CFONT face=3DArial size=3D3=3EDownload Now=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FA=3E=3C=2FP=3E =3CP=3E=3CSTRONG=3E=3CFONT color=3D#ff0080 face=3DArial size=3D4=3EExpress Email Blaster =3B=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3EExpress Email Blaster =3B is a very fast=2C powerful yet simple to use email sender=2E Utilizing multiple threads=2Fconnections=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3E =3Band multiple SMTP servers your emails will be sent out fast and easily=2E There are User Information=2C Attach Files=2C =3B=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3EAddress and Mail Logs four tabbed area for the E-mails details for sending=2E About 25 SMTP servers come with the =3B=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3Edemo version=2C and users may Add and Delete SMTP servers=2E About =3CFONT color=3D#008000=3E=3CB=3E60=2C000=3C=2FB=3E=3C=2FFONT=3E E-mails will be sent out per hour=2E=22=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CSTRONG=3E=3CA href=3D=22http=3A=2F=2Fwww=2Ewldinfo=2Ecom=2Fbj=5Fdownload=2Fbeeb=5Fset=2Ezip=22=3E=3CFONT face=3DArial size=3D3=3EDownload Now=3C=2FFONT=3E=3C=2FA=3E=3C=2FSTRONG=3E=3C=2FP=3E =3CP=3E=3CSTRONG=3E=3CFONT color=3D#ff0080 face=3DArial size=3D4=3EExpress Email Address Extractor=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FP=3E=3CFONT size=3D4=3E =3CP=3E=3CFONT color=3D#008000 size=3D3=3EThis program is the most efficient=2C easy to use email address collector available on the =3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3E=3CFONT color=3D#008000 size=3D3=3E =3Binternet! =3C=2FFONT=3E=3CFONT color=3D#000000 size=3D3=3EBeijing Express Email Address Extractor =28ExpressEAE=29 is designed to extract=3C=2FFONT=3E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT color=3D#000000 size=3D3=3E =3Be-mail addresses from web-pages on the Internet =28using HTTP protocols=29 =2EExpressEAE=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT color=3D#000000 size=3D3=3E =3Bsupports operation through many proxy-server and works very fast=2C as it is able of =3B=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT color=3D#000000 size=3D3=3Eloading several pages simultaneously=2C and requires very few resources=2E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT color=3D#000000 face=3DArial=3E=3CFONT size=3D3=3EWith it=2C you will be able to=3C=2FFONT=3E=3CFONT size=3D2=3E =3C=2FFONT=3E=3CFONT size=3D3=3Euse targeted searches to crawl the world wide web=2C extracting =3B=3C=2FFONT=3E=3C=2FFONT=3E =3CP=3E=3CFONT color=3D#000000 face=3DArial size=3D3=3Ethousands of clean=2C fresh email addresses=2E Ably Email address Extractor is unlike other =3B=3C=2FFONT=3E =3CP=3E=3CFONT color=3D#000000 face=3DArial size=3D3=3Eaddress collecting programs=2C which limit you to one or two search engines and are unable=3C=2FFONT=3E =3CP=3E=3CFONT color=3D#000000 face=3DArial size=3D3=3E =3Bto do auto searches HUGE address=2E Most of them collect a high percentage of incomplete=2C =3B=3C=2FFONT=3E =3CP=3E=3CFONT color=3D#000000 face=3DArial size=3D3=3Eunusable addresses which will cause you serious problems when using them in a mailing=2E =3B=3C=2FFONT=3E =3CUL=3E =3CLI=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EEasier to learn and use than any other email address collector program available=2E=3C=2FFONT=3E =3CLI=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EAccesses eight search engines =3B=3C=2FFONT=3E =3CLI=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EAdd your own URLs to the list to be searched=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial size=3D3=3E=3CFONT color=3D#008000=3ESupports operation through =3C=2FFONT=3E=3CFONT color=3D#ff00ff=3Ea lot of=3C=2FFONT=3E=3CFONT color=3D#008000=3E proxy-server and works very fast =28HTTP Proxy=29=3C=2FFONT=3E=3C=2FFONT=3E =3CLI=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EAble of loading several pages simultaneously=3C=2FFONT=3E =3CLI=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3ERequires very few resources=3C=2FFONT=3E =3CLI=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3ETimeout feature allows user to limit the amount of time crawling in dead sites and traps=2E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial size=3D3=3E=3CFONT color=3D#008000=3EEasy to make =3C=2FFONT=3E=3CFONT color=3D#ff00ff=3EHuge=3C=2FFONT=3E=3CFONT color=3D#008000=3E address list=3C=2FFONT=3E=3C=2FFONT=3E =3CLI=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EPause=2Fcontinue extraction at any time=2E=3C=2FFONT=3E =3CLI=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EAuto connection to the Internet=3C=2FFONT=3E =3C=2FLI=3E=3C=2FUL=3E =3CDIV=3E=3CSTRONG=3E=3CA href=3D=22http=3A=2F=2Fwww=2Ewldinfo=2Ecom=2Fbj=5Fdownload=2Feeae=5Fset=2Ezip=22=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EDownload Now=3C=2FFONT=3E=3C=2FA=3E=3C=2FSTRONG=3E =3C=2FDIV=3E =3CDIV=3E=3CSTRONG=3E=3CFONT color=3D#ff0080 face=3DArial=3EExpress Email Address Downloader=3C=2FFONT=3E=3C=2FSTRONG=3E =3C=2FDIV=3E =3CUL=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT color=3D#006600 size=3D2=3EExpressEAD =3B =3C=2FFONT=3E=3CFONT color=3D#000000 size=3D2=3Eis a 32 bit Windows Program for e-mail marketing=2E It is intended for easy and convenient=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT color=3D#000000 size=3D2=3E =3Bsearch large e-mail address lists from mail servers=2E The program can be operated on =3B=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT color=3D#000000 size=3D2=3EWindows 95=2F98=2FME=2F2000 and NT=2E=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT color=3D#006600 size=3D2=3EExpressEAD =3B =3C=2FFONT=3E=3CFONT color=3D#000000 size=3D2=3Esupport multi-threads =28up to 1024 connections=29=2E=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT color=3D#006600 size=3D2=3EExpressEAD =3B =3C=2FFONT=3E=3CFONT color=3D#000000 size=3D2=3Ehas the ability =3B to reconnect to the mail server if the server has disconnected and =3B=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT color=3D#000000 size=3D2=3Econtinue the searching at the point where it has been interrupted=2E=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT color=3D#006600 size=3D2=3EExpressEAD =3B =3C=2FFONT=3E=3CFONT color=3D#000000 size=3D2=3Ehas an ergonomic interface that is easy to set up and simple to use=2E=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3C=2FLI=3E=3C=2FUL=3E =3CDIV=3E=3CFONT face=3DArial=3E =3B=3C=2FFONT=3E =3C=2FDIV=3E =3CP=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT color=3D#008000 face=3DArial size=3D4=3EFeatures=3A=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CUL type=3Ddisc=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT face=3DArial size=3D2=3Esupport multi-threads=2E=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT face=3DArial size=3D2=3Eauto get smtp server address=2Csupport multi-smtp servers=2E=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT face=3DArial size=3D2=3Eauto save =3B E-Mail Lists=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3CLI=3E=3CFONT face=3DArial=3E=3CSTRONG=3E=3CFONT face=3DArial size=3D2=3Eoffers an easy-to-use interface!=3C=2FFONT=3E=3C=2FSTRONG=3E=3C=2FFONT=3E =3C=2FLI=3E=3C=2FUL=3E =3CDIV=3E=3CSTRONG=3E=3CA href=3D=22http=3A=2F=2Fwww=2Ewldinfo=2Ecom=2Fbj=5Fdownload=2Feead=5Fset=2Ezip=22=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EDownload Now=3C=2FFONT=3E=3C=2FA=3E=3C=2FSTRONG=3E =3C=2FDIV=3E =3CDIV=3E=3CFONT face=3DArial=3E =3B=3C=2FFONT=3E =3C=2FDIV=3E =3CDIV=3E=3CSTRONG=3E=3CFONT color=3D#ff0080 face=3DArial=3EExpress Maillist Manager=3C=2FFONT=3E=3C=2FSTRONG=3E =3C=2FDIV=3E =3CDIV=3E=3CFONT face=3DArial=3E =3B=3C=2FFONT=3E =3C=2FDIV=3E =3CDIV=3E=3CFONT size=3D2=3E =3CP=3E=3CFONT face=3DArial=3E=3CFONT color=3Dblack size=3D3=3EThis program was designed to be a complement to the =3C=2FFONT=3E=3CFONT color=3D#800080 size=3D3=3EDirect Email Blaster =3B =3C=2FFONT=3E=3CFONT color=3Dblack size=3D3=3Eand =3C=2FFONT=3E=3CFONT color=3D#800080 size=3D3=3EEmail Blaster =3C=2FFONT=3E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3E=3CFONT color=3Dblack size=3D3=3Esuite of bulk email software programs=2E Its purpose is to organize your email lists in order to be more =3B=3C=2FFONT=3E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3E=3CFONT color=3Dblack size=3D3=3Eeffective with your email marketing campaign=2E Some of its features include=3A=3C=2FFONT=3E=3C=2FFONT=3E=3C=2FP=3E =3CP=3E=3CB=3E=3CFONT color=3D#008000 face=3DArial=3E=3CFONT size=3D3=3E‿=3BCombine several lists into one file=2E=3C=2FFONT=3E=3CBR=3E=3CFONT size=3D3=3E‿=3BSplit up larger lists to make them more manageable=2E=3C=2FFONT=3E=3CBR=3E=3CFONT size=3D3=3E‿=3BRemove addresses from file=2E=3C=2FFONT=3E=3CBR=3E=3CFONT size=3D3=3E‿=3BManual editing=2C adding=2C and deleting of addresses=2E=3C=2FFONT=3E=3CBR=3E=3CFONT size=3D3=3E‿=3BAbility to auto clean lists=2C that is=2C remove any duplicate or unwanted addresses=2E=3C=2FFONT=3E=3CBR=3E=3CFONT size=3D3=3E‿=3BMaintain all your address lists within the program so you no =3B longer need to keep all your=3C=2FFONT=3E=3C=2FFONT=3E=3C=2FB=3E=3C=2FP=3E =3CP=3E=3CB=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3E =3Blists saved as separate text files=2E=3C=2FFONT=3E=3C=2FB=3E=3C=2FP=3E =3CP=3E=3CSTRONG=3E=3CA href=3D=22http=3A=2F=2Fwww=2Ewldinfo=2Ecom=2Fbj=5Fdownload=2Fbemm=5Fset=2Ezip=22=3E=3CFONT color=3D#008000 face=3DArial size=3D3=3EDownload Now=3C=2FFONT=3E=3C=2FA=3E=3C=2FSTRONG=3E=3C=2FP=3E =3CP=3E=3CFONT face=3DArial=3E =3B=3C=2FFONT=3E=3C=2FP=3E =3CP=3E =3B=3C=2FP=3E =3CDIV=3E=3CFONT face=3DArial=3Eif you want to remove your email=2C please send email to =3CA href=3D=22mailto=3Atargetemailremoval=40btamail=2Enet=2Ecn=22=3Etargetemailremoval=40btamail=2Enet=2Ecn=3C=2FA=3E=3C=2FFONT=3E =3C=2FDIV=3E =3CDIV=3E=3CFONT face=3DArial=3E =3B=3C=2FFONT=3E =3C=2FDIV=3E =3CDIV=3E=3CFONT face=3DArial=3E =3B=3C=2FFONT=3E =3C=2FDIV=3E =3CDIV=3E =3B=3C=2FDIV=3E=3C=2FFONT=3E=3C=2FDIV=3E=3C=2FFONT=3E=3C=2FNOBR=3E=3C=2FFONT=3E=3C=2FFONT=3E=3C=2FDIV=3E=3C=2FBODY=3E=3C=2FHTML=3E From joshm@taconic.net Thu Jun 20 22:45:50 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5L3joa03008 for ; Thu, 20 Jun 2002 22:45:50 -0500 Received: from gatekeeper (mcha-ae015.taconic.net [205.231.29.143]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g5L3jlG25589 for ; Thu, 20 Jun 2002 23:45:48 -0400 (EDT) Message-ID: <0eff01c218d5$86e57300$4596e7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: <20020614222604.31108@triqs.com> Date: Thu, 20 Jun 2002 23:41:24 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] debug assert question Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, We're seeing an odd behavior and hoped that someone (perhaps JCW?) had some thoughts on it. Successive versions of our app each modify the format of our database tables (as needed). We track this by keeping a record in the db of the "current" version (it's native structure). By checknig this value, each release is able to migrate the db to the most recent structure accordingly. Simple, and it works (so far). We're currently statically linking with MK 2.4.6, but I believe that previous versions linked with 2.4.5, or possibly even 2.4.4. What we're seeing is that in the debug build, the current version of our app asserts somewhere in the MK code (not sure where) on this line: _down.Size() <= NumHandlers() but the thing is, it only does this the first time the app runs using and older db structure. Once the db is brought up to the current format, the app never again makes this assert when run. Questions: Does this have something to do with changing MK versions? Is this something we should worry about? If the universe were shaped like a burrito, would we still have giraffes? Help! -- josh From MAILER-DAEMON2740@google.com Fri Jun 21 09:42:01 2002 Received: from google.com (inetwork-206.187.rev.afripa.com [194.9.187.206] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g5LEfsa01205 for ; Fri, 21 Jun 2002 09:41:57 -0500 Message-Id: <200206211441.g5LEfsa01205@trixie.triqs.com> From: "Farmgirl6520" To: X-Priority: 3 X-Mailer: The Bat! (v1.53d) Date: Fri, 21 Jun 2002 18:45:09 +0400 Mime-Version: 1.0 Content-Type: text/html; charset="ISO-8859-2" Status: R X-Status: N X-Set: phwdnlwChtxl71frp@8956 Subject: [Metakit] Real ZOO web site, welcome! ID Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The BEST zoo site on the @net!
Sex With Dogs
Horse Blow Jobs.
Snake Fuck.
REAL ANIMAL FUCKING!
100% HARDCORE!
ww1.only-beasts.com
unsub
Good luck, .

From lvirden@cas.org Fri Jun 21 09:51:51 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5LEpoa02201 for ; Fri, 21 Jun 2002 09:51:51 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.11.6+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g5LEph304848 for ; Fri, 21 Jun 2002 10:51:43 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g5LEpgm16379; Fri, 21 Jun 2002 10:51:42 -0400 (EDT) Date: Fri, 21 Jun 2002 10:51:42 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020621105142.AAB16359@cas.org> To: Subject: [Metakit] Spamming on the list Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Apparently this list's email address has been sold now to the spammers - I've seen two msgs so far that were off topic. Perhaps it is time for this list to only automatically post msgs from members? -- Support Internet Radio Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Fri Jun 21 09:59:22 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5LExLa02889 for ; Fri, 21 Jun 2002 09:59:22 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Spamming on the list Date: Fri, 21 Jun 2002 16:58:35 +0200 Message-Id: <20020621145835.12169@triqs.com> In-Reply-To: <20020621105142.AAB16359@cas.org> References: <20020621105142.AAB16359@cas.org> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >Apparently this list's email address has been sold now to the spammers - I've >seen two msgs so far that were off topic. Ouch. I didn't see those, SpamAssassin probably caught them. >Perhaps it is time for this list to only automatically post msgs from >members? Yes, done, thanks. The net sure is becoming more toxic :( -jcw From jcw@equi4.com Tue Jun 25 07:07:42 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5PC7ea21134; Tue, 25 Jun 2002 07:07:41 -0500 From: Jean-Claude Wippler To: Joshua Muskovitz , metakit Subject: Re: [Metakit] debug assert question Date: Tue, 25 Jun 2002 14:05:41 +0200 Message-Id: <20020625120541.26373@triqs.com> In-Reply-To: <0eff01c218d5$86e57300$4596e7cd@gatekeeper> References: <0eff01c218d5$86e57300$4596e7cd@gatekeeper> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joshua Muskovitz wrote: [Assert in src/derived.cpp] > int c4_SortSeq::Compare(int index_, c4_Cursor cursor_) const > { > d4_assert(cursor_._seq != 0); > > const char* down = (const char*) _down.Contents(); > d4_assert(_down.Size() <= NumHandlers()); [...] >but the thing is, it only does this the first time the app runs using and >older db structure. Once the db is brought up to the current format, the >app never again makes this assert when run. > >Questions: > >Does this have something to do with changing MK versions? I doubt it. MK 2.4.x have not changed w.r.t. on-the-fly restructuring. >Is this something we should worry about? My hunch is that the conversion is interfering with your logic. The one thing to keep in mind is that when restructuring, MK will not immediately remove deleted properties - is does that on commit. So if you restructure in such a way that a property is no longer in the new structure, the property will still show up. To avoid this, my suggestion is that when you detect a change and do a restructuring step, end by doing a commit. After that, the datafile will be in the same structure as what you get on a subsequent open. >If the universe were shaped like a burrito, would we still have giraffes? Tell me, please! -jcw From mike.collins@riley.army.mil Tue Jun 25 09:41:20 2002 Received: from rilex06.riley.army.mil (emh1.riley.army.mil [144.246.219.66]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5PEfJa32012 for ; Tue, 25 Jun 2002 09:41:20 -0500 Received: by emh1.riley.army.mil with Internet Mail Service (5.5.2653.19) id ; Tue, 25 Jun 2002 09:39:38 -0500 Message-ID: <73C2ED40DAD3D311934E00500400935F03951E5F@rilex11.riley.army.mil> From: "Collins, Mike DAC DOIM" To: "'metakit@equi4.com'" Date: Tue, 25 Jun 2002 09:44:21 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C21C56.CACF65B0" Subject: [Metakit] Is there a problem with creating a single-file wrapped executable ? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C21C56.CACF65B0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C21C56.CACF65B0" ------_=_NextPart_001_01C21C56.CACF65B0 Content-Type: text/plain; charset="iso-8859-1" I've tried it according to the instructions (via the web) Scripted documents and I get an "unkown_pending(MZ" error? Michael E. Collins DOIM, Senior Network Tech. 200 Henry Ave. Ft. Riley, KS 66442 mike.collins@riley.army.mil Phone: 785-239-3662 DSN: 856-3662 Fax: 785-239-0555 Online Fax: 785-239-6660 ------_=_NextPart_001_01C21C56.CACF65B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
 
I've tried it=20 according to the instructions (via the web) Scripted documents and I = get an=20 "unkown_pending(MZ" error?
 
Michael E. Collins
DOIM, Senior Network = Tech.
200 Henry Ave.
Ft. Riley, KS 66442
mike.collins@riley.army.mil<= /A>
Phone: = 785-239-3662         DSN:=20 856-3662
Fax:=20 785-239-0555          =   =20 Online Fax:=20 785-239-6660
------_=_NextPart_001_01C21C56.CACF65B0-- ------_=_NextPart_000_01C21C56.CACF65B0 Content-Type: image/jpeg; name="Notebook.jpg" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="Notebook.jpg" Content-ID: <204501721@24062002-2f4b> /9j/4AAQSkZJRgABAgEASABIAAD/7QSyUGhvdG9zaG9wIDMuMAA4QklNA+kAAAAAAHgAAwAAAEgA SAAAAAADBgJS//f/9wMPAlsDRwUoA/wAAgAAAEgASAAAAAAC2AIoAAEAAABkAAAAAQADAwMAAAAB Jw8AAQABAAAAAAAAAAAAAAAAYAgAGQGQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA4 QklNA+0AAAAAABAASAAAAAEAAQBIAAAAAQABOEJJTQPzAAAAAAAIAAAAAAAAAAA4QklNBAoAAAAA AAEAADhCSU0nEAAAAAAACgABAAAAAAAAAAI4QklNA/UAAAAAAEgAL2ZmAAEAbGZmAAYAAAAAAAEA L2ZmAAEAoZmaAAYAAAAAAAEAMgAAAAEAWgAAAAYAAAAAAAEANQAAAAEALQAAAAYAAAAAAAE4QklN A/gAAAAAAHAAAP////////////////////////////8D6AAAAAD///////////////////////// ////A+gAAAAA/////////////////////////////wPoAAAAAP////////////////////////// //8D6AAAOEJJTQQAAAAAAAACAAA4QklNBAIAAAAAAAIAADhCSU0ECAAAAAAAEAAAAAEAAAJAAAAC QAAAAAA4QklNBAkAAAAAAqIAAAABAAAAgAAAAAIAAAGAAAADAAAAAoYAGAAB/9j/4AAQSkZJRgAB AgEASABIAAD//gAnRmlsZSB3cml0dGVuIGJ5IEFkb2JlIFBob3Rvc2hvcKggNC4wAP/uAA5BZG9i ZQBkgAAAAAH/2wCEAAwICAgJCAwJCQwRCwoLERUPDAwPFRgTExUTExgRDAwMDAwMEQwMDAwMDAwM DAwMDAwMDAwMDAwMDAwMDAwMDAwBDQsLDQ4NEA4OEBQODg4UFA4ODg4UEQwMDAwMEREMDAwMDAwR DAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDP/AABEIAAIAgAMBIgACEQEDEQH/3QAEAAj/xAE/ AAABBQEBAQEBAQAAAAAAAAADAAECBAUGBwgJCgsBAAEFAQEBAQEBAAAAAAAAAAEAAgMEBQYHCAkK CxAAAQQBAwIEAgUHBggFAwwzAQACEQMEIRIxBUFRYRMicYEyBhSRobFCIyQVUsFiMzRygtFDByWS U/Dh8WNzNRaisoMmRJNUZEXCo3Q2F9JV4mXys4TD03Xj80YnlKSFtJXE1OT0pbXF1eX1VmZ2hpam tsbW5vY3R1dnd4eXp7fH1+f3EQACAgECBAQDBAUGBwcGBTUBAAIRAyExEgRBUWFxIhMFMoGRFKGx QiPBUtHwMyRi4XKCkkNTFWNzNPElBhaisoMHJjXC0kSTVKMXZEVVNnRl4vKzhMPTdePzRpSkhbSV xNTk9KW1xdXl9VZmdoaWprbG1ub2JzdHV2d3h5ent8f/2gAMAwEAAhEDEQA/APROif0Kv6X81T9L j+ar/m/5K0F8rJJIfqlJfKySKn6pSXyskkp+qUl8rJJKfqlJfKySSn6pSXyskkp+qUl8rJJKfqlJ fKySSn//2ThCSU0EBgAAAAAABwABAAAAAQEA//4AJ0ZpbGUgd3JpdHRlbiBieSBBZG9iZSBQaG90 b3Nob3CoIDQuMAD/7gAOQWRvYmUAZIAAAAAB/9sAhAAMCAgNCQ0VDAwVGhQQFBogGxoaGyAiFxcX FxciEQwMDAwMDBEMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMAQ0NDREOERsRERsUDg4OFBQO Dg4OFBEMDAwMDBERDAwMDAwMEQwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCAAYBaAD ASIAAhEBAxEB/90ABABa/8QBPwAAAQUBAQEBAQEAAAAAAAAAAwABAgQFBgcICQoLAQABBQEBAQEB AQAAAAAAAAABAAIDBAUGBwgJCgsQAAEEAQMCBAIFBwYIBQMMMwEAAhEDBCESMQVBUWETInGBMgYU kaGxQiMkFVLBYjM0coLRQwclklPw4fFjczUWorKDJkSTVGRFwqN0NhfSVeJl8rOEw9N14/NGJ5Sk hbSVxNTk9KW1xdXl9VZmdoaWprbG1ub2N0dXZ3eHl6e3x9fn9xEAAgIBAgQEAwQFBgcHBgU1AQAC EQMhMRIEQVFhcSITBTKBkRShsUIjwVLR8DMkYuFygpJDUxVjczTxJQYWorKDByY1wtJEk1SjF2RF VTZ0ZeLys4TD03Xj80aUpIW0lcTU5PSltcXV5fVWZnaGlqa2xtbm9ic3R1dnd4eXp7fH/9oADAMB AAIRAxEAPwCv0T+n4/8AxrP+qavW15J0U/r+P/xrP+qavWg8eKElsWSHZfXWYe4A+ZUMjIFTJBE/ Fc1kXbg63mJP+amk0uesGqdc19Sup2ZrLmWGQxwLR4B35v8A0V0qKlJJJIqUkkkkpSSSSSlJJJJK UkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSS SSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJ KUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpS SSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJ JKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkp SSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJJJKUkkkkpSSSSSlJJ JJKUkkkkpSSSSSn/0J9G6oKn04zKKXOdYA6x7d1kOP8Ag/3Hs/MXY/sOl/0hYfCT/wCQavnZJArQ /S1HTXVN21+weYa7/vqzcroeQ+Q3XcYOn/mbF89pIaJfpboXRK+k1uDQPUsILyONPotZ/JatRfKq SSX6qSXyqkip+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJ KfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp +qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6 qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqp JfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl 8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXy qkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKq SSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJ KfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp +qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn6 qSXyqkkp+qkl8qpJKfqpJfKqSSn6qSXyqkkp+qkl8qpJKfqpJfKqSSn/2Q== ------_=_NextPart_000_01C21C56.CACF65B0-- From niki@vintech.bg Wed Jun 26 02:36:14 2002 Received: from www.vintech.bg (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5Q7aBa17622 for ; Wed, 26 Jun 2002 02:36:11 -0500 Received: from vintech.bg (sierra [10.0.0.7]) by www.vintech.bg (8.11.6/8.8.7) with ESMTP id g5Q7tww04176 for ; Wed, 26 Jun 2002 10:56:00 +0300 Message-ID: <3D196FB9.7000209@vintech.bg> Date: Wed, 26 Jun 2002 10:39:37 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0rc3) Gecko/20020523 X-Accept-Language: bg, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] converting 2.01 - 2.4.7 loses string columns Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I tryed new 2.4.7 Mk4py on some of my projects. On some of them when upgrading file format like this: import Mk4py db = Mk4py.storage('zz.mkt',1) db.commit() new storage lost data in all string columns. If i remove second (bigger) view all is fine. This is on linux and windows with python 2.1.3. Samle data and test program on request. If i export data to text file and then import is all seems to be ok. regards, Niki Spahiev From shmul@vself.com Thu Jun 27 08:00:12 2002 Received: from neo.vself.com ([192.117.144.254]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g5RD0Ba29831 for ; Thu, 27 Jun 2002 08:00:12 -0500 Received: from EXPERIENCE.vself.com (EXPERIENCE [10.0.0.80]) by neo.vself.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id MYWMRR69; Thu, 27 Jun 2002 16:01:35 +0200 Message-Id: <5.1.0.14.0.20020627062811.030faea8@10.0.0.3> X-Sender: shmul@10.0.0.3 X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Thu, 27 Jun 2002 07:01:44 -0700 To: metakit@equi4.com From: Shmulik Regev Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Why is Find so slow? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Running with a profiler, we discovered that calls to View::Find (on a hashed view) took a considerable amount of time mainly due to the copy of the key RowRef. Is this copying really required ? Is there a way to speed it up by passing in an already allocated raw (instead of allocating and destroying it every call) ? Cheers, Shmul From mbloore@yahoo.com Thu Jun 27 09:38:05 2002 Received: from web14311.mail.yahoo.com (web14311.mail.yahoo.com [216.136.224.61]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g5REc5a06491 for ; Thu, 27 Jun 2002 09:38:05 -0500 Message-ID: <20020627143801.44467.qmail@web14311.mail.yahoo.com> Received: from [64.229.17.230] by web14311.mail.yahoo.com via HTTP; Thu, 27 Jun 2002 07:38:01 PDT Date: Thu, 27 Jun 2002 07:38:01 -0700 (PDT) From: mARK bLOORE To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] hashed view problem Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i am using metakit 2.4.7 with C++ under Linux. if i make a hashed view, thus: char* desc = "info[folder:S,files:V[name:S,size:L]]"; c4_View infoRaw = infoStore.GetAs (desc); c4_View infoHash = infoStore.GetAs ("info_hash[_H:I,_R:I]"); c4_View info = infoRaw.Hash (infoHash, 1); i get a "pure virtual function called" error upon accessing the sixth file in any folder in 'info'. does this have something to do with the subview, or am i just doing this wrong? i got the code from http://www.equi4.com/metakit/wiki.cgi/127.html, where it is described as "untested". __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com From FabienB@gilian.com Tue Jul 2 12:10:48 2002 Received: from gilianmail.gilian.com (gw-guilian.ser.netvision.net.il [199.203.171.166]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g62HAka03820 for ; Tue, 2 Jul 2002 12:10:47 -0500 Received: by gilianmail with Internet Mail Service (5.5.2650.21) id ; Tue, 2 Jul 2002 19:57:18 +0300 Message-ID: <1797E7C8F378D411A1CE00D0B77E59CEC7C538@gilianmail> From: Fabien Benichou To: "'metakit@equi4.com'" Date: Tue, 2 Jul 2002 19:57:17 +0300 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="windows-1255" Subject: [Metakit] How to set the same value of a property in all the rows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I need to set the same value of a given property in all the rows of my view (or table). I wrote the following function: void SetAllValues(c4_View &v, const c4_Property &prop, const c4_Bytes &value) { long num_rows = v.GetSize(); for (long i = 0; i < num_rows; i++) prop(v[i]).SetData(value); } For sure, it is really not optimized!!! Do you know a way to optimize it? (Is there is any other way???) Thx, Fabien. From jcw@equi4.com Tue Jul 2 12:24:20 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g62HOJa04986 for ; Tue, 2 Jul 2002 12:24:19 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] How to set the same value of a property in all the rows Date: Tue, 2 Jul 2002 19:20:04 +0200 Message-Id: <20020702172004.5748@triqs.com> In-Reply-To: <1797E7C8F378D411A1CE00D0B77E59CEC7C538@gilianmail> References: <1797E7C8F378D411A1CE00D0B77E59CEC7C538@gilianmail> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Fabien Benichou wrote: >I need to set the same value of a given property in all the rows of my view >(or table). >I wrote the following function: > > void SetAllValues(c4_View &v, const c4_Property &prop, const >c4_Bytes &value) > { > long num_rows = v.GetSize(); Try adding: if (num_rows > 0) prop(v[num_rows-1]).SetData(value); > for (long i = 0; i < num_rows; i++) > prop(v[i]).SetData(value); > } > >For sure, it is really not optimized!!! >Do you know a way to optimize it? (Is there is any other way???) I'm getting that you're setting string or byte properties, which are variable-sized. If the above does not help, then I'm afraid there is little else to do. What number of rows, what performance, what expectations do you have? -jcw From jcw@equi4.com Tue Jul 2 12:28:51 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g62HSoa05412 for ; Tue, 2 Jul 2002 12:28:50 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] hashed view problem Date: Tue, 2 Jul 2002 19:27:16 +0200 Message-Id: <20020702172716.20092@triqs.com> In-Reply-To: <20020627143801.44467.qmail@web14311.mail.yahoo.com> References: <20020627143801.44467.qmail@web14311.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: >i am using metakit 2.4.7 with C++ under Linux. > >if i make a hashed view, thus: > > char* desc = "info[folder:S,files:V[name:S,size:L]]"; Please remove the ":V". I'm not sure it is allowed there. > c4_View infoRaw = infoStore.GetAs (desc); > c4_View infoHash = infoStore.GetAs ("info_hash[_H:I,_R:I]"); > c4_View info = infoRaw.Hash (infoHash, 1); > >i get a "pure virtual function called" error upon accessing the sixth >file in any folder in 'info'. > >does this have something to do with the subview, or am i just doing >this wrong? Looks good to me, apart from the above comment. >i got the code from http://www.equi4.com/metakit/wiki.cgi/127.html, >where it is described as "untested". It may have been back then, but it should be ok now. I'm surprised by this error - are you able to reproduce it? How did you build it? As for examples, look also at files in the examples/ directory in the source distribution, there are by now more examples of hashing, also in C++. -jcw From jcw@equi4.com Tue Jul 2 12:25:13 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g62HPBa05083 for ; Tue, 2 Jul 2002 12:25:11 -0500 Resent-Message-Id: <200207021725.g62HPBa05083@trixie.triqs.com> From: Jean-Claude Wippler (by way of Jean-Claude Wippler ) To: Shmulik Regev Subject: Re: [Metakit] Why is Find so slow? Date: Tue, 2 Jul 2002 19:22:48 +0200 Message-Id: <20020702172249.15982@triqs.com> In-Reply-To: <5.1.0.14.0.20020627062811.030faea8@10.0.0.3> References: <5.1.0.14.0.20020627062811.030faea8@10.0.0.3> X-Mailer: CTM PowerMail 3.1.2 Resent-Date: Tue, 2 Jul 2002 19:23:39 +0200 Resent-From: Jean-Claude Wippler MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Shmulik Regev wrote: >Running with a profiler, we discovered that calls to View::Find (on a >hashed view) took a considerable amount of time mainly due to the copy of >the key RowRef. Is this copying really required ? Is there a way to speed >it up by passing in an already allocated raw (instead of allocating and >destroying it every call) ? I'm not sure. At some point I know I had to use copies, because the underlying data was too volatile to keep as reference (if it points to a tmep buf, and fetching uses that temp buf too, then the find/compare will cause trouble). I'm sure there are cases where one could improve on this. If you are willing to construct a stand-alone example in C++, Python, or Tcl, I can try to optimize. -jcw From jcw@equi4.com Tue Jul 2 12:50:21 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g62HoKa07065 for ; Tue, 2 Jul 2002 12:50:21 -0500 From: Jean-Claude Wippler To: metakit Date: Tue, 2 Jul 2002 19:45:07 +0200 Message-Id: <20020702174507.13548@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Re: wrapped scripted document with tclkit.exe Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: >I'm very happy to create now a wrapped scripted document with tclkit.exe, >so I can put the tclkit.exe in the programm.exe. >On windows I see a small picture (Icon) with the letters TK in the >filemaneger or if I put it on the desktop I see this Icon there. > >Is the a function to include another icon into the wrapped exe to show >on the desctop? No. I think older MSVC (5? 4?) can take an exe and replace the icon, but that's about it. It's a resource (in the Win32 sense) inside the executable. I'd love to have a way to set that icon. Maybe a hard binary patch (from tcl) will do, but icons seems to take variable space... Open for suggestions on this one. TclKit gets buried more and more. That Tk icon on the exe is one of the last remnants (note that window icons can be replaced with "wm", AFAIK). -jcw From jcw@equi4.com Tue Jul 2 12:52:02 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g62Hq1a07213 for ; Tue, 2 Jul 2002 12:52:02 -0500 From: Jean-Claude Wippler To: "'metakit@equi4.com'" Subject: Re: [Metakit] Is there a problem with creating a single-file wrapped executable ? Date: Tue, 2 Jul 2002 19:50:24 +0200 Message-Id: <20020702175024.6854@triqs.com> In-Reply-To: <73C2ED40DAD3D311934E00500400935F03951E5F@rilex11.riley.army.mil> References: <73C2ED40DAD3D311934E00500400935F03951E5F@rilex11.riley.army.mil> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Collins, Mike DAC DOIM wrote: >I've tried it according to the instructions (via the web) Scripted documents >and I get an "unkown_pending(MZ" error? This is strange - 'MZ" tells me you used a Windows executable. What platform are you on? Where is the error occurring exactly? -jcw From niki@vintech.bg Wed Jul 3 02:18:37 2002 Received: from www.vintech.bg (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g637IXa29333 for ; Wed, 3 Jul 2002 02:18:34 -0500 Received: from vintech.bg (sierra [10.0.0.7]) by www.vintech.bg (8.11.6/8.8.7) with ESMTP id g637eAw12827 for ; Wed, 3 Jul 2002 10:40:10 +0300 Message-ID: <3D22A64D.4030604@vintech.bg> Date: Wed, 03 Jul 2002 10:22:53 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0rc3) Gecko/20020523 X-Accept-Language: bg, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Re: wrapped scripted document with tclkit.exe References: <20020702174507.13548@triqs.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Guenther Fischer wrote: >>Is the a function to include another icon into the wrapped exe to show >>on the desctop? > > > No. I think older MSVC (5? 4?) can take an exe and replace the icon, but > that's about it. It's a resource (in the Win32 sense) inside the executable. Windws NT can do that. Python installers use this. HTH Niki Spahiev From mike.collins@riley.army.mil Wed Jul 3 13:45:04 2002 Received: from rilex06.riley.army.mil (emh1.riley.army.mil [144.246.219.66]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g63Ij3a12297 for ; Wed, 3 Jul 2002 13:45:03 -0500 Received: by emh1.riley.army.mil with Internet Mail Service (5.5.2653.19) id ; Wed, 3 Jul 2002 13:43:18 -0500 Message-ID: <73C2ED40DAD3D311934E00500400935F03951EBA@rilex11.riley.army.mil> From: "Collins, Mike DAC DOIM" To: "'metakit@equi4.com'" Date: Wed, 3 Jul 2002 13:48:23 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] RE: Creating single file executable with TCLKIT.exe, Item 6. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm on Windows 2k using TclKit.exe as the executable. I think I followed that from the instructions with the "-prefix" command. The procedure actually seems to run ok, and creates a new executable but, you try to run the created executable that is the error that pops up in a dialog box. -----Original Message----- Message: 6 From: Jean-Claude Wippler To: "'metakit@equi4.com'" Subject: Re: [Metakit] Is there a problem with creating a single-file wrapped executable ? Date: Tue, 2 Jul 2002 19:50:24 +0200 Collins, Mike DAC DOIM wrote: >I've tried it according to the instructions (via the web) Scripted documents >and I get an "unkown_pending(MZ" error? This is strange - 'MZ" tells me you used a Windows executable. What platform are you on? Where is the error occurring exactly? -jcw --__--__-- From jcw@equi4.com Wed Jul 3 15:30:35 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g63KUYa20988 for ; Wed, 3 Jul 2002 15:30:35 -0500 From: Jean-Claude Wippler To: "'metakit@equi4.com'" Subject: Re: [Metakit] RE: Creating single file executable with TCLKIT.exe, Item 6. Date: Wed, 3 Jul 2002 22:28:09 +0200 Message-Id: <20020703202809.16697@triqs.com> In-Reply-To: <73C2ED40DAD3D311934E00500400935F03951EBA@rilex11.riley.army.mil> References: <73C2ED40DAD3D311934E00500400935F03951EBA@rilex11.riley.army.mil> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Collins, Mike DAC DOIM wrote: >I'm on Windows 2k using TclKit.exe as the executable. [...] >>and I get an "unkown_pending(MZ" error? You may have gotten a flakey build. Please fetch a new build, e.g. If that still fails, I'll have to ask you to email me a transcript of what you did - there must be something silly we're missing. -jcw From whisper@oz.net Fri Jul 5 13:04:03 2002 Received: from serverc.sea.theriver.com (serverC.sea.theriver.com [216.39.128.33]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g65I42a15125 for ; Fri, 5 Jul 2002 13:04:02 -0500 Received: (qmail 28669 invoked from network); 5 Jul 2002 18:04:01 -0000 Received: from sense-sea-megasub-1-1008.oz.net (HELO lion) (216.39.170.247) by serverc.sea.theriver.com with SMTP; 5 Jul 2002 18:04:01 -0000 From: "David LeBlanc" To: Date: Fri, 5 Jul 2002 11:04:01 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Mk Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi; I saw Gordon McMillan's response to my request for help on c.l.py, but I am still unclear on some things. I do have it up and running some of it's demos and such (version 3.4.7-37) on Python 2.2.1 (Win 2k-pro sp2), but there is some confusion about the documentation etc. I recall that in Tcl, one could create a nested view: is this also possible in Python? I did one test that suggests that a nested view got created, but I don't see how to insert values for the sub-view: import metakit as mk db = mk.storage('testfile.mk', 1) vu = db.getas("Packages[category:S,name:S,location:S,lastVisited:S,history[date:S ,result:S]]") vu.append(category='Database Library',name='Metakit', location='www.equi4.com/metakit', lastVisited='20020703') #vu.append.history(date='20020704', result='new') vu.append(category='Base Language',name='Python',location='www.python.org',lastVisited='20020704') rec = ['GUI-Framework', 'WxPython', 'www.wxpython.org', '20020704'] #rec = ['GUI-Framework', 'WxPython', 'www.wxpython.org', '20020704', ['20020704', 'new']] idx = vu.append(rec) db.commit() mk.dump(vu) ppty = mk.property('S','version') vu.addproperty(ppty) for s in vu.structure(): print s.type, s.name This is the output: category name location lastVisited history ---------------- -------- -------------------- ----------- ------- Database Library Metakit www.equi4.com/met... 20020703 0 rows Base Language Python www.python.org 20020704 0 rows GUI-Framework WxPython www.wxpython.org 20020704 0 rows Database Library Metakit www.equi4.com/met... 20020703 0 rows Base Language Python www.python.org 20020704 0 rows GUI-Framework WxPython www.wxpython.org 20020704 0 rows ---------------- -------- -------------------- ----------- ------- Total: 6 rows S category S name S location S lastVisited V history S version While "history" seems to be a _V_iew, attempting to get/set it's fields doesn't seem to be possible. What i'm shooting for here is a repeating group of history events. Is this even possible for Mk? Gordon, I don't see what you mean by "insert/append the containing row" - isn't that what I'm doing in the above code snippet? I also had some other questions: Can you permanently modify the view of an existing database? How? Gordon, how is the getas(...) going to modify the view without losing data? How do you put a pickle in a database? Are the various attributes of that pickle accessable while it's in metakit storage? How about the pickled object's attributes? How would you enforce a uniqueness constraint on a key field? Actually, this last question brought up an idea or two: 1. Allow the attachment of snippets of python script that trigger on read/write/update event(s) on a per-view and per-column(property) basis. This would allow a user to create arbitrary constraints as they choose. It also would not cost performance if not used. 2. Allow a Python class to be the type of a column. This class would have to be descended from an Mk class with the necessary primative read/write/?whatever? interfaces to fit into the Mk type scheme (and perhaps have other constraints as well). Date type anyone? :). This idea comes from an interest in experimenting with attributed objects where each object can have differing numbers of attributes. They could be saved as pickles, but I doubt that would offer me access to the state of the object that's been pickled (perhaps I'm wrong about that though...). If the current python interface does not support Mk features that would make any of this possible, have you considered the Boost library? It allows C++ code to instantiate python classes and vice versa and for C++ code to have python script callbacks. Regards, David LeBlanc Seattle, WA USA From gmcm@hypernet.com Fri Jul 5 13:46:05 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g65Ik5a16988 for ; Fri, 5 Jul 2002 13:46:05 -0500 Received: from PARANOIA (204.176.40.43) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Fri, 5 Jul 2002 14:45:04 -0400 From: "Gordon McMillan" To: "David LeBlanc" , Date: Fri, 5 Jul 2002 14:45:34 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Mk Python Reply-to: gmcm@hypernet.com Message-ID: <3D25B10E.10294.3F53A60@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 5 Jul 2002 at 11:04, David LeBlanc wrote: > I recall that in Tcl, one could create a nested > view: is this also possible in Python? Yes. > I did one > test that suggests that a nested view got created, > but I don't see how to insert values for the > sub-view: > db.getas("Packages[category:S,name:S,location:S,lastVis > ited:S,history[date:S ,result:S]]") > > vu.append(category='Database Library',name='Metakit', > location='www.equi4.com/metakit', > lastVisited='20020703') > #vu.append.history(date='20020704', result='new') ndx = vu.append(category=...) vu[ndx].history.append(....) [...] > I also had some other questions: > > Can you permanently modify the view of an existing > database? How? Gordon, how is the getas(...) going to > modify the view without losing data? If you delete a column or change it's type, you will lose data. Only the first "getas" will affect the ordering of the columns. All else follows from that. > How do you put a pickle in a database? Are the various > attributes of that pickle accessable while it's in > metakit storage? How about the pickled object's > attributes? Already answered - use type B. If you didn't understand, substitute "file" for "metakit storage" and answer it yourself <0.01 wink>. > How would you enforce a uniqueness constraint on a > key field? Answered that, too. > Actually, this last question brought up an idea or two: > > 1. Allow the attachment of snippets of python > script that trigger on read/write/update event(s) on > a per-view and per-column(property) basis. This > would allow a user to create arbitrary constraints > as they choose. It also would not cost performance > if not used. It's easy to wrap a view in a Python object and do this. There's no Python in Metakit, however. It's all in Mk4py. > 2. Allow a Python class to be the type of a column. Again, Metakit is (scripting) language agnostic, and only knows about primitive types. If the attribute names and types match, you can already do view.append(instance) The view will look for it's columns names as attrs of instance. > This idea comes from an interest in experimenting > with attributed objects where each object can have > differing numbers of attributes. Metakit views are homogeneous. > They could be saved as pickles, but I doubt that > would offer me access to the state of the object > that's been pickled (perhaps I'm wrong about that > though...). No, you are correct. > If the current python interface does not support Mk > features that would make any of this possible, have > you considered the Boost library? It allows C++ code > to instantiate python classes and vice versa and for > C++ code to have python script callbacks. It will also mean that Mk4py.so / .dll would be huge, and that your Python would have to be compiled with C++. That kills my interest. In fact, I wrote scxx because CXX has those problems. -- Gordon http://www.mcmillan-inc.com/ From whisper@oz.net Fri Jul 5 14:07:36 2002 Received: from serverc.sea.theriver.com (serverC.sea.theriver.com [216.39.128.33]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g65J7Za17896 for ; Fri, 5 Jul 2002 14:07:35 -0500 Received: (qmail 29822 invoked from network); 5 Jul 2002 19:07:34 -0000 Received: from sense-sea-megasub-1-1008.oz.net (HELO lion) (216.39.170.247) by serverc.sea.theriver.com with SMTP; 5 Jul 2002 19:07:34 -0000 From: "David LeBlanc" To: Subject: RE: [Metakit] Mk Python Date: Fri, 5 Jul 2002 12:07:34 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <3D25B10E.10294.3F53A60@localhost> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of Gordon McMillan > Sent: Friday, July 05, 2002 11:46 > To: David LeBlanc; metakit@equi4.com > Subject: Re: [Metakit] Mk Python > > > On 5 Jul 2002 at 11:04, David LeBlanc wrote: > > I did one > > test that suggests that a nested view got created, > > but I don't see how to insert values for the > > sub-view: > > > db.getas("Packages[category:S,name:S,location:S,lastVis > > ited:S,history[date:S ,result:S]]") > > > > vu.append(category='Database Library',name='Metakit', > > location='www.equi4.com/metakit', > > lastVisited='20020703') > > #vu.append.history(date='20020704', result='new') > > ndx = vu.append(category=...) > vu[ndx].history.append(....) > I'm sorry, maybe I'm being obtuse, but this still isn't clear to me. What does ndx have to do with it, and how does ndx end up in the db? What is the type of ndx? The doc has a very brief reference to "rowrefs" and that mentions the word "ndx", but that's about it. > > I also had some other questions: > > Actually, this last question brought up an idea or two: > > > > 1. Allow the attachment of snippets of python > > script that trigger on read/write/update event(s) on > > a per-view and per-column(property) basis. This > > would allow a user to create arbitrary constraints > > as they choose. It also would not cost performance > > if not used. > > It's easy to wrap a view in a Python object and do > this. There's no Python in Metakit, however. It's > all in Mk4py. Yes, but it doesn't intrinsically support retaining the code as part of the db. > > 2. Allow a Python class to be the type of a column. > > Again, Metakit is (scripting) language agnostic, and > only knows about primitive types. If the attribute > names and types match, you can already do > view.append(instance) > The view will look for it's columns names as attrs > of instance. Actually, a "ScriptLanguage="myfavoritescriptinglanguage" would keep it agnostic :-) I wasn't able to figure out how to use an instance (see code above) where nested data came into play. I sort of hoped/expected that an Mk row would be like a list and nested data like a list nested in the outer list. > > This idea comes from an interest in experimenting > > with attributed objects where each object can have > > differing numbers of attributes. > > Metakit views are homogeneous. Huh? > > If the current python interface does not support Mk > > features that would make any of this possible, have > > you considered the Boost library? It allows C++ code > > to instantiate python classes and vice versa and for > > C++ code to have python script callbacks. > > It will also mean that Mk4py.so / .dll would be huge, > and that your Python would have to be compiled > with C++. That kills my interest. In fact, I wrote > scxx because CXX has those problems. Granted mk.dll would be larger, but that much larger? Do added benefits outweigh the size cost? I also don't see why Python would have to be compiled as a C++ app in order to use Boost? I would expect some "extern C" wrappers to cope with that...? I had the impression there where many things you could do with a Boost-ed interface that was not possible with other C/C++<->python adaptors. > -- Gordon > http://www.mcmillan-inc.com/ > From gmcm@hypernet.com Fri Jul 5 14:55:50 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g65Jtoa19825 for ; Fri, 5 Jul 2002 14:55:50 -0500 Received: from PARANOIA (204.176.40.43) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Fri, 5 Jul 2002 15:55:32 -0400 From: "Gordon McMillan" To: "David LeBlanc" , Date: Fri, 5 Jul 2002 15:56:06 -0400 MIME-Version: 1.0 Subject: RE: [Metakit] Mk Python Reply-to: gmcm@hypernet.com Message-ID: <3D25C196.19746.435CDAF@localhost> Priority: normal In-reply-to: References: <3D25B10E.10294.3F53A60@localhost> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 5 Jul 2002 at 12:07, David LeBlanc wrote: > > ndx = vu.append(category=...) > > vu[ndx].history.append(....) > I'm sorry, maybe I'm being obtuse, but this still > isn't clear to me. What does ndx have to do with it, > and how does ndx end up in the db? What is the type > of ndx? The doc has a very brief reference to > "rowrefs" and that mentions the word "ndx", but > that's about it. A view is a sequence. From this one can deduce that ndx is an integer. Or you could poke at it in the interpreter. > I also don't see why Python would have to be > compiled as a C++ app in order to use Boost? Static initializers. If you would like to do a Boost-ed Python binding to Metakit, please do. -- Gordon http://www.mcmillan-inc.com/ From jcw@equi4.com Fri Jul 5 17:10:55 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g65MAsa26103; Fri, 5 Jul 2002 17:10:54 -0500 From: Jean-Claude Wippler To: metakit , announce Date: Sat, 6 Jul 2002 00:10:23 +0200 Message-Id: <20020705221023.12229@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] TclKit 8.4b1 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For those with an interest in TclKit, I've built and uploaded new versions based on the Tcl/Tk 8.4b1 release on SourceForge (well, I used CVS, actually): http://www.equi4.com/tclkit/ These builds are based on the "genkit" script, which is available along with the CVS snapshots used to create these builds - see the above URL for details. TclKit is now considered stable, it will essentially only track updates of the various components from now on. There will be no release next month BTW, due to the summer recess. -jcw From FabienB@gilian.com Sun Jul 7 05:06:41 2002 Received: from gilianmail.gilian.com (gw-guilian.ser.netvision.net.il [199.203.171.166]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g67A6da05629 for ; Sun, 7 Jul 2002 05:06:40 -0500 Received: by gilianmail with Internet Mail Service (5.5.2650.21) id ; Sun, 7 Jul 2002 13:09:45 +0300 Message-ID: <1797E7C8F378D411A1CE00D0B77E59CEC7C54B@gilianmail> From: Fabien Benichou To: "'metakit@equi4.com'" Date: Sun, 7 Jul 2002 13:09:44 +0300 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] RE: metakit digest, Vol 1 #241 - 9 msgs Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: HI, here are my answers/questions: > Try adding: > if (num_rows > 0) > prop(v[num_rows-1]).SetData(value); I'm not sure I get it. Does setting the value of last row will set all the others with the same value?? > I'm getting that you're setting string or byte properties, which are > variable-sized. Some values are string properties other are long properties. Does it make a difference? If yes, do you have a better optimization for properties of fixed-sized > What number of rows, what performance, what > expectations do you have? Number of rows: ~10 000 Performance: the best that I can have bec. I use the method SetAllValues very often in my code Thx, Fabien. ---------------------------------------------------------------------------- --------------- Message: 2 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] How to set the same value of a property in all the rows Date: Tue, 2 Jul 2002 19:20:04 +0200 Fabien Benichou wrote: >I need to set the same value of a given property in all the rows of my view >(or table). >I wrote the following function: > > void SetAllValues(c4_View &v, const c4_Property &prop, const >c4_Bytes &value) > { > long num_rows = v.GetSize(); Try adding: if (num_rows > 0) prop(v[num_rows-1]).SetData(value); > for (long i = 0; i < num_rows; i++) > prop(v[i]).SetData(value); > } > >For sure, it is really not optimized!!! >Do you know a way to optimize it? (Is there is any other way???) I'm getting that you're setting string or byte properties, which are variable-sized. If the above does not help, then I'm afraid there is little else to do. What number of rows, what performance, what expectations do you have? -jcw From FabienB@gilian.com Sun Jul 7 05:24:51 2002 Received: from gilianmail.gilian.com (gw-guilian.ser.netvision.net.il [199.203.171.166]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g67AOoa05886 for ; Sun, 7 Jul 2002 05:24:50 -0500 Received: by gilianmail with Internet Mail Service (5.5.2650.21) id ; Sun, 7 Jul 2002 13:27:56 +0300 Message-ID: <1797E7C8F378D411A1CE00D0B77E59CEC7C54C@gilianmail> From: Fabien Benichou To: metakit@equi4.com Date: Sun, 7 Jul 2002 13:27:55 +0300 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="windows-1255" Subject: [Metakit] New (eventual) usefule method Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, As I needed to get from an index in a base view, the corresponding index in a derived view (the same as the method c4_View::GetIndexOf but in the other way) I added the following code. It may be useful for other people, so perhaps you may insert this change in your code...??? File mk4.h, Ln 279 (class c4_View): ... int GetIndexOf(const c4_RowRef&) const; int GetRevIndexOf(const c4_RowRef&) const; <- line added ... File mk4.h, Ln 851 (class c4_Sequence): ... virtual int RemapIndex(int, const c4_Sequence*) const; virtual int GetRevIndex(int) const; <- line added ... File view.cpp, Ln 833: /** Return the index of the container row in the derived view (or -1) <- line added * <- line added */ <- line added int c4_View::GetRevIndexOf(const c4_RowRef& row_) const <- line added { <- line added c4_Cursor cursor = &row_; <- line added return _seq->GetRevIndex(cursor._index); <- line added } <- line added File derived.cpp, Ln 50 (class c4_FilterSeq) ... virtual int RemapIndex(int, const c4_Sequence*) const; virtual int GetRevIndex(int) const; <- line added ... File derived.cpp, Ln 263 int c4_FilterSeq::GetRevIndex(int index_) const <- line added { <- line added return _revMap.GetAt(index_); <- line added } <- line added File store.h, Ln 75(class c4_DerivedSeq) ... virtual int RemapIndex(int, const c4_Sequence*) const; virtual int GetRevIndex(int) const; <- line added ... File store.cpp, Ln 452 int c4_DerivedSeq::GetRevIndex(int index_) const <- line added { <- line added return index_; <- line added } <- line added File viewx.cpp, Ln 151 int c4_Sequence::GetRevIndex(int index_) const <- line added { <- line added return index_; <- line added } <- line added Fabien Benichou Software Engineer Gilian Technologies "The last line of defense against hacker sabotage!" www.gilian.com Tel: 972-9-956-0036 x233 FAX: 972-9-956-5668 From jcw@equi4.com Sun Jul 7 05:32:56 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g67AWta06012 for ; Sun, 7 Jul 2002 05:32:55 -0500 From: Jean-Claude Wippler To: "'metakit@equi4.com'" Subject: Re: [Metakit] RE: metakit digest, Vol 1 #241 - 9 msgs Date: Sun, 7 Jul 2002 12:32:34 +0200 Message-Id: <20020707103234.3971@triqs.com> In-Reply-To: <1797E7C8F378D411A1CE00D0B77E59CEC7C54B@gilianmail> References: <1797E7C8F378D411A1CE00D0B77E59CEC7C54B@gilianmail> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Fabien Benichou wrote: >> Try adding: >> if (num_rows > 0) >> prop(v[num_rows-1]).SetData(value); > >I'm not sure I get it. Does setting the value of last row will set all the >others with the same value?? No, you still need the loop. But in the case of strings, it pre- allocates a bit more (only if the number of rows changes, btw). >Some values are string properties other are long properties. Does it make a >difference? If yes, do you have a better optimization for properties of >fixed-sized Yes, fixed size is far more efficient. Keep in mind that MK stores its data column-wise. >> What number of rows, what performance, what >> expectations do you have? > >Number of rows: ~10 000 >Performance: the best that I can have bec. I use the method SetAllValues >very often in my code In that case, try changing the logic. You could have an extra property saying "use a fixed default", and store that defaults somewhere. That would work efficiently even for strings, because the loop would set an int. To make the most out of this, store 0 and -1 in that int (it forces byte- wide alloc, instead of a slightly less efficient but more compact 0/1 bit field): ... store default value somewhere ... c4_IntProp prop_d ("blah_d"); for (long i = 0; i < num_rows; i++) prop_d (v[i]) = -1; Then, when you store an individual value, store 0. And when you fetch, check the prop_d value to decide whether to use prop or the common default you stored. -jcw From jcw@equi4.com Sun Jul 7 05:45:34 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g67AjXa06186; Sun, 7 Jul 2002 05:45:33 -0500 From: Jean-Claude Wippler To: Jean-Claude Wippler , "'metakit@equi4.com'" Subject: Re: [Metakit] RE: metakit digest, Vol 1 #241 - 9 msgs Date: Sun, 7 Jul 2002 12:45:11 +0200 Message-Id: <20020707104511.17325@triqs.com> In-Reply-To: <20020707103234.3971@triqs.com> References: <20020707103234.3971@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Fabien Benichou wrote: >>Number of rows: ~10 000 >>Performance: the best that I can have bec. I use the method SetAllValues >>very often in my code [me] >In that case, try changing the logic. You could have [...] There's a more efficient way to do this: - store a "generation" counter in each row, next to the data value - store the current generation and default somewhere (once!) - to set all values to a new default: increment current gen and store the new common default - to get a value: get gen counter value from that row if same as current gen, use common default else get stored data value and use that - to store a value: store -1 as gen counter in that row store data value in that row You need to deal with a few boundary conditions and counter rollover but that's about it - all operations will become O(1), i.e. constant time. -jcw From mike.collins@riley.army.mil Mon Jul 8 11:14:50 2002 Received: from rilex06.riley.army.mil (emh1.riley.army.mil [144.246.219.66]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g68GEoa18595 for ; Mon, 8 Jul 2002 11:14:50 -0500 Received: by emh1.riley.army.mil with Internet Mail Service (5.5.2653.19) id ; Mon, 8 Jul 2002 11:13:03 -0500 Message-ID: <73C2ED40DAD3D311934E00500400935F03951EC0@rilex11.riley.army.mil> From: "Collins, Mike DAC DOIM" To: "'metakit@equi4.com'" Date: Mon, 8 Jul 2002 11:18:18 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] RE: Creating single file executable with TCLKIT.exe, Item 6. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks that's, better. I think I had tried this newer version but didn't like the way the errors showed up on the console so I went back to the other version. I have suggestion for you. Rather than create the batch file, since were on Windows anyway why not just create the output file with a new extension like ".tkg" or something else that could just be associated with the tclkit executable. And then with the "-prefix" option have it create the output file with ".exe" . Just a suggestion thanks for you help. -----Original Message----- From: metakit-request@equi4.com [mailto:metakit-request@equi4.com] Sent: Thursday, July 04, 2002 12:01 PM To: metakit@equi4.com Subject: metakit digest, Vol 1 #242 - 3 msgs Send metakit mailing list submissions to metakit@equi4.com To subscribe or unsubscribe via the World Wide Web, visit http://www.equi4.com/mailman/listinfo/metakit or, via email, send a message with subject or body 'help' to metakit-request@equi4.com You can reach the person managing the list at metakit-admin@equi4.com When replying, please edit your Subject line so it is more specific than "Re: Contents of metakit digest..." Today's Topics: 1. RE: Creating single file executable with TCLKIT.exe, Item 6. (Collins, Mike DAC DOIM) 2. Re: RE: Creating single file executable with TCLKIT.exe, Item 6. (Jean-Claude Wippler) --__--__-- Message: 1 From: "Collins, Mike DAC DOIM" To: "'metakit@equi4.com'" Date: Wed, 3 Jul 2002 13:48:23 -0500 Subject: [Metakit] RE: Creating single file executable with TCLKIT.exe, Item 6. I'm on Windows 2k using TclKit.exe as the executable. I think I followed that from the instructions with the "-prefix" command. The procedure actually seems to run ok, and creates a new executable but, you try to run the created executable that is the error that pops up in a dialog box. -----Original Message----- Message: 6 From: Jean-Claude Wippler To: "'metakit@equi4.com'" Subject: Re: [Metakit] Is there a problem with creating a single-file wrapped executable ? Date: Tue, 2 Jul 2002 19:50:24 +0200 Collins, Mike DAC DOIM wrote: >I've tried it according to the instructions (via the web) Scripted documents >and I get an "unkown_pending(MZ" error? This is strange - 'MZ" tells me you used a Windows executable. What platform are you on? Where is the error occurring exactly? -jcw -- __--__-- --__--__-- Message: 2 From: Jean-Claude Wippler To: "'metakit@equi4.com'" Subject: Re: [Metakit] RE: Creating single file executable with TCLKIT.exe, Item 6. Date: Wed, 3 Jul 2002 22:28:09 +0200 Collins, Mike DAC DOIM wrote: >I'm on Windows 2k using TclKit.exe as the executable. [...] >>and I get an "unkown_pending(MZ" error? You may have gotten a flakey build. Please fetch a new build, e.g. If that still fails, I'll have to ask you to email me a transcript of what you did - there must be something silly we're missing. -jcw --__--__-- _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit End of metakit Digest From nicolasb@maich.gr Tue Jul 9 08:17:09 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g69DH8a11867 for ; Tue, 9 Jul 2002 08:17:08 -0500 Received: FROM maich.gr BY mail-server ; Tue Jul 09 16:42:04 2002 +0300 Message-ID: <3D2AE2F6.10904@maich.gr> Date: Tue, 09 Jul 2002 16:19:50 +0300 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Thread Enabled Tclkit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello JC and all, Is it possible to putup a thread enabled version of the current tclkit for windows? sincerely, nicolas boretos From jcw@equi4.com Wed Jul 10 03:16:06 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6A8G1a01027; Wed, 10 Jul 2002 03:16:02 -0500 From: Jean-Claude Wippler To: metakit Cc: Vince Darley , Will Duquette , "Daniel A. Steffen" Date: Wed, 10 Jul 2002 10:15:51 +0200 Message-Id: <20020710081551.9771@triqs.com> X-Mailer: CTM PowerMail 4.0b3 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Starkit mailing list Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For those who have an interest in tclkit and scripted documents... While finishing the name change from scripted documents to starkits, and as a first step to announcing this in public, I thought it might be useful to try to organize some of the email dicussion going on lately into a mailing list. That way I don't have to try to figure out who wants to hear about what, but more importantly there will be a mailing list archive to refer to. This is an invitation to join a new mailing list about starkits, starpacks, and tclkit - if you know of others who might be interested, please feel free to pass this on. The list is managed by mailman, like all other lists I administer - the main page is at: If you are also a member of the metakit@equi4.com mailing list, please note that this new list is about all aspects of tclkit from now on, while the metakit mailing list remains for those who use metakit by itself, from C++, Python, and Tcl. Oh, and just to clarify one more point: this is an attempt to *reduce*, or at least optimize, email traffic and workload, not provide yet another chatty outlet. We all have plenty to do! -jcw From tcoram@pobox.com Wed Jul 10 12:37:05 2002 Received: from localhost.localdomain ([209.225.31.120]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g6AHb5a32022 for ; Wed, 10 Jul 2002 12:37:05 -0500 Message-Id: <200207101737.g6AHb5a32022@trixie.triqs.com> Received: (qmail 2985 invoked from network); 10 Jul 2002 17:36:58 -0000 Received: from localhost (HELO there) (127.0.0.1) by localhost with SMTP; 10 Jul 2002 17:36:58 -0000 Content-Type: text/plain; charset="iso-8859-1" From: Todd Coram To: metakit@equi4.com Date: Wed, 10 Jul 2002 13:36:57 -0400 X-Mailer: KMail [version 1.3.2] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Metakit] Scaling up into the mega-megabytes... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I remember reading that metakit scales up into the megabytes (for medium sized data needs) , but what happens if I crank it up into the 100MB+ range? I have a very simple view (basically I am persisting Tcl arrays) with no real performance requirements. Right now I am running a test where the database is at 60MB and counting...every few seconds a dozen or so strings (less than 100 bytes) are added to the view and a some are removed seconds later. But, some have a long lifetime and this is where the 100MB+ comes into play. I am running a fairly recent version (2.47) from tclkit. What the heck am I doing? Essentially, I am using TclKit to implement a persistent storage backed tuplespace (think: java space). The tuplespace is used for short-lived (but reliable) message transfers (hence the adds and almost immediate deletes) and storing LDAP-like data that is truly persistent. -- todd From jcw@equi4.com Wed Jul 10 12:49:21 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6AHnKa00304 for ; Wed, 10 Jul 2002 12:49:20 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Scaling up into the mega-megabytes... Date: Wed, 10 Jul 2002 19:47:37 +0200 Message-Id: <20020710174737.9266@triqs.com> In-Reply-To: <200207101737.g6AHb5a32022@trixie.triqs.com> References: <200207101737.g6AHb5a32022@trixie.triqs.com> X-Mailer: CTM PowerMail 4.0b3 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Todd Coram wrote: >I remember reading that metakit scales up into the megabytes (for medium >sized data needs) , but what happens if I crank it up into the 100MB+ range? > > I have a very simple view (basically I am persisting Tcl arrays) with no >real performance requirements. Right now I am running a test where the >database is at 60MB and counting...every few seconds a dozen or so strings >(less than 100 bytes) are added to the view and a some are removed seconds >later. But, some have a long lifetime and this is where the 100MB+ comes into >play. > >I am running a fairly recent version (2.47) from tclkit. > >What the heck am I doing? >Essentially, I am using TclKit to implement a persistent storage backed >tuplespace (think: java space). The tuplespace is used for short-lived (but >reliable) message transfers (hence the adds and almost immediate deletes) and >storing LDAP-like data that is truly persistent. You can go higher, but the current Mk4tcl api doesn't make it as easy as Python or C++. There's an object oriented interface, but it's essentially undocumented (I can only point at tcl/mk4tcl.h in the sources). However, all is not desperate. See the examples/ dir in the sources, for example bigblock.tcl, it shows how to set up a "blocked" view. -jcw From jcw@equi4.com Fri Jul 12 05:29:14 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6CATEa19568 for ; Fri, 12 Jul 2002 05:29:14 -0500 From: Jean-Claude Wippler To: Metakit Subject: Re: [Metakit] Thread Enabled Tclkit? Date: Fri, 12 Jul 2002 12:27:52 +0200 Message-Id: <20020712102752.18051@triqs.com> In-Reply-To: <3D2AE2F6.10904@maich.gr> References: <3D2AE2F6.10904@maich.gr> X-Mailer: CTM PowerMail 4.0b3 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas Boretos wrote: >Is it possible to putup a thread enabled version of the current tclkit >for windows? I've had some running threads with the latest tclkits - there seems to be an initialization problem with interps causing tclkit to fail on startup. I'd like to see this resolved, but it most likely won't happen before Tcl 8.4 final (unless someone else steps forward). -jcw From jcw@equi4.com Fri Jul 12 07:54:14 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6CCsDa26825 for ; Fri, 12 Jul 2002 07:54:13 -0500 From: Jean-Claude Wippler To: Metakit Subject: Re: [Metakit] Thread Enabled Tclkit? Date: Fri, 12 Jul 2002 14:52:52 +0200 Message-Id: <20020712125252.5896@triqs.com> In-Reply-To: <20020712102752.18051@triqs.com> References: <20020712102752.18051@triqs.com> X-Mailer: CTM PowerMail 4.0b3 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Whoops.. I wrote: >I've had some running threads with the latest tclkits - there seems to be I meant "I've had some trouble running with threads enabled". -jcw From jcw@equi4.com Fri Jul 12 10:57:24 2002 Received: from [10.0.1.194] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6CFvNa04604; Fri, 12 Jul 2002 10:57:23 -0500 From: Jean-Claude Wippler To: Metakit , Starkit Date: Fri, 12 Jul 2002 17:55:39 +0200 Message-Id: <20020712155539.13891@triqs.com> X-Mailer: CTM PowerMail 4.0b3 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Away 19-7 through 20-8 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Just a little note to say that I'll be away from July 19th through August 20th. Arrangements are being made to take care of the basics, like website access emergencies, but for a while you'll be far more likely to find me on faraway beaches, mountains, treetops, deserts, and seas... than on the net :) -jcw From dbaciu@hotmail.com Mon Jul 15 07:26:17 2002 Received: from hotmail.com (oe68.law4.hotmail.com [216.33.148.164]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6FCQHa13718 for ; Mon, 15 Jul 2002 07:26:17 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Mon, 15 Jul 2002 05:26:11 -0700 X-Originating-IP: [193.231.111.154] From: "Daniel B." To: Date: Mon, 15 Jul 2002 15:29:56 +0300 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_022A_01C22C14.796A1A80" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Message-ID: X-OriginalArrivalTime: 15 Jul 2002 12:26:11.0770 (UTC) FILETIME=[CDFFB1A0:01C22BFA] Subject: [Metakit] Using Selection and Sorting simultaneous Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_022A_01C22C14.796A1A80 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Good day.=20 I just discovered MetaKit, and I started to make a simple DB application = using Borland C++ 5.0. I am using MetaKit 2.4.7.=20 First of all I whant to congratulate you for this grate tool. Ok, now I am stuck, and I can't figure out how to solve one problem. I = need to use selection and filtering simultaneous on the same tables, and = to make maintenance against it. (Is this possible?). I read about some = limitation of the selection viewers...=20 The code I used is: c4_Storage *gpDBStorage gpDBStorage =3D new c4_Storage ("mydb.db", true); [...] c4_IntProp pMonth("Month"), pYear("Year"); c4_View view1, view2, view3; view1 =3D = gpDBStorage->GetAs("invoice[DocNr:I,Month:I,Year:I,Price:F]"); view2 =3D view1.Select (pMonth [giCrtMonth] + pYear = [giCrtYear]); view3 =3D view2.SortOn ((pMonth, pYear)); [...] view3.RemoveAll(); gpDBStorage->Commit(); Neighter RemoveAll called on view2 nor view3 has no effect. I have tried to switch selection with sorting but I get the same rezult: = none.=20 Do I make something wrong or is this a limitation of MetaKit ? I have tried to work around, by takeing the index of the underlying = table but is behaveing strange (I get index of other line that I am = expected): =20 [...] int iX =3D 3; // the index of element to be removed int index =3D view1.GetIndexOf( view3.ElementAt( iX )); view1.SetAt(index, row); Is this correct ? Can someone help me ? Regards,=20 Daniel. ------=_NextPart_000_022A_01C22C14.796A1A80 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
 
Good day.
I just discovered MetaKit, and I started to make = a simple=20 DB application using Borland C++ 5.0.
I am using MetaKit 2.4.7.
First of all I whant to congratulate you = for this=20 grate tool.
 
Ok, now I am stuck, and I can't figure out = how to=20 solve one problem. I need to use selection and filtering = simultaneous=20 on the same tables, and to make maintenance against it. = (Is this=20 possible?). I read about some limitation of the selection=20 viewers... 
The code I used is:
 
        = c4_Storage=20 *gpDBStorage
        = gpDBStorage =3D=20 new c4_Storage ("mydb.db", true);
        [...]
 
       =20 c4_IntProp pMonth("Month"),=20 pYear("Year");
        c4_View = view1,=20 view2, view3;
 
        view1 =3D=20 gpDBStorage->GetAs("invoice[DocNr:I,Month:I,Year:I,Price:F]");<= /DIV>
 
        view2 =3D=20 view1.Select (pMonth [giCrtMonth] + pYear [giCrtYear]);
        = view3 =3D=20 view2.SortOn ((pMonth, pYear));
 
        = [...]

        = view3.RemoveAll();
        = gpDBStorage->Commit();
Neighter RemoveAll called on view2 nor view3 has no = effect.
I have tried to switch selection with sorting but I get the = same=20 rezult: none.
Do I make something wrong or is this a limitation of MetaKit = ?
I have tried to work around, by takeing the index of the underlying = table=20 but is behaveing strange (I get index of other line that I am = expected):
   
        [...]
        int iX =3D 3; // the = index of=20 element to be removed
        int index =3D = view1.GetIndexOf(=20 view3.ElementAt( iX=20 ));
           =     =20 view1.SetAt(index, row);

Is this correct ?
Can someone help me ?
 
Regards,
Daniel.
 
 ------=_NextPart_000_022A_01C22C14.796A1A80-- From imadjunk@hotmail.com Wed Jul 17 09:45:28 2002 Received: from hotmail.com (oe54.law10.hotmail.com [64.4.14.47]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6HEjSa13780 for ; Wed, 17 Jul 2002 09:45:28 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 17 Jul 2002 07:45:22 -0700 X-Originating-IP: [207.180.130.87] From: "Imad Jureidini" To: Date: Wed, 17 Jul 2002 10:45:13 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Message-ID: X-OriginalArrivalTime: 17 Jul 2002 14:45:22.0862 (UTC) FILETIME=[9476B4E0:01C22DA0] Subject: [Metakit] Building metakit with VC7 and q4_STD Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm having problems building metakit 2.4.7 with Visual Studio.NET using the q4_STD flag. I get a couple of warnings regarding signed/unsigned comparisons when building, and some of the regression tests fail, such as: r00 - Simple insert Failed: A(_refData[i] == _prop (_attached[i])) If I build the project using VC6 everything works fine. The VC7 project is created by converting the original VC6 one. This isn't a big deal since I can still use q4_MFC, but it seems like something that should be fixed. Thanks, Imad Jureidini From mbloore@yahoo.com Thu Jul 18 14:09:51 2002 Received: from web14307.mail.yahoo.com (web14307.mail.yahoo.com [216.136.173.83]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g6IJ9na24477 for ; Thu, 18 Jul 2002 14:09:50 -0500 Message-ID: <20020718190945.39583.qmail@web14307.mail.yahoo.com> Received: from [64.229.24.159] by web14307.mail.yahoo.com via HTTP; Thu, 18 Jul 2002 12:09:45 PDT Date: Thu, 18 Jul 2002 12:09:45 -0700 (PDT) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] commit-aside confusion Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i have just started using commit-extend and commit-aside, which promise much good for my application when it needs to do many small updates to a large database with simultaneous readers. i have some confusions from reading the "new transaction modes" page (http://www.equi4.com/metakit/wiki.cgi/82): the second bullet under Commit choices says commit-extend requires mode 1 or 2. should that be mode 2 only? the third bullet says "commit-aside can be stacked. See below for more details" but stacking is not mentioned again. the last para under Commit-extend says there are ways to get the main file to minumum size, but gives no hint as to what they are. the third para under Commit-aside says commits to the main file cause "a 'real' commit on the aside file to be issued". but i find that i have to call commit on the aside file myself. is the doc wrong or should i be doing something differently? in this case i have the main file open in mode 0, and the aside file in mode 2. the second bullet in that section says that a "full" commit will roll aside-file changes into the main file. i have not been able to make this happen, with various combinations of open modes for the two files, and commiting one or both in various orders (with an argument of "true" to Commit). whatever i do, the main and aside files stay the same size. i can still read and update them. __________________________________________________ Do You Yahoo!? Yahoo! Autos - Get free new car price quotes http://autos.yahoo.com From umbright@taltrade.com Mon Jul 22 14:17:41 2002 Received: from talchiexch03.taltrade.com (talchiexch03.taltrade.com [207.48.80.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6MJHea08871 for ; Mon, 22 Jul 2002 14:17:40 -0500 Received: from mail pickup service by talchiexch03.taltrade.com with Microsoft SMTPSVC; Mon, 22 Jul 2002 14:17:39 -0500 x-gfisavedcharset: iso-8859-1 Content-Type: text/plain; charset="iso-8859-1" Received: from talchiexch02.taltrade.com ([10.207.64.102]) by talchiexch03.taltrade.com with Microsoft SMTPSVC(5.0.2195.2966); Mon, 22 Jul 2002 14:17:37 -0500 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Date: Mon, 22 Jul 2002 14:17:28 -0500 Message-ID: <45CE1FBFCDAD2E4DA84B49F2F866FC4F18319E@talchiexch02.taltrade.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: mk4py equivilent of GetIndexOf? Thread-Index: AcIxtHBow4A8/LEfQW6TXTOyOovNtQ== From: "Guy Umbright" To: X-OriginalArrivalTime: 22 Jul 2002 19:17:38.0080 (UTC) FILETIME=[7113EA00:01C231B4] Subject: [Metakit] mk4py equivilent of GetIndexOf? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is there a way to get the index of a row like you can via=20 GetIndexOf in C++? This electronic mail message and any attached files contain information = intended for the exclusive use of the individual or entity to whom it is = addressed and may contain information that is proprietary, privileged, = confidential and/or exempt from disclosure under applicable law. If you = are not the intended recipient, you are hereby notified that any = viewing, copying, disclosure or distribution of this information may be = subject to legal restriction or sanction. Please notify the sender, by = electronic mail or telephone, of any unintended recipients and delete = the original message without making any copies. From tom.krehbiel@motorola.com Wed Jul 24 13:25:14 2002 Received: from motgate4.mot.com (motgate4.mot.com [144.189.100.102]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6OIPEa17193; Wed, 24 Jul 2002 13:25:14 -0500 Received: [from pobox.mot.com (pobox.mot.com [129.188.137.100]) by motgate4.mot.com (motgate4 2.1) with ESMTP id LAA24935; Wed, 24 Jul 2002 11:25:13 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by pobox.mot.com (MOT-pobox 2.0) with ESMTP id LAA06737; Wed, 24 Jul 2002 11:25:12 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id MRC539GX; Wed, 24 Jul 2002 11:25:12 -0700 Message-ID: <3D3EF108.BA740EAA@motorola.com> Date: Wed, 24 Jul 2002 11:25:12 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 To: Mel , starkit@equi4.com, metakit@equi4.com References: <20020724180247.44688.qmail@web12305.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Re: [Starkit] (no subject) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mel wrote: > > itk3.2 is included in the kitten library! > whay to i have to do to include it in my kitten application? > > many thanks > Mel > --- Tom Krehbiel wrote: > > > like the document said, i do 'kitten.kit -l' and i do not see itk > > or > > > itcl listed! > > > > > > am i missing something? > > > > Itcl is include in tclkit but Itk is not include. > > > > -tjk I don't know :-( , I'm still trying to get tclkit to build on Solaris. MetaKit seems to be broken on Solaris. I downloaded metakit-2.4.7-37 from the MetaKit site and it doesn't build so there is a problem with MetaKit. The error looks like this: % ../unix/configure checking for Python configuration... not enabled checking for Tcl headers... not enabled checking for g++... g++ checking for C++ compiler default output... a.out checking whether the C++ compiler works... configure: error: cannot run C++ compiled programs. If you meant to cross compile, use `--host'. On HP I got tclkit to build and I can create a starkit with script that work, but as soon as I try to load a shared object the os complains /usr/lib/dld.sl: Call to mmap() failed - TEXT /var/tmp/tcla02761 /usr/lib/dld.sl: Permission denied 200704couldn't load file "/var/tmp/tcla02761": permission denied while executing "load [file join $root lib Tktable2.7 Tktable.sl.2.7]" (file "test/main.tcl" line 24) invoked from within "source test/main.tcl" ("uplevel" body line 1) invoked from within "uplevel [list source [file join $self main.tcl]]" Any help on either of these problems would be appreciated. Tom K. From lvirden@cas.org Sat Jul 27 14:21:14 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6RJLEa28610 for ; Sat, 27 Jul 2002 14:21:14 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.11.6+Sun/m4_8.11.6/CAS_MAIL_HUB-1.15) with ESMTP id g6RJL7O15196 for ; Sat, 27 Jul 2002 15:21:07 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id g6RJL7908625; Sat, 27 Jul 2002 15:21:07 -0400 (EDT) Date: Sat, 27 Jul 2002 15:21:07 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020727152107.AAB8615@cas.org> To: metakit Subject: [Metakit] segfault in Mk4tcl Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Today on I found this msg: I'm away from my usual mail, so I'm denied permission to post this to the Metakit mailing list. The following simple script segfaults: package require Mk4tcl mk::cursor create I mk::cursor create I exit [2]DGP ---- Anyone have a solution for Don? -- Tcl'2002 Sept 16, 2002, Vancouver, BC http://www.tcl.tk/community/tcl2002/ Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From mmartin6@cfl.rr.com Mon Jul 29 13:28:29 2002 Received: from orllnx01.orl.rbd.com (orllnx01.orl.rbd.com [208.61.212.238]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g6TISTa25364 for ; Mon, 29 Jul 2002 13:28:29 -0500 Received: (qmail 12890 invoked from network); 29 Jul 2002 18:28:29 -0000 Received: from orl-portal01.rbd.com (HELO cfl.rr.com) (208.61.212.254) by orllnx01.orl.rbd.com with SMTP; 29 Jul 2002 18:28:29 -0000 Message-ID: <3D458942.5030906@cfl.rr.com> Date: Mon, 29 Jul 2002 14:28:18 -0400 From: Marcus Martin User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0rc2) Gecko/20020512 Netscape/7.0b1 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Zlib and Metakit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Has anyone ever attempted or succeeded at integrating zlib and metakit together? I was looking to reimpliment the c4_File Strategy using the zlib library. This seems like a topic that must have come up before but I have seen no mention of it in the mailing lists. I do not want to reinvent the wheel, has this been done before? Thanks for the help, Marcus From tom.krehbiel@motorola.com Mon Jul 29 14:02:37 2002 Received: from ftpbox.mot.com (ftpbox.mot.com [129.188.136.101]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g6TJ2ba27613 for ; Mon, 29 Jul 2002 14:02:37 -0500 Received: [from pobox3.mot.com (pobox3.mot.com [10.64.251.242]) by ftpbox.mot.com (ftpbox 2.1) with ESMTP id MAA18625 for ; Mon, 29 Jul 2002 12:02:36 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by pobox3.mot.com (MOT-pobox3 2.0) with ESMTP id MAA04931 for ; Mon, 29 Jul 2002 12:01:04 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id PSDC8MWZ; Mon, 29 Jul 2002 12:02:35 -0700 Message-ID: <3D45914B.1E629C0A@motorola.com> Date: Mon, 29 Jul 2002 12:02:35 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 CC: metakit@equi4.com Subject: Re: [Metakit] Zlib and Metakit References: <3D458942.5030906@cfl.rr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Marcus, > Has anyone ever attempted or succeeded at integrating zlib and metakit > together? I think the answere is yes because the tclkit build for starkit includes zlib and I believe it is used by MetaKit. -tjk From mbloore@yahoo.com Thu Aug 1 12:09:44 2002 Received: from web14305.mail.yahoo.com (web14305.mail.yahoo.com [216.136.173.81]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g71H9ha29894 for ; Thu, 1 Aug 2002 12:09:43 -0500 Message-ID: <20020801170943.56546.qmail@web14305.mail.yahoo.com> Received: from [64.229.24.159] by web14305.mail.yahoo.com via HTTP; Thu, 01 Aug 2002 10:09:43 PDT Date: Thu, 1 Aug 2002 10:09:43 -0700 (PDT) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] changing c4_BytesProp in place? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i have an application that stores data in a large number of c4_Bytes objects (two columns, 98,000 rows). at each update, a few of them get data appended, and much data shuffling ensues. i allocate and fill a new object of the new size, then assign it to the appropriate column and row. it would be better if i could pre-allocate those objects, and fill unused space as needed, only increasing their sizes when space runs out. i can see no obvious way to do this, since c4_Bytes::Contents() returns a pointer to const, and even if i did update that buffer there is no way to tell MetaKit that the data has changed. is there some way to modify a bytes object in place and get it commited? ===== -- mARK bLOORE __________________________________________________ Do You Yahoo!? Yahoo! Health - Feel better, live better http://health.yahoo.com From ac@AcmeDrivers.com Fri Aug 2 08:09:46 2002 Received: from mail12.atl.registeredsite.com (nobody@mail12.atl.registeredsite.com [64.224.219.86]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g72D9ka20246 for ; Fri, 2 Aug 2002 08:09:46 -0500 Received: from mail.acmedrivers.com (mail.acmedrivers.com [216.247.140.7]) by mail12.atl.registeredsite.com (8.12.2/8.12.5) with ESMTP id g72D9jY6009506 for ; Fri, 2 Aug 2002 09:09:45 -0400 Received: from alex [216.247.140.7] by mail.acmedrivers.com with ESMTP (SMTPD32-6.06) id A477836010C; Fri, 02 Aug 2002 09:09:11 -0400 From: "Anthony Corriveau" To: Date: Fri, 2 Aug 2002 09:09:45 -0400 Message-ID: <000201c23a25$e01a82f0$0201000a@alex> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2616 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] why was c4_Storage::Store() dropped? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: why was c4_Storage::Store() dropped? is there a replacement functionality? Anthony Corriveau ac@AcmeDrivers.com From mbloore@yahoo.com Fri Aug 2 10:04:16 2002 Received: from web14306.mail.yahoo.com (web14306.mail.yahoo.com [216.136.173.82]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g72F4Ga26597 for ; Fri, 2 Aug 2002 10:04:16 -0500 Message-ID: <20020802150415.59395.qmail@web14306.mail.yahoo.com> Received: from [64.229.24.159] by web14306.mail.yahoo.com via HTTP; Fri, 02 Aug 2002 08:04:15 PDT Date: Fri, 2 Aug 2002 08:04:15 -0700 (PDT) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] Re: why was c4_Storage::Store() dropped? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Anthony Corriveau wrote: > why was c4_Storage::Store() dropped? > is there a replacement functionality? taking a guess from the extensive documentation, i would say that c4_Storage::Store() let you put an existing view into an existing store. if i'm wrong, stop reading now. depending on what you need to do, this may now take more or less work. (warning, i'm still guessing! i have done only one small part of this.) if you only want the one view in the storage object, you might use c4_Storage::c4_Storage(const c4_View&) and be done in one step. if the storage exists and has other views in it, there is c4_View GetAs(const char *). this adds an empty view. if you don't have or can't hard-code the description string, then you can use c4_View::NumProperties() and c4_View::NthProperty() to go through the columns, and c4_Property::Name() and c4_Property::Type() to build the string. schemas are remarkably malleable in metakit. then, of course, you have to copy all the data. i don't know why the old routine was dropped. __________________________________________________ Do You Yahoo!? Yahoo! Health - Feel better, live better http://health.yahoo.com From ac@AcmeDrivers.com Sat Aug 3 11:04:15 2002 Received: from mail5.atl.registeredsite.com (nobody@mail5.atl.registeredsite.com [64.224.219.79]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g73G4Fa24476 for ; Sat, 3 Aug 2002 11:04:15 -0500 Received: from mail.acmedrivers.com (mail.acmedrivers.com [216.247.140.7]) by mail5.atl.registeredsite.com (8.12.2/8.12.5) with ESMTP id g73G4EZ0030683 for ; Sat, 3 Aug 2002 12:04:14 -0400 Received: from alex [216.247.140.7] by mail.acmedrivers.com with ESMTP (SMTPD32-6.06) id AEDC18390138; Sat, 03 Aug 2002 12:03:40 -0400 From: "Anthony Corriveau" To: Subject: FW: [Metakit] why was c4_Storage::Store() dropped? Date: Sat, 3 Aug 2002 12:04:16 -0400 Message-ID: <000301c23b07$6b72ca10$0201000a@alex> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2616 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: So there is no way to save changes to a view without redefining it with GetAs()? For example if I do this: c4_View vAddress = storage.GetAs("address[name:S,country:S]"); c4_IntProp pAge ("age"); vAddress.AddProperty(pAge ); This change cannot be saved? I have to do this? : vAddress = storage.GetAs("address[name:S,country:S,age:I]"); It seems strange that c4_View and c4_Prop provide operators for easy manipulation and modification, but are rendered useless because the changes cannot be saved. I can write my own Store function, as you indicate below, but I feel like I am missing something. Anthony Corriveau ac@AcmeDrivers.com > -----Original Message----- > From: mARK bLOORE [mailto:mbloore@yahoo.com] > Sent: Friday, August 02, 2002 11:02 AM > To: Anthony Corriveau > Subject: Re: [Metakit] why was c4_Storage::Store() dropped? > > > > --- Anthony Corriveau wrote: > > why was c4_Storage::Store() dropped? > > is there a replacement functionality? > > taking a guess from the extensive documentation, i would say that > c4_Storage::Store() let you put an existing view into an existing > store. if i'm wrong, stop reading now. > > depending on what you need to do, this may now take more or > less work. (warning, i'm still guessing! i have done only > one small part of this.) > > if you only want the one view in the storage object, you > might use c4_Storage::c4_Storage(const c4_View&) and be done > in one step. > > if the storage exists and has other views in it, there is > c4_View GetAs(const char *). this adds an empty view. > > if you don't have or can't hard-code the description string, > then you can use c4_View::NumProperties() and > c4_View::NthProperty() to go through the columns, and > c4_Property::Name() and > c4_Property::Type() to build the string. schemas are > remarkably malleable in metakit. > > then, of course, you have to copy all the data. > > i don't know why the old routine was dropped. > > > ===== > -- > mARK bLOORE > > __________________________________________________ > Do You Yahoo!? > Yahoo! Health - Feel better, live better http://health.yahoo.com > From mbloore@yahoo.com Tue Aug 6 10:13:27 2002 Received: from web14302.mail.yahoo.com (web14302.mail.yahoo.com [216.136.173.78]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g76FDQa17618 for ; Tue, 6 Aug 2002 10:13:26 -0500 Message-ID: <20020806151326.27332.qmail@web14302.mail.yahoo.com> Received: from [64.229.24.159] by web14302.mail.yahoo.com via HTTP; Tue, 06 Aug 2002 08:13:26 PDT Date: Tue, 6 Aug 2002 08:13:26 -0700 (PDT) From: mARK bLOORE Subject: Re: FW: [Metakit] why was c4_Storage::Store() dropped? To: metakit In-Reply-To: <000301c23b07$6b72ca10$0201000a@alex> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Anthony Corriveau wrote: > So there is no way to save changes to a view without redefining it > with > GetAs()? > For example if I do this: > > c4_View vAddress = storage.GetAs("address[name:S,country:S]"); > c4_IntProp pAge ("age"); > vAddress.AddProperty(pAge ); > > This change cannot be saved? > I have to do this? : > > vAddress = storage.GetAs("address[name:S,country:S,age:I]"); >... well, it's easy enough to try! i just did, and the answer is that you can add a property that way, if you Commit() it. on subsequent opens, of course, you need to give the new description string. from your original message i had thought that you wanted to add a view which had been created without a storage object to an existing storage object. that would be different. __________________________________________________ Do You Yahoo!? Yahoo! Health - Feel better, live better http://health.yahoo.com From mbloore@yahoo.com Tue Aug 6 14:31:53 2002 Received: from web14305.mail.yahoo.com (web14305.mail.yahoo.com [216.136.173.81]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g76JVra03994 for ; Tue, 6 Aug 2002 14:31:53 -0500 Message-ID: <20020806193152.9953.qmail@web14305.mail.yahoo.com> Received: from [64.229.24.159] by web14305.mail.yahoo.com via HTTP; Tue, 06 Aug 2002 12:31:52 PDT Date: Tue, 6 Aug 2002 12:31:52 -0700 (PDT) From: mARK bLOORE Subject: RE: FW: [Metakit] why was c4_Storage::Store() dropped? To: metakit In-Reply-To: <000001c23d6d$8f1590c0$0201000a@alex> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Anthony Corriveau wrote: > Any hint as to what I'm doing wrong then? I tried 2.4.7 and am now > using > the CVS kit. > > int main() > { > c4_Storage storage ("test1.dat", true); > c4_IntProp prop1 ("prop1"); > c4_IntProp prop2 ("prop1"); > c4_View view = storage.GetAs("test[prop1:I]"); > view.AddProperty(prop2); > > view.Add(prop1[4321]+prop2[2345]); > storage.Commit(); > > return 0; > } two hints: what does metakit do if two properties in one view have the same name? what does metakit do with the string passed to GetAs()? __________________________________________________ Do You Yahoo!? Yahoo! Health - Feel better, live better http://health.yahoo.com From mbloore@yahoo.com Tue Aug 6 15:57:13 2002 Received: from web14307.mail.yahoo.com (web14307.mail.yahoo.com [216.136.173.83]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g76KvCa10575 for ; Tue, 6 Aug 2002 15:57:12 -0500 Message-ID: <20020806205712.35871.qmail@web14307.mail.yahoo.com> Received: from [64.229.24.159] by web14307.mail.yahoo.com via HTTP; Tue, 06 Aug 2002 13:57:12 PDT Date: Tue, 6 Aug 2002 13:57:12 -0700 (PDT) From: mARK bLOORE Subject: RE: FW: [Metakit] why was c4_Storage::Store() dropped? To: metakit In-Reply-To: <000101c23d83$f68f6d00$0201000a@alex> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i haven't tried mkdump, but when i GetAs with the existing structure, use AddProperty and Commit, then GetAs again with the new structure (in a new run), my added property and its values are there. perhaps mkdump gets confused somehow. i am using 2.4.7 with C++. btw, the second arg to the c4_Storage ctor should be an int. it used to be bool, and bool will work because of the way it gets converted to int. --- Anthony Corriveau wrote: > Oops. fixed the prop name, but it still doesn't save the Added > Property. > > > int main() > { > > c4_Storage storage ("test1.dat", true); > c4_IntProp prop1 ("prop1"); > c4_IntProp prop2 ("prop2"); > c4_View view = storage.GetAs("test[prop1:I]"); > view.AddProperty(prop2); > > view.Add(prop1[4321]+prop2[2345]); > storage.Commit(true); > return 0; > > } > > mkdump returns: > > test1.dat: 1 properties > test[prop1:I] > > VIEW 1 rows = test:V > 0: subview 'test' > VIEW 1 rows = prop1:I > 0: 4321 > > > > > -----Original Message----- > > From: metakit-admin@equi4.com > > [mailto:metakit-admin@equi4.com] On Behalf Of mARK bLOORE > > Sent: Tuesday, August 06, 2002 3:32 PM > > To: metakit > > Subject: RE: FW: [Metakit] why was c4_Storage::Store() dropped? > > > > > > > > --- Anthony Corriveau wrote: > > > Any hint as to what I'm doing wrong then? I tried 2.4.7 and am > now > > > using the CVS kit. > > > > > > int main() > > > { > > > c4_Storage storage ("test1.dat", true); > > > c4_IntProp prop1 ("prop1"); > > > c4_IntProp prop2 ("prop1"); > > > c4_View view = storage.GetAs("test[prop1:I]"); > > > view.AddProperty(prop2); > > > > > > view.Add(prop1[4321]+prop2[2345]); > > > storage.Commit(); > > > > > > return 0; > > > } > > > > > > two hints: > > what does metakit do if two properties in one view have the > > same name? what does metakit do with the string passed to GetAs()? __________________________________________________ Do You Yahoo!? Yahoo! Health - Feel better, live better http://health.yahoo.com From mbloore@yahoo.com Tue Aug 6 19:59:04 2002 Received: from web14307.mail.yahoo.com (web14307.mail.yahoo.com [216.136.173.83]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g770x3a21570 for ; Tue, 6 Aug 2002 19:59:03 -0500 Message-ID: <20020807005903.61641.qmail@web14307.mail.yahoo.com> Received: from [64.229.24.159] by web14307.mail.yahoo.com via HTTP; Tue, 06 Aug 2002 17:59:03 PDT Date: Tue, 6 Aug 2002 17:59:03 -0700 (PDT) From: mARK bLOORE Subject: RE: FW: [Metakit] why was c4_Storage::Store() dropped? To: metakit In-Reply-To: <000201c23d99$17cd8230$0201000a@alex> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Anthony Corriveau wrote: > I can't get it to store the property or the data unless the structure > is > defined in GetAs(). Looking at the database file in an editor, I can > see > that only the name of the first prop is there. > > Do you have some working sample code you code send me? i modified this from some code i used to try out blobs. in editing it for readability i found that it didn't work as i had described earlier; i must have made a mistake interpreting my own code! it seems that to make added propertied persist you must call GetAs with the new description before comitting the change. actually, i'm a little surprised that that second call to GetAs doesn't wipe out the value that was given to the new propery. if necessary, you can generate the new descpripion string automatically. run the following three times, with different values for 'stage': 1 to create the original data file. 2 to add the new property. 3 to show that the new property was saved. my output is: Hello World, this is MetaKit! 7865 0: 1 0: 2 Goodbye World Hello World, this is MetaKit! 7889 0: 1 12: 2 Goodbye World Hello World, this is MetaKit! 7913 0: 1 12: 2 Goodbye World #include #include #include c4_View vInfo; c4_Row row; c4_BytesProp pBlob ("blob"); c4_LongProp pSize ("size"); #define stage 3 static void show (void) { for (int i = 0; i < vInfo.GetSize(); i++) { c4_RowRef row = vInfo[i]; c4_Bytes b = pBlob.Get (row); cout << pSize (row) << ": " << *reinterpret_cast(b.Contents()) << endl; } } int main () { cout << endl << endl << "Hello World, this is MetaKit! " << getpid() << endl; c4_Storage sStorage ("/home/mark/tests/metakit/myfile.mk4", 1); #if stage <= 2 vInfo = sStorage.GetAs ("info[blob:B]"); #else vInfo = sStorage.GetAs ("info[blob:B,size:L]"); #endif #if stage == 1 int one = 1; pBlob (row) = c4_Bytes (&one, sizeof one); vInfo.Add (row); int two = 2; pBlob (row) = c4_Bytes (&two, sizeof two); vInfo.Add (row); #endif #if stage == 2 vInfo.AddProperty (pSize); pSize (vInfo[1]) = 12; vInfo = sStorage.GetAs ("info[blob:B,size:L]"); #endif sStorage.Commit (true); show(); cout << "Goodbye World" << endl; return 0; } __________________________________________________ Do You Yahoo!? Yahoo! Health - Feel better, live better http://health.yahoo.com From dgporter@erols.com Fri Aug 9 14:54:29 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g79JsTa13680 for ; Fri, 9 Aug 2002 14:54:29 -0500 X-Info: This message was accepted for relay by smtp01.mrf.mail.rcn.net as the sender used SMTP authentication X-Trace: UmFuZG9tSVbUSbQQaeF7TT+hb4RMd3s+QHLnkcUTjJtlObv10jS+WMt/pZfZ68SDu7nosVB3qpk= Received: from hamming.cam.nist.gov ([129.6.88.137] helo=erols.com) by smtp01.mrf.mail.rcn.net with asmtp (Exim 3.35 #6) id 17dFq8-0000ef-00; Fri, 09 Aug 2002 15:54:28 -0400 Message-ID: <3D541DF4.40103@erols.com> Date: Fri, 09 Aug 2002 15:54:28 -0400 From: Don Porter Reply-To: dgporter@erols.com User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.0) Gecko/20020609 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Utter newbie confusion Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Don't know why I can never get any joy with Mk4tcl... Can someone explain this simple failure? % package require Mk4tcl 2.4.7 % mk::file open foo foo.mk foo % mk::row append foo.bar name dgp foo.bar!0 % mk::select foo.bar -exact name dgp 0 % mk::file commit foo foo % mk::select foo.bar -exact name dgp % So [mk::select] works before [mk::file commit], but fails after?! What's up with that? So many others use and love Metakit, and I get stopped by the simplest operations. What's wrong with my mental model? How do I get the right one? -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@nist.gov Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From dgporter@erols.com Fri Aug 9 22:18:44 2002 Received: from smtp02.mrf.mail.rcn.net (smtp02.mrf.mail.rcn.net [207.172.4.61]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7A3Iia29508 for ; Fri, 9 Aug 2002 22:18:44 -0500 Received: from 66-44-10-180.s180.apx2.lnh.md.dialup.rcn.com ([66.44.10.180] helo=there) by smtp02.mrf.mail.rcn.net with smtp (Exim 3.35 #6) id 17dMm3-0004pA-00 for metakit@equi4.com; Fri, 09 Aug 2002 23:18:43 -0400 Content-Type: text/plain; charset="iso-8859-1" From: Don Porter Reply-To: dgporter@erols.com To: metakit@equi4.com Date: Fri, 9 Aug 2002 23:13:59 -0400 X-Mailer: KMail [version 1.3.2] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: Subject: [Metakit] sad saga continues... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Roseman replied to my earlier question, pointing out that I needed to have a property created in a [mk::view layout] command before Metakit would commit it to disk. I find that confusing, but at least it's an answer. I'd be much happier with a system that either forced me to be explicit about everything (so I could depend on getting error messages when I try to do something unexpected -- like misspell a property name), or one that automatically took care of all things for me (so I'd get a complete ease-of-use experience). A mixed approach delivers neither virtue. Next issue: Apparently a [mk::view delete] does not take effect on an in-memory dataset until a [mk::file commit] on the corresponding datafile? See the following session: % package require Mk4tcl 2.4.7 % mk::file open test test.mk test % mk::view layout test.test name test.test % mk::row append test.test name dgp test.test!0 % mk::view size test.test 1 % mk::view delete test.test test.test % mk::view size test.test 1 % mk::row append test.test name dgp test.test!1 % mk::view size test.test 2 % mk::view delete test.test test.test % mk::view size test.test 2 % mk::file commit test test % mk::view size test.test 0 Does that make sense? Not to me. So how does this work? Do you just get used to the quirks after some period of usage, and from then on intuitively avoid the commands that don't work sensibly? -- | Don Porter dgporter@erols.com | | "Some days you just can't get rid of a bomb!" | | -- Adam West as BATMAN | |______________________________________________________________________| From aag@stanford.edu Mon Aug 12 03:07:59 2002 Received: from smtp1.Stanford.EDU (smtp1.Stanford.EDU [171.64.14.23]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7C87wa32469 for ; Mon, 12 Aug 2002 03:07:58 -0500 Received: from smtp1.Stanford.EDU (localhost [127.0.0.1]) by smtp1.Stanford.EDU (8.11.6/8.11.6) with ESMTP id g7C87vs17190 for ; Mon, 12 Aug 2002 01:07:57 -0700 (PDT) Received: from infobox (zoostation.Stanford.EDU [128.12.146.107]) by smtp1.Stanford.EDU (8.11.6/8.11.6) with SMTP id g7C87uE17184 for ; Mon, 12 Aug 2002 01:07:57 -0700 (PDT) From: "Adrian Graham" To: Date: Mon, 12 Aug 2002 01:07:52 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] MetaKit and C++ Builder Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'd like to use MetaKit in a Borland C++ Builder 6.0 project. After some failed attempts at getting BCB6 to work with the prebuilt dll (through IMPLIB), I've decided it's probably best for me to build a C++ Builder static .lib from the source. I'm curious if anyone else has already done this and might have some tips (like what compiler flags need to be set) or a project file to share with me. (I tried a quick compile earlier and ran into several errors and quite a few warnings. I can work through these, but I thought I'd ask if anyone's already done this first.) Much thanks, and I'm looking forward to learning more about MetaKit! Adrian From dgporter@erols.com Wed Aug 14 15:36:07 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7EKa7a32188 for ; Wed, 14 Aug 2002 15:36:07 -0500 X-Info: This message was accepted for relay by smtp01.mrf.mail.rcn.net as the sender used SMTP authentication X-Trace: UmFuZG9tSVb5B5CE0yjfkZQ6eKHOKDhTSMPGHn5bX1PiZ3x9twKmCSSbFLo2sw1AOb7/xSvKd0w= Received: from hamming.cam.nist.gov ([129.6.88.137] helo=erols.com) by smtp01.mrf.mail.rcn.net with asmtp (Exim 3.35 #6) id 17f4sA-00032S-00; Wed, 14 Aug 2002 16:36:06 -0400 Message-ID: <3D5ABF2F.50005@erols.com> Date: Wed, 14 Aug 2002 16:35:59 -0400 From: Donald G Porter Reply-To: dgporter@erols.com User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.1b) Gecko/20020812 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Mk4tcl segfault on Linux/Alpha Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This may not be surprising; I don't think Mk4tcl (Metakit as well?) supports 64-bit platforms, but for the record: % package require Mk4tcl 2.4.7 % mk::file open foo foo % mk::view layout foo.test name Segmentation fault Should Mk4tcl check at [package ifneeded] time, and refuse to register itself for loading on 64-bit platforms? -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@nist.gov Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From dgporter@erols.com Wed Aug 14 15:42:37 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7EKgaa32639 for ; Wed, 14 Aug 2002 15:42:36 -0500 X-Info: This message was accepted for relay by smtp01.mrf.mail.rcn.net as the sender used SMTP authentication X-Trace: UmFuZG9tSVZj3eXVifeGhkcfpNBMwCQCrT8BFmTNKQPwgsT8HdmdU7VrFV3Qy7aW5xN53mP/rRE= Received: from hamming.cam.nist.gov ([129.6.88.137] helo=erols.com) by smtp01.mrf.mail.rcn.net with asmtp (Exim 3.35 #6) id 17f4yS-0004Q5-00; Wed, 14 Aug 2002 16:42:36 -0400 Message-ID: <3D5AC0BC.7050001@erols.com> Date: Wed, 14 Aug 2002 16:42:36 -0400 From: Donald G Porter Reply-To: dgporter@erols.com User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.1b) Gecko/20020812 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] more brokenness in Mk4tcl Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A Tcl list can have many different string representations, without changing what the list is. Tcl commands that accept list arguments should not be sensitive to the particular string rep. If they are, it's a sign that they are using string-processing operations where list-processing operations would be correct. With that in mind, consider the following fault in [mk::view layout]: % package require Mk4tcl 2.4.7 % mk::file open foo foo % mk::view layout foo.test1 {name {email {address}}} foo.test1 % mk::view layout foo.test2 {name {email address}} ;# same list foo.test2 % mk::view layout foo.test1 ;# this is good name {email {address}} % mk::view layout foo.test2 ;# bad; list nesting has been removed name email address If you pause to imagine a programmer constructing their layout with [list ...] commands where they do not even have precise control over the string rep., it's clear this is very bad. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@nist.gov Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From dgporter@erols.com Thu Aug 15 09:52:28 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7FEqSa19365 for ; Thu, 15 Aug 2002 09:52:28 -0500 X-Info: This message was accepted for relay by smtp01.mrf.mail.rcn.net as the sender used SMTP authentication X-Trace: UmFuZG9tSVYDdGHCADKTzkKBRmhdRvI8c8lAFC2+iS1u8tu5zhKqnV/UiMfQbzPUEMXD7/IcCQ8= Received: from hamming.cam.nist.gov ([129.6.88.137] helo=erols.com) by smtp01.mrf.mail.rcn.net with asmtp (Exim 3.35 #6) id 17fLz9-0000gB-00; Thu, 15 Aug 2002 10:52:27 -0400 Message-ID: <3D5BC02A.1060008@erols.com> Date: Thu, 15 Aug 2002 10:52:26 -0400 From: Donald G Porter Reply-To: dgporter@erols.com User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.0) Gecko/20020609 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Mk4tcl & multiple writers Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sample session and running commentary: % package require Mk4tcl 2.4.7 % mk::file open test test.mk test % mk::file open test test.mk Any introspection that returns a file name ought to return the full absolute path; otherwise, the information is useless if there's been a [cd] any time between the open and the query. % mk::file open test test.mk file already open Seems good. Multiple writers opening the same file is prevented. % mk::file open foo test.mk foo D'oh! Take that back. I expected an error from Mk4tcl here. % mk::view layout test.test {test} test.test % mk::row append test.test test foo test.test!0 Added some data to "test"... % mk::file views foo % ... that is not visible in "foo" -- apparently separate copies; good. % mk::file commit test test % mk::file rollback foo foo % mk::file views foo test Confirm the two tags reference the same file, now share the same data. But since Mk4tcl did not prevent multiple writers, we can step on each other's updates. % mk::row append test.test test bar test.test!1 % mk::file commit test test % mk::row append foo.test test snafu foo.test!1 % mk::file commit foo foo % mk::file rollback test test % mk::loop X test.test {puts [mk::get $X]} test foo test snafu So, the lesson seems to be that the burden is completely on the Mk4tcl user to prevent multiple writers; the Mk4tcl interface itself offers no checking and no help. It's impossible for the Mk4tcl user to do this checking if any [cd] has been done, or if multiple interps are involved. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@nist.gov Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From fret@memecode.com Wed Aug 21 06:25:25 2002 Received: from mail009.syd.optusnet.com.au (mail009.syd.optusnet.com.au [210.49.20.137]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7LBPOa03401 for ; Wed, 21 Aug 2002 06:25:24 -0500 Received: from default (c19152.carlnfd1.nsw.optusnet.com.au [210.49.145.241]) by mail009.syd.optusnet.com.au (8.11.1/8.11.1) with SMTP id g7LBPMM02110 for ; Wed, 21 Aug 2002 21:25:22 +1000 Message-Id: <200208211125.g7LBPMM02110@mail009.syd.optusnet.com.au> Date: Wed, 21 Aug 2002 09:23:01 +1000 X-Mailer: InScribe v1.78 (Win32 Debug) To: "MetaKit " From: "Matthew Allen" MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit Subject: [Metakit] Newbie questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Newbie question? How do I delete a property from a view? Which is probably the same as how do I delete a view from a storage? regards -- Matthew Allen (fret@memecode.com) (http://www.memecode.com) "Microsoft Windows - would you use a car that broke down every day?" From dgporter@erols.com Fri Aug 23 17:32:06 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7NMW6a16757 for ; Fri, 23 Aug 2002 17:32:06 -0500 X-Info: This message was accepted for relay by smtp01.mrf.mail.rcn.net as the sender used SMTP authentication X-Trace: UmFuZG9tSVbd4Auwgi6VRQAX3zx69Vb9klGiNVcmEhMbTVA7SM463X+G3rny2TJYFjZrXawW4Uo= Received: from hamming.cam.nist.gov ([129.6.88.137] helo=erols.com) by smtp01.mrf.mail.rcn.net with asmtp (Exim 3.35 #6) id 17iMyL-0001OZ-00; Fri, 23 Aug 2002 18:32:05 -0400 Message-ID: <3D66B7DB.7060105@erols.com> Date: Fri, 23 Aug 2002 18:31:55 -0400 From: Donald G Porter Reply-To: dgporter@erols.com User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.0) Gecko/20020609 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Mk4tcl documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've found the Mk4tcl documentation quite incomplete. To start a remedy, I've started putting together documentation on the Metakit Wiki. See http://www.equi4.com/metakit/wiki.cgi/147 I'm filling it in slowly, based on observation, "trial and error message", and source code diving. I think it will go much faster if some others with Mk4tcl experience will jump in and contribute, Wiki-style. Thanks. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@nist.gov Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From tom.krehbiel@motorola.com Mon Aug 26 10:17:34 2002 Received: from motgate4.mot.com (motgate4.mot.com [144.189.100.102]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7QFHXa23568 for ; Mon, 26 Aug 2002 10:17:33 -0500 Received: [from pobox4.mot.com (pobox4.mot.com [10.64.251.243]) by motgate4.mot.com (motgate4 2.1) with ESMTP id IAA08379 for ; Mon, 26 Aug 2002 08:17:33 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by pobox4.mot.com (MOT-pobox4 2.0) with ESMTP id IAA16584 for ; Mon, 26 Aug 2002 08:17:32 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id PSDC96JY; Mon, 26 Aug 2002 08:17:31 -0700 Message-ID: <3D6A468B.89624A23@motorola.com> Date: Mon, 26 Aug 2002 08:17:31 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 CC: metakit@equi4.com Subject: Re: [Metakit] Mk4tcl documentation References: <3D66B7DB.7060105@erols.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Donald, Thanks for working on the documentation. I noticed the comment at the bottom about use of '$view'. Can the problem be resolved by writing the lines as * $view [close] * $view [space] ..etc Are you also planning on documenting the view operations? Regards, Tom K. > I've found the Mk4tcl documentation quite incomplete. To start a > remedy, I've started putting together documentation on the Metakit Wiki. > See http://www.equi4.com/metakit/wiki.cgi/147 > > I'm filling it in slowly, based on observation, "trial and error > message", and source code diving. I think it will go much faster if > some others with Mk4tcl experience will jump in and contribute, Wiki-style. From dgporter@erols.com Mon Aug 26 10:49:23 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7QFnNa25665 for ; Mon, 26 Aug 2002 10:49:23 -0500 X-Info: This message was accepted for relay by smtp01.mrf.mail.rcn.net as the sender used SMTP authentication X-Trace: UmFuZG9tSVbeyg+ySgYDerLV6gyTdIKsXxrP2f/eW0hwZe0KlRHnM0QH8Y0j1OcxXDo3vr5iorA= Received: from hamming.cam.nist.gov ([129.6.88.137] helo=erols.com) by smtp01.mrf.mail.rcn.net with asmtp (Exim 3.35 #6) id 17jM7G-0004rt-00; Mon, 26 Aug 2002 11:49:22 -0400 Message-ID: <3D6A4DFD.6060507@erols.com> Date: Mon, 26 Aug 2002 11:49:17 -0400 From: Donald G Porter Reply-To: dgporter@erols.com User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.0) Gecko/20020609 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Tom Krehbiel CC: metakit@equi4.com Subject: Re: [Metakit] Mk4tcl documentation References: <3D66B7DB.7060105@erols.com> <3D6A468B.89624A23@motorola.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tom Krehbiel wrote: > Thanks for working on the documentation. I noticed the comment at the bottom > about use of '$view'. Can the problem be resolved by writing the lines as > > * $view [close] > * $view [space] The problem is the Wiki does not allow page titles to begin with a $ or a <, and also does not allow any formatting. All-caps was the only substitute I could find to highlight the first word of a page title. > Are you also planning on documenting the view operations? I've done all of them I can. The rest I do not understand. They seem to have something to do with the underlying structures used to store the view, but there's a limit to how deep I want to dive into C++ code just to get usable Tcl documentation. I hope the Wiki-style will prevail and others will fill in the gaps. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@nist.gov Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From tom.krehbiel@motorola.com Mon Aug 26 12:10:21 2002 Received: from ftpbox.mot.com (ftpbox.mot.com [129.188.136.101]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7QHALa31595 for ; Mon, 26 Aug 2002 12:10:21 -0500 Received: [from pobox4.mot.com (pobox4.mot.com [10.64.251.243]) by ftpbox.mot.com (ftpbox 2.1) with ESMTP id KAA20746 for ; Mon, 26 Aug 2002 10:10:20 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by pobox4.mot.com (MOT-pobox4 2.0) with ESMTP id KAA05740 for ; Mon, 26 Aug 2002 10:10:20 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id PSDC966N; Mon, 26 Aug 2002 10:10:19 -0700 Message-ID: <3D6A60FB.2794896D@motorola.com> Date: Mon, 26 Aug 2002 10:10:19 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 CC: metakit@equi4.com Subject: Re: [Metakit] Mk4tcl documentation References: <3D66B7DB.7060105@erols.com> <3D6A468B.89624A23@motorola.com> <3D6A4DFD.6060507@erols.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Donald, As a starting point for documentation of the other commands you can look at the python MetaKit documentation at http://www.equi4.com/metakit/python.html. It's the same set of commands so the behavior should be the same. Regards, Tom K. > Tom Krehbiel wrote: > > Thanks for working on the documentation. I noticed the comment at the bottom > > about use of '$view'. Can the problem be resolved by writing the lines as > > > > * $view [close] > > * $view [space] > > The problem is the Wiki does not allow page titles to > begin with a $ or a <, and also does not allow any formatting. > All-caps was the only substitute I could find to highlight > the first word of a page title. > > > Are you also planning on documenting the view operations? > > I've done all of them I can. The rest I do not understand. They > seem to have something to do with the underlying structures used > to store the view, but there's a limit to how deep I want to dive > into C++ code just to get usable Tcl documentation. > > I hope the Wiki-style will prevail and others will fill in the gaps. From mann@earthling.net Tue Aug 27 13:46:03 2002 Received: from che.kiza.com (IDENT:root@[198.144.196.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7RIk3a14662 for ; Tue, 27 Aug 2002 13:46:03 -0500 Received: from tonylaptop (puntarenas-a50.racsa.co.cr [196.40.52.179]) by che.kiza.com (8.11.2/8.11.2) with SMTP id g7RJENt01939 for ; Tue, 27 Aug 2002 12:14:25 -0700 Message-ID: <008101c24df9$f8211bb0$a2e0fea9@tonylaptop> From: "Tony Mann" To: "Metakit List" Date: Tue, 27 Aug 2002 12:45:45 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Removing property causes assert error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We are having a problem removing properties from our db. We have a string set up for use with GetAs that looks like this #define MYVIEW "MyView[Prop1:I,Prop2:I]" This works fine. Now, we want to remove Prop2, and add a new Prop3 and Prop 4. #define MYVIEW "MyView[Prop1:I,Prop3:I,Prop4:I]" When we try this, and I call c4_View::Search to locate a record using Prop1, we get an assert error in c4_SortSeq::Compare, on line 696 of derived.cpp: int c4_SortSeq::Compare(int index_, c4_Cursor cursor_) const { d4_assert(cursor_._seq != 0); const char* down = (const char*) _down.Contents(); d4_assert(_down.Size() <= NumHandlers()); ///// THIS ASSERT FAILS ... } The next time we open the db, the assert error does not occur. We do a Commit before we call the Search method, so my guess is that the disk version of the db does not have the extraneous property info floating around like the in-memory one, and therefore the problem is gone the next time the db is loaded from disk. We have also had some db corruption, and while we could not pinpoint the source, it always occurred when properties were added or removed. So we consider this a very serious problem. Has anyone else run into this? What is the approved way of removing properties? If we have to leave old properties in, we will, but my preference is to keep the db clean by removing them. The C++ docs clearly state that adding and removing properties and easy to do in Metakit, so I have been surprised by this behavior. I thought that I had seen some posts about this, but I could not find them, so if this has been asked and answered before, forgive me. ..tony.. From mann@earthling.net Tue Aug 27 14:14:37 2002 Received: from che.kiza.com (IDENT:root@[198.144.196.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7RJEaa16697 for ; Tue, 27 Aug 2002 14:14:36 -0500 Received: from tonylaptop (puntarenas-a50.racsa.co.cr [196.40.52.179]) by che.kiza.com (8.11.2/8.11.2) with SMTP id g7RJgvt02052; Tue, 27 Aug 2002 12:42:58 -0700 Message-ID: <009701c24dfd$f588e7d0$a2e0fea9@tonylaptop> From: "Tony Mann" To: "Metakit List" Cc: References: <008101c24df9$f8211bb0$a2e0fea9@tonylaptop> <3D6BCC29.4050505@cfl.rr.com> Subject: Re: [Metakit] Removing property causes assert error Date: Tue, 27 Aug 2002 13:14:19 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Adding the Commit worked!! Thanks so much. ..tony.. ----- Original Message ----- From: "Marcus Martin" To: "Tony Mann" Sent: Tuesday, August 27, 2002 12:59 PM Subject: Re: [Metakit] Removing property causes assert error > > > Tony Mann wrote: > > We are having a problem removing properties from our db. We have a string > > set up for use with GetAs that looks like this > > > > #define MYVIEW "MyView[Prop1:I,Prop2:I]" > > > > This works fine. > > > > Now, we want to remove Prop2, and add a new Prop3 and Prop 4. > > > > #define MYVIEW "MyView[Prop1:I,Prop3:I,Prop4:I]" > > > > When we try this, and I call c4_View::Search to locate a record using Prop1, > > we get an assert error in c4_SortSeq::Compare, on line 696 of derived.cpp > > Are you doing a commit() immediately after the GetAs() call? I believe > you need to do this for adding/removing a property to work correctly. > > Marcus > From walter.van.holst@xs4all.nl Tue Aug 27 15:17:38 2002 Received: from smtpzilla1.xs4all.nl (smtpzilla1.xs4all.nl [194.109.127.137]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7RKHba20710 for ; Tue, 27 Aug 2002 15:17:37 -0500 Received: from [192.168.1.2] (fossiel.xs4all.nl [213.84.32.42]) by smtpzilla1.xs4all.nl (8.12.0/8.12.0) with ESMTP id g7RKHYfl013451 for ; Tue, 27 Aug 2002 22:17:35 +0200 (CEST) From: "Walter H. van Holst" To: metakit@equi4.com Content-Type: multipart/mixed; boundary="=-CYDSfOevdCEHvsME8/R5" Organization: COMECON Message-Id: <1030479359.23092.25.camel@Walter> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.1.0.99 (Preview Release) Date: 27 Aug 2002 22:16:00 +0200 Subject: [Metakit] Proper spec file for Metakit 2.4.7-37 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=-CYDSfOevdCEHvsME8/R5 Content-Type: text/plain Content-Transfer-Encoding: 7bit Hello, The .spec file included in the tarball release of Metakit 2.4.7 is severely outdated. After some tinkering I have changed it to the .spec attached to this message. It works on my RH box and it results in the following (S)RPMs: metakit-2.4.7-37.srpm metakit-2.4.7-37.i386.rpm metakit-devel-2.4.7-37.i386.rpm Python and TCL bindings won't build. They caused me more troubles than I was willing to fix since I only needed Metakit to build the SimGear library. Anyone willing to look into these is welcome. Building the packages is fairly straightforward: Put the .spec file in /usr/src/redhat/SPECS Put the tarball in /usr/src/redhat/SOURCES Execute rpm -ba /usr/src/redhat/SPECS/metakit.spec and off you go. I am going to unsubscribe from this mailing list after this message, so please cc: any replies to my own address. Feedback is welcome, of course. Regards, Walter -- You have a morbid aversion to dying. You probably resent the fact that you're at war and might get your head blown off any second. - Col. Sanderson in Catch-22 --=-CYDSfOevdCEHvsME8/R5 Content-Disposition: attachment; filename=metakit.spec Content-Type: text/plain; name=metakit.spec; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit %define name metakit %define ver 2.4.7 %define extension tar.gz %define prefix /usr Summary: The MetaKit Library 2.0 Name: %{name} Version: %{ver} Release: 37 Copyright: X/MIT style Group: Applications/Databases Source: %{name}-%{ver}-%{release}.%{extension} #Patch: metakit-install.patch URL: http://www.equi4.com/metakit/ Buildroot: /tmp/%{name}-%{ver}-root Packager: Walter H. van Holst # Previous packager was Sean Summers %description MetaKit is an embeddable database which runs on Unix, Windows, Macintosh, and other platforms. It lets you build applications which store their data efficiently, in a portable way, and which will not need a complex runtime installation. In terms of the data model, MetaKit takes the middle ground between RDBMS, OODBMS, and flat-file databases - yet it is quite different from each of them. WHAT IT ISN'T - MetaKit is not: 1) an SQL database, 2) multi-user/-threading, 3) scalable to gigabytes, 4) proprietary software, 5) a toy. %package devel Summary: Development Libraries for The MetaKit Library 2.0 Group: Development/Libraries %description devel The %{name}-devel package contains the libraries and header files necessary for writing programs that make use of the MetaKit library. #%package python #Group: Development/Libraries #Summary: Python module for The MetaKit Library 2.0 #%description python #The %{name}-python package contains the libraries necessary #for using the MetaKit as a python module. #%package tcl #Group: Development/Libraries #Summary: TCL module for The MetaKit Library 2.0 #%description tcl #The %{name}-tcl package contains the libraries necessary #for using the MetaKit as a tcl module. %prep %setup #%patch -p1 %build cd builds ../unix/configure --prefix=${prefix} #--enable-tcl ## maybe TCL_DECLARE_MUTEX is too new for RH6.1.92? make ${RPM_BUILD_OPTS} # Testing takes a while #rm tests/\!keepme.txt make ${RPM_BUILD_OPTS} test %install cd builds make prefix=${RPM_BUILD_ROOT}/usr install DESTDIR=${RPM_BUILD_ROOT} #mske install TOPDIR=${RPM_BUILD_ROOT} libtool --finish ${RPM_BUILD_ROOT}${prefix}/lib/ ##python setup #make python #install -Ds libmk4py.la ${RPM_BUILD_ROOT}/usr/lib/python2.2/site-packages/libmk4py.la #%install tcl #make Mk4tcl.so %clean rm -rf $RPM_BUILD_ROOT %files %doc CHANGES README MetaKit.html %doc doc/e4s.gif doc/format.html /usr/lib/libmk4.so #/usr/lib/libmk4.so.0 #/usr/lib/libmk4.so.0.0.0 /usr/lib/libmk4.la %files devel %doc doc/api/ demos/ /usr/lib/libmk4.a /usr/include/mk4.h /usr/include/mk4.inl /usr//include/mk4str.h /usr/include/mk4str.inl #%files python #%doc python/*.py #%doc doc/python.* #/usr/lib/python2.2/site-packages/Mk4pymodule.so #%files tcl #%doc tcl/*.tcl #%doc doc/tcl.* #/usr/lib/tcl --=-CYDSfOevdCEHvsME8/R5-- From scusack@helpisit.com.au Tue Aug 27 17:48:28 2002 Received: from ams1.justnet.info (ams1.justnet.info [213.206.75.141]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7RMmRa28331 for ; Tue, 27 Aug 2002 17:48:28 -0500 Received: by ams1.justnet.info (Postfix, from userid 2266) id BCF9D1AA58; Tue, 27 Aug 2002 22:48:24 +0000 (GMT) Received: from ams1.justnet.info (localhost [127.0.0.1]) by ams1.justnet.info (Postfix) with SMTP id BFFFE1A762 for ; Tue, 27 Aug 2002 22:48:22 +0000 (GMT) Received: from 203.41.160.165 (SquirrelMail authenticated user u12655e) by webmail.justnet.info with HTTP; Tue, 27 Aug 2002 22:48:22 -0000 (GMT) Message-ID: <3554.203.41.160.165.1030488502.squirrel@webmail.justnet.info> Date: Tue, 27 Aug 2002 22:48:22 -0000 (GMT) From: "Simon Cusack" To: X-Priority: 3 Importance: Normal X-MSMail-Priority: Normal Reply-To: scusack@helpisit.com.au X-Mailer: JustNet webmail v1.2.6 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Status: No, hits=0.0 required=12.0 tests= version=2.20 X-Spam-Level: Subject: [Metakit] metakit across a network Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I have been experiencing some corruption of a metakit file during the following scenario; 1. The client is running on WinNT 4.0 2. The Metakit file is on a Novell or NT Server Everything appears to go ok for a while and then I stop getting results from views. When I look at the file using the Metakit viewer, it opens ok, thinking that it is a valid file, and the views definitions appear to be ok, but the contents of the views is jumbled up. Rows have run together, etc. This has happened using Python and the c++ api. Are there any special features or flags that I should be aware of when using metakit across a network like this? I haven't managed to get it to happen consistently so I haven't included an example, I just thought I would check that there wasn't something obvious that I missing first. If not I'll get an example together. Regards, Simon. From mmartin6@cfl.rr.com Tue Aug 27 22:19:39 2002 Received: from smtp-server3.tampabay.rr.com (smtp-server3.tampabay.rr.com [65.32.1.41]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7S3Jca05802 for ; Tue, 27 Aug 2002 22:19:39 -0500 Received: from [192.168.0.3] (51.12.33.65.cfl.rr.com [65.33.12.51]) by smtp-server3.tampabay.rr.com (8.12.2/8.12.2) with ESMTP id g7S3JacB029970; Tue, 27 Aug 2002 23:19:37 -0400 (EDT) Subject: Re: [Metakit] metakit across a network From: Marcus A Martin To: scusack@helpisit.com.au Cc: metakit@equi4.com In-Reply-To: <3554.203.41.160.165.1030488502.squirrel@webmail.justnet.info> References: <3554.203.41.160.165.1030488502.squirrel@webmail.justnet.info> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.8 Date: 27 Aug 2002 23:17:26 -0500 Message-Id: <1030508247.5169.8.camel@morwin.cfl.rr.com> Mime-Version: 1.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Tue, 2002-08-27 at 17:48, Simon Cusack wrote: > Hi, > > I have been experiencing some corruption of a metakit file during the > following scenario; > 1. The client is running on WinNT 4.0 > 2. The Metakit file is on a Novell or NT Server > > Everything appears to go ok for a while and then I stop getting results > from views. > When I look at the file using the Metakit viewer, it opens ok, thinking > that it is a valid file, and the views definitions appear to be ok, but > the contents of the views is jumbled up. Rows have run together, etc. > This has happened using Python and the c++ api. Are there any special > features or flags that I should be aware of when using metakit across a > network like this? > I haven't managed to get it to happen consistently so I haven't included > an example, I just thought I would check that there wasn't something > obvious that I missing first. If not I'll get an example together. > Regards, Simon. This is a very big WAG, but does NOVELL use a different end of line parameter than windows? Metakit opens binary files, which are different on unix and windows. If you open a binary file in windows mode it can helpfully alter carrige-return and line-feed into just line-feed. I have had this problem with windows/unix compatability before. Marcus From mann@earthling.net Wed Aug 28 02:02:56 2002 Received: from che.kiza.com (IDENT:root@[198.144.196.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7S72ua14986 for ; Wed, 28 Aug 2002 02:02:56 -0500 Received: from tonylaptop (puntarenas-a43.racsa.co.cr [196.40.52.172]) by che.kiza.com (8.11.2/8.11.2) with SMTP id g7S7VMt04698 for ; Wed, 28 Aug 2002 00:31:23 -0700 Message-ID: <01c901c24e60$e65ce5f0$a2e0fea9@tonylaptop> From: "Tony Mann" To: "Metakit List" Date: Wed, 28 Aug 2002 01:02:30 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Major thrashing problems when adding a property Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We have a view with several small props and one binary property. Each binary property has around 3K of data, and with over 1000 records, this is many megabytes. All was working fine until I added a new property to the view (and yes I now do a Commit right after the GetAs call). Suddenly opening the database takes over a minute when it used to take a second or two, with massive disk thrashing. If we start with a blank db, there are no thrashing problems at all. It only happens when we add the property to a db with data already in it. I assume the problem is the large amount of binary data in this view. Does anyone know if moving this data into another view will help? Or do I need to move it into another db file? Another option is to write all the data from the old db into a new db, with the new prop added. Does this seem like a sound approach? And does anyone know if there is a way to detect when a db is very fragmented? Frankly, I am disappointed by all the problems I am having adding a property, since the column-wise storage of Metakit was supposed to make this a quick and painless operation. What the heck is going on? ..tony.. From fret@memecode.com Wed Aug 28 04:08:55 2002 Received: from mail005.syd.optusnet.com.au (mail005.syd.optusnet.com.au [210.49.20.136]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7S98sa20758 for ; Wed, 28 Aug 2002 04:08:54 -0500 Received: from default (c19152.carlnfd1.nsw.optusnet.com.au [210.49.145.241]) by mail005.syd.optusnet.com.au (8.11.1/8.11.1) with SMTP id g7S98pa20155 for ; Wed, 28 Aug 2002 19:08:52 +1000 Message-Id: <200208280908.g7S98pa20155@mail005.syd.optusnet.com.au> Date: Wed, 28 Aug 2002 19:08:41 +1000 X-Mailer: InScribe v1.78 (Win32 Debug) To: "MetaKit " From: "Matthew Allen" Reply-To: "Matthew Allen" Subject: Re: [Metakit] Newbie questions MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g7S98sa20758 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > On Tue, 2002-08-20 at 18:23, Matthew Allen wrote: > > Newbie question? > > > > How do I delete a property from a view? > > > > Which is probably the same as how do I delete a view from a storage? > > Did anyone ever answer you? Nope... I was going to try again tonight... but I guess this will do. It seems the way to change structure is the "GetAs" function of the storage. But this is not good for reasonably nested views. Because you have to parse the description string, which is non-trivial and then do another "GetAs". I'm not even sure whether that is going to work for nested stuff that well. What I'd like to see is a "DeleteProperty" function in the Views to match the "AddProperty". FYI I'm making a little generic DB interface... with a list of tables in a tree view with the fields/records displayed in a list view. So far I can add tables, add fields, add records and so I want to add the equivalent delete functions those objects. This is all as an introduction to see if I can use it to store email, contacts and calendar information in a Mail Client. regards -- Matthew Allen (fret@memecode.com) (http://www.memecode.com) "Microsoft Windows - would you use a car that broke down every day?" ------ From gmcm@hypernet.com Wed Aug 28 08:17:22 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SDHMa01452 for ; Wed, 28 Aug 2002 08:17:22 -0500 Received: from PARANOIA (204.176.40.94) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Wed, 28 Aug 2002 09:15:54 -0400 From: "Gordon McMillan" To: "Tony Mann" , "Metakit List" Date: Wed, 28 Aug 2002 09:16:33 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Major thrashing problems when adding a property Reply-to: gmcm@hypernet.com Message-ID: <3D6C94F1.16224.B4CB74FD@localhost> Priority: normal In-reply-to: <01c901c24e60$e65ce5f0$a2e0fea9@tonylaptop> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 28 Aug 2002 at 1:02, Tony Mann wrote: > We have a view with several small props and one binary > property. Each binary property has around 3K of data, > and with over 1000 records, this is many megabytes. > > All was working fine until I added a new property > to the view (and yes I now do a Commit right after > the GetAs call). Suddenly opening the database takes > over a minute when it used to take a second or two, > with massive disk thrashing. FWIW, I do lots of dynamic restructures (using Mk4py) and have never had this problem. Before getting drastic, I'd use SaveTo and see if that clears it up (opens should be almost instantaneous). -- Gordon http://www.mcmillan-inc.com/ From mann@earthling.net Wed Aug 28 10:47:18 2002 Received: from che.kiza.com (IDENT:root@[198.144.196.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SFlHa10655 for ; Wed, 28 Aug 2002 10:47:18 -0500 Received: from tonylaptop (puntarenas-a83.racsa.co.cr [196.40.52.212]) by che.kiza.com (8.11.2/8.11.2) with SMTP id g7SGFja01369; Wed, 28 Aug 2002 09:15:47 -0700 Message-ID: <022101c24eaa$25376770$a2e0fea9@tonylaptop> From: "Tony Mann" To: "Metakit List" Cc: References: <3D6C94F1.16224.B4CB74FD@localhost> Subject: Re: [Metakit] Major thrashing problems when adding a property Date: Wed, 28 Aug 2002 09:43:48 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Well, I tried this. My db file shrunk by several megabytes, which is nice, but the thrashing problem is still there! This surprises me, since as I said, a new db file does not have any of these problems. Somehow saving to a stream and then reloading from it preserved the problems that are causing the thrashing So, now can I get drastic? :-) ..tony.. ----- Original Message ----- From: "Gordon McMillan" To: "Tony Mann" ; "Metakit List" Sent: Wednesday, August 28, 2002 7:16 AM Subject: Re: [Metakit] Major thrashing problems when adding a property > On 28 Aug 2002 at 1:02, Tony Mann wrote: > > > We have a view with several small props and one binary > > property. Each binary property has around 3K of data, > > and with over 1000 records, this is many megabytes. > > > > All was working fine until I added a new property > > to the view (and yes I now do a Commit right after > > the GetAs call). Suddenly opening the database takes > > over a minute when it used to take a second or two, > > with massive disk thrashing. > > FWIW, I do lots of dynamic restructures (using Mk4py) > and have never had this problem. Before getting > drastic, I'd use SaveTo and see if that clears it up > (opens should be almost instantaneous). > > -- Gordon > http://www.mcmillan-inc.com/ > From jcw@equi4.com Wed Aug 28 11:18:16 2002 Received: from [10.0.1.193] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SGIFa12386 for ; Wed, 28 Aug 2002 11:18:15 -0500 From: "Jean-Claude Wippler" To: "Metakit List" Subject: Re: [Metakit] Major thrashing problems when adding a property Date: Wed, 28 Aug 2002 18:02:07 +0200 Message-Id: <20020828160207.16495@triqs.com> In-Reply-To: <022101c24eaa$25376770$a2e0fea9@tonylaptop> References: <022101c24eaa$25376770$a2e0fea9@tonylaptop> X-Mailer: CTM PowerMail 4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tony Mann wrote: >> > We have a view with several small props and one binary >> > property. Each binary property has around 3K of data, >> > and with over 1000 records, this is many megabytes. >> > >> > All was working fine until I added a new property >> > to the view (and yes I now do a Commit right after >> > the GetAs call). Suddenly opening the database takes >> > over a minute when it used to take a second or two, >> > with massive disk thrashing. You seem to be hitting a bad spot in MK's optimization strategy. Above a certain size, heuristics kick in to store items individually instead of as column. To turn this behavior off, you need to make a change to the MK sources: in src/format.cpp, c4_FormatB::ShouldBeMemo, comment out the body and replace it by "return false;". The downside is that commits will write out potentially huge columns. FWIW, this change in 2.3/2.4 is what made the "M" datatype (memo) obsolete - it still exists, but under the hood and adaptively. It looks like more sophisticated logic is needed to avoid this rather sudden degradation. I've had one report of a corrupted datafile when this switchover happens, but have not found a way to reproduce and figure out the exact problem. -jcw From jcw@equi4.com Wed Aug 28 11:18:18 2002 Received: from [10.0.1.193] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SGIHa12390 for ; Wed, 28 Aug 2002 11:18:18 -0500 From: "Jean-Claude Wippler" To: "Metakit List" Date: Wed, 28 Aug 2002 18:13:38 +0200 Message-Id: <20020828161338.26135@triqs.com> X-Mailer: CTM PowerMail 4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Properties vs. commit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: To respond to a couple of emails regarding adding/deleting properties and why things happen as they and when they do: MK views consist of one or more persistent properties, optionally followed by non-persistent properties. GetAs adjusts the properties as follows: - persistent props get re-ordered as specified - props no-longer listed get moved to end and become non-persistent - non-persistent props which are listed get moved to the front - new props get inserted with default values On commit, persistent props are saved, making all structural changes permanent, while non-persistent props are deleted from the in-memory storage/view objects. The above has several implications: - props do not get deleted by "getas", but by the subsequent commit - one can have temp data props, simply by setting props in a row - you can convert props by copying them over to a new one (diff name) - getas can be done even after props have been added to a view - all of the above holds for subview props too This behavior is not necessarily the most intuitive, but there are a number of reasons why things are as they are for now - some are historical, btw. -jcw From jcw@equi4.com Wed Aug 28 11:18:20 2002 Received: from [10.0.1.193] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SGIJa12394 for ; Wed, 28 Aug 2002 11:18:19 -0500 From: "Jean-Claude Wippler" To: "Metakit List" Subject: Re: [Metakit] Mk4tcl segfault on Linux/Alpha Date: Wed, 28 Aug 2002 18:16:09 +0200 Message-Id: <20020828161609.30646@triqs.com> In-Reply-To: <3D5ABF2F.50005@erols.com> References: <3D5ABF2F.50005@erols.com> X-Mailer: CTM PowerMail 4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Donald G Porter wrote: >This may not be surprising; I don't think Mk4tcl (Metakit as well?) >supports 64-bit platforms, but for the record: > >% package require Mk4tcl >2.4.7 >% mk::file open foo >foo >% mk::view layout foo.test name >Segmentation fault > >Should Mk4tcl check at [package ifneeded] time, and refuse to register >itself for loading on 64-bit platforms? No - you seem to be hitting a bug. MK (i.e. the C++ core) has supported 64-bit Alpha for years. Maybe this is an issue of the Mk4tcl layer, or maybe Tcl's 64-bit changes broke something. -jcw From jyl@best.com Wed Aug 28 12:03:38 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g7SH3ca15136 for ; Wed, 28 Aug 2002 12:03:38 -0500 Received: (qmail 52507 invoked by uid 19667); 28 Aug 2002 17:03:00 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 28 Aug 2002 17:03:00 -0000 Message-ID: <3D6D03F3.1030901@best.com> Date: Wed, 28 Aug 2002 10:10:11 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: Metakit List Subject: Re: [Metakit] Properties vs. commit References: <20020828161338.26135@triqs.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Just to clarify my understanding, it is the latest GetAs that determines which properties are mde persistent in the next commit, right? So by "non-persistent properties that are mentioned" what do you mean? Do you mean properties that have been added by a previous GetAs but not yet made persistent? --JYL Jean-Claude Wippler wrote: >To respond to a couple of emails regarding adding/deleting properties and >why things happen as they and when they do: > >MK views consist of one or more persistent properties, optionally >followed by non-persistent properties. > >GetAs adjusts the properties as follows: > - persistent props get re-ordered as specified > - props no-longer listed get moved to end and become non-persistent > - non-persistent props which are listed get moved to the front > - new props get inserted with default values > >On commit, persistent props are saved, making all structural changes >permanent, while non-persistent props are deleted from the in-memory >storage/view objects. > >The above has several implications: > - props do not get deleted by "getas", but by the subsequent commit > - one can have temp data props, simply by setting props in a row > - you can convert props by copying them over to a new one (diff name) > - getas can be done even after props have been added to a view > - all of the above holds for subview props too > >This behavior is not necessarily the most intuitive, but there are a >number of reasons why things are as they are for now - some are >historical, btw. > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From jcw@equi4.com Wed Aug 28 12:10:35 2002 Received: from [10.0.1.193] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SHAYa15563 for ; Wed, 28 Aug 2002 12:10:34 -0500 From: "Jean-Claude Wippler" To: "Metakit List" Subject: Re: [Metakit] Properties vs. commit Date: Wed, 28 Aug 2002 19:08:51 +0200 Message-Id: <20020828170851.26148@triqs.com> In-Reply-To: <3D6D03F3.1030901@best.com> References: <3D6D03F3.1030901@best.com> X-Mailer: CTM PowerMail 4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >Just to clarify my understanding, it is the latest GetAs that determines >which properties are mde persistent in the next commit, right? Right. >So by >"non-persistent properties that are mentioned" what do you mean? Do you >mean properties that have been added by a previous GetAs but not yet >made persistent? Or props added simply by setting them. You can start with an empty view with no props, and simply add values. It'll collect all props as temporary. -jcw From joshm@taconic.net Wed Aug 28 13:07:53 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SI7qa18731; Wed, 28 Aug 2002 13:07:52 -0500 Received: from Dimension (mcha-ad047.taconic.net [205.231.26.238]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g7SI7p915035; Wed, 28 Aug 2002 14:07:51 -0400 (EDT) Message-ID: <064d01c24ebd$d09709a0$700a0a0a@Dimension> From: "Joshua Muskovitz" To: "Jean-Claude Wippler" , "Metakit List" References: <022101c24eaa$25376770$a2e0fea9@tonylaptop> <20020828160207.16495@triqs.com> Subject: Re: [Metakit] Major thrashing problems when adding a property Date: Wed, 28 Aug 2002 14:07:45 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > You seem to be hitting a bad spot in MK's optimization strategy. Above a > certain size, heuristics kick in to store items individually instead of > as column. Your theory that it is hitting a heuristic doesn't make sense in our case. What Tony didn't mention is that this problem came up when we added the new properties, and the behavior was consistent REGARDLESS of the actual number of records. (In my identically formatted db, I have less than 100 records, yet my copy thrashed in exactly the same way as Tony's, immediately after making the same change to the db structure.) So we obviously suspect that it has more to do with adding the columns than the size of the data. -- josh From jcw@equi4.com Wed Aug 28 14:48:13 2002 Received: from [10.0.1.193] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SJmCa24998; Wed, 28 Aug 2002 14:48:12 -0500 From: "Jean-Claude Wippler" To: "Walter H. van Holst" , "Metakit List" Subject: Re: [Metakit] Proper spec file for Metakit 2.4.7-37 Date: Wed, 28 Aug 2002 21:45:58 +0200 Message-Id: <20020828194558.1402@triqs.com> In-Reply-To: <1030479359.23092.25.camel@Walter> References: <1030479359.23092.25.camel@Walter> X-Mailer: CTM PowerMail 4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Walter H. van Holst wrote: >The .spec file included in the tarball release of Metakit 2.4.7 is >severely outdated. After some tinkering I have changed it to the .spec >attached to this message. It works on my RH box and it results in the >following (S)RPMs: > >metakit-2.4.7-37.srpm >metakit-2.4.7-37.i386.rpm >metakit-devel-2.4.7-37.i386.rpm > >Python and TCL bindings won't build. They caused me more troubles than I >was willing to fix since I only needed Metakit to build the SimGear >library. Anyone willing to look into these is welcome. > >Building the packages is fairly straightforward: > >Put the .spec file in /usr/src/redhat/SPECS >Put the tarball in /usr/src/redhat/SOURCES > >Execute rpm -ba /usr/src/redhat/SPECS/metakit.spec and off you go. > >I am going to unsubscribe from this mailing list after this message, so >please cc: any replies to my own address. Feedback is welcome, of >course. Thanks for doing this (yes, the .spec is ancient, I had forgotten all about it). The Tcl build needs a "--with-tcl=/path" arg for configure to work, and Python needs "--enable-python" if memory serves me right. They generate libmk4tcl.so and libmk4py.so files, which need to be installed as Mk4tcl.so and Mk4py.so, respectively. You may want to have a look at the latest code in CVS, there was a fix on July 1st to install Tcl properly (with pkgIndex.tcl), see: http://www.equi4.com/metakit/CHANGES I'll fold in your .spec fixes in the next round of source changes - thx. -jcw From walter.van.holst@xs4all.nl Wed Aug 28 16:31:40 2002 Received: from smtpzilla3.xs4all.nl (smtpzilla3.xs4all.nl [194.109.127.139]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7SLVda31895; Wed, 28 Aug 2002 16:31:39 -0500 Received: from [192.168.1.2] (fossiel.xs4all.nl [213.84.32.42]) by smtpzilla3.xs4all.nl (8.12.0/8.12.0) with ESMTP id g7SLVcJ5067669; Wed, 28 Aug 2002 23:31:38 +0200 (CEST) Subject: Re: [Metakit] Proper spec file for Metakit 2.4.7-37 From: "Walter H. van Holst" To: Jean-Claude Wippler Cc: Metakit List In-Reply-To: <20020828194558.1402@triqs.com> References: <1030479359.23092.25.camel@Walter> <20020828194558.1402@triqs.com> Content-Type: text/plain Organization: COMECON Message-Id: <1030570204.23092.205.camel@Walter> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.1.0.99 (Preview Release) Date: 28 Aug 2002 23:30:04 +0200 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Wed, 2002-08-28 at 21:45, Jean-Claude Wippler wrote: > The Tcl build needs a "--with-tcl=/path" arg for configure to work, and > Python needs "--enable-python" if memory serves me right. They generate > libmk4tcl.so and libmk4py.so files, which need to be installed as > Mk4tcl.so and Mk4py.so, respectively. Trying to do this for Python results in the following build error: creating myio /bin/sh ./libtool --mode=compile g++ -c -O2 -I../unix/../include -I../unix/../python/scxx -I/include/python2.2 ../unix/../python/PyProperty.cpp g++ -c -O2 -I../unix/../include -I../unix/../python/scxx -I/include/python2.2 ../unix/../python/PyProperty.cpp -fPIC -DPIC -o .libs/PyProperty.o In file included from ../python/PyProperty.h:14, from ../unix/../python/PyProperty.cpp:10: ../python/PyHead.h:13:20: Python.h: No such file or directory In file included from ../python/scxx/PWONumber.h:8, from ../unix/../python/PyProperty.cpp:11: ../python/scxx/PWOBase.h:10:20: Python.h: No such file or directory make: *** [PyProperty.o] Error 1 At first sight (but I am neither an .spec nor an automake expert, on the contrary) this seems to be a Makefile issue and not a .spec file issue. > > You may want to have a look at the latest code in CVS, there was a fix on > July 1st to install Tcl properly (with pkgIndex.tcl), see: > http://www.equi4.com/metakit/CHANGES > > I'll fold in your .spec fixes in the next round of source changes - thx. Please note that you have to update the %version string in the top of the .spec file in case it is a new version, in case of just a new release you have to change the %release string. Otherwise you end up with an outdated .spec again. Regards, Walter -- You have a morbid aversion to dying. You probably resent the fact that you're at war and might get your head blown off any second. - Col. Sanderson in Catch-22 From mbloore@yahoo.com Wed Aug 28 19:26:46 2002 Received: from web14310.mail.yahoo.com (web14310.mail.yahoo.com [216.136.224.60]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g7T0Qja08280 for ; Wed, 28 Aug 2002 19:26:46 -0500 Message-ID: <20020829002639.60484.qmail@web14310.mail.yahoo.com> Received: from [65.95.158.100] by web14310.mail.yahoo.com via HTTP; Wed, 28 Aug 2002 17:26:39 PDT Date: Wed, 28 Aug 2002 17:26:39 -0700 (PDT) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] bug in Add() with ordered view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: if you add a row to an ordered view, Add() returns the index of the last row, instead of the index of the newly added row (as documented). for an unordered view that is, of course, the correct return value. i am using metakit 2.4.7 under RH7.2. the following program prints 0 1 2 a b c but it should print 0 1 1 a b c since the third addition goes into position 1, moving the old 1 to 2. #include #include #include c4_View vInfo; c4_StringProp pName ("name"); using namespace std; int main () { cout << endl << endl << "Hello World, this is MetaKit! " << getpid() << endl; c4_Storage sStorage ("/home/mark/tests/metakit/myfile.mk4", 1); vInfo = sStorage.GetAs ("info[name:S]") .Ordered (1); int newI = vInfo.Add (pName["a"]); cout << newI << endl; newI = vInfo.Add (pName["c"]); cout << newI << endl; newI = vInfo.Add (pName["b"]); cout << newI << endl; cout << endl; cout << pName (vInfo[0]) << endl; cout << pName (vInfo[1]) << endl; cout << pName (vInfo[2]) << endl; cout << "committing" << endl; sStorage.Commit (true); cout << "Goodbye World." << endl; return 0; } ===== -- mARK bLOORE __________________________________________________ Do You Yahoo!? Yahoo! Finance - Get real-time stock quotes http://finance.yahoo.com From mann@earthling.net Thu Aug 29 14:46:15 2002 Received: from che.kiza.com (IDENT:root@[198.144.196.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7TJkFa09195 for ; Thu, 29 Aug 2002 14:46:15 -0500 Received: from tonylaptop (puntarenas-a13.racsa.co.cr [196.40.52.142]) by che.kiza.com (8.11.2/8.11.2) with SMTP id g7TKEvO02251 for ; Thu, 29 Aug 2002 13:14:58 -0700 Message-ID: <00a501c24f94$b0844dd0$0100a8c0@tonylaptop> From: "Tony Mann" To: "Metakit List" Date: Thu, 29 Aug 2002 13:45:44 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Wrong version number in 2.4.7 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The sources for 2.4.7 have the version number set at 2.4.6, at least on the web site. That is, d4_MetaKitLibraryVersion is defined as "246". Don't know about in CVS. ..tony.. From joseph.jude@skynet.be Thu Aug 29 23:45:40 2002 Received: from riker.skynet.be (riker.skynet.be [195.238.3.89]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g7U4jda01122 for ; Thu, 29 Aug 2002 23:45:39 -0500 Received: from JosephLT (14.202-200-80.adsl.skynet.be [80.200.202.14]) by riker.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.20) with SMTP id g7U4jbI13170 for ; Fri, 30 Aug 2002 06:45:37 +0200 (MET DST) (envelope-from ) Message-ID: <003f01c24fe1$09ad1850$0200a8c0@JosephLT> From: "Joseph Jude" To: "Metakit List" References: <00a501c24f94$b0844dd0$0100a8c0@tonylaptop> Subject: [Metakit] Compilation error Date: Fri, 30 Aug 2002 06:52:23 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello list members, When I try to compile MK, I get the following errors: Error : undefined identifier '_O_NOINHERIT' fileio.cpp line 206 int flags = _O_BINARY | _O_NOINHERIT | (mode_ > 0 ? _O_RDWR : _O_RDONLY); Error : undefined identifier '_S_IREAD' fileio.cpp line 228 fd = _open(fname_, flags | _O_CREAT, _S_IREAD | _S_IWRITE); I use: W2K CW5.3 Thanks for any support ----------------------------------------- Joseph Jude From mss@mawhrin.net Mon Sep 9 12:34:49 2002 Received: from only.mawhrin.net (host4.transas.vianw.co.uk [194.119.138.213] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g89HYna05852 for ; Mon, 9 Sep 2002 12:34:49 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id BF0121F175; Mon, 9 Sep 2002 18:34:50 +0100 (BST) Date: Mon, 9 Sep 2002 18:34:50 +0100 To: metakit@equi4.com Message-ID: <20020909173450.GA3155@mawhrin.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i From: mss@mawhrin.net (Mikhail Sobolev) Subject: [Metakit] python interface Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi I am trying to build python interface. Are there any special tricks to accomplish the task? The problem I have is the following error message: PyView.cpp: In function `struct PyObject * PyView_relocrows(PyView *, PyObject *)': PyView.cpp:654: no matching function for call to `PyView::IsCompatibleWith (PyView &)' I cannot find any mention of IsCompatibleWith function. Any hints? Thanks -- Misha From mss@mawhrin.net Mon Sep 9 12:53:02 2002 Received: from only.mawhrin.net (host4.transas.vianw.co.uk [194.119.138.213] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g89Hr1a07581 for ; Mon, 9 Sep 2002 12:53:01 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id C2F951F6DE; Mon, 9 Sep 2002 18:53:02 +0100 (BST) Date: Mon, 9 Sep 2002 18:53:02 +0100 To: metakit@equi4.com Subject: Re: [Metakit] python interface Message-ID: <20020909175302.GA3250@mawhrin.net> References: <20020909173450.GA3155@mawhrin.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020909173450.GA3155@mawhrin.net> User-Agent: Mutt/1.4i From: mss@mawhrin.net (Mikhail Sobolev) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: After _careful_ investigation, it turned out that the versions of the installed headers and the headers required to build the extension are different. -- Misha On Mon, Sep 09, 2002 at 06:34:50PM +0100, Mikhail Sobolev wrote: > Hi > > I am trying to build python interface. Are there any special tricks to > accomplish the task? > > The problem I have is the following error message: > > PyView.cpp: In function `struct PyObject * PyView_relocrows(PyView *, PyObject *)': > PyView.cpp:654: no matching function for call to `PyView::IsCompatibleWith (PyView &)' > > I cannot find any mention of IsCompatibleWith function. > > Any hints? > > Thanks > > -- > Misha > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From ukumar@in.firstrain.com Tue Sep 10 05:51:44 2002 Received: from frdelhi.nd.in.firstrain.com (ptil-234-132-del.primus-india.net [203.196.132.234] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8AApfa12440 for ; Tue, 10 Sep 2002 05:51:42 -0500 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Date: Tue, 10 Sep 2002 16:22:01 +0530 X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Message-ID: <205C5F16CA92F94484F054E94520F4D709AF59@mail.in.firstrain.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Metakit versus flat-file system Thread-Index: AcJYuBfXDNxr0dTCSZWmRYTnx9d5+Q== From: "Ubhay Kumar" To: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g8AApfa12440 Subject: [Metakit] Metakit versus flat-file system Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi , I have a very simple, storage system requirement to store and retrieve (Not much relational queries required) my application data. The size of the data for my application can grow in order of dozen of MB's. To clarify further a bit every record will have only two fields one is integer "Id" and other a string (of unknown size). Now i have two questions regarding this - ?. I first wanted to understand whats the benefit of using metakit database over flat file system in terms of file access efficiency. That is what is the difference between a file i/o (open,seek,read commands) and using metakit (which uses memory map i/o internally). ?. Will metakit be useful in my case (as described above). Thanx in advance, Regards, Ubhay. From rjones@ekit-inc.com Fri Sep 13 00:07:03 2002 Received: from crown.off.ekorp.com (ekorp-203-63-137-225.eoff.ekorp.com [203.63.137.225]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8D572a03269 for ; Fri, 13 Sep 2002 00:07:03 -0500 Received: (from uucp@localhost) by crown.off.ekorp.com (8.9.3/8.9.3) id FAA18490 for metakit@equi4.com; Fri, 13 Sep 2002 05:07:00 GMT Received: from localhost(127.0.0.1), claiming to be "outpost.off.ekorp.com" via SMTP by localhost, id smtpdAAASLa4eK; Fri Sep 13 05:06:57 2002 From: Richard Jones Organization: ekit.com Inc To: metakit@equi4.com Date: Fri, 13 Sep 2002 15:06:56 +1000 User-Agent: KMail/1.4.6 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit Content-Disposition: inline Message-Id: <200209131506.56697.rjones@ekit-inc.com> Subject: [Metakit] How do I build Mk4py? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I followed the instructions in the README, by: % cd builds % ../unix/configure --prefix=/app/roundup --enable-python % make % make test % make install and that all seemed to go without a hitch. Only there's no Mk4py to be found anywhere in the metakit source tree... Richard From niki@vintech.bg Fri Sep 13 02:25:18 2002 Received: from mail.vintech.bg (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8D7PFa08460 for ; Fri, 13 Sep 2002 02:25:16 -0500 Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id BC85223481; Fri, 13 Sep 2002 10:25:08 +0300 (EEST) Message-ID: <3D819340.3070202@vintech.bg> Date: Fri, 13 Sep 2002 10:26:56 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: bg, en MIME-Version: 1.0 To: Richard Jones Cc: metakit@equi4.com Subject: Re: [Metakit] How do I build Mk4py? References: <200209131506.56697.rjones@ekit-inc.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Richard Jones wrote: > I followed the instructions in the README, by: > > % cd builds > % ../unix/configure --prefix=/app/roundup --enable-python > % make > % make test > % make install > > and that all seemed to go without a hitch. Only there's no Mk4py to be found > anywhere in the metakit source tree... > Chack that configure finds python include and lib. If it can't you must specify them. HTH Niki Spahiev From rjones@ekit-inc.com Fri Sep 13 02:35:35 2002 Received: from crown.off.ekorp.com (ekorp-203-63-137-225.eoff.ekorp.com [203.63.137.225]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8D7ZYa08818 for ; Fri, 13 Sep 2002 02:35:34 -0500 Received: (from uucp@localhost) by crown.off.ekorp.com (8.9.3/8.9.3) id HAA05169; Fri, 13 Sep 2002 07:35:15 GMT Received: from localhost(127.0.0.1), claiming to be "outpost.off.ekorp.com" via SMTP by localhost, id smtpdAAABvai_j; Fri Sep 13 07:35:04 2002 From: Richard Jones Organization: ekit.com Inc To: Niki Spahiev Subject: Re: [Metakit] How do I build Mk4py? Date: Fri, 13 Sep 2002 17:35:03 +1000 User-Agent: KMail/1.4.6 Cc: metakit@equi4.com References: <200209131506.56697.rjones@ekit-inc.com> <3D819340.3070202@vintech.bg> In-Reply-To: <3D819340.3070202@vintech.bg> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit Content-Disposition: inline Message-Id: <200209131735.03737.rjones@ekit-inc.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Fri, 13 Sep 2002 5:26 pm, Niki Spahiev wrote: > Richard Jones wrote: > > I followed the instructions in the README, by: > > > > % cd builds > > % ../unix/configure --prefix=/app/roundup --enable-python > > % make > > % make test > > % make install > > > > and that all seemed to go without a hitch. Only there's no Mk4py to be > > found anywhere in the metakit source tree... > > Chack that configure finds python include and lib. If it can't you must > specify them. It definitely found python, and compiled a bunch of python-looking stuff. No Mk4py though... Richard From jcw@equi4.com Fri Sep 13 03:10:30 2002 Received: from [10.0.1.190] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8D8ASa10070 for ; Fri, 13 Sep 2002 03:10:29 -0500 From: "Jean-Claude Wippler" To: Subject: Re: [Metakit] How do I build Mk4py? Date: Fri, 13 Sep 2002 10:10:30 +0200 Message-Id: <20020913081030.32384@triqs.com> In-Reply-To: <200209131506.56697.rjones@ekit-inc.com> References: <200209131506.56697.rjones@ekit-inc.com> X-Mailer: CTM PowerMail 4.0.1 carbon MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Richard Jones wrote: >I followed the instructions in the README, by: > >% cd builds >% ../unix/configure --prefix=/app/roundup --enable-python >% make >% make test >% make install > >and that all seemed to go without a hitch. Only there's no Mk4py >to be found anywhere in the metakit source tree... The build does not properly install Mk4py in 2.4.7 - there are some small changes in the CVS tree which fix this. Alternately, you can copy libmk4py.so to Mk4py.so - it should be in the builds/ area (or in builds/.libs/). I'll bump things up to a new 2.4.8 release later this month. -jcw From mss@mawhrin.net Fri Sep 13 03:16:05 2002 Received: from only.mawhrin.net (host4.transas.vianw.co.uk [194.119.138.213] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8D8G5a10256 for ; Fri, 13 Sep 2002 03:16:05 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id B078B1EA63; Fri, 13 Sep 2002 09:16:03 +0100 (BST) Date: Fri, 13 Sep 2002 09:16:03 +0100 To: metakit@equi4.com Subject: Re: [Metakit] How do I build Mk4py? Message-ID: <20020913081603.GA863@mawhrin.net> Mail-Followup-To: metakit@equi4.com References: <200209131506.56697.rjones@ekit-inc.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="YiEDa0DAkWCtVeE4" Content-Disposition: inline In-Reply-To: <200209131506.56697.rjones@ekit-inc.com> User-Agent: Mutt/1.4i From: mss@mawhrin.net (Mikhail Sobolev) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --YiEDa0DAkWCtVeE4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Of course, it's not exactly what you want. However, it did help me. I just wrote down a setup.py file, and copied all the stuff in a separate directory. The setup.py file is attached. -- Misha PS Please note that in the same directory I put metakit.py file, which is not in original tarball. On Fri, Sep 13, 2002 at 03:06:56PM +1000, Richard Jones wrote: > I followed the instructions in the README, by: > > % cd builds > % ../unix/configure --prefix=/app/roundup --enable-python > % make > % make test > % make install > > and that all seemed to go without a hitch. Only there's no Mk4py to be found > anywhere in the metakit source tree... > > > Richard --YiEDa0DAkWCtVeE4 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="setup.py" #! /usr/bin/python from distutils.core import setup from distutils.extension import Extension setup (name = 'Mk4Py', version = '2.4.7', ext_modules = [ Extension (name = 'Mk4py', sources = [ 'PyProperty.cpp', 'PyRowRef.cpp', 'PyStorage.cpp', 'PyView.cpp', 'scxx/PWOImp.cpp' ], include_dirs = ['scxx'], libraries = ['mk4']) ], py_modules = [ 'metakit' ]) --YiEDa0DAkWCtVeE4-- From rjones@ekit-inc.com Fri Sep 13 03:26:12 2002 Received: from crown.off.ekorp.com (ekorp-203-63-137-225.eoff.ekorp.com [203.63.137.225]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8D8Q6a10576; Fri, 13 Sep 2002 03:26:11 -0500 Received: (from uucp@localhost) by crown.off.ekorp.com (8.9.3/8.9.3) id IAA10305; Fri, 13 Sep 2002 08:26:04 GMT Received: from localhost(127.0.0.1), claiming to be "outpost.off.ekorp.com" via SMTP by localhost, id smtpdAAAHfai9t; Fri Sep 13 08:26:02 2002 From: Richard Jones Organization: ekit.com Inc To: "Jean-Claude Wippler" , Subject: Re: [Metakit] How do I build Mk4py? Date: Fri, 13 Sep 2002 18:26:02 +1000 User-Agent: KMail/1.4.6 References: <200209131506.56697.rjones@ekit-inc.com> <20020913081030.32384@triqs.com> In-Reply-To: <20020913081030.32384@triqs.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit Content-Disposition: inline Message-Id: <200209131826.02691.rjones@ekit-inc.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Fri, 13 Sep 2002 6:10 pm, Jean-Claude Wippler wrote: > Richard Jones wrote: > >I followed the instructions in the README, by: > > > >% cd builds > >% ../unix/configure --prefix=/app/roundup --enable-python > >% make > >% make test > >% make install > > > >and that all seemed to go without a hitch. Only there's no Mk4py > >to be found anywhere in the metakit source tree... > > The build does not properly install Mk4py in 2.4.7 - there are some small > changes in the CVS tree which fix this. > > Alternately, you can copy libmk4py.so to Mk4py.so - it should be in the > builds/ area (or in builds/.libs/). Ahah! Righto, now on to playing with it ;) Richard From ukumar@in.firstrain.com Thu Sep 19 03:10:55 2002 Received: from frdelhi.nd.in.firstrain.com (ptil-234-132-del.primus-india.net [203.196.132.234] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8J8Apa09635 for ; Thu, 19 Sep 2002 03:10:53 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Date: Thu, 19 Sep 2002 13:41:04 +0530 X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 content-class: urn:content-classes:message Message-ID: <205C5F16CA92F94484F054E94520F4D709AF5D@mail.in.firstrain.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Metakit concurrency Thread-Index: AcJftBmnXjKp4UeyQ4G2AtIZRK5J/A== From: "Ubhay Kumar" To: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g8J8Apa09635 Subject: [Metakit] Metakit concurrency Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi .. I have certain question regarding metakit concurrency ... 1. Can we read/write on DIFFERENT Views/Subviews in storage concurrently? 2. Is there some concurrency wrapper available for metakit? Thanx in advance, Regards, Ubhay. PS: Could anyone of you please answer my last question too. From rking35@ford.com Thu Sep 19 16:05:29 2002 Received: from dymwsm18.mailwatch.com (dymwsm18.mailwatch.com [204.253.83.220]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8JL5Ta07515 for ; Thu, 19 Sep 2002 16:05:29 -0500 Received: from mwsc0206.mw4.mailwatch.com (mwsc0206.mw4.mailwatch.com [204.253.83.136]) by dymwsm18.mailwatch.com (8.11.0/8.11.0) with ESMTP id g8JL5dw26905 for ; Thu, 19 Sep 2002 17:05:39 -0400 Received: from mail pickup service by mwsc0206.mw4.mailwatch.com with Microsoft SMTPSVC; Thu, 19 Sep 2002 17:05:39 -0400 Received: from 204.253.83.34 ([204.253.83.34]) by MWSC0206 with SMTP id 00020006832c8769-bba2-4eae-be97-c8fad5ae60ca; Thu, 19 Sep 2002 17:05:34 -0500 Received: from ecpo1.azell.com (ecpo1.azell.com [136.1.7.4]) by dymwsm12.mailwatch.com (8.11.0/8.11.0) with ESMTP id g8JL5XQ08075 for ; Thu, 19 Sep 2002 17:05:33 -0400 Received: from na1fcs04.dearborn.ford.com ([19.59.114.65]) by ecpo1.azell.com (Mirapoint Messaging Server MOS 3.2.0.52-EA) with ESMTP id AEV63572; Thu, 19 Sep 2002 17:05:30 -0400 (EDT) Received: by na1fcs04.dearborn.ford.com with Internet Mail Service (5.5.2655.15) id ; Thu, 19 Sep 2002 17:05:30 -0400 Message-ID: From: "King, Rick (R.B.)" To: "'metakit@equi4.com'" Date: Thu, 19 Sep 2002 17:05:30 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.15) Content-Type: text/plain; charset="iso-8859-1" HOP-COUNT: 1 X-MAILWATCH-INSTANCEID: 01020006832c8769-bba2-4eae-be97-c8fad5ae60ca X-OriginalArrivalTime: 19 Sep 2002 21:05:39.0461 (UTC) FILETIME=[4EA76B50:01C26020] Subject: [Metakit] Help with first time installation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I want to use metakit with python. At first, I thought that binaries were available for python, but that does not appear to be the case. I can build with msvc6, but it sure would be nice to get the binaries so I don't have to bother. But, there is no "mk4py.py", or "mk.py", or anything like that anywhere that I can find. Am I missing it on the website somewhere? Thanks for your help! -Rick From gmcm@hypernet.com Thu Sep 19 16:45:35 2002 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8JLjZa08714 for ; Thu, 19 Sep 2002 16:45:35 -0500 Received: from PARANOIA (204.176.40.98) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Thu, 19 Sep 2002 17:44:07 -0400 From: "Gordon McMillan" To: "King, Rick (R.B.)" , "'metakit@equi4.com'" Date: Thu, 19 Sep 2002 17:44:56 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Help with first time installation Reply-to: gmcm@hypernet.com Message-ID: <3D8A0D18.4869.BB2EBC2@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 19 Sep 2002 at 17:05, King, Rick (R.B.) wrote: > Hi, > I want to use metakit with python. > > At first, I thought that binaries were available > for python, but that does not appear to be the case. > I can build with msvc6, but it sure would be nice to > get the binaries so I don't have to bother. There's a 2.4.6 (one minor behind) built for Python 2.2 at http://www.equi4.com/pub/mk/mk-2.4.6-win32/. The build is not at all painful (there's a dsw for msvc6). I'd recommend building (to get up to 2.4.7). > But, there is no "mk4py.py", or "mk.py", or > anything like that anywhere that I can find. Am I > missing it on > the website somewhere? All you need is Mk4py.dll and metakit.py, which is at http://www.equi4.com/pub/mk/ (and you don't even really need that; it just gives you a nice dump utility). It's part of the source download, too. -- Gordon http://www.mcmillan-inc.com/ From mbloore@yahoo.com Mon Sep 23 09:57:17 2002 Received: from web14306.mail.yahoo.com (web14306.mail.yahoo.com [216.136.173.82]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g8NEvGa03475 for ; Mon, 23 Sep 2002 09:57:16 -0500 Message-ID: <20020923145730.50517.qmail@web14306.mail.yahoo.com> Received: from [64.231.192.130] by web14306.mail.yahoo.com via HTTP; Mon, 23 Sep 2002 07:57:30 PDT Date: Mon, 23 Sep 2002 07:57:30 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] changing c4_BytesProp in place? To: metakit In-Reply-To: <20020801170943.56546.qmail@web14305.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i seem to have found a solution, or at least the best possible: first, use c4_BytesRef::operator c4_Bytes() to get direct access to the data. using c4_BytesProp::Get(const c4_RowRef &) or c4_BytesRef::Access(t4_i32, int) makes a copy. then, use c4_BytesRef::Modify(const c4_Bytes &, t4_i32, int) on the original BytesRef object to make changes. in most cases, this writes through to the on-disk data, rather than making any temporary copies. the only exception arises when the data is split across segments in the disk file, in which case the cast operator is forced to copy data from the two segments. this still causes my application to spend considerable time in memory allocation, though it is greatly improved from the original version. i don't suppose i can get control of the segmentation? --- mARK bLOORE wrote: > i have an application that stores data in a large number of c4_Bytes > objects (two columns, 98,000 rows). at each update, a few of them > get > data appended, and much data shuffling ensues. i allocate and fill a > new object of the new size, then assign it to the appropriate column > and row. > > it would be better if i could pre-allocate those objects, and fill > unused space as needed, only increasing their sizes when space runs > out. i can see no obvious way to do this, since c4_Bytes::Contents() > returns a pointer to const, and even if i did update that buffer > there > is no way to tell MetaKit that the data has changed. > > is there some way to modify a bytes object in place and get it > commited? > > > ===== > -- > mARK bLOORE > > __________________________________________________ > Do You Yahoo!? > Yahoo! Health - Feel better, live better > http://health.yahoo.com > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit ===== -- mARK bLOORE __________________________________________________ Do you Yahoo!? New DSL Internet Access from SBC & Yahoo! http://sbc.yahoo.com From jcw@equi4.com Mon Sep 30 08:18:38 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g8UDIba28150 for ; Mon, 30 Sep 2002 08:18:38 -0500 Date: Mon, 30 Sep 2002 15:18:56 +0200 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <205C5F16CA92F94484F054E94520F4D7111B88@mail.in.firstrain.com> Message-Id: <2C3C6820-D477-11D6-9FB5-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.546) Subject: [Metakit] Re: Metakit concurrency urgent Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ubhay Kumar wrote: My apologies for not responding sooner. > > 1. Can we read/write on DIFFERENT Views/Subviews in storage > concurrently? Reading should work (the only thing a read does is adjust a per-view property cache), writing definitely will not - there is too much shared underneth, i.e. the c4_Storage and c4_Persist classes. > 2. Is there some concurrency wrapper available for metakit? > I don't recall with certainty, but remember seeing mention of this (over a year ago?). Not sure whether it was available - though you can create one yourself. The only global data structure is a property symbol table, which has been properly mutex-protected (on Win32, Unix should not be hard to do as well, see view.cpp where everything is centralized. The Mk4tcl wrapper introduces a monolithic access lock, so this is definitely feasible. > So, My doubts are: > > 1. Is it true, that, in a multithreaded application if thread t1 > "created/opened" > a database then ONLY t1 can "modify/update" the database. All other > threads (like > t2) needs to send request to t1 for any "modification/update". Yes. > If answer to above is true, will that mean, even synchronizing the > "write" access using some locks (mutex etc) > between t1 and t2 would not suffice. Yes - because the write would be ok, but read access would not be safe while such writes are taking place. > 2. If answer to above is true, is there any workaround to that(like > Concurrency Wrapper)? And > how that will help. My suggestions (talk is cheap, I know) would be to consider a single worker thread, with all other threads passing on work to the worker thread for both read and write. If concurrent read is essential, and writes are secondary, you could open a datafile N times read-only, each in a thread, and once "read-extend", which is a mode that only appends changes to the end of a datafile, hence the readers won't see it. Then, after commit, you will need to find a way to signal all your reader threads to re-open the file, at which point they will see the new state. You will also need to occasionally compact the (ever-growing) datafile when no-one is using it (serialize to a new file and replace the old file with it). This is not as far-fetched as it seems, AFAIK the Mac OS X standard AddressBook does exactly this. It'll all depend on requirements - if the datafile is large, you may need more tricks to make sure they work off a shared memory-mapped file area (which has not been done before to my knowledge, but should be feasible). A more simplistic approach might perhaps also be an option for you: keep a datafile for multi-thread read-only use, and another copy to have a single thread write to it, then notify and collaborate with all threads to do a swap/copy on commit. Clearly, each of these approaches has its trade-offs. -jcw From jcw@equi4.com Thu Oct 3 04:09:33 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9399Wa20992; Thu, 3 Oct 2002 04:09:32 -0500 Date: Thu, 3 Oct 2002 11:09:54 +0200 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) Cc: andreask@ActiveState.com To: MetaKit mailing list From: Jean-Claude Wippler Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.546) Subject: [Metakit] Building on HP-UX? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, In the quest to make MK conquer the world, Andreas Kupries of ActiveState and I have been having a go at building MetaKit for HP-UX on two different setups (hppa2.0 hpux11.11 hppa1.0 hpux11.00 and hppa2.0 hpux11.11). Some config/make issues came up, but things now seem to work for the statically-linked case. The trouble is that we really would like to solve the shared-library case as well. So far we solved details such as .sl vs .so, but the resulting code appears to crash (this is the Mk4tcl binding, though it may well be unrelated to Tcl). It may be a problem of trying to load a C++ extension in a C main, or other C++-specific troubles. I am going to look a bit deeper, but maybe someone on this list has hints and tips which can help us along? It goes without saying that my aim is to make MK build out of the box, static as well as shared. Let me also add that MK has been built on several more 64-bit systems recently (Linux variants, mostly), all leading to minor tweaks which indeed make it build out of the box. For an overview of the 20+ builds far, see http://www.equi4.com/tclkit/ - these include MK statically. Getting onto more and more platforms is a major goal for me, which includes the upper 64-bit architectures as well as the lower-end PDA's (WinCE is there, things like Palm OS 5 and Zaurus/Linux are on my radar). If you have suggestions w.r.t. building for HP-UX (AIX is next target, btw), please share them if you can, either on this list or in private email. -jcw From jcw@equi4.com Fri Oct 4 06:17:19 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g94BHGa05386; Fri, 4 Oct 2002 06:17:18 -0500 Date: Fri, 4 Oct 2002 13:17:38 +0200 Subject: Summary: [Metakit] Building on HP-UX? Content-Type: multipart/mixed; boundary=Apple-Mail-11--411516078 Mime-Version: 1.0 (Apple Message framework v546) Cc: Steve.Ball@zveno.com, andreask@ActiveState.com, MetaKit mailing list To: Peter Farmer From: Jean-Claude Wippler In-Reply-To: <200210040120.g941KcM31776@mycroftv> Message-Id: X-Mailer: Apple Mail (2.546) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-11--411516078 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Peter, Thank you for jumping in here, and thanks also Steve B, for passing this on within your organization. The great news: it looks like Andreas has solved things (aside from predictable issues such as .sl vs .so) for now. His solution was to link to -lCsup, and to add a small assembler (!) stub to get static C++ init chains right, I think. Andreas mentioned that he found this on the net through Google. As I understand it, this logic is needed when a C main program calls a C++ shared lib extension. Which in the case of MK remains relatively simple because I've avoided all uses of static initializers with objects (constructors/destructors, to be precise). I'm copying the information below. It is not quite clear to me how I can adjust the config/make files in my project to handle this, but the good news is that this manual tweaking seems to have produced a shared library that works just fine. > * if your using the HPUX compiler tools , I'd suggest dropping them in > favour > of a GCC toolchain. I find that most apps are a lot easier to port > with gcc > and gmake et. al. Essentially all the tricky problems have been > handled by > the gcc folks 8-). It may not deliver the best optimised binary , but > [...] I have to admit that the trade-offs are as clear as mud to me. In fact, I'm having trouble understanding which HP-UX platforms are compatible and how many builds one may eventually need to support. So instead, let me simply pass this advice on to others. If someone on the mailing list has suggestions on how to integrate this, or better still wants to go ahead and create a patch set, please do! Regards, Jean-Claude [Andreas] >> A bit more experimentation and I think now know how to do it. >> Here is what I did. >> >> Uno) Tell libtool to link the .sl against -lCsup >> (aC++ runtime support library). This gets rid of >> unresolved __shlinit and introduces some new unresolved >> symbols. >> >> Duo) Compile the attached assembler file. >> Cmd: as -o cpprt0_stubs.o cpprt_stubs.s > > Might be better to use -o .libs/cpprt0_stubs.o to keep in sync with > what > libtool is doing > >> >> Tres) libtools has to be convinced somehow to add this object file >> to the library (I did it manually, i.e. I ran the makefile, >> captured the output extracted the commands generated by libtool >> (containing the .o files) and extended these commands. > > Use -Wl,cpprt0_stubs.o as flag in the libtool lines (mode=link) > to forward this external object file to the linker. > > (See docs > http://www.gnu.org/software/libtool/manual.html#Link%20mode) > > Note: > All of the above is for > > CXX=aCC > CC=cc > > This will not apply to builds involving g++ and/or gcc. > >> Voila. > > Phewwww. Done. --Apple-Mail-11--411516078 Content-Disposition: attachment; filename=cpprt0_stubs.s Content-Transfer-Encoding: 7bit Content-Type: application/octet-stream; x-unix-mode=0444; name="cpprt0_stubs.s" .code ; stubs for static constructors in a.out .export __StaticCtorTable_Start,data .export __StaticCtorTable_End,data __StaticCtorTable_Start __StaticCtorTable_End .data ; stubs for static constructors in a.out, compiled with +z/+Z .export __ZStaticCtorTable_Start,data .export __ZStaticCtorTable_End,data __ZStaticCtorTable_Start __ZStaticCtorTable_End --Apple-Mail-11--411516078 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed --Apple-Mail-11--411516078-- From bertram@bwh.harvard.edu Mon Oct 7 09:59:19 2002 Received: from clifford.bwh.harvard.edu (clifford.bwh.harvard.edu [134.174.9.41]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g97ExJa23295 for ; Mon, 7 Oct 2002 09:59:19 -0500 Received: from stephanie.bwh.harvard.edu (stephanie [134.174.9.197]) by clifford.bwh.harvard.edu (8.10.2+Sun/8.11.0) with ESMTP id g97Exco20325 for ; Mon, 7 Oct 2002 10:59:38 -0400 (EDT) Received: from bwh.harvard.edu (localhost [127.0.0.1]) by stephanie.bwh.harvard.edu (8.10.2+Sun/8.11.0) with ESMTP id g97Exch23736 for ; Mon, 7 Oct 2002 10:59:38 -0400 (EDT) Message-ID: <3DA1A15A.12CE154B@bwh.harvard.edu> Date: Mon, 07 Oct 2002 10:59:38 -0400 From: "Bertram G. Herzog" Reply-To: Bertram.Herzog@web.de X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.8 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Binaries for Solaris 5.8 - "Error in startup script" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi to all! Does someone have compiled and linkes Shared Objects for the Solaris 5.8-OS. I tried to compile and link the Metakit 2.4.7. as explained in the Metakit-Tutorial by Mark Roseman in the appendix, but now only have a direktory full of different *.o-files. Apparently, they don't have ALL the information needed for one Shared Object. The file libmk4tcl.so sure seems to be the file-of-choice, but when loading this one in my application, the following error-message occurs: Error in startup script: couldn't find procedure Mk_Init while executing "load mypath/work/metakit2.4.7./libmk4tcl.so" (file "/mypath/work/tcltest/test.tcl" line 4) Thank you for your help, Bertram Herzog P.S.: If this is the wrong mailing-list for problems with Metak-Kit, I apologize for the appearance of this mail (and am glad to hear where to post such things.) From jcw@equi4.com Mon Oct 7 10:16:31 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g97FGVa24605 for ; Mon, 7 Oct 2002 10:16:31 -0500 Date: Mon, 7 Oct 2002 17:16:59 +0200 Subject: Re: [Metakit] Binaries for Solaris 5.8 - "Error in startup script" Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3DA1A15A.12CE154B@bwh.harvard.edu> Message-Id: X-Mailer: Apple Mail (2.546) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bertram G. Herzog wrote: > Does someone have compiled and linkes Shared Objects for the Solaris > 5.8-OS. I tried to compile and link the Metakit 2.4.7. as explained in > the Metakit-Tutorial by Mark Roseman in the appendix, but now only have > a direktory full of different *.o-files. See http://www.equi4.com/metakit/CHANGES > Apparently, they don't have ALL the information needed for one Shared > Object. The file libmk4tcl.so sure seems to be the file-of-choice, but > when loading this one in my application, the following error-message > occurs: > Error in startup script: couldn't find procedure Mk_Init > while executing > "load mypath/work/metakit2.4.7./libmk4tcl.so" > (file "/mypath/work/tcltest/test.tcl" line 4) Copy libmk4tcl.so to Mk4tcl.so, then load that. > P.S.: If this is the wrong mailing-list for problems with Metak-Kit, I > apologize for the appearance of this mail (and am glad to hear where to > post such things.) Right spot. But I had to manually approve your post - please subscribe (this measure was added to prevent spam from being distributed on this list). See the link listed below. -jcw From mbloore@yahoo.com Mon Oct 7 14:56:02 2002 Received: from web14310.mail.yahoo.com (web14310.mail.yahoo.com [216.136.224.60]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g97Ju2a15067 for ; Mon, 7 Oct 2002 14:56:02 -0500 Message-ID: <20021007195626.96040.qmail@web14310.mail.yahoo.com> Received: from [65.95.159.152] by web14310.mail.yahoo.com via HTTP; Mon, 07 Oct 2002 12:56:26 PDT Date: Mon, 7 Oct 2002 12:56:26 -0700 (PDT) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] limited number of columns Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i have an application that, among other data, keeps some 98,000 rows in two columns. it spends a lot of time allocating memory when getting data that is split across segments, so, without really expecting it to help or even to work, i tested organizing it as two rows in 98,000 columns. and indeed, it didn't work! i am working in C++. i tried to make a std::vector of 99,000 c4_StringProp objects, with unique names. the test program starts looping after some 32,000 have been done (at 32K, i would guess; i didn't check) in the call to the c4_StringProp constructor. is the property ID rolling over? ===== -- mARK bLOORE __________________________________________________ Do you Yahoo!? Faith Hill - Exclusive Performances, Videos & More http://faith.yahoo.com From jcw@equi4.com Mon Oct 7 16:48:48 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g97Lmla22308; Mon, 7 Oct 2002 16:48:47 -0500 Date: Mon, 7 Oct 2002 23:49:15 +0200 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) Cc: MetaKit mailing list To: Starkit list From: Jean-Claude Wippler In-Reply-To: Message-Id: <9F8A5256-DA3E-11D6-8313-00050251EF2F@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.546) Subject: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavior Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: john_zouck@jhuapl.edu wrote: > I have had a problem with tclkit for linux when running on an SGI > computer > with Redhat linux. It seems to run fine, but then when I exit it hangs, > and leaves processes that can't be killed without rebooting. I just > downloaded a new tclkit from equi4 and the behavior is the same as the > older ones: > > jhz[\H:(~)]8: ./tclkit-linux-x86 > % exit > > At this point I can't ctrl-c or ctrl-z. [...] > Can't kill -9 either. Ouch, that is a major bug. You're not the first one to run into it, nor is it limited to Linux. First let's get this straight: a user-mode program which causes the OS to leave it in an uniterruptable state is an *OS* bug. So the most *accurate* advice I can give is: report this as a major bug to your OS vendor, RedHat in this case. I cannot repeat this enough: a user-mode program, not using any root permissions, and doing nothing but publicly documented system calls, should not be able to cause this. If I had mischief in mind, I could probably use this flaw as DOS-attack: just launch processes forever, each hitting this OS bug... Unfortunately, this all comes down to the simple fact that MK is triggering a serious bug on three different OS'es so far (RH, HPUX, OSX), so I have no option other than try and find a workaround. The world is too big, wheels turn too slowly, and MK is not likely to make anyone change things - even when a reproducible hard bug like this proves them to be causing it. Sigh... Ok, next step is for *me* to be able to reproduce this. 1) What version of RedHat are you using? If I can grab a copy of it somewhere, I'll set up a machine and repeat your runs. 2) What is the size/date of the tclkit executable you are using, and also: what does the Tcl command "puts $vfs::tclkit_version" return? With a bit of luck, your hang may be one I can reproduce - *then* I can start digging in. -jcw PS. FWIW, the problem seems to be related to having a datafile open, writing to it, even extending it at times, while there is a mmap on it (which is R/O) and with some of the writes using pointers into that map as data. With page-size allocations, a write to a page using data from that same page (never overlapping) may well mightily confuse an imperfect OS, as it tries to copy-on-write perhaps, page-faulting with a lock pending on the write while the read is in use. Whatever. I may even try to create a small example in an attempt to force OS vendors to fix this after all... But first, I need a setup to reproduce this behavior. As if there isn't enough work... :) From Murat_Berk@bmc.com Mon Oct 7 16:58:41 2002 Received: from creeper.bmc.com (firebird.bmc.com [198.207.223.228]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g97Lwfa22808; Mon, 7 Oct 2002 16:58:41 -0500 Received: from ec02-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2/8.10.2) with ESMTP id g97LvbH15902; Mon, 7 Oct 2002 16:57:37 -0500 (CDT) Received: by ec02-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Mon, 7 Oct 2002 16:57:51 -0500 Message-ID: From: "Berk, Murat" To: "'Jean-Claude Wippler'" , Starkit list Cc: MetaKit mailing list Subject: RE: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavio r Date: Mon, 7 Oct 2002 16:57:49 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is the data on NFS mounted directories? We used to have this kind of problems also where we build some databases as part of build process into NFS. Some memory access patterns was causing big problems to us between Auspex NFS server and Solaris workstations (Solaris NFS server was OK) THen we had similar problems on other ones. (we used to go infinite loop and take 100% on the build machine the whole night) I think mmap across nfs was creating some problems which we never saw under very heavy load using local databases. This might be totally unrelated. I hope the above will be helpfull as a datapoint for this kind of issues... Murat -----Original Message----- From: Jean-Claude Wippler [mailto:jcw@equi4.com] Sent: Monday, October 07, 2002 5:49 PM To: Starkit list Cc: MetaKit mailing list Subject: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavior john_zouck@jhuapl.edu wrote: > I have had a problem with tclkit for linux when running on an SGI > computer > with Redhat linux. It seems to run fine, but then when I exit it hangs, > and leaves processes that can't be killed without rebooting. I just > downloaded a new tclkit from equi4 and the behavior is the same as the > older ones: > > jhz[\H:(~)]8: ./tclkit-linux-x86 > % exit > > At this point I can't ctrl-c or ctrl-z. [...] > Can't kill -9 either. Ouch, that is a major bug. You're not the first one to run into it, nor is it limited to Linux. First let's get this straight: a user-mode program which causes the OS to leave it in an uniterruptable state is an *OS* bug. So the most *accurate* advice I can give is: report this as a major bug to your OS vendor, RedHat in this case. I cannot repeat this enough: a user-mode program, not using any root permissions, and doing nothing but publicly documented system calls, should not be able to cause this. If I had mischief in mind, I could probably use this flaw as DOS-attack: just launch processes forever, each hitting this OS bug... Unfortunately, this all comes down to the simple fact that MK is triggering a serious bug on three different OS'es so far (RH, HPUX, OSX), so I have no option other than try and find a workaround. The world is too big, wheels turn too slowly, and MK is not likely to make anyone change things - even when a reproducible hard bug like this proves them to be causing it. Sigh... Ok, next step is for *me* to be able to reproduce this. 1) What version of RedHat are you using? If I can grab a copy of it somewhere, I'll set up a machine and repeat your runs. 2) What is the size/date of the tclkit executable you are using, and also: what does the Tcl command "puts $vfs::tclkit_version" return? With a bit of luck, your hang may be one I can reproduce - *then* I can start digging in. -jcw PS. FWIW, the problem seems to be related to having a datafile open, writing to it, even extending it at times, while there is a mmap on it (which is R/O) and with some of the writes using pointers into that map as data. With page-size allocations, a write to a page using data from that same page (never overlapping) may well mightily confuse an imperfect OS, as it tries to copy-on-write perhaps, page-faulting with a lock pending on the write while the read is in use. Whatever. I may even try to create a small example in an attempt to force OS vendors to fix this after all... But first, I need a setup to reproduce this behavior. As if there isn't enough work... :) _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From tom.krehbiel@motorola.com Mon Oct 7 18:16:34 2002 Received: from motgate2.mot.com (motgate2.mot.com [136.182.1.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g97NGYa26109; Mon, 7 Oct 2002 18:16:34 -0500 Received: [from mothost.mot.com (mothost.mot.com [129.188.137.101]) by motgate2.mot.com (motgate2 2.1) with ESMTP id QAA12813; Mon, 7 Oct 2002 16:17:02 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by mothost.mot.com (MOT-pobox 2.0) with ESMTP id QAA00286; Mon, 7 Oct 2002 16:16:58 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id S47YSD5S; Mon, 7 Oct 2002 16:16:57 -0700 Message-ID: <3DA215E9.9F4D2CD2@motorola.com> Date: Mon, 07 Oct 2002 16:16:57 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Starkit list , MetaKit mailing list Subject: Re: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavior References: <9F8A5256-DA3E-11D6-8313-00050251EF2F@equi4.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude, <..snip..> > Unfortunately, this all comes down to the simple fact that MK is > triggering a serious bug on three different OS'es so far (RH, HPUX, > OSX), so I have no option other than try and find a workaround. The > world is too big, wheels turn too slowly, and MK is not likely to make > anyone change things - even when a reproducible hard bug like this > proves them to be causing it. > > Sigh... > > Ok, next step is for *me* to be able to reproduce this. > > 1) What version of RedHat are you using? If I can grab a copy of it > somewhere, I'll set up a machine and repeat your runs. > > 2) What is the size/date of the tclkit executable you are using, and > also: > what does the Tcl command "puts $vfs::tclkit_version" return? > > With a bit of luck, your hang may be one I can reproduce - *then* I can > start digging in. > > -jcw > > PS. FWIW, the problem seems to be related to having a datafile open, > writing to it, even extending it at times, while there is a mmap on it > (which is R/O) and with some of the writes using pointers into that map > as data. With page-size allocations, a write to a page using data from > that same page (never overlapping) may well mightily confuse an > imperfect OS, as it tries to copy-on-write perhaps, page-faulting with > a lock pending on the write while the read is in use. Whatever. I may > even try to create a small example in an attempt to force OS vendors to > fix this after all... My sysadmin has been out the passed couple of days. When he returns I'm going to ask him if there are any patches for mmap bugs for my HP. It is possible that the bug has been reported and fixed but my system hasn't been patch since the strategy here is "never patch a working system". I'll let you know what I find out when he returns. Tom K. From tom.krehbiel@motorola.com Mon Oct 7 18:22:46 2002 Received: from motgate2.mot.com (motgate2.mot.com [136.182.1.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g97NMja26397; Mon, 7 Oct 2002 18:22:45 -0500 Received: [from pobox4.mot.com (pobox4.mot.com [10.64.251.243]) by motgate2.mot.com (motgate2 2.1) with ESMTP id QAA16161; Mon, 7 Oct 2002 16:23:13 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by pobox4.mot.com (MOT-pobox4 2.0) with ESMTP id QAA10644; Mon, 7 Oct 2002 16:23:09 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id S47YSD6P; Mon, 7 Oct 2002 16:23:09 -0700 Message-ID: <3DA2175D.27740634@motorola.com> Date: Mon, 07 Oct 2002 16:23:09 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 CC: "'Jean-Claude Wippler'" , Starkit list , MetaKit mailing list Subject: Re: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavior References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Berk, > Is the data on NFS mounted directories? > > We used to have this kind of problems also where we build some databases > as part of build process into NFS. > > Some memory access patterns was causing big problems to us between > Auspex NFS server and Solaris workstations (Solaris NFS server was OK) > THen we had similar problems on other ones. (we used to go infinite loop > and take 100% on the build machine the whole night) > > I think mmap across nfs was creating some problems which we never saw under > very heavy load using local databases. > > This might be totally unrelated. I hope the above will be helpfull as a > datapoint for this kind of issues... Interesting :-/ The problem on my HP showed up when I was doing a save from wikit. The problem is stable. All my files are on a file server an I thought there was the possibility of a problem between NFS as implemented on HP and NFS as implemented on the file server. Can I test the hypothesis by storing the db in /tmp on the host? I thought about trying it but haven't done it yet. Tom K. From nicolasb@maich.gr Tue Oct 8 01:55:36 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g986tZa17509 for ; Tue, 8 Oct 2002 01:55:35 -0500 Received: FROM maich.gr BY mail-server ; Tue Oct 08 10:25:28 2002 +0300 Message-ID: <3DA282B5.6080902@maich.gr> Date: Tue, 08 Oct 2002 10:01:09 +0300 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0rc2) Gecko/20020510 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Starkit list , MetaKit mailing list Subject: Re: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavior References: <9F8A5256-DA3E-11D6-8313-00050251EF2F@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Sigh... > > Ok, next step is for *me* to be able to reproduce this. > > 1) What version of RedHat are you using? If I can grab a copy of it > somewhere, I'll set up a machine and repeat your runs. > > 2) What is the size/date of the tclkit executable you are using, and also: > what does the Tcl command "puts $vfs::tclkit_version" return? > Hi, I have a RH 7.1 server and could run some tests... regards, nicolas boretos From john_zouck@jhuapl.edu Tue Oct 8 08:44:49 2002 Received: from houston.jhuapl.edu (houston.jhuapl.edu [128.244.26.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g98Dina09654; Tue, 8 Oct 2002 08:44:49 -0500 Received: from CONVERSION-DAEMON by houston.jhuapl.edu (PMDF V5.2-32 #40040) id <0H3O002010V3LV@houston.jhuapl.edu>; Tue, 8 Oct 2002 09:45:04 -0400 (EDT) Received: from aplexus.jhuapl.edu (aplexus.jhuapl.edu [128.244.28.6]) by houston.jhuapl.edu (PMDF V5.2-32 #40040) with ESMTP id <0H3O00FHG0UXPP@houston.jhuapl.edu>; Tue, 08 Oct 2002 09:45:03 -0400 (EDT) Received: from localhost (jhz@localhost) by aplexus.jhuapl.edu (8.10.2+Sun/8.10.2) with ESMTP id g98Diua08154; Tue, 08 Oct 2002 09:44:57 -0400 (EDT) Date: Tue, 08 Oct 2002 09:44:56 -0400 (EDT) From: john_zouck@jhuapl.edu In-reply-to: <9F8A5256-DA3E-11D6-8313-00050251EF2F@equi4.com> X-Sender: jhz@aplexus.jhuapl.edu To: Jean-Claude Wippler Cc: Starkit list , MetaKit mailing list Message-id: MIME-version: 1.0 Content-type: TEXT/PLAIN; charset=US-ASCII Content-transfer-encoding: 7BIT X-Authentication-warning: aplexus.jhuapl.edu: jhz owned process doing -bs Subject: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavior Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude, I have included the information you wanted in-line with your email text. I agree that the OS should not allow such things to happen. Since it is linux, there is a chance we could even fix the bug, but I have neither the time or familiarity with linux internals to tackle this at this time. I will look into letting the linux community and RedHat know about this. Might help improve their system. Even as 'su' the process could not be killed. Maybe it has to do with the SGI SMP aspects... By the way, you and the other developers deservemany thanks for the fine work and ideas you have contributed to bring the starkit concept and implementation to this point. On Mon, 7 Oct 2002, Jean-Claude Wippler wrote: > john_zouck@jhuapl.edu wrote: > > > I have had a problem with tclkit for linux when running on an SGI > > computer > > with Redhat linux. It seems to run fine, but then when I exit it hangs, > > and leaves processes that can't be killed without rebooting. I just > > downloaded a new tclkit from equi4 and the behavior is the same as the > > older ones: > > > > jhz[\H:(~)]8: ./tclkit-linux-x86 > > % exit > > > > At this point I can't ctrl-c or ctrl-z. > [...] > > Can't kill -9 either. > > Ouch, that is a major bug. You're not the first one to run into it, > nor is it limited to Linux. > > First let's get this straight: a user-mode program which causes the OS > to leave it in an uniterruptable state is an *OS* bug. So the most > *accurate* advice I can give is: report this as a major bug to your OS > vendor, RedHat in this case. > > I cannot repeat this enough: a user-mode program, not using any root > permissions, and doing nothing but publicly documented system calls, > should not be able to cause this. If I had mischief in mind, I could > probably use this flaw as DOS-attack: just launch processes forever, > each hitting this OS bug... > > Unfortunately, this all comes down to the simple fact that MK is > triggering a serious bug on three different OS'es so far (RH, HPUX, > OSX), so I have no option other than try and find a workaround. The > world is too big, wheels turn too slowly, and MK is not likely to make > anyone change things - even when a reproducible hard bug like this > proves them to be causing it. > > Sigh... > > Ok, next step is for *me* to be able to reproduce this. > > 1) What version of RedHat are you using? If I can grab a copy of it > somewhere, I'll set up a machine and repeat your runs. Trying 128.244.28.62... Connected to ardana.jhuapl.edu. Escape character is '^]'. Red Hat Linux release 6.2 (Zoot) Kernel 2.2.15-3SGI_39smp on an i686 login: jhz Password: Last login: Mon Oct 7 17:15:17 from aplexus.jhuapl.edu > > 2) What is the size/date of the tclkit executable you are using, and > also: > what does the Tcl command "puts $vfs::tclkit_version" return? jhz[\H:(~)]1: recent total 57892 -rwxr-xr-x 1 jhz users 2864054 Oct 7 17:11 tclkit-linux-x86* jhz[\H:(~)]1: sum tclkit-* 47402 2797 jhz[\H:(~)]4: ./tclkit-linux-x86 % puts $vfs::tclkit_version 200209 > > With a bit of luck, your hang may be one I can reproduce - *then* I can > start digging in. > > -jcw > > PS. FWIW, the problem seems to be related to having a datafile open, > writing to it, even extending it at times, while there is a mmap on it > (which is R/O) and with some of the writes using pointers into that map > as data. With page-size allocations, a write to a page using data from > that same page (never overlapping) may well mightily confuse an > imperfect OS, as it tries to copy-on-write perhaps, page-faulting with > a lock pending on the write while the read is in use. Whatever. I may > even try to create a small example in an attempt to force OS vendors to > fix this after all... > > But first, I need a setup to reproduce this behavior. > > As if there isn't enough work... :) > > _______________________________________________ > starkit mailing list > starkit@equi4.com > http://www.equi4.com/mailman/listinfo/starkit > ============================ John H. Zouck The Johns Hopkins University Applied Physics Laboratory ============================ From john_zouck@jhuapl.edu Tue Oct 8 08:47:54 2002 Received: from houston.jhuapl.edu (houston.jhuapl.edu [128.244.26.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g98Dlsa09935; Tue, 8 Oct 2002 08:47:54 -0500 Received: from CONVERSION-DAEMON by houston.jhuapl.edu (PMDF V5.2-32 #40040) id <0H3O0030110894@houston.jhuapl.edu>; Tue, 8 Oct 2002 09:48:09 -0400 (EDT) Received: from aplexus.jhuapl.edu (aplexus.jhuapl.edu [128.244.28.6]) by houston.jhuapl.edu (PMDF V5.2-32 #40040) with ESMTP id <0H3O00FGN100EI@houston.jhuapl.edu>; Tue, 08 Oct 2002 09:48:07 -0400 (EDT) Received: from localhost (jhz@localhost) by aplexus.jhuapl.edu (8.10.2+Sun/8.10.2) with ESMTP id g98Dm0308291; Tue, 08 Oct 2002 09:48:00 -0400 (EDT) Date: Tue, 08 Oct 2002 09:48:00 -0400 (EDT) From: john_zouck@jhuapl.edu Subject: RE: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavio r In-reply-to: X-Sender: jhz@aplexus.jhuapl.edu To: "Berk, Murat" Cc: "'Jean-Claude Wippler'" , Starkit list , MetaKit mailing list Message-id: MIME-version: 1.0 Content-type: TEXT/PLAIN; charset=US-ASCII Content-transfer-encoding: 7BIT X-Authentication-warning: aplexus.jhuapl.edu: jhz owned process doing -bs Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, 7 Oct 2002, Berk, Murat wrote: > Is the data on NFS mounted directories? All the files are on the local disk: jhz[\H:(~)]4: df Filesystem 1k-blocks Used Available Use% Mounted on /dev/scsi/host0/bus0/target0/lun0/part5 4134900 2506264 1418588 64% / /dev/scsi/host0/bus0/target0/lun0/part1 31079 8412 21063 29% /boot /dev/scsi/host0/bus0/target0/lun0/part6 13227316 7121508 5433892 57% /home > > We used to have this kind of problems also where we build some databases > as part of build process into NFS. > > Some memory access patterns was causing big problems to us between > Auspex NFS server and Solaris workstations (Solaris NFS server was OK) > THen we had similar problems on other ones. (we used to go infinite loop > and take 100% on the build machine the whole night) > > I think mmap across nfs was creating some problems which we never saw under > very heavy load using local databases. > > This might be totally unrelated. I hope the above will be helpfull as a > datapoint for this kind of issues... > > Murat > > > -----Original Message----- > From: Jean-Claude Wippler [mailto:jcw@equi4.com] > Sent: Monday, October 07, 2002 5:49 PM > To: Starkit list > Cc: MetaKit mailing list > Subject: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavior > > > john_zouck@jhuapl.edu wrote: > > > I have had a problem with tclkit for linux when running on an SGI > > computer > > with Redhat linux. It seems to run fine, but then when I exit it hangs, > > and leaves processes that can't be killed without rebooting. I just > > downloaded a new tclkit from equi4 and the behavior is the same as the > > older ones: > > > > jhz[\H:(~)]8: ./tclkit-linux-x86 > > % exit > > > > At this point I can't ctrl-c or ctrl-z. > [...] > > Can't kill -9 either. > > Ouch, that is a major bug. You're not the first one to run into it, > nor is it limited to Linux. > > First let's get this straight: a user-mode program which causes the OS > to leave it in an uniterruptable state is an *OS* bug. So the most > *accurate* advice I can give is: report this as a major bug to your OS > vendor, RedHat in this case. > > I cannot repeat this enough: a user-mode program, not using any root > permissions, and doing nothing but publicly documented system calls, > should not be able to cause this. If I had mischief in mind, I could > probably use this flaw as DOS-attack: just launch processes forever, > each hitting this OS bug... > > Unfortunately, this all comes down to the simple fact that MK is > triggering a serious bug on three different OS'es so far (RH, HPUX, > OSX), so I have no option other than try and find a workaround. The > world is too big, wheels turn too slowly, and MK is not likely to make > anyone change things - even when a reproducible hard bug like this > proves them to be causing it. > > Sigh... > > Ok, next step is for *me* to be able to reproduce this. > > 1) What version of RedHat are you using? If I can grab a copy of it > somewhere, I'll set up a machine and repeat your runs. > > 2) What is the size/date of the tclkit executable you are using, and > also: > what does the Tcl command "puts $vfs::tclkit_version" return? > > With a bit of luck, your hang may be one I can reproduce - *then* I can > start digging in. > > -jcw > > PS. FWIW, the problem seems to be related to having a datafile open, > writing to it, even extending it at times, while there is a mmap on it > (which is R/O) and with some of the writes using pointers into that map > as data. With page-size allocations, a write to a page using data from > that same page (never overlapping) may well mightily confuse an > imperfect OS, as it tries to copy-on-write perhaps, page-faulting with > a lock pending on the write while the read is in use. Whatever. I may > even try to create a small example in an attempt to force OS vendors to > fix this after all... > > But first, I need a setup to reproduce this behavior. > > As if there isn't enough work... :) > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > _______________________________________________ > starkit mailing list > starkit@equi4.com > http://www.equi4.com/mailman/listinfo/starkit > ============================ John H. Zouck The Johns Hopkins University Applied Physics Laboratory ============================ From steve@digital-smarties.com Tue Oct 8 09:50:36 2002 Received: from digital-smarties.com (IDENT:qmailr@gw.digital-smarties.com [203.59.125.206]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g98EoZa14248 for ; Tue, 8 Oct 2002 09:50:35 -0500 Received: (qmail 9664 invoked by uid 8); 8 Oct 2002 14:50:58 -0000 Received: from pc-00060 (192.168.1.60, claiming to be "[192.168.1.60]") by gateway.digital-smarties.com with SMTP id smtpdaH9dZj; Tue, 08 Oct 2002 10:50:52 EDT From: "Steve Landers" To: , "Jean-Claude Wippler" Cc: "Starkit list" , "MetaKit mailing list" Date: Tue, 8 Oct 2002 22:50:49 +0800 Message-Id: <20021008145049.12430@mail.digital-smarties.com> In-Reply-To: References: X-Mailer: CTM PowerMail 4.0.1 carbon MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Re: [Starkit] Strange SGI Linux tclkit exit behavior Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John, > Even as 'su' the process could not be >killed. Maybe it has to do with the SGI SMP aspects... That would not suprise me ... I'll be having a look at this tomorrow to see if it occurs on RedHat 7.2 Steve From eric.lequien@laposte.net Thu Oct 10 08:01:43 2002 Received: from mail.libertysurf.net (mail.libertysurf.net [213.36.80.91]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9AD1ga24852 for ; Thu, 10 Oct 2002 08:01:43 -0500 Received: from ELN-DESK.laposte.net (213.36.65.203) by mail.libertysurf.net (6.5.026) id 3DA0C01A00079AAC for metakit@equi4.com; Thu, 10 Oct 2002 15:02:03 +0200 Message-Id: <5.1.0.14.2.20021006213350.04e5d008@mailandnews.com> X-Sender: eric.lequien@pop.laposte.net (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Thu, 10 Oct 2002 15:01:37 +0200 To: metakit@equi4.com From: Eric Lequien Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] MetaKit vs. competitors Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all, Just discovered the Equi4 site during search and because I'm looking for a reliable, fast, easy to maintain (and affordable in term of cost) embedded database librarie for C/C++. Well, the quick summary I've read is very impressive considering the lib/dll/api weight :-) At this point I've just downloaded the useful files for my own case (ie. MetaKit 2.4.7 using C++ under Win32) and browsed the entire site as the links from it (also, I've found a kindly comment in the "Available C++ Libraries FAQ" at , dated 2000). To go further or not, I've to check some points for wish you may have the answer (surely) and which can be summarized like this : "What's about MetaKit vs. others libraries ?". By "others libraries", I mean the others "C/C++ [relational] database libraries to build standalone app for Win32" whatever be the database file format (proprietary or not). For example : - Birdstep RDM Embedded 6.1 - Sequiter Codebase 6.5 - Faircom C-TREE Plus - Softfocus Btree/ISAM - Borland Interbase ... Of course, these libraries (for the majority) are far from the open-source concept and sold at a so expensive price that it's quite discouraging for an independent developer like me, but my question is more on the technical side (size/speed/reliability/maintability performance) and about perenity (will MetaKit continue), forgetting the money aspect. Don't know if some of you will reply to this kind of question, but maybe a comparaison would be nice for the entire community... Have a good day Cordially Eric Lequien eric.lequien@laposte.net From tom.krehbiel@motorola.com Thu Oct 10 10:24:58 2002 Received: from motgate3.mot.com (motgate3.mot.com [144.189.100.103]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9AFOwa06926 for ; Thu, 10 Oct 2002 10:24:58 -0500 Received: [from mothost.mot.com (mothost.mot.com [129.188.137.101]) by motgate3.mot.com (motgate3 2.1) with ESMTP id IAA03512 for ; Thu, 10 Oct 2002 08:22:58 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by mothost.mot.com (MOT-pobox 2.0) with ESMTP id IAA19039 for ; Thu, 10 Oct 2002 08:25:23 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id S47YS2WN; Thu, 10 Oct 2002 08:24:36 -0700 Message-ID: <3DA59BB4.D347A134@motorola.com> Date: Thu, 10 Oct 2002 08:24:36 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 To: Eric Lequien CC: metakit@equi4.com Subject: Re: [Metakit] MetaKit vs. competitors References: <5.1.0.14.2.20021006213350.04e5d008@mailandnews.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Eric, > To go further or not, I've to check some points for wish you may have the > answer (surely) and which can be summarized like this : "What's about > MetaKit vs. others libraries ?". > > By "others libraries", I mean the others "C/C++ [relational] database > libraries to build standalone app for Win32" whatever be the database file > format (proprietary or not). > > For example : > - Birdstep RDM Embedded 6.1 > - Sequiter Codebase 6.5 > - Faircom C-TREE Plus > - Softfocus Btree/ISAM > - Borland Interbase > ... I'm familiar with Oracle, Interbase and MySQL all of which are "real" RDBM system. MetaKit isn't targeted at the RDBM system space it is targeted at the imbedded database space. If your are familiar with RDBM systems then some of the things you won't find are: 1) SQL language support 2) maintenance utilities 3) db shadowing, mirroring, etc. 4) user defined function 5) triggers 6) two phase commit 7) database "views" 8) journaling MetaKit is however powerful enough (with the tcl API) to produce equivalent results if you need to. Tom K. From nicolasb@maich.gr Thu Oct 10 11:23:08 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g9AGN7a14227 for ; Thu, 10 Oct 2002 11:23:07 -0500 Received: FROM maich.gr BY mail-server ; Thu Oct 10 19:53:09 2002 +0300 Message-ID: <3DA5AAC1.2050803@maich.gr> Date: Thu, 10 Oct 2002 19:28:49 +0300 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0rc2) Gecko/20020510 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Eric Lequien CC: metakit@equi4.com Subject: Re: [Metakit] MetaKit vs. competitors References: <5.1.0.14.2.20021006213350.04e5d008@mailandnews.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, There are a couple of other issues that might attract a Metakit user; Data file portability(Unix,win,mac etc..) Multiple Interfaces(AFAIK, c++, tcl, python ...) WRT some of the products you mentioned, I can only comment on Faircom, as it was a consideration for us a while back. IIRC, its 16,000USD cost was out of the question for us.... What we've settled on was metakit through its tcl interface. With this combination we have been able to evaluate, code, and usually solve many tasks/applications. As for performance, our 5-60 MByte data files dont even make Metakit even breathe hard. As for support, and "perenity" as you call it, I've been a metakit user for ~ 4 years, seen metakit evolve consistently, and have been supported by the author and the metakit community for years....I do not see why the authors commiment would suddelnly change. Ultimately, as with open source, you have what you need to go on alone should there be some un-foreseen snag.... I am surprised that you do not mention Sqlite, and Berkely Db in you comparisons as this these type of libs might be more in line with Metakit... Anyway, highly recommended product for the appropriate applications... regards, nicolas boretos From lvirden@cas.org Thu Oct 10 11:31:12 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9AGVCa15348 for ; Thu, 10 Oct 2002 11:31:12 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.5/m8.12.5/CAS_MAIL_HUB-2.00) with ESMTP id g9AGVRok026503 for ; Thu, 10 Oct 2002 12:31:27 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id g9AGVQK10316; Thu, 10 Oct 2002 12:31:26 -0400 (EDT) Date: Thu, 10 Oct 2002 12:31:26 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20021010123126.AAB10305@cas.org> Subject: Re: [Metakit] MetaKit vs. competitors In-Reply-To: Your message of Thu, 10 Oct 2002 19:28:49 +0300 To: metakit@equi4.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Nicolas Boretos > I am surprised that you do not mention Sqlite, and Berkely Db in you > comparisons as this these type of libs might be more in line with Metakit... It's just been a few weeks ago that the authors of Sqlite and Metakit were attending a conference together, swapping ideas, etc... -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jyl@best.com Thu Oct 10 20:35:35 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g9B1ZZa08958 for ; Thu, 10 Oct 2002 20:35:35 -0500 Received: (qmail 87577 invoked by uid 19667); 11 Oct 2002 01:33:24 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 11 Oct 2002 01:33:24 -0000 Message-ID: <3DA62D32.2070606@best.com> Date: Thu, 10 Oct 2002 18:45:22 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Tom Krehbiel CC: Eric Lequien , metakit@equi4.com Subject: Re: [Metakit] MetaKit vs. competitors References: <5.1.0.14.2.20021006213350.04e5d008@mailandnews.com> <3DA59BB4.D347A134@motorola.com> Content-Type: multipart/alternative; boundary="------------070901000009060508050007" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------070901000009060508050007 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit e4Graph (http://e4graph.sourceforge.net) provides some of the missing pieces: * Stored procedues for the Tcl binding (I think that's what you mean with "user defined functions") * Triggers Also, I believe there's a very good SQL emulation layer on top of Metakit. Check out http://www.equi4.com/metakit/mksql.html --JYL Tom Krehbiel wrote: >Eric, > >>To go further or not, I've to check some points for wish you may have the >>answer (surely) and which can be summarized like this : "What's about >>MetaKit vs. others libraries ?". >> >>By "others libraries", I mean the others "C/C++ [relational] database >>libraries to build standalone app for Win32" whatever be the database file >>format (proprietary or not). >> >>For example : >>- Birdstep RDM Embedded 6.1 >>- Sequiter Codebase 6.5 >>- Faircom C-TREE Plus >>- Softfocus Btree/ISAM >>- Borland Interbase >>... >> > >I'm familiar with Oracle, Interbase and MySQL all of which are "real" RDBM system. MetaKit isn't targeted at the RDBM system space >it is targeted at the imbedded database space. If your are familiar with RDBM systems then some of the things you won't find are: > >1) SQL language support >2) maintenance utilities >3) db shadowing, mirroring, etc. >4) user defined function >5) triggers >6) two phase commit >7) database "views" >8) journaling > >MetaKit is however powerful enough (with the tcl API) to produce equivalent results if you need to. > >Tom K. >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > --------------070901000009060508050007 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit e4Graph (http://e4graph.sourceforge.net) provides some of the missing pieces:

* Stored procedues for the Tcl binding (I think that's what you mean with "user defined functions")
* Triggers

Also, I believe there's a very good SQL emulation layer on top of Metakit. Check out http://www.equi4.com/metakit/mksql.html

--JYL

Tom Krehbiel wrote:
Eric,

To go further or not, I've to check some points for wish you may have the
answer (surely) and which can be summarized like this : "What's about
MetaKit vs. others libraries ?".

By "others libraries", I mean the others "C/C++ [relational] database
libraries to build standalone app for Win32" whatever be the database file
format (proprietary or not).

For example :
- Birdstep RDM Embedded 6.1
- Sequiter Codebase 6.5
- Faircom C-TREE Plus
- Softfocus Btree/ISAM
- Borland Interbase
...

I'm familiar with Oracle, Interbase and MySQL all of which are "real" RDBM system. MetaKit isn't targeted at the RDBM system space
it is targeted at the imbedded database space. If your are familiar with RDBM systems then some of the things you won't find are:

1) SQL language support
2) maintenance utilities
3) db shadowing, mirroring, etc.
4) user defined function
5) triggers
6) two phase commit
7) database "views"
8) journaling

MetaKit is however powerful enough (with the tcl API) to produce equivalent results if you need to.

Tom K.
_______________________________________________
metakit mailing list  -  metakit@equi4.com
http://www.equi4.com/mailman/listinfo/metakit


--------------070901000009060508050007-- From lvirden@cas.org Fri Oct 11 04:24:56 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9B9Ota04255 for ; Fri, 11 Oct 2002 04:24:55 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.5/m8.12.5/CAS_MAIL_HUB-2.00) with ESMTP id g9B9PFok008488 for ; Fri, 11 Oct 2002 05:25:15 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id g9B9PF820504; Fri, 11 Oct 2002 05:25:15 -0400 (EDT) Date: Fri, 11 Oct 2002 05:25:15 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20021011052515.AAB20482@cas.org> Subject: Forwarded: Re: [Metakit] MetaKit vs. competitors To: "Metakit" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I suspect that bbaker meant this for the list... Forwarded mail follows: >From bbaker@priefert.com Thu Oct 10 14:22:35 2002 Date: Thu, 10 Oct 2002 13:22:33 -0500 >From: bbaker User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.1) Gecko/20020826 MIME-Version: 1.0 To: "Larry W. Virden" Subject: Re: [Metakit] MetaKit vs. competitors If I were going to make comparisons and recommendations, the only real problem with MetaKit from my point of view is the half-baked index support. Don't get me wrong, I'm grateful for what is there. But to be useful you must be able to define multiple indexes on a file and write updates to the file with the indexes being updated automatically. I use the term "index" liberally here to include btree, hashes, ordered, sorted or whatever. For those familiar with FairCom's CTree or BTrieve, you know what I'm talking about. wlb -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Fri Oct 11 05:37:59 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BAbwa09826 for ; Fri, 11 Oct 2002 05:37:58 -0500 Date: Fri, 11 Oct 2002 12:38:28 +0200 Subject: Re: Forwarded: Re: [Metakit] MetaKit vs. competitors Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Jean-Claude Wippler To: "Metakit" Content-Transfer-Encoding: 7bit In-Reply-To: <20021011052515.AAB20482@cas.org> Message-Id: <9452A870-DD05-11D6-B9B9-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.546) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: > I suspect that bbaker meant this for the list... (Thx for forwarding this) > Forwarded mail follows: >> From bbaker@priefert.com Thu Oct 10 14:22:35 2002 [...] > If I were going to make comparisons and recommendations, the only real > problem with MetaKit from my point of view is the half-baked index > support. Don't get me wrong, I'm grateful for what is there. But to > be > useful you must be able to define multiple indexes on a file and write > updates to the file with the indexes being updated automatically. Correct - it's in there for single indexes, but multi-index updates are not quite ready for prime time. Basic idea is: v1 = mydata v2 = v1.Hash(...) Here. v2 can be seen as a view on mydata which maintains a hash index and is O(1). Next steps need to be, hypothetically: v3 = v2.ReorderProps(...) v4 = v3.KeepSortedIndex(...) Now v4 is a layered "cake" of views, which maintains two distinct index views. There is a first go at this, called c4_View::Indexed, but it is incomplete. The other approach I have been considering, is a flatter structure: v2a = v1.Indexable(...) Then have calls to attach one or more index views to it. > I use the term "index" liberally here to include btree, hashes, > ordered, > sorted or whatever. > > For those familiar with FairCom's CTree or BTrieve, you know > what I'm talking about. In a previous life, I wrote an in-house multi-user DB which had all that. And although indexes will be absolutely essential in some contexts - so far MK only has the hash to really move away from O(N) to O(1) - at the same I'd like to re-iterate that brute-force searching in MK is on a different level, and often surprisingly effective. Measurements would be great to have - all I get are the outliers. As I hope can be shown, MK's O(N) frequently outperforms those who offer O(log N), even O(1), also in 10K/100K-row cases - but evidently there is a break-even point! Until MK gets fancier multi-index support, the only advice I can repeat is: see if it works for your scenario, because there are limits. Measure it, please. -jcw From eric.lequien@laposte.net Fri Oct 11 05:51:25 2002 Received: from mail.libertysurf.net (mail.libertysurf.net [213.36.80.91]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BApOa10739 for ; Fri, 11 Oct 2002 05:51:24 -0500 Received: from ELN-DESK.laposte.net (213.36.66.188) by mail.libertysurf.net (6.5.026) id 3DA568EA0001FED2 for metakit@equi4.com; Fri, 11 Oct 2002 12:51:46 +0200 Message-Id: <5.1.0.14.2.20021011122838.00ab0370@mailandnews.com> X-Sender: eric.lequien@pop.laposte.net X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 11 Oct 2002 12:50:24 +0200 To: metakit@equi4.com From: Eric Lequien Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Re: MetaKit vs. competitors (Eric Lequien) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks to Tom, Nicolas, Larry and Jacob for your comment and advice. To well understand why I may talk about basic subjects, I have to tell you I'm mainly a treatment developer rather than a database one. So, maybe some confusions, at the beginning of my investigation, about RDMS and embedded db but, now, it's understood. In fact in "my" list, the main competitors I did have in mind were Sequiter Codebase and BirdStep RDM Embedded (forgotting cost consideration). Since my main concern is to produce downloadable stuffs, I have to take care of the final package size and embedded seems to be the right way to achieve this task. About Sqllite and Berkeley db, I will take a look, but don't know anything about those at this time... What can you tell me about advantages and drawbacks about MK vs. Sqllite vs. Berkeley db ? About the lack of an embedded way like MetaKit is against a RDMS, it seems it's mainly a matter of environment (I mean IDE) and some "components" around standard and common practices (eg. SQL). Well, e4Graph and SQL emulation layer may surely help a lot (not seen at the time I'm writing this message). So, what's the real difference, considering we can add tools around a database library to gain facility/ability of development as done in a RDBMS context ? However, thanks again. "Cordialement" Eric From bbaker@priefert.com Fri Oct 11 09:14:52 2002 Received: from hardy.priefert.com ([206.160.158.141]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BEEqa26894 for ; Fri, 11 Oct 2002 09:14:52 -0500 Received: from priefert.com (itdepartment.priefert.com [192.168.4.97]) by hardy.priefert.com (8.11.6/8.11.6) with ESMTP id g9B9DSG30639 for ; Fri, 11 Oct 2002 04:13:28 -0500 Message-ID: <3DA6DCF9.8030203@priefert.com> Date: Fri, 11 Oct 2002 09:15:21 -0500 From: bbaker User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: Forwarded: Re: [Metakit] MetaKit vs. competitors References: <9452A870-DD05-11D6-B9B9-00050251EF2F@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Next steps need to be, hypothetically: > v3 = v2.ReorderProps(...) > v4 = v3.KeepSortedIndex(...) > Now v4 is a layered "cake" of views, which maintains two distinct > index views. There is a first go at this, called c4_View::Indexed, > but it is incomplete. > > The other approach I have been considering, is a flatter structure: > v2a = v1.Indexable(...) > Then have calls to attach one or more index views to it. If we could vote, I would vote for the flatter structure. Although I am more than grateful for anything which works, the flatter structure seems easier to manipulate. Too many layers make me think of the leaning tower of Piza. > Measurements would be great to have - all I get are the outliers. As > I hope can be shown, MK's O(N) frequently outperforms those who offer > O(log N), even O(1), also in 10K/100K-row cases - but evidently there > is a break-even point! I tried it. The problem is that my app has to respond the user entry quickly. Every product number entered by the user has to be looked up in four tables. The machine is a PocketPC with 32MB RAM and a StrongArm processor at 204 MHz. The lookups work, but the app looses its responsiveness. For someone who enters 100's of products each day, every day of the week, the difference between "snappy" and "a second" is the difference between "I like this" and "Throw this away." My larger tables typically contain 20K records. I now have two methods of configuration: straight PocketPC databases, and a hybrid with some tables in PocketPC and others in MetaKit. PocketPC has a native database engine with some hard limitations on database size. If the database gets too big, the device simply locks up, and sometimes boots as a cold boot....which wipes out the databases system entirely since it is completely in RAM. Messy. But alas, PocketPC can handle my multiple indexes with ease. I can insert into the database without caring what indexes are defined on a given table...they are updated automatically. bbaker From jcw@equi4.com Fri Oct 11 09:32:00 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BEVxa28500 for ; Fri, 11 Oct 2002 09:31:59 -0500 Date: Fri, 11 Oct 2002 16:32:31 +0200 Subject: Re: Forwarded: Re: [Metakit] MetaKit vs. competitors Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3DA6DCF9.8030203@priefert.com> Message-Id: <466F05C4-DD26-11D6-B9B9-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.546) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: bbaker wrote: [plain search] > I tried it. The problem is that my app has to respond the user > entry quickly. Every product number entered by the user has to > be looked up in four tables. The machine is a PocketPC with > 32MB RAM and a StrongArm processor at 204 MHz. The > lookups work, but the app looses its responsiveness. [...] Is hashing a solution? One hash on each table? Available now. > But alas, PocketPC can handle my multiple indexes with > ease. I can insert into the database without caring what > indexes are defined on a given table...they are updated > automatically. Ok, thanks for the input - looks like multi-indexes needs attention. -jcw From jeff@k2.com Fri Oct 11 09:40:12 2002 Received: from kobalt.k2.com (IDENT:root@dsl092-155-098.sea1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BEeBa29295 for ; Fri, 11 Oct 2002 09:40:11 -0500 Received: from Klee ([10.6.18.41]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id g9BEeXp19078; Fri, 11 Oct 2002 10:40:34 -0400 From: "Jeffrey Kay" To: "'Eric Lequien'" , Subject: RE: [Metakit] Re: MetaKit vs. competitors (Eric Lequien) Date: Fri, 11 Oct 2002 10:40:33 -0400 Message-ID: <001f01c27134$27fbb2c0$2912060a@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal In-Reply-To: <5.1.0.14.2.20021011122838.00ab0370@mailandnews.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Eric -- I haven't contributed much to this list, but I've been using MetaKit now for several months as an embedded DB in a product I'm developing. In general, I like the tool very much, but there are some usage factors that are worth being aware of before selecting MK. (After this list, I'll compare to Ctree.) 1) Multithreading. MK does not support multithreaded access. The application I'm developing has ten or more threads that often need to read or update parts of a single database. To better handle this better, I've been using the convention of opening the database once (since multiple opens also cause problems) and associating a CritSec with it (Win32). My software uses two functions to wrap this -- getStore() and releaseStore(): c4_Storage& getStore() { m_critsec.enter(); return *m_store; } void releaseStore() { m_critsec.leave(); } By using this convention, I ensure that all access to the db is sequential. For some applications this would be a real problem, but for mine it isn't at all. I could further improve performance by opening multiple handles, one with read only access and another with read/write, but it's not that important for my app at the moment. 2) Views for Searching. It took me a little while to get the hang of views, but they are really quite pleasant to use. A view is sort of like a SELECT in SQL, a set of rows that you can cycle through. The view gives you access to all of the fields in each row, so it's not completely like a SELECT. Views can also be nested. Your database in MK is essentially one big view, with a set of subviews, in contrast to more traditional RDBMS tables. For example, let's say you want two tables -- customers, products. Each of these becomes a view in an MK database. From the perspective of the entire DB, "customers" and "products" are actually two fields in the "master" view. In the customer view, you have the typical fields and a subview called "phone_numbers", a list of the various phone numbers that an individual has, each value containing the number and a type. This "phone_numbers" view is nested within the "customers" view. The database description would look something like this: customers[name:S,address:S,phone_numbers[number:S,type:S]],products[id:S ,description:S] This should help give you some idea of the somewhat hierarchical, and object oriented nature of MK. Square brackets represent the contents of a view. If you were doing this in your typical table oriented database, you'd have to break out phone_numbers as a separate table that used a customer id for matching. You'd then do a JOIN to get the values. In MK you find the customer that you want and then dive into the subview to retrieve the phone numbers. There are some capabilities to flatten out subviews using JoinProp that are documented, but I haven't experimented much with them to see how they work. 3) Views for Modification. This subcategory of view issues is another tricky area. Generally when you remove a row from a view, you need to operate on a base view, not a derived view. This means that if you select a subset of a view, using the Select() function, and decide that one of the rows needs to be deleted, you have to ask the view for an index into a base view, to which you apply the remove. That long and complicated sentences looks like this in code: #define CUSTOMER_VIEW customers[name:S,address:S,phone_numbers[number:S,type:S]] c4_StringProp pCustomer("name"); c4_Storage storage("customer.db",true); c4_View v = storage.GetAs(CUSTOMER_VIEW); // gets the base view out of the database c4_View vs = v.Select(pCustomer["bob"]); // selects the rows that contain customer ID "bob" while(vs.GetSize()) { // we're going to remove all of the "bob" records int idx = v.GetIndexOf(vs[0]); // get the index of the first record in the derived view vs within the base view v.RemoveAt(idx); // remove the record from the base view } Fortunately all of the derived views update when a base view is altered, so this works great. Similar issues seem to exist when adding rows also. 4) Scope. Again, another really tricky issue for me. I've found that I often want to read a row, close the views, do some operations and then go back and update the row. Since everything is easier to create on the stack (as in my last example), I find myself throwing in some extra braces to force the stack to clear and shut down the views. MK seems much happier when I do that. Function foo() { { // some MK operations c4_View v = ... } // some other operations { // some more MK operations c4_View v = ... } } This forces the views to close when I'm done and doesn't leave stuff hanging around. 5) File size. The files don't get smaller when the data is deleted, so you have to clean up once in a while to shrink the file sizes. The technique for doing this isn't obvious, so if you get to this point I'll send you the code to do it. Essentially, you need to rewrite the file into a new file and replace the old one (it's easier than this but that's essentially what the operation looks like). Okay, so that's the list off the top of my head. On the minus side, MK isn't conventional when thinking about databases -- if you are looking for tables and rows and columns, MK might not do it for you. On the plus side, it's fast and small. The MK DLL for Windows is 126k, smaller than the DLL for the XML parser I use (expat at 135k). I've no complaints about the speed, although I haven't populated a table with a few million rows yet. I have several tables that have individual properties (fields) that exceed 10 MB in size and MK hasn't broken into a sweat yet. I've used Ctree for my embedded db needs for years. The good news about Ctree is that it too is fast. I'm not that crazy about having my indices stored in multiple files, but that's not too ugly. Ctree has some of the same multithreading problems that MK has. In theory the multithreading is supposed to work, but in reality, it doesn't. The Ctree DLL from the last project I did is about 315k, so it's a little larger. The schema creation in Ctree is clunky compared to MK, but that's because MK is so freeform. Ctree has better searching than MK does, however -- there's no way to find all of the rows where "id > 10 && id < 50", as far as I can tell. I made my decision about using MK by looking around a several different tools. The only one in your list that I didn't come across was SQLite, so I can't speak to that, except to say that the documentation doesn't seem to infer that SQLite is any better about multithreading than MK is. Beyond that I was looking for speed, size and license. Ctree has gotten expensive, otherwise I would probably have used them just out of habit (I know those guys pretty well -- they are a class act and build good software). I've found MK fun to use at times, and exasperating at others. I suspect that the biggest problem with MK is the lack of good example code. The programs that are available that use it are fine to look at, but a tutorial that shows specific small examples of how to select, how to add, how to sort, etc. would be extremely helpful. I figured out a lot of this by trial and error. Hope this helps -- good luck. jeffrey kay personal weblog "first get your facts, then you can distort them at your leisure" -- mark twain "golf is an endless series of tragedies obscured by the occasional miracle" -- sports illustrated "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of Eric Lequien > Sent: Friday, October 11, 2002 6:50 AM > To: metakit@equi4.com > Subject: [Metakit] Re: MetaKit vs. competitors (Eric Lequien) > > > Thanks to Tom, Nicolas, Larry and Jacob for your comment and > advice. To > well understand why I may talk about basic subjects, I have > to tell you I'm > mainly a treatment developer rather than a database one. > > So, maybe some confusions, at the beginning of my > investigation, about RDMS > and embedded db but, now, it's understood. In fact in "my" > list, the main > competitors I did have in mind were Sequiter Codebase and > BirdStep RDM > Embedded (forgotting cost consideration). Since my main concern is to > produce downloadable stuffs, I have to take care of the final > package size > and embedded seems to be the right way to achieve this task. > > About Sqllite and Berkeley db, I will take a look, but don't > know anything > about those at this time... What can you tell me about advantages and > drawbacks about MK vs. Sqllite vs. Berkeley db ? > > About the lack of an embedded way like MetaKit is against a > RDMS, it seems > it's mainly a matter of environment (I mean IDE) and some > "components" > around standard and common practices (eg. SQL). Well, e4Graph and SQL > emulation layer may surely help a lot (not seen at the time > I'm writing > this message). So, what's the real difference, considering we > can add tools > around a database library to gain facility/ability of > development as done > in a RDBMS context ? > > However, thanks again. > "Cordialement" > Eric > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From bbaker@priefert.com Fri Oct 11 10:50:01 2002 Received: from hardy.priefert.com ([206.160.158.141]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BFo0a02794 for ; Fri, 11 Oct 2002 10:50:01 -0500 Received: from priefert.com (itdepartment.priefert.com [192.168.4.97]) by hardy.priefert.com (8.11.6/8.11.6) with ESMTP id g9BAmaG30814 for ; Fri, 11 Oct 2002 05:48:36 -0500 Message-ID: <3DA6F345.7000209@priefert.com> Date: Fri, 11 Oct 2002 10:50:29 -0500 From: bbaker User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit Subject: Re: Forwarded: Re: [Metakit] MetaKit vs. competitors References: <466F05C4-DD26-11D6-B9B9-00050251EF2F@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hashing is a solution on some tables. Other tables have indexes which are not unique, ie, find all products of brand FOFO ordered by item. I would agree with a previous poster that one of the best things that could happen to metakit would be more examples and more docs. I myself could get busy with the WIKI. bbaker AKA wlb > [plain search] > >> I tried it. The problem is that my app has to respond the user >> entry quickly. Every product number entered by the user has to >> be looked up in four tables. The machine is a PocketPC with >> 32MB RAM and a StrongArm processor at 204 MHz. The >> lookups work, but the app looses its responsiveness. [...] > > > Is hashing a solution? One hash on each table? Available now. > >> But alas, PocketPC can handle my multiple indexes with >> ease. I can insert into the database without caring what >> indexes are defined on a given table...they are updated >> automatically. > > > Ok, thanks for the input - looks like multi-indexes needs attention. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From tom.krehbiel@motorola.com Fri Oct 11 11:18:23 2002 Received: from motgate4.mot.com (motgate4.mot.com [144.189.100.102]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BGIMa05063 for ; Fri, 11 Oct 2002 11:18:23 -0500 Received: [from mothost.mot.com (mothost.mot.com [129.188.137.101]) by motgate4.mot.com (motgate4 2.1) with ESMTP id JAA28080 for ; Fri, 11 Oct 2002 09:18:46 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by mothost.mot.com (MOT-pobox 2.0) with ESMTP id JAA11004 for ; Fri, 11 Oct 2002 09:18:45 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id S47YSLGD; Fri, 11 Oct 2002 09:18:45 -0700 Message-ID: <3DA6F9E5.1F0F7AE8@motorola.com> Date: Fri, 11 Oct 2002 09:18:45 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 To: Jeffrey Kay CC: metakit@equi4.com Subject: Re: [Metakit] Re: MetaKit vs. competitors (Eric Lequien) References: <001f01c27134$27fbb2c0$2912060a@k2.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeffrey, Have you looked at kitView http://www.digital-smarties.com/kitview.html It uses Tequila (http://www.equi4.com/tequila/) to implement distributed locking on databases/tables/rows/cell. The implementation is really clever and is apparently being used on a major application. Tom K. From jtaylor@shift-right.com Fri Oct 11 11:26:35 2002 Received: from mactufus.net2atlanta.com (ds2.net2atlanta.com [207.51.14.3]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BGQYa05792 for ; Fri, 11 Oct 2002 11:26:35 -0500 Received: from lefty (208.147.135.177 [208.147.135.177]) by mactufus.net2atlanta.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2656.59) id 4V7JMM65; Fri, 11 Oct 2002 12:27:58 -0400 From: "John Taylor" To: Date: Fri, 11 Oct 2002 12:26:57 -0400 Message-ID: <000d01c27143$077a2c70$6401a8c0@lefty> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Concurrency Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have several questions related to providing a C++ Multithreading wrapper to Metakit. The Wiki documentation says the following for version 1.8.5 (http://www.equi4.com/metakit/wiki.cgi/13): "... MetaKit cam be used in multi-threaded applications and in DLLs, as long as no datafile or view is used in more than one thread. This the case right now for Windows, it would be very easy to add this for use in Unix (one mutex)." And "... Wouldn't locking solve multi-user access? No. As a matter of fact, locking was added to the Tcl interface (Mk4tcl) recently, to enforce the above rules. That's the easy part. The main issue which must be solved is cache coherence. Right now, there's some information which gets loaded on file open, and then cached. This caching increases as the file is being modified. To solve the problem, either some form of IPC, or some sort of change propagation is needed." My first question is: are the above statements still true for the newer versions of Metakit v2.4.x? Next, are the following statements true? 1. Give the above statements, then simply providing mutex protection (and/or critical sections) around my code that access my metakit databases (both reads and writes actions) is not sufficient because of "cache coherence" issues. 2. My only option for a multithread wrapper is do "...introduce a client/server mechanism" to isoloate all operations on a given metakit database to the thread that the database was originally opened/created in. Next question. From a previous newsgroup posting, the comment has been made "... The only global data structure is a property symbol table, which has been properly mutex-protected (on Win32, Unix should not be hard to do as well.." So for my non-win32 builds, for complete concurrency protection I must also provide mutex protection of the property symbol table. Is this correct? Are there any other platform specific "gotchas" with respect to concurrency? Last question. In the source code (header.h) there is a preprocessor option "q4_MULTI". What does this option do currently (v2.4.x)? What is the planned functionality in the future? John T. Taylor Shift-Right Technologies, LLC. 433 Comanche Trl Lawrenceville, GA 30044 Tel: 678-344-3115 Fax: 770-736-9276 email: jtaylor@shift-right.com url: www.shift-right.com From stefan.kowski@gmx.de Fri Oct 11 11:24:40 2002 Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g9BGOea05586 for ; Fri, 11 Oct 2002 11:24:40 -0500 Received: (qmail 20918 invoked by uid 0); 11 Oct 2002 16:25:01 -0000 Received: from p508fd0e2.dip.t-dialin.net (HELO hegel) (80.143.208.226) by mail.gmx.net (mp009-rz3) with SMTP; 11 Oct 2002 16:25:01 -0000 From: "Stefan Kowski" To: Date: Fri, 11 Oct 2002 18:27:38 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Subject: [Metakit] mk-2.7.0/win: growing file size Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am using MetaKit for quite a while and have implemented a server storing data for multiple clients. The database consists of about 15 views with up to 50 rows each, one view has 500 rows (i.e. a very small database). The update frequency is low, mostly single records are being modified. What I see now is that the database file size grows larger than I would expect for this small amount of records. For the database mentiond above, it is 366 K. Does MetaKit increase the file size when no Inserts are being performed, only Updates, e.g. due to Commits? Is there a possibility to shrink the file to a minimum size required for the data? Has anyone seen a similar behaviour? Thank you for your answers, Stefan From tom.krehbiel@motorola.com Fri Oct 11 11:45:47 2002 Received: from motgate3.mot.com (motgate3.mot.com [144.189.100.103]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BGjla07342 for ; Fri, 11 Oct 2002 11:45:47 -0500 Received: [from mothost.mot.com (mothost.mot.com [129.188.137.101]) by motgate3.mot.com (motgate3 2.1) with ESMTP id JAA04707 for ; Fri, 11 Oct 2002 09:43:42 -0700 (MST)] Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by mothost.mot.com (MOT-pobox 2.0) with ESMTP id JAA20648 for ; Fri, 11 Oct 2002 09:46:09 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2654.52) id S47YSLK2; Fri, 11 Oct 2002 09:45:23 -0700 Message-ID: <3DA70023.376CCF@motorola.com> Date: Fri, 11 Oct 2002 09:45:23 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 To: Jacob Levy CC: metakit@equi4.com Subject: Re: [Metakit] MetaKit vs. competitors References: <5.1.0.14.2.20021006213350.04e5d008@mailandnews.com> <3DA59BB4.D347A134@motorola.com> <3DA62D32.2070606@best.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob, > e4Graph (http://e4graph.sourceforge.net) provides some of the missing > pieces: Tequila (http://www.equi4.com/tequila/) is a pure tcl extension that has been used to implement persistence in kitView (http://www.digital-smarties.com/kitview.html). > * Stored procedues for the Tcl binding (I think that's what you mean with > "user defined functions") > * Triggers > > Also, I believe there's a very good SQL emulation layer on top of Metakit. > Check out http://www.equi4.com/metakit/mksql.html Thanks for the reference I had missed this page. If you are using tcl with MetaKit then you need to be aware that the current documentation is serious incomplete. Check the bottom of the mk/tcl/mk4tcl.h file for all the object commands that are available (but undocumented at this time). Tom K. From jcw@equi4.com Fri Oct 11 11:54:28 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9BGsRa08045 for ; Fri, 11 Oct 2002 11:54:27 -0500 Date: Fri, 11 Oct 2002 18:55:00 +0200 Subject: Re: [Metakit] Concurrency Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <000d01c27143$077a2c70$6401a8c0@lefty> Message-Id: <2E11E7EF-DD3A-11D6-B9B9-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.546) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John Taylor wrote: > (http://www.equi4.com/metakit/wiki.cgi/13): > > "... MetaKit cam be used in multi-threaded applications and in DLLs, as > long as no datafile or view is used in more than one thread. This the > case right now for Windows, it would be very easy to add this for use > in > Unix (one mutex)." > > And > "... Wouldn't locking solve multi-user access? > > No. As a matter of fact, locking was added to the Tcl interface > (Mk4tcl) > recently, to enforce the above rules. That's the easy part. The main > issue which must be solved is cache coherence. Right now, there's some > information which gets loaded on file open, and then cached. This > caching increases as the file is being modified. To solve the problem, > either some form of IPC, or some sort of change propagation is needed." > > My first question is: are the above statements still true for the newer > versions of Metakit v2.4.x? Yes. > Next, are the following statements true? > 1. Give the above statements, then simply providing mutex protection > (and/or critical sections) around my code that access my metakit > databases (both reads and writes actions) is not sufficient because of > "cache coherence" issues. Multi-thread != multi-user (multi-process/multi-machine). With threading, all threads see the same consistent address space (assuming you open MK's datafile once). Cache coherence just means that as soon as MK datafiles are opened more than once, then some cached data between won't be synced. That is only the case with multiple processes, not with multiple threads inside a single process. > 2. My only option for a multithread wrapper is do "...introduce a > client/server mechanism" to isoloate all operations on a given metakit > database to the thread that the database was originally opened/created > in. Yes. Or wrap all accesses (read and write) to pass through a single mutex. > Next question. From a previous newsgroup posting, the comment has been > made "... The only global data structure is a property symbol table, > which has been properly mutex-protected (on Win32, Unix > should not be hard to do as well.." > So for my non-win32 builds, for complete concurrency protection I must > also provide mutex protection of the property symbol table. Is this > correct? Are there any other platform specific "gotchas" with respect > to concurrency? To summarize - there are two contexts: 1) Global data, which on Win32 is properly protected when q4_MULTI is enabled, and which should be extended for Unix - this is a matter of replicating the Win32 logic (all in src/view.cpp) with pthreads calls. 2) Storages, views, rows - these are not thread safe (but also have no global data usage at all. You either need to open and use a storage in one thread at all times, or implement a scheme over MK itself which lets you make sure you never make *any* MK calls based on the same storage or view or row from more than one thread. The "any MK calls" is the trouble-maker. There may be a way to support N readers, but right now that is not safe because readers affect some cached data. The current state is that you must treat all calls to storage/view/row objects as non-re-entrant. You may however have each thread do things on its own storage/view/row - concurrently. I.e. you can scan/modify things in one datafile in one thread, and do the same for another datafiel in anotehr thread. The new commit-extend mode was implemented to find a way out of this. You can open the same datafile N times R/, each from a thread, and 1 more time R/W even (in extend mode). The readers will not see the changes. The writer merely writes at the end of the file. At some point you will need to sync and comnit in non-extend mode. And you will need to compact the file (commit extend can grow a datafile very quickly). The commit-extend mode also works multi-process and multi-machine, i.e. you can have N readers anywhere, and a single writer. The trickiness is syncing them up again. It's not perfect, but it works today. On Mac OSX, this is done all the time in its central AddressBook database. It is not as clumsy as it may seem - this design allows total lock-free parallelism, which is in fact a performance advantage. > Last question. In the source code (header.h) there is a preprocessor > option "q4_MULTI". What does this option do currently (v2.4.x)? What > is the planned functionality in the future? It enables Win32 mutexes for the global data, i.e. property symbol table. It should do the same once Unix (pthreads) locking is added. -jcw From robertoschler@hotmail.com Fri Oct 18 12:27:27 2002 Received: from hotmail.com (oe56.law9.hotmail.com [64.4.8.191]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9IHRRa22883 for ; Fri, 18 Oct 2002 12:27:27 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Fri, 18 Oct 2002 10:27:54 -0700 X-Originating-IP: [24.52.33.1] From: "Robert Oschler" To: Date: Fri, 18 Oct 2002 13:29:11 -0400 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_01D6_01C276AA.583E9710" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Message-ID: X-OriginalArrivalTime: 18 Oct 2002 17:27:54.0843 (UTC) FILETIME=[B183AAB0:01C276CB] Subject: [Metakit] Where is mk4py.dll? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_01D6_01C276AA.583E9710 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable The download link to get the latest mk4py: = http://www.equi4.com/pub/download.html is broken. This link can be = found on the page: http://www.equi4.com/metakit/python.html . Where can = I find it? thx ------=_NextPart_000_01D6_01C276AA.583E9710 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
The download link to get the latest = mk4py: http://www.equi4.com/pub/download.html  is = broken. =20 This link can be found on the page: http://www.equi4.com/me= takit/python.html . =20 Where can I find it?
 
thx
 
------=_NextPart_000_01D6_01C276AA.583E9710-- From mbloore@yahoo.com Fri Oct 18 18:00:53 2002 Received: from web14310.mail.yahoo.com (web14310.mail.yahoo.com [216.136.224.60]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g9IN0qa11614 for ; Fri, 18 Oct 2002 18:00:53 -0500 Message-ID: <20021018230126.15969.qmail@web14310.mail.yahoo.com> Received: from [65.95.159.152] by web14310.mail.yahoo.com via HTTP; Fri, 18 Oct 2002 16:01:26 PDT Date: Fri, 18 Oct 2002 16:01:26 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] Where is mk4py.dll? To: metakit In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Robert Oschler wrote: > The download link to get the latest mk4py: > http://www.equi4.com/pub/download.html is broken. This link can be > found on the page: http://www.equi4.com/metakit/python.html . Where > can I find it? try http://www.equi4.com/pub/mk/ ===== -- mARK bLOORE __________________________________________________ Do you Yahoo!? Faith Hill - Exclusive Performances, Videos & More http://faith.yahoo.com From bowmanjm@attbi.com Sun Oct 20 22:52:33 2002 Received: from sccrmhc03.attbi.com (sccrmhc03.attbi.com [204.127.202.63]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9L3qXa22001 for ; Sun, 20 Oct 2002 22:52:33 -0500 Received: from there ([65.96.178.255]) by sccrmhc03.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with SMTP id <20021021035303.HRKS24979.sccrmhc03.attbi.com@there> for ; Mon, 21 Oct 2002 03:53:03 +0000 Content-Type: text/plain; charset="iso-8859-1" From: Jeremy Bowman To: metakit@equi4.com Date: Sun, 20 Oct 2002 23:50:40 -0400 X-Mailer: KMail [version 1.3.2] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <20021021035303.HRKS24979.sccrmhc03.attbi.com@there> Subject: [Metakit] MetaKit on the Sharp Zaurus Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I just released the first version of a personal database application for the Sharp Zaurus and other Qtopia-based Linux PDAs. I'm mentioning this here because it uses MetaKit for the underlying database work (storage, sorting, etc.) After the CVS bugfix from 2002/10/09 it seems to be working fine. I separated the MetaKit library into its own package so it can be installed separately from PortaBase for use by other applications (like the financial tracking application I plan to write next...) So if anybody is interested in PortaBase itself, using MetaKit on the Zaurus for their own application, or just seeing another example of code using MetaKit, feel free to take a look at: Homepage: http://portabase.sourceforge.net Project page: http://sourceforge.net/projects/portabase Jeremy Bowman P.S. - It should be possible to enable support for Mk4py on the Zaurus...I haven't needed this yet, but I'll probably get around to trying it in a few weeks or months if nobody else does it first. From ac@AcmeDrivers.com Thu Oct 24 07:44:24 2002 Received: from mail5.atl.registeredsite.com (nobody@mail5.atl.registeredsite.com [64.224.219.79]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9OCiOa25864 for ; Thu, 24 Oct 2002 07:44:24 -0500 Received: from mail.acmedrivers.com (mail.acmedrivers.com [216.247.140.7]) by mail5.atl.registeredsite.com (8.12.2/8.12.5) with ESMTP id g9OCj1TG014825 for ; Thu, 24 Oct 2002 08:45:01 -0400 Received: from alex [216.247.140.7] by mail.acmedrivers.com with ESMTP (SMTPD32-6.06) id AB4C41FC00E0; Thu, 24 Oct 2002 08:45:00 -0400 From: "Anthony Corriveau" To: Date: Thu, 24 Oct 2002 08:45:54 -0400 Message-ID: <000501c27b5b$4b8430b0$1b00000a@alex> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2616 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] 3 byte mem leak in c4_Storage? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It's not really a big deal, but I think there is a tiny 3 byte memory leak in c4_Storage. I was unable to track it down. I'm using Visual C++ 7.0. This program: #include int main(int argc, char *argv[]) { _CrtSetDbgFlag(_CrtSetDbgFlag(_CRTDBG_REPORT_FLAG) | _CRTDBG_LEAK_CHECK_DF); c4_Storage * pStore= new c4_Storage("test",true); delete pStore; } outputs: 'Pogo.exe': Loaded 'E:\code\ac\Pogo\Debug\Pogo.exe', Symbols loaded. 'Pogo.exe': Loaded 'C:\WINDOWS\system32\ntdll.dll', No symbols loaded. 'Pogo.exe': Loaded 'C:\WINDOWS\system32\kernel32.dll', No symbols loaded. 'Pogo.exe': Loaded 'C:\WINDOWS\system32\msvcp70d.dll', Symbols loaded. 'Pogo.exe': Loaded 'C:\WINDOWS\system32\msvcr70d.dll', Symbols loaded. Detected memory leaks! Dumping objects -> e:\sd\vstudio7\vc7\include\crtdbg.h(674) : {158} normal block at 0x00325BE0, 3 bytes long. Data: < > F9 00 00 Object dump complete. The program '[1272] Pogo.exe: Native' has exited with code 0 (0x0). Anthony Corriveau ac@AcmeDrivers.com From baspey@yahoo.com Thu Oct 24 23:56:58 2002 Received: from web11507.mail.yahoo.com (web11507.mail.yahoo.com [216.136.172.39]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g9P4uva30467 for ; Thu, 24 Oct 2002 23:56:57 -0500 Message-ID: <20021025045736.63762.qmail@web11507.mail.yahoo.com> Received: from [213.36.79.108] by web11507.mail.yahoo.com via HTTP; Thu, 24 Oct 2002 21:57:36 PDT Date: Thu, 24 Oct 2002 21:57:36 -0700 (PDT) From: Pascal Baspeyras Subject: [Metakit] 3 byte mem leak in c4_Storage? To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Anthony Corriveau wrote: > It's not really a big deal, but I think there is a > tiny 3 byte memory leak in c4_Storage. This was already discussed on November 2001. JCW's answer was: > The "leak" you report is, ehm... intentional. It is > caused by lines 173 and 178 in src/string.cpp - > where a static copy of an empty string is kept for > efficiency (sharing it and bumping refcounts instead > of allocating a new one). > > The bad news is that this will not be fixed. The > good news, is that the 3-byte leak does not ever > increase or repeat: no matter how much you use and > allocate things in MK, there will be at most *one* > such 3-byte object leak in the whole application. > > -jcw hth. __________________________________________________ Do you Yahoo!? Y! Web Hosting - Let the expert host your web site http://webhosting.yahoo.com/ From jyl@best.com Fri Oct 25 00:14:28 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g9P5ESa31278 for ; Fri, 25 Oct 2002 00:14:28 -0500 Received: (qmail 91408 invoked by uid 19667); 25 Oct 2002 05:11:49 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 25 Oct 2002 05:11:49 -0000 Message-ID: <3DB8D5D2.6060402@best.com> Date: Thu, 24 Oct 2002 22:25:38 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Pascal Baspeyras CC: metakit@equi4.com Subject: Re: [Metakit] 3 byte mem leak in c4_Storage? References: <20021025045736.63762.qmail@web11507.mail.yahoo.com> Content-Type: multipart/alternative; boundary="------------000407050500070203010508" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------000407050500070203010508 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit One interesting aspect of this is that (having just had the chance to debug/purify an application that uses Metakit), purify on Solaris does not report this leak. In fact I'm happy to report that Metakit has absolutely no leaks at least as far as reported by purify. YMMV, etc. My system: Solaris 8, purify (don't have the version handy), MK 2.4.7-37 built with gcc 2.95. --JYL Pascal Baspeyras wrote: >Anthony Corriveau wrote: > >>It's not really a big deal, but I think there is a >>tiny 3 byte memory leak in c4_Storage. >> > > >This was already discussed on November 2001. > > >JCW's answer was: > >>The "leak" you report is, ehm... intentional. It is >>caused by lines 173 and 178 in src/string.cpp - >>where a static copy of an empty string is kept for >>efficiency (sharing it and bumping refcounts instead >>of allocating a new one). >> >>The bad news is that this will not be fixed. The >>good news, is that the 3-byte leak does not ever >>increase or repeat: no matter how much you use and >>allocate things in MK, there will be at most *one* >>such 3-byte object leak in the whole application. >> >>-jcw >> > > > >hth. > > >__________________________________________________ >Do you Yahoo!? >Y! Web Hosting - Let the expert host your web site >http://webhosting.yahoo.com/ >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > --------------000407050500070203010508 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit One interesting aspect of this is that (having just had the chance to debug/purify an application that uses Metakit), purify on Solaris does not report this leak. In fact I'm happy to report that Metakit has absolutely no leaks at least as far as reported by purify. YMMV, etc.

My system: Solaris 8, purify (don't have the version handy), MK 2.4.7-37 built with gcc 2.95.

--JYL

Pascal Baspeyras wrote:
Anthony Corriveau wrote:
It's not really a big deal, but I think there is a
tiny 3 byte memory leak in c4_Storage.


This was already discussed on November 2001.


JCW's answer was:
The "leak" you report is, ehm... intentional.  It is
caused by lines 173 and 178 in src/string.cpp -
where a static copy of an empty string is kept for
efficiency (sharing it and bumping refcounts instead
of allocating a new one).

The bad news is that this will not be fixed.  The
good news, is that the 3-byte leak does not ever
increase or repeat: no matter how much you use and
allocate things in MK, there will be at most *one*
such 3-byte object leak in the whole application.

-jcw



hth.


__________________________________________________
Do you Yahoo!?
Y! Web Hosting - Let the expert host your web site
http://webhosting.yahoo.com/
_______________________________________________
metakit mailing list  -  metakit@equi4.com
http://www.equi4.com/mailman/listinfo/metakit


--------------000407050500070203010508-- From baspey@yahoo.com Fri Oct 25 03:46:03 2002 Received: from web11506.mail.yahoo.com (web11506.mail.yahoo.com [216.136.172.38]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g9P8k3a12416 for ; Fri, 25 Oct 2002 03:46:03 -0500 Message-ID: <20021025084641.77213.qmail@web11506.mail.yahoo.com> Received: from [212.83.141.37] by web11506.mail.yahoo.com via HTTP; Fri, 25 Oct 2002 01:46:41 PDT Date: Fri, 25 Oct 2002 01:46:41 -0700 (PDT) From: Pascal Baspeyras Subject: [Metakit] 3 byte mem leak in c4_Storage? To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > I'm happy to report that Metakit has absolutely no > leaks at least as far as reported by purify. As I understand it, you're reporting that purify is bad at detecting memory leaks ;-) Under win32, those 3 bytes are the sole leak detected by VC6, and I learned to forget about them (not such a big deal, really). __________________________________________________ Do you Yahoo!? Y! Web Hosting - Let the expert host your web site http://webhosting.yahoo.com/ From j.p.fletcher@aston.ac.uk Fri Oct 25 04:28:23 2002 Received: from mercury.aston.ac.uk (mercury.aston.ac.uk [134.151.79.53]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9P9SNa15102 for ; Fri, 25 Oct 2002 04:28:23 -0500 Received: from [134.151.79.13] (helo=email.aston.ac.uk) by mercury.aston.ac.uk with esmtp (Exim 3.30 #1) id 1850kv-0006YY-00; Fri, 25 Oct 2002 10:27:49 +0100 Received: from ceac034 ([134.151.70.134]) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 1850ls-0003ro-00; Fri, 25 Oct 2002 10:28:48 +0100 From: "John Fletcher" To: "Bob Hicks" Date: Fri, 25 Oct 2002 10:27:12 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] Ruby bindings CC: Priority: normal In-reply-to: <000901c1d817$41dacbb0$6401a8c0@homepcxuhigci7> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Bob Hicks" To: Subject: [Metakit] Ruby bindings Date sent: Sat, 30 Mar 2002 13:18:13 -0500 > Would anyone be in the process of creating Ruby bindings for Metakit? Is > anyone interested in such a beast? > > Bob > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > Is anything happening to link Ruby and Metakit? John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From bobhicks@adelphia.net Fri Oct 25 09:08:32 2002 Received: from smtprelay8.dc2.adelphia.net (smtprelay8.dc2.adelphia.net [64.8.50.40]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9PE8Wa30546 for ; Fri, 25 Oct 2002 09:08:32 -0500 Received: from home5j8ddwdscw ([24.53.138.107]) by smtprelay8.dc2.adelphia.net (Netscape Messaging Server 4.15) with ESMTP id H4JJB402.TAA; Fri, 25 Oct 2002 10:09:04 -0400 From: "Bob H" To: "'John Fletcher'" Cc: Subject: RE: [Metakit] Ruby bindings Date: Fri, 25 Oct 2002 10:09:04 -0400 Message-ID: <000401c27c30$16aa9e30$6b8a3518@home5j8ddwdscw> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 In-reply-to: Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I never received a response. Bob -----Original Message----- From: John Fletcher [mailto:J.P.Fletcher@aston.ac.uk] Sent: Friday, October 25, 2002 5:27 AM To: Bob Hicks Cc: metakit@equi4.com Subject: Re: [Metakit] Ruby bindings From: "Bob Hicks" To: Subject: [Metakit] Ruby bindings Date sent: Sat, 30 Mar 2002 13:18:13 -0500 > Would anyone be in the process of creating Ruby bindings for Metakit? > Is anyone interested in such a beast? > > Bob > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > Is anything happening to link Ruby and Metakit? John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From ac@AcmeDrivers.com Fri Oct 25 10:06:15 2002 Received: from mail8.atl.registeredsite.com (nobody@mail8.atl.registeredsite.com [64.224.219.82]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9PF6Fa01923 for ; Fri, 25 Oct 2002 10:06:15 -0500 Received: from mail.acmedrivers.com (mail.acmedrivers.com [216.247.140.7]) by mail8.atl.registeredsite.com (8.12.2/8.12.5) with ESMTP id g9PF6rrm010421 for ; Fri, 25 Oct 2002 11:06:54 -0400 Received: from alex [216.247.140.7] by mail.acmedrivers.com with ESMTP (SMTPD32-6.06) id AE0A471200EA; Fri, 25 Oct 2002 11:06:50 -0400 From: "Anthony Corriveau" To: Subject: RE: [Metakit] 3 byte mem leak in c4_Storage? Date: Fri, 25 Oct 2002 11:06:50 -0400 Message-ID: <000001c27c38$263b52b0$1b00000a@alex> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2616 Importance: Normal In-Reply-To: <20021025045736.63762.qmail@web11507.mail.yahoo.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I agree that this is not a problem. At most, it's a minor annoyance for people using VC memory leak detection. But for the pure academic value, for C++ novices like myself, I have a question. Could the nullVec be defined as: d4_reentrant static unsigned char nullVec[3]; and then simply remove the nullVec init at line 180: if (!nullVec) { // obtain a valid new empty string buffer to keep around static unsigned char* nv = new unsigned char [3]; nv[0] = nv[1] = nv[2] = 0; // only set static value after item is fully inited (avoid MT race) nullVec = nv; } Or is there a problem with that? Wouldn't nullVec be initialized to {0,0,0} even before global constructors are run? Anthony Corriveau ac@AcmeDrivers.com > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of Pascal Baspeyras > Sent: Friday, October 25, 2002 12:58 AM > To: metakit@equi4.com > Subject: [Metakit] 3 byte mem leak in c4_Storage? > > > Anthony Corriveau wrote: > > It's not really a big deal, but I think there is a > > tiny 3 byte memory leak in c4_Storage. > > > This was already discussed on November 2001. > > > JCW's answer was: > > The "leak" you report is, ehm... intentional. It is > > caused by lines 173 and 178 in src/string.cpp - > > where a static copy of an empty string is kept for > > efficiency (sharing it and bumping refcounts instead > > of allocating a new one). > > > > The bad news is that this will not be fixed. The > > good news, is that the 3-byte leak does not ever > > increase or repeat: no matter how much you use and > > allocate things in MK, there will be at most *one* > > such 3-byte object leak in the whole application. > > > > -jcw > > > > hth. > > > __________________________________________________ > Do you Yahoo!? > Y! Web Hosting - Let the expert host your web site http://webhosting.yahoo.com/ _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From rking35@ford.com Mon Oct 28 12:18:32 2002 Received: from dymwsm18.mailwatch.com (dymwsm18.mailwatch.com [204.253.83.220]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9SIIWa26238 for ; Mon, 28 Oct 2002 12:18:32 -0600 Received: from mwsc0220.mw4.mailwatch.com (mwsc0220.mw4.mailwatch.com [204.253.83.142]) by dymwsm18.mailwatch.com (8.11.0/8.11.0) with ESMTP id g9SIJDp11532 for ; Mon, 28 Oct 2002 13:19:13 -0500 Received: from mail pickup service by mwsc0220.mw4.mailwatch.com with Microsoft SMTPSVC; Mon, 28 Oct 2002 13:19:13 -0500 Received: from 204.253.83.39 ([204.253.83.39]) by MWSC0220 with SMTP id 000201141c1f528f-e9b3-4424-8d59-1fa87cdc2d7f; Mon, 28 Oct 2002 13:19:00 -0500 Received: from fmpo1.azell.com (fmpo1.azell.com [136.1.7.9]) by dymwsm15.mailwatch.com (8.12.6/8.12.6) with ESMTP id g9SIIxXE005484 for ; Mon, 28 Oct 2002 13:18:59 -0500 Received: from na1ecs04.dearborn.ford.com ([19.5.112.74]) by fmpo1.azell.com (Mirapoint Messaging Server MOS 3.2.0.52-EA) with ESMTP id AGQ87884; Mon, 28 Oct 2002 13:18:58 -0500 (EST) Received: by na1ecs04.dearborn.ford.com with Internet Mail Service (5.5.2655.15) id ; Mon, 28 Oct 2002 13:18:58 -0500 Message-ID: From: "King, Rick (R.B.)" To: "'metakit@equi4.com'" Date: Mon, 28 Oct 2002 13:18:56 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.15) Content-Type: text/plain; charset="iso-8859-1" HOP-COUNT: 1 X-MAILWATCH-INSTANCEID: 010201141c1f528f-e9b3-4424-8d59-1fa87cdc2d7f X-OriginalArrivalTime: 28 Oct 2002 18:19:13.0264 (UTC) FILETIME=[8486EF00:01C27EAE] Subject: [Metakit] Questions about deriving a new view using select Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am using Metakit with Python. I want to do something like this: "give me all records from this view where English != ' ' " apparently view.select() only allows me to select specific records, such as: "give me all records from this view where English = ' ' " so I did this and then used view.minus(): MyView = View.minus( View.select(English=' ')) (or something very close to this.) The problem is the resulting view is not ordered in the same way that the original view is. Am I missing something about select? Is there any way to do this and maintain ordering? Thanks! -Rick King Southfield MI USA From steve@improvision.com Mon Oct 28 12:55:02 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9SIt1a28990 for ; Mon, 28 Oct 2002 12:55:01 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA27741 for ; Mon, 28 Oct 2002 18:51:22 GMT Received: from improvision.com ([192.168.1.80]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971 for ; Mon, 28 Oct 2002 18:55:42 -0000 Date: Mon, 28 Oct 2002 18:55:41 +0000 Mime-Version: 1.0 (Apple Message framework v548) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Steve Baxter To: metakit@equi4.com Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.548) Subject: [Metakit] assert in Metakit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, We are occasionally getting an assertion in Metakit. This seems to happen when we are adding a lot of data to our database. Here is the assertion: Assertion (!UsesMap(ptr)) failed in file 'column.cpp' at line 287. And the call stack: _assert(void * 0x0137092c `string', void * 0x013707a4 `string', unsigned int 287) line 256 c4_Column::CopyNow(long 0) line 287 + 48 bytes c4_ColIter::BufSave() line 82 c4_Column::InsertData(long 0, long 999, unsigned char 1) line 817 + 19 bytes c4_ColOfInts::Set(int 0, const c4_Bytes & {...}) line 1285 c4_ColOfInts::SetInt(int 0, long 21) line 1329 + 46 bytes c4_FormatB::Commit(c4_SaveContext & {...}) line 828 c4_SaveContext::CommitSequence(c4_HandlerSeq & {...}, unsigned char 0) line 850 + 33 bytes c4_FormatV::Commit(c4_SaveContext & {...}) line 1251 c4_SaveContext::CommitSequence(c4_HandlerSeq & {...}, unsigned char 0) line 850 + 33 bytes c4_FormatV::Commit(c4_SaveContext & {...}) line 1251 c4_SaveContext::CommitSequence(c4_HandlerSeq & {...}, unsigned char 1) line 850 + 33 bytes c4_SaveContext::SaveIt(c4_HandlerSeq & {...}, c4_Allocator * * 0x022998c8, c4_Bytes & {...}) line 626 c4_Persist::Commit(unsigned char 0) line 952 c4_Storage::Commit(unsigned char 0) line 384 + 50 bytes Has anyone else come across this? jcw - any idea what can cause this (and where we should look to try and nail it down to a test case)? Cheers, Steve. -- Steve Baxter Software Development Manager Improvision +44-2476-692229 steve@improvision.com From mcfletch@rogers.com Tue Oct 29 07:03:06 2002 Received: from fep01-mail.bloor.is.net.cable.rogers.com (fep01-mail.bloor.is.net.cable.rogers.com [66.185.86.71]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9TD35a27643 for ; Tue, 29 Oct 2002 07:03:05 -0600 Received: from rogers.com ([24.43.65.252]) by fep01-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021029130342.DLTP4829.fep01-mail.bloor.is.net.cable.rogers.com@rogers.com> for ; Tue, 29 Oct 2002 08:03:42 -0500 Message-ID: <3DBE870F.3000504@rogers.com> Date: Tue, 29 Oct 2002 08:03:11 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep01-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Tue, 29 Oct 2002 08:03:42 -0500 Subject: [Metakit] Any samples of how to use a blocked-ordered (BTree) table in Python? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've been trying to get a blocked-ordered (BTree'd) table working in Python, and seem to be missing something. As I understand it, I should be able to do this with: """Simple sample for creating a BTree in Python This takes a little documentation to be obvious, basically, you only create a single view using getas, this has a single property which is a sub- view with the "target" structure of your table. You then call table.blocked().ordered( numkeys ) to get a table with the BTree-like semantics. """ import metakit import os file = 'testBTree.mk' try: os.remove( file ) except: pass db = metakit.storage( file, 1 ) _words = db.getas( '_words[_B[word:S,ID:I]]' ).blocked() words = _words.ordered(1) words.append( ('this',1) ) words.append( ('those',1) ) words.append( ('this',2) ) db.commit() print "words (should be 2 records, no duplicate of 'this')" print words.structure() metakit.dump( words ) and, indeed, that works (as in it finishes running), but the result is _3_ records, i.e. the key 'this' is getting recorded twice in the table, instead of overwriting the first instance. I'm not sure what I'm doing wrong, but there's obviously something fishy about my code. As another note, I think the reference docs for blocked may be incorrect in the signature for the method. As another note, if I append ('this',1) instead of ('this',2) the second time, there are only 2 records (almost as if the entire record was being used as a key). Thoughts appreciated, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From FabienB@gilian.com Wed Oct 30 17:27:51 2002 Received: from gilianmail.gilian.com (gw-guilian.ser.netvision.net.il [199.203.171.166]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9UNRoa14959 for ; Wed, 30 Oct 2002 17:27:51 -0600 Received: by GILIANMAIL with Internet Mail Service (5.5.2653.19) id ; Thu, 31 Oct 2002 01:37:13 +0200 Message-ID: <1797E7C8F378D411A1CE00D0B77E59CEC7C644@GILIANMAIL> From: Fabien Benichou To: "'metakit@equi4.com'" Date: Thu, 31 Oct 2002 01:37:12 +0200 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Can I add a property to a hash map view? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I would like to add a property to an already hash mapped view. Is it possible or forbidden? If yes, why the following code is crashing? Thx a lot, Fabien. Code that crashed (note that I do not use GetAs to create the map view): c4_StringProp pKey ("key"); c4_IntProp pC1("C1"); c4_View data, datah; data.AddProperty(pKey); c4_View map; c4_IntProp _H("_H"); c4_IntProp _R("_R"); map.AddProperty(_H); map.AddProperty(_R); //Create the hashed view datah = data.Hash(map, 1); //Add another property datah.AddProperty(pC1); //Add a row c4_Row row; pKey (row) = "hello"; //THE CRASH HAPPENS HERE!!! pC1 (row) = 0; datah.Add(row); From mcfletch@rogers.com Wed Oct 30 22:31:48 2002 Received: from fep04-mail.bloor.is.net.cable.rogers.com (fep04-mail.bloor.is.net.cable.rogers.com [66.185.86.74]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g9V4Vma00381 for ; Wed, 30 Oct 2002 22:31:48 -0600 Received: from rogers.com ([24.43.65.252]) by fep04-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021031043152.MMZI4298.fep04-mail.bloor.is.net.cable.rogers.com@rogers.com> for ; Wed, 30 Oct 2002 23:31:52 -0500 Message-ID: <3DC0B22B.9090606@rogers.com> Date: Wed, 30 Oct 2002 23:31:39 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep04-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Wed, 30 Oct 2002 23:31:52 -0500 Subject: [Metakit] Any scenarios where data can be lost like this? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Here's the scenario: * method that adds around 10,000 items to the database mapping "normalised" forms of words to sets of word-ids. As it does so, it checks to be sure that the ID really is in the sub-view "words" of normalised. This always suceeds (i.e. the database reports that ID is in words immediately after insertion). * method that re-opens the database and runs through to check that all items are really in the database. This method finds ~50 entries in "normalised" where the words sub-table is empty. * database is in autocommit mode, and commit is called explicitly at the end of the add method just in case. * there is no code anywhere to delete entries from the database. * I don't explicitly close the database at any point in time, just drop the objects and create new ones as needed. * If I run addWords() twice, the words are all present and accounted for on all subsequent runs. However, if I run it only once, then the 50 entries (always the same 50) are missing (they have 0-length words sub-views, which was explicitly checked against in the adding pass). Each time I'm running addWords (or checkWords) I'm re-opening the database. (I can't do multiple addWords in real-life to work around the problem, as it can take 8 minutes just to do an insert for a large (750,000 word) dictionary, and that's already a bit more than is really desirable). database table is structured as follows: 'normalised[_B[norm:S,words[ID:I]]]' So, basically, are their practices in the above which raise red-flags for "don't do that you'll lose data"? Or do I need to rip out enough of the app to be able to reproduce the failure? Thoughts appreciated, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From apwd@yahoo.com Fri Nov 1 14:03:49 2002 Received: from web11503.mail.yahoo.com (web11503.mail.yahoo.com [216.136.172.35]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA1K3na21401 for ; Fri, 1 Nov 2002 14:03:49 -0600 Message-ID: <20021101200433.20721.qmail@web11503.mail.yahoo.com> Received: from [24.206.89.101] by web11503.mail.yahoo.com via HTTP; Fri, 01 Nov 2002 12:04:33 PST Date: Fri, 1 Nov 2002 12:04:33 -0800 (PST) From: Cory To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] Database examples Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have written a generic, cross-platform Metakit database viewer and was wondering if anyone had some example databases I could use to test it. Cory Thomas apwd@yahoo.com __________________________________________________ Do you Yahoo!? HotJobs - Search new jobs daily now http://hotjobs.yahoo.com/ From tbrowder@spamcop.net Sun Nov 3 14:21:10 2002 Received: from mailgate.cesmail.net (mailgate.cesmail.net [204.214.92.120]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA3KLAa20790 for ; Sun, 3 Nov 2002 14:21:10 -0600 Received: (qmail 4383 invoked from network); 3 Nov 2002 20:21:56 -0000 Received: from unknown (HELO gamma.cesmail.net) (192.168.1.20) by mailgate.cesmail.net with SMTP; 3 Nov 2002 20:21:56 -0000 Received: (qmail 10345 invoked by uid 99); 3 Nov 2002 20:21:56 -0000 Received: from ip68-14-187-206.pn.at.cox.net (ip68-14-187-206.pn.at.cox.net [68.14.187.206]) by webmail.spamcop.net (IMP) with HTTP for ; Sun, 3 Nov 2002 15:21:56 -0500 Message-ID: <1036354916.3dc585643a681@webmail.spamcop.net> Date: Sun, 3 Nov 2002 15:21:56 -0500 From: tbrowder@spamcop.net To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs Subject: [Metakit] Version 2.4.8 and GCC 3.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I just downloaded the latest version of MetaKit and eagerly tried to compile it under a standard i386 RedHat 8.0 distribution with the '-Dg4_STL' option in CXXFLAGS and it core dumps during 'make test' (it also happens without the STL option). (I also had to make line 23 identical to line 21 in std.h.) The immediate problem seems to be the use of void pointers in the c4_ArrayT when a 0 is passed in column.cpp at line 556. Using a 'static_cast(0)' gets past the immediate function call but the cascading calls to wrapped functions fail with the same problem. Scott Meyers has a long discussion of the problems with passing null generic pointers in his "Effective C++," item 25. Perhaps that needs to be looked at here. Has anyone else had the same problem, or am I doing something stupid? Thanks. Tom From tbrowder@spamcop.net Sun Nov 3 15:13:25 2002 Received: from mailgate.cesmail.net (mailgate.cesmail.net [204.214.92.120]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA3LDPa21984 for ; Sun, 3 Nov 2002 15:13:25 -0600 Received: (qmail 10336 invoked from network); 3 Nov 2002 21:14:11 -0000 Received: from unknown (HELO gamma.cesmail.net) (192.168.1.20) by mailgate.cesmail.net with SMTP; 3 Nov 2002 21:14:11 -0000 Received: (qmail 12761 invoked by uid 99); 3 Nov 2002 21:14:11 -0000 Received: from ip68-14-187-206.pn.at.cox.net (ip68-14-187-206.pn.at.cox.net [68.14.187.206]) by webmail.spamcop.net (IMP) with HTTP for ; Sun, 3 Nov 2002 16:14:11 -0500 Message-ID: <1036358051.3dc591a379d87@webmail.spamcop.net> Date: Sun, 3 Nov 2002 16:14:11 -0500 From: tbrowder@spamcop.net To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs Subject: [Metakit] Re: New Version and GCC Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Uh, I don't know what I messed up, but the stock configuration works fine, now. Sorry for the false alarm. Tom From jcw@equi4.com Mon Nov 4 03:04:00 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA493xa06198; Mon, 4 Nov 2002 03:03:59 -0600 Date: Mon, 4 Nov 2002 10:04:46 +0100 Mime-Version: 1.0 (Apple Message framework v546) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: announce@equi4.com, MetaKit mailing list Content-Transfer-Encoding: 7bit Message-Id: <7725F048-EFD4-11D6-A213-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.546) Subject: [Metakit] MetaKit 2.4.8 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is an announcement of a new release of MetaKit. It consolidates minor changes and several bug fixes and build tweaks. Extract from the change history at http://www.equi4.com/metakit/CHANGES : - tweaks to better support build on HPUX, WinCE, IRIX, AIX, MacOS X - added unix pthreads to match win32 code (no new features, just a port) - PDA's: WinCE adds win\msevc3 subdir, iPaq/Linux on ARM just works - fixed bug in handling blocked views containing subviews - Mk4py and Mk4tcl now install as real packages, in the corresponding dirs As always, the home page is http://www.equi4.com/metakit - note that the web site is undergoing some changes, so if things are inaccurate / missing / broken / unexpected, please let me know. Happy programming! -jcw From jgodfrey@optinest.com Tue Nov 5 14:53:21 2002 Received: from linux2.optinest.com (node-20.65.210.128.in-addr.arpa [65.210.128.20] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA5KrKa18750 for ; Tue, 5 Nov 2002 14:53:20 -0600 Received: from nt40ld (nt4-0ld [65.210.128.93]) by linux2.optinest.com (8.11.6/8.11.3) with SMTP id gA5KqeY28081 for ; Tue, 5 Nov 2002 14:53:37 -0600 From: "Jeff Godfrey" To: Date: Tue, 5 Nov 2002 14:55:34 -0600 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Where is the newest Mk4tcl.dll for Windows? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I am currently running on WinNT using Mk4tcl 2.4.6. According to the equi4 website, at least 2.4.7 should be available, and I think 2.4.8, but I can't seem to *find* them. What is newest Windows version of Mk4tcl.dll (pre-built), and where can I find it? Thanks, Jeff Godfrey Optimation USA Phone: 816.836.2000 Fax : 816.836.8589 Email: mailto:jgodfrey@optinest.com From mcfletch@rogers.com Wed Nov 6 05:01:37 2002 Received: from fep03-mail.bloor.is.net.cable.rogers.com (fep03-mail.bloor.is.net.cable.rogers.com [66.185.86.73]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA6B1ba01509 for ; Wed, 6 Nov 2002 05:01:37 -0600 Received: from rogers.com ([24.43.65.252]) by fep03-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021106110110.XIKI4286.fep03-mail.bloor.is.net.cable.rogers.com@rogers.com> for ; Wed, 6 Nov 2002 06:01:10 -0500 Message-ID: <3DC8F680.3090801@rogers.com> Date: Wed, 06 Nov 2002 06:01:20 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep03-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Wed, 6 Nov 2002 06:01:10 -0500 Subject: [Metakit] Utility module for recursively printing dbase structure Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've just written a Python module for recursively describing a storage or view's structure. It's a rather trivial little module, but it does make it easy to see what's going on in your database. Here's a sample of the output: storage: view: 'phonetic_0_R' 2 rows property: _B (V) view: '_B' 0 rows property: norm (S) property: words (B) view: 'phonetic_0' 2 rows property: _B (V) view: '_B' 0 rows property: norm (S) property: words (B) ... Some things I noticed while writing it: * views do not appear to know their own name? * you don't seem to be able to get at a definition for a sub-view from the parent unless there is a row in the parent? * there are no StorageType or PropertyType objects in the metakit namespace * row-refs don't seem to have any accessible (documented) properties, such as "view" or "index" or "ndx" * storage doesn't seem to have any way to tell you: o whether it's in auto-commit mode o what file it's stored in (if any) * views don't seem to be able to tell you whether they are hashed/blocked/ordered Anyway, the module is available here: http://members.rogers.com/mcfletch/programming/ Hope it's of some use to someone, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From mcfletch@rogers.com Wed Nov 6 06:30:59 2002 Received: from fep03-mail.bloor.is.net.cable.rogers.com (fep03-mail.bloor.is.net.cable.rogers.com [66.185.86.73]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA6CUxa06516 for ; Wed, 6 Nov 2002 06:30:59 -0600 Received: from rogers.com ([24.43.65.252]) by fep03-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021106123035.ZNEV4286.fep03-mail.bloor.is.net.cable.rogers.com@rogers.com> for ; Wed, 6 Nov 2002 07:30:35 -0500 Message-ID: <3DC90B74.20709@rogers.com> Date: Wed, 06 Nov 2002 07:30:44 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep03-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Wed, 6 Nov 2002 07:30:34 -0500 Subject: [Metakit] Use more databases, or larger databases? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've got two major approaches I can take going forward; large numbers of databases, or large databases with large numbers of tables, and I'm wondering which is more "metakit-ish" in practice. Here's an estimate of the scales I'm dealing with: the estimates are very rough (real-world installs are likely to have fewer languages installed, and often will have fewer word-sets): Constants: ~41 tables/word-set ~10 word-sets/active dictionary (~6-7 system + 1-3 user) ~20 word-sets/language ~3 installed languages each language (all word-sets) is likely to be around 20MB of data when stored in the dbase <3 active dictionaries w/ totally disjoint word-sets (potentially many more that share word-sets) Option 1: 1 database per word-set Gives: ~30 open database files (common case ~10) ~60 + (1-3 * number of users) total database files on disk (common case ~30 files) ~41 tables in each database largest dbases would be ~12 MB Option 2: 1 system database, 1 user database (per-user) Gives: 2 open databases 2 database files (+1 per user beyond first) (41 tables/wordset)*(3 languages)*(20 wordsets/language) ~2460 tables in the two files probably file sizes of 50MB (system) and 10MB (user) in size for 3 languages So, basically, I'm looking for wisdom of the group regarding which is more likely to bring happiness and joy? Large numbers of tables in small numbers of databases, or small numbers of tables in large numbers of databases? BTW: all of my larger tables (large = ~ 2MB each, w/ 2 to 4 of those per language) are static/read-only, while some smaller tables (the user tables) will be constantly growing/changing. Thoughts appreciated, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From jeff_godfrey@pobox.com Wed Nov 6 12:28:09 2002 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA6IS8a31210 for ; Wed, 6 Nov 2002 12:28:09 -0600 Received: (qmail 24587 invoked from network); 6 Nov 2002 18:28:56 -0000 Received: from wbrg-dialup-216-74-200-106.iland.net (HELO quantex) (216.74.200.106) by chewey.iland.net with SMTP; 6 Nov 2002 18:28:56 -0000 Message-ID: <01c101c285c2$00e12560$0180d241@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: Date: Wed, 6 Nov 2002 12:26:14 -0600 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_01BA_01C2858F.B2C70480" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Newest Mk4tcl.dll - another try Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_01BA_01C2858F.B2C70480 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi All, Where can I find the newest Mk4tcl.dll file for Windows? I think it = should be either version 2.4.7 or 2.4.8. I can't seem to find it on the = equi4 website (only 2.4.6, which still has the "crash-on-exit" bug). Thanks, Jeff ------=_NextPart_000_01BA_01C2858F.B2C70480 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi All,
 
Where can I find the newest Mk4tcl.dll = file for=20 Windows?  I think it should be either version 2.4.7 or 2.4.8.  = I can't=20 seem to find it on the equi4 website (only 2.4.6, which still has the=20 "crash-on-exit" bug).

Thanks,

Jeff
------=_NextPart_000_01BA_01C2858F.B2C70480-- From jeff_godfrey@pobox.com Wed Nov 6 12:15:11 2002 Received: from hewey.iland.net (qmailr@hewey.iland.net [205.242.230.200]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA6IFBa30245 for ; Wed, 6 Nov 2002 12:15:11 -0600 Received: (qmail 14174 invoked from network); 6 Nov 2002 18:15:28 -0000 Received: from wbrg-dialup-216-74-200-106.iland.net (HELO quantex) (216.74.200.106) by hewey.iland.net with SMTP; 6 Nov 2002 18:15:28 -0000 Message-ID: <015501c285c0$33256920$0180d241@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: Date: Wed, 6 Nov 2002 12:12:58 -0600 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0152_01C2858D.D84CF400" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Newest Mk4tcl.dll - another try Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0152_01C2858D.D84CF400 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi All, Where can I find the newest Mk4tcl.dll file for Windows. I think it = should be either version 2.4.7 or 2.4.8. I can't seem to find it on the = equi4 website (only 2.4.6, which still has the "crash-on-exit" bug). Thanks, Jeff ------=_NextPart_000_0152_01C2858D.D84CF400 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi All,
 
Where can I find the newest Mk4tcl.dll = file for=20 Windows.  I think it should be either version 2.4.7 or 2.4.8.  = I can't=20 seem to find it on the equi4 website (only 2.4.6, which still has the=20 "crash-on-exit" bug).

Thanks,

Jeff
------=_NextPart_000_0152_01C2858D.D84CF400-- From jeff_godfrey@pobox.com Wed Nov 6 14:44:53 2002 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA6Kiqa06579 for ; Wed, 6 Nov 2002 14:44:52 -0600 Received: (qmail 9567 invoked from network); 6 Nov 2002 20:45:40 -0000 Received: from wbrg-dialup-216-74-200-106.iland.net (HELO quantex) (216.74.200.106) by chewey.iland.net with SMTP; 6 Nov 2002 20:45:40 -0000 Message-ID: <027601c285d5$0837d440$0180d241@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: Date: Wed, 6 Nov 2002 14:42:26 -0600 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0271_01C285A2.B9AFD660" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Repeat messages Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0271_01C285A2.B9AFD660 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sorry for the recent stack of like messages. Some weirdness going on = here... (when there's something strange in your neighborhood... who you = gonna' call?) Jeff ------=_NextPart_000_0271_01C285A2.B9AFD660 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Sorry for the recent stack of like = messages. =20 Some weirdness going on here... (when there's something strange in your=20 neighborhood... who you gonna' call?)
 
Jeff
------=_NextPart_000_0271_01C285A2.B9AFD660-- From jcw@equi4.com Wed Nov 6 15:19:07 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA6LJ6a08584; Wed, 6 Nov 2002 15:19:07 -0600 Date: Wed, 6 Nov 2002 22:19:55 +0100 Subject: Re: [Metakit] Newest Mk4tcl.dll - another try Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v546) Cc: Starkit list To: MetaKit mailing list From: Jean-Claude Wippler In-Reply-To: <015501c285c0$33256920$0180d241@quantex> Message-Id: <7EC72A2D-F1CD-11D6-BEE8-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.546) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id gA6LJ6a08584 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: > Where can I find the newest Mk4tcl.dll file for Windows.  I think it > should be either version 2.4.7 or 2.4.8.  I can't seem to find it on > the equi4 website (only 2.4.6, which still has the "crash-on-exit" > bug). (Mk4tcl 2.4.6 has no crash-on-exit, that's Tclkit AFAIK) Anyway... "your wish is my command": http://www.equi4.com/pub/mk/mk-2.4.8-win32/ I'll try to update Linux and MacOS soon as well. -jcw From jeff_godfrey@pobox.com Wed Nov 6 15:43:12 2002 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA6LhCa09941 for ; Wed, 6 Nov 2002 15:43:12 -0600 Received: (qmail 12204 invoked from network); 6 Nov 2002 21:44:01 -0000 Received: from wbrg-dialup-216-74-201-175.iland.net (HELO quantex) (216.74.201.175) by chewey.iland.net with SMTP; 6 Nov 2002 21:44:01 -0000 Message-ID: <001701c285dd$6a903ee0$0180d241@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "MetaKit mailing list" , "Jean-Claude Wippler" Cc: "Starkit list" References: <7EC72A2D-F1CD-11D6-BEE8-00050251EF2F@equi4.com> Subject: Re: [Metakit] Newest Mk4tcl.dll - another try Date: Wed, 6 Nov 2002 15:42:23 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: JCW, Thanks for the updated DLL. I currently *do* have an intermittent crash on exit bug with Mk4tcl. I'll cross my fingers and hope it goes away with the new version. I'll keep you posted... BTW, do you have any idea why my posts keep showing up in double? I got some kind of "Post by non-member to a members-only list" email today (I assume) because I have changed my email address, so I re-subscribed to the Metakit mailing list today. Do you think that's related? Thanks, Jeff ----- Original Message ----- From: "Jean-Claude Wippler" To: "MetaKit mailing list" Cc: "Starkit list" Sent: Wednesday, November 06, 2002 3:19 PM Subject: Re: [Metakit] Newest Mk4tcl.dll - another try Jeff Godfrey wrote: > Where can I find the newest Mk4tcl.dll file for Windows. I think it > should be either version 2.4.7 or 2.4.8. I can't seem to find it on > the equi4 website (only 2.4.6, which still has the "crash-on-exit" > bug). (Mk4tcl 2.4.6 has no crash-on-exit, that's Tclkit AFAIK) Anyway... "your wish is my command": http://www.equi4.com/pub/mk/mk-2.4.8-win32/ I'll try to update Linux and MacOS soon as well. -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From jeff_godfrey@pobox.com Wed Nov 6 15:52:21 2002 Received: from hewey.iland.net (qmailr@hewey.iland.net [205.242.230.200]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA6LqLa10541 for ; Wed, 6 Nov 2002 15:52:21 -0600 Received: (qmail 14343 invoked from network); 6 Nov 2002 21:52:35 -0000 Received: from wbrg-dialup-216-74-201-175.iland.net (HELO quantex) (216.74.201.175) by hewey.iland.net with SMTP; 6 Nov 2002 21:52:35 -0000 Message-ID: <002a01c285de$b4aa78a0$0180d241@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: Date: Wed, 6 Nov 2002 15:51:20 -0600 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0027_01C285AC.59DB2B40" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Mk4tcl - Windows crash on exit bug Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0027_01C285AC.59DB2B40 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi All, I was having an occasional "crash-on-exit" problem with Mk4tcl under = Windows (both Win98 and WinNT). I just upgraded from 2.4.6 to 2.4.8 = hoping that my problem might go away - unfortunately it did not. This = *seems* very similarly to the crash-on-exit bug some of us reported in = tclkit not so long ago. Like that problem, it seems very elusive - I = can't make it happen all the time. JCW seems unaware that this was = (is?) a problem with Mk4tcl. Has anyone else experienced this? I (sometimes) get something similar to the following on exit:=20 Wish84 caused an invalid page fault in module MK4TCL.DLL at = 017f:012ab4f0 Registers: Any info appreciated... Jeff ------=_NextPart_000_0027_01C285AC.59DB2B40 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi All,
 
I was having an occasional = "crash-on-exit" problem=20 with Mk4tcl under Windows (both Win98 and WinNT).  I just upgraded = from=20 2.4.6 to 2.4.8 hoping that my problem might go away - unfortunately it = did=20 not.  This *seems* very similarly to the crash-on-exit bug some of = us=20 reported in tclkit not so long ago.  Like that problem, it seems = very=20 elusive - I can't make it happen all the time.  JCW seems unaware = that this=20 was (is?) a problem with Mk4tcl.  Has anyone else experienced=20 this?
 
I (sometimes) get something similar to = the=20 following on exit:
 
Wish84 caused an invalid page fault in = module=20 MK4TCL.DLL at 017f:012ab4f0
Registers:
<a bunch of cryptic = stuff...>
 
Any info appreciated...
 
Jeff
------=_NextPart_000_0027_01C285AC.59DB2B40-- From nicolasb@maich.gr Thu Nov 7 02:22:09 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA78M7a05861 for ; Thu, 7 Nov 2002 02:22:08 -0600 Received: FROM maich.gr BY mail-server ; Thu Nov 07 10:54:10 2002 +0200 Message-ID: <3DCA2433.4020100@maich.gr> Date: Thu, 07 Nov 2002 10:28:35 +0200 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0rc2) Gecko/20020510 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jeff Godfrey CC: metakit@equi4.com Subject: Re: [Metakit] Mk4tcl - Windows crash on exit bug References: <002a01c285de$b4aa78a0$0180d241@quantex> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: > Hi All, > > I was having an occasional "crash-on-exit" problem with Mk4tcl under > Windows (both Win98 and WinNT). I just upgraded from 2.4.6 to 2.4.8 > hoping that my problem might go away - unfortunately it did not. This > *seems* very similarly to the crash-on-exit bug some of us reported in > tclkit not so long ago. Like that problem, it seems very elusive - I > can't make it happen all the time. JCW seems unaware that this was > (is?) a problem with Mk4tcl. Has anyone else experienced this? > > I (sometimes) get something similar to the following on exit: > > Wish84 caused an invalid page fault in module MK4TCL.DLL at 017f:012ab4f0 > Registers: > > > Any info appreciated... > > Jeff FWIW, an old post to this group, dont know if you are doing anything like this... Hi, Have tried to update a tclkit based app from an OLD version of tclkit (8.4a1 based), I believe from ~September 2000 to the latest version, but getting alot of crashes. I've narrowed it down to the fact that the OLD tclkit did not mind calls to a non-existing db.table, and did not crash on exit. The new tclkits still do not mind the above calls, but crash on exit. ##Test crash on exit % mk::file open exe C:/scripdoc-utils/tclkit.exe % set c [mk::view size test.table] 0 % exit Bad coding aside, this approach made it very convenient to initialize application variables to point to the begining of tables, without worrying whether the file was newley created (existing) or not. So my question is basically whether this is considered normal/designed behavior in the newer releases. regards, nicolas boretos From jyl@best.com Thu Nov 7 10:57:32 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gA7GvWa01791 for ; Thu, 7 Nov 2002 10:57:32 -0600 Received: (qmail 47564 invoked by uid 19667); 7 Nov 2002 16:44:38 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 7 Nov 2002 16:44:38 -0000 Message-ID: <3DCA9B25.30802@best.com> Date: Thu, 07 Nov 2002 08:56:05 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: "Mike C. Fletcher" CC: Metakit List Subject: Re: [Metakit] Use more databases, or larger databases? References: <3DC90B74.20709@rogers.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mike I use databases as units of consistency, i.e. I know a database is either in a new or old state. So I divide the contents up along consistency boundaries. If I need three views to always be consistent with each other, I'll put that into one database and commit changes to these views as a unit. So rather than design the database schema to take into account scaling, I design it based on what belongs together and which pieces of data do not relate to each other and thus are not required to preserve some kind of consistency relationship. As to scaling, I found MK to scale to large data sets, depending on the types of data you use. If you use integers, MK scales to enormous databases of many millions of records. You can also experience really good scalability with floating point data. If you use strings or blobs, there used to be some scalability limitations that JC has already discussed here -- browse the archives. I'm not sure whether all these have been addressed. If you use nested views, MK's scalability is again limited to data sets with several million records, so unless you want to catalogue the entire US population again you should be fine :). I know JC did a lot of work on scaling recently, he may want to chime in. In my work I avoid using nested views and I do not use derived views, so I cannot really speak for those features. It's just that I haven't really needed them, not that I'm avoiding them. YMMV as usual, $0.02 worth of advice, etc. --JYL Mike C. Fletcher wrote: > I've got two major approaches I can take going forward; > large numbers of databases, > or large databases with large numbers of tables, > and I'm wondering which is more "metakit-ish" in practice. > > Here's an estimate of the scales I'm dealing with: the estimates are > very rough (real-world installs are likely to have fewer languages > installed, and often will have fewer word-sets): > > Constants: > ~41 tables/word-set > ~10 word-sets/active dictionary (~6-7 system + 1-3 user) > ~20 word-sets/language > ~3 installed languages > each language (all word-sets) is likely to be around 20MB of data > when stored in the dbase > <3 active dictionaries w/ totally disjoint word-sets (potentially > many more that share word-sets) > > Option 1: > 1 database per word-set > Gives: > ~30 open database files (common case ~10) > ~60 + (1-3 * number of users) total database files on disk > (common case ~30 files) > ~41 tables in each database > largest dbases would be ~12 MB > > Option 2: > 1 system database, 1 user database (per-user) > Gives: > 2 open databases > 2 database files (+1 per user beyond first) > (41 tables/wordset)*(3 languages)*(20 wordsets/language) ~2460 > tables in the two files > probably file sizes of 50MB (system) and 10MB (user) in size > for 3 languages > > So, basically, I'm looking for wisdom of the group regarding which is > more likely to bring happiness and joy? Large numbers of tables in > small numbers of databases, or small numbers of tables in large > numbers of databases? BTW: all of my larger tables (large = ~ 2MB > each, w/ 2 to 4 of those per language) are static/read-only, while > some smaller tables (the user tables) will be constantly > growing/changing. > > Thoughts appreciated, > Mike > > _______________________________________ > Mike C. Fletcher > Designer, VR Plumber, Coder > http://members.rogers.com/mcfletch/ > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From mcfletch@rogers.com Fri Nov 8 02:15:21 2002 Received: from fep03-mail.bloor.is.net.cable.rogers.com (fep03-mail.bloor.is.net.cable.rogers.com [66.185.86.73]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA88FKa18081 for ; Fri, 8 Nov 2002 02:15:21 -0600 Received: from rogers.com ([24.43.65.252]) by fep03-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021108081603.RAZY4292.fep03-mail.bloor.is.net.cable.rogers.com@rogers.com>; Fri, 8 Nov 2002 03:16:03 -0500 Message-ID: <3DCB72B8.90604@rogers.com> Date: Fri, 08 Nov 2002 03:15:52 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jacob Levy CC: Metakit List Subject: Re: [Metakit] Use more databases, or larger databases? References: <3DC90B74.20709@rogers.com> <3DCA9B25.30802@best.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep03-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Fri, 8 Nov 2002 03:16:03 -0500 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Well, these data sets are word-sets of between 1000 and 1,000,000 strings, with each string stored ~4 times. Anyway, I think I'll likely go with option 2 and hope the scaling in a single db works well for strings as well :) . Thanks for the feedback, Mike Jacob Levy wrote: > Mike ... > As to scaling, I found MK to scale to large data sets, depending on > the types of data you use. If you use integers, MK scales to enormous > databases of many millions of records. You can also experience really > good scalability with floating point data. If you use strings or > blobs, there used to be some scalability limitations that JC has > already discussed here -- browse the archives. I'm not sure whether > all these have been addressed. If you use nested views, MK's > scalability is again limited to data sets with several million > records, so unless you want to catalogue the entire US population > again you should be fine :). I know JC did a lot of work on scaling > recently, he may want to chime in. In my work I avoid using nested > views and I do not use derived views, so I cannot really speak for > those features. It's just that I haven't really needed them, not that > I'm avoiding them. > > YMMV as usual, $0.02 worth of advice, etc. > > --JYL ... >> Option 2: >> 1 system database, 1 user database (per-user) >> Gives: >> 2 open databases >> 2 database files (+1 per user beyond first) >> (41 tables/wordset)*(3 languages)*(20 wordsets/language) ~2460 >> tables in the two files >> probably file sizes of 50MB (system) and 10MB (user) in size >> for 3 languages >> >> So, basically, I'm looking for wisdom of the group regarding which is >> more likely to bring happiness and joy? Large numbers of tables in >> small numbers of databases, or small numbers of tables in large >> numbers of databases? BTW: all of my larger tables (large = ~ 2MB >> each, w/ 2 to 4 of those per language) are static/read-only, while >> some smaller tables (the user tables) will be constantly >> growing/changing. > ... From jcw@equi4.com Fri Nov 8 06:07:16 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA8C7Fa26376 for ; Fri, 8 Nov 2002 06:07:15 -0600 Date: Fri, 8 Nov 2002 13:08:03 +0100 Subject: Re: [Metakit] Use more databases, or larger databases? Content-Type: text/plain; charset=WINDOWS-1252; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Jean-Claude Wippler To: Metakit List In-Reply-To: <3DC90B74.20709@rogers.com> Message-Id: X-Mailer: Apple Mail (2.546) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id gA8C7Fa26376 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mike C. Fletcher wrote: > I've got two major approaches I can take going forward; > large numbers of databases, > or large databases with large numbers of tables, > and I'm wondering which is more "metakit-ish" in practice. To add to Jacob's sound advice... > Option 1: > 1 database per word-set > Gives: > ~30 open database files (common case ~10) > ~60 + (1-3 * number of users) total database files on disk > (common case ~30 files) > ~41 tables in each database > largest dbases would be ~12 MB > > Option 2: > 1 system database, 1 user database (per-user) > Gives: > 2 open databases > 2 database files (+1 per user beyond first) > (41 tables/wordset)*(3 languages)*(20 wordsets/language) ~2460 > tables in the two files > probably file sizes of 50MB (system) and 10MB (user) in size > for 3 languages I'd indeed tend to go for 2 (as you indicated in a follow-up post). Number of views is a few thousand - not a problem (views, subviews, it's all the same inside MK). > Thoughts appreciated, Well, in that case... You're looking for word typo's, and the main tabel at least is essentially R/O, right? How about this - coming from someone who has never worked with such datasets, so please add salt and pick only the tasty bits: Create a monster table with all possible typo's. But instead of doing this with strings, and using scanning, or sorted searches, store MD5's and use hashing. I.e. from word -> MD5 -> hash lookup (O(1)) -> an index (one int). That index points to the corrected word, a second table since the same word is the outcome of various typo's. Well, or some variation - partial MD5, just CRC, whatever. Since it's pre-computed, you can tweak the MD5 hash or CRC to avoid collisions. It would not work for user dicts perhaps, but it would allow you to use MK's hashing. There's also "perfect hashing" as a technique, though I wouldn't bother. Having said that, the advantage of a 32-bit hash would be that you end up with an int vector which is considerably more efficient than a string or byte string (startup times, mem overhead, and access). So maybe perfect hashing would be worth it after all. Then, you don;t even need MK's hash - you just apply the perfect hash and out comes an index into the "fixed typo" view. Maybe all this is futile - I'm sure lots and lots of minds have worked on these issues before - but then again, maybe it helps you find some way to resolve this with not too much effort. I would expect that algorithmic changes might offer far more gains than representation tweaks. That applies to any solution and database of course, not just Metakit. Just my €.02 ... -jcw From tom.krehbiel@motorola.com Fri Nov 8 09:31:16 2002 Received: from motgate3.mot.com (motgate3.mot.com [144.189.100.103]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA8FVFa02644 for ; Fri, 8 Nov 2002 09:31:15 -0600 Received: from pobox4.mot.com (pobox4.mot.com [10.64.251.243]) by motgate3.mot.com (Motorola/Motgate3) with ESMTP id gA8FT5Rj019783 for ; Fri, 8 Nov 2002 08:29:06 -0700 (MST) Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by pobox4.mot.com (MOT-pobox4 2.0) with ESMTP id IAA21252 for ; Fri, 8 Nov 2002 08:32:03 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2656.59) id V7XB09B8; Fri, 8 Nov 2002 08:32:03 -0700 Message-ID: <3DCBD911.E0A65116@motorola.com> Date: Fri, 08 Nov 2002 08:32:33 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 CC: Metakit List Subject: Re: [Metakit] Use more databases, or larger databases? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mike, I missed the rest of this thread so I may be way of base with the following comments. I sounds like your creating a word index of miss spelled words. If so I would recommend you look at the ispell code (very old). They use a word dictionary that includes a suffix rule for each word. The suffix rule is used to create word variations. The word list is also hashed. Tom K. > Mike C. Fletcher wrote: > > > I've got two major approaches I can take going forward; > > large numbers of databases, > > or large databases with large numbers of tables, > > and I'm wondering which is more "metakit-ish" in practice. > > To add to Jacob's sound advice... > > > Option 1: > > 1 database per word-set > > Gives: > > ~30 open database files (common case ~10) > > ~60 + (1-3 * number of users) total database files on disk > > (common case ~30 files) > > ~41 tables in each database > > largest dbases would be ~12 MB > > > > Option 2: > > 1 system database, 1 user database (per-user) > > Gives: > > 2 open databases > > 2 database files (+1 per user beyond first) > > (41 tables/wordset)*(3 languages)*(20 wordsets/language) ~2460 > > tables in the two files > > probably file sizes of 50MB (system) and 10MB (user) in size > > for 3 languages > > I'd indeed tend to go for 2 (as you indicated in a follow-up post). > > Number of views is a few thousand - not a problem (views, subviews, > it's all the same inside MK). > > > Thoughts appreciated, > > Well, in that case... > > You're looking for word typo's, and the main tabel at least is > essentially R/O, right? How about this - coming from someone who has > never worked with such datasets, so please add salt and pick only the > tasty bits: > > Create a monster table with all possible typo's. But instead of doing > this with strings, and using scanning, or sorted searches, store MD5's > and use hashing. I.e. from word -> MD5 -> hash lookup (O(1)) -> an > index (one int). That index points to the corrected word, a second > table since the same word is the outcome of various typo's. > > Well, or some variation - partial MD5, just CRC, whatever. Since it's > pre-computed, you can tweak the MD5 hash or CRC to avoid collisions. > > It would not work for user dicts perhaps, but it would allow you to use > MK's hashing. > > There's also "perfect hashing" as a technique, though I wouldn't > bother. Having said that, the advantage of a 32-bit hash would be that > you end up with an int vector which is considerably more efficient than > a string or byte string (startup times, mem overhead, and access). So > maybe perfect hashing would be worth it after all. Then, you don;t > even need MK's hash - you just apply the perfect hash and out comes an > index into the "fixed typo" view. > > Maybe all this is futile - I'm sure lots and lots of minds have worked > on these issues before - but then again, maybe it helps you find some > way to resolve this with not too much effort. I would expect that > algorithmic changes might offer far more gains than representation > tweaks. That applies to any solution and database of course, not just > Metakit. > > Just my ?.02 ... > > -jcw From gregorb@clear.net.nz Fri Nov 8 12:36:43 2002 Received: from smtp1.clear.net.nz (smtp1.clear.net.nz [203.97.33.27]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA8Iaha19537 for ; Fri, 8 Nov 2002 12:36:43 -0600 Received: from clear.net.nz (203-167-167-49.dialup.clear.net.nz [203.167.167.49]) by smtp1.clear.net.nz (CLEAR Net Mail) with ESMTPA id <0H5900BGST2HRE@smtp1.clear.net.nz> for metakit@equi4.com; Sat, 09 Nov 2002 07:37:32 +1300 (NZDT) Date: Sat, 09 Nov 2002 07:36:05 +1300 From: Gregor Bruce Subject: Re: [Metakit] Mk4tcl - Windows crash on exit bug To: metakit@equi4.com Message-id: <3DCC0415.1070705@clear.net.nz> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7bit X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.1b) Gecko/20020721 References: <002a01c285de$b4aa78a0$0180d241@quantex> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: >Hi All, > >I was having an occasional "crash-on-exit" problem with Mk4tcl under Windows (both Win98 and WinNT). I just upgraded from 2.4.6 to 2.4.8 hoping that my problem might go away - unfortunately it did not. This *seems* very similarly to the crash-on-exit bug some of us reported in tclkit not so long ago. Like that problem, it seems very elusive - I can't make it happen all the time. JCW seems unaware that this was (is?) a problem with Mk4tcl. Has anyone else experienced this? > >I (sometimes) get something similar to the following on exit: > >Wish84 caused an invalid page fault in module MK4TCL.DLL at 017f:012ab4f0 >Registers: > > >Any info appreciated... > >Jeff > > > Hi, I have a similar problem using 2.4.3. To avoid crash on exit I comment out a line in mk4tcl.cpp: static void ExitProc(ClientData cd_) { // avoid exit crash from shell // delete (MkWorkspace*) cd_; } Gregor Bruce From jcw@equi4.com Fri Nov 8 13:05:16 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA8J5Fa21787; Fri, 8 Nov 2002 13:05:15 -0600 Date: Fri, 8 Nov 2002 20:06:02 +0100 Subject: Re: [Metakit] Mk4tcl - Windows crash on exit bug Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) Cc: metakit@equi4.com To: Gregor Bruce From: Jean-Claude Wippler In-Reply-To: <3DCC0415.1070705@clear.net.nz> Message-Id: <1FF1C669-F34D-11D6-A337-00050251EF2F@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.546) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gregor Bruce wrote: > I have a similar problem using 2.4.3. To avoid crash on exit I > comment out a line in mk4tcl.cpp: > > static void ExitProc(ClientData cd_) > { > // avoid exit crash from shell > // delete (MkWorkspace*) cd_; > } You're in essence turning off automatic cleanup. Things like having a commit done automatically on exit will be disabled, I assume (if not, PLEASE let me know, the there might be a case of double-cleanup!). Apart from the fact that you should really upgrade from all versions 2.4.2..2.4.6 because of a serious bug (see change log), I would really like to know whether this problem has been resolved in the latest builds. So if you have a chance to test against 2.4.8, that would be grand... -jcw From rjent@rjent.pair.com Fri Nov 8 16:56:47 2002 Received: from portal2.isz (pppoe-66-112-23-17.rb.spt.centurytel.net [66.112.23.17]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA8Muka10258 for ; Fri, 8 Nov 2002 16:56:46 -0600 Received: from rjent.pair.com (localhost [127.0.0.1]) by portal2.isz (8.12.5/8.12.5) with ESMTP id gA8KvVCh007382 for ; Fri, 8 Nov 2002 14:57:32 -0600 (CST) (envelope-from rjent@rjent.pair.com) From: "RJ Ent" To: metakit@equi4.com Date: Fri, 8 Nov 2002 15:57:31 -0500 Message-Id: <20021108205731.M25967@rjent.pair.com> X-Mailer: Open WebMail 1.70 20020712 X-OriginatingIP: 192.168.2.34 (rjent) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] Linux-MetaKit 2.4.8 Mk4tcl.so load error? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, I have encountered a problem with loading the latest linux Metakit (2.4.8). Below is the output from my linux workstation: Error in startup script: couldn't load file "./Mk4tcl.so": ./Mk4tcl.so: undefined symbol: _ZTVN10__cxxabiv120__si_class_type_infoE while executing "load ./Mk4tcl.so" I can put my old Mk4tcl.so back and all is well. Any ideas what I am missing to be able to use the latest MetaKit? Many thanks! -- RJEnt -- http://www.rjent.pair.com From jcw@equi4.com Fri Nov 8 17:08:17 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA8N8Ha11819 for ; Fri, 8 Nov 2002 17:08:17 -0600 Date: Sat, 9 Nov 2002 00:09:05 +0100 Subject: Re: [Metakit] Linux-MetaKit 2.4.8 Mk4tcl.so load error? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20021108205731.M25967@rjent.pair.com> Message-Id: <13E2C751-F36F-11D6-A337-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.546) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: RJ Ent wrote: > I have encountered a problem with loading the latest linux Metakit > (2.4.8). > Below is the output from my linux workstation: > > Error in startup script: couldn't load file > "./Mk4tcl.so": ./Mk4tcl.so: undefined symbol: > _ZTVN10__cxxabiv120__si_class_type_infoE > while executing > "load ./Mk4tcl.so" > > I can put my old Mk4tcl.so back and all is well. Any ideas what I am > missing > to be able to use the latest MetaKit? Urk - are you using the binaries from the web site? I think I built with gcc 3.2 - i.e. a very recent one. Not sure. I can go back to 2.95.3, but then it'll probably trip someone else. Looks like RTTI info (which I do not need at all), though this error may well be hiding a slew of other undefineds - what output do you get from "ldd Mk4tcl.so"? Does anyone have a suggestion on how to best deal with this? Is there a set of options to hand to gcc to get better binaries? Or is the inevitable answer going to be to revert to the classical configure/make sing-and-dance routine? Ain't binary linking a grand concept... :( -jcw From mcfletch@rogers.com Sat Nov 9 04:26:59 2002 Received: from fep04-mail.bloor.is.net.cable.rogers.com (fep04-mail.bloor.is.net.cable.rogers.com [66.185.86.74]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA9AQxa06044; Sat, 9 Nov 2002 04:26:59 -0600 Received: from rogers.com ([24.43.65.252]) by fep04-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021109102744.YGEY4992.fep04-mail.bloor.is.net.cable.rogers.com@rogers.com>; Sat, 9 Nov 2002 05:27:44 -0500 Message-ID: <3DCCE30A.5040108@rogers.com> Date: Sat, 09 Nov 2002 05:27:22 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Metakit List Subject: Re: [Metakit] Use more databases, or larger databases? References: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep04-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Sat, 9 Nov 2002 05:27:43 -0500 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The problem with pre-generating is that, though for a single-edit distance there are a reasonable number of "typos" for a given word; once you get beyond 1 edit-distance, you're looking at 10s of thousands (at 2) through millions at 3 and ridiculous numbers beyond that (assuming that your words/sounds allow for any of 255 characters (null is excluded)) of possible typos for any given word. At single-edit-distance I do exhaustively generate the possible edits from the source-word (the typo for which I'm trying to find a "good" word (or, really a thousand such, so the user can choose)) and do direct searches for each of those result targets, save for "insertions", for which I do a scan beyond the first character (which gives around ~2000 database checks). (I explain this a little more clearly later). I'm using the best algorithm for this type of search (phonetically-compressed edit-distance) that I could find (at least according to the various papers I read it's the best). I've added a major optimisation (of my own design) for d>1 searches which, though it loses a few typos, strongly limits the maximum time to search (and, nicely, actually gets faster with longer words), varying primarily by the size of the dictionary (for any given d, with linear instead of combinatoric growth for larger values of d). In practice, this lets me do an average of 3000 scans (with no checks) to find d of 2, 3, 4, or 5. With a simple loop, I can scan from 1->5 progressively until I have a thousand or so possible answers and still get around .1s total search time on average. Basically, the technique is based on phonetic compression (similar to what you see in soundex) and a primitive function that counts the number of changes required between any two items. For each "good" word, you calculate a "compressed" form (which is, roughly, what the word "sounds like", with certain other simplifications) and keep a table mapping sound:[words]. To find a similar word, at d==1, you just check all possible edits which are the same length or one shorter, then check char+word for char in charset, and finally, scan from firstChar (of source) through firstChar+1 in the length+1 table. For higher d, you select a small sub-string at the beginning and end of the word. You scan all tables from length-d through length + d for prefix/postfix:prefix/postfix+1 , checking each sound for the edit-distance from the source-sound. (This requires a reversed-string table to do the postfix scans, of course). At the moment the two tables have the same structure, but I could, instead, make the reversed-string table just store the phonetic compression, and do a .find() for every matching sound... I'm just not that motivated to introduce another 1000 or so finds into the algorithm, and the ordered character is critical for the scans, so I can't use the hash for the base table... so, for right now I've got 4x the data instead of 3x. Oh, one other nice thing about this approach, the same table _can_ act as your primary "does a word exist" table as well (I don't have it doing so at the moment, but it's the same mechanism). This area of research really is quite fascinating once you start delving into it. For instance, the use of a phonetic compressor has an ambiguous effect on the "quality" of results from the engine. There are lots of cases where the more intelligent phonetic compressors actually get in the way (because they are too focussed on capturing nuances of higher-level sounds (e.g. seperating hard-g and soft-gh) when the compression really wants them to show up as very similar (in case someone left off an h)). On the other hand, without a phonetic compressor, lots of common errors return 0 possible matches because the user really is just sounding the thing out. One of the other major contenders for fuzzy-search algos is (phonetic-)feature extraction (taking slices of the string and recording them against the words). That approach was actually why I first tried Metakit on this project (since to do this you _need_ wordIDs, as you're indexing each word len(word)+1 times). That approach does a nice job of finding words when you're looking at very badly mispelled data ( rezrvan -> reservation, for instance), because it can pull out infrequently-used letter-combinations (start-of-word+'r', 'rv' and 'va' in the example) and give words which share multiple such combinations higher scores. My initial implementation used a seperate words + IDs table (BTree indexed). Looking back on it, with a hash and using the natural index as the ID, it probably would have performed acceptably (it was abysmally slow with the initial implementation)). I may add it (after all, this is a toolkit for constructing these kinds of engines, why not add another algorithm for people to play with). Oh, a neat thing: you want collisions in most similarity-search algos I've seen discussed... if you could get an algorithm that always had "similar" (according to human understanding), and only (a small number of) similar words collide in the key-space, you'd have an almost perfect algo. There are lots of fascinating ways you can try to build that with classic compression approaches (e.g. feature extraction above), but you really do have to test the results to have any idea if they work well in practice... fun little games all over :) ... Okay, have blathered at you enough, time to read the next email, Mike Jean-Claude Wippler wrote: > Mike C. Fletcher wrote: ... > Well, in that case... > > You're looking for word typo's, and the main tabel at least is > essentially R/O, right? How about this - coming from someone who has > never worked with such datasets, so please add salt and pick only the > tasty bits: > > Create a monster table with all possible typo's. But instead of doing > this with strings, and using scanning, or sorted searches, store MD5's > and use hashing. I.e. from word -> MD5 -> hash lookup (O(1)) -> an > index (one int). That index points to the corrected word, a second > table since the same word is the outcome of various typo's. > > Well, or some variation - partial MD5, just CRC, whatever. Since it's > pre-computed, you can tweak the MD5 hash or CRC to avoid collisions. > > It would not work for user dicts perhaps, but it would allow you to > use MK's hashing. > > There's also "perfect hashing" as a technique, though I wouldn't > bother. Having said that, the advantage of a 32-bit hash would be > that you end up with an int vector which is considerably more > efficient than a string or byte string (startup times, mem overhead, > and access). So maybe perfect hashing would be worth it after all. > Then, you don;t even need MK's hash - you just apply the perfect hash > and out comes an index into the "fixed typo" view. > > Maybe all this is futile - I'm sure lots and lots of minds have worked > on these issues before - but then again, maybe it helps you find some > way to resolve this with not too much effort. I would expect that > algorithmic changes might offer far more gains than representation > tweaks. That applies to any solution and database of course, not just > Metakit. > > Just my €.02 ... > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > -- _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From mcfletch@rogers.com Sat Nov 9 04:34:26 2002 Received: from fep02-mail.bloor.is.net.cable.rogers.com (fep02-mail.bloor.is.net.cable.rogers.com [66.185.86.72]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA9AYQa06308 for ; Sat, 9 Nov 2002 04:34:26 -0600 Received: from rogers.com ([24.43.65.252]) by fep02-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021109103511.FRWV4594.fep02-mail.bloor.is.net.cable.rogers.com@rogers.com>; Sat, 9 Nov 2002 05:35:11 -0500 Message-ID: <3DCCE4C9.6020201@rogers.com> Date: Sat, 09 Nov 2002 05:34:49 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Tom Krehbiel CC: Metakit List Subject: Re: [Metakit] Use more databases, or larger databases? References: <3DCBD911.E0A65116@motorola.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep02-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Sat, 9 Nov 2002 05:35:11 -0500 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Tom, I'm actually looking at the aspell code (aspell's a few generations beyond ispell (via pspell IIUC))... well, not really the code so much as the documentation for the code. Suffix-reduction isn't something I'm currently doing, as it's likely to lose such things as: liftd -> lifted As the "d" wouldn't likely be seen as a suffix by the reducer, but the "ed" would be. Thanks for the pointer, though, I may take a look to see what they were using it for, Mike Tom Krehbiel wrote: >Mike, > >I missed the rest of this thread so I may be way of base with the following comments. I sounds like your creating a word index of >miss spelled words. If so I would recommend you look at the ispell code (very old). They use a word dictionary that includes a >suffix rule for each word. The suffix rule is used to create word variations. The word list is also hashed. > >Tom K. > > ... From gmcm@hypernet.com Sat Nov 9 07:40:22 2002 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA9DeLa12002 for ; Sat, 9 Nov 2002 07:40:22 -0600 Received: from PARANOIA (204.176.40.61) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Sat, 9 Nov 2002 08:40:02 -0500 From: "Gordon McMillan" To: "Mike C. Fletcher" Date: Sat, 9 Nov 2002 08:40:31 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Utility module for recursively printing dbase structure Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3DCCC9FF.6618.9670C6A@localhost> Priority: normal In-reply-to: <3DC8F680.3090801@rogers.com> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 6 Nov 2002 at 6:01, Mike C. Fletcher wrote: [Mike makes comments on the Python interface...] Some maybe partially satisfying answers... > Some things I noticed while writing it: > > * views do not appear to know their own name? Correct. Consider v.join(v2, v.shoesize). > * you don't seem to be able to get at a definition > for a sub-view > from the parent unless there is a row in the > parent? You can get it from the storage. > * there are no StorageType or PropertyType objects > in the metakit > namespace They are easy to expose. I never saw a need. > * row-refs don't seem to have any accessible > (documented) > properties, such as "view" or "index" or "ndx" No, the underlying C++ objects are opaque. Conceptually, they're a (view, index) pair, but the view might be anonymous and lazily evaluated (v.join(v2...) returns a "view" but it's really an iterator), and index is only meaningful in the context of it's particular view. > * storage doesn't seem to have any way to tell you: > o whether it's in auto-commit mode > o what file it's stored in (if any) Again, just wrapping the C++ object. Filename is something I always end up dealing with, so I'll think about adding that to the Python wrapper. > * views don't seem to be able to tell you whether > they are > hashed/blocked/ordered Heh. In fact, "ordered" is how you tell MK that the view is ordered. (Believe it or not, I have still never used hashed or blocked). Because views are composited out of other views (with differing capabilities), there are a gazillion flavors. The Python types are a *manufactured* type system designed to let you know what the general capabilities of a view are. As far as wrapping goes, there's only one C++ base class (and only one set of Python methods). All the other Python view types are created by cloning and using whiteout. That and some "what do you get when you cross a X with a Y" rules to decide what type object to point to. -- Gordon http://www.mcmillan-inc.com/ From rjent@rjent.pair.com Sat Nov 9 16:54:43 2002 Received: from portal2.isz (pppoe-66-112-23-17.rb.spt.centurytel.net [66.112.23.17]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gA9Msha29279 for ; Sat, 9 Nov 2002 16:54:43 -0600 Received: from rjent.pair.com (localhost [127.0.0.1]) by portal2.isz (8.12.5/8.12.5) with ESMTP id gA9KtVCh011264 for ; Sat, 9 Nov 2002 14:55:31 -0600 (CST) (envelope-from rjent@rjent.pair.com) From: "RJ Ent" To: metakit@equi4.com Date: Sat, 9 Nov 2002 15:55:31 -0500 Message-Id: <20021109205531.M80837@rjent.pair.com> X-Mailer: Open WebMail 1.70 20020712 X-OriginatingIP: 192.168.2.34 (rjent) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] Re: Linux-MetaKit 2.4.8 Mk4tcl.so load error? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, > RJ Ent wrote: > > > I have encountered a problem with loading the latest linux Metakit > > (2.4.8). > > Below is the output from my linux workstation: > > > > Error in startup script: couldn't load file > > "./Mk4tcl.so": ./Mk4tcl.so: undefined symbol: > > _ZTVN10__cxxabiv120__si_class_type_infoE > > while executing > > "load ./Mk4tcl.so" > > > > I can put my old Mk4tcl.so back and all is well. Any ideas what I am > > missing > > to be able to use the latest MetaKit? > > Urk - are you using the binaries from the web site? Yes. > I think I built > with gcc 3.2 - i.e. a very recent one. Not sure. I can go back to > 2.95.3, but then it'll probably trip someone else. > > Looks like RTTI info (which I do not need at all), though this error > may well be hiding a slew of other undefineds - what output do you get > from "ldd Mk4tcl.so"? libc.so.6 => /lib/libc.so.6 (0x4005a000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) > > Does anyone have a suggestion on how to best deal with this? Is there > a set of options to hand to gcc to get better binaries? Or is the > inevitable answer going to be to revert to the classical configure/make > sing-and-dance routine? > > Ain't binary linking a grand concept... :( > > -jcw -- RJEnt -- http://www.rjent.pair.com From jcw@equi4.com Sat Nov 9 19:37:16 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAA1bEa01510; Sat, 9 Nov 2002 19:37:15 -0600 Date: Sun, 10 Nov 2002 02:38:03 +0100 Subject: Re: [Metakit] Re: Linux-MetaKit 2.4.8 Mk4tcl.so load error? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v546) Cc: metakit@equi4.com To: "RJ Ent" From: Jean-Claude Wippler In-Reply-To: <20021109205531.M80837@rjent.pair.com> Message-Id: <0DAB72AD-F44D-11D6-A3C7-00050251EF2F@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.546) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: RJ Ent wrote: >>> Error in startup script: couldn't load file >>> "./Mk4tcl.so": ./Mk4tcl.so: undefined symbol: >>> _ZTVN10__cxxabiv120__si_class_type_infoE >>> while executing >>> "load ./Mk4tcl.so" >>> >>> I can put my old Mk4tcl.so back and all is well. Any ideas what I am >>> missing to be able to use the latest MetaKit? >> >> Urk - are you using the binaries from the web site? > > Yes. > >> I think I built >> with gcc 3.2 - i.e. a very recent one. Not sure. I can go back to >> 2.95.3, but then it'll probably trip someone else. I've replaced the binaries with 2.95.3 builds - maybe that helps. -jcw From gregorb@clear.net.nz Sun Nov 10 12:27:22 2002 Received: from smtp2.clear.net.nz (smtp2.clear.net.nz [203.97.37.27]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAAIRMa09041 for ; Sun, 10 Nov 2002 12:27:22 -0600 Received: from clear.net.nz (203-167-167-22.dialup.clear.net.nz [203.167.167.22]) by smtp2.clear.net.nz (CLEAR Net Mail) with ESMTPA id <0H5D000ZPHYYYX@smtp2.clear.net.nz> for metakit@equi4.com; Mon, 11 Nov 2002 07:28:12 +1300 (NZDT) Date: Mon, 11 Nov 2002 07:26:43 +1300 From: Gregor Bruce Subject: Re: [Metakit] Mk4tcl - Windows crash on exit bug To: metakit@equi4.com Message-id: <3DCEA4E3.8000204@clear.net.nz> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7bit X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.1b) Gecko/20020721 References: <1FF1C669-F34D-11D6-A337-00050251EF2F@equi4.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Gregor Bruce wrote: > >> I have a similar problem using 2.4.3. To avoid crash on exit I >> comment out a line in mk4tcl.cpp: >> >> static void ExitProc(ClientData cd_) >> { >> // avoid exit crash from shell >> // delete (MkWorkspace*) cd_; >> } > > > You're in essence turning off automatic cleanup. Things like having a > commit done automatically on exit will be disabled, I assume (if not, > PLEASE let me know, the there might be a case of double-cleanup!). I checked and made sure that ExitProc was being called once only. > > Apart from the fact that you should really upgrade from all versions > 2.4.2..2.4.6 because of a serious bug (see change log), I would really > like to know whether this problem has been resolved in the latest > builds. So if you have a chance to test against 2.4.8, that would be > grand... > This upgrade is on my to-do list, however I am not having any other problems with MK, so its down the priority list I'm afraid. Gregor Bruce From mcfletch@rogers.com Tue Nov 12 01:46:17 2002 Received: from fep03-mail.bloor.is.net.cable.rogers.com (fep03-mail.bloor.is.net.cable.rogers.com [66.185.86.73]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAC7kGa24429 for ; Tue, 12 Nov 2002 01:46:16 -0600 Received: from rogers.com ([24.43.65.252]) by fep03-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021112074703.IXEY4292.fep03-mail.bloor.is.net.cable.rogers.com@rogers.com>; Tue, 12 Nov 2002 02:47:03 -0500 Message-ID: <3DD0B1C6.6010909@rogers.com> Date: Tue, 12 Nov 2002 02:46:14 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gmcm@hypernet.com CC: metakit@equi4.com Subject: Re: [Metakit] Utility module for recursively printing dbase structure References: <3DCCC9FF.6618.9670C6A@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep03-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Tue, 12 Nov 2002 02:47:03 -0500 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: (Comments interspersed below) Gordon McMillan wrote: >On 6 Nov 2002 at 6:01, Mike C. Fletcher wrote: > >[Mike makes comments on the Python interface...] > >Some maybe partially satisfying answers... > > > >>Some things I noticed while writing it: >> >> * views do not appear to know their own name? >> >> > >Correct. Consider v.join(v2, v.shoesize). > Had considered it, but it still feels like named views should know their name. >> * you don't seem to be able to get at a definition >> for a sub-view >> from the parent unless there is a row in the >> parent? >> >> > >You can get it from the storage. > This is actually one of the reasons I wanted the views to know their names. Consider the case where a regular named view is passed to a function, it can't get the information on the sub-view (as far as I know) without knowing the name of the parent view (can it?) >> * there are no StorageType or PropertyType objects >> in the metakit >> namespace >> >> > >They are easy to expose. I never saw a need. > Just makes it possible to easily switch utility code on the type of object passed to the utility. ... >> * storage doesn't seem to have any way to tell you: >> o whether it's in auto-commit mode >> o what file it's stored in (if any) >> >> > >Again, just wrapping the C++ object. Filename >is something I always end up dealing with, so >I'll think about adding that to the Python wrapper. > > Appreciated. This is very much just a penchant for introspection APIs on my part, not a critical need for any current project. >> * views don't seem to be able to tell you whether >> they are >> hashed/blocked/ordered >> >> > >Heh. In fact, "ordered" is how you tell MK that >the view is ordered. (Believe it or not, I have still >never used hashed or blocked). > >Because views are composited out of other views >(with differing capabilities), there are a gazillion >flavors. The Python types are a *manufactured* type >system designed to let you know what the general >capabilities of a view are. As far as wrapping goes, >there's only one C++ base class (and only one set >of Python methods). All the other Python view types >are created by cloning and using whiteout. That and >some "what do you get when you cross a X with a Y" >rules to decide what type object to point to. > Hmm, thanks for the summary. I hadn't realised there were that many view types, nor that they were created at run-time. >-- Gordon >http://www.mcmillan-inc.com/ > > Thanks again, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From tom.krehbiel@motorola.com Tue Nov 12 09:14:29 2002 Received: from motgate.mot.com (motgate.mot.com [129.188.136.100]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gACFETa22356 for ; Tue, 12 Nov 2002 09:14:29 -0600 Received: from pobox4.mot.com (pobox4.mot.com [10.64.251.243]) by motgate.mot.com (Motorola/Motgate) with ESMTP id gACFFLGE026177 for ; Tue, 12 Nov 2002 08:15:21 -0700 (MST) Received: [from az33exm25.corp.mot.com (az33exm25.corp.mot.com [10.64.65.157]) by pobox4.mot.com (MOT-pobox4 2.0) with ESMTP id IAA25351 for ; Tue, 12 Nov 2002 08:15:21 -0700 (MST)] Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exm25.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2656.59) id V7XCAAF7; Tue, 12 Nov 2002 08:15:20 -0700 Message-ID: <3DD11B2C.3EB3BC6A@motorola.com> Date: Tue, 12 Nov 2002 08:15:56 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 To: "Mike C. Fletcher" CC: Metakit List Subject: Re: [Metakit] Use more databases, or larger databases? References: <3DCBD911.E0A65116@motorola.com> <3DCCE4C9.6020201@rogers.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mike, > Hi Tom, > > I'm actually looking at the aspell code (aspell's a few generations > beyond ispell (via pspell IIUC))... well, not really the code so much as > the documentation for the code. Suffix-reduction isn't something I'm > currently doing, as it's likely to lose such things as: > > liftd -> lifted It sounds like you understand this problem better than I do (it is interesting :-). I use nedit as my primary text editor but I'm dislexic and it doesn't have a built in spelling checker so I added a spelling checker dialog and run ispell on the text in the editor, hightlight the next mispelled word and show a suggestion list. I thought you might be interested to know that when I entered 'liftd' it provides the following suggestions: lift lifted lifts > > As the "d" wouldn't likely be seen as a suffix by the reducer, but the > "ed" would be. > > Thanks for the pointer, though, I may take a look to see what they were > using it for, > Mike > > Tom Krehbiel wrote: > > >Mike, > > > >I missed the rest of this thread so I may be way of base with the following comments. I sounds like your creating a word index of > >miss spelled words. If so I would recommend you look at the ispell code (very old). They use a word dictionary that includes a > >suffix rule for each word. The suffix rule is used to create word variations. The word list is also hashed. > > > >Tom K. > > > > > ... From mcfletch@rogers.com Tue Nov 12 11:53:20 2002 Received: from fep02-mail.bloor.is.net.cable.rogers.com (fep02-mail.bloor.is.net.cable.rogers.com [66.185.86.72]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gACHrKa04454 for ; Tue, 12 Nov 2002 11:53:20 -0600 Received: from rogers.com ([24.43.65.252]) by fep02-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021112175407.JKBH4594.fep02-mail.bloor.is.net.cable.rogers.com@rogers.com>; Tue, 12 Nov 2002 12:54:07 -0500 Message-ID: <3DD1400A.8010401@rogers.com> Date: Tue, 12 Nov 2002 12:53:14 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Tom Krehbiel CC: Metakit List Subject: Re: [Metakit] Use more databases, or larger databases? References: <3DCBD911.E0A65116@motorola.com> <3DCCE4C9.6020201@rogers.com> <3DD11B2C.3EB3BC6A@motorola.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep02-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Tue, 12 Nov 2002 12:54:07 -0500 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Here's what my current code (with a very primitive ranking algorithm comes up with) running on the standard-English (no American or British-specific words) aspell dictionary produces: Source: liftd Suggest: RankedSet(11): lifts lifted lift livid lived lifter lift's lifting lifters lifter's liverwort (That 11 is the number of similar words at phonetic-edit-distance == 1) Source: firgoden Suggest: RankedSet(1): forgotten Source: missurey Suggest: RankedSet(1): misery The ranker I'm currently using just lumps together everything a given edit-distance (measured without phonetic compression) away, where eventually I'm going to use something like aspell's weighted distances (which distinguish between the weight of a delete, particular sets of characters that get commonly over-typed, etceteras). That should make lifter, lift's, lifting and lifters show up before livid and lived (particularly I'm considering measuring lengths of common sub-strings as a decent ranker, though that can really fall down for entirely phonetic matches (such as firgoden and missurey above) if there's a lot of similar textual matches). Oh, just for fun, here's what it gets in "bad speller mode" (phonetic distance=3 search): Source: liftd Suggest: RankedSet(473): lifts lifted lift ltd livid lived lits lite lit lists listed list lipid linty lints linted lint lined limed lilts Basically, without a decent ranker the "good" suggestions get lost in the crowd there. Will have to look at the ispell code some day to see what they're doing with their "suffix reduction" (esp. since it doesn't seem to be as naive as the phrase sounds). Seems likely it's part of a compression scheme (similar to phonetic compression) to make similar words equate. Enjoy all, Mike Tom Krehbiel wrote: >Mike, > > ... >I use nedit as my primary text editor but I'm >dislexic and it doesn't have a built in spelling checker so I added a spelling checker dialog and run ispell on the text in the >editor, hightlight the next mispelled word and show a suggestion list. I thought you might be interested to know that when I entered >'liftd' it provides the following suggestions: > lift > lifted > lifts > > ... >>Thanks for the pointer, though, I may take a look to see what they were >>using it for, >>Mike >> >> ... _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From mike.collins@riley.army.mil Tue Nov 12 13:11:27 2002 Received: from rildoimml02.riley.army.mil (rildoimml02.riley.army.mil [144.246.27.62]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gACJBQa11413 for ; Tue, 12 Nov 2002 13:11:26 -0600 Received: by rildoimml02.riley.army.mil with Internet Mail Service (5.5.2653.19) id ; Tue, 12 Nov 2002 13:13:07 -0600 Message-ID: <464A43842A2EF54F9183D07E0DEC43564D066C@rildoimml05> From: "Collins, Mike DAC DOIM" To: "'metakit@equi4.com'" Date: Tue, 12 Nov 2002 13:11:10 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Tequila Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude did I see somewhere that Tequila is now included with Metakit? From dave@davidcook.org Wed Nov 13 19:15:42 2002 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gAE1Fga16256 for ; Wed, 13 Nov 2002 19:15:42 -0600 Received: from ip68-8-139-9.sd.sd.cox.net (HELO localhost.localdomain) (david?martin?cook@68.8.139.9 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 14 Nov 2002 01:16:35 -0000 Date: Wed, 13 Nov 2002 17:16:35 -0800 From: "David M. Cook" To: metakit@equi4.com Message-ID: <20021114011635.GA25711@davidcook.org> Mail-Followup-To: "David M. Cook" , metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Subject: [Metakit] Mk4py import error on RH8. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I see that someone already had this problem with the TCL binding. After building and installing metakit on a stock RH8 system I get Python 2.2.1 (#1, Aug 30 2002, 12:15:30) [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import Mk4py Traceback (most recent call last): File "", line 1, in ? ImportError: ./Mk4py.so: undefined symbol: _ZTVN10__cxxabiv117__class_type_infoE I tried building with -fno-rtti and get Python 2.2.1 (#1, Aug 30 2002, 12:15:30) [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import Mk4py Traceback (most recent call last): File "", line 1, in ? ImportError: ./Mk4py.so: undefined symbol: __gxx_personality_v0 Which looks even stranger. Dave Cook From mcfletch@rogers.com Wed Nov 13 21:24:32 2002 Received: from fep01-mail.bloor.is.net.cable.rogers.com (fep01-mail.bloor.is.net.cable.rogers.com [66.185.86.71]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAE3OWa25842 for ; Wed, 13 Nov 2002 21:24:32 -0600 Received: from rogers.com ([24.43.65.252]) by fep01-mail.bloor.is.net.cable.rogers.com (InterMail vM.5.01.05.06 201-253-122-126-106-20020509) with ESMTP id <20021114032519.WKBN4718.fep01-mail.bloor.is.net.cable.rogers.com@rogers.com> for ; Wed, 13 Nov 2002 22:25:19 -0500 Message-ID: <3DD31794.5010109@rogers.com> Date: Wed, 13 Nov 2002 22:25:08 -0500 From: "Mike C. Fletcher" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH PLAIN at fep01-mail.bloor.is.net.cable.rogers.com from [24.43.65.252] using ID at Wed, 13 Nov 2002 22:25:19 -0500 Subject: [Metakit] Add drop a table/view to the documentation? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I just came across a recipe (haven't tried it yet) for deleting a view: storage.getas( 'viewname' ) wondering if the summary text for getas could include a note that you can also delete using the method. Just a thought, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From gregorb@clear.net.nz Sun Nov 17 14:17:46 2002 Received: from smtp2.clear.net.nz (smtp2.clear.net.nz [203.97.37.27]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAHKHka25805 for ; Sun, 17 Nov 2002 14:17:46 -0600 Received: from clear.net.nz (203-167-169-45.dialup.clear.net.nz [203.167.169.45]) by smtp2.clear.net.nz (CLEAR Net Mail) with ESMTPA id <0H5Q00D0OLR40M@smtp2.clear.net.nz> for metakit@equi4.com; Mon, 18 Nov 2002 09:18:42 +1300 (NZDT) Date: Mon, 18 Nov 2002 09:18:11 +1300 From: Gregor Bruce Subject: Re: [Metakit] Mk4tcl - Windows crash on exit bug To: metakit@equi4.com Message-id: <3DD7F983.9080600@clear.net.nz> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7bit X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.1b) Gecko/20020721 References: <1FF1C669-F34D-11D6-A337-00050251EF2F@equi4.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Gregor Bruce wrote: > >> I have a similar problem using 2.4.3. To avoid crash on exit I >> comment out a line in mk4tcl.cpp: >> >> static void ExitProc(ClientData cd_) >> { >> // avoid exit crash from shell >> // delete (MkWorkspace*) cd_; >> } > > > You're in essence turning off automatic cleanup. Things like having a > commit done automatically on exit will be disabled, I assume (if not, > PLEASE let me know, the there might be a case of double-cleanup!). > > Apart from the fact that you should really upgrade from all versions > 2.4.2..2.4.6 because of a serious bug (see change log), I would really > like to know whether this problem has been resolved in the latest > builds. So if you have a chance to test against 2.4.8, that would be > grand... > > -jcw Well I updated to 2.4.8 over the weekend and so far the crash-on-exit has not occured. Previously it was occuring every time the programme exited. AGB From lev@oduurl.ru Tue Nov 19 23:34:10 2002 Received: from odusv.oduurl.ru (odusv.oduurl.ru [195.12.73.193] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gAK5Y5a05453 for ; Tue, 19 Nov 2002 23:34:09 -0600 Received: from [10.0.0.16] by odusv.oduurl.ru with SMTP id KAA07052 (8.6.12 for ); Wed, 20 Nov 2002 10:31:34 +0500 Message-Id: <200211200531.KAA07052@odusv.oduurl.ru> To: "metakit@equi4.com" Date: Wed, 20 Nov 02 10:34:30 -0500 From: "Ilja V.Levinson" X-Mailer: E-Mail Connection v2.5.03 Subject: [Metakit] Aside feature question Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello All, first, I would like to introduce a new port of the Metakit library. It was successfully compiled and installed on the OS-9/68k (http://www.microware.com) system. Note - it's a real time operating system, not Mac related OS! Anyhow, I have some question about it, especially related to the 'Storage Aside' feature. We plan to use a ROM-based (i.e. Read-Only Memory, like BIOS flash ROM in PC world) mk4-database using LoadFromStream strategy. Also we have small (very small) disk storage, and want to use it for aside operations. Unfortunatelly, aside strategy uses very much assitional space. For example, for small database (10000 records, ~200K) changing one field in one row requires about 20K (10% of the full data size!) for *each*. Adding one record uses full database size (100%!). Is this a normal behaviour (or just my errors)? May be, I should manually use some Differ() methods? Thanks in advance, Ilja P.S. Here is the code: ----------------- c4_Storage storage ("myfile.dat", true); //main storage (read-only) c4_Storage bkp("add.dat", 1); //additional storage.SetAside(bkp); //example only, ~10000 records c4_View vAddress = storage.GetAs("address[name:S,country:S,age:I]"); pAge (vAddress[2]) = 55; //change one field in one row storage.Commit(0); bkp.Commit(0); ------------------------ code end -- Ilja V.Levinson, Yekaterinburg, Russia OS-9 Software http://www.os9samba.com --------------------------------------------- * Great talkers are never great doers. From jcw@equi4.com Wed Nov 20 04:13:47 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAKADka23360 for ; Wed, 20 Nov 2002 04:13:46 -0600 Date: Wed, 20 Nov 2002 11:14:02 +0100 Content-Type: text/plain; charset=US-ASCII; format=flowed Resent-Date: Wed, 20 Nov 2002 11:14:44 +0100 Content-Transfer-Encoding: 7bit Resent-Message-Id: Resent-To: MetaKit mailing list Mime-Version: 1.0 (Apple Message framework v548) Subject: Re: [Metakit] Aside feature question From: Jean-Claude Wippler To: Starkit list Message-Id: Resent-From: Jean-Claude Wippler X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ilja V.Levinson wrote: > Hello All, > > first, I would like to introduce a new port of the Metakit > library. It was successfully compiled and installed on > the OS-9/68k (http://www.microware.com) system. > > Note - it's a real time operating system, not Mac related OS! Great - congratulations! Did you have to make any changes? Anything that needs to be folded back into the basic distribution so you can rebuild future versions more easily? > [...] Unfortunatelly, aside strategy uses very much assitional space. > [...] Yes. The design of commit aside was to create a small difference set, but the code actually track differences and come up with a good small set of "change descriptions" has not been implemented so far. Commit aside was the spin-off of a project which was never fully completed, and for a long long time my focus has been to just get the new 2.3/2.4 file format and the rest of the code rock-solid again. > for small database (10000 records, ~200K) changing one field in one > row requires about 20K (10% of the full data size!) for *each*. Adding > one record uses full database size (100%!). The mechanism to store things efficiently is there. But right now, MK stores entire columns as "change sets", instead of offset/data pairs to describe differences. Finishing this code is not hard, but nevertheless a sizeable amount fo work to get right. I'm not sure when I'll be able to get to this. Then again, I do tend to accept paid work commissioned to me / Equi4 Software > May be, I should manually use some Differ() methods? You could store view insert/delete/modify requests and replay them on open - but that approach may lead to relatively high memory use, as changes cause all affected views to be loaded. -jcw -jcw From jyl@best.com Wed Nov 20 09:54:23 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gAKFsNa12428 for ; Wed, 20 Nov 2002 09:54:23 -0600 Received: (qmail 14921 invoked by uid 19667); 20 Nov 2002 15:55:21 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 20 Nov 2002 15:55:21 -0000 Message-ID: <3DDBB356.7040701@best.com> Date: Wed, 20 Nov 2002 08:07:50 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: "Ilja V.Levinson" CC: "metakit@equi4.com" Subject: Re: [Metakit] Aside feature question References: <200211200531.KAA07052@odusv.oduurl.ru> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ilya If you want to use commit-aside, you need disk space that is big enough for at least one full save of the storage plus a number of increments for the commit-aside saved data. For background informatin, I believe the amount of disk space used is proportional to the number of commit-aside operations you do, that's the first order factor, and secondarily it's related to how many views you modify. So if you commit-aside after every modification then it's likely to eat up disk space, because a copy of at least one view will be saved every commit-aside. I'm not sure whether it's possible to, or how to, tell Metakit to discard older commit-aside save points. Jean-Claude, is that possible? If so, what is the API? --JYL Ilja V.Levinson wrote: >Hello All, > >first, I would like to introduce a new port of the Metakit >library. It was successfully compiled and installed on >the OS-9/68k (http://www.microware.com) system. > >Note - it's a real time operating system, not Mac related OS! > >Anyhow, I have some question about it, especially related to the >'Storage Aside' feature. We plan to use a ROM-based (i.e. Read-Only Memory, >like BIOS flash ROM in PC world) mk4-database using LoadFromStream >strategy. Also we have small (very small) disk storage, and want to use it >for aside operations. > >Unfortunatelly, aside strategy uses very much assitional space. For example, >for small database (10000 records, ~200K) changing one field in one >row requires about 20K (10% of the full data size!) for *each*. Adding >one record uses full database size (100%!). > >Is this a normal behaviour (or just my errors)? >May be, I should manually use some Differ() methods? > >Thanks in advance, > Ilja > >P.S. Here is the code: ----------------- > > c4_Storage storage ("myfile.dat", true); //main storage (read-only) > c4_Storage bkp("add.dat", 1); //additional > storage.SetAside(bkp); > > //example only, ~10000 records > c4_View vAddress = storage.GetAs("address[name:S,country:S,age:I]"); > > pAge (vAddress[2]) = 55; //change one field in one row > storage.Commit(0); > bkp.Commit(0); > >------------------------ code end > >-- >Ilja V.Levinson, Yekaterinburg, Russia >OS-9 Software http://www.os9samba.com >--------------------------------------------- >* Great talkers are never great doers. > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From jeff_godfrey@pobox.com Wed Nov 20 13:00:10 2002 Received: from hewey.iland.net (qmailr@hewey.iland.net [205.242.230.200]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gAKJ0Aa27914 for ; Wed, 20 Nov 2002 13:00:10 -0600 Received: (qmail 10424 invoked from network); 20 Nov 2002 19:00:59 -0000 Received: from wbrg-dialup-216-74-200-203.iland.net (HELO quantex) (216.74.200.203) by hewey.iland.net with SMTP; 20 Nov 2002 19:00:59 -0000 Message-ID: <001e01c290c7$1d769ec0$0180d241@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: Date: Wed, 20 Nov 2002 13:00:11 -0600 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0016_01C29094.C274F6C0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] metakit - persistent *AND* temporary storage Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0016_01C29094.C274F6C0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi All, I am using Metakit as the database behind a tcl application (through = Mk4tcl). I have written some simple API functions to access and = manipulate the database from the tcl level. I now have the need to = access some additional *external* data while the program is running. = This data is stored in, and loaded through a tcl source file. Rather = than write new procedures to access and manipulate this data, I thought = it would be nice to just plug it in as an additional view of my existing = Metakit database at run-time and access it using my existing API = functions. To me, this makes good sense, except that I don't want the = additional data permanently stored in the database. Is this doable, and if so, what's the simplest way? Ahhhh, it just dawned on me... Put the data in a *new* database opened = with "-nocommit" and don't explicitly commit it anywhere in my code? If = that's the best plan, I need to change my API functions to accept a = database tag, as they currently do not. Is there any way to do this = with just a single database? Thanks, Jeff ------=_NextPart_000_0016_01C29094.C274F6C0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi All,
 
I am using Metakit as the database = behind a tcl=20 application (through Mk4tcl).  I have written some simple API = functions to=20 access and manipulate the database from the tcl level.  I now = have the=20 need to access some additional *external* data while the program is=20 running.  This data is stored in, and loaded through a tcl source=20 file.  Rather than write new procedures to access and manipulate = this data,=20 I thought it would be nice to just plug it in as an = additional view of=20 my existing Metakit database at run-time and access it using my existing = API=20 functions.  To me, this makes good sense, except that I don't want = the=20 additional data permanently stored in the database.
 
Is this doable, and if so, what's the = simplest=20 way?
 
Ahhhh, it just dawned on me... Put the = data in a=20 *new* database opened with "-nocommit" and don't explicitly commit it = anywhere=20 in my code?  If that's the best plan, I need to change my API = functions to=20 accept a database tag, as they currently do not.  Is there any way = to do=20 this with just a single database?
 
Thanks,

Jeff
------=_NextPart_000_0016_01C29094.C274F6C0-- From mbloore@yahoo.com Wed Nov 20 15:16:00 2002 Received: from web14301.mail.yahoo.com (web14301.mail.yahoo.com [216.136.173.77]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gAKLG0a05674 for ; Wed, 20 Nov 2002 15:16:00 -0600 Message-ID: <20021120211658.55438.qmail@web14301.mail.yahoo.com> Received: from [65.95.158.51] by web14301.mail.yahoo.com via HTTP; Wed, 20 Nov 2002 13:16:58 PST Date: Wed, 20 Nov 2002 13:16:58 -0800 (PST) From: mARK bLOORE Subject: Re: [Metakit] metakit - persistent *AND* temporary storage To: metakit In-Reply-To: <001e01c290c7$1d769ec0$0180d241@quantex> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Jeff Godfrey wrote: > Ahhhh, it just dawned on me... Put the data in a *new* database > opened with "-nocommit" and don't explicitly commit it anywhere in my > code? If that's the best plan, I need to change my API functions to > accept a database tag, as they currently do not. Is there any way to > do this with just a single database? it might work to make the second file an aside file (c4_Storage::SetAside). i think you have to create it as such to begin with, and commit a new view in it. after that the combination of base and aside files should work like a single file, but show all the views from both. ===== -- mARK bLOORE __________________________________________________ Do you Yahoo!? Yahoo! Web Hosting - Let the expert host your site http://webhosting.yahoo.com From jcw@equi4.com Wed Nov 20 16:54:49 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAKMsma12360 for ; Wed, 20 Nov 2002 16:54:48 -0600 Date: Wed, 20 Nov 2002 23:55:47 +0100 Subject: Re: [Metakit] metakit - persistent *AND* temporary storage Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v548) From: Jean-Claude Wippler To: metakit Content-Transfer-Encoding: 7bit In-Reply-To: <20021120211658.55438.qmail@web14301.mail.yahoo.com> Message-Id: <35053CB0-FCDB-11D6-A2B4-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: > --- Jeff Godfrey wrote: >> Ahhhh, it just dawned on me... Put the data in a *new* database >> opened with "-nocommit" and don't explicitly commit it anywhere in my >> code? If that's the best plan, I need to change my API functions to >> accept a database tag, as they currently do not. Is there any way to >> do this with just a single database? > > it might work to make the second file an aside file > (c4_Storage::SetAside). i think you have to create it as such to begin > with, and commit a new view in it. after that the combination of base > and aside files should work like a single file, but show all the views > from both. The commit aside trick is one way to prevent changes from reaching the original data file, but if you don't want to make the data persist at all it is probably simpler to open in -nocommit mode (or even read-only!) and then make sure no commits are done at all. In separate db's things are even simpler: just open a db without file name. It won't persist (though you can still stream it out - save to file - if you want after all). In the same db would be trivial in C++, just store data in a subview which is not defined in the structure. That data will live only as long as the storage object is open. In Tcl/Python, you can do the same for simple data values - store info in a property which has not been declared part of the layout/structure. For subviews, things get a bit trickier, but I think the following should work - I'll use Tcl as example since you do: mk::file open db blah.dat mk::view layout db.names {first last} # ... here you have a view with two string properties mk::view layout db.names {first last {phones {type number}}} # ... now you also have a phones subview mk::view layout db.names {first last} # ... not anymore, BUT WAIT THAT'S NOT TRUE... After these lines you'll still see a phones subview in there. So you can do things like store things in db.names!123.phones, etc. The moment this ceases to exist is the moment you commit. Properties get added upon use (temporary) and upon layout/structure changes (permanent). There is no way to have temporary properties which outlives a commit. On commit, all extraneous data is dropped, all buffers are cleared, and all access is made to come from disk again. So if you want to keep temp data around independent of commits on the main data file, the best bet is probably to maintain a parallel in-memory view. It does everything a normal stored one does, including layout definitions and changes, but you cannot commit it. As for the "need to change my API functions to accept a database tag" - yes, you may need to fdo this, but note that if you're passing view names, then the database can simply be passed in front, i.e. instead of "names" you'd pass in "db.names", "tempdb.tames", etc. -jcw From mbloore@yahoo.com Wed Nov 20 18:48:59 2002 Received: from web14303.mail.yahoo.com (web14303.mail.yahoo.com [216.136.173.79]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gAL0mxa19371 for ; Wed, 20 Nov 2002 18:48:59 -0600 Message-ID: <20021121004958.42168.qmail@web14303.mail.yahoo.com> Received: from [65.95.158.51] by web14303.mail.yahoo.com via HTTP; Wed, 20 Nov 2002 16:49:58 PST Date: Wed, 20 Nov 2002 16:49:58 -0800 (PST) From: mARK bLOORE Subject: Re: [Metakit] metakit - persistent *AND* temporary storage To: metakit In-Reply-To: <20021120211658.55438.qmail@web14301.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- mARK bLOORE wrote: > it might work to make the second file an aside file i just re-read your message, and realized that this won't do what you want. i take it that the idea is to commit changes to the main file but not the second one. that is just the opposite of how an aside file works; if you make changes to a view that came from the main file they will go into the aside file. i suppose you could try a nasty hack: open the second file as the main one, and the first file as the aside file. but i suppose that you don't always have the second file, and (as mentioned here recently) aside files tend to grow very fast. also, as i recall, aside files can't be used with just any main file; it has to be the same one each time. ===== -- mARK bLOORE __________________________________________________ Do you Yahoo!? Yahoo! Web Hosting - Let the expert host your site http://webhosting.yahoo.com From lev@oduurl.ru Wed Nov 20 21:55:02 2002 Received: from odusv.oduurl.ru (odusv.oduurl.ru [195.12.73.193] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gAL3t1a29289 for ; Wed, 20 Nov 2002 21:55:01 -0600 Received: from [10.0.0.16] by odusv.oduurl.ru with SMTP id IAA14062 (8.6.12 for ); Thu, 21 Nov 2002 08:44:26 +0500 Message-Id: <200211210344.IAA14062@odusv.oduurl.ru> To: "metakit@equi4.com" Date: Thu, 21 Nov 02 08:47:25 -0500 From: "Ilja V.Levinson" X-Mailer: E-Mail Connection v2.5.03 Subject: [Metakit] Re: metakit digest, Vol 1 #302 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello : thanks to all for your useful replies. > Message: 2 > From: Jean-Claude Wippler > > first, I would like to introduce a new port of the Metakit > > library. It was successfully compiled and installed on > > the OS-9/68k (http://www.microware.com) system. > > > > Note - it's a real time operating system, not Mac related OS! > > Great - congratulations! > > Did you have to make any changes? Anything that needs to be folded > back into the basic distribution so you can rebuild future versions > more easily? The OS-9 package is not jet fully tested. Anyhow, for successful compilation I've made the following changes: -- OS9\config.h -- #define HAVE_MMAP 0 #define HAVE_MEMMOVE 1 #define HAVE_BCOPY 0 #define SIZEOF_LONG 4 #define HAVE_LONG_LONG 0 #define q4_UNIX 1 #define q4_OS9 1 #define q4_TINY 1 //just to decrease library size --------------------- -- src\fileio.c diff ---- 30c30 < #if q4_UNIX && !q4_OS9 --- > #if q4_UNIX 202c202 < #if q4_UNIX && !q4_OS9 --- > #if q4_UNIX 223c223 < #if q4_UNIX && !q4_OS9 --- > #if q4_UNIX --------------------------- That's all. -- Ilja V.Levinson, Yekaterinburg, Russia OS-9 Software http://www.os9samba.com --------------------------------------------- * Great talkers are never great doers. From jyl@best.com Wed Nov 20 23:21:38 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gAL5Lba00549 for ; Wed, 20 Nov 2002 23:21:38 -0600 Received: (qmail 16987 invoked by uid 19667); 21 Nov 2002 05:22:36 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 21 Nov 2002 05:22:36 -0000 Message-ID: <3DDC7088.9090504@best.com> Date: Wed, 20 Nov 2002 21:35:04 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Subject: Re: [Metakit] metakit - persistent *AND* temporary storage References: <35053CB0-FCDB-11D6-A2B4-00050251EF2F@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: At the C++ level how do you open a storage without creating a disk file? --JYL Jean-Claude Wippler wrote: > mARK bLOORE wrote: > >> --- Jeff Godfrey wrote: >> >>> Ahhhh, it just dawned on me... Put the data in a *new* database >>> opened with "-nocommit" and don't explicitly commit it anywhere in my >>> code? If that's the best plan, I need to change my API functions to >>> accept a database tag, as they currently do not. Is there any way to >>> do this with just a single database? >> >> >> it might work to make the second file an aside file >> (c4_Storage::SetAside). i think you have to create it as such to begin >> with, and commit a new view in it. after that the combination of base >> and aside files should work like a single file, but show all the views >> from both. > > > The commit aside trick is one way to prevent changes from reaching the > original data file, but if you don't want to make the data persist at > all it is probably simpler to open in -nocommit mode (or even > read-only!) and then make sure no commits are done at all. > > In separate db's things are even simpler: just open a db without file > name. It won't persist (though you can still stream it out - save to > file - if you want after all). > > In the same db would be trivial in C++, just store data in a subview > which is not defined in the structure. That data will live only as > long as the storage object is open. > > In Tcl/Python, you can do the same for simple data values - store info > in a property which has not been declared part of the layout/structure. > > For subviews, things get a bit trickier, but I think the following > should work - I'll use Tcl as example since you do: > > mk::file open db blah.dat > mk::view layout db.names {first last} > # ... here you have a view with two string properties > mk::view layout db.names {first last {phones {type number}}} # > ... now you also have a phones subview > mk::view layout db.names {first last} > # ... not anymore, BUT WAIT THAT'S NOT TRUE... > > After these lines you'll still see a phones subview in there. So you > can do things like store things in db.names!123.phones, etc. The > moment this ceases to exist is the moment you commit. Properties get > added upon use (temporary) and upon layout/structure changes (permanent). > > There is no way to have temporary properties which outlives a commit. > On commit, all extraneous data is dropped, all buffers are cleared, > and all access is made to come from disk again. > > So if you want to keep temp data around independent of commits on the > main data file, the best bet is probably to maintain a parallel > in-memory view. It does everything a normal stored one does, > including layout definitions and changes, but you cannot commit it. > > As for the "need to change my API functions to accept a database tag" > - yes, you may need to fdo this, but note that if you're passing view > names, then the database can simply be passed in front, i.e. instead > of "names" you'd pass in "db.names", "tempdb.tames", etc. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jcw@equi4.com Thu Nov 21 02:31:22 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAL8VMa09090 for ; Thu, 21 Nov 2002 02:31:22 -0600 Date: Thu, 21 Nov 2002 09:32:21 +0100 Subject: Re: [Metakit] metakit - persistent *AND* temporary storage Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v548) From: Jean-Claude Wippler To: metakit Content-Transfer-Encoding: 7bit In-Reply-To: <3DDC7088.9090504@best.com> Message-Id: X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > At the C++ level how do you open a storage without creating a disk > file? c4_Storage s; or with an existing storage, replacing it and dropping ref to previous one: s = c4_Storage (); -jcw From jeff_godfrey@pobox.com Thu Nov 21 08:04:04 2002 Received: from dewey.iland.net (dewey.iland.net [205.242.230.201]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gALE44a26243 for ; Thu, 21 Nov 2002 08:04:04 -0600 Received: (qmail 11793 invoked from network); 21 Nov 2002 14:05:01 -0000 Received: from wbrg-dialup-216-74-200-43.iland.net (HELO quantex) (216.74.200.43) by dewey.iland.net with SMTP; 21 Nov 2002 14:05:01 -0000 Message-ID: <003301c29166$e6f4f4a0$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "metakit" References: <35053CB0-FCDB-11D6-A2B4-00050251EF2F@equi4.com> Subject: Re: [Metakit] metakit - persistent *AND* temporary storage Date: Thu, 21 Nov 2002 08:03:59 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Jean-Claude Wippler" To: "metakit" Sent: Wednesday, November 20, 2002 4:55 PM Subject: Re: [Metakit] metakit - persistent *AND* temporary storage > So if you want to keep temp data around independent of commits on the > main data file, the best bet is probably to maintain a parallel > in-memory view. It does everything a normal stored one does, including > layout definitions and changes, but you cannot commit it. Hmmmmm.... This seems to be what I *thought* I wanted (I've since changed my mind - see below), but I'm still not sure how to go about it. How does one create a "parallel in-memory view"? Basically, what I wanted was the ability to add an additional view to my database that would never be committed but instead, would just disappear into the mist when the program was shut down. It seems that I can do this with an additional database *file*, but if I can do it by adding a additional *view* to an existing database, I'm not sure how. Also, currently the writes to my database file are immediately committed, if that's a consideration with any solution similar to the above... > As for the "need to change my API functions to accept a database tag" - > yes, you may need to fdo this, but note that if you're passing view > names, then the database can simply be passed in front, i.e. instead of > "names" you'd pass in "db.names", "tempdb.tames", etc. Good call! I am *already* passing view names as you describe "db.myview", so the database tag thing I mentioned is completely unnecessary. I wonder how long it would have taken me to realize that? Probably about the time I wrote some code like this: "myUpdatedAPIfunction db db.tag $myData" .....Ummmmm, wait a minute, why do I need "db" in there twice? ;^) Anyway, as mentioned above, I have kind of changed directions. After some thought, I've decided there is no reason I can't just permanently store the data from this external file in the Metakit database as needed - so, problem solved. Thanks for the info provide though, and I would *still* like to know how to add a *temporary* view to an existing database, if it is possible... Jeff From jeff_godfrey@pobox.com Thu Nov 21 08:06:09 2002 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gALE68a26385 for ; Thu, 21 Nov 2002 08:06:08 -0600 Received: (qmail 21975 invoked from network); 21 Nov 2002 14:07:08 -0000 Received: from wbrg-dialup-216-74-200-43.iland.net (HELO quantex) (216.74.200.43) by chewey.iland.net with SMTP; 21 Nov 2002 14:07:08 -0000 Message-ID: <003601c29167$34b60bc0$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "mARK bLOORE" , "metakit" References: <20021121004958.42168.qmail@web14303.mail.yahoo.com> Subject: Re: [Metakit] metakit - persistent *AND* temporary storage Date: Thu, 21 Nov 2002 08:06:31 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark, Thanks for the reply and the ideas. Since posting the question, I've changed my mind about necessity of making the data "temporary", so I really don't have a problem any more. See my response to JCW on the Metakit mailing list if your interested in more details... Thanks again, Jeff ----- Original Message ----- From: "mARK bLOORE" To: "metakit" Sent: Wednesday, November 20, 2002 6:49 PM Subject: Re: [Metakit] metakit - persistent *AND* temporary storage > > --- mARK bLOORE wrote: > > it might work to make the second file an aside file > > i just re-read your message, and realized that this won't do what you > want. > i take it that the idea is to commit changes to the main file but not > the second one. that is just the opposite of how an aside file works; > if you make changes to a view that came from the main file they will go > into the aside file. > i suppose you could try a nasty hack: open the second file as the main > one, and the first file as the aside file. but i suppose that you > don't always have the second file, and (as mentioned here recently) > aside files tend to grow very fast. also, as i recall, aside files > can't be used with just any main file; it has to be the same one each time. > > ===== > -- > mARK bLOORE From tdc@phreaker.net Sat Nov 23 00:12:17 2002 Received: from snickers.hotpop.com (snickers.hotpop.com [204.57.55.49]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAN6CGa18215 for ; Sat, 23 Nov 2002 00:12:17 -0600 Received: from phreaker.net (kubrick.hotpop.com [204.57.55.16]) by snickers.hotpop.com (Postfix) with SMTP id 19B9C72933 for ; Sat, 23 Nov 2002 06:12:45 +0000 (UTC) Received: from phreaker.net (unknown [194.149.99.36]) by smtp-2.hotpop.com (Postfix) with ESMTP id 1B9681B8B60 for ; Fri, 22 Nov 2002 12:53:16 +0000 (UTC) Message-ID: <3DDE2908.6090004@phreaker.net> Date: Fri, 22 Nov 2002 13:54:32 +0100 From: tdc User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020826 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: multipart/mixed; boundary="------------070509040702040700040700" X-HotPOP: ----------------------------------------------- Sent By HotPOP.com FREE Email Get your FREE POP email at www.HotPOP.com ----------------------------------------------- Subject: [Metakit] one (possible) BUG, one patch and some notices for MK 2.4.8 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------070509040702040700040700 Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Hi, I'm pretty new to MetaKit, but I have some troubles (at least with Python) with it from first try ;-) First the most painful thing: when using precompiled binary (Mk4py.so) downloaded from the web, i get SIGABORT when trying to dir(view). When i use my own compiled version, i get SIGSEGV: --- Python 2.2 (#1, Feb 24 2002, 16:21:58) [GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)] on linux-i386 Type "help", "copyright", "credits" or "license" for more information. >>> import metakit >>> db = metakit.storage('test.db',1) >>> users = db.view('users') >>> dir(users) (SIGSEGV) [user@host]$ _ --- It doesn't matter if i use 'getas' or 'view'.. Sure, there is no need to 'dir' a view, but that's the way i work and i think it SHOULD work. Is this some common thing (didn't found much info about python+mk in archives)?? Now the simple things: - In the source for MetaKit 2.4.8 there is one small glitch in Makefile.in (patch included) - In newer distributions (Mandrake, RedHat, possible others) there is usually Python.h header in python2.x/Python.h. Sure there are ways how to force use of the correct path, but for newbies, it can be confusing - Again, in new distros be sure to compile AND link the module with g++ and with the same version as python was compiled by. When python was compiled with 2.96 (as in mdk8.2) and you try to compile and link the Mk4py.so module with gcc-3.0.4 or g++-3.0.4 instead of g++-2.96, you'll fail to load it (example of linking with gcc-3.0.4): --- Python 2.2 (#1, Feb 24 2002, 16:21:58) [GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)] on linux-i386 Type "help", "copyright", "credits" or "license" for more information. >>> import metakit Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.2/site-packages/metakit.py", line 12, in ? from Mk4py import * ImportError: /usr/lib/python2.2/site-packages/Mk4py.so: undefined symbol: __gxx_personality_v0 >>> --- Bye, Dave --------------070509040702040700040700 Content-Type: text/plain; name="Makefile.in.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="Makefile.in.patch" --- Makefile.in.old Sat Nov 2 23:29:13 2002 +++ Makefile.in Fri Nov 22 13:17:35 2002 @@ -116,7 +116,7 @@ install-python: Mk4py@SHLIB_SUFFIX@ $(INSTALL_PROGRAM) Mk4py@SHLIB_SUFFIX@ $(DESTDIR)$(pylibdir) - $(INSTALL_PROGRAM) $(srcdir)/../python/metakit.py $$(DESTDIR)(pylibdir) + $(INSTALL_PROGRAM) $(srcdir)/../python/metakit.py $(DESTDIR)$(pylibdir) clean: $L rm -f *.la *.o *.lo --------------070509040702040700040700-- From sanford@selznick.com Tue Nov 26 15:18:18 2002 Received: from selznick.com (barracuda.pulvertech.com [64.72.162.2] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gAQLIIx18504 for ; Tue, 26 Nov 2002 15:18:18 -0600 Received: from [10.0.0.11] [169.197.48.235] by selznick.com with ESMTP (SMTPD32-7.07) id A54436390078; Tue, 26 Nov 2002 14:19:00 -0700 Mime-Version: 1.0 X-Sender: sanford@selznick.com@mail.webemporium.com Message-Id: Date: Tue, 26 Nov 2002 14:19:13 -0700 To: metakit@equi4.com From: Sanford Selznick Content-Type: text/plain; charset="us-ascii" ; format="flowed" Subject: [Metakit] Derived View + InsertAt() Crash Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The code below that uses MetaKit is crashing. JCW confirmed this as a bug in MetaKit and is working on fixing it. Apparently, modifying a view with InserAt after it's derived to another view is corrupting memory somehow. In any event, JCW recommended the following temporary fix: before performing the InsertAt(), reset the derived view with something like found_view = c4_View(); My question is, do I have to reset /all/ views derived from a c4_Storage object, or only clear the views derived from the same view that I'm inserting into? Thanks, Sanford PS - That crashing code... void QuickTest(void) { c4_Storage storage("c_test.dat", true); c4_View main_view = storage.GetAs("val:I"); c4_IntProp p("val"); c4_View found_view; { c4_Row row; // populate the database int intlist[] = { 0, 1, 2, 0, 1, 2, 0, 1, 2, 0, -1 }; for (int c = 0; -1 != intlist[c]; c++) { p(row) = intlist[c]; main_view.Add(row); } // find all "1" in list p(row) = 1; found_view = main_view.Select(row); cout << found_view.GetSize(); } { c4_Row row; // insert "6", a value into the _found_ list at index 2 p(row) = 6; c4_RowRef found_row = found_view.GetAt(2); // index 2 int main_insert = main_view.GetIndexOf(found_row); main_view.InsertAt(main_insert, row); } { c4_Row row; // try another insertion p(row) = 6; main_view.InsertAt(3, row); } // from dump.cpp ViewDisplay(found_view); ViewDisplay(main_view); } From njriley@theremin.cs.uiuc.edu Mon Dec 2 21:33:34 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB33XYx15853 for ; Mon, 2 Dec 2002 21:33:34 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id VAA2962289 for metakit@equi4.com; Mon, 2 Dec 2002 21:34:42 -0600 (CST) Date: Mon, 2 Dec 2002 21:34:41 -0600 From: Nicholas Riley To: metakit@equi4.com Message-ID: <20021203033441.GE2890677@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Subject: [Metakit] Can't use longs in Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I've been using and enjoying MetaKit on Mac OS X over the past couple of days, but have run into a problem. If I try to store a long int, I sometimes get an coercion error, and sometimes get -1. Here's an example that demonstrates the latter: % cat longtest.py import metakit x = metakit.storage() v = x.getas('hi[x:I,y:L]') v.append(x=221400115,y=221400115) v.append(x=221400115,y=2214001150) print v[0].x, v[0].y print v[1].x, v[1].y % python2.2 longtest.py 221400115 221400115 221400115 -1 I need to store large integers so this is somewhat of a problem. Any ideas what could be causing it? If I use an 'I' column instead of the 'L' column, I get this error: ValueError: int() literal too large: 2214001150 The coercion error is very strange, as the backtrace is incorrect and it occurs _after_ the column is set, which makes me suspect some kind of corruption at work. metric3 count 2214001150 L set Traceback (most recent call last): File "idvise.py", line 8, in ? ioms.addModelsFromIOMOX('~/Documents/Pablo/Idvisor/iomox-output') File "iom.py", line 31, in addModelsFromIOMOX [IOModel(self, os.path.join(self._iomoxDirPath, fn)) for fn in files] File "iom.py", line 116, in __init__ for node in iomodel.getElementsByTagName('profile')] File "iom.py", line 136, in __init__ models.addFragmentMetrics(IOFragment(fragmentNode), model, self) File "iom.py", line 69, in addFragmentMetrics for munit, mval in m.valDict.iteritems(): OverflowError: long int too large to convert to int Obviously the for loop is not causing this issue :-) In addition to OS X 10.2.2/Python 2.2.2 (compiled with GCC 3.1), I've also tested on Linux with Python 2.2.1 (compiled with GCC 2.95.4). I tested the Tcl binding as best I could - not being a Tcl person - and it seemed that long ints can successfully be stored with the Tcl binding. There weren't any type specifiers so it was hard to tell! Thanks, -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign From mikel@evins.net Tue Dec 3 23:34:28 2002 Received: from pimout3-ext.prodigy.net (pimout3-ext.prodigy.net [207.115.63.102]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB45YSx28377 for ; Tue, 3 Dec 2002 23:34:28 -0600 Received: from evins.net (adsl-66-127-237-50.dsl.sntc01.pacbell.net [66.127.237.50]) by pimout3-ext.prodigy.net (8.12.3 da nor stuldap/8.12.3) with ESMTP id gB45Za6X412324 for ; Wed, 4 Dec 2002 00:35:36 -0500 Date: Tue, 3 Dec 2002 21:35:37 -0800 Mime-Version: 1.0 (Apple Message framework v548) Content-Type: text/plain; charset=US-ASCII; format=flowed From: mikel evins To: metakit@equi4.com Content-Transfer-Encoding: 7bit Message-Id: <37E96494-074A-11D7-BD59-0003939376E6@evins.net> X-Mailer: Apple Mail (2.548) Subject: [Metakit] unable to use Mk4py on Mac OS X (jaguar, 10.2.2) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've tried dowloading the prebuilt .dylib and other files from the Metakit homepage. I've tried buildding them from source. I've tried using them with the python 2.2.1 that ships with Jaguar, and MachoPython, and a brand spanking new python 2.2.2 built from source. Nothing seems to work; invariably, nor matter where I put Mk4py.dylib (under whatver name), Python complains on importing metakit.py that the Module Mk4py does not exist or can't be loaded. I went so far as to reinstall Jaguar and subsequent updates in an attempt to address any possible misconfigurations. No dice. Anybody have any ideas? The corresponding install and usage on Windows 2000 was drop-dead easy. --me From njriley@theremin.cs.uiuc.edu Wed Dec 4 14:24:30 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB4KOTx30572 for ; Wed, 4 Dec 2002 14:24:29 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id OAA3038245; Wed, 4 Dec 2002 14:25:34 -0600 (CST) Date: Wed, 4 Dec 2002 14:25:34 -0600 From: Nicholas Riley To: mikel evins Cc: metakit@equi4.com Subject: Re: [Metakit] unable to use Mk4py on Mac OS X (jaguar, 10.2.2) Message-ID: <20021204202534.GB3050976@uiuc.edu> References: <37E96494-074A-11D7-BD59-0003939376E6@evins.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <37E96494-074A-11D7-BD59-0003939376E6@evins.net> User-Agent: Mutt/1.4i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Tue, Dec 03, 2002 at 09:35:37PM -0800, mikel evins wrote: > I've tried dowloading the prebuilt .dylib and other files from the > Metakit homepage. I've tried buildding them from source. I've tried > using them with the python 2.2.1 that ships with Jaguar, and > MachoPython, and a brand spanking new python 2.2.2 built from source. > > Nothing seems to work; invariably, nor matter where I put Mk4py.dylib > (under whatver name), Python complains on importing metakit.py that the > Module Mk4py does not exist or can't be loaded. Right - the makefile is passing GNU libtool the wrong options on OS X. There are some other problems too if you use a framework build (--enable-framework) of Python, which you probably didn't run into. The framework stuff is necessary if you want to be able to use Tkinter or wxPython. The attached patch should fix the problems with Python, and permit both Python and Tcl interfaces to work with framework builds. You need to use the Fink version of GNU libtool (or manually apply Fink's patches to the corresponding libtool version). Let me know if you have any questions... BTW, if you run into the long long problem I did with Mk4py (affects all platforms, not just Mac OS X), let me know and I can send you some preliminary patches. Mk4py's treatment of Python ints/longs is broken, and I am currently working on a more thorough patch that correctly handles overflows. -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign From njriley@theremin.cs.uiuc.edu Wed Dec 4 14:26:12 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB4KQBx30707 for ; Wed, 4 Dec 2002 14:26:11 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id OAA3024550; Wed, 4 Dec 2002 14:27:19 -0600 (CST) Date: Wed, 4 Dec 2002 14:27:19 -0600 From: Nicholas Riley To: mikel evins Cc: metakit@equi4.com Subject: Re: [Metakit] unable to use Mk4py on Mac OS X (jaguar, 10.2.2) Message-ID: <20021204202719.GC3050976@uiuc.edu> References: <37E96494-074A-11D7-BD59-0003939376E6@evins.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="xgyAXRrhYN0wYx8y" Content-Disposition: inline In-Reply-To: <37E96494-074A-11D7-BD59-0003939376E6@evins.net> User-Agent: Mutt/1.4i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --xgyAXRrhYN0wYx8y Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sorry, forgot to send the patch. (I realize I forgot to send my revised patch to Jean-Claude last time too, so here it is :-) -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign --xgyAXRrhYN0wYx8y Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="metakit-2.4.8+njr1.diff" --- metakit-2.4.8/unix/Makefile.in Sat Nov 2 16:29:13 2002 +++ metakit-2.4.8+njr1/unix/Makefile.in Thu Nov 28 16:33:56 2002 @@ -34,12 +34,15 @@ srcdir = @srcdir@ top_builddir = . -pyincludedir = @includedir@/python2.2 -pylibdir = @libdir@/python2.2/site-packages +pyincludedir = @PY_INCLUDE_DIR@ +pylibdir = @PY_LIB_DIR@ tclsh = tclsh tclincludedir = @TCL_INCLUDE_DIR@ -tcllibdir = @TCL_INCLUDE_DIR@/../lib +tcllibdir = @TCL_LIB_DIR@ + +CPPFLAGS = @CPPFLAGS@ +LDFLAGS = @LDFLAGS@ # Compiling without frame pointers can play tricks with exception handling # (e.g. in Mk4py). This does not affect standard operation, *only* errors. @@ -55,7 +58,8 @@ INSTALL_DATA = $L --mode=install @INSTALL_DATA@ INSTALL_PROGRAM = $L --mode=install @INSTALL_PROGRAM@ -LIBTOOL_FLAGS = @LIBTOOL_FLAGS@ +LIBTOOL_SHLIB_FLAGS = $(LDFLAGS) @LIBTOOL_SHLIB_FLAGS@ +LIBTOOL_MODULE_FLAGS = $(LDFLAGS) @LIBTOOL_MODULE_FLAGS@ STRIP_FLAGS = @STRIP_FLAGS@ @@ -63,20 +67,20 @@ #---------- Do not change, shorthand only -CXX_SWITCHES = $(CXXFLAGS) \ +CXX_SWITCHES = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(srcdir)/../src -I. -CXX_SWITCHES_TCL = $(CXXFLAGS) \ +CXX_SWITCHES_TCL = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(tclincludedir)/generic \ -I$(tclincludedir) -CXX_SWITCHES_PY = $(CXXFLAGS) \ +CXX_SWITCHES_PY = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(srcdir)/../python/scxx \ -I$(pyincludedir) -CXX_SWITCHES_LUA = $(CXXFLAGS) \ +CXX_SWITCHES_LUA = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include -CXX_SWITCHES_TEST = $(CXXFLAGS) \ +CXX_SWITCHES_TEST = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include #---------- The targets normally specified when calling "make" @@ -87,7 +91,7 @@ tcl: Makefile Mk4tcl@SHLIB_SUFFIX@ -python: Makefile Mk4py@SHLIB_SUFFIX@ +python: Makefile Mk4py@MODULE_SUFFIX@ test: Makefile libmk4.la regress test -d tests || mkdir tests @@ -114,9 +118,9 @@ $(INSTALL_PROGRAM) Mk4tcl@SHLIB_SUFFIX@ $(DESTDIR)$(tcllibdir)/Mk4tcl echo 'package ifneeded Mk4tcl 2.4.8 [list load [file join $$dir Mk4tcl@SHLIB_SUFFIX@] Mk4tcl]' >$(DESTDIR)$(tcllibdir)/Mk4tcl/pkgIndex.tcl -install-python: Mk4py@SHLIB_SUFFIX@ - $(INSTALL_PROGRAM) Mk4py@SHLIB_SUFFIX@ $(DESTDIR)$(pylibdir) - $(INSTALL_PROGRAM) $(srcdir)/../python/metakit.py $$(DESTDIR)(pylibdir) +install-python: Mk4py@MODULE_SUFFIX@ + $(INSTALL_PROGRAM) Mk4py@MODULE_SUFFIX@ $(DESTDIR)$(pylibdir) + $(INSTALL_PROGRAM) $(srcdir)/../python/metakit.py $(DESTDIR)$(pylibdir) clean: $L rm -f *.la *.o *.lo @@ -138,7 +142,7 @@ cd $(srcdir) && autoconf libmk4.la: $(LIBOBJS) $(LINK_SPECIAL_FILES) - $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_FLAGS) -avoid-version \ + $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_SHLIB_FLAGS) -avoid-version \ -rpath $(libdir) $(SHLOBJS) $(LINK_SPECIAL_FLAGS) Mk4tcl@SHLIB_SUFFIX@: libmk4tcl.la @@ -146,15 +150,15 @@ -strip $(STRIP_FLAGS) $@ 2>/dev/null libmk4tcl.la: mk4tcl.o mk4too.o $(LIBOBJS) $(LINK_SPECIAL_FILES) - $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_FLAGS) -avoid-version \ + $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_SHLIB_FLAGS) -avoid-version \ -rpath $(libdir) mk4tcl.lo mk4too.lo $(SHLOBJS) $(LINK_SPECIAL_FLAGS) -Mk4py@SHLIB_SUFFIX@: libmk4py.la - cp .libs/libmk4py@SHLIB_SUFFIX@ $@ +Mk4py@MODULE_SUFFIX@: libmk4py.la + cp .libs/libmk4py@MODULE_SUFFIX@ $@ -strip $(STRIP_FLAGS) $@ 2>/dev/null libmk4py.la: $(PYOBJS) $(LIBOBJS) - $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_FLAGS) -avoid-version \ + $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_MODULE_FLAGS) -avoid-version \ -rpath $(libdir) $(SPYOBJS) $(SHLOBJS) Mk4lua@SHLIB_SUFFIX@: mk4lua.o libmk4.la --- metakit-2.4.8/unix/configure.in Fri Oct 4 15:54:02 2002 +++ metakit-2.4.8+njr1/unix/configure.in Thu Nov 28 16:27:13 2002 @@ -8,25 +8,71 @@ MK_TARGETS="core" MK_INSTALL="install-mk" -AC_ARG_ENABLE(python, [ --enable-python build the Mk4py extension for Python], [python_ok=$enableval], [python_ok=no]) +AC_ARG_WITH(python, +[ --with-python=PATH build the Mk4py extension for Python with headers in + `PATH/include/python2.2'; install the Mk4py modules in + `PATH/lib/python2.2/site-packages'. If PATH is of the + form `HEADER:LIB', search for header files in HEADER, + and install Mk4py in LIB. If you omit the `=PATH' + part completely, the configure script will use the + provided includedir and libdir.], + [with_python=${withval}], [with_python=no]) AC_MSG_CHECKING([for Python configuration]) -if test "$python_ok" = yes; then +case "$with_python" in + "yes" ) + PY_INCLUDE_DIR='${includedir}/python2.2' + PY_LIB_DIR='${libdir}/python2.2/site-packages' + ;; + "no" ) + PY_INCLUDE_DIR="" + ;; + *":"* ) + PY_INCLUDE_DIR="`echo $with_python | sed -e 's/:.*$//'`" + PY_LIB_DIR="`echo $with_python | sed -e 's/^.*://'`" + ;; + * ) + PY_INCLUDE_DIR="$with_python/include/python2.2" + PY_LIB_DIR="$with_python/lib/python2.2/site-packages" + ;; +esac + +if test x"${PY_INCLUDE_DIR}" != x; then MK_TARGETS="$MK_TARGETS python" MK_INSTALL="$MK_INSTALL install-python" - AC_MSG_RESULT(enabled) + AC_MSG_RESULT($PY_INCLUDE_DIR and $PY_LIB_DIR) else AC_MSG_RESULT(not enabled) fi -AC_ARG_WITH(tcl, [ --with-tcl=DIR location of tcl.h header file], [TCL_INCLUDE_DIR=${withval}]) +AC_ARG_WITH(tcl, +[ --with-tcl=PATH build the Mk4tcl extension for Tcl with tcl.h in + `PATH/include/tcl.h'; install the Mk4tcl package + in `PATH/lib'. If PATH is of the form `HEADER:LIB', + search for tcl.h in HEADER, and install Mk4tcl in + LIB.], [with_tcl=${withval}], [with_tcl=no]) AC_MSG_CHECKING([for Tcl headers]) +case "$with_tcl" in + "no" ) + TCL_INCLUDE_DIR="" + ;; + *":"* ) + TCL_INCLUDE_DIR="`echo $with_tcl | sed -e 's/:.*$//'`" + TCL_LIB_DIR="`echo $with_tcl | sed -e 's/^.*://'`" + ;; + * ) + TCL_INCLUDE_DIR="$with_tcl/include" + TCL_LIB_DIR="$with_tcl/lib" + ;; +esac if test x"${TCL_INCLUDE_DIR}" != x ; then if test -f "${TCL_INCLUDE_DIR}/tcl.h" ; then AC_MSG_RESULT(found ${TCL_INCLUDE_DIR}/tcl.h) MK_TARGETS="$MK_TARGETS tcl" MK_INSTALL="$MK_INSTALL install-tcl" + AC_MSG_CHECKING(for Tcl package installation location) + AC_MSG_RESULT(${TCL_LIB_DIR}) else AC_MSG_ERROR([${TCL_INCLUDE_DIR} directory doesn't contain tcl.h]) fi @@ -59,24 +105,29 @@ # Deal with shared lib differences SHLIB_SUFFIX=".so" +MODULE_SUFFIX=".so" STRIP_FLAGS= -LIBTOOL_FLAGS=-module +LIBTOOL_SHLIB_FLAGS= +LIBTOOL_MODULE_FLAGS=-module case $build_os in darwin*) SHLIB_SUFFIX=".dylib" + # MODULE_SUFFIX remains .so (Darwin doesn't care) STRIP_FLAGS=-S - LIBTOOL_FLAGS= ;; hpux*) SHLIB_SUFFIX=".sl" + MODULE_SUFFIX=".sl" if test "X$build_cpu" = "Xia64" ; then - # libtools insists on .so files for HPUX/Itanium :( + # libtool insists on .so files for HP-UX/Itanium :( SHLIB_SUFFIX=".so" + MODULE_SUFFIX=".so" fi ;; cyg*) SHLIB_SUFFIX=".dll" + MODULE_SUFFIX=".dll" ;; esac @@ -126,10 +177,17 @@ AC_SUBST(MK_TARGETS) AC_SUBST(MK_INSTALL) +AC_SUBST(PY_INCLUDE_DIR) +AC_SUBST(PY_LIB_DIR) AC_SUBST(TCL_INCLUDE_DIR) +AC_SUBST(TCL_LIB_DIR) AC_SUBST(EXEEXT) +AC_SUBST(CPPFLAGS) +AC_SUBST(LDFLAGS) AC_SUBST(SHLIB_SUFFIX) -AC_SUBST(LIBTOOL_FLAGS) +AC_SUBST(MODULE_SUFFIX) +AC_SUBST(LIBTOOL_SHLIB_FLAGS) +AC_SUBST(LIBTOOL_MODULE_FLAGS) AC_SUBST(STRIP_FLAGS) AC_OUTPUT(Makefile) --xgyAXRrhYN0wYx8y-- From mikel@reactivity.com Wed Dec 4 16:03:02 2002 Received: from MAILSERVER.reactivity.com ([63.90.156.191]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB4M32x05693 for ; Wed, 4 Dec 2002 16:03:02 -0600 Received: from reactivity.com ([10.0.5.75]) by MAILSERVER.reactivity.com over TLS secured channel with Microsoft SMTPSVC(5.0.2195.4905); Wed, 4 Dec 2002 14:03:42 -0800 Date: Wed, 4 Dec 2002 14:03:40 -0800 Subject: Re: [Metakit] unable to use Mk4py on Mac OS X (jaguar, 10.2.2) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v548) Cc: metakit@equi4.com To: Nicholas Riley From: mikel evins In-Reply-To: <20021204202719.GC3050976@uiuc.edu> Message-Id: <3F297459-07D4-11D7-9EE9-0003939376E6@reactivity.com> X-Mailer: Apple Mail (2.548) X-OriginalArrivalTime: 04 Dec 2002 22:03:42.0648 (UTC) FILETIME=[02309380:01C29BE1] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id gB4M32x05693 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Wednesday, December 4, 2002, at 12:27 PM, Nicholas Riley wrote: > Sorry, forgot to send the patch. (I realize I forgot to send my > revised patch to Jean-Claude last time too, so here it is :-) > Thanks for the patch; presumably I'm doing something wrong, because I get a build failure with 2.4.8-38. I unpacked the tarball, applied the patch, substituted glibtool in place of the libtool that the ddistribution builds, and, voilá: ld: Undefined symbols: __Unwind_Resume __Unwind_GetDataRelBase __Unwind_GetIP __Unwind_GetLanguageSpecificData __Unwind_GetRegionStart __Unwind_GetTextRelBase __Unwind_SetGR __Unwind_SetIP __Unwind_DeleteException __Unwind_RaiseException make: *** [demo] Error 1 Any ideas? From njriley@theremin.cs.uiuc.edu Thu Dec 5 16:57:52 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB5Mvpx25038 for ; Thu, 5 Dec 2002 16:57:51 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id QAA3099029; Thu, 5 Dec 2002 16:58:50 -0600 (CST) Date: Thu, 5 Dec 2002 16:58:50 -0600 From: Nicholas Riley To: mikel evins Cc: metakit@equi4.com Subject: Re: [Metakit] unable to use Mk4py on Mac OS X (jaguar, 10.2.2) Message-ID: <20021205225850.GB3096194@uiuc.edu> References: <20021204202719.GC3050976@uiuc.edu> <3F297459-07D4-11D7-9EE9-0003939376E6@reactivity.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3F297459-07D4-11D7-9EE9-0003939376E6@reactivity.com> User-Agent: Mutt/1.4i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Wed, Dec 04, 2002 at 02:03:40PM -0800, mikel evins wrote: > > On Wednesday, December 4, 2002, at 12:27 PM, Nicholas Riley wrote: > > >Sorry, forgot to send the patch. (I realize I forgot to send my > >revised patch to Jean-Claude last time too, so here it is :-) > > > > Thanks for the patch; presumably I'm doing something wrong, because I > get a build failure with 2.4.8-38. I unpacked the tarball, applied the > patch, substituted glibtool in place of the libtool that the > ddistribution builds, and, voilá: Did you rerun libtoolize/aclocal/autoconf? This should happen automatically when you run configure, at least the second half of it, but it may not have. > ld: Undefined symbols: > __Unwind_Resume > __Unwind_GetDataRelBase > __Unwind_GetIP > __Unwind_GetLanguageSpecificData > __Unwind_GetRegionStart > __Unwind_GetTextRelBase > __Unwind_SetGR > __Unwind_SetIP > __Unwind_DeleteException > __Unwind_RaiseException I never saw anything like that before. -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign From njriley@theremin.cs.uiuc.edu Thu Dec 5 17:53:02 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB5Nr2x27761 for ; Thu, 5 Dec 2002 17:53:02 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id RAA3091354; Thu, 5 Dec 2002 17:54:09 -0600 (CST) Date: Thu, 5 Dec 2002 17:54:09 -0600 From: Nicholas Riley To: mikel evins Cc: metakit@equi4.com Subject: Re: [Metakit] unable to use Mk4py on Mac OS X (jaguar, 10.2.2) Message-ID: <20021205235409.GA3065679@uiuc.edu> References: <1039131118.1C931EDE@w5.dngr.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Thu, Dec 05, 2002 at 03:45:57PM -0800, mikel evins wrote: > > On Thursday, December 5, 2002, at 03:31 PM, Nicholas Riley wrote: > > > >>> > >>>Did you rerun libtoolize/aclocal/autoconf? This should happen > >>>automatically when you run configure, at least the second half of it, > >>>but it may not have. > >>> > >> > >>Tried it; same result. > > > >Can you post your entire build log somewhere? > > Do you just want me to pipe the build output to a file or is a log > constructed in the filetree somewhere in the normal course of things? > > In order to save one potential trouble, is there implicit a more > complicated set of instructions for running libtoolize/aclocal/autoconf > than simply executing them simple-mindedly in the unix subdir? To really be sure, here's what I generally do: % rm aclocal.m4 % rm -rf autom4te.cache % glibtoolize --copy --force % aclocal % autoconf If any of those give error messages such as missing macros, then something is probably wrong. Make sure that the versions of autoconf/automake and libtool running are all configured to work with one another - i.e., if you're using glibtool from Fink, then the other two should be from there too. The reason for this is that there are a lot of interdependencies between the packages and some item can be out of date but still appear to work at the auto* stage, but fail on configure/make. I went through about three hours of pain a few days ago trying to get Metakit to compile on Linux until I figured out that they weren't all using the same m4 directories. -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign From mikel@reactivity.com Thu Dec 5 17:28:11 2002 Received: from MAILSERVER.reactivity.com ([63.90.156.191]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB5NSBx26535 for ; Thu, 5 Dec 2002 17:28:11 -0600 Received: from reactivity.com ([66.127.237.50]) by MAILSERVER.reactivity.com over TLS secured channel with Microsoft SMTPSVC(5.0.2195.4905); Thu, 5 Dec 2002 15:29:10 -0800 Date: Thu, 5 Dec 2002 15:29:16 -0800 Subject: Re: [Metakit] unable to use Mk4py on Mac OS X (jaguar, 10.2.2) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v548) Cc: metakit@equi4.com To: Nicholas Riley From: mikel evins In-Reply-To: <20021205225850.GB3096194@uiuc.edu> Message-Id: <5EB52ECA-08A9-11D7-9EE9-0003939376E6@reactivity.com> X-Mailer: Apple Mail (2.548) X-OriginalArrivalTime: 05 Dec 2002 23:29:10.0789 (UTC) FILETIME=[1D368B50:01C29CB6] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id gB5NSBx26535 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Thursday, December 5, 2002, at 02:58 PM, Nicholas Riley wrote: > On Wed, Dec 04, 2002 at 02:03:40PM -0800, mikel evins wrote: >> >> On Wednesday, December 4, 2002, at 12:27 PM, Nicholas Riley wrote: >> >>> Sorry, forgot to send the patch. (I realize I forgot to send my >>> revised patch to Jean-Claude last time too, so here it is :-) >>> >> >> Thanks for the patch; presumably I'm doing something wrong, because I >> get a build failure with 2.4.8-38. I unpacked the tarball, applied the >> patch, substituted glibtool in place of the libtool that the >> ddistribution builds, and, voilá: > > Did you rerun libtoolize/aclocal/autoconf? This should happen > automatically when you run configure, at least the second half of it, > but it may not have. > Tried it; same result. From njriley@theremin.cs.uiuc.edu Thu Dec 5 22:14:33 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB64EXx06330 for ; Thu, 5 Dec 2002 22:14:33 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id WAA3114632; Thu, 5 Dec 2002 22:15:42 -0600 (CST) Date: Thu, 5 Dec 2002 22:15:41 -0600 From: Nicholas Riley To: mikel evins Cc: metakit@equi4.com Subject: Re: [Metakit] unable to use Mk4py on Mac OS X (jaguar, 10.2.2) Message-ID: <20021206041541.GA3074155@uiuc.edu> References: <20021205235409.GA3065679@uiuc.edu> <618F2921-08B0-11D7-9EE9-0003939376E6@evins.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <618F2921-08B0-11D7-9EE9-0003939376E6@evins.net> User-Agent: Mutt/1.4i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Thu, Dec 05, 2002 at 04:19:27PM -0800, mikel evins wrote: > > On Thursday, December 5, 2002, at 03:54 PM, Nicholas Riley wrote: > > >To really be sure, here's what I generally do: > > > >% rm aclocal.m4 > >% rm -rf autom4te.cache > >% glibtoolize --copy --force > >% aclocal > >% autoconf > > > > > > bash-2.05a$ rm aclocal.m4 > bash-2.05a$ rm -rf autom4te.cache > bash-2.05a$ glibtoolize --copy --force > You should add the contents of `/sw/share/aclocal/libtool.m4' to > `aclocal.m4'. > Putting files in AC_CONFIG_AUX_DIR, `scripts'. > bash-2.05a$ /sw/bin/aclocal > bash-2.05a$ /sw/bin/autoconf > bash-2.05a$ cd ../builds/ > bash-2.05a$ ../unix/configure --enable-python > checking for Python configuration... not enabled The syntax of the python option has changed with my patch - see configure --help. > loading cache /dev/null within ltconfig > ltconfig: you must specify a host type if you use `--no-verify' > Try `ltconfig --help' for more information. > configure: error: libtool configure failed Yup, this was my problem with MetaKit on Linux. There's some mismatch - are you sure you're using glibtoolize from Fink? -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign From nigel@bprj.co.uk Fri Dec 6 02:23:18 2002 Received: from cmailm1.svr.pol.co.uk (cmailm1.svr.pol.co.uk [195.92.193.18]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB68NEx15541 for ; Fri, 6 Dec 2002 02:23:17 -0600 Received: from modem-479.cheetah.dialup.pol.co.uk ([217.134.97.223] helo=jeeves.bprj.co.uk) by cmailm1.svr.pol.co.uk with esmtp (Exim 3.35 #1) id 18KDmS-0006Bi-00 for metakit@equi4.com; Fri, 06 Dec 2002 08:24:16 +0000 Received: from tuppy (tuppy.bprj.co.uk [192.168.1.199]) by jeeves.bprj.co.uk (8.12.3/8.12.3/SuSE Linux 0.6) with SMTP id gB68O6ub005323 for ; Fri, 6 Dec 2002 08:24:06 GMT From: "Nigel Hathaway" To: Date: Fri, 6 Dec 2002 08:24:32 -0000 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_021E_01C29D00.E7104840" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal X-MS-TNEF-Correlator: X-Virus-Scanned: by amavis-milter (http://amavis.org/) Subject: [Metakit] Wrong docs in 2.4.8 distro Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_021E_01C29D00.E7104840 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit The documents the come as part of the MetaKit 2.4.8 distribution refer to a much older version of the product (as I eventually found out). I would be helpful if the distribution could contain the current document set. I am not getting very far in trying to download the current documents either... They come in a special 'kit' file, which my browser (IE5.5) freezes on trying to download then shows as garbage (I think the web server tells the browser that it is a text MIME type for display). When I try to download the kit viewer, the zip file downloaded is not a zip file but an HTML file containing an error message. Please could the latest documents be made available for download in one of the conventional zip/gz formats? Thanks ************************************ Barnabas Projects Limited Tel: +44 117 937 4197 Internet: http://www.bprj.co.uk ------=_NextPart_000_021E_01C29D00.E7104840 Content-Type: application/ms-tnef; name="winmail.dat" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="winmail.dat" eJ8+IiAIAQaQCAAEAAAAAAABAAEAAQeQBgAIAAAA5AQAAAAAAADoAAEIgAcAGAAAAElQTS5NaWNy b3NvZnQgTWFpbC5Ob3RlADEIAQ2ABAACAAAAAgACAAEGgAMADgAAANIHDAAGAAgAGAAAAAUAEAEB A5AGAAgOAAAjAAAACwACAAEAAAALACMAAAAAAAMAJgAAAAAACwApAAEAAAACATEAAQAAANAAAABQ Q0RGRUIwOQABAAIAUAAAAAAAAAA4obsQBeUQGqG7CAArKlbCAABQU1RQUlguRExMAAAAAAAAAABO SVRB+b+4AQCqADfZbgAAAEg6XE91dGxvb2tcbWFpbGJveC5wc3QAGAAAAAAAAADjJVzP8aHUEb8I AE9JBiz9woEAAAAAAAAYAAAAAAAAAOMlXM/xodQRvwgAT0kGLP3CgAAAEAAAANREJfZF3aBLqGFm 8X7QJ0AbAAAAV3JvbmcgZG9jcyBpbiAyLjQuOCBkaXN0cm8AAwA2AAAAAAAeAHAAAQAAABsAAABX cm9uZyBkb2NzIGluIDIuNC44IGRpc3RybwAAAgFxAAEAAAAWAAAAAcKdAOR8SFLU4vLHT3qFxM4p hRKFHAAAAgEdDAEAAAAWAAAAU01UUDpOSUdFTEBCUFJKLkNPLlVLAAAACwABDgAAAABAAAYOAJDf 0wCdwgECAQoOAQAAABgAAAAAAAAA4yVcz/Gh1BG/CABPSQYs/cKAAAALAB8OAQAAAAIBCRABAAAA iAkAAIQJAADKFgAATFpGdUbV94sDAAoAcmNwZzEyNXIyDGBjMQMwAQcLYG6RDhAwMzMPFmZlD5JP AfcCpANjAgBjaArAc4RldALRcHJxMgAAkioKoW5vElAgMAHQhQHQNg+gMDUwNBQh8wHQFBA0fQdt AoMAUAPU+xH/EwtiE+EUUBOyGPQU0BsHEwKDMwHAEY4yMzjFF1QgB20gQ0UaBQ/ATxqeFEAbvxzF eXIaBTW5EY4xNhYxHx8DgkcJ0eprGgU2IR82GnEibwOC9lQIcBoFNySPJ7Il3wOCkChIZWIJcHcp GgX/G6En/huvKgYHEAGgDeAq5ro5K384JHEszwOCQgdA9nQN4BoEOAvCFnsbiAcTvR0XOBY/Hsg0 NSB2OB2x7xZsIhg0NCO5OB7hOA4lpvs0NCcnOCEBOA0o5zQ0KnzvMCI9jjO8Lhw4J8E4DTAnrzQ0 MbYCkQjmOwlvMEaf+mUOMDVHykjhSJ9JqUe0/0nSSD9MD0vNS09Jf0fPEGD8MjhRmlKxUm9TeUe0 U6K/Ug9V31WdVR9TT1cUOQ5QH1pkW8FT41vAAoJzdHmqbAeQaAngdAAAcQMhPGxpAUAFEAFAA/Bk Y+x0bAqxAGBzCrBfABbgEV9CbnVtAgBhYXXEdG8AYGRqdV0wBRB4Z2h0XmEKAV4wCgFp+QGQcDAD MTUiDAEPUgHQfjUnwBAYCNAJwF7AYwNuHnBjWWTwD5IDMHNuZX54FzAHsAWwAMACcxMQYy5zD5AD MGDAZGIgaXadE4BEARBgcDHQIFAKwMZhCcBiQGggRgIhXPPGMSEAXfJmaS0PkDJxz14warNefCqg ZHIJUGxS8xagbFJ3NFEBFwBlAWex/19vYH9hhmqzYi9jP2RPZV7+Ym4QCYACIHFRZkNqEGjQWWGB dC1pcANhOjCgb6F3EFN1YmoFkHR3EOBEYXRlOmnEJHBqT/9rX2xvbX9uj2+fcKxdgH4Q/wuADhJx UQwwcYQOUHIPcx9ndC91P3ZHUmVoMBcBIP8qgH4QBJBpxCfAeN9573r/33wPfR4IYF7wC4BlXWBo MN5sAUB+H38vgDQwkFAI0HJiCrB0OIDIceQPkDI/HuAQFpIRgsQTUBdwb29+ZoMfhC+FN5DghlAL UHneL2mAirALEYbFc2nELJD/h7+Iz4nfiu99H43vcK9xv/+CT5QPlR+WL5cydzJ21HgJfy8AXf9f D5z/YS+fRJCTOT+fj6Cfoa+iv4Uoq8BEb+5jqQAJ8AVATZ9QZxab8/tn5qvnYwBAr4hnkqhAhoFP AiA0YV0gBaBtcBNiRbsAwAMQU11CAdBc8zIAUP+m36fvqP+qD59PrK+tv67P94TrttC1sC210gYA ukC04Lp0CHBlRcMTEEawdgJRoCB7VW5rE1B3FdHmfQqhtJF3MVuwE+DDFP5oIhA3wJDht8HCsLfw mqH9AYBud5AAYAnwZ+Cw4JNBpngLYAJAb3kJ8FxmYP5wXYAAIAuQE1BooZqguHDPAOECMAJgAIBi ZAwwE1H9CrBjARAFsGigAgFm0IASKGVcaAWwesXyZGfZAMByZwuBytBoyQMwEHkBQWd2y5nKUbpA nsE37jAAUMwhzRU5OpHLYspQ/8NwzBPO43egxwAAcAswxfGXXUC0kA5QdgiQd2sLgP5kHuDQ0gTw B0AQYQFADgDfmnJdkLnQ0jUCEG+6cLhw/ZbAdMLAjTHFQtPHumAAwP+5kLhwk0HIcLpgudEJMroA /41QAlAHQAuQ1tECUdEhtzDv09DRwbfwAmB315MCUQAg5wnAtzDFcHJryXGywhchHxLyeCC1YMZh E4BDOlzqXHZgb2khbWlwAxAHkD3bEE0N4ANgtMABgCBP9wEgDeDKMFzcxg+TxuC10vouhQB0wwAX ELfwl1C60v9mYAFAxcHV0ASQyjDe0o0i3b1yMxPw3sRodGPCQRMC4wCABZBsdmwAULkRBQD/uUDi YQGQACDi4tEhsSEBwX/iURbgD3AAALkQDNABkCD+LsGU4mYOUOMCx7Hjb+R//eWPbA/AuRAFgec/ 6E/pX35sHuC5ENgA5w/rz+zUKd/lzCEA6p/vf+y0YipgApF/8J/ik8PB7l/zH/Qv9T9s/yfA9nLj T/ff+O/lzCyQ9n//+//9D/4f4sAvAPr/AI8Bn/8CpAr5tyC3D7gfuS+6P7tPX7xfvW++e0XxszZU XZAg98XxsQMtUHQRQbVBGEAJoF4gCWFowJOgEgNNFyBhgksMcCAyLjQukOCrZ/ALUmIK0GlpkCBG sJWSgHISAG8SkCBt4yD3aWDVsJehIIZyFQITJZNxXmTjIGjAReKzNigSoUm2IIZhBuB11/CWwCDT UA51xdATEArQKS4NCm0HciAalRkQd8eRl1Bil9rQE2AJUGZooCBpEzT/FHu1QBuytUAG4AfyEhMn MPdGsLExEXYgOLEaj9uByOD9aMBn2fAx4IBgFpIZwRrRvx6SImAiAhWhxNHUcG8LELsevxG0ZQxw 3/EYGoUanL8RMRnAElMIAe0QtIFjQiL5CHBxddNw2tDREAuxKTRrGdDbwSwbgGguYGlgbT8ZwD+Q wrC1kCXrLgBJRXg1LjXxACvKLyBHIHr/2+AruxURIw8kEwgQztIlcP0SoWcJcITAIcAscRIBqsDa axIDdz+AIAFyhnESAP+NUMfgEgMrRRIBeBAckDSRjzDxMzFmcLFQSU1FEgDf0IEZ0RWAFHHakXka cCow/lcwcjHRImEjfimRFpDQ4ceXsCqQEhJ6aXAqQyOX/0bANNIhgu0QOccU0RKQCBDwSFRNTDn0 HkUiAjxh/5ewk4AVgLEQQLAxgeWwEIn9GpxQXWCE0R3VEhLaogRwXxFpG/HAoAUgEpB2teFh/dZA ZSz8NjIjtSu7CAFpkH/a0BMlHkEZQhUBKOE5wS/cZ3o2EsCg1WA/Pv8nGf3QAGsuKxqcB2y2zwkf Ci//Cz8MTw1fDm8Pf2ngmKCNkP1UIGYWUWfx4iJWQarAQeEDStEsQkFVVE9URcBYVExJU1RNoGnw 9CAiwI4iGAAGoVZBFsCfaLFXc5GvkrKTVSAqXQ/1Xh8qGpVCTrC04ITBdlHGb3eiJXBMaW1RsFZw hyc2jZB3ECs0NCDDgFA3IDkzYmA0zlA3PxsGBuCXsGZQd9FQ4HRwUDovL3dkgC6QoHLEai61QC51 a0rCS1j/B29Nv07PT99Q71H/Uw9UH/++qAdfZ09oX2lvan9rj2yfJ22vD9xlqX0AeZALAACACCAG AAAAAADAAAAAAAAARgAAAAADhQAAAAAAAAMAAoAIIAYAAAAAAMAAAAAAAABGAAAAABCFAAAAAAAA AwAIgAggBgAAAAAAwAAAAAAAAEYAAAAAAYUAAAAAAAADAB2ACCAGAAAAAADAAAAAAAAARgAAAABS hQAAfW4BAB4AHoAIIAYAAAAAAMAAAAAAAABGAAAAAFSFAAABAAAABAAAADkuMAALAB+ACCAGAAAA AADAAAAAAAAARgAAAAAGhQAAAAAAAAsAI4AIIAYAAAAAAMAAAAAAAABGAAAAAA6FAAAAAAAAAwAk gAggBgAAAAAAwAAAAAAAAEYAAAAAEYUAAAAAAAADACaACCAGAAAAAADAAAAAAAAARgAAAAAYhQAA AAAAAAsAP4AIIAYAAAAAAMAAAAAAAABGAAAAAIKFAAABAAAAAgH4DwEAAAAQAAAA4yVcz/Gh1BG/ CABPSQYs/QIB+g8BAAAAEAAAAOMlXM/xodQRvwgAT0kGLP0CAfsPAQAAAFAAAAAAAAAAOKG7EAXl EBqhuwgAKypWwgAAUFNUUFJYLkRMTAAAAAAAAAAATklUQfm/uAEAqgA32W4AAABIOlxPdXRsb29r XG1haWxib3gucHN0AAMA/g8FAAAAAwANNP03AAACAX8AAQAAADAAAAA8TkVCQkxQQUlPREdLQkZN Sk5ORk1HRU1GTktBQS5uaWdlbEBicHJqLmNvLnVrPgADAAYQW7jXpwMABxC3AgAAAwAQEAAAAAAD ABEQAAAAAB4ACBABAAAAZQAAAFRIRURPQ1VNRU5UU1RIRUNPTUVBU1BBUlRPRlRIRU1FVEFLSVQy NDhESVNUUklCVVRJT05SRUZFUlRPQU1VQ0hPTERFUlZFUlNJT05PRlRIRVBST0RVQ1QoQVNJRVZF TlRVQUwAAAAA+ME= ------=_NextPart_000_021E_01C29D00.E7104840-- From lvirden@cas.org Fri Dec 6 11:51:01 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB6Hp1x17207 for ; Fri, 6 Dec 2002 11:51:01 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.5/m8.12.5/CAS_MAIL_HUB-2.00) with ESMTP id gB6Hq5L7001427 for ; Fri, 6 Dec 2002 12:52:05 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id gB6Hq4G17291; Fri, 6 Dec 2002 12:52:04 -0500 (EST) Date: Fri, 6 Dec 2002 12:52:04 -0500 (EST) From: "Larry W. Virden" Message-Id: <20021206125204.AAB17286@cas.org> To: "Metakit" Subject: [Metakit] From the tcler's wiki... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Today, the following question came up on the Tcler's Wiki, and I was hoping to get an answer for the person: Fwongo Hmm... do any other projects use Metakit for a format? (not that I need convincing that Starkits are a good thing, I'm just wondering) So, my question is - what open source Tcl related projects use Metakit databases? -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jyl@best.com Fri Dec 6 12:12:31 2002 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gB6ICVx19189 for ; Fri, 6 Dec 2002 12:12:31 -0600 Received: (qmail 14700 invoked by uid 19667); 6 Dec 2002 18:13:42 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 6 Dec 2002 18:13:42 -0000 Message-ID: <3DF0EC06.5070604@best.com> Date: Fri, 06 Dec 2002 10:27:18 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: "Larry W. Virden" CC: Metakit Subject: Re: [Metakit] From the tcler's wiki... References: <20021206125204.AAB17286@cas.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: e4graph is one: http://www.e4graph.com and http://e4graph.sourceforge.net --JYL Larry W. Virden wrote: >Today, the following question came up on the Tcler's Wiki, and I >was hoping to get an answer for the person: > >Fwongo Hmm... do any other projects use Metakit for a format? (not that I need convincing that Starkits are a good thing, I'm just wondering) > > > >So, my question is - what open source Tcl related projects use >Metakit databases? > From rjent@rjent.pair.com Fri Dec 6 21:04:21 2002 Received: from portal2.rjent.pair.com (adsl-66.112.47.157-rb.spt.centurytel.net [66.112.47.157]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB734Lx11357 for ; Fri, 6 Dec 2002 21:04:21 -0600 Received: from rjent.pair.com (localhost [127.0.0.1]) by portal2.rjent.pair.com (8.12.5/8.12.5) with ESMTP id gB716TAi001038 for ; Fri, 6 Dec 2002 19:06:29 -0600 (CST) (envelope-from rjent@rjent.pair.com) From: "RJ Ent" To: metakit@equi4.com Date: Fri, 6 Dec 2002 20:06:29 -0500 Message-Id: <20021207010629.M57249@rjent.pair.com> X-Mailer: Open WebMail 1.80 20021122 X-OriginatingIP: 192.168.2.34 (rjent) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] Metakit web database with local application front end? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, I have been considering the ability to have a tcl front end around a metakit database ran similar to the wikit and using standard http or https for transport to the database. This would allow me to taylor the app front end like I want but provide simple models for security. Would this idea work in application? Many thanks and thoughts welcome! -- RJEnt -- http://www.rjent.pair.com Providing solutions with FreeBSD http://www.freebsd.org From jcw@equi4.com Sat Dec 7 19:11:08 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB81B7x11590 for ; Sat, 7 Dec 2002 19:11:07 -0600 Date: Sun, 8 Dec 2002 02:12:19 +0100 Subject: Re: [Metakit] Wrong docs in 2.4.8 distro Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v548) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <195914AA-0A4A-11D7-BA90-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nigel Hathaway wrote: > The documents the come as part of the MetaKit 2.4.8 distribution refer > to a > much older version of the product (as I eventually found out). Whoops. > I am not getting very far in trying to download the current documents > either... > > They come in a special 'kit' file, which my browser (IE5.5) freezes on > trying to download then shows as garbage (I think the web server tells > the > browser that it is a text MIME type for display). Please try again - I think it should be fixed now. > When I try to download > the kit viewer, the zip file downloaded is not a zip file but an HTML > file > containing an error message. What is the URL? > Please could the latest documents be made available for download in > one of > the conventional zip/gz formats? I've repackaged it, see: http://www.equi4.com/pub/mk/metakit-2.4.7-html.tar.gz -jcw From jcw@equi4.com Sat Dec 7 19:15:12 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB81FCx11712 for ; Sat, 7 Dec 2002 19:15:12 -0600 Date: Sun, 8 Dec 2002 02:16:21 +0100 Subject: Re: [Metakit] From the tcler's wiki... Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v548) From: Jean-Claude Wippler To: "Metakit" Content-Transfer-Encoding: 7bit In-Reply-To: <20021206125204.AAB17286@cas.org> Message-Id: X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: > So, my question is - what open source Tcl related projects use > Metakit databases? [Jacob Levy] > e4graph is one: http://www.e4graph.com and > http://e4graph.sourceforge.net Yes - just Tcl & Open Source? No idea - I saw some OSS ones at SourceForge, but not Tcl. -jcw From jcw@equi4.com Sat Dec 7 19:18:22 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB81ILx11804; Sat, 7 Dec 2002 19:18:22 -0600 Date: Sun, 8 Dec 2002 02:19:34 +0100 Subject: Re: [Metakit] Metakit web database with local application front end? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v548) Cc: metakit@equi4.com To: "RJ Ent" From: Jean-Claude Wippler In-Reply-To: <20021207010629.M57249@rjent.pair.com> Message-Id: <1C505310-0A4B-11D7-BA90-00050251EF2F@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: RJ Ent wrote: > I have been considering the ability to have a tcl front end around a > metakit > database ran similar to the wikit and using standard http or https for > transport to the database. This would allow me to taylor the app > front end > like I want but provide simple models for security. Would this idea > work in > application? I can't see why not - it's being done all the time. You just need to think locking through - either a single process or lock files. -jcw From rjent@rjent.pair.com Sat Dec 7 20:13:05 2002 Received: from portal2.rjent.pair.com (adsl-66.112.47.157-rb.spt.centurytel.net [66.112.47.157]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB82D4x12946; Sat, 7 Dec 2002 20:13:04 -0600 Received: from rjent.pair.com (localhost [127.0.0.1]) by portal2.rjent.pair.com (8.12.5/8.12.5) with ESMTP id gB80FFAi004945; Sat, 7 Dec 2002 18:15:15 -0600 (CST) (envelope-from rjent@rjent.pair.com) From: "RJ Ent" To: Jean-Claude Wippler CC: metakit@equi4.com Subject: Re: [Metakit] Metakit web database with local application front end? Date: Sat, 7 Dec 2002 19:15:15 -0500 Message-Id: <20021208001515.M25441@rjent.pair.com> In-Reply-To: <1C505310-0A4B-11D7-BA90-00050251EF2F@equi4.com> References: <20021207010629.M57249@rjent.pair.com> <1C505310-0A4B-11D7-BA90-00050251EF2F@equi4.com> X-Mailer: Open WebMail 1.80 20021122 X-OriginatingIP: 192.168.2.34 (rjent) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, On Sun, 8 Dec 2002 02:19:34 +0100, Jean-Claude Wippler wrote > RJ Ent wrote: > > > I have been considering the ability to have a tcl front end around a > > metakit > > database ran similar to the wikit and using standard http or https for > > transport to the database. This would allow me to taylor the app > > front end > > like I want but provide simple models for security. Would this idea > > work in > > application? > > I can't see why not - it's being done all the time. > > You just need to think locking through - either a single process or > lock files. well here is where I show my lack of knowledge on the topic. I have read http://www.equi4.com/metakit/wiki.cgi/82 and still do not really understand, so would you be willing to explain a bit more on the when I would want to consider lock files?:) This model assists me by ease of install since all is self contained.:) Thanks for the response and all your efforts on Metakit! > > -jcw -- RJEnt -- http://www.rjent.pair.com Providing solutions with FreeBSD http://www.freebsd.org From nigel@bprj.co.uk Sun Dec 8 14:29:13 2002 Received: from cmailg1.svr.pol.co.uk (cmailg1.svr.pol.co.uk [195.92.195.171]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB8KTDx07127 for ; Sun, 8 Dec 2002 14:29:13 -0600 Received: from modem-1261.cheetah.dialup.pol.co.uk ([217.134.100.237] helo=jeeves.bprj.co.uk) by cmailg1.svr.pol.co.uk with esmtp (Exim 3.35 #1) id 18L84H-0000FZ-00 for metakit@equi4.com; Sun, 08 Dec 2002 20:30:25 +0000 Received: from tuppy (tuppy.bprj.co.uk [192.168.1.199]) by jeeves.bprj.co.uk (8.12.3/8.12.3/SuSE Linux 0.6) with SMTP id gB8KGuub020873 for ; Sun, 8 Dec 2002 20:16:56 GMT From: "Nigel Hathaway" To: "MetaKit mailing list" Subject: RE: [Metakit] Wrong docs in 2.4.8 distro Date: Sun, 8 Dec 2002 20:17:34 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal In-Reply-To: <195914AA-0A4A-11D7-BA90-00050251EF2F@equi4.com> X-Virus-Scanned: by amavis-milter (http://amavis.org/) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: http://www.equi4.com/pub/sk/mk4dok.kit This still gets interpreted by IE 5.5 as a text file for viewing in the browser (rather than a non-viewable file for downloading). This is the error on downloading the kit viewer (http://www.equi4.com/metakit/pub/kviewr18.zip): Search

Search

Searched for "kviewr18.zip" (in page titles):

  • No matches found

Tip: append an asterisk to search the page contents as well as titles.

Updated on 9 Apr 1999, 09:58 GMT
Recent Changes - About Wikit - Go to Equi4 Software - Help I can't find the page where this link is referenced again, and your search only searches page titles (arguably not that useful). Not very helpful, I know... Thanks for the docs. Nigel Hathaway ************************************ Barnabas Projects Limited Tel: +44 117 937 4197 Internet: http://www.bprj.co.uk -----Original Message----- From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com] On Behalf Of Jean-Claude Wippler Sent: 08 December 2002 01:12 To: MetaKit mailing list Subject: Re: [Metakit] Wrong docs in 2.4.8 distro Nigel Hathaway wrote: > The documents the come as part of the MetaKit 2.4.8 distribution refer > to a > much older version of the product (as I eventually found out). Whoops. > I am not getting very far in trying to download the current documents > either... > > They come in a special 'kit' file, which my browser (IE5.5) freezes on > trying to download then shows as garbage (I think the web server tells > the > browser that it is a text MIME type for display). Please try again - I think it should be fixed now. > When I try to download > the kit viewer, the zip file downloaded is not a zip file but an HTML > file > containing an error message. What is the URL? > Please could the latest documents be made available for download in > one of > the conventional zip/gz formats? I've repackaged it, see: http://www.equi4.com/pub/mk/metakit-2.4.7-html.tar.gz -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Sun Dec 8 15:18:22 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB8LILx09615 for ; Sun, 8 Dec 2002 15:18:21 -0600 Date: Sun, 8 Dec 2002 22:19:28 +0100 Subject: Re: [Metakit] Wrong docs in 2.4.8 distro Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v548) From: Jean-Claude Wippler To: "MetaKit mailing list" Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nigel Hathaway wrote: > http://www.equi4.com/pub/sk/mk4dok.kit > > This still gets interpreted by IE 5.5 as a text file for viewing in the > browser (rather than a non-viewable file for downloading). Right-click or use another browser, then. The mime type is "application/octet-stream" - if IE isn't treating that as a binary download then there's nothing I can do. > This is the error on downloading the kit viewer > (http://www.equi4.com/metakit/pub/kviewr18.zip): The www.equi4.com website has no reference to "kviewr18" - I have no idea where you got this link from. FYI, the Equi4 *and* the Metakit homepages both point to the KitViewer download area: http://www.equi4.com/pub/kv/ > Tip: append an asterisk to search the page contents as well as titles. [...] > I can't find the page where this link is referenced again, and your > search > only searches page titles (arguably not that useful). The following URL would have searched in titles + page contents: http://www.equi4.com/metakit/pub/kviewr18.zip* (Indeed - not found. If you ever find the origin of that link, please let me know) Anyway, I've adjusted the website to point to the proper spot. -jcw From brunobl@ifrance.com Mon Dec 9 12:02:12 2002 Received: from th22.opsion.fr (th22.opsion.fr [62.39.122.32]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gB9I26x14922 for ; Mon, 9 Dec 2002 12:02:11 -0600 Received: from 193.253.181.114 [193.253.181.114] by th22.opsion.fr id 200212091744.32bd; Mon, 9 Dec 2002 17:44:50 GMT User-Agent: Microsoft-Entourage/10.1.0.2006 Date: Mon, 09 Dec 2002 18:45:30 +0100 From: Bruno Blondeau To: Message-ID: In-Reply-To: <200212081801.gB8I18x03579@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] mmap Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, Could someone tell me how to force changes to the disk when mmap is being used by a Metakit database? I'm not used to work with mmap, so any sample code which would be integrated with Metakit would be helpful. Thanks, Bruno Blondeau http://www.iOrganizeX.com _____________________________________________________________________ GRAND JEU SMS : Pour gagner un NOKIA 7650, envoyez le mot IF au 61321 (prix d'un SMS + 0.35 euro). Un SMS vous dira si vous avez gagné. Règlement : http://www.ifrance.com/_reloc/sign.sms From jcw@equi4.com Mon Dec 9 15:27:23 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gB9LRMx31579 for ; Mon, 9 Dec 2002 15:27:23 -0600 Date: Mon, 9 Dec 2002 22:28:34 +0100 Subject: Re: [Metakit] Derived View + InsertAt() Crash Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v548) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <2C40D0FF-0BBD-11D7-BA90-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sanford Selznick wrote: > The code below that uses MetaKit is crashing. JCW confirmed this as a > bug in MetaKit and is working on fixing it. Apparently, modifying a > view with InserAt after it's derived to another view is corrupting > memory somehow. The exact case is that a row is inserted in a view that has a derived selection active on it, but which is not including the row at that position. I.e. an insert in pos 1 of a view with (1, 0) as contents, on which a selection is active containing only 1's. This was a pretty bad bug, if you use dynamic selections and insert in the underlying view without using GetIndexOf to map from existing selections only. (If none of this makes sense, please ignore it - you are most likely not creating derived selection views, nor making changes to the underlying view.) Thanks for reporting this, with a splendid short test case - and my apologies for taking so long to resolve this. > In any event, JCW recommended the following temporary fix: before > performing the InsertAt(), reset the derived view with something like > found_view = c4_View(); Yes - but with the latest code in CVS, things should be ok. The bug only existed in insert mode, btw. Given that the other cases look ok, I must have missed this in the 2.3/2.4 code changes. I've also added a new test case (n14) to make sure this bug never slips through again. Please let me know if anything else doesn't work as expected. -jcw From bvthadden@onlinehome.de Tue Dec 10 06:49:31 2002 Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBACnUG28506 for ; Tue, 10 Dec 2002 06:49:30 -0600 Received: from [212.227.126.161] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 18LjqW-0007xX-00 for metakit@equi4.com; Tue, 10 Dec 2002 13:50:44 +0100 Received: from [80.142.70.87] (helo=THADDEN) by mrelayng.kundenserver.de with asmtp (Exim 3.35 #1) id 18LjqW-0001Li-00 for metakit@equi4.com; Tue, 10 Dec 2002 13:50:44 +0100 Message-ID: <000a01c2a04a$c1a89470$1416a8c0@THADDEN> From: "Bodo von Thadden" To: Date: Tue, 10 Dec 2002 13:50:38 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0007_01C2A053.1F4F36D0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Subject: [Metakit] Compiling metakit under cygwin Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0007_01C2A053.1F4F36D0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello I know, this topic was often already discussed, but I have problems to = compile it with cygwin and gcc 3.2.1 The compiler alway's stop's at string.cpp. He say's to me, that the = method strcasecmp() was delared as extern later as static. I was pointed to a thread from early messages at = flightgear-users@flightgear.org to a solution for that. So I changed the line 33 to #if (!q4_MSVC && !q4_WATC) || ((defined(__CYGWIN__) || = defined(__MINGW32__)) && __GNUC__ < 3) // #if !q4_MSVC && !q4_WATC // MS C/C++ has this handy stricmp: a case-insensitive version of = strcmp // This version only works with 7-bit ASCII characters 0x00 through = 0x7F =20 static int stricmp(const char* p1, const char* p2) { .... .... } =20 #endif But it really didn't help. So the next time, I removed static from the = method and got some other error's Undefined reference to ___gcc_personality_sj0 in libmk4.a ... .. Could any body please help me to solve this problem. Thanks Bodo ------=_NextPart_000_0007_01C2A053.1F4F36D0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hello
I know, this topic was often already = discussed, but=20 I have problems to compile it with cygwin and gcc 3.2.1
The compiler alway's stop's at = string.cpp.  He=20 say's to me, that the method strcasecmp() was delared as = extern
later as static.
I was pointed to a thread from early = messages at flightgear-users@flightge= ar.org=20 to a solution for that.
So  I changed the line 33 = to
 
#if (!q4_MSVC && !q4_WATC) ||=20 ((defined(__CYGWIN__) || defined(__MINGW32__)) && __GNUC__ < = 3)
//=20 #if !q4_MSVC && !q4_WATC
 
  // MS C/C++ has this handy = stricmp: a=20 case-insensitive version of strcmp
  // This version only works = with=20 7-bit ASCII characters 0x00 through 0x7F
    =
  =20 static int stricmp(const char* p1, const char* p2)
  = {
  =20 ....
   ....
  }
  =
#endif
 
But it really didn't help. So the next = time, I=20 removed static from the method
and got some other error's
 
Undefined reference to = ___gcc_personality_sj0 in=20 libmk4.a
...
..
 
Could any body please help me to solve = this=20 problem.
 
Thanks
Bodo
 
 
------=_NextPart_000_0007_01C2A053.1F4F36D0-- From nigel@bprj.co.uk Wed Dec 11 11:22:41 2002 Received: from cmailm3.svr.pol.co.uk (cmailm3.svr.pol.co.uk [195.92.193.19]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBBHMdG27002 for ; Wed, 11 Dec 2002 11:22:40 -0600 Received: from modem-158.gorilla.dialup.pol.co.uk ([81.78.96.158] helo=jeeves.bprj.co.uk) by cmailm3.svr.pol.co.uk with esmtp (Exim 3.35 #1) id 18MAaP-0002ly-00 for metakit@equi4.com; Wed, 11 Dec 2002 17:23:54 +0000 Received: from tuppy (tuppy.bprj.co.uk [192.168.1.199]) by jeeves.bprj.co.uk (8.12.3/8.12.3/SuSE Linux 0.6) with SMTP id gBBGmU7E008683 for ; Wed, 11 Dec 2002 16:48:30 GMT From: "Nigel Hathaway" To: "MetaKit mailing list" Date: Wed, 11 Dec 2002 16:48:30 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal In-Reply-To: X-Virus-Scanned: by amavis-milter (http://amavis.org/) Subject: [Metakit] Logical omission - or maybe some clarification required? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: You could argue that there is a logical omission from the data type set in MetaKit, when it comes to looking at storage and access efficiencies. The current data (i.e. column) types, with their format letters, are as follows (taken from format.html, dated 1999): S - string (null-byte terminated) I - integer (1..32 bits, adapting at run time) F - 32-bit floating point (obsolete, in favour of D ??) D - 64-bit floating point B - untyped binary data M - "memo" binary data (now obsolete, replaced by B ??) ...and, of course, sub-views. From the code (format.cpp), it would seem that I, L, F, D, B, S and V are actually supported (L being a 64-bit integer, V being a sub-view). From reading the docs data is stored as individual columns in the file, and the fastest way to locate a data item is to make an offset from the start of the column data given by the index of the data item multiplied by its size, provided the data is of fixed size. This is true for F, D, L and I (which I don't follow, in terms of how the 'adaptive' part works. You don't specify how wide you want the integer to be when you set the column up, so if you fill it with Boolean values - 1's or 0's - and then add an 8-bit value in, does this mean the whole column gets re-written as 8-bits for each row?). For non-fixed types (S, B), there are two methods of storage. Strings are stored end-to-end, separated by the null-terminator on each, so the code has to search all the way down the column (reading each byte) in order to find an entry at a particular index - this seems a bit inefficient for large tables. Binary columns use a two stage look-up: it uses two columns the data stored end-to-end like strings and a column of integer data sizes, so again, you have to read all the way down a column to calculate the position and length, and then do the look-up - so perhaps not as lengthy as strings, but not as efficient as the obsoleted memo column which used two columns: one for size and the other for file offset? Is all this correct so far? What seems to be missing is fixed-sized binary columns (that can't be contained in integers up to 8 bytes long). But why would you want them anyway? If it is fixed sized then it implies that you know the structure, so it can be broken down into multiple columns of fundamental data types - true in some cases. Fixed sized strings and variants (data unions) are where it would be logical to use such things. You could say that strings are never fixed-size, so you waste a lot of space making them so, so you may not actually get any retrieval efficiency gains. Data unions (variants) which contain a code to identify type followed by a varying length of data depending on that code. Is there a performance gain to be had from this (over B columns)? Is it something that can be easily implemented using existing underlying access methods, or would it simply break the whole scheme and file format? Regards, Nigel Hathaway ************************************ Barnabas Projects Limited Tel: +44 117 937 4197 Internet: http://www.bprj.co.uk From v.digilio@sisdata.it Thu Dec 12 06:11:50 2002 Received: from smtp3.cp.tin.it (vsmtp3.tin.it [212.216.176.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBCCBnG11939 for ; Thu, 12 Dec 2002 06:11:49 -0600 Received: from vikwin2k (80.116.219.42) by smtp3.cp.tin.it (6.5.029) id 3DDD78820080CBD2 for metakit@equi4.com; Thu, 12 Dec 2002 13:13:04 +0100 Message-ID: <001401c2a1d8$45c261d0$1500a8c0@vikwin2k> From: "Vittorio Digilio" To: Date: Thu, 12 Dec 2002 13:16:17 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0011_01C2A1E0.A7437F40" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] c4_Property shouldn't require a virtual destructor? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Messaggio in formato MIME composto da più parti. ------=_NextPart_000_0011_01C2A1E0.A7437F40 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello,=20 I'm quite new at the metakit engine but I already like it very much. Unfortunately it lacks (almost completely :-) ) a full C++ documentation = (anybody, as a long-term user wrote down something and is willing to = share, thanks :-) ), so I started experimenting and inspecting the C++ = src code. I noticed that c4_Property, though being the base class for the other = properties, provides a non-virtual destructor. This could be justified by the fact that it holds only two simple member = variables (short and char) that would be anyway destroyed if a = heap-allocated c4_Property-derived instance, e.g. c4_IntProp, is = destroyed (delete). But it also looks like the c4_Property destructor (and constructor) uses = to reference-count ( c4_Property::Ref(int) ) about some internal static = variables (with thread-locking involved). In this scenario deleting the heap-allocated derived class shouldn't = call the base class c4_Property::~c4_Property() destructor and the = reference wouldn't be released. I mean : c4_IntProp *pMyInt=3Dnew c4_IntProp("age"); //.... //.... delete pMyInt; // c4_Property::~c4_Property() should'nt be called -> = Refs(-1) isn't called either Perhaps I'm missing here something really big and the destructor should = be non-virtual ?! TIA and best regards, Vittorio Digilio Sisdata srl -------------------------------------------------------------------------= ------- Borland didn't invent compilers. Microsoft didn't invent operating = systems. Novell didn't invent networking. Sun didn't invent Unix. Apple = didn't invent the graphical user interface. Oracle didn't invent the = database. It turns out that nearly all successful software companies = have concentrated on constructing better implementations of already = existing technologies. The market rewarded these companies because they = provided the market with what it wanted: products, not ideas. These = companies didn't have a horde of researchers working in obscure fields = in the hope that one of them would discover something useful. In the = software industry ideas are like air. The hard part is deciding which = ideas to choose. The focus of these companies was on "doing it right" = rather than on "doing it first" or "doing it differently". By allowing = other companies to monopolize new technologies, patents strike at the = very essence of the software industry's business philosophy. ------=_NextPart_000_0011_01C2A1E0.A7437F40 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hello,
I'm quite new at the metakit engine but = I already=20 like it very much.
 
Unfortunately it lacks (almost = completely :-) ) a=20 full C++ documentation (anybody, as a long-term user wrote down = something=20 and is willing to share, thanks :-) ), so I started experimenting and = inspecting=20 the C++ src code.
 
I noticed that c4_Property, though = being the=20 base class for the other properties, provides a non-virtual=20 destructor.
 
This could be justified by the fact = that it holds=20 only two simple member variables (short and char) that would be anyway = destroyed=20 if a heap-allocated c4_Property-derived instance, e.g. c4_IntProp, is = destroyed=20 (delete).
But it also looks like the c4_Property = destructor=20 (and constructor) uses to reference-count ( c4_Property::Ref(int) ) = about some=20 internal static variables (with thread-locking involved).
 
In this scenario deleting the = heap-allocated=20 derived class shouldn't call the base class c4_Property::~c4_Property()=20 destructor and the reference wouldn't be released.
 
I mean :
 
c4_IntProp *pMyInt=3Dnew=20 c4_IntProp("age");
//....
//....
delete pMyInt; // = c4_Property::~c4_Property()=20 should'nt be called -> Refs(-1) isn't called either
 
Perhaps I'm missing here something = really big and=20 the destructor should be non-virtual ?!
 
TIA and best regards,
 
Vittorio Digilio
Sisdata srl
 
Borland didn't invent compilers. = Microsoft didn't=20 invent operating systems. Novell didn't invent networking. Sun didn't = invent=20 Unix. Apple didn't invent the graphical user interface. Oracle didn't = invent the=20 database. It turns out that nearly all successful software companies = have=20 concentrated on constructing better implementations of already existing=20 technologies. The market rewarded these companies because they provided = the=20 market with what it wanted: products, not ideas. These companies didn't = have a=20 horde of researchers working in obscure fields in the hope that one of = them=20 would discover something useful. In the software industry ideas are like = air.=20 The hard part is deciding which ideas to choose. The focus of these = companies=20 was on "doing it right" rather than on "doing it first" or "doing it=20 differently". By allowing other companies to monopolize new = technologies,=20 patents strike at the very essence of the software industry's business=20 philosophy.
------=_NextPart_000_0011_01C2A1E0.A7437F40-- From jcw@equi4.com Thu Dec 12 06:21:24 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBCCLMG13307 for ; Thu, 12 Dec 2002 06:21:23 -0600 Date: Thu, 12 Dec 2002 13:22:38 +0100 Subject: Re: [Metakit] Logical omission - or maybe some clarification required? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v548) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <671C0AD0-0DCC-11D7-91BB-00050251EF2F@equi4.com> X-Mailer: Apple Mail (2.548) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nigel Hathaway wrote: [...] > From reading the docs data is stored as individual columns in the > file, and > the fastest way to locate a data item is to make an offset from the > start of > the column data given by the index of the data item multiplied by its > size, > provided the data is of fixed size. This is true for F, D, L and I > (which I > don't follow, in terms of how the 'adaptive' part works. You don't > specify > how wide you want the integer to be when you set the column up, so if > you > fill it with Boolean values - 1's or 0's - and then add an 8-bit value > in, > does this mean the whole column gets re-written as 8-bits for each > row?). Yes. If you are concerned about this, you can store a big value first to "widen" the datatype. There will at some point be a way to reduce it again, but so far that only happens when copying views (just like there will at some point to be be a "compaction" function, the current design is starting to reach limits which can be avoided). > For non-fixed types (S, B), there are two methods of storage. Strings > are > stored end-to-end, separated by the null-terminator on each, so the > code has > to search all the way down the column (reading each byte) in order to > find > an entry at a particular index - this seems a bit inefficient for large > tables. It is - which is why the 2.3/2.4 file format was changed a bit, see below. > Binary columns use a two stage look-up: it uses two columns the > data stored end-to-end like strings and a column of integer data > sizes, so > again, you have to read all the way down a column to calculate the > position > and length, and then do the look-up - so perhaps not as lengthy as > strings, > but not as efficient as the obsoleted memo column which used two > columns: > one for size and the other for file offset? The B, S, and M datatypes have all been merged into one. The new "B" datatype stores things in three low-level columns: * a set of concatenated strings * a vector of field sizes * a variable-sized "run" of strings which were stored out-of-band For small strings and/or a low number of rows (there's a heuristic there), things are stored as a concatenation, but scans for terminating null bytes are not needed, because all field sizes are stored in a separate int column. Large items are stored out-of-band, i.e. as a separate area in the datafile. The third column then contains information about this, as a sequence of triples: skip, size, position The skip value indicates how many in-band rows to skip, then comes one out-of-band item, of the indicated size and at the indicated file position. A column with only out-of-band data looks like: the data column is empty the offsets column is empty the out-of-band column contains: 0, size0, pos0, 0, size1, pos1, 0, size2, pos2, etc... > What seems to be missing is fixed-sized binary columns (that can't be > contained in integers up to 8 bytes long). But why would you want them > anyway? If it is fixed sized then it implies that you know the > structure, > so it can be broken down into multiple columns of fundamental data > types - > true in some cases. Fixed sized strings and variants (data unions) are > where it would be logical to use such things. You could say that > strings > are never fixed-size, so you waste a lot of space making them so, so > you may > not actually get any retrieval efficiency gains. Data unions > (variants) > which contain a code to identify type followed by a varying length of > data > depending on that code. > > Is there a performance gain to be had from this (over B columns)? Yes, this is an omission I have been planning to get to for some time. The plan is not to introduce a new datatype, but to autodetect integer columns in which each value is the same. So this would end up being a B-type property, with three columns, the first containing the data, the second containing the "constant-int" column (which is a few bytes, and easily singled out), and the third is empty. The B handlers would recognize this format and avoid setting up a cumulative offset vector. The benefit is that it would also cover other cases, and that it would work for all styles: B, S, and what used to be M (i.e. large, but constant-size chunks, think disk blocks, for example). > Is it something that can be easily implemented using existing > underlying > access methods, or would it simply break the whole scheme and file > format? I don't know. My first concern has been for quite some time now to get MK 2.4 up to the same level of stability and robustness as MK 2.01 was. That is now the case, I think. So looking into this would be feasible - though I don't have time for it this very moment. Having said that, note that you can quite effectively simulate this mechanism in a different way: have one huge B-type, i.e. binary property, i.e. one row. Then use the partial Access/Modify calls to maintain such a single item as an array of fixed-size chunks. You wouldn't be able to search and operate on it with other View operations, but even that can probably be simulated by creating a custom viewer which maps each chunk to a fake property. One more comment w.r.t performance in variable-sized B's and S's when there are a very large number of rows: this is exactly what "blocked" views are designed to improve upon. They turn a view which logically looks like one big one into one which is physically stored in "chunky format". Each of those chunks is a subview, which in turn is quick to access and modify. The number of scans to generate cumulative offset vectors is drastically reduced, or rather: split up and delayed until needed. Combined with the lazy opening scheme that 2.3 added over 2.0, this makes opening of huge views with B/S data in it a lot quicker. If you think that's the end of the story... nope, there's more. I've experimented a few times in the past with a scheme to build and maintain cumulative size vectors, which are so important in this column-wise concatenated world of MK, in a more gradual and dynamic manner. The idea is to use a bisection-type tracking and invalidation algorithm to locate and update cumulative vectors. The logic is such that it can be applied to columns of variable sized B & S data, but also to blocked views, which are in fact the same thing yet again, but as second layer on top of low-level columns. The reasons I've been holding off, are many - apart from making MK work *correctly* first, and faster second. An important one is that some new interesting generalities have started to emerge in the 2.3/2.4 file format - columns and out-of-band items are one and the same thing, to name an example. So instead of rushing into more file format decisions, I wanted to see just how far the current design can be pushed. Keep in mind that MK was originally aimed at perhaps a few thousand rows of data, and that on-the-fly restructuring came as an afterthought... there are some hugely exciting opportunities left, and the current file format was designed with such changes in mind. Compressed and/or encrypted columns? Heterogenous subviews? Storage-in-a-storage with mmap support? Derived/calculated properties and subviews which persist? Compact commit-aside storage? Combined commit-extend and -aside storage in a single datafile? Versioned/translucent storage? Lock-free concurrent access and modification? Don't hold your breath, but the core can evolve to each of these IMO. So far it's all pure vaporware. But I hope it explains why I'm rather cautious in slapping new features onto MK. At the top of my priorities, is making sure that the MK datafile format will last another decade (or more). There's already a lot of data out there, in a file format that has a lot of growth left in it. IMNSHO. -jcw From nigel@bprj.co.uk Thu Dec 12 07:45:26 2002 Received: from cmailg2.svr.pol.co.uk (cmailg2.svr.pol.co.uk [195.92.195.172]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBCDjPG24392 for ; Thu, 12 Dec 2002 07:45:26 -0600 Received: from modem-241.cougar.dialup.pol.co.uk ([217.134.224.241] helo=jeeves.bprj.co.uk) by cmailg2.svr.pol.co.uk with esmtp (Exim 3.35 #1) id 18MTff-0004RB-00 for metakit@equi4.com; Thu, 12 Dec 2002 13:46:35 +0000 Received: from tuppy (tuppy.bprj.co.uk [192.168.1.199]) by jeeves.bprj.co.uk (8.12.3/8.12.3/SuSE Linux 0.6) with SMTP id gBCDUc7E013902 for ; Thu, 12 Dec 2002 13:30:38 GMT From: "Nigel Hathaway" To: "MetaKit mailing list" Subject: RE: [Metakit] Logical omission - or maybe some clarification required? Date: Thu, 12 Dec 2002 13:30:38 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal In-Reply-To: <671C0AD0-0DCC-11D7-91BB-00050251EF2F@equi4.com> X-Virus-Scanned: by amavis-milter (http://amavis.org/) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks, that's very helpful. ************************************ Barnabas Projects Limited Tel: +44 117 937 4197 Internet: http://www.bprj.co.uk -----Original Message----- From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com] On Behalf Of Jean-Claude Wippler Sent: 12 December 2002 12:23 To: MetaKit mailing list Subject: Re: [Metakit] Logical omission - or maybe some clarification required? Nigel Hathaway wrote: [...] > From reading the docs data is stored as individual columns in the > file, and > the fastest way to locate a data item is to make an offset from the > start of > the column data given by the index of the data item multiplied by its > size, > provided the data is of fixed size. This is true for F, D, L and I > (which I > don't follow, in terms of how the 'adaptive' part works. You don't > specify > how wide you want the integer to be when you set the column up, so if > you > fill it with Boolean values - 1's or 0's - and then add an 8-bit value > in, > does this mean the whole column gets re-written as 8-bits for each > row?). Yes. If you are concerned about this, you can store a big value first to "widen" the datatype. There will at some point be a way to reduce it again, but so far that only happens when copying views (just like there will at some point to be be a "compaction" function, the current design is starting to reach limits which can be avoided). > For non-fixed types (S, B), there are two methods of storage. Strings > are > stored end-to-end, separated by the null-terminator on each, so the > code has > to search all the way down the column (reading each byte) in order to > find > an entry at a particular index - this seems a bit inefficient for large > tables. It is - which is why the 2.3/2.4 file format was changed a bit, see below. > Binary columns use a two stage look-up: it uses two columns the > data stored end-to-end like strings and a column of integer data > sizes, so > again, you have to read all the way down a column to calculate the > position > and length, and then do the look-up - so perhaps not as lengthy as > strings, > but not as efficient as the obsoleted memo column which used two > columns: > one for size and the other for file offset? The B, S, and M datatypes have all been merged into one. The new "B" datatype stores things in three low-level columns: * a set of concatenated strings * a vector of field sizes * a variable-sized "run" of strings which were stored out-of-band For small strings and/or a low number of rows (there's a heuristic there), things are stored as a concatenation, but scans for terminating null bytes are not needed, because all field sizes are stored in a separate int column. Large items are stored out-of-band, i.e. as a separate area in the datafile. The third column then contains information about this, as a sequence of triples: skip, size, position The skip value indicates how many in-band rows to skip, then comes one out-of-band item, of the indicated size and at the indicated file position. A column with only out-of-band data looks like: the data column is empty the offsets column is empty the out-of-band column contains: 0, size0, pos0, 0, size1, pos1, 0, size2, pos2, etc... > What seems to be missing is fixed-sized binary columns (that can't be > contained in integers up to 8 bytes long). But why would you want them > anyway? If it is fixed sized then it implies that you know the > structure, > so it can be broken down into multiple columns of fundamental data > types - > true in some cases. Fixed sized strings and variants (data unions) are > where it would be logical to use such things. You could say that > strings > are never fixed-size, so you waste a lot of space making them so, so > you may > not actually get any retrieval efficiency gains. Data unions > (variants) > which contain a code to identify type followed by a varying length of > data > depending on that code. > > Is there a performance gain to be had from this (over B columns)? Yes, this is an omission I have been planning to get to for some time. The plan is not to introduce a new datatype, but to autodetect integer columns in which each value is the same. So this would end up being a B-type property, with three columns, the first containing the data, the second containing the "constant-int" column (which is a few bytes, and easily singled out), and the third is empty. The B handlers would recognize this format and avoid setting up a cumulative offset vector. The benefit is that it would also cover other cases, and that it would work for all styles: B, S, and what used to be M (i.e. large, but constant-size chunks, think disk blocks, for example). > Is it something that can be easily implemented using existing > underlying > access methods, or would it simply break the whole scheme and file > format? I don't know. My first concern has been for quite some time now to get MK 2.4 up to the same level of stability and robustness as MK 2.01 was. That is now the case, I think. So looking into this would be feasible - though I don't have time for it this very moment. Having said that, note that you can quite effectively simulate this mechanism in a different way: have one huge B-type, i.e. binary property, i.e. one row. Then use the partial Access/Modify calls to maintain such a single item as an array of fixed-size chunks. You wouldn't be able to search and operate on it with other View operations, but even that can probably be simulated by creating a custom viewer which maps each chunk to a fake property. One more comment w.r.t performance in variable-sized B's and S's when there are a very large number of rows: this is exactly what "blocked" views are designed to improve upon. They turn a view which logically looks like one big one into one which is physically stored in "chunky format". Each of those chunks is a subview, which in turn is quick to access and modify. The number of scans to generate cumulative offset vectors is drastically reduced, or rather: split up and delayed until needed. Combined with the lazy opening scheme that 2.3 added over 2.0, this makes opening of huge views with B/S data in it a lot quicker. If you think that's the end of the story... nope, there's more. I've experimented a few times in the past with a scheme to build and maintain cumulative size vectors, which are so important in this column-wise concatenated world of MK, in a more gradual and dynamic manner. The idea is to use a bisection-type tracking and invalidation algorithm to locate and update cumulative vectors. The logic is such that it can be applied to columns of variable sized B & S data, but also to blocked views, which are in fact the same thing yet again, but as second layer on top of low-level columns. The reasons I've been holding off, are many - apart from making MK work *correctly* first, and faster second. An important one is that some new interesting generalities have started to emerge in the 2.3/2.4 file format - columns and out-of-band items are one and the same thing, to name an example. So instead of rushing into more file format decisions, I wanted to see just how far the current design can be pushed. Keep in mind that MK was originally aimed at perhaps a few thousand rows of data, and that on-the-fly restructuring came as an afterthought... there are some hugely exciting opportunities left, and the current file format was designed with such changes in mind. Compressed and/or encrypted columns? Heterogenous subviews? Storage-in-a-storage with mmap support? Derived/calculated properties and subviews which persist? Compact commit-aside storage? Combined commit-extend and -aside storage in a single datafile? Versioned/translucent storage? Lock-free concurrent access and modification? Don't hold your breath, but the core can evolve to each of these IMO. So far it's all pure vaporware. But I hope it explains why I'm rather cautious in slapping new features onto MK. At the top of my priorities, is making sure that the MK datafile format will last another decade (or more). There's already a lot of data out there, in a file format that has a lot of growth left in it. IMNSHO. -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From v.digilio@sisdata.it Thu Dec 12 12:47:20 2002 Received: from smtp2.cp.tin.it (vsmtp2.tin.it [212.216.176.222]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBCIlKG23046 for ; Thu, 12 Dec 2002 12:47:20 -0600 Received: from w2k (62.211.56.206) by smtp2.cp.tin.it (6.5.029) id 3DEE435A00387EE5 for metakit@equi4.com; Thu, 12 Dec 2002 19:48:30 +0100 Message-ID: <003201c2a20f$246eb740$2800a8c0@w2k> From: "Vittorio Digilio" To: Date: Thu, 12 Dec 2002 19:49:03 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] c4_Property Shouldn't require a virtual destructor? [typo] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sorry for the typo, obviusly I meant : c4_Property *pMyInt=new c4_IntProp("age"); // Had to be c4_Property * ! //.... //.... delete pMyInt; // c4_Property::~c4_Property() should'nt be called // and c4_Property::Refs(-1) isn't called either. Cheers, vdg From edmund@troche.org Thu Dec 12 21:45:41 2002 Received: from txsmtp03.texas.rr.com (smtp3.texas.rr.com [24.93.36.231]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBD3jeG24951 for ; Thu, 12 Dec 2002 21:45:41 -0600 Received: from [192.168.1.102] (cs2875-42.austin.rr.com [24.28.75.42]) by txsmtp03.texas.rr.com (8.12.5/8.12.2) with ESMTP id gBD3iAUq028021; Thu, 12 Dec 2002 22:44:11 -0500 (EST) Subject: Re: [Metakit] mmap From: Edmund Troche To: Bruno Blondeau Cc: metakit@equi4.com In-Reply-To: References: Content-Type: text/plain; charset=ISO-8859-15 Organization: Message-Id: <1039751039.26458.4.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.0.99 Date: 12 Dec 2002 21:44:00 -0600 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Did you try msync() ? Edmund Troche On Mon, 2002-12-09 at 11:45, Bruno Blondeau wrote: > Hello, > > Could someone tell me how to force changes to the disk when mmap is being > used by a Metakit database? > > I'm not used to work with mmap, so any sample code which would be integrated > with Metakit would be helpful. > > Thanks, > Bruno Blondeau http://www.iOrganizeX.com > > > > > > > > _____________________________________________________________________ > GRAND JEU SMS : Pour gagner un NOKIA 7650, envoyez le mot IF au 61321 > (prix d'un SMS + 0.35 euro). Un SMS vous dira si vous avez gagné. > Règlement : http://www.ifrance.com/_reloc/sign.sms > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- Edmund Troche From njriley@theremin.cs.uiuc.edu Fri Dec 13 00:06:40 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBD66YG29188 for ; Fri, 13 Dec 2002 00:06:40 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id AAA3117808 for metakit@equi4.com; Fri, 13 Dec 2002 00:07:50 -0600 (CST) Date: Fri, 13 Dec 2002 00:07:50 -0600 From: Nicholas Riley To: metakit@equi4.com Message-ID: <20021213060750.GE3370954@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Subject: [Metakit] Python bindings and 'throw 1' Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I've been working on fixing the overflow/long long support problems with Mk4py and have been finding many issues with the bindings that I've been fixing as I go along. I've also written the beginnings of an automated test suite for Mk4py. I'm cleaning everything up and will post a preliminary patch later tonight if all goes well. I notice about a year ago the PWException class was removed, and exceptions are now signaled by 'throw 1'. The CHANGES file has this to say: Simplified Mk4py - by removing a layer of exception handler classes in scxx. Errors now set info and throw a plain int. It was only after readding my own PWDException class that I discovered a PWException class was present in scxx before you excised it. The PWDException ("D" is for "dummy") is about as simple as it can be: class PWDException { private: PWDException(); } *PWDPyException(); My intuition is that throwing PWDPyException should not be any more expensive than throwing 1, and it should be a lot easier to understand what is happening. Is this OK? -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign From scusack@helpisit.com.au Fri Dec 13 02:34:38 2002 Received: from mail010.syd.optusnet.com.au (mail010.syd.optusnet.com.au [210.49.20.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBD8YbG05112 for ; Fri, 13 Dec 2002 02:34:38 -0600 Received: from vfr (sdcax56-042.dialup.optusnet.com.au [198.142.217.42]) by mail010.syd.optusnet.com.au (8.11.1/8.11.1) with SMTP id gBD8ZpF22145 for ; Fri, 13 Dec 2002 19:35:52 +1100 From: Simon Cusack To: MetaKit Date: Fri, 13 Dec 2002 08:41:07 +1100 X-Priority: 3 (Normal) Reply-To: scusack@helpisit.com.au Message-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1252" X-Mailer: Opera 6.05 build 1140 Subject: [Metakit] small borland change for metakit-2.4.8 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The new 2.4.8 is great, I had to make a small change for borland to compile it. In src\univ.cpp I had change line 22 from: #if !q4_MSVC && !q4_WATC && !(q4_MWCW && defined(_WIN32)) to : #if !q4_BORC && !q4_MSVC && !q4_WATC && !(q4_MWCW && defined(_WIN32)) to build for borland builder 5 and 6. Simon From nigel@bprj.co.uk Fri Dec 13 05:19:30 2002 Received: from cmailm3.svr.pol.co.uk (cmailm3.svr.pol.co.uk [195.92.193.19]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBDBJTG13876 for ; Fri, 13 Dec 2002 05:19:30 -0600 Received: from modem-970.elephant.dialup.pol.co.uk ([217.134.243.202] helo=jeeves.bprj.co.uk) by cmailm3.svr.pol.co.uk with esmtp (Exim 3.35 #1) id 18Mns5-00043A-00 for metakit@equi4.com; Fri, 13 Dec 2002 11:20:45 +0000 Received: from tuppy (tuppy.bprj.co.uk [192.168.1.199]) by jeeves.bprj.co.uk (8.12.3/8.12.3/SuSE Linux 0.6) with SMTP id gBDB1f7E019800 for ; Fri, 13 Dec 2002 11:01:41 GMT From: "Nigel Hathaway" To: "MetaKit mailing list" Date: Fri, 13 Dec 2002 11:01:43 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal In-Reply-To: X-Virus-Scanned: by amavis-milter (http://amavis.org/) Subject: [Metakit] Sparse tables Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I suspect this has been addressed before, but since it is something I am looking to do in my current project, I'm posting this anyway. Let's say we have an ordinary table, and it has (in the limiting case) 250 columns and 100,000 rows. The data type of each column is known and fixed, as are the number of columns. Each row has 1 or more of its columns containing data, and the rest are not used or null. The table is sparsely populated, that is, 90% of all the cells contain nulls randomly but evenly distributed (for sake of argument). Metakit has currently no way of representing a null entry (as opposed to 0 or some other default). I was originally going to tackle this using a classic property list approach: each row is a sub-view of two columns; column 1 is a string which is the table column name and column 2 is a variant (or union) containing data appropriate to the column type - this was the motivation behind my last posting regarding fixed-size binary columns. Giving it some further thought, I have come up with another idea: Create a table with the 250 columns, but don't populate it in the conventional way. Instead, have an auxiliary table where each row corresponds to the row in the required sparse table, with the table containing a single column which is a two-column sub-view: column number (adaptive integer) and row number (adaptive integer), and these correspond the location of the real data in our 250-column table, which has no empty spaces. Of course, this means that the columns in the big table have a varying number of rows in them, so arguably what you need is 250 single-column tables (which in MetaKit's case is virtually the same anyway). The question is: how to implement? This is obviously a classic case for a custom viewer similar to the hash map (for example). You only access the data through the custom view, which itself maintains the integrity of both tables. Each time you add a record, you add an item onto the end of the relevant columns, and create entries for each in the lookup table. But what happens when you delete an entry? You take out items of data out of the middle of several columns, and those columns shift down one from the point of deletions. This invalidates the data in the lookup table for reference to those columns beyond the deletion points. Does this mean you have to iterate over the entire lookup table set and make adjustments, or is there a better way of doing this? Maybe this has already been done and I have missed it? Another question: can you chain different custom views on to another? For example, I have my sparse table custom viewer, but I still want to index the table for fast look-up, e.g. by hashing on the values of a certain column (ignoring the fact that there could be null values). Can I use the hash view which in turn uses the sparse view which in turn accesses the raw data? I am still evaluating storage libraries for my data. I am writing cross platform Macintosh (classic and OS X) and Windows, so this severely limits my choice. One other candidate is SQLite. However, MetaKit still comes out on top in the suitability stakes. Given that I need to write this anyway (and it won't be on JCW's agenda in my timescales) it makes sense for me to contribute this into the project, given also its generic nature. Assuming I go down this route, are you (JCW) open to this? - I would prefer not to be the maintainer of such a small chunk of code in the long term. Would you (JCW) be willing to give me further help/pointers to ensure that it conforms to your overall style and schema? Regards, Nigel Hathaway ************************************ Barnabas Projects Limited Tel: +44 117 937 4197 Internet: http://www.bprj.co.uk From njriley@theremin.cs.uiuc.edu Fri Dec 13 11:05:44 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBDH5hG30890 for ; Fri, 13 Dec 2002 11:05:44 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id LAA3446334 for metakit@equi4.com; Fri, 13 Dec 2002 11:07:00 -0600 (CST) Date: Fri, 13 Dec 2002 11:06:59 -0600 From: Nicholas Riley To: metakit@equi4.com Message-ID: <20021213170659.GA3439635@uiuc.edu> References: <20021203033441.GE2890677@uiuc.edu> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="ew6BAiZeqk4r7MaW" Content-Disposition: inline In-Reply-To: <20021203033441.GE2890677@uiuc.edu> User-Agent: Mutt/1.4i Subject: [Metakit] [PATCH] Can't use longs in Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --ew6BAiZeqk4r7MaW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Here's the patch. This includes: * the OS X build fixes I included in my previous patches, with a few tweaks (actually check for Python.h) * changes to the Python bindings to properly support 'L' fields, handle overflow, and throw dummy exceptions * the beginnings of a test suite for the Python bindings, using the test.regrtest module since it's the closest standard module Python has to MetaKit's Tcl tests; a corresponding test-python target in Makefile.in * a few more build changes - scxx dependencies were wrong A couple of further questions: * The other problem I had with Mk4py was its inability to deal with Unicode strings. Is there any precedent in the other language bindings for what to do here? Do 'S' fields have any encoding information associated with them? * Currently, the int type conversion matrix looks something like this: Python type MetaKit type Result ----------- ------------ ------ int I ok long int <= 32 bits I ok long int > 32 bits I TypeError float I ok, truncated float/int > 32 bits I TypeError long int <= 64 bits L ok long int > 64 bits L ValueError float L ok, truncated float/int > 64 bits L ValueError string I, L TypeError A few of these conversions are debatable - should Mk4py truncate a float? Should it try to convert a string to an int/long int if possible? I've got finals coming up so I won't be able to do anything further until next Tuesday. -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign --ew6BAiZeqk4r7MaW Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="metakit-2.4.8+njr2.diff" diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/python/PyRowRef.cpp metakit-2.4.8+njr2/python/PyRowRef.cpp --- metakit-2.4.8/python/PyRowRef.cpp Wed Dec 19 10:26:00 2001 +++ metakit-2.4.8+njr2/python/PyRowRef.cpp Fri Dec 13 09:56:04 2002 @@ -58,7 +58,7 @@ PyErr_Clear(); return Py_FindMethod(RowRefMethods, (PyObject* )o, nm); } - catch (...) { return 0; } + catch (...) { return NULL; } } static int PyRowRef_setattr(PyRowRef *o, char *nm, PyObject* v) { @@ -72,7 +72,7 @@ Py_DECREF(p); return 0; } - PyErr_SetString(PyExc_AttributeError, "delete of non-existing attribute"); + PyErr_SetString(PyExc_AttributeError, "delete of nonexistent attribute"); return -1; } catch (...) { return -1; } @@ -122,8 +122,8 @@ // with thanks to Niki Spahiev for improving conversions and error checks void PyRowRef::setFromPython(const c4_RowRef& row, const c4_Property& prop, PyObject* attr) { switch (prop.Type()) { - case 'I': - if (PyInt_Check(attr)) + case 'I': + if (PyInt_CheckExact(attr)) ((const c4_IntProp&) prop) (row) = PyInt_AS_LONG(attr); else if (attr != Py_None) { @@ -132,16 +132,20 @@ } break; #ifdef HAVE_LONG_LONG - case 'L': - if (PyInt_Check(attr)) + case 'L': + if (PyInt_CheckExact(attr)) ((const c4_LongProp&) prop) (row) = PyInt_AS_LONG(attr); - else if (PyLong_Check(attr)) - ((const c4_LongProp&) prop) (row) = PyLong_AsLong(attr); + else if (PyLong_Check(attr)) { + long long number = PyLong_AsLongLong(attr); + if (number == -1 && PyErr_Occurred() != NULL) + Fail(PyExc_ValueError, "long int too large to convert to C long long"); + ((const c4_LongProp&) prop) (row) = number; + } else if (attr != Py_None) - { - PWONumber number (attr); - ((const c4_LongProp&) prop) (row) = (long) number; - } + { + PWONumber number (attr); + ((const c4_LongProp&) prop) (row) = (long long) number; + } break; #endif case 'F': @@ -163,7 +167,7 @@ } break; case 'S': - if (PyString_Check(attr)) { + if (PyString_CheckExact(attr)) { c4_Bytes temp (PyString_AS_STRING(attr), PyString_GET_SIZE(attr) + 1, false); prop (row).SetData(temp); @@ -191,13 +195,16 @@ break; case 'B': case 'M': - if (PyString_Check(attr)) { + if (PyString_CheckExact(attr)) { c4_Bytes temp (PyString_AS_STRING(attr), PyString_GET_SIZE(attr), false); prop (row).SetData(temp); } else if (attr != Py_None) Fail(PyExc_TypeError, "wrong type for ByteProp"); + default: + PyErr_Format(PyExc_TypeError, "unknown property type '%c'", prop.Type()); + throw PWDPyException; } } @@ -214,9 +221,9 @@ case 'F': ((const c4_FloatProp&) prop) (row) = 0.0; break; - case 'D': - ((const c4_DoubleProp&) prop) (row) = 0.0; - break; + case 'D': + ((const c4_DoubleProp&) prop) (row) = 0.0; + break; case 'S': ((const c4_StringProp&) prop) (row) = ""; break; @@ -229,6 +236,10 @@ c4_Bytes temp; prop (row).SetData(temp); } + break; + default: + PyErr_Format(PyExc_TypeError, "unknown property type '%c'", prop.Type()); + throw PWDPyException; } } @@ -240,7 +251,7 @@ } #ifdef HAVE_LONG_LONG case 'L': { - return PyLong_FromLong((long) ((const c4_LongProp&)prop)(*this)); + return PyLong_FromLongLong((long long) ((const c4_LongProp&)prop)(*this)); } #endif case 'F': { @@ -265,6 +276,7 @@ PWOString rslt((const char*)temp.Contents(), temp.Size()); return rslt.disOwn(); } + default: + return PyErr_Format(PyExc_TypeError, "unknown property type '%c'", prop.Type()); } - return 0; } diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/python/scxx/PWOBase.h metakit-2.4.8+njr2/python/scxx/PWOBase.h --- metakit-2.4.8/python/scxx/PWOBase.h Mon Dec 3 16:50:59 2001 +++ metakit-2.4.8+njr2/python/scxx/PWOBase.h Fri Dec 13 09:11:20 2002 @@ -10,7 +10,17 @@ #include #include +// Dummy, minimal exception thrown when a Python exception is generated +// (after PyErr_Format, PyErr_SetString, etc.) +class PWDException +{ + public: + PWDException() {} +} extern const& PWDPyException; + +// functions throw PWDPyException void Fail(PyObject*, const char* msg); +void FailIfPyErr(); class PWOBase { diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/python/scxx/PWOImp.cpp metakit-2.4.8+njr2/python/scxx/PWOImp.cpp --- metakit-2.4.8/python/scxx/PWOImp.cpp Mon Dec 3 16:50:59 2001 +++ metakit-2.4.8+njr2/python/scxx/PWOImp.cpp Fri Dec 13 09:12:05 2002 @@ -7,6 +7,10 @@ #include "PWOMapping.h" #include "PWOCallable.h" +// dummy exception singleton +const PWDException PWDPyExceptionObj; +const PWDException& PWDPyException = PWDPyExceptionObj; + // incref new owner, and decref old owner, and adjust to new owner void PWOBase::GrabRef(PyObject* newObj) { @@ -37,24 +41,30 @@ static PWOTuple _empty; PyObject *rslt = PyEval_CallObjectWithKeywords(*this, _empty, NULL); if (rslt == 0) - throw 1; + throw PWDPyException; return rslt; } PWOBase PWOCallable::call(PWOTuple& args) const { PyObject *rslt = PyEval_CallObjectWithKeywords(*this, args, NULL); if (rslt == 0) - throw 1; + throw PWDPyException; return rslt; } PWOBase PWOCallable::call(PWOTuple& args, PWOMapping& kws) const { PyObject *rslt = PyEval_CallObjectWithKeywords(*this, args, kws); if (rslt == 0) - throw 1; + throw PWDPyException; return rslt; } void Fail(PyObject* exc, const char* msg) { PyErr_SetString(exc, msg); - throw 1; + throw PWDPyException; +} + +void FailIfPyErr() +{ + PyObject *exc = PyErr_Occurred(); + if (exc != NULL) throw PWDPyException; } diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/python/scxx/PWOMapping.h metakit-2.4.8+njr2/python/scxx/PWOMapping.h --- metakit-2.4.8/python/scxx/PWOMapping.h Fri Feb 1 09:32:51 2002 +++ metakit-2.4.8+njr2/python/scxx/PWOMapping.h Tue Dec 10 17:10:37 2002 @@ -56,7 +56,7 @@ //PyDict_GetItemString PWOMappingMmbr operator [] (const char* key) { PyObject* rslt = PyMapping_GetItemString(_obj, (char*) key); - if (rslt==0) + if (rslt == NULL) PyErr_Clear(); PWOString _key(key); return PWOMappingMmbr(rslt, *this, _key); @@ -64,7 +64,7 @@ //PyDict_GetItem PWOMappingMmbr operator [] (PyObject* key) { PyObject* rslt = PyDict_GetItem(_obj, key); - //if (rslt==0) + //if (rslt == NULL) // Fail(PyExc_KeyError, "Key not found"); return PWOMappingMmbr(rslt, *this, key); }; diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/python/scxx/PWONumber.h metakit-2.4.8+njr2/python/scxx/PWONumber.h --- metakit-2.4.8/python/scxx/PWONumber.h Mon Dec 3 16:50:59 2001 +++ metakit-2.4.8+njr2/python/scxx/PWONumber.h Fri Dec 13 09:56:23 2002 @@ -15,6 +15,9 @@ PWONumber(int i) : PWOBase (PyInt_FromLong(i)) { LoseRef(_obj); } PWONumber(long i) : PWOBase (PyInt_FromLong(i)) { LoseRef(_obj); } PWONumber(unsigned long i) : PWOBase (PyLong_FromUnsignedLong(i)) { LoseRef(_obj); } +#ifdef HAVE_LONG_LONG + PWONumber(long long i) : PWOBase (PyLong_FromLongLong(i)) { LoseRef(_obj); } +#endif PWONumber(double d) : PWOBase (PyFloat_FromDouble(d)) { LoseRef(_obj); } PWONumber(const PWONumber& other) : PWOBase(other) {}; @@ -35,27 +38,27 @@ virtual void _violentTypeCheck() { if (!PyNumber_Check(_obj)) { GrabRef(0); - Fail(PyExc_TypeError, "Not a number"); + Fail(PyExc_TypeError, "not a number"); } }; //PyNumber_Absolute PWONumber abs() const { PyObject* rslt = PyNumber_Absolute(_obj); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Failed to get absolute value"); return LoseRef(rslt); }; //PyNumber_Add PWONumber operator+(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Add(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for +"); return LoseRef(rslt); }; //PyNumber_And PWONumber operator&(const PWONumber& rhs) const { PyObject* rslt = PyNumber_And(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for &"); return LoseRef(rslt); }; @@ -63,115 +66,126 @@ //PyNumber_Divide PWONumber operator/(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Divide(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for /"); return LoseRef(rslt); }; //PyNumber_Divmod PWOSequence divmod(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Divmod(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for divmod"); return LoseRef(rslt); }; //PyNumber_Float - operator double () const { - PyObject* F = PyNumber_Float(_obj); - if (F==0) + operator double () const { + PyObject* F = PyNumber_Float(_obj); + if (F == NULL) Fail(PyExc_TypeError, "Cannot convert to double"); - double r = PyFloat_AS_DOUBLE(F); - Py_DECREF(F); - return r; - }; + double r = PyFloat_AS_DOUBLE(F); + Py_DECREF(F); + return r; + }; + /* // no easy, safe way to do this operator float () const { double rslt = (double) *this; - //if (rslt > INT_MAX) - // Fail(PyExc_TypeError, "Cannot convert to a float"); return (float) rslt; - }; + }; */ //PyNumber_Int - operator long () const { - PyObject* Int = PyNumber_Int(_obj); - if (Int==0) - Fail(PyExc_TypeError, "Cannot convert to long"); - long r = PyInt_AS_LONG(Int); - Py_DECREF(Int); - return r; - }; + operator long () const { + PyObject* Int = PyNumber_Int(_obj); + if (Int == NULL) + Fail(PyExc_TypeError, "can't convert to int"); + long r = PyInt_AsLong(_obj); + if (r == -1) FailIfPyErr(); + return r; + }; operator int () const { long rslt = (long) *this; if (rslt > INT_MAX) - Fail(PyExc_TypeError, "Cannot convert to an int"); + Fail(PyExc_ValueError, "int too large to convert to C int"); return (int) rslt; }; //PyNumber_Invert PWONumber operator~ () const { PyObject* rslt = PyNumber_Invert(_obj); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper type for ~"); return LoseRef(rslt); }; //PyNumber_Long +#ifdef HAVE_LONG_LONG + operator long long () const { + PyObject* Long = PyNumber_Long(_obj); + if (Long == NULL) + Fail(PyExc_TypeError, "can't convert to long int"); + long r = PyLong_AsLongLong(Long); + if (r == -1 && PyErr_Occurred() != NULL) + Fail(PyExc_ValueError, "long int too large to convert to C long long"); + Py_DECREF(Long); + return r; + }; +#endif //PyNumber_Lshift PWONumber operator<<(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Lshift(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for <<"); return LoseRef(rslt); }; //PyNumber_Multiply PWONumber operator*(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Multiply(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for *"); return LoseRef(rslt); }; //PyNumber_Negative PWONumber operator- () const { PyObject* rslt = PyNumber_Negative(_obj); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper type for unary -"); return LoseRef(rslt); }; //PyNumber_Or PWONumber operator|(const PWONumber& rhs) const { - PyObject* rslt = PyNumber_Or(_obj, rhs); - if (rslt==0) + PyObject* rslt = PyNumber_Or(_obj, rhs); + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for |"); return LoseRef(rslt); }; //PyNumber_Positive PWONumber operator+ () const { PyObject* rslt = PyNumber_Positive(_obj); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper type for unary +"); return LoseRef(rslt); }; //PyNumber_Remainder PWONumber operator%(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Remainder(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for %"); return LoseRef(rslt); }; //PyNumber_Rshift PWONumber operator>>(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Rshift(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for >>"); return LoseRef(rslt); }; //PyNumber_Subtract PWONumber operator-(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Subtract(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for -"); return LoseRef(rslt); }; //PyNumber_Xor PWONumber operator^(const PWONumber& rhs) const { PyObject* rslt = PyNumber_Xor(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for ^"); return LoseRef(rslt); }; diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/python/scxx/PWOSequence.h metakit-2.4.8+njr2/python/scxx/PWOSequence.h --- metakit-2.4.8/python/scxx/PWOSequence.h Mon Dec 3 16:50:59 2001 +++ metakit-2.4.8+njr2/python/scxx/PWOSequence.h Tue Dec 10 17:10:17 2002 @@ -35,7 +35,7 @@ //PySequence_Concat PWOSequence operator+(const PWOSequence& rhs) const { PyObject* rslt = PySequence_Concat(_obj, rhs); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_TypeError, "Improper rhs for +"); return LoseRef(rslt); }; @@ -82,7 +82,7 @@ //PySequence_Repeat PWOSequence operator * (int count) const { PyObject* rslt = PySequence_Repeat(_obj, count); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_RuntimeError, "sequence repeat failed"); return LoseRef(rslt); }; @@ -159,7 +159,7 @@ }; static PWOString format(const PWOString& fmt, PWOTuple& args){ PyObject * rslt =PyString_Format(fmt, args); - if (rslt==0) + if (rslt == NULL) Fail(PyExc_RuntimeError, "string format failed"); return LoseRef(rslt); }; diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/python/test/all.py metakit-2.4.8+njr2/python/test/all.py --- metakit-2.4.8/python/test/all.py Wed Dec 31 18:00:00 1969 +++ metakit-2.4.8+njr2/python/test/all.py Thu Dec 12 17:51:54 2002 @@ -0,0 +1,29 @@ +# all.py -- Run all tests for the MetaKit Python bindings +# $Id$ +# This is part of MetaKit, see http://www.equi4.com/metakit/ + +import sys +import os +import test.regrtest + +def canonicalPath(path): + """Do everything but resolve symbolic links to create an absolute path.""" + return os.path.abspath(os.path.expanduser(os.path.expandvars(path))) + +# from Python 2.2's regrtest module +def findtestdir(): + if __name__ == '__main__': + file = sys.argv[0] + else: + file = __file__ + testdir = os.path.dirname(file) or os.curdir + return testdir + +testdir = canonicalPath(findtestdir()) + +# Don't run the standard Python tests, just run MetaKit tests +test.regrtest.STDTESTS = [] +test.regrtest.NOTTESTS = [] + +# Take a look at the +test.regrtest.main(testdir=testdir) diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/python/test/test_inttypes.py metakit-2.4.8+njr2/python/test/test_inttypes.py --- metakit-2.4.8/python/test/test_inttypes.py Wed Dec 31 18:00:00 1969 +++ metakit-2.4.8+njr2/python/test/test_inttypes.py Fri Dec 13 09:57:14 2002 @@ -0,0 +1,178 @@ +# test_inttypes.py -- Test MetaKit Python bindings for integral types +# $Id$ +# This is part of MetaKit, see http://www.equi4.com/metakit/ + +from test.test_support import TestFailed, verbose +import os +import sys + +# Keep track of failures as they happen, but don't die on the first +# one unless it's unrecoverable. If failure_count > 0 when script +# finishes, raise TestFailed. +failure_count = 0 + +def fail(op, args, err=None, expected=None, actual=None): + global failure_count + print 'FAIL:', op, args + print ' ', + if err is not None: print err, + if actual is not None: print 'got', actual, actual.__class__, + if expected is not None: print 'expected', expected, + print + failure_count = failure_count + 1 + +# for overflow testing +MAXINT = sys.maxint +MININT = -MAXINT - 1 +MAXLONGLONG = 2**63 - 1 +MINLONGLONG = -2**63 +MAXULONGLONG = 2**64 + +# Make sure we're using modules from the builds directory, assuming +# that's the current directory at the time we're run. +sys.path.insert(0, os.getcwd()) + +import metakit + +# Check that data types are as we expect (will break with Python 2.4 +# int/long integration?) +try: + MAXLONGLONG = int(MAXLONGLONG) + raise TestFailed('not expecting 2^63 - 1 to be a valid Python integer') +except OverflowError: + pass + +storage = metakit.storage() +v = storage.getas('test[intf:I,longf:L]') + +# insert rows into view and Python array +arr = [] + +def checklen(**args): + alen = len(arr) + vlen = len(v) + if alen != vlen: + fail('append', args, 'view length mismatch', actual=vlen, expected=alen) + try: + print 'ARRAY CONTENTS:' + for arow in arr: print arow + metakit.dump(v, 'VIEW CONTENTS:') + except: pass + raise TestFailed('unexpected number of rows in view, aborting; run in verbose mode for details') + +def insert(**args): + try: + v.append(args) + arr.append(args) + except Exception, e: + fail('append', args, actual=e) + try: + checklen(**args) + except TestFailed: + raise + except Exception, e: + fail('append', args, 'spurious', actual=e) + +def reject(exception_class=Exception, **args): + try: + ix = v.append(args) + fail('append', args, 'succeeded', expected=exception_class) + v.delete(ix) + except Exception, e: + if isinstance(e, exception_class): + if verbose: + print 'PASS: rejected', args + print ' as expected <%s> %s' % (e.__class__, e) + else: + fail('append', args, expected=exception_class, actual=e) + try: + checklen(**args) + except TestFailed: + raise + except Exception, e: + fail('append', args, 'spurious', actual=e) + +# defaults +insert(intf=0, longf=0) + +# int field +insert(intf=1, longf=0) +insert(intf=-5, longf=0) +insert(intf=MAXINT, longf=0) +insert(intf=MININT, longf=0) +reject(TypeError, intf=MAXINT + 1, longf=0) +reject(TypeError, intf=MININT - 1, longf=0) + +# long field +insert(intf=0, longf=-1L) +insert(intf=0, longf=5L) +insert(intf=0, longf=MAXLONGLONG) +insert(intf=0, longf=MINLONGLONG) +reject(ValueError, intf=0, longf=MAXULONGLONG) +reject(ValueError, intf=0, longf=MAXLONGLONG + 1) +reject(ValueError, intf=0, longf=MAXULONGLONG) +reject(ValueError, intf=0, longf=MINLONGLONG - 1) + +# mixed valid int/long +insert(intf=1, longf=2) +insert(intf=-5, longf=-2**30) + +# implicit conversion to int +insert(intf=14L, longf=0) +insert(intf=-30L, longf=0) +insert(intf=45.0, longf=0) +insert(intf=21.4, longf=0) +reject(TypeError, intf=float(MAXINT + 1), longf=0) +reject(TypeError, intf=float(MININT - 1), longf=0) +reject(TypeError, intf='215', longf=0) +reject(TypeError, intf='-318.19', longf=0) +reject(TypeError, intf=str(MAXINT + 1), longf=0) + +# implicit conversion to long +insert(intf=0, longf=278) +insert(intf=0, longf=-213) +insert(intf=0, longf=95.0) +insert(intf=0, longf=27.3) +reject(ValueError, intf=0, longf=float(2 * MAXLONGLONG)) +reject(ValueError, intf=0, longf=float(2 * MINLONGLONG)) +reject(TypeError, intf=0, longf=str(MAXLONGLONG)) +reject(TypeError, intf=0, longf=str(MINLONGLONG)) +reject(TypeError, intf=0, longf='-21.39') +reject(TypeError, intf=0, longf=str(MAXULONGLONG)) + +# XXX should repeat with assignment instead of appending +# XXX test v.select() + +if verbose: + metakit.dump(v, 'VIEW CONTENTS:') + +# compare view with array +for arow, vrow in zip(arr, v): + failed = False + for f in arow.keys(): + try: + vf = getattr(vrow, f) + af = arow[f] + if af == vf: + continue + # Perform the same implicit coercion as Mk4py should + if type(af) != type(vf): + try: + af = type(vf)(af) + if af == vf: + continue + except: + pass + # If we get here, we got an exception or the values didn't match + # even with coercion + failed = True + fail('%s access' % f, arow, expected=af, actual=vf) + except Exception, e: + failed = True + fail('%s access' % f, arow, expected=arow[f], actual=e) + if not failed: + if verbose: + print 'PASS: retrieved', arow + +if failure_count > 0: + raise TestFailed('%d failures; run in verbose mode for details' % failure_count) diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/unix/Makefile.in metakit-2.4.8+njr2/unix/Makefile.in --- metakit-2.4.8/unix/Makefile.in Sat Nov 2 16:29:13 2002 +++ metakit-2.4.8+njr2/unix/Makefile.in Fri Dec 13 10:30:01 2002 @@ -34,12 +34,16 @@ srcdir = @srcdir@ top_builddir = . -pyincludedir = @includedir@/python2.2 -pylibdir = @libdir@/python2.2/site-packages +python = python +pyincludedir = @PY_INCLUDE_DIR@ +pylibdir = @PY_LIB_DIR@ tclsh = tclsh tclincludedir = @TCL_INCLUDE_DIR@ -tcllibdir = @TCL_INCLUDE_DIR@/../lib +tcllibdir = @TCL_LIB_DIR@ + +CPPFLAGS = @CPPFLAGS@ +LDFLAGS = @LDFLAGS@ # Compiling without frame pointers can play tricks with exception handling # (e.g. in Mk4py). This does not affect standard operation, *only* errors. @@ -55,7 +59,8 @@ INSTALL_DATA = $L --mode=install @INSTALL_DATA@ INSTALL_PROGRAM = $L --mode=install @INSTALL_PROGRAM@ -LIBTOOL_FLAGS = @LIBTOOL_FLAGS@ +LIBTOOL_SHLIB_FLAGS = $(LDFLAGS) @LIBTOOL_SHLIB_FLAGS@ +LIBTOOL_MODULE_FLAGS = $(LDFLAGS) @LIBTOOL_MODULE_FLAGS@ STRIP_FLAGS = @STRIP_FLAGS@ @@ -63,20 +68,20 @@ #---------- Do not change, shorthand only -CXX_SWITCHES = $(CXXFLAGS) \ +CXX_SWITCHES = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(srcdir)/../src -I. -CXX_SWITCHES_TCL = $(CXXFLAGS) \ +CXX_SWITCHES_TCL = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(tclincludedir)/generic \ -I$(tclincludedir) -CXX_SWITCHES_PY = $(CXXFLAGS) \ +CXX_SWITCHES_PY = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(srcdir)/../python/scxx \ -I$(pyincludedir) -CXX_SWITCHES_LUA = $(CXXFLAGS) \ +CXX_SWITCHES_LUA = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include -CXX_SWITCHES_TEST = $(CXXFLAGS) \ +CXX_SWITCHES_TEST = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include #---------- The targets normally specified when calling "make" @@ -87,7 +92,7 @@ tcl: Makefile Mk4tcl@SHLIB_SUFFIX@ -python: Makefile Mk4py@SHLIB_SUFFIX@ +python: Makefile Mk4py@MODULE_SUFFIX@ test: Makefile libmk4.la regress test -d tests || mkdir tests @@ -99,6 +104,9 @@ test-tcl: tcl cd $(srcdir)/../tcl/test && $(tclsh) all.tcl +test-python: python + $(python) $(srcdir)/../python/test/all.py + install: @MK_INSTALL@ install-mk: libmk4.la @@ -114,9 +122,9 @@ $(INSTALL_PROGRAM) Mk4tcl@SHLIB_SUFFIX@ $(DESTDIR)$(tcllibdir)/Mk4tcl echo 'package ifneeded Mk4tcl 2.4.8 [list load [file join $$dir Mk4tcl@SHLIB_SUFFIX@] Mk4tcl]' >$(DESTDIR)$(tcllibdir)/Mk4tcl/pkgIndex.tcl -install-python: Mk4py@SHLIB_SUFFIX@ - $(INSTALL_PROGRAM) Mk4py@SHLIB_SUFFIX@ $(DESTDIR)$(pylibdir) - $(INSTALL_PROGRAM) $(srcdir)/../python/metakit.py $$(DESTDIR)(pylibdir) +install-python: Mk4py@MODULE_SUFFIX@ + $(INSTALL_PROGRAM) Mk4py@MODULE_SUFFIX@ $(DESTDIR)$(pylibdir) + $(INSTALL_PROGRAM) $(srcdir)/../python/metakit.py $(DESTDIR)$(pylibdir) clean: $L rm -f *.la *.o *.lo @@ -138,7 +146,7 @@ cd $(srcdir) && autoconf libmk4.la: $(LIBOBJS) $(LINK_SPECIAL_FILES) - $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_FLAGS) -avoid-version \ + $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_SHLIB_FLAGS) -avoid-version \ -rpath $(libdir) $(SHLOBJS) $(LINK_SPECIAL_FLAGS) Mk4tcl@SHLIB_SUFFIX@: libmk4tcl.la @@ -146,15 +154,15 @@ -strip $(STRIP_FLAGS) $@ 2>/dev/null libmk4tcl.la: mk4tcl.o mk4too.o $(LIBOBJS) $(LINK_SPECIAL_FILES) - $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_FLAGS) -avoid-version \ + $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_SHLIB_FLAGS) -avoid-version \ -rpath $(libdir) mk4tcl.lo mk4too.lo $(SHLOBJS) $(LINK_SPECIAL_FLAGS) -Mk4py@SHLIB_SUFFIX@: libmk4py.la - cp .libs/libmk4py@SHLIB_SUFFIX@ $@ +Mk4py@MODULE_SUFFIX@: libmk4py.la + cp .libs/libmk4py@MODULE_SUFFIX@ $@ -strip $(STRIP_FLAGS) $@ 2>/dev/null libmk4py.la: $(PYOBJS) $(LIBOBJS) - $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_FLAGS) -avoid-version \ + $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_MODULE_FLAGS) -avoid-version \ -rpath $(libdir) $(SPYOBJS) $(SHLOBJS) Mk4lua@SHLIB_SUFFIX@: mk4lua.o libmk4.la @@ -202,8 +210,16 @@ $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) $? PyView.o: $(srcdir)/../python/PyView.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) $? -PWOImp.o: $(srcdir)/../python/scxx/PWOImp.cpp - $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) $? + +PWOImp.o: $(srcdir)/../python/scxx/PWOImp.cpp \ + $(srcdir)/../python/scxx/PWOBase.h \ + $(srcdir)/../python/scxx/PWOCallable.h \ + $(srcdir)/../python/scxx/PWOMSequence.h \ + $(srcdir)/../python/scxx/PWOMapping.h \ + $(srcdir)/../python/scxx/PWONumber.h \ + $(srcdir)/../python/scxx/PWOSequence.h + $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) \ + $(srcdir)/../python/scxx/PWOImp.cpp column.o: $(srcdir)/../src/column.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? diff -uNr -X metakit-files-to-ignore-when-diffing metakit-2.4.8/unix/configure.in metakit-2.4.8+njr2/unix/configure.in --- metakit-2.4.8/unix/configure.in Fri Oct 4 15:54:02 2002 +++ metakit-2.4.8+njr2/unix/configure.in Fri Dec 13 10:28:41 2002 @@ -8,25 +8,75 @@ MK_TARGETS="core" MK_INSTALL="install-mk" -AC_ARG_ENABLE(python, [ --enable-python build the Mk4py extension for Python], [python_ok=$enableval], [python_ok=no]) +AC_ARG_WITH(python, +[ --with-python=PATH build the Mk4py extension for Python with headers in + `PATH/include/python2.2'; install the Mk4py modules in + `PATH/lib/python2.2/site-packages'. If PATH is of the + form `HEADER:LIB', search for header files in HEADER, + and install Mk4py in LIB. If you omit the `=PATH' + part completely, the configure script will use the + provided includedir and libdir.], + [with_python=${withval}], [with_python=no]) AC_MSG_CHECKING([for Python configuration]) -if test "$python_ok" = yes; then - MK_TARGETS="$MK_TARGETS python" - MK_INSTALL="$MK_INSTALL install-python" - AC_MSG_RESULT(enabled) +case "$with_python" in + "yes" ) + PY_INCLUDE_DIR='${includedir}/python2.2' + PY_LIB_DIR='${libdir}/python2.2/site-packages' + ;; + "no" ) + PY_INCLUDE_DIR="" + ;; + *":"* ) + PY_INCLUDE_DIR="`echo $with_python | sed -e 's/:.*$//'`" + PY_LIB_DIR="`echo $with_python | sed -e 's/^.*://'`" + ;; + * ) + PY_INCLUDE_DIR="$with_python/include/python2.2" + PY_LIB_DIR="$with_python/lib/python2.2/site-packages" + ;; +esac + +if test x"${PY_INCLUDE_DIR}" != x; then + if test -f "${PY_INCLUDE_DIR}/Python.h" ; then + MK_TARGETS="$MK_TARGETS python" + MK_INSTALL="$MK_INSTALL install-python" + AC_MSG_RESULT($PY_INCLUDE_DIR and $PY_LIB_DIR) + else + AC_MSG_RESULT(can't find $(PY_INCLUDE_DIR)/Python.h) + fi else AC_MSG_RESULT(not enabled) fi -AC_ARG_WITH(tcl, [ --with-tcl=DIR location of tcl.h header file], [TCL_INCLUDE_DIR=${withval}]) +AC_ARG_WITH(tcl, +[ --with-tcl=PATH build the Mk4tcl extension for Tcl with tcl.h in + `PATH/include/tcl.h'; install the Mk4tcl package + in `PATH/lib'. If PATH is of the form `HEADER:LIB', + search for tcl.h in HEADER, and install Mk4tcl in + LIB.], [with_tcl=${withval}], [with_tcl=no]) AC_MSG_CHECKING([for Tcl headers]) +case "$with_tcl" in + "no" ) + TCL_INCLUDE_DIR="" + ;; + *":"* ) + TCL_INCLUDE_DIR="`echo $with_tcl | sed -e 's/:.*$//'`" + TCL_LIB_DIR="`echo $with_tcl | sed -e 's/^.*://'`" + ;; + * ) + TCL_INCLUDE_DIR="$with_tcl/include" + TCL_LIB_DIR="$with_tcl/lib" + ;; +esac if test x"${TCL_INCLUDE_DIR}" != x ; then if test -f "${TCL_INCLUDE_DIR}/tcl.h" ; then AC_MSG_RESULT(found ${TCL_INCLUDE_DIR}/tcl.h) MK_TARGETS="$MK_TARGETS tcl" MK_INSTALL="$MK_INSTALL install-tcl" + AC_MSG_CHECKING(for Tcl package installation location) + AC_MSG_RESULT(${TCL_LIB_DIR}) else AC_MSG_ERROR([${TCL_INCLUDE_DIR} directory doesn't contain tcl.h]) fi @@ -59,24 +109,29 @@ # Deal with shared lib differences SHLIB_SUFFIX=".so" +MODULE_SUFFIX=".so" STRIP_FLAGS= -LIBTOOL_FLAGS=-module +LIBTOOL_SHLIB_FLAGS= +LIBTOOL_MODULE_FLAGS=-module case $build_os in darwin*) SHLIB_SUFFIX=".dylib" + # MODULE_SUFFIX remains .so (Darwin doesn't care) STRIP_FLAGS=-S - LIBTOOL_FLAGS= ;; hpux*) SHLIB_SUFFIX=".sl" + MODULE_SUFFIX=".sl" if test "X$build_cpu" = "Xia64" ; then - # libtools insists on .so files for HPUX/Itanium :( + # libtool insists on .so files for HP-UX/Itanium :( SHLIB_SUFFIX=".so" + MODULE_SUFFIX=".so" fi ;; cyg*) SHLIB_SUFFIX=".dll" + MODULE_SUFFIX=".dll" ;; esac @@ -126,10 +181,17 @@ AC_SUBST(MK_TARGETS) AC_SUBST(MK_INSTALL) +AC_SUBST(PY_INCLUDE_DIR) +AC_SUBST(PY_LIB_DIR) AC_SUBST(TCL_INCLUDE_DIR) +AC_SUBST(TCL_LIB_DIR) AC_SUBST(EXEEXT) +AC_SUBST(CPPFLAGS) +AC_SUBST(LDFLAGS) AC_SUBST(SHLIB_SUFFIX) -AC_SUBST(LIBTOOL_FLAGS) +AC_SUBST(MODULE_SUFFIX) +AC_SUBST(LIBTOOL_SHLIB_FLAGS) +AC_SUBST(LIBTOOL_MODULE_FLAGS) AC_SUBST(STRIP_FLAGS) AC_OUTPUT(Makefile) --ew6BAiZeqk4r7MaW-- From niki@vintech.bg Sat Dec 14 04:29:31 2002 Received: from mail.vintech.bg (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBEATTG11945 for ; Sat, 14 Dec 2002 04:29:30 -0600 Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id DF73322C37; Sat, 14 Dec 2002 13:30:38 +0200 (EET) Message-ID: <3DFB099B.8020602@vintech.bg> Date: Sat, 14 Dec 2002 12:36:11 +0200 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.2) Gecko/20021126 X-Accept-Language: bg, en MIME-Version: 1.0 To: Nicholas Riley Cc: metakit@equi4.com Subject: Re: [Metakit] [PATCH] Can't use longs in Mk4py References: <20021203033441.GE2890677@uiuc.edu> <20021213170659.GA3439635@uiuc.edu> In-Reply-To: <20021213170659.GA3439635@uiuc.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: IMO PyString_CheckExact is Python 2.2 only. regards, Niki Spahiev From njriley@theremin.cs.uiuc.edu Sat Dec 14 10:24:31 2002 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBEGOVG21577 for ; Sat, 14 Dec 2002 10:24:31 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id KAA3489502; Sat, 14 Dec 2002 10:25:27 -0600 (CST) Date: Sat, 14 Dec 2002 10:25:26 -0600 From: Nicholas Riley To: Niki Spahiev Cc: metakit@equi4.com Subject: Re: [Metakit] [PATCH] Can't use longs in Mk4py Message-ID: <20021214162526.GA3488591@uiuc.edu> References: <20021203033441.GE2890677@uiuc.edu> <20021213170659.GA3439635@uiuc.edu> <3DFB099B.8020602@vintech.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3DFB099B.8020602@vintech.bg> User-Agent: Mutt/1.4i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Sat, Dec 14, 2002 at 12:36:11PM +0200, Niki Spahiev wrote: > IMO PyString_CheckExact is Python 2.2 only. Not only that, I was under the mistaken assumptions that I needed PyString_CheckExact: - to distinguish between Unicode and non-Unicode strings - to filter subclasses of 'str' because PyString_AS_STRING wouldn't work I've tested these cases (as I should have done in the first place) and they both appear to work fine with plain PyString_Check. Thanks! -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign From brooks@onyxtech.com Thu Dec 19 00:56:45 2002 Received: from gevalt.org (kvetch.gevalt.org [204.107.232.237]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBJ6ujG11508 for ; Thu, 19 Dec 2002 00:56:45 -0600 Received: from [4.63.231.143] (HELO [192.168.1.217]) by gevalt.org (CommuniGate Pro SMTP 3.5) with ESMTP id 953757 for metakit@equi4.com; Thu, 19 Dec 2002 01:58:01 -0500 User-Agent: Microsoft-Entourage/10.1.1.2418 Date: Thu, 19 Dec 2002 01:57:41 -0500 From: Brooks Bell To: Metakit Message-ID: In-Reply-To: <200211131801.gADI1Ha06554@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Deleting properties from views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm not sure what I'm missing. I have a view that has an ID number representing a location. Another view contains two properties. One is the same ID number and the second is the city name for that location. I want to join the two views so that I have the city name as a property in the first one. So... I do this: c4_View locView = mStorage->GetAs(kLocationFormat); c4_IntProp pCityId("cityId"); c4_View vCityView = mStorage->GetAs(kCityFormat); locView = locView.Join(pCityId, vCityView); This works - the city name is added to locView . Now I want to get rid of locView's city ID property. It is here I run into trouble. The Store() function sounded like what I wanted but it is depreciated. So I attempt to use GetAs() with a redefined format that does not contain the cityId but does contain the cityName: locView = mStorage->GetAs(kFlatLocFormat); This does not work. The cityId is indeed gone but the city name comes up as containing no data. It doesn't matter whether I commit first or not. Is there something I need to do to get the join to stick? TIA, Brooks Bell From v.digilio@sisdata.it Thu Dec 19 05:46:17 2002 Received: from smtp3.cp.tin.it (vsmtp3.tin.it [212.216.176.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBJBk8Q01038 for ; Thu, 19 Dec 2002 05:46:17 -0600 Received: from vikwin2k (80.116.221.150) by smtp3.cp.tin.it (6.5.029) id 3DDD788200AEB8AE for metakit@equi4.com; Thu, 19 Dec 2002 12:47:22 +0100 Message-ID: <001b01c2a754$e70ebf30$1500a8c0@vikwin2k> From: "Vittorio Digilio" To: Date: Thu, 19 Dec 2002 12:51:00 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0018_01C2A75D.482979B0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] small borland change for metakit 2.4.8 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Messaggio in formato MIME composto da più parti. ------=_NextPart_000_0018_01C2A75D.482979B0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello Mr. Cusack, I got a question : how did you set the preprocessor symbols to compile = and run smoothly the metakit dll in BCB6? (aside for the small change in = univ.cpp reported in the mailing list).=20 Did you compile and run successfully (unmodified) all of the regression = tests, demos and examples? =20 I did compile it as a dll in BCB6 but had to tweak quite a lot the = preprocessor statements to obtain a clean dll compile of the metakit = engine and have the recursion tests and example compile and run ok. =20 This was quite an interesting exercise w.r.t. acquire precious knowledge = of many of the inner details of the library itself, giving me also the = chance to have a full-browsable (as BCB-units and inline-members taken = into account in the class-viewer) src cleaned by the unrelated (in my = case) cross-platform preprocessor symbols code specific (I also = eliminated the MFC support). =20 Though such a modified src, while perfectly functioning, is well-suited = to reverse-engineering the metakit lib and try to understand what and = how things are (pretty big lack of documentation), it represents anyway = a deviation from the original library which I'd like to to stick to. So I consider my modified src as being useful for just my personal = learning purpouses. That's why I'd like to know how you managed with the preprocessor = symbols (i.e. their exact settings). =20 My biggest problems when I tried to compile the original metakit src as = a dll were :=20 - C4_String class didn't get exported (JWC's tweak to specify the = __declspec(dllexport) modifier aside from the class and function = definitions didn't seem to work in BCB6) - got a big number of warnings about impossibility to inline members and = functions containing return statements (so they didn't get inline, thing = I'd wish to occur :-) ) =20 Thanks for your patience (such a long mail), TIA and regards, Vittorio Digilio Senior SW Developer=20 ------=_NextPart_000_0018_01C2A75D.482979B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hello Mr. Cusack,
 
I got a question : how did you set=20 the preprocessor symbols to compile and run smoothly the metakit = dll in=20 BCB6? (aside for the small change in univ.cpp reported in the mailing = list).=20
Did you compile and run successfully = (unmodified)=20 all of the regression tests, demos and examples?
 
I did compile it as a dll in BCB6 but = had to tweak=20 quite a lot the preprocessor statements to obtain a clean dll = compile of=20 the metakit engine and have the recursion tests and example compile and = run=20 ok.
 
This was quite an = interesting exercise w.r.t.=20 acquire precious knowledge of many of the inner details of the = library=20 itself, giving me also the chance to have a full-browsable (as BCB-units = and=20 inline-members taken into account in the class-viewer) src = cleaned by the=20 unrelated (in my case) cross-platform preprocessor symbols=20 code specific (I also eliminated the MFC=20 support).   
 
Though such a modified src, while = perfectly=20 functioning, is well-suited to reverse-engineering the metakit lib and = try to=20 understand what and how things are (pretty big lack of documentation),=20 it represents anyway a deviation from the original library which = I'd like=20 to to stick to.
So I consider my modified src as = being useful=20 for just my personal learning purpouses.
 
That's why I'd like to know how you = managed with=20 the preprocessor symbols (i.e. their exact settings).
 
My biggest problems when I tried = to compile=20 the original metakit src as a dll were :
 
- C4_String class didn't get exported = (JWC's tweak=20 to specify the __declspec(dllexport) modifier aside from the class and = function=20 definitions didn't seem to work in BCB6)
- got a big number of warnings about = impossibility=20 to inline members and functions containing return statements (so they = didn't get=20 inline, thing I'd wish to occur :-) )
 
Thanks for your patience (such a long=20 mail),
 
TIA and regards,

Vittorio Digilio
Senior SW Developer=20
 
------=_NextPart_000_0018_01C2A75D.482979B0-- From brooks@onyxtech.com Thu Dec 19 10:46:37 2002 Received: from gevalt.org (kvetch.gevalt.org [204.107.232.237]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBJGkbQ31488 for ; Thu, 19 Dec 2002 10:46:37 -0600 Received: from [4.63.231.143] (HELO [192.168.1.217]) by gevalt.org (CommuniGate Pro SMTP 3.5) with ESMTP id 953953 for metakit@equi4.com; Thu, 19 Dec 2002 11:47:56 -0500 User-Agent: Microsoft-Entourage/10.1.1.2418 Date: Thu, 19 Dec 2002 11:47:37 -0500 From: Brooks Bell To: Metakit Message-ID: In-Reply-To: <200212141801.gBEI12G27668@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Join and persistance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I looked into this more and found this wiki page on "derived views": http://www.equi4.com/metakit/wiki.cgi/99 To wit: > Custom viewers are the most recent addition to MK, they do not propagate > changes, nor track such changes from underlying views. In most cases, they > will only work when the data remains unchanged. Custom viewers are completely > separate from data views, and do not deal with persistence and storage at all. ... > Custom viewers are used for all other view operations, including joins So... are comments 1, 2, 3 and 4 below correct? const char* kLocationFormat = "locationView[locName:S,cityId:I]"; const char* kCityFormat = "cityView[cityId:I,cityName:S]"; const char* kFlatLocFormat = "locationView[locName:S,cityName:S]"; c4_View locView = mStorage->GetAs(kLocationFormat); c4_View vCityView = mStorage->GetAs(kCityFormat); // 1. cityName property gets added to locView along with // data BUT it doesn't persist. locView = locView.Join(pCityId, vCityView); // 2. The view with the name "locationView" is not modified // at all by this commit because it was not modified by // the above join mStorage->Commit(); // 3. This treats cityName as a new property. The join // had no effect. Removing the commit above makes no // difference. c4_View locView2 = mStorage->GetAs(kjoinedLocViewFormat); // 4. There is no way to have the result of a join persist? From mbloore@yahoo.com Thu Dec 19 14:53:01 2002 Received: from web14308.mail.yahoo.com (web14308.mail.yahoo.com [216.136.173.156]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gBJKr1Q23083 for ; Thu, 19 Dec 2002 14:53:01 -0600 Message-ID: <20021219205423.40319.qmail@web14308.mail.yahoo.com> Received: from [65.95.158.146] by web14308.mail.yahoo.com via HTTP; Thu, 19 Dec 2002 12:54:23 PST Date: Thu, 19 Dec 2002 12:54:23 -0800 (PST) From: mARK bLOORE Subject: Re: [Metakit] Join and persistance To: metakit@equi4.com In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Brooks Bell wrote: > locView = locView.Join(pCityId, vCityView); > c4_View locView2 = mStorage->GetAs(kjoinedLocViewFormat); > > // 4. There is no way to have the result of a join persist? i would try looping through locView, adding its rows to locView2 (which is initially empty). when done, commit. i wouldn't expect a join to persist automatically. it only exists in memory, and i don't think it even has to derive from views in the same storage, so where would it persist? ===== -- mARK bLOORE __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com From brooks@onyxtech.com Thu Dec 19 15:24:21 2002 Received: from gevalt.org (kvetch.gevalt.org [204.107.232.237]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBJLOKQ26251 for ; Thu, 19 Dec 2002 15:24:21 -0600 Received: from [4.63.231.143] (HELO [192.168.1.217]) by gevalt.org (CommuniGate Pro SMTP 3.5) with ESMTP id 954134; Thu, 19 Dec 2002 16:25:40 -0500 User-Agent: Microsoft-Entourage/10.1.1.2418 Date: Thu, 19 Dec 2002 16:25:39 -0500 Subject: Re: [Metakit] Join and persistance From: Brooks Bell To: mARK bLOORE , Metakit Message-ID: In-Reply-To: <20021219205423.40319.qmail@web14308.mail.yahoo.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 12/19/02 3:54 PM, "mARK bLOORE" wrote: > i would try looping through locView, adding its rows to locView2 (which > is initially empty). when done, commit. > i wouldn't expect a join to persist automatically. it only exists in > memory, and i don't think it even has to derive from views in the same > storage, so where would it persist? I don't know. I think the confusion results from the "derived" views having the same type as "regular" views. There have been more than a few questions about saving sorting order and that kind of thing. Seems like the derived views should be a weaker subclass of c4_view to make that apparent. The solution I implemented is along the lines of what you suggest. I created a routine: ViewCopy(const c4_View& inView, const string& inNewViewName) that takes inView and creates a new "regular" view that can persist. The schema for the new view is pulled out of the old view and then the rows are copied over. There probably should be a third parameter for the storage object but in mine mStorage is an instance variable so it is already available. I didn't bother to step into the subviews as my data doesn't include them. I'm hoping the c4_View.AddRow method will just copy them. Seems like this function gets around a lot of the problems with derived views and something like it should be in the lib. Here's the code: void ApDir::ViewCopy(const c4_View& inView, const string& inNewViewName) { assert(mStorage); // Copy the schema string schema = inNewViewName; // newName c4_String types; bool hasData = false; bool hasSubs = false; schema += '['; // newName[ // retrieve header info and build the schema for (int n = 0; n < inView.NumProperties(); ++n) { c4_Property prop = inView.NthProperty(n); char t = prop.Type(); schema += (const char*) prop.Name(); // newName[propName schema += ':'; // newName[propName: schema += t; // newName[propName:I if (n != inView.NumProperties() - 1) schema += ','; // newName[propName:I, types += (c4_String) (prop.Type()); if (t == 'V') hasSubs = true; else hasData = true; } schema += ']'; // newName[propName:I, ... propName:I] // Format string complete. Create regular view. c4_View outView = mStorage->GetAs(schema.c_str()); // Now populate the view with a copy of the data. int numRows = inView.GetSize(); for (int rowNum = 0; rowNum < numRows; ++rowNum) { // if (hasData) *** Ignored subviews for now { c4_RowRef row = inView[rowNum]; outView.Add(row); } } } From scusack@helpisit.com.au Sat Dec 21 22:23:10 2002 Received: from mail025.syd.optusnet.com.au (mail025.syd.optusnet.com.au [210.49.20.147]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBM4N9Q25590 for ; Sat, 21 Dec 2002 22:23:10 -0600 Received: from vfr (sdcax58-233.dialup.optusnet.com.au [198.142.219.233]) by mail025.syd.optusnet.com.au (8.11.1/8.11.1) with SMTP id gBM4OUF05267 for ; Sun, 22 Dec 2002 15:24:31 +1100 From: Simon Cusack To: MetaKit Date: Sun, 22 Dec 2002 15:23:17 +1100 X-Priority: 3 (Normal) Reply-To: scusack@helpisit.com.au In-Reply-To: <001b01c2a754$e70ebf30$1500a8c0@vikwin2k> Message-Id: <75VP54LG041XWVLJ846176LIKGNH83B.3e053e35@vfr> Subject: Re: [Metakit] small borland change for metakit 2.4.8 MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1252" X-Mailer: Opera 6.05 build 1140 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Vittorio, It sounds like you have gone into it alot further than I did. I don't build it as a DLL just as a static library, as it is small I don't bother with a dll as then I have to remember to distribute it. Also I don't build the Python lib I just download the one provided. So for building the library I just looked at what was done in the VC makefile, then greated a project, added the necessary files, checked that the config.h was set properly (basically the same as VC) made the change already mentioned and that was it. What did you have to change? Simon. 19/12/2002 10:51:00 PM, "Vittorio Digilio" wrote: > > > From: "Vittorio Digilio" > > To: > Subject:[Metakit] small borland change for metakit 2.4.8 > Date: Thu, 19 Dec 2002 12:51:00 +0100 > > > > Hello Mr. Cusack, > > I got a question : how did you set the preprocessor symbols to compile and > run smoothly the metakit dll in BCB6? (aside for the small change in > > univ.cpp reported in the mailing list). > Did you compile and run successfully (unmodified) all of the regression > tests, demos and examples? > > I did compile it as a dll in BCB6 but had to tweak quite a lot the > > preprocessor statements to obtain a clean dll compile of the metakit engine > and have the recursion tests and example compile and run ok. > > This was quite an interesting exercise w.r.t. acquire precious knowledge of > many of the inner details of the library itself, giving me also the chance > > to have a full-browsable (as BCB-units and inline-members taken into account > in the class-viewer) src cleaned by the unrelated (in my case) cross- > platform preprocessor symbols code specific (I also eliminated the MFC > support). > > > Though such a modified src, while perfectly functioning, is well-suited to > reverse-engineering the metakit lib and try to understand what and how > things are (pretty big lack of documentation), it represents anyway a > deviation from the original library which I'd like to to stick to. > > So I consider my modified src as being useful for just my personal learning > purpouses. > > That's why I'd like to know how you managed with the preprocessor symbols > (i.e. their exact settings). > > > My biggest problems when I tried to compile the original metakit src as a > dll were : > > - C4_String class didn't get exported (JWC's tweak to specify the > __declspec(dllexport) modifier aside from the class and function definitions > > didn't seem to work in BCB6) > - got a big number of warnings about impossibility to inline members and > functions containing return statements (so they didn't get inline, thing I'd > wish to occur :-) ) > > > Thanks for your patience (such a long mail), > > TIA and regards, > > Vittorio Digilio > > Senior SW Developer > From nigel@bprj.co.uk Mon Dec 23 09:08:04 2002 Received: from cmailg4.svr.pol.co.uk (cmailg4.svr.pol.co.uk [195.92.195.174]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNF83Q00438 for ; Mon, 23 Dec 2002 09:08:04 -0600 Received: from modem-1271.chimpanzee.dialup.pol.co.uk ([217.134.116.247] helo=jeeves.bprj.co.uk) by cmailg4.svr.pol.co.uk with esmtp (Exim 3.35 #1) id 18QUCs-0001CX-00 for metakit@equi4.com; Mon, 23 Dec 2002 15:09:27 +0000 Received: from tuppy (tuppy.bprj.co.uk [192.168.1.199]) by jeeves.bprj.co.uk (8.12.3/8.12.3/SuSE Linux 0.6) with SMTP id gBNF8N7E022293 for ; Mon, 23 Dec 2002 15:08:24 GMT From: "Nigel Hathaway" To: "MetaKit mailing list" Subject: RE: [Metakit] Sparse tables Date: Mon, 23 Dec 2002 15:08:38 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal In-Reply-To: X-Virus-Scanned: by amavis-milter (http://amavis.org/) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Does anyone know of any other implementations of sparse table on Metakit? (Comments on the previous post below would also be appreciated) Regards, Nigel Hathaway ************************************ Barnabas Projects Limited Tel: +44 117 937 4197 Internet: http://www.bprj.co.uk -----Original Message----- From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com] On Behalf Of Nigel Hathaway Sent: 13 December 2002 11:02 To: MetaKit mailing list Subject: [Metakit] Sparse tables I suspect this has been addressed before, but since it is something I am looking to do in my current project, I'm posting this anyway. Let's say we have an ordinary table, and it has (in the limiting case) 250 columns and 100,000 rows. The data type of each column is known and fixed, as are the number of columns. Each row has 1 or more of its columns containing data, and the rest are not used or null. The table is sparsely populated, that is, 90% of all the cells contain nulls randomly but evenly distributed (for sake of argument). Metakit has currently no way of representing a null entry (as opposed to 0 or some other default). I was originally going to tackle this using a classic property list approach: each row is a sub-view of two columns; column 1 is a string which is the table column name and column 2 is a variant (or union) containing data appropriate to the column type - this was the motivation behind my last posting regarding fixed-size binary columns. Giving it some further thought, I have come up with another idea: Create a table with the 250 columns, but don't populate it in the conventional way. Instead, have an auxiliary table where each row corresponds to the row in the required sparse table, with the table containing a single column which is a two-column sub-view: column number (adaptive integer) and row number (adaptive integer), and these correspond the location of the real data in our 250-column table, which has no empty spaces. Of course, this means that the columns in the big table have a varying number of rows in them, so arguably what you need is 250 single-column tables (which in MetaKit's case is virtually the same anyway). The question is: how to implement? This is obviously a classic case for a custom viewer similar to the hash map (for example). You only access the data through the custom view, which itself maintains the integrity of both tables. Each time you add a record, you add an item onto the end of the relevant columns, and create entries for each in the lookup table. But what happens when you delete an entry? You take out items of data out of the middle of several columns, and those columns shift down one from the point of deletions. This invalidates the data in the lookup table for reference to those columns beyond the deletion points. Does this mean you have to iterate over the entire lookup table set and make adjustments, or is there a better way of doing this? Maybe this has already been done and I have missed it? Another question: can you chain different custom views on to another? For example, I have my sparse table custom viewer, but I still want to index the table for fast look-up, e.g. by hashing on the values of a certain column (ignoring the fact that there could be null values). Can I use the hash view which in turn uses the sparse view which in turn accesses the raw data? I am still evaluating storage libraries for my data. I am writing cross platform Macintosh (classic and OS X) and Windows, so this severely limits my choice. One other candidate is SQLite. However, MetaKit still comes out on top in the suitability stakes. Given that I need to write this anyway (and it won't be on JCW's agenda in my timescales) it makes sense for me to contribute this into the project, given also its generic nature. Assuming I go down this route, are you (JCW) open to this? - I would prefer not to be the maintainer of such a small chunk of code in the long term. Would you (JCW) be willing to give me further help/pointers to ensure that it conforms to your overall style and schema? Regards, Nigel Hathaway ************************************ Barnabas Projects Limited Tel: +44 117 937 4197 Internet: http://www.bprj.co.uk _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Mon Dec 23 13:22:42 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNJMfQ12058 for ; Mon, 23 Dec 2002 13:22:41 -0600 Date: Mon, 23 Dec 2002 20:23:35 +0100 Subject: Re: [Metakit] c4_Property shouldn't require a virtual destructor? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list In-Reply-To: <001401c2a1d8$45c261d0$1500a8c0@vikwin2k> Message-Id: <07B4D610-16AC-11D7-B0DB-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id gBNJMfQ12058 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Vittorio Digilio wrote: > Unfortunately it lacks (almost completely :-) ) a full C++ > documentation (anybody, as a long-term user wrote down something and > is willing to share, thanks :-) ), so I started experimenting and > inspecting the C++ src code. Does http://www.equi4.com/metakit/api/hierarchy.html help? > I noticed that c4_Property, though being the base class for the other > properties, provides a non-virtual destructor. [...] > In this scenario deleting the heap-allocated derived class shouldn't > call the base class c4_Property::~c4_Property() destructor and the > reference wouldn't be released. >   > I mean : >   > c4_Property *pMyInt=new c4_IntProp("age"); > //.... > //.... > delete pMyInt; // c4_Property::~c4_Property() should'nt be called > // and Refs(-1) isn't called either >   > Perhaps I'm missing here something really big and the destructor > should be non-virtual ?! You mean should be virtual? I don't know, but properties are not intended for the heap. Why not simply: c4_IntProp pMyInt ("age"); -jcw From jcw@equi4.com Mon Dec 23 13:22:58 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNJMvQ12076 for ; Mon, 23 Dec 2002 13:22:57 -0600 Date: Mon, 23 Dec 2002 20:24:21 +0100 Subject: Re: [Metakit] Deleting properties from views Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <23540D58-16AC-11D7-B98F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brooks Bell wrote: > Now I want to get rid of locView's city ID property. It is here I run > into > trouble. The Store() function sounded like what I wanted but it is > depreciated. So I attempt to use GetAs() with a redefined format that > does > not contain the cityId but does contain the cityName: > > locView = mStorage->GetAs(kFlatLocFormat); > > This does not work. The cityId is indeed gone but the city name comes > up as > containing no data. It doesn't matter whether I commit first or not. GetAs turns no-longer-used properties into "temporary" properties. Their value remains available until commit. After that, the view no longer has such a property, but note that accessing a property for a view always returns a value (empty string or zero) - MK does not throw an error for accessing properties which are not defined in the view. > Is there something I need to do to get the join to stick? To be answered in next email... -jcw From jcw@equi4.com Mon Dec 23 13:23:03 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNJN2Q12086 for ; Mon, 23 Dec 2002 13:23:02 -0600 Date: Mon, 23 Dec 2002 20:24:25 +0100 Subject: Re: [Metakit] Join and persistance Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <25FAC614-16AC-11D7-B98F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brooks Bell wrote: > So... are comments 1, 2, 3 and 4 below correct? > > const char* kLocationFormat = "locationView[locName:S,cityId:I]"; > const char* kCityFormat = "cityView[cityId:I,cityName:S]"; > const char* kFlatLocFormat = "locationView[locName:S,cityName:S]"; > > c4_View locView = mStorage->GetAs(kLocationFormat); > c4_View vCityView = mStorage->GetAs(kCityFormat); > > // 1. cityName property gets added to locView along with > // data BUT it doesn't persist. > locView = locView.Join(pCityId, vCityView); It may become clearer if you keep in mind that locview acts like a pointer. A join does not "alter" a view, it "constructs" a new one based on two others. What you did in the above is to join the old locView with vCityView and make locView point to the new result. The old locView data is still there and unaffected. > // 2. The view with the name "locationView" is not modified > // at all by this commit because it was not modified by > // the above join > mStorage->Commit(); Correct. You're not committing the view that resulted from the join, you're comitting whatever views were defined in the storage. > // 3. This treats cityName as a new property. The join > // had no effect. Removing the commit above makes no > // difference. > c4_View locView2 = mStorage->GetAs(kjoinedLocViewFormat); You're getting the underlying view again, has no relationship with the join result above. > // 4. There is no way to have the result of a join persist? There is, indirectly. You have to copy the (virtual) view data that the join produces: c4_View temp; temp.InsertAt(0, locView); Then copy that back into storage: c4_View locView2 = mStorage->GetAs(kjoinedLocViewFormat); locView2.SetSize(0); locView.InsertAt(0, temp); The resaon for copying twice is that locView is virtual, the join will use the underlying data when you access things in it. If you cleared locView2 (which wpoints to the same data as what the join uses) before making a copy, chances are that the join will be gone as well (not sure what happens, but I don't think it'd work). The Duplicate operation does the same insertat trick, so you can shorten the above to: c4_View temp = locView.Duplicate(); c4_View locView2 = mStorage->GetAs(kjoinedLocViewFormat); locView2.SetSize(0); locView.InsertAt(0, temp); I'm not quite sure what you want to achieve, though. if it's about dropping a property, then GetAs would be enough. If it's about adding one, then a new GetAs with a loop to copy values in might be more convenient. -jcw From jcw@equi4.com Mon Dec 23 13:23:06 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNJN5Q12094 for ; Mon, 23 Dec 2002 13:23:05 -0600 Date: Mon, 23 Dec 2002 20:24:28 +0100 Subject: Re: [Metakit] Sparse tables Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: "MetaKit mailing list" Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <27A7944F-16AC-11D7-B98F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nigel Hathaway wrote: > Let's say we have an ordinary table, and it has (in the limiting case) > 250 > columns and 100,000 rows. The data type of each column is known and > fixed, > as are the number of columns. Each row has 1 or more of its columns > containing data, and the rest are not used or null. The table is > sparsely > populated, that is, 90% of all the cells contain nulls randomly but > evenly > distributed (for sake of argument). > > Metakit has currently no way of representing a null entry (as opposed > to 0 > or some other default). Correct. One way out is to define a second property, containing 0/1 values to indicate "nullness". There's a price to pay if this is done for each column, and if there are lots of modifications (inserts/deletes in sub-byte columns is not truly efficient, lots of bit-shifting going on, even though the amount of data is small). Another way out is to move to a different model. You're saying you have an matrix of which only some slots are occupied with a value. If these are all of the same datatype, then you can represent this as a view of rows. If you also do things with sort order and keeping it sorted, then access can be relatively simple - it will all depend on what accesses you need. Maping this second approach into a generic custom viewer would be grand, IMO. > I was originally going to tackle this using a classic property list > approach: each row is a sub-view of two columns; column 1 is a string > which > is the table column name and column 2 is a variant (or union) > containing > data appropriate to the column type - this was the motivation behind > my last > posting regarding fixed-size binary columns. Subviews are nice, but 100,000 subviews are pushing MK a bit too far. Well, it does work, just like 1000's of properties per row do work, but you will find out that performance goes down (in the case of subviews, especially if all of them contain *some* data, there is a lot of work going on to track a large number of tiny vectors and the associated free space). > Giving it some further thought, I have come up with another idea: > > Create a table with the 250 columns, but don't populate it in the > conventional way. Instead, have an auxiliary table where each row > corresponds to the row in the required sparse table, with the table > containing a single column which is a two-column sub-view: column > number > (adaptive integer) and row number (adaptive integer), and these > correspond > the location of the real data in our 250-column table, which has no > empty > spaces. Of course, this means that the columns in the big table have a > varying number of rows in them, so arguably what you need is 250 > single-column tables (which in MetaKit's case is virtually the same > anyway). In terms of performance, in fact even in terms of storage, the following two structures are nearly equivalent: data[c1:I,c2:I,...,c249:I,c250:I] and data[cols[c:I]] (With 250 entries in the data view, each subview being one column). > The question is: how to implement? You will have to experiment (ant time!) things to approximate your particular access pattern. With the numbers you indicate, I'd stay away from 100,000 subviews. If nothing else, put it in a single view and manage ranges. Once you have the model that works for you, *then* you can consider re-doing it as a custom view, to hide the implementation from the rest of the app. > Another question: can you chain different custom views on to another? > For > example, I have my sparse table custom viewer, but I still want to > index the > table for fast look-up, e.g. by hashing on the values of a certain > column > (ignoring the fact that there could be null values). Can I use the > hash > view which in turn uses the sparse view which in turn accesses the raw > data? Yes. > I am still evaluating storage libraries for my data. I am writing > cross > platform Macintosh (classic and OS X) and Windows, so this severely > limits > my choice. One other candidate is SQLite. However, MetaKit still > comes out > on top in the suitability stakes. Given that I need to write this > anyway > (and it won't be on JCW's agenda in my timescales) it makes sense for > me to > contribute this into the project, given also its generic nature. > Assuming I > go down this route, are you (JCW) open to this? - I would prefer not > to be > the maintainer of such a small chunk of code in the long term. Would > you > (JCW) be willing to give me further help/pointers to ensure that it > conforms > to your overall style and schema? Well, as you can see I'm not always *totally* on top of things, and sometimes it takes time for me to get back with an answer. Basic rule is that if I can answer fully with a quick answer, I will - usually. If it takes more work, or if the answer is "it depends", or if it's something which ought to be clear from what there is already... well, then things tend to sink into a bigger pile of todo's. Generally speaking, I welcome *all* contributions which can be fitted in. There's a bunch of real valuable work waiting to be integrated and consolidated right now. -jcw From jcw@equi4.com Mon Dec 23 13:23:08 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNJN7Q12098 for ; Mon, 23 Dec 2002 13:23:07 -0600 Date: Mon, 23 Dec 2002 20:24:31 +0100 Subject: Re: [Metakit] mmap Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <292D1AF0-16AC-11D7-B98F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bruno Blondeau wrote: > Could someone tell me how to force changes to the disk when mmap is > being > used by a Metakit database? MK uses mmap in readonly mode. Changes written to file during a commit are written to the underlying file. The implementation for all I/O is concentrated in the c4_Strategy class, with c4_FileStrategy as the standard implementation of it. -jcw From jcw@equi4.com Mon Dec 23 13:23:11 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNJNAQ12106 for ; Mon, 23 Dec 2002 13:23:10 -0600 Date: Mon, 23 Dec 2002 20:24:33 +0100 Subject: Re: [Metakit] Compiling metakit under cygwin Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list In-Reply-To: <000a01c2a04a$c1a89470$1416a8c0@THADDEN> Message-Id: <2AC9B512-16AC-11D7-B98F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id gBNJNAQ12106 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bodo von Thadden wrote: > I know, this topic was often already discussed, but I have problems to > compile it with cygwin and gcc 3.2.1 > The compiler alway's stop's at string.cpp.  He say's to me, that the > method strcasecmp() was delared as extern > later as static. This was mentioned in another email I got on the 20th. Not sure what causes it - on Linux, gcc 3.2.1 compiles the latest CVS head of MK fine. > #if (!q4_MSVC && !q4_WATC) || ((defined(__CYGWIN__) || > defined(__MINGW32__)) && __GNUC__ < 3) > // #if !q4_MSVC && !q4_WATC Is this old? Are you using the latest version of MK? > Undefined reference to ___gcc_personality_sj0 in libmk4.a I've never seen this before - could it be a compiler versioning issue? With gcc 3, you have to be super careful to re-compile all sources, there have been incompatible changes. -jcw From jcw@equi4.com Mon Dec 23 13:23:13 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNJNCQ12116 for ; Mon, 23 Dec 2002 13:23:13 -0600 Date: Mon, 23 Dec 2002 20:24:36 +0100 Subject: Re: [Metakit] Python bindings and 'throw 1' Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20021213060750.GE3370954@uiuc.edu> Message-Id: <2C2B77DA-16AC-11D7-B98F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicholas Riley wrote: > I notice about a year ago the PWException class was removed, and > exceptions are now signaled by 'throw 1'. The CHANGES file has this to > say: > > Simplified Mk4py - by removing a layer of exception handler classes > in scxx. Errors now set info and throw a plain int. > > It was only after readding my own PWDException class that I discovered > a PWException class was present in scxx before you excised it. > > The PWDException ("D" is for "dummy") is about as simple as it can be: > > class PWDException > { > private: > PWDException(); > } *PWDPyException(); > > My intuition is that throwing PWDPyException should not be any more > expensive than throwing 1, and it should be a lot easier to > understand what is happening. Is this OK? I think so. The exception cleaning also resulted in reducing the number of spots where they were thrown, in fact all but one are now in SCXX. I agree - the "1" kludge does not seem useful. Have vague memories of some build issue with exceptions which caused me to throw it around in the first place, but I'd say go for it. Cleanup is good. -jcw From jcw@equi4.com Mon Dec 23 13:23:15 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBNJNFQ12122 for ; Mon, 23 Dec 2002 13:23:15 -0600 Date: Mon, 23 Dec 2002 20:24:38 +0100 Subject: Re: [Metakit] small borland change for metakit-2.4.8 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <2D899E55-16AC-11D7-B98F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Simon Cusack wrote: > The new 2.4.8 is great, I had to make a small change for borland to > compile > it. > > In src\univ.cpp I had change line 22 from: > > #if !q4_MSVC && !q4_WATC && !(q4_MWCW && defined(_WIN32)) > > to : > > #if !q4_BORC && !q4_MSVC && !q4_WATC && !(q4_MWCW && defined(_WIN32)) > > to build for borland builder 5 and 6. Ok, thx - added to CVS. -jcw From jeff@k2.com Thu Dec 26 10:52:24 2002 Received: from kobalt.k2.com (IDENT:root@dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBQGqNl09991 for ; Thu, 26 Dec 2002 10:52:23 -0600 Received: from Klee (dsl093-106-226.wdc1.dsl.speakeasy.net [66.93.106.226]) by kobalt-eth1.k2.com (8.10.2/8.10.2) with ESMTP id gBQGrmj15829 for ; Thu, 26 Dec 2002 11:53:48 -0500 From: "Jeffrey Kay" To: Date: Thu, 26 Dec 2002 11:53:47 -0500 Message-ID: <000d01c2acff$5c97ded0$aba8a8c0@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Case Sensitive Find/Select/Locate Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: How hard would it be to add another field type, say 's' (lowercase s) for supporting case sensitive strings? I've sort of run into a brick wall with this -- I have some tables that the case-insensitive search is fine, but others where I really need the case sensitivity. I'd prefer not to go the global variable approach presented in previous messages. I wouldn't mind setting an instance variable in a c4_View class, but not a static. The problem appears to be that there's no path f4_CompareFormat call in c4_Handler::Compare back up to the view that's calling it. Is that a fair statement? c4_Handler::Compare appears to be where all calls eventually end up, but there doesn't seem to be a way to reference the view from the places that call it. Even so, I'd still prefer either a field type or property setting that would flow down to the comparison function. I wouldn't mind contributing the code if you can point me in the right direction or give me a couple of hints about how you think this can be accomplished. Thanks -- jeffrey kay personal weblog pgp key "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein From jcw@equi4.com Thu Dec 26 11:15:07 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBQHF7l10533 for ; Thu, 26 Dec 2002 11:15:07 -0600 Date: Thu, 26 Dec 2002 18:16:32 +0100 Subject: Re: [Metakit] Case Sensitive Find/Select/Locate Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <000d01c2acff$5c97ded0$aba8a8c0@k2.com> Message-Id: X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeffrey Kay wrote: > How hard would it be to add another field type, say 's' (lowercase s) > for supporting case sensitive strings? I've sort of run into a brick > wall with this -- I have some tables that the case-insensitive search > is > fine, but others where I really need the case sensitivity. If it were trivial, it would have been in there... But you're right, this needs to be addressed. There may be ways to get us there without globals (I agree, app-wide modality would be a bad choice). One idea is to direct comparisons through the c4_Strategy class. This is per-storage (but one could play tricks, and make the comparison code do different things only for some specified properties). This should be doable with little risk and impact on the rest of MK, nor does it have to cost us in performance IMO. Another option would be to add a comparison member (or function pointer) to each property object (c4_Property, and all its derived classes). Again, no performance cost of substance IMO, but I'm not sure how far the effects of this would reach. Encoding sort choices in type case ("s" vs "S") would not be my favorite, because representation and use are really two different issues. Even using special property names ("name_nc:S" vs "name:S") would be preferable from my perspective, because it keeps this aspect ouf of the MK core. If comparisons are moved to the strategy class, then one could implement this on top of MK - it may well become a default, but at least it would become overridable (for those who need to maintain 100% compatibility). There is indeed no way to go from a view to its parent. This is unfortunate, but impossible to alter in the current design (unattached subviews can be referenced from multiple items). > I wouldn't mind contributing the code if you can point me in the right > direction or give me a couple of hints about how you think this can be > accomplished. Let me think about this. The other reason to push forward on some sort of custom sorting, is that we really need to get Unicode-aware sorting worked out, which can be considered another "custom" sort order (probably the default one, one day). -jcw From jeff@k2.com Thu Dec 26 14:36:20 2002 Received: from kobalt.k2.com (IDENT:root@dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBQKaHl16003 for ; Thu, 26 Dec 2002 14:36:17 -0600 Received: from Klee (dsl093-106-226.wdc1.dsl.speakeasy.net [66.93.106.226]) by kobalt-eth1.k2.com (8.10.2/8.10.2) with ESMTP id gBQKbcj27951 for ; Thu, 26 Dec 2002 15:37:39 -0500 From: "Jeffrey Kay" To: Date: Thu, 26 Dec 2002 15:37:36 -0500 Message-ID: <002f01c2ad1e$a1d266e0$aba8a8c0@k2.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0030_01C2ACF4.B8FC5EE0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <0C65A08B-18FD-11D7-8896-000A957664F8@equi4.com> Subject: [Metakit] Code to Support Case Sensitive Strings and Custom String Comparisons Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0030_01C2ACF4.B8FC5EE0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit All -- You can probably tell the urgency of this change for me by how quickly I went to work on it :-). I've attached changes that I made to the 2.4.7 baseline to add support for case sensitive string comparisons. I included the source code for a sample c4_caseStringProp class that replaces the case insensitive comparison with a case sensitive one. Support for this change is through the addition of a CustomCompare member to c4_Property where a custom comparison function can be inserted. In the attached ZIP file (hopefully this worked through the e-mail list -- if not, I'll post this on k2.com) there are 10 files: Replacement source files: -- handler.cpp, mk4.h, mk4.inl, view.cpp Diffs -- handler.cpp.diff, mk4.h.diff, mk4.inl.diff, view.cpp.diff Sample source code: -- c4_caseStringProp.cpp, c4_caseStringProp.h If you look through the diffs, you'll notice that I changed very little of the original code (one of my requirements). The four files that were changed in the 2.4.7 baseline received the following changes: Handler.cpp -- added code to test for existence of custom comparator in property class View.cpp -- added code to initialize c4_Property::CustomCompare private member to NULL in c4_Property constructors -- added c4_Property::SetCustomCompare() function to set CustomCompare value Mk4.h -- added CUSTOMCOMPARE and LPCUSTOMCOMPARE typedefs to c4_Property class -- added CustomCompare private member to c4_Property class -- added declarations for SetCustomCompare and GetCustomCompare() to c4_Property class Mk4.inl -- added c4_Property::GetCustomCompare() function to retrieve CustomCompare value I implemented the sample code within my software and then just replaced some of the c4_StringProp declarations with c4_caseStringProp for the particular properties that require case sensitive handling. After walking through the code in the debugger several times, everything seems to work as I anticipated. I would think that this would also work nicely for other custom string comparisons, including UTF-8 and Unicode. Similarly, this should work for properties other than strings. Let me know if you have any questions about these changes. jeffrey kay personal weblog pgp key "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein ------=_NextPart_000_0030_01C2ACF4.B8FC5EE0 Content-Type: application/x-zip-compressed; name="MetaKitChanges.zip" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="MetaKitChanges.zip" UEsDBBQAAAAIALFwmi1Z/DSdewAAALkAAAAQAAAAaGFuZGxlci5jcHAuZGlmZjO21DExTja25LJR UFBwTy1xqixJLdbIzEtJrYjXUUhJLEnUtOYFSvKCFWSmKWjEFxTlF6QWlVTqAZU7lxaX5Oc65+cW JBalamgqKNoq+IX6+GgqVIM0cCooFKWWlBbl4delqQGyRyc5v6ASYpmCQi2Xrq4ulx1uN3EBAFBL AwQUAAAACAC8cJotlk1GarwAAABnAQAADQAAAHZpZXcuY3BwLmRpZmYzNDQy1zE0NDZMNgSyuGwU FBRqeYEUL5hZlp+ZopBsEh9QlF+QWlRSaWUVnFriXFpckp/rnJ9bkFiUquET4BwaHOLv6+zvG+AY 5KqQnKypUA3RDQQoahVsgbLWvDBLdHV1uexATC6g/SZA+40NwFJWCvEllQWpChogMl5TB9UQDb9Q Hx9NuGY0xUCjzE2BRpkbwowCekCjAOj8eD331BLPFA1NoIFQLRDhECAbLIpqDVwPqrAmitXEGs4F AFBLAwQUAAAACADZcJotf3aYKqoAAAANAQAADAAAAG1rNC5pbmwuZGlmZjOxtEg2sbTgslEAASuF +MwUBQ2N4oz8ohJNhcyUeE0dhfiSyoJUBQ0QCeI6lxaX5Oc65+cWJBalahhocunq6nLZEamdy9TQ QsfUyDgZTBtaAu2t5QUSIJxiEp+Zl5OZl6qQbBIfUJRfkFpUUmll5RPgHBoc4u/r7O8b4Bjkiirp nlqC6p6y/MwUTYXk/LziEpCZ1bxgnxWllpQW5aE63ZoXYjnE+bVAzAUAUEsDBBQAAAAIAM9wmi3O yOy3IQEAAE8CAAAKAAAAbWs0LmguZGlmZpWQ0UrDQBBF3wP5h3kqiSTUhEza2CJoFBEsDVZ9DdvN CtE2uyTTSujPu93aSqIgvoXJzD3nLiIWiJE1BdtCHHHEWH/vbMv3fesSdhbGgYdxqJcSszSF4RDe lu/AikIUQBKajVKyJuCbhuQauFwrVjOSdaMj40hfa0R8ri8BqFWiEK9QVgTp8+JpPkvns+zq8dbh smp0RpRftySaASyD3IMfwzB3J3YnqZNyBg9ZZ6CXcawVxrrZKDGHOu2lFB8glTCWHnyzs3o/pXbg HtCGdeBtZVnAQlBqWqampHB6OOD86Nf/c9c/3QceMV+v/U85XS3xMAk5jgPDbIhRyQ+q6UqwaqPu KxJ1xVY3jJjjnvqoutwyEhe/qnY8J/ZJ7u946xNQSwMEFAAAAAgAFWeaLZ6Op2C6EAAAIVIAAAcA AABtazQuaW5s3Vx5b9tGFv+7AvIdJmmhyI4Ox+t0W8cxmthpY2zaGLZbdFEUwkgcydxQpMLDiprU n33fe3NwSA51pLKaXSNApOHMu39v3hxir8cmbw+6fhiwTqfR67GvzrxD3dS+YY+7X7P9vb393t7j 3t43bO/bw/39w4MnbDa8SdjL91P2FQ66uvYTBv+mPE5ZNGI/ipT/y0/bLL0W7DqaiCkfC+xwnabT w15vNpt1xbvMP+gOo0lvAr3f+mmv0ejt7rLvRn4gGmyXnWeDwB8yT4z80E/9KEzY7NofXjMeC5Yl GQ+COQMp/VB40B+Hb/IP9boQUeyJWHiKT8KSCHWakww8mPF5IgWELgMxiqAVHvsxG/lxkoKUoEnD O+jL4Wx40D/J4iSK8dNFNLsQo8PDaCpinkZxk7V22BDUTBsfGozFIs3ikPWHNOBp409pngvZzFkW +u8yMKrHgC1wRfPHSCudgzXQgOy5aWHR4D9imIIabMTh/wkoBdaLxTQWiQjThPGQwf8xWhSIj4No gPYlMhPuhyknHZP5ZBAFLOWDQHQZe8nBHYYHSMCTxB9jR0u+NhGRrosFEkvYDQ9AcJ6wIArH+H8C QQJPRyBXOAQjopl5amvEmHjvJ2kCbM9CNhYhWC1oW9y9h4mir3lb9MErRINPpxBUHMOJxVmI1E4j FkYpS1L0HlGBZxiEbZb4JIxyOBHAruOMxzxMBegJgiqeGOoJBy34MI4ScsY45hOw500kGSZd6ZUg 0FL7EFEzP73OB4dEAQQZ+zfCIup7XYrxPJbAJxhF50r/w8MfRHrmOUPI92T4bPyv0fjSH7H77w76 r9/89MPXB6zZZPc1HlrYRg92qP3V819e9k1bw1JlEEFMaRg8e8ZapAJLoQPQ5P02KzQM+ju2erzf DR4zGDagD8AKW/Z1yz7pXsPs6BN47bMjSZh9/FhilTM/snh/KULPH92J/SFHQQz84osZkVfZgcKd hdlkIGJMxwhsDDWKUkDpDfTvVkMJyVAYXfp/CGcgJeJd5/inbAKpK2ntmJx0cs3DsZDBCkORpeGD IX9JMMJvkCEg+wT+W8G8OcQ65Hcex5BF22weZWwIWQimheFbmXmGhi5kVKRM2B/GggMwxzm/mQ+Q AmSDnVkMouFYgtUfIo56YjKF9AAZJxPABvJQAMyFoeKncrwnApESxpFuwCEGkBawvILvY/j8Yt7H GQ7gmIJdQT9IzZCuUkikWQipvgLRmwiSnDHspTIsWjsUM/wM0YbfFHEZadLIFwI1a+X9dB+NZHD1 JLoBp0KaNg5uJQJSOdkEEhNqv9OtlUcSgGzUkoy1fHs2DzXZTCOQE5SmvCwYoCeBXEypGnzkBSJu M5iFkGNxujuXPXdztqqlPr5MBx1f6BMrxMiTMsbA/1Mx9Ec+pBuavTBA5AOwnOp5zSkgImiNrcRL xUmigmcASdfzZD7H0VkI034wx/iQYJEz6nfkfvCZJ973qSNq3BlwcD5Gi3xi9QQpXgZiIvtiB6A/ jKZzySeHyMKoeZ5SzEiuOj+ZAqJpmNjxo4fJIU3TxbbpWZiAKcC7JJEyKFBPqSCA7zyURiMTMFPl +R50gEkU6oAI6fhExyDSeCRPQDAl8CxITZxCyGDfkEG84kCkIrCWkEJMtBvQiQDLCYxTzpxChSGo cuDxOMMHi0EndQRTNL6wbQjBfeT23Re15lWj8hAEIaXqkvYwysK0LzvlqoNtUWPTt80mGdCHgDtm e+Auxmy3GXErnmsr+gUPSggbsyYpl36AygnFlDUEkcJiTfWSldg1vxE5LNDDefmVQMk3h2QMOU2B ROZuUGIgVKKEkngkMzNSySs3khKKYsCZzkEwSNLS80KMYvPgk1GlLO2e5GRFFpF29XGhkl8RWbkX i6nYdNUALDhCZ8miOFaFp2rs0vSL6qi89xyqyrATijGUiTc0HYuxiN31nhQf5uBzw2DhVP1Kpmd7 uv7el/CTehPQTRnd1kGTIs4tm2OHM6/PfjbLDkBiNPR5KrMm18AfUtdIV+1axyubH80hhiNMHJ3H 2IxTMSAbS+wR2NhbZAFUQte+5EQl4I5jQsEnyLql+5ApEHuFSQ6LAn9KSwMZ2zGtWTksBt7j2ieN syF0hXWBCBJRmexk3rjmsTXZnYpE06z3UqFTQTRV7Mh5r4QxbUfLDUAtww61Ud8HgpfiHaS1VjHr DKmpwPtUbJg3EHTwhtact+QrDR1BSUFl2oSmo5IExSpDcrhVn5hmYHF0eRzYCOXpwrQ3gJyoq1UI AVCMB8tXK4p501pDmKbqKiKvsuXaIf8ulxAn0QRwJ1owEnvs/QX5EU4Ldbj/KTrcb/G+lL1o2ysl lNwAoXkFMBSIBBclHGd8oOvJumfZomw9k9omOypZzC3VGAKcqtoVBTv+BMEGfRCG91e1EeRC8hWm 1FVM9WmuU0Kt4jltozUFO15dMOb6KwXakYmz6iL4L/yp9bPck1NrWZQQ8w8mfqw/5f6brlX09h0P EVZQDQ3JPh6tN6MsxX200uKhzQbQTsvbKAtoTpVbJGafjT0IsyBAkzzQG3WmQqa9Tx7Qkg5xLvkj NyT0oKlH/rb3+4OuLNFnUfw2MUWXHAGzKm5XodC0lpNTttyGgxKrXEQboxwemo/g0IP+pUrzTUz4 qmSSddEOBdIhpXXWauLjnTbryzm/pft80Gn+PBYdPzTpPdet6xKjaUukY+zRI53pHz2SfJ7mYbOL pZaZVM6jJF2DXR030LaAJMs2u6i31vfRo3w+Q0WteexTFO10tKKdziqKrs6ujtuqinY6uaLPvRtO i4fBHCObyshoNIIl3houfUa81ThdhssYgmequVb7F3wIkZ9N/4oInQUidJaLcJkN0pjL9JEvhRcI 4ZShKEK1cKzzB+uYQaVNHBkKng9rRCyiBiKdCdxgnkU6Qzh2BB1yWXLTfy7pjCyqS1cHrYkUb2Xr mMhwsJbpp+Aoh32MEGSnokjsUdVeIF1SEY42bhbgyBLTkqnNquYqiCnbcikW7VVTqWnI8QLxajVE aVjWlvRRlpVKa9Wee2VRZbgay3I9uKC2W0+HI6PCx4+NL9hS3Y5WU+14ZUHsCm5RGbaqnQrF16Ly aRHBv79okttjjSIa9Jmm+ehKGLpQGCqEWsj400nPpmzQYNWXeqcupv+VvaGSbWkWEMfqWVc1SSHU l669Z2raKITarKmpYsp3Zf3KStQWF8pJOjqNF5ztEvulyC8rWyindWMp2FpNiDOlnV4eqTatnbXA LEfTSuvFdcRZI0esRR7/ICj3n3SedB5/++03h8yLwoew6n2vTpA41dODttx/G5Il/AT3eRLWgZq4 gwfbE5HG/nBt4x2tZbvj/yndjtUyepXV59qBcLwsDo7XI89cf1tNiS/mqUjyBbVZStLlCjqFHPgh j/VtjGLmoMGUOekTVvxICBdUuPhr7eFqik5qWiMeJAI1xsp0qHd+EMQYKrhbDWVLRrt0UIzI1SOs APGeyOnLE3by66/sSfef9tq/IKpagtIJqqbehqUjQ+5LhJauwZ2/XVj2jlSJFoiwX1KHmioalRRq pQf9ARDe3SFia6vXs8rfyiGm5lMo6Eh8zdXS8ER1dqdx9dBZcudHAgMyEe1zJ07uqupWLOtP4eFB CTd6q1UN7b+OEnEClm3ZcyF8l06QJzjst9/BX7hfvbNIhUWbjDJS6FDQ7DJ26+BcmDZI0BKaVduS SaNX2iP2R/oEE88OJtwTbXOmP8Cd60DQQRJu53ed0XtrY45CMDcfsqQ8sgj+a//pEkrvaZerHt0u Cymz811b8bQbzuTn+DMU9NlJkw5OqkWZKXsAo+aOVUv1LRVqKm4tsRddIinUXGepmFDPctllmHbV vSbnVFHhespT3rJiCbLGUhlg2HL2bUmrBneWEJdaiHJYl0Sp1J71Mrh5L6jJtDTl6TJvXw6yzf2p aD8L04vKgkE2UpzLj2416MDdBGkRFPrZh80Lrm+3XZ399O87scnrKBxXjaJaySrq82dlFin790HE HR7VzSS9/vIZin8aZYNAVOU37aSA+fZ5xSXdKMRq6E6jkzJX1UK6Oa/7Pk8PX8KaxwUv004KmG+f oQa4m1CVX7WS9OrzZyi7LjBKwuf3la0vsg5l6Xyqr0T6Xl6SwBTbaiXXUZzuUDvMjNgTVgfYH76e ZEkaTfQ6dk8phaNBHAYTOJ5e0iYuLpGzEMd51s1xvPwB5U5NEVaUWk+4UNrUb0A5N+wt4+uyTe4i 1VxHoRNYuiqLtTedyrbloiy/rqNuvcorOL7jVoRb+LYle7kQdMour1VIaeWd+75LaBSUXKNvemk5 bcHQ0wWprmCEs1JEUsDmXuNeXQy9Pj/5+fLqzY8nb348f37xsvgQHF+MDKzWFJ97jQ/3ck6Fbk/v IcslQb7eX14BoXTVEghbdQ2En4176DYR/iYA/JLDOIfNw7OHbfXcsXtqKN/apGt6Xsh9VjPmzgI9 Z4x37P+xX+CKtfga3Gwh831aZ4luWFwuYNHWMqn0aHF0sWLPVMenFYsC4QLX5wm0tGqpqxFA+GmV l83HqE5d1eJ0IWdNDJf7tfwVUSmlenxHa9+tFNUOnOlmU1avh7TXi5CW074tEK/rq8CWD9sS2r4+ KLK9O7jlPJbjDaTaFN5ytjngnOQ3D7iqMzXinAJsEXHWes2BC9Oer9jWQ8b3i5BhUb8tkq/trcBh jdwCOjxa6JX43h08LCaL8aHk2hQ+LL4y8mrpbx4gDociQmol2DpC5GLfAZH8gbUpsB5ITheBxKZ/ W2JQ31/hxB68VaDYjO8OKTaXrULFZrx1rLic+rmAZZu7Tw4wmvZ8/2k9KL4oQrHqA4vDbZGFC4xm B6w4cgtQ1AxLnO8OixaTxVA0T9Thx6YwaQkgI305o81j0+FkOkpdKsrWJzS5t+nAUP7A2gNdD0WX y1Bk87gtMXHhKN+ILQ3eBpIspUvc7w5NNpeV4ETibQpKNvcClmq4bB5HLh9bQKqRY+sgwr1PB4R0 s9mGr4OPEz2/LENPTv62QN+FHH0IUBi2nflH7zHnfO8OLzmPFScf+SOjTQEmZ1+aeurYbB4xVe8W Jp46Qf6OeYenYjyv/rhsJuglH17Eznpv9Ltv/KR6I05f79CkDg/Pkl/w50r5SYF1uY/ujz1l6Lql Mq7zp/SR5wNoXhGXgsN+pH4slTdU0Wp+WGGPeh1Fb7OpI6LlYViThcXbGqp/E5/DI7oWN9TXrIv8 tBmLDOVrBi7wLS4o0NR1p1L/vKLE2xqKw+plWG7cNf50XOkfGZeWw4IOwcKhL5Jdu5+8EmQ9dR4z eVaHp3ci+krvEcvoxRYj+YNUfONN/uazBsGUnQIJlZoTdd1X3tZKI/aAB5MoSR/Io8xuJX2o+UH6 J88fu86Xju22zKDdHeIk/YsvxeGYjhO6c4cRM40jWJBOJJRfnJyfs4Puk31lRfPKEDwYlC/HSpC4 lFr/6mVnZWkx2y3+YdSuPKR0/a7ngnqg5MaK9GPW4htk2nSPFH84CaJevLrEo+PXry7rJEREUZQV XiXhEizXx/xmS3WWEu7uspfqVSd8CA5O2ozOvPH9OiQTMSm+NqKgSFp4+UpM782gRjxIpvdNXMmX b5hzbUjEAyHjLgqZ8PGVByzxPfN2Hfk6N5JJu8DxS9GyNWxv2a9dWfDWFZwmHEb7wXqLjTVj/f+Y SqlRej/NMkMtNNGGs9d/AVBLAwQUAAAACACbaJot8Z3qe30eAAAOfgAABQAAAG1rNC5otFx5cxs3 sv87rPJ3QJwtryRT1GHFeSXFrtVlh2tdRSr2vtq3xRoOQRHRcIYZDCXT2eSzv+7GMcAMeEi2WC6Z HADdQPevu3E0ZmuLjW/3WiO2udnY2mJ/aw/21YPmHdtp7e6x3e3t3a3tH7debbPd3f3d1/vbr9h9 fCfZ6ecJ+xu2uR4JyeDfJMoLlg3ZOS+iD6JoMsk5GxXFZH9r6/7+vsV/n4q9VpyNt8ZQ41YUW43G 1sYG+8dQJLzBNth5JFLTmiWin0f5jIk0TqYDznQlaPODGKYDPmS93vmHvd4vvV7jB/gpUu48gX5t frsPUMOBsuv29dkpfVv8sVWuR9wO6EwNqMn6M/ZPHqWbx0mEA/skJpOE5012ivJh3WxY3Ec5b7KL s5YlRF9OTrvHnfbVdfvy4iGd6Bb5NC6mOR+wQVRETBZZHt1wdi+KEQN1jKFzWyzPkqQfxbcsSgcs SzcHfIzfkiwaiPRG9YT+HB5/uLj8dHZ68v70/PTiurtCV1b9lHLLQFxc9jn0EHtxPsujJhtmORsD cNIbViDmJpmUop/wltLPcj2u/DEKvzg8b1+8Z8eXFx9hqCD3runqVef0Xftf+K1zen7Y+bBcDLbC cTaeAJhZIcacZZNCZKnE57/v9aj8MLmPZpIpTA9YJNkOg5FvN1kcTUQRJeILH1hqVzmf5FnMpYQ6 qg1RG2hqv0qt6edgN1D+HGnhd8Ngbf152bUkklK19z6xJqbLm2CdssC+FYAmbfYjHg14LksdziYc WNSIFZoWlqs+CBJBk0GfsmkhBRhFrBhZYu+TrB8lbDhNYysv/Aw1sXZa8DyNkiYgg8OIwYAYWBtP ZmaYRqX4OefjPs/r1MynS2MCZzSdTHgeR5Lbpu0UBpzGnN1FuYgAfLpxTzc9BLRK0zzJ7ivNgXIh YjamDjh8g81d7pbCWRaDIAx3lkZjX1/d+czfZfkY2oKfhkYgr2pj1dS2azIOvSHsTFNUbZzl/Nub WgNZb0Efj6d5ztOCgdY4sGdv2M72NoSFcfQbQPYl/MIfIqUfY4gVBU9RFdb9A+S1s9W+9iN0GvTL dvd+UkzYbmuv9ZPh0ISIM2OvwLAwyH37mMFOOOA4jxTGQCXktSiQ0aAJ5GhbHwW/P2DBD1CJ2B2U g09Wakhm6ADRk/tEQHzgAoJkkEgKHRjwz/C3yDRJv30nu+/w4dz2ILQhB/0A9IlAnqn2VRohAtA+ A/1AE5RCiPnRrOAyLAJoPJXga4DrBGtHeQZyYDc85TmYUl0QXRXf5nQEohoUSiXTCQIEPBmiyKNx PIUoOUa98KpITX8wGsVUTXwhU0SKstqVnEfjuTLtyyKPYvShWE01qxGICn4zC5AAAnIGfR9ThERU oW55PoziKr6u8gxGWsyhggOJYowgQIC8LyhK1pULPhYJHdRLzrL0Zk7RO5hAzGt2kk1BbnMKYeRi LlWCy5wyVJoq0p6Fnd7xfAa2B2bT5wmCsJysQrhgYgwTsDG4HrLUJkuzgoQyge4BwkDbFXl2YSqL mAnLM4JJVZJwCi3IgqRZsTVjS3UKQIAsTSLgsXcAkCl4bZq63UXJlIdVg7Yb1ky4hBQTLlJ6CZeR 5MNFSmPhMlQKlfhyOOETiDEgCFGzfpAD6sHMgkYgL7/tLwD7pGadq7W9yAoxFMHGS9teKZ+xKt+n iZbAA0x9jFHvfgSzZAjm9zAZ4MpPmjXTc1jJDZKkNXqOTxOR3uJs8uTsrOHM/9ja+Qd41Dv919Vl 57q7zl68YN/bMpiUfmhfQ/m6DbP2Edtp/ADqE0PbqwzXEkNEliSvNCAkUeyTMPVI2Dgb8ITtvN7s Q0WctCWsPxXJQPo96p1cqo6UT8577f953T2vd++6ffG/XufwQb1r2B1UyN8lWVVGMsMJjnYCEqZV vNVqwfR7MorYCAT1eo+6mYANqf6tld3pRVhtnf33v04feyJ6vbdOXfy/xnfffVf2s/epffFqt975 s8uL99DE7b56VB8A1IimSaH0C7pEd6ZWRJ/FeDrGWDbECR7NiKCzHp/2xVn74tTjox45fJ4GqEdZ nqDsj1++ZOb/I1Q5zys67x1dds4OL06Oe71107o/M+Om5UE+TWnVBA4c0axWRxEimib+gLIMy/Qs sEK/c32GOP4afJvB/NjaZhKm51leKDj1s0y56AJWNcTXHQ57+4Ztf/5xe9vlcXR56XBQ/yEPt6Fl 3O7AYnMBxxIjSL3PAQmc3cQxoiXOIP5BfZjHY0N8SmtqMZFXnUvcAcDlaEUb8kash3o7pa0XZG6+ F/mUm+/DKIFgWRHa+4tfkekWv4mrZt6DMlR3TSfILshf7/38cvjxlCyF/tiK/uOqdG2PAB85rnJu JSzGB/xTlOciq8Hx/NMprOsRjEqfaqm+hmMPdK0MAZ5mEI6qIbi/mVEhLEd1ZZKbWbXGWTqEGUeh t2bsVHZ/v/zOUJt6V+ZB6qjKQMR5JrNhwT4KiRMMMMyKBM67x72Ppx1rjOfdj8dLoN9UlfZan8mD pplaY05gTqp0b4iiSezshC1CPyn1CI41LV7vwRBoPGWLi8veRTfg69ktxL8btVK5ywR4/uiOto0y nM2x6Q2oAg2cf57kFK/L5Znaxmj8MAFhjyN2H+XoadcGQuIkf5/t7f70ah2ZQJCFZR2tHoAExDb0 SKAg15T1YJ9gaXlJ8YskobZPKPiD2sBjolb03LW6m6pGR6sF2VBhwggeTRC+d69PmPmUHiWqejj9 nBDGSi0i+hwVUiPQnVUQciz9a/2DyhvAvKFcODM54TFN1hBAtYlNlbKOakHKPKV1GkVPjrqPUtwe cPcPdJn6r4I3W2x5FmqLC4xYihu0mXgU5bjB1YdZ8gHxjGFlV4BczGYYRmxdocl4Ebdsz1XctzRB alhRvNr1Z5qIvKGak9Au1+u9vo592GIMC1Jw/+zVbl/33tDDFiGCFXrp3wuHJPyZERdFz5O0nqd4 lOJRluHuG5jNDI0N9ybBsaVEk5anN6jHWp9e7x1gd13vY61/3VYvHYLfphIMPOpVFiARSfvh+iH7 Q9fbOdBfQDx/HjQIubhqjsDtvnnD1tDYbaOo12Teg35vvdLm55WaPNHcy1t3Nf5oTPKsgAUpH+w3 mLuA3WA9yX+HVZlyF/uNrQ2IiKmSEBofOgD3F5AFpONaGY9imGEBY3WJvtleP6gUuhsqG7VStU9S fU7ScrYvXrCJ/tabUxN/vqCyv2wh/Sxrvyi1GmxYrvA2mP6ytq40d6DdE8ATVjA7O5u729vbCHOJ OwhiOHNCNh5tsfe8KDDsbDGpv6FrTqe0+6x3BpQc0Xahdld84SU3Rv2m+NXVZVCvSZXfbO6sH7CG Ke/wMcwLDpMEhwus9wO8BUDtTgww1nO13yGtEtXmHzY6LJCJ6YNXagT37/+wSh2nm6q9wbptbIWr aZ2qHmhutC9A5gM9aBd8bAfatJsOLxx+hlm1quFZaYC6uFYzLNoHRacMP+YpoxzKe3g4bzhKO6iL w8FgLTBc5NpOJeDVWFLC3T0hh5mqNk90WuM763boWt2FgwendA41g3Ej6rbEkygIshAUP8GEs2oN FXF3eJLFEMugU9JXD1Wnn3rUJzxKEHE0jaXZiLJbwaVabgu9/2qxfzEdX9k6pQUgZOpe4KIYmR8V HCKpd4Bzr7ha0MZ98KPZBUxMzZAhbG/4kCffcTIFs8YxrwUKjyFceAUaCpZ3veee1zLW1Ax5uhK6 VgLYR5CsjHOhFiKu/ZHQc3EHrom2oX3v3IXJemgI+Pwyna93v16Hw8pD8kr1GsDcnpv2HLdD6zYS 4EQ1O1F6w2vVA0YY4AUi/M1jNm9QuiIiP5vOrd+wvplgczlcNAqs2cF4CWu4Lo/yeBQYBFTSf7W1 qNAY1Bugz/f51tiD4xlMVxk4OI9ostjgLc1I5MtrHWcpGMkqjFPH5hy0O5oNW4AmAO54OjmaVThV EIgUwr2cQsSb37atjjdCTX9NBUxt1kJdgqJsBQtq48mMXAmXJ2JIhwMrVD0X6bQ6olAn/5mJFMfm V6XRNlUseEMLuQALbLrE4sMkPL1Hg8s0mYVc0C+RrOJwAcaPIATdYspCvegyh4UtFM1tS/bLB48b DJipMmgMazZqYUQJWzhOgX2/MMcfVGqdUYgN09zwiaq5OgRwPHVR4RTKinJah/RUharPdpU0zAXu gs9Z78wTE0WyUNPvH9/050e3fPt4no/v7tvVmvprL5pJgauJYbEEil0rJ229E+48bJgl9RYj8NrD d7XojxLM0lBn4pzfMn1uSyDI+hjWZNnr4CEcED7pXF5dnZ7slwlirl25FdxZhz9EYomrsT8Pvvki 1s1RymQvEp/dI6E1s2/dE/1x/FmVVR+yt+zH7W3cwJyPb21izgLdedhXK835EH9w65+/qvXbr2v9 VR1/UGOzseFkVGB6Cu5FqK0G5kD8CoIM5iPpE2+pNxKU867uVZTNTMKQk9hSaY4usEfFeqqFfZ23 t4HxQ1mZdfKWk2nHIpaio1WDYeXAvB0Quxhi7ihPLH2ulkCKCo76eZSMM1k8t3k1VpxW/BvMm/CX ZE36LQ0d5ld4JLaxpmm/hPXWUPJiPUizupCnMFfwycRsVYhCVcRty/soH0gQEyaJqu+eiK5yvkkr a9ocwsZ1IZV7Ly9f6o2ZEgKyWNrea24k7HZgwFftwOZmqAPL2nvNhdm4MCQOB3eUGtifgR5vYDGW aunPkcGb+iCOMAN3OlmVxOabQDe6075KK4rs3vkgQKUkUt3NMYQ6HIICbQKzgdBpj31e3HPcGb7P tHyktjOHnOURRiwskkM9M+Gqrm2HYmlZi6jRsafR3iK6Qm9h6O4aL2FmVWCMmGWsT1g8vBvh4IGH GKrjVS0QCtHgCqKkEZ46UPwpR+TyX50BZposYvL9KkyuNfWhyNGJK/ng6TpmgxWjKAVvCioc1KRZ Cw2P5XRDByT56szefu2w4CvJjXZuVxnd1wjSDO+hTN/OZ1qHaOzN++cAiDyaqZhpQNE+5EpwrUT5 0I7oY3mvhOSH8ffVkaucvxqoVaxdthb5Co4hcM9nGpiePXqYAbwtGO03EvACsC8Y9arMn2CJ4SYu AxeYnZYjPKRRlVM2gZlGxrBTjIg3UyH1JFVHlp4qLw/UmDoGSmGoZsT1UHJIh2pkJJTgiRYSpSpD DftA81rMI0k1t8o0LiDAoDHaKQ0e42WxiAq1hCTidt5ZCZIvvHlngKSTlI4mHsHiLDcZ3rHeezk2 Bc4emrcuNnSPzUQbnZ7Jw73R8ptKf/zOPMPdyqgsOQ4CJUDgKfBEykwZeDm6uOZnAuvrVpMZmMQ4 SgsRy5a52NFBV4hnU/wzTLR0DsdY8uSOq0RKmNO7/hPFK1vUzjmSJVJ0FMnJzxXUAWCY1T0w/s5y cSPSqDBhBFAwjgbc9urQPLUJ0jrhDm8iREURxSN93qAGxz/DrDOZUa6/PmFs+TbG9k1iSqnFckkY woFaESn6aeYcJFkkVObx1ZbDPBtba3ItyDeaFwuJkBQVJd8rzKFWM8ETbmDdUGfTZcfrjqDU2jIv ED7MDgwHyD2alh2M21OYdGM+r3OwhzBV+elmKW69Cm1wHdMpAdBbyVmpdTapfiTikVngxkSEp5E5 SMV1CI3L3fBS4nWXD6sFN/BJ4g7IIxKluqlVOsTDRJ1/Kq3pYnMyiufJnjt6ymBFp9sUq6Z4+sBU 1NJZRazXn+IBAvv3zuv/KMGqLGzWixLMUsopjSFDEXdhDhk31Uo/LfTliPtIYhIQT7PpzajVagGJ P8m9agYbEOW0I0HytMUixReOPyiw9xC9ZQhUAKM+a8SXP5UWUIgb5cJufrHanA9Wiu2h/4G2MJej 0yAMc6etOv2/jyZr/nM9Vi9FA+kSCSubYy2aymaNLe/y4oj0Y5f+9aJjgFNeruuXTslVJx1MO71e Op+e39i1BrVFfR7dcvQl7rb1WSbts6cEvUl/dSMIzi2WhA91tQoi3GaWYv6tpqI2yRUsDOW19QMW pFFemZ5KszNW7jhIfUWLjdSFlArRuLzE5R8ulXkby7iqzjbZLed2Zy7We5/S7N1X2Sp1YtJAfcuk w+O5bEyYk1NR2GttYeLuMYchXQa6JkhrSknPcZLhpWJMOgUJTAtwSFzS7E7fPi+z/LXtOipp+ElQ 9qjCGeC6dT2H0yI7JpprStK4ADX+wqqBmW91O65nWbjPdV6b4YYpQnjuslb2+EXJjH5TDpiq5B66 UXOvo+q80Q6ko2/j10obdlKNcyf8vyYJXa44V0oN/bMsGrwDRZfJd67zi+74deYXNVC99kBoDtJI xXjGXx4L0ONBKU91sOCTot4iPU+rzCdUPXCqEtHXwbr89xcgvay4RvD6B1pudT35/Fz31zW318YM a7p1HzBmY1xP4Pqelc7PZEU0/ngGvZKjLMdzjQFpG1OOe5jzeoBNwAR/69/qLEVcfqtUfX1TFQ0O r3+DrOUzx2+6ScfHv3avL8+PL8+vDjun1TjB+ju9Wqxg/V3QyDOHjkdjg51deQ+oasjh4ZTPpHiW yWO4DU6UaYopBgriRiQAQJBA3cc9lC6iLUy5al9/eXUcq3QaGgH5qV8Ep9D0w61W8UWUpub6KdQ3 vjvBcyhC5ygNvKeIc7t/YfhCjfoM3Gmkp7RD6R8NlX7FEFqYuPas8axRum2V4mSSESpwYHGs0FN9 /r7aEMm5PXIn4ThpSqeTtn4RxElURGvr1A3HnqscPPJQ+c8Dc4dki1J2bmAeXS5xWqVBtlWykDcb sepfuqJ18agEa+jVnHk9rmoE2hYGgKYmSA034PAUhJJdvVHobYncbPa06eLwatjwWSyhrW4aYM1V CHaXEzRQCqzlFC8XGFbotCEp3U2ELOWGT1N50hHdn8cVngSSrYZd0hup4DFphceqLBxqRL1KBwFn buHgvdUSfnhzm8w6iD9z4/6bAdASXB2BZZNFEFS3KpyBVCGor6g/CoOLieuLFg/B4GKCSzD4eu8B GDSMHgpCh8nKPIIodAi5bo/eC4CrjEk2F332pQ7fDH4lxdXx57RZCMBhcERVGJr3ITwOhyvw0Lsw D0LjCmQXYFJxnI/JKlwq3GgzLwjPMDor7B7DrYrTCkkXp+olFSASWIfTS36CSC1fMfINoKrNyKG5 OljdRgvROpgzripe7Us6HgfYldg8BrIrEf6GoK3xe2LYhvktB255K7ge67v05pQwgrv2PThBBD/C 1zoUV8ev22ghfqU3lipqu+YdMY9D7SLi5UgeBtgFNBfA1BXcylhVrB6AzRCX1ZhUARmi5LrTI5Hi /fAgBO3blr4VAkuCqwPQabMQf33a9p8DP/P2osehbznphyFvPr3lwKvfcVyGC8XN7CU/FIKBS5ir s5sDxhpNF5C01RCEI5Z8SzRaequDsWyyEIt3zhiqeDFbuI9C4jLCD8PhXGqrwFAdAayMQuT1GPT5 bFbhMhd0PqmnPLNyL717t/CDJz64o2be34AvecnFgN7qgInpdLOT6KphuZQJhIGseg+1dyKn18j9 FWrsIdhJ7ZHmtMW/tbsmzI64fmFeHCUJ5t6mMV93eDlgvObjSaIu0r5h2wdhdpW72OaNLFqZhqre 6tTnsJqaoLtT2e10EsCqUHctfRKmtpdY7J9lveNFrCd9+PI9UfBxk+Wqv6JQr55yX0Npp4KGDx21 LLzRrrvvNeguvdeOTaiquundsTeyg2O3d8NdJrWWnhyCTdSl8457/VvVe6JEKvNaTEq8Mz0Z8QTc g87kovTCXJ3NIFz6ahKDr0FreUfHSMeLDK45qGJWuXallC+zsQll6loJ4j6ZVbCEFxzXnCyGKsbp bGsFWiTmT7mwNwBrNJ9Q1OoQm6dxNJHThLaT4myAr2iuXOZvb12aN48K88LRiriJlidw57l2ORUV lEU1qMqP0APv0qcrfIZb/aQBtbfarOaTmMoUzbCykrCpXZN0sI0+rNXbt7UqHQ4R/51I+HlEt2f8 IerNcCxXbitUmHM5Upmi6nZ/eRiDiSGYX+Qw141O08HlkI469AN6LYgL42snA9sc9NNrAgu0KzyD x1yqO07vNqT3C+Irium42TgZSk+R7xIxmfCBC+vTPIfoTSABnw3qL6ClwEiAEFFhnOLDF55n2Al8 fA9uM7vVLruH9SG+eF6XMoqRJ5o0vdd2rIRaEsJ+RzH2u2FWfmV2EtSmN1m7RC9saFEGCGSxn4Rp Rci8SFgLsoc/uzqtiWwkSVRnyvsgmLue80TJz6bUmesshlAfVHdJDw2p6xG+YheGYEjZd1CO6GIn A4fEQVDJrXToYIurTLpEMO3KtCWCcZZMx2ml0RlPn9Bx6Ctv5KXT8pXGOGrtpXGKRwmrmHSW46sd KAP1/5u7mt62cSB6969Qc1jEQTa5BD046CFILwtst0XangtXllOhWTuwY6RG/3znmzMiJdc2stic AlkaUk/8mCHfPK6aermaJQ6sJs+tnWrUdLVCKXz1CE6Jek1p3FVi1mJSGlFV1mM+piCJrqCHucJp C9nXcyLmIlUR68oZtMxHqZebhxnY+N6giFuDGl7tv82FkmBtC7VGCQGmybZMaKY99llTY/btsyhm KZEGnFw0e7+sps/TbRghNUPxp2uid7EY7SFQOikX+Nbs8iVnqqC7JU3+cww+Fhtoq9xLKF+d7yH4 sYnyaOyld6HTzLwSbw85+/0jwCJO3wScPqL/urgBPozWWq/+DQ+Eqkcb9jD0WWoncxhEGpWjHgnV 4SzZg+GV6It4tEjtSR+TCQ3Bp2NX1lQYm+iZSqICqv993fpvMJkIH09SJMnFghJhenuEX67LVP7I 39dGXydXno1pIY7I5GYu3daODlhhmhP3ax2ERaLj6r3cZUeJKSQsRb8FcCddhI77aDm3rjL4+p02 anmywpOp70JK+9smXcDiYBC+Q15BihbB5M0MlSzxABLsp0JF78lkcry3GCt03gsLQjdVCjIvW2bq XEgrfht5myCiNOgei+E+TSZ/1zu4x+4YMwQfbJXDCSKxxtFfs4LCEf2gX22cX7XPqIsiGX27GHRR CCMkxhR9uWzqDsTS+TKYO4VM48kBWOY/f8K0KyUNFZR6OIk+yf8OkYFgskZi1w/TeP/tIrMuUEmx t2xwoHQkxUduqIfUPIQeROFpfcH03mdZLGELHmpWXhBcJPjaK0zqqUQSOIfxTB63gvKWwuXliOTy U9x4RchNTxAo9dvndobrMvMuMBhQd0DAwDcI3Y07nxfmb3C2QkzOjNU+aDUGL8bf3jgHaVNnt10M fDJbG4sjpwFZjtpZqMum7AXp0NeczZBG0hvKrQmain5ULf+YdbvHqKrgZ3hzE0YFXwDA8hcCo8w1 KdXQx/Gruf3Gkl3pclxxoZpjin3fjaO4g/N53XziCXiemqlyWINzEsJIP90qXsSmv0Vnha7qbH55 +YqP8Lj6QlTRycSeBj9i81j41DqttC7Ed4B86JONFPKckCXV3bgOTNNQ+2wQ+oPkBi/5eAGXjaXv qEumb3qeLTz5UgHByidM6sEt92lVnCii1MnOhU4fljebFBC8ZyEbW3tk9xmDLqLyY+NeCMv1a8NF 0ZIcem5LVjumYY6MPX/D2O1epDAxLljL/2mPodKl7ws6uW0rycl4aBJltT1syZR5eVwghCoYNkMt 9Twpk+i277KeVCfm655CCeOTkBln6WQ/y573p5S6yelQ1LftiBAJXc1JSXmo0YRVoZT/6VPastEs eeDXo6ENFnV8k6M4tHfg/RNn7SaJzAKqYazvWO/JWbOfo93udERhs94b56R2keuxliaiJ5+zO2BP ZyFaq6kHNtN0FmKjrWra9JjVkYnNduYb57pnGDwNJuYnsSloGL+vDGCw+4Vo/y2OqcZ+IgE7q5Kx /NWOy3eu3N9/MoKOFJ7P+bjZCjeWUywjFRn3EUNmrR9PdndWsdDTgdKDuPeYGvx0rXWCWxQ9XXgq dZqP4XEVazKXMBmzKoXubauf/ZzZAmzG58xwU97pEcCpif2R02p1oXt9dQh0zlqqVQbe66tx3GIv wFUi5WXMzyNRMxv7wxar58ETWtYB4GU2XRUDhsLp2olhmSGWsxKPhDEZ+f/iaHXsB3KIF1cAV7b4 ZIk1Q9YIPkcAazb2xzXUzsOqZg8BtmvUVXFHMrMrg7czyXeQcLnBU0iXFRTxMDtHzTEoGGdVPA2i Wdw/Uc49bri4IqsbivJt7wzHiCBKaq9Ebgx7+StacaMkz3Y+/8I7MLaRri7AekOm5xuc4Glmf7eE +7fd19J0j6zwHb1SaH5Zi0mMxCOaTDKyf5sxvl5qLGyCAuED2osnAFrFSi1F4ukdwBHlI4NN6UtH gKYm9odMGCG+dxHR5ACwElNIqjN8PsQLyZzawTXxiJuLdvFw8lJnhPgTx/TY9F9QSwMEFAAAAAgA JGeaLbYfWundDgAA1C0AAAsAAABoYW5kbGVyLmNwcLVabW/bRhL+HAH9DxNdIZOKrLf6Q2PX7iVp fDUucYLY6ZcgEChyZdGmSIakrLiB77ffzM6+kaIcB7gTWkda7s7MzjzzttzRCJZBGiWiGIZ5Dvv7 ndEIfj6LDt3hwS1MhpPnMB2Pp6PxwWgyhvH4cDo9PJjCJrwt4fXXHH6mlZfLuAT8Lw+KCrIFvBVV 8O+4GkApBCyrKj8cjTabzVB8WccHwzBbjVY44yauRp3OqN+Hfy7iRHSgD38y9xLKKisEREEVQJxC mCXrVbq/iUsBi6xYBRVOxrX/iNMwWUcCuksRRCj1suuOqa3UBnl5YywWSVQfYo71sRwFi0u5FkcX 8PTLwezs/M3Z+esWpnGa4FqRRvEC9/i//JDCw4OZUlWnc5vFkTNwePgqEUHx8q4SpYfD8ksP5uvF zEdFpmXV+dYBVHBQxSGEy6CAv0WRlZ9+/XzUwQdxWkEKx7A4mElCp1Jh3vsiw/1Xd54/vLzLhef7 Rzg5OpgFZYnjXgq/HUMZ/y3Q/JKeL6kRWySm5QBPPhtAio/vOx1iVhM9WyGGhEfjcRqJr7MBywyN ncg9AJAqChFUAgKcl98ROqBcBUkisYMARMMI2lMJ65JnrVZZCrO4EisitUAVgpVPEvGIxfBVllYC F3r+QPIcXuDuGj9o07/yTg0JYky6+Zeo2Ah6I/TAP/qp8xNpeQHeLFc6HeLUV2uE/Erv34enx3D+ 8c0bH7791HkCUIhqXaQPr/E9uWfaArEBuCe51EoyJ0/cZdCBUhmvJ+tsIwuVF1eEqovgVkgFfa16 bAwLhfGO1e+S6A+xiFPhEeoG9Ow9u9QuEtLAlQouy2yD3wVkSQRd3H0XNgEi6UaIHOIK1mkVJ5Bm kGTplSggFSIS0bYUjlkcjNXQhSbOskQq4oKwpOAmPYMsy4gkyn3IEd1I0hg5lX6xBfscnwwcitDr QarR06ast9ktu8GiyFZIl75WmZYE0SPHCSY0CuwOWxh0UKjoMAg78tkHsSIu6slEDxN1JAonzNuX g4BJAgfVlLNUmgkHNGomct/3tJfvRrAf+dSj3YX40unUfh4e1n6CZ1GFxuEvcguHMFM/0Rp6fAAz GfwBEYffyT1Qz/wjXa8+ZJuSfnW+AW3su4ztzx5km1QUDC013NfJdUsenjtUcqMnW7nUo1PE1ymN eIaGP5CmMEL3eGqL6CDDIgMeUTk+wn9+g/P1SlLEAHcEz57FDojkeA9YhGNgxrF/pJ/r2AEUjRCK Mn/qKMK/zoOVThIA8g96dzS/isTCUylGA+tFFCkNeRSlZDxXQYroD6BPIUDTclPO8THEFncN8/yn YR92HfZd6d9FllVvxK1IcItai0iRmB1xQK8BSc7iAU6U6CaWBPqznIQOOeaN0c/9kz+yF+sqU4HT quMPrH/C5Sn613vJmB/Z0Qusf4Ir4VXFWshHu42oayZtRtaSSARmxR6cV0utXFbVTAGoHF6ISmax sWYg8/0rlEB4/vYWNT4UaQZo3b7KMFYRR+4KGnaNZbTbMJxxA6dcUVnMGmA7Zsq1nGA+oMyegr4V S7nUsRbMeWKtP+ado27O2Yk8FRdVGYKZq48bKkO0QffT565UqXYUpJiKjfEg8Ghig9fTvhrsWN/V oKuBeEsv7BnamqYs0s7Yk96oMoST9Zv+NNMOpfkpb2/TZFzFWVozxJY6jZ0t177Gxk4jNcFvsppL VMej3Yq3wQoWjrJb1U3DH0SJDhWikJj1MFIFSSnUI2PDMUO0Zp3xI7aiPVaGlsXaVg2y0lOR3t0X eXISY1igYpvmw+8wxrTgxmXOtrIMuhIVFCgA1tdU3movhs0yDpewiq+WFUQZnI3eyTUmWJC+GlFi fz+EExlCtCzg7KsHS1pi40Zo5ZCSEPsUNSkijvUlBNSrSU3gWCHCNe4WI8adWkUqOCvP5RKk5hti RsiCI1rBEU26nYxmhU/FL61fDv8Myov1/DYWG6/w5Ti66Xr+Oq2KOy8cQOEPt63BhjDyEyW5d6l4 u3vN46xUwUei0n1uQ+ryyBm00ZSLqReVp8HGn3pM1aP3Hfffe1t3sbyG72j0MV0FeS6iF0liCRg8 KbgyofsHQCqKFcbFizwIxceSFMPgrKEEDRBu1wWhb4RzbfhY4PCkRxl6l5m3jby9G62FHRpwwrks Pvj7rNl2mMLpxCQCM3RsVrV0rpKJU8e0F311ZnpcR9COzXIGVEjSFnwcr51ouGXmMizinEN2LWva oIox5mMpLsUqfyl7X88N80OmMBeocQ0BHyPS7vDnBtRG3ai7qEL6hdr9aLQrf3QY7i3BZbWmgk1A iQFwjdXgXCzobGiDI0vcQyKoK1xhDMrWaeRi+oaxdtNWJt20oPrmcai+2UZ1ypzSJqrTNlSnTgB0 wH2je0cVD9qqPVbsEHk4FtpRvKcLyiu8QM9+oIZPF6aAx6+memctqS3SkjPqc2UdQScQZ5Gp8Wmf qiLXOsywJU/XQqnLmfUbZUEwyrZ4574Sg2g8aK1dGr0A1MvboTpeMDYYmBL0HgTmeieeI9QIl/JE QbHHTlr+xK43rdREt9s4gdjwbRfZjqJucCj1WxaYREH4cAzeyBUkIWJ8JUrM8XG15LOPIkI5sQRQ 1NTxR0gL4/QK4oopWbF32MxtnToWNn2kLasgt7pHQcpchHGQQBigEtGGG7GHLtgtjPsT8wAI35Ct 5RlwrhNpKLp8EDwXSbaRFDtOvaXigyp+eoxYGw8w5xr7YHkTRJEMVigLhS3I6HiP66Fb4iBSxLOg wi2S4WaHi9aDQerLucTRwVO6C0+dB2tSU5FWGGRNUaq30147WT1u1U9mA9e8geu2DVy3RLPrx0Wz 6x+KZqA+bUENuT2BeoLood2/IDUT5K45yKma7rpebaiaDpcMa4V6I4Zd+7ZyxwXSr4karXNzSjs1 ssoAVFv9xKm/uE6vnQLUWmlyjCN1ICnk2QFUwRyzT7ZJS3S8Uh2XGFbIRIgdpYKz65Ymt5au+df+ ST3om6wsD/CbjaJ8eSHDuT3kbGHkBk+eM9w6FZZM+Di0zsQA7UEWNVF4Dip376+9h5tPla0eolwr XGyC4+k7C5b3HBxU91wdzOZ3lej3Ia8KXbCUIllQIeT0b/S01rvZsywkEf8yhTIOSnuiFdJhO20c GWJB/1eQrIXXJyrK22x85oWyBiJspQirO1HpKAkK2Voi49GKLbnqd5nprHti5VdCXlQycJcyfHXh mTPmeU6M86HPCkp9nNTVzbeksh0JS6ddtIll8eD5CH/aT0n44zpxv9G800dKCM+O+YiePtq56S/r khoQruUJW/73VccHYNQQuLpzugl6aKV83DErGJlbqxciOQBHiAd7u3dJpDG91c3UTh2OdhWVO8+B KbJkRXxF3mtKgv0Tp5gcvlOPdQNW35B94aPJYP2mpVLFx45tnSZxji0vv7po9KoPH3/aQnhXytNV 3HJIXBQLI40M8eeI0snz58/Hz8e/HEK5CXIGTpxy0OeSjCqdKluHS3IYygorrGV0XmdC9DqSU2SJ tQpEcXkzBPO+fF1ivp/fybXKMf7CqdRXJVmIpS9hbNiuoddf0UDplaDEGqvXWWURvs/K2ksHmYWj slKvkPAbzWhCpaZHCke0Ylgb9etvOasJZtFpHVRhlqhjBPzSYhwcpgj5zbiWPVDAJxw8tY9Z2aQo 9ZlHnVHfOBFqGSXAMnCeYZUsWB1Uy4uvCDPtsuv5JRmO1g+0nkz0kCzqU7SijG/XVNrvgZi4hU0r 2a0lUzg2vJx1lldz78hElgE9udhtldXOJTQJPtUm03tv5Q7kwmKiWUjpxdT8JMmwUeuPasSXWVFR 8Y1uGywqOswgVsSzjUcPQfFjOsEV31WJK1AQXdNhAAnBlZreazUZNs/S1YOp86BHjOoUqcGkM0Om KYufZh3n8HBzEMdAgmMzFyFPd6Lc3a7ZnJ2oht2BfqeAotXGNxvjbTjFwDepRz6auW0InGessDVZ 3nQZy3YZg9se9yt7L/ewUtnQVQi0RhGHN3fkcJHAtowCI5o946sU8miGAkSqWj92/JZd7b3d0+VT rUbhqX0IaS/LCXWwKORbFAafeCbimRdl2wtpc8tpY6ECmFloVqrKKid2yPN3/LN/glP1q5BDfeTq TiYWyAcnT7cnN2eXNdIojLpL0ka5rFFuzDWTLVwUtshe8yK7EakD9qeovQujBJTSai+fosnw/3Dq 6JBWTBsrjNpyjP8l/h9OzIp7fd/pCXKfjPcP9unm2CEmU8xyCA+6gIOgoOhMd7roKEEgWNDj5GWc PIvpmg7+yVwPJdgVBKSAEswaPZPOCwaUWRFJQRjSAUkh9pE6ghNjFvX+6mbP0CV0SaHLOcdgKJc4 MQ5jDBDJnYEy8YorBPv5u0uYC0zvLqEsr+IVmiHSwAbZVRJPLNxWK4zIWL2V9laJkgbJ+EPzJoIa bdRfamJw7VaJBbs1UmpVXZtr8W3Nk05rEUHnbGxliOek4S92xhRnTGl5Y4b6hxHkeJ3FC0PF8Sst rSlfG6G/Uc+Y+K8qCVWwYNk1a3//6JYEs63yBA1C7OvxDBtccx+GCA/kPF5qCdOYvt91bC611e93 9Pt8tNcYlNeBtmZKLs59MumeSVbK+2a0sRr73KT8fr79bqDfz2ut89ZbAHtTxLwhVo+bryIe2a3Y PlQvJ630agW1jtxKyJ572qzGdr9EqL3i+nHJ6j2HJqZOLviKICmC0jHQtVM+paSrTRgucowcWB+U +jxOHtrY3N+xN9A2cZJgCU+XUu/odQQSvUVzcWiL4kKEFEEkLVlMRPZtDNGjGNK2+ZfrOInovqzc L5cqCvoI2HS94mqe2oKeFH/G136e0MdYWJ2i8FGZuuhyMDtLKzrPgfw9eN33XSw98lf47VVXQ5eo 8oxTHD8143wGwE/O8cm5XHuJ3y67jEZ9Tiglkq+pkMknJf5neEacPqkNfHbZ6fboWPKUyz+ln/2d 3ak6/XvolUdTCfDAew/qaSvuZxdDt3NlsXgn5/BJvwKRW7nErdiDkV4lG9mJ/1ktlfflJE15sqV9 wVo2RXsOlPFgIVf9Hy7K/RdQSwMEFAAAAAgA8miaLVuMv7SDKQAArI0AAAgAAAB2aWV3LmNwcO1d e3Mbx5H/O6jKdxjTVzRIgeDDuroqSlQiU0rMRJZ8pBznKpVjLYABsdZiF95dkKJfn/3614/Z2QVA 0YnkS6qsyp2J3Xn2u3t6evf33XXqb4bjxcLt7fX2991/nE2Ow7PBtTscHj50RwcHR/sHn+4f/Sf9 efzw8PjT/3I34+vKPX+7cP+Bbq9naeXof4ukrF0xdV/4OvlzWg9cPfNuVsz9IrnyaDCr68Xx/v7N zc3Qf7tMHw7HxXx/Tq3fpPV+r7e/u+t+P00z33O77my+yPzc53VSp0WOYedJmrtxllSVr1xe1C7N r4vs2k/oD7fwZZVWtc/H6E2DfZzm42w58W5r5pOJL4ezregZPUipZ/vheFnVxbz9jJ6Unh45+kdb nRalO/dZMU5qf17cVFHLaeqzyb1a6lrbE5V+nizwiJ5N3UffPrw8e/ni7OXzqMn8zcNhmmfUy+eT dEoAe5//gMnxw8vXs5Lg9aIYv+n1GNidh9/3FstRlo6Pe679xvV3HvXwkDt9XmQT+vE9/V/T3vFj bogfP0W/fuS+FbA9dvXDy/TTI/d0Mjn307782h4Qmmtqi5bv3s3P+CcQJ4B/ffaSZt3ext9ffPXi 9RlIErToXuXu6zT/9GjglpV3iRuXKa0zyVzlx0yedeEWZVHTLyb6q6wY4e3tfFRkrk5GmR/yQE+z qsAYlasWfpxSm5oBuFclU+9oxKzCWITy/YkfO/rtSj/1JejajYtlXlcYiMcStmuzSekXWTKm4bGK Remv02JZ8aKJgV6/uBi4m1k6ntFMOXEQDzPyeD9xN2k9K5Z1s7AyHb+pwFuT2zyZY7vZrcsKYqaJ e/bixSfVwCXgex6lBEOX9AZ0/+r0r/tPCTDX/q88d18m5X2/oAFepKMyKW93wlZeEuiO3WmxJHqY Jde6oi/+cPpJ5U5PFdgXCuskn7jTizS/yjzTnYqEAY+Ef6OlYGFcENeMiAdvIHwI8LSla4F5QtKj pqnKCe1pntQ1seTQBAcTwxcXfzkFLYATL16fn52+7n28KJOreeJukjKn6fuTtAJmj93Do4PDHTB9 XuREwvmExnX+LUmjCgvmzRy74XBonBt4+vFNmk9IOgxnT3rME8oBp+dnr89On764vHh++vrs1Ut3 BSgAAsJjMeMdH3f5UBnPubOcQJdk6Xe+A8T+dhhQ2G/NqOBO+f/xkM9zgtU/PNpP3eFeeEL3fYZT mdAZtS0ihEMuWVI4gvP0splo8vASZFLWfX7hTk7c3qH74QcXfj44FAEGYq6XZa5vHrsD9zuCJG2b pPkbP3nmxyWzXH9b5tvpufjfcdz4LO821h197LPKQ5JBKGSuIIItO9xc0UKuacWQCKDnil+7LRUZ ma+qLeLkimRLjzdIyiHN/T2p48c7O61D/n16/HRnl38WiYoYaeEenMj7ANEPoRXFKgma8C9kHZnU ekoLyTKVSiRfJ0mduBLc7IJNxHKIEFqS2BFFKM2qulyOaTMQzDyYGk2uX0xJbrAqIKmxxbpja+C2 krJMbukPEq9kK2RMIFuQzkI51S0ZP/OqEalYaOUSmqCawy5bFCmosjJiSmh2WlOzAxLnla9rEmyk 4GAEomXict6vc9dJtvRuUqjtlWQ0mErZAAPqIJK+xu55DJl2EnRRdhvpL++8kHsF8F3zikkzQSWx 1oM+oTFHt5goLd3B3iip2N6b+Leqyngo0r2TJdQew3+vUZk0bH27YFolI4yojFHjSSdRu0a18rSm DcfFIvWTgYhtmv7K14A6AQfzkPFWp6Rt8IisSBuBaJ5wYcq1YhOWNJUrRt/AKKB1BBI6F30ktOwn O1gSWcAEd8bLZFISY6MHb0a1q20o2ikNdbosq6I0pU4oJaj7kjTakn5m6RvYKor4FtinhLPiBlOy jiQRRVZqZXiHJeAOGULQtUfRto8xxO/RSbUtLeKC1Gp+9SW1cYuXyZzU/RbZC34Lks7akESUBk+v 8J78AXndtABUwBnWSYfCox134rb+VMxghGVZmsyrrdCKx7NGDz9tjwkecPNb/Mc6yK8hiZw+7U+W +HsSHLqlfRYfZInkwqAGEqE7GB9EB98u2c3QCVjA8kx9AEPf7qIdS65jd0l/ur78/r7RE/SOTI3+ R3gtKoQbnkBeEtdhA58T/DNfXqD/gWgnAuWYfkNgwhh+93rFq+HHvly/6FNu8hdusastW2s/oIX3 Yh1qbWRJ3WXLeLzqdsOfufacOH0Bk7QmAp1vADi/29U2rVXT2y/F1To+htnZtzayGwBfln7iDt4f AnLn54v6NmKlIvfGQrfrNoG+mOxLbbPNrVtb2bSiLkG1F6e42XuCkc8gNPsycrTuxa0b2+KLtfRh y8PPbd5Wa2n8YChE/P0m+HyRvGGnxNQCmzNgcPIgklxUGIFp2AvzhIUAJgkt7WTTUlq4/OjERQtS s+GSTLZmSRGim6b6uLV2sioAUzU6drF62RBZBH/0oNeKvRDR6CnpX+jjhBRG7sl5Ij/hVsURWwOq 2UytJOMx5DyDQwYIfdlin6A/e41fVX66zNiz6jBrZZKf1HxgG4EmxmWXxv1eN0Ao9gAUYC+LhSrP 95hcWfaNCvJWA+Bpi2fUrA87jGSlWmSk7ukvavTZLamMbXK2yEATEmJMbKBmAvbLembP+hiFIaxr 4yZ9mvFpDblMLDqkH88ILH2eIBDSxT3hfl2kk2YrF3dtxRb8fja0ZicXm3dCwoJsw9uBuypFHwM5 TBKEqNx78rVXt/K0/iO15s2wJRRvQqycbZhtz8m2op0E/pC27skJiPeCKKy/I1LvQn9qgwfuULkk yA+etG+TbYfBAzuQPhVb0fZjzL3VrDfMOoArvLPVok3QJIZni41+0FDkmZMxRDQtVtB+j3FmcIAG f+euASEX7RfbOsuhwbCdgYs2EkgxDZuKNRO21kj0BEbira0VZBgY3curFNbUxPsF/xwQ448ThEKq 5Yg5k1k2yYhUKzU4QcPE5pmEIUz8hv0+Wy7IeSAzrR8TJ4caU96RBIvI+yDgz5KS7fj8NpqwMoJq bCOyNJdZTRA6zUj69hUMeDYMYDoYMBHEIJImq3CqVgAV5Hzj7yA6kxcx0C70HbkjaU5Sbi6xLMjL IicqIKGJ2G+N+C+EIAaticsy8o4zDvuETd6QgYj2Gl+Z8GwW2QI4GlMWw8wrn137agPIFSixLNAm mAzwoP9hQX0jtINH9J/H7uVy/mWYh6DqHjxIhdXQcdjVybEcSXdEbyukZaKGyQBhMyUgv5bzXNy0 IDPYN4Nvjb2So5PXMZ+ZB6VdF0WVmvcqEUO1UtaxWyTrNhks3zcrv8vwoL2ccyvxj1/uEaWEXfWX kO7uO18WsbtHz3binTQukUVHELecpuxrRrtYs9iwqRjwJi5EBpKAfsyjRssIK+Q21D6mjNauaVw1 01Rm7gzDPA0I/kDv2oKvwS0phPTakzAinoIzdQcOdTXynrx0hLXCUkEOSgaQL8EAW0UwVhOw9dkt vK9+o/1IoOzyQhQ0zQTspcHjZTGUg56u/U4HOiymJHReIRyLYFBCLjVYnNA3NUgsc3jrEahHt7r9 +/OZ2nxjc0xd1VHV6c6Qd6dKDqqxGp4W8wWJ5JfFKW2lzzvdiRyDWDtoVEsf7B0GhD7ztBFxzQU1 WHODU/PA1zjgE+u56jwg2BcYc9UDH0e+9eHALY7o/z59ZK9YWInLS0DoNy1WHN+WJg4yXKMX/ViC hiiXsGOVxMy1XpCaCT+4w9nZIGXbuonFZyyKTKqsEZhEcaLDRL9KREVVpMm7RmYwU7QtrcZQWGdo oU3bBaFWefApWiYHSCx3T4iYGuJEvINxHUyt2DTR2bgBmSk7GphexwR5pF7Ylgv9yZIjG4fX87f0 7+LQADRtM/+sYtqv01HmvyY8d72sia/qyxVmJs79pKZ3tS/nHNlFnDJo2plIv8r1EUCStiNwPzfb UZB02JfZjSPymHG47q3sUlE9TbLK23HIRRN3OSTYmEvXfnME354Hj943DvWum6Fvv/1wh0bce6IP yNSp/duaYwGrvY/W9j5a29sAOSJ3zc3Jp5OoJ8lvBZ54cAMW4XpcDRe5UuDRUsnXrfgIgybG30cb wNOZBwMSNtmcAkOTY7QocB4X0KZTYN+EBV08cECzzOThH3DajUc//KCUh32utj5qtb7fAtl+i01H Wt2VD/smq0KiOhHJtJ/qxOHB/ablg8c1Bms/1ciFf7uAgrv2v9vpxQpmAqrBClgBsEHV3xk+89W4 TBf8C663UYx1Orp3JzuOOsR2J0eP2Oa/yosSGoMkGrRVZUroi4J2kdR1Mp6J68QR/6qY+5sZGUwO x04Abxu0pphKL55LVnDMms+AEVC+KrBmMhTRRGLo7mZWEMtzeFnkqRwFs0JjkCW1mxfX6MjLGPn6 xpNNQwLG5M1HO43pXs34+FePpWEswHNCb1o4TNWajJhUDqGT8op8Aj8vPqk4RN3yqiwthIMfZ7ks BBaKJKSMSAAQPLIlG74ciOdBCUpzwM4OSpTzSNuJe9BWDnFqB0vkaVnMQ1xBDrDaIlTekcUdhaz0 IOuxSTb9HXmsbk9GNjWC/k17/mUyLVI5vThWe+Aen8ggOM+WPx7HQYBHq811IaH9A3vyro68Iuom 61xZmqwt2nqkFJkd2Z+mPmkuTmCQWCwUqnSiLFosyzEf4lTct1nFNs8IEcA8S7JBdvBElkS/+b9P Tjo7M5GsqwABg1CQOECW2gDpA5MCSQPskibi3o+WZMDN/PhNZxErOlWUaGuOefIGBkgyJrqsbpKF +u4TmHJFmV4RBDLZ3Kq1IFAN5gJ2FIyFNdsZE1sggkhw11gBKL3SabgdUKKgO1HQRVhkUJmJoXAz IrnDNNEGsX3SX9GPHFzYe/L8LTkZ+ZV/npPQIYVv2EkHvd/Qv92Vfo0iH8g6qa3t+txjgwQZ5crW 0btFLfjQKw7qmHFI0pGoj+MRfTPe927Syu8QxEgVR3a8BlDYqypxoDoAicxJSHJeiQQt2GknO2mp STpN8JdImYUdmWtlxrKOLVw9PrYstS1ZJKfPCJDAEWS3jsN4dZmACHf1fTjl7Yw84HjIEkGWekl7 xCprlvp8UExKghb9TYEdko82Ty0PQT0RMAORO7Xa40MNsaLnxJsSukzUUpHYE8ehMRKvHpjQIWhy sWUkXCM+KsCM1WAjtosNcZkLGqC/xvmePrwUpHKDXY1arUG3roCQQPjkpW8IIaQwt35F9v83sl/l XadkueIxXsQHrySCAjGQEwEnVuhh4Hapq0oNNbA2EtA6+tH9CHwZUgzIhl6sRYoQ1TXZEf5XCvpX oKBzQUYU04poadAE/EQf8KmaRWXE4EBus0j/7gCT4ibXuBgZosgToZ3yqDaCUkIY4Beh3YEsbLhZ DjKeWOXNk3o8s/OnKCqCpJ9/d/plKmKr7aYo31TiToTTUvYbMJTgPFCWHMQSpNYsoXL9IXlBZXJj Tnog3so36VTwTBaCQTzY2USf3GX1MAtJxpd3qbk/pEjCMuLY5vb237sRLogVRjVXOyC9xH5/Rfov gfRzzNISSYb8jE+pWaYorjRkSS986UZk0E4qGNzjbFlpxH1lkFl6NbtcN8hysdg0yM8gOFnjNs9y B72ttazIhSqZ0OZyQvUrsb2D2CIaw1ggM0n0uAexmRrp6K00v1O8dJVPmpvy4RDaCsYtSNlgvmsa FWQi1DiA/hXdvwC6v5ZgfBfr9OhnoT1C+YA732VQ8JY5glj5K7D2fpWlY6Ro+mmimF4uAF2fT3aU EOwsOFkdhM2pgfMp7z3huHV6tcSlFpYeSMNl+LgtnmhLCE/Til2dvCHUsC3pOTG2qv3iknNUiB6J Ts6mRkN4wxlR/irBEaYECzlKqs3Z6LWsXaGElpEv1p3rV3VS1pccOaphsIrE1pzltP4kwwUUr1nO MprLkqoOuduvQ5qcHJxpnHJTTicTB72fFxMiKLtrtE7nAEISrEzLEJFkw13/ZvBsIA6aUgYwitAx rL/0vZssANIxQcdXacJJXxNk1zDK5LwbzSPxsNqWBhGjeoIYnCB1XMxHFi8kBSdBwV0gh4P7zFEa s86X8xFSDacBqandlZLxi0AQa5qKpVTfFMalkWxrEvGitIvWEsR9QWAbo6hCts3Eh/P3pwO+B8KS qEURm+UC9rg+jXIj0q2Xol1a34FmvsjYZHWu4PUc7xdy9B3lDTVaA9DRONyEM54mSAPoOIiKTybC ODODezMjXsqQdqLEycSMlNCaZ5PkC8WG3DoIESEW83bcUaXf2WWNOmguUwxCl3oYOtEbC6116EEo uuPSii8HHDL1bxO+1MM6rZa4sjqRQQWQhTbFLY4s8+UHFxKMnnXnwO8gk6bffQllkRDynE/G4pIw NsZFSYBdFPlEj43i3dL/9E4Dvdgb3e6hG60HNCLsT6j6rOG3cKpnOGNp3vC1BNwbZFJbOVCL2Bq6 Q7obuBdkK/O5adDHcVvBe20ZuB8aW18SDH8uP6PLPXAUYgO8LWNnoWVyH/JJMOROV/mYEWzgbDIu FAjiJdkrVp2aj+Fp1ZP2JMZ+0YWpblsWtchwrJZzL1eGvEsnOC/EJd0uL+PPGAuJG6UsGSpSZVlS MjPy7ByZsh0E9EYzqxlQ88Uo1hrIv+OrVdktH6/6mMCIbPgUlq3cb0CeWVG8UbJO6w9NLafMKz+T XrTTPSkmvutAA+FoeuL6JoQJdcyDyPLa+cCbPefJ1yUcFdkkzuRpXuTvkHA8oAJCBsnvhkZiJ9Oc 4L1cNKrhzqOGLoFGdIm+F//9wn3yx/NXX33pPvufTwIJwthlGkwm1wk5TleBVaYJ30tP6nDLsFou FkWJ5EJfWTxbbSTTsI1x2sRi555lJxgbiIb8RSBhkuLAlubgTbLDItvPcQu6VKEQyGJGsLNYI6yF FLepIzjIwT0vXUPNL8PBnKhVOR2VYUPo8o2/dX9leZ63lbTwOBYLjavN2vgBpF42dzd9ZCekTdjE GkvudmaGo7XKi9qWSJN8KCvuj4DxZ7drAtk0aWVBo2hZsWkZZWwxrghu3XHADtsKOQ0ecZJnMeW8 QINBZLhK282xI9qULVqZR1eqk9zBQHxwi6knlv5eDcTfjviJRuuYCRHDdOHWSGyDhPryddfsN5Jj C4WnY9kTrsDKrUmXxDz0oZB+yqUfPgjONX20i3LMzsAPbPteMa5ZFiWZAeUVQNOs5M4D+kAHalV0 IkrvF+pf5em3y5UbAJZvK+Dpb3388uOtOE9LcaVpq5p8wBng7ajMUsqZvMNdhiewzNkLEPum4tsT LTCYKj1NsvEyA5O0u5rVww5oCFwh54O9U75kaWJfwhzUlRgtRVCKRrqV3LiVeVXazT8U1X+Fxd/X XFE7RV4PDXV36Gaz6xLUZFG53Xirri/bvQu4kigmTtrdMMYAcpf1Xw3GZ7aHd8LZul4jJzsGNmi/ p6lGQqMkPtLGXWA/AJnjQYbfj5GuhxtYKZyYod/fjv7+s5irhXtV2VGQ5h1o3/rrq/MttgduCsbg vyven3EQAoVJ/nG8u97/I9oP/3m035/jCWd7E4WYlpUI+CDNiohAGQ4fzlU77raD2Muc3bxVDAy5 rsG/E/F8gS2vsUaYLppcipaLvnohiKyJ6TQdpwSCYxQ8EWC7Bs5izJPt4BdmoxWaveyzaUMxjQxr qFpI9G7fTHywcbBp/dtFgpDDwOJM3xRIj9zglWGHaa6JHdM1hqYMx+bWbiu7sHH+gkdBYJLNcMgh tgM4QsAjVHZ7u2WZ2hA3cJa2bAt4zaXAtj4UDfyJYAM+XiEDsSZpWWaXmvsQu1A4AeKlUm++j0L8 i1ip0E659MeCdt6qnOnwxSjL9L7T9gxLU+NT1iIz3MPksvI65GGCAACcRlLAkP2AEP2nTXysVMh2 LXPqOHGgDWUbiQuRrky0p2JRh7gfakj+sBcunPnL8HyE6o6PofvcgG6cA3BkMNyyY9RLVjcu5fGF AOXa+2H5Prg9pzFf0Rx35q2GRutzV3Fu4ifxqhO5EUyaVuQEuahZchtlKNvSxVetWM7MkzzBEXcS Sv1xfxqd2w7Am6EQmUsw0DSparmlWTTxDYnLIpySNudY8Lhpf2llR2lyRNMTz/rPwNCawEr3AD+s nzfdOckRQRCGdnWKiCK2yvvgtikXPZwgbb/HsaAo9oVLp1MoXPo7YPGGWuA0UV0CDvCU8S1ql4ao 0mq6wdOpVMNK6kGwgda0JEbh5xjIwE1/yA13jmd6O+ansQBpUiVXs2hzdnTGWB4ajAz4KzHyZhKF 3dbl58dng8vz47M1eiFtRVXtOiKnJ/DpVZMaAaRzvdN29JJBK3UOSr14xIfcds1c6CPc5ecdLdlR y30FS/pULirwHVeO8kjYLrW8DVZrFYpboQrjrdNCjpVRIWsYrTyR5vGZYYk7HNxc4CsnTlIsYEJ2 Zd3EOX2LBp/rAR1SOiu/evuW2ZzvxJ7YdSpcd3la9bcWvqCef4P6O74Y0Ivjs79bVay452xT18vP D+/uzRA8Qd7I8HP6kwthzQZSv4OaDYdDbtLcv+EilHjFj1FDA6U4eFNxEawV8cWjd/QJOFPSBoyz 75JtPIIKams/4DHW55OsCrsRKipWjO5g45ITxlUgVlUyF0NTM07ui01J06iE89Dk8Bmu5LhZi+EZ 72MkPrthepN5Q5E1TgIZJVnCVd+SJYn+pNbqqDF1J+6qKCaOhUaZTPxeMZ2GO3A8phwpg7WaoKNM FgvArgwxVjamN2M25ZI4otE1ZCdRbhNbqydoaotJPnaoDwJ5loeD6QJlCUZSzBIjcY9JuLOoUGFW wimDQlxXg0OCsVw1xRR3VJIzikMK1CZGuvyszQ0r7MAYFnb4TFbcjzgBb9dwAj8OV6nWsINrE9b7 PaUK67yDdazNfa0C2E4Rn3AWQjtGCr+0YwcZfbPXriFxzlVBbAxFWYL+hBQ1muoSp9a4YhOAZ1Ud EJL4eVmit00KYKzo8EcLQ5I9KOk+cn3T4MVFM5ja6pL5DWtE9k95y1zrkxLkOPOa1Sy/tRyM6InI /OhLPDqqG2OHxrxBGj1X6PH2NFtgGtbwjfLiHOe8wbBdk6PxS6tW3nu4svCvqVkj69SoBnlcJtK6 LMJ2aThzOdpjuY9xRjUqZATptoHRXgke+/fVV9a+o7LuwX5R2qpWZ4mzV9/BhjqG3NcmmML4nBPN CAZiYkQOAUaKs2TF+cJV/i1al+a9ikkeyGLRzaRF1hEGstUqB/LftjM1giM6kGRHOyZLNuXiWsas VbmBRZpWcjSxWEdzK7xMrYnIK1yljbyPqgmi0Hq/Nn7fEnNyC04trbIWEgX9kt8KYyWwneotvnMe qKqJooHs+TI537OfvJujjcHuw10iPe/DYHdxF0bZyGBrI/2M0PVmXOchZA/ZZz2n/zgOILC9k2ts DkseDvbdwIYMo3RKQUU0YuZJcyMFsIzDHX3yo/YOWxkk02UuyXKdMotEEcwKgRfZoGnlqo8091vT K0J6wRqvtFM0CSUSMeir6er1nbJYCaNLuV6yFPg/J24bbfT0RCEp77REGOfUSfkse65pxJElsL+P gDMqNdSxyuP0aE54AIO2l23tL7jpmptHYtfL/W/900qdR1i3RbYG20Zn6add3O/cgTt2Px20K1+1 cI3/y/3bWsoHbrqOFmVVPJN8claEVk2KFyACB+7NHqtWFZl7rL+bFNwEpriKhyS7IpzXszniqyZr hvjqg6QFxBZDq5hWALZkILDAMLprInIkEGrJXgcd89ZUnOt+Qu2lUI5NSnVITK0YE+7FfotLfGmt tynu8KwSJiC89l6RXhJrrjtyWW8AsSjegFx6UslIk9hDO8EWteMX6le0wntN9QPt+wQFe6w+zzmn ai5uiep5CY+s2EOWfHcrZYJQZ2EnVOY4dlwgQSot9nRVcvTTLo8hs4GLrAjsGprkGozS0o6TolpM LCdJfPN/T3jG4amoTF9anbBxfE+TGl/wVU3tFdWQbeCgjT4KcOBBuMyolrrutSonfCOVE76xygko nPBNUx0DTdJH+kP/I32bigsyZacQT6iL9ptxqKGw7Way+otuwTrJzcjFaABLplW1JBu095tQrXrv CSHA0PzAfTNwMy5wdzbBaR1X497p/cbR09OMMMA7liLdYfk82MzKrqHIpBTxZmDxiaLUxgTLHGCs EemXN9b9x57BZMrbP1m/cVqEclazVhnix97GKm5CM1q7jTgUheBNFOV8NSSyrE1+QbrcwcnMpVwy Xn7jjpOE/cQiIesWvUXDhb7QZJycQO3Y01ggDCtMU3Eh21XG3yjSu5dJ+eqHw0cwaIOuUyOVjCOy tvoZasECJcvALDwu+s5R6woN6N2TJ+5Q4NqwoGF2Hm6kNmVr9ve5Agyeuieuz5WVd/42/zsj/pkf 45tTh8ckXNknkA8E8HdcFOtY91wm5K9oyNOlPY0Ru4yUJD9oQi1x4qBd3Cw4JC3+KYIXCR/stkH8 ggv/bAAxa8tdKfPTPftodjZNYVlzaUBSNXNxeESXYEG0BD6U0ZDUUAewcV7PrE5T7naRYOnnPrvd lWpNt8KvUFCaaZyL/cj2uQzABZjKtCIak2gGEaZZChgXiqW6JcBjRFKEF5YfYwNolWqtqrSVgANH W9bVL1NSK3z/iuyvEV4mW5onYSOkcnSEYnGTYjmq2SaryNp287RCJqvqhFnzTSDaA0kTnCXIECME 0iTB2I6P6ow/KtSFFk50Kju0FVBEdNA+U+NUEAIECZSwWMTFrAiR2ekb7AOVB5mf1vOiWasMyAaC AB/ov4LXJqsR4z5KUGe+J9cyY/dfBjH7lNs2OLR8gI9OBu7xwD2h/38i5PzkpAsMfGpGOVO/NiXf R2IAM0DNYdHQSjRNQF0ebVrFIAcxC75j6bZ0s4+LxRPd+BaRE6nTt0O1BhpDmNQXl/YJqlUKAg2g MjFZVvCtTWakf1xq0TxqOm+UWiaNdtHYqhVhqqjzWpHTec/CJ1Tq+igULdzVWl3LpvQcx9jM7pIY Ump3G1gHzVIpEf4WTjqLq153M0szfJZgs8Yu+uEHkUdYQbQllYoHzSJIwv2JWOMzMA95ojf4ihlQ /OzZn9zTRQmBddB8GSY7AmHA/KzUgHZ8UrU03BwZco42YecooOdoE37QBG/XYuikg6Kjd+FoTYOW hjgiI7JRE8Yd4ZYf8fLbdFxclclipmH+PgvKg+HwJTmhw7Z2ML237vC9pX+jr0/wS/2EwgqWOGCV dKHJT0m0at+Vdyn37ZqHNM72tvw1ispyYS1SsT7dab6gYE86FXBDw8edduTnEWEcuweHcV2SnJUD LVW8QF3F76gV/TKz633+Eyuu+WKTyBp1G3GykdaRyou+fxSCWZw7FH1KSaWVFMNd+Y6DFECEZ2Tp RNF90SREDvhzUTLQnd/p4ZicRTwSjaHpaQIPE65lSJB7w5eDgl1CCuWZt2LZmovEpxW8FFKok3DZ PQQ+yvjbSKKnaJj/KZYShQYNS1l8HkyHEoNZ5hUnnMzmsZewV2mn6ly9R78TmkTQF/s3ta+lsRWE DyxFv2v+bpPMxmbK3Cc5nzNVWmiWhFd+tTN0hjio/mXFJ3scDYGyVyiHU5yAT1G2EwUGcroE3xwy h9+/TxK83gtLlMsAtsABwK+rrsieKQnJkk5T+yu2M6acuifgJ+PH48BMwgCo+t+scI1xZjfVSSTp 9mVNRnmdC8ocRhTCqfm7WVHmooXaLfMmRDbdU03mKf3VEncnmjqYKB8Yjh3lSj2IlAtilss8F5N1 6vRLZpr/4BF8mSoJuz4WJgUTR8s0Q/hHj2TEWWZMIFKNaorsB2l1gw43C1Hbms/jIvBJKE09aPEo AVKXxOUjsaYcs5KxCcwFAJy3vwjGJhIHNZXVJjEbuej7W+rX5UrCUqyeFbfm0BhHcqQAkLa5JR6l BSHqmR6Ftu7A4vMmQ14cEbmJIFiZzWfNdiRfxdJ5ZJBEshJbrFxJnJdva11FyJH8OlgZpNzg9vCk BpczkEEOxTfoDMfzSl2F6MKNUCg7HnrxTGLpPFiW1mRniLHDlkNCXJtwOZqkhJeM5nM7BKwQa4Mb FFFwynfPeDDaQeXnCU7sqmNjHLk3HK9SLe5KIhp5zWCQ0rJl9wNr+jUOLVUQ2rPbTuyqsi7iUT6+ k9F4pIkn092Fi+yRVMdPJk7OPq6b5Rihv89/+jVfoo5eT/7Ln3QCjfPHGY8bZiILHH6BeLX8bawf V/vE3q7Gtt81igvy68TJp3AevWtkC5zTf3/++NypNcVPzY7R/NzDf/T9S8Eit3WuZ5uyXtGo63b9 fWPbRC1XtrHjmi3vw/NqmF4uVEca1lT+8L5racHJdcKvl9EZggUfw9fm9CWeb4cBxApfM4Ka5DaG c/HY9lkgcg/CUGJco2UDpejDWYADpyy2L9mDh7QCA3GL3tZmeFjRZQbGKV8+oD/fAyzG609jMMgj cbYRTJEDUi41tCjTa02vR6Ohjtya7Y7JzL+lpc4kcrwbn/U0DTvh4XVldXmE4foQL7tT3SivtG/F eMX/48wwCc+2UK7R2A0479CHRX89f94k6vpjw4uBVR6sIeQkPn+0hyNlNe1PYKedJBznB2U0tDBq ff+B3pkIMPwq9TTCw8a96JRd3PzZP+d472f5+Nw+DWfPzj2yuPqH9tDO8cL8WovxQERBi6BNIDWN GxETHSMU6hw2NNV92eFTgdGlYuyZt1V/aFfLqMAsh89ghMhb+27TKKnsE7iwYJqvmVvf4GaRFcWh 5onnOhA52yZTtWvLQr0izRSzXkOz+/VmTCKXD9AS00kfjquTyZGU7GPlt1EwkF0/GQX+jlYcLefs 94ekKDs+tk+Pt3JvyDkq4K7cIlNXdbtGIjXng2/5qLljgGnu6Nd83KYf7taqRnrWIOPoN8INUKl9 kZcPTJIJUTUEVb/zafgAR46Q0jDPLXJMjlRV3YaTN94Np8LrTLx38nQyDltGqVo9DS1KlLWSu/rs SeZc/YKXeU2GUsIZLOqewKSjdkgne4N0bJiCWTri2KD2GccfiCb2jL5fDSnYaijfU3iKzyNTS5AR vu5TrTZ89jWBMW4n18ykIUu6fcR/5GJ/yt7Ila9dmbIfw7XsUjWFmYA14whpZHAx3qY1TBZjcKNn fEaM/NTlwsxo/hQgWqrW5QyOaOEidDsb4RhyMva9lT6yiahTtKv1vRpgaq8OdEMvkuC/7f127Z5I C8h3KC3c9eLL068uXr/64vTVF18+PX/uxmPSRr/l8V2rJYJx40e/lcF7rUGjH6QlZnAniGcb5dB8 hEpsw0v2gfrciPROe0Evv3rxovnQQmuPISbb3nkj/mMI9Vz7iX4UHTlpDKmrMhmJt8ApvQT7NDPf tyLHwFs8OMZos4IYzc0CIpp+1OpvyG0PEFDejNAQe/hCjnx3BF97oZYMRvlCLOH2JFoIH+6uhG33 9rghn+m3zgIFK4vVIcgypB7hxBfx7kVNbuJ3kpUiZ2yMZfiqe9HXw0KoWzbe7+8u3P+6Xf0o17b7 6eDt0YF++2V7m3fU+X7Xov3trnB2zPa+GsK0m8fNZtjOElAQB8jLCLYxUMjUwsZ07A5q4q231hBW Efal8Fw/S3Ps33pvn+pE3+Y7na4F+/Vt9Mi8O5osdRAM/M5A9lpgLyDscdyl6j84DN7WJh42o84e tb9sTOuTD3jBCkXuADGx8rQ8fk1/89M2a4c+7cc7XcMp5o01ptMmrD2xdI3Nu/yptU3zLqn13mHb zGu6rPPkVsCCcWR3zdzxyIFdW2BDG4GbvRDA8UpEq0U5ds1tTOQa6Ap6MTO31i2fyosOLjoAFOHV 5LhYwsUGYaBJFk8nnNDAIZHgBnPSSxO7FWPJgsVsFbUtmuaAGN8XC1aahDczCW3ur0EFwxM+FaJK l+v3xq/AwHv8ha3w84GioSuRQ96Kas4VrUF+L6zwfv3wMv30aHunzYnPZV8KpoGuDIgNJ1ndtT22 I9LwnVR8EaUsWgnuBeAqn0mu3BXiZr2PaenfksPx+fPTP/PAfJ4f7ByOe+HjUU0SPWebGKXQKPDY OYvVjCvZk+zodVEnWdV85mctFKKWtlcRlo0H+09qXN3YiMtxynm7HFCU8sEdizTz4YZL+P4ehuMw novwK6tsifK1Bp3Ixo9RNG76/h2t/wNQSwMEFAAAAAgAKXKaLcbvLouHAAAAqQEAABUAAABjNF9j YXNlU3RyaW5nUHJvcC5jcHDT11dINolPTixODS4pysxLDyjKL9BLLiiwUsjMLchJzU3NK0ksyczP U8hPUyjJSMVUrJCck1hcrMfLpa8PwtQAvFy8XMqZeck5pSmpCkrFJSmJaRV6GUoowja52SZ6GXZo IsUlRWBBZP2Y3oMYRdgdxACQOQrO+XlAm0uTQQGl75IKZ1MzQABQSwMEFAAAAAgAqnKaLeE8jVKC AQAApgMAABMAAABjNF9jYXNlU3RyaW5nUHJvcC5otZLbToNAEIavS8I7jDExYEwplB6kiUld8JBo bYoa7yYr3bYkZSHLtok2+uwuVGoVLxXCxc7M//HP7FgWRC5GNGehFDGfj0WaNRcexFwyMaMRg1kq QC5YvQyiJc3zpq5ZVvH9xaNrunYYz+BgymYxZ1NjePGExEUyDIPwfnI9uhxP7sZ4hUh8x+227QA7 JOig2+/6eNojNrb7Qc/t9N2u3z5FvB6Rmwc/8NFU3C0T/gpZecXbkOBjMIEzsFutlgpmgs4TCimP mDoxPlVVllUvLN5yhr/M1oNs9byMoyL1Fda1ja5tE56uNWoyI0p5LiFaUAHHnCbMVKBvBKOMKq0C NRohk2SVyzQhaZJRwYwa0fP89DNpDpTkrTDdWMdCrugS3usOTNjsynJJpWpB7RLsKJVFF89fJMuP 4NnGE6gFHaxMwlcDsLbB2Ovx2CzUTZKqZeUyN8yT8hzGr8wwS7vfxE5N7PwQO/tigBIgmFwJrn7d rDpYO9Uo3gbbW9y/5v/Y3Q9QSwECFAAUAAAACACxcJotWfw0nXsAAAC5AAAAEAAAAAAAAAABACAA toEAAAAAaGFuZGxlci5jcHAuZGlmZlBLAQIUABQAAAAIALxwmi2WTUZqvAAAAGcBAAANAAAAAAAA AAEAIAC2gakAAAB2aWV3LmNwcC5kaWZmUEsBAhQAFAAAAAgA2XCaLX92mCqqAAAADQEAAAwAAAAA AAAAAQAgALaBkAEAAG1rNC5pbmwuZGlmZlBLAQIUABQAAAAIAM9wmi3OyOy3IQEAAE8CAAAKAAAA AAAAAAEAIAC2gWQCAABtazQuaC5kaWZmUEsBAhQAFAAAAAgAFWeaLZ6Op2C6EAAAIVIAAAcAAAAA AAAAAQAgALaBrQMAAG1rNC5pbmxQSwECFAAUAAAACACbaJot8Z3qe30eAAAOfgAABQAAAAAAAAAB ACAAtoGMFAAAbWs0LmhQSwECFAAUAAAACAAkZ5otth9a6d0OAADULQAACwAAAAAAAAABACAAtoEs MwAAaGFuZGxlci5jcHBQSwECFAAUAAAACADyaJotW4y/tIMpAACsjQAACAAAAAAAAAABACAAtoEy QgAAdmlldy5jcHBQSwECFAAUAAAACAApcpotxu8ui4cAAACpAQAAFQAAAAAAAAABACAAtoHbawAA YzRfY2FzZVN0cmluZ1Byb3AuY3BwUEsBAhQAFAAAAAgAqnKaLeE8jVKCAQAApgMAABMAAAAAAAAA AQAgALaBlWwAAGM0X2Nhc2VTdHJpbmdQcm9wLmhQSwUGAAAAAAoACgBGAgAASG4AAAAA ------=_NextPart_000_0030_01C2ACF4.B8FC5EE0-- From graute@pueschner.com Fri Dec 27 13:33:17 2002 Received: from mail01d.rapidsite.net (mail01d.rapidsite.net [207.158.192.52]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gBRJXHl11998 for ; Fri, 27 Dec 2002 13:33:17 -0600 Received: from www.pueschner.com (207.158.216.170) by mail01d.rapidsite.net (RS ver 1.0.63s) with SMTP id 023644 for ; Fri, 27 Dec 2002 14:34:30 -0500 (EST) X-KENId: 00003B6AKEN15292CB1 X-KENRelayed: 00003B6AKEN15292CB1@eserver Received: from 192.168.211.1:2340 by KEN! (192.168.211.11:25) with SMTP ; Fri, 27 Dec 2002 20:33:21 +0100 Received: by PSERVER with Internet Mail Service (5.5.2448.0) id ; Fri, 27 Dec 2002 20:35:48 -0000 Date: Fri, 27 Dec 2002 20:35:48 -0000 From: Ralf Graute To: "'metakit@equi4.com'" Message-Id: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2ADE7.8A03FE90" X-Mailer: Internet Mail Service (5.5.2448.0) X-Loop-Detect: 1 Subject: [Metakit] properties with identical names but different types in different views (tcl) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2ADE7.8A03FE90 Content-Type: text/plain; charset="iso-8859-1" Hi all, I have a question concerning metakit in tcl: Is it possible to use properties with identical names but different property types in two different views? When I tried to do this with the following script, I got an error. While adding data to the second view, metakit checked the data for the type of the property with the same name in the first view. The test run on Win2000 with the ActiveState tcl distribution 8.4.1 ------------- puts [info patchlevel] puts [package require Mk4tcl] proc test {} { mk::file open datafile # create view view1 with two float properties mk::view layout datafile.view1 {property1:F property2:F} # append to 1st view mk::row append datafile.view1 property1 2.3 property2 6.5 # create view view2 with two string properties mk::view layout datafile.view2 {property1:S property2:S} # 1st try if {[catch {mk::row append datafile.view2 property1 moin}]} { puts $::errorInfo } # 2nd try set cur2 [mk::row append datafile.view2] if {[catch {mk::set $cur2 property1 moin}]} { puts $::errorInfo } } test ------------- The output: 8.4.1 2.4.7 expected floating-point number but got "moin" while executing "mk::row append datafile.view2 property1 moin" expected floating-point number but got "moin" while executing "mk::set $cur2 property1 moin" Thanks --rg ------_=_NextPart_001_01C2ADE7.8A03FE90 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable properties with identical names but different types in different = views (tcl)

Hi all,
I have a question concerning metakit = in tcl:

Is it possible to use properties with = identical names but different property types in two different = views?
When I tried to do this with the = following script, I got an error. While adding data to the second view, = metakit checked the data for the type of the property with the same = name in the first view.

The test run on Win2000 with the = ActiveState tcl distribution 8.4.1

-------------
puts [info patchlevel]
puts [package require = Mk4tcl]

proc test {} {
  mk::file open = datafile

  # create view view1 with = two float properties
  mk::view layout = datafile.view1 {property1:F property2:F}

  # append to 1st = view
  mk::row append = datafile.view1 property1 2.3 property2 6.5


  # create view view2 with = two string properties
  mk::view layout = datafile.view2 {property1:S property2:S}

  # 1st try
  if {[catch  = {mk::row append datafile.view2 property1 moin}]} {
    puts = $::errorInfo
  }

  # 2nd try
  set cur2 [mk::row append = datafile.view2]
  if {[catch {mk::set = $cur2 property1 moin}]} {
    puts = $::errorInfo
  }
}

test
-------------
The output:

8.4.1
2.4.7
expected floating-point number = but got "moin"
    while = executing
"mk::row append = datafile.view2 property1 moin"
expected floating-point number = but got "moin"
    while = executing
"mk::set $cur2 property1 = moin"

Thanks
--rg


------_=_NextPart_001_01C2ADE7.8A03FE90-- From graute@pueschner.com Sat Dec 28 04:24:08 2002 Received: from mail01d.rapidsite.net (mail01d.rapidsite.net [207.158.192.52]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id gBSAO8l32392 for ; Sat, 28 Dec 2002 04:24:08 -0600 Received: from www.pueschner.com (207.158.216.170) by mail01d.rapidsite.net (RS ver 1.0.63s) with SMTP id 081454 for ; Sat, 28 Dec 2002 05:25:21 -0500 (EST) X-KENId: 00007B93KEN03308309 X-KENRelayed: 00007B93KEN03308309@eserver Received: from 192.168.211.1:2539 by KEN! (192.168.211.11:25) with SMTP ; Sat, 28 Dec 2002 11:28:33 +0100 Received: by PSERVER with Internet Mail Service (5.5.2448.0) id ; Sat, 28 Dec 2002 11:31:02 -0000 Date: Sat, 28 Dec 2002 11:30:55 -0000 From: Ralf Graute To: "'metakit@equi4.com'" Message-Id: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2AE64.99D4F1E0" X-Mailer: Internet Mail Service (5.5.2448.0) X-Loop-Detect: 1 Subject: [Metakit] properties with identical names but different types in different views (tcl) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2AE64.99D4F1E0 Content-Type: text/plain; charset="iso-8859-1" Hi all, I have a question concerning metakit in tcl: Is it possible to use properties with identical names but different property types in two different views? When I tried to do this with the following script, I got an error. While adding data to the second view, metakit checked the data for the type of the property with the same name in the first view. The test run on Win2000 with the ActiveState tcl distribution 8.4.1 ------------- puts [info patchlevel] puts [package require Mk4tcl] proc test {} { mk::file open datafile # create view view1 with two float properties mk::view layout datafile.view1 {property1:F property2:F} # append to 1st view mk::row append datafile.view1 property1 2.3 property2 6.5 # create view view2 with two string properties mk::view layout datafile.view2 {property1:S property2:S} # 1st try if {[catch {mk::row append datafile.view2 property1 moin}]} { puts $::errorInfo } # 2nd try set cur2 [mk::row append datafile.view2] if {[catch {mk::set $cur2 property1 moin}]} { puts $::errorInfo } } test ------------- The output: 8.4.1 2.4.7 expected floating-point number but got "moin" while executing "mk::row append datafile.view2 property1 moin" expected floating-point number but got "moin" while executing "mk::set $cur2 property1 moin" Thanks --rg ------_=_NextPart_001_01C2AE64.99D4F1E0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable properties with identical names but different types in different = views (tcl)

Hi all,
I have a question concerning metakit = in tcl:

Is it possible to use properties with = identical names but different property types in two different = views?
When I tried to do this with the = following script, I got an error. While adding data to the second view, = metakit checked the data for the type of the property with the same = name in the first view.

The test run on Win2000 with the = ActiveState tcl distribution 8.4.1

-------------
puts [info patchlevel]
puts [package require = Mk4tcl]

proc test {} {
  mk::file open = datafile

  # create view view1 with = two float properties
  mk::view layout = datafile.view1 {property1:F property2:F}

  # append to 1st = view
  mk::row append = datafile.view1 property1 2.3 property2 6.5


  # create view view2 with = two string properties
  mk::view layout = datafile.view2 {property1:S property2:S}

  # 1st try
  if {[catch  = {mk::row append datafile.view2 property1 moin}]} {
    puts = $::errorInfo
  }

  # 2nd try
  set cur2 [mk::row append = datafile.view2]
  if {[catch {mk::set = $cur2 property1 moin}]} {
    puts = $::errorInfo
  }
}

test
-------------
The output:

8.4.1
2.4.7
expected floating-point number = but got "moin"
    while = executing
"mk::row append = datafile.view2 property1 moin"
expected floating-point number = but got "moin"
    while = executing
"mk::set $cur2 property1 = moin"

Thanks
--rg



------_=_NextPart_001_01C2AE64.99D4F1E0-- From jcw@equi4.com Sat Dec 28 04:43:18 2002 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id gBSAhHl32663 for ; Sat, 28 Dec 2002 04:43:17 -0600 Date: Sat, 28 Dec 2002 11:44:48 +0100 Subject: Re: [Metakit] properties with identical names but different types in different views (tcl) Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: MetaKit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <62A2F012-1A51-11D7-B702-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ralf Graute wrote: > I have a question concerning metakit in tcl: Yes, I know. You sent it yesterday. Do you really have to send it again in less than 24 hours to over 200 people, just because no one answered instantly in what is no doubt vacation time for many of us? I'll get to it, if no one else will. But I'm not going to let my priorities be dictated by someone who ask a question which can be answered from looking at MK's source code (yes, I know it's C++ and perhaps not easy to do for you - as I said, I'll get to it). -jcw From jyl@e4graph.com Sun Jan 5 10:22:53 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h05GMrc17567 for ; Sun, 5 Jan 2003 10:22:53 -0600 Received: (qmail 30728 invoked by uid 48); 5 Jan 2003 16:24:26 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl@e4graph.com) by mod3.net with HTTP; Sun, 5 Jan 2003 08:24:26 -0800 (PST) Message-ID: <64933.209.233.24.218.1041783866.squirrel@mod3.net> Date: Sun, 5 Jan 2003 08:24:26 -0800 (PST) From: To: X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] [ANNOUNCE] e4Graph 1.0a7 released Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am pleased to announce the 1.0a7 release of e4Graph, the seventh Alpha release. WHAT IS IT: e4Graph is a C++ library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. The e4Graph package also provides bindings in several other languages, currently Tcl and Java, and allows input/output of object graphs via an XML representation. The e4Graph package is built on top of Metakit 2.4.8 and also depends on Expat 1.95.5. WHERE TO GET: Downloads: http://sourceforge.net/projects/e4graph/ Homepage: http://www.e4graph.com/e4graph/ Changelog: http://www.e4graph.com/e4graph/changes.txt Installation: http://www.e4graph.com/e4graph/e4install.html From mszak50@hotmail.com Mon Jan 6 14:51:18 2003 Received: from hotmail.com (f109.law15.hotmail.com [64.4.23.109]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h06KpHc22869 for ; Mon, 6 Jan 2003 14:51:18 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Mon, 6 Jan 2003 12:52:04 -0800 Received: from 24.62.101.121 by lw15fd.law15.hotmail.msn.com with HTTP; Mon, 06 Jan 2003 20:52:04 GMT X-Originating-IP: [24.62.101.121] From: "Michael Skrzypczak" To: metakit@equi4.com Date: Mon, 06 Jan 2003 15:52:04 -0500 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 06 Jan 2003 20:52:04.0476 (UTC) FILETIME=[77E953C0:01C2B5C5] Subject: [Metakit] I'm clueless - using a "Select" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm trying to figure out how to use the C++ interface to metakit to do retrieve a set of rows that match multiple criteria. I've looked through the documentation I can find - including the TCP primer, and it looks like I should use the .Select call off of the c4_View object. The select call takes a const c4_RowRef & object. How do I create this object? For example, if I have a view with "lastName" and "zipCode" as properties...and I want to find the set of last names with a specific zip code, how do I do this? A code snippet would be wonderful if anyone could point me to one! Tia, Mike Skrzypzcak _________________________________________________________________ The new MSN 8 is here: Try it free* for 2 months http://join.msn.com/?page=dept/dialup From jeff@k2.com Mon Jan 6 15:07:59 2003 Received: from kobalt.k2.com (IDENT:root@dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h06L7wc23917 for ; Mon, 6 Jan 2003 15:07:58 -0600 Received: from Klee (dsl093-106-226.wdc1.dsl.speakeasy.net [66.93.106.226]) by kobalt-eth1.k2.com (8.10.2/8.10.2) with ESMTP id h06L9WG07990; Mon, 6 Jan 2003 16:09:32 -0500 From: "Jeffrey Kay" To: "'Michael Skrzypczak'" , Subject: RE: [Metakit] I'm clueless - using a "Select" Date: Mon, 6 Jan 2003 16:09:30 -0500 Message-ID: <000901c2b5c7$e8ba7b90$aba8a8c0@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mike -- Basically what you want to do is this: c4_StringProp pZip("zipCode"); c4_StringProp pLastName("lastName"); c4_Storage store("mystore.db",true); c4_View v = store.GetAs("addresses[zipCode:S,lastName:S]"); c4_View vs = v.Select(pZip["20005"] + pLastName["Skrzpzcak"]); This will create a view vs that will have only records that match the values "20005" and "Skrzpzcak". Keep in mind that this is a virtual view, so you can't remove records from it. If you want to remove the records, you need to do something like v.RemoveAt(v.GetIndexOf(vs[0])) to dereference the first record of vs into the base view v. Hope this helps. jeffrey kay personal weblog pgp key "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of Michael Skrzypczak > Sent: Monday, January 06, 2003 3:52 PM > To: metakit@equi4.com > Subject: [Metakit] I'm clueless - using a "Select" > > > > I'm trying to figure out how to use the C++ interface to > metakit to do > retrieve a set of rows that match multiple criteria. > > I've looked through the documentation I can find - including > the TCP primer, > and it looks like I should use the .Select call off of the > c4_View object. > > The select call takes a const c4_RowRef & object. How do I > create this > object? > > For example, if I have a view with "lastName" and "zipCode" as > properties...and I want to find the set of last names with a > specific zip > code, how do I do this? > > A code snippet would be wonderful if anyone could point me to one! > > Tia, > Mike Skrzypzcak > > > > > > > > > _________________________________________________________________ > The new MSN 8 is here: Try it free* for 2 months > http://join.msn.com/?page=dept/dialup > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From rcohen@dial.oleane.com Mon Jan 6 16:48:33 2003 Received: from mel-rto2.wanadoo.fr (smtp-out-2.wanadoo.fr [193.252.19.254]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h06MmXc29071 for ; Mon, 6 Jan 2003 16:48:33 -0600 Received: from mel-rta10.wanadoo.fr (193.252.19.193) by mel-rto2.wanadoo.fr (6.7.015) id 3E0C3370004F8682 for metakit@equi4.com; Mon, 6 Jan 2003 23:50:03 +0100 Received: from dial.oleane.com (81.49.70.81) by mel-rta10.wanadoo.fr (6.7.015) id 3E075B5C005A3319 for metakit@equi4.com; Mon, 6 Jan 2003 23:50:03 +0100 Message-ID: <3E1A15F5.CFB64785@dial.oleane.com> Date: Mon, 06 Jan 2003 23:49:09 +0000 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.79 [en] (WinNT; U) X-Accept-Language: en-US,fr,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Memory problems on macos9 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I dont know if some of you use macos9 compilers, in particular oldish code warrior 10 which is my case. I had last year some memory problems, and found the same kind of problems in my code also. Some frequent allocations may produce a HUGE bloc to be allocated. For instance I ask 1000 times 1000 bytes, and instead of allocating 1Mb, the application allocates 100 Mb ! The solution is to replace "malloc" and "free" by "NewPtr" and "FreePtr". The only problem is "realloc" that does not exist, but can be easily coded. I beleive that the bug is in old code warrior implementation of "malloc". But not sure. Hope this helps. -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From canis@users.sourceforge.net Sat Jan 11 11:16:11 2003 Received: from hive.gotdns.org (pc3-glfd1-5-cust137.glfd.cable.ntl.com [213.107.72.137]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0BHG9c29419 for ; Sat, 11 Jan 2003 11:16:10 -0600 Received: from hive.gotdns.org (canis@localhost.my.domain [IPv6:::1]) by hive.gotdns.org (8.12.6/8.12.6) with ESMTP id h0BHHBKV031093 for ; Sat, 11 Jan 2003 17:17:12 GMT Received: from localhost (canis@localhost) by hive.gotdns.org (8.12.6/8.12.6/Submit) with ESMTP id h0BHHB20005262 for ; Sat, 11 Jan 2003 17:17:11 GMT X-Authentication-Warning: hive.gotdns.org: canis owned process doing -bs Date: Sat, 11 Jan 2003 17:17:11 +0000 (GMT) From: Canis Lupus X-X-Sender: canis@hive.gotdns.org To: metakit@equi4.com In-Reply-To: <200301111702.h0BH20c29205@trixie.triqs.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Metakit under OpenBSD Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, Has anyone had any luck or lack of it with Metakit & OpenBSD? Checking the list archives and Googling hasn't turned much up either way. It seems to compile just fine, but I'm getting failures in the regression tests, and (if I plough ahead and use it anyway) core dumps, database corruption and commit failures (with a Python script that works fine on Windows, using the pre-built .DLL available on the Metakit website). This is with Metakit 2.4.8-38 and OpenBSD 3.2 on i386. The regression tests are OK up to e05: e05 - Test memory mapping Failed: A(s == "hi!hello") and continue OK after that until m02: m02 - Blocked view bug *** Signal 11 (core dump, and no more tests performed) This is a shame, as I'm finding Metakit to be very useful under Windows and really want to get some of that software running on ny OpenBSD brick. Any extra information I can provide, tests I can try out etc, just let me know and I'll do my best. Cheers, Canis From jyl@best.com Sat Jan 11 13:56:13 2003 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h0BJuDc32042 for ; Sat, 11 Jan 2003 13:56:13 -0600 Received: (qmail 2731 invoked by uid 19667); 11 Jan 2003 19:57:52 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 11 Jan 2003 19:57:52 -0000 Message-ID: <3E207694.3040205@best.com> Date: Sat, 11 Jan 2003 11:55:00 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Canis Lupus CC: metakit@equi4.com Subject: Re: [Metakit] Metakit under OpenBSD References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I compiled Metakit 2.4.8 on FreeBSD 4.3 (not exactly OpenBSD but close) on the compile farm of sourceforge.net and all regression tests pass. The output of gcc -v is attached below: bash-2.05a$ gcc -v Using builtin specs. gcc version 2.95.4 20020320 [FreeBSD] --JYL Canis Lupus wrote: >Hi all, > >Has anyone had any luck or lack of it with Metakit & OpenBSD? Checking the >list archives and Googling hasn't turned much up either way. It seems to >compile just fine, but I'm getting failures in the regression tests, and >(if I plough ahead and use it anyway) core dumps, database corruption and >commit failures (with a Python script that works fine on Windows, using >the pre-built .DLL available on the Metakit website). > >This is with Metakit 2.4.8-38 and OpenBSD 3.2 on i386. The regression >tests are OK up to e05: > e05 - Test memory mapping > Failed: A(s == "hi!hello") > >and continue OK after that until m02: > m02 - Blocked view bug > *** Signal 11 >(core dump, and no more tests performed) > >This is a shame, as I'm finding Metakit to be very useful under Windows >and really want to get some of that software running on ny OpenBSD brick. >Any extra information I can provide, tests I can try out etc, just let me >know and I'll do my best. > >Cheers, > Canis > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From dan.gregory@wcom.com Fri Jan 17 17:11:26 2003 Received: from dgesmtp02.wcom.com (dgesmtp02.wcom.com [199.249.16.17]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0HNBPc00777 for ; Fri, 17 Jan 2003 17:11:25 -0600 Received: from pmismtp01.wcomnet.com ([166.38.62.36]) by firewall.wcom.com (Iplanet MTA ) with ESMTP id <0H8V00M2ASGUDG@firewall.wcom.com> for metakit@equi4.com; Fri, 17 Jan 2003 23:12:30 +0000 (GMT) Received: from pmismtp01.wcomnet.com by pmismtp01.wcomnet.com (iPlanet Messaging Server 5.1 HotFix 0.7 (built May 7 2002)) with SMTP id <0H8V00101SGTLW@pmismtp01.wcomnet.com> for metakit@equi4.com; Fri, 17 Jan 2003 23:12:30 +0000 (GMT) Received: from wcom.com ([166.34.145.47]) by pmismtp01.wcomnet.com (iPlanet Messaging Server 5.1 HotFix 0.7 (built May 7 2002)) with ESMTP id <0H8V0007SSGS1A@pmismtp01.wcomnet.com>; Fri, 17 Jan 2003 23:12:29 +0000 (GMT) Date: Fri, 17 Jan 2003 16:15:45 -0700 From: Dan Gregory To: metakit@equi4.com Message-id: <3E288EA1.3080806@wcom.com> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_yHiOVHN5zXwYUheZpKZRCQ)" X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.2.1) Gecko/20021130 Subject: [Metakit] Compiling on Sparc Solaris 8 w/ CC and MetaKit 2.4.8 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Boundary_(ID_yHiOVHN5zXwYUheZpKZRCQ) Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7bit Has anyone tried to compile this package with CC, specifically(Sun WorkShop 6 update 1 C++ 5.2 Patch 109508-09 2002/07/08)? The symptoms I was getting are: I can compile ok, but the test cores on b14(actually running in the debugger can get it to run correctly) when the view is accessed via GetAs(). This tries to invoke ~c4_string and when it dereferences p, it cores. If are having this problem then you can replace this method: /const char* c4_HandlerSeq::Description() { return _field != 0 ? UseTempBuffer(Definition().DescribeSubFields()) : 0; } / with this one: /const char* c4_HandlerSeq::Description() { const char* returnPtr = (_field != 0) ? UseTempBuffer(Definition().DescribeSubFields()) : 0; return returnPtr; }/ I think the problem lies somewhere with the destruction of temporary variables. The temporary c4_String variable returned from DescribeSubFields get descructed before Definition() or DescribeSubFields() gets called. It seems totally hosed... I am in the process of discussing this with sun to see if it's a bug or not. Later, Dan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here is a stack dump from the core: program terminated by signal SEGV (access to address exceeded protections) Current function is fDec 116 --*p; (/opt/SUNWspro/bin/../WS6U1/bin/sparcv9/dbx) where =>[1] fDec(p = 0xa1e10 "\x9d\xe3\xbf\xa0\xf6'\xa0P\xf4'\xa0L\xf2'\xa0H\xf0'\xa0D\xe0^G\xa0D\xa0^D ^P\xe2^G\xa0H\xe4^G\xa0L\xe6^G\xa0P\x90^T"), line 116 in "string.cpp" [2] c4_String::~c4_String(this = 0xffbeef78), line 149 in "string.cpp" [3] c4_HandlerSeq::Description(this = 0xf4568), line 226 in "handler.cpp" [4] c4_View::Description(this = 0xffbef060), line 139 in "mk4.inl" [5] c4_Storage::Description(this = 0xffbef374, name_ = 0xf2152 "a"), line 361 in "store.cpp" [6] c4_Storage::GetAs(this = 0xffbef374, description_ = 0xd740c "a[p1:I,p2:S]"), line 279 in "store.cpp" [7] TestBasics1(), line 194 in "tbasic1.cpp" [8] main(), line 63 in "regress.cpp" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --Boundary_(ID_yHiOVHN5zXwYUheZpKZRCQ) Content-type: text/html; charset=us-ascii Content-transfer-encoding: 7bit
Has anyone tried to compile this package with CC, specifically(Sun WorkShop 6 update 1 C++ 5.2 Patch 109508-09 2002/07/08)?   The symptoms I was getting are:  I can compile ok, but the test cores on b14(actually running in the debugger can get it to run correctly) when the view is accessed via GetAs().  This tries to invoke ~c4_string and when it dereferences p, it cores.  If are having this problem then you can replace this method:

const char* c4_HandlerSeq::Description()
{
  return _field != 0 ? UseTempBuffer(Definition().DescribeSubFields()) : 0;
}

with this one:

const char* c4_HandlerSeq::Description()
{
  const char* returnPtr = (_field != 0) ? UseTempBuffer(Definition().DescribeSubFields()) : 0;

  return returnPtr;
}


I think the problem lies somewhere with the destruction of temporary variables.  The temporary c4_String variable returned from DescribeSubFields get descructed before Definition() or DescribeSubFields() gets called.  It seems totally hosed...  I am in the process of discussing this with sun to see if it's a bug or not.


Later,
Dan



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Here is a stack dump from the core:
program terminated by signal SEGV (access to address exceeded protections)
Current function is fDec
  116     --*p;
(/opt/SUNWspro/bin/../WS6U1/bin/sparcv9/dbx) where                          
=>[1] fDec(p = 0xa1e10 "\x9d\xe3\xbf\xa0\xf6'\xa0P\xf4'\xa0L\xf2'\xa0H\xf0'\xa0D\xe0^G\xa0D\xa0^D ^P\xe2^G\xa0H\xe4^G\xa0L\xe6^G\xa0P\x90^T"), line 116 in "string.cpp"
  [2] c4_String::~c4_String(this = 0xffbeef78), line 149 in "string.cpp"
  [3] c4_HandlerSeq::Description(this = 0xf4568), line 226 in "handler.cpp"
  [4] c4_View::Description(this = 0xffbef060), line 139 in "mk4.inl"
  [5] c4_Storage::Description(this = 0xffbef374, name_ = 0xf2152 "a"), line 361 in "store.cpp"
  [6] c4_Storage::GetAs(this = 0xffbef374, description_ = 0xd740c "a[p1:I,p2:S]"), line 279 in "store.cpp"
  [7] TestBasics1(), line 194 in "tbasic1.cpp"
  [8] main(), line 63 in "regress.cpp"

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--Boundary_(ID_yHiOVHN5zXwYUheZpKZRCQ)-- From altis@semi-retired.com Sat Jan 18 15:35:18 2003 Received: from mpls-qmqp-01.inet.qwest.net (mpls-qmqp-01.inet.qwest.net [63.231.195.112]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h0ILZIc22746 for ; Sat, 18 Jan 2003 15:35:18 -0600 Received: (qmail 78206 invoked by uid 0); 18 Jan 2003 21:37:02 -0000 Received: from mpls-pop-05.inet.qwest.net (63.231.195.5) by mpls-qmqp-01.inet.qwest.net with QMQP; 18 Jan 2003 21:37:02 -0000 Received: from gamer.semi-retired.com (HELO gamer) (63.225.95.173) by mpls-pop-05.inet.qwest.net with SMTP; 18 Jan 2003 21:37:02 -0000 Date: Sat, 18 Jan 2003 13:37:46 -0800 Message-ID: From: "Kevin Altis" To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4920.2300 Importance: Normal Subject: [Metakit] location of .mk file for Mac OS X AddressBook? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is MetaKit still used for the AddressBook on Mac OS X? If so, what is the location of the .mk database file? I would like to try inspecting the file from the Python interpreter and also move the file over to a Windows machine and try the same things. ka From njriley@theremin.cs.uiuc.edu Sat Jan 18 15:48:05 2003 Received: from theremin.cs.uiuc.edu (theremin.cs.uiuc.edu [128.174.237.236]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0ILm5c23736 for ; Sat, 18 Jan 2003 15:48:05 -0600 Received: (from njriley@localhost) by theremin.cs.uiuc.edu (8.9.3/8.9.3) id PAA5210700; Sat, 18 Jan 2003 15:49:49 -0600 (CST) Date: Sat, 18 Jan 2003 15:49:49 -0600 From: Nicholas Riley To: Kevin Altis Cc: metakit@equi4.com Subject: Re: [Metakit] location of .mk file for Mac OS X AddressBook? Message-ID: <20030118214949.GA5092851@uiuc.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Sat, Jan 18, 2003 at 01:37:46PM -0800, Kevin Altis wrote: > Is MetaKit still used for the AddressBook on Mac OS X? If so, what is the > location of the .mk database file? I would like to try inspecting the file > from the Python interpreter and also move the file over to a Windows machine > and try the same things. ~/Library/Application Support/AddressBook.data, but I tried opening 'em with Mk4py and had no luck. (Also discovered another bug in Mk4py at the same time, which I'll fix soon since I'm working on it today anyway.) You should really be using the AddressBook APIs instead, and vCard for transport; they expose pretty much everything and are rather well designed. -- =Nicholas Riley | Pablo Research Group, Department of Computer Science and Medical Scholars Program, University of Illinois at Urbana-Champaign From altis@semi-retired.com Sat Jan 18 18:19:44 2003 Received: from mpls-qmqp-01.inet.qwest.net (mpls-qmqp-01.inet.qwest.net [63.231.195.112]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h0J0Jic29697 for ; Sat, 18 Jan 2003 18:19:44 -0600 Received: (qmail 30529 invoked by uid 0); 19 Jan 2003 00:21:29 -0000 Received: from mpls-pop-05.inet.qwest.net (63.231.195.5) by mpls-qmqp-01.inet.qwest.net with QMQP; 19 Jan 2003 00:21:29 -0000 Received: from gamer.semi-retired.com (HELO gamer) (63.225.95.173) by mpls-pop-05.inet.qwest.net with SMTP; 19 Jan 2003 00:21:29 -0000 Date: Sat, 18 Jan 2003 16:22:12 -0800 Message-ID: From: "Kevin Altis" To: "Nicholas Riley" Cc: metakit@equi4.com Subject: RE: [Metakit] location of .mk file for Mac OS X AddressBook? MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <20030118214949.GA5092851@uiuc.edu> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4920.2300 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > From: Nicholas Riley > > On Sat, Jan 18, 2003 at 01:37:46PM -0800, Kevin Altis wrote: > > Is MetaKit still used for the AddressBook on Mac OS X? If so, > what is the > > location of the .mk database file? I would like to try > inspecting the file > > from the Python interpreter and also move the file over to a > Windows machine > > and try the same things. > > ~/Library/Application Support/AddressBook.data, but I tried opening > 'em with Mk4py and had no luck. (Also discovered another bug in Mk4py > at the same time, which I'll fix soon since I'm working on it today > anyway.) > > You should really be using the AddressBook APIs instead, and vCard for > transport; they expose pretty much everything and are rather well > designed. Thanks! If I was just using Mac OS X I would look at that API. Offlist, my friend, Bill Bumgarner already suggested using the AddressBook APIs from PyObjC. Bill said: Assuming you have PyObjC installed... in any case, AddressBook stores everything in vcard format. Python 2.3a1 (#1, Jan 11 2003, 15:30:09) [GCC 3.1 20020420 (prerelease)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import AddressBook >>> book = AddressBook.ABAddressBook.sharedAddressBook() >>> p = book.people()[0] >>> p.valueForProperty_('First') 'Bill' >>> p.valueForProperty_('HomePage') 'http://www.codefab.com/' >>> However, in my case, I have no interest in doing something that only runs on Mac OS X, thus my interest in accessing the data directly with metakit. I transferred the file over to a Windows 2000 box and tried accessing the data there and it seems to work. >>> import metakit >>> os.chdir(r'c:\tmp') >>> db = metakit.storage('AddressBook.data', 0) >>> db.description() 'ABDataType[UID:S,Property:S,_Property:I,Table:S,_Table:I,Type:I,_Type:I],_A BDataType[_H:I,_R:I],ABInfo[UID:S,Me:S,_Me:I,Synced:I,_Synced:I,Version:I,_V ersion:I],_ABInfo[_H:I,_R:I],ABGroup[UID:S,Creation:S,_Creation:I,Modificati on:S,_Modification:I,GroupName:S,_GroupName:I],_ABGroup[_H:I,_R:I],ABMailRec ent[UID:S,Last:S,_Last:I,PersonUID:S,_PersonUID:I,Email:S,_Email:I,First:S,_ First:I,MailLastDates:B,_MailLastDates:I],_ABMailRecent[_H:I,_R:I],ABGroupCo ntents[UID:S,distribution:B,_distribution:I,MemberUID:S,_MemberUID:I,GroupUI D:S,_GroupUID:I],_ABGroupContents[_H:I,_R:I],ABPerson[UID:S,Organization:S,_ Organization:I,AIMInstant:B,_AIMInstant:I,ICQInstant:B,_ICQInstant:I,MSNInst ant:B,_MSNInstant:I,Email:B,_Email:I,HomePage:S,_HomePage:I,FirstPhonetic:S, _FirstPhonetic:I,Middle:S,_Middle:I,MiddlePhonetic:S,_MiddlePhonetic:I,Last: S,_Last:I,MaidenName:S,_MaidenName:I,LastPhonetic:S,_LastPhonetic:I,ABPerson Flags:I,_ABPersonFlags:I,Phone:B,_Phone:I,Modification:S,_Modification:I,Yah ooInstant:B,_YahooInstant:I,JobTitle:S,_JobTitle:I,Title:S,_Title:I,Address: B,_Address:I,First:S,_First:I,Nickname:S,_Nickname:I,JabberInstant:B,_Jabber Instant:I,Note:S,_Note:I,Suffix:S,_Suffix:I,StoredPictureName:S,_StoredPictu reName:I,Creation:S,_Creation:I,Birthday:S,_Birthday:I],_ABPerson[_H:I,_R:I] ,ABLog[UID:S,DeletedUID:S,_DeletedUID:I],_ABLog[_H:I,_R:I]' >>> vw = db.view('ABPerson') >>> vw[0].First 'Kevin' >>> vw[0].Last 'Altis' I'll play some more, this looks promising. ka --- Kevin Altis altis@semi-retired.com http://radio.weblogs.com/0102677/ http://www.pythoncard.org/ From jcw@equi4.com Sun Jan 19 08:04:45 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0JE4ic00956 for ; Sun, 19 Jan 2003 08:04:45 -0600 Date: Sun, 19 Jan 2003 15:06:29 +0100 Subject: Re: [Metakit] Compiling on Sparc Solaris 8 w/ CC and MetaKit 2.4.8 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list In-Reply-To: <3E288EA1.3080806@wcom.com> Message-Id: <34B48B8E-2BB7-11D7-B0FE-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h0JE4ic00956 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Dan Gregory wrote: > I can compile ok, but the test cores on b14(actually running in the > debugger can get it to run correctly) when the view is accessed via > GetAs().  This tries to invoke ~c4_string and when it dereferences p, > it cores.  If are having this problem then you can replace this > method: > > const char* c4_HandlerSeq::Description() > { >   return _field != 0 ? UseTempBuffer(Definition().DescribeSubFields()) > : 0; > } > > with this one: > > const char* c4_HandlerSeq::Description() > { >   const char* returnPtr = (_field != 0) ? > UseTempBuffer(Definition().DescribeSubFields()) : 0; > >   return returnPtr; > } > > I think the problem lies somewhere with the destruction of temporary > variables.  The temporary c4_String variable returned from > DescribeSubFields get descructed before Definition() or > DescribeSubFields() gets called.  It seems totally hosed...  I am in > the process of discussing this with sun to see if it's a bug or not. I've made changes in the source - better safe than sorry. In CVS now. Thanks. -jcw From Michael_Scharf@gmx.de Sun Jan 19 23:45:55 2003 Received: from mailout03.sul.t-online.com (mailout03.sul.t-online.com [194.25.134.81]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0K5jsc25942 for ; Sun, 19 Jan 2003 23:45:54 -0600 Received: from fwd03.sul.t-online.de by mailout03.sul.t-online.com with smtp id 18aUmY-0008MJ-02; Mon, 20 Jan 2003 06:47:38 +0100 Received: from gmx.de (320007127165-0001@[217.228.177.175]) by fmrl03.sul.t-online.com with esmtp id 18aUmY-1KWJyCC; Mon, 20 Jan 2003 06:47:38 +0100 Message-ID: <3E2B8D77.6060701@gmx.de> Date: Mon, 20 Jan 2003 06:47:35 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Subject: [Metakit] Q: Basic questions about huge databases. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am new to metakit but what I have read about it, it seems to be a good database for what I want to do. I want to use metakit mostly from python, but also from delphi (because I want to visualize things in delphi) on windows. I have been playing with sqlite, but it seems to have a real performance problem, when there are millions of rows in the database (and the database is not in the disk cache of the OS). In principle I want to model something like a file system with symbolic links. There are directories and files. Files have a 300-500 byte annotation. Most files are empty initially. And file content is generated over time. I have to deal with 1-10 million files of a total of 10Gb-100Gb. There are also sets of files (directories) that I want to efficiently access together. If those files are spread on the disk, the access disk time gets the limiting factor. If they are close to each other and I can access them in the order they are on disk, then the access is much faster compared to a random distribution on disk. Actually, files accessed together are usually created together. If I would use a auto-increment-integer-key-field, there is a high probability that I want to access files with keys that are within a small range. What I essentially need is: - a huge index (a few million keys) with md5 keys (preferably a hash table) - the index points to some data structure with meta info (meta_data) (about 10 attributes with 300-500 bytes per row) - most of those data structures point to a ~3kb blob field. - some of those data structures point to a big file like data structure (a blob with varying sizes 20kb-10MB) - some data structure to maintain tree and graph like structures Questions: - Can metakit deal with such data (if 50 GB is to big, I could move the blob like data into a flat file)? - Can I quickly sort on some columns of the meta_data? - How would you model a tree/graph like data structure in metakit? - Can I access item in the order they are on disk. Say: I give metakit 100 or 1000 keys, can it fetch the data efficiently (in disk order)? - Can I store binary data with metakit (null characters in strings)? - How stable is metakit (in case of power failure, can the database be corrupted so that it is unrecoverable)? I don't care if I loose the last (not committed) entries. Michael From schlenk@uni-oldenburg.de Mon Jan 20 09:47:14 2003 Received: from mailout08.sul.t-online.com (mailout08.sul.t-online.com [194.25.134.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0KFlEc01247 for ; Mon, 20 Jan 2003 09:47:14 -0600 Received: from fwd10.sul.t-online.de by mailout08.sul.t-online.com with smtp id 18aeAT-0004Jc-04; Mon, 20 Jan 2003 16:48:57 +0100 Received: from pD9526E6A.dip.t-dialin.net (520047057634-0001@[217.82.99.229]) by fmrl10.sul.t-online.com with esmtp id 18aeAI-0up4bIC; Mon, 20 Jan 2003 16:48:46 +0100 Received: from uni-oldenburg.de (unknown [192.168.2.2]) by pD9526E6A.dip.t-dialin.net (Postfix) with ESMTP id D24DB7BEF; Mon, 12 Mar 2001 14:34:30 +0000 (/etc/localtime) Message-ID: <3E2C1C5C.70708@uni-oldenburg.de> Date: Mon, 20 Jan 2003 16:57:16 +0100 From: Michael Schlenker Organization: =?ISO-8859-1?Q?Universit=E4t_Oldenburg?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2a) Gecko/20020910 X-Accept-Language: de-de, en-us MIME-Version: 1.0 To: Michael Scharf Cc: Metakit List Subject: Re: [Metakit] Q: Basic questions about huge databases. References: <3E2B8D77.6060701@gmx.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 520047057634-0001@t-dialin.net Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Michael Scharf wrote: > Hi, > > I have been playing with sqlite, but it seems to have a real > performance problem, when there are millions of rows in the > database (and the database is not in the disk cache of the OS). Actually i found sqlite scales a bit better for huge numbers of rows, after i created indices for the things i needed, but not so much better than metakit... > What I essentially need is: > > - a huge index (a few million keys) with md5 keys (preferably a hash > table) > - the index points to some data structure with meta info (meta_data) > (about 10 attributes with 300-500 bytes per row) > - most of those data structures point to a ~3kb blob field. > - some of those data structures point to a big file like > data structure (a blob with varying sizes 20kb-10MB) > - some data structure to maintain tree and graph like > structures > > Questions: > - Can metakit deal with such data (if 50 GB is to big, I could > move the blob like data into a flat file)? As far as i know from talks with JCW last summer metakit doesn't play in the Gigabytes of Data League. So moving the large blobs out would probably help. > - Can I quickly sort on some columns of the meta_data? > - How would you model a tree/graph like data structure in > metakit? Do you know E4graph? It is designed on top of metakit for those kind of structures. http://www.e4graph.com/e4graph/index.html > - Can I store binary data with metakit (null characters in > strings)? At least the tcl binding can, so i assume the other bindings can do it as well. Michael Schlenker From jcw@equi4.com Mon Jan 20 10:08:53 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0KG8qc03204 for ; Mon, 20 Jan 2003 10:08:52 -0600 Date: Mon, 20 Jan 2003 17:10:35 +0100 Subject: Re: [Metakit] Q: Basic questions about huge databases. Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3E2C1C5C.70708@uni-oldenburg.de> Message-Id: X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Michael (2x), >> - a huge index (a few million keys) with md5 keys (preferably a hash >> table) Should be doable. You say python - have a look at the "hash" operator, which manages hash tables. And look at examples/find.py for lots more info. >> - the index points to some data structure with meta info (meta_data) >> (about 10 attributes with 300-500 bytes per row) 2,000,000 x 500 b = 1 Gb I think you will push MK too far. It mmap's all of the file, on a 32b address space machine that may become an issue. >> - most of those data structures point to a ~3kb blob field. You will have to use a second file, and manage it separately (perhaps with MK, but not being a MK file). >> - some of those data structures point to a big file like >> data structure (a blob with varying sizes 20kb-10MB) Same. >> - some data structure to maintain tree and graph like >> structures Several ways to do that. If you're talking millions of nodes... ouch. >> - Can metakit deal with such data (if 50 GB is to big, I could >> move the blob like data into a flat file)? MK cannot handle 50 Gb, or 5 Gb. Even 1 Gb is pushing it (but more because of other aspects - such as free space pool size). You will have to look into the hash and blocked viewer operations, without them this is a guaranteed dead end. > As far as i know from talks with JCW last summer metakit doesn't play > in the Gigabytes of Data League. So moving the large blobs out would > probably help. > >> - Can I quickly sort on some columns of the meta_data? Sort / select a million rows? Ouch. >> - Can I store binary data with metakit (null characters in >> strings)? > > At least the tcl binding can, so i assume the other bindings can do it > as well. Yes. Declare the fields as type "B". You have not mentioned the platform/OS. It may sound like heresy, but have you considered something like ReiserFS? It's geared at huge numbers of small (file) objects (I've got two > 50 Gb "datasets", i.e. disks, set up that way). I'd use that for most of the bits, and then see if MK can be used to provide access paths into such a storage system. Treating MK as a disk replacement is not going to be a fun experience. Oracle might be up to that, but not MK. Let me qualify that as "not yet", btw :) -jcw From jcw@equi4.com Mon Jan 20 11:44:22 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0KHiLc11201; Mon, 20 Jan 2003 11:44:21 -0600 Date: Mon, 20 Jan 2003 18:46:05 +0100 Subject: Re: [Metakit] Q: Basic questions about huge databases. Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) Cc: Starkit list To: Metakit mailing list From: Jean-Claude Wippler In-Reply-To: Message-Id: <0C87F0D7-2C9F-11D7-B0FE-000A957664F8@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Re discussion on limits and sizes... (crossposted to starkit mailing list). I just did a few tests (from tcl), using hashing on a fast Linux box. Wanted to follow up because I think I gave the impression that millions of rows is a lot... Well, I don't want to get into actual timings, given that it all depends on everything from etching process used to the phase of the moon anyway, but here's some info which may be of use. - databases of several hundreds Mb's can work really well - same for "millions of rows", it's no longer such a big deal - add many thousands of rows per second even from a scripting interface - as of MK 2.3/2.4, there are persistent hash indexes, and they really have O(1) performance (10's/100's of thousands of accesses per second) There continue to be effects which can surprise you, because column-wise data is something radically, totally, fundamentally different, but you have to keep in mind that these "surprises" can go both ways... At the C/C++ and Python and Tcl level, be prepared to see performance levels which seem unreal. There will be an order of magnitude difference between a hand-coded C/C++ in-memory hash table and the one MK offers persistently, but that should be about it. I've been seeing hash access rates of 25K searches/sec, using a new experimental Tcl wrapper (which appears to slow it down 4-fold, so there's a lot of leeway for optimization). I've never taken the time to compare this with other DB's (again: etching and moon phases), but it seems to me that MK packs a lot of oomph. Please feel free to follow up with teal-life stories and experiences of your own. -jcw From bmenzel@fs.fed.us Tue Jan 21 17:17:38 2003 Received: from rmaccsv2.rmacc.r2.fs.fed.us (rmaccsv2.rmacc.r2.fs.fed.us [166.2.208.11]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0LNHcc05412 for ; Tue, 21 Jan 2003 17:17:38 -0600 Received: from sv10.r2.fs.fed.us by rmaccsv2.rmacc.r2.fs.fed.us (USDA-FS/LakewoodCO/3-03-00) using ESMTP; id QAA17676 for ; Tue, 21 Jan 2003 16:19:23 -0700 To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.3 March 21, 2000 Message-ID: From: "Barbara Menzel" Date: Tue, 21 Jan 2003 16:19:52 -0700 X-MIMETrack: Serialize by Router on ENTR2B/E/USDAFS(Release 5.0.11 |July 24, 2002) at 01/21/2003 04:19:22 PM MIME-Version: 1.0 Content-type: multipart/mixed; Boundary="0__=08BBE626DFEC79028f9e8a93df938690918c08BBE626DFEC7902" Content-Disposition: inline Apparently-To: Subject: [Metakit] Question on Views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0__=08BBE626DFEC79028f9e8a93df938690918c08BBE626DFEC7902 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: quoted-printable Hello! We are using MetaKit with Visual C++. Often, we find there is a need t= o initialize or perform some initial action on a view within a new class.= I've tried passing the view into the object via the constructor and get= several compile errors, primarily, c4_view is not a recognized type. However, passing a view as a parameter in a member function, there are = no errors and everything works fine. The view can even be updated within = the member function and returned with the updates included. Has anyone tri= ed this or something similar with any success? (Embedded image moved to file: pic07483.gif) = --0__=08BBE626DFEC79028f9e8a93df938690918c08BBE626DFEC7902 Content-type: image/gif; name="pic07483.gif" Content-Disposition: attachment; filename="pic07483.gif" Content-transfer-encoding: base64 R0lGODlhrgF5APcAAAAAAAAA/yk5OSk5SilCKSlCSilCWilKOSlKcylSYylaYylaczExMTFShDFS jDFapTFjhDFjlDk5KTlCOTlKKTlKSjlKWjlSczlaYzljc0IpEEJapUJjhEJjlEKErUKEvUKEzkpK OUpKWkpSSkpjc0prhEprlEprpUpzvUpzxkpz1kp7rVIxGFI5KVI5OVJKKVJKOVJKSlJKc1JSa1Ja SlJaWlJjWlJja1Jrc1KErVKExlKMvVKMzmNCMWNCQmNKUmNSMWNSQmNSY2NSa2NSc2NaUmNjY2Nj c2NjhGNjjGNzlGNzpWN7zmOErWOEvWOM1muEpXNCKXNCQnNKUnNSMXNSQnNSY3NSa3NaQnNaUnNj UnNjY3NrY3Nrc3NrhHNrjHNzpXN7lHN7tXOUxnuEvXuMrXuMxoRKMYRKQoRaY4RjhIR7pYSUrYSU xoxKMYxKQoxSUoxaQoxaY4xaa4xjQoxjUoxjhIxrY4xrc4xzhIx7hIyEjJxKMZxKQpxaOZxaSpxa Upxja5xrQpxrSpxrWpxzWpxzc5xzjJylxqVaMaVra6Vrc6VzhKVzjKV7c6WEhKWElKWMlKWMnK1a Qq1aUq1rSq1rWq1zUq17WrVza7V7jLWEc7WMjL1aSr1jMb1jWr1zUr1za71zc717Ur17hL17lL2E c72Mc72MnL2Mpb2UjL2cpcZjQsZjUsZzSsZzWsaEWsaMWs57a857c857lM6EhM6Mc86MnM6UhM6U nM6ljM6lnM6lrdZjStZjUtZzStZzWtaEWtaUWtaltdaltdate9a1xt5aQt5zUt57a96Ua96UhN6U lN6lc96ta96tjN6tnN61reeEWueEa+eUhOecWuecpeelreete+e1ree9ree9xufGpefOvefe7+9r Uu97Uu+EUu+Uc++clO+lc++ljO+tnO+1a++1hO+1ve+9pe/OzveUWvelc/elrfetUve1jPe1nPe9 e/e9jPe9pffGnPfGpffGvffepffetffvzv+1rf/GhP/Opf/v1v///yH5BAAAAAAALAAAAACuAXkA AAj+AP8JHEiwoMGDCBMqXMiwocOHECNKnEixosWLGDNq3Mixo8ePIEOKHEmypMmTKFOu8JCDB48P Hj6AAPGBx46bMXes+LDDAw+aPnXocJnDw5McN3v2dLkjRw4dHnSgOOGhatUTK7GuXHGiq9aqK1Z6 4OqVrFevOVbkOBs2pdu3cOPKnXtR7VEdJ4QKnQmTB4oUQp/YZIKUh4eZPkHsePJEx00nPJ44ASH0 JQgUHz5MPYFia+eYH8SGxooVBdjOWsuuLKrjyZK0ZNXSnU27tu3bHJHu4LnDJo+nLn9D/vDUKuUc KU4gH7vTKgrYOrhGT94VBU2ZJ5KvAOFBa9YTmsH+OgnLdcVmqkVXNFniZAkUKG2oql2Cu779+/hr rxj6JIVLJk4wAeBieukAQloq/GXdDimEpd12KaggoWk7pWAhVxb+ZWEKKGCWwgcpgCChTMxxpsIT Y4iBVRMsOuFEDiyOUcYSXrlnggldsZjfjjz26KNGK/DQ4AqABReZCiztIJWE3KHgmA5OCOZEVAdO FRZmKoDAYYearSBhTFOpYCGIoZnWFU+XoUBGG22Q4YQZMJK3hHpskLHGCUoskeeNJXjFEAAGAVqQ oAQROpChAiH6j6KMBurooI8WGumhkyZa6aKXNgrpppJySqmnloKKqaiadmrqp6eGmuqoq5Z6UAn+ +znR2wdYpeDBYpEdtUJTjanAQ5D7PQXlCgB+8CSIKOikJWbMNihTmR78NVOHyY6FAhRLsBfWnOqt VWMba5SBCBRsQKHEjR10YEJDACDa7qDuxgvvvIXKWy+9h9qbL76J6tsvv4v6GzDA795r8L4H/5vw wAsXjPDDCkPMsMQOR2zxxBdXTFATkoFgHmBbMsjYUL86JZRTaqXnMQoqeLlWWE85kUJRM82kArK2 RkttVTlf2cQJ2HYVlhNe3WjCCiYsEYYJSoQFhbc3nvCQxlQLXDXBVmeN9dYNa9011xR7HTbYGYtd NtkYp331wv+Y0ATRlzX1W1ROtRQrE7uFlV3+Wg3qUJOt5un9XHp+V1sUb1J1tlJmO61AI41l/NxV GzR2haMJHXTlXhmYW95BBhlw0EFErqqK6ummp84q6qurXnrrsL8ue6a0k1p7q7ezPrvtvOPeO+qw kmeeCro18Vtaz60gM1WlnfVVZ2YCrlpsXrGkllVdIf9zecolPaNXHXBwggkllLAGGDiSAHoGJEDA QQQcTG222vOvPfbX+N+v/9n5878//Wizn/8GCMD+FfB//9CKlZyUFp1oBWUnIJq3zuM86gkNQ3oj Dfe45zznRa1zOOKK0cKXARyxAQzly5wJOJAB9zVgAeyqnwwDOEMDCvCABLyhDmv4vx3S8If+Npzf QI7mN6E0YT8naIIOkCOUqRzBBuU7mgbZwpnYcKUzl9lPWKKVGudlLl0niMAJOlACMJ6AAxwoX58s V8YT6KkE6EpXBMLXAQg04E+5c10eY7fH3fnuj7rroyB/p0dC8tGQfgwkIi9FEBOABQx5wMMXwNAE JaxhDXs43xf0AIkvHCEJM7iRWTp4QQuxJYPC8+AGMIcjdXVAjOFLo7qMRj4+Ga18JFghBOCIo678 6JfADKYw/wGrE4ABD3dIgxa40AU82AGZeHAEHrrQBS/g4Qg4SEIvL8gZDcLMKWNhonLS4pSimWCV IERj6E6gAAhA4JV1PEEJ0IgB0ZWABPP+LMECIgCB0C2gBBDg0zAHStCCziaESsADHgqRBWve4Q51 qEMa8JCHLhihC560wRGM0EsiOUVJ6XnKVHJAK+59Ri3jM6fo4PeAMEZgju5zXxlvRAIO4BOO9+RA Py+wgAbM8wERwKlBh0rUooakBDlgw0PTcAc7zCGidzDCF+7gBSMU4Qc/KIIQiECDGtwAqU9JS1lO 4LFRdpCDXenTLcf4SgdkIF3vVAD88Im5CPAJnbPkwAPKmAEMvHNdRg2sYAcLkRIs4RCMkGggiiAH ptZBC3eYg1aHMAMjCKEGRODCDGgwgxtss3pn7cpYmNOEHNDocuNDLbpKwE9+vlOmcOz+6wYy0CcS 4DOXt4WjuuhK2N769rf/yNMiClGHSxwiC3WQwxwiiwcubGEIXeCCEWYwgyJsQQsziIEIbkDK7npT LVBI7QdTOkcOmMB9HVgABBwQxpuqS520xVz4cGqCDMwglxuwAAaAy9/+GhQJTUCmIpqrBjzMAQ9p SINkhQBZNWwhBloYQhGGYIQjzEAENuCud7HyMrEuYQ1t+Jl5U1tG0bnTvAtYgDw5oF55kvC8EIiA bXdZXxJ0gK4mIAEUAevfHvu4RxTFgxrSoAbnpkGiW7DCIfBgXTvUYQtZ6AISqHtREYigBjPAwRqd N9qwbK8JZSjXGeEaxb4uwAEPoG3+BBxQRvjp1LUn2MAFfFpiNDqAAxjwKQcscIHPZeBP7Qo0uzAS aEEHqmqKokii+wWoRaOk0I62T6QTMukdKZMLeNiCdeVghS6ooQhFqMNVj6BQL2wU1EkgghW2cGEa iACfbCmRcpawnhyUgXJdCV+M4WfXz0FgjKLrAAaCmjkOXIBPEEjxAhTQABYrQL075UADLuAABwya dQBTNLZxaBGNocrbGqm0qcRdG3KzjdL1SYMVrnDdrGpaCFmIgRHmYAU1dEELzy2CEWxAAqva4Q5d xXIN+H20bwoNzGEIb+S68mtXmneFbp1jGOU4Rzq6080X4KlOe+qABmDgzAiwAAL+PqeAa6uOkRKp dKMpZW48VqrlEGk5oWAul5WTbmr1WXUa4JCGHxBBDVYoQgx+sIU74KERWrCq0LE8BCEYgQtIwAER alCDrtJACeX82WtWwIZysUhysVwhCcZIWzZywK37xEAYW3iBDFzAAhloQLURIPdmo7G+DpitAjpu clY5bOWNNtTfR+VtQTna5pgS/MwZBumIJfrwg3/X4gtG+cDPfPLjtpThK9Z4m0ce8YFfXeQd5S6M fd7yjA4YzUlShylA9MmgnoKViW70JW+BCDNIwhHSUIMhVFTfWd5oZ2+QBG0ecQW3nlETgpZaNG4A r+Id3zvr+FII1LMBclfvbPf+CYFnq3fkDbAj9q1/R0AbmvCqV5jhNU/51BUa3ZgPFeoTL//6o3zy kgd8+/2uqvWfn9Gbt372l3j7h3qSl34HiH7/13/5IoAOyIDoBzuzkQZbgAdWkAVwUARaoFVYpgV1 gExd8GAj0EzJhGl3sAXUBGpDIAIjUAE1QE1KEAZgVgZQsHx3Ik8sFVR9QkZ9EjolkAEPcGJpdAF7 xQEbgH0KgADJpl7VFn4ppgDShgF05wAw5HL9t3/8l3qiRzUnV38CqHpfuHgAeHKLJoYPmIVm2IBf qEho+CmKd4ZwOIYS2Hjf1oD2d3mrMhtZgFxJtoFCEGpbQHQTpUxaoF1d0Aj+j0WCWvAFdpAHerAH SFABFlABNvAFYVAuMrIeYHYj7rRLcCRPKbVCcOWD8+QAENBnL9VTD2CKneh2ChB3EJAA7vRCKYYA fReHEHiGEbhtneJ5qCOGWYhtpdeGEZiGAziMfwSM52eMuKiLhVSHd8iA9rJ6JFEEWWAFfigEP+B0 QiACXCAHyQQHVnBhQgCOHogHdWAFyGQHS7YGN1ADFmADNZAEX/AFSaAEJaAE+Kgu8DNf4xNFdYQ5 OmVPHWCKDuBXEYB9Z7YAbrdnd/ZCGecAfEaFCZBitxiNuXiM7peH/peRxOiLEhiMzCh/cHiAIOlH xph5xYiRuJgqyoiR9Bf+jdQYElnVej8gAliFjVrwA/gWUaImAjGwBXAwB2nQTIdgBcn1gYaAB14w ApM4cDhwBOQDR8WHj65lS+iiU8lWR8l2djpVbW51ATHWAHSHfRk3ZwnQAMzGUwqwdwhQcgsBaW/4 floIgMt4l3T5hiSJhwgol5tnl4cWLwE4MCapeITHl/+XmH+3mIKGl++Xf40ZeozHeYzZlwFIl1vI l7YRA0UQWUUgAn+4h/qWBlnAVHegBSIwBEVHb9aFB0LABXNACIVwCYWABzFAXTMwAgNnAzaAA1rm RkyTBAuQOSkVPu1zAdKGRlDIdq90RuuVbGK5bN23AAhAhSnWcdNGlrb++GM3xxEzuRGCN1DfCRdD dwVXEAMSZo1JZgRZsAVOJgfxFmoQdWQfWANZMJ+NEAh4QF1VNwM1QALZ9EQlgE19ogRJYARK4wVh oD43sAde4JsGSYS7dpALwAGymAET2QAGUJ0vpJ0it5VUyJ0x5xHjmRHhKUwl+hZTMAVNJwJWNQRE QATuBmU8dwVTUAT0uQVXkAV2kGBw8FCGUAd3UAS5+Z9WNo9JgAMjYFtJswRHoARcwAVFIKUK+gX8 NgMYUJAn4E5tt3cJeYpNuABzRoXMhgAZgABv+ZZtWQIiGhcL+BaY+UtvyiM0cHtJgARGMHBC8Ida BWp3kAVEegVyIFH+gBpqkHUHq/ZQCnUHXCAClSWPN4BNOMCbk3oEJWADIqAER7CpRvB0nvRVr9R2 7BVUGNBCaMqQ62WdTthTammK2MdCbRqrsioXy8RMXbBVXjAFQjADRPCZe5pVnFkHV2CNRWAFc9Ce TPWj4HgHmvAIh5AHR+AFvVoEvWlhvolN+ORVXtAFn3QEebAHFmUDeAY6+pSq1Xampiidy6adYtpx FVp3sxqv8loSRHAEkXAHXyAEX3Cr9aoFqWYEaZCbInAFaSAEN3pVezgFe/hUeMAJpWAIzpoHkIAH h5AKeuAFW0ADNqCxI5BhvHlfd3oENYAE2/oFe6AHesBRXZkB6vX+bEvIUw5Qne2UYslGiwxZhfOa szqrEVlAUUcAalW1qRZlWbpHXUXws0PwA0OwolnFtMh1B6SQCaXgCIVACIoQUYRgB5qgCkegsRUw AiJwBDfggp1lA0n6RBvFBXqABDVgBHnGcRnQhNiHAXOGZ2SZnSHnANR2AQlgfnNKaIjmI405ETQ3 uDtrHwrVBXlQr71Kj0aQVUZgaqq5BVygBe1pBEQgYVPwAzc6BVswBZeQCaJ7tT7puY4gCY6QB0ag m5OYYVRHXRgWlSWwb/q2BaxmbF2pAHDXlinGtz2FANR5ZkuYluUXl7w4omhjG8gIbiQqMQqRoof7 EB9oBJbbBT/+IFVTJgRHAAZIEJSNWGT4+gUoSARdAGpYZY2M8AqVkAmKsAVEWXQUWwekoAlewLYs eGE2YAQiwFn4ewOgFq17sAp6UAK0KLcFybcI8HYZtwBSmAAiV0YlYAF9Z6Kkx3L3gYwh6Z2igm7R axJ1EF1bYARGR01W8AOR+rNJoAcfaILL1FxfgATRulFDEAiW0AqZgI6K+m8+GQg3egRfUFGcxZs3 gAP7qwSuhgMzgKde8AVagANu5Wzdp7u/m31J6HYXUKr4hAFs6nJ6qXkWE5mlF3+OZ3pz+Zfst4Us N5hsc3oGaIcZQ2n4V5ej18EZUU3UqwVqcAfl+79LgATa+1D+jfhMXqAGeZAEdmBv+3oEW9AIi5AJ lgAIz2RghpAHEDVNFRbCj4AKmsBkVDcC+6ax/aZj1LWtRUADfbV31VZHe1azfRU6Wvw5N7XFcbmM adyMeiTG/leAGqmA+beShraGB5PLYch4MBmSwrzL80fHGOGNVzAEQ+AFVWUFW4UE+kbNeOwFeVBR X7CteexpH9xMi4oHm2AITmYIi7AI03QE3EqyXpAKkHAN7gANFLUFNaCbNKCxG2sDHVsDSrCxNdBX lIgBpWpTx5ZL83QjbWdb5XNtL+l3v3yivbiL6SeSWHiZLPnLbvi8chieKSmHXZiXGb3RyqwR2zgF RBCpUvr+h0cAw51agdOUzV+gUEdgB4ecB4OcB6dgB8wUpIraCIYgB0l3BAxGBFpwBIcwDNmQDobQ BY8ACZwQSUcrsvJYzyMwAtRaA5Oqz7yJAZOKAwD6sT94iqGDTxNMjHMIzArokSMpx5vijGZtOsDM 0SEtjG3NKcOchyMtEQNbAyKQZEFXBEQAw/sKTVwwVYzITIIcSV7QiI2QBUJQgR94B5EUwny6pzZg u6twDdFAyQrVCKdACMqkUPoGajVQASwItlWdzzQwAvhMqbwJoLZFRrZV1ilpyycKkivJko9iy7vY kWOc26fD29AI3B3p1nk9ETGAWTGaBb1KvpIACXegB+v+SE1doAeRVN2Mmt2ZRrVbUAdPRQiHsAXa G9VWFQP52wWpoAoKdXScgArmnLiZvAoOGgMtyIKSqJsb65T6XHWrjc+TalsAbn6BKceXp5kTrcZg CMZjSIeTGXqVl3qRKSmKWZcObcYXQ+EN3pcI87fH7RDaRb1Z8LO3GVG5oArPRE0/bK9MuQdMmQdc UGBaoAWF8KdzAAh1YAg4vo6MSqShlARdUAhRqlCpkAnj3AhNpVBSqm95agQ08AMVIIkVsNqTOIkd OwOmjdX8xptwhAOEBdEd/uUQAbY/SwTdWAPMdAiGYHTIRLlbgAQP5QWHYAdeoASUHF14YAmUMAV1 AAf+WaAJpcAIjkAImgAHPxAD9lYDd7AJOFwHnFALlyAKd2AIjaBQjlAH1/u4L9ipIThRn/nkTglq I4AAX2sDlEh1A0cCXY7XYL7qBxEDSf64YVsHA5YFy9SpoKaBH+iBQupMD4XmgBAIaSAHChsKoEAI l0AIhMAFQekImOAInFAJJ5gHnGAKlWAJiwBZgxAIDyUHzT2weDAFipsLuoALi1XP9awFNFABE4AB LWgAq13qXN7lHM7q9D4QP5AFlWtdoLYFgQAHe07rOtqrRxZqmaYJjiDZkvAIw/VYdbCHqJAJoTAI g2DpIqAHzODommAIiuAIpIAKzFAJlUAINo6OeGD+CHPAuZubVT9wB5iAB6qgB5HQrIaA7lZWAAVw 35JYABaw2iNQ7z7fphgorEUnalew5z9gnubZnoxgdFqAYHdwCYbgCJrMCZhg45tbBFdwCZbwCTYu ajHgCLYwCpsQCqIwv5zQCMwQCpkg8oUACMhu49uoVdolAllwCfAJsdewC45w8Po7AAMwATb/5AJQ AE5ZAT9/+D7Gk62nBXYQdHdACO0Z40R/r5UspIYQCI5ADKoADbVg8mmggVOq9m0vrDEgBJtgCYqA CstwC6OwCKFLDa9ACoMgB4GwrE3vqEmQmit4B4pwo1qQC7lgCJqQC81KdQUwAMdfAQdgABNgAQX+ QAOIH/38BQPsuYeQdYGtF4LOVQR4oJ+t11j8vgnRkA0lngmBoAhZ0HRbkAaWYO1wII4/YAWiIAqA 4AhqTwiNAPHUUPaLcAjgraMAkebKCBE1RhikoQpOkR9dahXCs8gQNFVFRlwcIABBgQEVCiCo8E/k SJIiAZwseRJASZYtXb6EGVNmSpQrZ94kidKkzpE2e6pU+XPlUKA+g/rEmVTpUqZNnTaNMSXNjxhZ 7sCBM+eOljRzuhDBU6cOni537uC5A83armucXgGCM6WIESNpFi2yVCdLlhqDLrWqQ2iWpTuFPjGz BYoUoTqONOFptKWICBozZIgQMeML1yJyRsH+yVTnjiMtI3CMqCCiAoEBEwS8HjCT586ntXEidWkU t+2cvVP+/mdT+E+TvXXzRp5c+fKbMYoIKVIji1g4d+oAop7mUCOzaepwuZMmV65huai9gvODiJYt WRaBahX6yo8ieAqBWiTnlSJCijK9+sQUTERRhBRMxtrCICIKmiGJI4qwoxFrbtHCEEriioEqES5C bYABDBigABBlC66nEplbbjfgTEQxOKN+UxEp3IYzUcYWb8QxR5xEKKJHqob4YYg68qpjiymyMGQQ QNIIBI+9UrFll2yUKWeWLLgoggghhghEkUXq+IEqIn6YIxBA7LBFkToUeUWWSkSR5ZVaRCn+JZRM ppjBi1TsoMsI9u5gJhrrHJkFGEB+GCGGiza0YIQQUKtggthkGqrGnGTElCjaShTuxZourbFT3Wxs qVIXO91U09xOnPHEFV19FVUdZ6W11n+qmkuILYv46oo05CoiCznEMuuOK0hhRpVrrBmnHGYWSeOI LYSYTqwpeMxykTsIISSNSyxhExhRLBHlk0VIeWSWUETpgoZHVDHkjiyMIIKIPHjRwg5HOFGGmmUm I+iiRitITVILCiDRUxZpZLi44oJi9VQXI+ZUJxopdngnoDK+mKiLgTOVY5ZIHXljUG1FOWXkMoRu iBmKyOMsRbQwAg8hrBDNuzTwEKWWU7b+iUYTQxwRRZM6rujRiCm0ALPHI67IRBRK9OpvkWTizGSZ VwappJBAHCnkyFBgGUXeIYqwwpEi8eAEF2jMiccRGlQjqAIaRgCxgtYmjWm4jiGWeGIWMTa54VM7 5ni3Vl19kXGRS334UsAHh9U3FStXOXPNZYohhspokCsNXFbBYwv2TNdiiivuIAVZWnYxJpWtHHEE j7CusCKLKX4Q4ocpoCuC27i4UAQQ/WaRepZKZJFFlEEIuVYTU2qp5SwrhPDCEC60uKMWauLZhxsu aIhBtQoa3XAECyJFmNLIHbf85MYbX9jhxTHeNGP7B0cc/pGJ01/lSLY/AgZwcwdE4Ej+YmCEGvAo C0QohCjihTstdEENdzDELRhhClrYwhB7wMMjZqetQJAiDVu4nnrUc4UaaGE6u5vCIjJBrnEVIhnJ mEUrZDGLT9RhCozIRCZQ4Yg7yOEORYjXWZ4Rj3hAwxE2MMgICqCauxFsAhUYkfs49TjDdfFTBOxi Af2GP1j1j2JH+ZhvHkc5AfJvYfRLYBwPGANdJW0LoghEHbSwxyJooXSlMAUeaBeJO9gBD4fgXiHq YIhccKIOVqDZ2ebCEPpkgT7BI4QrQjELQgAiGMCIEzLA0Yo5UKIVnTxFIQiBicmoLQtcwUU9aqGF 8sVgfQSjAMEIZgFJUcpiGgOgxjz+BqphpqomqrLYMZE5G6GISijEfJisgGk/ngBucoHDJhovJ0du 1qp8PepRHahxifa8MnVZuIIh0HKWPJBFLGmo3R04cQgiYmkIXaDZtERAHx7JpQ5wAEQmQoGXHJJi GsiYhSw+YYmAVmIU1EGbIfK4FS08whZ1uFv6CrAaAayPlyMQQDdFehPFjdSkJ8WRC4ogmSFcwRHP EksRrnAFplliE444xB3AwwXbFQGnddiE0bZnFU6o0wgMIQLLflcEqQxCLM9DRjI+4Qtk7NASlchE KygxC7jorgqAqESTsHWKNGyIBjXQpQAOoDePUKB9KIVrALcZV7rWNSYq/cEJ5aD+iUJY52i42wIh QhEM6/h1e1ygpxYcoQvRWEcO5uAEIeRwJKtYR3djClNevQOIUADjF8AIRygMAQhQVKISlkBGD+NS BL0wpAhzwwMtMYAaChAgUgIowBU58la7ivSavQVucF0Cg8kYYQ5qc2yR0iCHRcBBE6rwghq68AXb CbKwz6gdWhrBiV0cghGK2EUtnqEIR9SiDj0CVpiWJodywSkYycgLHAjxiVcAoxI+1N3uqoKoGGwh tqkZwQEGQIGMEKwAGw2JcBW8YAabdJ9aMoSa8vjPO0hkaJtoEiR2mgc9pME6j0iF7fIgNEegYhGK sMUz1mEKR+SiiN7JL1Mzy5X+QXyCFrO4hCI+MQj4SLUOMaDlD0SQXuf4KQYVUKsAriiAXHqENQ2G cpSljLIfbGELaeAE9zopQ0KcdhalIMRZziJIPQaiwqs4RFjgVYdGNGIRqthFNzShCWgo8g63UJvp piAZhuyOC4voJCBaoQgeAsIXloCDWLTgoy2UwhGGgN5eYCCpChzA0iOorYf4NmVOd9rTT3HOFMQy 6jkAwkx10MSXdmeWOhRCEVzokhxQ8Qjb6YF2WrjpJozhjVyIQheFGEQgFuFIUmxCNE5a2hT2LAdA YAcryJsDVxetF9/51BZaKIQwTlEHTAxiBDTQm4BDECkCECCkfUumMz32W8z+CZOZyTkKU+b6aXon EAb0YXWiA70I7mSBqaIOj9HSQIrGqKKdGNQjHvLACGN0wxCYyAUm4KCIEgbRFpbIBCjOiwdDmMIU ioDDXlLn002U3BBbqCV6zVmfPd7BCCEIwQQYEKm7SaojJKpfqtTIRjXOm6Qrene9hb5gysRgNGnI SpjvIDTR1MEImghEFvhahzlcQguN4HgdtLU2PDwjH4eARC5CY2ZHXAIQqyhQKRaBB0ZYwhTqIHhj 27kFPYiGjzVgahAsyVp/h6BHNLibQaw4AhvQYAIXwXkByWijy/lcKfdr99Al39sMde5bEnHSIqO+ hWNtOxNq8iP31DkWImb+4RaY4MQ63GwL8xKiEHPQBAZHMQtMWIJnoWhELk5hijlIBg+reAQt54Ly zvVRLJuQQyYO1aMRXJECi5rA4Q0fAhuMIPGMg3wZI/8xa7IqU9jMXzDXLbFRBXPy578RDERQFxPf QU1zIO/RsrCJXIylFHLYYyCKYGbuNUIvQc2Ee1CFRciFYsCOO+iCRiAEQwgFS+i1RdCE1nGEZRCF TeCER8DAPNACHhGBzsE7yigC61CEQrCF5zmFTdCCChgBFyA8wKMBGGgywLu+GYEj/4mR7oumMCIj 8ctBkfmlM0K/IESRGKgBK8ADp6qD+xILxqgDTlgFvSiFQJADIWCEPCL+Inn6pzk4hbQgBfAyh0x4 HoVzBEbghVBYhVUQBVKwhUy4hEpwBUtwKrMIix9gGqoogs4hwiMCG0wABFOIBVjAhFEQRELAgiDw HMC7okSxPi0yoMVrphhZo79Jpp37n8PRn5BJI5MRwk3kDRioAUPwjkSDi3+arEfgBbGQIT3aAtLA A0W4g0xwhN4zGlLIhDs4BW14iDDDA0fjhJsqBC0AhBH8D2UAhUEYBTgMjFGLC34aAR8wC0OAg0vI o0KIBUw4BWGwBViIBWHAAhoIASyAQSVrvhm8RLkyx1fhIp4THMjLROPIOXXkxHhkCix4hDwSRbjI in8qvXNxqimABEP+yIINMoTHyAK00ARRqINTuIZnIYQ5WARAcL1MSLSnGgSskIVCsIQ/QEKuYYyF +IEQALI6pANCoINBSB1M8Lh+gQVYEAZhgIUimABFCQECaJRF7BvLmZ+eQ0fMyUkw2p/j8Mk2yjnD kceiVIpF0oJEA6gL2YsTEwtDOIUv8S9HqoNbYDNG8CFCCCJAqAVosAWuwo5MoIRmIwSAGoQLGcVR eIVKAISHqoPnMUlLUhT6qIIq8AEsyARM0J0f2ARYGId2EAZlCAZlqAZh0AIYaJQYOIBHmQCS6j6I KYrym6bJfKZskkzB6UEAmhzJnI2gM8rPfAm/+ie8oITdkSjrmAL+WoypORAITlAkF6sDNaCdOsAF augZJNQCQriEiqQOQEADKsiCKogDQGgTXNgErEg0SwqCIggBCSgCLJiCIMACJNwdxYqHdlAGZYCH cuDOQkgUCjArR0kw0CTP8syRfxo1UkCPKRi7OpADRzOEf5oCv3pIVXBFxVIFTNiFcTgFR9iEqYGD NCCEsYQDH4gCrKgC3wmTOnCFV0DPUbvDHwiCU+jPLChJsaiCaYPIU1iGcWAicghMi+CQEHg+JDPP E0VR5RALOYADURCHUHAEU3jLRyqqvtKLVQwzGZowfeGEa6AGXBgETCgeOUiDJMGOOKiDuiyCIIAD LLjLOrCFU3D+NoCKixeAgSS0lqTUox9whCAFhCOxhXmYh3agBmX4sUchCM+ptANI0TZ1U6aAg0EA tlkIhlq4qCxItE0gL+gJuTl4hDqwA4RUNoArhVzQBUC4A1wgBexIA3yMCziQgjCpgk0ohEtItEW4 hI5Ezh+LAd1sNjQIuQKFgSw4EL3QgmeQB3Qoh1OoyDtMFEXBNNx601mlVZgQllAoBFNIQ1dyD0Xa BEvVnUVtNZDzoSKYA1FghmUgBVXgBXixg7EAOTiIAzd4gyzwgSqog1ZohUr4g7FsNqygA6V8gSyA BacChDf4JzgAAizAhEuoAzqog2d4hnmoxqcqBHDckA2ZgCf+q9V+9dc7IyJHwAS97JETS7Ra6Ecu cIQseATk9KdM0AVb4AT9dKS4YJJTOC0kTFIsoANXaIVvMTs4qISyBNc6aIEgqAQlIQRB8ItCEIRX AIWyHDUtOIVYCIxLkIVYUKQ6gLmLmAAJ8NegpdUt1QIz24osEFBR85LWuopFKITk/IEsyARm4ARU WAVoQAsfUi8X2iPdiQIfEDVnG844wIppxVZMeIMWqIOxrAOWBYRB6NJLCJsjkYtASAZuw4QUgwZM WE6ZjLlNu0nILJwvKpnLBL9nKorDMVx4S1xKFFzPNI50S7ftq6bODDptqtzFNU/f2RCmARM6nI6H bC6psA7+UyASuSAEaCg2xojPOdCdYMHTUJVWNAgTJaGESaAEP3iDOPgDONhdQbCEOOgBQKCESugD QWDCkdVaF+oRQ+grWOAEbOiHfIAG5rSICkDTn1OY99kioLPBl6AfTIRH5KjBkhpfd+Re8TXf7vWf SvxeSkRRLNjSKbACZfsBONACRTCFUIiLq8sCV3va1LmDcaA9ReAaRWDLO3jbb0XPuLBWOriESagE AkVOrAAEWwAEOpjOdnUouB01OzyvPiqELLCFVyCHfugHeKgGvTREGFBBm4QJMyKcRkzf8DsZxzU/ 5mg8nqRhkKGc/tkm8cVMmnBfy51VsB3URLvDYLGFZPD+tx8orExQpGXchGUYBUDIVLNrNrbshOu4 LyqVgiqYAimghE+ghOKNA1CFAzTgNqfKgjlAzyBt15hSFC0oRNYatWWwB3uIB3hQhmcoBi2ggRKF 4VVZoxmmYSHu4ezLPuVoxxvcyfcVI3hU5MgTSjZyPHlUSqWMqSKoYolcGlQUhUSTiy2QA78gBEso BEwIhUvwBV9oBV/ohFbAjlGrgiyIgjiA5VboBNwFqESgA0BoVyQkBL8wrdMFMvnFgmDxIWCzBXSY B36AB3qgh2dYtPQpZMg5ZDOCxPa15AHiYRTRRG/uYZ0sR0mOn3PUZgNq5PKsgkSrAiwYzt0ZhBLE 3+n+kAM5KMF/AqfPW5MuLIRK+ARt/YW1LN7ZhYMq6AEpGAVfAAZX4NZBQNKxVMu3ReCK3ARL6DJA KMQrnYIqcA46CNe3HIRN4Adq7gd8iIfGCAKYowFGtMRIVMdRaedDbrdMXgruG2cbNmRhciObThxm CuKZjmTyPIM3WIgjqQMfoINQwMdR8zA+xIopCIRAKIRXOONQYMtitgRKUJJm6100qIIo6AEfiIFY hmUzzmBAwN3EyNTrgItme1sfggHiCgIfEAFgE4SSpIMWYyJ7QGFzGJoqCgGYPmd0Zuf/uabyXecU cd9z9rn1lWTJTuzFJmr4LU8kVdcp8IEfYGoKfgP+uZiCGEhXsWieV0AeU0iGSzjjSmgFS+gEtkWD Qe2BHnCBbG2FgXYFUKCEbk0EUEgTQOhlgBKEP0ADiIznF4iBKgiCF8ACjFSSe8WCU4CGeUDheOgH XcCFIPi28QRfdAxfTAbqoRbvouYNyoajeWPknxbKGvxm+K1p8owCKRBtZSPtCSZe7Pi3qf6BIakE cWnlWv6nZktXrJACH0Bw0t4E4nWF4p1ltu5qWDhjWJ4E7PADks5dIHgBsE3CS2jDkpwOR0AHJqqH fpAHQeCea2ZEoaaJAUpcy71BcP6+FtnMzA2cjenmIRagzpScY5KcUuHxyjxcFHUBHwCCH4hUqpj+ Aq0iUCR9V95UtqtaBlf4BFB4akfFCjSg3QP/gdtmyzJ27bGcBD/oA63O7Vim5bGkAqlO0BfoASzA A7BaWWyNAiy4BFvAhnnABmVoh1iAgRB4ARK9vscTWsz+uUK3lQwRbTGuimajhDh4Z2kFqC+dDiWZ BkuwBPSwYAqe6gLtbLOugzcABDcg3ldIhEToAywOBm3VVtMqXjeogkqAg1Uuu4G9BEGAhQymXb+w hFMgh3YYB2VAB0wgrubz7hiWN0Q39JtU9lp5AYboAb3YnR84VzeogzD2gUd11H+ShVZ4hWnQVkq4 qrF89DhAUrKW30LI6LftpDP+A0Ag20uA6AX+JltAOAOUnQS/kOjAcCq/GAVB8INApANMQIdzqAZ4 eAYweUEKYNNDv41mX/aehngdcQ5KWtIMwS/6jgq5wApLCgxAwIVX3lZH9+qrSjQ0CIL+PoWHswVC EONX0p24+NrlqUg4cANphfQ40NYDSTTPjQP5ndHpiINCaIaiF4ZmGARDjL7GnPim50QgULYqoIIg gAEg2IvpeN29+Ccnj1NKAIaokgUiEcUCP3DnWEuIBAQ/+ANLeMM6GE5ILYQKH0s/IPU3yOWA58M4 ELXZpQMqEGkkBQRYUIZmsIVpGHxb4FsYpACnZ/wglAI4EOk3oAL7VbasxwIt0Lu4mAIsmLj+b0AG YKCOAv8nKQiC0o8BOkiGXUbzVkgGVih3N5iCQSgeuof3NzgDKVhbK09SQRA1JK2EKRCEty2EUViG 7PRjZRCHdigHjDqyxnd+yQMCNOB8z040JI2x+ha1vaCPUZgGaog70db8QQ2TFnAEafgGX/iFV6Yq Xn50tx8Epx5LPrB9N/iDx6eEYKgElg0GStDNd4dXtwcIKZuqxYPXrhw0ce2qYYERo8K/iBInSgxA 8SLGjBonWqwY4ONHiiA3jtTYcSPKlCpXsmzp8iXMmDJn0oz4Jk6dKUWmDJqE8w0WLXDqZIHzpg4c LVnqDEJWbt0upD+05KxTJ4iPHzBCuLj+NOuXL1/HWlG6BIgQIDhwFnEDNAUOIEqUqAzCNGgQoVGg AF2yJIgOnaFUfAQ5FatWuXPz+L1rd06Zs0FbKaQMWVOl5X8nK3LWbDLiZpGhL5Mubfo06tSqI1Lx c7MKFUF1/tBJCwcn0qJDp0xRhMzVqUJZpvhA+hZNDx8vYMAIEsqSokqT4laqBCjtlDrNbNU5I6UO qEpwKMmKBavSJVODLlnFAgdLkB8/rMAS1gxbvH7z+jUTho1cLHSEMAJmq13U0UkhWYRgZQUa+CCE EUo4IUs9IFXHG2ppiBN8L7jggw9AaJHdG65IMwslbqEhBRxSSPHWFEBQQUcPcrgCTCv+rXQiV1y2 9RBHJb+0Aoog4Q1iySWVDGIdWoDUQUccQQ0WBAxFnKKKI83Mw406tjjiiBaOlLMJDBA1mNmCC3KU oGYIguQmZ2y26dmboGXWpmV3KrimnR6NRiGggQo6qEQsABFFDz380EMVaNwExxQ/TLFUFVhEqpYs tVyC1htVSHGdFHEAMogop5RlCSWvtPIJj9YZlegUlrgSyiCCLEnIH6Na8gdScAA2RRU+9BCDQzH8 UEchmGxyCjerOGKVFqjEUgcQIWDG4Eh7ekZnntsy6C24cYLWWWhyfnvntuH+SSi77bpLmgaGJsfc D7f56ukUQWCRhVCXBAPLK68Agqr+W0ACEgUl06V4HcNpAfIHHBDDgdUPaICDjCytVNJJK35MUgla 08VBBRVALHVGDCk7pMUmVqmChxaGgFkEF3dhgQUNDqo7Lp7fpptgSWqmSyfPcx7op8/lkrvuu007 /TRKGrTAAhXDGhvEFFJU0eIUOFVBiC3B/OLKK8nINckbb1BxXRR+5DhJH2+kqOEbgCSioRQgvjAF MsBs4odsQ9WBxZNIBQFkHbYO0oNDQFTx5B11OBIUFoUMssUIRRQRBA0ENsjzuUUXLefQRP8Mumgi dba60qKjCzXssT/twg8usPDCD8oRFoQUw/0QBB2uJPMLMMDIMss0PN7GYlq2uDL+ti85VkLJjg+7 UUUcUuTOXBCWX0cHbXFAGQcacZyBhV10DPIkHS9Y+AcmdGShl1t1ZIJJFkVUsLkW1n6+MwDNNa7Q nUtopyudAL2lJzsJjWmyeyAEJxQD7f0gZSmTTwUrWIUelC1gwPBF2YLRim8gAxBZ+BQobGGJ6wxl EA7jEcPgMBgPuYAKlmjFriaxMLjcxg/ouYQLBwE4vBQCMI6A0idacYns1CF+WAgBEPTlv8rkSUFp 6hOeelYno22Ri35iYJzQtBltAU2BDowgGtOImhYYq4IhIBaIskIsTCTDFZ+QBTBccTxZgKOP4fBD dqyTIxxa5zpWuc0f/vAGNBD+IkmJqEIMCtGJP8glEX4ARNre4AY3DMIVl4DFkKrDBz8QYgqDeZYg lBQlOtThB0uhAg1CQAPKxG5dZ1QjLnOpywoYywchaMEIYkAYYmnBFMoARSt8Qbbj/QIc4bDFxX5R N6sQYlWd8EUlctUH6cQhbXWAhSAoUYeEXaIVk/iDH/wQt3S6AQ5RmAImrOKKUehoEpZAZnX8YLk4 tAIQo6BDsJKTBUxMIQQv8BxJ3qTQhTK0oQ59KEQjKtGJ3lKXFr1oRiw4gglUEAYi8EERYiGNYKhq VrIgXjKQEY5yyOJ5rzhDHMJzTR2RRS1vmA4c0ECFOkyCFersUcI6ocM/JKL+D244AxWqAAQlDeIV SaJEJVyho0G04g998MMfquMKtxTBBzHIwiCoEIQYILSWCsUoWtOKxhaACAYTkAAMvBqEUVCDGsBQ 1SVG8QpZiEMa4ihHXznWipuygnqfYMUkdpTVXEGsCnTowyR8Aoi49SFhifAEH27ihjhEIQhRcAEm ErExPywJEEqSTeJa0Yvq5MoPdKCCFpJqlSLQYAJqvS1ucwsoF1hQAgwIgXIGEY52zGKQoijnNKgh j3mIQxzhiMsforCq61ACsnCIGyC6+c63ZPaogHBD2u4W2bidIQpRqJpyRlEJWLDimp1IhCDI56tX /OIXndhYJQSBiUIUAj7+RcDCa7Wg2wETuMCXMRYMRsAVr/ogGOKQRSUsEQySfsIWuOhHPQA7jbS4 KFaZMIU5+/AwTcLBDWhADhUmUVRM2kZuiYBb3NTCG0TBIYlSpQQofMGKSgCmB3QQmy8+9rFKvGIU l3itfjdRCEMYuMlOfrJGXLCVrcQAWdUIhi0okaN2sGMatrgHPvoBj3JMww/vGQRvAtaKUMiFbqEI xSgAAQqCIscH5YtD9oySlh50+AdSQIMPFGHfTwiVFeb0Q9XQYIlfIKNjkQUZkWARC2y0AxNaKIKA oazpTRf4BSEIQQ/wQAhS2CIWy/hEJYDRCWrIAhC76Ac9yjEPXThMGsr+CAYdQtGKQYznEq+wRSiq UIlW3FiT513Rn6UAXk26QQo9AEIPEJWociaiFb9gBbb54IYsCMIVz6uvxiQtDHmQAxcibUz8fhAT AFyE3RRx90TgLRF5R4Te/7A3vtut73fvO979nve/6x3wew883/w+uL8RDnCFC5zhBHe4wRMu8YVP vOEVf/jFDR6DQdjiFMiaBSySgUwcAiIYd3h1PMjBDVwU0hbKCMc0piEkju0oFEKCwzRylKJQxO0N fjgDINQp4rjFYZPmRZQPWiCKhPWUY26gAhoE8YpwWBsZrHAFLPpjC0wAURjCUAYsRjGKQrwEAPQ2 +7vPrva0sz3ea3f+e9vn/Xa5x73ec7d73e99d73nHe1w/zvdAY93wfOd8H4PPOIHn/jCL/7win88 4yF/ePvFE2y1MEcwpIpDaPhDH/awhzq6kQv1wiEUMZ/G8BrtirDk6BvfSEQzLAEdNsPPEZsqhO0n QYdM4m0ohQiCVDsh/KIOJg7AqIQ0GG11+BWCKUCMhTCqMYil1CEWLXF85LOP/e3vnft97z74vy9+ w4ef/ONvfPnRf37Jp5/969c+vQkhCquMQhbLIGnAlDEPaKhDHeZoxikMwixsyiDIQpcFAzv00TeA wwIigy+EQzjggi0o2R0QAhwIwiZsgrI8QrJgwk6RjFI5Gx3EU13+mFMlWJLaBAFVUYIrhEPyXYIg iNUl2MUofJ1bBAEVYMEgsETEUZwPWtwPYlwQ9iAQFqEQGiERHqESJiETFpwTQtwTZlwUDuEUImEV LiFG1IHDXIItzIKqJMMznILGVMLIXYclvEItwII4KEM5tIM4IIPr/cIxfENhIUMwWKBVRM5OBQak 1EHkqAINZMEdjAyf0QUgIYtVCFEdxEERYYItjEIwVAOPJUeIBEUd2EI1EEIVxMjjYML1tR/8vZ/3 mR8pql8put8phmIqjqIptiIquqIqwiIrviIt/gMVDEUozIIshEKF2QM36NwfgIIlrA9TEAIlKMM0 lEM82EIy0SH+YiXWTFHCCu3XJjyDLrjBHzRbDyzFHQDBk2jBG0jBG0RBdZxB+VTBUsSBIFxCIUCf 1/0THQhLFWBFDBSGMqCZD0zBvmCCJ7rELMZiLf6jQILiQIoiQR6kQSbkKiLkQipkK/7AJroCScnC M/RDmBVPJwBCRl4IIIiCMohDPIhDwATDMfhCLxRWKyRCIuxIMgVDIQSClTRDdSRCHACLsJCMKe2C I8RBxKSTH0gBTKFBEBCFFHCcMMBC8xWBpCQKcxRCJeSED2RBUo3gDsJEE0IhVkphVlLhVlphV2Lh V16lVo4lV5KlV5olWKLlVVIBnpENLsQDPdTDOFTD2MAFKKBnQU7AgSiIAzmUAyy4XDlIg+uRBdww DCWIBStYRRU4gi48Wh+cQYscDrDUwTPYAjKZBSBAia/4QRSIjx+4QRMVAqQEARCECONMgZEBlCmx Uh2wB6e9JmzGpmzOJm3Wpm3eJm7mJm0GBAA7 --0__=08BBE626DFEC79028f9e8a93df938690918c08BBE626DFEC7902-- From jeff@k2.com Tue Jan 21 22:29:03 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0M4T2c32661 for ; Tue, 21 Jan 2003 22:29:02 -0600 Received: from Klee ([192.168.0.25]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h0M4Ulf26548; Tue, 21 Jan 2003 23:30:47 -0500 From: "Jeffrey Kay" To: "'Barbara Menzel'" , Subject: RE: [Metakit] Question on Views Date: Tue, 21 Jan 2003 23:30:47 -0500 Message-ID: <002a01c2c1cf$09617c50$1900a8c0@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal In-Reply-To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've been using MetaKit very successfully with VC++ and haven't had any trouble with it. If you're having compilation errors for passing in c4_View, it's likely that you didn't include the metakit header files in the right place. What I usually do is a forward class declaration in my header file and then include the metakit header files in the cpp file. So your .h file would have class c4_View; class myclass { myclass(c4_View &view); ~myclass(); }; And your .cpp file would include the files. If you want to post or e-mail back more details, I'll see if I can help you. Cheers -- jeffrey kay personal weblog pgp key "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of Barbara Menzel > Sent: Tuesday, January 21, 2003 6:20 PM > To: metakit@equi4.com > Subject: [Metakit] Question on Views > > > > Hello! > We are using MetaKit with Visual C++. Often, we find there > is a need to > initialize or perform some initial action on a view within a > new class. > I've tried passing the view into the object via the > constructor and get > several compile errors, primarily, c4_view is not a recognized type. > However, passing a view as a parameter in a member function, > there are no > errors and everything works fine. The view can even be > updated within the > member function and returned with the updates included. Has > anyone tried > this or something similar with any success? > (Embedded image moved to file: pic07483.gif) > From jcw@equi4.com Wed Jan 22 03:04:30 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0M94Tc15522 for ; Wed, 22 Jan 2003 03:04:29 -0600 Date: Wed, 22 Jan 2003 10:06:15 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.551) Subject: [Metakit] libtool Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have a question... For some reason I do not quite understand, MK builds shared libs with "ld". This completes and works as expected with C++ programs, but it causes runtime errors when loaded from a C main (for example Mk4tcl.so loaded from tclsh). It may also breaks down even with C++ in Unix systems which do not support shared library back-linking. The libtool 1.4.1 info docs sound ominous: Writing libraries for C++ ========================= Creating libraries of C++ code should be a fairly straightforward process, because its object files differ from C ones in only three ways: 1. Because of name mangling, C++ libraries are only usable by the C++ compiler that created them. This decision was made by the designers of C++ in order to protect users from conflicting implementations of features such as constructors, exception handling, and RTTI. 2. On some systems, the C++ compiler must take special actions for the dynamic linker to run dynamic (i.e., run-time) initializers. This means that we should not call `ld' directly to link such libraries, and we should use the C++ compiler instead. 3. C++ compilers will link some Standard C++ library in by default, but libtool does not know which are these libraries, so it cannot even run the inter-library dependence analyzer to check how to link it in. Therefore, running `ld' to link a C++ program or library is deemed to fail. However, running the C++ compiler directly may lead to problems related with inter-library dependencies. The conclusion is that libtool is not ready for general use for C++ libraries. You should avoid any global or static variable initializations that would cause an "initializer element is not constant" error if you compiled them with a standard C compiler. There are other ways of working around this problem, but they are beyond the scope of this manual. Furthermore, you'd better find out, at configure time, what are the C++ Standard libraries that the C++ compiler will link in by default, and explicitly list them in the link command line. Hopefully, in the future, libtool will be able to do this job by itself. My question is: would anyone have a suggestion how to deal with this in the most portable manner? I tend to use MK mostly in static-linked form, but evidently it would be nice to make this work in the most general way possible. The current CVS sources have a "-lstdc++" added to LDFLAGS, which solves it for Linux, but generate the following output on MacOS X (both are gcc 3.1/3.2): *** Warning: This library needs some functionality provided by -lstdc++. *** I have the capability to make that library automatically link in when *** you link to this library. But I can only do this if you have a *** shared version of the library, which you do not appear to have. *** The inter-library dependencies that have been dropped here will be *** automatically added whenever a program is linked with this library *** or is declared to -dlopen it. g++ -dynamiclib -flat_namespace -undefined suppress -o .libs/libmk4tcl.dylib mk4tcl.lo mk4too.lo column.lo custom.lo derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo -lc -install_name /usr/local/lib/libmk4tcl.dylib The resulting library does load in tclsh. Should the conclusion be to throw out libtool altogether? Frankly, I wouldn't mind one bit... -jcw From jcw@equi4.com Wed Jan 22 03:11:08 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0M9B7c16037 for ; Wed, 22 Jan 2003 03:11:07 -0600 Date: Wed, 22 Jan 2003 10:12:53 +0100 Subject: Re: [Metakit] Question on Views Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Barbara Menzel wrote: > We are using MetaKit with Visual C++. Often, we find there is a need > to > initialize or perform some initial action on a view within a new class. > I've tried passing the view into the object via the constructor and get > several compile errors, primarily, c4_view is not a recognized type. That's a typo, I assume: c4_View, not c4_view, right? > However, passing a view as a parameter in a member function, there are > no > errors and everything works fine. The view can even be updated within > the > member function and returned with the updates included. Has anyone > tried > this or something similar with any success? Could you post a brief extract of the code you would like to get working? I'm having trouble understanding exactly what part is not doing what you expect. -jcw From lvirden@cas.org Wed Jan 22 07:45:18 2003 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0MDjIc01718 for ; Wed, 22 Jan 2003 07:45:18 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.5/m8.12.5/CAS_MAIL_HUB-2.00) with ESMTP id h0MDkvj5026586 for ; Wed, 22 Jan 2003 08:46:58 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id h0MDkvs17717; Wed, 22 Jan 2003 08:46:57 -0500 (EST) Date: Wed, 22 Jan 2003 08:46:57 -0500 (EST) From: "Larry W. Virden" Message-Id: <20030122084657.AAB17689@cas.org> Subject: Re: [Metakit] libtool In-Reply-To: of Wed, 22 Jan 2003 10:06:15 +0100 To: Metakit mailing list Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler 1. Because of name mangling, C++ libraries are only usable by the C++ compiler that created them. This decision was made by the designers of C++ in order to protect users from conflicting implementations of features such as constructors, exception handling, and RTTI. LWV> Note that you should take this statement literally - that is to say, LWV> even a different VERSION of the same C++ compiler is likely to have LWV> problems with the library! This issue has nothing to do with libtool... -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jyl@best.com Wed Jan 22 09:46:11 2003 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h0MFkAc10959 for ; Wed, 22 Jan 2003 09:46:10 -0600 Received: (qmail 73676 invoked by uid 19667); 22 Jan 2003 15:47:57 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 22 Jan 2003 15:47:57 -0000 Message-ID: <3E2EBC96.8020306@best.com> Date: Wed, 22 Jan 2003 07:45:26 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: Metakit mailing list Subject: Re: [Metakit] libtool References: Content-Type: multipart/mixed; boundary="------------050807060200060806010800" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------050807060200060806010800 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Jean-Claude There are a lot of issues mixed up together in your email, and the libtool docs don't help to clarify. The major issue is that linking a C++ shared library with 'ld' does not always produce something that will dlopen successfully into a process (Note that dlopen is used both at runtime and by ld.so at process startup time, so if it links and the process starts up, you can also be sure it'll dlopen at runtime successfully). If you use a Makefile generated by configure then you can't get rid of libtool, I think it is used whenever you're asking configure to generate a Makefile for creating a shared library. And it's always generated even if not used. Since I did not want to use libtool, a tool I don't understand and which comes with that dire warning about C++, this is the primary reason why I use my own Makefiles for e4Graph and only use a Makefile.include that is created by configure, for configuration options. I've never had any issues loading e4Graph shared objects into tclsh or any other executable written in C, C++, even Kylix (Borland Pascal). Anyways, I've attached the Makefile for e4Graph to show how I link the shared library. Check line 183 or so for the details. There's more examples and a working implementation of using configure only for configuration (as intended) and not also for generating Makefiles, in the distros. (http://sourceforge.net/projects/e4graph). Jean-Claude Wippler wrote: > I have a question... > > For some reason I do not quite understand, MK builds shared libs with > "ld". This completes and works as expected with C++ programs, but it > causes runtime errors when loaded from a C main (for example Mk4tcl.so > loaded from tclsh). It may also breaks down even with C++ in Unix > systems which do not support shared library back-linking. > > The libtool 1.4.1 info docs sound ominous: > > Writing libraries for C++ > ========================= > > Creating libraries of C++ code should be a fairly straightforward > process, because its object files differ from C ones in only three ways: > > 1. Because of name mangling, C++ libraries are only usable by the C++ > compiler that created them. This decision was made by the > designers of C++ in order to protect users from conflicting > implementations of features such as constructors, exception > handling, and RTTI. > > 2. On some systems, the C++ compiler must take special actions for the > dynamic linker to run dynamic (i.e., run-time) initializers. This > means that we should not call `ld' directly to link such > libraries, and we should use the C++ compiler instead. > > 3. C++ compilers will link some Standard C++ library in by default, > but libtool does not know which are these libraries, so it cannot > even run the inter-library dependence analyzer to check how to > link it in. Therefore, running `ld' to link a C++ program or > library is deemed to fail. However, running the C++ compiler > directly may lead to problems related with inter-library > dependencies. > > The conclusion is that libtool is not ready for general use for C++ > libraries. You should avoid any global or static variable > initializations that would cause an "initializer element is not > constant" error if you compiled them with a standard C compiler. > > There are other ways of working around this problem, but they are > beyond the scope of this manual. > > Furthermore, you'd better find out, at configure time, what are the > C++ Standard libraries that the C++ compiler will link in by default, > and explicitly list them in the link command line. Hopefully, in the > future, libtool will be able to do this job by itself. > > > > My question is: would anyone have a suggestion how to deal with this > in the most portable manner? I tend to use MK mostly in static-linked > form, but evidently it would be nice to make this work in the most > general way possible. The current CVS sources have a "-lstdc++" added > to LDFLAGS, which solves it for Linux, but generate the following > output on MacOS X (both are gcc 3.1/3.2): > > *** Warning: This library needs some functionality provided by -lstdc++. > *** I have the capability to make that library automatically link in when > *** you link to this library. But I can only do this if you have a > *** shared version of the library, which you do not appear to have. > *** The inter-library dependencies that have been dropped here will be > *** automatically added whenever a program is linked with this library > *** or is declared to -dlopen it. > g++ -dynamiclib -flat_namespace -undefined suppress -o > .libs/libmk4tcl.dylib mk4tcl.lo mk4too.lo column.lo custom.lo > derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo > std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo -lc > -install_name /usr/local/lib/libmk4tcl.dylib > > The resulting library does load in tclsh. > > Should the conclusion be to throw out libtool altogether? Frankly, I > wouldn't mind one bit... > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > --------------050807060200060806010800 Content-Type: text/plain; name="makefile" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="makefile" # makefile -- # # Generic Makefile for building e4Graph on UNIX with GNU C++. # # Author: Jacob Levy, jyl@best.com # # Copyright (c) 2000-2003, JYL Software Inc. # # Permission is hereby granted, free of charge, to any person obtaining # a copy of this software and associated documentation files (the # "Software"), to deal in the Software without restriction, including # without limitation the rights to use, copy, modify, merge, publish, # distribute, sublicense, and/or sell copies of the Software, and to # permit persons to whom the Software is furnished to do so, subject to # the following conditions: # # The above copyright notice and this permission notice shall be # included in all copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE, EVEN IF # JYL SOFTWARE INC. IS MADE AWARE OF THE POSSIBILITY OF SUCH DAMAGE. # Include the parameters determined at configure time: include ../../all/makefile.include # Which libraries to link with the libe4graph: LIBS = -L$(METAKIT_LIB) -lmk4 # The directories containing the source code: INTF = ../intf IMPL = ../impl # The Metakit driver directory: MKDRV = ../mkdriver # The directory containing the include files: INCLUDE = ../include # What are the sources for e4Graph: CPPSOURCES = \ $(MKDRV)/mkstorage.cpp \ $(INTF)/e4refcount.cpp \ $(INTF)/e4id.cpp \ $(INTF)/e4node.cpp \ $(INTF)/e4storage.cpp \ $(INTF)/e4vertex.cpp \ $(INTF)/e4nvisitor.cpp \ $(INTF)/e4svisitor.cpp \ $(INTF)/e4vvisitor.cpp \ $(IMPL)/i4drivers.cpp \ ${IMPL}/i4dstring.cpp \ $(IMPL)/i4intstack.cpp \ ${IMPL}/i4node.cpp \ ${IMPL}/i4storage.cpp \ ${IMPL}/i4vertex.cpp \ ${IMPL}/i4hash.cpp # What include flags to pass to the compiler: INCLUDES = \ -I${INCLUDE} -I${MKDRV} -I${METAKIT_INC} # The object files: OBJECTS = \ mkstorage.o \ e4refcount.o \ e4id.o \ e4node.o \ e4storage.o \ e4vertex.o \ e4nvisitor.o \ e4svisitor.o \ e4vvisitor.o \ i4dstring.o \ i4drivers.o \ i4intstack.o \ i4node.o \ i4storage.o \ i4vertex.o \ i4hash.o # What optimization options to pass to the compiler: CXXFLAGS += $(INCLUDES) -DE4_DLL -DDEBUG # Main target: makes libe4tree.so: all: libe4graph.so # Install the library: install: all -@mkdir -p $(includedir) cp -p ../include/e4graph.h $(includedir)/e4graph.h -@mkdir -p $(libdir) cp -p libe4graph.so $(libdir)/libe4graph.so.0.0.0 -@/bin/rm -f $(libdir)/libe4graph.so $(libdir)/libe4graph.so.0 -@ln -s $(libdir)/libe4graph.so.0.0.0 $(libdir)/libe4graph.so -@ln -s $(libdir)/libe4graph.so.0.0.0 $(libdir)/libe4graph.so.0 # Uninstall the library: uninstall: /bin/rm -f $(includedir)/e4graph.h /bin/rm -f $(libdir)/libe4graph.so.0.0.0 /bin/rm -f $(libdir)/libe4graph.so /bin/rm -f $(libdir)/libe4graph.so.0 # Release engineering: release: all mkdir -p $(release_tmp_dir)/include cp -p ../include/e4graph.h $(release_tmp_dir)/include/e4graph.h mkdir -p $(release_tmp_dir)/lib cp -p libe4graph.so $(release_tmp_dir)/lib/libe4graph.so.0.0.0 cp -p libe4graph.so $(release_tmp_dir)/lib/libe4graph.so cp -p libe4graph.so $(release_tmp_dir)/lib/libe4graph.so.0 # Individual file targets: i4hash.o: ${IMPL}/i4hash.cpp $(CXX) -c $(CXXFLAGS) ${IMPL}/i4hash.cpp i4drivers.o: ${IMPL}/i4drivers.cpp $(CXX) -c $(CXXFLAGS) ${IMPL}/i4drivers.cpp i4dstring.o: ${IMPL}/i4dstring.cpp $(CXX) -c $(CXXFLAGS) ${IMPL}/i4dstring.cpp i4intstack.o: ${IMPL}/i4intstack.cpp $(CXX) -c $(CXXFLAGS) ${IMPL}/i4intstack.cpp i4vertex.o: ${IMPL}/i4vertex.cpp $(CXX) -c $(CXXFLAGS) ${IMPL}/i4vertex.cpp mkstorage.o: ${MKDRV}/mkstorage.cpp $(CXX) -c $(CXXFLAGS) ${MKDRV}/mkstorage.cpp i4node.o: ${IMPL}/i4node.cpp $(CXX) -c $(CXXFLAGS) ${IMPL}/i4node.cpp i4storage.o: ${IMPL}/i4storage.cpp $(CXX) -c $(CXXFLAGS) ${IMPL}/i4storage.cpp e4refcount.o: $(INTF)/e4refcount.cpp $(CXX) -c $(CXXFLAGS) $(INTF)/e4refcount.cpp e4id.o: $(INTF)/e4id.cpp $(CXX) -c $(CXXFLAGS) $(INTF)/e4id.cpp e4vertex.o: $(INTF)/e4vertex.cpp $(CXX) -c $(CXXFLAGS) $(INTF)/e4vertex.cpp e4node.o: $(INTF)/e4node.cpp $(CXX) -c $(CXXFLAGS) $(INTF)/e4node.cpp e4storage.o: $(INTF)/e4storage.cpp $(CXX) -c $(CXXFLAGS) $(INTF)/e4storage.cpp e4vvisitor.o: $(INTF)/e4vvisitor.cpp $(CXX) -c $(CXXFLAGS) $(INTF)/e4vvisitor.cpp e4nvisitor.o: $(INTF)/e4nvisitor.cpp $(CXX) -c $(CXXFLAGS) $(INTF)/e4nvisitor.cpp e4svisitor.o: $(INTF)/e4svisitor.cpp $(CXX) -c $(CXXFLAGS) $(INTF)/e4svisitor.cpp libe4graph.so: $(OBJECTS) $(CXX) -shared -o libe4graph.so $(OBJECTS) $(LIBS) # Clean: clean: -@/bin/rm -f ${OBJECTS} -@/bin/rm -f *~ #* -@/bin/rm -f libe4graph.so --------------050807060200060806010800-- From pknight@eurologic.com Fri Jan 24 11:02:22 2003 Received: from best.eurologic.com (no-dns-yet.demon.co.uk [80.177.61.148] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0OH2Gc15513 for ; Fri, 24 Jan 2003 11:02:17 -0600 Received: from eurologic.com (budvar [192.168.7.63]) by best.eurologic.com (8.9.3+Sun/8.9.3) with ESMTP id RAA01654 for ; Fri, 24 Jan 2003 17:04:01 GMT Message-ID: <3E3171DD.30409@eurologic.com> Date: Fri, 24 Jan 2003 17:03:25 +0000 From: Pat Knight Organization: Eurologic User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021130 X-Accept-Language: en, en-us MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Blocked/Ordered views with Mk4py using 2.4.8 on Linux Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm trying to use an ordered view on top of a blocked view but keep getting a segmentation fault. Before I start reporting bugs, please could someone have a quick look at some sample code and see if I'm setting up the views correctly. The code is as follows: import Mk4py as mk ORDERED_ONLY, BLOCKED_ONLY, BOTH = range(3) option = BOTH filedb = mk.storage() if option == BOTH: words = filedb.getas('_B[k1[k2:I,v:I]]') blocked = words.blocked() ordered = blocked.ordered() ordered.append(k2=2,v=3) elif option == BLOCKED_ONLY: words = filedb.getas('_B[k1[k2:I,v:I]]') blocked = words.blocked() blocked.append(k2=2,v=3) elif option == ORDERED_ONLY: words = filedb.getas('k1[k2:I,v:I]') ordered = words.ordered() ordered.append(k2=2,v=3) When option == BOTH, I get the segmentation fault. When it is either of the other two values, there's no seg fault. I'm not expecting anyone to diagnose the seg fault yet - I'll investigate that further myself if people say this code should work. -- Cheers, Pat Tel: 0117 930 9621 (Intl: +44-117-930-9621). From bmenzel@fs.fed.us Fri Jan 24 13:41:46 2003 Received: from rmaccsv2.rmacc.r2.fs.fed.us (rmaccsv2.rmacc.r2.fs.fed.us [166.2.208.11]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0OJfjc28793; Fri, 24 Jan 2003 13:41:45 -0600 Received: from sv10.r2.fs.fed.us by rmaccsv2.rmacc.r2.fs.fed.us (USDA-FS/LakewoodCO/3-03-00) using ESMTP; id MAA23634 for multiple recipients; Fri, 24 Jan 2003 12:43:29 -0700 Subject: Re: [Metakit] Question on Views To: Jean-Claude Wippler Cc: Metakit mailing list , metakit-admin@equi4.com X-Mailer: Lotus Notes Release 5.0.3 March 21, 2000 Message-ID: From: "Barbara Menzel" Date: Fri, 24 Jan 2003 12:43:58 -0700 X-MIMETrack: Serialize by Router on ENTR2B/E/USDAFS(Release 5.0.11 |July 24, 2002) at 01/24/2003 12:43:29 PM MIME-Version: 1.0 Content-type: multipart/mixed; Boundary="0__=08BBE62BDFF8F92E8f9e8a93df938690918c08BBE62BDFF8F92E" Content-Disposition: inline Apparently-To: Apparently-To: Apparently-To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0__=08BBE62BDFF8F92E8f9e8a93df938690918c08BBE62BDFF8F92E Content-type: text/plain; charset=us-ascii Here's an example of the type of constructor code I'm getting errors on... CAreaBased::CAreaBased(c4_View vJustMeasured, c4_View vTreeTable) { int nRows = vTreeTable.GetSize(); if(nRows > 0) vJustMeasured.SetSize(); } It is appropriately prototyped in the header file and when the object is instantiated, there would be a view (vJustMeasured) equal in size to the tree table which is then visible to the class CAreaBased. Using c4_View in the constructor parameter list is what generates the unrecognized type error. (Embedded image moved to file: pic11773.gif) Jean-Claude Wippler To: Metakit mailing list cc: Sent by: Subject: Re: [Metakit] Question on Views metakit-admin@eq ui4.com 01/22/2003 02:12 AM Barbara Menzel wrote: > We are using MetaKit with Visual C++. Often, we find there is a need > to > initialize or perform some initial action on a view within a new class. > I've tried passing the view into the object via the constructor and get > several compile errors, primarily, c4_view is not a recognized type. That's a typo, I assume: c4_View, not c4_view, right? > However, passing a view as a parameter in a member function, there are > no > errors and everything works fine. The view can even be updated within > the > member function and returned with the updates included. Has anyone > tried > this or something similar with any success? Could you post a brief extract of the code you would like to get working? I'm having trouble understanding exactly what part is not doing what you expect. -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit --0__=08BBE62BDFF8F92E8f9e8a93df938690918c08BBE62BDFF8F92E Content-type: image/gif; name="pic11773.gif" Content-Disposition: attachment; filename="pic11773.gif" Content-transfer-encoding: base64 R0lGODlhrgF5APcAAAAAAAAA/yk5OSk5SilCKSlCSilCWilKOSlKcylSYylaYylaczExMTFShDFS jDFapTFjhDFjlDk5KTlCOTlKKTlKSjlKWjlSczlaYzljc0IpEEJapUJjhEJjlEKErUKEvUKEzkpK OUpKWkpSSkpjc0prhEprlEprpUpzvUpzxkpz1kp7rVIxGFI5KVI5OVJKKVJKOVJKSlJKc1JSa1Ja SlJaWlJjWlJja1Jrc1KErVKExlKMvVKMzmNCMWNCQmNKUmNSMWNSQmNSY2NSa2NSc2NaUmNjY2Nj c2NjhGNjjGNzlGNzpWN7zmOErWOEvWOM1muEpXNCKXNCQnNKUnNSMXNSQnNSY3NSa3NaQnNaUnNj UnNjY3NrY3Nrc3NrhHNrjHNzpXN7lHN7tXOUxnuEvXuMrXuMxoRKMYRKQoRaY4RjhIR7pYSUrYSU xoxKMYxKQoxSUoxaQoxaY4xaa4xjQoxjUoxjhIxrY4xrc4xzhIx7hIyEjJxKMZxKQpxaOZxaSpxa Upxja5xrQpxrSpxrWpxzWpxzc5xzjJylxqVaMaVra6Vrc6VzhKVzjKV7c6WEhKWElKWMlKWMnK1a Qq1aUq1rSq1rWq1zUq17WrVza7V7jLWEc7WMjL1aSr1jMb1jWr1zUr1za71zc717Ur17hL17lL2E c72Mc72MnL2Mpb2UjL2cpcZjQsZjUsZzSsZzWsaEWsaMWs57a857c857lM6EhM6Mc86MnM6UhM6U nM6ljM6lnM6lrdZjStZjUtZzStZzWtaEWtaUWtaltdaltdate9a1xt5aQt5zUt57a96Ua96UhN6U lN6lc96ta96tjN6tnN61reeEWueEa+eUhOecWuecpeelreete+e1ree9ree9xufGpefOvefe7+9r Uu97Uu+EUu+Uc++clO+lc++ljO+tnO+1a++1hO+1ve+9pe/OzveUWvelc/elrfetUve1jPe1nPe9 e/e9jPe9pffGnPfGpffGvffepffetffvzv+1rf/GhP/Opf/v1v///yH5BAAAAAAALAAAAACuAXkA AAj+AP8JHEiwoMGDCBMqXMiwocOHECNKnEixosWLGDNq3Mixo8ePIEOKHEmypMmTKFOu8JCDB48P Hj6AAPGBx46bMXes+LDDAw+aPnXocJnDw5McN3v2dLkjRw4dHnSgOOGhatUTK7GuXHGiq9aqK1Z6 4OqVrFevOVbkOBs2pdu3cOPKnXtR7VEdJ4QKnQmTB4oUQp/YZIKUh4eZPkHsePJEx00nPJ44ASH0 JQgUHz5MPYFia+eYH8SGxooVBdjOWsuuLKrjyZK0ZNXSnU27tu3bHJHu4LnDJo+nLn9D/vDUKuUc KU4gH7vTKgrYOrhGT94VBU2ZJ5KvAOFBa9YTmsH+OgnLdcVmqkVXNFniZAkUKG2oql2Cu779+/hr rxj6JIVLJk4wAeBieukAQloq/GXdDimEpd12KaggoWk7pWAhVxb+ZWEKKGCWwgcpgCChTMxxpsIT Y4iBVRMsOuFEDiyOUcYSXrlnggldsZjfjjz26KNGK/DQ4AqABReZCiztIJWE3KHgmA5OCOZEVAdO FRZmKoDAYYearSBhTFOpYCGIoZnWFU+XoUBGG22Q4YQZMJK3hHpskLHGCUoskeeNJXjFEAAGAVqQ oAQROpChAiH6j6KMBurooI8WGumhkyZa6aKXNgrpppJySqmnloKKqaiadmrqp6eGmuqoq5Z6UAn+ +znR2wdYpeDBYpEdtUJTjanAQ5D7PQXlCgB+8CSIKOikJWbMNihTmR78NVOHyY6FAhRLsBfWnOqt VWMba5SBCBRsQKHEjR10YEJDACDa7qDuxgvvvIXKWy+9h9qbL76J6tsvv4v6GzDA795r8L4H/5vw wAsXjPDDCkPMsMQOR2zxxBdXTFATkoFgHmBbMsjYUL86JZRTaqXnMQoqeLlWWE85kUJRM82kArK2 RkttVTlf2cQJ2HYVlhNe3WjCCiYsEYYJSoQFhbc3nvCQxlQLXDXBVmeN9dYNa9011xR7HTbYGYtd NtkYp331wv+Y0ATRlzX1W1ROtRQrE7uFlV3+Wg3qUJOt5un9XHp+V1sUb1J1tlJmO61AI41l/NxV GzR2haMJHXTlXhmYW95BBhlw0EFErqqK6ummp84q6qurXnrrsL8ue6a0k1p7q7ezPrvtvOPeO+qw kmeeCro18Vtaz60gM1WlnfVVZ2YCrlpsXrGkllVdIf9zecolPaNXHXBwggkllLAGGDiSAHoGJEDA QQQcTG222vOvPfbX+N+v/9n5878//Wizn/8GCMD+FfB//9CKlZyUFp1oBWUnIJq3zuM86gkNQ3oj Dfe45zznRa1zOOKK0cKXARyxAQzly5wJOJAB9zVgAeyqnwwDOEMDCvCABLyhDmv4vx3S8If+Npzf QI7mN6E0YT8naIIOkCOUqRzBBuU7mgbZwpnYcKUzl9lPWKKVGudlLl0niMAJOlACMJ6AAxwoX58s V8YT6KkE6EpXBMLXAQg04E+5c10eY7fH3fnuj7rroyB/p0dC8tGQfgwkIi9FEBOABQx5wMMXwNAE JaxhDXs43xf0AIkvHCEJM7iRWTp4QQuxJYPC8+AGMIcjdXVAjOFLo7qMRj4+Ga18JFghBOCIo678 6JfADKYw/wGrE4ABD3dIgxa40AU82AGZeHAEHrrQBS/g4Qg4SEIvL8gZDcLMKWNhonLS4pSimWCV IERj6E6gAAhA4JV1PEEJ0IgB0ZWABPP+LMECIgCB0C2gBBDg0zAHStCCziaESsADHgqRBWve4Q51 qEMa8JCHLhihC560wRGM0EsiOUVJ6XnKVHJAK+59Ri3jM6fo4PeAMEZgju5zXxlvRAIO4BOO9+RA Py+wgAbM8wERwKlBh0rUooakBDlgw0PTcAc7zCGidzDCF+7gBSMU4Qc/KIIQiECDGtwAqU9JS1lO 4LFRdpCDXenTLcf4SgdkIF3vVAD88Im5CPAJnbPkwAPKmAEMvHNdRg2sYAcLkRIs4RCMkGggiiAH ptZBC3eYg1aHMAMjCKEGRODCDGgwgxtss3pn7cpYmNOEHNDocuNDLbpKwE9+vlOmcOz+6wYy0CcS 4DOXt4WjuuhK2N769rf/yNMiClGHSxwiC3WQwxwiiwcubGEIXeCCEWYwgyJsQQsziIEIbkDK7npT LVBI7QdTOkcOmMB9HVgABBwQxpuqS520xVz4cGqCDMwglxuwAAaAy9/+GhQJTUCmIpqrBjzMAQ9p SINkhQBZNWwhBloYQhGGYIQjzEAENuCud7HyMrEuYQ1t+Jl5U1tG0bnTvAtYgDw5oF55kvC8EIiA bXdZXxJ0gK4mIAEUAevfHvu4RxTFgxrSoAbnpkGiW7DCIfBgXTvUYQtZ6AISqHtREYigBjPAwRqd N9qwbK8JZSjXGeEaxb4uwAEPoG3+BBxQRvjp1LUn2MAFfFpiNDqAAxjwKQcscIHPZeBP7Qo0uzAS aEEHqmqKokii+wWoRaOk0I62T6QTMukdKZMLeNiCdeVghS6ooQhFqMNVj6BQL2wU1EkgghW2cGEa iACfbCmRcpawnhyUgXJdCV+M4WfXz0FgjKLrAAaCmjkOXIBPEEjxAhTQABYrQL075UADLuAABwya dQBTNLZxaBGNocrbGqm0qcRdG3KzjdL1SYMVrnDdrGpaCFmIgRHmYAU1dEELzy2CEWxAAqva4Q5d xXIN+H20bwoNzGEIb+S68mtXmneFbp1jGOU4Rzq6080X4KlOe+qABmDgzAiwAAL+PqeAa6uOkRKp dKMpZW48VqrlEGk5oWAul5WTbmr1WXUa4JCGHxBBDVYoQgx+sIU74KERWrCq0LE8BCEYgQtIwAER alCDrtJACeX82WtWwIZysUhysVwhCcZIWzZywK37xEAYW3iBDFzAAhloQLURIPdmo7G+DpitAjpu clY5bOWNNtTfR+VtQTna5pgS/MwZBumIJfrwg3/X4gtG+cDPfPLjtpThK9Z4m0ce8YFfXeQd5S6M fd7yjA4YzUlShylA9MmgnoKViW70JW+BCDNIwhHSUIMhVFTfWd5oZ2+QBG0ecQW3nlETgpZaNG4A r+Id3zvr+FII1LMBclfvbPf+CYFnq3fkDbAj9q1/R0AbmvCqV5jhNU/51BUa3ZgPFeoTL//6o3zy kgd8+/2uqvWfn9Gbt372l3j7h3qSl34HiH7/13/5IoAOyIDoBzuzkQZbgAdWkAVwUARaoFVYpgV1 gExd8GAj0EzJhGl3sAXUBGpDIAIjUAE1QE1KEAZgVgZQsHx3Ik8sFVR9QkZ9EjolkAEPcGJpdAF7 xQEbgH0KgADJpl7VFn4ppgDShgF05wAw5HL9t3/8l3qiRzUnV38CqHpfuHgAeHKLJoYPmIVm2IBf qEho+CmKd4ZwOIYS2Hjf1oD2d3mrMhtZgFxJtoFCEGpbQHQTpUxaoF1d0Aj+j0WCWvAFdpAHerAH SFABFlABNvAFYVAuMrIeYHYj7rRLcCRPKbVCcOWD8+QAENBnL9VTD2CKneh2ChB3EJAA7vRCKYYA fReHEHiGEbhtneJ5qCOGWYhtpdeGEZiGAziMfwSM52eMuKiLhVSHd8iA9rJ6JFEEWWAFfigEP+B0 QiACXCAHyQQHVnBhQgCOHogHdWAFyGQHS7YGN1ADFmADNZAEX/AFSaAEJaAE+Kgu8DNf4xNFdYQ5 OmVPHWCKDuBXEYB9Z7YAbrdnd/ZCGecAfEaFCZBitxiNuXiM7peH/peRxOiLEhiMzCh/cHiAIOlH xph5xYiRuJgqyoiR9Bf+jdQYElnVej8gAliFjVrwA/gWUaImAjGwBXAwB2nQTIdgBcn1gYaAB14w ApM4cDhwBOQDR8WHj65lS+iiU8lWR8l2djpVbW51ATHWAHSHfRk3ZwnQAMzGUwqwdwhQcgsBaW/4 floIgMt4l3T5hiSJhwgol5tnl4cWLwE4MCapeITHl/+XmH+3mIKGl++Xf40ZeozHeYzZlwFIl1vI l7YRA0UQWUUgAn+4h/qWBlnAVHegBSIwBEVHb9aFB0LABXNACIVwCYWABzFAXTMwAgNnAzaAA1rm RkyTBAuQOSkVPu1zAdKGRlDIdq90RuuVbGK5bN23AAhAhSnWcdNGlrb++GM3xxEzuRGCN1DfCRdD dwVXEAMSZo1JZgRZsAVOJgfxFmoQdWQfWANZMJ+NEAh4QF1VNwM1QALZ9EQlgE19ogRJYARK4wVh oD43sAde4JsGSYS7dpALwAGymAET2QAGUJ0vpJ0it5VUyJ0x5xHjmRHhKUwl+hZTMAVNJwJWNQRE QATuBmU8dwVTUAT0uQVXkAV2kGBw8FCGUAd3UAS5+Z9WNo9JgAMjYFtJswRHoARcwAVFIKUK+gX8 NgMYUJAn4E5tt3cJeYpNuABzRoXMhgAZgABv+ZZtWQIiGhcL+BaY+UtvyiM0cHtJgARGMHBC8Ida BWp3kAVEegVyIFH+gBpqkHUHq/ZQCnUHXCAClSWPN4BNOMCbk3oEJWADIqAER7CpRvB0nvRVr9R2 7BVUGNBCaMqQ62WdTthTammK2MdCbRqrsioXy8RMXbBVXjAFQjADRPCZe5pVnFkHV2CNRWAFc9Ce TPWj4HgHmvAIh5AHR+AFvVoEvWlhvolN+ORVXtAFn3QEebAHFmUDeAY6+pSq1Xampiidy6adYtpx FVp3sxqv8loSRHAEkXAHXyAEX3Cr9aoFqWYEaZCbInAFaSAEN3pVezgFe/hUeMAJpWAIzpoHkIAH h5AKeuAFW0ADNqCxI5BhvHlfd3oENYAE2/oFe6AHesBRXZkB6vX+bEvIUw5Qne2UYslGiwxZhfOa szqrEVlAUUcAalW1qRZlWbpHXUXws0PwA0OwolnFtMh1B6SQCaXgCIVACIoQUYRgB5qgCkegsRUw AiJwBDfggp1lA0n6RBvFBXqABDVgBHnGcRnQhNiHAXOGZ2SZnSHnANR2AQlgfnNKaIjmI405ETQ3 uDtrHwrVBXlQr71Kj0aQVUZgaqq5BVygBe1pBEQgYVPwAzc6BVswBZeQCaJ7tT7puY4gCY6QB0ag m5OYYVRHXRgWlSWwb/q2BaxmbF2pAHDXlinGtz2FANR5ZkuYluUXl7w4omhjG8gIbiQqMQqRoof7 EB9oBJbbBT/+IFVTJgRHAAZIEJSNWGT4+gUoSARdAGpYZY2M8AqVkAmKsAVEWXQUWwekoAlewLYs eGE2YAQiwFn4ewOgFq17sAp6UAK0KLcFybcI8HYZtwBSmAAiV0YlYAF9Z6Kkx3L3gYwh6Z2igm7R axJ1EF1bYARGR01W8AOR+rNJoAcfaILL1FxfgATRulFDEAiW0AqZgI6K+m8+GQg3egRfUFGcxZs3 gAP7qwSuhgMzgKde8AVagANu5Wzdp7u/m31J6HYXUKr4hAFs6nJ6qXkWE5mlF3+OZ3pz+Zfst4Us N5hsc3oGaIcZQ2n4V5ej18EZUU3UqwVqcAfl+79LgATa+1D+jfhMXqAGeZAEdmBv+3oEW9AIi5AJ lgAIz2RghpAHEDVNFRbCj4AKmsBkVDcC+6ax/aZj1LWtRUADfbV31VZHe1azfRU6Wvw5N7XFcbmM adyMeiTG/leAGqmA+beShraGB5PLYch4MBmSwrzL80fHGOGNVzAEQ+AFVWUFW4UE+kbNeOwFeVBR X7CteexpH9xMi4oHm2AITmYIi7AI03QE3EqyXpAKkHAN7gANFLUFNaCbNKCxG2sDHVsDSrCxNdBX lIgBpWpTx5ZL83QjbWdb5XNtL+l3v3yivbiL6SeSWHiZLPnLbvi8chieKSmHXZiXGb3RyqwR2zgF RBCpUvr+h0cAw51agdOUzV+gUEdgB4ecB4OcB6dgB8wUpIraCIYgB0l3BAxGBFpwBIcwDNmQDobQ BY8ACZwQSUcrsvJYzyMwAtRaA5Oqz7yJAZOKAwD6sT94iqGDTxNMjHMIzArokSMpx5vijGZtOsDM 0SEtjG3NKcOchyMtEQNbAyKQZEFXBEQAw/sKTVwwVYzITIIcSV7QiI2QBUJQgR94B5EUwny6pzZg u6twDdFAyQrVCKdACMqkUPoGajVQASwItlWdzzQwAvhMqbwJoLZFRrZV1ilpyycKkivJko9iy7vY kWOc26fD29AI3B3p1nk9ETGAWTGaBb1KvpIACXegB+v+SE1doAeRVN2Mmt2ZRrVbUAdPRQiHsAXa G9VWFQP52wWpoAoKdXScgArmnLiZvAoOGgMtyIKSqJsb65T6XHWrjc+TalsAbn6BKceXp5kTrcZg CMZjSIeTGXqVl3qRKSmKWZcObcYXQ+EN3pcI87fH7RDaRb1Z8LO3GVG5oArPRE0/bK9MuQdMmQdc UGBaoAWF8KdzAAh1YAg4vo6MSqShlARdUAhRqlCpkAnj3AhNpVBSqm95agQ08AMVIIkVsNqTOIkd OwOmjdX8xptwhAOEBdEd/uUQAbY/SwTdWAPMdAiGYHTIRLlbgAQP5QWHYAdeoASUHF14YAmUMAV1 AAf+WaAJpcAIjkAImgAHPxAD9lYDd7AJOFwHnFALlyAKd2AIjaBQjlAH1/u4L9ipIThRn/nkTglq I4AAX2sDlEh1A0cCXY7XYL7qBxEDSf64YVsHA5YFy9SpoKaBH+iBQupMD4XmgBAIaSAHChsKoEAI l0AIhMAFQekImOAInFAJJ5gHnGAKlWAJiwBZgxAIDyUHzT2weDAFipsLuoALi1XP9awFNFABE4AB LWgAq13qXN7lHM7q9D4QP5AFlWtdoLYFgQAHe07rOtqrRxZqmaYJjiDZkvAIw/VYdbCHqJAJoTAI g2DpIqAHzODommAIiuAIpIAKzFAJlUAINo6OeGD+CHPAuZubVT9wB5iAB6qgB5HQrIaA7lZWAAVw 35JYABaw2iNQ7z7fphgorEUnalew5z9gnubZnoxgdFqAYHdwCYbgCJrMCZhg45tbBFdwCZbwCTYu ajHgCLYwCpsQCqIwv5zQCMwQCpkg8oUACMhu49uoVdolAllwCfAJsdewC45w8Po7AAMwATb/5AJQ AE5ZAT9/+D7Gk62nBXYQdHdACO0Z40R/r5UspIYQCI5ADKoADbVg8mmggVOq9m0vrDEgBJtgCYqA CstwC6OwCKFLDa9ACoMgB4GwrE3vqEmQmit4B4pwo1qQC7lgCJqQC81KdQUwAMdfAQdgABNgAQX+ QAOIH/38BQPsuYeQdYGtF4LOVQR4oJ+t11j8vgnRkA0lngmBoAhZ0HRbkAaWYO1wII4/YAWiIAqA 4AhqTwiNAPHUUPaLcAjgraMAkebKCBE1RhikoQpOkR9dahXCs8gQNFVFRlwcIABBgQEVCiCo8E/k SJIiAZwseRJASZYtXb6EGVNmSpQrZ94kidKkzpE2e6pU+XPlUKA+g/rEmVTpUqZNnTaNMSXNjxhZ 7sCBM+eOljRzuhDBU6cOni537uC5A83armucXgGCM6WIESNpFi2yVCdLlhqDLrWqQ2iWpTuFPjGz BYoUoTqONOFptKWICBozZIgQMeML1yJyRsH+yVTnjiMtI3CMqCCiAoEBEwS8HjCT586ntXEidWkU t+2cvVP+/mdT+E+TvXXzRp5c+fKbMYoIKVIji1g4d+oAop7mUCOzaepwuZMmV65huai9gvODiJYt WRaBahX6yo8ieAqBWiTnlSJCijK9+sQUTERRhBRMxtrCICIKmiGJI4qwoxFrbtHCEEriioEqES5C bYABDBigABBlC66nEplbbjfgTEQxOKN+UxEp3IYzUcYWb8QxR5xEKKJHqob4YYg68qpjiymyMGQQ QNIIBI+9UrFll2yUKWeWLLgoggghhghEkUXq+IEqIn6YIxBA7LBFkToUeUWWSkSR5ZVaRCn+JZRM ppjBi1TsoMsI9u5gJhrrHJkFGEB+GCGGiza0YIQQUKtggthkGqrGnGTElCjaShTuxZourbFT3Wxs qVIXO91U09xOnPHEFV19FVUdZ6W11n+qmkuILYv46oo05CoiCznEMuuOK0hhRpVrrBmnHGYWSeOI LYSYTqwpeMxykTsIISSNSyxhExhRLBHlk0VIeWSWUETpgoZHVDHkjiyMIIKIPHjRwg5HOFGGmmUm I+iiRitITVILCiDRUxZpZLi44oJi9VQXI+ZUJxopdngnoDK+mKiLgTOVY5ZIHXljUG1FOWXkMoRu iBmKyOMsRbQwAg8hrBDNuzTwEKWWU7b+iUYTQxwRRZM6rujRiCm0ALPHI67IRBRK9OpvkWTizGSZ VwappJBAHCnkyFBgGUXeIYqwwpEi8eAEF2jMiccRGlQjqAIaRgCxgtYmjWm4jiGWeGIWMTa54VM7 5ni3Vl19kXGRS334UsAHh9U3FStXOXPNZYohhspokCsNXFbBYwv2TNdiiivuIAVZWnYxJpWtHHEE j7CusCKLKX4Q4ocpoCuC27i4UAQQ/WaRepZKZJFFlEEIuVYTU2qp5SwrhPDCEC60uKMWauLZhxsu aIhBtQoa3XAECyJFmNLIHbf85MYbX9jhxTHeNGP7B0cc/pGJ01/lSLY/AgZwcwdE4Ej+YmCEGvAo C0QohCjihTstdEENdzDELRhhClrYwhB7wMMjZqetQJAiDVu4nnrUc4UaaGE6u5vCIjJBrnEVIhnJ mEUrZDGLT9RhCozIRCZQ4Yg7yOEORYjXWZ4Rj3hAwxE2MMgICqCauxFsAhUYkfs49TjDdfFTBOxi Af2GP1j1j2JH+ZhvHkc5AfJvYfRLYBwPGANdJW0LoghEHbSwxyJooXSlMAUeaBeJO9gBD4fgXiHq YIhccKIOVqDZ2ebCEPpkgT7BI4QrQjELQgAiGMCIEzLA0Yo5UKIVnTxFIQiBicmoLQtcwUU9aqGF 8sVgfQSjAMEIZgFJUcpiGgOgxjz+BqphpqomqrLYMZE5G6GISijEfJisgGk/ngBucoHDJhovJ0du 1qp8PepRHahxifa8MnVZuIIh0HKWPJBFLGmo3R04cQgiYmkIXaDZtERAHx7JpQ5wAEQmQoGXHJJi GsiYhSw+YYmAVmIU1EGbIfK4FS08whZ1uFv6CrAaAayPlyMQQDdFehPFjdSkJ8WRC4ogmSFcwRHP EksRrnAFplliE444xB3AwwXbFQGnddiE0bZnFU6o0wgMIQLLflcEqQxCLM9DRjI+4Qtk7NASlchE KygxC7jorgqAqESTsHWKNGyIBjXQpQAOoDePUKB9KIVrALcZV7rWNSYq/cEJ5aD+iUJY52i42wIh QhEM6/h1e1ygpxYcoQvRWEcO5uAEIeRwJKtYR3djClNevQOIUADjF8AIRygMAQhQVKISlkBGD+NS BL0wpAhzwwMtMYAaChAgUgIowBU58la7ivSavQVucF0Cg8kYYQ5qc2yR0iCHRcBBE6rwghq68AXb CbKwz6gdWhrBiV0cghGK2EUtnqEIR9SiDj0CVpiWJodywSkYycgLHAjxiVcAoxI+1N3uqoKoGGwh tqkZwQEGQIGMEKwAGw2JcBW8YAabdJ9aMoSa8vjPO0hkaJtoEiR2mgc9pME6j0iF7fIgNEegYhGK sMUz1mEKR+SiiN7JL1Mzy5X+QXyCFrO4hCI+MQj4SLUOMaDlD0SQXuf4KQYVUKsAriiAXHqENQ2G cpSljLIfbGELaeAE9zopQ0KcdhalIMRZziJIPQaiwqs4RFjgVYdGNGIRqthFNzShCWgo8g63UJvp piAZhuyOC4voJCBaoQgeAsIXloCDWLTgoy2UwhGGgN5eYCCpChzA0iOorYf4NmVOd9rTT3HOFMQy 6jkAwkx10MSXdmeWOhRCEVzokhxQ8Qjb6YF2WrjpJozhjVyIQheFGEQgFuFIUmxCNE5a2hT2LAdA YAcryJsDVxetF9/51BZaKIQwTlEHTAxiBDTQm4BDECkCECCkfUumMz32W8z+CZOZyTkKU+b6aXon EAb0YXWiA70I7mSBqaIOj9HSQIrGqKKdGNQjHvLACGN0wxCYyAUm4KCIEgbRFpbIBCjOiwdDmMIU ioDDXlLn002U3BBbqCV6zVmfPd7BCCEIwQQYEKm7SaojJKpfqtTIRjXOm6Qrene9hb5gysRgNGnI SpjvIDTR1MEImghEFvhahzlcQguN4HgdtLU2PDwjH4eARC5CY2ZHXAIQqyhQKRaBB0ZYwhTqIHhj 27kFPYiGjzVgahAsyVp/h6BHNLibQaw4AhvQYAIXwXkByWijy/lcKfdr99Al39sMde5bEnHSIqO+ hWNtOxNq8iP31DkWImb+4RaY4MQ63GwL8xKiEHPQBAZHMQtMWIJnoWhELk5hijlIBg+reAQt54Ly zvVRLJuQQyYO1aMRXJECi5rA4Q0fAhuMIPGMg3wZI/8xa7IqU9jMXzDXLbFRBXPy578RDERQFxPf QU1zIO/RsrCJXIylFHLYYyCKYGbuNUIvQc2Ee1CFRciFYsCOO+iCRiAEQwgFS+i1RdCE1nGEZRCF TeCER8DAPNACHhGBzsE7yigC61CEQrCF5zmFTdCCChgBFyA8wKMBGGgywLu+GYEj/4mR7oumMCIj 8ctBkfmlM0K/IESRGKgBK8ADp6qD+xILxqgDTlgFvSiFQJADIWCEPCL+Inn6pzk4hbQgBfAyh0x4 HoVzBEbghVBYhVUQBVKwhUy4hEpwBUtwKrMIix9gGqoogs4hwiMCG0wABFOIBVjAhFEQRELAgiDw HMC7okSxPi0yoMVrphhZo79Jpp37n8PRn5BJI5MRwk3kDRioAUPwjkSDi3+arEfgBbGQIT3aAtLA A0W4g0xwhN4zGlLIhDs4BW14iDDDA0fjhJsqBC0AhBH8D2UAhUEYBTgMjFGLC34aAR8wC0OAg0vI o0KIBUw4BWGwBViIBWHAAhoIASyAQSVrvhm8RLkyx1fhIp4THMjLROPIOXXkxHhkCix4hDwSRbjI in8qvXNxqimABEP+yIINMoTHyAK00ARRqINTuIZnIYQ5WARAcL1MSLSnGgSskIVCsIQ/QEKuYYyF +IEQALI6pANCoINBSB1M8Lh+gQVYEAZhgIUimABFCQECaJRF7BvLmZ+eQ0fMyUkw2p/j8Mk2yjnD kceiVIpF0oJEA6gL2YsTEwtDOIUv8S9HqoNbYDNG8CFCCCJAqAVosAWuwo5MoIRmIwSAGoQLGcVR eIVKAISHqoPnMUlLUhT6qIIq8AEsyARM0J0f2ARYGId2EAZlCAZlqAZh0AIYaJQYOIBHmQCS6j6I KYrym6bJfKZskkzB6UEAmhzJnI2gM8rPfAm/+ie8oITdkSjrmAL+WoypORAITlAkF6sDNaCdOsAF augZJNQCQriEiqQOQEADKsiCKogDQGgTXNgErEg0SwqCIggBCSgCLJiCIMACJNwdxYqHdlAGZYCH cuDOQkgUCjArR0kw0CTP8syRfxo1UkCPKRi7OpADRzOEf5oCv3pIVXBFxVIFTNiFcTgFR9iEqYGD NCCEsYQDH4gCrKgC3wmTOnCFV0DPUbvDHwiCU+jPLChJsaiCaYPIU1iGcWAicghMi+CQEHg+JDPP E0VR5RALOYADURCHUHAEU3jLRyqqvtKLVQwzGZowfeGEa6AGXBgETCgeOUiDJMGOOKiDuiyCIIAD LLjLOrCFU3D+NoCKixeAgSS0lqTUox9whCAFhCOxhXmYh3agBmX4sUchCM+ptANI0TZ1U6aAg0EA tlkIhlq4qCxItE0gL+gJuTl4hDqwA4RUNoArhVzQBUC4A1wgBexIA3yMCziQgjCpgk0ohEtItEW4 hI5Ezh+LAd1sNjQIuQKFgSw4EL3QgmeQB3Qoh1OoyDtMFEXBNNx601mlVZgQllAoBFNIQ1dyD0Xa BEvVnUVtNZDzoSKYA1FghmUgBVXgBXixg7EAOTiIAzd4gyzwgSqog1ZohUr4g7FsNqygA6V8gSyA BacChDf4JzgAAizAhEuoAzqog2d4hnmoxqcqBHDckA2ZgCf+q9V+9dc7IyJHwAS97JETS7Ra6Ecu cIQseATk9KdM0AVb4AT9dKS4YJJTOC0kTFIsoANXaIVvMTs4qISyBNc6aIEgqAQlIQRB8ItCEIRX AIWyHDUtOIVYCIxLkIVYUKQ6gLmLmAAJ8NegpdUt1QIz24osEFBR85LWuopFKITk/IEsyARm4ARU WAVoQAsfUi8X2iPdiQIfEDVnG844wIppxVZMeIMWqIOxrAOWBYRB6NJLCJsjkYtASAZuw4QUgwZM WE6ZjLlNu0nILJwvKpnLBL9nKorDMVx4S1xKFFzPNI50S7ftq6bODDptqtzFNU/f2RCmARM6nI6H bC6psA7+UyASuSAEaCg2xojPOdCdYMHTUJVWNAgTJaGESaAEP3iDOPgDONhdQbCEOOgBQKCESugD QWDCkdVaF+oRQ+grWOAEbOiHfIAG5rSICkDTn1OY99kioLPBl6AfTIRH5KjBkhpfd+Re8TXf7vWf SvxeSkRRLNjSKbACZfsBONACRTCFUIiLq8sCV3va1LmDcaA9ReAaRWDLO3jbb0XPuLBWOriESagE AkVOrAAEWwAEOpjOdnUouB01OzyvPiqELLCFVyCHfugHeKgGvTREGFBBm4QJMyKcRkzf8DsZxzU/ 5mg8nqRhkKGc/tkm8cVMmnBfy51VsB3URLvDYLGFZPD+tx8orExQpGXchGUYBUDIVLNrNrbshOu4 LyqVgiqYAimghE+ghOKNA1CFAzTgNqfKgjlAzyBt15hSFC0oRNYatWWwB3uIB3hQhmcoBi2ggRKF 4VVZoxmmYSHu4ezLPuVoxxvcyfcVI3hU5MgTSjZyPHlUSqWMqSKoYolcGlQUhUSTiy2QA78gBEso BEwIhUvwBV9oBV/ohFbAjlGrgiyIgjiA5VboBNwFqESgA0BoVyQkBL8wrdMFMvnFgmDxIWCzBXSY B36AB3qgh2dYtPQpZMg5ZDOCxPa15AHiYRTRRG/uYZ0sR0mOn3PUZgNq5PKsgkSrAiwYzt0ZhBLE 3+n+kAM5KMF/AqfPW5MuLIRK+ARt/YW1LN7ZhYMq6AEpGAVfAAZX4NZBQNKxVMu3ReCK3ARL6DJA KMQrnYIqcA46CNe3HIRN4Adq7gd8iIfGCAKYowFGtMRIVMdRaedDbrdMXgruG2cbNmRhciObThxm CuKZjmTyPIM3WIgjqQMfoINQwMdR8zA+xIopCIRAKIRXOONQYMtitgRKUJJm6100qIIo6AEfiIFY hmUzzmBAwN3EyNTrgItme1sfggHiCgIfEAFgE4SSpIMWYyJ7QGFzGJoqCgGYPmd0Zuf/uabyXecU cd9z9rn1lWTJTuzFJmr4LU8kVdcp8IEfYGoKfgP+uZiCGEhXsWieV0AeU0iGSzjjSmgFS+gEtkWD Qe2BHnCBbG2FgXYFUKCEbk0EUEgTQOhlgBKEP0ADiIznF4iBKgiCF8ACjFSSe8WCU4CGeUDheOgH XcCFIPi28QRfdAxfTAbqoRbvouYNyoajeWPknxbKGvxm+K1p8owCKRBtZSPtCSZe7Pi3qf6BIakE cWnlWv6nZktXrJACH0Bw0t4E4nWF4p1ltu5qWDhjWJ4E7PADks5dIHgBsE3CS2jDkpwOR0AHJqqH fpAHQeCea2ZEoaaJAUpcy71BcP6+FtnMzA2cjenmIRagzpScY5KcUuHxyjxcFHUBHwCCH4hUqpj+ Aq0iUCR9V95UtqtaBlf4BFB4akfFCjSg3QP/gdtmyzJ27bGcBD/oA63O7Vim5bGkAqlO0BfoASzA A7BaWWyNAiy4BFvAhnnABmVoh1iAgRB4ARK9vscTWsz+uUK3lQwRbTGuimajhDh4Z2kFqC+dDiWZ BkuwBPSwYAqe6gLtbLOugzcABDcg3ldIhEToAywOBm3VVtMqXjeogkqAg1Uuu4G9BEGAhQymXb+w hFMgh3YYB2VAB0wgrubz7hiWN0Q39JtU9lp5AYboAb3YnR84VzeogzD2gUd11H+ShVZ4hWnQVkq4 qrF89DhAUrKW30LI6LftpDP+A0Ag20uA6AX+JltAOAOUnQS/kOjAcCq/GAVB8INApANMQIdzqAZ4 eAYweUEKYNNDv41mX/aehngdcQ5KWtIMwS/6jgq5wApLCgxAwIVX3lZH9+qrSjQ0CIL+PoWHswVC EONX0p24+NrlqUg4cANphfQ40NYDSTTPjQP5ndHpiINCaIaiF4ZmGARDjL7GnPim50QgULYqoIIg gAEg2IvpeN29+Ccnj1NKAIaokgUiEcUCP3DnWEuIBAQ/+ANLeMM6GE5ILYQKH0s/IPU3yOWA58M4 ELXZpQMqEGkkBQRYUIZmsIVpGHxb4FsYpACnZ/wglAI4EOk3oAL7VbasxwIt0Lu4mAIsmLj+b0AG YKCOAv8nKQiC0o8BOkiGXUbzVkgGVih3N5iCQSgeuof3NzgDKVhbK09SQRA1JK2EKRCEty2EUViG 7PRjZRCHdigHjDqyxnd+yQMCNOB8z040JI2x+ha1vaCPUZgGaog70db8QQ2TFnAEafgGX/iFV6Yq Xn50tx8Epx5LPrB9N/iDx6eEYKgElg0GStDNd4dXtwcIKZuqxYPXrhw0ce2qYYERo8K/iBInSgxA 8SLGjBonWqwY4ONHiiA3jtTYcSPKlCpXsmzp8iXMmDJn0oz4Jk6dKUWmDJqE8w0WLXDqZIHzpg4c LVnqDEJWbt0upD+05KxTJ4iPHzBCuLj+NOuXL1/HWlG6BIgQIDhwFnEDNAUOIEqUqAzCNGgQoVGg AF2yJIgOnaFUfAQ5FatWuXPz+L1rd06Zs0FbKaQMWVOl5X8nK3LWbDLiZpGhL5Mubfo06tSqI1Lx c7MKFUF1/tBJCwcn0qJDp0xRhMzVqUJZpvhA+hZNDx8vYMAIEsqSokqT4laqBCjtlDrNbNU5I6UO qEpwKMmKBavSJVODLlnFAgdLkB8/rMAS1gxbvH7z+jUTho1cLHSEMAJmq13U0UkhWYRgZQUa+CCE EUo4IUs9IFXHG2ppiBN8L7jggw9AaJHdG65IMwslbqEhBRxSSPHWFEBQQUcPcrgCTCv+rXQiV1y2 9RBHJb+0Aoog4Q1iySWVDGIdWoDUQUccQQ0WBAxFnKKKI83Mw406tjjiiBaOlLMJDBA1mNmCC3KU oGYIguQmZ2y26dmboGXWpmV3KrimnR6NRiGggQo6qEQsABFFDz380EMVaNwExxQ/TLFUFVhEqpYs tVyC1htVSHGdFHEAMogop5RlCSWvtPIJj9YZlegUlrgSyiCCLEnIH6Na8gdScAA2RRU+9BCDQzH8 UEchmGxyCjerOGKVFqjEUgcQIWDG4Eh7ekZnntsy6C24cYLWWWhyfnvntuH+SSi77bpLmgaGJsfc D7f56ukUQWCRhVCXBAPLK68Agqr+W0ACEgUl06V4HcNpAfIHHBDDgdUPaICDjCytVNJJK35MUgla 08VBBRVALHVGDCk7pMUmVqmChxaGgFkEF3dhgQUNDqo7Lp7fpptgSWqmSyfPcx7op8/lkrvuu007 /TRKGrTAAhXDGhvEFFJU0eIUOFVBiC3B/OLKK8nINckbb1BxXRR+5DhJH2+kqOEbgCSioRQgvjAF MsBs4odsQ9WBxZNIBQFkHbYO0oNDQFTx5B11OBIUFoUMssUIRRQRBA0ENsjzuUUXLefQRP8Mumgi dba60qKjCzXssT/twg8usPDCD8oRFoQUw/0QBB2uJPMLMMDIMss0PN7GYlq2uDL+ti85VkLJjg+7 UUUcUuTOXBCWX0cHbXFAGQcacZyBhV10DPIkHS9Y+AcmdGShl1t1ZIJJFkVUsLkW1n6+MwDNNa7Q nUtopyudAL2lJzsJjWmyeyAEJxQD7f0gZSmTTwUrWIUelC1gwPBF2YLRim8gAxBZ+BQobGGJ6wxl EA7jEcPgMBgPuYAKlmjFriaxMLjcxg/ouYQLBwE4vBQCMI6A0idacYns1CF+WAgBEPTlv8rkSUFp 6hOeelYno22Ri35iYJzQtBltAU2BDowgGtOImhYYq4IhIBaIskIsTCTDFZ+QBTBccTxZgKOP4fBD dqyTIxxa5zpWuc0f/vAGNBD+IkmJqEIMCtGJP8glEX4ARNre4AY3DMIVl4DFkKrDBz8QYgqDeZYg lBQlOtThB0uhAg1CQAPKxG5dZ1QjLnOpywoYywchaMEIYkAYYmnBFMoARSt8Qbbj/QIc4bDFxX5R N6sQYlWd8EUlctUH6cQhbXWAhSAoUYeEXaIVk/iDH/wQt3S6AQ5RmAImrOKKUehoEpZAZnX8YLk4 tAIQo6BDsJKTBUxMIQQv8BxJ3qTQhTK0oQ59KEQjKtGJ3lKXFr1oRiw4gglUEAYi8EERYiGNYKhq VrIgXjKQEY5yyOJ5rzhDHMJzTR2RRS1vmA4c0ECFOkyCFersUcI6ocM/JKL+D244AxWqAAQlDeIV SaJEJVyho0G04g998MMfquMKtxTBBzHIwiCoEIQYILSWCsUoWtOKxhaACAYTkAAMvBqEUVCDGsBQ 1SVG8QpZiEMa4ihHXznWipuygnqfYMUkdpTVXEGsCnTowyR8Aoi49SFhifAEH27ihjhEIQhRcAEm ErExPywJEEqSTeJa0Yvq5MoPdKCCFpJqlSLQYAJqvS1ucwsoF1hQAgwIgXIGEY52zGKQoijnNKgh j3mIQxzhiMsforCq61ACsnCIGyC6+c63ZPaogHBD2u4W2bidIQpRqJpyRlEJWLDimp1IhCDI56tX /OIXndhYJQSBiUIUAj7+RcDCa7Wg2wETuMCXMRYMRsAVr/ogGOKQRSUsEQySfsIWuOhHPQA7jbS4 KFaZMIU5+/AwTcLBDWhADhUmUVRM2kZuiYBb3NTCG0TBIYlSpQQofMGKSgCmB3QQmy8+9rFKvGIU l3itfjdRCEMYuMlOfrJGXLCVrcQAWdUIhi0okaN2sGMatrgHPvoBj3JMww/vGQRvAtaKUMiFbqEI xSgAAQqCIscH5YtD9oySlh50+AdSQIMPFGHfTwiVFeb0Q9XQYIlfIKNjkQUZkWARC2y0AxNaKIKA oazpTRf4BSEIQQ/wQAhS2CIWy/hEJYDRCWrIAhC76Ac9yjEPXThMGsr+CAYdQtGKQYznEq+wRSiq UIlW3FiT513Rn6UAXk26QQo9AEIPEJWociaiFb9gBbb54IYsCMIVz6uvxiQtDHmQAxcibUz8fhAT AFyE3RRx90TgLRF5R4Te/7A3vtut73fvO979nve/6x3wew883/w+uL8RDnCFC5zhBHe4wRMu8YVP vOEVf/jFDR6DQdjiFMiaBSySgUwcAiIYd3h1PMjBDVwU0hbKCMc0piEkju0oFEKCwzRylKJQxO0N fjgDINQp4rjFYZPmRZQPWiCKhPWUY26gAhoE8YpwWBsZrHAFLPpjC0wAURjCUAYsRjGKQrwEAPQ2 +7vPrva0sz3ea3f+e9vn/Xa5x73ec7d73e99d73nHe1w/zvdAY93wfOd8H4PPOIHn/jCL/7win88 4yF/ePvFE2y1MEcwpIpDaPhDH/awhzq6kQv1wiEUMZ/G8BrtirDk6BvfSEQzLAEdNsPPEZsqhO0n QYdM4m0ohQiCVDsh/KIOJg7AqIQ0GG11+BWCKUCMhTCqMYil1CEWLXF85LOP/e3vnft97z74vy9+ w4ef/ONvfPnRf37Jp5/969c+vQkhCquMQhbLIGnAlDEPaKhDHeZoxikMwixsyiDIQpcFAzv00TeA wwIigy+EQzjggi0o2R0QAhwIwiZsgrI8QrJgwk6RjFI5Gx3EU13+mFMlWJLaBAFVUYIrhEPyXYIg iNUl2MUofJ1bBAEVYMEgsETEUZwPWtwPYlwQ9iAQFqEQGiERHqESJiETFpwTQtwTZlwUDuEUImEV LiFG1IHDXIItzIKqJMMznILGVMLIXYclvEItwII4KEM5tIM4IIPr/cIxfENhIUMwWKBVRM5OBQak 1EHkqAINZMEdjAyf0QUgIYtVCFEdxEERYYItjEIwVAOPJUeIBEUd2EI1EEIVxMjjYML1tR/8vZ/3 mR8pql8put8phmIqjqIptiIquqIqwiIrviIt/gMVDEUozIIshEKF2QM36NwfgIIlrA9TEAIlKMM0 lEM82EIy0SH+YiXWTFHCCu3XJjyDLrjBHzRbDyzFHQDBk2jBG0jBG0RBdZxB+VTBUsSBIFxCIUCf 1/0THQhLFWBFDBSGMqCZD0zBvmCCJ7rELMZiLf6jQILiQIoiQR6kQSbkKiLkQipkK/7AJroCScnC M/RDmBVPJwBCRl4IIIiCMohDPIhDwATDMfhCLxRWKyRCIuxIMgVDIQSClTRDdSRCHACLsJCMKe2C I8RBxKSTH0gBTKFBEBCFFHCcMMBC8xWBpCQKcxRCJeSED2RBUo3gDsJEE0IhVkphVlLhVlphV2Lh V16lVo4lV5KlV5olWKLlVVIBnpENLsQDPdTDOFTD2MAFKKBnQU7AgSiIAzmUAyy4XDlIg+uRBdww DCWIBStYRRU4gi48Wh+cQYscDrDUwTPYAjKZBSBAia/4QRSIjx+4QRMVAqQEARCECONMgZEBlCmx Uh2wB6e9JmzGpmzOJm3Wpm3eJm7mJm0GBAA7 --0__=08BBE62BDFF8F92E8f9e8a93df938690918c08BBE62BDFF8F92E-- From steve@improvision.com Fri Jan 24 14:29:53 2003 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0OKTqc32069; Fri, 24 Jan 2003 14:29:52 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id UAA16381; Fri, 24 Jan 2003 20:26:47 GMT Received: from improvision.com ([213.105.144.233]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971; Fri, 24 Jan 2003 20:31:39 -0000 Date: Fri, 24 Jan 2003 20:31:35 +0000 Subject: Re: [Metakit] Question on Views Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) Cc: Jean-Claude Wippler , Metakit mailing list , metakit-admin@equi4.com To: "Barbara Menzel" From: Steve Baxter In-Reply-To: Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Barbara, The problem might be that you are passing vJustMeasured and vTreeTable to CAreaBased by value. This makes a copy of the objects, which may not be allowed. You might be better passing a reference: > CAreaBased::CAreaBased(c4_View& vJustMeasured, c4_View& vTreeTable) > { > int nRows = vTreeTable.GetSize(); > if(nRows > 0) vJustMeasured.SetSize(); > } Copying the c4_View might be the cause of your error and is definitely quite inefficient! Cheers, Steve. On Friday, Jan 24, 2003, at 19:43 Europe/London, Barbara Menzel wrote: > > Here's an example of the type of constructor code I'm getting errors > on... > > CAreaBased::CAreaBased(c4_View vJustMeasured, c4_View vTreeTable) > { > int nRows = vTreeTable.GetSize(); > if(nRows > 0) vJustMeasured.SetSize(); > } > > It is appropriately prototyped in the header file and when the object > is > instantiated, there would be a view (vJustMeasured) equal in size to > the > tree table which is then visible to the class CAreaBased. Using > c4_View in > the constructor parameter list is what generates the unrecognized type > error. > (Embedded image moved to file: pic11773.gif) > > > > Jean-Claude > Wippler To: Metakit > mailing list > cc: > Sent by: Subject: Re: [Metakit] > Question on Views > metakit-admin@eq > ui4.com > > > 01/22/2003 02:12 > AM > > > > > > > Barbara Menzel wrote: > >> We are using MetaKit with Visual C++. Often, we find there is a need >> to >> initialize or perform some initial action on a view within a new >> class. >> I've tried passing the view into the object via the constructor and >> get >> several compile errors, primarily, c4_view is not a recognized type. > > That's a typo, I assume: c4_View, not c4_view, right? > >> However, passing a view as a parameter in a member function, there are >> no >> errors and everything works fine. The view can even be updated within >> the >> member function and returned with the updates included. Has anyone >> tried >> this or something similar with any success? > > Could you post a brief extract of the code you would like to get > working? I'm having trouble understanding exactly what part is not > doing what you expect. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > > > From jcw@equi4.com Fri Jan 24 14:53:52 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0OKrpc01252 for ; Fri, 24 Jan 2003 14:53:51 -0600 Date: Fri, 24 Jan 2003 21:55:41 +0100 Subject: Re: [Metakit] Question on Views Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <33339D02-2FDE-11D7-A08F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Steve Baxter wrote: > The problem might be that you are passing vJustMeasured and vTreeTable > to CAreaBased by value. This makes a copy of the objects, which may > not be allowed. You might be better passing a reference: > >> CAreaBased::CAreaBased(c4_View& vJustMeasured, c4_View& vTreeTable) >> { >> int nRows = vTreeTable.GetSize(); >> if(nRows > 0) vJustMeasured.SetSize(); >> } No, that's not it. Objects of type c4_View are "smart pointers", using refcounts internally to avoid copying. I suspect that the class contains a member which is not being initialized. The above code does nothing at all for a CAreaBased object: it just sets the size of one view given to to, to the size of another view given to it. Can't say much more to help, without seeing at least the relevant part of the class definition. >> Barbara Menzel wrote: >> >>> We are using MetaKit with Visual C++. Often, we find there is a need >>> to >>> initialize or perform some initial action on a view within a new >>> class. >>> I've tried passing the view into the object via the constructor and >>> get >>> several compile errors, primarily, c4_view is not a recognized type. Please describe the error messages in more detail, and include the code snippet that causes it. -jcw From steve@improvision.com Fri Jan 24 15:01:16 2003 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0OL1Gc01785; Fri, 24 Jan 2003 15:01:16 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id UAA18424; Fri, 24 Jan 2003 20:58:11 GMT Received: from improvision.com ([213.105.144.233]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971; Fri, 24 Jan 2003 21:03:04 -0000 Date: Fri, 24 Jan 2003 21:03:00 +0000 Subject: Re: [Metakit] Question on Views Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) Cc: Metakit mailing list To: Jean-Claude Wippler From: Steve Baxter In-Reply-To: <33339D02-2FDE-11D7-A08F-000A957664F8@equi4.com> Message-Id: <3872C532-2FDF-11D7-AC6B-0003934D0DA4@improvision.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Serves me right for answering without looking at the source code (I'm at home) :) On Friday, Jan 24, 2003, at 20:55 Europe/London, Jean-Claude Wippler wrote: > Steve Baxter wrote: > >> The problem might be that you are passing vJustMeasured and >> vTreeTable to CAreaBased by value. This makes a copy of the objects, >> which may not be allowed. You might be better passing a reference: >> >>> CAreaBased::CAreaBased(c4_View& vJustMeasured, c4_View& vTreeTable) >>> { >>> int nRows = vTreeTable.GetSize(); >>> if(nRows > 0) vJustMeasured.SetSize(); >>> } > > No, that's not it. Objects of type c4_View are "smart pointers", > using refcounts internally to avoid copying. > > I suspect that the class contains a member which is not being > initialized. The above code does nothing at all for a CAreaBased > object: it just sets the size of one view given to to, to the size of > another view given to it. > > Can't say much more to help, without seeing at least the relevant part > of the class definition. > >>> Barbara Menzel wrote: >>> >>>> We are using MetaKit with Visual C++. Often, we find there is a >>>> need >>>> to >>>> initialize or perform some initial action on a view within a new >>>> class. >>>> I've tried passing the view into the object via the constructor and >>>> get >>>> several compile errors, primarily, c4_view is not a recognized type. > > Please describe the error messages in more detail, and include the > code snippet that causes it. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From dan.gregory@wcom.com Tue Jan 28 10:32:55 2003 Received: from pmesmtp02.wcom.com (pmesmtp02.wcom.com [199.249.20.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0SGWsc20427; Tue, 28 Jan 2003 10:32:55 -0600 Received: from dgismtp03.wcomnet.com ([166.38.58.143]) by firewall.wcom.com (Iplanet MTA ) with ESMTP id <0H9F001MYN9NNP@firewall.wcom.com>; Tue, 28 Jan 2003 16:32:11 +0000 (GMT) Received: from dgismtp03.wcomnet.com by dgismtp03.wcomnet.com (iPlanet Messaging Server 5.1 HotFix 0.7 (built May 7 2002)) with SMTP id <0H9F00D01N9HO1@dgismtp03.wcomnet.com>; Tue, 28 Jan 2003 16:32:10 +0000 (GMT) Received: from wcom.com ([166.34.145.47]) by dgismtp03.wcomnet.com (iPlanet Messaging Server 5.1 HotFix 0.7 (built May 7 2002)) with ESMTP id <0H9F00BHLN9J9H@dgismtp03.wcomnet.com>; Tue, 28 Jan 2003 16:32:08 +0000 (GMT) Date: Tue, 28 Jan 2003 09:36:07 -0700 From: Dan Gregory Subject: Re: [Metakit] Compiling on Sparc Solaris 8 w/ CC and MetaKit 2.4.8 In-reply-to: <34B48B8E-2BB7-11D7-B0FE-000A957664F8@equi4.com> To: Jean-Claude Wippler Cc: Metakit mailing list Message-id: <3E36B177.3020807@wcom.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7bit X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.2.1) Gecko/20021130 References: <34B48B8E-2BB7-11D7-B0FE-000A957664F8@equi4.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude, I got an update about the compiler issue... don't know what you want to do about it (change the code back and edit the Makefiles, or just leave it changed for now), but here is the answer that works: ============================= Do you get the same problem if you compile with this option: -features=%all,no%iddollar,tmplife The compiler assumes -features=%all,no%iddollar,no%tmplife when no -features option is given, and this affects when temporary objects are destroyed. Interestingly, there is the following comment on the man page: The behavior of a program might change when you use -features=tmplife. Testing whether the pro- gram works both with and without the -features=tmplife option is one way to test the program's portability ============================= Givin the last statement, I think that you should keep the code changed, but that's just my opinion. Later, Dan Jean-Claude Wippler wrote: > Dan Gregory wrote: > >> I can compile ok, but the test cores on b14(actually running in the >> debugger can get it to run correctly) when the view is accessed via >> GetAs(). This tries to invoke ~c4_string and when it dereferences p, >> it cores. If are having this problem then you can replace this > >> method: >> >> const char* c4_HandlerSeq::Description() >> { >> return _field != 0 ? >> UseTempBuffer(Definition().DescribeSubFields()) : 0; >> } >> >> with this one: >> >> const char* c4_HandlerSeq::Description() >> { >> const char* returnPtr = (_field != 0) ? >> UseTempBuffer(Definition().DescribeSubFields()) : 0; >> >> return returnPtr; >> } >> >> I think the problem lies somewhere with the destruction of temporary >> variables. The temporary c4_String variable returned from >> DescribeSubFields get descructed before Definition() or >> DescribeSubFields() gets called. It seems totally hosed... I am in >> the process of discussing this with sun to see if it's a bug or not. > > > I've made changes in the source - better safe than sorry. In CVS now. > > Thanks. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From jcw@equi4.com Tue Jan 28 10:41:48 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h0SGflc21268 for ; Tue, 28 Jan 2003 10:41:47 -0600 Date: Tue, 28 Jan 2003 17:43:39 +0100 Subject: Re: [Metakit] Compiling on Sparc Solaris 8 w/ CC and MetaKit 2.4.8 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list In-Reply-To: <3E36B177.3020807@wcom.com> Message-Id: X-Mailer: Apple Mail (2.551) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h0SGflc21268 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Dan Gregory wrote: [comments about Sun CC options:] > -features=%all,no%iddollar,tmplife > > The compiler assumes -features=%all,no%iddollar,no%tmplife when no > -features option is given, and this affects when temporary objects are > destroyed. [...] I changed the line you mentioned:   return _field != 0 ? UseTempBuffer(Definition().DescribeSubFields()) : 0; To something that ought to be safe (changes in CVS): const char* returnPtr = (_field != 0) ? UseTempBuffer(Definition().DescribeSubFields()) : 0; return returnPtr; I can tweak it further if needed. As you suggest, I would prefer to have code which works with default compiler settings. I'm sure my change won't affect other compilers that much, and the code in question is not truly performance critical anyway. -jcw From bkelley@wi.mit.edu Wed Jan 29 11:05:02 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h0TH52c14187 for ; Wed, 29 Jan 2003 11:05:02 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.0.0.44) with SMTP id M2003012912065421077 for ; Wed, 29 Jan 2003 12:06:55 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id H9HJJI00.G6M for ; Wed, 29 Jan 2003 12:06:54 -0500 Message-ID: <3E380B9B.9090706@wi.mit.edu> Date: Wed, 29 Jan 2003 12:12:59 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0) Gecko/20020530 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: multipart/mixed; boundary="------------050107050805090407030503" Subject: [Metakit] Python metakit question about metakit.wrap Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------050107050805090407030503 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Last year there was a post from Tom Jones about database of recipes and ingredients: http://trixie.triqs.com/pipermail/metakit/2002-June/000729.html There was some wierd behavior using metakit.rwrap and view.minus It turns out that metakit.wrap wrapped views act strangely for quite a few view operations. I have updated the example above to compare different ways of performing the same task which is performing a test to see if a list of ingredients from a recipe is contained in another list of available ingredients. In short four approaches to producing the list of available ingredients are: 1) making a new table in the database (works) 2) making a temporary non-stored view (works) 3) wrapping a python dictionary (fails) 4) wrapping a python list (fails) When inspected under metakit.dump all four views appear to be correct and identical. I'll make this into a little metakit HOW-TO if someone can explain why views created by metakit.wrap don't seem to work :) I have a python version of metakit.wrap that seems to do what I want. (which is essentially 2) If this is the "correct" behavior I'll post it as well. Attached is the test code -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 --------------050107050805090407030503 Content-Type: text/plain; name="test_meta.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="test_meta.py" import metakit # metakit set operations example db = metakit.storage() recipes = db.getas("recipes[name:S,ingredients[ingredient:S]]") ts = 'tabasco sauce' v = 'vodka' tj = 'tomato juice' cj = 'cramberry juice' lj = 'lime juice' i = recipes.append(name='bloody mary') recipes[i].ingredients.append(ingredient=ts) recipes[i].ingredients.append(ingredient=v) recipes[i].ingredients.append(ingredient=tj) i = recipes.append(name='cosmopolitan') recipes[i].ingredients.append(ingredient=v) recipes[i].ingredients.append(ingredient=cj) recipes[i].ingredients.append(ingredient=lj) def checkView(view, recipes=recipes): metakit.dump(view) found = [] for r in recipes: left = r.ingredients.minus(view) if len(left) == 0: found.append(r.name) if "bloody mary" not in found: print "\tERORR: failed finding bloody mary" if "cosmopolitan" not in found: print "\tERROR: failed finding cosmopolitan" views = [] print "*"*44 print "make a new table in the view" ingredients = db.getas("ingredients[ingredient:S]") ingredients.append(ingredient=v) ingredients.append(ingredient=cj) ingredients.append(ingredient=lj) ingredients.append(ingredient=tj) ingredients.append(ingredient=ts) checkView(ingredients) views.append(('new table', ingredients)) print "*"*44 print "make a new non stored temporary view" ingredients = metakit.view() ingredients.addproperty(metakit.property('S', "ingredient")) ingredients.append(ingredient=v) ingredients.append(ingredient=cj) ingredients.append(ingredient=lj) ingredients.append(ingredient=tj) ingredients.append(ingredient=ts) checkView(ingredients) views.append(('temporary table', ingredients)) # print "*"*44 print "make a view from a wrapped dictionary" have = [ { 'ingredient' : v }, { 'ingredient' : cj }, { 'ingredient' : lj}, { 'ingredient' : tj }, { 'ingredient' : ts }, ] ingredients = metakit.wrap( have, [metakit.property('S', "ingredient")] ) checkView(ingredients) views.append(('wrapped dictionary', ingredients)) # print "*"*44 print "make a view from a wrapped list" have = [("vodka",), ("cramberry juice",), ("lime juice",), ("tomato juice",), (ts,)] ingredients = metakit.wrap( have, [metakit.property('S', "ingredient")], 1 ) checkView(ingredients) views.append(('wrapped list', ingredients)) print "testing tables versus wrapped tables" i = 0 for n, v in views: i += 1 for n2, v2 in views[i:]: if len(v.minus(v2)) != 0: print "minus:\t\t", `n`, "minus test failed with", `n2` if len(v.intersect(v2)) != len(v): print "intersection:\t", `n`, "intersection", `n2`, "failed" --------------050107050805090407030503-- From kris@cs.ucsb.edu Tue Feb 4 15:27:53 2003 Received: from letters.cs.ucsb.edu (letters.cs.ucsb.edu [128.111.41.13]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h14LRrc22034 for ; Tue, 4 Feb 2003 15:27:53 -0600 Received: from merrimack.cs.ucsb.edu (merrimack.cs.ucsb.edu [128.111.68.227]) by letters.cs.ucsb.edu (8.11.6+Sun/8.11.6) with ESMTP id h14LTmj23950 for ; Tue, 4 Feb 2003 13:29:48 -0800 (PST) From: "Kristian G. Kvilekval" To: metakit@equi4.com Content-Type: text/plain Organization: Message-Id: <1044394188.27369.25.camel@merrimack> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.1 Date: 04 Feb 2003 13:29:48 -0800 Content-Transfer-Encoding: 7bit Subject: [Metakit] Beginner Q's Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am adding metakit to zinf (a music player) to manage the music metadata. So far I have been very pleased, but I ran into something that I thought would be really simple, but can't find any docs or examples (I've looked through the mailing list too). Sooo.. I have database with const char dbview[] = "dbview[url:S,type:I,artist:S,album:S,title:S, comment:S,genre:S,track:I,year:I,time:I,size:I, playcount:I,guid:S]"; and I want to extract tracks or playlist files. How? m_dbview = m_database->GetAs (dbview); c4_View vtitles = m_dbview.Select(m_pType[kTypeTrack]+m_pType[kTypePlaylist]); would appear to ask for things that are both a track *and* a playlist. What's the proper way to ask for a track *or* playlist? Also somewhere I saw that Find is faster than Select if you are just going to iterate through the selected view. Is this correct? Thx. -- email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris From jcw@equi4.com Tue Feb 4 18:23:16 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h150NGc28705 for ; Tue, 4 Feb 2003 18:23:16 -0600 Date: Wed, 5 Feb 2003 01:25:13 +0100 Subject: Re: [Metakit] Beginner Q's Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <1044394188.27369.25.camel@merrimack> Message-Id: <4AD5EEEC-38A0-11D7-B0C4-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Kristian G. Kvilekval wrote: (manually accepted post - please consider subscribing) > const char dbview[] = > "dbview[url:S,type:I,artist:S,album:S,title:S, > comment:S,genre:S,track:I,year:I,time:I,size:I, > playcount:I,guid:S]"; > > and I want to extract tracks or playlist files. How? > > m_dbview = m_database->GetAs (dbview); > c4_View vtitles = > m_dbview.Select(m_pType[kTypeTrack]+m_pType[kTypePlaylist]); > > would appear to ask for things that are both a track *and* a playlist. > What's the proper way to ask for a track *or* playlist? probably: c4_View v1 = m_dbview.Select(m_pType[kTypeTrack]); c4_View v2 = m_dbview.Select(m_pType[kTypePlaylist]); c4_View vtitles = v1.Union(v2). > Also somewhere I saw that Find is faster than Select if you are just > going to iterate through the selected view. Is this correct? Yes, maybe. Under say 10,000 rows I wouldn't expect much difference. -jcw From cco+metakit@asitturnsout.org Wed Feb 5 16:37:14 2003 Received: from albert.asitturnsout.org (postfix@dsl-156-051.atm02.sea.blarg.net [206.124.156.51]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h15MbEc12991 for ; Wed, 5 Feb 2003 16:37:14 -0600 Received: from webmail.asitturnsout.org (localhost [127.0.0.1]) by albert.asitturnsout.org (Postfix) with ESMTP id DD0468D7F0 for ; Wed, 5 Feb 2003 14:39:11 -0800 (PST) From: "Chris Olds" To: metakit@equi4.com Date: Wed, 5 Feb 2003 14:39:11 -0800 Message-Id: <20030205223911.M99554@asitturnsout.org> X-Mailer: Open WebMail 1.81 20021127 X-OriginatingIP: 206.124.156.51 (cco) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] Q: rowids? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm working on a calendering/PIM application, and I have several independent datasets I need to work with (all in the same MK storage). Can anyone give me a recipe for generating surrogate key values? Since I'm only allowing one opener for the storage, I suppose I could get the last key value used by sorting the view and giving new rows a key value greater than the last value used, but that doesn't feel like the best solution - I've spent years avoiding just this kind of thing in SQL... To make this concrete, here's how it would look in Python: import metakit as mk db = mk.storage() v = db.getas("contact[key:I,name:S,address:S,city:S,state:S,zip:I]") nextkey = 1 if v: nextkey = v.sort(v.key)[-1]+1 and then any inserts into the view would use the current value of nextkey and increment it (all of this is actually in a class that keeps all of the ugly stuff hidden, so the keeping track part isn't as bad as it looks here). I don't want to use the row numbers returned when I add a row, because then I'd be up a creek if I ever deleted a row (which I don't expect to do a lot of, but I don't want to preclude). So - is there a better way? /cco -- GPG Key Fingerprint: B375 A4E7 752B DB8C 4359 852E C3CF BF64 379A E9B2 Debian Project (http://www.debian.org) From 1jerry@usd1.com Wed Feb 5 22:22:37 2003 Received: from bfish.hurrah.com (root@bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h164Mbc04693 for ; Wed, 5 Feb 2003 22:22:37 -0600 Received: from CTL (12-211-33-9.client.attbi.com [12.211.33.9]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id h164OYo18713; Wed, 5 Feb 2003 20:24:34 -0800 Date: Wed, 5 Feb 2003 20:18:58 -0800 From: Jerry McRae <1jerry@usd1.com> X-Mailer: The Bat! (v1.60q) Personal Organization: usd1 X-Priority: 3 (Normal) Message-ID: <122186717553.20030205201858@usd1.com> To: "Chris Olds" CC: metakit@equi4.com Subject: Re: [Metakit] Q: rowids? In-Reply-To: <20030205223911.M99554@asitturnsout.org> References: <20030205223911.M99554@asitturnsout.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Chris, My response to what you wrote me on Wednesday, February 5, 2003 at 2:39 PM: Why not always append and get the next key with: nextkey = v[-1].key + 1 or have you already rejected this? -- /\_/\ Jerry McRae \|||/ mail safely: The Bat! 1.60q (~o o~) (. .) )'Y'( _______________-o00-(_)-00o-___________________________ ( ) / If I had a nickel for every time Windows crashes... Oh yea, I do! --Bill Gates ========== your original message below ========== From: Chris Olds To: metakit@equi4.com on: 2:39:11 PM re: [Metakit] Q: rowids? wise words: CO> I'm working on a calendering/PIM application, and I have several independent CO> datasets I need to work with (all in the same MK storage). Can anyone give me CO> a recipe for generating surrogate key values? Since I'm only allowing one CO> opener for the storage, I suppose I could get the last key value used by CO> sorting the view and giving new rows a key value greater than the last value CO> used, but that doesn't feel like the best solution - I've spent years avoiding CO> just this kind of thing in SQL... CO> To make this concrete, here's how it would look in Python: CO> import metakit as mk CO> db = mk.storage() CO> v = db.getas("contact[key:I,name:S,address:S,city:S,state:S,zip:I]") CO> nextkey = 1 CO> if v: CO> nextkey = v.sort(v.key)[-1]+1 CO> and then any inserts into the view would use the current value of nextkey and CO> increment it (all of this is actually in a class that keeps all of the ugly CO> stuff hidden, so the keeping track part isn't as bad as it looks here). CO> I don't want to use the row numbers returned when I add a row, because then CO> I'd be up a creek if I ever deleted a row (which I don't expect to do a lot CO> of, but I don't want to preclude). CO> So - is there a better way? CO> /cco CO> -- CO> GPG Key Fingerprint: B375 A4E7 752B DB8C 4359 852E C3CF BF64 379A E9B2 CO> Debian Project (http://www.debian.org) CO> _______________________________________________ CO> metakit mailing list - metakit@equi4.com CO> http://www.equi4.com/mailman/listinfo/metakit From forstercolin@yahoo.co.uk Thu Feb 6 04:34:18 2003 Received: from web11301.mail.yahoo.com (web11301.mail.yahoo.com [216.136.131.204]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h16AYIc29895 for ; Thu, 6 Feb 2003 04:34:18 -0600 Message-ID: <20030206103617.88612.qmail@web11301.mail.yahoo.com> Received: from [165.76.33.39] by web11301.mail.yahoo.com via HTTP; Thu, 06 Feb 2003 10:36:17 GMT Date: Thu, 6 Feb 2003 10:36:17 +0000 (GMT) From: =?iso-8859-1?q?Colin=20Forster?= To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] Need some advice for Find/Locate/Search Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I have a DB which includes a start date property and end date property. Given a date I'd like MK to return the the rows where the date is greater-than-or-equal-to the start date and less-than-or-equal-to the end date. Note that a single call to return all rows is not necessary; multiple calls is fine. I've found documentation that says it can be done but nothing to say how it is done. BTW, I'm coding using the C++ API. Any advice muchly appreciated. Col __________________________________________________ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com From gmcm@hypernet.com Thu Feb 6 08:52:50 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h16Eqnc16428 for ; Thu, 6 Feb 2003 08:52:49 -0600 Received: from PARANOIA (204.176.40.73) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3) for ; Thu, 6 Feb 2003 09:58:20 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Thu, 6 Feb 2003 09:54:34 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Q: rowids? Reply-to: gmcm@hypernet.com Message-ID: <3E4230DA.12267.DDC8AB0B@localhost> Priority: normal In-reply-to: <20030205223911.M99554@asitturnsout.org> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 5 Feb 2003 at 14:39, Chris Olds wrote: > I'm working on a calendering/PIM application, and I > have several independent datasets I need to work > with (all in the same MK storage). Can anyone give > me a recipe for generating surrogate key values? I have a couple techniques, depending on usage. One involves a view "maxids[viewname:S,id:I]", whose use should be pretty obvious. The downsides are (1) remembering to wrap all append / inserts, (2) remembering to initialize new storages, and (3) you may eventually have problems with very volatile data. In apps where I can afford some over head on open, I'll often use a "...,_isdel:I]" flag. I delete anything where _isdel is 1 when I open the storage. During the run, it's safe to use row numbers. The first technique is good for CGI, the 2nd for GUIs. > Since I'm only allowing one opener for the storage, > I suppose I could get the last key value used by > sorting the view and giving new rows a key value > greater than the last value used, but that doesn't > feel like the best solution - I've spent years > avoiding just this kind of thing in SQL... Why sort it? Scan on open for the maxid, and maintain that in memory. It's lower overhead, and doesn't interfere with whatever ordering the app might want to see or maintain. -- Gordon http://www.mcmillan-inc.com/ From jcw@equi4.com Thu Feb 6 09:15:21 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h16FFKc18282 for ; Thu, 6 Feb 2003 09:15:20 -0600 Date: Thu, 6 Feb 2003 16:17:17 +0100 Subject: Re: [Metakit] Q: rowids? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3E4230DA.12267.DDC8AB0B@localhost> Message-Id: <14427500-39E6-11D7-8730-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: > Why sort it? Scan on open for the maxid, and > maintain that in memory. It's lower overhead, and > doesn't interfere with whatever ordering the app > might want to see or maintain. Worth repeating, because it highlights a fundamental aspect of MK's column-wise data storage model. You can open a datafile of any size, point at a view with 10s of thousands of "rows" of any complexity, and still do the above scan-on-open with no other overhead than one read of a few Kb off the disk. This is to efficient, that any other approach is a waste of effort in most cases. Column-wise data storage means a scan over one property is f a s t. It would be even faster if MK had C-coded loop aggregate functions such as "max", but hey - there need to be some goodies saved up for later :) One last point on this. This sort of tight max-scan loop takes maximum advantage of CPU caches (even more so if coded in C). On modern CPU's, that equates to "warp drive". -jcw From kris@cs.ucsb.edu Wed Feb 12 15:37:36 2003 Received: from letters.cs.ucsb.edu (letters.cs.ucsb.edu [128.111.41.13]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1CLbZc11198 for ; Wed, 12 Feb 2003 15:37:36 -0600 Received: from merrimack.cs.ucsb.edu (merrimack.cs.ucsb.edu [128.111.68.227]) by letters.cs.ucsb.edu (8.11.6+Sun/8.11.6) with ESMTP id h1CLddw28305 for ; Wed, 12 Feb 2003 13:39:39 -0800 (PST) Subject: [Metakit] Beginner Q's From: "Kristian G. Kvilekval" To: metakit@equi4.com Content-Type: text/plain Organization: Message-Id: <1045085979.21138.53.camel@merrimack> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 12 Feb 2003 13:39:39 -0800 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >> const char dbview[] => "dbview[url:S,type:I,artist:S,album:S,title:S,> comment:S,genre:S,track:I,year:I,time:I,size:I,> playcount:I,guid:S]";>> and I want to extract tracks or playlist files. How? >> m_dbview = m_database->GetAs (dbview); > c4_View vtitles = > m_dbview.Select(m_pType[kTypeTrack]+m_pType[kTypePlaylist]); >> would appear to ask for things that are both a track *and* a playlist.> What's the proper way to ask for a track *or* playlist? >probably: > c4_View v1 = m_dbview.Select(m_pType[kTypeTrack]); > c4_View v2 = m_dbview.Select(m_pType[kTypePlaylist]); > c4_View vtitles = v1.Union(v2). > Also somewhere I saw that Find is faster than Select if you are just> going to iterate through the selected view. Is this correct? >Yes, maybe. Under say 10,000 rows I wouldn't expect much difference. So is there also a way to do an *or* query using Find? -- email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris From tp@diffenbach.org Thu Feb 13 01:43:02 2003 Received: from hostica.com (ww25.hostica.com [216.116.124.25]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h1D7h1c18257 for ; Thu, 13 Feb 2003 01:43:01 -0600 Received: (qmail 32687 invoked from network); 13 Feb 2003 07:45:05 -0000 Received: from pool-138-88-61-147.res.east.verizon.net (HELO W2K) (138.88.61.147) by ww25.hostica.com with SMTP; 13 Feb 2003 07:45:05 -0000 Reply-To: From: To: Date: Thu, 13 Feb 2003 02:44:31 -0500 Message-ID: <000501c2d333$c2323ba0$f2a7fea9@W2K> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] string.cpp: static linkage of strcasecmp conflicts with Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm having trouble compiling metakit under cygwin with gcc 3.0.4: g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/string.cpp -DDLL_EXPORT -DPIC ../src/string.cpp: In function `int strcasecmp(const char*, const char*)': ../src/string.cpp:39: `int strcasecmp(const char*, const char*)' was declared `extern' and later `static' /usr/include/string.h:60: previous declaration of `int strcasecmp(const char*, const char*)' make: *** [string.o] Error 1 The compiler's seeing the Standard definition in because is included by mk4str.h, which is included by header.h, and q4_STD is (presumably) defined (line 30 of mk4str.h). Removing "static" at line 38 of string.cpp works, but I probably should have found something better. Please advise. -- Help me live longer! Help yourself to live longer! Run folding@home on your computer and help cure disease! Go to http://folding.stanford.edu P.S.: Please mail me ONLY at the address tp@diffenbach.org. Mail to other addresses will be discarded unread. -- end signature block -- ------------------------- From jcw@equi4.com Thu Feb 13 05:22:24 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1DBMNc02229 for ; Thu, 13 Feb 2003 05:22:23 -0600 Date: Thu, 13 Feb 2003 12:24:29 +0100 Subject: Re: [Metakit] string.cpp: static linkage of strcasecmp conflicts with Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <000501c2d333$c2323ba0$f2a7fea9@W2K> Message-Id: X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: wrote: > ../src/string.cpp: In function `int strcasecmp(const char*, const > char*)': > ../src/string.cpp:39: `int strcasecmp(const char*, const char*)' was > declared > `extern' and later `static' > /usr/include/string.h:60: previous declaration of `int strcasecmp(const > char*, > const char*)' > make: *** [string.o] Error 1 Can you telll me what source code you are using (latest entry in CHANGES, for example)? I'm currently making some changes in that area. May have broken something. -jcw From 1jerry@usd1.com Thu Feb 13 12:30:25 2003 Received: from bfish.hurrah.com (root@bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1DIUPc07566 for ; Thu, 13 Feb 2003 12:30:25 -0600 Received: from CTL (12-211-33-9.client.attbi.com [12.211.33.9]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id h1DIWT803622 for ; Thu, 13 Feb 2003 10:32:30 -0800 Date: Thu, 13 Feb 2003 10:25:59 -0800 From: Jerry McRae <1jerry@usd1.com> X-Mailer: The Bat! (v1.60q) Personal Organization: usd1 X-Priority: 3 (Normal) Message-ID: <9315730184.20030213102559@usd1.com> To: Metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] finding rows in Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Am I dense or is there a way to find the original row from a 'viewer' instance? >>> vw=db.getas('test[name,pay[key,val:I]') >>> r=vw[1] >>> vw1=r.pay.select(key='b',val=1) # now I have a subset Now I have the subset I wish to work with, but I need to modify or delete the *original* rows. In this instance, I have no urid (unique row id). I have the same problem with vw.sortrev(), which I have to do a vw.find(urid=...) for every record I want to process. (Didn't sortrev used to pass changes through?) Is there any way to identify or access the original row without adding a dummy urid to each one and searching? -- Thanks, Jerry McRae MetaKit 2.4.8, Python 2.2, wxPython 2.2.5, Win98, 384M RAM From bowmanjm@attbi.com Thu Feb 13 21:05:55 2003 Received: from sccrmhc03.attbi.com (sccrmhc03.attbi.com [204.127.202.63]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1E35tc17020 for ; Thu, 13 Feb 2003 21:05:55 -0600 Message-Id: <200302140305.h1E35tc17020@trixie.triqs.com> Received: from there (h002078c644e3.ne.client2.attbi.com[65.96.178.255]) by sccrmhc03.attbi.com (sccrmhc03) with SMTP id <2003021403075400300dhb61e>; Fri, 14 Feb 2003 03:07:54 +0000 Content-Type: text/plain; charset="iso-8859-1" From: Jeremy Bowman To: Metakit mailing list Date: Thu, 13 Feb 2003 22:06:48 -0500 X-Mailer: KMail [version 1.3.2] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Metakit] Re: Metakit string sorting Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: (Moving to the mailing list as requested...) --- Jean-Claude Wippler wrote: > Jeremy, > > (Please consider taking this discussion to the MK mailing list!) > > > The issue of how to allow more versatile string sorting (particularly > > case-sensitive sorting and Unicode sorting) seems to have come up on > > the mailing list several times...do you have any plans [...] > > Yes, see Jeffey Kay's post to this list on Dec 26, 2002, for example. > > I had started to look into that and had to conclude that the change we > thought would work was not dealing with all cases - unfortunately. > > > You've mentioned some of the issues before...as I see it, the ideal > > solution would allow: > > > > 1. Case sensitive or insensitive sorting > > 2. Sorting by Unicode code point > > 3. Using locale-dependent sorting rules > > 4. Set the default sorting rules for the application as a whole > > 5. Override the default sorting for particular properties > > Thanks for summarizing. > > There are two different sides to this - sorting as view operation (be > it one-shot or dynamically tracking), and maintaining persistent views > sorted. > > The latter can be managed manually, i.e. find the right spot to insert, > etc. It's not perfect, but it works - and the code to locate the spot > can be outside MK and use whatever collation rules it needs. I hadn't really considered the persistently sorted case, although I can see how it would be useful in many applications. In my case, I've written a data management tool which lets the user sort the records any way they want, and change quickly between user-defined sortings. In fact, these sortings can be by any sequence of properties, each in either ascending or descending order. For this, a fast and flexible sorting operation is critical. The current SortOnReverse() is perfect, except for the limited string comparison capabilities. > Now the sort operator. > > I've been pondering on this on and off indeed, since the current lack > of support even just for UTF-8 is really a serious weak spot in MK. > It's surprising how far we got without even configurable case-sens vs. > case-insens, in fact. > > All the considerations you list are important. Global sort settings > would be awkward - for a library. Per-view sort settings need to be > carefully thought out - since comparisons are at times between > different views. Per storage settings might be technically easier > (without losing efficiency), but too coarse. > > But I think there is a solution in another direction. > > I'm starting to see MK as a number of independent concepts. On the one > side are storage and representation, on the other are manipulation and > derivation. MK 2.3/2.4 added a lot of custom views, but these are in > fact almost completely independent of the rest of the library. > > The "sort object" used to capture a sorted view, for example, is > nothing but a permutation vector. > > We have RemapWith as operation. This means that if you create a > permutation vector, it's a small step to "apply" it to a view and > perform the sort. > > My suggestion would be to look for a solution in this direction. We > can define one or more sort operations outside what is currently MK. > For a first approximation, the permutation can be a simple view with > one int, being row indices. That would work today, i.e. > c4_View sorted = origview.RemapWith(permutationview); > > If performance becomes an issue, I would *much* rather work on > overcoming that with say a batch vector transfer, or variant of > RemapWith which takes an int[]. > > The path I would like to go, is to actually reduce the MK core and > create a separate manipulation / operator library on top. With very > strong attention to performance issues - in fact, I think there is > still a lot of slack left to take advantage of. This makes sense, especially since there's already the beginnings of such a separation. The important thing would be to allow easy customization of these operations without needing to re-implement large portions of them. For example, it would be much easier to call something like SortOnReverse() with a custom set of value comparison functions either passed as an argument or set as a property somewhere, than have to define a new sorting function that re-implements all the logic needed to sort a combination of properties (each in either ascending or descending order). While I could do the latter now and use RemapWith() to apply the results, it's an awful lot of coding for something I can almost do with the built-in operation. It might work well to design the view operations as classes which can be subclassed easily, overriding only the methods which implement behavior you want to change; it's hard to do that with the current design, where a lot of the steps in the process are performed by global functions or static member functions (like everything dealing with value comparisons...) > For Python/Tcl, all of the above would work well. With as escape an > *additional* comparison API which calls back into those languages. For > flexibility only, not for speed. > > Note that this has implications in several other areas. This same > remapping can also be used for selection, repetition, reversal, > slicing. In fact, that's how many of the current new view operations > work. The most important one for me is complex selection - with an > additional expression parsing layer, it can generate flag columns > (calculated fields, essentially). One ("filter") run over those and we > get row numbers of all "true" entries, so to speak - ready for > remapping. I've already implemented a form of "complex selection" in PortaBase. It's actually easier to do this than to customize the sorting order, because you only need to track "does this record get added or not?" instead of "what index does this record go at?" So I defined a "Condition" class that defines and applies a single-property selection; it runs through all the values of that property and does either numeric comparisons (equality, inequality, etc.) or string comparisons (equality, inequality, starts with, and contains; each either case-sensitive or case-insensitive) against a constant value, returning a new view containing only the records that match the condition. A "Filter" class then combines a sequence of conditions by passing the results of one into the next. I don't need to worry about sorting order during all this because I apply the current sorting to the final, filtered view (this also has the benefit of needing to sort fewer records than if the entire view was sorted). Performance seems to be respectable, people are using this on PDAs with views containing over 10,000 records and have been impressed with the speed. So back to the original topic...all I really need in the foreseeable future is the ability to specify the string comparison function used in sorting (and preferably in selecting also; both currently use f4_CompareFormat()). It can be a function pointer or a virtual member function that can be overridden by subclassing, whatever works. If the Sort(), SortOn(), and SortOnReverse() methods are among the things you'd like to see moved up into a higher layer, this would make a decent set of things to move into one class (although a few other things like Indexed() use SortOn(), so those would also have to be either moved into the higher layer or reworked a bit). Then there could be an overridable comparison method (possibly calling a set of other methods that could be overridden, so you could override just the string comparison behavior, etc.) This method could be either part of the sorting class itself or part of a separate comparison class either inherited by or used by the sorting class (the latter would make it easier to reuse for a filtering class). Let me know if you have more questions/comments, want to start a design yourself, or want me to attempt a more detailed design. Jeremy ------------------------------------------------------------------------- Original message, for those of you who didn't see it: The issue of how to allow more versatile string sorting (particularly case-sensitive sorting and Unicode sorting) seems to have come up on the mailing list several times...do you have any plans to address this in the near future? I'm currently using a version of 2.4.8 patched to allow case-sensitive comparisons globally, but the inability to sort Unicode strings correctly is an open bug in my program (PortaBase, at http://portabase.sourceforge.net). It'd be nice to be able to fix this sometime in the next few months, and I can probably contribute some code for it or help with the design if you'd like. I just released versions of PortaBase for Debian and Windows, and especially for Debian it would be nice to eventually be able to use an official shared library version of Metakit instead of statically linking to my patched version. You've mentioned some of the issues before...as I see it, the ideal solution would allow: 1. Case sensitive or insensitive sorting 2. Sorting by Unicode code point 3. Using locale-dependent sorting rules 4. Set the default sorting rules for the application as a whole 5. Override the default sorting for particular properties (1) isn't too difficult to do at a global level, code for this has been posted before (and I'm currently using a variant of that.) (2) could be done, but isn't useful as often as (3) and would require either assuming or letting developers specify a text encoding (usually UTF-8, I imagine.) I strongly suspect (3) is too complicated to implement as part of Metakit without large amounts of work and a substantial increase in the size of the library. (4) and (5) are points that should be kept in mind when designing the API in order to accommodate a variety of use cases. Since class libraries like Qt (http://www.trolltech.com) and IBM's ICU (http://oss.software.ibm.com/icu/) already have implementations of (3), I think the cleanest solution is to add a mechanism for applications to define their own functions to use for comparison (with a default provided that implements the current behavior). At first glance, doing this would require the following: - Define an API for the comparison function; I guess use a function pointer for this? Maybe "typedef int ((*CompareFunc)(const c4_Bytes&, const c4_Bytes&));" (never used function pointers before, I hope that's right...) This definition would have the advantage of making it possible to similarly define custom sorting rules for other property types as well. - Add a static property of type CompareFunc somewhere (in c4_Property?) to use as the default string comparison function throughout the application; by default it would have a value equivalent to the current c4_FormatS::DoCompare(). The value of this pointer could be changed to dynamically change the global CompareFunc. (Again, by defining a static CompareFunc for each property type, this extends to allow custom sorting of any kind of property.) - Add "CompareFunc c4_Property::GetCompareFunc()" and "void c4_Property::SetCompareFunc(CompareFunc)". If not explicitly set (or if reset by passing a null pointer), the current global CompareFunc for the property's type would be returned by the get function. - In "c4_Handler::Compare()", replace "f4_CompareFormat(Property().Type(), data, copy)" with "(*(Property().GetCompareFunc()))(data, copy)". I'd expect this to be competitive with the current implementation in terms of performance, maybe just a little faster or slower. - Expose this somehow to the Python and Tcl wrappers. I can't imagine that defining a comparison function in Python and calling it from C++ for each comparison would do anything good for performance. It shouldn't be too hard to create methods allowing you to choose between a few default implementations coded in C++ (case sensitive, case insensitive, maybe UTF-8 code point order), but this still wouldn't really allow for locale-dependent sorting. Might be possible to use Python's internal C code for doing locale-specific string comparison (exposed in Python as locale.strcoll(string1, string2)) as one of the default options. (Can you tell that I've never used Tcl...?) Open to ideas on this. Let me know what you think, Jeremy From kris@cs.ucsb.edu Fri Feb 14 00:59:14 2003 Received: from fed1mtao03.cox.net (fed1mtao03.cox.net [68.6.19.242]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1E6xEc29335 for ; Fri, 14 Feb 2003 00:59:14 -0600 Received: from [192.168.123.139] ([68.6.94.123]) by fed1mtao03.cox.net (InterMail vM.5.01.04.05 201-253-122-122-105-20011231) with ESMTP id <20030214070056.HWJF12976.fed1mtao03.cox.net@[192.168.123.139]> for ; Fri, 14 Feb 2003 02:00:56 -0500 From: Kristian Kvilekval To: Metakit mailing list In-Reply-To: <200302140305.h1E35tc17020@trixie.triqs.com> References: <200302140305.h1E35tc17020@trixie.triqs.com> Content-Type: text/plain Organization: Message-Id: <1045206056.31789.9.camel@krispc> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 13 Feb 2003 23:00:56 -0800 Content-Transfer-Encoding: 7bit Subject: [Metakit] metakit<-->XML routine? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Has anybody created the routines that would allow the database to be stored as a simple XML document? For example: paper[title:S,url:S,author[last:S,first:S] Would be dumped as: A paper http://somewhere.edu someone .... Any pointers would be appreciated.. Thx. -- Kristian G. Kvilekval email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris From nicolasb@maich.gr Fri Feb 14 01:31:19 2003 Received: from mail-server ([193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h1E7VIc31252 for ; Fri, 14 Feb 2003 01:31:18 -0600 Received: FROM maich.gr BY mail-server ; Fri Feb 14 10:06:48 2003 +0200 Message-ID: <3E4C9D2B.6060305@maich.gr> Date: Fri, 14 Feb 2003 09:39:23 +0200 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0rc2) Gecko/20020510 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kristian Kvilekval CC: Metakit mailing list Subject: Re: [Metakit] metakit<-->XML routine? References: <200302140305.h1E35tc17020@trixie.triqs.com> <1045206056.31789.9.camel@krispc> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Kristian Kvilekval wrote: > Has anybody created the routines that would allow the database > to be stored as a simple XML document? > > For example: > > paper[title:S,url:S,author[last:S,first:S] > > Would be dumped as: > > > > A paper > http://somewhere.edu > > someone > > .... > > > > Any pointers would be appreciated.. > > > Thx. > > Hi, IIRC, JC did some mk<->xml stuff awhile back. This might help http://www.equi4.com/pub/etc/xml2.README regards, nicolas boretos From MStanczak@znet.com Fri Feb 14 03:06:56 2003 Received: from la.znet.com (la.znet.com [207.167.96.1]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1E96uc05347 for ; Fri, 14 Feb 2003 03:06:56 -0600 Received: from trying (lats02-59.znet.net [207.167.96.59]) (authenticated bits=0) by la.znet.com (8.12.6/8.12.6/le1-la) with ESMTP id h1E98v0B020970 for ; Fri, 14 Feb 2003 01:08:58 -0800 (PST) X-Envelope-From: MStanczak@znet.com X-Envelope-To: From: Mariusz Stanczak To: metakit@equi4.com Date: Fri, 14 Feb 2003 04:09:00 -0500 Organization: Forth Bits Message-ID: X-Mailer: Forte Agent 1.93/32.576 English (American) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Errors from MKTEST Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I'm a new user. After compiling the 2.4.8.38 version with MSVC6 the mktest.exe run results in a large number of failed tests. Starting with test r00, with the exception of tests s42, and m01, all the tests output "Failed: A(fp)"... this can't be good, can it? I used the provided MSVC workspace/project files with no changes on a W2K/Athlon system. Thanks for any pointers/elucidations, From j.p.fletcher@aston.ac.uk Fri Feb 14 03:49:46 2003 Received: from mercury.aston.ac.uk (mercury.aston.ac.uk [134.151.79.53]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1E9njc08330 for ; Fri, 14 Feb 2003 03:49:46 -0600 Received: from [134.151.79.13] (helo=email.aston.ac.uk) by mercury.aston.ac.uk with esmtp (Exim 3.30 #1) id 18jcTa-0006Cu-00; Fri, 14 Feb 2003 09:49:46 +0000 Received: from ceac133 ([134.151.50.133]) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 18jcVW-00035e-00; Fri, 14 Feb 2003 09:51:46 +0000 From: "John Fletcher" To: Kristian Kvilekval Date: Fri, 14 Feb 2003 09:49:43 -0000 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] metakit<-->XML routine? CC: metakit@equi4.com Priority: normal References: <200302140305.h1E35tc17020@trixie.triqs.com> In-reply-to: <1045206056.31789.9.camel@krispc> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Kristian Kvilekval To: Metakit mailing list Organization: Subject: [Metakit] metakit<-->XML routine? Date sent: 13 Feb 2003 23:00:56 -0800 > > Has anybody created the routines that would allow the database > to be stored as a simple XML document? > > For example: > > paper[title:S,url:S,author[last:S,first:S] > > Would be dumped as: > > > > A paper > http://somewhere.edu > > someone > > .... > > > > Any pointers would be appreciated.. > > > Thx. > > > -- > Kristian G. Kvilekval > email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > Look at the e4graph extension of metakit. John ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From jcw@equi4.com Fri Feb 14 05:25:33 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1EBPWc14042 for ; Fri, 14 Feb 2003 05:25:33 -0600 Date: Fri, 14 Feb 2003 12:27:37 +0100 Subject: Re: [Metakit] Errors from MKTEST Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <51F77621-400F-11D7-ACAD-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mariusz Stanczak wrote: > I'm a new user. After compiling the 2.4.8.38 version with MSVC6 the > mktest.exe run results in a large number of failed tests. Starting > with > test r00, with the exception of tests s42, and m01, all the tests > output > "Failed: A(fp)"... this can't be good, can it? > I used the provided MSVC workspace/project files with no changes on a > W2K/Athlon system. It sounds like the "tests" directory is not there. In the current directory during the tests, do "mkdir tests", then run again. The regression test program only standard stdio/stdlib functions - it has no way of doing the mkdir cross-platform. -jcw From jyl@best.com Fri Feb 14 08:51:21 2003 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h1EEpLc25628 for ; Fri, 14 Feb 2003 08:51:21 -0600 Received: (qmail 5849 invoked by uid 19667); 14 Feb 2003 14:53:26 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 14 Feb 2003 14:53:26 -0000 Message-ID: <3E4D02D3.2080309@best.com> Date: Fri, 14 Feb 2003 06:53:07 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Nicolas Boretos CC: Kristian Kvilekval , Metakit mailing list Subject: Re: [Metakit] metakit<-->XML routine? References: <200302140305.h1E35tc17020@trixie.triqs.com> <1045206056.31789.9.camel@krispc> <3E4C9D2B.6060305@maich.gr> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Also take a look at http://www.e4graph.com/e4graph/e4xml.html as well as http://www-106.ibm.com/developerworks/webservices/library/ws-xtcl.html?dwzone=webservices JYL Nicolas Boretos wrote: > Kristian Kvilekval wrote: > >> Has anybody created the routines that would allow the database >> to be stored as a simple XML document? >> >> For example: >> >> paper[title:S,url:S,author[last:S,first:S] >> >> Would be dumped as: >> >> >> >> A paper >> http://somewhere.edu >> >> someone >> >> .... >> >> >> >> Any pointers would be appreciated.. >> >> >> Thx. >> >> > > Hi, > > IIRC, JC did some mk<->xml stuff awhile back. This might help > > http://www.equi4.com/pub/etc/xml2.README > > regards, > > nicolas boretos > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From kris@cs.ucsb.edu Fri Feb 14 13:33:42 2003 Received: from letters.cs.ucsb.edu (letters.cs.ucsb.edu [128.111.41.13]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1EJXgc15844 for ; Fri, 14 Feb 2003 13:33:42 -0600 Received: from merrimack.cs.ucsb.edu (merrimack.cs.ucsb.edu [128.111.68.227]) by letters.cs.ucsb.edu (8.11.6+Sun/8.11.6) with ESMTP id h1EJZgw13232 for ; Fri, 14 Feb 2003 11:35:42 -0800 (PST) Subject: Re: [Metakit] metakit<-->XML routine? From: "Kristian G. Kvilekval" To: Metakit mailing list In-Reply-To: <3E4C9D2B.6060305@maich.gr> References: <200302140305.h1E35tc17020@trixie.triqs.com> <1045206056.31789.9.camel@krispc> <3E4C9D2B.6060305@maich.gr> Content-Type: text/plain Organization: Message-Id: <1045251341.3645.51.camel@merrimack> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 14 Feb 2003 11:35:41 -0800 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks for pointers! To summarize what I've found so far, I'll use the following example doc. Help with XML Joe jacker Happy Hacker http://www.equi4.com/pub/etc/xml2.README Describes converting a single XML document to several metakit views one for XML element. Attributes become become columns in tables. "book[_V:S,id:S],title[_V:S],author[_V:S] book[0]._V = "" book[0].id = 101 title[0]._V ="Help with XML" author[0]._V = "Joe jacker" author[1]._V = "Happy Hacker" http://www.equi4.com/metakit/xml/ Straight forward translation of the XML structure into a set of nested views. "xml[name:S,value:S,elements[ref:I,extra:S],attribs[name:S,value:S]]" xml[0].name = "book" xml[0].attribs[0].name = "id" .. xml[1].name = "title" ... http://www.e4graph.com/e4graph/e4xml.html Parses e4graph files looking for special graph tags. Can be used to parse general XML files, but all elements end up being e4nodes. None of these seem to fit the bill. It seems that there is a natural mapping between many XML document types and the metakit format. The ideal would be the document would discovered to be: "book[id:I,title:S, author[name:S]]" Single leaf XML elements or attributes would create columns and nesting elements would create nested views. This structure could be discovered in three ways: 1. Providing a metakit tag at the beginning of the document metakit view="book[id:I,title:S, author[name:S]] /> 2. Reading the DTD of the document if available. 3. Scanning the entire document for the elements. I think implementing #1 would not be too bad, then 3 and finally 2. Have I forgotten something important or missed the boat completely? Any comments or suggestions are appreciated. -- email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris From 1jerry@usd1.com Fri Feb 14 14:13:02 2003 Received: from bfish.hurrah.com (root@bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1EKD2c19443 for ; Fri, 14 Feb 2003 14:13:02 -0600 Received: from CTL (12-211-33-9.client.attbi.com [12.211.33.9]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id h1EKF7815220; Fri, 14 Feb 2003 12:15:07 -0800 Date: Fri, 14 Feb 2003 12:06:43 -0800 From: Jerry McRae <1jerry@usd1.com> X-Mailer: The Bat! (v1.60q) Personal Organization: usd1 X-Priority: 3 (Normal) Message-ID: <15108188255.20030214120643@usd1.com> To: "Kristian G. Kvilekval" CC: Metakit Subject: Re[2]: [Metakit] metakit<-->XML routine? In-Reply-To: <1045251341.3645.51.camel@merrimack> References: <200302140305.h1E35tc17020@trixie.triqs.com> <1045206056.31789.9.camel@krispc> <3E4C9D2B.6060305@maich.gr> <1045251341.3645.51.camel@merrimack> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Kristian, My response to what you wrote on Friday, February 14, 2003 at 11:35 AM: Thanks for this research. My only comment on your suggestion is that maybe a new "view" for each top level tag. This would get what you want, and if you ever decided to add another top level besides "book", it would handle it automatically. I look forward to a helpful conclusion to this thread. -- /\_/\ Jerry McRae \|||/ mail safely: The Bat! 1.60q (~o o~) 1(360)-574-4444 (. .) mailto:1jerry@usd1.com )'Y'( _______________-o00-(_)-00o-___________________________ ( ) / Where am I going, and what am I doing in this handbasket? ========== your original message below ========== From: Kristian G. Kvilekval To: Metakit mailing list on: 11:35:41 AM re: [Metakit] metakit<-->XML routine? wise words: KGK> Thanks for pointers! KGK> To summarize what I've found so far, I'll use the following example doc. KGK> KGK> Help with XML KGK> Joe jacker KGK> Happy Hacker KGK> KGK> http://www.equi4.com/pub/etc/xml2.README KGK> Describes converting a single XML document to several metakit views KGK> one for XML element. Attributes become become columns in tables. KGK> "book[_V:S,id:S],title[_V:S],author[_V:S] KGK> book[0]._V = "" book[0].id = 101 KGK> title[0]._V ="Help with XML" KGK> author[0]._V = "Joe jacker" KGK> author[1]._V = "Happy Hacker" KGK> http://www.equi4.com/metakit/xml/ KGK> Straight forward translation of the XML structure into a set of KGK> nested views. KGK> "xml[name:S,value:S,elements[ref:I,extra:S],attribs[name:S,value:S]]" KGK> xml[0].name = "book" xml[0].attribs[0].name = "id" .. KGK> xml[1].name = "title" KGK> ... KGK> http://www.e4graph.com/e4graph/e4xml.html KGK> Parses e4graph files looking for special graph tags. Can be KGK> used to parse general XML files, but all elements end up being KGK> e4nodes. KGK> None of these seem to fit the bill. It seems that there is a natural KGK> mapping between many XML document types and the metakit format. KGK> The ideal would be the document would discovered to be: KGK> "book[id:I,title:S, author[name:S]]" KGK> Single leaf XML elements or attributes would create columns and KGK> nesting elements would create nested views. KGK> This structure could be discovered in three ways: KGK> 1. Providing a metakit tag at the beginning of the document KGK> metakit view="book[id:I,title:S, author[name:S]] /> KGK> 2. Reading the DTD of the document if available. KGK> KGK> KGK> KGK> KGK> 3. Scanning the entire document for the elements. KGK> I think implementing #1 would not be too bad, then 3 and finally 2. KGK> Have I forgotten something important or missed the boat completely? KGK> Any comments or suggestions are appreciated. From jcw@equi4.com Fri Feb 14 15:36:04 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1ELa2c26227 for ; Fri, 14 Feb 2003 15:36:02 -0600 Date: Fri, 14 Feb 2003 22:38:07 +0100 Subject: Re: [Metakit] metakit<-->XML routine? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <1045251341.3645.51.camel@merrimack> Message-Id: <9B7896BC-4064-11D7-BF61-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Kristian G. Kvilekval wrote: > To summarize what I've found so far, I'll use the following example > doc. Great overview - good to see several options being brought forward. > > Help with XML > Joe jacker > Happy Hacker > This is one example. But how about boundary cases, say: some Joe jacker more Happy Hacker text The above may be hard to automatically structure, but even with hints: what is the "right" structure? Probably at least as important is: what are your goals with this? Do you need to collect XML and represent it more efficiently for your own purposes? Are you aiming to deal with arbitrary XML? Do you want to export data from any MK dataset? I would expect that each of the options you've summed up has benefits over the other, depending on what your requirements are. Very interesting issues, btw - IMO. -jcw From kris@cs.ucsb.edu Fri Feb 14 16:29:29 2003 Received: from letters.cs.ucsb.edu (letters.cs.ucsb.edu [128.111.41.13]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1EMTSc29772; Fri, 14 Feb 2003 16:29:28 -0600 Received: from merrimack.cs.ucsb.edu (merrimack.cs.ucsb.edu [128.111.68.227]) by letters.cs.ucsb.edu (8.11.6+Sun/8.11.6) with ESMTP id h1EMVXw27962; Fri, 14 Feb 2003 14:31:33 -0800 (PST) Subject: Re: [Metakit] metakit<-->XML routine? From: "Kristian G. Kvilekval" To: Jean-Claude Wippler Cc: Metakit mailing list In-Reply-To: <9B7896BC-4064-11D7-BF61-000A957664F8@equi4.com> References: <9B7896BC-4064-11D7-BF61-000A957664F8@equi4.com> Content-Type: text/plain Organization: Message-Id: <1045261893.3645.105.camel@merrimack> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 14 Feb 2003 14:31:33 -0800 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Fri, 2003-02-14 at 13:38, Jean-Claude Wippler wrote: > This is one example. But how about boundary cases, say: > > > some > Joe jacker > more > Happy Hacker > text > > > The above may be hard to automatically structure, but even with hints: > what is the "right" structure? My gut reaction is that "free" text should be collected and 1. Ignored 2. Concatenated into a value column book.value="somemoretext" 3. Or tagged by location either in the book or by the closest child a) "book[pcdata[value:S],title...]" book.pcdata[0] =" some" b) "book[pcdata[value:S, id:I]]" where id is some unique id and representing the preceding tag My preference would be A) 1 or 2 for most DB applications b) 3a for most others. c) 3b where input must be exactly duplicated on output. > Probably at least as important is: what are your goals with this? Do > you need to collect XML and represent it more efficiently for your own > purposes? Are you aiming to deal with arbitrary XML? Do you want to > export data from any MK dataset? Yes. Yes. and Yes. I have few goals really. The first is I see a natural fit between tree semi-structured data and metakit. Both support extensible schemas and provide a semantic structuring to the data. Metakit could really be a great XML manipulation tool. Currently, XML tool kits are pretty clunky manipulating things at the tree syntax level instead of the document semantic level. By translating an XML document into a metakit DB, I get a "better" mapping of the semantic meaning of data (I think). Plus I get all the functionality of a DB. The second, I have not really explored but I think you might be able to use views as ways of remapping XML documents easily. If you have tried to use XSLT then you know that we need something else. Reading an XML document, restructuring the view and exporting would just be too nice :) Finally the most arbitrary reason, is that I am unix guy and fundamentally distrust binary formatted databases. I have several projects that I would like to use metakit, but prefer to store them as XML so that other tools can read them directly (web stuff). The best option for me would be to open/save my metakit databases as XML documents (but only if they appear natural). -- email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris From jyl@best.com Fri Feb 14 22:01:57 2003 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h1F41vc12533 for ; Fri, 14 Feb 2003 22:01:57 -0600 Received: (qmail 93610 invoked by uid 19667); 15 Feb 2003 04:04:03 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 15 Feb 2003 04:04:03 -0000 Message-ID: <3E4DBC1F.3010704@best.com> Date: Fri, 14 Feb 2003 20:03:43 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: "Kristian G. Kvilekval" CC: Metakit mailing list Subject: Re: [Metakit] metakit<-->XML routine? References: <200302140305.h1E35tc17020@trixie.triqs.com> <1045206056.31789.9.camel@krispc> <3E4C9D2B.6060305@maich.gr> <1045251341.3645.51.camel@merrimack> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Small correction regardig e4graph's XML support: it parses *any* XML and stores it in a Metakit storage as nodes and vertices. You can also convert any e4graph node and vertex structure into XML. Every solution has its plusses and minuses. For e4graph, you have to buy into representing the XML as a tree of nodes. If you store things in Metakit, you have to store it as a view with a fixed structure, and that may not be ideal for all the lookups you want to do. Etc. Life is full of tradeoffs:) JYL Kristian G. Kvilekval wrote: >Thanks for pointers! > >To summarize what I've found so far, I'll use the following example doc. > > > Help with XML > Joe jacker > Happy Hacker > > >http://www.equi4.com/pub/etc/xml2.README > > Describes converting a single XML document to several metakit views > one for XML element. Attributes become become columns in tables. > > "book[_V:S,id:S],title[_V:S],author[_V:S] > > book[0]._V = "" book[0].id = 101 > title[0]._V ="Help with XML" > author[0]._V = "Joe jacker" > author[1]._V = "Happy Hacker" > >http://www.equi4.com/metakit/xml/ > > Straight forward translation of the XML structure into a set of > nested views. > "xml[name:S,value:S,elements[ref:I,extra:S],attribs[name:S,value:S]]" > > xml[0].name = "book" xml[0].attribs[0].name = "id" .. > xml[1].name = "title" > ... > >http://www.e4graph.com/e4graph/e4xml.html > > Parses e4graph files looking for special graph tags. Can be > used to parse general XML files, but all elements end up being > e4nodes. > > >None of these seem to fit the bill. It seems that there is a natural >mapping between many XML document types and the metakit format. > >The ideal would be the document would discovered to be: > > "book[id:I,title:S, author[name:S]]" > > Single leaf XML elements or attributes would create columns and > nesting elements would create nested views. > > > This structure could be discovered in three ways: > > 1. Providing a metakit tag at the beginning of the document > metakit view="book[id:I,title:S, author[name:S]] /> > > 2. Reading the DTD of the document if available. > > > > > > > 3. Scanning the entire document for the elements. > > >I think implementing #1 would not be too bad, then 3 and finally 2. > >Have I forgotten something important or missed the boat completely? >Any comments or suggestions are appreciated. > > From yeksoon@myrealbox.com Wed Feb 19 10:23:36 2003 Received: from smtp-send.myrealbox.com (smtp-send.myrealbox.com [192.108.102.143]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1JGNZc20962 for ; Wed, 19 Feb 2003 10:23:36 -0600 Received: from zeus.myrealbox.com yeksoon@smtp-send.myrealbox.com [218.186.64.147] by smtp-send.myrealbox.com with NetMail SMTP Agent $Revision: 3.28 $ on Novell NetWare via secured & encrypted transport (TLS); Wed, 19 Feb 2003 09:25:46 -0700 Message-Id: <5.2.0.9.2.20030220001457.0294b438@pop3.myrealbox.com> X-Sender: yeksoon@pop3.myrealbox.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Thu, 20 Feb 2003 00:19:26 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-1C4D204A; boundary="=======40C5FA7=======" Subject: [Metakit] Tk with tclkit problem? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======40C5FA7======= Content-Type: text/plain; x-avg-checked=avg-ok-1C4D204A; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit I encounter the following problem when testing Tk with tclkit under Linux (eg. ./tclkit hello.tcl) Error message as follows: == invalid command name "wm" while executing "wm title . Hello" (file "hello.tcl" line 2) ==== Here's my simple script: === wm title . Hello frame .top -borderwidth 10 pack .top -side top -fill x button .top.hello -text Hello -command {puts stdout "Hello, World!"} button .top.quit -text Quit -command exit pack .top.quit .top.hello -padx 20 -pady 10 == What is the likely cause of it? Thanks YekSoon --=======40C5FA7======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-1C4D204A Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.455 / Virus Database: 255 - Release Date: 13-Feb-03 --=======40C5FA7=======-- From jcw@equi4.com Wed Feb 19 10:30:31 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1JGUUc21509; Wed, 19 Feb 2003 10:30:30 -0600 Date: Wed, 19 Feb 2003 17:32:39 +0100 Subject: Re: [Metakit] Tk with tclkit problem? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) Cc: Metakit mailing list To: Lok Yek Soon From: Jean-Claude Wippler In-Reply-To: <5.2.0.9.2.20030220001457.0294b438@pop3.myrealbox.com> Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Lok Yek Soon wrote: > I encounter the following problem when testing Tk with tclkit under > Linux (eg. ./tclkit hello.tcl) > > Error message as follows: > == > invalid command name "wm" > while executing > "wm title . Hello" > (file "hello.tcl" line 2) > ==== Add the following line before calling "wm": package require Tk Please post tclkit-related Q's to the starkit mailing list instead: http://www.equi4.com/mailman/listinfo/starkit -jcw From 1jerry@usd1.com Wed Feb 19 10:41:50 2003 Received: from bfish.hurrah.com (root@bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1JGfoc22415 for ; Wed, 19 Feb 2003 10:41:50 -0600 Received: from CTL (12-211-33-9.client.attbi.com [12.211.33.9]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id h1JGi1n11772 for ; Wed, 19 Feb 2003 08:44:01 -0800 Date: Thu, 13 Feb 2003 10:25:59 -0800 From: Jerry McRae <1jerry@usd1.com> X-Mailer: The Bat! (v1.60q) Personal Organization: usd1 X-Priority: 3 (Normal) Message-ID: <9315730184.20030213102559@usd1.com> To: Metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] finding rows in Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Am I dense or is there a way to find the original row from a 'viewer' instance? >>> vw=db.getas('test[name,pay[key,val:I]') >>> r=vw[1] >>> vw1=r.pay.select(key='b',val=1) # now I have a subset Now I have the subset I wish to work with, but I need to modify or delete the *original* rows. In this instance, I have no urid (unique row id). I have the same problem with vw.sortrev(), which I have to do a vw.find(urid=...) for every record I want to process. (Didn't sortrev used to pass changes through?) Is there any way to identify or access the original row without adding a dummy urid to each one and searching? -- Thanks, Jerry McRae MetaKit 2.4.8, Python 2.2, wxPython 2.2.5, Win98, 384M RAM From gmcm@hypernet.com Wed Feb 19 15:32:49 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1JLWnc11720 for ; Wed, 19 Feb 2003 15:32:49 -0600 Received: from PARANOIA (204.176.40.94) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Wed, 19 Feb 2003 16:36:52 -0500 From: "Gordon McMillan" To: Jerry McRae <1jerry@usd1.com>, Metakit Date: Wed, 19 Feb 2003 16:33:32 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] finding rows in Python Reply-to: gmcm@hypernet.com Message-ID: <3E53B1DC.12687.202E533B@localhost> Priority: normal In-reply-to: <9315730184.20030213102559@usd1.com> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 13 Feb 2003 at 10:25, Jerry McRae wrote: > Am I dense or is there a way to find the original row from a 'viewer' > instance? > > >>> vw=db.getas('test[name,pay[key,val:I]') > >>> r=vw[1] > >>> vw1=r.pay.select(key='b',val=1) # now I have a subset > > Now I have the subset I wish to work with, but I need to modify or > delete the *original* rows. In this instance, I have no urid (unique > row id). Check out view.remove, view.map, view.indices, view.filter. All the capabilities should be there, but you may have to order your code differently (eg, do the work in a callback). > I have the same problem with vw.sortrev(), which I have to do > a vw.find(urid=...) for every record I want to process. (Didn't sortrev > used to pass changes through?) Is there any way to identify or access the > original row without adding a dummy urid to each one and searching? Select & the sorts should allow mutating the original through the derived view - at least the Mk4py code is doing a _base->GetIndexOf(). Hmm. Didn't realize any of this had broken, but I use map & friends when changing mulitple rows. Unique IDs should only be needed when you need to remember a row across sessions / interactions. -- Gordon http://www.mcmillan-inc.com/ From jcw@equi4.com Thu Feb 20 04:56:24 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1KAuNc22036; Thu, 20 Feb 2003 04:56:23 -0600 Date: Thu, 20 Feb 2003 11:58:31 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list , announce@equi4.com Content-Transfer-Encoding: 7bit Message-Id: <400C5E26-44C2-11D7-A9C3-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Subject: [Metakit] Metakit 2.4.9 released Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There is a new release of Metakit. It's mostly a bug fix release, plus some smaller changes to extend the Tcl binding a bit more on the OO side. Extract from changelog at http://www.equi4.com/metakit/CHANGES: 2003-02-19 ### MK 2.4.9 2003-02-18 Fix bug in blocked view delete and hash byteorder 2003-02-17 Configure tweaks for hpux/ia64 2003-02-14 Bug found in blocked viewer modification 2003-02-14 Some changes to OO interface in Tcl 2003-02-14 Enable stdio buffering 2003-02-07 Tweaks to restore broken MK ports 2003-02-07 Changed code to avoid compiler warning 2003-02-02 Work around optimizer bug in gcc 3.2.1 2003-01-24 Fixed cleanup order bug in Mk4tcl 2003-01-22 Add missing -lstdc++ 2003-01-19 Tweak to temp object use 2003-01-17 Add synonym for mk4tcl "info" command 2003-01-16 Allow access to root view in Mk4tcl 2003-01-15 Use strdup 2003-01-10 Build improvements, Mk4py long and Mac improvements 2003-01-09 String compare tweak, Mac Carbon runtime mmap code 2002-12-23 Tweak for Borland builder 5 & 6 2002-12-09 Fixed bug in selection view change propagation 2002-12-02 Fixed bug in MK old-file format conversion 2002-11-24 Fixed Mk4tcl threaded build 2002-11-22 Configure tweak for HPUX/Itanium 2002-11-16 Tweaks to compile on Mac 2002-11-04 Fixed typo in Makefile 2002-11-03 ### MK 2.4.8 The homepage is, as before, http://www.equi4.com/metakit - FYI, Andreas Kupries has documented the 2.3/2.4 file format, see doc section. FYI also, there is now a bug tracking system at http://www.equi4.com/bugs The 2.4.9 release passes all its 140+ tests on Windows, Linux, and MacOS X - but there probably remain some portability issues in the makefiles and headers. Metakit is Open Source Software, and will always remain so. If you would like to support this, please share bug details, tricks/insights, and porting tips - it's a very effective way to help take it yet further. Happy programming :) -jcw From dgporter@erols.com Thu Feb 20 07:21:56 2003 Received: from smtp02.mrf.mail.rcn.net (smtp02.mrf.mail.rcn.net [207.172.4.61]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1KDLtc32057; Thu, 20 Feb 2003 07:21:55 -0600 Received: from 66-44-5-217.s1487.apx1.lnh.md.dialup.rcn.com ([66.44.5.217] helo=there) by smtp02.mrf.mail.rcn.net with smtp (Exim 3.35 #4) id 18lqgH-0001xl-00; Thu, 20 Feb 2003 08:24:05 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Don Porter Reply-To: dgporter@erols.com To: Jean-Claude Wippler , Metakit mailing list , announce@equi4.com Date: Thu, 20 Feb 2003 08:16:22 -0500 X-Mailer: KMail [version 1.3.2] References: <400C5E26-44C2-11D7-A9C3-000A957664F8@equi4.com> In-Reply-To: <400C5E26-44C2-11D7-A9C3-000A957664F8@equi4.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: Subject: [Metakit] Re: [Announce] Metakit 2.4.9 released Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Thursday 20 February 2003 05:58, Jean-Claude Wippler wrote: > There is a new release of Metakit. It's mostly a bug fix release, > plus some smaller changes to extend the Tcl binding a bit more on the > OO side. Could someone familiar with the extensions to the Tcl binding please update http://www.equi4.com/metakit/wiki.cgi/147 ? It's not great, but better than no documentation at all. -- | Don Porter dgporter@erols.com | | "Some days you just can't get rid of a bomb!" | | -- Adam West as BATMAN | |______________________________________________________________________| From Michael_Scharf@gmx.de Sun Feb 23 21:02:34 2003 Received: from mailout01.sul.t-online.com (mailout01.sul.t-online.com [194.25.134.80]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1O32Yc07438 for ; Sun, 23 Feb 2003 21:02:34 -0600 Received: from fwd08.sul.t-online.de by mailout01.sul.t-online.com with smtp id 18n8v8-0007ZC-00; Mon, 24 Feb 2003 04:04:46 +0100 Received: from gmx.de (320007127165-0001@[217.88.238.67]) by fmrl08.sul.t-online.com with esmtp id 18n8uv-1suoFMC; Mon, 24 Feb 2003 04:04:33 +0100 Message-ID: <3E598BC2.3090102@gmx.de> Date: Mon, 24 Feb 2003 04:04:34 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Subject: [Metakit] Bug in metakit2.4.9 python bindings (+fix) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This code works well with 2.4.8 with and with 2.4.9 (using the Mk4py.dll from the 2.4.8 distribution) import metakit s=metakit.storage() v=s.getas('x[b:B]') v.append(b='b') With the 2.4.9 Mk4py.dll I get: Traceback (most recent call last): File "c:/cygwin/tmp/x.py", line 4, in ? v.append(b='b') Looking at the sourcecode reveals a missing break statement in PyRowRef::setFromPython (I have not compiled it, but the missing break is an obvouis problem): diff -c PyRowRef.cpp.orig PyRowRef.cpp *** PyRowRef.cpp.orig Mon Feb 24 04:02:06 2003 --- PyRowRef.cpp Mon Feb 24 03:57:12 2003 *************** *** 202,207 **** --- 202,208 ---- } else if (attr != Py_None) Fail(PyExc_TypeError, "wrong type for ByteProp"); + break; default: PyErr_Format(PyExc_TypeError, "unknown property type '%c'", prop.Type()); throw PWDPyException; Michael From jcw@equi4.com Mon Feb 24 03:25:59 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1O9Pxc23618 for ; Mon, 24 Feb 2003 03:25:59 -0600 Date: Mon, 24 Feb 2003 10:28:10 +0100 Subject: Re: [Metakit] Bug in metakit2.4.9 python bindings (+fix) Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3E598BC2.3090102@gmx.de> Message-Id: <4A88F10A-47DA-11D7-8423-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Michael Scharf wrote: > With the 2.4.9 Mk4py.dll I get: > Traceback (most recent call last): > File "c:/cygwin/tmp/x.py", line 4, in ? > v.append(b='b') > > Looking at the sourcecode reveals a missing break > statement in PyRowRef::setFromPython (I have not > compiled it, but the missing break is an obvouis > problem): > > diff -c PyRowRef.cpp.orig PyRowRef.cpp > *** PyRowRef.cpp.orig Mon Feb 24 04:02:06 2003 > --- PyRowRef.cpp Mon Feb 24 03:57:12 2003 > *************** > *** 202,207 **** > --- 202,208 ---- > } > else if (attr != Py_None) > Fail(PyExc_TypeError, "wrong type for ByteProp"); > + break; Thank you - for pinpointing and solving the issue fully! I've checked in your change and have rebuilt the Windows and Linux binaries. It goes to show that Mk4py really needs a test suite. Nicholas Riley has already made a start with this - I'll need to look into adopting it. -jcw From Andreas.theDragon@gmx.net Tue Feb 25 06:50:35 2003 Received: from mx0.gmx.net (mx0.gmx.net [213.165.64.100]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h1PCoZc14821 for ; Tue, 25 Feb 2003 06:50:35 -0600 Received: (qmail 12315 invoked by uid 0); 25 Feb 2003 12:52:45 -0000 Date: Tue, 25 Feb 2003 13:52:45 +0100 (MET) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: metakit@equi4.com MIME-Version: 1.0 X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [217.194.32.49] Message-ID: <28312.1046177565@www13.gmx.net> X-Mailer: WWW-Mail 1.6 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] Data loss? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I've been using Metakit for several years now and had no problems. Unfortunately 2 users reported corrupted databases so all their data was lost (I use Metakit mainly as a storage base, we talk about ca. 30-40'000 rows with 1-2kByte each). I am not sure what can cause a corruption, especially because I cannot reproduce it so my question is: Which operations are most critical? A first guess was that Windows was shut down while the program was still running and the memory mapped file couldn't be written back completely. Or I do some kind of reorganisation to reduce the filesize if the user has deleted records: FILE *myNewFile = fopen (strTemp, "wb"); c4_FileStream myNewStream (myNewFile, true); m_Storage->SaveTo( myNewStream ); fclose (myNewFile); // 2. release metakit storage and views CloseMetakit(); // 3. Delete original file and move new one DeleteFile(strMeta); MoveFile( strTemp, strMeta ); // 4. reload Metakit InitMetakit(); Could that cause errors? Regards, Andreas From jcw@equi4.com Tue Feb 25 07:22:47 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1PDMkc17408 for ; Tue, 25 Feb 2003 07:22:46 -0600 Date: Tue, 25 Feb 2003 14:24:59 +0100 Subject: Re: [Metakit] Data loss? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list In-Reply-To: <28312.1046177565@www13.gmx.net> Message-Id: <89FCF989-48C4-11D7-8423-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h1PDMkc17408 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas Mügge wrote: > I've been using Metakit for several years now and had no problems. > Unfortunately 2 users reported corrupted databases so all their data > was lost (I use Metakit mainly as a storage base, we talk about ca. > 30-40'000 rows with 1-2kByte each). > > I am not sure what can cause a corruption, especially because I > cannot reproduce it so my question is: > > Which operations are most critical? A first guess was that Windows > was shut down while the program was still running and the memory > mapped file couldn't be written back completely. No. This cannot happen - short of a disk head causing damage on power down, but even that is terribly unlikely these days AFAIK, as heads are made to retract automatically. Failures to write have no effect on integrity, transactions are defined by the very last write in each commit. > Or I do some kind of reorganisation to reduce the filesize if the user > has deleted records: > > FILE *myNewFile = fopen (strTemp, "wb"); > c4_FileStream myNewStream (myNewFile, true); > m_Storage->SaveTo( myNewStream ); > fclose (myNewFile); > > // 2. release metakit storage and views > CloseMetakit(); > > // 3. Delete original file and move new one > DeleteFile(strMeta); > MoveFile( strTemp, strMeta ); > > // 4. reload Metakit > InitMetakit(); > > Could that cause errors? No, this is fine. The only explanations I have are: - multiple simultaneous opens in r/w mode - failing to test for commit success - out of order disk block flushing to disk The former is always trouble. The second issue is important to catch disk-full situations. Make sure you check every commit (and don't use auto-commit, because then you can't verify the closing commit). MK will always return a failure from the commit call if any write fails. The last issue is tricky. In Unix, there are "fsync" system calls, to make sure file changes are actually completely written before the call returns. But MK's default file I/O is 100% posix, and neither fsync nor Win32 equivalents are part of standard Metakit. If you feel that relying on stdio's "fflush" is not enough, then you will need to define an overriding strategy class which does things yet more strictly. The "c4_FileStrategy::DataCommit" member gets called twice on each commit (first has zero arg, second non-zero). It is essential for data integrity that in both calls, it only returns after data has been really written. With that rule, you can pull the plug at any microsecond in time, and things will be safe. One thing to be very careful about is file servers. Best rule: don't put a database on a file server. There are way too many trouble spots to feel comfortable with (locking, caches). Oh, btw, you don't mention Win32 or MK versions. There have been bugs. Make sure you check with the changelog to see if anything could have bitten you. Having said all that, the short answer remains as before: MK has been designed to be 100% failsafe, short of stray memory overwrites from applications and hardware damage. I'm very interested to hear what the explanation ends up being, and really hope you can narrow it down. If it's a bug in MK, it is of the utmost importance to find it. In my book, a less than 100% reliable database is not a database but a time bomb. -jcw From jcw@equi4.com Tue Feb 25 18:18:44 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1Q0Ihc11673; Tue, 25 Feb 2003 18:18:43 -0600 Date: Wed, 26 Feb 2003 01:20:55 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list , Starkit list Content-Transfer-Encoding: 7bit Message-Id: <2BE0B7B2-4920-11D7-8423-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Subject: [Metakit] OOMK 0.1 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: (This is one of those posts which fits into neither mailing list) For adventurous Tcl coders: OOMK is a new pure-Tcl wrapper around Mk4tcl, which brings the OO model which C++ and Python have so long enjoyed to Tcl. Sometimes code shows it all more easily - a complete example: package require oomk mkstorage db db define v1 {s1 s2} {11 aa 22 bb 33 cc} db define v2 {s1 s3} {11 eleven 22 twenty-two 44 forty-four 22 hi!} [$v1 join $v2 s1] as v3 $v3 dump \t Output: s1 s3 s2 -- -- ---------- 11 aa eleven 22 bb hi! 22 bb twenty-two -- -- ---------- This is release 0.1 (IOW, it's experimental). I will help fix things where possible, but am not committing to this as being *the* future API for MK + Tcl. The OO model is based on Will Duquette's neat "Snit" package (version 0.8). It requires the very latest release of Metakit (version 2.4.9). More details can be found here: http://www.equi4.com/pub/sk/oomk.kit/lib/app-oomk/oomk.txt The actual code itself is in a single script: http://www.equi4.com/pub/sk/oomk.kit/lib/oomk/oomk.tcl Or download it all as a Starkit (it'll be added to sdarchive): http://www.equi4.com/pub/sk/oomk.kit -jcw From Michael_Scharf@gmx.de Thu Feb 27 18:27:05 2003 Received: from mailout04.sul.t-online.com (mailout04.sul.t-online.com [194.25.134.18]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1S0R5c31174 for ; Thu, 27 Feb 2003 18:27:05 -0600 Received: from fwd08.sul.t-online.de by mailout04.sul.t-online.com with smtp id 18oYOt-0004pb-00; Fri, 28 Feb 2003 01:29:19 +0100 Received: from gmx.de (320007127165-0001@[80.132.63.124]) by fmrl08.sul.t-online.com with esmtp id 18oYOg-0qn78yC; Fri, 28 Feb 2003 01:29:06 +0100 Message-ID: <3E5EAD79.9040206@gmx.de> Date: Fri, 28 Feb 2003 01:29:45 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Subject: [Metakit] Some questions (FAQ?) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm still experimenting with metakit and I like it more and more :-). I'm using Mk4py but I also experimented with the C++ interface. The only missing thing seems documentation. I read a good fraction of what is on the web plus some mails. But still there are some questions I could not find answers: Q: How to delete an entire view (the opposite of: storage.getas(...))? Q: How to delete an entire column (how to transform "view[c1:S,c2:S]" to "view[c1:S]" persistently and freeing column c2)? Q: How to shrink a database? (I read making a copy might do the job. But is there a generic way to make a copy of an arbitrary data structure)? Q: How do I know what views are persistent and which views are transient (blocked, hash, ordered seem persistent, sort, select, union etc. seem transient...)? Q: Can I make derived views (like union, or sort) persistent? Q: It is said that metakit uses memory mapped files. When I create a new database, the file size remains 0 (on window) until I commit. Is this a problem of windows, or is the view in memory until (the first) commit? Michael From Michael_Scharf@gmx.de Thu Feb 27 20:43:03 2003 Received: from mailout05.sul.t-online.com (mailout05.sul.t-online.com [194.25.134.82]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1S2h2c09673 for ; Thu, 27 Feb 2003 20:43:03 -0600 Received: from fwd08.sul.t-online.de by mailout05.sul.t-online.com with smtp id 18oaWU-0003fF-00; Fri, 28 Feb 2003 03:45:18 +0100 Received: from gmx.de (320007127165-0001@[80.132.63.124]) by fmrl08.sul.t-online.com with esmtp id 18oaWP-18jh1UC; Fri, 28 Feb 2003 03:45:13 +0100 Message-ID: <3E5ECD61.1010306@gmx.de> Date: Fri, 28 Feb 2003 03:45:53 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Subject: [Metakit] caution: duplicate property names in one view can causes strange commit behaviour (Mk4py) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I have been looking for hours for a very strange bug. It seemed that commit sometimes worked and sometimes not (actually the commit committed the changes that the previous commit should have committed).... Anyway, I tried to distill the problem and it turned out, that in one of the (nested (blocked)) views I defined twice the same property. This property was committed 'delayed' (each time I opened the database, the column was one version behind, as if the 2 versions of the column were swapped on each commit). Unfortunately, I could not create a small sample reproducing this behavior. Therefore the warning: duplicated properties view[...prop:I,...,prop:I...] can cause problems (at least with the python bindings)! Michael From gmcm@hypernet.com Fri Feb 28 07:08:10 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1SD89c17985 for ; Fri, 28 Feb 2003 07:08:09 -0600 Received: from PARANOIA (204.176.40.49) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Fri, 28 Feb 2003 08:12:31 -0500 From: "Gordon McMillan" To: Michael Scharf , Metakit List Date: Fri, 28 Feb 2003 08:09:15 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Some questions (FAQ?) Reply-to: gmcm@hypernet.com Message-ID: <3E5F192B.2180.4CB9E210@localhost> Priority: normal In-reply-to: <3E5EAD79.9040206@gmx.de> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 28 Feb 2003 at 1:29, Michael Scharf wrote: > I'm still experimenting with metakit and I like it more and > more :-). I'm using Mk4py but I also experimented with the > C++ interface. > > The only missing thing seems documentation. I read a good > fraction of what is on the web plus some mails. But still > there are some questions I could not find answers: > > Q: How to delete an entire view (the opposite of: storage.getas(...))? Not sure if there's an easier way. You can create a 2nd storage with the views you want and use slice assignment to copy the data you want: v1[:] = v2. > Q: How to delete an entire column (how to transform > "view[c1:S,c2:S]" to "view[c1:S]" persistently and > freeing column c2)? storage.getas(). > Q: How to shrink a database? (I read making a copy might do the > job. But is there a generic way to make a copy of an > arbitrary data structure)? storage.save(). A non-read-only database will generally be 2x the size of the data, so shrinking it is not likely to be very useful - it will just double on the next modification. (There are situations where it's a useful trick, but they're pretty esoteric.) > Q: How do I know what views are persistent and which > views are transient (blocked, hash, ordered seem > persistent, sort, select, union etc. seem transient...)? What's persistent is what storage.description() tells you. There are no view-returning methods that do in-place mutations. Even with the mapping views (where MK will do the maintenance for you), you need to inform MK that you expect this on each open. (In other words, if I have an ordered view, and I open it and don't use v = v.ordered(1) and then do an append, it will likely destroy the ordering.) Basically, there's no metadata except what storage.description() tells you. > Q: Can I make derived views (like union, or sort) persistent? v[:] = v.sort() is perfectly valid. If you're sorting on leftmost properties, you can subsequently use v.ordered(). You could similarly save a union, but you'd have to define a persistent view with the right properties. Note that derived views are generally iterators - they don't do much until you access them. So v[:] = v.sort() is very expensive since it forces an in-memory copy of v. > Q: It is said that metakit uses memory mapped files. When I > create a new database, the file size remains 0 (on window) > until I commit. Is this a problem of windows, or is the view > in memory until (the first) commit? The latter. -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Fri Feb 28 07:08:10 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1SD8Ac17990 for ; Fri, 28 Feb 2003 07:08:10 -0600 Received: from PARANOIA (204.176.40.49) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Fri, 28 Feb 2003 08:12:33 -0500 From: "Gordon McMillan" To: Michael Scharf , Metakit List Date: Fri, 28 Feb 2003 08:09:15 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] caution: duplicate property names in one view can causes strange commit behaviour (Mk4py) Reply-to: gmcm@hypernet.com Message-ID: <3E5F192B.31386.4CB9E256@localhost> Priority: normal In-reply-to: <3E5ECD61.1010306@gmx.de> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 28 Feb 2003 at 3:45, Michael Scharf wrote: > I have been looking for hours for a very strange bug. > It seemed that commit sometimes worked and sometimes > not (actually the commit committed the changes that > the previous commit should have committed).... > > Anyway, I tried to distill the problem and it turned out, > that in one of the (nested (blocked)) views I defined > twice the same property. This property was committed > 'delayed' (each time I opened the database, the column > was one version behind, as if the 2 versions of the column > were swapped on each commit). The pair (name, type) uniquely identifies a property. Defining a view with multiple instances of the same property *should* be impossible. Alas, it's not. In the Python bindings, it's basically assumed that (view, propertyname) will be unique. You could work around that by always using explicit property objects instead of view.propertyname. > Unfortunately, I could not create a small sample > reproducing this behavior. > > Therefore the warning: duplicated properties > view[...prop:I,...,prop:I...] > can cause problems (at least with the python > bindings)! > > > Michael > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- Gordon http://www.mcmillan-inc.com/ From jeff@k2.com Fri Feb 28 07:38:38 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1SDccc21216 for ; Fri, 28 Feb 2003 07:38:38 -0600 Received: from Klee ([192.168.0.21]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h1SDftd02693; Fri, 28 Feb 2003 08:41:55 -0500 From: "Jeffrey Kay" To: , "'Michael Scharf'" , "'Metakit List'" Subject: RE: [Metakit] Some questions (FAQ?) Date: Fri, 28 Feb 2003 08:40:26 -0500 Message-ID: <002701c2df2e$f39e4240$017410ac@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal In-Reply-To: <3E5F192B.2180.4CB9E210@localhost> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm a C++ interface user, so all my answers are in C++. > > Q: How to delete an entire view (the opposite of: > storage.getas(...))? > > Not sure if there's an easier way. You can create a 2nd > storage with the views you want and use slice assignment > to copy the data you want: v1[:] = v2. The easiest way is to do a GetAs on the view without specifying any properties, e.g. c4_Storage storage("mystore.db",true); storage.GetAs("view"); storage.Commit(); > > Q: How to delete an entire column (how to transform > > "view[c1:S,c2:S]" to "view[c1:S]" persistently and > > freeing column c2)? Same thing really ... Just get the new view without the extra property and commit it to storage. That should free the column. c4_Storage storage("mystore.db",true); storage.GetAs("view[c1:S]"); storage.Commit(); Property c2 will then be gone. > > Q: How to shrink a database? (I read making a copy might do the > > job. But is there a generic way to make a copy of an > > arbitrary data structure)? > > storage.save(). A non-read-only database will generally be 2x the > size of the data, so shrinking it is not likely to be very useful - it > will just double on the next modification. (There are > situations where > it's a useful trick, but they're pretty esoteric.) The size of the db will depend heavily on how much you put in and delete, so shrinking in many applications is necessary. Here's the code to do it -- FILE *tf; if ((tf = fopen("mystore.db","rb")) != NULL) { fclose(tf); CFileStream fs1(fopen("~mystore.db", "wb"), true); c4_Storage st("mystore.db", 0); st.SaveTo(fs1); // now ~mystore.db has the compressed copy of mystore.db. CopyFile("~mystore.db","mystore.db",true); DeleteFile("~mystore.db"); } Apologies for the Win32 calls for copying and deleting, but I think you'll get the idea. > > > Q: How do I know what views are persistent and which > > views are transient (blocked, hash, ordered seem > > persistent, sort, select, union etc. seem transient...)? > > What's persistent is what storage.description() tells you. > There are no view-returning methods that do in-place mutations. > Even with the mapping views (where MK will do the > maintenance for you), you need to inform MK that you > expect this on each open. (In other words, if I have an > ordered view, and I open it and don't use > v = v.ordered(1) > and then do an append, it will likely destroy the ordering.) > > Basically, there's no metadata except what > storage.description() tells you. In my experience, I assume that any derived view (beyond the one I get from GetAs()) is transient and will not allow me to do updates. I always use GetIndexOf() to get the index of a row in the derived view as a row in the base view (the one from GetAs()) and do my manipulations in the base view. > > Q: Can I make derived views (like union, or sort) persistent? > > v[:] = v.sort() is perfectly valid. If you're sorting on leftmost > properties, you can subsequently use v.ordered(). > > You could similarly save a union, but you'd have to > define a persistent view with the right properties. > > Note that derived views are generally iterators - they > don't do much until you access them. So > v[:] = v.sort() > is very expensive since it forces an in-memory > copy of v. Sorry, haven't tried this. > > > Q: It is said that metakit uses memory mapped files. When I > > create a new database, the file size remains 0 (on window) > > until I commit. Is this a problem of windows, or is the view > > in memory until (the first) commit? > > The latter. Good luck. I've been using MetaKit extensively in an app that I've developed and I'm pretty pleased with everything except concurrent access by multiple threads. To get around the concurrency problems, I open the database and then provide getStore() and releaseStore() functions, which essentially enter and leave a critical section (critsec) to prevent concurrent access. In my application, database access isn't a time critical thing, so this is really no trouble and ensures that I don't screw up the database. jeffrey kay weblog pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein From Michael_Scharf@gmx.de Fri Feb 28 08:05:59 2003 Received: from mailout09.sul.t-online.com (mailout09.sul.t-online.com [194.25.134.84]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1SE5xc23535 for ; Fri, 28 Feb 2003 08:05:59 -0600 Received: from fwd01.sul.t-online.de by mailout09.sul.t-online.com with smtp id 18olBN-0005zI-08; Fri, 28 Feb 2003 15:08:13 +0100 Received: from gmx.de (320007127165-0001@[217.88.235.39]) by fmrl01.sul.t-online.com with esmtp id 18olBI-0FkBcWC; Fri, 28 Feb 2003 15:08:08 +0100 Message-ID: <3E5F6D55.5070408@gmx.de> Date: Fri, 28 Feb 2003 15:08:21 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 CC: Metakit List Subject: Re: [Metakit] caution: duplicate property names in one view can causes strange commit behaviour (Mk4py) References: <3E5F192B.31386.4CB9E256@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: > The pair (name, type) uniquely identifies a property. > Defining a view with multiple instances of the same > property *should* be impossible. Alas, it's not. Well, it usually makes no sense. I did that accidentally. So, I don't request that feature ;-). In most other data description langueges, you get an error if you use the same property name twice in a structure..... Michael From jcw@equi4.com Fri Feb 28 08:20:35 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1SEKYc24654 for ; Fri, 28 Feb 2003 08:20:34 -0600 Date: Fri, 28 Feb 2003 15:22:46 +0100 Subject: Re: [Metakit] caution: duplicate property names in one view can causes strange commit behaviour (Mk4py) Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit List Content-Transfer-Encoding: 7bit In-Reply-To: <3E5F6D55.5070408@gmx.de> Message-Id: <1BE4C0BE-4B28-11D7-8423-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Michael Scharf wrote: > Well, it usually makes no sense. I did that accidentally. > So, I don't request that feature ;-). > In most other data description langueges, you get an error > if you use the same property name twice in a structure..... This stems from an old design choice whereby one could leave off names: a[a:I,b:I] could be written as a[:I,:I] - not allowing access by name of course. The code that accepted it had to deal with duplicates (empty names). And now it has become a historical choice that comes back to bite us... FWIW, there will at one point again be a way to specify structure without any names at all. The above will then become "[II]", note the absence of even the view name and some of the punctuation. -jcw From Michael_Scharf@gmx.de Fri Feb 28 09:24:25 2003 Received: from mailout08.sul.t-online.com (mailout08.sul.t-online.com [194.25.134.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1SFOPc29675 for ; Fri, 28 Feb 2003 09:24:25 -0600 Received: from fwd01.sul.t-online.de by mailout08.sul.t-online.com with smtp id 18omPI-0002fn-0F; Fri, 28 Feb 2003 16:26:40 +0100 Received: from gmx.de (320007127165-0001@[217.88.235.39]) by fmrl01.sul.t-online.com with esmtp id 18omP4-0tM3PcC; Fri, 28 Feb 2003 16:26:26 +0100 Message-ID: <3E5F7FB1.50108@gmx.de> Date: Fri, 28 Feb 2003 16:26:41 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Subject: [Metakit] Mk4py adding derrived views does not work.... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, The following code causes an exception: import metakit,os s=metakit.storage() v1=s.getas('v1[x:I]') v2=s.getas('v2[x:I]') v3=s.getas('v2[x:I]') v=v1+v2 v=v1+v2+v3 That's the exception: Traceback (most recent call last): File "z4.py", line 7, in ? v=v1+v2+v3 TypeError: unsupported operand types for +: 'PyViewer' and 'PyView' The + operator on view calls c4_View::Concat. I don't see any reason, why the new derived view cannot be used as argument for another concat. It does NOT modify any of the argument views, instead it creates a new view. Here's how I fixed it: I added sq_concat (and sq_repeat) to ViewerAsSeq. It might be a performance bottleneck to concat too many views, because they get stacked, but that's another problem ;-) *** metakit-2.4.9.orig/python/PyView.cpp Mon Mar 25 21:41:37 2002 --- metakit-2.4.9/python/PyView.cpp Fri Feb 28 16:22:57 2003 *************** *** 923,930 **** }; static PySequenceMethods ViewerAsSeq = { (inquiry)PyView_length, //sq_length ! (binaryfunc)0, //sq_concat ! (intargfunc)0, //sq_repeat (intargfunc)PyView_getitem, //sq_item (intintargfunc)PyView_getslice, //sq_slice (intobjargproc)0, //sq_ass_item --- 923,930 ---- }; static PySequenceMethods ViewerAsSeq = { (inquiry)PyView_length, //sq_length ! (binaryfunc)PyView_concat, //sq_concat ! (intargfunc)PyView_repeat, //sq_repeat (intargfunc)PyView_getitem, //sq_item (intintargfunc)PyView_getslice, //sq_slice (intobjargproc)0, //sq_ass_item From Michael_Scharf@gmx.de Fri Feb 28 09:43:36 2003 Received: from mailout02.sul.t-online.com (mailout02.sul.t-online.com [194.25.134.17]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1SFhac31184 for ; Fri, 28 Feb 2003 09:43:36 -0600 Received: from fwd01.sul.t-online.de by mailout02.sul.t-online.com with smtp id 18omhs-0006RU-06; Fri, 28 Feb 2003 16:45:52 +0100 Received: from gmx.de (320007127165-0001@[217.88.235.39]) by fmrl01.sul.t-online.com with esmtp id 18omhc-1WTwsCC; Fri, 28 Feb 2003 16:45:36 +0100 Message-ID: <3E5F842F.4030203@gmx.de> Date: Fri, 28 Feb 2003 16:45:51 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Subject: [Metakit] My4py many memory leaks Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I discovered quite somem memory leaks in the My4py implementation. Here are two examples: import metakit def test(n): s=metakit.storage() v=s.getas('view[x:I]') for i in xrange(n): v.append(0) # the returned int leaks # numbers < 1000 arse shared # to see the leak, we need to add > 1000 if i%10000==0: v[:]=[] test(10000000) Thhe append leaks. It returns the position of the new item. There seems to be a refcount problem with the returned int. Here's a fix: *** metakit-2.4.9.orig/python/PyView.cpp Mon Mar 25 21:41:37 2002 --- metakit-2.4.9/python/PyView.cpp Fri Feb 28 16:28:11 2003 *************** *** 63,75 **** static PyObject* PyView_append(PyView *o, PyObject* _args, PyObject* kwargs) { try { ! PWOSequence args(_args); ! PWONumber ndx(o->GetSize()); ! if (args.len() == 0) o->insertAt(ndx, kwargs); ! else ! o->insertAt(ndx, args[0]); ! return ndx.disOwn(); } catch (...) { return 0; } } --- 63,75 ---- static PyObject* PyView_append(PyView *o, PyObject* _args, PyObject* kwargs) { try { ! int ndx=o->GetSize(); ! if (PySequence_Length(_args) == 0) o->insertAt(ndx, kwargs); ! else { ! o->insertAt(ndx, PySequence_Fast_GET_ITEM(_args,0)); ! } ! return PyInt_FromLong(ndx); } catch (...) { return 0; } } Another example is view.description: import metakit def test(n): s=metakit.storage() v=s.getas('view[x:I]') for i in xrange(n): v.structure() test(10000000) The fix is: *** metakit-2.4.9.orig/python/PyView.cpp Mon Mar 25 21:41:37 2002 --- metakit-2.4.9/python/PyView.cpp Fri Feb 28 16:43:23 2003 *************** *** 37,42 **** --- 37,43 ---- static PyObject* PyView_structure(PyView *o, PyObject* _args) { try { + if(!PyArg_NoArgs(_args)) return NULL; return o->structure(); } catch (...) { return 0; } *************** *** 1117,1127 **** PyObject* PyView::structure() { int n = NumProperties(); ! PWOList rslt(n); for (int i = 0; i < n; i++) { ! rslt[i] = new PyProperty(NthProperty(i)); } ! return rslt.disOwn(); } PyView *PyView::getSlice(int s, int e) { --- 1118,1128 ---- PyObject* PyView::structure() { int n = NumProperties(); ! PyObject* list=PyList_New(n); for (int i = 0; i < n; i++) { ! PyList_SET_ITEM(list, i, new PyProperty(NthProperty(i))); } ! return list; } PyView *PyView::getSlice(int s, int e) { In general, I'm not really convinced that the PW objects are a good choice. I's prefer to see classical PyArg_Parse calls. Michael From Michael_Scharf@gmx.de Fri Feb 28 09:55:25 2003 Received: from mailout09.sul.t-online.com (mailout09.sul.t-online.com [194.25.134.84]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h1SFtPc32144 for ; Fri, 28 Feb 2003 09:55:25 -0600 Received: from fwd01.sul.t-online.de by mailout09.sul.t-online.com with smtp id 18omt5-0005tX-0O; Fri, 28 Feb 2003 16:57:27 +0100 Received: from gmx.de (320007127165-0001@[217.88.235.39]) by fmrl01.sul.t-online.com with esmtp id 18omt2-08zwESC; Fri, 28 Feb 2003 16:57:24 +0100 Message-ID: <3E5F86F2.5000601@gmx.de> Date: Fri, 28 Feb 2003 16:57:38 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Michael Scharf CC: Metakit List Subject: Re: [Metakit] Mk4py adding derrived views does not work.... References: <3E5F7FB1.50108@gmx.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: OOPS, incomplete patch! (I forgot to include the new typecheck in PyView_concat!) *** metakit-2.4.9.orig/python/PyView.cpp Mon Mar 25 21:41:37 2002 --- metakit-2.4.9/python/PyView.cpp Fri Feb 28 16:56:11 2003 *************** *** 849,855 **** static PyObject* PyView_concat(PyView *o, PyView *other) { try { ! if (other->ob_type != &PyViewtype) Fail(PyExc_TypeError, "Not a PyView"); return new PyView(o->Concat(*other), 0, o->computeState(RWVIEWER)); } --- 849,855 ---- static PyObject* PyView_concat(PyView *o, PyView *other) { try { ! if (other->ob_type != &PyViewtype && other->ob_type != &PyViewertype) Fail(PyExc_TypeError, "Not a PyView"); return new PyView(o->Concat(*other), 0, o->computeState(RWVIEWER)); } *************** *** 923,930 **** }; static PySequenceMethods ViewerAsSeq = { (inquiry)PyView_length, //sq_length ! (binaryfunc)0, //sq_concat ! (intargfunc)0, //sq_repeat (intargfunc)PyView_getitem, //sq_item (intintargfunc)PyView_getslice, //sq_slice (intobjargproc)0, //sq_ass_item --- 923,930 ---- }; static PySequenceMethods ViewerAsSeq = { (inquiry)PyView_length, //sq_length ! (binaryfunc)PyView_concat, //sq_concat ! (intargfunc)PyView_repeat, //sq_repeat (intargfunc)PyView_getitem, //sq_item (intintargfunc)PyView_getslice, //sq_slice (intobjargproc)0, //sq_ass_item From jyl@best.com Fri Feb 28 10:08:17 2003 Received: from jyl.best.vwh.net (jyl.best.vwh.net [192.220.76.211]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h1SG8Hc00808 for ; Fri, 28 Feb 2003 10:08:17 -0600 Received: (qmail 51549 invoked by uid 19667); 28 Feb 2003 16:10:27 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 28 Feb 2003 16:10:27 -0000 Message-ID: <3E5F8A29.6080807@best.com> Date: Fri, 28 Feb 2003 08:11:21 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: gmcm@hypernet.com CC: Michael Scharf , Metakit List Subject: Re: [Metakit] Some questions (FAQ?) References: <3E5F192B.2180.4CB9E210@localhost> Content-Type: multipart/alternative; boundary="------------040602010406080102090208" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------040602010406080102090208 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit > > >>Q: How to delete an entire view (the opposite of: storage.getas(...))? >> >Not sure if there's an easier way. You can create a 2nd >storage with the views you want and use slice assignment >to copy the data you want: v1[:] = v2 > Or you could do storage.getas() with an empty format ("") on the view you want to delete and then storage.commit(). >>Q: How to delete an entire column (how to transform >> "view[c1:S,c2:S]" to "view[c1:S]" persistently and >> freeing column c2)? >> > >storage.getas(). > Actually to make it persist, storage.getas() followed by storage.commit(). >>Q: How to shrink a database? (I read making a copy might do the >> job. But is there a generic way to make a copy of an >> arbitrary data structure)? >> > >storage.save(). A non-read-only database will generally be 2x the >size of the data, so shrinking it is not likely to be very useful - it >will just double on the next modification. (There are situations where >it's a useful trick, but they're pretty esoteric.) > >>Q: How do I know what views are persistent and which >> views are transient (blocked, hash, ordered seem >> persistent, sort, select, union etc. seem transient...)? >> > >What's persistent is what storage.description() tells you. >There are no view-returning methods that do in-place mutations. >Even with the mapping views (where MK will do the >maintenance for you), you need to inform MK that you >expect this on each open. (In other words, if I have an >ordered view, and I open it and don't use > v = v.ordered(1) >and then do an append, it will likely destroy the ordering.) > >Basically, there's no metadata except what >storage.description() tells you. > This is slightly incorrect IMHO. If you created a new view and haven't committed the storage yet, I believe storage.description will happily tell you about the not-yet-committed view. The only way I know of to do what you ask is to reopen the same storage in read-only mode and then do a storage.description(). That will tell you what's persisted inside the storage NOW. >>Q: Can I make derived views (like union, or sort) persistent? >> > >v[:] = v.sort() is perfectly valid. If you're sorting on leftmost >properties, you can subsequently use v.ordered(). > >You could similarly save a union, but you'd have to >define a persistent view with the right properties. > >Note that derived views are generally iterators - they >don't do much until you access them. So > v[:] = v.sort() >is very expensive since it forces an in-memory >copy of v. > > >>Q: It is said that metakit uses memory mapped files. When I >> create a new database, the file size remains 0 (on window) >> until I commit. Is this a problem of windows, or is the view >> in memory until (the first) commit? >> > >The latter. > >-- Gordon >http://www.mcmillan-inc.com/ > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > --------------040602010406080102090208 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit 
Q: How to delete an entire view (the opposite of: storage.getas(...))?
Not sure if there's an easier way. You can create a 2nd
storage with the views you want and use slice assignment
to copy the data you want: v1[:] = v2
Or you could do storage.getas() with an empty format ("") on the view you want to delete and then storage.commit().
Q: How to delete an entire column (how to transform
    "view[c1:S,c2:S]" to "view[c1:S]" persistently and
    freeing column c2)?

storage.getas().
Actually to make it persist, storage.getas() followed by storage.commit().
Q: How to shrink a database? (I read making a copy might do the
    job. But is there a generic way to make a copy of an
    arbitrary data structure)?

storage.save(). A non-read-only database will generally be 2x the 
size of the data, so shrinking it is not likely to be very useful - it
will just double on the next modification. (There are situations where 
it's a useful trick, but they're pretty esoteric.)
Q: How do I know what views are persistent and which
    views are transient (blocked, hash, ordered seem
    persistent, sort, select, union etc. seem transient...)?

What's persistent is what storage.description() tells you.
There are no view-returning methods that do in-place mutations.
Even with the mapping views (where MK will do the
maintenance for you), you need to inform MK that you
expect this on each open. (In other words, if I have an 
ordered view, and I open it and don't use 
 v = v.ordered(1)
and then do an append, it will likely destroy the ordering.)
 
Basically, there's no metadata except what
storage.description() tells you.
This is slightly incorrect IMHO. If you created a new view and haven't committed the storage yet, I believe storage.description will happily tell you about the not-yet-committed view. The only way I know of to do what you ask is to reopen the same storage in read-only mode and then do a storage.description(). That will tell you what's persisted inside the storage NOW.
Q: Can I make derived views (like union, or sort) persistent?

v[:] = v.sort() is perfectly valid. If you're sorting on leftmost
properties, you can subsequently use v.ordered().

You could similarly save a union, but you'd have to 
define a persistent view with the right properties.

Note that derived views are generally iterators - they
don't do much until you access them. So
 v[:] = v.sort()
is very expensive since it forces an in-memory
copy of v.
 
Q: It is said that metakit uses memory mapped files. When I
    create a new database, the file size remains 0 (on window)
    until I commit. Is this a problem of windows, or is the view
    in memory until (the first) commit?

The latter.

-- Gordon
http://www.mcmillan-inc.com/

_______________________________________________
metakit mailing list  -  metakit@equi4.com
http://www.equi4.com/mailman/listinfo/metakit


--------------040602010406080102090208-- From jrankin@oneil.com Mon Mar 3 09:26:54 2003 Received: from meridius.oneil.com (inetservicesout.oneil.com [208.154.227.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h23FQrc02535 for ; Mon, 3 Mar 2003 09:26:53 -0600 Received: from [192.168.1.20] ([192.168.1.20]:52494 "EHLO daytonnt20.dayton.oneil.com" ident: "NO-IDENT-SERVICE[2]") by meridius.oneil.com with ESMTP id ; Mon, 3 Mar 2003 11:20:42 -0500 Received: from nsmail.dayton.oneil.com (unverified) by daytonnt20.dayton.oneil.com (Content Technologies SMTPRS 4.3.6) with ESMTP id for ; Mon, 3 Mar 2003 10:28:54 -0500 Received: from jrankin ([192.168.1.149]) by nsmail.dayton.oneil.com (Netscape Messaging Server 3.62) with SMTP id 614 for ; Mon, 3 Mar 2003 10:28:51 -0500 Reply-To: From: "Jeff Rankin" To: Date: Mon, 3 Mar 2003 10:28:49 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal X-Spam-Status: No, hits=0.8 required=5.0 tests=SPAM_PHRASE_00_01,USER_AGENT_OUTLOOK version=2.42 X-Spam-Level: Subject: [Metakit] how to determining a db's meta data Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi: I building a few Metakit management tools using Tclkit and Metakit. Using Mk4tcl, is there a method to determine a Metakit databases 'meta' data? For example, is there a way to determine what views have been set up in the database, and perhaps the composition of these views? I've looked in the mkshow.tcl file in the source distribution but I don't see any obvious way to do this. Sincerely, W. Jeffrey Rankin Lead Web Application Developer O'NEIL & ASSOCIATES, INC. 495 Byers Rd. Miamisburg, Ohio 45342-3662 Phone: (937) 865-0800 ext. 3504 Fax: (937) 865-5858 E-mail: jrankin@oneil.com From jcw@equi4.com Mon Mar 3 18:25:41 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h240Pec21872; Mon, 3 Mar 2003 18:25:40 -0600 Date: Tue, 4 Mar 2003 01:27:58 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: announce@equi4.com, Metakit mailing list , Starkit list Content-Transfer-Encoding: 7bit Message-Id: <2694A91A-4DD8-11D7-A01D-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Subject: [Metakit] Metakit 2.4.9.1 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Only two weeks after the 2.4.9 release: here is a new update. The changes are minor, most changes affect only Mk4tcl or Mk4py. The reason to release again so soon was not because of some serious bug, but to synchronize the Metakit source code and version number with a new release of Tclkit (8.4.2). 2003-03-03 ############################################ MK 2.4.9.1 2003-03-02 Fixes to Mk4py (gmcm) 2003-03-01 Revert changes in Mk4py/scxx, avoid Mk4tcl warning 2003-02-28 Fixed relaxed layout in Mk4tcl, several Mk4py fixes 2003-02-27 Added support for HPUX aCC "long long" 2003-02-26 Added 64-bit int support to Mk4tcl, fix mingw 2003-02-24 Break was missing in switch Mk4py 2003-02-20 Remove a stray include, remove CR's 2003-02-18 ############################################## MK 2.4.9 As of now, a new 4-level version number is used, to indicate that only bug fixes and no feature changes whatsoever are planned for this code base henceforth. http://www.equi4.com/metakit -jcw From nicolasb@maich.gr Tue Mar 4 04:56:34 2003 Received: from mail-server ([193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h24AuXc04219 for ; Tue, 4 Mar 2003 04:56:33 -0600 Received: FROM maich.gr BY mail-server ; Tue Mar 04 13:32:39 2003 +0200 Message-ID: <3E64885D.3000001@maich.gr> Date: Tue, 04 Mar 2003 13:05:01 +0200 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0rc2) Gecko/20020510 X-Accept-Language: en-us, en MIME-Version: 1.0 To: jrankin@oneil.com CC: metakit@equi4.com Subject: Re: [Metakit] how to determining a db's meta data References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Rankin wrote: > Hi: > > I building a few Metakit management tools using Tclkit and Metakit. > > Using Mk4tcl, is there a method to determine a Metakit databases 'meta' > data? For example, is there a way to determine what views have been set up > in the database, and perhaps the composition of these views? Hi, Maybe I'm missing something, but... mk::file views db should return existing views in the db mk::view info db.view should return a property list.. hope this helps, regards, nicolas boretos > > I've looked in the mkshow.tcl file in the source distribution but I don't > see any obvious way to do this. > > Sincerely, > W. Jeffrey Rankin > Lead Web Application Developer > > O'NEIL & ASSOCIATES, INC. > 495 Byers Rd. > Miamisburg, Ohio 45342-3662 > Phone: (937) 865-0800 ext. 3504 > Fax: (937) 865-5858 > E-mail: jrankin@oneil.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > . > From jcw@equi4.com Wed Mar 5 17:04:45 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h25N4jC13772 for ; Wed, 5 Mar 2003 17:04:45 -0600 Date: Thu, 6 Mar 2003 00:07:04 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: <2E1E1305-4F5F-11D7-A01D-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Subject: [Metakit] Thread bugs fixed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Two bugs have just been found and resolved in Metakit. These caused potential race conditions in threaded C++ builds. The race is only possible between threads, and hence only when two or more threads each have a datafile open. This is related to creating or destroying c4_Property objects, something which happens mostly during open and close. This may explain why these problems were never detected. Tcl and Python are not affected (they have an extra mutex layer of their own). The changelog gives details. Changes (to view.cpp) have been checked into CVS. (What better way to bring out bugs than a new release, hah!) -jcw From Michael_Scharf@gmx.de Wed Mar 5 22:44:11 2003 Received: from mailout03.sul.t-online.com (mailout03.sul.t-online.com [194.25.134.81]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h264iAC06746 for ; Wed, 5 Mar 2003 22:44:10 -0600 Received: from fwd05.sul.t-online.de by mailout03.sul.t-online.com with smtp id 18qnH3-0002uS-01; Thu, 06 Mar 2003 05:46:29 +0100 Received: from gmx.de (320007127165-0001@[80.132.57.91]) by fmrl05.sul.t-online.com with esmtp id 18qnH1-1ZsJHsC; Thu, 6 Mar 2003 05:46:27 +0100 Message-ID: <3E66D2B7.10206@gmx.de> Date: Thu, 06 Mar 2003 05:46:47 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: multipart/mixed; boundary="------------070904020102050903030401" X-Sender: 320007127165-0001@t-dialin.net Subject: [Metakit] are there any examples for c4_View::Indexed? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------070904020102050903030401 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, is there any example or testcase for an indexed view? I did some experiments but the view does not quite hold what it promises: /** Create mapped view which maintains an index permutation * * This is an identity view which somewhat resembles the ordered view, it * maintains a secondary "map" view to contain the permutation to act as * an index. The indexed view presents the same order of rows as the * underlying view, but the index map is set up in such a way that binary * search is possible on the keys specified. When the "unique" parameter * is true, insertions which would create a duplicate key are ignored. * * This view is modifiable. Careful: when a row is changed in such a way * that its key is the same as in another row, that other row will be * deleted from the view. */ find is actually slower than linear search (unless, I completely misunderstand the comment) I attached the tests (in python and in C++). I hope I use indexed views the way they are supposed to be used. >mkdemo.exe linear 1729mS indexed 2740mS > python speed_indexed.py test[a:I,b:I] 100000 rows with b = a modulo 100; indexed on b search all 100 values of b linear search 4.05599999428 indexed search 5.4580000639 test[a:I,b:I 100000 rows with b = 100000-a; indexed on b; search 100 random values of b linear search 1.42199993134 indexed search 2.17299997807 Is there another way to efficiently search on some (arbitrary) columns? Michael --------------070904020102050903030401 Content-Type: text/plain; name="speed_indexed.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="speed_indexed.py" import metakit def speedtestA(N,M): print 'test[a:I,b:I] %i rows with b = a modulo %d; indexed on b search all %d values of b' %(N,M,M) s=metakit.storage() base=s.getas('base[a:I,b:I]') index=s.getas('base_index_b[ndx:I]') indexed=base.indexed(index,metakit.property('I','b')) # fill the table with b is a modulo M for i in xrange(N): base.append(a=i,b=i%M) # a search test method def findall(N,M,view): import time t0=time.time() for m in xrange(M): i=view.find(b=m) while i>=0: #print view[i].a,view[i].b i=view.find(b=m,start=i+1) return time.time()-t0 print 'linear search',findall(N,M,base) print 'indexed search',findall(N,M,indexed) def speedtestB(N,M): print 'test[a:I,b:I %i rows with b = %i-a; indexed on b; search %d random values of b' %(N,N,M) s=metakit.storage() base=s.getas('base[a:I,b:I]') index=s.getas('base_index_b[ndx:I]') indexed=base.indexed(index,metakit.property('I','b')) # fill the table with b in inverse order for i in xrange(N): base.append(a=i,b=N-i) # a search test method def findall(N,M,view): import time,random # make sure to get the same random order r=random.Random(42) t0=time.time() for m in xrange(100): i=view.find(b=r.randint(0,N)) #print view[i].a,view[i].b return time.time()-t0 print 'linear search',findall(N,M,base) print 'indexed search',findall(N,M,indexed) if __name__=='__main__': speedtestA(100000,100) speedtestB(100000,100) --------------070904020102050903030401 Content-Type: text/plain; name="demo.cpp" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="demo.cpp" // An example using the MetaKit C++ persistence library ///////////////////////////////////////////////////////////////////////////// // // This code demonstrates: // // - Creating a persistent view and adding two data rows to it. // - Adding a third data row using MetaKit's operator shorthands. // - Adding an additional property without losing the existing data. // - Storing an additional view in the data file later on. // - Inserting a new record into one of the views in the datafile. // - Real persistence, the data file will grow each time this is run. // ///////////////////////////////////////////////////////////////////////////// #include "mk4.h" #include "mk4str.h" #include #ifdef WIN32 #define WIN32_LEAN_AND_MEAN #include long ticks () { LARGE_INTEGER t; static double f = 0.0; if (f == 0.0) { QueryPerformanceFrequency(&t); f = (double) t.QuadPart / 10000000000.0; } QueryPerformanceCounter(&t); return (long) (f * t.QuadPart); } #else #include #include #include long ticks() { struct timeval tv; struct timezone tz; gettimeofday(&tv, &tz); return tv.tv_sec * 1000 + tv.tv_usec; } #endif long test(c4_View& view,int N,int M) { c4_IntProp pA("a"); long t0=ticks(); for(int i=0;i<100;i++) { view.Find(pA[N/M*i]); } return (ticks()-t0); } int #if _WIN32_WCE _cdecl #endif main() { c4_IntProp pA("a"); c4_IntProp pB("b"); c4_Storage storage; c4_View vBase = storage.GetAs("test[a:I,b:I]"); c4_Row row; const int N=100000; for(int i=0;i; Wed, 5 Mar 2003 22:50:07 -0600 Received: from fwd05.sul.t-online.de by mailout03.sul.t-online.com with smtp id 18qnMo-0000Tv-02; Thu, 06 Mar 2003 05:52:26 +0100 Received: from gmx.de (320007127165-0001@[80.132.57.91]) by fmrl05.sul.t-online.com with esmtp id 18qnMh-24daVcC; Thu, 6 Mar 2003 05:52:19 +0100 Message-ID: <3E66D418.1000808@gmx.de> Date: Thu, 06 Mar 2003 05:52:40 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20021120 Netscape/7.01 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: multipart/mixed; boundary="------------060102030006000006000401" X-Sender: 320007127165-0001@t-dialin.net Subject: [Metakit] crash in view.indexed in Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------060102030006000006000401 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit I attached a little unittest for view.indexed in python. Unfortunately, one test (commented out) crashes python and I cannot get the debug version of Mk4py to run (it crashes on importing). And because I am not a msdev user (I don't even have online help with msdev) I don't know how to debug the code. (I could move the stuff to a linux machine, but this is a purely windows project, and that would add quite some extra complication) I wrote similar code in C++ and it seems to work fine.... Michael --------------060102030006000006000401 Content-Type: text/plain; name="test_indexed.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="test_indexed.py" # # import unittest import metakit def rmfile(filename): import os try: os.remove(filename) except os.error: pass class IndexedTestCase(unittest.TestCase): def fillView(self,view): view.append(a='P',b='M') view.append(a='y',b='e') view.append(a='t',b='t') view.append(a='h',b='a') view.append(a='o',b='k') view.append(a='n',b='i') view.append(a='!',b='t') def asserViewEquals(self,view1,view2): self.assertEquals(len(view1),len(view2)) for i in range(len(view1)): self.assertEquals(view1[i].a,view1[i].a) self.assertEquals(view1[i].b,view1[i].b) def testAccessIndexed(self): s=metakit.storage() base=s.getas('base[a:S,b:S]') index=s.getas('base_index_b[ndx:I]') indexed=base.indexed(index,metakit.property('S','b')) self.fillView(base) self.asserViewEquals(base,indexed) def testAccessIndexedAfterBase(self): s=metakit.storage() base=s.getas('base[a:S,b:S]') self.fillView(base) # # create the indexed view after the original view # index=s.getas('base_index_b[ndx:I]') indexed=base.indexed(index,metakit.property('S','b')) self.asserViewEquals(base,indexed) def testFillIndexed(self): s=metakit.storage() base=s.getas('base[a:S,b:S]') index=s.getas('base_index_b[ndx:I]') indexed=base.indexed(index,metakit.property('S','b')) ######################################################################## # # CRASH! # ######################################################################## #self.fillView(indexed) def testReopen(self): filename='testReopen.mk4' # # add something in indexed mode # rmfile(filename) s=metakit.storage(filename,1) base=s.getas('base[a:S,b:S]') index=s.getas('base_index_b[ndx:I]') base.append(a='A',b='B') indexed=base.indexed(index,metakit.property('S','b')) s.commit() del base,index,indexed,s # # add something in base (no index) # s=metakit.storage(filename,1) base=s.getas('base[a:S,b:S]') self.fillView(base) s.commit() del base,s # # check if the index is updated # s=metakit.storage(filename,1) base=s.getas('base[a:S,b:S]') index=s.getas('base_index_b[ndx:I]') indexed=base.indexed(index,metakit.property('S','b')) self.asserViewEquals(base,indexed) rmfile(filename) if __name__=='__main__': unittest.main() --------------060102030006000006000401 Content-Type: text/plain; name="crash_indexed.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="crash_indexed.py" import metakit s=metakit.storage() base=s.getas('base[a:S,b:S]') index=s.getas('base_index_b[ndx:I]') indexed=base.indexed(index,metakit.property('S','b')) # # THE FOLLOWING LINE CRASHES PYTHON # indexed.append(a='x',b='y') --------------060102030006000006000401 Content-Type: text/plain; name="demo.cpp" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="demo.cpp" // An example using the MetaKit C++ persistence library ///////////////////////////////////////////////////////////////////////////// // // This code demonstrates: // // - Creating a persistent view and adding two data rows to it. // - Adding a third data row using MetaKit's operator shorthands. // - Adding an additional property without losing the existing data. // - Storing an additional view in the data file later on. // - Inserting a new record into one of the views in the datafile. // - Real persistence, the data file will grow each time this is run. // ///////////////////////////////////////////////////////////////////////////// #include "mk4.h" #include "mk4str.h" #include int #if _WIN32_WCE _cdecl #endif main() { c4_StringProp pA("a"); c4_StringProp pB("b"); c4_Storage storage; c4_View vBase = storage.GetAs("test[a:S,b:S]"); c4_View vIndex=storage.GetAs("inrex_test_b[idx:I]"); c4_View vIndexed=vBase.Indexed(vIndex,pB); c4_Row row; pA(row)="a"; pB(row)="a"; // THIS CRASHES IN PYTHON vIndexed.Add(row); return 0; } ///////////////////////////////////////////////////////////////////////////// --------------060102030006000006000401-- From jcw@equi4.com Fri Mar 7 15:59:41 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h27LxeC14824 for ; Fri, 7 Mar 2003 15:59:40 -0600 Date: Fri, 7 Mar 2003 23:02:01 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: <6CD9EC0E-50E8-11D7-83E1-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Subject: [Metakit] blocked viewer multi-deletion fixes Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bugs are my speciality, it seems! :( Some fixes have been checked in which at last should resolve all the issues concerning multi-row deletions in blocked views. A lot more assertion checking is now taking place to verify consistency in debug builds (q4_CHECK). A new "m07" test was added which now verifies proper operation of all the boundaries cases I can think of (well over a dozen). The effect of these problems was that after some multi-row deletions an internal vector of row offsets had incorrect values, causing subsequent access and modifications to fail - even potentially crash. The problem "only" affects deletions of more than one row at once, in specific boundary conditions, and only in blocked views. All changes have been checked into CVS. All regression tests pass on Win32, Linux, and MacOS X. -jcw From lvirden@cas.org Mon Mar 10 12:25:07 2003 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2AIP7C02246 for ; Mon, 10 Mar 2003 12:25:07 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.8/8.12.8/CAS_MAIL_HUB-2.00) with ESMTP id h2AIROsL016070 for ; Mon, 10 Mar 2003 13:27:24 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id h2AIROO13382; Mon, 10 Mar 2003 13:27:24 -0500 (EST) Date: Mon, 10 Mar 2003 13:27:24 -0500 (EST) From: "Larry W. Virden" Message-Id: <20030310132724.AAB13373@cas.org> To: Metakit mailing list Subject: [Metakit] mk::file open fail Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: If a script is attempting to open a metakit file and the open fails , is there a way to get more details about why the failure occured? The error that I'm seeing is: file open failed -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From mmartin6@cfl.rr.com Tue Mar 11 13:00:49 2003 Received: from orllnx01.orl.rbd.com (orllnx01.orl.rbd.com [208.61.212.238]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h2BJ0mC30638 for ; Tue, 11 Mar 2003 13:00:48 -0600 Message-Id: <200303111900.h2BJ0mC30638@trixie.triqs.com> Received: (qmail 3350 invoked from network); 11 Mar 2003 19:02:56 -0000 Received: from host252.orl.rbd.com (HELO developer) (208.61.212.252) by orllnx01.orl.rbd.com with SMTP; 11 Mar 2003 19:02:56 -0000 From: Marcus Martin To: Date: Tue, 11 Mar 2003 14:00:54 -0500 Reply-To: X-Account: Master RR X-Mailer: Express Plus Version 1.0.5.2 Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h2BJ0mC30638 Subject: [Metakit] zlib and metakit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am attempting to write a class to allow for metakit databases to be stored with zlib compression on the whole file. In order to this successful and maintain JD's extremely ruthless control over the files, I am following the paradigm set forth in the c4_FileStategy class as closely as possible. I have hit one snag though. If you open a new metakit file and try to write to it, the library will do a read first and then a write. I do not understand why this occurs. I see that a read is done on a empty file and something is returned, then writing begins. In zlib, reading an empty file returns a -2 (zlib's flag for a stream error). What is the purpose of performing a read on an empty file? I need to understand inorder to finish the class I am writing. Thanks, Marcus From mmartin6@cfl.rr.com Tue Mar 11 13:13:51 2003 Received: from orllnx01.orl.rbd.com (orllnx01.orl.rbd.com [208.61.212.238]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h2BJDoC32380 for ; Tue, 11 Mar 2003 13:13:50 -0600 Message-Id: <200303111913.h2BJDoC32380@trixie.triqs.com> Received: (qmail 3487 invoked from network); 11 Mar 2003 19:16:10 -0000 Received: from host252.orl.rbd.com (HELO developer) (208.61.212.252) by orllnx01.orl.rbd.com with SMTP; 11 Mar 2003 19:16:10 -0000 From: Marcus Martin To: , Date: Tue, 11 Mar 2003 14:14:08 -0500 Reply-To: X-Account: Master RR X-Mailer: Express Plus Version 1.0.5.2 Subject: RE: [Metakit] zlib and metakit Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h2BJDoC32380 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What the... My mail client has butchered my very own email. Let me try it again. I am attempting to write a class to allow for metakit databases to be stored with zlib compression on the whole file. In order to this successful and maintain JD's extremely ruthless control over the files, I am following the paradigm set forth in the c4_FileStrategy class. I have hit one snag though. If you open a new metakit file and try to write to it, the library will do a read first and then a write. I do not understand why this occurs. I see that a read is done on a empty file and something is returned, then writing begins. When i attempt to read an empty zlib file, it correctly returns a -2 (zlib's stream error flag). I do not understand why metakit performs a read on an empty file and what I should do when this occurs. Any advice would be appreciated. Thanks, Marcus ----- Original message ----------------------------------------> From: Marcus Martin To: Received: Tue, 11 Mar 2003 14:00:54 -0500 Subject: [Metakit] zlib and metakit >I am attempting to write a class to allow for metakit databases to be stored with zlib >compression on the whole file. In order to this successful and maintain JD's extremely >ruthless control over the files, I am following the paradigm set forth in the c4_ >I have hit one snag though. If you open a new metakit file and try to write to it, the >library will do a read first and then a write. I do not understand why this occurs. I see >that a read is done on a empty file and something is returned, then writing be >Thanks, >Marcus >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit From bkelley@wi.mit.edu Thu Mar 13 12:49:18 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h2DInIC24926 for ; Thu, 13 Mar 2003 12:49:18 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.0.0.44) with SMTP id M2003031313514412254 for ; Thu, 13 Mar 2003 13:51:44 -0500 Received: from wi.mit.edu (stockwell19.wi.mit.edu [18.157.1.112]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HBPB2800.IR6 for ; Thu, 13 Mar 2003 13:51:44 -0500 Message-ID: <3E70D33F.40906@wi.mit.edu> Date: Thu, 13 Mar 2003 13:51:43 -0500 From: "Brian Kelley" Reply-To: bkelley@wi.mit.edu Organization: Whitehead Institute for Biomedical Research User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.2.1) Gecko/20021130 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Compiling Python-Metakit for MACOSX gcc Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The default unix build was causing some headaches under MACOSX using gcc 3 from the apple developers kit. I expect that this is because apple only ships libstdc++ as a static library. You can create a usable build by 1) following the make file instructions (you will be in the build directory) 2) watching it fail and then 3) execute the following (again from the build directory): g++ -bundle -undefined suppress -flat_namespace -lstdc++ \ PWOImp.o PyView.o field.o persist.o string.o viewx.o \ PyProperty.o column.o fileio.o remap.o table.o PyRowRef.o \ custom.o format.o std.o univ.o PyStorage.o derived.o \ handler.o store.o view.o -o .libs/libmk4py.so I'm not sure that all the flags are needed but they are what swig uses so I went with them. The resulting Mk4py.so file is a little large (420k) but it works. I'll be happy to email this in if desired. Since I don't really need to make a dynamic library, I could contribute a setup.py file that will to create and install this file along with the python portion of the library. Brian Kelley Whitehead Institute for Biomedical Research From schlenk@uni-oldenburg.de Sun Mar 16 07:25:09 2003 Received: from mailout09.sul.t-online.com (mailout09.sul.t-online.com [194.25.134.84]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2GDP8C16496 for ; Sun, 16 Mar 2003 07:25:08 -0600 Received: from fwd02.sul.t-online.de by mailout09.sul.t-online.com with smtp id 18uYAn-0000Wz-0B; Sun, 16 Mar 2003 14:27:33 +0100 Received: from pD9526E6A.dip.t-dialin.net (520047057634-0001@[217.229.109.113]) by fmrl02.sul.t-online.com with esmtp id 18uYAm-0NHvoOC; Sun, 16 Mar 2003 14:27:32 +0100 Received: from uni-oldenburg.de (unknown [192.168.2.2]) by pD9526E6A.dip.t-dialin.net (Postfix) with ESMTP id 470E97BEF; Sat, 31 Mar 2001 15:48:51 +0000 (/etc/localtime) Message-ID: <3E747C53.1020001@uni-oldenburg.de> Date: Sun, 16 Mar 2003 14:29:55 +0100 From: Michael Schlenker Organization: =?ISO-8859-1?Q?Universit=E4t_Oldenburg?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2a) Gecko/20020910 X-Accept-Language: de-de, en-us MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 520047057634-0001@t-dialin.net Subject: [Metakit] Documentation for OO style tcl binding? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I was looking at the OO-style tcl bindings, as used in the examples dir, for comparing hashed, blocked and ordered access. I didn't find much docs (other than the sparse collection of pages dgp put up at the metakit wiki http://www.equi4.com/metakit/wiki.cgi/147 ). Does documentation exists, is it available without decyphering C++ code (I'm not in the mood for that)? Basically i'm looking for docs, what blocked views and hashed views and ordered views are all about, what are the benefits, what are the drawbacks, how do i use them correctly? (Usage example, for the tcl binding preferred) One direct question: Is my assumption correct, that for "VIEW view hash " arg gives the number of buckets in the hash table, or what is it? Michael Schlenker From jcw@equi4.com Sun Mar 16 16:51:50 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2GMpnC31163 for ; Sun, 16 Mar 2003 16:51:49 -0600 Date: Sun, 16 Mar 2003 23:54:12 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: <34ABEC22-5802-11D7-868F-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Subject: [Metakit] Metakit 2.4.9.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a minor bug release. 2003-03-16 ############################################ MK 2.4.9.2 2003-03-10 Fixes for sparc64 configure and AIX strcasecmp 2003-03-07 Fix more bugs in blocked view, add 64-bit Sparc 2003-03-05 Fixed two potential races in C++ threaded build 2003-03-03 ############################################ MK 2.4.9.1 My apologies for the rapid succession of releases. Though 2.4.9.1 was supposed to last, the threading and blocked view bugs were substantial enough for those who need this functionality to warrant *one* more release. -jcw From borco@go.ro Mon Mar 17 01:54:38 2003 Received: from borco.local.unidec.ro ([81.196.100.134]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2H7sbC15557 for ; Mon, 17 Mar 2003 01:54:38 -0600 Received: from localhost ([127.0.0.1] ident=borco) by borco.local.unidec.ro with esmtp (Exim 3.36 #1 (Debian)) id 18upUW-0000N0-00; Mon, 17 Mar 2003 09:57:04 +0200 From: Ionutz Borcoman To: metakit@equi4.com Content-Type: text/plain Organization: Message-Id: <1047887824.1376.1.camel@borco.local.unidec.ro> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 17 Mar 2003 09:57:04 +0200 Content-Transfer-Encoding: 7bit Subject: [Metakit] newbye f.a.q.'s Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I have discovered recently the MetaKit and started playing with it (using the Python bindings). I still have a couple of things that I don't understand fully (probably because of my lack of understanding and experience with databases ...): 1. How do I add values to a view with sub-views ? I've seen that you can retrieve the index of the main view and use it to access the sub-view. Is there a straighter approach ? Now I'm doing something like this: vw = db.getas("archive[name:S,dirs[name:S,parent:I,files[name:S]]]") idx1 = vw.append(name = 'ar1') idx2 = vw[idx1].dirs.append(name='dir1',parent=0) vw[idx1].dirs[idx2].append(name='file1') # didn't test this yet ... Can you please include such an example in the documentation of the 'Metakit for Python' documentation ? 2. Can I declare a column to store unique values ? If not, what's the best way to simulate/replace this ? 3. I need some table to be joined by some foreign key (an index, preferably). What's the best approach for this ? Is there any way to declare a column to store unique indexes auto-generated by metakit ? Is there a simple/recommended way to do this ? Using the index returned by the metakit on appending is not very useful here, because the index is going to change on row removal; if I use it, I'll have to check all related views and update them, too. Is there a solution to this ? 4. Can I create a complex search criteria ? Can I use the results of select() to delete the rows ? Why does the example derive.py doesn't work ? Is this a bug, is the example obsolete or is the example based on a future feature ? (The example fails when trying to delete from the view returned by select, v2, as the view is immutable). Bellow is a simple test program I used to test removing from the database. Now I can select based on a single key, but I don't know how to select based on multiple keys/criteria. Can I use ranges ? Like "value > 1 and value < 10 or name = 'ar1'" ? ################### test_remove.py start ####################### import os import metakit def tolist(v): return [(x.name,x.value) for x in v] try: os.remove("test.mk") except: pass db = metakit.storage("test.mk",1) vw = db.getas("test[name:S,value:I]") vw.append(name = 'ar1', value = 1) vw.append(name = 'ar2', value = 2) vw.append(name = 'ar3', value = 3) vw.append(name = 'ar1', value = 2) vw.append(name = 'ar2', value = 1) vw.append(name = 'ar1', value = 1) vw.append(name = 'ar3', value = 1) db.commit() print 'View is', tolist(vw) v1 = vw.select(name='ar1') # why can't I simply remove lines returned by v1 ? print 'Selected only those with "ar1"', tolist(v1) while 1: idx = vw.find(name = 'ar1') # how do I find the index of rows with # name = 'ar1' or value > 1 and value < 2? if idx == -1: break print 'Delete %d from %s' % (idx, tolist(vw)) vw.delete(idx) print 'After deletion view is :', tolist(vw) db.commit() db = None ################### test_remove.py end ######################## TIA, Ionutz From mike.collins@riley.army.mil Mon Mar 17 13:37:12 2003 Received: from rildoimml01.riley.army.mil (rildoimml01.riley.army.mil [144.246.27.61]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2HJbBC31075 for ; Mon, 17 Mar 2003 13:37:11 -0600 Received: by rildoimml01.riley.army.mil with Internet Mail Service (5.5.2653.19) id ; Mon, 17 Mar 2003 13:35:34 -0600 Message-ID: <464A43842A2EF54F9183D07E0DEC43560102E194@rildoimml05.riley.army.mil> From: "Collins, Mike DAC DOIM" To: "'metakit@equi4.com'" Date: Mon, 17 Mar 2003 13:39:30 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Tequila Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all. I've been playing around with Tequila as of late and I saw a blurp somewhere about storing scripts on the Tequila Server. If I were going to do something like that do I understand that it would be just as any other variable in TCL and that you would just store those variables in fields in a metakit datafile on the Server, so to speak. Or is there something a little more exotic going on, that I missed? Anybody also got any working examples of using the Tequila::proxy function? Thanks in advance. From bkelley@wi.mit.edu Tue Mar 18 12:51:52 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h2IIpqC20350 for ; Tue, 18 Mar 2003 12:51:52 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.0.0.44) with SMTP id M2003031813541123405 for ; Tue, 18 Mar 2003 13:54:12 -0500 Received: from wi.mit.edu (stockwell19.wi.mit.edu [18.157.1.112]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HBYKIB00.UEM for ; Tue, 18 Mar 2003 13:54:11 -0500 Message-ID: <3E776B53.6070104@wi.mit.edu> Date: Tue, 18 Mar 2003 13:54:11 -0500 From: "Brian Kelley" Reply-To: bkelley@wi.mit.edu Organization: Whitehead Institute for Biomedical Research User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.2.1) Gecko/20021130 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Compiling Python-Metakit for MACOSX gcc References: <3E70D33F.40906@wi.mit.edu> In-Reply-To: <3E70D33F.40906@wi.mit.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I missed a step, after the g++ command just type make again Brian Kelley wrote: > The default unix build was causing some headaches under MACOSX using gcc > 3 from the apple developers kit. I expect that this is because apple > only ships libstdc++ as a static library. > > You can create a usable build by > 1) following the make file instructions (you will be in the build directory) > 2) watching it fail and then > 3) execute the following (again from the build directory): > > g++ -bundle -undefined suppress -flat_namespace -lstdc++ \ > PWOImp.o PyView.o field.o persist.o string.o viewx.o \ > PyProperty.o column.o fileio.o remap.o table.o PyRowRef.o \ > custom.o format.o std.o univ.o PyStorage.o derived.o \ > handler.o store.o view.o -o .libs/libmk4py.so > 4) type make again > I'm not sure that all the flags are needed but they are what swig uses > so I went with them. The resulting Mk4py.so file is a little large > (420k) but it works. > > I'll be happy to email this in if desired. > > Since I don't really need to make a dynamic library, I could contribute > a setup.py file that will to create and install this file along with the > python portion of the library. > > Brian Kelley > Whitehead Institute for Biomedical Research > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Wed Mar 19 02:39:33 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2J8dWC13977; Wed, 19 Mar 2003 02:39:32 -0600 Date: Wed, 19 Mar 2003 09:41:56 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; delsp=yes; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list , Starkit list Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.551) Subject: [Metakit] Metakit performance datapoint Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: With the caveat that benchmarks can be made to prove anything, and that I will mention only what suits me best, here's a pointer to some recent benchmarks in which Metakit was included: http://sourceforge.net/mailarchive/ forum.php?thread_id=1811280&forum_id=1100 (the sqlite timings are hard to put in perspective: different machine) I'll just add one comment, about the fact that one MK timing appears to have O(N^2) performance: this has been coded without blocked or hashed views. Thanks to Gordon for pointing these timings out. Make sure you get the orders of magnitude right when interpreting these numbers :) -jcw From jcw@equi4.com Wed Mar 19 03:10:06 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2J9A5C15901; Wed, 19 Mar 2003 03:10:05 -0600 Date: Wed, 19 Mar 2003 10:12:30 +0100 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) Cc: Starkit list To: Metakit mailing list From: Jean-Claude Wippler Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.551) Subject: [Metakit] Free-riding on open source software Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is an issue which I want to bring up *before* it becomes a serious problem. As you know, Metakit is open source software. It has evolved and grown as an independent product over a period of 7 years now, the second half of which in open source form, using the liberal X11/MIT open source license. As you may or may not know, my company also offers a commercial "Metakit Enterprise License" for those who require a more formal level of support (one year at a time) and for those who wish to support me and my work on Metakit to make it a sustainable activity. See http://www.equi4.com/metakit/license.html I am pleased to report that the number of Enterprise License customers is increasing, with a few particularly loyal customers renewing the license on a yearly basis. But at the same time, I see an alarming trend. There are *several* major companies by now, which have based some of their commercial offerings heavily on Metakit, yet who have not - for quite some time in some cases - taken the step to purchase an Enterprise License. I am willing to assume that this is because I do not tend to make a fuss about that side of things - the option may not even be known to some. The "Prisoner's Dilemma" insight, which comes from game theory, illustrates how it is very logical for parties to leave cost issues up to others, i.e. "if others pay, why stick my neck out?", and "if they are not paying, why should I?". Unfortunately, in the case of MK, theory and practice need to be in sync. Like most mortals on this planet, "food on the table" as well as "financial reward" are very real concepts to me. I have chosen the path of making Metakit open source, and sharing inordinate amounts of my time with others, on the basis that commercial projects are what will keep me going, while MK (and the derived Tclkit project) are "spin-offs". And they are, and it works (yes, trust me, it does!). Equi4 Software has allowed me to be independent for almost 14 years by now, and I see no reason why that could not last. As I have said a few times in the past, independence is more than not having a boss: it also is a crucial foundation for thinking outside the box. But there is an inequality which worries me. Some highly commercial companies, for which the cost of a Metakit Enterprise License is almost too low to take seriously, are relying on Metakit, its data structures, and most importantly: the data of their customers. They make their business model work using open source, yet do not "give back" or "do the right thing". The reason this concerns me is not what you probably think. I'm not talking about my bottom line, or having a hard time making a living (though I've seen better years, yes). The problem is the future of my work. I am currently working on a number of new implementation ideas, which take some of the lessons and experiences learned from Metakit *way* further. That means I'm contemplating moving away more from paid contract work, and focusing almost exclusively on design, architecture, and algorithms. Having a more independent level of funding would free me more to work on such things. And share them, as I *very* much would like to. But to do so, I urge those of you who have incorporated Metakit into your commercial products, or plan to do so, to broaden your perspective and think ahead just a bit further. Without a certain level of financial support, I will be forced to reconsider the proprietary option. I'm not saying this lightly. FWIW, I strongly believe in the strength of open source software - and the quality benefits it brings to everyone involved. For something as fundamental as long-lived data storage, it's more than a nice ideology - it's crucial IMO. It's hard for me as techie to refrain from talking about the ideas themselves here, I can assure you. All I'll say for now is: you ain't seen nothin' yet, the data concepts which lie underneath Metakit are in their infancy. I'm talking orders of magnitude in both performance and scalability, and with today's CPU architectures the trade-offs are in fact getting more and more in favor of MK's approach. With your help, we can sit on the same side of the table and have a shared interest. Please don't force me to work in secrecy. -jcw From borco@go.ro Wed Mar 19 03:36:22 2003 Received: from borco.local.unidec.ro ([81.196.100.134]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2J9aLC17931 for ; Wed, 19 Mar 2003 03:36:21 -0600 Received: from localhost ([127.0.0.1] ident=borco) by borco.local.unidec.ro with esmtp (Exim 3.36 #1 (Debian)) id 18va25-0001Qi-00; Wed, 19 Mar 2003 11:38:49 +0200 Subject: Re: [Metakit] newbye f.a.q.'s From: Ionutz Borcoman To: metakit@equi4.com In-Reply-To: <1047887824.1376.1.camel@borco.local.unidec.ro> References: <1047887824.1376.1.camel@borco.local.unidec.ro> Content-Type: text/plain Organization: Message-Id: <1048066728.839.10.camel@borco.local.unidec.ro> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 19 Mar 2003 11:38:48 +0200 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm still waiting for some answers :-/ Is the lack of any reply meaning that MetaKit is not what I need and that it can't use it for my project ? If that's the case, I'll go with SQLite, even though it'll be harder for me in some areas (it has no binary storage, so I'll have to use binhex conversions, and, from your benchmarks, it looks slower), but at least it has unique indexes and I can delete/update/find rows based on complex criteria. TIA, Ionutz On Mon, 2003-03-17 at 09:57, Ionutz Borcoman wrote: > Hi, > > I have discovered recently the MetaKit and started playing with it > (using the Python bindings). > > I still have a couple of things that I don't understand fully (probably > because of my lack of understanding and experience with databases ...): > > 1. How do I add values to a view with sub-views ? I've seen that you can > retrieve the index of the main view and use it to access the sub-view. > Is there a straighter approach ? Now I'm doing something like this: > > vw = db.getas("archive[name:S,dirs[name:S,parent:I,files[name:S]]]") > idx1 = vw.append(name = 'ar1') > idx2 = vw[idx1].dirs.append(name='dir1',parent=0) > vw[idx1].dirs[idx2].append(name='file1') # didn't test this yet ... > > Can you please include such an example in the documentation of the > 'Metakit for Python' documentation ? > > 2. Can I declare a column to store unique values ? If not, what's the > best way to simulate/replace this ? > > 3. I need some table to be joined by some foreign key (an index, > preferably). What's the best approach for this ? Is there any way to > declare a column to store unique indexes auto-generated by metakit ? Is > there a simple/recommended way to do this ? Using the index returned by > the metakit on appending is not very useful here, because the index is > going to change on row removal; if I use it, I'll have to check all > related views and update them, too. Is there a solution to this ? > > 4. Can I create a complex search criteria ? Can I use the results of > select() to delete the rows ? Why does the example derive.py doesn't > work ? Is this a bug, is the example obsolete or is the example based on > a future feature ? (The example fails when trying to delete from the > view returned by select, v2, as the view is immutable). > > Bellow is a simple test program I used to test removing from the > database. Now I can select based on a single key, but I don't know how > to select based on multiple keys/criteria. Can I use ranges ? Like > "value > 1 and value < 10 or name = 'ar1'" ? > > ################### test_remove.py start ####################### > import os > import metakit > > def tolist(v): > return [(x.name,x.value) for x in v] > > try: os.remove("test.mk") > except: pass > > db = metakit.storage("test.mk",1) > vw = db.getas("test[name:S,value:I]") > vw.append(name = 'ar1', value = 1) > vw.append(name = 'ar2', value = 2) > vw.append(name = 'ar3', value = 3) > vw.append(name = 'ar1', value = 2) > vw.append(name = 'ar2', value = 1) > vw.append(name = 'ar1', value = 1) > vw.append(name = 'ar3', value = 1) > db.commit() > > print 'View is', tolist(vw) > v1 = vw.select(name='ar1') > # why can't I simply remove lines returned by v1 ? > > print 'Selected only those with "ar1"', tolist(v1) > while 1: > idx = vw.find(name = 'ar1') > # how do I find the index of rows with > # name = 'ar1' or value > 1 and value < 2? > if idx == -1: > break > print 'Delete %d from %s' % (idx, tolist(vw)) > vw.delete(idx) > print 'After deletion view is :', tolist(vw) > > db.commit() > db = None > > ################### test_remove.py end ######################## > > TIA, > > Ionutz From borco@go.ro Wed Mar 19 10:25:40 2003 Received: from borco.local.unidec.ro ([81.196.100.134]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2JGPeC16676 for ; Wed, 19 Mar 2003 10:25:40 -0600 Received: from localhost ([127.0.0.1] ident=borco) by borco.local.unidec.ro with esmtp (Exim 3.36 #1 (Debian)) id 18vgQ6-0000GO-00; Wed, 19 Mar 2003 18:28:02 +0200 Subject: Re: [Metakit] newbye f.a.q.'s From: Ionutz Borcoman To: gmcm@hypernet.com Cc: Metakit In-Reply-To: <3E783CEB.7107.AEEE76B4@localhost> References: <1047887824.1376.1.camel@borco.local.unidec.ro> <3E783CEB.7107.AEEE76B4@localhost> Content-Type: text/plain Organization: Message-Id: <1048091281.695.13.camel@borco.local.unidec.ro> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 19 Mar 2003 18:28:02 +0200 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Thanks for the reply. On Wed, 2003-03-19 at 16:48, Gordon McMillan wrote: > On 19 Mar 2003 at 11:38, Ionutz Borcoman wrote: > > > Is the lack of any reply meaning that MetaKit is not what I need and > > that it can't use it for my project ? > > No, it means people are busy and you don't appear to have done > much investigation. I have try to follow the demos and many are broken. It's not easy to investigate this way. Also, the docs are minimal. I've also search the list archive to little success. > > If that's the case, I'll go with SQLite, even though it'll be harder for > > me in some areas (it has no binary storage, so I'll have to use binhex > > conversions, and, from your benchmarks, it looks slower), but at least > > it has unique indexes and I can delete/update/find rows based on complex > > criteria. > > View.select can do ranges. Unique indices are easy to construct & maintain. > MkSQL proves that Metakit can do anything SQL can. I suggest you sit > at the interactive prompt and check out the doc strings of the view methods. I've seen your MkSQL. It was on my list to try using it, but I wanted first to get a grasp on metakit itself. I'll go with MkSQL if you say it can give me all I want ;-) Can you please explain the problem with UPDATE ? On the "Grammar page" you are saying the UPDATE is dangerous ? Is it dangerous even if you identify the row by it's UNIQUE id ? How else can I change data in database ? Only by DELETE and INSERT a new row ? Isn't this a bit extreme ? Isn't this memory/CPU consuming if the row contains a big BLOB but I want to change only a short text stored on other column ? TIA, Ionutz From gmcm@hypernet.com Wed Mar 19 12:16:56 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2JIGuC29349 for ; Wed, 19 Mar 2003 12:16:56 -0600 Received: from PARANOIA (204.176.40.135) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Wed, 19 Mar 2003 13:21:11 -0500 From: "Gordon McMillan" To: Ionutz Borcoman , Metakit Date: Wed, 19 Mar 2003 13:17:51 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] newbye f.a.q.'s Reply-to: gmcm@hypernet.com Message-ID: <3E786DFF.27145.AFAE29CD@localhost> Priority: normal References: <3E783CEB.7107.AEEE76B4@localhost> In-reply-to: <1048091281.695.13.camel@borco.local.unidec.ro> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 19 Mar 2003 at 18:28, Ionutz Borcoman wrote: > I have try to follow the demos and many are broken. It's not easy to > investigate this way. Also, the docs are minimal. I've also search the > list archive to little success. What can I say. It's one of a number of OS projects I work on. I try to keep the doc strings accurate. Some things that used to work don't anymore, but since they're not in my usage patterns I don't have much motivation to fix them. People send in code patches, but doc patches are very, very rare. > Can you please explain the problem with UPDATE ? On the "Grammar page" > you are saying the UPDATE is dangerous ? Is it dangerous even if you > identify the row by it's UNIQUE id ? How else can I change data in > database ? Only by DELETE and INSERT a new row ? Isn't this a bit > extreme ? Isn't this memory/CPU consuming if the row contains a big BLOB > but I want to change only a short text stored on other column ? Nothing wrong with UPDATE. What's evil is "SELECT FOR UPDATE", which consumes enormous resources. If you can't figure out how to do a mass update in one UPDATE statement, then SELECT in a r/o cursor and UPDATE them one at a time (with checks that they haven't been updated). The difference in scalability is at least an order of magnitude. I wasn't saying you should use MkSQL. But it demonstrates how to use view.filter, view.remove, set up unique indices and other advanced features. -- Gordon http://www.mcmillan-inc.com/ From bkelley@wi.mit.edu Sat Mar 22 11:10:16 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h2MHAFC11831 for ; Sat, 22 Mar 2003 11:10:16 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.0.0.44) with SMTP id M2003032212124830049 for ; Sat, 22 Mar 2003 12:12:48 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HC5UHC00.N6D for ; Sat, 22 Mar 2003 12:12:48 -0500 Message-ID: <3E7C9B1E.9000408@wi.mit.edu> Date: Sat, 22 Mar 2003 12:19:26 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20020530 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: multipart/mixed; boundary="------------010704070303050704060604" Subject: [Metakit] Metakit Python Tutorial Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------010704070303050704060604 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Somewhat based on the statement by Gordon McMillan: I don't have much motivation to fix them. People send in code patches, but doc patches are very, very rare. I spent some time dumping some of my knowledge of metakit into a small tutorial. If some kind soul could double check the following code and comments I will compile it into a python-style mini-howto. Suggestions are welcome and I will be updating this (read "making it more verbose and adding more examples") as I learn more about metakit. Cheers. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 --------------010704070303050704060604 Content-Type: text/plain; name="metakit_tutorial.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="metakit_tutorial.py" # This is some informal documentation and test for metakit # It basically is designed so that I can remember what # the hell I'm doing import metakit, os, marshal try: os.remove("test.mk") except: pass READONLY = 0 READWRITE = 1 COMMITEXTEND = 2 print "#######################################################" print "making the database" db = metakit.storage("test.mk", READWRITE) # let's make a table table = db.getas("test[testid:I,string:S,marshal:B]") # The marshalling stores basic python types # This needs to be stored as a "B"inary type data = [(0,"foo",marshal.dumps("I am a foo!")), (1,"bar",marshal.dumps(123)), (2,"baz",marshal.dumps([3,4,5,3,4,5,6,7,7,8]))] for row in data: # append the data into the rows (all this # stuff is coerced to live in the database) table.append(row) db.commit() print "#######################################################" print "making temporary table for joining" # we are going to perform a query by joining two # tables together # now let's make a temporary indices list # and coerrce it into a row indices = db.getas("_temp[testid:I]") indices.append(2) # now we can make a subview of the table as a result # of this join subview = table.join(indices, indices.testid) assert len(subview) == 1 for row in subview: print row.testid, row.string, marshal.loads(row.marshal) # we should probably drop the temporary table # in case a commit is called later del indices[:] # this can also be done indices[:] = [] # I'm committing here just to test whether the indices table has # been dropped or not later db.commit() print "#######################################################" print "wrapping a sequence of dictionaries" # alternatively we can "wrap" a python sequence into a # temporary view. This wraps a sequences using # the respective database types. so below # [2,...], [metakit.property("I", "testid")... # the first column is wrapped by an Integer column named testid indices = metakit.wrap([{"testid":x} for x in [2]], [metakit.property("I", "testid")]) subview = table.join(indices, indices.testid) # for row in subview: print row.testid, row.string, marshal.loads(row.marshal) print "\n#######################################################" print "wrapping a sequence of dictionaries using the tables properties" # If you don't want to remember how to make the property # lists, you can just use the ones from another table indices = metakit.wrap([{"testid":x} for x in [2]], [table.testid]) subview = table.join(indices, indices.testid) for row in subview: print row.testid, row.string, marshal.loads(row.marshal) del db # This just tests to see if the temporary # table we made has been dropped or not db = metakit.storage("test.mk", READWRITE) table = db.getas("test[testid:I,string:S,marshal:B]") test = db.getas("_temp[testid:I]") assert len(test) == 0 print "\n#############################################################" print "Things that don't work as I expect" print print "select only uses the first two criteria" v = table.select({"testid":0}, {"testid":1}, {"testid":2}) print len(v), "-> should be 3" v = table.select({"testid":1}, {"testid":2}, {"string":"foo"}) print len(v), "-> should be 1" print "\n#############################################################" print "view.filter doesn't return the whole row, just the matching" print "indices in a table with an index attribute" print "Not much can be done with these now, hopefully when the" print "Mapped Index view is completed we can use this output" print "to create a subview that has access to the rows of the" print "original data" vw = table.filter(lambda x: 1) metakit.dump(vw) print "\n#############################################################" print "here is a version of filter that is a little better," print "the returned subview has the same rows but the result" print "is read only." print "This is perhaps the most confusing aspect of metakit" print "Most subview returned are readonly, only selects and" print "sorts are really tracked in this way. See" print "http://www.equi4.com/metakit/wiki.cgi/99" def filter(view, func): result = view i = 0 for row in view: if not func(row): result = result.minus(view[i:i+1]) i += 1 return result subview = filter(table, lambda x:x.testid<2) assert len(subview) == 2 try: subview[0].string = "changed" except TypeError: print "caught TypeError, this was expected" print "\n#############################################################" print "here is an enhanced filter that returns a trackable" print "subview, view must be a data table not another" print "subview" print "You'll see a new column in the database but this won't" print "be saved when the data is commited" def filter(view, func): view.addproperty(metakit.property("I", "_temp__344_343")) for row in view: if func(row): row._temp__344_343 = 1 else: row._temp__344_343 = 0 return view.select(_temp__344_343=1) subview = filter(table, lambda x:x.testid<2) assert len(subview) == 2 metakit.dump(subview) subview[0].string = "changed" --------------010704070303050704060604-- From Michael_Scharf@gmx.de Sun Mar 23 08:58:51 2003 Received: from mailout03.sul.t-online.com (mailout03.sul.t-online.com [194.25.134.81]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2NEwoC01679; Sun, 23 Mar 2003 08:58:51 -0600 Received: from fwd02.sul.t-online.de by mailout03.sul.t-online.com with smtp id 18x6yQ-0006F0-08; Sun, 23 Mar 2003 16:01:22 +0100 Received: from gmx.de (320007127165-0001@[80.132.63.87]) by fmrl02.sul.t-online.com with esmtp id 18x6yG-03SU9AC; Sun, 23 Mar 2003 16:01:12 +0100 Message-ID: <3E7DCC5A.2030104@gmx.de> Date: Sun, 23 Mar 2003 16:01:46 +0100 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 CC: Metakit mailing list , Starkit list Subject: Re: [Metakit] Free-riding on open source software References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 320007127165-0001@t-dialin.net Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude, your posting made me think. I am trying at the moment (besides my normal job) to create my own independent business. At the moment I make no money at all with my idea. I'm still in the development phase. Because I don't want to have to high initial investment costs (besides my time), I try to choose open source libraries as basis of my work. There are 3 things I like about it: - initially no investment (in money) - though in case of metakit the lack of extensive documentation yields quite some time investment. - source is available (it helped me to overcome the documentation deficiencies. I could also report bugs with fixes...) - there are support groups (also helpful) Once my company makes money, I have absolutely no problem to pay for the open source libraries (if the price is reasonable). A negative example for me is For BSD-DB 4.x: SleepyCat wants between $25,000 and $300,000 plus $5,000-$60,000 annual maintenance. Which is simply outside any realistic considerations for a product that will cost $20 per license. I would have to sell 15,000 copies just to pay for the SleepyCat license... I told them that's far too expensive for me and the sales representative answered: "...we could talk about financing a buyout for you, where you paid the cost over a year or two, which would alleviate the up-front large-payment problem." Actually, they even want me to pay if I make my product freeware but not open source... Interestingly, bsddb is free for open source products and it therefore is widely distributed and used. It's also free when used on a single host (like a web server). What I want to say is that I think the concept "free for open source and some payment for commercial products" is OK, unless the payment is reasonable. I think if you "force" commercial users to pay the reasonably low fee and keep the product open source, the number of registered users could increase. As you pointed out, for some of the companies the annual fee is just "nothing". I think, they would not want to risk to get a legal problem and would pay. Unless you increase the price drastically, I think even one-person shareware programmers could afford a license. So, my suggestion is: keep it open source, but "force" commercial user to pay a "little" fee.... Michael From steve@DigitalSmarties.com Mon Mar 24 04:58:01 2003 Received: from digitalsmarties.com (qmailr@gw.digital-smarties.com [203.59.125.206] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h2OAvxC21931 for ; Mon, 24 Mar 2003 04:57:59 -0600 Received: (qmail 30997 invoked from network); 24 Mar 2003 11:00:32 -0000 Received: from server.digital-smarties.com (192.168.1.2) by gateway.digitalsmarties.com (192.168.1.1) with ESMTP; 24 Mar 2003 11:00:32 -0000 Subject: Re: [Metakit] Tequila From: Steve Landers To: "Collins, Mike DAC DOIM" Cc: "metakit@equi4.com" In-Reply-To: <1048502501.2396.135.camel@server> References: <464A43842A2EF54F9183D07E0DEC43560102E194@rildoimml05.riley.army.mil> <1048502501.2396.135.camel@server> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.8 Date: 24 Mar 2003 18:55:31 +0800 Message-Id: <1048503331.2396.145.camel@server> Mime-Version: 1.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mike, > I've been playing around with Tequila as of late and I saw a blurp > somewhere about storing scripts on the Tequila Server. My "Kitview" paper from the Tcl2001 conference talks a bit about this. > If I were going to do something like that do I understand that it > would be just as any other variable in TCL and that you would just > store those variables in fields in a metakit datafile on the Server, > so to speak. Or is there something a little more exotic going on, > that I missed? Nope - you got it. Store in an element within the Tequila attached array, and just eval it when needed (ideally within a safe interpreter). Steve From gb@murphy.bofh.ms Mon Mar 24 12:08:59 2003 Received: from muenster.westfalen.de (mail@muenster.westfalen.de [62.153.201.130]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2OI8wC17132 for ; Mon, 24 Mar 2003 12:08:58 -0600 Received: from goggle.westfalen.de ([62.153.201.251] ident=mail) by muenster.westfalen.de with esmtp (Exim 3.35 #1 (Debian)) id 18xWPz-0003YH-00 for ; Mon, 24 Mar 2003 19:11:31 +0100 Received: from murphy.bofh.ms [10.0.0.123] by goggle.westfalen.de with esmtp (Exim 3.12 #1) id 18xWPj-0006m1-00 (Debian); Mon, 24 Mar 2003 19:11:15 +0100 Date: Mon, 24 Mar 2003 19:10:23 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Georg Bauer To: metakit@equi4.com Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.551) Subject: [Metakit] Bus error with metakit 2.4.9 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I am fiddeling around with blocked/ordered views in metakit with python. I do the following interactively: [murphy:~] gb% python Python 2.2.2 (#2, Jan 10 2003, 18:01:54) [GCC 2.95.2 19991024 (release)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import metakit >>> metakit.version '2.4.9.2' >>> db = metakit.storage('db.mk', 1) >>> words = db.getas('_B[liste[key:S,value:S]]').blocked().ordered(1) >>> words.append(key='gb', value='Georg') Bus error What is wrong with this? I thought this should create a blocked view with ordering by key. But it segfaults. Any pointers? I found a similar report from Januar by somebody else, but no reply to that problem. So maybe it's already known or I am doing something wrong, but I can't see what exactly. The problem with the blocked view stuff is, it is a bit terse in the documentation. A short example how to actually use it would help. bye, Georg (who otherwise really likes metakit - it's just that I will need to manage very large tables in future projects and would like to use ordered blocked views for that, as it seems to be the way to go for big tables) From gb@murphy.bofh.ms Mon Mar 24 12:18:34 2003 Received: from muenster.westfalen.de (mail@muenster.westfalen.de [62.153.201.130]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2OIIWC17834 for ; Mon, 24 Mar 2003 12:18:33 -0600 Received: from goggle.westfalen.de ([62.153.201.251] ident=mail) by muenster.westfalen.de with esmtp (Exim 3.35 #1 (Debian)) id 18xWZF-0003zW-00 for ; Mon, 24 Mar 2003 19:21:05 +0100 Received: from murphy.bofh.ms [10.0.0.123] by goggle.westfalen.de with esmtp (Exim 3.12 #1) id 18xWYy-0006pI-00 (Debian); Mon, 24 Mar 2003 19:20:48 +0100 Date: Mon, 24 Mar 2003 19:19:56 +0100 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: multipart/mixed; boundary=Apple-Mail-2--644163810 From: Georg Bauer To: metakit@equi4.com Message-Id: <374E4FFE-5E25-11D7-8E66-0050E4304D13@murphy.bofh.ms> X-Mailer: Apple Mail (2.551) Subject: [Metakit] patched Makefile.in for Mac OS X Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2--644163810 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hi! Attached is a Makefile.in that will be able to build your metakit on Mac OS X. One problem: Mac OS X doesn't come with python 2.2 development libraries, so usually you will need your own local installation of python2.2. The patch adds the necessary stuff to build a dynamic bundle, so Mk4py.so can be loaded into python. The problem: I am not really a professional makefile hacker, especially when it comes down to autoconf, so I don't know how to only patch this in when darwin is detected as platform. Maybe someone else on the list can do that? bye, Georg --Apple-Mail-2--644163810 Content-Disposition: attachment; filename=Makefile.in Content-Transfer-Encoding: 7bit Content-Type: application/octet-stream; x-unix-mode=0755; name="Makefile.in" # If this file has the name "Makefile.in" then it is a template for a # Makefile; to generate the actual Makefile, run "./configure", which # is a configuration script generated by the "autoconf" program. SHELL = /bin/sh LIBOBJS = column.o custom.o derived.o fileio.o field.o format.o \ handler.o persist.o remap.o std.o store.o string.o table.o \ univ.o view.o viewx.o SHLOBJS = column.lo custom.lo derived.lo fileio.lo field.lo format.lo \ handler.lo persist.lo remap.lo std.lo store.lo string.lo table.lo \ univ.lo view.lo viewx.lo PYOBJS = PyProperty.o PyRowRef.o PyStorage.o PyView.o PWOImp.o SPYOBJS = PyProperty.lo PyRowRef.lo PyStorage.lo PyView.lo PWOImp.lo TSTOBJS = regress.o tbasic1.o tbasic2.o tcusto1.o tcusto2.o tdiffer.o \ textend.o tformat.o tlimits.o tmapped.o tnotify.o tresize.o \ tstore1.o tstore2.o tstore3.o tstore4.o tstore5.o DEMOS = demo dump myio LINK_SPECIAL_FLAGS = @LINK_SPECIAL_FLAGS@ LINK_SPECIAL_FILES = @LINK_SPECIAL_FILES@ #---------- Autoconf settings, can be overriden as "make" args prefix = @prefix@ exec_prefix = @exec_prefix@ includedir = @includedir@ libdir = @libdir@ srcdir = @srcdir@ top_builddir = . python = python pyincludedir = @PY_INCLUDE_DIR@ pylibdir = @PY_LIB_DIR@ tclsh = tclsh tclincludedir = @TCL_INCLUDE_DIR@ tcllibdir = @TCL_LIB_DIR@ CPPFLAGS = @CPPFLAGS@ LDFLAGS = @LDFLAGS@ # Compiling without frame pointers can play tricks with exception handling # (e.g. in Mk4py). This does not affect standard operation, *only* errors. # Happens in gcc 2.95.3, it has been disabled for now (even though slower). #CXXFLAGS = -fomit-frame-pointer @CXXFLAGS@ @MK_THREADS@ CXXFLAGS = @CXXFLAGS@ @MK_THREADS@ #CXXFLAGS = -Dq4_CHECK @CXXFLAGS@ @MK_THREADS@ #CXXFLAGS = -Wall -pedantic -Wno-unused @CXXFLAGS@ @MK_THREADS@ CXX = @CXX@ L = @LIBTOOL@ INSTALL = @INSTALL@ INSTALL_DATA = $L --mode=install @INSTALL_DATA@ INSTALL_PROGRAM = $L --mode=install @INSTALL_PROGRAM@ LIBTOOL_SHLIB_FLAGS = $(LDFLAGS) @LIBTOOL_SHLIB_FLAGS@ LIBTOOL_MODULE_FLAGS = $(LDFLAGS) @LIBTOOL_MODULE_FLAGS@ STRIP_FLAGS = @STRIP_FLAGS@ DESTDIR = #---------- Do not change, shorthand only CXX_SWITCHES = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(srcdir)/../src -I. CXX_SWITCHES_TCL = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(tclincludedir)/generic \ -I$(tclincludedir) CXX_SWITCHES_PY = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include \ -I$(srcdir)/../python/scxx \ -I$(pyincludedir) CXX_SWITCHES_LUA = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include CXX_SWITCHES_TEST = $(CPPFLAGS) $(CXXFLAGS) \ -I$(srcdir)/../include #---------- The targets normally specified when calling "make" all: @MK_TARGETS@ core: Makefile libmk4.la $(DEMOS) tcl: Makefile Mk4tcl@SHLIB_SUFFIX@ python: Makefile Mk4py@MODULE_SUFFIX@ test: Makefile libmk4.la regress test -d tests || mkdir tests -test -d ../tests/ok/CVS && (test -d tests/CVS || mkdir tests/CVS) test -f reversed || cp $(srcdir)/reversed . ./regress diff $(srcdir)/../tests/ok tests test-tcl: tcl cd $(srcdir)/../tcl/test && $(tclsh) all.tcl test-python: python $(python) $(srcdir)/../python/test/all.py install: @MK_INSTALL@ install-mk: libmk4.la mkdir -p $(DESTDIR)$(includedir) $(DESTDIR)$(libdir) $(INSTALL_DATA) $(srcdir)/../include/mk4.h \ $(srcdir)/../include/mk4.inl \ $(srcdir)/../include/mk4str.h \ $(srcdir)/../include/mk4str.inl $(DESTDIR)$(includedir) $(INSTALL_PROGRAM) libmk4.la $(DESTDIR)$(libdir) install-tcl: Mk4tcl@SHLIB_SUFFIX@ mkdir -p $(DESTDIR)$(tcllibdir)/Mk4tcl $(INSTALL_PROGRAM) Mk4tcl@SHLIB_SUFFIX@ $(DESTDIR)$(tcllibdir)/Mk4tcl echo 'package ifneeded Mk4tcl 2.4.8 [list load [file join $$dir Mk4tcl@SHLIB_SUFFIX@] Mk4tcl]' >$(DESTDIR)$(tcllibdir)/Mk4tcl/pkgIndex.tcl install-python: Mk4py@MODULE_SUFFIX@ $(INSTALL_PROGRAM) Mk4py@MODULE_SUFFIX@ $(DESTDIR)$(pylibdir) $(INSTALL_PROGRAM) $(srcdir)/../python/metakit.py $(DESTDIR)$(pylibdir) clean: $L rm -f *.la *.o *.lo rm -f $(DEMOS) struct regress myfile.dat secret.dat rm -rf tests/[a-z]* .libs distclean: clean rm -f Makefile libtool config.h config.status config.log config.cache #---------- This defines what each of the targets does Makefile: $(srcdir)/Makefile.in config.status $(SHELL) ./config.status config.status: $(srcdir)/configure $(SHELL) ./config.status --recheck $(srcdir)/configure: $(srcdir)/configure.in cd $(srcdir) && autoconf libmk4.la: $(LIBOBJS) $(LINK_SPECIAL_FILES) $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_SHLIB_FLAGS) -avoid-version \ -rpath $(libdir) $(SHLOBJS) $(LINK_SPECIAL_FLAGS) Mk4tcl@SHLIB_SUFFIX@: libmk4tcl.la cp .libs/libmk4tcl@SHLIB_SUFFIX@ $@ -strip $(STRIP_FLAGS) $@ 2>/dev/null libmk4tcl.la: mk4tcl.o mk4too.o $(LIBOBJS) $(LINK_SPECIAL_FILES) $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_SHLIB_FLAGS) -avoid-version \ -rpath $(libdir) mk4tcl.lo mk4too.lo $(SHLOBJS) $(LINK_SPECIAL_FLAGS) Mk4py@MODULE_SUFFIX@: libmk4py@MODULE_SUFFIX@ cp libmk4py@MODULE_SUFFIX@ $@ -strip $(STRIP_FLAGS) $@ 2>/dev/null libmk4py@MODULE_SUFFIX@: libmk4py.la $(CC) -Wl,-F. -Wl,-flat_namespace,-U,_environ -bundle \ -bundle_loader ${prefix}/bin/python2.2 \ $(SPYOBJS) $(SHLOBJS) \ ${prefix}/lib/python2.2/config/libpython2.2.a \ -lstdc++ -o $@ libmk4py.la: $(PYOBJS) $(LIBOBJS) $L --mode=link $(CXX) -o $@ $(CXX_SWITCHES) $(LIBTOOL_MODULE_FLAGS) -avoid-version \ -rpath $(libdir) $(SPYOBJS) $(SHLOBJS) Mk4lua@SHLIB_SUFFIX@: mk4lua.o libmk4.la $(CXX) -o $@ $(CXX_SWITCHES) -shared mk4lua.lo $(SHLOBJS) \ -L$(libdir) -llua -llualib demo: $(srcdir)/../demos/demo.cpp libmk4.la $L --mode=link $(CXX) $(CXX_SWITCHES) -static -o $@@EXEEXT@ \ $(srcdir)/../demos/demo.cpp libmk4.la dump: $(srcdir)/../demos/dump.cpp libmk4.la $L --mode=link $(CXX) $(CXX_SWITCHES) -static -o $@@EXEEXT@ \ $(srcdir)/../demos/dump.cpp libmk4.la myio: $(srcdir)/../demos/myio.cpp libmk4.la $L --mode=link $(CXX) $(CXX_SWITCHES) -o $@@EXEEXT@ \ $(srcdir)/../demos/myio.cpp libmk4.la struct: $(srcdir)/../demos/struct.cpp $L --mode=link $(CXX) $(CXX_SWITCHES) -o $@@EXEEXT@ \ $(srcdir)/../demos/struct.cpp -lmk4 regress: $(TSTOBJS) libmk4.la $L --mode=link $(CXX) -static -o $@@EXEEXT@ $(TSTOBJS) libmk4.la #---------- Dependencies # Hack. # Configuration: HPUX, CXX=aCC, CC=cc, cpu PA-RISC # => Have to link this object file to satisfy symbols. # cpprt0_stubs.o: $(srcdir)/cpprt0_stubs.s as -o $@ $? mk4tcl.o: $(srcdir)/../tcl/mk4tcl.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES_TCL) $? mk4too.o: $(srcdir)/../tcl/mk4too.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES_TCL) $? PyProperty.o: $(srcdir)/../python/PyProperty.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) $? PyRowRef.o: $(srcdir)/../python/PyRowRef.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) $? PyStorage.o: $(srcdir)/../python/PyStorage.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) $? PyView.o: $(srcdir)/../python/PyView.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) $? PWOImp.o: $(srcdir)/../python/scxx/PWOImp.cpp \ $(srcdir)/../python/scxx/PWOBase.h \ $(srcdir)/../python/scxx/PWOCallable.h \ $(srcdir)/../python/scxx/PWOMSequence.h \ $(srcdir)/../python/scxx/PWOMapping.h \ $(srcdir)/../python/scxx/PWONumber.h \ $(srcdir)/../python/scxx/PWOSequence.h $L --mode=compile $(CXX) -c $(CXX_SWITCHES_PY) \ $(srcdir)/../python/scxx/PWOImp.cpp column.o: $(srcdir)/../src/column.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? custom.o: $(srcdir)/../src/custom.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? derived.o: $(srcdir)/../src/derived.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? field.o: $(srcdir)/../src/field.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? fileio.o: $(srcdir)/../src/fileio.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? format.o: $(srcdir)/../src/format.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? handler.o: $(srcdir)/../src/handler.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? persist.o: $(srcdir)/../src/persist.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? remap.o: $(srcdir)/../src/remap.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? std.o: $(srcdir)/../src/std.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? store.o: $(srcdir)/../src/store.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? string.o: $(srcdir)/../src/string.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? table.o: $(srcdir)/../src/table.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? univ.o: $(srcdir)/../src/univ.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? view.o: $(srcdir)/../src/view.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? viewx.o: $(srcdir)/../src/viewx.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES) $? mk4lua.o: $(srcdir)/../lua/mk4lua.cpp $L --mode=compile $(CXX) -c $(CXX_SWITCHES_LUA) $? regress.o: $(srcdir)/../tests/regress.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tbasic1.o: $(srcdir)/../tests/tbasic1.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tbasic2.o: $(srcdir)/../tests/tbasic2.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tcusto1.o: $(srcdir)/../tests/tcusto1.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tcusto2.o: $(srcdir)/../tests/tcusto2.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tdiffer.o: $(srcdir)/../tests/tdiffer.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ textend.o: $(srcdir)/../tests/textend.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tformat.o: $(srcdir)/../tests/tformat.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tlimits.o: $(srcdir)/../tests/tlimits.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tmapped.o: $(srcdir)/../tests/tmapped.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tnotify.o: $(srcdir)/../tests/tnotify.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tresize.o: $(srcdir)/../tests/tresize.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tstore1.o: $(srcdir)/../tests/tstore1.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tstore2.o: $(srcdir)/../tests/tstore2.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tstore3.o: $(srcdir)/../tests/tstore3.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tstore4.o: $(srcdir)/../tests/tstore4.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ tstore5.o: $(srcdir)/../tests/tstore5.cpp $(CXX) -c $(CXX_SWITCHES_TEST) $? -o $@ #---------- End --Apple-Mail-2--644163810-- From niki@vintech.bg Wed Mar 26 06:22:00 2003 Received: from intra.vintech.l (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2QCLxC06972 for ; Wed, 26 Mar 2003 06:21:59 -0600 Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id 0BA2257136 for ; Wed, 26 Mar 2003 14:31:24 +0200 (EET) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id 037F922C39 for ; Wed, 26 Mar 2003 14:24:25 +0200 (EET) Message-ID: <3E819B42.5030104@vintech.bg> Date: Wed, 26 Mar 2003 14:21:22 +0200 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.3b) Gecko/20030210 X-Accept-Language: bg, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] mk4py 2.4.9.2 error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This code does not work and there is no exception: import Mk4py; print Mk4py.version class test: def __init__( self, c, p, t ): self.c = c self.p = p self.t = t st = Mk4py.storage( 'mk2492.tst', 1 ) v = st.getas('demo[c:I,p:D,t:s]') v.append( test(77, 3.14, 'boza') ) print v[0].c, v[0].p ----------- 2.4.9.2 0 0.0 expected result: 77 3.14 Niki Spahiev From niki@vintech.bg Wed Mar 26 09:59:43 2003 Received: from intra.vintech.l (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2QFxgC25187 for ; Wed, 26 Mar 2003 09:59:42 -0600 Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id BF99F57136 for ; Wed, 26 Mar 2003 18:09:14 +0200 (EET) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id 6F39222C39 for ; Wed, 26 Mar 2003 18:02:14 +0200 (EET) Message-ID: <3E81CE4F.7070505@vintech.bg> Date: Wed, 26 Mar 2003 17:59:11 +0200 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.3b) Gecko/20030210 X-Accept-Language: bg, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] mk4py 2.4.9.2 error References: <3E819B42.5030104@vintech.bg> In-Reply-To: <3E819B42.5030104@vintech.bg> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Apparently PySequence_Check(o) (in PyView::makeRow) is true for instances. Replacing it with PyList_Check(o) solves this problem. Byt why there is no error? Defaults are used instead. Niki Spahiev From gb@murphy.bofh.ms Thu Mar 27 06:02:47 2003 Received: from murphy.bofh.ms (gb@pD950C38F.dip.t-dialin.net [217.80.195.143]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2RC2kC18994 for ; Thu, 27 Mar 2003 06:02:46 -0600 Received: from murphy.bofh.ms (localhost [127.0.0.1]) by murphy.bofh.ms (8.12.7/8.12.2) with ESMTP id h2RBxQMK002622 for ; Thu, 27 Mar 2003 12:59:26 +0100 (CET) Received: (from gb@localhost) by murphy.bofh.ms (8.12.7/8.12.2/Submit) id h2RBxQMF002621 for metakit@equi4.com; Thu, 27 Mar 2003 12:59:26 +0100 (CET) Date: Thu, 27 Mar 2003 12:59:26 +0100 (CET) From: Georg Bauer Message-Id: <200303271159.h2RBxQMF002621@murphy.bofh.ms> To: metakit@equi4.com Subject: [Metakit] More on the segfault/buserror with blocked+ordered views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I started it under Linux, this time. Same metakit version 2.4.9.2. I used gdb to get a bit more infos: gb@sdok-light:~/.PyDS/tools$ gdb python2.2 GNU gdb 2002-04-01-cvs Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-linux"...(no debugging symbols found)... (gdb) run Starting program: /usr/bin/python2.2 (no debugging symbols found)...(no debugging symbols found)...[New Thread 1024 (LWP 7436)] (no debugging symbols found)...(no debugging symbols found)... Python 2.2.1 (#1, Sep 7 2002, 14:34:30) [GCC 2.95.4 20011002 (Debian prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. (no debugging symbols found)...(no debugging symbols found)... (no debugging symbols found)...(no debugging symbols found)... >>> import metakit (no debugging symbols found)...(no debugging symbols found)... db>>> db = metakit.storage() >>> table = db.getas('B[zeile[key:S,value:S]]') >>> btable = table.blocked() >>> stable = btable.ordered(1) >>> stable.append(key='blah',value='blubb') (no debugging symbols found)...(no debugging symbols found)... Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1024 (LWP 7436)] 0x402278ae in c4_HandlerSeq::NthHandler () from /usr/lib/python2.2/site-packages/Mk4py.so Does this help to pin down the problem? Any hints on what I could do to help find this one? It's a show-stopper for me, as I need blocked and ordered views because of the expected volume of records in an application I am playing around currently with. bye, Georg From jcw@equi4.com Thu Mar 27 06:08:58 2003 Received: from equi4.com (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2RC8vC19327 for ; Thu, 27 Mar 2003 06:08:57 -0600 Date: Thu, 27 Mar 2003 13:11:32 +0100 Subject: Re: [Metakit] More on the segfault/buserror with blocked+ordered views Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <200303271159.h2RBxQMF002621@murphy.bofh.ms> Message-Id: <3F9A6648-604D-11D7-B52B-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Georg Bauer wrote: > I started it under Linux, this time. Same metakit version 2.4.9.2. I > used > gdb to get a bit more infos: > > gb@sdok-light:~/.PyDS/tools$ gdb python2.2 [...] >>>> table = db.getas('B[zeile[key:S,value:S]]') >>>> btable = table.blocked() >>>> stable = btable.ordered(1) >>>> stable.append(key='blah',value='blubb') > (no debugging symbols found)...(no debugging symbols found)... > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 1024 (LWP 7436)] > 0x402278ae in c4_HandlerSeq::NthHandler () > from /usr/lib/python2.2/site-packages/Mk4py.so Ouch, I should have spotted this sooner. You need to give blocked() a view with a subview called "_B": table = db.getas('zeile[_B[key:S,value:S]]') -jcw From gb@murphy.bofh.ms Thu Mar 27 08:43:44 2003 Received: from murphy.bofh.ms (gb@pD950C38F.dip.t-dialin.net [217.80.195.143]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h2REhhC27941 for ; Thu, 27 Mar 2003 08:43:43 -0600 Received: from murphy.bofh.ms (localhost [127.0.0.1]) by murphy.bofh.ms (8.12.7/8.12.2) with ESMTP id h2REeOMK002760 for ; Thu, 27 Mar 2003 15:40:24 +0100 (CET) Received: (from gb@localhost) by murphy.bofh.ms (8.12.7/8.12.2/Submit) id h2REeO4h002759 for metakit@equi4.com; Thu, 27 Mar 2003 15:40:24 +0100 (CET) Date: Thu, 27 Mar 2003 15:40:24 +0100 (CET) From: Georg Bauer Message-Id: <200303271440.h2REeO4h002759@murphy.bofh.ms> To: metakit@equi4.com Subject: [Metakit] question regarding aside files Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I am playing around with aside commits under python. Is the following code the right way to use aside storage? import metakit mainstorage = metakit.storage('test.data', 1) asidestorage = metakit.storage('test.aside', 1) mainstorage.aside(asidestorage) .... do your processing and adding lots of records .... mainstorage.commit(1) When I do the above, I get a problem: it works, but memory consumption grows exorbitantly. Looks like the aside storage file isn't used but everything is kept in memory (don't really know what is kept in memory, but the consumption is equivalent in size roughly). And after the full commit the memory doesn't shrink, it's still growing (I wove in several full commits every 100 record inserts). When I leave out the full commits and the aside storage, everything works fine although slowly (starts with 20 record adds in the first 200 records, drops down over time to about 2 records per second after roundabout 2000 records, I didn't try beyond that, because usually at that point with aside storage it consumes about 250 MB memory). Any hints? bye, Georg PS: the tip with the blocked storage did work. I just can't reply, as I am not near my main machine and have to rely on bsd mail tool and don't have access to the incoming mail ;-) ) From Michael_Scharf@gmx.de Wed Apr 2 10:47:41 2003 Received: from mail.gmx.net (pop.gmx.net [213.165.65.60]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h32GlfN08889 for ; Wed, 2 Apr 2003 10:47:41 -0600 Received: (qmail 27484 invoked by uid 65534); 2 Apr 2003 16:47:40 -0000 Received: from p50843A29.dip.t-dialin.net (EHLO gmx.de) (80.132.58.41) by mail.gmx.net (mp006-rz3) with SMTP; 02 Apr 2003 18:47:40 +0200 Message-ID: <3E8B13CA.3090001@gmx.de> Date: Wed, 02 Apr 2003 18:46:02 +0200 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit java bindings? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I switched my project from delphi/python to java. And I still beleive that MetaKit is the best database for my application. Now I wonder if anybody has written some JNI code to access MetaKit from java. If not, I might write some bindings.... Michael From jcw@equi4.com Wed Apr 2 11:28:46 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h32HSjN11948 for ; Wed, 2 Apr 2003 11:28:45 -0600 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 744C283BA7; Wed, 2 Apr 2003 12:28:43 -0500 (EST) Date: Wed, 2 Apr 2003 19:28:42 +0200 Subject: Re: [Metakit] MetaKit java bindings? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) Cc: Christian Tismer To: Metakit List From: Jean-Claude Wippler In-Reply-To: <3E8B13CA.3090001@gmx.de> Message-Id: <8CFCB214-6530-11D7-A473-000A957664F8@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.551) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Michael Scharf wrote: > I switched my project from delphi/python to java. And I still beleive > that MetaKit is the best database for my application. Now I wonder > if anybody has written some JNI code to access MetaKit from java. Whee... a trans-lingual MK afeccionado! :) It depends on what you want to do. An experiment which was done about a year ago (by Christian Tismer), was to generate a *fixed* binding, given an existing database structure. Changes to the schema means you have to regenerate and recompile the wrapper. The reason to mention this, is that the binding is C, not C++ (though inside, is of course still a C++ core for now). That means SWIG should have no trouble at all wrapping it to various languages, including Java. This was in fact one reason for doing it. As far as I can remember it was definitely functional code, though not exposing most of the MK view operators, just basic access/modify functionality for views and rows. The project was shelved, to await better focus and actual need. In case you're interested - it's all available in a CVS project on equi4.com (follow same checkout instructions as metakit, but use "metable" as module name). It would be fantastic if you can make such a binding work for Java, in some form or other (metable is just food for thought, there are of course many more ways to go about this). It also matches my conviction that data storage has longer lifetimes than languages, i.e. over time I expect an increasing interest to bind to a second language (as teams and projects evolve). It's been some time since I mentioned it, so let me re-iterate that I continue to be interested in making more language bindings happen, also to Perl, Ruby, etc. My main hurdle is not knowing enough about each of the respective languages to be able to do things in a "natural" way myself. But I'll definitely do my best to help where I can. -jcw From tismer@tismer.com Wed Apr 2 11:49:24 2003 Received: from centera.de (mail@[213.133.98.72]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h32HnON13551; Wed, 2 Apr 2003 11:49:24 -0600 Received: from dsl-217-199-78-176.berlikomm.net ([217.199.78.176] helo=tismer.com) by centera.de with asmtp (Exim 3.36 #1 (Debian)) id 190lYk-0002pY-00; Wed, 02 Apr 2003 18:57:58 +0200 Message-ID: <3E8B22A7.40306@tismer.com> Date: Wed, 02 Apr 2003 19:49:27 +0200 From: Christian Tismer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.2) Gecko/20021126 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Metakit List Subject: Re: [Metakit] MetaKit java bindings? References: <8CFCB214-6530-11D7-A473-000A957664F8@equi4.com> In-Reply-To: <8CFCB214-6530-11D7-A473-000A957664F8@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Michael Scharf wrote: > >> I switched my project from delphi/python to java. And I still beleive >> that MetaKit is the best database for my application. Now I wonder >> if anybody has written some JNI code to access MetaKit from java. Aber Michael, was machst Du denn für Sachen! TssTss... > Whee... a trans-lingual MK afeccionado! :) > > It depends on what you want to do. An experiment which was done about a > year ago (by Christian Tismer), was to generate a *fixed* binding, given > an existing database structure. Changes to the schema means you have to > regenerate and recompile the wrapper. The reason to mention this, is > that the binding is C, not C++ (though inside, is of course still a C++ > core for now). That means SWIG should have no trouble at all wrapping > it to various languages, including Java. This was in fact one reason > for doing it. > > As far as I can remember it was definitely functional code, though not > exposing most of the MK view operators, just basic access/modify > functionality for views and rows. The project was shelved, to await > better focus and actual need. In case you're interested - it's all > available in a CVS project on equi4.com (follow same checkout > instructions as metakit, but use "metable" as module name). Yeah, I think it was in a usable state, already. With a fully automated generator that creates all the bindings for an existing MK database structure. Well, the generator *is* in Python, and it uses Metakit to do the introspection. I'd be happy to see something happening with this stuff. I think it was quite good. Probably, a much nicer thing could be written for C++, since I'm faking something like classes by static structs, but anyway, the SWIGability is a nice feature which I wasn't aware of! Let me know if you need help. cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From Michael_Scharf@gmx.de Wed Apr 2 12:21:59 2003 Received: from mail.gmx.net (mail.gmx.net [213.165.65.60]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h32ILxN16145 for ; Wed, 2 Apr 2003 12:21:59 -0600 Received: (qmail 17677 invoked by uid 65534); 2 Apr 2003 18:21:58 -0000 Received: from p50843A29.dip.t-dialin.net (EHLO gmx.de) (80.132.58.41) by mail.gmx.net (mp011-rz3) with SMTP; 02 Apr 2003 20:21:58 +0200 Message-ID: <3E8B29E5.4060809@gmx.de> Date: Wed, 02 Apr 2003 20:20:21 +0200 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit List Subject: Re: [Metakit] MetaKit java bindings? References: <8CFCB214-6530-11D7-A473-000A957664F8@equi4.com> <3E8B22A7.40306@tismer.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Christian Tismer wrote: > Aber Michael, was machst Du denn für Sachen! TssTss... Und du hast auch deine finger ueberall drin ;-) > Yeah, I think it was in a usable state, already. With a fully > automated generator that creates all the bindings for an existing > MK database structure. > Well, the generator *is* in Python, and it uses Metakit to do > the introspection. Cool, I'll take look at it. > I'd be happy to see something happening with this stuff. > I think it was quite good. > Probably, a much nicer thing could be written for C++, since > I'm faking something like classes by static structs, but > anyway, the SWIGability is a nice feature which I wasn't aware of! I am not a big fan of SWIG, but it might have changed in the last years. I have written my own generator (in python, what else), where it is pretty staight forward to customize the generated code (unsing an old version of Bobo (zope) DTML)... Michael From Michael_Scharf@gmx.de Wed Apr 2 12:26:54 2003 Received: from mail.gmx.net (imap.gmx.net [213.165.65.60]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h32IQsN16515 for ; Wed, 2 Apr 2003 12:26:54 -0600 Received: (qmail 5524 invoked by uid 65534); 2 Apr 2003 18:26:53 -0000 Received: from p50843A29.dip.t-dialin.net (EHLO gmx.de) (80.132.58.41) by mail.gmx.net (mp009-rz3) with SMTP; 02 Apr 2003 20:26:53 +0200 Message-ID: <3E8B283A.3020509@gmx.de> Date: Wed, 02 Apr 2003 20:13:14 +0200 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 CC: Metakit List Subject: Re: [Metakit] MetaKit java bindings? References: <8CFCB214-6530-11D7-A473-000A957664F8@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > It depends on what you want to do. An experiment which was done about a > year ago (by Christian Tismer), was to generate a *fixed* binding, given > an existing database structure. :-D I know Christian he's a good developer :-D > As far as I can remember it was definitely functional code, though not > exposing most of the MK view operators, just basic access/modify > functionality for views and rows. The project was shelved, to await > better focus and actual need. In case you're interested - it's all > available in a CVS project on equi4.com (follow same checkout > instructions as metakit, but use "metable" as module name). Ok, I'll take a look at it... > It would be fantastic if you can make such a binding work for Java, in > some form or other (metable is just food for thought, there are of > course many more ways to go about this). It also matches my conviction > that data storage has longer lifetimes than languages, i.e. over time I > expect an increasing interest to bind to a second language (as teams and > projects evolve). I have been looking around and I could not find anything comparable to metakit for java. Something similar might exists but I could not find it. With jdk1.4 there exists a new low leve IO library nio (http://java.sun.com/j2se/1.4.1/docs/api/java/nio/package-summary.html) which allows access to memory mapped files, but I have not yet found a database that makes use of it.... > It's been some time since I mentioned it, so let me re-iterate that I > continue to be interested in making more language bindings happen, also > to Perl, Ruby, etc. My main hurdle is not knowing enough about each of > the respective languages to be able to do things in a "natural" way > myself. But I'll definitely do my best to help where I can. That's good. If I'll do it, I'd like to add some unit tests to the system. I very much beleive in unittests and writing test cases helps a lot in understanding the system :-) Michael From ian.fairclough@intelligent-ms.com Tue Apr 15 06:44:02 2003 Received: from scratchy.base.ims ([194.74.0.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3FBi1N06847 for ; Tue, 15 Apr 2003 06:44:02 -0500 Received: from grpware.intelligent-ms.com (IDENT:root@[192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id h3FBnUA29103 for ; Tue, 15 Apr 2003 12:49:30 +0100 Received: from imsgsx6tvznbng ([192.168.200.181]) by grpware.intelligent-ms.com (8.11.2/8.11.2) with SMTP id h3FBVTv02819 for ; Tue, 15 Apr 2003 12:31:29 +0100 Message-ID: <02d301c30344$4d902e60$242cfc3e@imsgsx6tvznbng> From: "Ian Fairclough" To: Date: Tue, 15 Apr 2003 12:43:17 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_02D0_01C3034C.96E936D0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4920.2300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4920.2300 Subject: [Metakit] Use of Select Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_02D0_01C3034C.96E936D0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, Forgive me if I appear dumb ! I am struggling to ascertain how I can implement some more complex = select statements to obtain a sub-set of rows from a view. For example, I would like to : select where "name 1" NOT EQUAL "value 1" select where "name 1" EQUAL "value 1" AND "name 2" EQUAL "value 2" AND = "name 3" NOT EQUAL "value 3" select where "name 1" EQUAL "value 1" OR "name 2" EQUAL "value 2" select where ("name 1" EQUAL "value 1" AND "name 2" EQUAL "value 2") OR = "name 3" EQUAL "value3" etc etc. Is it easy to implement these kinds of select statements using metakit ? = If so, can you point me at some code examples (or give me a clue what = the code should look like). Thanks very much in advance for any information you can provide. Best regards, Ian Fairclough. ------=_NextPart_000_02D0_01C3034C.96E936D0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi,
 
Forgive me if I appear dumb = !
 
I am struggling to ascertain how I can = implement=20 some more complex select statements to obtain a sub-set of rows from a=20 view.
 
For example, I would like to = :
 
select where "name 1" NOT EQUAL "value=20 1"
select where "name 1" EQUAL "value 1" = AND "name 2"=20 EQUAL "value 2" AND "name 3" NOT EQUAL "value 3"
select where "name 1" EQUAL "value 1" = OR "name 2"=20 EQUAL "value 2"
select where ("name 1" EQUAL "value 1" = AND "name 2"=20 EQUAL "value 2") OR "name 3" EQUAL "value3"
 
etc etc.
 
Is it easy to implement these kinds of = select=20 statements using metakit ?  If so, can you point me at some code = examples=20 (or give me a clue what the code should look like).
 
Thanks very much in advance for any = information you=20 can provide.
 
Best regards,
Ian = Fairclough.
------=_NextPart_000_02D0_01C3034C.96E936D0-- From tom.krehbiel@motorola.com Tue Apr 15 09:05:53 2003 Received: from motgate2.mot.com (motgate2.mot.com [136.182.1.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3FE5rN16536 for ; Tue, 15 Apr 2003 09:05:53 -0500 Received: from az33exr01.mot.com (az33exr01.mot.com [10.64.251.231]) by motgate2.mot.com (Motorola/Motgate2) with ESMTP id h3FE67d9001747 for ; Tue, 15 Apr 2003 07:06:07 -0700 (MST) Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exr01.mot.com (Motorola/az33exr01) with ESMTP id h3FE65Oq006283 for ; Tue, 15 Apr 2003 09:06:05 -0500 Message-ID: <3E9C11CD.67E0FB25@motorola.com> Date: Tue, 15 Apr 2003 07:06:05 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DDL X-Mailer: Mozilla 4.78 [en] (X11; U; HP-UX B.11.00 9000/785) X-Accept-Language: en MIME-Version: 1.0 CC: metakit@equi4.com Subject: Re: [Metakit] Use of Select References: <02d301c30344$4d902e60$242cfc3e@imsgsx6tvznbng> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ian, From the tcl bindings man page. EXAMPLES Select a range of entries: foreach i [mk::select db.phonebook -min date 19980101 -max date 19980131] { puts "Dated Jan 1998: [mk::get db.phonebook!$i name]" } Search for a unique match ('-count 2' speeds up selection when many entries match): set v [mk::select db.phonebook -count 2 -glob name "John*"] switch [llength $v] { 0 {puts "not found"} 1 {puts "found: [mk::get db.phonebook![lindex $v 0] name]"} 2 {puts "there is more than one entry matching 'John*'"} } Sort by descending date and by ascending name: foreach i [mk::select db.phonebook -rsort date -sort name] { puts "Change log: [mk::get db.phonebook!$i date name]" } Tom K. > > Hi, > > Forgive me if I appear dumb ! > > I am struggling to ascertain how I can implement some more complex select statements to obtain a sub-set of rows from a view. > > For example, I would like to : > > select where "name 1" NOT EQUAL "value 1" > select where "name 1" EQUAL "value 1" AND "name 2" EQUAL "value 2" AND "name 3" NOT EQUAL "value 3" > select where "name 1" EQUAL "value 1" OR "name 2" EQUAL "value 2" > select where ("name 1" EQUAL "value 1" AND "name 2" EQUAL "value 2") OR "name 3" EQUAL "value3" > > etc etc. > > Is it easy to implement these kinds of select statements using metakit ? If so, can you point me at some code examples (or give me a clue what the code should look like). > > Thanks very much in advance for any information you can provide. > > Best regards, > Ian Fairclough. From marek@tranglos.com Thu Apr 17 19:22:49 2003 Received: from sack.dreamhost.com (postfix@sack.dreamhost.com [66.33.213.6]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3I0MnN23223 for ; Thu, 17 Apr 2003 19:22:49 -0500 Received: from 81.190.8.88 (8-88.lodz.mm.pl [81.190.8.88]) by sack.dreamhost.com (Postfix) with ESMTP id E8E4313E247 for ; Thu, 17 Apr 2003 17:23:03 -0700 (PDT) Date: Fri, 18 Apr 2003 02:23:17 +0200 From: Marek Jedlinski X-Mailer: The Bat! (v1.62 Christmas Edition) Educational Reply-To: Marek Jedlinski Organization: Tranglos X-Priority: 3 (Normal) Message-ID: <12639423117.20030418022317@tranglos.com> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit from a Delphi application? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: (first-time post; I've googled and searched the online docs and mailing list archives) I've been looking long for a flexible storage system for my open-source app, written in Delphi. I've now found MetaKit and it looks very much like something I could use, but I find no mention of Delphi/Object Pascal bindings. Has anyone done this before? Or how would I go about translating the API? (Fair disclosure: I only write code for fun; fun includes Delphi and Perl, but never extended as far as C++ :) I do have some experience with Win32 API, so I realize what's involved, in general, but just by looking at MetaKit source, I can't figure out where to start.) And a more general question: if I were using MK from Delphi, is MK suitable for use as a generic storage mechanism? That is, my app is not database-backed, and I need no database-specific functionality, just an ability to store and load small and med-sized chunks of data dynamically, e.g. via streams or raw bufders. Currently, my app is using a flat file and loads it whole into memory, which is becoming a major hassle and a bottleneck in one. Just to give an example of what I've been looking for - I've considered using OLE compound files, but they're hairy and too tightly bound to Windows for my liking, while I'd like to maintain at least a semblance of portability to Kylix. Thanks a lot in advance, Marek Jedlinski -- No ads, no nags freeware: http://www.tranglos.com (KeyNote, PhoneDeck, KookieJar, Oubliette) "Well I can get the 'a'. But how do I put the circle around it?" From lvirden@cas.org Wed Apr 23 11:07:29 2003 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3NG7SN17451 for ; Wed, 23 Apr 2003 11:07:29 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.9/8.12.9/CAS_MAIL_HUB-2.00) with ESMTP id h3NG7gKw021098 for ; Wed, 23 Apr 2003 12:07:43 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id h3NG7gT14379; Wed, 23 Apr 2003 12:07:42 -0400 (EDT) Date: Wed, 23 Apr 2003 12:07:42 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20030423120742.AAB14345@cas.org> To: metakit@equi4.com Subject: [Metakit] ISO: a general metakit database browser Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Do metakit databases generally contain enough self-referential data that one could write an application that could be pointed to any metakit database so that it could display the contents of that database in a fashion useful for learning about the database? Has anyone made an attempt to do that sort of thing? I've seen applications before that could be pointed to oracle databases, or odbc databases, and provide the user a gui browser to see metadata about table and column names, row contents, relationships, etc. I'm wondering whether metakit databases are similarly self contained, or if some of the metadata is contained in the applications which read and/or write the databases. -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jtate@mi-corporation.com Wed Apr 23 11:12:08 2003 Received: from mail2.mi-corporation.com ([209.116.71.23]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3NGC8N17898 for ; Wed, 23 Apr 2003 11:12:08 -0500 Received: from jtatework (unknown [204.85.2.183]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mail2.mi-corporation.com (Postfix) with SMTP id 13ED45623E; Wed, 23 Apr 2003 12:12:35 -0400 (EDT) From: "Joseph Tate" To: "Larry W. Virden" , Subject: RE: [Metakit] ISO: a general metakit database browser Date: Wed, 23 Apr 2003 12:12:28 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 In-Reply-To: <20030423120742.AAB14345@cas.org> Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Take a look at KitViewer. > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of Larry W. Virden > Sent: Wednesday, April 23, 2003 12:08 PM > To: metakit@equi4.com > Subject: [Metakit] ISO: a general metakit database browser > > > Do metakit databases generally contain enough self-referential data that > one could write an application that could be pointed to any > metakit database > so that it could display the contents of that database in a fashion useful > for learning about the database? > > Has anyone made an attempt to do that sort of thing? > > I've seen applications before that could be pointed to oracle databases, > or odbc databases, and provide the user a gui browser to see > metadata about > table and column names, row contents, relationships, etc. > > I'm wondering whether metakit databases are similarly self contained, or > if some of the metadata is contained in the applications which read and/or > write the databases. > -- > Tcl - The glue of a new generation. > Larry W. Virden http://www.purl.org/NET/lvirden/> > Even if explicitly stated to the contrary, nothing in this posting should > be construed as representing my employer's opinions. > -><- > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From lvirden@cas.org Wed Apr 23 11:15:37 2003 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3NGFaN18242 for ; Wed, 23 Apr 2003 11:15:36 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.9/8.12.9/CAS_MAIL_HUB-2.00) with ESMTP id h3NGFlKw024487 for ; Wed, 23 Apr 2003 12:15:49 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id h3NGFkk14630; Wed, 23 Apr 2003 12:15:46 -0400 (EDT) Date: Wed, 23 Apr 2003 12:15:46 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20030423121545.AAB14608@cas.org> Subject: Re: [Metakit] ISO: a general metakit database browser In-Reply-To: Your message of Wed, 23 Apr 2003 12:12:28 -0400 To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Has anyone tried to write something like kitviewer in a way that it would compile and run on other platforms - either in python, tcl, perl, or at least tried to modify the code so that it compiles on other platforms? I seldom have access to windows platforms, so it's hard to check it out. -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jtate@mi-corporation.com Wed Apr 23 11:37:23 2003 Received: from mail2.mi-corporation.com ([209.116.71.23]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3NGbNN20121 for ; Wed, 23 Apr 2003 11:37:23 -0500 Received: from jtatework (unknown [204.85.2.183]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mail2.mi-corporation.com (Postfix) with SMTP id 880505623E; Wed, 23 Apr 2003 12:37:50 -0400 (EDT) From: "Joseph Tate" To: "Larry W. Virden" , Subject: RE: [Metakit] ISO: a general metakit database browser Date: Wed, 23 Apr 2003 12:37:44 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 In-Reply-To: <20030423121545.AAB14608@cas.org> Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: You should be able to look at the code at least. See how it's done. It can open any arbitrary db file, so the question that you asked in your previous e-mail has an affirmative answer. > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of Larry W. Virden > Sent: Wednesday, April 23, 2003 12:16 PM > To: metakit@equi4.com > Subject: Re: [Metakit] ISO: a general metakit database browser > > > Has anyone tried to write something like kitviewer in a way that it > would compile and run on other platforms - either in python, tcl, perl, > or at least tried to modify the code so that it compiles on other > platforms? > > I seldom have access to windows platforms, so it's hard to check it out. > -- > Tcl - The glue of a new generation. > Larry W. Virden http://www.purl.org/NET/lvirden/> > Even if explicitly stated to the contrary, nothing in this posting should > be construed as representing my employer's opinions. > -><- > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From lvirden@cas.org Wed Apr 23 12:02:01 2003 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3NH20N22187 for ; Wed, 23 Apr 2003 12:02:00 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.9/8.12.9/CAS_MAIL_HUB-2.00) with ESMTP id h3NH2FKw010021 for ; Wed, 23 Apr 2003 13:02:15 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id h3NH2EX15533; Wed, 23 Apr 2003 13:02:14 -0400 (EDT) Date: Wed, 23 Apr 2003 13:02:14 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20030423130214.AAB15524@cas.org> Subject: Re: [Metakit] ISO: a general metakit database browser In-Reply-To: Your message of Wed, 23 Apr 2003 12:37:44 -0400 To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Right you are. I apologize. -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From 1jerry@usd1.com Wed Apr 23 12:55:19 2003 Received: from bfish.hurrah.com (root@bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3NHtJN26967; Wed, 23 Apr 2003 12:55:19 -0500 Received: from CTL (12-211-33-9.client.attbi.com [12.211.33.9]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id h3NHteX06277; Wed, 23 Apr 2003 10:55:40 -0700 Date: Wed, 23 Apr 2003 10:50:47 -0700 From: Jerry McRae <1jerry@usd1.com> X-Mailer: The Bat! (v1.60q) Personal Organization: usd1 X-Priority: 3 (Normal) Message-ID: <16587595276.20030423105047@usd1.com> To: metakit-admin@equi4.com, "Larry W. Virden" CC: metakit@equi4.com Subject: Re[2]: [Metakit] ISO: a general metakit database browser OT In-Reply-To: <20030423130214.AAB15524@cas.org> References: <20030423130214.AAB15524@cas.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi metakit-admin, My response to what you wrote me on Wednesday, April 23, 2003 at 10:02 AM: Off Topic. Your e-mail program seems to create headers outside the semi-"standard" format. The result is that your replies do not end up threaded with the original messages (in mail readers that can do this). This would make it much easier (for me, anyway) to track subjects. There is also no X-Mailer in the header that identifies the program. If this is your own program, can you please fill in the In-Reply-To: in the header with the original message ID (not subject)? If not, then....never mind. -- /\_/\ Jerry McRae \|||/ mail safely: The Bat! 1.60q (~o o~) 1(360)-574-4444 (. .) mailto:1jerry@usd1.com )'Y'( _______________-o00-(_)-00o-___________________________ ( ) / He that's secure is not safe. --Benjamin Franklin ========== your original message below ========== From: Larry W. Virden To: metakit@equi4.com on: 10:02:14 AM re: [Metakit] ISO: a general metakit database browser wise words: LWV> Right you are. I apologize. From jcw@equi4.com Wed Apr 23 14:23:39 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3NJNcN02330 for ; Wed, 23 Apr 2003 14:23:38 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id A6AF383BE7 for ; Wed, 23 Apr 2003 15:23:52 -0400 (EDT) Date: Wed, 23 Apr 2003 21:23:54 +0200 Subject: Re: [Metakit] ISO: a general metakit database browser Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20030423120742.AAB14345@cas.org> Message-Id: <1F3BB7DD-75C1-11D7-8907-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: > Do metakit databases generally contain enough self-referential data > that > one could write an application that could be pointed to any metakit > database > so that it could display the contents of that database in a fashion > useful > for learning about the database? Enough to decode the data? Yes. Enough to present a meaningful textual dump? Depends. There are two sub-commands in sdx (all Tcl) which let you view the structure, and dump contents: "sdx mkinfo" and "sdx mkshow" (both are command-line based). The info MK stores is sufficient to fully access all data as strings, numbers, subview structures, etc. What it does not store is some additional info such as whether an int is really used as a date or as a currency, whether a B-type field holds text or binary data, and such. Another way to put it is that only a limited datatype info is tracked in the datafile (int, string, float, and such). Because that's all the MK core cares about. > Has anyone made an attempt to do that sort of thing? As was pointed out, "KitViewer" is a Win-only app which displays any MK datafile. It was written in Borland C++ Builder, because that contains a very effective "data aware grid", which can display data matrices of any size very very efficiently. It'd be great if something more portable could be written. This has been brought up by quite a few people before. Let me just say that both the Python and the Tcl API's let you get at the structure and type of all data items - so technically this is most definitely feasible from a scripted language. -jcw From bkelley@wi.mit.edu Thu Apr 24 09:53:22 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h3OErLN07700; Thu, 24 Apr 2003 09:53:21 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.0.0.44) with SMTP id M2003042410533517496 ; Thu, 24 Apr 2003 10:53:35 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HDUS1B00.B5O; Thu, 24 Apr 2003 10:53:35 -0400 Message-ID: <3EA7FAC7.3050509@wi.mit.edu> Date: Thu, 24 Apr 2003 10:55:03 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com CC: Jean-Claude Wippler Subject: Re: [Metakit] ISO: a general metakit database browser References: <1F3BB7DD-75C1-11D7-8907-000A957664F8@equi4.com> In-Reply-To: <1F3BB7DD-75C1-11D7-8907-000A957664F8@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > > It'd be great if something more portable could be written. This has > been brought up by quite a few people before. Let me just say that > both the Python and the Tcl API's let you get at the structure and > type of all data items - so technically this is most definitely > feasible from a scripted language. > > -jcw > Ask and you shall receive: requires metakit, Python 2.2 http://www.python.org/, wxPython 2.3+ http://www.wxpython.org/ A windows binary is located here http://staffa.wi.mit.edu/people/kelley/ features: view all tables and subtables in a metakit storage object. Handles large amounts of data easily. Only 150 lines of code (!) Minuses: wxPython has a huge overhead (10MB) so this is not as small as the original kitviewer. ToDo: export to flat/file excel, html (for printing purposes) Can't alter data currently (easy to add though) -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From bkelley@wi.mit.edu Thu Apr 24 09:55:39 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h3OEtcN07913; Thu, 24 Apr 2003 09:55:38 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.0.0.44) with SMTP id M2003042410560005519 ; Thu, 24 Apr 2003 10:56:00 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HDUS5C00.V70; Thu, 24 Apr 2003 10:56:00 -0400 Message-ID: <3EA7FB58.5040705@wi.mit.edu> Date: Thu, 24 Apr 2003 10:57:28 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Subject: Re: [Metakit] ISO: a general metakit database browser References: <1F3BB7DD-75C1-11D7-8907-000A957664F8@equi4.com> In-Reply-To: <1F3BB7DD-75C1-11D7-8907-000A957664F8@equi4.com> Content-Type: multipart/mixed; boundary="------------080904030703080503060605" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------080904030703080503060605 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit This time I'll actually add the source :) You can also download from http://staffa.wi.mit.edu/people/kelley/ -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 --------------080904030703080503060605 Content-Type: text/plain; name="KitViewer.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="KitViewer.py" ##################################################################### ## Simple MetaKit Table decomposer ##################################################################### import metakit, sys, os from wxPython.wx import * from wxPython.grid import * try: import cStringIO as StringIO except ImportError: import StringIO class T: def __init__(self, db): """A wrapper around a metakit database. T.tables = list of available tables T["tablename"] -> returns a table wrapper""" self.db = db self.tables = [x.name for x in db.contents().structure()] def __getitem__(self, table): return Table(self.db.view(table)) class Table: def __init__(self, view): """Wrapper around a metakit table. Table.props -> list of metakit props Table.headers -> list of column names Table.rows -> number of rows Table.cols -> number of cols Table[row, col] -> data at table position i,j print Table -> dump the current table to text """ self.view = view self.props = view.structure() self.headers = [p.name for p in self.props] self.rows = len(view) self.cols = len(self.props) def __len__(self): return len(self.view) def __getitem__(self, key): row, col = key _row = self.view[row] name = self.headers[col] prop = getattr(self.view, name) data = getattr(_row, name) if prop.type == "V": return Table(data) else: return data def __str__(self): stdout = sys.stdout sys.stdout = StringIO.StringIO() metakit.dump(self.view) text = sys.stdout.getvalue() sys.stdout = stdout return text ####################################################################### ## wxPython viewing code using the above class KitTable(wxPyGridTableBase): """ A custom wxGrid Table using a metakit view """ def __init__(self, table): wxPyGridTableBase.__init__(self) self.table = table def GetNumberCols(self): return self.table.cols def GetNumberRows(self): return self.table.rows def GetColLabelValue(self, col): return self.table.headers[col] def GetValue(self, row, col): data = self.table[row, col] if type(data) == types.InstanceType and data.__class__.__name__ == "Table": return "SUBTABLE" return str( self.table[row, col] ) def SetValue(self, row, col, value): raise NotImplementedError class KitGrid(wxGrid): def __init__(self, parent, table, sbar=None): wxGrid.__init__(self, parent, -1) self.table = table ktable = KitTable(table) self.SetTable(ktable, true) attr = wxGridCellAttr() attr.SetBackgroundColour(wxCYAN) # set all of the table columns to a different background col = 0 for prop in self.table.props: if prop.type == "V": self.SetColAttr(col, attr) col += 1 EVT_GRID_CELL_LEFT_DCLICK(self, self.OnCellLeftDClick) if sbar: sbar.SetStatusText("Double Click subtables to expand") def OnCellLeftDClick(self,evt): col = evt.GetCol() row = evt.GetRow() if self.table.props[col].type == "V": subtable = self.table[row, col] frame=KitViewerSingle(self, subtable, name="SUBTABLE %s"%self.table.headers[col]) frame.Show() evt.Skip() class KitViewerSingle(wxFrame): def __init__(self, parent, table, name="KitViewer"): wxFrame.__init__(self, parent, -1, name, size=(640,480)) self.CreateStatusBar() grid = KitGrid(self, table, sbar=self) class KitViewer(wxFrame): def __init__(self, parent, db=None, name="KitViewer"): wxFrame.__init__(self, parent, -1, name, size=(640,480)) self.nb = wxNotebook(self, -1) if db: self.db = T(db) self._reload() else: self.db = None self.mainmenu = wxMenuBar() menu = wxMenu() self.mainmenu.Append(menu, '&File') exitID = wxNewId() openID = wxNewId() menu.Append(openID, '&Open\tCtrl-O', 'Open a project to analyze') menu.Append(exitID, 'E&xit\tAlt-X', 'Get the heck outta here!') EVT_MENU(self, openID, self.OnOpen) EVT_MENU(self, exitID, self.OnExit) self.SetMenuBar(self.mainmenu) self.CreateStatusBar() self.SetStatusText("Double click subtables to expand") def _reload(self): nb = self.nb nb.DeleteAllPages() for key in self.db.tables: grid = KitGrid(nb, self.db[key], sbar=self) nb.AddPage(grid, key) def OnExit(self, evt): self.Close() def OnOpen(self, evt): fdlg = wxFileDialog(self, "Choose a file", ".", "", "*.*", wxOPEN) if fdlg.ShowModal() == wxID_OK: path = fdlg.GetPaths()[0] if os.path.exists(path): try: db = metakit.storage(path, 0) self.db = T(db) self._reload() except: dlg = wxMessageDialog(frame, 'Selected File does not appear to be a metakit database.', 'A Message Box', wxOK | wxICON_INFORMATION) dlg.ShowModal() dlg.Destroy() raise else: dlg = wxMessageDialog(frame, 'File does not exist!', 'A Message Box', wxOK | wxICON_INFORMATION) dlg.ShowModal() dlg.Destroy() fdlg.Destroy() if __name__ == "__main__": app = wxPySimpleApp() frame = KitViewer(None, None) frame.Show(true) app.MainLoop() --------------080904030703080503060605-- From jcw@equi4.com Thu Apr 24 10:55:07 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3OFt7N13316 for ; Thu, 24 Apr 2003 10:55:07 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 76CE183987 for ; Thu, 24 Apr 2003 11:55:22 -0400 (EDT) Date: Thu, 24 Apr 2003 17:55:23 +0200 Subject: Re: [Metakit] ISO: a general metakit database browser Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3EA7FAC7.3050509@wi.mit.edu> Message-Id: <28543C59-766D-11D7-8179-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: [wishful thinking...] > Ask and you shall receive: > requires metakit, Python 2.2 http://www.python.org/, wxPython 2.3+ > http://www.wxpython.org/ > A windows binary is located here > http://staffa.wi.mit.edu/people/kelley/ > > features: > view all tables and subtables in a metakit storage object. Handles > large amounts of data easily. > Only 150 lines of code (!) > Minuses: > wxPython has a huge overhead (10MB) so this is not as small as the > original kitviewer. > ToDo: > export to flat/file excel, html (for printing purposes) > Can't alter data currently (easy to add though) Wow, this is cool. I've taken the liberty to add a link to it on the MK homepage (under "Other links"). It's unfortunate that the support code is so big, and that a few different pieces need to be installed to make this work on various Unices (a recurring theme cq. pet peeve of mine), but still - this is going to help a *lot* IMO. Thanks for sharing! -jcw From jeff_godfrey@pobox.com Thu Apr 24 11:07:12 2003 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h3OG7AN14498 for ; Thu, 24 Apr 2003 11:07:10 -0500 Received: (qmail 9444 invoked from network); 24 Apr 2003 16:07:30 -0000 Received: from wbrg-dialup-216-74-200-123.iland.net (HELO quantex) (216.74.200.123) by chewey.iland.net with SMTP; 24 Apr 2003 16:07:30 -0000 Message-ID: <015f01c30a7b$54256d60$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Metakit mailing list" References: <28543C59-766D-11D7-8179-000A957664F8@equi4.com> Subject: Re: [Metakit] ISO: a general metakit database browser Date: Thu, 24 Apr 2003 11:05:24 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 4/17/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Jean-Claude Wippler" To: "Metakit mailing list" Sent: Thursday, April 24, 2003 10:55 AM Subject: Re: [Metakit] ISO: a general metakit database browser Here is some tcl code that I sent privately to Larry in response to his original enquiry. It's very much unfinished (just a hack, really), but does allow the viewing, modification, and deletion of data in a general metakit database. The "add record" button is not yet functional. It is pure tcl, but does require the BWidget and Tablelist extensions (also pure tcl) as well as the mk4tcl library. Also, sorry if this is "larger" than I should post directly to the mailing list but I don't really have an online place for it currently... Jeff ========== Code follows ============ package require Mk4tcl package require BWidget package require Tablelist package require Tk proc main {} { #console show getImages buildUI } proc dbOpen {} { set file [tk_getOpenFile] if {![file exists $file]} { return } # --- close any currently open db's foreach {tag oldFile} [mk::file open] { mk::file close $tag } mk::file open db $file set ::db(file) $file set ::db(views) [mk::file views db] foreach view $::db(views) { set ::db(field,$view) [mk::view info db.$view] } populateTree } proc buildUI {} { # Menu description set menuDesc { "&File" all file 0 { {command "&Open..." {} "Open database" {} -command dbOpen} {separator} {command "E&xit" {} "Exit" {} -command exit} } } set mf [MainFrame .mf \ -menu $menuDesc \ -separator top \ -textvariable ::indicator(help)] $mf addindicator -width 12 -textvar ::indicator(status) $mf showstatusbar status set uf [$mf getframe] set tb1 [$mf addtoolbar] set bbox1 [ButtonBox $tb1.bbox1 -spacing 0 -padx 1 -pady 1] $bbox1 add -image $::image(open) \ -highlightthickness 0 -takefocus 0 -relief link -borderwidth 1 -padx 1 -pady 1 \ -helptext "Open a MetaKit database file" -command dbOpen pack $bbox1 -side left -anchor w set sep [Separator $tb1.sep -orient vertical] pack $sep -side left -fill y -padx 4 -anchor w set bbox2 [ButtonBox $tb1.bbox2 -spacing 0 -padx 1 -pady 1] $bbox2 add -image $::image(refresh) \ -highlightthickness 0 -takefocus 0 -relief link -borderwidth 1 -padx 1 -pady 1 \ -helptext "Refresh view" $bbox2 add -image $::image(add) \ -highlightthickness 0 -takefocus 0 -relief link -borderwidth 1 -padx 1 -pady 1 \ -helptext "Add record" $bbox2 add -image $::image(delete) \ -highlightthickness 0 -takefocus 0 -relief link -borderwidth 1 -padx 1 -pady 1 \ -helptext "Delete record(s)" -command dbDelete pack $bbox2 -side left -anchor w set swTree [ScrolledWindow $uf.swTree -auto both -scrollbar both] set swTable [ScrolledWindow $uf.swTable -auto both -scrollbar both] set tree [Tree $swTree.tree \ -padx 16 \ -redraw true \ -selectcommand nodeClicked \ -width 15] set table [tablelist::tablelist $swTable.table \ -bg white \ -width 100 \ -height 22 \ -stretch all \ -selectmode extended \ -labelcommand {tablelist::sortByColumn}] set ::widgets(tree) $tree set ::widgets(table) $table set body [$table bodypath] bind $body {selectRecord %W} $swTree setwidget $tree $swTable setwidget $table pack $swTree -fill y -side left pack $swTable -expand 1 -fill both -side left pack $mf -expand 1 -fill both } proc selectRecord {body} { set tree $::widgets(tree) set thisNode [$tree selection get] set parent [$tree parent $thisNode] if {[string equal $parent "root"]} { set view $thisNode } else { set view $parent } set parent [winfo parent $body] set selectedRow [$parent curselection] if {[llength $selectedRow] == 1} { set recID [lindex [$parent get $selectedRow] 0] editRecord $view $recID $selectedRow } } proc dbDelete {} { set tree $::widgets(tree) set thisNode [$tree selection get] if {![string length $thisNode]} { return } set parent [$tree parent $thisNode] if {[string equal $parent "root"]} { set view $thisNode } else { set view $parent } set table $::widgets(table) set recordList [$table curselection] set recCount [llength $recordList] if {!$recCount} { set msg "No records selected" set title "Delete" tk_messageBox \ -message $msg \ -title $title \ -icon info return } if {$recCount == 1} { set type "record" } else { set type "records" } set msg "Permanently delete $recCount $type?" set title "Delete Warning" set resp [tk_messageBox \ -message $msg \ -title $title \ -icon question \ -type yesno] if {[string equal $resp "no"]} { return } # --- Delete the selected records from both the tablelist and the db. # Note, they are deleted from the bottom up as the database re-assigns # the record ID's with each deletion foreach tableID $recordList { lappend dbIDList [lindex [$table get $tableID] 0] } foreach dbID [lsort -integer -decreasing $dbIDList] { mk::cursor create thisCursor $view $dbID mk::row delete $thisCursor } $table selection clear 0 end $tree selection set $thisNode } proc editRecord {view recID tableRow} { mk::cursor create thisCursor $view $recID set record [mk::get $thisCursor] array set ::thisData $record set dialog [Dialog .dialog \ -title "Edit Record in view \"$view\"" \ -anchor e] set uf [$dialog getframe] set count 0 foreach {header data} $record { incr count set label [label $uf.l$count -text "${header}:" -anchor e] set entry [entry $uf.e$count -textvar ::thisData($header) -width 30] grid $label $entry -padx 2 -pady 2 -sticky ew } grid columnconfigure $uf 1 -weight 1 $dialog add -text "Submit" -command [list $dialog enddialog "submit"] $dialog add -text "Cancel" -command [list $dialog enddialog "Cancel"] set result [$dialog draw] if {[string equal $result "submit"]} { # --- put the data in the db eval mk::set $thisCursor [array get ::thisData] # --- update the tablelist with the new data from the db set table $::widgets(table) set record [mk::get $thisCursor] # --- get the record number (for col #1) from the current db cursor val set thisData [lindex [split $thisCursor "!"] 1] foreach {header data} $record { lappend thisData $data } $table rowconfigure $tableRow -text $thisData } destroy $dialog catch {unset ::thisData} } proc nodeClicked {tree node} { set parent [$tree parent $node] if {[string equal $parent "root"]} { set view $node } else { set view $parent } loadTable $view } proc loadTable {view {filter *}} { set table $::widgets(table) $table configure -cursor watch after 200 update # --- get all columns from the requested view set colList [mk::view info $view] set colList [linsert $colList 0 "rec_no"] # --- set up each column's width, text, and justification foreach col $colList { lappend cols 20 $col left } # --- delete the existing tablelist data... $table delete 0 end # --- configure the tablelist with the column info $table configure -columns $cols # --- set all columns but the "rec_no" to stretch upon resize... for {set i 1} {$i < [llength $colList]} {incr i} { lappend stretchList $i } $table configure -stretch $stretchList # --- get *all* data from the selected database view set finalList [list] foreach rowID [mk::select $view $filter] { mk::cursor create thisCursor $view $rowID set temp $rowID foreach {col val} [mk::get $thisCursor] { lappend temp $val } lappend finalList $temp #$table insert end $temp } eval $table insert end $finalList set ::indicator(status) "[$table size] records" $table configure -cursor {} update } proc populateTree {} { set tree $::widgets(tree) # --- delete all existing nodes... catch {$tree delete [$tree nodes root]} # --- Load the database descripiton into the tree... # Each view is denoted by a top level node, and each column within the # view becomes a child node of the view node. foreach view $::db(views) { $tree insert end "root" db.$view -text $view -selectable 1 -image [Bitmap::get file] foreach field $::db(field,$view) { $tree insert end db.$view $view.$field -text $field -image [Bitmap::get oplink] } } # --- automatically select the first view node... $tree selection set [lindex [$tree nodes "root"] 0] } proc getImages {} { set ::image(open) [image create photo -data { R0lGODlhFgAWAIYAAPwCBAQCBCQWDCwaDDwmFPSubPzGhPzCfPy2dOSmZPzK lPzSnPzOlPzKjBQODPzChPzWnPy2bPSmXPyuZOyeXIRSLEQuFEwyHEQqFDQi FCweDKRuPFRSTPT29PTy9Ozq7OTi3Nze3NTW1MzOzMTGxMTCxLy6tLSytKyu rDQyNMzKxOTm5OTi5Nza1NTS1MTCvLS2tLSyrKSmpJyenJSWlIyKjHx+fFxe XBwaHKxuPMzKzLy6vIyOjHx6fDw6NPy6dGxubLy+vISChCQmJNza3KyqrBQS FLR2RKSinJyanGxqZAwGBJSSlCwqLAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAA LAAAAAAWABYAAAf/gACCg4SFhoeDAYqKiIeLj4wBjQCMhY+NkoiLk5qbhQID oJyGBAUGBwgEo4MECQoLDA2pDrS1tKQPEAwHERITE77AvxKqhAQNDA8UFRYX Fs8YBAQZGqGPxw0RGxwdHR4eHyAhIiMkJSYnKCgpBAYPEhcqHyssLS4kLzAx MjM0NTY3cBA4UCAHBw8gVnhgEcKFjhc7UPDj0cMHAAI/KFgY4YLFio/jRpTY sW8GDyCSCEQw2DChOHIqgsCQSEPIEEEEJFhAoUNECCJEyOk4d6KIyRtGcB7h IJKjixEjHu4oimSGEIs4d8IIUoKECnNB0ElMwkNJJgBLlJBAcQKGiR07KGAU RVGViY0mhIwwSTKjr99+THjUoIg0r48hTRIrRtxkiOMhDgrZCQQAIf5oQ3Jl YXRlZCBieSBCTVBUb0dJRiBQcm8gdmVyc2lvbiAyLjUNCqkgRGV2ZWxDb3Ig MTk5NywxOTk4LiBBbGwgcmlnaHRzIHJlc2VydmVkLg0KaHR0cDovL3d3dy5k ZXZlbGNvci5jb20AOw==}] set ::image(refresh) [image create photo -data { R0lGODlhFgAWAIYAAPwCBCRSFCRSHBw+DBxCFCQ6FBwyDBQWBBxGFCxyLGTC hMzqzLzmvHzKjDyOTER+RERyNDSqXNzy3LzivFS+fCyCPBQmBCQiBBxKFBQq DOTy3LTitES2dDR+PCxuJOT25KTarCx+PESSTCxKHDSeVCyKRNT21ESWVDSG PBQyDAQCBBQSFDRuLDSyZDySTGzChCRiJKSmpExKTDS2ZGzGhLy+vGxqbISC hDSKRMzKzGxubDQ2NIyOjCQiJCwqLBQWFCwuLKSipERCRERGRHR2dAwKDDw6 PFRWVIyKjCQmJFRSVBwaHKyurAQGBExOTBweHFxeXAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAA LAAAAAAWABYAAAf9gACCg4IBAgMEBQYHhI2ECAkKCwwNDg8QBAOOhAQREhMU FQgWBxcHGBmbggkaGxwPB4yDB6SbBJIKHQaqtY0eHyAhsqrDgx4aCiKpqoQH AyMjJBMKJaSxzAAHIRsmJgonKA0LHSmDKiuOBywRLSQuLyEwwyoxMuiN6iUz NBXy5jU2bsgoJugABBz95uXQsUMGD3vpPPgTpKIGwx4+HMr4kW4YkCA2hAzx AQSIECI+imBTwVIFESNHerRUgc0cEiFHkjiiyYzeDiVLdvLcySSkkKGEWiZV weSGkIHMmvQosoQlkaZOjvhosvKJjIAxoOAsgpRZkQNLnvSoqspAIAAh/mhD cmVhdGVkIGJ5IEJNUFRvR0lGIFBybyB2ZXJzaW9uIDIuNQ0KqSBEZXZlbENv ciAxOTk3LDE5OTguIEFsbCByaWdodHMgcmVzZXJ2ZWQuDQpodHRwOi8vd3d3 LmRldmVsY29yLmNvbQA7}] set ::image(add) [image create photo -data { R0lGODlhEAAQAIQAAPwCBARCZPz+/Mzq9BTC3ITC1HSyzGSivAw+XBSStIS6 zHy2zAQCDFyatAQOHFSStEyOtAQSJBSq1DR2nCxunCRmlAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAAAQABAAAAVbICCOQTme aCAMRIC+AVu47xkTBl2Ld16XQNYBQTQBVIOkMHFQJBeMI4tAbSSu2IRDSnhA vFfI97sFRM6RwOMacbjLKckVvgvIJ5EdSU7J648VgXQ7Dmd/hyJ+IQAh/mhD cmVhdGVkIGJ5IEJNUFRvR0lGIFBybyB2ZXJzaW9uIDIuNQ0KqSBEZXZlbENv ciAxOTk3LDE5OTguIEFsbCByaWdodHMgcmVzZXJ2ZWQuDQpodHRwOi8vd3d3 LmRldmVsY29yLmNvbQA7}] set ::image(delete) [image create photo -data { R0lGODlhEAAQAIMAAPwCBARCZPz+/Mzq9BTC3AQCDAQOHFSStAQSJAAAAAAA AAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAAAQABAAAAQwEMhJq704681n +GAISoEwnGg6EAUQrEQsz4ThEkeu77kNIAagMEj0dY7IpHI58UcAACH+aENy ZWF0ZWQgYnkgQk1QVG9HSUYgUHJvIHZlcnNpb24gMi41DQqpIERldmVsQ29y IDE5OTcsMTk5OC4gQWxsIHJpZ2h0cyByZXNlcnZlZC4NCmh0dHA6Ly93d3cu ZGV2ZWxjb3IuY29tADs=}] } main From jeff_godfrey@pobox.com Thu Apr 24 11:26:06 2003 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h3OGQ6N16499 for ; Thu, 24 Apr 2003 11:26:06 -0500 Received: (qmail 23678 invoked from network); 24 Apr 2003 16:26:26 -0000 Received: from wbrg-dialup-216-74-200-123.iland.net (HELO quantex) (216.74.200.123) by chewey.iland.net with SMTP; 24 Apr 2003 16:26:26 -0000 Message-ID: <01a001c30a7d$f69ed520$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Jeff Godfrey" , "Metakit mailing list" References: <28543C59-766D-11D7-8179-000A957664F8@equi4.com> <015f01c30a7b$54256d60$0101a8c0@quantex> Subject: Re: [Metakit] ISO: a general metakit database browser Date: Thu, 24 Apr 2003 11:24:15 -0500 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_019D_01C30A54.09DB3F40" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 4/17/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_019D_01C30A54.09DB3F40 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit ----- Original Message ----- From: "Jeff Godfrey" To: "Metakit mailing list" Sent: Thursday, April 24, 2003 11:05 AM Subject: Re: [Metakit] ISO: a general metakit database browser > ----- Original Message ----- > From: "Jean-Claude Wippler" > To: "Metakit mailing list" > Sent: Thursday, April 24, 2003 10:55 AM > Subject: Re: [Metakit] ISO: a general metakit database browser > > Here is some tcl code that I sent privately to Larry in response to his > original enquiry. It's very much unfinished (just a hack, really), but does > allow the viewing, modification, and deletion of data in a general metakit > database. The "add record" button is not yet functional. It is pure tcl, > but does require the BWidget and Tablelist extensions (also pure tcl) as > well as the mk4tcl library. Ooooh, that's ugly... I'm going to try again as an attachment... Sorry for the inbox clutter... ------=_NextPart_000_019D_01C30A54.09DB3F40 Content-Type: application/octet-stream; name="db_manager.tcl" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="db_manager.tcl" package require Mk4tcl package require BWidget package require Tablelist package require Tk proc main {} { #console show getImages buildUI } proc dbOpen {} { set file [tk_getOpenFile] if {![file exists $file]} { return } # --- close any currently open db's foreach {tag oldFile} [mk::file open] { mk::file close $tag } mk::file open db $file set ::db(file) $file set ::db(views) [mk::file views db] foreach view $::db(views) { set ::db(field,$view) [mk::view info db.$view] } populateTree } proc buildUI {} { # Menu description set menuDesc { "&File" all file 0 { {command "&Open..." {} "Open database" {} -command dbOpen} {separator} {command "E&xit" {} "Exit" {} -command exit} } } set mf [MainFrame .mf \ -menu $menuDesc \ -separator top \ -textvariable ::indicator(help)] $mf addindicator -width 12 -textvar ::indicator(status) $mf showstatusbar status set uf [$mf getframe] set tb1 [$mf addtoolbar] set bbox1 [ButtonBox $tb1.bbox1 -spacing 0 -padx 1 -pady 1] $bbox1 add -image $::image(open) \ -highlightthickness 0 -takefocus 0 -relief link -borderwidth 1 = -padx 1 -pady 1 \ -helptext "Open a MetaKit database file" -command dbOpen pack $bbox1 -side left -anchor w set sep [Separator $tb1.sep -orient vertical] pack $sep -side left -fill y -padx 4 -anchor w set bbox2 [ButtonBox $tb1.bbox2 -spacing 0 -padx 1 -pady 1] $bbox2 add -image $::image(refresh) \ -highlightthickness 0 -takefocus 0 -relief link -borderwidth 1 = -padx 1 -pady 1 \ -helptext "Refresh view" $bbox2 add -image $::image(add) \ -highlightthickness 0 -takefocus 0 -relief link -borderwidth 1 = -padx 1 -pady 1 \ -helptext "Add record" $bbox2 add -image $::image(delete) \ -highlightthickness 0 -takefocus 0 -relief link -borderwidth 1 = -padx 1 -pady 1 \ -helptext "Delete record(s)" -command dbDelete pack $bbox2 -side left -anchor w set swTree [ScrolledWindow $uf.swTree -auto both -scrollbar both] set swTable [ScrolledWindow $uf.swTable -auto both -scrollbar both] set tree [Tree $swTree.tree \ -padx 16 \ -redraw true \ -selectcommand nodeClicked \ -width 15] set table [tablelist::tablelist $swTable.table \ -bg white \ -width 100 \ -height 22 \ -stretch all \ -selectmode extended \ -labelcommand {tablelist::sortByColumn}] set ::widgets(tree) $tree set ::widgets(table) $table set body [$table bodypath] bind $body {selectRecord %W} $swTree setwidget $tree $swTable setwidget $table pack $swTree -fill y -side left pack $swTable -expand 1 -fill both -side left pack $mf -expand 1 -fill both } proc selectRecord {body} { set tree $::widgets(tree) set thisNode [$tree selection get] set parent [$tree parent $thisNode] if {[string equal $parent "root"]} { set view $thisNode } else { set view $parent } set parent [winfo parent $body] set selectedRow [$parent curselection] if {[llength $selectedRow] =3D=3D 1} { set recID [lindex [$parent get $selectedRow] 0] editRecord $view $recID $selectedRow } } proc dbDelete {} { set tree $::widgets(tree) set thisNode [$tree selection get] if {![string length $thisNode]} { return } set parent [$tree parent $thisNode] if {[string equal $parent "root"]} { set view $thisNode } else { set view $parent } set table $::widgets(table) set recordList [$table curselection] set recCount [llength $recordList] if {!$recCount} { set msg "No records selected" set title "Delete" tk_messageBox \ -message $msg \ -title $title \ -icon info return } if {$recCount =3D=3D 1} { set type "record" } else { set type "records" } set msg "Permanently delete $recCount $type?" set title "Delete Warning" set resp [tk_messageBox \ -message $msg \ -title $title \ -icon question \ -type yesno] if {[string equal $resp "no"]} { return } # --- Delete the selected records from both the tablelist and the = db. # Note, they are deleted from the bottom up as the database = re-assigns # the record ID's with each deletion foreach tableID $recordList { lappend dbIDList [lindex [$table get $tableID] 0] } foreach dbID [lsort -integer -decreasing $dbIDList] { mk::cursor create thisCursor $view $dbID mk::row delete $thisCursor } $table selection clear 0 end $tree selection set $thisNode } proc editRecord {view recID tableRow} { mk::cursor create thisCursor $view $recID set record [mk::get $thisCursor] array set ::thisData $record set dialog [Dialog .dialog \ -title "Edit Record in view \"$view\"" \ -anchor e] set uf [$dialog getframe] set count 0 foreach {header data} $record { incr count set label [label $uf.l$count -text "${header}:" -anchor e] set entry [entry $uf.e$count -textvar ::thisData($header) -width = 30] grid $label $entry -padx 2 -pady 2 -sticky ew } grid columnconfigure $uf 1 -weight 1 $dialog add -text "Submit" -command [list $dialog enddialog = "submit"] $dialog add -text "Cancel" -command [list $dialog enddialog = "Cancel"] set result [$dialog draw] if {[string equal $result "submit"]} { # --- put the data in the db eval mk::set $thisCursor [array get ::thisData] # --- update the tablelist with the new data from the db set table $::widgets(table) set record [mk::get $thisCursor] # --- get the record number (for col #1) from the current db = cursor val set thisData [lindex [split $thisCursor "!"] 1] foreach {header data} $record { lappend thisData $data } $table rowconfigure $tableRow -text $thisData } destroy $dialog catch {unset ::thisData} } proc nodeClicked {tree node} { set parent [$tree parent $node] if {[string equal $parent "root"]} { set view $node } else { set view $parent } loadTable $view } proc loadTable {view {filter *}} { set table $::widgets(table) $table configure -cursor watch after 200 update # --- get all columns from the requested view set colList [mk::view info $view] set colList [linsert $colList 0 "rec_no"] # --- set up each column's width, text, and justification foreach col $colList { lappend cols 20 $col left } # --- delete the existing tablelist data... $table delete 0 end # --- configure the tablelist with the column info $table configure -columns $cols # --- set all columns but the "rec_no" to stretch upon resize... for {set i 1} {$i < [llength $colList]} {incr i} { lappend stretchList $i } $table configure -stretch $stretchList # --- get *all* data from the selected database view set finalList [list] foreach rowID [mk::select $view $filter] { mk::cursor create thisCursor $view $rowID set temp $rowID foreach {col val} [mk::get $thisCursor] { lappend temp $val } lappend finalList $temp #$table insert end $temp } eval $table insert end $finalList set ::indicator(status) "[$table size] records" $table configure -cursor {} update } proc populateTree {} { set tree $::widgets(tree) # --- delete all existing nodes... catch {$tree delete [$tree nodes root]} # --- Load the database descripiton into the tree... # Each view is denoted by a top level node, and each column = within the # view becomes a child node of the view node. foreach view $::db(views) { $tree insert end "root" db.$view -text $view -selectable 1 = -image [Bitmap::get file] foreach field $::db(field,$view) { $tree insert end db.$view $view.$field -text $field -image = [Bitmap::get oplink] } } # --- automatically select the first view node... $tree selection set [lindex [$tree nodes "root"] 0] } proc getImages {} { set ::image(open) [image create photo -data { R0lGODlhFgAWAIYAAPwCBAQCBCQWDCwaDDwmFPSubPzGhPzCfPy2dOSmZPzK lPzSnPzOlPzKjBQODPzChPzWnPy2bPSmXPyuZOyeXIRSLEQuFEwyHEQqFDQi FCweDKRuPFRSTPT29PTy9Ozq7OTi3Nze3NTW1MzOzMTGxMTCxLy6tLSytKyu rDQyNMzKxOTm5OTi5Nza1NTS1MTCvLS2tLSyrKSmpJyenJSWlIyKjHx+fFxe XBwaHKxuPMzKzLy6vIyOjHx6fDw6NPy6dGxubLy+vISChCQmJNza3KyqrBQS FLR2RKSinJyanGxqZAwGBJSSlCwqLAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAA LAAAAAAWABYAAAf/gACCg4SFhoeDAYqKiIeLj4wBjQCMhY+NkoiLk5qbhQID oJyGBAUGBwgEo4MECQoLDA2pDrS1tKQPEAwHERITE77AvxKqhAQNDA8UFRYX Fs8YBAQZGqGPxw0RGxwdHR4eHyAhIiMkJSYnKCgpBAYPEhcqHyssLS4kLzAx MjM0NTY3cBA4UCAHBw8gVnhgEcKFjhc7UPDj0cMHAAI/KFgY4YLFio/jRpTY sW8GDyCSCEQw2DChOHIqgsCQSEPIEEEEJFhAoUNECCJEyOk4d6KIyRtGcB7h IJKjixEjHu4oimSGEIs4d8IIUoKECnNB0ElMwkNJJgBLlJBAcQKGiR07KGAU RVGViY0mhIwwSTKjr99+THjUoIg0r48hTRIrRtxkiOMhDgrZCQQAIf5oQ3Jl YXRlZCBieSBCTVBUb0dJRiBQcm8gdmVyc2lvbiAyLjUNCqkgRGV2ZWxDb3Ig MTk5NywxOTk4LiBBbGwgcmlnaHRzIHJlc2VydmVkLg0KaHR0cDovL3d3dy5k ZXZlbGNvci5jb20AOw=3D=3D}] set ::image(refresh) [image create photo -data { R0lGODlhFgAWAIYAAPwCBCRSFCRSHBw+DBxCFCQ6FBwyDBQWBBxGFCxyLGTC hMzqzLzmvHzKjDyOTER+RERyNDSqXNzy3LzivFS+fCyCPBQmBCQiBBxKFBQq DOTy3LTitES2dDR+PCxuJOT25KTarCx+PESSTCxKHDSeVCyKRNT21ESWVDSG PBQyDAQCBBQSFDRuLDSyZDySTGzChCRiJKSmpExKTDS2ZGzGhLy+vGxqbISC hDSKRMzKzGxubDQ2NIyOjCQiJCwqLBQWFCwuLKSipERCRERGRHR2dAwKDDw6 PFRWVIyKjCQmJFRSVBwaHKyurAQGBExOTBweHFxeXAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAA LAAAAAAWABYAAAf9gACCg4IBAgMEBQYHhI2ECAkKCwwNDg8QBAOOhAQREhMU FQgWBxcHGBmbggkaGxwPB4yDB6SbBJIKHQaqtY0eHyAhsqrDgx4aCiKpqoQH AyMjJBMKJaSxzAAHIRsmJgonKA0LHSmDKiuOBywRLSQuLyEwwyoxMuiN6iUz NBXy5jU2bsgoJugABBz95uXQsUMGD3vpPPgTpKIGwx4+HMr4kW4YkCA2hAzx AQSIECI+imBTwVIFESNHerRUgc0cEiFHkjiiyYzeDiVLdvLcySSkkKGEWiZV weSGkIHMmvQosoQlkaZOjvhosvKJjIAxoOAsgpRZkQNLnvSoqspAIAAh/mhD cmVhdGVkIGJ5IEJNUFRvR0lGIFBybyB2ZXJzaW9uIDIuNQ0KqSBEZXZlbENv ciAxOTk3LDE5OTguIEFsbCByaWdodHMgcmVzZXJ2ZWQuDQpodHRwOi8vd3d3 LmRldmVsY29yLmNvbQA7}] set ::image(add) [image create photo -data { R0lGODlhEAAQAIQAAPwCBARCZPz+/Mzq9BTC3ITC1HSyzGSivAw+XBSStIS6 zHy2zAQCDFyatAQOHFSStEyOtAQSJBSq1DR2nCxunCRmlAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAAAQABAAAAVbICCOQTme aCAMRIC+AVu47xkTBl2Ld16XQNYBQTQBVIOkMHFQJBeMI4tAbSSu2IRDSnhA vFfI97sFRM6RwOMacbjLKckVvgvIJ5EdSU7J648VgXQ7Dmd/hyJ+IQAh/mhD cmVhdGVkIGJ5IEJNUFRvR0lGIFBybyB2ZXJzaW9uIDIuNQ0KqSBEZXZlbENv ciAxOTk3LDE5OTguIEFsbCByaWdodHMgcmVzZXJ2ZWQuDQpodHRwOi8vd3d3 LmRldmVsY29yLmNvbQA7}] set ::image(delete) [image create photo -data { R0lGODlhEAAQAIMAAPwCBARCZPz+/Mzq9BTC3AQCDAQOHFSStAQSJAAAAAAA AAAAAAAAAAAAAAAAAAAAACH5BAEAAAAALAAAAAAQABAAAAQwEMhJq704681n +GAISoEwnGg6EAUQrEQsz4ThEkeu77kNIAagMEj0dY7IpHI58UcAACH+aENy ZWF0ZWQgYnkgQk1QVG9HSUYgUHJvIHZlcnNpb24gMi41DQqpIERldmVsQ29y IDE5OTcsMTk5OC4gQWxsIHJpZ2h0cyByZXNlcnZlZC4NCmh0dHA6Ly93d3cu ZGV2ZWxjb3IuY29tADs=3D}] } main ------=_NextPart_000_019D_01C30A54.09DB3F40-- From jeff_godfrey@pobox.com Thu Apr 24 16:26:41 2003 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h3OLQfN12480 for ; Thu, 24 Apr 2003 16:26:41 -0500 Received: (qmail 22249 invoked from network); 24 Apr 2003 21:21:39 -0000 Received: from wbrg-dialup-216-74-200-84.iland.net (HELO quantex) (216.74.200.84) by chewey.iland.net with SMTP; 24 Apr 2003 21:21:39 -0000 Message-ID: <006201c30aa7$6d03fd20$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: Date: Thu, 24 Apr 2003 16:20:56 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 4/23/03), Outbound message X-Antivirus-Status: Clean Subject: [Metakit] metakit - 1 read/write app, multiple reader apps - question Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have 2 separate tcl based apps using a single metakit database. One app manages the database, so it opens the database in read/write mode. The other app (of which there can be multiple instances), just needs access to the data in the database, so it opens the database in readonly mode. Here's the issue: If I have both apps running, and I change some of the data stored in the database (via the "manager app"), how can I access the changes in the reader app? Each time the reader app needs data, it gets it directly from the database, but it seems to be a copy of the data as it existed when the database was first opened by the reader. How can I get to the "changed" data without restarting the reader app (or closing and opening the database)? Hmmm... Re-reading the docs, this seems to have something to do with "open -extend" and "rollback", although I'm not sure I fully understand what I am reading. Does anyone have any advice and/or tcl examples of this? Thanks, Jeff From jcw@equi4.com Fri Apr 25 06:04:28 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h3PB4SN28792 for ; Fri, 25 Apr 2003 06:04:28 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 71CF283BED; Fri, 25 Apr 2003 07:04:44 -0400 (EDT) Date: Fri, 25 Apr 2003 13:04:45 +0200 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: "Robert W.Techentin" To: Metakit mailing list From: Jean-Claude Wippler Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.552) Subject: [Metakit] FYI: Debian Linux compile farm build logs Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: While tweaking things to make MK build better on Debian Linux, I found out that they are running a pretty nifty compile farm with pretty detailed logging: http://buildd.debian.org/build.php?pkg=libmetakit2.4.9.2 There's a general MK page at: http://packages.qa.debian.org/libm/libmetakit2.4.9.2.html This may be of use to you as reference perhaps, if you're dealing with one or more of the platforms they do: ia64, hpp1, s390, sparc, powerpc, alpha, m68k, mipsel, mips -jcw PS. Oh, and btw, MK make/config has been updated/tweaked :) - also, Gordon McMillan has just fixed some issues with the Python binding. All in CVS. From jeff_godfrey@pobox.com Tue Apr 29 10:09:44 2003 Received: from lewey.iland.net (lewey.iland.net [205.242.230.202]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h3TF9iN22185 for ; Tue, 29 Apr 2003 10:09:44 -0500 Received: (qmail 27663 invoked from network); 29 Apr 2003 15:10:10 -0000 Received: from wbrg-dialup-216-74-201-70.iland.net (HELO quantex) (216.74.201.70) by lewey.iland.net with SMTP; 29 Apr 2003 15:10:10 -0000 Message-ID: <00f001c30e61$69e2e580$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: Date: Tue, 29 Apr 2003 10:09:54 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 4/23/03), Outbound message X-Antivirus-Status: Clean Subject: [Metakit] 1 read/write app, multiple reader apps - question Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Seeing no response to a post from 5 days ago, I thought I'd try again. Thanks for any advice... Jeff ======= I have 2 separate tcl based apps using a single metakit database. One app manages the database, so it opens the database in read/write mode. The other app (of which there can be multiple instances), just needs access to the data in the database, so it opens the database in readonly mode. Here's the issue: If I have both apps running, and I change some of the data stored in the database (via the "manager app"), how can I access the changes in the reader app? Each time the reader app needs data, it gets it directly from the database, but it seems to be a copy of the data as it existed when the database was first opened by the reader. How can I get to the "changed" data without restarting the reader app (or closing and reopening the database)? Hmmm... Re-reading the docs, this seems to have something to do with "open -extend" and "rollback", although I'm not sure I fully understand what I am reading. Does anyone have any advice and/or tcl examples of this? Thanks, Jeff From mbloore@yahoo.com Fri May 2 13:09:10 2003 Received: from web14309.mail.yahoo.com (web14309.mail.yahoo.com [216.136.224.59]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h42I99N03818 for ; Fri, 2 May 2003 13:09:10 -0500 Message-ID: <20030502180938.6819.qmail@web14309.mail.yahoo.com> Received: from [65.93.220.149] by web14309.mail.yahoo.com via HTTP; Fri, 02 May 2003 11:09:38 PDT Date: Fri, 2 May 2003 11:09:38 -0700 (PDT) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] assertion failure with large DB Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: the following test program runs for a while (on our test machine, till it is duplicating 64K records), then dies on an assertion failure at line 92 of univ.cpp, which indicates a realloc failure. i am using MK2.4.9.2, but i had the same problem with 2.4.7. during adding of records memory use grows hugely (to around 700MB), then drops during commit. the machine we ran this on has 1GB of RAM and 4 GB virtual space, with an Athlon 1600+ CPU, running win2k. from zero it takes 31 CPU minutes to fail. a second run, starting from a DB of 64K records, takes 22 min. the DB file reaches 313MB. i realize that is rather large. our actual application is being asked to handle very large amounts of data now. on a much smaller machine this program started page faulting rather slowly while using no CPU, and i killed it after some hours. this is a show stopper for us. any workaround would be welcome. i am about to try doing frequent commits. TIA. #include #include #include #include int main(int argc, char **argv) { c4_Storage storage ("bigbug.db", true); c4_View v = storage.GetAs ("blobs[stuff:B]"); #if 1 const c4_BytesProp blobP ("stuff"); c4_Bytes more; char* data = (char*) more.SetBuffer (5000); memcpy (data, "test", 5); v.Add (blobP [more]); #endif for (int j = 0; j < 20; j++) { int lim = v.GetSize(); printf ("duping %d records.\n", lim); for (int i = 0; i < lim; i++) { v.Add (v[i]); } printf (" committing.\n"); storage.Commit(); } return 0; } ===== -- mARK bLOORE __________________________________ Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. http://search.yahoo.com From jcw@equi4.com Mon May 5 05:50:23 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h45AoNN24290 for ; Mon, 5 May 2003 05:50:23 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 924F983953 for ; Mon, 5 May 2003 06:50:47 -0400 (EDT) Date: Mon, 5 May 2003 12:50:49 +0200 Subject: Re: [Metakit] assertion failure with large DB Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20030502180938.6819.qmail@web14309.mail.yahoo.com> Message-Id: <6EB919CC-7EE7-11D7-BFF5-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: > the following test program runs for a while (on our test machine, till > it is duplicating 64K records), then dies on an assertion failure at > line 92 of univ.cpp, which indicates a realloc failure. [...] > on a much smaller machine this program started page faulting rather > slowly while using no CPU, and i killed it after some hours. [...] > for (int j = 0; j < 20; j++) > { > int lim = v.GetSize(); > printf ("duping %d records.\n", lim); > > for (int i = 0; i < lim; i++) > { > v.Add (v[i]); > } > > printf (" committing.\n"); > storage.Commit(); > } My suggestion would be to switch to blocked views, and to commit more often (data is collected in memory until committed). Replace this: c4_View v = storage.GetAs ("_B[blobs[stuff:B]]") with: c4_View v = storage.GetAs ("blobs[stuff:B]") v = v.Blocked(); Blocked views have dramatic better scalability, one test I did went up to 100,000,000 million (small) rows. But the hard limit of 2 Gb data, probably even quite a bit less due to virtual address space limits. Is there any way to reduce the data? Compression? More compact representations? Or splitting things into multiple smaller datafiles, and keeping only one of them open? -jcw From mbloore@yahoo.com Mon May 5 10:24:47 2003 Received: from web14305.mail.yahoo.com (web14305.mail.yahoo.com [216.136.173.81]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h45FOlN13179 for ; Mon, 5 May 2003 10:24:47 -0500 Message-ID: <20030505152518.90064.qmail@web14305.mail.yahoo.com> Received: from [64.231.192.109] by web14305.mail.yahoo.com via HTTP; Mon, 05 May 2003 08:25:18 PDT Date: Mon, 5 May 2003 08:25:18 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] assertion failure with large DB To: metakit In-Reply-To: <6EB919CC-7EE7-11D7-BFF5-000A957664F8@equi4.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Jean-Claude Wippler wrote: > mARK bLOORE wrote: >[snip] > My suggestion would be to switch to blocked views, and to commit more > often (data is collected in memory until committed). > > Replace this: > c4_View v = storage.GetAs ("_B[blobs[stuff:B]]") > with: > c4_View v = storage.GetAs ("blobs[stuff:B]") > v = v.Blocked(); > thanks, i will try that. i don't know what blocked views are, but if it is that easy... what does "_B" mean in that GetAs call? it does not appear in my code. should it have? or is it supposed to be on the new GetAs call? does that mean that i can't use my existing database files with blocked access? (the big ones can take a day to build.) i did try doing more frequent commits, and it is working much better (ie it does write the database file!). i ran into another sort-of problem when i did that: the program failed another assertion in fileio.cpp, when it couldn't memory-map the file. (in this case, i believe it is supposed to fail-over to regular file i/o, and that seems to work fine.) i take it that the d4_assert() call is inactive in release code, but active in debug code. the result was that my debug-mode test program failed, while my release-mode application did not! for testing, i commented out the asserts in ResetFileMapping(), but perhaps there is a better way. i find that reading the (very large) database after it is built is causing disk activity, where before (with slightly smaller DBs) it did not. could that be due to not using memory-mapped files? or perhaps because the data file is, in fact, even bigger than expected, suggesting that there is dead space caused by the frequent Commit calls? the disk activity is slowing my application's responsiveness. > Is there any way to reduce the data? Compression? More compact > representations? Or splitting things into multiple smaller > datafiles, and keeping only one of them open? compression of some of the largest data may be possible; i will check. keeping only one of several files open won't work, i need access to all the data all the time. keeping several smaller files open together would work, until one Commit fails and the whole set becomes inconsistent. i hope to avoid having to handle that case, since links among various data items are widespread and deep. ===== -- mARK bLOORE __________________________________ Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. http://search.yahoo.com From jcw@equi4.com Mon May 5 10:58:02 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h45Fw1N18005 for ; Mon, 5 May 2003 10:58:01 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 83E6083953 for ; Mon, 5 May 2003 11:58:25 -0400 (EDT) Date: Mon, 5 May 2003 17:58:27 +0200 Subject: Re: [Metakit] assertion failure with large DB Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20030505152518.90064.qmail@web14305.mail.yahoo.com> Message-Id: <6882E190-7F12-11D7-BFF5-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: >> Replace this: >> c4_View v = storage.GetAs ("_B[blobs[stuff:B]]") >> with: >> c4_View v = storage.GetAs ("blobs[stuff:B]") >> v = v.Blocked(); >> > > thanks, i will try that. i don't know what blocked views are, but if > it is that easy... > > what does "_B" mean in that GetAs call? it does not appear in my code. > should it have? or is it supposed to be on the new GetAs call? does > that mean that i can't use my existing database files with blocked > access? (the big ones can take a day to build.) You're going to have to rebuild, there is no conversion utility. The "_B" is required to apply v.Blocked(). What this does is switch from a view of rows to a view of subviews of rows, Blocked() makes it all appear as one big flat view again. > doing more frequent commits, and it is working much better > (ie it does write the database file!). i ran into another sort-of > problem when i did that: the program failed another assertion in > fileio.cpp, when it couldn't memory-map the file. (in this case, i > believe it is supposed to fail-over to regular file i/o, and that seems > to work fine.) i take it that the d4_assert() call is inactive in > release code, but active in debug code. the result was that my > debug-mode test program failed, while my release-mode application did > not! for testing, i commented out the asserts in ResetFileMapping(), > but perhaps there is a better way. You're hitting limits of memory mapped files in a 32-bit address space. The code will work without memory mapping, but it may be affect memory use even more, so this is probably bad. The other way to handle large datasets, is to use MK to manage the data allocation, but not the data itself (this can scale indefinitely). Basically, you can store two ints (or long ints) instead of each blob, and then store the blob in a regular file. The ints are position and offset, for example. You would then implement a free space manager and store that in MK. This need not affect the rest of the code, if you create a "custom viewer" class which hides this extra level. It also need not affect transaction/commit logic. The whole idea is to create a special view which hides everything. If you look into things like the Hash and the Blocking view, you will see that they too simply play tricks as custom viewers. One could for sure implement a generic layer which stores blobs of unlimited size in an external file (as DBase did ages ago with their memo's). There's really a ton of potential in the whole view model of MK, much of which is probably still way under-utilized. But it's hard for me to justify adding such capability until there is a project which actually needs (and funds) it. > i find that reading the (very large) database after it is built is > causing disk activity, where before (with slightly smaller DBs) it did > not. could that be due to not using memory-mapped files? or perhaps > because the data file is, in fact, even bigger than expected, > suggesting that there is dead space caused by the frequent Commit > calls? the disk activity is slowing my application's responsiveness. Multiple commits lead to unused areas of space. That gets re-used on subsequent commits. > compression of some of the largest data may be possible; i will check. > keeping only one of several files open won't work, i need access to all > the data all the time. keeping several smaller files open together > would work, until one Commit fails and the whole set becomes > inconsistent. i hope to avoid having to handle that case, since links > among various data items are widespread and deep. Ok, understand. Your long term best bet seems to be a custom viewer, as outlined above. That would still need to play some tricks, given that you're reaching the limits of mapping everything into memory at once. Another option would be to move up to a 64-but architecture... -jcw From frank@canyon-medical.com Mon May 5 14:38:09 2003 Received: from mail3.centurytel.net (mail3.centurytel.net [209.142.136.99]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h45Jc8N04379 for ; Mon, 5 May 2003 14:38:08 -0500 Received: from aSqueakSystem (pppoe0220.sm.centurytel.net [64.91.3.223]) by mail3.centurytel.net (8.12.9/8.12.9) with SMTP id h45Jcb7o007258 for ; Mon, 5 May 2003 14:38:37 -0500 (CDT) Message-Id: <200305051938.h45Jcb7o007258@mail3.centurytel.net> X-Mailer: Celeste 2.0.4956 Date: Mon, 5 May 2003 14:39:46 -0600 Subject: Re: [Metakit] assertion failure with large DB Cc: Metakit mailing list From: Frank Sergeant Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Another option would be to move up to a 64-but architecture... I think Broadway chorus lines use a 64-butt architecture. -- Frank From mbloore@yahoo.com Mon May 5 15:41:42 2003 Received: from web14305.mail.yahoo.com (web14305.mail.yahoo.com [216.136.173.81]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h45KfgN09644 for ; Mon, 5 May 2003 15:41:42 -0500 Message-ID: <20030505204213.39835.qmail@web14305.mail.yahoo.com> Received: from [64.231.192.109] by web14305.mail.yahoo.com via HTTP; Mon, 05 May 2003 13:42:13 PDT Date: Mon, 5 May 2003 13:42:13 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] assertion failure with large DB To: Metakit mailing list In-Reply-To: <200305051938.h45Jcb7o007258@mail3.centurytel.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Frank Sergeant wrote: > > Another option would be to move up to a 64-but architecture... > > I think Broadway chorus lines use a 64-butt architecture. and if you put that 64-butt line on a 1MHz bus, you get 128MFlops. __________________________________ Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. http://search.yahoo.com From jgodfrey@optinest.com Tue May 6 16:02:46 2003 Received: from linux2.optinest.com ([208.188.206.240]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h46L2kN00413 for ; Tue, 6 May 2003 16:02:46 -0500 Received: from nt40ld (nt4-0ld [208.188.206.93]) by linux2.optinest.com (8.11.6/8.11.3) with SMTP id h46L3CQ06099 for ; Tue, 6 May 2003 16:03:12 -0500 From: "Jeff Godfrey" To: Date: Tue, 6 May 2003 16:04:34 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Mk4tcl - corrupted database over network - WinNT Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I have been using a Metakit database in a tcl based starpack application for about 10 months now with no problems. The application is packaged to run under Windows. My versions are as follows: Mk4tcl - 2.4.9.1 Tcl/Tk - 8.4.2 Windows NT 4.0 The application is designed to read/write a Metakit based database file found in the same directory as the application itself. Today, for the first time, I tried to access the application over a network - using both a mapped network drive, and just "wading" to it over the Windows "Network Neighborhood". In both scenarios, I have been able seriously corrupt the database by successively adding / deleting data. The corruptions seem to be in the form of either complete garbage, or in having data (or portions of it) from one view show up in another view. Again, I have used the application (and the database code) quite extensively for about 10 months now without problems, so (I think) I am confident in the stability of my tcl code. It seems that I can regularly corrupt things when accessing the metakit file over the network. Has anyone else experienced this? I'm not even sure where to start looking. Any advice is appreciated. Thanks, Jeff Godfrey Optimation USA Phone: 816.836.2000 Fax : 816.836.8589 Email: mailto:jgodfrey@optinest.com From Garth.Lancaster@mbf.com.au Wed May 7 19:38:29 2003 Received: from fs1-smtp.mbf.com.au ([61.88.251.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h480cSN28247 for ; Wed, 7 May 2003 19:38:29 -0500 Received: from MBF-INT1-MTA by fs1-smtp.mbf.com.au with Novell_GroupWise; Thu, 08 May 2003 10:38:10 +1000 Message-Id: X-Mailer: Novell GroupWise Internet Agent 6.0.2 Date: Thu, 08 May 2003 10:39:24 +1000 From: "Garth Lancaster" To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Guinevere: 2.0.9 ; Subject: [Metakit] a humbling experience - a trial with view.search Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I recently decided to wean myself off sql type databases - I had toyed with Metakit and simple brute force view traversals a while ago, so decided it had the right 'fit' for persisting my data now .. so I started with the following (I also tried to break the c way of doing things and try in c++ for a change) ps .. I forgot to upload my source from home last night (it was 12:30), so Im doing this from memory - so there may be minor changes in the real code) -- persist.h -- #ifndef __PERSIST_H__ #define __PERSIST_H__ #include "..\metakit-2.4.9.2\include\mk4.h" #include "..\metakit-2.4.9.2\include\mk4str.h" #if _MSC_VER >= 1000 #pragma once #endif // _MSC_VER >= 1000 class CPersist { public: c4_Storage m_storage; c4_View m_view; CPersist(const CString& filename_); ~CPersist(); int CPersist::find(const CString& strccyymm); void CPersist::SetupNewMonth(const CString& ccyymm_, const int sdd_, const int edd_); protected: private: }; #endif // __PERSIST_H__ -- persist.cpp -- // persist.cpp: implementation of the CPersist class. // ////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include "persist.h" ////////////////////////////////////////////////////////////////////// // Construction/Destruction ////////////////////////////////////////////////////////////////////// CPersist::CPersist(const CString& filename_) : m_storage (filename_, true ), m_view (m_storage.GetAs("history[ccyymm:S,sdd:I,edd:I,cxx:I]")){ m_storage.AutoCommit(); } CPersist::~CPersist() { } int CPersist::find(const CString& strccyymm) { c4_StringProp pccyymm ("ccyymm"); int i = m_view.Search(pccyymm [strccyymm]); return i; } void CPersist::SetupNewMonth(const CString& ccyymm_, const int sdd_, const int edd_) { c4_Row row; c4_StringProp pccyymm ("ccyymm"); c4_IntProp psdd ("sdd"); c4_IntProp pedd ("edd"); c4_IntProp pcxx ("cxx"); m_view.Add(pccyymm [ccyymm_] + psdd[sdd_] + pedd[edd_] + pcxx[0]); m_view.Add(row); m_storage.Commit(); } ... I was happy that I could do // Set Up Which Store We Want - each outside entity will use a different store ... // strIdent = ["clayko"|"neller"] from command line CString strPersist = strIdent; strPersist += ".mk4"; // Open Persist Table CPersist trackrecord(strPersist); // Build Key - This Month As CCYYMM CTime t = CTime::GetCurrentTime(); CString strccyymm = t.Format("%Y%m"); // See If We Have A Record For This Month int i; if ((i = trackrecord.find(strccyymm)) == 0) { trackrecord.SetupNewMonth(strccyymm, 15, 20); } but the trackrecord.find call never found the row ['200305', 15, 20, 0] in the table after adding it the first time and always added another row .. a debug always showed that the search in the find method returned 0.. not shown here is that I tried to sort the view in the find method, I tried creating a sorted copy of the view in the find method .. to no avail .. I know If I create a reverse sorted view that the latest month ended up in row 0 as one would expect ... I guess I could create a view using Select, something like CString strccyymm = "200305"; c4_View vs = m_view.Select(pccyymm[strccyymm]); and check if number of rows returned is 0... but Ive got the feeling Im about to be humbled by my ineptitude in simply trying to return an indicator of the existance of the row in a simpler way .. can someone tell me what I need to do CPersist::find to make it work - its ok, I can take it .. in the meantime I'll see if I can try 'Find' .. since my store will only ever hold ccyymm and 3 integers per month, its not going to grow large or fast, so Im happy to use Find regards & hopefully thanks, Garth oh, Im using mk 2.4.9.2 as the includes should show, on NT 4.0 SP6a with MSVC++ 6 sp 5 I LOVE deadlines - particularly the whooshing sound they make as they go by ...... Garth Lancaster Team Leader : Integration & Storage Mgmt IT Services : Group Operations garth.lancaster@mbf.com.au Ph : +61 2 9323 9534 Fax : +61 2 9267 7359 This email contains confidential information intended only for the addressee. If you received it in error, please inform us by reply email or by calling us on 131 137. Please also delete this email and destroy any hard copy. You must not disclose or use the information in this email. From Garth.Lancaster@mbf.com.au Thu May 8 00:26:58 2003 Received: from fs1-smtp.mbf.com.au ([61.88.251.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h485QvN12523 for ; Thu, 8 May 2003 00:26:58 -0500 Received: from MBF-INT1-MTA by fs1-smtp.mbf.com.au with Novell_GroupWise; Thu, 08 May 2003 15:26:33 +1000 Message-Id: X-Mailer: Novell GroupWise Internet Agent 6.0.2 Date: Thu, 08 May 2003 15:27:44 +1000 From: "Garth Lancaster" To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Guinevere: 2.0.9 ; Subject: [Metakit] Apology Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Having Set My Mail Properties To Digest, I dont know (and wont) till later if anyone looked at my code and laughed !!! I thank anyone who replied ... However, I revisited it earlier today (when I was fully caffienated ie), and I found that I could get the right result from the main() of a test stub ... so I went back to the class and, shock horrer, in the CPersist.Find method I had used view.SortOn, vs what I used in the main() ... view.Sort() so, please accept my apologies and write me off as a total fool 'G' This email contains confidential information intended only for the addressee. If you received it in error, please inform us by reply email or by calling us on 131 137. Please also delete this email and destroy any hard copy. You must not disclose or use the information in this email. From jcw@equi4.com Thu May 8 03:51:42 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h488pgN25128 for ; Thu, 8 May 2003 03:51:42 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id C8FBA83953 for ; Thu, 8 May 2003 04:52:08 -0400 (EDT) Date: Thu, 8 May 2003 10:52:10 +0200 Subject: Re: [Metakit] Apology Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <5B0D041C-8132-11D7-BD77-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Garth Lancaster wrote: [apologies] No worries. Glad you sorted it out. Sometimes the path to a solution is bumpy. I know how it feels :) -jcw From steve@DigitalSmarties.com Thu May 8 05:00:14 2003 Received: from digitalsmarties.com (qmailr@gw.digital-smarties.com [203.59.125.206] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h48A0CN30099 for ; Thu, 8 May 2003 05:00:12 -0500 Received: (qmail 791 invoked from network); 8 May 2003 10:00:41 -0000 Received: from gateway.digitalsmarties.com (HELO DigitalSmarties.com) (192.168.1.1) by gateway.digitalsmarties.com (192.168.1.1) with ESMTP; 08 May 2003 10:00:41 -0000 Date: Thu, 8 May 2003 18:00:39 +0800 Subject: Re: [Metakit] Apology Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: To: "Garth Lancaster" From: Steve Landers In-Reply-To: Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Garth, > so, please accept my apologies and write me off as a total fool Nah - you'll have to work hard then that for us to write you off as a total fool :-) But seriously, we've all done similar - I certainly don't think it reflects negatively on you in any way (quite the opposite, since you were quick to respond when you found the problem). One thing you might not be aware of - there are bindings to Metakit for both Python and Tcl - exposing most (but unfortunately not all) of the Metakit functionality. Maybe you could find it an interesting exercise to try one of these scripting languages? Best regards Steve -- Steve Landers Software Design Solutions Digital Smarties steve@DigitalSmarties.com Perth, Western Australia DigitalSmarties.com From steve@DigitalSmarties.com Thu May 8 05:49:00 2003 Received: from digitalsmarties.com (qmailr@gw.digital-smarties.com [203.59.125.206] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h48AmwN01208 for ; Thu, 8 May 2003 05:48:58 -0500 Received: (qmail 2058 invoked from network); 8 May 2003 10:49:28 -0000 Received: from gateway.digitalsmarties.com (HELO DigitalSmarties.com) (192.168.1.1) by gateway.digitalsmarties.com (192.168.1.1) with ESMTP; 08 May 2003 10:49:28 -0000 Date: Thu, 8 May 2003 18:49:26 +0800 Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: To: "Jeff Godfrey" From: Steve Landers In-Reply-To: Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jeff, > I have been using a Metakit database in a tcl based starpack > application for > about 10 months now with no problems. The application is packaged to > run > under Windows. My versions are as follows: > > Mk4tcl - 2.4.9.1 > Tcl/Tk - 8.4.2 > Windows NT 4.0 Likewise, but I'm using the latest Tclkit for windows - which has Mk4tcl 2.4.9.2 > The application is designed to read/write a Metakit based database file > found in the same directory as the application itself. Yep - I do that often > Today, for the first > time, I tried to access the application over a network - using both a > mapped > network drive, and just "wading" to it over the Windows "Network > Neighborhood". In both scenarios, I have been able seriously corrupt > the > database by successively adding / deleting data. The corruptions seem > to be > in the form of either complete garbage, or in having data (or portions > of > it) from one view show up in another view. Hmm - I've definitely not seen this :-( > > Again, I have used the application (and the database code) quite > extensively > for about 10 months now without problems, so (I think) I am confident > in the > stability of my tcl code. It seems that I can regularly corrupt > things when > accessing the metakit file over the network. Bummer > Has anyone else experienced this? I'm not even sure where to start > looking. > Any advice is appreciated. What sort of SMB server is it (i.e. WinNT, Win2k, WinXP, Samba on *nix, etc)? It's a long shot, but I wonder if there's some extra buffering happening at the client end and some how this is causing the problem. Like I said - it's a long shot. But we've got to start diagnosing this somehow Steve -- Steve Landers Software Design Solutions Digital Smarties steve@DigitalSmarties.com Perth, Western Australia DigitalSmarties.com From bkelley@wi.mit.edu Thu May 8 11:19:36 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h48GJZN32087 for ; Thu, 8 May 2003 11:19:35 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003050812200522454 for ; Thu, 08 May 2003 12:20:05 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HEKTDH00.0RZ for ; Thu, 8 May 2003 12:20:05 -0400 Message-ID: <3EBA841B.2090808@wi.mit.edu> Date: Thu, 08 May 2003 12:21:47 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Second release of KitViewer Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: KitViewer 1.1 Metakit embedded data viewer. KitViewer can view metakit embedded databases. Use it to browse through data and subtables. Features Add/Remove rows from a database Edit database contents Import Database Tables from CSV files Export Database Tables to CSV files Embedded Python scripting language. Full access to database features. Changes are updated in the view. Open databases by dragging the executable to the KitViewer Desktop shortcut (windows only) Windows binary is available now. Source code will be uploaded this afternoon. Better documentation and tips will be uploaded in version 1.2 To use start the application and open a metakit database. Please forward bugs to bkelley@wi.mit.edu -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From bkelley@wi.mit.edu Thu May 8 12:56:40 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h48HueN08828 for ; Thu, 8 May 2003 12:56:40 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003050813571023586 for ; Thu, 08 May 2003 13:57:10 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HEKXVA00.ESS for ; Thu, 8 May 2003 13:57:10 -0400 Message-ID: <3EBA9ADB.4020605@wi.mit.edu> Date: Thu, 08 May 2003 13:58:51 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Second release of KitViewer References: <3EBA841B.2090808@wi.mit.edu> In-Reply-To: <3EBA841B.2090808@wi.mit.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: KitViewer is available at http://staffa.wi.mit.edu/people/kelley/ There is also a link from http://www.equi4.com/metakit/ -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From jeff_godfrey@pobox.com Thu May 8 18:43:23 2003 Received: from hewey.iland.net (qmailr@hewey.iland.net [205.242.230.200]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h48NhMN09821 for ; Thu, 8 May 2003 18:43:22 -0500 Received: (qmail 32571 invoked from network); 8 May 2003 23:43:42 -0000 Received: from wbrg-dialup-216-74-200-96.iland.net (HELO quantex) (216.74.200.96) by hewey.iland.net with SMTP; 8 May 2003 23:43:42 -0000 Message-ID: <025301c315bb$9430ce40$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Steve Landers" Cc: References: Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Date: Thu, 8 May 2003 18:42:40 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/2/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Steve Landers" To: "Jeff Godfrey" Cc: Sent: Thursday, May 08, 2003 5:49 AM Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT > What sort of SMB server is it (i.e. WinNT, Win2k, WinXP, Samba on *nix, > etc)? It's a long shot, but I wonder if there's some extra buffering > happening at the client end and some how this is causing the problem. Hi Steve, I'm no networking guru, but from what I know, it's just a run-of-the-mill Windows NT network. It definitely doesn't involve *nix and/or Samba... Well, at least I'm glad to *not* be hearing - "Oh, yeah. You can't / shouldn't do that, because it doesn't work..." I'm afraid this may be hard to track down, as I can't consistently reproduce it. I thought it was somehow related to a particular laptop system I was running on, because when I switched systems the problem mysteriously disappeared. Unfortunately, several hours later, it reappeared on the 2nd system. I then started really beating on things in earnest, and haven't seen it since... All I know for sure is that when it happens, it happens in a big way... ;^( Jeff From bkelley@wi.mit.edu Fri May 9 12:35:32 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h49HZWN23929 for ; Fri, 9 May 2003 12:35:32 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003050913360232633 for ; Fri, 09 May 2003 13:36:02 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HEMRK200.N01; Fri, 9 May 2003 13:36:02 -0400 Message-ID: <3EBBE767.8030605@wi.mit.edu> Date: Fri, 09 May 2003 13:37:43 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com CC: dndposey@jps.net Subject: Re: [Metakit] Second release of KitViewer References: <3EBA841B.2090808@wi.mit.edu> <3EBA9ADB.4020605@wi.mit.edu> In-Reply-To: <3EBA9ADB.4020605@wi.mit.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > KitViewer is available at http://staffa.wi.mit.edu/people/kelley/ > There is also a link from http://www.equi4.com/metakit/ > In case you were wondering, the links work now :) -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From jcw@equi4.com Fri May 9 12:45:05 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h49Hj5N24901 for ; Fri, 9 May 2003 12:45:05 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id A35F583953 for ; Fri, 9 May 2003 13:45:31 -0400 (EDT) Date: Fri, 9 May 2003 19:45:32 +0200 Subject: Re: [Metakit] Second release of KitViewer Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3EBBE767.8030605@wi.mit.edu> Message-Id: <07E70A33-8246-11D7-BD77-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: >> KitViewer is available at http://staffa.wi.mit.edu/people/kelley/ >> There is also a link from http://www.equi4.com/metakit/ >> > In case you were wondering, the links work now :) Indeed I was - yeay! - cool :) -jcw From bkelley@wi.mit.edu Mon May 12 16:22:13 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.4.1.89]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4CLMCN29388 for ; Mon, 12 May 2003 16:22:12 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.0.0.44) with SMTP id M2003051217224422430 for ; Mon, 12 May 2003 17:22:44 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HESM1W00.S2E for ; Mon, 12 May 2003 17:22:44 -0400 Message-ID: <3EC0110B.1050109@wi.mit.edu> Date: Mon, 12 May 2003 17:24:27 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Simple Patch for Metakit to Store Python Objects Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've been playing with a simple patch to allow metakit to transparently store python objects that are picklable. The new database type is "P". "P" is really an "S" in disguise. import metakit db = metakit.storage() vw = db.getas("test[python:P]") values = (None,1,2,3,4,5,6,None) vw.append((values,)) print 'type', vw.python print 'value', vw[0].python ================================= output type Property('P', 'python') value (None, 1, 2, 3, 4, 5, 6, None) ================================= It's not going to be blazingly fast, currently the pickle module is imported every time a variable is accessed and sorting sorts on the internal text string, not the python object, but it's fairly useful for storing arbitrary python objects. I also don't do error checking for bad pickles or not being able to actually load the pickle module so consider this a test :) I've made a patched distribution (with windows builds) that you can download from http://staffa.wi.mit.edu/people/kelley/metakit-python-patched.zip ToDo (maybe): a) sort on arbitrary python objects (this will be slow since every compared object will need to be unpickled for comparison) b) provide a compressed storage using zlib. c) make a generic metakit scheme for conversion callback function(s)? -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From jeff_godfrey@pobox.com Wed May 14 13:41:41 2003 Received: from hewey.iland.net (lewey.iland.net [205.242.230.202]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4EIffN30216 for ; Wed, 14 May 2003 13:41:41 -0500 Received: (qmail 9657 invoked from network); 14 May 2003 18:42:16 -0000 Received: from wbrg-dialup-216-74-200-18.iland.net (HELO quantex) (216.74.200.18) by lewey.iland.net with SMTP; 14 May 2003 18:42:16 -0000 Message-ID: <01de01c31a48$61560260$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: References: Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Date: Wed, 14 May 2003 13:41:00 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Jeff Godfrey" To: Sent: Tuesday, May 06, 2003 4:04 PM Subject: [Metakit] Mk4tcl - corrupted database over network - WinNT > The application is designed to read/write a Metakit based database file > found in the same directory as the application itself. Today, for the first > time, I tried to access the application over a network - using both a mapped > network drive, and just "wading" to it over the Windows "Network > Neighborhood". In both scenarios, I have been able seriously corrupt the > database by successively adding / deleting data. The corruptions seem to be > in the form of either complete garbage, or in having data (or portions of > it) from one view show up in another view. Hi All, Well, my random metakit database corruption appears to be happening again (always only over a network). Unfortunately, this time it happened at a customer site... Yuck! So far, I still have no idea what is causing it. I have been hammering on the database (over the network) for several hours today and haven't been able to break anything. I wrote a dedicated database layer for the application in question about a year ago and have really never looked back until these corruptions started. Essentially, each view of my DB has a dedicated function to add and/or update records. Below is one such function. It either updates the record (if it already exists) or adds a new record. Can anyone spot anything dangerous in this code? One thing I did notice - I don't check any return values from any of the [mk::?] commands, but the docs don't specify that anything *is* returned. Is anything returned that might indicate a failure of some sort? proc ::db::storeDataset {dsName dsDescr dsPath dsLock} { set view "db.dataset" set recLoc [mk::select $view -exact dsName $dsName] if {[llength $recLoc]} { mk::cursor create thisCursor $view $recLoc } else { set thisCursor [mk::row append $view] } mk::set $thisCursor dsName $dsName dsDescr $dsDescr dsPath $dsPath dsLock $dsLock mk::file commit db } Any help or suggestions appreciated. Thanks, Jeff From jcw@equi4.com Wed May 14 14:10:25 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4EJAON01447 for ; Wed, 14 May 2003 14:10:24 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 0CC108396D for ; Wed, 14 May 2003 15:10:55 -0400 (EDT) Date: Wed, 14 May 2003 21:10:56 +0200 Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <01de01c31a48$61560260$0101a8c0@quantex> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff, >> The application is designed to read/write a Metakit based database >> file >> found in the same directory as the application itself. Today, for the > first >> time, I tried to access the application over a network - using both a > mapped >> network drive, and just "wading" to it over the Windows "Network >> Neighborhood". In both scenarios, I have been able seriously corrupt >> the >> database by successively adding / deleting data. The corruptions >> seem to > be >> in the form of either complete garbage, or in having data (or >> portions of >> it) from one view show up in another view. > > Hi All, > > Well, my random metakit database corruption appears to be happening > again > (always only over a network). Unfortunately, this time it happened at > a > customer site... Yuck! So far, I still have no idea what is causing > it. I > have been hammering on the database (over the network) for several > hours > today and haven't been able to break anything. Ouch. > I wrote a dedicated database layer for the application in question > about a > year ago and have really never looked back until these corruptions > started. [...] > Is anything returned that might indicate a failure of some sort? No - all failures throw errors in Tcl, so you'd automatically see them. I suspect this is related to servers, caching, moon phase - or a mix. You have NT4, good - I have it too (SP6). What you didn't mention is how the server is set up. Presumably NT4 is your desktop. What's the server? How much confidence do you have that everything has been configured properly? Be sure to run through the change log to verify that 2.4.9.1 vs 2.4.9.2 have no issues which might affect you: http://www.equi4.com/pub/mk/metakit-2.4.9.2.kit/CHANGES FWIW, I just checked, I didn't see anything disturbing. If you can somehow create a test script which independently creates a case that fails, it would be great. It is hard for me to chase this right now. Is it a big datafile? Is performance critical? If not, then there may be tricks to make I/O work differently (slower) but at least correct. What I'm thinking of is a hack to prevent MK from using memory mapped files. -jcw From jcw@equi4.com Wed May 14 14:15:03 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4EJF2N02059 for ; Wed, 14 May 2003 14:15:02 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 6776C8396D for ; Wed, 14 May 2003 15:15:33 -0400 (EDT) Date: Wed, 14 May 2003 21:15:35 +0200 Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <025301c315bb$9430ce40$0101a8c0@quantex> Message-Id: <705441CC-8640-11D7-9E32-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: > I'm no networking guru, but from what I know, it's just a > run-of-the-mill > Windows NT network. It definitely doesn't involve *nix and/or Samba... [...] > All I know for sure is that when it happens, it happens in a big > way... ;^( That's to be expected in MK's case. Damage doesn't affect rows but columns. You are absolutely certain that the datafile is *never* opened twice at the same time, right? (easy to run into this, when on a network server) Because it there is even a remote chance, you'll need to prevent it, i.e. add locking. -jcw From jeff_godfrey@pobox.com Wed May 14 14:52:15 2003 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4EJqFN06310 for ; Wed, 14 May 2003 14:52:15 -0500 Received: (qmail 24114 invoked from network); 14 May 2003 19:52:52 -0000 Received: from wbrg-dialup-216-74-200-18.iland.net (HELO quantex) (216.74.200.18) by chewey.iland.net with SMTP; 14 May 2003 19:52:52 -0000 Message-ID: <020401c31a52$3a88f340$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Jean-Claude Wippler" , "Metakit mailing list" References: Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Date: Wed, 14 May 2003 14:51:29 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Jean-Claude Wippler" To: "Metakit mailing list" Sent: Wednesday, May 14, 2003 2:10 PM Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT > > Is anything returned that might indicate a failure of some sort? > > No - all failures throw errors in Tcl, so you'd automatically see them. OK, that's good... > I suspect this is related to servers, caching, moon phase - or a mix. > > You have NT4, good - I have it too (SP6). What you didn't mention is > how the server is set up. Presumably NT4 is your desktop. What's the > server? How much confidence do you have that everything has been > configured properly? I'm no network expert, so I'll have to check the details with my SysAdmin and let you know. Worse though, I don't (obviously) have control over these settings at a customer site. Any idea how susceptible metakit is to network tweaks? Apparently somewhat, at least. My hopes are somewhat raised by the fact that no one else seems to have seen this before. Here's hoping the entire design (of my software) doesn't come crashing down due to writing to metakit over a network :-) > Be sure to run through the change log to verify that 2.4.9.1 vs 2.4.9.2 > have no issues which might affect you: > http://www.equi4.com/pub/mk/metakit-2.4.9.2.kit/CHANGES > FWIW, I just checked, I didn't see anything disturbing. Thanks. I checked it earlier and nothing really stood out... > If you can somehow create a test script which independently creates a > case that fails, it would be great. It is hard for me to chase this > right now. Agreed. That would be nice, but so far - no dice. I'll keep working on it. > Is it a big datafile? Is performance critical? If not, then there may > be tricks to make I/O work differently (slower) but at least correct. > What I'm thinking of is a hack to prevent MK from using memory mapped > files. No, the file is *very* small as far as DB's go. Currently it contains 13 separate views. Some of them have only (and will never have more than) just a few records. The largest view will have maybe 30-60 records with about 5 properties.... Right now, I'd be more than willing to trade speed for reliability. What are you thinking? Thanks for the help, Jeff From jeff_godfrey@pobox.com Wed May 14 14:53:54 2003 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4EJrsN06502 for ; Wed, 14 May 2003 14:53:54 -0500 Received: (qmail 25453 invoked from network); 14 May 2003 19:54:32 -0000 Received: from wbrg-dialup-216-74-200-18.iland.net (HELO quantex) (216.74.200.18) by chewey.iland.net with SMTP; 14 May 2003 19:54:32 -0000 Message-ID: <020701c31a52$769320e0$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Jean-Claude Wippler" , "Metakit mailing list" References: <705441CC-8640-11D7-9E32-000A957664F8@equi4.com> Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Date: Wed, 14 May 2003 14:53:11 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Jean-Claude Wippler" To: "Metakit mailing list" Sent: Wednesday, May 14, 2003 2:15 PM Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT > You are absolutely certain that the datafile is *never* opened twice at > the same time, right? (easy to run into this, when on a network > server) Because it there is even a remote chance, you'll need to > prevent it, i.e. add locking. Yes, my app has a locking mechanism built in - and I'm confident that it is working correctly. Also, when I *saw* this happen a week or two ago, I'm sure that I was the only one accessing anything - it's still in development (by only me) after all... :-) Jeff From jcw@equi4.com Wed May 14 15:31:52 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4EKVqN10558 for ; Wed, 14 May 2003 15:31:52 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id C47FC8396D for ; Wed, 14 May 2003 16:32:22 -0400 (EDT) Date: Wed, 14 May 2003 22:32:25 +0200 Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <020401c31a52$3a88f340$0101a8c0@quantex> Message-Id: <2C0EE03F-864B-11D7-9E32-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff, > No, the file is *very* small as far as DB's go. Currently it contains > 13 > separate views. Some of them have only (and will never have more > than) just > a few records. The largest view will have maybe 30-60 records with > about 5 > properties.... > > Right now, I'd be more than willing to trade speed for reliability. > What > are you thinking? Several options. One is to save differently, to a new file and then replace the datafile with the changed one. That would keep memory mapping on, but would never change the file on the server - other than replacing it completely (better make sure it's well tested, or you might risk losing the datafile completely). The logic is to open with -readonly, and then to save: set fd [open s:/path/on/server/outfile w] mk::file save db $fd close $fd file rename -force s:/path/on/server/outfile s:/path/on/server/datafile The above means you're only reading and doing full saves, so you have to get rid of commit calls. The other, even more drastic, measure is to also load the database entirely via a read, and then use it as in-mem DB. That approach means again you cannot commit. To open in this manner: mk::view open db ;# note: no filename given set fd [open s:/path/on/server/datafile] mk::file load db $fd close $fd At this point, you have an in-memory datafile. To be honest, I'd be hesitant to do either one. Running away from the problem is not going to solve it - but it's your call, and maybe you have no choice. A script that triggers the problem could help a lot. Make it commit more than once, make it save substantial amounts of data, close and reopen occasionally. Let it run overnight, whatever... Another way to try nailing this down, is to create a trace of all MK requests (even in code shipped to your customer, if need be), which when things go wrong can be inspected and even replayed and pruned until the trouble is localized in a specific sequence of requests. See the "mktrace.tcl" script, which is old but should still work - or can as least give you some thoughts on how to go about that: http://www.equi4.com/pub/etc/mktrace.tcl Note that this logs *all* MK calls, so if you're using starkits, you might want to add logic to filter out some of the calls. -jcw PS. I'll re-send this email to the starkit mailing list, in case you want to discuss Tcl-specific aspects of all this. Though if it's a case of MK damaging files, then this is indeed the right mailing list for such issues. From jeff_godfrey@pobox.com Wed May 14 16:38:04 2003 Received: from chewey.iland.net (chewey.iland.net [205.242.230.203]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4ELc4N17363 for ; Wed, 14 May 2003 16:38:04 -0500 Received: (qmail 14373 invoked from network); 14 May 2003 21:38:39 -0000 Received: from wbrg-dialup-216-74-200-18.iland.net (HELO quantex) (216.74.200.18) by chewey.iland.net with SMTP; 14 May 2003 21:38:39 -0000 Message-ID: <023301c31a60$ff097ba0$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Jean-Claude Wippler" , "Metakit mailing list" References: <2C0EE03F-864B-11D7-9E32-000A957664F8@equi4.com> Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Date: Wed, 14 May 2003 16:37:12 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: JC, Thanks for the input and the options. FWIW, I agree with you - I'd much rather find the problem and see it fixed than finding a way to work around it. At this point, I'm not sure how quickly I am going to find myself backed into a corner and *need* a fix for my existing app. If it ends up being soon (which is very possible), I'll investigate some of the options you've mentioned. Again, thanks for the help. I *really* appreciate it. Jeff ----- Original Message ----- From: "Jean-Claude Wippler" To: "Metakit mailing list" Sent: Wednesday, May 14, 2003 3:32 PM Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT > Jeff, > > > No, the file is *very* small as far as DB's go. Currently it contains > > 13 > > separate views. Some of them have only (and will never have more > > than) just > > a few records. The largest view will have maybe 30-60 records with > > about 5 > > properties.... > > > > Right now, I'd be more than willing to trade speed for reliability. > > What > > are you thinking? > > Several options. > > One is to save differently, to a new file and then replace the datafile > with the changed one. That would keep memory mapping on, but would > never change the file on the server - other than replacing it > completely (better make sure it's well tested, or you might risk losing > the datafile completely). > > The logic is to open with -readonly, and then to save: > > set fd [open s:/path/on/server/outfile w] > mk::file save db $fd > close $fd > file rename -force s:/path/on/server/outfile s:/path/on/server/datafile > > The above means you're only reading and doing full saves, so you have > to get rid of commit calls. > > The other, even more drastic, measure is to also load the database > entirely via a read, and then use it as in-mem DB. That approach means > again you cannot commit. To open in this manner: > > mk::view open db ;# note: no filename given > set fd [open s:/path/on/server/datafile] > mk::file load db $fd > close $fd > > At this point, you have an in-memory datafile. > > To be honest, I'd be hesitant to do either one. Running away from the > problem is not going to solve it - but it's your call, and maybe you > have no choice. > > A script that triggers the problem could help a lot. Make it commit > more than once, make it save substantial amounts of data, close and > reopen occasionally. Let it run overnight, whatever... > > Another way to try nailing this down, is to create a trace of all MK > requests (even in code shipped to your customer, if need be), which > when things go wrong can be inspected and even replayed and pruned > until the trouble is localized in a specific sequence of requests. See > the "mktrace.tcl" script, which is old but should still work - or can > as least give you some thoughts on how to go about that: > http://www.equi4.com/pub/etc/mktrace.tcl > > Note that this logs *all* MK calls, so if you're using starkits, you > might want to add logic to filter out some of the calls. > > -jcw > > PS. I'll re-send this email to the starkit mailing list, in case you > want to discuss Tcl-specific aspects of all this. Though if it's a > case of MK damaging files, then this is indeed the right mailing list > for such issues. > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From jeff_godfrey@pobox.com Wed May 14 22:12:53 2003 Received: from hewey.iland.net (qmailr@hewey.iland.net [205.242.230.200]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4F3CrN12512 for ; Wed, 14 May 2003 22:12:53 -0500 Received: (qmail 17529 invoked from network); 15 May 2003 03:13:15 -0000 Received: from wbrg-dialup-216-74-202-16.iland.net (HELO quantex) (216.74.202.16) by hewey.iland.net with SMTP; 15 May 2003 03:13:15 -0000 Message-ID: <009701c31a8f$cec57b40$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Jean-Claude Wippler" , "Metakit mailing list" References: <2C0EE03F-864B-11D7-9E32-000A957664F8@equi4.com> Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Date: Wed, 14 May 2003 22:11:57 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Jean-Claude Wippler" To: "Metakit mailing list" Sent: Wednesday, May 14, 2003 3:32 PM Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT > A script that triggers the problem could help a lot. Make it commit > more than once, make it save substantial amounts of data, close and > reopen occasionally. Let it run overnight, whatever... Hi JC, Well, I've put together a "Metakit stress tester" script of sorts. Although I haven't seen any corruption yet, I have seen something I don't understand. Since I am working from home, I can't exactly mimic the offending network setup, but here's what I have: - A Win98 network with a single WinNT laptop hanging off of it. - On a Win98 machine, I've started my stress-tester script. The script is designed to constantly (via an [after idle] script) add and/or update records in a Metakit database that is stored on a mapped network drive on the WinNT system. Basically, the Metakit file is opened in write mode, hammered on for "X" minutes, the closed. - While that is running, I've started a separate tcl app on the WinNT system. It's only purpose is to open the *same* Metakit file (this time in -readonly mode), spit some of it's content to the console window, and close the file. Well, I can open the file without problems. I can retrieve some data without problems. But when I try to close the file, the script just hangs. Actually, my local mk4tcl.dll file *was* 2.4.9 - it actually generated a doctor Watson error. I've since upgraded both systems to 2.4.9.2, and now the close of the "-readonly" session just hangs - permanently it appears. Should I be able to do this? Should I even worry about closing a file that I opened in read-only mode? Currently, this is how the *real* apps in question are handling things. I can't say that I've ever had the actual "-readonly" app hang on close, but then it's unlikely that the Metakit file was being written to (by the other app) during the close - in this testing scenario it's *very* likely. Does any of the above provide any hints, or does this seem "unrelated" to the real problem? Thanks for any input. Jeff From lista@nakawe.se Wed May 14 23:02:32 2003 Received: from smtp.telenordia.se (franklin.telenor.se [213.150.135.136]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4F42WN15777 for ; Wed, 14 May 2003 23:02:32 -0500 Message-Id: <200305150402.h4F42WN15777@trixie.triqs.com> Received: from adslsmtp.tninet.se (h71n1c1o1018.bredband.skanova.com [217.208.212.71]) by franklin.telenor.se (BMR ErlangTM/OTP 3.1) with ESMTP id 297010.971389.1052.1s5926350franklin for ; Thu, 15 May 2003 06:03:09 +0200 X-Mailer: Ultrafunk Popcorn release 1.40b3 (25-April-2003) X-URL: http://www.ultrafunk.com/products/popcorn/ X-Priority: 3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 Date: Thu, 15 May 2003 06:06:32 +0100 From: Veronica Loell To: metakit@equi4.com Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Reply-To: lista@nakawe.se Organization: Nakawe data Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I can't say anything on the technical bits. But my gut tells me that I would choose a "locking" solution that included 1 server to handle all requests even the readonly ones. Perhaps returning the results in a separate MK-database for the readonly requests. The server would then work something like this: - open, read/write, close - - open, read/write, close - etc. Of course including locking information for any client that wishes to actually change data. However as jcw _and_ you said, it is very important to chase down the actual problem not just try to go around it. - Veronica Loell > >Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT > From: "Jeff Godfrey" > Date: Wed, 14 May 2003 22:11:57 -0500 > To: "Jean-Claude Wippler" , > "Metakit mailing list" > >----- Original Message ----- >From: "Jean-Claude Wippler" >To: "Metakit mailing list" >Sent: Wednesday, May 14, 2003 3:32 PM >Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT > >> A script that triggers the problem could help a lot. Make it commit >> more than once, make it save substantial amounts of data, close and >> reopen occasionally. Let it run overnight, whatever... > >Hi JC, > >Well, I've put together a "Metakit stress tester" script of sorts. Although >I haven't seen any corruption yet, I have seen something I don't understand. > >Since I am working from home, I can't exactly mimic the offending network >setup, but here's what I have: > >- A Win98 network with a single WinNT laptop hanging off of it. >- On a Win98 machine, I've started my stress-tester script. The script is >designed to constantly (via an [after idle] script) add and/or update >records in a Metakit database that is stored on a mapped network drive on >the WinNT system. Basically, the Metakit file is opened in write mode, >hammered on for "X" minutes, the closed. >- While that is running, I've started a separate tcl app on the WinNT >system. It's only purpose is to open the *same* Metakit file (this time >in -readonly mode), spit some of it's content to the console window, and >close the file. > >Well, I can open the file without problems. I can retrieve some data without >problems. But when I try to close the file, the script just hangs. >Actually, my local mk4tcl.dll file *was* 2.4.9 - it actually generated a >doctor Watson error. I've since upgraded both systems to 2.4.9.2, and now >the close of the "-readonly" session just hangs - permanently it appears. > >Should I be able to do this? Should I even worry about closing a file that >I opened in read-only mode? Currently, this is how the *real* apps in >question are handling things. I can't say that I've ever had the actual >"-readonly" app hang on close, but then it's unlikely that the Metakit file >was being written to (by the other app) during the close - in this testing >scenario it's *very* likely. > >Does any of the above provide any hints, or does this seem "unrelated" to >the real problem? > >Thanks for any input. > > >Jeff > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit From scusack@helpisit.com.au Thu May 15 02:53:45 2003 Received: from mail.iinet.net.au (mail-09.iinet.net.au [203.59.3.41]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4F7rhN03038 for ; Thu, 15 May 2003 02:53:44 -0500 Received: (qmail 11436 invoked from network); 15 May 2003 07:49:18 -0000 Received: from unknown (HELO kwaka) (203.217.17.36) by mail.iinet.net.au with SMTP; 15 May 2003 07:49:18 -0000 Date: Thu, 15 May 2003 17:49:21 +1000 From: Simon Cusack X-Mailer: The Bat! (v1.62i) Personal Reply-To: Simon Cusack X-Priority: 3 (Normal) Message-ID: <8412908921.20030515174921@helpisit.com.au> To: "Metakit mailing list" Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT In-Reply-To: <009701c31a8f$cec57b40$0101a8c0@quantex> References: <2C0EE03F-864B-11D7-9E32-000A957664F8@equi4.com> <009701c31a8f$cec57b40$0101a8c0@quantex> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, first sorry about that previous blank email, accidentally hit the send key and couldn't stop it in time. I had this problem a while ago, I was using the c++ api, the client could be nt4 or win2000, the server could be a novell server or a nt4 box. I think it was metakit 2.4.8 but I can't be sure. I posted to the list but I found it very hard to replicate the problem and other pressures caused me to just limit it to working locally. I always meant to get back to it and write a script should the problem. However I could run test scripts for you if you email them to me, this might help track down the problem in different situations. How about a script that repeatedly writes the same rows, closes, opens and then reads to check that it gets back the same rows? regards, Simon. Thursday, May 15, 2003, 1:11:57 PM, you wrote: JG> ----- Original Message ----- JG> From: "Jean-Claude Wippler" JG> To: "Metakit mailing list" JG> Sent: Wednesday, May 14, 2003 3:32 PM JG> Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT >> A script that triggers the problem could help a lot. Make it commit >> more than once, make it save substantial amounts of data, close and >> reopen occasionally. Let it run overnight, whatever... JG> Hi JC, JG> Well, I've put together a "Metakit stress tester" script of sorts. Although JG> I haven't seen any corruption yet, I have seen something I don't understand. JG> Since I am working from home, I can't exactly mimic the offending network JG> setup, but here's what I have: JG> - A Win98 network with a single WinNT laptop hanging off of it. JG> - On a Win98 machine, I've started my stress-tester script. The script is JG> designed to constantly (via an [after idle] script) add and/or update JG> records in a Metakit database that is stored on a mapped network drive on JG> the WinNT system. Basically, the Metakit file is opened in write mode, JG> hammered on for "X" minutes, the closed. JG> - While that is running, I've started a separate tcl app on the WinNT JG> system. It's only purpose is to open the *same* Metakit file (this time JG> in -readonly mode), spit some of it's content to the console window, and JG> close the file. JG> Well, I can open the file without problems. I can retrieve some data without JG> problems. But when I try to close the file, the script just hangs. JG> Actually, my local mk4tcl.dll file *was* 2.4.9 - it actually generated a JG> doctor Watson error. I've since upgraded both systems to 2.4.9.2, and now JG> the close of the "-readonly" session just hangs - permanently it appears. JG> Should I be able to do this? Should I even worry about closing a file that JG> I opened in read-only mode? Currently, this is how the *real* apps in JG> question are handling things. I can't say that I've ever had the actual JG> "-readonly" app hang on close, but then it's unlikely that the Metakit file JG> was being written to (by the other app) during the close - in this testing JG> scenario it's *very* likely. JG> Does any of the above provide any hints, or does this seem "unrelated" to JG> the real problem? JG> Thanks for any input. JG> Jeff JG> _______________________________________________ JG> metakit mailing list - metakit@equi4.com JG> http://www.equi4.com/mailman/listinfo/metakit From scusack@helpisit.com.au Thu May 15 02:53:45 2003 Received: from mail.iinet.net.au (mail-04.iinet.net.au [203.59.3.36]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4F7rhN03037 for ; Thu, 15 May 2003 02:53:44 -0500 Received: (qmail 25758 invoked from network); 15 May 2003 07:43:51 -0000 Received: from unknown (HELO kwaka) (203.217.17.36) by mail.iinet.net.au with SMTP; 15 May 2003 07:43:51 -0000 Date: Thu, 15 May 2003 17:43:54 +1000 From: Simon Cusack X-Mailer: The Bat! (v1.62i) Personal Reply-To: Simon Cusack X-Priority: 3 (Normal) Message-ID: <2512582140.20030515174354@helpisit.com.au> To: "Metakit mailing list" Subject: Re[2]: [Metakit] Mk4tcl - corrupted database over network - WinNT In-Reply-To: <009701c31a8f$cec57b40$0101a8c0@quantex> References: <2C0EE03F-864B-11D7-9E32-000A957664F8@equi4.com> <009701c31a8f$cec57b40$0101a8c0@quantex> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jeff, Thursday, May 15, 2003, 1:11:57 PM, you wrote: JG> ----- Original Message ----- JG> From: "Jean-Claude Wippler" JG> To: "Metakit mailing list" JG> Sent: Wednesday, May 14, 2003 3:32 PM JG> Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT >> A script that triggers the problem could help a lot. Make it commit >> more than once, make it save substantial amounts of data, close and >> reopen occasionally. Let it run overnight, whatever... JG> Hi JC, JG> Well, I've put together a "Metakit stress tester" script of sorts. Although JG> I haven't seen any corruption yet, I have seen something I don't understand. JG> Since I am working from home, I can't exactly mimic the offending network JG> setup, but here's what I have: JG> - A Win98 network with a single WinNT laptop hanging off of it. JG> - On a Win98 machine, I've started my stress-tester script. The script is JG> designed to constantly (via an [after idle] script) add and/or update JG> records in a Metakit database that is stored on a mapped network drive on JG> the WinNT system. Basically, the Metakit file is opened in write mode, JG> hammered on for "X" minutes, the closed. JG> - While that is running, I've started a separate tcl app on the WinNT JG> system. It's only purpose is to open the *same* Metakit file (this time JG> in -readonly mode), spit some of it's content to the console window, and JG> close the file. JG> Well, I can open the file without problems. I can retrieve some data without JG> problems. But when I try to close the file, the script just hangs. JG> Actually, my local mk4tcl.dll file *was* 2.4.9 - it actually generated a JG> doctor Watson error. I've since upgraded both systems to 2.4.9.2, and now JG> the close of the "-readonly" session just hangs - permanently it appears. JG> Should I be able to do this? Should I even worry about closing a file that JG> I opened in read-only mode? Currently, this is how the *real* apps in JG> question are handling things. I can't say that I've ever had the actual JG> "-readonly" app hang on close, but then it's unlikely that the Metakit file JG> was being written to (by the other app) during the close - in this testing JG> scenario it's *very* likely. JG> Does any of the above provide any hints, or does this seem "unrelated" to JG> the real problem? JG> Thanks for any input. JG> Jeff JG> _______________________________________________ JG> metakit mailing list - metakit@equi4.com JG> http://www.equi4.com/mailman/listinfo/metakit From jeff_godfrey@pobox.com Thu May 15 08:34:17 2003 Received: from hewey.iland.net (lewey.iland.net [205.242.230.202]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4FDYHN09206 for ; Thu, 15 May 2003 08:34:17 -0500 Received: (qmail 18710 invoked from network); 15 May 2003 13:34:55 -0000 Received: from wbrg-dialup-216-74-201-47.iland.net (HELO quantex) (216.74.201.47) by lewey.iland.net with SMTP; 15 May 2003 13:34:55 -0000 Message-ID: <00b401c31ae6$818fefc0$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Jean-Claude Wippler" , "Metakit mailing list" References: <2C0EE03F-864B-11D7-9E32-000A957664F8@equi4.com> <023301c31a60$ff097ba0$0101a8c0@quantex> Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Date: Thu, 15 May 2003 08:32:57 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks to all who have responded with ideas, tips, and offers to help. I'll try to respond to each posting soon - I just don't have the time right now. Upon further use of my "test" app, I am more concerned about the apparent fragilityy of the database close from the "-readonly" side (although it may be completely unrelated to my real problem - database corruption). Just a quick recap - I have a tcl/metakit based app running on a Win98 box, designed to create a metakit database file on a mapped network drive (on a WinNT laptop). It just runs for "X" minutes and randomly adds or updates 1 record in the metakit file every 200 milliseconds. At the same time, I have a 2nd tcl/metakit based app running on the WinNT laptop (actually - it just [package require]'s Mk4tcl and opens a console). Now, I interactively open the same Metakit file that is being (actively) updated from the Win98 box - but this time in "-readonly" mode. So far so good. But, if I now try to close the file opened as "-readonly" - I get varying results: - Sometimes, it just closes cleanly - although this is rare - Sometimes, it just permanently hangs the wish session - Sometimes, it triggers a Dr. Watson error in wish84 Again, I can't say that I've ever seen the hang or crash issues noted above in my "real" application. I'm not sure why my testing environment seems to be so fragile... Also, I am using Mk4tcl 2.4.9.2 and ActiveTCL 8.4.2 Any (more) ideas? Thanks, Jeff From jcw@equi4.com Thu May 15 08:59:53 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4FDxrN12161 for ; Thu, 15 May 2003 08:59:53 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 297D38396D for ; Thu, 15 May 2003 10:00:25 -0400 (EDT) Date: Thu, 15 May 2003 16:00:24 +0200 Subject: Re: [Metakit] Mk4tcl - corrupted database over network - WinNT Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: "Metakit mailing list" Content-Transfer-Encoding: 7bit In-Reply-To: <00b401c31ae6$818fefc0$0101a8c0@quantex> Message-Id: <936124A8-86DD-11D7-9E32-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: > Upon further use of my "test" app, I am more concerned about the > apparent > fragilityy of the database close from the "-readonly" side (although > it may > be completely unrelated to my real problem - database corruption). I agree - what you describe is *weird*! > Just a quick recap - I have a tcl/metakit based app running on a Win98 > box, > designed to create a metakit database file on a mapped network drive > (on a > WinNT laptop). It just runs for "X" minutes and randomly adds or > updates 1 > record in the metakit file every 200 milliseconds. > > At the same time, I have a 2nd tcl/metakit based app running on the > WinNT > laptop (actually - it just [package require]'s Mk4tcl and opens a > console). > Now, I interactively open the same Metakit file that is being > (actively) > updated from the Win98 box - but this time in "-readonly" mode. > > So far so good. But, if I now try to close the file opened as > "-readonly" - > I get varying results: > > - Sometimes, it just closes cleanly - although this is rare > - Sometimes, it just permanently hangs the wish session > - Sometimes, it triggers a Dr. Watson error in wish84 > > Again, I can't say that I've ever seen the hang or crash issues noted > above > in my "real" application. I'm not sure why my testing environment > seems to > be so fragile... > > Also, I am using Mk4tcl 2.4.9.2 and ActiveTCL 8.4.2 Very very strange. The open code on Win32 is: int flags = _O_BINARY | _O_NOINHERIT | (mode_ > 0 ? _O_RDWR : _O_RDONLY); int fd = _open(fname_, flags); I wonder if we need some magic flags for Win32 servers? And whether that depends on context, client or server Windows version, moonphase? The problem you describe is absolutely new to me. How about a plain file open instead of MK readonly? Does that hang too? -jcw From jyl@mod3.net Thu May 15 10:26:01 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4FFQ1N22201 for ; Thu, 15 May 2003 10:26:01 -0500 Received: (qmail 27700 invoked by uid 48); 15 May 2003 15:24:58 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 15 May 2003 08:24:57 -0700 (PDT) Message-ID: <65038.209.233.24.218.1053012297.squirrel@mod3.net> Date: Thu, 15 May 2003 08:24:57 -0700 (PDT) From: "Jacob Levy" To: X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] [HEADS UP] Proposed changes in e4Graph APIs and terminology Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I wanted everyone to have an opportunity to have a say in this discussion, so that I can make the right decision. The background is that I'm finding the current terminology of e4Graph hard to explain and understand, and non-standard as far as graph theory goes. So the following changes are intended to make the APIs more conformant to the common terminology as well as making e4Graph richer, more capable and easier to understand. I want to make the changes *now*, before e4Graph 1.0final comes out, so I will not have to support two very different APIs forever. But I will do what the majority of those responding ask for, so your input is very important. Some of you may know already that e4Graph is a way to store arbitrary directed graphs of objects in persistent storages. I use Metakit as the storage mechanism, with great success -- Metakit has been extremely robust, fast, portable and useful, and besides, it's supported by the world's most responsive programmer, JCW :). e4Graph has language bindings for Tcl and Java, and now Python is being added. I'm also planning to make e4Graph use other DBs as storage, e.g. mySQL, postgress, etc. e4Graph currently has just two concepts, nodes and vertices. Nodes are ordered collections of vertices, and vertices are a mechanism to transition from a node to a value (be it an integer, double, string, binary or other node). You'll note that the terminology is not graph-theory-standard; usually what e4Graph calls vertice is called edge in graph-theoretic terminology. The new e4Graph API will be based on three concepts (instead of the existing two): * Nodes stay the same. * Vertices that lead from a node to another node are now edges. * Vertices whose value is a scalar, string or binary are now attributes. An attribute is simply an association between a string name and a typed value. In the new API, attributes can have *anything* as their value, including scalars, string, binary, edge or node. Nodes thus now become ordered collections of edges. Additionally, edges and nodes can have any number of attributes. Previously verices were named. The name now becomes one of the attributes on the equivalent edge. All of this implies large changes in the APIs of all language bindings. I formally can do this now, because e4Graph is still in Alpha (and I kept it in Alpha for a very long time for the express purpose of getting everything right the first time). However, I recognize that many projects already use e4Graph, some commercially. Therefore I solicit feedback. Two alternatives, besides making the changes now: * The first is to bring e4Graph 1.0 to the release version with the current API and then immediately abandon it. New development will go into e4Graph 2.0 which will be finished much faster than 1.0 was. * Not make any changes, ever. e4Graph 1.0 is the only version, and we live with the non-standard terminology and without the flexibility afforded by the new attribute mechanism. As might be obvious, I prefer the proposal I'm making here, and as a less desirable alternative, to finish 1.0 and move on immediately to 2.0 while supporting 1.0 with bug fixes (but NO NEW DEVELOPMENT). Please send in your comments. I will wait for a couple of weeks before deciding anything. --JYL From MStanczak@znet.com Thu May 15 13:08:16 2003 Received: from la.znet.com (la.znet.com [207.167.96.1]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4FI8GN07751 for ; Thu, 15 May 2003 13:08:16 -0500 Received: from trying (lats10-236.znet.net [207.167.97.236]) (authenticated bits=0) by la.znet.com (8.12.6/8.12.6/le1-la) with ESMTP id h4FI8n63012088 for ; Thu, 15 May 2003 11:08:51 -0700 (PDT) X-Envelope-From: MStanczak@znet.com X-Envelope-To: From: Mariusz Stanczak To: metakit@equi4.com Subject: Re: [Metakit] [HEADS UP] Proposed changes in e4Graph APIs and terminology Date: Thu, 15 May 2003 21:08:51 +0300 Organization: Forth Bits Message-ID: References: <65038.209.233.24.218.1053012297.squirrel@mod3.net> In-Reply-To: <65038.209.233.24.218.1053012297.squirrel@mod3.net> X-Mailer: Forte Agent 1.93/32.576 English (American) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Thu, 15 May 2003 08:24:57 -0700 (PDT), Jacob Levy wrote: >The new e4Graph API will be based on three concepts (instead of the >existing two): > >* Nodes stay the same. >* Vertices that lead from a node to another node are now edges. >* Vertices whose value is a scalar, string or binary are now attributes. >An attribute is simply an association between a string name and a typed >value. In the new API, attributes can have *anything* as their value, >including scalars, string, binary, edge or node. > >Nodes thus now become ordered collections of edges. Additionally, edges >and nodes can have any number of attributes. Previously verices were >named. The name now becomes one of the attributes on the equivalent edge. This is a revolution ;-), but to get an answer to your question out of the way, mine is a +1 vote for your choice on how to proceed. Now, could you clarify what would be the difference between an Edge, and an Attribute? Semantic sugar? If such would be the case, then wouldn't it be beneficial for e4Graph to be conceptually reduced to just Nodes and Attributes (with no loss in flexibility, and simpler to explain too)? In your brief outline above, what's to prevent an Edge to have one of it's attributes be of type Node, i.e. be an Edge? Unless you plan an Edge to be a special case whose type's only attribute be of type string (a name). > >--JYL /Mariusz From jeff_godfrey@pobox.com Fri May 16 11:00:26 2003 Received: from dewey.iland.net (dewey.iland.net [205.242.230.201]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4GG0PN15249 for ; Fri, 16 May 2003 11:00:25 -0500 Received: (qmail 12533 invoked from network); 16 May 2003 15:55:59 -0000 Received: from wbrg-dialup-216-74-201-15.iland.net (HELO quantex) (216.74.201.15) by dewey.iland.net with SMTP; 16 May 2003 15:55:59 -0000 Message-ID: <01e601c31bc3$7c29e940$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Jean-Claude Wippler" , "Metakit mailing list" References: <705441CC-8640-11D7-9E32-000A957664F8@equi4.com> Date: Fri, 16 May 2003 10:54:23 -0500 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_01E3_01C31B99.82F5D100" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Subject: [Metakit] Mk4tcl - corrupted database over NT network - update [LONG] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_01E3_01C31B99.82F5D100 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Hi All, After 2 days of (very confusing) testing, I *think* I have a tcl script and a "procedure" to consistently reproduce my database corruption. First, an explanation of my test systems: - ActiveTcl 8.4.2 - Mk4tcl 2.4.9.2 - 2 pc's running NT 4.0 Workstation (Service pack 6A) - Directly connected through a network hub (no other systems connected, ie no other network traffic) Next, an explanation of the (attached) tcl program: When executed, it opens a very sparse GUI that will allow you to define a directory for the storage of a canned metakit database file (test_db.mk). It also provides a "Start" and "Stop" button. After you define a directory and press "Start", here's what happens: - A metakit file is created in the specified directory with a single view (test) containng 2 properties ("user" and "timestamp") - Now, every 200 milliseconds, it either adds or updates a record in the database based on the following: - It randomly picks a number between 1 and 100 - If that record already exists in the database, it is updated (with a new timestamp) - If the selected record number does not exist, a new record is appended to the database - So, ultimately, once 100 records have been added, all further changes are updates - Once a record is written, it is immediately read back from the database - The "Written" and the "Read" records are displayed on the screen for inspection - If you "Stop" the program, the database file is closed and the program waits for you to "Start" it again. - You can "Start" and "Stop" the program as many times as you like in a single run Now, for the "procedure" required to corrupt the database... - First, if the database file is stored on the "local" system, I *cannot*, under any circumstances, cause a corruption - The metakit file must be stored on the networked system to cause corruption, but only under certain circumstances: - If the program is run from "System A", and the file is stored on "System B", and nothing else is done - all seems to be OK. That is, (I don't think) I've ever seen corruption this way. - OK, here's where it gets weird.... - Open a Windows Explorer Window and point it at the directory the Metakit file is (will be) stored in. In my experience, this can be done from *either* system. - Now, start the program. Everything seems to run fine until I *refresh* the Window Explorer view (F5). Once the view is refreshed - either immediately (or very shortly thereafter), the database is corrupted ( (some of) the "Read" lines no longer match the "Written" lines). - I don't know exactly why this causes a problem. It seems to be somehow related to another process needing to collect file attribute information about the Metakit file. - We also use MKS Toolkit (a "Unix-like" environment for Windows) in house. Just for grins, I opened a shell window in the directory containing the "Metakit" file. An "ls" on the directory doesn't appear to cause problems, but an "ls -l" (while the test program is running) seems to cause the file to be corrupted. I don't know exactly what Win API's the MKS "ls -l" command is using, but I assume that it is somehow accessing file attribute information - which seems to cause the corruption. Now, back in my *real* application, I can cause corruption over the network without all "voodoo" required in the above scenario. That is, it seems able to corrupt the data all on it's own, but not in a reproducible manner. The above is the best way I have discovered to "see" the problem. So, does any of the above mean anything to anyone? - because I'm really grasping at straws here. I would *really* appreciate knowing if anyone can reproduce the described behavior - at least then I would know that I'm not going crazy. Any additional input, ideas for further testing, or just words of encouragement ;^) are greatly appreciated.... Thanks, Jeff ------=_NextPart_000_01E3_01C31B99.82F5D100 Content-Type: application/octet-stream; name="db_stress_test2.tcl" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="db_stress_test2.tcl" package provide app-db_stress_test2 1.0 package require Mk4tcl proc main {} { console show buildUI #openDB ./ #defineViews } proc buildUI {} { set f1 [frame .f1] set f2 [frame .f2] set lb1 [label $f1.lb1 -text "Output Directory:"] set en1 [entry $f1.en1 -width 40] set btn1 [button $f1.b1 -width 10 -text "Browse..." -command [list = getDir $en1]] set btn2 [button $f2.b2 -width 10 -text "Start" -command [list start = $en1]] set btn3 [button $f2.b3 -width 10 -text "Stop" -command stop] pack $f1 $f2 -fill x -pady 2 pack $lb1 $en1 $btn1 -side left -padx 2 -pady 2 pack $btn3 $btn2 -side right -padx 2 -pady 2 } proc getDir {w} { set file [tk_chooseDirectory] if {$file ne ""} { $w delete 0 end $w insert 0 $file } } proc start {w} { set ::stop 0 set dir [$w get] if {$dir eq ""} { set dir "./" } if {![file isdirectory $dir]} { tk_messageBox \ -message "Directory \"$dir\" does not exist" \ -type ok \ -icon error return } set dbFile [file join $dir "test_db.mk"] openDB $dbFile defineViews loop } proc loop {} { if {!$::stop} { addData after 200 loop } } proc stop {} { set ::stop 1 catch {mk::file close db} } proc random_int {upperLimit {lowerLimit 1}} { expr {int(rand()*($upperLimit-$lowerLimit+1)+$lowerLimit)} } proc openDB {dbFile} { if {[file exists $dbFile]} { set resp [tk_messageBox \ -title "DB File Exists" \ -message "$dbFile already exists\n\nDelete it?" \ -type yesno \ -icon warning] if {$resp eq "yes"} { catch {file delete -force $dbFile} puts "\n***** Deleted $dbFile *****\n" } } mk::file open db $dbFile } proc defineViews {} { mk::view layout db.test {user timestamp} } proc addData {} { set view "db.test" # get a random index set index [random_int 100] if {[mk::view size $view] <=3D $index} { set thisCursor [mk::row append $view] set word "Added " } else { mk::cursor create thisCursor $view $index set word "Updated" } set writeData [list user "Test User" timestamp [clock seconds]] eval mk::set $thisCursor $writeData mk::file commit db set readData [mk::get $thisCursor] set recNo [lindex [split $thisCursor "!"] end] puts "$word recNo $recNo --> $writeData" puts "Read recNo $recNo --> $readData" puts "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D" } main ------=_NextPart_000_01E3_01C31B99.82F5D100-- From mmartin6@cfl.rr.com Fri May 16 12:05:47 2003 Received: from orllnx01.orl.rbd.com (orllnx01.orl.rbd.com [208.61.212.238]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4GH5lN22528 for ; Fri, 16 May 2003 12:05:47 -0500 Received: (qmail 15447 invoked from network); 16 May 2003 17:06:16 -0000 Received: from host252.orl.rbd.com (HELO cfl.rr.com) (208.61.212.252) by orllnx01.orl.rbd.com with SMTP; 16 May 2003 17:06:16 -0000 Message-ID: <3EC51986.70902@cfl.rr.com> Date: Fri, 16 May 2003 13:01:58 -0400 From: Marcus Martin User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030416 Thunderbird/0.1a X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jeff Godfrey CC: Jean-Claude Wippler , Metakit mailing list Subject: Re: [Metakit] Mk4tcl - corrupted database over NT network - update [LONG] References: <705441CC-8640-11D7-9E32-000A957664F8@equi4.com> <01e601c31bc3$7c29e940$0101a8c0@quantex> In-Reply-To: <01e601c31bc3$7c29e940$0101a8c0@quantex> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > - Now, start the program. Everything seems to run fine until I >*refresh* the Window Explorer view (F5). Once the view is refreshed - >either immediately (or very shortly thereafter), the database is corrupted >( (some of) the "Read" lines no longer match the "Written" lines). > - I don't know exactly why this causes a problem. It seems to be >somehow related to another process needing to collect file attribute >information about the Metakit file. > - We also use MKS Toolkit (a "Unix-like" environment for Windows) in >house. Just for grins, I opened a shell window in the directory containing >the "Metakit" file. An "ls" on the directory doesn't appear to cause >problems, but an "ls -l" (while the test program is running) seems to cause >the file to be corrupted. I don't know exactly what Win API's the MKS >"ls -l" command is using, but I assume that it is somehow accessing file >attribute information - which seems to cause the corruption. > I am not sure if this will help, but I do know, from experimentation that when you hit F5 on a Windows Explorer view, it opens every file in the directory and reads in the first X bytes to determine what type of program file it is associated with if the file has an extension that is not in the known file extension list for your wndows isntallation. I do not see this by itself causing coruption, but it may play a contributing factor. Marcus From jcw@equi4.com Fri May 16 12:06:25 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4GH6ON22630 for ; Fri, 16 May 2003 12:06:25 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 855588396D for ; Fri, 16 May 2003 13:06:57 -0400 (EDT) Date: Fri, 16 May 2003 19:06:59 +0200 Subject: Re: [Metakit] Mk4tcl - corrupted database over NT network - update [LONG] Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <01e601c31bc3$7c29e940$0101a8c0@quantex> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: > After 2 days of (very confusing) testing, I *think* I have a tcl > script and > a "procedure" to consistently reproduce my database corruption. [... valuable info ...] > Any additional input, ideas for further testing, or just words of > encouragement ;^) are greatly appreciated.... Very quick first reaction - kudo's for diving in: "When the going gets tough, the tough get going" I'll have a good look at this. I don't have two NT boxes (but I can create two virtual ones with vmware, if need be). Will see to what extent this matters. Your assertion that local mode use never corrupts is good, at least not the entire building is caving in on us! -jcw From schlenk@uni-oldenburg.de Fri May 16 13:14:29 2003 Received: from mailout11.sul.t-online.com (mailout11.sul.t-online.com [194.25.134.85]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4GIESN29839 for ; Fri, 16 May 2003 13:14:28 -0500 Received: from fwd06.sul.t-online.de by mailout11.sul.t-online.com with smtp id 19GjjX-0004Wm-01; Fri, 16 May 2003 20:15:07 +0200 Received: from pD9526E6A.dip.t-dialin.net (520047057634-0001@[217.229.97.126]) by fmrl06.sul.t-online.com with esmtp id 19GjjS-0DIwimC; Fri, 16 May 2003 20:15:02 +0200 Received: from uni-oldenburg.de (unknown [192.168.2.2]) by pD9526E6A.dip.t-dialin.net (Postfix) with ESMTP id 59F577BEF; Thu, 26 Apr 2001 23:54:39 +0000 (/etc/localtime) Message-ID: <3EC52B77.2000600@uni-oldenburg.de> Date: Fri, 16 May 2003 20:18:31 +0200 From: Michael Schlenker Organization: =?ISO-8859-1?Q?Universit=E4t_Oldenburg?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2a) Gecko/20020910 X-Accept-Language: de-de, en-us MIME-Version: 1.0 To: Jeff Godfrey Cc: Metakit mailing list Subject: Re: [Metakit] Mk4tcl - corrupted database over NT network - update [LONG] References: <705441CC-8640-11D7-9E32-000A957664F8@equi4.com> <01e601c31bc3$7c29e940$0101a8c0@quantex> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Sender: 520047057634-0001@t-dialin.net Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: >Hi All, > > >Now, for the "procedure" required to corrupt the database... > >- First, if the database file is stored on the "local" system, I *cannot*, >under any circumstances, cause a corruption >- The metakit file must be stored on the networked system to cause >corruption, but only under certain circumstances: > - If the program is run from "System A", and the file is stored on >"System B", and nothing else is done - all seems to be OK. That is, (I >don't think) I've ever seen corruption this way. > - OK, here's where it gets weird.... > - Open a Windows Explorer Window and point it at the directory the >Metakit file is (will be) stored in. In my experience, this can be done >from *either* system. > - Now, start the program. Everything seems to run fine until I >*refresh* the Window Explorer view (F5). Once the view is refreshed - >either immediately (or very shortly thereafter), the database is corrupted >( (some of) the "Read" lines no longer match the "Written" lines). > - I don't know exactly why this causes a problem. It seems to be >somehow related to another process needing to collect file attribute >information about the Metakit file. > - We also use MKS Toolkit (a "Unix-like" environment for Windows) in >house. Just for grins, I opened a shell window in the directory containing >the "Metakit" file. An "ls" on the directory doesn't appear to cause >problems, but an "ls -l" (while the test program is running) seems to cause >the file to be corrupted. I don't know exactly what Win API's the MKS >"ls -l" command is using, but I assume that it is somehow accessing file >attribute information - which seems to cause the corruption. > >Now, back in my *real* application, I can cause corruption over the network >without all "voodoo" required in the above scenario. That is, it seems able >to corrupt the data all on it's own, but not in a reproducible manner. The >above is the best way I have discovered to "see" the problem. > >So, does any of the above mean anything to anyone? - because I'm really >grasping at straws here. I would *really* appreciate knowing if anyone can >reproduce the described behavior - at least then I would know that I'm not >going crazy. > > > Maybe on your server the Microsoft index service is running (it is by default) and opens you metakit database in a similar way similar to your experiments. This would cause random corruption. Michael Schlenker From jeff_godfrey@pobox.com Fri May 16 14:16:37 2003 Received: from dewey.iland.net (dewey.iland.net [205.242.230.201]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4GJGbN03547 for ; Fri, 16 May 2003 14:16:37 -0500 Received: (qmail 8808 invoked from network); 16 May 2003 19:17:06 -0000 Received: from wbrg-dialup-216-74-201-15.iland.net (HELO quantex) (216.74.201.15) by dewey.iland.net with SMTP; 16 May 2003 19:17:06 -0000 Message-ID: <030f01c31bdf$92611a00$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Marcus Martin" Cc: "Metakit mailing list" References: <705441CC-8640-11D7-9E32-000A957664F8@equi4.com> <01e601c31bc3$7c29e940$0101a8c0@quantex> <3EC51986.70902@cfl.rr.com> Subject: Re: [Metakit] Mk4tcl - corrupted database over NT network - update [LONG] Date: Fri, 16 May 2003 14:15:27 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Marcus Martin" To: "Jeff Godfrey" Cc: "Jean-Claude Wippler" ; "Metakit mailing list" Sent: Friday, May 16, 2003 12:01 PM Subject: Re: [Metakit] Mk4tcl - corrupted database over NT network - update [LONG] > I am not sure if this will help, but I do know, from experimentation > that when you hit F5 on a Windows Explorer view, it opens every file in > the directory and reads in the first X bytes to determine what type of > program file it is associated with if the file has an extension that is > not in the known file extension list for your wndows isntallation. I do > not see this by itself causing coruption, but it may play a contributing > factor. > > Marcus Marcus, That's all news to me. I'll keep it in mind. Thanks, Jeff From jeff_godfrey@pobox.com Fri May 16 16:02:33 2003 Received: from hewey.iland.net (qmailr@hewey.iland.net [205.242.230.200]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4GL2XN13996 for ; Fri, 16 May 2003 16:02:33 -0500 Received: (qmail 12123 invoked from network); 16 May 2003 21:02:43 -0000 Received: from wbrg-dialup-216-74-201-15.iland.net (HELO quantex) (216.74.201.15) by hewey.iland.net with SMTP; 16 May 2003 21:02:43 -0000 Message-ID: <039101c31bee$58a54b60$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Metakit mailing list" References: <705441CC-8640-11D7-9E32-000A957664F8@equi4.com> <01e601c31bc3$7c29e940$0101a8c0@quantex> <3EC51986.70902@cfl.rr.com> <030f01c31bdf$92611a00$0101a8c0@quantex> Date: Fri, 16 May 2003 16:01:11 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Subject: [Metakit] Mk4tcl - corrupted database - update Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I've got some updated news, but I give it with this note of caution: So far, this is very much a moving target - probably because I don't yet understand exactly what factors ultimately contribute to the problem (and I'm running out of ideas *to* understand the factors). As I am not entirely confident in the soundness of this info I want to be cautious, as I don't want to misguide anyone elses efforts... 1. So far, all corruptions seem to have happened when the test program is *running on* a system using NT4.0 service pack *6*. We have done limited testing on several systems running service pack *5*, and so far have not caused any corruption. I want to be clear here - the metakit file is always stored (in our testing) over the network on a system running service pack *6*, but the program has been run under both service pack 5 and 6. When running under 5, no corruption has yet occurred. 2. Another way to (fairly) consistently corrupt the file seems to be the following: On the system where the metakit file is local, double-click the file in Windows Explorer (while the test program is running from the other system). Assuming the "mk" extension is not associated with any app, Windows will open a "Open With" dialog. Just quit from there (don't actually open the dialog). Immediately, or very shortly thereafter, the test program will begin to read garbage from the file... Has anyone managed to recreate the described problem? I'm very concerned that with the apparent "goofy-ness" of my scenarios, it may be difficult or impossible to reproduce elsewhere... Thanks again, Jeff From jcw@equi4.com Fri May 16 16:37:40 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4GLbeN17837 for ; Fri, 16 May 2003 16:37:40 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 561B68396D for ; Fri, 16 May 2003 17:38:12 -0400 (EDT) Date: Fri, 16 May 2003 23:38:12 +0200 Subject: Re: [Metakit] Mk4tcl - corrupted database - update Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <039101c31bee$58a54b60$0101a8c0@quantex> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff, > I've got some updated news, but I give it with this note of caution: Yah, until we figure this one out, all bets are off. > 1. [NT4 SP5 vs SP6] Other factors can come in here: config diffs, even CPU speed diffs. I'm still not ruling out lunar eclipses. > 2. Another way to (fairly) consistently corrupt the file seems to be > the > following: On the system where the metakit file is local, > double-click the > file in Windows Explorer (while the test program is running from the > other > system). Assuming the "mk" extension is not associated with any app, > Windows will open a "Open With" dialog. Just quit from there (don't > actually open the dialog). Immediately, or very shortly thereafter, > the > test program will begin to read garbage from the file... This is a serious hint that memory mapping on networked file systems is not working properly. Several years ago, the first problem of this kind appeared in 95/98, hinting that mmaps were internally faked with reads/writes to the file descriptor. The conclusion then was that when writing to a file from a buffer which was in the memory map, things could go sour because the write messed with the file pointer. A "real" mmap does not care one bit about file pointers, but apparently Win9x was actually doing reads (from the wrong spot, as I found out). The bug has a long standing workaround, now used for all Win32 systems (also to avoid adding OS version detection code): copy to a temp buf, and write from there. See line 389 and on in src/fileio.cpp for details. The key was to copy to temp, and *then* seek to the write spot, because once in a while the "mmap read" affected the write position (!). The same workaround "fixes" hard system deadlocks on hpux and osx, albeit at a cost in commit performance. It now looks like Windows is again messing with a file pointer, but from another process accessing the file locally (huh???). And it's affecting the mmaps or reading/writing of MK data to that file via file server calls. If this turns out to be the correct analysis, and if a workaround does not present itself, then I think the only option left is to switch back to non-memory mapped files. This is such a terrible step back w.r.t. performance and memory consumption in many cases, that I see no other way than to expose this choice through the MK API and let people themselves indicate when safe, ehm, "commit" needs to be practiced. Yuck. The best answer is to never put MK datafiles on a file server, but I suppose one does not always have that choice... One thing I don't yet fully understand: you've described how even *reads* can mess things up, but what about never touching the file from more than one spot at a time, i.e. guarding things with a lock and not even opening/reading the file? In all you've described - it seems that would imply no problems could occur? > Has anyone managed to recreate the described problem? I'm very > concerned > that with the apparent "goofy-ness" of my scenarios, it may be > difficult or > impossible to reproduce elsewhere... We'll see. Lots of pieces in this puzzle. Thanks for the very detailed searches, trials, and information. -jcw From jeff_godfrey@pobox.com Fri May 16 17:00:31 2003 Received: from hewey.iland.net (qmailr@hewey.iland.net [205.242.230.200]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4GM0VN20226 for ; Fri, 16 May 2003 17:00:31 -0500 Received: (qmail 30187 invoked from network); 16 May 2003 22:00:57 -0000 Received: from wbrg-dialup-216-74-201-15.iland.net (HELO quantex) (216.74.201.15) by hewey.iland.net with SMTP; 16 May 2003 22:00:57 -0000 Message-ID: <03c401c31bf6$7a5e21c0$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: "Metakit mailing list" References: Subject: Re: [Metakit] Mk4tcl - corrupted database - update Date: Fri, 16 May 2003 16:59:25 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/14/03), Outbound message X-Antivirus-Status: Clean Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ----- Original Message ----- From: "Jean-Claude Wippler" To: "Metakit mailing list" Sent: Friday, May 16, 2003 4:38 PM Subject: Re: [Metakit] Mk4tcl - corrupted database - update > If this turns out to be the correct analysis, and if a workaround does > not present itself, then I think the only option left is to switch back > to non-memory mapped files. This is such a terrible step back w.r.t. > performance and memory consumption in many cases, that I see no other > way than to expose this choice through the MK API and let people > themselves indicate when safe, ehm, "commit" needs to be practiced. Right now, I'd welcome the ability to "turn off the problem", even if it means a (large) performance penalty. But, for the general case, I definitely understand your concerns... > Yuck. The best answer is to never put MK datafiles on a file server, > but I suppose one does not always have that choice... Agreed. Unfortunately, I think that is *very* limiting - and as you stated, not always under the developer's control. > One thing I don't yet fully understand: you've described how even > *reads* can mess things up, but what about never touching the file from > more than one spot at a time, i.e. guarding things with a lock and not > even opening/reading the file? In all you've described - it seems that > would imply no problems could occur? OK, I'm not sure I understand this one. I could manage the single access at a time issue within my 2 separate programs if need be (although even that doesn't sound like fun), but.... If "OS related" operations (like refreshing the Explorer window or attempting to open the file with an external application) really *are* causing problems like they appear to be - how can I prevent that? > We'll see. Lots of pieces in this puzzle. Yes, unfortunately more all the time... > Thanks for the very detailed searches, trials, and information. Your welcome. Thank you (and everyone else) for lending a hand, as I'm really in *way* over my head... ;^) Jeff From niki@vintech.bg Sat May 17 05:39:02 2003 Received: from intra.vintech.l (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4HAd0N01370 for ; Sat, 17 May 2003 05:39:00 -0500 Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id 447A056CC6; Sat, 17 May 2003 12:43:59 +0300 (EEST) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id 5700B22C39; Sat, 17 May 2003 12:39:34 +0300 (EEST) Message-ID: <3EC6113E.7000908@vintech.bg> Date: Sat, 17 May 2003 13:38:54 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: bg, en MIME-Version: 1.0 To: Jeff Godfrey Cc: Metakit mailing list Subject: Re: [Metakit] Mk4tcl - corrupted database - update References: <03c401c31bf6$7a5e21c0$0101a8c0@quantex> In-Reply-To: <03c401c31bf6$7a5e21c0$0101a8c0@quantex> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: > > OK, I'm not sure I understand this one. I could manage the single access at > a time issue within my 2 separate programs if need be (although even that > doesn't sound like fun), but.... If "OS related" operations (like > refreshing the Explorer window or attempting to open the file with an > external application) really *are* causing problems like they appear to be - > how can I prevent that? Use maximum possible locking e.g. sopen( ..., _SH_DENYRW ) Maybe SMB oplock code is brocken or badly configured in some WinNT versions. Niki Spahiev From Conor.ODoherty@ace-hellas.gr Tue May 20 17:45:28 2003 Received: from logos.hol.gr (logos.hol.gr [194.30.192.15]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4KMjQN08360 for ; Tue, 20 May 2003 17:45:27 -0500 Received: from smtp.hol.gr (aperion.mail.dc.hol.net [192.168.20.34]) by logos.hol.gr (8.11.6/8.11.6) with ESMTP id h4KMjgA19871 for ; Wed, 21 May 2003 01:45:42 +0300 Received: from atlas.ace-hellas.gr (atlas.ace-hellas.gr [195.97.19.130]) by smtp.hol.gr (8.11.6/8.11.6) with ESMTP id h4KMjfw23954 for ; Wed, 21 May 2003 01:45:41 +0300 Received: from localhost (localhost [127.0.0.1]) by atlas.ace-hellas.gr (8.12.8/linuxconf) with ESMTP id h4KMjdtm017007 for ; Wed, 21 May 2003 01:45:40 +0300 Received: from mfdev ([192.168.2.165] [192.168.2.165]) by atlas.ace-hellas.gr (AvMailGate-2.0.1.11) id 16997-5A96597D; Wed, 21 May 2003 01:45:39 +0300 From: "Conor O'Doherty" To: Date: Wed, 21 May 2003 01:43:58 +0300 Message-ID: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0001_01C31F3A.723B50C0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 X-AntiVirus: checked by AntiVir Milter 1.0.2; AVE 6.19.0.3; VDF 6.19.0.19 X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.11; AVE: 6.19.0.3; VDF: 6.19.0.19; host: atlas.ace-hellas.gr) X-RAVMilter-Version: 8.3.3(snapshot 20020312) (aperion) Subject: [Metakit] WinCE complilation error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0001_01C31F3A.723B50C0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit Just downloaded metakit for the first time to test it. I tried to compile the Windows CE version and gave an error at line:226 in file:string.ccp. I read a post from tp@diffenbach.org in the archive, who seem to have the same problem. The question is has this been address or when will this be addressed can anybody tell me. Here is the line in question and its error !!!! return Data() == str ? 0 : strcasecmp(Data(), str); error C2065: 'stricmp' : undeclared identifier Kind Regards Conor ACE-Hellas AE Conor O’Doherty Chief Programmer Tel : +30 (210) 6068 678 Email : Conor.ODoherty@ace-hellas.gr ICQ : 146827126 http://www.ace-hellas.gr ------=_NextPart_000_0001_01C31F3A.723B50C0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Just = downloaded=20 metakit for the first time to test it.
I = tried to compile=20 the Windows CE version and gave an error at line:226 in file:string.ccp. =
I = read a post from tp@diffenbach.org in = the archive,=20 who seem to have the same problem.
The question is has this been = address or=20 when will this be addressed can anybody tell=20 me.
Here is the line in question = and its error=20 !!!!
 
    return Data()=20 =3D=3D str ? 0 : strcasecmp(Data(), = str);
   =20 error C2065: 'stricmp' : undeclared identifier
 
Kind=20 Regards
Conor
 
 
 

ACE-Hellas AE

 

Conor = O=92Doherty

Chief=20 Programmer

Tel : +30 = (210) 6068=20 678

Email : Conor.ODoherty@ace-hellas.gr=

ICQ : 146827126 

 

http://www.ace-hellas.gr

 
------=_NextPart_000_0001_01C31F3A.723B50C0-- From jeff_godfrey@pobox.com Fri May 23 08:33:04 2003 Received: from dewey.iland.net (dewey.iland.net [205.242.230.201]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4NDX4N17997 for ; Fri, 23 May 2003 08:33:04 -0500 Received: (qmail 18072 invoked from network); 23 May 2003 13:33:45 -0000 Received: from wbrg-dialup-216-74-200-117.iland.net (HELO quantex) (216.74.200.117) by dewey.iland.net with SMTP; 23 May 2003 13:33:45 -0000 Message-ID: <00fa01c3212f$b3e64ce0$0101a8c0@quantex> Reply-To: "Jeff Godfrey" From: "Jeff Godfrey" To: Date: Fri, 23 May 2003 08:31:37 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Antivirus: avast! (VPS 5/19/03), Outbound message X-Antivirus-Status: Clean Subject: [Metakit] Metakit - mk4tcl db corruption - request to list members Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I am still having on-going troubles with metakit file corruption under WinNT when the storage file is created on a networked system. I created (and attached) a tcl based test program and listed a series of steps to a post sent to this list on 16-May-03, under a similar subject heading. Using that script and the attached information, I am able to faithfully re-create the described problem. I have had this corruption happen both on the network at my place of business, as well as a customer's network. I am working with JC on a few work-around ideas (actually, I am just testing JC's ideas...), but it would be really nice to find and fix this problem in the metakit source... So far, I have been given no confirmation that anyone else has even seen the problem. So, I am asking for volunteers (with access to an NT network) to run my test program and follow the list of steps in order to see how "reproducible" (?) this problem is. Again, I can make it happen on 2 separate NT networks, so I hope others can see it also. Currently, JC does not have access to 2 networked NT systems to accurately test for the problem. So, if anyone can try this out and pass the results on , I would *really* appreciate it. Also, if you are able to see the problem, any insight you might have would be great. If you need me to repost (or email directly) the test program and the list of steps, I'd be happy to. Just ask... Thanks, Jeff From mbloore@yahoo.com Fri May 23 11:22:16 2003 Received: from web14309.mail.yahoo.com (web14309.mail.yahoo.com [216.136.224.59]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4NGMGN02179 for ; Fri, 23 May 2003 11:22:16 -0500 Message-ID: <20030523162301.25734.qmail@web14309.mail.yahoo.com> Received: from [65.95.156.7] by web14309.mail.yahoo.com via HTTP; Fri, 23 May 2003 09:23:01 PDT Date: Fri, 23 May 2003 09:23:01 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] Metakit - mk4tcl db corruption To: metakit@equi4.com In-Reply-To: <00fa01c3212f$b3e64ce0$0101a8c0@quantex> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i seem to have run across this problem in my own app just yesterday. i go through a directory of files, indexing info about them. i start with a fresh MK file on a shared volume, and do a commit every 1000 files. i get a crash on the second commit, every time. an MK routine that gets a string from the DB fails on an assertion that the string is zero-terminated. the string it appears to be looking for exists, but the pointer it uses points elsewhere in memory (to just before a string that describes the main view, if that means anything). doing a small set of files, involving only one commit, is OK. on a local disk, i can index 200,000 files without a problem. ===== -- mARK bLOORE __________________________________ Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. http://search.yahoo.com From jyl@mod3.net Fri May 23 12:15:32 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4NHFVN07627 for ; Fri, 23 May 2003 12:15:31 -0500 Received: (qmail 7977 invoked by uid 48); 23 May 2003 17:14:08 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Fri, 23 May 2003 10:14:08 -0700 (PDT) Message-ID: <64936.209.233.24.218.1053710048.squirrel@mod3.net> Date: Fri, 23 May 2003 10:14:08 -0700 (PDT) Subject: Re: [Metakit] Metakit - mk4tcl db corruption From: "Jacob Levy" To: In-Reply-To: <20030523162301.25734.qmail@web14309.mail.yahoo.com> References: <00fa01c3212f$b3e64ce0$0101a8c0@quantex> <20030523162301.25734.qmail@web14309.mail.yahoo.com> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark What OS is the disk shared from? With what OS are you running the indexer? Are you running Metakit 2.4.9.2, the latest? --JYL > i seem to have run across this problem in my own app just yesterday. i > go through a directory of files, indexing info about them. i start with > a fresh MK file on a shared volume, and do a commit every 1000 files. i > get a crash on the second commit, every time. an MK routine that gets a > string from the DB fails on an assertion that the string is > zero-terminated. the string it appears to be looking for exists, but > the pointer it uses points elsewhere in memory (to just before a string > that describes the main view, if that means anything). > > doing a small set of files, involving only one commit, is OK. > > on a local disk, i can index 200,000 files without a problem. > > ===== > -- > mARK bLOORE > > __________________________________ > Do you Yahoo!? > The New Yahoo! Search - Faster. Easier. Bingo. > http://search.yahoo.com > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From mbloore@yahoo.com Fri May 23 13:29:03 2003 Received: from web14306.mail.yahoo.com (web14306.mail.yahoo.com [216.136.173.82]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4NIT2N15024 for ; Fri, 23 May 2003 13:29:02 -0500 Message-ID: <20030523182947.43279.qmail@web14306.mail.yahoo.com> Received: from [65.95.156.7] by web14306.mail.yahoo.com via HTTP; Fri, 23 May 2003 11:29:47 PDT Date: Fri, 23 May 2003 11:29:47 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] Metakit - mk4tcl db corruption To: metakit@equi4.com In-Reply-To: <20030523162301.25734.qmail@web14309.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: hmm, some important info seems to be missing! my app is running on win2k. the shared volume is on a freeBSD machine, using samba to share files with windows. i am using MK 2.4.9.2; i started with an earlier version, but i don't think we ever used a shared volume before, at least not doing more than one commit per indexing session. i have just tried it with 2.4.7, which gave the same result. my description isn't quite right: the second commit succeeds; the error happens processing the next file, which involves searching the database for the directory it is in (which searches for the path string). it is the very next DB access that fails, a Find() on a string in an ordinary view of the DB. (i don't use any derived views.) --- mARK bLOORE wrote: > i seem to have run across this problem in my own app just yesterday. > i > go through a directory of files, indexing info about them. i start > with a fresh MK file on a shared volume, and do a commit every 1000 > files. i get a crash on the second commit, every time. an MK > routine > that gets a string from the DB fails on an assertion that the string > is > zero-terminated. the string it appears to be looking for exists, but > the pointer it uses points elsewhere in memory (to just before a > string > that describes the main view, if that means anything). > > doing a small set of files, involving only one commit, is OK. > > on a local disk, i can index 200,000 files without a problem. > > ===== > -- > mARK bLOORE > > __________________________________ > Do you Yahoo!? > The New Yahoo! Search - Faster. Easier. Bingo. > http://search.yahoo.com > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________ Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. http://search.yahoo.com From lista@nakawe.se Fri May 23 23:15:40 2003 Received: from m3.azalea.se (m3.azalea.se [217.75.96.207]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4O4FdN19850 for ; Fri, 23 May 2003 23:15:40 -0500 Received: (qmail 7953 invoked by uid 0); 24 May 2003 04:16:26 -0000 Received: from unknown (HELO webmail.nakawe.se) (127.0.0.1) by m3.azalea.se with SMTP; 24 May 2003 04:16:26 -0000 Received: from 195.190.202.212 (SquirrelMail authenticated user lista@nakawe.se) by webmail.azalea.se with HTTP; Sat, 24 May 2003 06:16:26 +0200 (CEST) Message-ID: <34387.195.190.202.212.1053749786.squirrel@webmail.azalea.se> Date: Sat, 24 May 2003 06:16:26 +0200 (CEST) Subject: [Fwd: Re: [Metakit] Metakit - mk4tcl db corruption - request to list members] From: lista@nakawe.se To: metakit@equi4.com User-Agent: SquirrelMail/1.4.0 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ---------------------------- Original Message ---------------------------- Subject: Re: [Metakit] Metakit - mk4tcl db corruption - request to list members Wrom: OQKEDOTWFAOBUZXUWL Date: Sat, May 24, 2003 6:13 am To: "Jeff Godfrey" -------------------------------------------------------------------------- I should be able to check this after I get home from Tcl'europe (june 3rd). I have just aquired a used laptop and installed NT4 on it. I run w2k on my desktop so that should qualify as 2 NT, and I need to set up networking anyways so it's no extra work for me. But as I said I can't do anything for about a week. - Veronica Loell > Hi All, > > I am still having on-going troubles with metakit file corruption under WinNT > when the storage file is created on a networked system. I created (and attached) a tcl based test program and listed a series of steps to a post sent to this list on 16-May-03, under a similar subject heading. Using that > script and the attached information, I am able to faithfully re-create the described problem. I have had this corruption happen both on the network at > my place of business, as well as a customer's network. > > I am working with JC on a few work-around ideas (actually, I am just testing > JC's ideas...), but it would be really nice to find and fix this problem in > the metakit source... > > So far, I have been given no confirmation that anyone else has even seen the > problem. So, I am asking for volunteers (with access to an NT network) to run my test program and follow the list of steps in order to see how "reproducible" (?) this problem is. Again, I can make it happen on 2 separate NT networks, so I hope others can see it also. Currently, JC does > not have access to 2 networked NT systems to accurately test for the problem. > > So, if anyone can try this out and pass the results on , I would *really* appreciate it. Also, if you are able to see the problem, any insight you might have would be great. > > If you need me to repost (or email directly) the test program and the list of steps, I'd be happy to. Just ask... > > Thanks, > > > Jeff > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From angus@europeonline.net Wed May 28 09:37:08 2003 Received: from unicorn.europeonline.net (unicorn.europeonline.net [194.177.33.81]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4SEb7N18454 for ; Wed, 28 May 2003 09:37:07 -0500 Received: from groovinhigh (pppoe231-luxdsl-031.pt.lu [213.135.231.31]) (authenticated bits=0) by unicorn.europeonline.net (8.12.9/8.12.9) with ESMTP id h4SEbtVE068390 for ; Wed, 28 May 2003 16:37:56 +0200 (CEST) (envelope-from angus@europeonline.net) From: "Angus Lord" To: Date: Wed, 28 May 2003 16:37:55 +0200 Message-ID: <0b6a01c32526$ba0dfe10$196c0550@groovinhigh> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h4SEb7N18454 Subject: [Metakit] Redundant data Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm using metakit partly as structured storage for my app. Setting up the database is fine, however if I modify one of the entries, then I get a copy of the data stored in the file (only once, I can modify it many times). This is fine for small bits of data, but if I am storing and modifying files (a few kb) then I am potentially wasting a lot of space. Is there any way to turn this feature off? For a single record, this can be tested with the code (C++) c4_StringProp pTempData("data"); c4_Storage storage("myfile.dat",true); c4_Row row; pTempData(row) = "test"; c4_View vTest = storage.GetAs("test[data:S]"); if (vTest.GetSize() >= 1) vTest.SetSize(0); vTest.Add(row); storage.Commit(); Cheers Angus From jcw@equi4.com Wed May 28 09:55:51 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4SEtpN20772 for ; Wed, 28 May 2003 09:55:51 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 1BDA683AEA for ; Wed, 28 May 2003 10:56:35 -0400 (EDT) Date: Wed, 28 May 2003 16:56:34 +0200 Subject: Re: [Metakit] Redundant data Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Content-Transfer-Encoding: 7bit In-Reply-To: <0b6a01c32526$ba0dfe10$196c0550@groovinhigh> Message-Id: <931365F5-911C-11D7-8877-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Angus Lord wrote: > I'm using metakit partly as structured storage for my app. Setting up > the > database is fine, however if I modify one of the entries, then I get a > copy > of the data stored in the file (only once, I can modify it many > times). This > is fine for small bits of data, but if I am storing and modifying > files (a > few kb) then I am potentially wasting a lot of space. Is there any way > to > turn this feature off? You're seeing the consequences of "stable storage" - the mechanism which ensures commit/rollback robustness. Comes with Metakit, which is a database manager that will continue to function with a consistent dataset regardless of aborts and pulling plugs at the most awkward times. You can compress, by saving to a new file and switching over to it (see SaveTo). The space is not wasted or lost. It gets re-used later. -jcw From j.p.fletcher@aston.ac.uk Thu May 29 04:25:25 2003 Received: from mercury.aston.ac.uk (mercury.aston.ac.uk [134.151.79.53]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4T9PON08079 for ; Thu, 29 May 2003 04:25:25 -0500 Received: from [134.151.79.13] (helo=email.aston.ac.uk) by mercury.aston.ac.uk with esmtp (Exim 3.30 #1) id 19LJd1-0003Pp-00 for metakit@equi4.com; Thu, 29 May 2003 10:23:19 +0100 Received: from ceac133 ([134.151.50.133]) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 19LJfm-0002tu-00 for metakit@equi4.com; Thu, 29 May 2003 10:26:10 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Thu, 29 May 2003 10:21:16 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Priority: normal In-reply-to: <20030523182947.43279.qmail@web14306.mail.yahoo.com> References: <20030523162301.25734.qmail@web14309.mail.yahoo.com> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Subject: [Metakit] Installing Mk4tcl on Red Hat Linux 8.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am attempting to use Mk4tcl on a computer running Red Hat Linux. Just for now I am using Metakit 2.4.7 as I had already built it for C++ use. The problem is that the Tcl command package require Mk4tcl failed whereever I put the file, renamed from libmk4tcl.so to Mk4tcl.so I was doing that at home last night. This morning I have found some information in Mark Roseman's helpful notes on this http://www.markroseman.com/tcl/mktcl.html The problem seems to be that Red Hat Linux regards Tcl as part of the operating system and installs its packages in /usr/lib instead of /usr/local/lib The installation reports that tcl_pkgPath contains ONLY /usr/lib What can be done so that the test files which use the package equire command will work? Thanks John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From jcw@equi4.com Thu May 29 05:13:19 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TADJN10766 for ; Thu, 29 May 2003 05:13:19 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 982EA83AEA for ; Thu, 29 May 2003 06:14:03 -0400 (EDT) Date: Thu, 29 May 2003 12:14:03 +0200 Subject: Re: [Metakit] Installing Mk4tcl on Red Hat Linux 8.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <461DC260-91BE-11D7-8877-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John Fletcher wrote: > package require Mk4tcl > > failed whereever I put the file, renamed from libmk4tcl.so to > Mk4tcl.so [...] > The problem seems to be that Red Hat Linux regards Tcl as part of > the operating system and installs its packages in > > /usr/lib instead of /usr/local/lib > > The installation reports that tcl_pkgPath contains ONLY /usr/lib > > What can be done so that the test files which use the package > equire command will work? Set $auto_path accordingly. FYI, "package require" needs a pkgIndex.tcl file, it doesn't know how to load .so's, that's what "load" is for. -jcw From JHu@CoreLab.com Thu May 29 08:49:39 2003 Received: from exchangegw2.CORELAB.COM (pix.corelab.com [12.45.169.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TDndN24383 for ; Thu, 29 May 2003 08:49:39 -0500 Received: from VSMAIL1.CORELAB.COM ([172.16.3.27]) by exchangegw2.CORELAB.COM with Microsoft SMTPSVC(5.0.2195.5329); Thu, 29 May 2003 08:50:24 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C325E9.4123CDF7" Date: Thu, 29 May 2003 08:50:24 -0500 Message-ID: <875F35D078FEAD48A3CA5A32B5ACC06B1C73D4@VSMAIL1.CORELAB.COM> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Help on Int property Thread-Index: AcMl6UEx3QXHEKbGQQCv6+tHMjvi7g== From: "Jim Hu" To: X-OriginalArrivalTime: 29 May 2003 13:50:24.0831 (UTC) FILETIME=[41381CF0:01C325E9] Subject: [Metakit] Help on Int property Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------_=_NextPart_001_01C325E9.4123CDF7 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, all, I am just starting to use Metakit (2.9.4.2). I used the demo.cpp as = an example and wrote the following piece of code. When I use the dump to dump the result, the = integer part of values are all zeros. Where did I do wrong? Please advise. #include "mk4.h" #include "mk4str.h" #include main() { c4_StringProp pName ("name"); c4_IntProp pValue ("value"); c4_Storage storage("kdeck.dat", true); c4_View kdeck =3D storage.GetAs("kdeck[name:S, value:I]"); c4_Row row; pName (row) =3D "proc_cdp_first"; pValue(row) =3D 1; kdeck.Add(row); pName (row) =3D "proc_cdp_last"; pValue(row) =3D 676; storage.Commit(); int v1 =3D pValue(row); printf("V1 =3D %d\n", v1); kdeck.Add(row); int v2 (pValue (kdeck[1])); printf("V2 =3D %d\n", v2); pName (row) =3D "proc_cdp_inc"; pValue(row) =3D 1; kdeck.Add(row); storage.Commit(); return(0); } Sincerely, Jim Hu ------_=_NextPart_001_01C325E9.4123CDF7 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Help on Int property

Hi, all,

     I am just = starting to use Metakit (2.9.4.2). I used the demo.cpp as an example and = wrote
the following piece of code. When I = use the dump to dump the result, the integer part of values
are all zeros. Where did I do wrong? = Please advise.

#include "mk4.h"
#include "mk4str.h"

#include <stdio.h>

main()
{

       = c4_StringProp pName ("name");
       = c4_IntProp pValue ("value");
       = c4_Storage storage("kdeck.dat", true);
       = c4_View kdeck =3D storage.GetAs("kdeck[name:S, = value:I]");
       = c4_Row row;
       = pName (row) =3D "proc_cdp_first";
       = pValue(row) =3D 1;
       = kdeck.Add(row);
       = pName (row) =3D "proc_cdp_last";
       = pValue(row) =3D 676;
       = storage.Commit();
       = int v1 =3D pValue(row);
       = printf("V1 =3D %d\n", v1);
       = kdeck.Add(row);
       = int v2 (pValue (kdeck[1]));
       = printf("V2 =3D %d\n", v2);
       = pName (row) =3D "proc_cdp_inc";
       = pValue(row) =3D 1;
       = kdeck.Add(row);
       = storage.Commit();

       = return(0);
}

Sincerely,

Jim Hu

------_=_NextPart_001_01C325E9.4123CDF7-- From jeff@k2.com Thu May 29 09:03:34 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TE3YN25325 for ; Thu, 29 May 2003 09:03:34 -0500 Received: from Klee ([192.168.0.21]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h4TDucN00595; Thu, 29 May 2003 09:56:38 -0400 From: "Jeffrey Kay" To: "'Jim Hu'" Cc: Subject: RE: [Metakit] Help on Int property Date: Thu, 29 May 2003 10:02:58 -0400 Message-ID: <00f301c325eb$02c19850$017410ac@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal In-Reply-To: <875F35D078FEAD48A3CA5A32B5ACC06B1C73D4@VSMAIL1.CORELAB.COM> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: See if this simplifies what you are trying to do ... #include "mk4.h" #include "mk4str.h" #include main() { c4_StringProp pName ("name"); c4_IntProp pValue ("value"); c4_Storage storage("kdeck.dat", true); c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); kdeck.Add(pName["proc_cdp_first"] + pValue[1]); kdeck.Add(pName["proc_cdp_last"] + pValue[676]); storage.Commit(); for(int j=0;j pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein -----Original Message----- From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com] On Behalf Of Jim Hu Sent: Thursday, May 29, 2003 9:50 AM To: metakit@equi4.com Subject: [Metakit] Help on Int property Hi, all, I am just starting to use Metakit (2.9.4.2). I used the demo.cpp as an example and wrote the following piece of code. When I use the dump to dump the result, the integer part of values are all zeros. Where did I do wrong? Please advise. #include "mk4.h" #include "mk4str.h" #include main() { c4_StringProp pName ("name"); c4_IntProp pValue ("value"); c4_Storage storage("kdeck.dat", true); c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); c4_Row row; pName (row) = "proc_cdp_first"; pValue(row) = 1; kdeck.Add(row); pName (row) = "proc_cdp_last"; pValue(row) = 676; storage.Commit(); int v1 = pValue(row); printf("V1 = %d\n", v1); kdeck.Add(row); int v2 (pValue (kdeck[1])); printf("V2 = %d\n", v2); pName (row) = "proc_cdp_inc"; pValue(row) = 1; kdeck.Add(row); storage.Commit(); return(0); } Sincerely, Jim Hu From jcw@equi4.com Thu May 29 09:29:06 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TET6N27238 for ; Thu, 29 May 2003 09:29:06 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 0109883AEA for ; Thu, 29 May 2003 10:29:50 -0400 (EDT) Date: Thu, 29 May 2003 16:29:49 +0200 Subject: Re: [Metakit] Installing Mk4tcl on Red Hat Linux 8.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <010C7F0C-91E2-11D7-8877-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John Fletcher wrote: >> Set $auto_path accordingly. > > What does this do? Standard Tcl - please google for it or look at man pages. > Is there any way in which the installation can do the building of the > package index file? If not, then the demos and tests, which use > "package require" will all fail, unless instructions for how to get > round this are given as well. Yes, the pkgIndex.tcl file gets created by "make install", see the makefile for how/where it does that. The trouble right now is that something is broken in the 2.4.9.2 autoconf/libtool config (help!). I probably should not have followed someone's recent advice (in private email) to update to a newer autoconf/libtool combo :( > Alternatively, there could be some tests which use load instead. $ cd ../tcl/test $ tclsh all.tcl Processing 9 scripts... mk1basic mk2chan mk3struct mk4commit mk5object mk6fixed mk7limit mk8fail mk9crash Passed 33 tests $ > Incidentally, I did a build of 2.4.9.2 and I noticed that the > libmk4tcl.so and Mk4tcl.so files are not the same size, so it is not > just a case of renaming, as was said in the readme for 2.4.7. I > eventually found the built libaries in the hidden folder builds/.lib $ ls -l .libs/libmk4tcl.so -rwxr-xr-x 1 jcw users 1189052 May 29 16:09 .libs/libmk4tcl.so $ ls -l Mk4tcl.so -rwxr-xr-x 1 jcw users 340468 May 29 16:09 Mk4tcl.so $ cp -a .libs/libmk4tcl.so blah $ strip blah $ ls -l blah -rwxr-xr-x 1 jcw users 340468 May 29 16:25 blah $ -jcw From j.p.fletcher@aston.ac.uk Thu May 29 09:29:59 2003 Received: from mercury.aston.ac.uk (mercury.aston.ac.uk [134.151.79.53]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TETxN27304 for ; Thu, 29 May 2003 09:29:59 -0500 Received: from [134.151.79.13] (helo=email.aston.ac.uk) by mercury.aston.ac.uk with esmtp (Exim 3.30 #1) id 19LONm-00028F-00 for metakit@equi4.com; Thu, 29 May 2003 15:27:54 +0100 Received: from ceac133 ([134.151.50.133]) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 19LOQX-0006BM-00 for metakit@equi4.com; Thu, 29 May 2003 15:30:45 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Thu, 29 May 2003 15:25:52 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: (Fwd) Re: [Metakit] Installing Mk4tcl on Red Hat Linux 8.0 Priority: normal X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Subject: Re: [Metakit] Installing Mk4tcl on Red Hat Linux 8.0 From: Jean-Claude Wippler To: Metakit mailing list Date sent: Thu, 29 May 2003 12:14:03 +0200 > John Fletcher wrote: > > > package require Mk4tcl > > > > failed whereever I put the file, renamed from libmk4tcl.so to > > Mk4tcl.so > [...] > > The problem seems to be that Red Hat Linux regards Tcl as part of > > the operating system and installs its packages in > > > > /usr/lib instead of /usr/local/lib > > > > The installation reports that tcl_pkgPath contains ONLY /usr/lib > > > > What can be done so that the test files which use the package > > equire command will work? > > Set $auto_path accordingly. What does this do? > FYI, "package require" needs a > pkgIndex.tcl file, it doesn't know how to load .so's, that's what > "load" is for. > > -jcw > Is there any way in which the installation can do the building of the package index file? If not, then the demos and tests, which use "package require" will all fail, unless instructions for how to get round this are given as well. Alternatively, there could be some tests which use load instead. Incidentally, I did a build of 2.4.9.2 and I noticed that the libmk4tcl.so and Mk4tcl.so files are not the same size, so it is not just a case of renaming, as was said in the readme for 2.4.7. I eventually found the built libaries in the hidden folder builds/.lib Cheers John ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From JHu@CoreLab.com Thu May 29 10:00:26 2003 Received: from exchangegw1.CORELAB.COM (pix.corelab.com [12.45.169.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TF0QN29399 for ; Thu, 29 May 2003 10:00:26 -0500 Received: from VSMAIL1.CORELAB.COM ([172.16.3.27]) by exchangegw1.CORELAB.COM with Microsoft SMTPSVC(5.0.2195.5329); Thu, 29 May 2003 10:01:11 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: [Metakit] Help on Int property Date: Thu, 29 May 2003 10:01:11 -0500 Message-ID: <875F35D078FEAD48A3CA5A32B5ACC06B1C73D6@VSMAIL1.CORELAB.COM> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Metakit] Help on Int property Thread-Index: AcMl6zWY7XY8D0LZQoukBGk0dV9b8gAAQYWQAAGphvA= From: "Jim Hu" To: "Jeffrey Kay" Cc: X-OriginalArrivalTime: 29 May 2003 15:01:11.0526 (UTC) FILETIME=[24725860:01C325F3] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h4TF0QN29399 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Dear Jeffrey, Further to my last email, I quickly install metakit on Solaris 5.6 with gcc 2.95.3. I tested demo and dump, they gave correct results. But when I run your code, I got V0 = -268436744 V1 = -268436744 and dump the result gave me the same as on Linux kdeck.dat: 1 properties kdeck[name:S, value:I] VIEW 1 rows = kdeck:V 0: subview 'kdeck' VIEW 3 rows = name:S value:I 0: 'proc_cdp_first' 0 1: 'proc_cdp_last' 0 2: 'proc_cdp_inc' 0 I cannot figure out what I have done wrong Best regards Jim -----Original Message----- From: Jim Hu Sent: Thursday, May 29, 2003 9:19 AM To: 'Jeffrey Kay' Subject: RE: [Metakit] Help on Int property Dear Jeffrey, Thank you so much for your reply. I wonder if my there is anything wrong with my installation as your code gives the wrong result as well (there is a bug in the line 20). The running results are V0 = 134525096 V1 = 134525096 and the dump resluts are kdeck.dat: 1 properties kdeck[name:S, value:I] VIEW 1 rows = kdeck:V 0: subview 'kdeck' VIEW 3 rows = name:S value:I 0: 'proc_cdp_first' 0 1: 'proc_cdp_last' 0 2: 'proc_cdp_inc' 0 would you help me run the code on your system. I am use RedHat 6.1. Best regards Jim -----Original Message----- From: Jeffrey Kay [mailto:jeff@k2.com] Sent: Thursday, May 29, 2003 9:03 AM To: Jim Hu Cc: metakit@equi4.com Subject: RE: [Metakit] Help on Int property See if this simplifies what you are trying to do ... #include "mk4.h" #include "mk4str.h" #include main() { c4_StringProp pName ("name"); c4_IntProp pValue ("value"); c4_Storage storage("kdeck.dat", true); c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); kdeck.Add(pName["proc_cdp_first"] + pValue[1]); kdeck.Add(pName["proc_cdp_last"] + pValue[676]); storage.Commit(); for(int j=0;j pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein -----Original Message----- From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com] On Behalf Of Jim Hu Sent: Thursday, May 29, 2003 9:50 AM To: metakit@equi4.com Subject: [Metakit] Help on Int property Hi, all, I am just starting to use Metakit (2.9.4.2). I used the demo.cpp as an example and wrote the following piece of code. When I use the dump to dump the result, the integer part of values are all zeros. Where did I do wrong? Please advise. #include "mk4.h" #include "mk4str.h" #include main() { c4_StringProp pName ("name"); c4_IntProp pValue ("value"); c4_Storage storage("kdeck.dat", true); c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); c4_Row row; pName (row) = "proc_cdp_first"; pValue(row) = 1; kdeck.Add(row); pName (row) = "proc_cdp_last"; pValue(row) = 676; storage.Commit(); int v1 = pValue(row); printf("V1 = %d\n", v1); kdeck.Add(row); int v2 (pValue (kdeck[1])); printf("V2 = %d\n", v2); pName (row) = "proc_cdp_inc"; pValue(row) = 1; kdeck.Add(row); storage.Commit(); return(0); } Sincerely, Jim Hu From jyl@mod3.net Thu May 29 11:09:12 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4TG9CN02281 for ; Thu, 29 May 2003 11:09:12 -0500 Received: (qmail 25163 invoked by uid 48); 29 May 2003 16:07:33 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 29 May 2003 09:07:33 -0700 (PDT) Message-ID: <65082.209.233.24.218.1054224453.squirrel@mod3.net> Date: Thu, 29 May 2003 09:07:33 -0700 (PDT) Subject: RE: [Metakit] Help on Int property From: "Jacob Levy" To: In-Reply-To: <875F35D078FEAD48A3CA5A32B5ACC06B1C73D6@VSMAIL1.CORELAB.COM> References: <875F35D078FEAD48A3CA5A32B5ACC06B1C73D6@VSMAIL1.CORELAB.COM> X-Priority: 3 Importance: Normal Cc: , Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jim The versions of Solaris and GCC you're using are quite old. I'm not sure whether Metakit has been tested with these. I suggest you do a 'make test' in Metakit first, to ensure that you're getting a good build. I don't really expect any problems, but it's just one more thing that you should check to eliminate any possible question marks later on. I see you're using Metakit 2.4.9.2, that's great, that's the right version to use. In all, I've found this version to be the best yet, rock solid and very easy to build/install. Also, please repost the code you are running with Jeff's modifications, so we can see if there's any bugs introduced by your adoption of Jeff's suggestions. Thanks! --JYL > Dear Jeffrey, > > Further to my last email, I quickly install metakit on Solaris 5.6 > with gcc 2.95.3. > I tested demo and dump, they gave correct results. But when I run your > code, I got > > V0 = -268436744 > V1 = -268436744 > > and dump the result gave me the same as on Linux > > kdeck.dat: 1 properties > kdeck[name:S, value:I] > > VIEW 1 rows = kdeck:V > 0: subview 'kdeck' > VIEW 3 rows = name:S value:I > 0: 'proc_cdp_first' 0 > 1: 'proc_cdp_last' 0 > 2: 'proc_cdp_inc' 0 > > I cannot figure out what I have done wrong > > Best regards > > Jim > > -----Original Message----- > From: Jim Hu > Sent: Thursday, May 29, 2003 9:19 AM > To: 'Jeffrey Kay' > Subject: RE: [Metakit] Help on Int property > > > Dear Jeffrey, > > Thank you so much for your reply. I wonder if my there is anything > wrong with > my installation as your code gives the wrong result as well (there is a > bug in the line 20). The running results are > > V0 = 134525096 > V1 = 134525096 > > and the dump resluts are > > kdeck.dat: 1 properties > kdeck[name:S, value:I] > > VIEW 1 rows = kdeck:V > 0: subview 'kdeck' > VIEW 3 rows = name:S value:I > 0: 'proc_cdp_first' 0 > 1: 'proc_cdp_last' 0 > 2: 'proc_cdp_inc' 0 > > would you help me run the code on your system. I am use RedHat 6.1. > > Best regards > > Jim > > -----Original Message----- > From: Jeffrey Kay [mailto:jeff@k2.com] > Sent: Thursday, May 29, 2003 9:03 AM > To: Jim Hu > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > See if this simplifies what you are trying to do ... > > #include "mk4.h" > #include "mk4str.h" > #include > main() > { > c4_StringProp pName ("name"); > c4_IntProp pValue ("value"); > c4_Storage storage("kdeck.dat", true); > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); > > storage.Commit(); > > for(int j=0;j printf("V%d = %d\n",j,pValue(kdeck[j])); > } > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > storage.Commit(); > return(0); > } > > I'm not sure exactly where your logic was heading, but this is the > simplest way to add and retrieve values from rows in a view. To > dereference a particular row in a view, the syntax is c4_View[rownum]. > To dereference a particular value in a particular row, it's > c4_Prop(c4_View[rownum]). You can assign a particular row to a row > variable also, but for most things I usually use the approach shown > above. Similarly, if I'm tracking down a particular row, I use Find or > Select and then use the row number to make the property changes as shown > in this example. There are a couple of things that you need to be aware > of if you use derived views, but we'll cross that bridge when you get to > it. > > jeffrey kay > weblog pgp key aim > share files with me -- get shinkuro -- > > "first get your facts, then you can distort them at your leisure" -- > mark twain > "if the person in the next lane at the stoplight rolls up the window and > locks the door, support their view of life by snarling at them" -- a > biker's guide to life > "if A equals success, then the formula is A equals X plus Y plus Z. X is > work. Y is play. Z is keep your mouth shut." -- albert einstein > > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com] On Behalf > Of Jim Hu > Sent: Thursday, May 29, 2003 9:50 AM > To: metakit@equi4.com > Subject: [Metakit] Help on Int property > > > Hi, all, > I am just starting to use Metakit (2.9.4.2). I used the demo.cpp as > an example and wrote > the following piece of code. When I use the dump to dump the result, the > integer part of values > are all zeros. Where did I do wrong? Please advise. > > #include "mk4.h" > #include "mk4str.h" > #include > main() > { > c4_StringProp pName ("name"); > c4_IntProp pValue ("value"); > c4_Storage storage("kdeck.dat", true); > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > c4_Row row; > pName (row) = "proc_cdp_first"; > pValue(row) = 1; > kdeck.Add(row); > pName (row) = "proc_cdp_last"; > pValue(row) = 676; > storage.Commit(); > int v1 = pValue(row); > printf("V1 = %d\n", v1); > kdeck.Add(row); > int v2 (pValue (kdeck[1])); > printf("V2 = %d\n", v2); > pName (row) = "proc_cdp_inc"; > pValue(row) = 1; > kdeck.Add(row); > storage.Commit(); > return(0); > } > Sincerely, > Jim Hu > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From JHu@CoreLab.com Thu May 29 11:19:45 2003 Received: from exchangegw1.CORELAB.COM (pix.corelab.com [12.45.169.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TGJjN03075 for ; Thu, 29 May 2003 11:19:45 -0500 Received: from VSMAIL1.CORELAB.COM ([172.16.3.27]) by exchangegw1.CORELAB.COM with Microsoft SMTPSVC(5.0.2195.5329); Thu, 29 May 2003 11:20:30 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C325FE.38E221A9" Date: Thu, 29 May 2003 11:20:30 -0500 Message-ID: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB4@VSMAIL1.CORELAB.COM> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Can String and Integer be mixed to initialize a database Thread-Index: AcMl/jjt/VSXxtZkTQGntTk7T3nChw== From: "Jim Hu" To: X-OriginalArrivalTime: 29 May 2003 16:20:30.0526 (UTC) FILETIME=[39081DE0:01C325FE] Subject: [Metakit] Can String and Integer be mixed to initialize a database Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------_=_NextPart_001_01C325FE.38E221A9 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, All, Further to my post on 'Help on Int property', I slightly modify the = demp.cpp as below. The result becomes wrong. I got=20 ./dump myfile.dat myfile.dat: 2 properties address[name:S,country:S, age:I],info[version:I] VIEW 1 rows =3D address:V info:V 0: subview 'address' VIEW 4 rows =3D name:S country:S age:I 0: 'John Williams' 'UK' 0 1: 'Paco Pena' 'Spain' 0 2: 'Julian Bream' '' 0 3: 'Julien Coco' 'Netherlands' 0 0: subview 'info' VIEW 1 rows =3D version:I 0: 100 The above is from ReadHat 6.1 with gcc version egcs-2.91.66 = 19990314/Linux (egcs-1.1.2 release) Sincerely, Jim Hu ---------------------------- The slightly modified demo.cpp #include "mk4.h" #include "mk4str.h" #include int #if _WIN32_WCE _cdecl #endif main() { // These properties could just as well have been declared globally. c4_StringProp pName ("name"); c4_StringProp pCountry ("country"); c4_IntProp pAge ("age"); // Note: be careful with the lifetime of views vs. storage objects. // When a storage object goes away, all associated views are = cleared. c4_Storage storage ("myfile.dat", true); // There are two ways to make views persistent, but the c4_View::Store = call // call used in previous demos will be dropped, use "c4_View::GetAs" = instead. =20 // Start with an empty view of the proper structure. c4_View vAddress =3D storage.GetAs("address[name:S,country:S, = age:I]"); // Let's add two rows of data to the view. c4_Row row; pName (row) =3D "John Williams"; pCountry (row) =3D "UK"; pAge(row) =3D 28; vAddress.Add(row); pName (row) =3D "Paco Pena"; pCountry (row) =3D "Spain"; pAge(row) =3D 44; vAddress.Add(row); // A simple check to prove that the data is in the view. c4_String s1 (pName (vAddress[1])); c4_String s2 (pCountry (vAddress[1])); printf("The country of %s is: %s\n", (const char*) s1, (const char*) = s2); // This saves the data to file. storage.Commit(); // Data file now contains 2 addresses. // A very compact notation to create and add a third row. vAddress.Add(pName ["Julien Coco"] + pCountry ["Netherlands"]); storage.Commit(); // Data file now contains 3 addresses. // Add a second view to the data file, leaving the first view = intact. c4_View vInfo =3D storage.GetAs("info[version:I]"); // Add some data, a single integer in this case. c4_IntProp pVersion ("version"); vInfo.Add(pVersion [100]); storage.Commit(); // Data file now contains 3 addresses and 1 info = rec. // Insert a row into the address view. Note that another = (duplicate) // property definition is used here - just to show it can be done. c4_IntProp pYears ("age"); // On file this is still the "age" field. vAddress.InsertAt(2, pName ["Julian Bream"] + pYears [50]); // Preceding commits were only included for demonstration purposes. storage.Commit(); // Datafile now contains 4 addresses and 1 info rec. return 0; } ------_=_NextPart_001_01C325FE.38E221A9 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Can String and Integer be mixed to initialize a database

Hi, All,

     Further to my = post on 'Help on Int property', I slightly modify the demp.cpp as below. = The result
becomes wrong. I got

./dump myfile.dat
myfile.dat: 2 properties
  address[name:S,country:S, = age:I],info[version:I]

 VIEW     1 = rows =3D address:V info:V
    0: subview = 'address'
   = VIEW     4 rows =3D name:S country:S  = age:I
      0: = 'John Williams' 'UK' 0
      1: = 'Paco Pena' 'Spain' 0
      2: = 'Julian Bream' '' 0
      3: = 'Julien Coco' 'Netherlands' 0
    0: subview = 'info'
   = VIEW     1 rows =3D version:I
      0: = 100

   The above is from ReadHat = 6.1 with gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 = release)

Sincerely,

Jim Hu

---------------------------- The = slightly modified demo.cpp

#include "mk4.h"
#include "mk4str.h"

#include <stdio.h>

int
#if _WIN32_WCE
_cdecl
#endif
main()
{
    // These properties = could just as well have been declared globally.
  c4_StringProp pName = ("name");
  c4_StringProp pCountry = ("country");
  c4_IntProp pAge = ("age");

    // Note: be careful = with the lifetime of views vs. storage objects.
    // When a storage = object goes away, all associated views are cleared.
  c4_Storage storage = ("myfile.dat", true);

// There are two ways to make views = persistent, but the c4_View::Store call
// call used in previous demos will be = dropped, use "c4_View::GetAs" instead.
 
    // Start with an = empty view of the proper structure.
  c4_View vAddress =3D = storage.GetAs("address[name:S,country:S, age:I]");

    // Let's add two = rows of data to the view.
  c4_Row row;

  pName (row) =3D "John = Williams";
  pCountry (row) =3D = "UK";
  pAge(row) =3D 28;
  vAddress.Add(row);

  pName (row) =3D "Paco = Pena";
  pCountry (row) =3D = "Spain";
  pAge(row) =3D 44;
  vAddress.Add(row);

    // A simple check to = prove that the data is in the view.
  c4_String s1 (pName = (vAddress[1]));
  c4_String s2 (pCountry = (vAddress[1]));
  printf("The country of %s = is: %s\n", (const char*) s1, (const char*) s2);

    // This saves the = data to file.
  storage.Commit(); // Data file = now contains 2 addresses.

    // A very compact = notation to create and add a third row.
  vAddress.Add(pName = ["Julien Coco"] + pCountry ["Netherlands"]);

  storage.Commit(); // Data file = now contains 3 addresses.

    // Add a second view = to the data file, leaving the first view intact.
  c4_View vInfo =3D = storage.GetAs("info[version:I]");
    // Add some data, a = single integer in this case.
  c4_IntProp pVersion = ("version");
  vInfo.Add(pVersion = [100]);

  storage.Commit(); // Data file = now contains 3 addresses and 1 info rec.

    // Insert a row into = the address view.  Note that another (duplicate)
    // property = definition is used here - just to show it can be done.
  c4_IntProp pYears = ("age");  // On file this is still the "age" = field.

  vAddress.InsertAt(2, pName = ["Julian Bream"] + pYears [50]);

    // Preceding commits = were only included for demonstration purposes.
  storage.Commit(); // Datafile = now contains 4 addresses and 1 info rec.

  return 0;
}

------_=_NextPart_001_01C325FE.38E221A9-- From JHu@CoreLab.com Thu May 29 11:28:37 2003 Received: from exchangegw2.CORELAB.COM (pix.corelab.com [12.45.169.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TGSbN03827 for ; Thu, 29 May 2003 11:28:37 -0500 Received: from VSMAIL1.CORELAB.COM ([172.16.3.27]) by exchangegw2.CORELAB.COM with Microsoft SMTPSVC(5.0.2195.5329); Thu, 29 May 2003 11:29:23 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: [Metakit] Help on Int property Date: Thu, 29 May 2003 11:29:22 -0500 Message-ID: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB5@VSMAIL1.CORELAB.COM> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Metakit] Help on Int property Thread-Index: AcMl/MQFFJM+w0pIR+a50KAyoBasCAAAa/MA From: "Jim Hu" To: Cc: X-OriginalArrivalTime: 29 May 2003 16:29:23.0092 (UTC) FILETIME=[76773D40:01C325FF] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h4TGSbN03827 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob, Many thanks for your reply. I did build the test and they are all right to me. I only change Jeff's code line 20 from kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); to kdeck.Add(pName["proc_cdp_inc"] + pValue[1]); I have just posted another message regarding mixture of string and integer. I will try an newer version of ReadHat and gcc to see if the problem still happens. Best regards Jim -----Original Message----- From: Jacob Levy [mailto:jyl@mod3.net] Sent: Thursday, May 29, 2003 11:08 AM To: Jim Hu Cc: jeff@k2.com; metakit@equi4.com Subject: RE: [Metakit] Help on Int property Jim The versions of Solaris and GCC you're using are quite old. I'm not sure whether Metakit has been tested with these. I suggest you do a 'make test' in Metakit first, to ensure that you're getting a good build. I don't really expect any problems, but it's just one more thing that you should check to eliminate any possible question marks later on. I see you're using Metakit 2.4.9.2, that's great, that's the right version to use. In all, I've found this version to be the best yet, rock solid and very easy to build/install. Also, please repost the code you are running with Jeff's modifications, so we can see if there's any bugs introduced by your adoption of Jeff's suggestions. Thanks! --JYL > Dear Jeffrey, > > Further to my last email, I quickly install metakit on Solaris 5.6 > with gcc 2.95.3. > I tested demo and dump, they gave correct results. But when I run your > code, I got > > V0 = -268436744 > V1 = -268436744 > > and dump the result gave me the same as on Linux > > kdeck.dat: 1 properties > kdeck[name:S, value:I] > > VIEW 1 rows = kdeck:V > 0: subview 'kdeck' > VIEW 3 rows = name:S value:I > 0: 'proc_cdp_first' 0 > 1: 'proc_cdp_last' 0 > 2: 'proc_cdp_inc' 0 > > I cannot figure out what I have done wrong > > Best regards > > Jim > > -----Original Message----- > From: Jim Hu > Sent: Thursday, May 29, 2003 9:19 AM > To: 'Jeffrey Kay' > Subject: RE: [Metakit] Help on Int property > > > Dear Jeffrey, > > Thank you so much for your reply. I wonder if my there is anything > wrong with > my installation as your code gives the wrong result as well (there is a > bug in the line 20). The running results are > > V0 = 134525096 > V1 = 134525096 > > and the dump resluts are > > kdeck.dat: 1 properties > kdeck[name:S, value:I] > > VIEW 1 rows = kdeck:V > 0: subview 'kdeck' > VIEW 3 rows = name:S value:I > 0: 'proc_cdp_first' 0 > 1: 'proc_cdp_last' 0 > 2: 'proc_cdp_inc' 0 > > would you help me run the code on your system. I am use RedHat 6.1. > > Best regards > > Jim > > -----Original Message----- > From: Jeffrey Kay [mailto:jeff@k2.com] > Sent: Thursday, May 29, 2003 9:03 AM > To: Jim Hu > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > See if this simplifies what you are trying to do ... > > #include "mk4.h" > #include "mk4str.h" > #include > main() > { > c4_StringProp pName ("name"); > c4_IntProp pValue ("value"); > c4_Storage storage("kdeck.dat", true); > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); > > storage.Commit(); > > for(int j=0;j printf("V%d = %d\n",j,pValue(kdeck[j])); > } > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > storage.Commit(); > return(0); > } > > I'm not sure exactly where your logic was heading, but this is the > simplest way to add and retrieve values from rows in a view. To > dereference a particular row in a view, the syntax is c4_View[rownum]. > To dereference a particular value in a particular row, it's > c4_Prop(c4_View[rownum]). You can assign a particular row to a row > variable also, but for most things I usually use the approach shown > above. Similarly, if I'm tracking down a particular row, I use Find or > Select and then use the row number to make the property changes as shown > in this example. There are a couple of things that you need to be aware > of if you use derived views, but we'll cross that bridge when you get to > it. > > jeffrey kay > weblog pgp key aim > share files with me -- get shinkuro -- > > "first get your facts, then you can distort them at your leisure" -- > mark twain > "if the person in the next lane at the stoplight rolls up the window and > locks the door, support their view of life by snarling at them" -- a > biker's guide to life > "if A equals success, then the formula is A equals X plus Y plus Z. X is > work. Y is play. Z is keep your mouth shut." -- albert einstein > > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com] On Behalf > Of Jim Hu > Sent: Thursday, May 29, 2003 9:50 AM > To: metakit@equi4.com > Subject: [Metakit] Help on Int property > > > Hi, all, > I am just starting to use Metakit (2.9.4.2). I used the demo.cpp as > an example and wrote > the following piece of code. When I use the dump to dump the result, the > integer part of values > are all zeros. Where did I do wrong? Please advise. > > #include "mk4.h" > #include "mk4str.h" > #include > main() > { > c4_StringProp pName ("name"); > c4_IntProp pValue ("value"); > c4_Storage storage("kdeck.dat", true); > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > c4_Row row; > pName (row) = "proc_cdp_first"; > pValue(row) = 1; > kdeck.Add(row); > pName (row) = "proc_cdp_last"; > pValue(row) = 676; > storage.Commit(); > int v1 = pValue(row); > printf("V1 = %d\n", v1); > kdeck.Add(row); > int v2 (pValue (kdeck[1])); > printf("V2 = %d\n", v2); > pName (row) = "proc_cdp_inc"; > pValue(row) = 1; > kdeck.Add(row); > storage.Commit(); > return(0); > } > Sincerely, > Jim Hu > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From JHu@CoreLab.com Thu May 29 11:58:48 2003 Received: from exchangegw1.CORELAB.COM (pix.corelab.com [12.45.169.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TGwmN06105 for ; Thu, 29 May 2003 11:58:48 -0500 Received: from VSMAIL1.CORELAB.COM ([172.16.3.27]) by exchangegw1.CORELAB.COM with Microsoft SMTPSVC(5.0.2195.5329); Thu, 29 May 2003 11:59:33 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: [Metakit] Help on Int property Date: Thu, 29 May 2003 11:59:33 -0500 Message-ID: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB6@VSMAIL1.CORELAB.COM> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Metakit] Help on Int property Thread-Index: AcMl/MQFFJM+w0pIR+a50KAyoBasCAAAa/MAAAEiyKA= From: "Jim Hu" To: Cc: X-OriginalArrivalTime: 29 May 2003 16:59:33.0948 (UTC) FILETIME=[ADD1EFC0:01C32603] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h4TGwmN06105 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob, I have just built and installed metakit 2.4.9.2 on ReadHat Linux 7.1.2 gcc 2.96. I have got exactly the same wrong reslut as on Solaris 5.6 with gcc 2.95.3 and ReadHat 6.1 with gcc 2.91.66. Best regards JH -----Original Message----- From: Jim Hu Sent: Thursday, May 29, 2003 11:29 AM To: 'jyl@mod3.net' Cc: metakit@equi4.com Subject: RE: [Metakit] Help on Int property Jacob, Many thanks for your reply. I did build the test and they are all right to me. I only change Jeff's code line 20 from kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); to kdeck.Add(pName["proc_cdp_inc"] + pValue[1]); I have just posted another message regarding mixture of string and integer. I will try an newer version of ReadHat and gcc to see if the problem still happens. Best regards Jim -----Original Message----- From: Jacob Levy [mailto:jyl@mod3.net] Sent: Thursday, May 29, 2003 11:08 AM To: Jim Hu Cc: jeff@k2.com; metakit@equi4.com Subject: RE: [Metakit] Help on Int property Jim The versions of Solaris and GCC you're using are quite old. I'm not sure whether Metakit has been tested with these. I suggest you do a 'make test' in Metakit first, to ensure that you're getting a good build. I don't really expect any problems, but it's just one more thing that you should check to eliminate any possible question marks later on. I see you're using Metakit 2.4.9.2, that's great, that's the right version to use. In all, I've found this version to be the best yet, rock solid and very easy to build/install. Also, please repost the code you are running with Jeff's modifications, so we can see if there's any bugs introduced by your adoption of Jeff's suggestions. Thanks! --JYL > Dear Jeffrey, > > Further to my last email, I quickly install metakit on Solaris 5.6 > with gcc 2.95.3. > I tested demo and dump, they gave correct results. But when I run your > code, I got > > V0 = -268436744 > V1 = -268436744 > > and dump the result gave me the same as on Linux > > kdeck.dat: 1 properties > kdeck[name:S, value:I] > > VIEW 1 rows = kdeck:V > 0: subview 'kdeck' > VIEW 3 rows = name:S value:I > 0: 'proc_cdp_first' 0 > 1: 'proc_cdp_last' 0 > 2: 'proc_cdp_inc' 0 > > I cannot figure out what I have done wrong > > Best regards > > Jim > > -----Original Message----- > From: Jim Hu > Sent: Thursday, May 29, 2003 9:19 AM > To: 'Jeffrey Kay' > Subject: RE: [Metakit] Help on Int property > > > Dear Jeffrey, > > Thank you so much for your reply. I wonder if my there is anything > wrong with > my installation as your code gives the wrong result as well (there is a > bug in the line 20). The running results are > > V0 = 134525096 > V1 = 134525096 > > and the dump resluts are > > kdeck.dat: 1 properties > kdeck[name:S, value:I] > > VIEW 1 rows = kdeck:V > 0: subview 'kdeck' > VIEW 3 rows = name:S value:I > 0: 'proc_cdp_first' 0 > 1: 'proc_cdp_last' 0 > 2: 'proc_cdp_inc' 0 > > would you help me run the code on your system. I am use RedHat 6.1. > > Best regards > > Jim > > -----Original Message----- > From: Jeffrey Kay [mailto:jeff@k2.com] > Sent: Thursday, May 29, 2003 9:03 AM > To: Jim Hu > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > See if this simplifies what you are trying to do ... > > #include "mk4.h" > #include "mk4str.h" > #include > main() > { > c4_StringProp pName ("name"); > c4_IntProp pValue ("value"); > c4_Storage storage("kdeck.dat", true); > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); > > storage.Commit(); > > for(int j=0;j printf("V%d = %d\n",j,pValue(kdeck[j])); > } > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > storage.Commit(); > return(0); > } > > I'm not sure exactly where your logic was heading, but this is the > simplest way to add and retrieve values from rows in a view. To > dereference a particular row in a view, the syntax is c4_View[rownum]. > To dereference a particular value in a particular row, it's > c4_Prop(c4_View[rownum]). You can assign a particular row to a row > variable also, but for most things I usually use the approach shown > above. Similarly, if I'm tracking down a particular row, I use Find or > Select and then use the row number to make the property changes as shown > in this example. There are a couple of things that you need to be aware > of if you use derived views, but we'll cross that bridge when you get to > it. > > jeffrey kay > weblog pgp key aim > share files with me -- get shinkuro -- > > "first get your facts, then you can distort them at your leisure" -- > mark twain > "if the person in the next lane at the stoplight rolls up the window and > locks the door, support their view of life by snarling at them" -- a > biker's guide to life > "if A equals success, then the formula is A equals X plus Y plus Z. X is > work. Y is play. Z is keep your mouth shut." -- albert einstein > > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com] On Behalf > Of Jim Hu > Sent: Thursday, May 29, 2003 9:50 AM > To: metakit@equi4.com > Subject: [Metakit] Help on Int property > > > Hi, all, > I am just starting to use Metakit (2.9.4.2). I used the demo.cpp as > an example and wrote > the following piece of code. When I use the dump to dump the result, the > integer part of values > are all zeros. Where did I do wrong? Please advise. > > #include "mk4.h" > #include "mk4str.h" > #include > main() > { > c4_StringProp pName ("name"); > c4_IntProp pValue ("value"); > c4_Storage storage("kdeck.dat", true); > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > c4_Row row; > pName (row) = "proc_cdp_first"; > pValue(row) = 1; > kdeck.Add(row); > pName (row) = "proc_cdp_last"; > pValue(row) = 676; > storage.Commit(); > int v1 = pValue(row); > printf("V1 = %d\n", v1); > kdeck.Add(row); > int v2 (pValue (kdeck[1])); > printf("V2 = %d\n", v2); > pName (row) = "proc_cdp_inc"; > pValue(row) = 1; > kdeck.Add(row); > storage.Commit(); > return(0); > } > Sincerely, > Jim Hu > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jeff@k2.com Thu May 29 13:29:45 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TITiN12717 for ; Thu, 29 May 2003 13:29:44 -0500 Received: from Klee ([192.168.0.21]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h4TIMdN21222; Thu, 29 May 2003 14:22:39 -0400 From: "Jeffrey Kay" To: "'Jim Hu'" , Cc: Subject: RE: [Metakit] Help on Int property Date: Thu, 29 May 2003 14:29:00 -0400 Message-ID: <010501c32610$2cdb7d20$017410ac@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal In-Reply-To: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB5@VSMAIL1.CORELAB.COM> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Well, you got me on this one also. I've been using int props in other places without trouble, but this test program doesn't work. I'm looking into it further. FWIW, I'm running Metakit 2.4.7 on Windows. jeffrey kay weblog pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of Jim Hu > Sent: Thursday, May 29, 2003 12:29 PM > To: jyl@mod3.net > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jacob, > > Many thanks for your reply. I did build the test and they > are all right to me. I only change Jeff's > code line 20 from > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > > to > > kdeck.Add(pName["proc_cdp_inc"] + pValue[1]); > > I have just posted another message regarding mixture of > string and integer. > I will try an newer version of ReadHat and gcc to see if the > problem still happens. > > Best regards > > Jim > > -----Original Message----- > From: Jacob Levy [mailto:jyl@mod3.net] > Sent: Thursday, May 29, 2003 11:08 AM > To: Jim Hu > Cc: jeff@k2.com; metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jim > > The versions of Solaris and GCC you're using are quite old. > I'm not sure > whether Metakit has been tested with these. I suggest you do > a 'make test' > in Metakit first, to ensure that you're getting a good build. I don't > really expect any problems, but it's just one more thing that > you should > check to eliminate any possible question marks later on. > > I see you're using Metakit 2.4.9.2, that's great, that's the > right version > to use. In all, I've found this version to be the best yet, > rock solid and > very easy to build/install. > > Also, please repost the code you are running with Jeff's > modifications, so > we can see if there's any bugs introduced by your adoption of Jeff's > suggestions. > > Thanks! --JYL > > > Dear Jeffrey, > > > > Further to my last email, I quickly install metakit on > Solaris 5.6 > > with gcc 2.95.3. > > I tested demo and dump, they gave correct results. But when > I run your > > code, I got > > > > V0 = -268436744 > > V1 = -268436744 > > > > and dump the result gave me the same as on Linux > > > > kdeck.dat: 1 properties > > kdeck[name:S, value:I] > > > > VIEW 1 rows = kdeck:V > > 0: subview 'kdeck' > > VIEW 3 rows = name:S value:I > > 0: 'proc_cdp_first' 0 > > 1: 'proc_cdp_last' 0 > > 2: 'proc_cdp_inc' 0 > > > > I cannot figure out what I have done wrong > > > > Best regards > > > > Jim > > > > -----Original Message----- > > From: Jim Hu > > Sent: Thursday, May 29, 2003 9:19 AM > > To: 'Jeffrey Kay' > > Subject: RE: [Metakit] Help on Int property > > > > > > Dear Jeffrey, > > > > Thank you so much for your reply. I wonder if my > there is anything > > wrong with > > my installation as your code gives the wrong result as well > (there is a > > bug in the line 20). The running results are > > > > V0 = 134525096 > > V1 = 134525096 > > > > and the dump resluts are > > > > kdeck.dat: 1 properties > > kdeck[name:S, value:I] > > > > VIEW 1 rows = kdeck:V > > 0: subview 'kdeck' > > VIEW 3 rows = name:S value:I > > 0: 'proc_cdp_first' 0 > > 1: 'proc_cdp_last' 0 > > 2: 'proc_cdp_inc' 0 > > > > would you help me run the code on your system. I am use RedHat 6.1. > > > > Best regards > > > > Jim > > > > -----Original Message----- > > From: Jeffrey Kay [mailto:jeff@k2.com] > > Sent: Thursday, May 29, 2003 9:03 AM > > To: Jim Hu > > Cc: metakit@equi4.com > > Subject: RE: [Metakit] Help on Int property > > > > > > See if this simplifies what you are trying to do ... > > > > #include "mk4.h" > > #include "mk4str.h" > > #include > > main() > > { > > c4_StringProp pName ("name"); > > c4_IntProp pValue ("value"); > > c4_Storage storage("kdeck.dat", true); > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > > > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); > > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); > > > > storage.Commit(); > > > > for(int j=0;j > printf("V%d = %d\n",j,pValue(kdeck[j])); > > } > > > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > > storage.Commit(); > > return(0); > > } > > > > I'm not sure exactly where your logic was heading, but this is the > > simplest way to add and retrieve values from rows in a view. To > > dereference a particular row in a view, the syntax is > c4_View[rownum]. > > To dereference a particular value in a particular row, it's > > c4_Prop(c4_View[rownum]). You can assign a particular row to a row > > variable also, but for most things I usually use the approach shown > > above. Similarly, if I'm tracking down a particular row, I > use Find or > > Select and then use the row number to make the property > changes as shown > > in this example. There are a couple of things that you > need to be aware > > of if you use derived views, but we'll cross that bridge > when you get to > > it. > > > > jeffrey kay > > weblog pgp key aim > > share files with me -- get shinkuro -- > > > > "first get your facts, then you can distort them at your leisure" -- > > mark twain > > "if the person in the next lane at the stoplight rolls up > the window and > > locks the door, support their view of life by snarling at them" -- a > > biker's guide to life > > "if A equals success, then the formula is A equals X plus Y > plus Z. X is > > work. Y is play. Z is keep your mouth shut." -- albert einstein > > > > -----Original Message----- > > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf > > Of Jim Hu > > > Sent: Thursday, May 29, 2003 9:50 AM > > To: metakit@equi4.com > > Subject: [Metakit] Help on Int property > > > > > > Hi, all, > > I am just starting to use Metakit (2.9.4.2). I used > the demo.cpp as > > an example and wrote > > the following piece of code. When I use the dump to dump > the result, the > > integer part of values > > are all zeros. Where did I do wrong? Please advise. > > > > #include "mk4.h" > > #include "mk4str.h" > > #include > > main() > > { > > c4_StringProp pName ("name"); > > c4_IntProp pValue ("value"); > > c4_Storage storage("kdeck.dat", true); > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > c4_Row row; > > pName (row) = "proc_cdp_first"; > > pValue(row) = 1; > > kdeck.Add(row); > > pName (row) = "proc_cdp_last"; > > pValue(row) = 676; > > storage.Commit(); > > int v1 = pValue(row); > > printf("V1 = %d\n", v1); > > kdeck.Add(row); > > int v2 (pValue (kdeck[1])); > > printf("V2 = %d\n", v2); > > pName (row) = "proc_cdp_inc"; > > pValue(row) = 1; > > kdeck.Add(row); > > storage.Commit(); > > return(0); > > } > > Sincerely, > > Jim Hu > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jeff@k2.com Thu May 29 13:38:48 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TIciN13582 for ; Thu, 29 May 2003 13:38:47 -0500 Received: from Klee ([192.168.0.21]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h4TIVbN22076; Thu, 29 May 2003 14:31:37 -0400 From: "Jeffrey Kay" To: "'Jim Hu'" , Cc: Subject: RE: [Metakit] Help on Int property Date: Thu, 29 May 2003 14:37:58 -0400 Message-ID: <010601c32611$6d52d370$017410ac@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal In-Reply-To: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB6@VSMAIL1.CORELAB.COM> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Okay, here's a program that works and why (AFAIK). c4_IntProp does not have an automatic cast to an int. The operator() returns a c4_IntRef, which doesn't seem happy about automatically being cast to an int. If you assign the value directly to an int, or presumably force the cast, then it should work fine. I'm surprised that I never noticed this, but perhaps it's because I don't usually printf anything. #include "mk4.h" #include "mk4str.h" #include main() { c4_StringProp pName("name"); c4_IntProp pValue("value"); c4_Storage storage("kdeck.db",true); c4_View kdeck = storage.GetAs("kdeck[name:S,value:I]"); kdeck.Add(pName["proc_cdp_first"] + pValue[1]); kdeck.Add(pName["proc_cdp_last"] + pValue[676]); storage.Commit(); for(int j=0;j pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of Jim Hu > Sent: Thursday, May 29, 2003 1:00 PM > To: jyl@mod3.net > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jacob, > > I have just built and installed metakit 2.4.9.2 on > ReadHat Linux 7.1.2 > gcc 2.96. I have got exactly the same wrong reslut as on > Solaris 5.6 with gcc 2.95.3 > and ReadHat 6.1 with gcc 2.91.66. > > Best regards > > JH > > -----Original Message----- > From: Jim Hu > Sent: Thursday, May 29, 2003 11:29 AM > To: 'jyl@mod3.net' > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jacob, > > Many thanks for your reply. I did build the test and they > are all right to me. I only change Jeff's > code line 20 from > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > > to > > kdeck.Add(pName["proc_cdp_inc"] + pValue[1]); > > I have just posted another message regarding mixture of > string and integer. > I will try an newer version of ReadHat and gcc to see if the > problem still happens. > > Best regards > > Jim > > -----Original Message----- > From: Jacob Levy [mailto:jyl@mod3.net] > Sent: Thursday, May 29, 2003 11:08 AM > To: Jim Hu > Cc: jeff@k2.com; metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jim > > The versions of Solaris and GCC you're using are quite old. > I'm not sure > whether Metakit has been tested with these. I suggest you do > a 'make test' > in Metakit first, to ensure that you're getting a good build. I don't > really expect any problems, but it's just one more thing that > you should > check to eliminate any possible question marks later on. > > I see you're using Metakit 2.4.9.2, that's great, that's the > right version > to use. In all, I've found this version to be the best yet, > rock solid and > very easy to build/install. > > Also, please repost the code you are running with Jeff's > modifications, so > we can see if there's any bugs introduced by your adoption of Jeff's > suggestions. > > Thanks! --JYL > > > Dear Jeffrey, > > > > Further to my last email, I quickly install metakit on > Solaris 5.6 > > with gcc 2.95.3. > > I tested demo and dump, they gave correct results. But when > I run your > > code, I got > > > > V0 = -268436744 > > V1 = -268436744 > > > > and dump the result gave me the same as on Linux > > > > kdeck.dat: 1 properties > > kdeck[name:S, value:I] > > > > VIEW 1 rows = kdeck:V > > 0: subview 'kdeck' > > VIEW 3 rows = name:S value:I > > 0: 'proc_cdp_first' 0 > > 1: 'proc_cdp_last' 0 > > 2: 'proc_cdp_inc' 0 > > > > I cannot figure out what I have done wrong > > > > Best regards > > > > Jim > > > > -----Original Message----- > > From: Jim Hu > > Sent: Thursday, May 29, 2003 9:19 AM > > To: 'Jeffrey Kay' > > Subject: RE: [Metakit] Help on Int property > > > > > > Dear Jeffrey, > > > > Thank you so much for your reply. I wonder if my > there is anything > > wrong with > > my installation as your code gives the wrong result as well > (there is a > > bug in the line 20). The running results are > > > > V0 = 134525096 > > V1 = 134525096 > > > > and the dump resluts are > > > > kdeck.dat: 1 properties > > kdeck[name:S, value:I] > > > > VIEW 1 rows = kdeck:V > > 0: subview 'kdeck' > > VIEW 3 rows = name:S value:I > > 0: 'proc_cdp_first' 0 > > 1: 'proc_cdp_last' 0 > > 2: 'proc_cdp_inc' 0 > > > > would you help me run the code on your system. I am use RedHat 6.1. > > > > Best regards > > > > Jim > > > > -----Original Message----- > > From: Jeffrey Kay [mailto:jeff@k2.com] > > Sent: Thursday, May 29, 2003 9:03 AM > > To: Jim Hu > > Cc: metakit@equi4.com > > Subject: RE: [Metakit] Help on Int property > > > > > > See if this simplifies what you are trying to do ... > > > > #include "mk4.h" > > #include "mk4str.h" > > #include > > main() > > { > > c4_StringProp pName ("name"); > > c4_IntProp pValue ("value"); > > c4_Storage storage("kdeck.dat", true); > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > > > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); > > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); > > > > storage.Commit(); > > > > for(int j=0;j > printf("V%d = %d\n",j,pValue(kdeck[j])); > > } > > > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > > storage.Commit(); > > return(0); > > } > > > > I'm not sure exactly where your logic was heading, but this is the > > simplest way to add and retrieve values from rows in a view. To > > dereference a particular row in a view, the syntax is > c4_View[rownum]. > > To dereference a particular value in a particular row, it's > > c4_Prop(c4_View[rownum]). You can assign a particular row to a row > > variable also, but for most things I usually use the approach shown > > above. Similarly, if I'm tracking down a particular row, I > use Find or > > Select and then use the row number to make the property > changes as shown > > in this example. There are a couple of things that you > need to be aware > > of if you use derived views, but we'll cross that bridge > when you get to > > it. > > > > jeffrey kay > > weblog pgp key aim > > share files with me -- get shinkuro -- > > > > "first get your facts, then you can distort them at your leisure" -- > > mark twain > > "if the person in the next lane at the stoplight rolls up > the window and > > locks the door, support their view of life by snarling at them" -- a > > biker's guide to life > > "if A equals success, then the formula is A equals X plus Y > plus Z. X is > > work. Y is play. Z is keep your mouth shut." -- albert einstein > > > > -----Original Message----- > > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf > > Of Jim Hu > > > Sent: Thursday, May 29, 2003 9:50 AM > > To: metakit@equi4.com > > Subject: [Metakit] Help on Int property > > > > > > Hi, all, > > I am just starting to use Metakit (2.9.4.2). I used > the demo.cpp as > > an example and wrote > > the following piece of code. When I use the dump to dump > the result, the > > integer part of values > > are all zeros. Where did I do wrong? Please advise. > > > > #include "mk4.h" > > #include "mk4str.h" > > #include > > main() > > { > > c4_StringProp pName ("name"); > > c4_IntProp pValue ("value"); > > c4_Storage storage("kdeck.dat", true); > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > c4_Row row; > > pName (row) = "proc_cdp_first"; > > pValue(row) = 1; > > kdeck.Add(row); > > pName (row) = "proc_cdp_last"; > > pValue(row) = 676; > > storage.Commit(); > > int v1 = pValue(row); > > printf("V1 = %d\n", v1); > > kdeck.Add(row); > > int v2 (pValue (kdeck[1])); > > printf("V2 = %d\n", v2); > > pName (row) = "proc_cdp_inc"; > > pValue(row) = 1; > > kdeck.Add(row); > > storage.Commit(); > > return(0); > > } > > Sincerely, > > Jim Hu > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From mac@ceti.pl Thu May 29 14:31:14 2003 Received: from mx2.ceti.pl (mx2.ceti.pl [62.121.128.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TJVDN17654 for ; Thu, 29 May 2003 14:31:14 -0500 Received: from tau.ceti.pl (tau.ceti.pl [62.121.128.11]) by mx2.ceti.pl (Postfix) with ESMTP id 2849211E6C3 for ; Thu, 29 May 2003 21:31:52 +0200 (CEST) Received: by tau.ceti.pl (Postfix, from userid 2967) id 87A91202FEC; Thu, 29 May 2003 21:31:57 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by tau.ceti.pl (Postfix) with ESMTP id 84BC71E70C7 for ; Thu, 29 May 2003 21:31:57 +0200 (CEST) Date: Thu, 29 May 2003 21:31:52 +0200 (CEST) From: Maciej Starzyk To: metakit@equi4.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Saving nested views. (fwd) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I created an in-memory db. Added two views, one stored as an attribute in the other. Both views are created with getas(), so they should be persistent. After saving the database with save(), and reloading it, the child view cannot be retrieved from the parent view. Does anyone have idea why this doesn't work? I've attached the script and the output below. I am using Python 2.3b1 on win xp prof and metakit 2.4.9.2 Thanks, Maciej Here is the script: ############################################## import metakit as m def showDb (db, msg): print "=======================" print msg print "=======================" p = db.view('parent') c = p[0].children print "View 'parent':" m.dump(p) print "View 'child':" m.dump(c) ## ## ## mem_db = m.storage() p = mem_db.getas('parent[name:S,children:V]') c = mem_db.getas('child[name:S]') c.append(name='a child') p.append(name='a parent', children=c) filename = 'qq.mk' mem_db.save(open(filename, 'wb')) file_db = m.storage(filename, 1) showDb(mem_db , "DB in memory:") showDb(file_db, "DB in file:") file_db = None import os os.remove(filename) ######################################## And here is the output: ======================= DB in memory: ======================= View 'parent': name children -------- -------- a parent 1 rows -------- -------- Total: 1 rows View 'child': name ------- a child ------- Total: 1 rows ======================= DB in file: ======================= View 'parent': name children -------- -------- a parent 1 rows -------- -------- Total: 1 rows View 'child': Total: 1 rows From JHu@CoreLab.com Thu May 29 14:36:42 2003 Received: from exchangegw1.CORELAB.COM (pix.corelab.com [12.45.169.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TJafN18072 for ; Thu, 29 May 2003 14:36:42 -0500 Received: from VSMAIL1.CORELAB.COM ([172.16.3.27]) by exchangegw1.CORELAB.COM with Microsoft SMTPSVC(5.0.2195.5329); Thu, 29 May 2003 14:37:27 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: [Metakit] Help on Int property Date: Thu, 29 May 2003 14:37:25 -0500 Message-ID: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB7@VSMAIL1.CORELAB.COM> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Metakit] Help on Int property Thread-Index: AcMmEaEjg4sc5lTcTkaDswgXuOyhJgABuc4w From: "Jim Hu" To: "Jeffrey Kay" , Cc: X-OriginalArrivalTime: 29 May 2003 19:37:27.0498 (UTC) FILETIME=[BC7EF2A0:01C32619] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h4TJafN18072 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeffrey & Jacob, Thank you so much for help. The real problem to cuase wrong data saved into database I have just find out is because I left a space in front the word value in the line c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); What you found is also right. I wonder if the space problem is a Metakit bug or something I do not know. Please advise. Many thanks. Jim -----Original Message----- From: Jeffrey Kay [mailto:jeff@k2.com] Sent: Thursday, May 29, 2003 1:38 PM To: Jim Hu; jyl@mod3.net Cc: metakit@equi4.com Subject: RE: [Metakit] Help on Int property Okay, here's a program that works and why (AFAIK). c4_IntProp does not have an automatic cast to an int. The operator() returns a c4_IntRef, which doesn't seem happy about automatically being cast to an int. If you assign the value directly to an int, or presumably force the cast, then it should work fine. I'm surprised that I never noticed this, but perhaps it's because I don't usually printf anything. #include "mk4.h" #include "mk4str.h" #include main() { c4_StringProp pName("name"); c4_IntProp pValue("value"); c4_Storage storage("kdeck.db",true); c4_View kdeck = storage.GetAs("kdeck[name:S,value:I]"); kdeck.Add(pName["proc_cdp_first"] + pValue[1]); kdeck.Add(pName["proc_cdp_last"] + pValue[676]); storage.Commit(); for(int j=0;j pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of Jim Hu > Sent: Thursday, May 29, 2003 1:00 PM > To: jyl@mod3.net > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jacob, > > I have just built and installed metakit 2.4.9.2 on > ReadHat Linux 7.1.2 > gcc 2.96. I have got exactly the same wrong reslut as on > Solaris 5.6 with gcc 2.95.3 > and ReadHat 6.1 with gcc 2.91.66. > > Best regards > > JH > > -----Original Message----- > From: Jim Hu > Sent: Thursday, May 29, 2003 11:29 AM > To: 'jyl@mod3.net' > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jacob, > > Many thanks for your reply. I did build the test and they > are all right to me. I only change Jeff's > code line 20 from > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > > to > > kdeck.Add(pName["proc_cdp_inc"] + pValue[1]); > > I have just posted another message regarding mixture of > string and integer. > I will try an newer version of ReadHat and gcc to see if the > problem still happens. > > Best regards > > Jim > > -----Original Message----- > From: Jacob Levy [mailto:jyl@mod3.net] > Sent: Thursday, May 29, 2003 11:08 AM > To: Jim Hu > Cc: jeff@k2.com; metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jim > > The versions of Solaris and GCC you're using are quite old. > I'm not sure > whether Metakit has been tested with these. I suggest you do > a 'make test' > in Metakit first, to ensure that you're getting a good build. I don't > really expect any problems, but it's just one more thing that > you should > check to eliminate any possible question marks later on. > > I see you're using Metakit 2.4.9.2, that's great, that's the > right version > to use. In all, I've found this version to be the best yet, > rock solid and > very easy to build/install. > > Also, please repost the code you are running with Jeff's > modifications, so > we can see if there's any bugs introduced by your adoption of Jeff's > suggestions. > > Thanks! --JYL > > > Dear Jeffrey, > > > > Further to my last email, I quickly install metakit on > Solaris 5.6 > > with gcc 2.95.3. > > I tested demo and dump, they gave correct results. But when > I run your > > code, I got > > > > V0 = -268436744 > > V1 = -268436744 > > > > and dump the result gave me the same as on Linux > > > > kdeck.dat: 1 properties > > kdeck[name:S, value:I] > > > > VIEW 1 rows = kdeck:V > > 0: subview 'kdeck' > > VIEW 3 rows = name:S value:I > > 0: 'proc_cdp_first' 0 > > 1: 'proc_cdp_last' 0 > > 2: 'proc_cdp_inc' 0 > > > > I cannot figure out what I have done wrong > > > > Best regards > > > > Jim > > > > -----Original Message----- > > From: Jim Hu > > Sent: Thursday, May 29, 2003 9:19 AM > > To: 'Jeffrey Kay' > > Subject: RE: [Metakit] Help on Int property > > > > > > Dear Jeffrey, > > > > Thank you so much for your reply. I wonder if my > there is anything > > wrong with > > my installation as your code gives the wrong result as well > (there is a > > bug in the line 20). The running results are > > > > V0 = 134525096 > > V1 = 134525096 > > > > and the dump resluts are > > > > kdeck.dat: 1 properties > > kdeck[name:S, value:I] > > > > VIEW 1 rows = kdeck:V > > 0: subview 'kdeck' > > VIEW 3 rows = name:S value:I > > 0: 'proc_cdp_first' 0 > > 1: 'proc_cdp_last' 0 > > 2: 'proc_cdp_inc' 0 > > > > would you help me run the code on your system. I am use RedHat 6.1. > > > > Best regards > > > > Jim > > > > -----Original Message----- > > From: Jeffrey Kay [mailto:jeff@k2.com] > > Sent: Thursday, May 29, 2003 9:03 AM > > To: Jim Hu > > Cc: metakit@equi4.com > > Subject: RE: [Metakit] Help on Int property > > > > > > See if this simplifies what you are trying to do ... > > > > #include "mk4.h" > > #include "mk4str.h" > > #include > > main() > > { > > c4_StringProp pName ("name"); > > c4_IntProp pValue ("value"); > > c4_Storage storage("kdeck.dat", true); > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > > > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); > > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); > > > > storage.Commit(); > > > > for(int j=0;j > printf("V%d = %d\n",j,pValue(kdeck[j])); > > } > > > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > > storage.Commit(); > > return(0); > > } > > > > I'm not sure exactly where your logic was heading, but this is the > > simplest way to add and retrieve values from rows in a view. To > > dereference a particular row in a view, the syntax is > c4_View[rownum]. > > To dereference a particular value in a particular row, it's > > c4_Prop(c4_View[rownum]). You can assign a particular row to a row > > variable also, but for most things I usually use the approach shown > > above. Similarly, if I'm tracking down a particular row, I > use Find or > > Select and then use the row number to make the property > changes as shown > > in this example. There are a couple of things that you > need to be aware > > of if you use derived views, but we'll cross that bridge > when you get to > > it. > > > > jeffrey kay > > weblog pgp key aim > > share files with me -- get shinkuro -- > > > > "first get your facts, then you can distort them at your leisure" -- > > mark twain > > "if the person in the next lane at the stoplight rolls up > the window and > > locks the door, support their view of life by snarling at them" -- a > > biker's guide to life > > "if A equals success, then the formula is A equals X plus Y > plus Z. X is > > work. Y is play. Z is keep your mouth shut." -- albert einstein > > > > -----Original Message----- > > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf > > Of Jim Hu > > > Sent: Thursday, May 29, 2003 9:50 AM > > To: metakit@equi4.com > > Subject: [Metakit] Help on Int property > > > > > > Hi, all, > > I am just starting to use Metakit (2.9.4.2). I used > the demo.cpp as > > an example and wrote > > the following piece of code. When I use the dump to dump > the result, the > > integer part of values > > are all zeros. Where did I do wrong? Please advise. > > > > #include "mk4.h" > > #include "mk4str.h" > > #include > > main() > > { > > c4_StringProp pName ("name"); > > c4_IntProp pValue ("value"); > > c4_Storage storage("kdeck.dat", true); > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > c4_Row row; > > pName (row) = "proc_cdp_first"; > > pValue(row) = 1; > > kdeck.Add(row); > > pName (row) = "proc_cdp_last"; > > pValue(row) = 676; > > storage.Commit(); > > int v1 = pValue(row); > > printf("V1 = %d\n", v1); > > kdeck.Add(row); > > int v2 (pValue (kdeck[1])); > > printf("V2 = %d\n", v2); > > pName (row) = "proc_cdp_inc"; > > pValue(row) = 1; > > kdeck.Add(row); > > storage.Commit(); > > return(0); > > } > > Sincerely, > > Jim Hu > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jeff@k2.com Thu May 29 14:42:14 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TJgDN18553 for ; Thu, 29 May 2003 14:42:13 -0500 Received: from Klee ([192.168.0.21]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h4TJZ3N26860; Thu, 29 May 2003 15:35:03 -0400 From: "Jeffrey Kay" To: "'Jim Hu'" , Cc: Subject: RE: [Metakit] Help on Int property Date: Thu, 29 May 2003 15:41:24 -0400 Message-ID: <010c01c3261a$49fabc40$017410ac@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal In-Reply-To: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB7@VSMAIL1.CORELAB.COM> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Yup -- that too. I checked my code that uses MetaKit and there are no spaces in the property names. I didn't realize that I did that when I put together the test code. jeffrey kay weblog pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of Jim Hu > Sent: Thursday, May 29, 2003 3:37 PM > To: Jeffrey Kay; jyl@mod3.net > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Jeffrey & Jacob, > > Thank you so much for help. The real problem to cuase wrong data > saved into database I have just find out is because I left a space in > front the word value in the line > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > What you found is also right. I wonder if the space problem > is a Metakit bug or something I do not know. Please advise. > > Many thanks. > > Jim > > -----Original Message----- > From: Jeffrey Kay [mailto:jeff@k2.com] > Sent: Thursday, May 29, 2003 1:38 PM > To: Jim Hu; jyl@mod3.net > Cc: metakit@equi4.com > Subject: RE: [Metakit] Help on Int property > > > Okay, here's a program that works and why (AFAIK). > c4_IntProp does not > have an automatic cast to an int. The operator() returns a c4_IntRef, > which doesn't seem happy about automatically being cast to an int. If > you assign the value directly to an int, or presumably force the cast, > then it should work fine. > > I'm surprised that I never noticed this, but perhaps it's because I > don't usually printf anything. > > #include "mk4.h" > #include "mk4str.h" > #include > > main() > { > c4_StringProp pName("name"); > c4_IntProp pValue("value"); > > c4_Storage storage("kdeck.db",true); > > c4_View kdeck = storage.GetAs("kdeck[name:S,value:I]"); > > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); > > storage.Commit(); > > for(int j=0;j int x = pValue(kdeck[j]); > printf("V%d = %d\n",j,x); > } > > kdeck.Add(pName["proc_cdp_inc"] + pValue[1]); > storage.Commit(); > > return(0); > } > > > > jeffrey kay > weblog pgp key aim > share files with me -- get shinkuro -- > > "first get your facts, then you can distort them at your leisure" -- > mark twain > "if the person in the next lane at the stoplight rolls up the > window and > locks the door, support their view of life by snarling at them" -- a > biker's guide to life > "if A equals success, then the formula is A equals X plus Y > plus Z. X is > work. Y is play. Z is keep your mouth shut." -- albert einstein > > > > -----Original Message----- > > From: metakit-admin@equi4.com > > [mailto:metakit-admin@equi4.com] On Behalf Of Jim Hu > > Sent: Thursday, May 29, 2003 1:00 PM > > To: jyl@mod3.net > > Cc: metakit@equi4.com > > Subject: RE: [Metakit] Help on Int property > > > > > > Jacob, > > > > I have just built and installed metakit 2.4.9.2 on > > ReadHat Linux 7.1.2 > > gcc 2.96. I have got exactly the same wrong reslut as on > > Solaris 5.6 with gcc 2.95.3 > > and ReadHat 6.1 with gcc 2.91.66. > > > > Best regards > > > > JH > > > > -----Original Message----- > > From: Jim Hu > > Sent: Thursday, May 29, 2003 11:29 AM > > To: 'jyl@mod3.net' > > Cc: metakit@equi4.com > > Subject: RE: [Metakit] Help on Int property > > > > > > Jacob, > > > > Many thanks for your reply. I did build the test and they > > are all right to me. I only change Jeff's > > code line 20 from > > > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > > > > to > > > > kdeck.Add(pName["proc_cdp_inc"] + pValue[1]); > > > > I have just posted another message regarding mixture of > > string and integer. > > I will try an newer version of ReadHat and gcc to see if the > > problem still happens. > > > > Best regards > > > > Jim > > > > -----Original Message----- > > From: Jacob Levy [mailto:jyl@mod3.net] > > Sent: Thursday, May 29, 2003 11:08 AM > > To: Jim Hu > > Cc: jeff@k2.com; metakit@equi4.com > > Subject: RE: [Metakit] Help on Int property > > > > > > Jim > > > > The versions of Solaris and GCC you're using are quite old. > > I'm not sure > > whether Metakit has been tested with these. I suggest you do > > a 'make test' > > in Metakit first, to ensure that you're getting a good > build. I don't > > really expect any problems, but it's just one more thing that > > you should > > check to eliminate any possible question marks later on. > > > > I see you're using Metakit 2.4.9.2, that's great, that's the > > right version > > to use. In all, I've found this version to be the best yet, > > rock solid and > > very easy to build/install. > > > > Also, please repost the code you are running with Jeff's > > modifications, so > > we can see if there's any bugs introduced by your adoption of Jeff's > > suggestions. > > > > Thanks! --JYL > > > > > Dear Jeffrey, > > > > > > Further to my last email, I quickly install metakit on > > Solaris 5.6 > > > with gcc 2.95.3. > > > I tested demo and dump, they gave correct results. But when > > I run your > > > code, I got > > > > > > V0 = -268436744 > > > V1 = -268436744 > > > > > > and dump the result gave me the same as on Linux > > > > > > kdeck.dat: 1 properties > > > kdeck[name:S, value:I] > > > > > > VIEW 1 rows = kdeck:V > > > 0: subview 'kdeck' > > > VIEW 3 rows = name:S value:I > > > 0: 'proc_cdp_first' 0 > > > 1: 'proc_cdp_last' 0 > > > 2: 'proc_cdp_inc' 0 > > > > > > I cannot figure out what I have done wrong > > > > > > Best regards > > > > > > Jim > > > > > > -----Original Message----- > > > From: Jim Hu > > > Sent: Thursday, May 29, 2003 9:19 AM > > > To: 'Jeffrey Kay' > > > Subject: RE: [Metakit] Help on Int property > > > > > > > > > Dear Jeffrey, > > > > > > Thank you so much for your reply. I wonder if my > > there is anything > > > wrong with > > > my installation as your code gives the wrong result as well > > (there is a > > > bug in the line 20). The running results are > > > > > > V0 = 134525096 > > > V1 = 134525096 > > > > > > and the dump resluts are > > > > > > kdeck.dat: 1 properties > > > kdeck[name:S, value:I] > > > > > > VIEW 1 rows = kdeck:V > > > 0: subview 'kdeck' > > > VIEW 3 rows = name:S value:I > > > 0: 'proc_cdp_first' 0 > > > 1: 'proc_cdp_last' 0 > > > 2: 'proc_cdp_inc' 0 > > > > > > would you help me run the code on your system. I am use > RedHat 6.1. > > > > > > Best regards > > > > > > Jim > > > > > > -----Original Message----- > > > From: Jeffrey Kay [mailto:jeff@k2.com] > > > Sent: Thursday, May 29, 2003 9:03 AM > > > To: Jim Hu > > > Cc: metakit@equi4.com > > > Subject: RE: [Metakit] Help on Int property > > > > > > > > > See if this simplifies what you are trying to do ... > > > > > > #include "mk4.h" > > > #include "mk4str.h" > > > #include > > > main() > > > { > > > c4_StringProp pName ("name"); > > > c4_IntProp pValue ("value"); > > > c4_Storage storage("kdeck.dat", true); > > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > > > > > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); > > > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); > > > > > > storage.Commit(); > > > > > > for(int j=0;j > > printf("V%d = %d\n",j,pValue(kdeck[j])); > > > } > > > > > > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); > > > storage.Commit(); > > > return(0); > > > } > > > > > > I'm not sure exactly where your logic was heading, but this is the > > > simplest way to add and retrieve values from rows in a view. To > > > dereference a particular row in a view, the syntax is > > c4_View[rownum]. > > > To dereference a particular value in a particular row, it's > > > c4_Prop(c4_View[rownum]). You can assign a particular > row to a row > > > variable also, but for most things I usually use the > approach shown > > > above. Similarly, if I'm tracking down a particular row, I > > use Find or > > > Select and then use the row number to make the property > > changes as shown > > > in this example. There are a couple of things that you > > need to be aware > > > of if you use derived views, but we'll cross that bridge > > when you get to > > > it. > > > > > > jeffrey kay > > > weblog pgp key aim > > > share files with me -- get shinkuro -- > > > > > > "first get your facts, then you can distort them at your > leisure" -- > > > mark twain > > > "if the person in the next lane at the stoplight rolls up > > the window and > > > locks the door, support their view of life by snarling at > them" -- a > > > biker's guide to life > > > "if A equals success, then the formula is A equals X plus Y > > plus Z. X is > > > work. Y is play. Z is keep your mouth shut." -- albert einstein > > > > > > -----Original Message----- > > > From: metakit-admin@equi4.com > > [mailto:metakit-admin@equi4.com] On Behalf > > > Of Jim Hu > > > > > Sent: Thursday, May 29, 2003 9:50 AM > > > To: metakit@equi4.com > > > Subject: [Metakit] Help on Int property > > > > > > > > > Hi, all, > > > I am just starting to use Metakit (2.9.4.2). I used > > the demo.cpp as > > > an example and wrote > > > the following piece of code. When I use the dump to dump > > the result, the > > > integer part of values > > > are all zeros. Where did I do wrong? Please advise. > > > > > > #include "mk4.h" > > > #include "mk4str.h" > > > #include > > > main() > > > { > > > c4_StringProp pName ("name"); > > > c4_IntProp pValue ("value"); > > > c4_Storage storage("kdeck.dat", true); > > > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); > > > c4_Row row; > > > pName (row) = "proc_cdp_first"; > > > pValue(row) = 1; > > > kdeck.Add(row); > > > pName (row) = "proc_cdp_last"; > > > pValue(row) = 676; > > > storage.Commit(); > > > int v1 = pValue(row); > > > printf("V1 = %d\n", v1); > > > kdeck.Add(row); > > > int v2 (pValue (kdeck[1])); > > > printf("V2 = %d\n", v2); > > > pName (row) = "proc_cdp_inc"; > > > pValue(row) = 1; > > > kdeck.Add(row); > > > storage.Commit(); > > > return(0); > > > } > > > Sincerely, > > > Jim Hu > > > > > > _______________________________________________ > > > metakit mailing list - metakit@equi4.com > > > http://www.equi4.com/mailman/listinfo/metakit > > > > > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jcw@equi4.com Thu May 29 14:51:46 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TJpjN19256 for ; Thu, 29 May 2003 14:51:45 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 3005083AEA for ; Thu, 29 May 2003 15:52:30 -0400 (EDT) Date: Thu, 29 May 2003 21:52:29 +0200 Subject: Re: [Metakit] Help on Int property Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB7@VSMAIL1.CORELAB.COM> Message-Id: <14778738-920F-11D7-8877-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jim Hu wrote: > Jeffrey & Jacob, > > Thank you so much for help. (From me too, much appreciated) > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); MK has never supported spaces in the description string. > What you found is also right. Indeed - good catch: printf's don't cause the args to be cast (they cannot - this was one of the reasons why C++ introduced "cout << x" for printing). Conclusion: either assign to an int, or add a cast. (hundreds of quoted lines snipped) Please don't leave so much text in a discussion, it just adds to the noise. -jcw From bkelley@wi.mit.edu Thu May 29 14:55:57 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4TJtuN19610 for ; Thu, 29 May 2003 14:55:56 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003052915564322457 for ; Thu, 29 May 2003 15:56:43 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HFNZEJ00.NB2 for ; Thu, 29 May 2003 15:56:43 -0400 Message-ID: <3ED66650.2040909@wi.mit.edu> Date: Thu, 29 May 2003 15:58:08 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] New KitViewer distribution Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Many lovely things added including documentation. * Sort/Search on tables * Generate Excel reports * Plugin-in architecture for viewing non-standard data. I have an example that shows molecules embedded in a database. Next week, I'll distribute a simple application that shows this capability. url http://staffa.wi.mit.edu/people/kelley/ documentation http://staffa.wi.mit.edu/people/kelley/KitViewer/docs/KitViewer_Docs.html -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From jcw@equi4.com Thu May 29 15:18:00 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TKHxN21701 for ; Thu, 29 May 2003 15:17:59 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 9557A83AEA for ; Thu, 29 May 2003 16:18:44 -0400 (EDT) Date: Thu, 29 May 2003 22:18:42 +0200 Subject: Re: [Metakit] New KitViewer distribution Content-Type: text/plain; delsp=yes; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3ED66650.2040909@wi.mit.edu> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > Many lovely things added including documentation. > > * Sort/Search on tables > * Generate Excel reports > > * Plugin-in architecture for viewing non-standard data. I have an > example that shows molecules embedded in a database. Next week, I'll > distribute a simple application that shows this capability. > > url > http://staffa.wi.mit.edu/people/kelley/ > > documentation > http://staffa.wi.mit.edu/people/kelley/KitViewer/docs/ > KitViewer_Docs.html Cool! Are you using the "wrap" logic of Mk4py for plugging in other data formats? I haven't looked at the source, but with mk4py.wrap(), the result is a MK view, on which all of MK's view relational & set operators can be applied. It might even be possible to make Gordon McMillan's MkSQL run on top, which means that you could tie into various formats - show them with your KitViewer, and do all sorts of pretty powerful stuff *across* multiple datafiles. It's probably too soon to actually try this, but IMO there is quite some potential in that direction. -jcw From gmcm@hypernet.com Thu May 29 15:26:30 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TKQTN22486 for ; Thu, 29 May 2003 15:26:29 -0500 Received: from PARANOIA (204.176.40.136) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Thu, 29 May 2003 16:30:08 -0400 From: "Gordon McMillan" To: Maciej Starzyk Date: Thu, 29 May 2003 16:27:28 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Saving nested views. (fwd) Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3ED634F0.14780.A90417B@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 29 May 2003 at 21:31, Maciej Starzyk wrote: > p = mem_db.getas('parent[name:S,children:V]') > c = mem_db.getas('child[name:S]') > c.append(name='a child') > p.append(name='a parent', children=c) What you want is: p = mem_db.getas('parent[name:S,children[name:S]]') i = p.append(name='a parent') c[i].children.append(name='a child') -- Gordon http://www.mcmillan-inc.com/ From jcw@equi4.com Thu May 29 15:40:27 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TKeRN23702 for ; Thu, 29 May 2003 15:40:27 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 58C4483AEA for ; Thu, 29 May 2003 16:41:12 -0400 (EDT) Date: Thu, 29 May 2003 22:41:10 +0200 Subject: Re: [Metakit] Saving nested views. (fwd) Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: metakit@equi4.com Content-Transfer-Encoding: 7bit In-Reply-To: <3ED634F0.14780.A90417B@localhost> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: > What you want is: > p = mem_db.getas('parent[name:S,children[name:S]]') > i = p.append(name='a parent') > c[i].children.append(name='a child') Yes, but you see, on Gordon's keyboard, "c" and "p" are mapped to the same scancode - on other keyboards, that last line is better written as: p[i].children.append(name='a child') -jpw From bkelley@wi.mit.edu Thu May 29 15:50:06 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4TKo5N24561; Thu, 29 May 2003 15:50:05 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003052916505223070 ; Thu, 29 May 2003 16:50:52 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HFO1WS00.K9I; Thu, 29 May 2003 16:50:52 -0400 Message-ID: <3ED67301.9090701@wi.mit.edu> Date: Thu, 29 May 2003 16:52:17 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler , metakit@equi4.com Subject: Re: [Metakit] New KitViewer distribution References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > > Cool! > > Are you using the "wrap" logic of Mk4py for plugging in other data > formats? Nothing so fancy. You essentially assign a column to a specific renderer. For instance there is a column named zmol in my compound databases. The plugin architecture says that any column named "zmol" gets viewed as a molecule. This allows joins and selects to work appropriately when viewing the new types. Eventally I'll probably add meta_data information similar to the TCL KitViewer proposals. > > I haven't looked at the source, but with mk4py.wrap(), the result is > a MK view, on which all of MK's view relational & set operators can > be applied. It might even be possible to make Gordon McMillan's > MkSQL run on top, which means that you could tie into various formats > - show them with your KitViewer, and do all sorts of pretty powerful > stuff *across* multiple datafiles. My next step is actually to layer atop MkSQL and allow sql expressions. I need to understand the internal format for MkSQL a little better though... I've already had great success sucking in CSV files and using the scripting window to manipulate the data. Once I figured out how to use it, MetaKit is incredibly powerful. By the way, I have some additions I would like to make to the python documentation that should ease people into using the python wrapper. > It's probably too soon to actually try this, but IMO there is quite > some potential in that direction. > > -jcw -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From mac@ceti.pl Thu May 29 16:17:20 2003 Received: from mx2.ceti.pl (mx2.ceti.pl [62.121.128.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TLHJN27132 for ; Thu, 29 May 2003 16:17:20 -0500 Received: from tau.ceti.pl (tau.ceti.pl [62.121.128.11]) by mx2.ceti.pl (Postfix) with ESMTP id C70EC11EA2D; Thu, 29 May 2003 23:18:05 +0200 (CEST) Received: by tau.ceti.pl (Postfix, from userid 2967) id D9714202FE8; Thu, 29 May 2003 23:18:05 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by tau.ceti.pl (Postfix) with ESMTP id D8F261E70C7; Thu, 29 May 2003 23:18:05 +0200 (CEST) Date: Thu, 29 May 2003 23:18:00 +0200 (CEST) From: Maciej Starzyk To: Gordon McMillan Cc: metakit@equi4.com Subject: Re: [Metakit] Saving nested views. (fwd) In-Reply-To: <3ED634F0.14780.A90417B@localhost> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Thu, 29 May 2003, Gordon McMillan wrote: > On 29 May 2003 at 21:31, Maciej Starzyk wrote: > > > p = mem_db.getas('parent[name:S,children:V]') > > c = mem_db.getas('child[name:S]') > > c.append(name='a child') > > p.append(name='a parent', children=c) > > What you want is: > p = mem_db.getas('parent[name:S,children[name:S]]') > i = p.append(name='a parent') > c[i].children.append(name='a child') Gordon, Thanks for your reply. So is it not possible to persist nested views with different structures this way? For example - modifying the original code I would like to be able to say: p = mem_db.getas('parent[name:S,children:V]') c = mem_db.getas('child[name:S]') c2 = mem_db.getas('child_2[first:S,last:S]") c.append(name='a child') c2.append(first='other', last='childk') p.append(name='a parent', children=c) p.append(name='other parent', children=c2) -- Maciej From jyl@mod3.net Thu May 29 16:35:02 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4TLZ1N28723 for ; Thu, 29 May 2003 16:35:02 -0500 Received: (qmail 24468 invoked by uid 48); 29 May 2003 21:33:22 -0000 Received: from 192.18.42.10 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 29 May 2003 14:33:22 -0700 (PDT) Message-ID: <48131.192.18.42.10.1054244002.squirrel@mod3.net> Date: Thu, 29 May 2003 14:33:22 -0700 (PDT) Subject: RE: [Metakit] Help on Int property From: "Jacob Levy" To: In-Reply-To: <010501c32610$2cdb7d20$017410ac@k2.com> References: <875F35D078FEAD48A3CA5A32B5ACC06B41EFB5@VSMAIL1.CORELAB.COM> <010501c32610$2cdb7d20$017410ac@k2.com> X-Priority: 3 Importance: Normal Cc: , , Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I dont know if it matters (I think it actually does) but this test program is using a GetAs() with a format that contains a space. AFAIK (and that ain't much) the format parser is not all that tolerant of spaces. --JYL > Well, you got me on this one also. I've been using int props in other > places without trouble, but this test program doesn't work. I'm looking > into it further. FWIW, I'm running Metakit 2.4.7 on Windows. > > jeffrey kay > weblog pgp key aim > share files with me -- get shinkuro -- > > "first get your facts, then you can distort them at your leisure" -- > mark twain > "if the person in the next lane at the stoplight rolls up the window and > locks the door, support their view of life by snarling at them" -- a > biker's guide to life > "if A equals success, then the formula is A equals X plus Y plus Z. X is > work. Y is play. Z is keep your mouth shut." -- albert einstein > > >> -----Original Message----- >> From: metakit-admin@equi4.com >> [mailto:metakit-admin@equi4.com] On Behalf Of Jim Hu >> Sent: Thursday, May 29, 2003 12:29 PM >> To: jyl@mod3.net >> Cc: metakit@equi4.com >> Subject: RE: [Metakit] Help on Int property >> >> >> Jacob, >> >> Many thanks for your reply. I did build the test and they >> are all right to me. I only change Jeff's >> code line 20 from >> >> kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); >> >> to >> >> kdeck.Add(pName["proc_cdp_inc"] + pValue[1]); >> >> I have just posted another message regarding mixture of >> string and integer. >> I will try an newer version of ReadHat and gcc to see if the >> problem still happens. >> >> Best regards >> >> Jim >> >> -----Original Message----- >> From: Jacob Levy [mailto:jyl@mod3.net] >> Sent: Thursday, May 29, 2003 11:08 AM >> To: Jim Hu >> Cc: jeff@k2.com; metakit@equi4.com >> Subject: RE: [Metakit] Help on Int property >> >> >> Jim >> >> The versions of Solaris and GCC you're using are quite old. >> I'm not sure >> whether Metakit has been tested with these. I suggest you do >> a 'make test' >> in Metakit first, to ensure that you're getting a good build. I don't >> really expect any problems, but it's just one more thing that >> you should >> check to eliminate any possible question marks later on. >> >> I see you're using Metakit 2.4.9.2, that's great, that's the >> right version >> to use. In all, I've found this version to be the best yet, >> rock solid and >> very easy to build/install. >> >> Also, please repost the code you are running with Jeff's >> modifications, so >> we can see if there's any bugs introduced by your adoption of Jeff's >> suggestions. >> >> Thanks! --JYL >> >> > Dear Jeffrey, >> > >> > Further to my last email, I quickly install metakit on >> Solaris 5.6 >> > with gcc 2.95.3. >> > I tested demo and dump, they gave correct results. But when >> I run your >> > code, I got >> > >> > V0 = -268436744 >> > V1 = -268436744 >> > >> > and dump the result gave me the same as on Linux >> > >> > kdeck.dat: 1 properties >> > kdeck[name:S, value:I] >> > >> > VIEW 1 rows = kdeck:V >> > 0: subview 'kdeck' >> > VIEW 3 rows = name:S value:I >> > 0: 'proc_cdp_first' 0 >> > 1: 'proc_cdp_last' 0 >> > 2: 'proc_cdp_inc' 0 >> > >> > I cannot figure out what I have done wrong >> > >> > Best regards >> > >> > Jim >> > >> > -----Original Message----- >> > From: Jim Hu >> > Sent: Thursday, May 29, 2003 9:19 AM >> > To: 'Jeffrey Kay' >> > Subject: RE: [Metakit] Help on Int property >> > >> > >> > Dear Jeffrey, >> > >> > Thank you so much for your reply. I wonder if my >> there is anything >> > wrong with >> > my installation as your code gives the wrong result as well >> (there is a >> > bug in the line 20). The running results are >> > >> > V0 = 134525096 >> > V1 = 134525096 >> > >> > and the dump resluts are >> > >> > kdeck.dat: 1 properties >> > kdeck[name:S, value:I] >> > >> > VIEW 1 rows = kdeck:V >> > 0: subview 'kdeck' >> > VIEW 3 rows = name:S value:I >> > 0: 'proc_cdp_first' 0 >> > 1: 'proc_cdp_last' 0 >> > 2: 'proc_cdp_inc' 0 >> > >> > would you help me run the code on your system. I am use RedHat 6.1. >> > >> > Best regards >> > >> > Jim >> > >> > -----Original Message----- >> > From: Jeffrey Kay [mailto:jeff@k2.com] >> > Sent: Thursday, May 29, 2003 9:03 AM >> > To: Jim Hu >> > Cc: metakit@equi4.com >> > Subject: RE: [Metakit] Help on Int property >> > >> > >> > See if this simplifies what you are trying to do ... >> > >> > #include "mk4.h" >> > #include "mk4str.h" >> > #include >> > main() >> > { >> > c4_StringProp pName ("name"); >> > c4_IntProp pValue ("value"); >> > c4_Storage storage("kdeck.dat", true); >> > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); >> > >> > kdeck.Add(pName["proc_cdp_first"] + pValue[1]); >> > kdeck.Add(pName["proc_cdp_last"] + pValue[676]); >> > >> > storage.Commit(); >> > >> > for(int j=0;j> > printf("V%d = %d\n",j,pValue(kdeck[j])); >> > } >> > >> > kdeck.Add(pValue["proc_cdp_inc"] + pValue[1]); >> > storage.Commit(); >> > return(0); >> > } >> > >> > I'm not sure exactly where your logic was heading, but this is the >> simplest way to add and retrieve values from rows in a view. To >> dereference a particular row in a view, the syntax is >> c4_View[rownum]. >> > To dereference a particular value in a particular row, it's >> > c4_Prop(c4_View[rownum]). You can assign a particular row to a row >> variable also, but for most things I usually use the approach shown >> above. Similarly, if I'm tracking down a particular row, I >> use Find or >> > Select and then use the row number to make the property >> changes as shown >> > in this example. There are a couple of things that you >> need to be aware >> > of if you use derived views, but we'll cross that bridge >> when you get to >> > it. >> > >> > jeffrey kay >> > weblog pgp key aim >> > share files with me -- get shinkuro -- >> > >> > "first get your facts, then you can distort them at your leisure" -- >> mark twain >> > "if the person in the next lane at the stoplight rolls up >> the window and >> > locks the door, support their view of life by snarling at them" -- a >> biker's guide to life >> > "if A equals success, then the formula is A equals X plus Y >> plus Z. X is >> > work. Y is play. Z is keep your mouth shut." -- albert einstein >> > >> > -----Original Message----- >> > From: metakit-admin@equi4.com >> [mailto:metakit-admin@equi4.com] On Behalf >> > Of Jim Hu >> > >> Sent: Thursday, May 29, 2003 9:50 AM >> > To: metakit@equi4.com >> > Subject: [Metakit] Help on Int property >> > >> > >> > Hi, all, >> > I am just starting to use Metakit (2.9.4.2). I used >> the demo.cpp as >> > an example and wrote >> > the following piece of code. When I use the dump to dump >> the result, the >> > integer part of values >> > are all zeros. Where did I do wrong? Please advise. >> > >> > #include "mk4.h" >> > #include "mk4str.h" >> > #include >> > main() >> > { >> > c4_StringProp pName ("name"); >> > c4_IntProp pValue ("value"); >> > c4_Storage storage("kdeck.dat", true); >> > c4_View kdeck = storage.GetAs("kdeck[name:S, value:I]"); >> c4_Row row; >> > pName (row) = "proc_cdp_first"; >> > pValue(row) = 1; >> > kdeck.Add(row); >> > pName (row) = "proc_cdp_last"; >> > pValue(row) = 676; >> > storage.Commit(); >> > int v1 = pValue(row); >> > printf("V1 = %d\n", v1); >> > kdeck.Add(row); >> > int v2 (pValue (kdeck[1])); >> > printf("V2 = %d\n", v2); >> > pName (row) = "proc_cdp_inc"; >> > pValue(row) = 1; >> > kdeck.Add(row); >> > storage.Commit(); >> > return(0); >> > } >> > Sincerely, >> > Jim Hu >> > >> > _______________________________________________ >> > metakit mailing list - metakit@equi4.com >> > http://www.equi4.com/mailman/listinfo/metakit >> >> >> >> _______________________________________________ >> metakit mailing list - metakit@equi4.com >> http://www.equi4.com/mailman/listinfo/metakit >> > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From lvirden@cas.org Thu May 29 18:31:17 2003 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h4TNVHN06165 for ; Thu, 29 May 2003 18:31:17 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.12.9/8.12.9/CAS_MAIL_HUB-2.00) with ESMTP id h4TNW0Vr021955 for ; Thu, 29 May 2003 19:32:00 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.11.6+Sun/m4_8.11.6/CAS_CLIENT-1.18) id h4TNW0r00160; Thu, 29 May 2003 19:32:00 -0400 (EDT) Date: Thu, 29 May 2003 19:32:00 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20030529193200.AAB149@cas.org> Subject: Re: [Metakit] Help on Int property In-Reply-To: Your message of Thu, 29 May 2003 09:07:33 -0700 (PDT) To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Jacob Levy" > The versions of Solaris and GCC you're using are quite old. I'm not sure > whether Metakit has been tested with these. Well, at the very least, the amount of metakit needed for starkits has been tested with this combo of OS and gcc - that's what I use. I am not familar enough with metakit and c++ to provide any debugging assistance though. -- Tcl - The glue of a new generation. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From bkelley@wi.mit.edu Fri May 30 11:49:43 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h4UGnhN06238; Fri, 30 May 2003 11:49:43 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003053012503029984 ; Fri, 30 May 2003 12:50:30 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HFPLG600.CGI; Fri, 30 May 2003 12:50:30 -0400 Message-ID: <3ED78C2A.3030304@wi.mit.edu> Date: Fri, 30 May 2003 12:51:54 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler , metakit@equi4.com Subject: Re: [Metakit] Help on Int property References: <14778738-920F-11D7-8877-000A957664F8@equi4.com> In-Reply-To: <14778738-920F-11D7-8877-000A957664F8@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > > MK has never supported spaces in the description string. By support do you mean unintended consequences? I've noticed that it seems to work just fine on windows, at least through the python wrapper: import metakit db = metakit.storage() vw = db.getas("test[first field, second field]") print vw.structure() # [Property('S', 'first field'), Property('S', ' second field')] property = metakit.property("S", " abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#-$%^&*()~`,./<>?:\";'") print property #Property('S', # ' abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#-$%^&*()~`,./<>?:";'') vw.addproperty(property) print vw.structure() #[Property('S', 'first field'), Property('S', ' second field'), # Property('S', ' abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#-$%^&*()~`,./<>?:";'')] This has actually been quite useful for KitViewer in that I can take *any* header description from a CSV file as long as I strip out commas. [As a side note, you cannot use the standard (value = row.abcde 123) technique to get values, you have to use (getattr(row, "abcde 123")] Now from the python side, a more critical error is that sometimes the following happens: vw = db.getas("bad_table[A:S,B:S,A:I,B:I]") vw.structure() ==> SystemError: error return without exception set vw.append(("this", "is", 0, 0)) Traceback (most recent call last): File "", line 1, in ? SystemError: NULL result without error in PyObject_Call I think after the first failure to db.getas no other error is caught, I'll look at the source code to see if an exception can be set properly. I've even had the interpreter crash on me. My solution was to write a wrapper around db.getas(...) that parses the command for errors like this and returns a reasonable error message. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From mac@ceti.pl Tue Jun 3 18:50:51 2003 Received: from mx2.ceti.pl (mx2.ceti.pl [62.121.128.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h53NooN17880 for ; Tue, 3 Jun 2003 18:50:51 -0500 Received: from tau.ceti.pl (tau.ceti.pl [62.121.128.11]) by mx2.ceti.pl (Postfix) with ESMTP id 0D00311E5A3 for ; Wed, 4 Jun 2003 01:51:42 +0200 (CEST) Received: from mstarzyk (unknown [195.46.43.178]) by tau.ceti.pl (Postfix) with ESMTP id 12743202FEB for ; Wed, 4 Jun 2003 01:51:41 +0200 (CEST) Date: Wed, 4 Jun 2003 01:51:51 +0200 From: Maciej Starzyk X-Mailer: The Bat! (v1.62r) Personal Reply-To: Maciej Starzyk X-Priority: 3 (Normal) Message-ID: <8510399318.20030604015151@ceti.pl> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Problem with sortrev Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, Today I found a strange behavior of sortrev. Here is the script: H:\maciek\mk_test> type ss.py import metakit as m db = m.storage() v = db.getas('a[x:S]') for x in ['a', 'c', 'b']: v.append(x=x) def show (view, label): print "%-4s: %s" % (label, " ".join([r.x for r in view])) ## Original view show(v, "orig") ## Sorted by x show(v.sort(v.x), "sort") ## Reverse sorted by x show(v.sortrev((), (v.x,)), "rev") ## Now add a new property `idx' to the view with ## the same values as `x' v.addproperty(m.property('S', 'idx')) def mkidx(row): row.idx = row.x v.map(mkidx) print m.dump(v) print ## Original view show(v, "orig") ## Sorted by idx show(v.sort(v.idx), "sort") ## Reverse sorted by idx show(v.sortrev((), (v.idx,)), "rev") And here is the output: H:\maciek\mk_test> python ss.py orig: a c b sort: a b c rev : c b a x idx - --- a a c c b b - --- Total: 3 rows orig: a c b sort: a b c rev : a b c I was expecting the sorting on idx would have the same effect as sorting on x. Sort looks good, but sortrev seems to be broken. Could anyone please explain it ? Thanks, Maciej From gmcm@hypernet.com Tue Jun 3 20:36:42 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h541agN24389 for ; Tue, 3 Jun 2003 20:36:42 -0500 Received: from PARANOIA (204.176.40.94) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Tue, 3 Jun 2003 21:37:32 -0400 From: "Gordon McMillan" To: Maciej Starzyk , metakit@equi4.com Date: Tue, 3 Jun 2003 21:34:41 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Problem with sortrev Reply-to: gmcm@hypernet.com Message-ID: <3EDD1471.17969.25695067@localhost> Priority: normal In-reply-to: <8510399318.20030604015151@ceti.pl> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 4 Jun 2003 at 1:51, Maciej Starzyk wrote: > Hello, > > Today I found a strange behavior of sortrev. > Here is the script: [...] > show(v.sortrev((), (v.x,)), "rev") This should be v.sortrev((v.x,), (v.x,)) With your simple view it works as expected here, but not in general. The syntax may look odd, but it lets you do complex ordering: v.sortrev((v.a, v.b, v.c), (v.b,)) is ascending on a, descending on b, ascending on c. -- Gordon http://www.mcmillan-inc.com/ From mac@ceti.pl Wed Jun 4 13:03:58 2003 Received: from mx2.ceti.pl (mx2.ceti.pl [62.121.128.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h54I3vN28514 for ; Wed, 4 Jun 2003 13:03:57 -0500 Received: from tau.ceti.pl (tau.ceti.pl [62.121.128.11]) by mx2.ceti.pl (Postfix) with ESMTP id 085F311E9B1; Wed, 4 Jun 2003 20:04:45 +0200 (CEST) Received: from mstarzyk (unknown [195.46.43.178]) by tau.ceti.pl (Postfix) with ESMTP id 79E0C202FEA; Wed, 4 Jun 2003 20:04:50 +0200 (CEST) Date: Wed, 4 Jun 2003 20:04:57 +0200 From: Maciej Starzyk X-Mailer: The Bat! (v1.62r) Personal Reply-To: Maciej Starzyk X-Priority: 3 (Normal) Message-ID: <1951809253.20030604200457@ceti.pl> To: Gordon McMillan Cc: metakit@equi4.com Subject: Re[2]: [Metakit] Problem with sortrev In-Reply-To: <3EDD1471.17969.25695067@localhost> References: <3EDD1471.17969.25695067@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Wednesday, June 4, 2003, 3:34:41 AM, you wrote: GM> This should be GM> v.sortrev((v.x,), (v.x,)) GM> With your simple view it works as expected here, but not in GM> general. GM> The syntax may look odd, but it lets you do complex ordering: GM> v.sortrev((v.a, v.b, v.c), (v.b,)) GM> is ascending on a, descending on b, ascending on c. Thanks Gordon, now it makes sense. I thought sortrev sorted a view in ascending order on the properties from the first param tuple, and in descending order on properties from the second tuple. The api description is a little ambiguous in that matter :) Maciej From j.p.fletcher@aston.ac.uk Mon Jun 9 05:28:39 2003 Received: from mercury.aston.ac.uk (mercury.aston.ac.uk [134.151.79.53]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h59AScN12393 for ; Mon, 9 Jun 2003 05:28:39 -0500 Received: from [134.151.79.13] (helo=email.aston.ac.uk) by mercury.aston.ac.uk with esmtp (Exim 3.30 #1) id 19PJrG-0003MV-00; Mon, 09 Jun 2003 11:26:34 +0100 Received: from ceac133 ([134.151.50.133]) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 19PJu8-0003YD-00; Mon, 09 Jun 2003 11:29:32 +0100 From: "John Fletcher" To: "Bob Hicks" Date: Mon, 9 Jun 2003 11:23:49 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT CC: metakit@equi4.com Priority: normal In-reply-to: <000c01c32624$e5ed2cf0$25354644@ptah> References: X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Subject: [Metakit] Ruby bindings using SWIG Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bob I have made quite a lot of progress with binding Metakit, using SWIG (http://www.swig.org/). All that this needs, once you have SWIG, is to make a local copy of the Metakit header and run this SWIG file ----------------------- /* File : mk4.i */ %module mk4 %{ #include "mk4.h" %} /* First attempt at mk4 interface */ %include "mk4.h" ---------------------- Here is a translation of part of the Metakit demo. Notice (1) that all the Metakit classes have an initial capital letter. This is a requirement of Ruby and the translation is done by SWIG, which also gives a warning. (2) In this simple version some functionality is lost, in particular [ ] operations on the view. Some of this can be restored by adding things to the SWIG interface. I was concerned to get something operational. (3) I have only tried this on Linux using SWIG 1.3.16 and gcc 3.2. ---------------------------- # test file for ruby interface for Metakit # load the package, which must have a lower case name. require 'mk4' include Mk4 puts "A first test of Metakit running via Ruby" puts "John Fletcher June 2003 J.P.Fletcher@aston.ac.uk" puts "This is intended as a translation of demo.cpp" pName = C4_StringProp.new("name") pCountry = C4_StringProp.new("country") storage = C4_Storage.new("myfile.dat",1) vAddress = storage.GetAs("address[name:S,country:S]") row = C4_Row.new() # These do not translate directly #pName (row) = "John Williams" #pCountry (row) = "UK" #row = pName["John Williams"] #row.pName = "John Williams" # This has to be used instead. pName.Set(row,"John Williams") pCountry.Set(row,"UK") vAddress.Add(row) pName.Set(row,"Paco Pena") pCountry.Set(row,"Spain") vAddress.Add(row) # This has added two records. # Here is how to get data back again s1 = pName.Get(vAddress.GetAt(1)) s2 = pCountry.Get(vAddress.GetAt(1)) puts "The country of #{s1} is #{s2}" storage.Commit() ------------------- Example output: A first test of Metakit running via Ruby John Fletcher June 2003 J.P.Fletcher@aston.ac.uk This is intended as a translation of demo.cpp The country of Paco Pena is Spain John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From jcw@equi4.com Mon Jun 9 15:09:00 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h59K90N20061 for ; Mon, 9 Jun 2003 15:09:00 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 0251283B12 for ; Mon, 9 Jun 2003 16:09:54 -0400 (EDT) Date: Mon, 9 Jun 2003 22:09:54 +0200 Subject: Re: [Metakit] Ruby bindings using SWIG Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <55EE9C0A-9AB6-11D7-93F6-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John, > I have made quite a lot of progress with binding Metakit, using > SWIG (http://www.swig.org/). All that this needs, once you have > SWIG, is to make a local copy of the Metakit header and run this > SWIG file > > ----------------------- > /* File : mk4.i */ > %module mk4 > > %{ > #include "mk4.h" > %} > > /* First attempt at mk4 interface */ > > %include "mk4.h" > ---------------------- Thanks for sharing. Looks like SWIG has come a *long* way since I last looked at it! Whee - this means Perl and Java binding might now also be within easy reach... Is it ok to add your .i file and Ruby sample code on one of the pages on www.equi4.com? I'd like to support this and help people who want to create more bindings and sample code. Or perhaps wait for things to settle a bit more? -jcw From j.p.fletcher@aston.ac.uk Tue Jun 10 04:16:29 2003 Received: from mercury.aston.ac.uk (mercury.aston.ac.uk [134.151.79.53]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5A9GTN12862 for ; Tue, 10 Jun 2003 04:16:29 -0500 Received: from [134.151.79.13] (helo=email.aston.ac.uk) by mercury.aston.ac.uk with esmtp (Exim 3.30 #1) id 19PfD1-0006et-00 for metakit@equi4.com; Tue, 10 Jun 2003 10:14:27 +0100 Received: from ceac133 ([134.151.50.133]) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 19PfBA-00030A-00 for metakit@equi4.com; Tue, 10 Jun 2003 10:12:32 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Tue, 10 Jun 2003 10:06:47 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] Ruby bindings using SWIG Priority: normal References: In-reply-to: <55EE9C0A-9AB6-11D7-93F6-000A957664F8@equi4.com> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Subject: Re: [Metakit] Ruby bindings using SWIG From: Jean-Claude Wippler To: Metakit mailing list Date sent: Mon, 9 Jun 2003 22:09:54 +0200 > John, > > > I have made quite a lot of progress with binding Metakit, using > > SWIG (http://www.swig.org/). All that this needs, once you have > > SWIG, is to make a local copy of the Metakit header and run this > > SWIG file > > > > ----------------------- > > /* File : mk4.i */ > > %module mk4 > > > > %{ > > #include "mk4.h" > > %} > > > > /* First attempt at mk4 interface */ > > > > %include "mk4.h" > > ---------------------- > > Thanks for sharing. Looks like SWIG has come a *long* way since I last > looked at it! Whee - this means Perl and Java binding might now also > be within easy reach... > Yes it has. Support for templates and overloaded operators is much improved. > Is it ok to add your .i file and Ruby sample code on one of the pages > on www.equi4.com? I'd like to support this and help people who want to > create more bindings and sample code. Or perhaps wait for things to > settle a bit more? > See below > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > JCW Have a look at http://www.equi4.com/metakit/wiki.cgi/224 on the metakit wiki. I have already put the same code up there. If you want to make it Ruby page on your web site as well, feel free to do so. I suggest you put in a link to SWIG. I searched for SWIG and Ruby on the MK wiki before I started this and found nothing. I am sure what is posted can be improved to add back the [ ] operator. = operator has to be renamed as e.g. equals != operator is not needed (implemented in Ruby as !(a == b) ) The big one is whether to rename all the classes in Metakit with capital letters. I have a version of the mk4.i where I use %rename a lot, which gets rid of a lot of the SWIG warnings. In most cases I think this will do all that is needed. The exception I have found (with Metakit) is when a renamed object is a member in a C++ class which is also translated. e.g. I have a class which has a member object of type e4_Storage (the E4Graph equivalent of c4_Storage ). I cannot get that properly renamed by SWIG to be type E4_Storage and access to it from Ruby fails. The workaround is to reimplement the needed member functions in the upper class. This means that from a strategic point of view, implementations will start to be target language specific. For example the Ruby interface has direct support for the STL vector and string via an stl.i I am happy to post all that I have done so far on Metakit/Ruby. I will do so in the next few days. I am not interested to develop the interface Metakit interface myself, as my main interest is in E4Graph, which I have also done, including E4xml. This hides me from direct use of Metakit. It may be for the most efficient operation that SWIG is a stepping stone to a crafted implementation. What it certainly does is generate something that works very quickly. Cheers John ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From isaac.ben@brin.com Tue Jun 10 12:42:02 2003 Received: from mailrelay.brin.com (mailrelay.brin.com [12.23.97.23]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5AHg2N32380 for ; Tue, 10 Jun 2003 12:42:02 -0500 Received: (qmail 32321 invoked from network); 10 Jun 2003 17:23:33 -0000 Received: from cl-10150.brin.com (HELO ?172.18.1.82?) (12.23.101.50) by mailrelay.brin.com with SMTP; 10 Jun 2003 17:23:33 -0000 From: Isaac Ben Reply-To: isaac.ben@brin.com To: metakit Content-Type: text/plain Organization: Message-Id: <1055266952.25472.11.camel@ib.dev.brin.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.4- Date: 10 Jun 2003 11:42:32 -0600 Content-Transfer-Encoding: 7bit Subject: [Metakit] make install error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I've been trying to get metakit (both cvs and latest tarball) to compile with no success. Actually it compiles but won't install. The following is the output of the make install command: make install mkdir -p /usr/include /usr/lib /bin/sh ./libtool --mode=install /bin/install -c -m 644 ../unix/../include/mk4.h \ ../unix/../include/mk4.inl \ ../unix/../include/mk4str.h \ ../unix/../include/mk4str.inl /usr/include /bin/install -c -m 644 ../unix/../include/mk4.h /usr/include/mk4.h /bin/install -c -m 644 ../unix/../include/mk4.inl /usr/include/mk4.inl /bin/install -c -m 644 ../unix/../include/mk4str.h /usr/include/mk4str.h /bin/install -c -m 644 ../unix/../include/mk4str.inl /usr/include/mk4str.inl /bin/sh ./libtool --mode=install /bin/install -c libmk4.la /usr/lib /bin/install -c .libs/libmk4.lai /usr/lib/libmk4.la /bin/install: cannot stat `.libs/libmk4.lai': No such file or directory make: *** [install-mk] Error 1 As far as I can tell the file libmk4.lai doesn't exist anywhere. Any suggestions would be appreciated. Thanks, IB Isaac Ben --------------- I learned to play guitar just to get the girls, and anyone who says they didn't is just lyin'! -- Willie Nelson From Jamshed.Qureshi@umit.com Fri Jun 13 08:46:40 2003 Received: from jamshed ([212.15.17.4]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5DDkXN29764 for ; Fri, 13 Jun 2003 08:46:36 -0500 Received: from umit.com ([127.0.0.1]) by jamshed with Microsoft SMTPSVC(5.0.2195.5329); Fri, 13 Jun 2003 16:49:30 +0300 Message-ID: <3EE9D66A.5000403@umit.com> Date: Fri, 13 Jun 2003 16:49:30 +0300 From: Jamshed Qureshi Reply-To: Jamshed Qureshi - UMIT Teknoloji User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en, tr MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 13 Jun 2003 13:49:30.0769 (UTC) FILETIME=[9D312810:01C331B2] Subject: [Metakit] MK4TCL 2.4.9.2 Error: no storage with this name Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, Tcl version 8.4. I'm kind of locked in this strange scenario i hop i can get some help here. I have a proc which takes params and wirtes to a mkdata file using mk::row append. This view is simply a eventlog view. so i'm logging info/warning/error events into it. When i run the tcl program i'm getting this error: "no storage with this name" Also when i put the mk::row append code between the program code it works. I'm getting this error when i call this proc to do the append view. Has anyone seen this before.. regards, Jamshed From Jamshed.Qureshi@umit.com Fri Jun 13 09:13:45 2003 Received: from jamshed ([212.15.17.4]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5DEDgN31799 for ; Fri, 13 Jun 2003 09:13:43 -0500 Received: from umit.com ([127.0.0.1]) by jamshed with Microsoft SMTPSVC(5.0.2195.5329); Fri, 13 Jun 2003 17:18:10 +0300 Message-ID: <3EE9DD21.4010207@umit.com> Date: Fri, 13 Jun 2003 17:18:09 +0300 From: Jamshed Qureshi Reply-To: Jamshed Qureshi - UMIT Teknoloji User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en, tr MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] MK4TCL 2.4.9.2 Error: no storage with this name References: <3EE9D66A.5000403@umit.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-OriginalArrivalTime: 13 Jun 2003 14:18:10.0201 (UTC) FILETIME=[9E0DA890:01C331B6] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sorry guys for the call. i finally figured it out it. best regards, Jamshed Jamshed Qureshi wrote: > Hi all, > Tcl version 8.4. > I'm kind of locked in this strange scenario i hop i can get some help > here. > I have a proc which takes params and wirtes to a mkdata file using > mk::row append. > This view is simply a eventlog view. so i'm logging > info/warning/error events into it. > When i run the tcl program i'm getting this error: > "no storage with this name" > > Also when i put the mk::row append code between the program code it > works. I'm getting this error when i call this proc to do the append > view. > > Has anyone seen this before.. > regards, > Jamshed > > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- Ümit Teknoloji Systems Management Experts Tel: +90(216) 356 09 47 / 386 66 16 Fax:+90(216) 356 60 27 Support: support@umit.com Web: www.umit.com The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. From IchWeissAlles@t-online.de Wed Jun 18 09:01:00 2003 Received: from mailout05.sul.t-online.com (mailout05.sul.t-online.com [194.25.134.82]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5IE0xN17072 for ; Wed, 18 Jun 2003 09:00:59 -0500 Received: from fwd10.aul.t-online.de by mailout05.sul.t-online.com with smtp id 19SdVk-0003V8-00; Wed, 18 Jun 2003 16:02:04 +0200 Received: from t-online.de (ZkDizBZXreqzBjpkovAe8VWDAxdO+fUkbm6+PTzqLJXbmcabOMqDkS@[217.84.89.193]) by fwd10.sul.t-online.com with esmtp id 19SdVP-12LRY00; Wed, 18 Jun 2003 16:01:43 +0200 Message-ID: <3EF07120.6090406@t-online.de> Date: Wed, 18 Jun 2003 16:03:12 +0200 From: IchWeissAlles@t-online.de (juri) User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030401 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Seen: false X-ID: ZkDizBZXreqzBjpkovAe8VWDAxdO+fUkbm6+PTzqLJXbmcabOMqDkS@t-dialin.net Subject: [Metakit] newbie needs help Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: hi, hi, this is the first time i use a database in one of my programms. i have a few questions on the c++ API,hope that someone can help me. 1) how do i search the database ? let´s say i have 20 entry´s "adresses". 8 of them live in "arlington road". now how do i get these 8 entry´s ? 2)about view in views: how do i add a "sub-view" phone numbers (like in the example" with the SAME name to EVERY adress..and how can i access them ?? (i want to access them via adress[x].number..) 3) can i add sub-views with the exact name and dynamic size like: 20 adresses. each with a sub-view telephone,private or buissiness. now some have no buisiness numbers,some have up to 200... when i tryed this i could always only get thenumberof(last entry). i have no idea how the syntax would look for adressOfAlexander.phoneprivate.schowemall or adressOfAlexander.phoneprivate.how many are there ? 4) how am i supposed to find out myself ? i read the doc´s and the examples but it´s pretty hard to get into.also most examples like catfish are only running on windows. i´m using linux/gcc. thx a lot,jurriaan //sorry for my bad englisch From bkelley@wi.mit.edu Fri Jun 20 08:46:07 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5KDk6N15764 for ; Fri, 20 Jun 2003 08:46:07 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003062009470910014 for ; Fri, 20 Jun 2003 09:47:09 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HGS8YL00.CC3 for ; Fri, 20 Jun 2003 09:47:10 -0400 Message-ID: <3EF310B8.4070308@wi.mit.edu> Date: Fri, 20 Jun 2003 09:48:40 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Metakit writer/reader question Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been playing around with metakit using a single writer and multiple readers. The setup is something like this multiple threaded mysql oracle ---> single writer ---> storage <--- multiple readers etc queries I realize that if the writer writes to the storage the readers need to be "reset" otherwise the view will be corrupted. But, on inspection, this seems to only affect the table(s) that are being written modified. So my question is if you have a reader looking at a view that is not being modified while another view is being modified do you have to update the reader on commits to avoid corruption or am I just being lucky? As you can probably tell from the diagram, I am using metakit as a local "sandbox" to play with data extracted from various sql queries. I can't tell you how incredibly powerful this is, especially in combination with Gordon's MkSQL package. KitViewer 2.0 will include this external database import feature in the near future. Now if we only had NULL support ;) -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From janko@mind-nest.com Mon Jun 23 13:49:02 2003 Received: from dopey.kelt.si ([195.246.8.57]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5NIn1N06174 for ; Mon, 23 Jun 2003 13:49:01 -0500 Received: (qmail 11663 invoked from network); 23 Jun 2003 18:50:12 -0000 Received: from vo243-90.dial-up.volja.net (HELO jankotov) (janko?mind-nest.com@217.72.90.243) by 195.246.8.57 with SMTP; 23 Jun 2003 18:50:12 -0000 Message-ID: <016901c339b7$ebaa22a0$2dc8fea9@jankotov> From: To: Date: Mon, 23 Jun 2003 20:47:36 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0166_01C339C8.ADD3FEE0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Subject: [Metakit] commit trouble Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0166_01C339C8.ADD3FEE0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Hi I just tried metakit for python.... I bumpet in some unexpected = behaviour in my first explorations... Is this a bug or am I doing = something wrong... >>> import metakit >>> db =3D metakit.storage("datafile.mk",1) >>> vw =3D db.getas("people[first:S,last:S,shoesize:I]") >>> vw.append(first=3D'John',last=3D'Lennon',shoesize=3D44) 1 >>> db.commit() >>> db =3D metakit.storage("datafile.mk",0) >>> vw =3D db.getas("people[first:S,last:S,shoesize:I]") >>> vw.append(first=3D'Flash',last=3D'Gordon',shoesize=3D42) 2 >>> db.commit() Traceback (most recent call last): File "", line 1, in ? db.commit() IOError: commit failed >>>=20 As you see, if I open existed db I dails to commit. If I open new it = commits. I am doing this on=20 win98 tahanks form making metakit :janko ------=_NextPart_000_0166_01C339C8.ADD3FEE0 Content-Type: text/html; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable
Hi I just tried metakit for = python.... I bumpet=20 in some unexpected behaviour in my first explorations... Is this a bug = or am I=20 doing something wrong...
 
>>> import = metakit
>>> db =3D=20 metakit.storage("datafile.mk",1)
>>> vw =3D=20 db.getas("people[first:S,last:S,shoesize:I]")
>>>=20 vw.append(first=3D'John',last=3D'Lennon',shoesize=3D44)
1
>>&= gt;=20 db.commit()
>>> db =3D=20 metakit.storage("datafile.mk",0)
>>> vw =3D=20 db.getas("people[first:S,last:S,shoesize:I]")
>>>=20 vw.append(first=3D'Flash',last=3D'Gordon',shoesize=3D42)
2
>>= >=20 db.commit()
Traceback (most recent call last):
  File=20 "<pyshell#8>", line 1, in ?
    = db.commit()
IOError:=20 commit failed
>>>
 
As you see, if I open existed db I = dails to=20 commit. If I open new it commits. I am doing this on
win98
 
tahanks form making = metakit
:janko
------=_NextPart_000_0166_01C339C8.ADD3FEE0-- From gmcm@hypernet.com Mon Jun 23 13:57:00 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5NIv0N06816 for ; Mon, 23 Jun 2003 13:57:00 -0500 Received: from PARANOIA (204.176.40.55) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Mon, 23 Jun 2003 14:59:33 -0400 From: "Gordon McMillan" To: Date: Mon, 23 Jun 2003 14:57:09 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] commit trouble Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3EF71545.28201.25A0515B@localhost> Priority: normal In-reply-to: <016901c339b7$ebaa22a0$2dc8fea9@jankotov> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 23 Jun 2003 at 20:47, janko@mind-nest.com wrote: > >>> db = metakit.storage("datafile.mk",0) > >>> vw = db.getas("people[first:S,last:S,shoesize:I]") > >>> vw.append(first='Flash',last='Gordon',shoesize=42) > 2 > >>> db.commit() > Traceback (most recent call last): > File "", line 1, in ? > db.commit() > IOError: commit failed > >>> > > As you see, if I open existed db I dails to commit. If I open new it commits. No, you opened it read-only. 1 means "for update". (The append is fine because it only modified memory). -- Gordon http://www.mcmillan-inc.com/ From janko@mind-nest.com Mon Jun 23 14:09:50 2003 Received: from dopey.kelt.si ([195.246.8.57]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5NJ9nN07889 for ; Mon, 23 Jun 2003 14:09:49 -0500 Received: (qmail 13772 invoked from network); 23 Jun 2003 19:10:58 -0000 Received: from vo243-90.dial-up.volja.net (HELO jankotov) (janko?mind-nest.com@217.72.90.243) by 195.246.8.57 with SMTP; 23 Jun 2003 19:10:58 -0000 Message-ID: <01ab01c339ba$d3ac71a0$2dc8fea9@jankotov> From: To: Subject: Re: [Metakit] commit trouble Date: Mon, 23 Jun 2003 21:08:26 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_01A8_01C339CB.96B82260" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_01A8_01C339CB.96B82260 Content-Type: text/plain; charset="windows-1250" Content-Transfer-Encoding: quoted-printable Thank you very much Gordon. I was a little mislead by the pyndex docs = which stated that 1 is for creating and 0 is for opening existing db. I = will notify Amir too. :janko www.mind-nest.com ------=_NextPart_000_01A8_01C339CB.96B82260 Content-Type: text/html; charset="windows-1250" Content-Transfer-Encoding: quoted-printable
Thank you very much Gordon. I was a = little=20 mislead by the pyndex docs which stated that 1 is for creating and 0 is = for=20 opening existing db. I will notify Amir too.
 
:janko
www.mind-nest.com
------=_NextPart_000_01A8_01C339CB.96B82260-- From guenther.fischer@hrz.tu-chemnitz.de Tue Jun 24 06:57:05 2003 Received: from tom.hrz.tu-chemnitz.de (tom.hrz.tu-chemnitz.de [134.109.132.38]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5OBv4N18094 for ; Tue, 24 Jun 2003 06:57:04 -0500 Received: from hedwig.hrz.tu-chemnitz.de ([134.109.132.173] helo=mailbox.hrz.tu-chemnitz.de) by tom.hrz.tu-chemnitz.de with esmtp (Exim 4.12) id 19UmRC-0006Qp-00 for metakit@equi4.com; Tue, 24 Jun 2003 13:58:14 +0200 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46]) by mailbox.hrz.tu-chemnitz.de with esmtp (Exim 4.20) id 19UmRB-0000m7-Qg for metakit@equi4.com; Tue, 24 Jun 2003 13:58:13 +0200 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.11.6/8.11.6) with ESMTP id h5OBwDe01917 for ; Tue, 24 Jun 2003 13:58:13 +0200 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Tue, 24 Jun 2003 13:58:13 +0200 (MEST) From: Guenther Fischer To: metakit@equi4.com In-Reply-To: <3EBA9ADB.4020605@wi.mit.edu> Message-ID: References: <3EBA841B.2090808@wi.mit.edu> <3EBA9ADB.4020605@wi.mit.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -1.4 (-) X-Scanner: exiscan for exim4 (http://duncanthrax.net/exiscan/) *19UmRC-0006Qp-00*rOMvomEtry6* Subject: [Metakit] corrupt database Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I got a corrupt database from a user. My application (tcl) got "Speicherzugriffsfehler" - I think its segmentation violation. mk_manager.kit says the same. With kitviewer, I can load the db, but if I try some kind of operation the same error. Is the a simple programm like mk_cp db1 db2, which tries to copy correct parts of the corrupt db? The corrupt db come from windows - I don't know how this happens, but because of the positive loading in kitviewer I had some hope to save some parts of it. If I try to read step by step with tcl I can read the first record but no one after it ... -- ~Guenther Fischer From jyl@mod3.net Tue Jun 24 10:04:46 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5OF4kN00629 for ; Tue, 24 Jun 2003 10:04:46 -0500 Received: (qmail 30213 invoked by uid 48); 24 Jun 2003 15:05:55 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Tue, 24 Jun 2003 08:05:54 -0700 (PDT) Message-ID: <64783.209.233.24.218.1056467154.squirrel@mod3.net> Date: Tue, 24 Jun 2003 08:05:54 -0700 (PDT) Subject: Re: [Metakit] corrupt database From: "Jacob Levy" To: In-Reply-To: References: <3EBA841B.2090808@wi.mit.edu> <3EBA9ADB.4020605@wi.mit.edu> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther, You (and we) need a little more information: * If you can, ask your user what they did, as best they can remember, to produce the database that causes the crashing. Even better, if you can walk them through the steps and reproduce the problem. * You need to ask them what version of Metakit they were using. What version of Tcl were they using? I think you said the DB came from Windows, what version (win2k, win98, winnt, etc.)? * You should also tell us what version of Metakit *you* are using and on what OS. Did you build Metakit yourself? If so, with what compiler and options? What version of Tcl were you using? * If you have a simple way of reproducing the failure, please post a small Tcl script that will demonstrate how you produced the crash. Does this happen only on this one DB or on all DBs? * If you can run the program under a debugger (gdb on Linux, for example) and produce a stack trace, that would help. Once some more information is available, perhaps it'll be possible to help you figure this out. Thanks! --JYL > Hi, > I got a corrupt database from a user. My application (tcl) got > "Speicherzugriffsfehler" - I think its segmentation violation. > mk_manager.kit says the same. With kitviewer, I can load the db, but if > I try some kind of operation the same error. > > Is the a simple programm like mk_cp db1 db2, which tries to copy correct > parts of the corrupt db? > > The corrupt db come from windows - I don't know how this happens, but > because of the positive loading in kitviewer I had some hope to save > some parts of it. > > If I try to read step by step with tcl I can read the first record but > no one after it ... > -- > ~Guenther Fischer > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From guenther.fischer@hrz.tu-chemnitz.de Tue Jun 24 11:06:06 2003 Received: from meg.hrz.tu-chemnitz.de (meg.hrz.tu-chemnitz.de [134.109.132.57]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5OG65N05238 for ; Tue, 24 Jun 2003 11:06:06 -0500 Received: from hedwig.hrz.tu-chemnitz.de ([134.109.132.173] helo=mailbox.hrz.tu-chemnitz.de) by meg.hrz.tu-chemnitz.de with esmtp (Exim 4.12) id 19UqKB-0008N1-00; Tue, 24 Jun 2003 18:07:15 +0200 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46]) by mailbox.hrz.tu-chemnitz.de with esmtp (Exim 4.20) id 19UqKB-0001WO-GQ; Tue, 24 Jun 2003 18:07:15 +0200 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.11.6/8.11.6) with ESMTP id h5OG7FO22714; Tue, 24 Jun 2003 18:07:15 +0200 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Tue, 24 Jun 2003 18:07:15 +0200 (MEST) From: Guenther Fischer To: Jacob Levy cc: metakit@equi4.com Subject: Re: [Metakit] corrupt database In-Reply-To: <64783.209.233.24.218.1056467154.squirrel@mod3.net> Message-ID: References: <3EBA841B.2090808@wi.mit.edu> <3EBA9ADB.4020605@wi.mit.edu> <64783.209.233.24.218.1056467154.squirrel@mod3.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -2.9 (--) X-Scanner: exiscan for exim4 (http://duncanthrax.net/exiscan/) *19UqKB-0008N1-00*tsj.oR37wrE* Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thank you I wil try: On Tue, 24 Jun 2003, Jacob Levy wrote: > Guenther, > > You (and we) need a little more information: > > * If you can, ask your user what they did, as best they can remember, to > produce the database that causes the crashing. Even better, if you can > walk them through the steps and reproduce the problem. I've written to the user - but I didn't have much hope about such informations - he only a simple Windows user and a newbe too ,,, > * You need to ask them what version of Metakit they were using. What > version of Tcl were they using? I think you said the DB came from Windows, > what version (win2k, win98, winnt, etc.)? It is a starpack with the latest version of tclkit - the windows version is build on linux. tclkit is the build fron equi4.com. > * You should also tell us what version of Metakit *you* are using and on > what OS. Did you build Metakit yourself? If so, with what compiler and > options? What version of Tcl were you using? ... I use the same tclkit on a linux box > * If you have a simple way of reproducing the failure, please post a small > Tcl script that will demonstrate how you produced the crash. Does this > happen only on this one DB or on all DBs? The error comes only with this one DB created with my application (a wine databes programm). There are many other users - I never see it before. I user tclkit/metakit for some years for this project (free software). I think there are some bad data in the database (disk error or what ever) and this data are needed for indexing or so. > * If you can run the program under a debugger (gdb on Linux, for example) > and produce a stack trace, that would help. (gdb) run Starting program: /home/fischer/rolle/Wein/wkv099i (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)... Program received signal SIGSEGV, Segmentation fault. 0x4207c45c in memcpy () from /lib/tls/libc.so.6 (gdb) bt #0 0x4207c45c in memcpy () from /lib/tls/libc.so.6 #1 0x080d61c0 in TclpUnloadFile () #2 0x080e5683 in TclpUnloadFile () #3 0x080e5fc7 in TclpUnloadFile () #4 0x080dcb4c in TclpUnloadFile () #5 0x080d4133 in TclpUnloadFile () #6 0x080d4f06 in TclpUnloadFile () #7 0x080c3310 in TclpUnloadFile () #8 0x080c68b7 in TclpUnloadFile () #9 0x080c6d20 in TclpUnloadFile () #10 0x080ca4ce in TclpUnloadFile () #11 0x08051bcd in TclInvokeObjectCommand () #12 0x08073c3d in TclInvokeObjectCommand () #13 0x080730b9 in TclInvokeObjectCommand () #14 0x0809f6a8 in TclObjInterpProc () #15 0x08051bcd in TclInvokeObjectCommand () #16 0x08073c3d in TclInvokeObjectCommand () #17 0x080730b9 in TclInvokeObjectCommand () #18 0x08052ad3 in TclInvokeObjectCommand () #19 0x08059dc8 in TclInvokeObjectCommand () #20 0x08051bcd in TclInvokeObjectCommand () #21 0x08052562 in TclInvokeObjectCommand () #22 0x0808b412 in TclChannelEventScriptInvoker () #23 0x0805e612 in TclInvokeObjectCommand () #24 0x08051bcd in TclInvokeObjectCommand () #25 0x08052562 in TclInvokeObjectCommand () #26 0x0808b412 in TclChannelEventScriptInvoker () #27 0x0805e612 in TclInvokeObjectCommand () #28 0x08051bcd in TclInvokeObjectCommand () #29 0x08052562 in TclInvokeObjectCommand () #30 0x0808b412 in TclChannelEventScriptInvoker () #31 0x080911a9 in Tcl_FSLstat () #32 0x0804d278 in strcpy () #33 0x420156a4 in __libc_start_main () from /lib/tls/libc.so.6 (gdb) and strace says: (the last lines) read(8, "\0Weingut Selbach-Oster 54492 Zel"..., 2124) = 2124 _llseek(8, 0, [0], SEEK_SET) = 0 _llseek(8, 135168, [135168], SEEK_SET) = 0 read(8, "\0Weingut Selbach-Oster 54492 Zel"..., 2116) = 2116 read(8, "\200\0\2\364\0\0\232i", 4096) = 8 _llseek(8, 135168, [135168], SEEK_SET) = 0 read(8, "\0Weingut Selbach-Oster 54492 Zel"..., 4096) = 2124 _llseek(8, 0, [0], SEEK_SET) = 0 read(8, "JL\32\0\0\2\30LH\0H\0it\0\377?H\0H\0H\00012.30\0\0001\0"..., 4096) = 4096 fstat64(8, {st_mode=S_IFREG|0666, st_size=137292, ...}) = 0 _llseek(8, 135168, [135168], SEEK_SET) = 0 read(8, "\0Weingut Selbach-Oster 54492 Zel"..., 4096) = 2124 _llseek(8, 0, [0], SEEK_SET) = 0 read(8, "JL\32\0\0\2\30LH\0H\0it\0\377?H\0H\0H\00012.30\0\0001\0"..., 4096) = 4096 brk(0) = 0x8345000 brk(0x8346000) = 0x8346000 brk(0) = 0x8346000 brk(0x8347000) = 0x8347000 brk(0) = 0x8347000 brk(0x8348000) = 0x8348000 brk(0) = 0x8348000 brk(0x8349000) = 0x8349000 brk(0) = 0x8349000 brk(0x834a000) = 0x834a000 brk(0) = 0x834a000 brk(0x834b000) = 0x834b000 brk(0) = 0x834b000 brk(0x834c000) = 0x834c000 brk(0) = 0x834c000 brk(0x834d000) = 0x834d000 brk(0) = 0x834d000 brk(0x834e000) = 0x834e000 brk(0) = 0x834e000 brk(0x834f000) = 0x834f000 brk(0) = 0x834f000 brk(0x8350000) = 0x8350000 brk(0) = 0x8350000 brk(0x8351000) = 0x8351000 brk(0) = 0x8351000 brk(0x8352000) = 0x8352000 --- SIGSEGV (Segmentation fault) @ 0 (0) --- +++ killed by SIGSEGV +++ -- ~Guenther Fischer From rcohen@dial.oleane.com Wed Jun 25 08:59:11 2003 Received: from mwinf0402.wanadoo.fr (smtp5.wanadoo.fr [193.252.22.27]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5PDxAN18300 for ; Wed, 25 Jun 2003 08:59:11 -0500 Received: from dial.oleane.com (ATours-101-1-2-57.w217-128.abo.wanadoo.fr [217.128.52.57]) by mwinf0402.wanadoo.fr (SMTP Server) with ESMTP id 8AB88800086 for ; Wed, 25 Jun 2003 16:00:13 +0200 (CEST) Message-ID: <3EF9AAF8.6050604@dial.oleane.com> Date: Wed, 25 Jun 2003 16:00:24 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com References: <3EBA841B.2090808@wi.mit.edu> <3EBA9ADB.4020605@wi.mit.edu> <64783.209.233.24.218.1056467154.squirrel@mod3.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Is it a good idea ? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I had an old library that I need to replace. It is a btree class that allow efficient basic keyed data stores which may reside either in memory or on disk, and that accept arbitrary sized data with hash key of any type (byte array). I wonder if metakit would be a good replacement ?? I know that is is robust and have many interesting features, but I also know that the search is not optimized. (as far as I remember from the discussions last year). In particular, I need sometimes to access data through keys that are not strings (a byte array that must work on any endian system) What I need is simply a sort of hash table storing data on the disk. What is your opinion ? Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From rcohen@dial.oleane.com Wed Jun 25 11:45:49 2003 Received: from mwinf0403.wanadoo.fr (smtp5.wanadoo.fr [193.252.22.27]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5PGjmN31002 for ; Wed, 25 Jun 2003 11:45:49 -0500 Received: from dial.oleane.com (ATours-101-1-2-57.w217-128.abo.wanadoo.fr [217.128.52.57]) by mwinf0403.wanadoo.fr (SMTP Server) with ESMTP id 024EB50003B6 for ; Wed, 25 Jun 2003 18:46:54 +0200 (CEST) Message-ID: <3EF9D208.7020002@dial.oleane.com> Date: Wed, 25 Jun 2003 18:47:04 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] help on "join" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Sorry this is basic, but I cant manage to do it. I have 2 views : c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); c4_View table2=database.GetAs("table2[index:I,val:D]"); I search in table2 : c4_View tableselect=table2.SelectRange(row_start,row_end); and want all records in table1 that have the field index present in "tableselect" : c4_View result=table1.Join(table2,tableselect); I cant understand in which order I must write table1, table2, tableselect in this call to Join. I tried every possibility without success, having sometimes nothing, sometimes rows I dont want etc... Thanks for any help. -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jcw@equi4.com Thu Jun 26 04:24:04 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5Q9O4N02242 for ; Thu, 26 Jun 2003 04:24:04 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 7FBEC83B12 for ; Thu, 26 Jun 2003 05:25:10 -0400 (EDT) Date: Thu, 26 Jun 2003 11:25:10 +0200 Subject: Re: [Metakit] Is it a good idea ? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3EF9AAF8.6050604@dial.oleane.com> Message-Id: <1520DE8A-A7B8-11D7-9EAE-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > I had an old library that I need to replace. It is a btree class that > allow efficient basic keyed data stores which may reside either in > memory or on disk, and that accept arbitrary sized data with hash key > of any type (byte array). > > I wonder if metakit would be a good replacement ?? I know that is is > robust and have many interesting features, but I also know that the > search is not optimized. (as far as I remember from the discussions > last year). In particular, I need sometimes to access data through > keys that are not strings (a byte array that must work on any endian > system) > > What I need is simply a sort of hash table storing data on the disk. > What is your opinion ? Check out the hashed views, see examples/ and tests/ dirs for some code using it. Access is instant and O(1). If you also need frequent updates and serious scalability, look at blocked views (which can be combined with hashes in a few ways). -jcw From jcw@equi4.com Thu Jun 26 04:27:00 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5Q9R0N02440 for ; Thu, 26 Jun 2003 04:27:00 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 974C583B12 for ; Thu, 26 Jun 2003 05:28:06 -0400 (EDT) Date: Thu, 26 Jun 2003 11:28:05 +0200 Subject: Re: [Metakit] help on "join" Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3EF9D208.7020002@dial.oleane.com> Message-Id: <7D9C10C0-A7B8-11D7-9EAE-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > I have 2 views : > c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); > c4_View table2=database.GetAs("table2[index:I,val:D]"); > > I search in table2 : > > c4_View tableselect=table2.SelectRange(row_start,row_end); > > and want all records in table1 that have the field index present in > "tableselect" : You want rows in tableselect, expanded with matching rows in table2, based on the value of the index property: c4_IntProp pIndex ("index"); c4_View result = tableselect.Join(table2, pIndex); -jcw From jcw@equi4.com Thu Jun 26 04:38:32 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5Q9cWN03201 for ; Thu, 26 Jun 2003 04:38:32 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 9276883B12 for ; Thu, 26 Jun 2003 05:39:38 -0400 (EDT) Date: Thu, 26 Jun 2003 11:39:37 +0200 Subject: Re: [Metakit] corrupt database Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <1A6E6A57-A7BA-11D7-9EAE-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: > On Tue, 24 Jun 2003, Jacob Levy wrote: >> You (and we) need a little more information: [...] > It is a starpack with the latest version of tclkit - the windows > version > is build on linux. tclkit is the build fron equi4.com. For issues regarding starpacks and tclkit, it is probably more effective to post to the starkit mailing list, see: http://www.equi4.com/mailman/listinfo/starkit > The error comes only with this one DB created with my application (a > wine > databes programm). There are many other users - I never see it before. > I > user tclkit/metakit for some years for this project (free software). I > think there are some bad data in the database (disk error or what ever) > and this data are needed for indexing or so. The one unexplained problem on Windows, and it might even be a regression from previous releases, is a reported corruption when the datafile is on a file server. So this should definitely be something to find out. Every other case I know of was caused by opening more than once. The bad news: datafile corruption in MK tends to damage real bad. It usually does not damage records but entire *columns*. So when trying to extract data, your best bet is to try to not extract all properties. Quick things to try: - does "sdx mkinfo " give meaningful details? - also see the mk2tcl starkit on http://mini.net/sdarchive/ - sometimes readkit.tcl can read what MK itself cannot -jcw From jcw@equi4.com Thu Jun 26 05:06:08 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5QA67N04877 for ; Thu, 26 Jun 2003 05:06:07 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 22D0283B12 for ; Thu, 26 Jun 2003 06:07:14 -0400 (EDT) Date: Thu, 26 Jun 2003 12:07:13 +0200 Subject: Re: [Metakit] Metakit writer/reader question Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3EF310B8.4070308@wi.mit.edu> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > I have been playing around with metakit using a single writer and > multiple readers. > > The setup is something like this > > multiple > threaded > mysql > oracle ---> single writer ---> storage <--- multiple readers > etc > queries > I realize that if the writer writes to the storage the readers need to > be "reset" otherwise the view will be corrupted. But, on inspection, > this seems to only affect the table(s) that are being written > modified. > > So my question is if you have a reader looking at a view that is not > being modified while another view is being modified do you have to > update the reader on commits to avoid corruption or am I just being > lucky? Hm, good point. I think it should essentially work, but there is probably theoretically still a chance of reading bad info. A number of things have to come together for that to happen: - more than one commit between open and use in the reader - the reader has to not yet have accessed the view - free space is such that the view location info is overwritten - it can probably only happen in sub-views, not at the top level I suspect that you'll have a *very* hard time making it happen. But the reason I think it can still happen is that MK loads view info lazily - so it will only know exactly where the data is located once you start accessing it. So the file header is read on open, but navigation towards the requested data happens later. You're right that the view itself, once located, cannot be affected - only columns and views which are changed ever get touched on the disk. > As you can probably tell from the diagram, I am using metakit as a > local "sandbox" to play with data extracted from various sql queries. > I can't tell you how incredibly powerful this is, especially in > combination with Gordon's MkSQL package. KitViewer 2.0 will include > this external database import feature in the near future. Very cool. > Now if we only had NULL support ;) I suspect that many discussions have gone by on whether NULLs are good or bad, with all sorts of variations such as NULLs only in non-key fields, etc. NULL support can be added to MK, quite efficiently in fact, albeit not very conveniently yet: you can either reserve a special value (and get bitten by the fact that view operators will have no idea), or you can create a second property with 0/1 values for each property that allows NULL. It might be awkward to use, but it could be done. The one missing capability in MK 2.4 which would make either approach a lot more practical, is property expressions. Once those are implemented, complex selections and more sophisticated view operators would allow the use of NULLs. -jcw From jyl@mod3.net Thu Jun 26 11:07:13 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5QG7DN28735 for ; Thu, 26 Jun 2003 11:07:13 -0500 Received: (qmail 19972 invoked by uid 48); 26 Jun 2003 16:08:23 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 26 Jun 2003 09:08:23 -0700 (PDT) Message-ID: <65021.209.233.24.218.1056643703.squirrel@mod3.net> Date: Thu, 26 Jun 2003 09:08:23 -0700 (PDT) Subject: Re: [Metakit] corrupt database From: "Jacob Levy" To: In-Reply-To: <1A6E6A57-A7BA-11D7-9EAE-000A957664F8@equi4.com> References: <1A6E6A57-A7BA-11D7-9EAE-000A957664F8@equi4.com> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: JC Is it a problem when one process opens the same storage several times or is it only a problem when more than one process opens the same storage? I'm under the (mistaken?) impression that there's only a problem if more than one process opens the storage concurrently. Please clarify -- thanks! --JYL > Guenther Fischer wrote: > >> On Tue, 24 Jun 2003, Jacob Levy wrote: >>> You (and we) need a little more information: > [...] >> It is a starpack with the latest version of tclkit - the windows >> version >> is build on linux. tclkit is the build fron equi4.com. > > For issues regarding starpacks and tclkit, it is probably more > effective to post to the starkit mailing list, see: > http://www.equi4.com/mailman/listinfo/starkit > >> The error comes only with this one DB created with my application (a >> wine >> databes programm). There are many other users - I never see it before. >> I >> user tclkit/metakit for some years for this project (free software). I >> think there are some bad data in the database (disk error or what >> ever) and this data are needed for indexing or so. > > The one unexplained problem on Windows, and it might even be a > regression from previous releases, is a reported corruption when the > datafile is on a file server. So this should definitely be something > to find out. > > Every other case I know of was caused by opening more than once. > > The bad news: datafile corruption in MK tends to damage real bad. It > usually does not damage records but entire *columns*. So when trying > to extract data, your best bet is to try to not extract all properties. > > Quick things to try: > - does "sdx mkinfo " give meaningful details? > - also see the mk2tcl starkit on http://mini.net/sdarchive/ > - sometimes readkit.tcl can read what MK itself cannot > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From lista@nakawe.se Thu Jun 26 11:17:35 2003 Received: from smtp.telenordia.se (franklin.telenor.se [213.150.135.136]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5QGHYN29611 for ; Thu, 26 Jun 2003 11:17:34 -0500 Message-Id: <200306261617.h5QGHYN29611@trixie.triqs.com> Received: from adslsmtp.tninet.se (h153n3c1o1018.bredband.skanova.com [217.209.44.153]) by franklin.telenor.se (BMR ErlangTM/OTP 3.1) with ESMTP id 783141.644324.1056.1s121945franklin for ; Thu, 26 Jun 2003 18:18:44 +0200 X-Mailer: Ultrafunk Popcorn release 1.40b3 (25-April-2003) X-URL: http://www.ultrafunk.com/products/popcorn/ X-Priority: 3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 Date: Thu, 26 Jun 2003 18:24:10 +0100 From: Veronica Loell To: metakit@equi4.com Subject: Re: [Metakit] corrupt database Reply-To: lista@nakawe.se Organization: Nakawe data Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Concurrent access is what has been discussed before as causing problems. - Veronica > >Subject: Re: [Metakit] corrupt database > From: "Jacob Levy" > Date: Thu, 26 Jun 2003 09:08:23 -0700 (PDT) > To: > Cc: > >JC > >Is it a problem when one process opens the same storage several times or >is it only a problem when more than one process opens the same storage? >I'm under the (mistaken?) impression that there's only a problem if more >than one process opens the storage concurrently. Please clarify -- thanks! > >--JYL > >> Guenther Fischer wrote: >> >>> On Tue, 24 Jun 2003, Jacob Levy wrote: >>>> You (and we) need a little more information: >> [...] >>> It is a starpack with the latest version of tclkit - the windows >>> version >>> is build on linux. tclkit is the build fron equi4.com. >> >> For issues regarding starpacks and tclkit, it is probably more >> effective to post to the starkit mailing list, see: >> http://www.equi4.com/mailman/listinfo/starkit >> >>> The error comes only with this one DB created with my application (a >>> wine >>> databes programm). There are many other users - I never see it before. >>> I >>> user tclkit/metakit for some years for this project (free software). I >>> think there are some bad data in the database (disk error or what >>> ever) and this data are needed for indexing or so. >> >> The one unexplained problem on Windows, and it might even be a >> regression from previous releases, is a reported corruption when the >> datafile is on a file server. So this should definitely be something >> to find out. >> >> Every other case I know of was caused by opening more than once. >> >> The bad news: datafile corruption in MK tends to damage real bad. It >> usually does not damage records but entire *columns*. So when trying >> to extract data, your best bet is to try to not extract all properties. >> >> Quick things to try: >> - does "sdx mkinfo " give meaningful details? >> - also see the mk2tcl starkit on http://mini.net/sdarchive/ >> - sometimes readkit.tcl can read what MK itself cannot >> >> -jcw >> >> _______________________________________________ >> metakit mailing list - metakit@equi4.com >> http://www.equi4.com/mailman/listinfo/metakit > > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Thu Jun 26 12:17:48 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5QHHlN02232 for ; Thu, 26 Jun 2003 12:17:47 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 4138783B12 for ; Thu, 26 Jun 2003 13:18:54 -0400 (EDT) Date: Thu, 26 Jun 2003 19:18:53 +0200 Subject: Re: [Metakit] corrupt database Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <65021.209.233.24.218.1056643703.squirrel@mod3.net> Message-Id: <42D464BC-A7FA-11D7-9EAE-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > Is it a problem when one process opens the same storage several times > or > is it only a problem when more than one process opens the same storage? Any scenario, whether in-process or multi-process. Each open db has its own cache, which gets out of sync when changed underneath it. Open a storage once, use it within a single thread. In Mk4tcl, there's a global mutex, so there you can open a storage once with the "-shared" flag and use it in multiple interpreters and threads. For every other scenario: use a client/server approach. -jcw From jyl@mod3.net Thu Jun 26 12:24:14 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5QHODN02784 for ; Thu, 26 Jun 2003 12:24:13 -0500 Received: (qmail 3821 invoked by uid 48); 26 Jun 2003 17:25:23 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 26 Jun 2003 10:25:23 -0700 (PDT) Message-ID: <65122.209.233.24.218.1056648323.squirrel@mod3.net> Date: Thu, 26 Jun 2003 10:25:23 -0700 (PDT) Subject: Re: [Metakit] corrupt database From: "Jacob Levy" To: In-Reply-To: <42D464BC-A7FA-11D7-9EAE-000A957664F8@equi4.com> References: <65021.209.233.24.218.1056643703.squirrel@mod3.net> <42D464BC-A7FA-11D7-9EAE-000A957664F8@equi4.com> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks JCW! I'm glad I asked, 'cause..: For those of you on this list that use e4Graph, I actually implemented a similar mechanism to what JCW describes in mk4tcl, so that its safe to reopen the storage. All storages are opened shared by default, so no special flag is needed, in e4Graph. I originally implemented this as an optimization, just wanting to save file descriptors etc. I did not know it was actually *required* :) Of course this only helps for the in-process case when one process opens a storage multiple times. No solution yet for the multi-process case. --JYL > Jacob Levy wrote: > >> Is it a problem when one process opens the same storage several times >> or >> is it only a problem when more than one process opens the same >> storage? > > Any scenario, whether in-process or multi-process. Each open db has > its own cache, which gets out of sync when changed underneath it. Open > a storage once, use it within a single thread. > > In Mk4tcl, there's a global mutex, so there you can open a storage once > with the "-shared" flag and use it in multiple interpreters and threads. > > For every other scenario: use a client/server approach. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From rcohen@dial.oleane.com Fri Jun 27 05:42:38 2003 Received: from mwinf0201.wanadoo.fr (smtp7.wanadoo.fr [193.252.22.29]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5RAgbN13753 for ; Fri, 27 Jun 2003 05:42:37 -0500 Received: from dial.oleane.com (unknown [80.11.89.243]) by mwinf0201.wanadoo.fr (SMTP Server) with ESMTP id 3F1DD30003D8 for ; Fri, 27 Jun 2003 12:43:44 +0200 (CEST) Message-ID: <3EFC1FE5.9050904@dial.oleane.com> Date: Fri, 27 Jun 2003 12:43:49 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] help on "join" References: <7D9C10C0-A7B8-11D7-9EAE-000A957664F8@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Now it works, but I wonder something : This suppose that table1 and table2 have a prop with same name. Is it possible to do it with 2 props with different names, but containing same kind of data ? For instance in my table customers I have a field id, but in my table invoices, the field is called customer_id. Should I first use "Rename()" ? Thanks Jean-Claude Wippler wrote: > Riccardo Cohen wrote: > >> I have 2 views : >> c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); >> c4_View table2=database.GetAs("table2[index:I,val:D]"); >> >> I search in table2 : >> >> c4_View tableselect=table2.SelectRange(row_start,row_end); >> >> and want all records in table1 that have the field index present in >> "tableselect" : > > > You want rows in tableselect, expanded with matching rows in table2, > based on the value of the index property: > c4_IntProp pIndex ("index"); > c4_View result = tableselect.Join(table2, pIndex); > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From bkelley@wi.mit.edu Fri Jun 27 08:37:37 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5RDbbN24564 for ; Fri, 27 Jun 2003 08:37:37 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003062709384607643 for ; Fri, 27 Jun 2003 09:38:46 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HH578M00.Q3E; Fri, 27 Jun 2003 09:38:46 -0400 Message-ID: <3EFC4938.8020300@wi.mit.edu> Date: Fri, 27 Jun 2003 09:40:08 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com CC: Riccardo Cohen Subject: Re: [Metakit] help on "join" References: <7D9C10C0-A7B8-11D7-9EAE-000A957664F8@equi4.com> <3EFC1FE5.9050904@dial.oleane.com> In-Reply-To: <3EFC1FE5.9050904@dial.oleane.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > Now it works, but I wonder something : > This suppose that table1 and table2 have a prop with same name. > > Is it possible to do it with 2 props with different names, but > containing same kind of data ? > For instance in my table customers I have a field id, but in my table > invoices, the field is called customer_id. > Should I first use "Rename()" ? Yes. You have to be a bit careful though consider table1=database.GetAs("test[id:I]") table2=database.GetAs("test2[customer_id:I,id:I]") table3 = table2.Rename("customer_id", "id") The original "id" column from table 2 will not be visible from the derived view table3. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From rcohen@dial.oleane.com Fri Jun 27 09:58:38 2003 Received: from mwinf0201.wanadoo.fr (smtp7.wanadoo.fr [193.252.22.29]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5REwcN30424 for ; Fri, 27 Jun 2003 09:58:38 -0500 Received: from dial.oleane.com (unknown [80.11.89.243]) by mwinf0201.wanadoo.fr (SMTP Server) with ESMTP id 0E80530003C6 for ; Fri, 27 Jun 2003 16:59:45 +0200 (CEST) Message-ID: <3EFC5BE6.10403@dial.oleane.com> Date: Fri, 27 Jun 2003 16:59:50 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Content-Type: multipart/mixed; boundary="------------010806090908070605050009" Subject: [Metakit] macos9 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------010806090908070605050009 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable Hi again, Sorry to ask many things, but it is now ok for windows. I try on my macos= 9 (no carbon and old compiler). I could compile with 2 modifs : - config.h : #define HAVE_LONG_LONG 0 - string.cpp : add the stricmp function that was written for q4_WINCE bef= ore "#define strcasecmp=20 stricmp" since my library is too old and dont have that function. I compiled the demo.cpp, It creates always an 8 bytes files "myfile.dat" = which seems to be empty.=20 (attached in this mail) I changed a bit the demo : c4_String s1 (pName (vAddress[1])); c4_String s2 (pCountry (vAddress[1])); printf("The country of %s is: %s\n", (const char*) s1, (const char*) s= 2); // This saves the data to file. storage.Commit(); // Data file now contains 2 addresses. c4_String st1 (pName (vAddress[1])); c4_String st2 (pCountry (vAddress[1])); printf("The country of %s is: %s\n", (const char*) st1, (const char*) = st2); and the output is : The country of Paco Pena is: Spain The country of ountry:S] is: =FF=7F=FF=7F=FF=7F=FF=7F=FF=FF=FF=FF=F0=01 o= untry:S] the commit() between both printf() made all wrong. Could you please tell me where to search for any mistake ? Could it be an endian problem ? Is there any #define I should add somewhere ? What other test could give me a clue of what happens ? Thanks --=20 Riccardo Cohen Articque Les Roches 37230 Fondettes France email =3D rcohen@dial.oleane.com web =3D http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 --------------010806090908070605050009 Content-Type: application/octet-stream; name="myfile.dat" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="myfile.dat" TEoaAAAAAGU= --------------010806090908070605050009-- From bkelley@wi.mit.edu Fri Jun 27 10:12:15 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5RFCFN31652 for ; Fri, 27 Jun 2003 10:12:15 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003062711132408691 for ; Fri, 27 Jun 2003 11:13:24 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HH5BMC00.920 for ; Fri, 27 Jun 2003 11:13:24 -0400 Message-ID: <3EFC5F66.4010706@wi.mit.edu> Date: Fri, 27 Jun 2003 11:14:46 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Bug appending to ordered tables? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm not sure if this should be classified as a bug but consider the following python code: import metakit db = metakit.storage() table = db.getas("test[foo]").ordered() idx = table.append({"foo":"B"}) assert table[idx].foo == "B" idx = table.append({"foo":"A"}) # this fails... assert table[idx].foo == "A" Basically for ordered tables, the idx returned from an append is not correct. I assume that this is because the table is ordered after the idx is generated for the return value. The question is, should an ordered table return the pre-ordered idx or the ordered idx? -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From gmcm@hypernet.com Fri Jun 27 10:30:51 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5RFUpN00957 for ; Fri, 27 Jun 2003 10:30:51 -0500 Received: from PARANOIA (204.176.40.71) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Fri, 27 Jun 2003 11:33:35 -0400 From: "Gordon McMillan" To: "Brian Kelley" , Metakit mailing list Date: Fri, 27 Jun 2003 11:31:09 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Bug appending to ordered tables? Reply-to: gmcm@hypernet.com Message-ID: <3EFC2AFD.32643.397D27D8@localhost> Priority: normal In-reply-to: <3EFC5F66.4010706@wi.mit.edu> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 27 Jun 2003 at 11:14, Brian Kelley wrote: > I'm not sure if this should be classified as a bug but consider the > following python code: Ugh. It's a bug. Please enter it on equi4's bug tracker. -- Gordon http://www.mcmillan-inc.com/ > import metakit > > db = metakit.storage() > table = db.getas("test[foo]").ordered() > > idx = table.append({"foo":"B"}) > assert table[idx].foo == "B" > idx = table.append({"foo":"A"}) > # this fails... > assert table[idx].foo == "A" > > Basically for ordered tables, the idx returned from an append is not > correct. I assume that this is because the table is ordered after the > idx is generated for the return value. > > The question is, should an ordered table return the pre-ordered idx or > the ordered idx? > > -- > Brian Kelley bkelley@wi.mit.edu > Whitehead Institute for Biomedical Research 617 258-6191 > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From bkelley@wi.mit.edu Fri Jun 27 10:36:36 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h5RFaaN01467 for ; Fri, 27 Jun 2003 10:36:36 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003062711374508952 for ; Fri, 27 Jun 2003 11:37:45 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HH5CQX00.B44; Fri, 27 Jun 2003 11:37:45 -0400 Message-ID: <3EFC651B.10102@wi.mit.edu> Date: Fri, 27 Jun 2003 11:39:07 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gmcm@hypernet.com CC: Metakit mailing list Subject: Re: [Metakit] Bug appending to ordered tables? References: <3EFC2AFD.32643.397D27D8@localhost> In-Reply-To: <3EFC2AFD.32643.397D27D8@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: >On 27 Jun 2003 at 11:14, Brian Kelley wrote: > > > >>I'm not sure if this should be classified as a bug but consider the >>following python code: >> >> > >Ugh. It's a bug. Please enter it on equi4's bug tracker. > > > done. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From Mailer-Daemon@LVMPD.com Fri Jun 27 12:02:53 2003 Received: from clark2-red.co.clark.nv.us (clark2-red.co.clark.nv.us [198.200.132.62]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5RH2qN08976 for ; Fri, 27 Jun 2003 12:02:53 -0500 Received: from auto01.co.clark.nv.us by clark2-red.co.clark.nv.us via smtpd (for equi4.com [216.110.35.178]) with ESMTP; Fri, 27 Jun 2003 10:04:05 -0700 Received: from CONVERSION-DAEMON.mail.Co.Clark.NV.US by mail.Co.Clark.NV.US (PMDF V6.2 #30750) id <01KXL2YCYRXS004BIF@mail.Co.Clark.NV.US> for metakit@equi4.com; Fri, 27 Jun 2003 10:03:50 -0700 (PST) Received: from lvmpd.com ("port 4819"@lvmpd-mail.lvmpd.com [172.19.1.32]) by mail.Co.Clark.NV.US (PMDF V6.2 #30750) with ESMTP id <01KXL2YCJJSG00HVV2@mail.Co.Clark.NV.US> for metakit@equi4.com; Fri, 27 Jun 2003 10:03:49 -0700 (PST) Received: from LVMPDGate1-MTA by lvmpd.com with Novell_GroupWise; Fri, 27 Jun 2003 10:03:40 -0700 Date: Fri, 27 Jun 2003 10:03:32 -0700 From: Charles Parker To: metakit@equi4.com Reply-to: C4791P@LVMPD.com Message-id: MIME-version: 1.0 X-Mailer: Novell GroupWise Internet Agent 6.0.3 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-disposition: inline Subject: [Metakit] Re: metakit digest, Vol 1 #409 - 8 msgs (Vacation) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: My wife and I are celebrating our 35th wedding anniversary in Hawaii, and I won't return to the office until July 15, 2003. Please contact Supply Customer Service @ 229-2637 or 229-2680 for assistance while I am gone. Or please call the Supply Manager, John Krueger, directly @ 229-3164. Thanks. From rcohen@dial.oleane.com Mon Jun 30 10:26:57 2003 Received: from mwinf0302.wanadoo.fr (smtp6.wanadoo.fr [193.252.22.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5UFQvN30389 for ; Mon, 30 Jun 2003 10:26:57 -0500 Received: from dial.oleane.com (ATours-101-1-2-72.w217-128.abo.wanadoo.fr [217.128.52.72]) by mwinf0302.wanadoo.fr (SMTP Server) with ESMTP id 2F634C000204 for ; Mon, 30 Jun 2003 17:28:06 +0200 (CEST) Message-ID: <3F005709.10507@dial.oleane.com> Date: Mon, 30 Jun 2003 17:28:09 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] wild char and seatching Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: 1) I read the tcl tutorial, and found that in TCL you can set -min property value -regexp property value etc... But how can you do that in C++ ? I tried that code : dpath(strrow)="%Be%"; strfound=datas.Select(strrow); to extract all rows with a "Be" inside the string property dpath ? I tried % and * and it does not work. 2) I suppose that Find() would behave the same, and Search also ??? Is there any document that discribe the difference between these 3 functions ? Actually it seems that only the syntax is the same, but the actual work is the same. 3) Hash() function is not documented and the samples are a bit difficult for me. (I dont know tcl). Is there any other doc that discribes how it works, because I dont understand. What I want to do is having a hash table on a view to access keys more quickly than with select. Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From olivetti@itc.it Mon Jun 30 11:30:31 2003 Received: from camelot.itc.it (camelot.itc.it [195.223.171.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5UGUUN02301 for ; Mon, 30 Jun 2003 11:30:30 -0500 Received: from orchestra.itc.it (orchestra [10.0.10.11]) by camelot.itc.it (8.12.8/8.12.8) with ESMTP id h5UGViv2002926 for ; Mon, 30 Jun 2003 18:31:44 +0200 (MET DST) Received: from eloy.itc.it. (eloy [10.30.0.71]) by orchestra.itc.it (8.11.6/8.11.6) with ESMTP id h5UGViB14387; Mon, 30 Jun 2003 18:31:44 +0200 Received: (from olivetti@localhost) by eloy.itc.it. (8.11.6/8.11.2) id h5UGVim23264; Mon, 30 Jun 2003 18:31:44 +0200 Date: Mon, 30 Jun 2003 18:31:44 +0200 From: Emanuele Olivetti To: metakit@equi4.com Message-ID: <20030630183144.N21978@eloy.itc.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i Subject: [Metakit] how to use view.filter result? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm a newbie of Mk4py (wonderful) trying to understand how to use this library in the correct way. I had some problem with 'filter' method of views: how can I use the result of the filtering to access the initial records? Something like this (doesn't work): ... table=db.getas("income[VALUE:I]") ... myfilter=lambda r: r.VALUE>12 result=table.filter(myfilter) for i in result: print i.VALUE #doesn't work # or : print table[i].VALUE , but still doesn't work I saw an old post by McMillan, suggesting "._index": --------------------- view = db.view('customers') myfilter = lambda(row: row.lname == 'Smi') indices = view.filter(myfilter) for indexrec in indices: # do something with view[indexrec._index] --------------------- but doesn't work too (._index no more supported?) After that one, another messages said: --------------------------- print "\n#############################################################" print "view.filter doesn't return the whole row, just the matching" print "indices in a table with an index attribute" print "Not much can be done with these now, hopefully when the" print "Mapped Index view is completed we can use this output" print "to create a subview that has access to the rows of the" print "original data" ---------------------------- Is there something new to use filter results now? What can I do with a PyRowRef ? Thanks in advance, Emanuele From gmcm@hypernet.com Mon Jun 30 12:04:57 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h5UH4vN04573 for ; Mon, 30 Jun 2003 12:04:57 -0500 Received: from PARANOIA (204.176.40.75) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Mon, 30 Jun 2003 13:07:20 -0400 From: "Gordon McMillan" To: Emanuele Olivetti , metakit@equi4.com Date: Mon, 30 Jun 2003 13:04:59 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] how to use view.filter result? Reply-to: gmcm@hypernet.com Message-ID: <3F00357B.19244.9536026@localhost> Priority: normal In-reply-to: <20030630183144.N21978@eloy.itc.it> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 30 Jun 2003 at 18:31, Emanuele Olivetti wrote: > Hi, > I'm a newbie of Mk4py (wonderful) trying to understand how > to use this library in the correct way. > I had some problem with 'filter' method of views: how can > I use the result of the filtering to access the initial > records? > > Something like this (doesn't work): > > ... > table=db.getas("income[VALUE:I]") > ... > myfilter=lambda r: r.VALUE>12 > result=table.filter(myfilter) > > for i in result: > print i.VALUE #doesn't work > # or : print table[i].VALUE , but still doesn't work result.structure() would tell you it has one int property "index". You could then use table[i.index]. I usually use filter with remapwith or remove. In your example: table.remapwith(result) should give exactly the same result as table.select({'VALUE':13},{'VALUE':sys.maxint}) > I saw an old post by McMillan, suggesting "._index": Sorry. It's just "index". [...] > Is there something new to use filter results now? What can I do > with a PyRowRef ? That's what table[n] is: An object whose attr's are the columns (in your case, table[n].VALUE). BTW: Metakit is case insensitive, but Python is case sensitive. There are a couple places where that can cause trouble (at least one of them fixed in the CVS version). You're safe if you always use lowercase for columns (or upper, I suppose). -- Gordon http://www.mcmillan-inc.com/ From olivetti@itc.it Tue Jul 1 03:38:07 2003 Received: from camelot.itc.it (camelot.itc.it [195.223.171.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h618c5N19476 for ; Tue, 1 Jul 2003 03:38:06 -0500 Received: from orchestra.itc.it (orchestra [10.0.10.11]) by camelot.itc.it (8.12.8/8.12.8) with ESMTP id h618dJv2004565; Tue, 1 Jul 2003 10:39:19 +0200 (MET DST) Received: from eloy.itc.it. (eloy [10.30.0.71]) by orchestra.itc.it (8.11.6/8.11.6) with ESMTP id h618dIB04449; Tue, 1 Jul 2003 10:39:18 +0200 Received: (from olivetti@localhost) by eloy.itc.it. (8.11.6/8.11.2) id h618dH524684; Tue, 1 Jul 2003 10:39:17 +0200 Date: Tue, 1 Jul 2003 10:39:17 +0200 From: Emanuele Olivetti To: Gordon McMillan , metakit@equi4.com Subject: Re: [Metakit] how to use view.filter result? Message-ID: <20030701103917.A24340@eloy.itc.it> References: <20030630183144.N21978@eloy.itc.it> <3F00357B.19244.9536026@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <3F00357B.19244.9536026@localhost>; from gmcm@hypernet.com on Mon, Jun 30, 2003 at 01:04:59PM -0400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, Jun 30, 2003 at 01:04:59PM -0400, Gordon McMillan wrote: > result.structure() would tell you it has one int property "index". > You could then use table[i.index]. Perfect. Now my code works well and I've understood new things on metakit (4py ;-) > I usually use filter with > remapwith or remove. In your example: > table.remapwith(result) > should give exactly the same result as > table.select({'VALUE':13},{'VALUE':sys.maxint}) I'll try to mesure performances of the two solutions: I need to manage large tables (~200.000 rows) with few (float) values in the receord and to perform selection based on those kind of constraints (e.g. 100 > Is there something new to use filter results now? What can I do > > with a PyRowRef ? > > That's what table[n] is: An object whose attr's are the columns > (in your case, table[n].VALUE). Now it's all clear. I made mistakes referring to the wrong table. > BTW: Metakit is case insensitive, but Python is case sensitive. ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > There are a couple places where that can cause trouble (at least > one of them fixed in the CVS version). You're safe if you always > use lowercase for columns (or upper, I suppose). Interesting. This information is very important and should be put on the Mk4py page, in my opinion. Thanks a lot for your answers and the beautiful work done with Mk4py and to the Metakit team for their database. Emanuele ================================================================= Emanuele Olivetti ITC-IRST phone: +39 0461 314 321 Via Sommarive 18 fax : +39 0461 302 040 38050 Povo (Trento), ITALY home-page : http://sra.itc.it/people/olivetti/ ================================================================= From j.mwebb@comcast.net Thu Jul 3 15:35:56 2003 Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h63KZuv24424 for ; Thu, 3 Jul 2003 15:35:56 -0500 Message-Id: <200307032035.h63KZuv24424@trixie.triqs.com> Received: from rwcrwbc04 (unknown[204.127.197.114](misconfigured sender)) by attbi.com (rwcrmhc12) with SMTP id <2003070320355001400f6egle>; Thu, 3 Jul 2003 20:35:50 +0000 Received: from [199.5.214.34] by rwcrwbc04; Thu, 03 Jul 2003 20:35:49 +0000 From: j.mwebb@comcast.net To: metakit@equi4.com Date: Thu, 03 Jul 2003 20:35:49 +0000 X-Mailer: AT&T Message Center Version 1 (Jun 26 2003) X-Authenticated-Sender: ai5td2ViYkBjb21jYXN0Lm5ldA== MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="NextPart_Webmail_9m3u9jl4l_24495_1057264549" Subject: [Metakit] patch for ./unix/configure.in for installing under linux Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --NextPart_Webmail_9m3u9jl4l_24495_1057264549 Content-Type: text/plain Content-Transfer-Encoding: 8bit Under a RH 8.0 libtool --mode=install complains about a .lai file not existing. Attached is a patch for the unix/configure.in file to fix this. Jeff Webb --NextPart_Webmail_9m3u9jl4l_24495_1057264549 Content-Type: application/octet-stream; name="unix_configure.patch" Content-Transfer-Encoding: 7bit diff -Naur unix_orig/configure.in unix/configure.in --- unix_orig/configure.in 2003-03-10 17:09:04.000000000 -0700 +++ unix/configure.in 2003-07-03 14:24:17.000000000 -0600 @@ -241,6 +241,10 @@ LDFLAGS="${LDFLAGS} -lstdc++" fi ;; + linux*) + LDFLAGS="${LDFLAGS} -lstdc++" + LINK_SPECIAL_FLAGS="-rpath ${libdir}" + ;; *) LDFLAGS="${LDFLAGS} -lstdc++" ;; --NextPart_Webmail_9m3u9jl4l_24495_1057264549-- From mac@ceti.pl Sat Jul 5 03:22:54 2003 Received: from mx2.ceti.pl (mx2.ceti.pl [62.121.128.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h658MWv25607 for ; Sat, 5 Jul 2003 03:22:53 -0500 Received: from tau.ceti.pl (tau.ceti.pl [62.121.128.11]) by mx2.ceti.pl (Postfix) with ESMTP id 511E611E59F for ; Sat, 5 Jul 2003 10:22:10 +0200 (CEST) Received: from mstarzyk (unknown [195.46.43.178]) by tau.ceti.pl (Postfix) with ESMTP id 1BD40202FEA for ; Sat, 5 Jul 2003 10:22:09 +0200 (CEST) Date: Sat, 5 Jul 2003 10:22:14 +0200 From: Maciej Starzyk X-Mailer: The Bat! (v1.62r) Personal Reply-To: Maciej Starzyk X-Priority: 3 (Normal) Message-ID: <1895147373.20030705102214@ceti.pl> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Mk4py - updating rows in derived views. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hello, I tried to update the rows in views created with sort() and sortrev(). The sortrev view could not be updated at all. For sort() it 'almost' worked, except in one case. After the base view object went out of scope, and some other view object created, the sorted view did not allow updates any more. I checked this with Python 2.2.3 and mk4py 2.4.9.2, and with Python 2.3b2 and the latest mk4py version from cvs and the result was the same. Please see the attached test and its output. Could anyone explain if this is a bug in mk4py ? Thanks, Maciej ############################################## ## Test ############################################## import metakit import unittest class MkTest(unittest.TestCase): def setUp (self): self.db = metakit.storage() self.db.getas("a[x:S]") def sorted (self): v = self._view() print "view in sort :", v return v.sort(v.x), v def _view (self): return self.db.view('a') def _someOtherView (self): return metakit.view() def doTestSort (self, keepBase, makeNewView): v = self._view() v.append(x='aaa') print print "view :", v if keepBase: v_sorted, v_base = self.sorted() else: v_sorted = self.sorted()[0] if makeNewView: print "some other view :", self._someOtherView() v_sorted[0].x = 'bbb' self.assertEquals(v_sorted[0].x, 'bbb') ## ## Tests ## def testSort (self): self.doTestSort(keepBase=False, makeNewView=False) def testSortKeepBase (self): self.doTestSort(keepBase=True, makeNewView=False) def testSortWithNew (self): self.doTestSort(keepBase=False, makeNewView=True) def testSortWithNewKeepBase (self): self.doTestSort(keepBase=True, makeNewView=True) def testSortRev (self): v = self._view() v.append(x='aaa') v_sorted = v.sortrev((v.x,), (v.x,)) v_sorted[0].x = 'bbb' self.assertEquals(v_sorted[0].x, 'bbb') if __name__ == "__main__": import sys sys.argv.append('-v') unittest.main() ############################################## ## Output ############################################## testSort (__main__.MkTest) ... view : view in sort : ok testSortKeepBase (__main__.MkTest) ... view : view in sort : ok testSortRev (__main__.MkTest) ... FAIL testSortWithNew (__main__.MkTest) ... view : view in sort : some other view : FAIL testSortWithNewKeepBase (__main__.MkTest) ... view : view in sort : some other view : ok ====================================================================== FAIL: testSortRev (__main__.MkTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "mt.py", line 57, in testSortRev self.assertEquals(v_sorted[0].x, 'bbb') File "h:\dev\py23\lib\unittest.py", line 292, in failUnlessEqual raise self.failureException, \ AssertionError: 'aaa' != 'bbb' ====================================================================== FAIL: testSortWithNew (__main__.MkTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "mt.py", line 47, in testSortWithNew self.doTestSort(keepBase=False, makeNewView=True) File "mt.py", line 35, in doTestSort self.assertEquals(v_sorted[0].x, 'bbb') File "h:\dev\py23\lib\unittest.py", line 292, in failUnlessEqual raise self.failureException, \ AssertionError: 'aaa' != 'bbb' ---------------------------------------------------------------------- Ran 5 tests in 0.015s FAILED (failures=2) From olivetti@itc.it Sun Jul 6 12:11:15 2003 Received: from camelot.itc.it (camelot.itc.it [195.223.171.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h66HBDv14795 for ; Sun, 6 Jul 2003 12:11:14 -0500 Received: from orchestra.itc.it (orchestra [10.0.10.11]) by camelot.itc.it (8.12.8/8.12.8) with ESMTP id h66HBBdu025677 for ; Sun, 6 Jul 2003 19:11:11 +0200 (MET DST) Received: from wonder.itc.it (wonder [10.0.10.40]) by orchestra.itc.it (8.11.6/8.11.6) with ESMTP id h66HBAB16019; Sun, 6 Jul 2003 19:11:10 +0200 From: Emanuele Olivetti Received: (from apache@localhost) by wonder.itc.it (8.11.6/8.11.2) id h66HBAV25897; Sun, 6 Jul 2003 19:11:10 +0200 X-Authentication-Warning: wonder.itc.it: apache set sender to olivetti@itc.it using -f Received: from 62.123.124.44 (SquirrelMail authenticated user olivetti) by webmail.itc.it with HTTP; Sun, 6 Jul 2003 19:11:10 +0200 (CEST) Message-ID: <39372.62.123.124.44.1057511470.squirrel@webmail.itc.it> Date: Sun, 6 Jul 2003 19:11:10 +0200 (CEST) To: X-Priority: 3 Importance: Normal X-MSMail-Priority: Normal X-Mailer: SquirrelMail (version 1.2.7) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] find,search,locate with LIMIT Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I'm using metakit 4 python and trying to solve this problem: given an ordered view, find the first record with a certain field greater that a given value. It is like a MySQL select with the keyword LIMIT=1 , to stop the process after the first match. I'd like find the most performance-effective solution; I've read about ordered and blocked view but didn't understand how to use in practice. Can I use view.serch/view.locate to solve this problem? My current (trivial) solution: ----- db=metakit.storage("example.db",0) # ~1 million of records table=db.getas("[time:D,amplitude:D]") # records were inserted in order with respect to the first row and # there are no duplicates threshold=1000.0 r=table.filter(lambda r: r.time>threshold)[0] # assumption: filtering goes from first row on, and the first # result of the filter is the first row of the resulting table. # Is this a too strong assumtion? i=r.index print i,table2[i].time, table2[i].amplitude ------ Since I've to perform this task many times (with different thresholds) I'd like to find and efficient solution. Thanks in advance, Emanuele From jcw@equi4.com Tue Jul 8 08:31:50 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h68DVnd16388 for ; Tue, 8 Jul 2003 08:31:49 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 2022683B16 for ; Tue, 8 Jul 2003 09:31:43 -0400 (EDT) Date: Tue, 8 Jul 2003 15:31:38 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: <80C913CB-B148-11D7-ACCC-000A957664F8@equi4.com> X-Mailer: Apple Mail (2.552) Subject: [Metakit] List should be back up Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a test. If you get this message, then it means that the mailing list is back up after a total server rebuild. The archives should be back as well, I don't think any messages or new posts were lost. Please don't reply to this message, I can track it on the server to see whether things are ok. -jcw From mac@ceti.pl Sat Jul 12 04:55:11 2003 Received: from mx2.ceti.pl (mx2.ceti.pl [62.121.128.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6C9tAm27434 for ; Sat, 12 Jul 2003 04:55:10 -0500 Received: from tau.ceti.pl (tau.ceti.pl [62.121.128.11]) by mx2.ceti.pl (Postfix) with ESMTP id CF81611E521 for ; Sat, 12 Jul 2003 11:55:06 +0200 (CEST) Received: from mstarzyk (unknown [195.46.43.178]) by tau.ceti.pl (Postfix) with ESMTP id DD929202FE7 for ; Sat, 12 Jul 2003 11:55:05 +0200 (CEST) Date: Sat, 12 Jul 2003 11:55:06 +0200 From: Maciej Starzyk X-Mailer: The Bat! (v1.62r) Personal Reply-To: Maciej Starzyk X-Priority: 3 (Normal) Message-ID: <1046359663.20030712115506@ceti.pl> To: metakit@equi4.com In-Reply-To: <1895147373.20030705102214@ceti.pl> References: <1895147373.20030705102214@ceti.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Re: Mk4py - updating rows in derived views. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hello, Saturday, July 5, 2003, 10:22:14 AM, I wrote: MS> I tried to update the rows in views created with MS> sort() and sortrev(). The sortrev view could not be MS> updated at all. MS> [...] Some messages in this list's archive suggest it should be possible to modify rows in derived views, and the base view would be updated. Since it doesn't seem to work in this case, the other way would be to modify the rows in the base view directly. In mk4py there is however no equivalent of GetIndexOf. Is there some other method to get to the base view's row given the row in the derived view ? Without introducing some kind of row ids ? Thanks, Maciej From gmcm@hypernet.com Sat Jul 12 08:55:46 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6CDtjm02070 for ; Sat, 12 Jul 2003 08:55:46 -0500 Received: from PARANOIA (204.176.40.67) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Sat, 12 Jul 2003 09:57:16 -0400 From: "Gordon McMillan" To: Maciej Starzyk , metakit@equi4.com Date: Sat, 12 Jul 2003 09:54:26 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Re: Mk4py - updating rows in derived views. Reply-to: gmcm@hypernet.com Message-ID: <3F0FDAD2.1046.3CED6CE@localhost> Priority: normal In-reply-to: <1046359663.20030712115506@ceti.pl> References: <1895147373.20030705102214@ceti.pl> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 12 Jul 2003 at 11:55, Maciej Starzyk wrote: > Saturday, July 5, 2003, 10:22:14 AM, I wrote: > MS> I tried to update the rows in views created with > MS> sort() and sortrev(). The sortrev view could not be > MS> updated at all. > MS> [...] > > Some messages in this list's archive suggest it should be > possible to modify rows in derived views, and the base > view would be updated. > Since it doesn't seem to work in this case, the other way > would be to modify the rows in the base view directly. > In mk4py there is however no equivalent of GetIndexOf. > Is there some other method to get to the base view's row > given the row in the derived view ? Without introducing > some kind of row ids ? First, I consider it a bug that it doesn't work with sortrev(), if you would like to log that on equi4's bug tracker. Second, you can use view.indices(subset): >>> v = db.getas('v[nm:S,x:I]') >>> v.append(nm='Zelda',x=33) 0 >>> v.append(nm='Wilbur',x=35) 1 >>> v.append(nm='Adam',x=37) 2 >>> w = v.sortrev(['x'],['x']) >>> metakit.dump(w) nm x ------ -- Adam 37 Wilbur 35 Zelda 33 ------ -- Total: 3 rows >>> metakit.dump(v.indices(w)) index ----- 2 1 0 ----- Total: 3 rows If possible, I would highly recommend using ordered or hashed instead of sorted views. -- Gordon http://www.mcmillan-inc.com/ From jyl@mod3.net Sat Jul 12 10:38:16 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6CFcGm04953 for ; Sat, 12 Jul 2003 10:38:16 -0500 Received: (qmail 16010 invoked by uid 48); 12 Jul 2003 15:36:18 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Sat, 12 Jul 2003 08:36:18 -0700 (PDT) Message-ID: <65160.209.233.24.218.1058024178.squirrel@mod3.net> Date: Sat, 12 Jul 2003 08:36:18 -0700 (PDT) From: "Jacob Levy" To: X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] Problem building Metakit 2.4.9.2 on Solaris 2.8 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Could someone tell me if I'm doing anything wrong? bash-2.03$ ../unix/configure bash-2.03$ gcc -v Reading specs from /opt/sfw/lib/gcc-lib/sparc-sun-solaris2.8/2.95.3/specs gcc version 2.95.3 20010315 (release) bash-2.03$ find / -name "libstdc*" .... /opt/sfw/lib/gcc-lib/sparc-sun-solaris2.8/2.95.3/libstdc++.a /opt/sfw/lib/gcc-lib/sparc-sun-solaris2.8/2.95.3/libstdc++.so /opt/sfw/lib/libstdc++.a.2.10.0 .... bash-2.03$ make .... /bin/sh ./libtool --mode=link g++ -o libmk4.la -g -O2 -I../unix/../include -I../unix/../src -I. -lstdc++ -avoid-version \ -rpath /usr/local/lib column.lo custom.lo derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo rm -fr .libs/libmk4.la .libs/libmk4.* .libs/libmk4.* /usr/ccs/bin/ld -G -h libmk4.so -o .libs/libmk4.so column.lo custom.lo derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo -lstdc++ -lc ld: fatal: library -lstdc++: not found ld: fatal: File processing errors. No output written to .libs/libmk4.so *** Error code 1 make: Fatal error: Command failed for target `libmk4.la' Any ideas appreciated. --JYL From jyl@mod3.net Sat Jul 12 10:47:22 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6CFlMm05288 for ; Sat, 12 Jul 2003 10:47:22 -0500 Received: (qmail 28557 invoked from network); 12 Jul 2003 15:45:24 -0000 Received: from adsl-209-233-24-218.dsl.snfc21.pacbell.net (HELO mod3.net) (jyl@209.233.24.218) by mod3.net with SMTP; 12 Jul 2003 15:45:24 -0000 Message-ID: <3F102D75.6060705@mod3.net> Date: Sat, 12 Jul 2003 08:47:01 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com CC: jyl@mod3.net Subject: Re: [Metakit] Problem building Metakit 2.4.9.2 on Solaris 2.8 References: <65160.209.233.24.218.1058024178.squirrel@mod3.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: OK, the problem is that libtool uses /usr/ccs/bin/ld which does not have the slightest idea where gcc installed libstdc++.so. Editing libtool by hand to replace /usr/ccs/bin/ld with 'gcc -shared' and the link step works. Yuck! --JYL Jacob Levy wrote: >Could someone tell me if I'm doing anything wrong? > >bash-2.03$ ../unix/configure >bash-2.03$ gcc -v >Reading specs from /opt/sfw/lib/gcc-lib/sparc-sun-solaris2.8/2.95.3/specs >gcc version 2.95.3 20010315 (release) >bash-2.03$ find / -name "libstdc*" >.... >/opt/sfw/lib/gcc-lib/sparc-sun-solaris2.8/2.95.3/libstdc++.a >/opt/sfw/lib/gcc-lib/sparc-sun-solaris2.8/2.95.3/libstdc++.so >/opt/sfw/lib/libstdc++.a.2.10.0 >.... >bash-2.03$ make >.... >/bin/sh ./libtool --mode=link g++ -o libmk4.la -g -O2 >-I../unix/../include -I../unix/../src -I. -lstdc++ -avoid-version \ > -rpath /usr/local/lib column.lo custom.lo derived.lo fileio.lo >field.lo format.lo handler.lo persist.lo remap.lo std.lo store.lo >string.lo table.lo univ.lo view.lo viewx.lo >rm -fr .libs/libmk4.la .libs/libmk4.* .libs/libmk4.* >/usr/ccs/bin/ld -G -h libmk4.so -o .libs/libmk4.so column.lo custom.lo >derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo >std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo -lstdc++ -lc >ld: fatal: library -lstdc++: not found >ld: fatal: File processing errors. No output written to .libs/libmk4.so >*** Error code 1 >make: Fatal error: Command failed for target `libmk4.la' > >Any ideas appreciated. > >--JYL > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > > > From david@rebirthing.co.nz Sat Jul 12 22:05:36 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6D35Ym31176 for ; Sat, 12 Jul 2003 22:05:35 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19bXAz-0007If-00 for ; Sun, 13 Jul 2003 15:05:25 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058065486.22672.30.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 13 Jul 2003 15:04:46 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Gratitude Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi Just a message of thanks to the developers of Metakit, and the python interface. I'm building a DHTML framework in Python, with the requirement that it be able to run on all of those budget web hosts - the ones that alwsys seem to have the latest PHP, but only Python 1.5.2 with no SQL interface wrapper. To my delight, the Metakit python interface builds and runs just fine for Python 1.5.2 (with just a couple of Makefile hacks). I built the binary shared lib on my own box, and uploaded it with metakit.py to one such budget host (running x86, redhat, python 1.5.2), and the damn thing works just fine! :) You folks have certainly filled a gap. While the unexplainable prejudice against Python prevails, and even after, Metakit will fill a desperate gap for years to come. Thanks for your efforts. -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From jyl@mod3.net Sun Jul 13 10:28:04 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6DFS3m28631 for ; Sun, 13 Jul 2003 10:28:04 -0500 Received: (qmail 4918 invoked by uid 48); 13 Jul 2003 15:26:02 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Sun, 13 Jul 2003 08:26:02 -0700 (PDT) Message-ID: <65124.209.233.24.218.1058109962.squirrel@mod3.net> Date: Sun, 13 Jul 2003 08:26:02 -0700 (PDT) From: "Jacob Levy" To: X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] [Fwd: Strange problem, only on FreeBSD] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I sent this to JCW and forgot to also include the mailing list, apologies. In case anyone has any insights, similar experiences+workaround, or any suggestions, please (**PLEASE**) do send. In case you want to try it yourself, you can grab the e4Graph could out of the CVS repository (CVS on http://www.sourceforge.net/projects/e4graph) or email me and I'll send you the latest ZIPped sources. --JYL -------- Original Message -------- Subject: Strange problem, only on FreeBSD From: "Jacob Levy" Date: Sat, July 12, 2003 10:22 pm To: I have the following strange behavior with Metakit 2.4.9.2 and e4Graph 1.0a8 compiled with gcc 2.95.4. This is probably an e4Graph problem, but I need a bit of help understanding what I might be doing wrong. As I said the problem only shows up on FreeBSD, not on Linux, where I use the same gcc. Digging a bit I found that c4_IntRef::operator= calls c4_Sequence::Set with a c4_Property whose Type() is 0x0. I added a printout in f4_CreateFormat (in format.cpp) to print the value returned by c4_Property:: c4_Handler* f4_CreateFormat(const c4_Property& prop_, c4_HandlerSeq& seq_) { char fmt = prop_.Type(); fprintf(stderr, "Format is %c (0x%x)\n", fmt, fmt); switch (prop_.Type()) { case 'I': return d4_new c4_FormatX (prop_, seq_); #if !q4_TINY case 'L': return d4_new c4_FormatL (prop_, seq_); case 'F': return d4_new c4_FormatF (prop_, seq_); case 'D': return d4_new c4_FormatD (prop_, seq_); #endif case 'B': return d4_new c4_FormatB (prop_, seq_); case 'S': return d4_new c4_FormatS (prop_, seq_); case 'V': return d4_new c4_FormatV (prop_, seq_); } d4_assert(0); return 0; } I rebuilt Metakit with -Dq4_CHECK, so that asserts are turned on. I also did 'make test' in Metakit to ensure that everything is clean, and it is. However, when I use Metakit from e4Graph, I get an assert at the end of this routine: [jyl@jupiter all] make test (cd ../main/unix; make all) (cd ../test/unix; make all) (cd ../test/unix; make runtest) /home/jyl/devel/e4graph-1.0a8/all/../builds/i386-unknown-freebsd4.8/bin/testcore: libe4graph.so => /home/jyl/devel/e4graph-1.0a8/all/../builds/i386-unknown-freebsd4.8/libs/libe4graph.so (0x280d8000) libmk4.so => /home/jyl/devel/e4graph-1.0a8/all/../../install/lib/libmk4.so (0x28112000) libstdc++.so.3 => /usr/lib/libstdc++.so.3 (0x28152000) libm.so.2 => /usr/lib/libm.so.2 (0x28199000) libc.so.4 => /usr/lib/libc.so.4 (0x281b5000) Running e4_DString tests: ..... Running e4_Refcount tests: Format is V (0x56) Format is I (0x49) Format is (0x0) assertion "0" failed: file "../unix/../src/format.cpp", line 1302 Abort (core dumped) *** Error code 134 Stop in /home/jyl/devel/e4graph-1.0a8/test/unix. *** Error code 1 Stop in /home/jyl/devel/e4graph-1.0a8/all. Running under GDB I get the following stack trace: [jyl@jupiter all] gdb `which testcore` GNU gdb 4.18 (FreeBSD) Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-freebsd"... (no debugging symbols found)... (gdb) set args a (gdb) run Starting program: /home/jyl/devel/install/bin/testcore a Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c line 2627 in elfstab_build_psymtabs Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c line 933 in fill_symbuf Running e4_DString tests: ..... Running e4_Refcount tests: Format is V (0x56) Format is I (0x49) Format is (0x0) assertion "0" failed: file "../unix/../src/format.cpp", line 1302 (no debugging symbols found)... Program received signal SIGABRT, Aborted. 0x281ed9c8 in kill () from /usr/lib/libc.so.4 (gdb) where #0 0x281ed9c8 in kill () from /usr/lib/libc.so.4 #1 0x28230651 in abort () from /usr/lib/libc.so.4 #2 0x2820ba5a in __assert () from /usr/lib/libc.so.4 #3 0x28133867 in f4_CreateFormat (prop_=@0x28110990, seq_=@0x80bd5c0) at ../unix/../src/format.cpp:1302 #4 0x28135095 in c4_HandlerSeq::CreateHandler (this=0x80bd5c0, prop_=@0x28110990) at ../unix/../src/handler.cpp:145 #5 0x2814a223 in c4_Sequence::PropIndex (this=0x80bd5c0, prop_=@0x28110990) at ../unix/../src/viewx.cpp:325 #6 0x2814a43d in c4_Sequence::Set (this=0x80bd5c0, index_=0, prop_=@0x28110990, buf_=@0xbfbff3e4) at ../unix/../src/viewx.cpp:362 #7 0x2814ab34 in c4_IntRef::operator= (this=0xbfbff634, value_=1) at ../unix/../include/mk4.inl:426 #8 0x280e7697 in e4_MetakitStorageImpl::Initialize () from /home/jyl/devel/install/lib/libe4graph.so #9 0x280e6f33 in e4_MetakitStorageImpl::e4_MetakitStorageImpl () from /home/jyl/devel/install/lib/libe4graph.so #10 0x280e60ee in e4_MetakitStorageImpl::GetStorage () from /home/jyl/devel/install/lib/libe4graph.so #11 0x2810b008 in e4_StorageImpl::GetStorage () from /home/jyl/devel/install/lib/libe4graph.so #12 0x28100af8 in e4_Storage::e4_Storage () from /home/jyl/devel/install/lib/libe4graph.so #13 0x804bfce in clean_storage () #14 0x807a1c7 in test_refcount1 () #15 0x807b484 in test_refcount () #16 0x804c241 in main () #17 0x804beee in _start () (gdb) The relevant code in e4Graph is: e4_MetakitStorageImpl::e4_MetakitStorageImpl(const char *fname, int state, int pps) : e4_StorageImpl(fname, E4_METAKIT, pps), needsGC(false), idStack1(NULL), idStack2(NULL) { bool openmode = ((pps & E4_SPMODIFY) == 0) ? false : true; c4_Storage ls(fname, openmode); /* * Open the storage in exclusive read-write mode or in read-only mode * depending on the passed permissions. */ storage = ls; /* * Check that the open succeeded. */ if (!storage.Strategy().IsValid()) { return; } /* * Check that initialization succeeded. */ if (!Initialize(state, openmode)) { return; } } /* * Helper function for initializing a storage. */ bool e4_MetakitStorageImpl::Initialize(int state, int canmodify) { int i, j, cnt, flags; unused = storage.GetAs(MK4_GRAPHUNUSED1_4); if (unused.GetSize() == 0) { /* * If this is a totally new storage and we cannot set up * our data structures, return false. */ if (!canmodify) { return false; } unused.SetSize(MK4_GRAPHUNUSEDSIZE); (****) pFirst(unused[MK4_GRAPHSTORAGEMAJORVER]) = (int) E4_MKSTORAGE_MAJORVER; pFirst(unused[MK4_GRAPHSTORAGEMINORVER]) = (int) E4_MKSTORAGE_MINORVER; The assert occurs on the line marked (****). That line is the first time I access any value in a view in the newly opened storage. The property itself is declared in the beginning of the same file, as follows: c4_IntProp pFirst("first"); This is the format for the view being accessed: #define MK4_GRAPHUNUSED1_4 "e4GraphUnused[first:I]" I'm not sure what to think. Is this a compiler bug, or is this my bug, or maybe I'm touching an esoteric bug in MK? Of course I suspect my own code first, but I'm not sure how to proceed. I did not attach the core file, since I didn't want this message to grow too big. If you need that, or the e4Graph sources, or anything else, just ask, I'll be happy to supply. Any insights appreciated! --JYL From jyl@mod3.net Sun Jul 13 13:44:43 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6DIigX05671 for ; Sun, 13 Jul 2003 13:44:43 -0500 Received: (qmail 23983 invoked from network); 13 Jul 2003 18:42:39 -0000 Received: from adsl-209-233-24-218.dsl.snfc21.pacbell.net (HELO mod3.net) (jyl@209.233.24.218) by mod3.net with SMTP; 13 Jul 2003 18:42:39 -0000 Message-ID: <3F11A86D.4000903@mod3.net> Date: Sun, 13 Jul 2003 11:43:57 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com CC: jcw@equi4.com Content-Type: multipart/alternative; boundary="------------040807010007080201000807" Subject: [Metakit] [Fwd: Strange problem, only on FreeBSD] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --------------040807010007080201000807 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit I solved this problem, The good news is that this problem has nothing to do with Metakit, which, as always, performs admirably. The bad news: if you use tclConfig.sh in your configure scripts, and specifically rely on the value of SHLIB_LD, you might get a bad value on some platforms (at least Solaris 8, OpenBSD, and FreeBSD confirmed so far). The suggested value was 'ld -shareable -x' on FreeBSD and OpenBSD; when I changed it to 'gcc -shared' and rebuilt, all my problems were solved and the e4Graph package built and passed all the tests. OT but relevant question: is there a portable way to figure out what to use to link shared libraries? --JYL -------- Original Message -------- Subject: Strange problem, only on FreeBSD Date: Sat, 12 Jul 2003 22:22:09 -0700 (PDT) From: "Jacob Levy" Reply-To: jyl@mod3.net To: CC: , I have the following strange behavior with Metakit 2.4.9.2 and e4Graph 1.0a8 compiled with gcc 2.95.4. This is probably an e4Graph problem, but I need a bit of help understanding what I might be doing wrong. As I said the problem only shows up on FreeBSD, not on Linux, where I use the same gcc. Digging a bit I found that c4_IntRef::operator= calls c4_Sequence::Set with a c4_Property whose Type() is 0x0. I added a printout in f4_CreateFormat (in format.cpp) to print the value returned by c4_Property:: c4_Handler* f4_CreateFormat(const c4_Property& prop_, c4_HandlerSeq& seq_) { char fmt = prop_.Type(); fprintf(stderr, "Format is %c (0x%x)\n", fmt, fmt); switch (prop_.Type()) { case 'I': return d4_new c4_FormatX (prop_, seq_); #if !q4_TINY case 'L': return d4_new c4_FormatL (prop_, seq_); case 'F': return d4_new c4_FormatF (prop_, seq_); case 'D': return d4_new c4_FormatD (prop_, seq_); #endif case 'B': return d4_new c4_FormatB (prop_, seq_); case 'S': return d4_new c4_FormatS (prop_, seq_); case 'V': return d4_new c4_FormatV (prop_, seq_); } d4_assert(0); return 0; } I rebuilt Metakit with -Dq4_CHECK, so that asserts are turned on. I also did 'make test' in Metakit to ensure that everything is clean, and it is. However, when I use Metakit from e4Graph, I get an assert at the end of this routine: [jyl@jupiter all] make test (cd ../main/unix; make all) (cd ../test/unix; make all) (cd ../test/unix; make runtest) /home/jyl/devel/e4graph-1.0a8/all/../builds/i386-unknown-freebsd4.8/bin/testcore: libe4graph.so => /home/jyl/devel/e4graph-1.0a8/all/../builds/i386-unknown-freebsd4.8/libs/libe4graph.so (0x280d8000) libmk4.so => /home/jyl/devel/e4graph-1.0a8/all/../../install/lib/libmk4.so (0x28112000) libstdc++.so.3 => /usr/lib/libstdc++.so.3 (0x28152000) libm.so.2 => /usr/lib/libm.so.2 (0x28199000) libc.so.4 => /usr/lib/libc.so.4 (0x281b5000) Running e4_DString tests: ..... Running e4_Refcount tests: Format is V (0x56) Format is I (0x49) Format is (0x0) assertion "0" failed: file "../unix/../src/format.cpp", line 1302 Abort (core dumped) *** Error code 134 Stop in /home/jyl/devel/e4graph-1.0a8/test/unix. *** Error code 1 Stop in /home/jyl/devel/e4graph-1.0a8/all. Running under GDB I get the following stack trace: [jyl@jupiter all] gdb `which testcore` GNU gdb 4.18 (FreeBSD) Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-freebsd"... (no debugging symbols found)... (gdb) set args a (gdb) run Starting program: /home/jyl/devel/install/bin/testcore a Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c line 2627 in elfstab_build_psymtabs Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c line 933 in fill_symbuf Running e4_DString tests: ..... Running e4_Refcount tests: Format is V (0x56) Format is I (0x49) Format is (0x0) assertion "0" failed: file "../unix/../src/format.cpp", line 1302 (no debugging symbols found)... Program received signal SIGABRT, Aborted. 0x281ed9c8 in kill () from /usr/lib/libc.so.4 (gdb) where #0 0x281ed9c8 in kill () from /usr/lib/libc.so.4 #1 0x28230651 in abort () from /usr/lib/libc.so.4 #2 0x2820ba5a in __assert () from /usr/lib/libc.so.4 #3 0x28133867 in f4_CreateFormat (prop_=@0x28110990, seq_=@0x80bd5c0) at ../unix/../src/format.cpp:1302 #4 0x28135095 in c4_HandlerSeq::CreateHandler (this=0x80bd5c0, prop_=@0x28110990) at ../unix/../src/handler.cpp:145 #5 0x2814a223 in c4_Sequence::PropIndex (this=0x80bd5c0, prop_=@0x28110990) at ../unix/../src/viewx.cpp:325 #6 0x2814a43d in c4_Sequence::Set (this=0x80bd5c0, index_=0, prop_=@0x28110990, buf_=@0xbfbff3e4) at ../unix/../src/viewx.cpp:362 #7 0x2814ab34 in c4_IntRef::operator= (this=0xbfbff634, value_=1) at ../unix/../include/mk4.inl:426 #8 0x280e7697 in e4_MetakitStorageImpl::Initialize () from /home/jyl/devel/install/lib/libe4graph.so #9 0x280e6f33 in e4_MetakitStorageImpl::e4_MetakitStorageImpl () from /home/jyl/devel/install/lib/libe4graph.so #10 0x280e60ee in e4_MetakitStorageImpl::GetStorage () from /home/jyl/devel/install/lib/libe4graph.so #11 0x2810b008 in e4_StorageImpl::GetStorage () from /home/jyl/devel/install/lib/libe4graph.so #12 0x28100af8 in e4_Storage::e4_Storage () from /home/jyl/devel/install/lib/libe4graph.so #13 0x804bfce in clean_storage () #14 0x807a1c7 in test_refcount1 () #15 0x807b484 in test_refcount () #16 0x804c241 in main () #17 0x804beee in _start () (gdb) The relevant code in e4Graph is: e4_MetakitStorageImpl::e4_MetakitStorageImpl(const char *fname, int state, int pps) : e4_StorageImpl(fname, E4_METAKIT, pps), needsGC(false), idStack1(NULL), idStack2(NULL) { bool openmode = ((pps & E4_SPMODIFY) == 0) ? false : true; c4_Storage ls(fname, openmode); /* * Open the storage in exclusive read-write mode or in read-only mode * depending on the passed permissions. */ storage = ls; /* * Check that the open succeeded. */ if (!storage.Strategy().IsValid()) { return; } /* * Check that initialization succeeded. */ if (!Initialize(state, openmode)) { return; } } /* * Helper function for initializing a storage. */ bool e4_MetakitStorageImpl::Initialize(int state, int canmodify) { int i, j, cnt, flags; unused = storage.GetAs(MK4_GRAPHUNUSED1_4); if (unused.GetSize() == 0) { /* * If this is a totally new storage and we cannot set up * our data structures, return false. */ if (!canmodify) { return false; } unused.SetSize(MK4_GRAPHUNUSEDSIZE); (****) pFirst(unused[MK4_GRAPHSTORAGEMAJORVER]) = (int) E4_MKSTORAGE_MAJORVER; pFirst(unused[MK4_GRAPHSTORAGEMINORVER]) = (int) E4_MKSTORAGE_MINORVER; The assert occurs on the line marked (****). That line is the first time I access any value in a view in the newly opened storage. The property itself is declared in the beginning of the same file, as follows: c4_IntProp pFirst("first"); This is the format for the view being accessed: #define MK4_GRAPHUNUSED1_4 "e4GraphUnused[first:I]" I'm not sure what to think. Is this a compiler bug, or is this my bug, or maybe I'm touching an esoteric bug in MK? Of course I suspect my own code first, but I'm not sure how to proceed. I did not attach the core file, since I didn't want this message to grow too big. If you need that, or the e4Graph sources, or anything else, just ask, I'll be happy to supply. Any insights appreciated! --JYL --------------040807010007080201000807 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit I solved this problem,

The good news is that this problem has nothing to do with Metakit, which, as always, performs admirably.

The bad news: if you use tclConfig.sh in your configure scripts, and specifically rely on the value of SHLIB_LD, you might get a bad value on some platforms (at least Solaris 8, OpenBSD, and FreeBSD confirmed so far). The suggested value was 'ld -shareable -x' on FreeBSD and OpenBSD; when I changed it to 'gcc -shared' and rebuilt, all my problems were solved and the e4Graph package built and passed all the tests.

OT but relevant question: is there a portable way to figure out what to use to link shared libraries?

--JYL

-------- Original Message --------
Subject: Strange problem, only on FreeBSD
Date: Sat, 12 Jul 2003 22:22:09 -0700 (PDT)
From: "Jacob Levy" <jyl@mod3.net>
Reply-To: jyl@mod3.net
To: <jcw@equi4.com>
CC: <jyl@mod3.net>, <jyl@eng.sun.com> 


I have the following strange behavior with Metakit 2.4.9.2 and e4Graph
1.0a8 compiled with gcc 2.95.4. This is probably an e4Graph problem, but I
need a bit of help understanding what I might be doing wrong. As I said
the problem only shows up on FreeBSD, not on Linux, where I use the same
gcc.

Digging a bit I found that c4_IntRef::operator= calls c4_Sequence::Set
with a c4_Property whose Type() is 0x0. I added a printout in
f4_CreateFormat (in format.cpp) to print the value returned by
c4_Property::

c4_Handler* f4_CreateFormat(const c4_Property& prop_, c4_HandlerSeq& seq_)
{
  char fmt = prop_.Type();

  fprintf(stderr, "Format is %c (0x%x)\n", fmt, fmt);

  switch (prop_.Type()) {
    case 'I': return d4_new c4_FormatX (prop_, seq_);
#if !q4_TINY
    case 'L': return d4_new c4_FormatL (prop_, seq_);
    case 'F': return d4_new c4_FormatF (prop_, seq_);
    case 'D': return d4_new c4_FormatD (prop_, seq_);
#endif
    case 'B': return d4_new c4_FormatB (prop_, seq_);
    case 'S': return d4_new c4_FormatS (prop_, seq_);
    case 'V': return d4_new c4_FormatV (prop_, seq_);
  }

  d4_assert(0);
  return 0;
}

I rebuilt Metakit with -Dq4_CHECK, so that asserts are turned on. I also
did 'make test' in Metakit to ensure that everything is clean, and it is.
However, when I use Metakit from e4Graph, I get an assert at the end of
this routine:

[jyl@jupiter all] make test
(cd ../main/unix; make  all)
(cd ../test/unix; make  all)
(cd ../test/unix; make runtest)
/home/jyl/devel/e4graph-1.0a8/all/../builds/i386-unknown-freebsd4.8/bin/testcore:
        libe4graph.so =>
/home/jyl/devel/e4graph-1.0a8/all/../builds/i386-unknown-freebsd4.8/libs/libe4graph.so
(0x280d8000)
        libmk4.so =>
/home/jyl/devel/e4graph-1.0a8/all/../../install/lib/libmk4.so
(0x28112000)
        libstdc++.so.3 => /usr/lib/libstdc++.so.3 (0x28152000)
        libm.so.2 => /usr/lib/libm.so.2 (0x28199000)
        libc.so.4 => /usr/lib/libc.so.4 (0x281b5000)
Running e4_DString tests: .....
Running e4_Refcount tests: Format is V (0x56)
Format is I (0x49)
Format is  (0x0)
assertion "0" failed: file "../unix/../src/format.cpp", line 1302
Abort (core dumped)
*** Error code 134

Stop in /home/jyl/devel/e4graph-1.0a8/test/unix.
*** Error code 1

Stop in /home/jyl/devel/e4graph-1.0a8/all.

Running under GDB I get the following stack trace:

[jyl@jupiter all] gdb `which testcore`
GNU gdb 4.18 (FreeBSD)
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-unknown-freebsd"...
(no debugging symbols found)...
(gdb) set args a
(gdb) run
Starting program: /home/jyl/devel/install/bin/testcore a
Deprecated bfd_read called at
/usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c
line 2627 in elfstab_build_psymtabs
Deprecated bfd_read called at
/usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c
line 933 in fill_symbuf
Running e4_DString tests: .....
Running e4_Refcount tests: Format is V (0x56)
Format is I (0x49)
Format is  (0x0)
assertion "0" failed: file "../unix/../src/format.cpp", line 1302
(no debugging symbols found)...
Program received signal SIGABRT, Aborted.
0x281ed9c8 in kill () from /usr/lib/libc.so.4
(gdb) where
#0  0x281ed9c8 in kill () from /usr/lib/libc.so.4
#1  0x28230651 in abort () from /usr/lib/libc.so.4
#2  0x2820ba5a in __assert () from /usr/lib/libc.so.4
#3  0x28133867 in f4_CreateFormat (prop_=@0x28110990, seq_=@0x80bd5c0)
    at ../unix/../src/format.cpp:1302
#4  0x28135095 in c4_HandlerSeq::CreateHandler (this=0x80bd5c0,
    prop_=@0x28110990) at ../unix/../src/handler.cpp:145
#5  0x2814a223 in c4_Sequence::PropIndex (this=0x80bd5c0, prop_=@0x28110990)
    at ../unix/../src/viewx.cpp:325
#6  0x2814a43d in c4_Sequence::Set (this=0x80bd5c0, index_=0,
    prop_=@0x28110990, buf_=@0xbfbff3e4) at ../unix/../src/viewx.cpp:362
#7  0x2814ab34 in c4_IntRef::operator= (this=0xbfbff634, value_=1)
    at ../unix/../include/mk4.inl:426
#8  0x280e7697 in e4_MetakitStorageImpl::Initialize ()
   from /home/jyl/devel/install/lib/libe4graph.so
#9  0x280e6f33 in e4_MetakitStorageImpl::e4_MetakitStorageImpl ()
   from /home/jyl/devel/install/lib/libe4graph.so
#10 0x280e60ee in e4_MetakitStorageImpl::GetStorage ()
   from /home/jyl/devel/install/lib/libe4graph.so
#11 0x2810b008 in e4_StorageImpl::GetStorage ()
   from /home/jyl/devel/install/lib/libe4graph.so
#12 0x28100af8 in e4_Storage::e4_Storage ()
   from /home/jyl/devel/install/lib/libe4graph.so
#13 0x804bfce in clean_storage ()
#14 0x807a1c7 in test_refcount1 ()
#15 0x807b484 in test_refcount ()
#16 0x804c241 in main ()
#17 0x804beee in _start ()
(gdb)

The relevant code in e4Graph is:

e4_MetakitStorageImpl::e4_MetakitStorageImpl(const char *fname,
                                             int state,
                                             int pps)
    : e4_StorageImpl(fname, E4_METAKIT, pps),
      needsGC(false),
      idStack1(NULL),
      idStack2(NULL)
{
    bool openmode = ((pps & E4_SPMODIFY) == 0) ? false : true;
    c4_Storage ls(fname, openmode);

    /*
     * Open the storage in exclusive read-write mode or in read-only mode
     * depending on the passed permissions.
     */

    storage = ls;

    /*
     * Check that the open succeeded.
     */

    if (!storage.Strategy().IsValid()) {
        return;
    }

    /*
     * Check that initialization succeeded.
     */

    if (!Initialize(state, openmode)) {
        return;
    }
}

/*
 * Helper function for initializing a storage.
 */

bool
e4_MetakitStorageImpl::Initialize(int state, int canmodify)
{
    int i, j, cnt, flags;

    unused = storage.GetAs(MK4_GRAPHUNUSED1_4);
    if (unused.GetSize() == 0) {

        /*
         * If this is a totally new storage and we cannot set up
         * our data structures, return false.
         */

        if (!canmodify) {
            return false;
        }

        unused.SetSize(MK4_GRAPHUNUSEDSIZE);
(****)  pFirst(unused[MK4_GRAPHSTORAGEMAJORVER]) = (int)
E4_MKSTORAGE_MAJORVER;
        pFirst(unused[MK4_GRAPHSTORAGEMINORVER]) = (int)
E4_MKSTORAGE_MINORVER;

The assert occurs on the line marked (****). That line is the first time I
access any value in a view in the newly opened storage.

The property itself is declared in the beginning of the same file, as
follows:

c4_IntProp pFirst("first");

This is the format for the view being accessed:

#define MK4_GRAPHUNUSED1_4      "e4GraphUnused[first:I]"

I'm not sure what to think. Is this a compiler bug, or is this my bug, or
maybe I'm touching an esoteric bug in MK? Of course I suspect my own code
first, but I'm not sure how to proceed.

I did not attach the core file, since I didn't want this message to grow
too big. If you need that, or the e4Graph sources, or anything else, just
ask, I'll be happy to supply.

Any insights appreciated!

--JYL
--------------040807010007080201000807-- From david@rebirthing.co.nz Sun Jul 13 21:47:48 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6E2lkX24549 for ; Sun, 13 Jul 2003 21:47:47 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19btNJ-0004RS-00 for ; Mon, 14 Jul 2003 14:47:37 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058150816.1189.14.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 14 Jul 2003 14:46:56 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Problem with join Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I'm learning Metakit, and trying to do a join between 2 different views. Basically, one view has the fields 'first', 'last', 'shoesize' and the other view has fields 'first', 'last', 'city'. I'm trying to end up with a view with 'first', 'last', 'shoesize' and 'city'. But the derived view has erroneous rows. Can someone please help? (Python code and output appears below) ----------------------------------- import metakit # create one database db = metakit.storage("datafile1.mk",1) # add a view vw1 = db.getas("people[first:S,last:S,shoesize:I]") # Add two rows vw1.append(first='John',last='Lennon',shoesize=44) vw1.append(first='Flash',last='Gordon',shoesize=42) vw1.append(first='John',last='Doe', shoesize=40) # create another view vw2 = db.getas("persons[first:S,last:S,city:S]") # Add three rows vw2.append(first='John',last='Lennon',city="San Diego") vw2.append(first='Flash',last='Gordon', city="Houston") vw2.append(first='Andy',last='Young',city="Sydney") # Commit the changes to file: db.commit() # Try to get a view which accurately merges both views vwn = vw1.join(vw2, 'first', 'last') for r in vwn: print "%s %s %s %d" % (r.first, r.last, r.city, r.shoesize) ------------------ The above code produces: Flash Gordon Sydney 42 Flash Gordon Houston 42 Flash Gordon San Diego 42 John Doe Sydney 40 John Doe Houston 40 John Doe San Diego 40 John Lennon Sydney 44 John Lennon Houston 44 John Lennon San Diego 44 It should (or, I want it to) produce: Flash Gordon Houston 42 John Lennon San Diego 44 -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From bkelley@wi.mit.edu Mon Jul 14 09:36:53 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6EEarX23743 for ; Mon, 14 Jul 2003 09:36:53 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071410364921980 for ; Mon, 14 Jul 2003 10:36:49 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI0R9D00.OBJ; Mon, 14 Jul 2003 10:36:49 -0400 Message-ID: <3F12C05A.6040701@wi.mit.edu> Date: Mon, 14 Jul 2003 10:38:18 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: David McNab , metakit@equi4.com Subject: Re: [Metakit] Problem with join References: <1058150816.1189.14.camel@rebirth> In-Reply-To: <1058150816.1189.14.camel@rebirth> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: David McNab wrote: In your join statement you have to use metakit properties, not string values. I had this problem for a while as well which is why I'm rewriting (portions) of the metakit documentation. > ># Try to get a view which accurately merges both views >vwn = vw1.join(vw2, 'first', 'last') > > vwn = vw1.join(vw2, vw1.first, vw1.last) -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From david@rebirthing.co.nz Mon Jul 14 09:48:09 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6EEm7X24563 for ; Mon, 14 Jul 2003 09:48:08 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19c4cN-0007Af-00 for ; Tue, 15 Jul 2003 02:47:55 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058194034.1211.31.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 15 Jul 2003 02:47:14 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] view.select criteria = '!=', '<', '>=', etc Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I see in the doco for view.select() that criteria can only be specified for equality. Is there any way to sort on a field *not* equal to a value, or less than a value, greater or equal to a value etc? -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From jeff@k2.com Mon Jul 14 10:16:08 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc1.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6EFG8X27178 for ; Mon, 14 Jul 2003 10:16:08 -0500 Received: from Klee (dsl093-106-226.wdc1.dsl.speakeasy.net [66.93.106.226]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h6EFIe215320; Mon, 14 Jul 2003 11:18:41 -0400 From: "Jeffrey Kay" To: "'David McNab'" , Subject: RE: [Metakit] view.select criteria = '!=', '<', '>=', etc Date: Mon, 14 Jul 2003 11:12:38 -0400 Message-ID: <001c01c34a1a$5d6c7560$017410ac@k2.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4024 In-Reply-To: <1058194034.1211.31.camel@rebirth> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: AFAIK, the only way to select based on inequality is by building two views and subtracting one view from the other, using c4_View::Minus. There are several functions in the c4_View class that allow you combine, merge, and compute the differences between two views. jeffrey kay weblog pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein > -----Original Message----- > From: metakit-admin@equi4.com > [mailto:metakit-admin@equi4.com] On Behalf Of David McNab > Sent: Monday, July 14, 2003 10:47 AM > To: metakit@equi4.com > Subject: [Metakit] view.select criteria = '!=', '<', '>=', etc > > > Hi, > > I see in the doco for view.select() that criteria can only be > specified > for equality. > > Is there any way to sort on a field *not* equal to a value, > or less than > a value, greater or equal to a value etc? > > > -- > Kind regards > David > > -- > > leave this line intact so your email gets through my junk mail filter > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From gmcm@hypernet.com Mon Jul 14 10:25:11 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6EFPAX28133 for ; Mon, 14 Jul 2003 10:25:11 -0500 Received: from PARANOIA (204.176.40.66) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3) for ; Mon, 14 Jul 2003 11:26:47 -0400 From: "Gordon McMillan" To: metakit@equi4.com Date: Mon, 14 Jul 2003 11:23:56 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] view.select criteria = '!=', '<', '>=', etc Reply-to: gmcm@hypernet.com Message-ID: <3F1292CC.7504.E6D7F77@localhost> Priority: normal In-reply-to: <1058194034.1211.31.camel@rebirth> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 15 Jul 2003 at 2:47, David McNab wrote: > I see in the doco for view.select() that criteria can only be > specified for equality. > > Is there any way to sort on a field *not* equal to a value, > or less than a value, greater or equal to a value etc? There's a way of doing ranges: v.select({'shoesize':9},{'shoesize':13}) For complex criteria, use v.filter and then v.remapwith w = v.filter(lambda row: row.shoesize % 2) w = v.remapwith(w) gives me all odd shoesize rows. -- Gordon http://www.mcmillan-inc.com/ From bkelley@wi.mit.edu Mon Jul 14 11:11:45 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6EGBjX00703 for ; Mon, 14 Jul 2003 11:11:45 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071412114123154 for ; Mon, 14 Jul 2003 12:11:41 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI0VNH00.ADT for ; Mon, 14 Jul 2003 12:11:41 -0400 Message-ID: <3F12D696.2080203@wi.mit.edu> Date: Mon, 14 Jul 2003 12:13:10 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com References: <39372.62.123.124.44.1057511470.squirrel@webmail.itc.it> In-Reply-To: <39372.62.123.124.44.1057511470.squirrel@webmail.itc.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Neat Trick: using python's bisect with ordered columns Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: (This reply never made it to the mailing list, I thought the solution was pretty neat so here it is) Euele Olivetti wrote: > Hi, > I'm using metakit 4 python and trying to solve this problem: > given an ordered view, find the first record with a certain field > greater that a given value. It is like a MySQL select with the > keyword LIMIT=1 , to stop the process after the first match. > I'm not sure that metakit supports this type of search but we would still like to use the fact that the column you are searching is ordered. Having said that, here is my solution. Fact #1 python has a bisect package that supports this type of thing. Fact #2 it only works on lists. Solution -> make a metakit column look like a list. Hope this works out for you. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 ***CUT HERE *** import metakit, bisect db = metakit.storage() vw = db.getas("table[time:f]").ordered() data = [1.2, 0.9, 1.3, 1.6, 0.5] for x in data: vw.append(x) metakit.dump(vw) class ColumnToList: def __init__(self, view, column): """Wrap a metakit column to (sort of) look like a list""" self.view = view self.column = column def __getitem__(self, index): return getattr(self.view[index], self.column) def __len__(self): return len(self.view) fakelist = ColumnToList(vw, "time") print fakelist[1] print "index for 0.2 is", bisect.bisect(fakelist, 0.2) print "index for 1.5 is", bisect.bisect(fakelist, 1.5) threshold = 1.4 index = bisect.bisect(fakelist, threshold) while index < len(fakelist): if vw[index].time > threshold: break index += 1 if index < len(view): print "found first value above", threshold, vw[index].time, "at index", index else: print "not found" -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From olivetti@itc.it Mon Jul 14 11:24:35 2003 Received: from camelot.itc.it (camelot.itc.it [195.223.171.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6EGOYX02298 for ; Mon, 14 Jul 2003 11:24:35 -0500 Received: from orchestra.itc.it (orchestra [10.0.10.11]) by camelot.itc.it (8.12.8/8.12.8) with ESMTP id h6EGOXdu005842 for ; Mon, 14 Jul 2003 18:24:33 +0200 (MET DST) Received: from eloy.itc.it. (eloy [10.30.0.71]) by orchestra.itc.it (8.11.6/8.11.6) with ESMTP id h6EGOXB19970; Mon, 14 Jul 2003 18:24:33 +0200 Received: (from olivetti@localhost) by eloy.itc.it. (8.11.6/8.11.2) id h6EGOXJ01767; Mon, 14 Jul 2003 18:24:33 +0200 Date: Mon, 14 Jul 2003 18:24:32 +0200 From: Emanuele Olivetti To: metakit@equi4.com Message-ID: <20030714182432.J21981@eloy.itc.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i Subject: [Metakit] bug in Mk4py ? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: metakit v2.4.9.2 , python v2.2.2 on linux box (redhat 7.3 and mdk 9.1) try this to get a core (foo.bd doesn't exist or doesn't have the 'bar' view): --- import metakit db=metakit.storage("foo.db",1) print db.description("bar") --- Segmentation fault (core dumped) I tried to go deeper (I've already made python bindings for a C library but not for sip/C++) but haven't found a solution because I still don't know metakit architecture. The corresponding C++ example does not give a core (db.Description just returns a null value, as expected) so I think it's a problem of the python wrapper. Another (very small) bug. In the installation instructions (README) it's written: To build the Mk4py extension on Unix, change the configure to: % ../unix/configure --enable-python it is not true (at least for my configuration); you've to do: % ../unix/configure --with-python=PATH with the correct python PATH. Hope this helps, Emanuele From gmcm@hypernet.com Mon Jul 14 13:49:38 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6EIncX16371 for ; Mon, 14 Jul 2003 13:49:38 -0500 Received: from PARANOIA (204.176.40.40) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Mon, 14 Jul 2003 14:51:14 -0400 From: "Gordon McMillan" To: "Brian Kelley" , metakit@equi4.com Date: Mon, 14 Jul 2003 14:48:18 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Neat Trick: using python's bisect with ordered columns Reply-to: gmcm@hypernet.com Message-ID: <3F12C2B2.20187.F289A43@localhost> Priority: normal In-reply-to: <3F12D696.2080203@wi.mit.edu> References: <39372.62.123.124.44.1057511470.squirrel@webmail.itc.it> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 14 Jul 2003 at 12:13, Brian Kelley wrote: > (This reply never made it to the mailing list, I thought the solution > was pretty neat so here it is) Clever, but if the view is properly ordered, and Metakit knows that (through the use of ordered or hash etc.), then find is already a binary search. If Metakit doesn't know it's ordered, use search or locate. > Euele Olivetti wrote: > > > Hi, > > I'm using metakit 4 python and trying to solve this problem: > > given an ordered view, find the first record with a certain field > > greater that a given value. It is like a MySQL select with the keyword > > LIMIT=1 , to stop the process after the first match. > > > I'm not sure that metakit supports this type of search but we would > still like to use the fact that the column you are searching is ordered. > Having said that, here is my solution. > > Fact #1 python has a bisect package that supports this type of thing. > Fact #2 it only works on lists. Solution -> make a metakit column look > like a list. > > Hope this works out for you. > > -- > Brian Kelley bkelley@wi.mit.edu > Whitehead Institute for Biomedical Research 617 258-6191 > ***CUT HERE *** > > import metakit, bisect > > db = metakit.storage() > vw = db.getas("table[time:f]").ordered() > data = [1.2, 0.9, 1.3, 1.6, 0.5] > for x in data: vw.append(x) > metakit.dump(vw) > > class ColumnToList: > def __init__(self, view, column): > """Wrap a metakit column to (sort of) look like a list""" > self.view = view > self.column = column > def __getitem__(self, index): > return getattr(self.view[index], self.column) > def __len__(self): > return len(self.view) > > fakelist = ColumnToList(vw, "time") > print fakelist[1] > > print "index for 0.2 is", bisect.bisect(fakelist, 0.2) > print "index for 1.5 is", bisect.bisect(fakelist, 1.5) > > threshold = 1.4 > index = bisect.bisect(fakelist, threshold) > while index < len(fakelist): > if vw[index].time > threshold: > break > index += 1 > if index < len(view): > print "found first value above", threshold, vw[index].time, "at > index", index > else: > print "not found" > > > > > > > > -- > Brian Kelley bkelley@wi.mit.edu > Whitehead Institute for Biomedical Research 617 258-6191 > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Mon Jul 14 13:49:39 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6EIndX16374 for ; Mon, 14 Jul 2003 13:49:39 -0500 Received: from PARANOIA (204.176.40.40) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Mon, 14 Jul 2003 14:51:16 -0400 From: "Gordon McMillan" To: Emanuele Olivetti , metakit@equi4.com Date: Mon, 14 Jul 2003 14:48:18 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] bug in Mk4py ? Reply-to: gmcm@hypernet.com Message-ID: <3F12C2B2.16892.F289967@localhost> Priority: normal In-reply-to: <20030714182432.J21981@eloy.itc.it> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 14 Jul 2003 at 18:24, Emanuele Olivetti wrote: > metakit v2.4.9.2 , python v2.2.2 on linux box (redhat 7.3 and mdk 9.1) > > try this to get a core (foo.bd doesn't exist or doesn't have the > 'bar' view): > --- > import metakit > db=metakit.storage("foo.db",1) > print db.description("bar") > --- > > Segmentation fault (core dumped) Please log it in the bug tracker on equi4. > Another (very small) bug. In the installation instructions > (README) it's written: > > To build the Mk4py extension on Unix, change the configure to: > % ../unix/configure --enable-python > > it is not true (at least for my configuration); you've to do: > % ../unix/configure --with-python=PATH > with the correct python PATH. If you pull metakit from CVS, don't mention python to configure. Once Metakit is built: cd python python setup.py install -- Gordon http://www.mcmillan-inc.com/ From bkelley@wi.mit.edu Mon Jul 14 16:04:39 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6EL4dX28827 for ; Mon, 14 Jul 2003 16:04:39 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071417043526503 for ; Mon, 14 Jul 2003 17:04:35 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI197O00.QH6; Mon, 14 Jul 2003 17:04:36 -0400 Message-ID: <3F131B3C.2080004@wi.mit.edu> Date: Mon, 14 Jul 2003 17:06:04 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gmcm@hypernet.com, metakit@equi4.com, olivetti@itc.it Subject: Re: [Metakit] Neat Trick: using python's bisect with ordered columns References: <39372.62.123.124.44.1057511470.squirrel@webmail.itc.it> <3F12C2B2.20187.F289A43@localhost> In-Reply-To: <3F12C2B2.20187.F289A43@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: >On 14 Jul 2003 at 12:13, Brian Kelley wrote: > > > >>(This reply never made it to the mailing list, I thought the solution >>was pretty neat so here it is) >> >> > >Clever, but if the view is properly ordered, and Metakit knows that >(through the use of ordered or hash etc.), then find is already >a binary search. If Metakit doesn't know it's ordered, use search >or locate. > find won't work to find the insertion point for threshold type values. Search or locate are what I was missing. Although it seems that they only work on ordered columns (see below). In summary (please correct me if I am wrong)... vw.search on ordered columns finds the proper insertion point for new values ala bisect. vw.search on unordered columns is undefined. import metakit db = metakit.storage() # works #vw = db.getas("test[time:D]").ordered() # doesn't work vw = db.getas("test[time:D]") for t in [0.0, 0.7, 0.3, 1.0, 0.2]: vw.append(t) metakit.dump(vw) # find the insertion point for a given value index = vw.search(time=0.4) for i in range(index, len(vw)): print i, vw[i].time assert vw[i].time > 0.4 # find inserttion point and number of replicates (identical # values) index, replicates = vw.locate(time=0.2) print index, replicates -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From david@rebirthing.co.nz Mon Jul 14 17:16:26 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6EMGPX02602 for ; Mon, 14 Jul 2003 17:16:25 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19cBcH-0000fU-00 for ; Tue, 15 Jul 2003 10:16:17 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058220935.1223.3.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 15 Jul 2003 10:15:35 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] determining if it's a PyStorage, or a PyView, or PyROViewer Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, How can I determine if an arg passed to a function is a storage type, a view type or a derived view type? The entity 'PyStorage' (as returned by 'type(db)', doesn't seem to be available anywhere. And strangely, db.__class__ causes an exception (!?) If feels like the Mk4Py layer might be a little over-zealous about hiding its internals. Ideas anyone? -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From david@rebirthing.co.nz Mon Jul 14 22:18:33 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6F3IRX24099 for ; Mon, 14 Jul 2003 22:18:33 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19cGKT-0001ss-00 for ; Tue, 15 Jul 2003 15:18:13 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058239050.1459.65.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 15 Jul 2003 15:17:30 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Metakit Plus - a Convenience Wrapper (python) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I've created a 'convenience wrapper' for Metakit, called 'Metakit Plus', which IMO adds significant extensibility to Metakit for Python. http://www.freenet.org.nz/python/metakit Some features: - the 'dump' routine can also dump out the schema of an entire storage object - 'storage' and 'view' are implemented as classes, and behave identically to Metakit's storage and view classes. This allows for easy subclassing in client programs (For example, methods of storage and view objects which return views, will give you wrapped views instead). - When creating a 'storage' (or 'storage'-derived) object, you can specify the class to use for creating views (via a 'viewclass=myViewClass' keyword arg). This allows you to subclass Metakit Plus's view class and ensure that all created and derived views are of your subclass. - Some convenience methods have been added to the 'view' class, allowing one-step modification/deletion of rows meeting criteria. For instance, 'removeWhen', 'changeWhen', 'purgeDuplicates'. This will save coding time and help make code more readable. - the 'dump' function is now also a method of 'storage' and 'view' I've worked hard to make it plug-compatible with the metakit.py module, so doing 'import metakitplus' instead of 'import metakit' should see your progs working unchanged. I'd be grateful for feedback/suggestions/flames/ideas/bugs with this module. -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From olivetti@itc.it Tue Jul 15 08:37:19 2003 Received: from camelot.itc.it (camelot.itc.it [195.223.171.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6FDbDX30777 for ; Tue, 15 Jul 2003 08:37:14 -0500 Received: from orchestra.itc.it (orchestra [10.0.10.11]) by camelot.itc.it (8.12.8/8.12.8) with ESMTP id h6FDbAdu024469; Tue, 15 Jul 2003 15:37:10 +0200 (MET DST) Received: from eloy.itc.it. (eloy [10.30.0.71]) by orchestra.itc.it (8.11.6/8.11.6) with ESMTP id h6FDb9B10288; Tue, 15 Jul 2003 15:37:09 +0200 Received: (from olivetti@localhost) by eloy.itc.it. (8.11.6/8.11.2) id h6FDb8703413; Tue, 15 Jul 2003 15:37:08 +0200 Date: Tue, 15 Jul 2003 15:37:08 +0200 From: Emanuele Olivetti To: metakit@equi4.com, Gordon McMillan Subject: Re: [Metakit] bug in Mk4py ? Message-ID: <20030715153708.B3183@eloy.itc.it> References: <20030714182432.J21981@eloy.itc.it> <3F12C2B2.16892.F289967@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <3F12C2B2.16892.F289967@localhost>; from gmcm@hypernet.com on Mon, Jul 14, 2003 at 02:48:18PM -0400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Mon, Jul 14, 2003 at 02:48:18PM -0400, Gordon McMillan wrote: > > Segmentation fault (core dumped) > > Please log it in the bug tracker on equi4. Oops, just did it. > > Another (very small) bug. In the installation instructions > > (README) it's written: > > > > To build the Mk4py extension on Unix, change the configure to: > > % ../unix/configure --enable-python > > > > it is not true (at least for my configuration); you've to do: > > % ../unix/configure --with-python=PATH > > with the correct python PATH. > > If you pull metakit from CVS, don't mention python to configure. > Once Metakit is built: > cd python > python setup.py install My metakit is the current stable release that comes from : http://www.equi4.com/pub/mk/metakit-2.4.9.2.tar.gz (not CVS); shall I use 'python setup.py install' in this case? Why it is not mentioned in the installation instructions? Thanks in advance, Emanuele From Adrian.Davis@lewisham.gov.uk Tue Jul 15 10:17:00 2003 Received: from pixfirewall (newpost.lewisham.gov.uk [193.195.199.153]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6FFGxX06255 for ; Tue, 15 Jul 2003 10:16:59 -0500 Received: from lmx.lewisham.gov.uk (lmx.lewisham.gov.uk [193.195.207.18]) by pixfirewall (8.9.3/8.9.3) with SMTP id QAA25482 for ; Tue, 15 Jul 2003 16:04:42 +0100 Received: by lmx.lewisham.gov.uk(Lotus SMTP MTA v4.6.7 (934.1 12-30-1999)) id 80256D64.005411AC ; Tue, 15 Jul 2003 16:18:15 +0100 X-Lotus-FromDomain: LEWISHAM From: Adrian.Davis@lewisham.gov.uk To: metakit@equi4.com Message-ID: <80256D64.00540FF6.00@lmx.lewisham.gov.uk> Date: Tue, 15 Jul 2003 16:18:11 +0100 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Subject: [Metakit] Deleting rows in multi-user environment. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I am developing a multi-user application using TCL/metakit. The users access/update the database through a "server" process. When a user deletes a row using the row-ID the row-IDs of subsequent rows change. Therefore row-ID references that other users have become invalid. Is there a "metakit" method to identify a row so that when a row is deleted by one user, other users references are still valid? Many Thanks, =Adrian= Information in this message is confidential and may also be legally privileged. It is intended solely for the person to whom it is addressed. If you are not the intended recipient, please notify the sender, and then please delete the message from your system immediately. From tom.krehbiel@motorola.com Tue Jul 15 11:08:29 2003 Received: from motgate6.mot.com (motgate6.mot.com [144.189.100.106]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6FG8SX11101; Tue, 15 Jul 2003 11:08:29 -0500 Received: from az33exr01.mot.com (az33exr01.mot.com [10.64.251.231]) by motgate6.mot.com (Motorola/Motgate6) with ESMTP id h6FG8SIP001093; Tue, 15 Jul 2003 09:08:28 -0700 (MST) Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exr01.mot.com (Motorola/az33exr01) with ESMTP id h6FG8R4e014778; Tue, 15 Jul 2003 11:08:28 -0500 Message-ID: <3F1426F9.2020800@motorola.com> Date: Tue, 15 Jul 2003 09:08:25 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DMO User-Agent: Mozilla/5.0 (X11; U; HP-UX 9000/785; en-US; rv:1.2) Gecko/20021219 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Starkit mailing list CC: metakit@equi4.com Subject: Re: [Metakit] Deleting rows in multi-user environment. References: <80256D64.00540FF6.00@lmx.lewisham.gov.uk> In-Reply-To: <80256D64.00540FF6.00@lmx.lewisham.gov.uk> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Adrian, > I am developing a multi-user application using TCL/metakit. The users > access/update the database through a "server" process. > When a user deletes a row using the row-ID the row-IDs of subsequent rows > change. Therefore row-ID references that other users have become invalid. > > Is there a "metakit" method to identify a row so that when a row is deleted by > one user, other users references are still valid? Since your using Tcl, you should use tclkit as the executable (which has metakit included). See http://www.equi4.com/ for infor on Tclkit and Starkit info. Also at this sight is Tequila which is the solution to your problem. It provides an elegent shared global array throuth a server which can be tied to the metakit database. Tom K. From seanamckay@juno.com Tue Jul 15 16:52:47 2003 Received: from m10.nyc.untd.com (m10.nyc.untd.com [64.136.22.73]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6FLqlX07983 for ; Tue, 15 Jul 2003 16:52:47 -0500 Received: from cookie.untd.com by cookie.untd.com for <"qWseXT3rq5SaoRv93RsNRHHFxiy/EewO22F8r3LGIwZgAqkdMSbNNw==">; Tue, 15 Jul 2003 17:52:32 EDT Received: (from seanamckay@juno.com) by m10.nyc.untd.com (jqueuemail) id H46M9X65; Tue, 15 Jul 2003 17:52:32 EDT To: metakit@equi4.com Date: Tue, 15 Jul 2003 16:51:32 -0500 Message-ID: <20030715.165137.-1408747.1.seanamckay@juno.com> X-Mailer: Juno 4.0.11 MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Juno-Line-Breaks: 1-12,14-15,17-20,22-24 X-Juno-Att: 0 X-Juno-RefParts: 0 From: Sean A McKay Subject: [Metakit] MkSQL MemoryError error on fetchall() Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I encountered the following error while going through the MkSQL user's guide: >>> from MkSQL.MkSQLDBI import * >>> conn = connect(file_name) >>> cur = conn.cursor() >>> cur.execute("select * from Status ") >>> row0 = cur.fetchone() >>> acouple = cur.fetchmany(size=2) >>> therest = cur.fetchall() Traceback (most recent call last): File "", line 1, in ? therest = cur.fetchall() File "F:\PYTHON22\Lib\site-packages\MkSQL\MkSQLDBI.py", line 146, in fetchall slice = self.rs[self.ndx:] File "F:\PYTHON22\Lib\site-packages\MkSQL\SQL.py", line 90, in __getslice__ iv = metakit.wrap(range(lo, hi), [metakit.property('I','index')]) MemoryError Any thoughts on how to resolve this, or is this a bug? (Btw, I know that the Status table had a total of 5 rows) Thanks, Sean ________________________________________________________________ The best thing to hit the internet in years - Juno SpeedBand! Surf the web up to FIVE TIMES FASTER! Only $14.95/ month - visit www.juno.com to sign up today! From seanamckay@juno.com Tue Jul 15 17:25:47 2003 Received: from m10.nyc.untd.com (m10.nyc.untd.com [64.136.22.73]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6FMPkX10978 for ; Tue, 15 Jul 2003 17:25:46 -0500 Received: from cookie.untd.com by cookie.untd.com for <"qWseXT3rq5SaoRv93RsNRHHFxiy/EewOCQ9rjWYZIunlTyCqySEhdA==">; Tue, 15 Jul 2003 18:25:14 EDT Received: (from seanamckay@juno.com) by m10.nyc.untd.com (jqueuemail) id H46P5TXL; Tue, 15 Jul 2003 18:25:14 EDT To: metakit@equi4.com Date: Tue, 15 Jul 2003 17:24:19 -0500 Message-ID: <20030715.172419.-1408747.4.seanamckay@juno.com> X-Mailer: Juno 4.0.11 MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Juno-Line-Breaks: 1-16,18-19 X-Juno-Att: 0 X-Juno-RefParts: 0 From: Sean A McKay Subject: [Metakit] MkSql Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I encountered the following errors w/ MkSql (btw, I'm using 2.4.9.2 and the latest MkSql) >>> from MkSQL.MkSQLDBI import * >>> conn = connect(file_name) >>> cur = conn.cursor() >>> cur.execute("select * from Status ") >>> row0 = cur.fetchone() >>> acouple = cur.fetchmany(size=2) >>> row0 ('Completed', 0) >>> acouple Traceback (most recent call last): File "", line 1, in ? acouple File "F:\PYTHON22\Lib\site-packages\MkSQL\SQL.py", line 41, in __repr__ tmp = [repr(row) for row in self] File "F:\PYTHON22\Lib\site-packages\MkSQL\SQL.py", line 70, in __getitem__ return self.wrapper(self.bv[self.iv[ndx].index], self.colmap) ValueError: Object has no usable attributes ________________________________________________________________ The best thing to hit the internet in years - Juno SpeedBand! Surf the web up to FIVE TIMES FASTER! Only $14.95/ month - visit www.juno.com to sign up today! From bkelley@wi.mit.edu Wed Jul 16 08:46:58 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6GDkwX06880 for ; Wed, 16 Jul 2003 08:46:58 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071609465412389 for ; Wed, 16 Jul 2003 09:46:54 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI4EA600.P9E; Wed, 16 Jul 2003 09:46:54 -0400 Message-ID: <3F1557A4.1070504@wi.mit.edu> Date: Wed, 16 Jul 2003 09:48:20 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Sean A McKay CC: metakit@equi4.com Subject: Re: [Metakit] MkSql References: <20030715.172419.-1408747.4.seanamckay@juno.com> In-Reply-To: <20030715.172419.-1408747.4.seanamckay@juno.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Sean A McKay wrote: >I encountered the following errors w/ MkSql (btw, I'm using 2.4.9.2 and >the latest MkSql) > > > >>>>from MkSQL.MkSQLDBI import * >>>>conn = connect(file_name) >>>>cur = conn.cursor() >>>>cur.execute("select * from Status ") >>>>row0 = cur.fetchone() >>>>acouple = cur.fetchmany(size=2) >>>>row0 >>>> >>>> Could you send me the database files or perhaps an example where you create and load the database with the relevant data and produce this error? I'm wondering if the problem is fetchall() called after fetchmany(). What happens if you just use fetchall()? -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From bkelley@wi.mit.edu Wed Jul 16 08:52:34 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6GDqYX07367 for ; Wed, 16 Jul 2003 08:52:34 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071609523012464 for ; Wed, 16 Jul 2003 09:52:30 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI4EJI00.HBG; Wed, 16 Jul 2003 09:52:30 -0400 Message-ID: <3F1558F4.3000305@wi.mit.edu> Date: Wed, 16 Jul 2003 09:53:56 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: David McNab CC: metakit@equi4.com Subject: Re: [Metakit] Metakit Plus - a Convenience Wrapper (python) References: <1058239050.1459.65.camel@rebirth> In-Reply-To: <1058239050.1459.65.camel@rebirth> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: David McNab wrote: >Hi, > >I've created a 'convenience wrapper' for Metakit, called 'Metakit Plus', >which IMO adds significant extensibility to Metakit for Python. > >http://www.freenet.org.nz/python/metakit > > I've also created a convenience wrapper that acts as an observable. That is, when new views are created or modified an observable is notified so that external views can be updated accordingly. The two convenience functions that I've added are 1) view.filterView which acts like view.filter but returns a new view 2) view.joinDifferent(view2, (id1,id2), (id3,id4)) which joins two views on different ids. This is equivalent to select * from vw1,vw2 where vw1.id=vw2.other_id It also keeps a functional history of how a view was created so that it can be regenerated at a later date. (This almost works :)) Perhaps we should combine efforts? -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From bkelley@wi.mit.edu Wed Jul 16 09:09:43 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6GE9hX09115 for ; Wed, 16 Jul 2003 09:09:43 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071610093912659 for ; Wed, 16 Jul 2003 10:09:39 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI4FC300.LAU; Wed, 16 Jul 2003 10:09:39 -0400 Message-ID: <3F155CF9.4000806@wi.mit.edu> Date: Wed, 16 Jul 2003 10:11:05 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 CC: Sean A McKay , metakit@equi4.com Subject: Re: [Metakit] MkSql References: <20030715.172419.-1408747.4.seanamckay@juno.com> <3F1557A4.1070504@wi.mit.edu> In-Reply-To: <3F1557A4.1070504@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Yep, fetchmany appears to be broken. I'll file a bug report. If you want to get into the internals of mksql, see below from MkSQL.MkSQLDBI import * import metakit c1 = """\ create table drinkers ( name char unique, age integer ) """ conn = connect("testdb.db") cur = conn.cursor() cur.execute(c1) for name, age in ("norm", 22), ("bart", 4), ("foo", 44): cur.execute("insert into drinkers (name, age) values (:name, :age)", {"name":name, "age":age}) cur.execute("select * from drinkers") res = cur.fetchmany(2) print len(res) metakit.dump(res.bv) print res.iv # for some reason the internal index view appears to be broken. # res.iv contains no usable attributes (this is needed for fetchmany # I believe) metakit.dump(res.iv) -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From bkelley@wi.mit.edu Wed Jul 16 09:11:36 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6GEBZX09344 for ; Wed, 16 Jul 2003 09:11:36 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071610113512683 for ; Wed, 16 Jul 2003 10:11:35 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI4FFB00.NAM; Wed, 16 Jul 2003 10:11:35 -0400 Message-ID: <3F155D6D.5060705@wi.mit.edu> Date: Wed, 16 Jul 2003 10:13:01 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Sean A McKay , metakit@equi4.com Subject: Re: [Metakit] MkSQL MemoryError error on fetchall() References: <20030715.165137.-1408747.1.seanamckay@juno.com> In-Reply-To: <20030715.165137.-1408747.1.seanamckay@juno.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Sean A McKay wrote: >I encountered the following error while going through the MkSQL user's >guide: > > > >>>>from MkSQL.MkSQLDBI import * >>>>conn = connect(file_name) >>>>cur = conn.cursor() >>>>cur.execute("select * from Status ") >>>>row0 = cur.fetchone() >>>>acouple = cur.fetchmany(size=2) >>>>therest = cur.fetchall() >>>> fetchmany currently appears to be broken. To solve this problem, don't use it :) I'll file a bug report, see by previous email if you want more details. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From bkelley@wi.mit.edu Wed Jul 16 09:48:14 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6GEmEX13368 for ; Wed, 16 Jul 2003 09:48:14 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071610481013131 for ; Wed, 16 Jul 2003 10:48:10 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI4H4A00.OB6; Wed, 16 Jul 2003 10:48:10 -0400 Message-ID: <3F156601.4010500@wi.mit.edu> Date: Wed, 16 Jul 2003 10:49:37 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 CC: seanamckay@juno.com, metakit@equi4.com Subject: Re: [Metakit] MkSql References: <20030715.172419.-1408747.4.seanamckay@juno.com> <3F1557A4.1070504@wi.mit.edu> <3F155CF9.4000806@wi.mit.edu> In-Reply-To: <3F155CF9.4000806@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > Yep, fetchmany appears to be broken. I'll file a bug report. > If you want to get into the internals of mksql, see below > The bug is in the python metakit wrapper. Namely the "wrap" function is now broken. import metakit print metakit.version iv = metakit.wrap([0,1,2], [metakit.property("I","index")]) print iv[0].index -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From seanamckay@juno.com Wed Jul 16 09:52:01 2003 Received: from m10.nyc.untd.com (m10.nyc.untd.com [64.136.22.73]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6GEq1X13825 for ; Wed, 16 Jul 2003 09:52:01 -0500 Received: from cookie.untd.com by cookie.untd.com for <"qWseXT3rq5SaoRv93RsNRHHFxiy/EewOe+X8ZDbyU5wxr0X0cuhrlA==">; Wed, 16 Jul 2003 10:50:50 EDT Received: (from seanamckay@juno.com) by m10.nyc.untd.com (jqueuemail) id H48GJHRL; Wed, 16 Jul 2003 10:50:50 EDT To: bkelley@wi.mit.edu Cc: metakit@equi4.com Date: Wed, 16 Jul 2003 09:44:16 -0500 Subject: Re: [Metakit] MkSql Message-ID: <20030716.094416.-853401.0.seanamckay@juno.com> X-Mailer: Juno 4.0.11 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=--__JNP_000_6268.544d.5d39 X-Juno-Line-Breaks: 0-8,11-16,18-49,49-32767 X-Juno-Att: 1 X-Juno-RefParts: 0 From: Sean A McKay Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ----__JNP_000_6268.544d.5d39 Content-Type: text/plain Content-Transfer-Encoding: 7bit Brian, You are correct, if I call fetchall after a fetchone or a fetchmany, it doesn't work, BUT, if I call it by itself (e.g. w/o any preceding fetchone or fetchmany), it works fine. Attached is a copy of the table that the example dies on.. Sean On Wed, 16 Jul 2003 09:48:20 -0400 "Brian Kelley" writes: > Sean A McKay wrote: > > >I encountered the following errors w/ MkSql (btw, I'm using 2.4.9.2 > and > >the latest MkSql) > > > > > > > >>>>from MkSQL.MkSQLDBI import * > >>>>conn = connect(file_name) > >>>>cur = conn.cursor() > >>>>cur.execute("select * from Status ") > >>>>row0 = cur.fetchone() > >>>>acouple = cur.fetchmany(size=2) > >>>>row0 > >>>> > >>>> > Could you send me the database files or perhaps an example where you > create and load the database with the relevant data and produce this > error? > > I'm wondering if the problem is fetchall() called after fetchmany(). > > What happens if you just use fetchall()? > > -- > Brian Kelley bkelley@wi.mit.edu > Whitehead Institute for Biomedical Research 617 258-6191 > > > ----__JNP_000_6268.544d.5d39 Content-Type: application/octet-stream; name="Status.mk" Content-Transfer-Encoding: base64 SkwaAAAAALmAgBAyZAoMCAwVgIWDir/GhY2ASSxkZXNjcmlwdGlvbjpTXYGCiIAAAAAAAAAugAAA JAAAAApKTAoAAAAAe0NvbXBsZXRlZABJbiBQcm9ncmVzcwBXYWl0aW5nAE5vdCBTdGFydGVkAE5v IExvbmdlciBBcHBsaWNhYmxlAICfU3RhdHVzW3R5cGVfaWQ6SSxkZXNjcmlwdGlvbjpTXYGJkoAA AAAAAACpgAAAJAAAAIU= ----__JNP_000_6268.544d.5d39-- ________________________________________________________________ The best thing to hit the internet in years - Juno SpeedBand! Surf the web up to FIVE TIMES FASTER! Only $14.95/ month - visit www.juno.com to sign up today! From bkelley@wi.mit.edu Wed Jul 16 12:13:47 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6GHDlL31640 for ; Wed, 16 Jul 2003 12:13:47 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003071613134314915 for ; Wed, 16 Jul 2003 13:13:43 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HI4NUV00.1CD; Wed, 16 Jul 2003 13:13:43 -0400 Message-ID: <3F15881D.4020806@wi.mit.edu> Date: Wed, 16 Jul 2003 13:15:09 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Brian Kelley CC: seanamckay@juno.com, metakit@equi4.com, Gordon McMillan Subject: Re: [Metakit] MkSql References: <20030715.172419.-1408747.4.seanamckay@juno.com> <3F1557A4.1070504@wi.mit.edu> <3F155CF9.4000806@wi.mit.edu> <3F156601.4010500@wi.mit.edu> In-Reply-To: <3F156601.4010500@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > Brian Kelley wrote: > >> Yep, fetchmany appears to be broken. I'll file a bug report. >> If you want to get into the internals of mksql, see below >> > The bug is in the python metakit wrapper. Namely the "wrap" function > is now broken. > > import metakit > print metakit.version > iv = metakit.wrap([0,1,2], [metakit.property("I","index")]) > print iv[0].index > > metakit.wrap works fine for dictionaries. To fix fetchmany make the following change to SQL.py in MkSQL class ResultSet(_RS): ... ... def __getslice__(self, lo, hi): change iv = metakit.wrap(range(lo, hi), [metakit.property('I','index')]) to iv = metakit.wrap([{"index":x} for x in range(lo, hi)], [metakit.property('I','index')]) and fetchmany will work once again. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From Adrian.Davis@lewisham.gov.uk Thu Jul 17 06:35:36 2003 Received: from pixfirewall (newpost.lewisham.gov.uk [193.195.199.153]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6HBZYL10886 for ; Thu, 17 Jul 2003 06:35:36 -0500 Received: from lmx.lewisham.gov.uk (lmx.lewisham.gov.uk [193.195.207.18]) by pixfirewall (8.9.3/8.9.3) with SMTP id MAA09824 for ; Thu, 17 Jul 2003 12:23:15 +0100 Received: by lmx.lewisham.gov.uk(Lotus SMTP MTA v4.6.7 (934.1 12-30-1999)) id 80256D66.003FCBED ; Thu, 17 Jul 2003 12:36:49 +0100 X-Lotus-FromDomain: LEWISHAM From: Adrian.Davis@lewisham.gov.uk To: metakit@equi4.com Message-ID: <80256D66.003FCA43.00@lmx.lewisham.gov.uk> Date: Thu, 17 Jul 2003 12:36:44 +0100 Mime-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-Disposition: inline Subject: [Metakit] One reader many writers? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I've looked at what MetaKit documentation I can find, but I am still unsure as to whether it is possible to have the database open for writing by one process and another process has it open for reading. Is this possible? If so, are there any special considerations or limitations? Many Thanks, =Adrian= Information in this message is confidential and may also be legally privileged. It is intended solely for the person to whom it is addressed. If you are not the intended recipient, please notify the sender, and then please delete the message from your system immediately. From david@rebirthing.co.nz Thu Jul 17 06:43:26 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6HBhOL11335 for ; Thu, 17 Jul 2003 06:43:25 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19d7AI-0002Oa-00 for ; Thu, 17 Jul 2003 23:43:14 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058442146.1228.2.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 17 Jul 2003 23:42:26 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Question - editable joins? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I note that normal views are of type 'PyView', whereas views returned by a join() are of type 'PyROViewer', and that I can't edit any of the data in these views. Is there any way I can edit the values in a join()-derived view, without having to hunt down the corresponding rows/fields in the original view(s)? (Using Mk4py) -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From gmcm@hypernet.com Thu Jul 17 07:26:09 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6HCQ9L13812 for ; Thu, 17 Jul 2003 07:26:09 -0500 Received: from PARANOIA (204.176.40.47) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Thu, 17 Jul 2003 08:27:51 -0400 From: "Gordon McMillan" To: Adrian.Davis@lewisham.gov.uk, metakit@equi4.com Date: Thu, 17 Jul 2003 08:24:55 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] One reader many writers? Reply-to: gmcm@hypernet.com Message-ID: <3F165D57.9137.1D3CAE56@localhost> Priority: normal In-reply-to: <80256D66.003FCA43.00@lmx.lewisham.gov.uk> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 17 Jul 2003 at 12:36, Adrian.Davis@lewisham.gov.uk wrote: > I've looked at what MetaKit documentation I can find, but I am still > unsure as to whether it is possible to have the database open for > writing by one process and another process has it open for reading. > > Is this possible? If so, are there any special considerations or > limitations? In normal commit-mode, there are (a tad oversimpified) 2 dbs mapped into memory: [old,new] which on mutation / commit becomes [newer,new] and then [newer,newest] (that is, they swap). A reader will therefor be safe through one commit, but unless he reopens, he'll get trashed on the next one. The extend and aside modes leave readers safe (still reading unmodified until they reopen), but then you need to periodically compact. Note that opening is very cheap. -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Thu Jul 17 07:26:08 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6HCQ7L13809 for ; Thu, 17 Jul 2003 07:26:07 -0500 Received: from PARANOIA (204.176.40.47) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Thu, 17 Jul 2003 08:27:49 -0400 From: "Gordon McMillan" To: David McNab , metakit@equi4.com Date: Thu, 17 Jul 2003 08:24:55 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Question - editable joins? Reply-to: gmcm@hypernet.com Message-ID: <3F165D57.8731.1D3CADF2@localhost> Priority: normal In-reply-to: <1058442146.1228.2.camel@rebirth> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 17 Jul 2003 at 23:42, David McNab wrote: > I note that normal views are of type 'PyView', whereas views > returned by a join() are of type 'PyROViewer', and that I > can't edit any of the data in these views. > > Is there any way I can edit the values in a join()-derived > view, without having to hunt down the corresponding > rows/fields in the original view(s)? > > (Using Mk4py) No. The reason filter returns an "index view" instead of doing it's own remapwith is so code can get to the proper rows in the base view. You might also look at view.map(). The approach used by filter and map (passing in a callable) generally result in faster code than doing the looping in Python. -- Gordon http://www.mcmillan-inc.com/ From david@rebirthing.co.nz Thu Jul 17 09:11:59 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6HEBqL20795 for ; Thu, 17 Jul 2003 09:11:58 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19d9Tv-00053S-00; Fri, 18 Jul 2003 02:11:39 +1200 Subject: Re: [Metakit] Question - editable joins? From: David McNab To: gmcm@hypernet.com Cc: metakit@equi4.com In-Reply-To: <3F165D57.8731.1D3CADF2@localhost> References: <3F165D57.8731.1D3CADF2@localhost> Content-Type: text/plain Message-Id: <1058451051.1224.6.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 18 Jul 2003 02:10:51 +1200 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Fri, 2003-07-18 at 00:24, Gordon McMillan wrote: > > Is there any way I can edit the values in a join()-derived > > view, without having to hunt down the corresponding > > rows/fields in the original view(s)? > > > > (Using Mk4py) > > No. No probs - I'm updating the wrapper to make derived views editable. In other words, if you change a derived view, the changes propagate 'upstream' to the ancestor view(s). Cheers David From david@rebirthing.co.nz Thu Jul 17 21:39:49 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6I2dlL20570 for ; Thu, 17 Jul 2003 21:39:48 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19dL9n-0000WQ-00 for ; Fri, 18 Jul 2003 14:39:39 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058495930.1548.2.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 18 Jul 2003 14:38:50 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Performance - storage.view() versus storage.getas() Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I was thinking about opening my existing views with getas(), but am worried about performance. Will performance suffer unduly if I do: vw = db.getas("people[first:S,last:S]") instead of: try: vw = db.view('people') except: vw = db.getas('people[first:S,last:S]') ? -- Cheers David -- leave this line intact so your email gets through my junk mail filter From david@rebirthing.co.nz Thu Jul 17 21:48:42 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6I2meL21073 for ; Thu, 17 Jul 2003 21:48:41 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19dLIP-0000Yx-00 for ; Fri, 18 Jul 2003 14:48:33 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058496464.1547.7.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 18 Jul 2003 14:47:44 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Deleting views from a storage object/file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Is there any way to remove a view and its data from a storage file (like MySQL 'DROP TABLE tablename")? Or is it a matter of creating a new storage file, and copying across all the views sans the unwanted one, committing it and renaming it to the old file? -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From gmcm@hypernet.com Fri Jul 18 07:37:47 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6ICblL23124 for ; Fri, 18 Jul 2003 07:37:47 -0500 Received: from PARANOIA (204.176.40.68) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Fri, 18 Jul 2003 08:39:31 -0400 From: "Gordon McMillan" To: David McNab , metakit@equi4.com Date: Fri, 18 Jul 2003 08:36:36 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Deleting views from a storage object/file Reply-to: gmcm@hypernet.com Message-ID: <3F17B194.6300.226DBE48@localhost> Priority: normal In-reply-to: <1058496464.1547.7.camel@rebirth> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 18 Jul 2003 at 14:47, David McNab wrote: > Is there any way to remove a view and its data from a storage file (like > MySQL 'DROP TABLE tablename")? db.getas(tablename) > Or is it a matter of creating a new storage file, and copying > across all the views sans the unwanted one, committing it and > renaming it to the old file? Hee hee. I thought so at one time :-). -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Fri Jul 18 07:37:48 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6ICblL23127 for ; Fri, 18 Jul 2003 07:37:48 -0500 Received: from PARANOIA (204.176.40.68) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Fri, 18 Jul 2003 08:39:33 -0400 From: "Gordon McMillan" To: David McNab Date: Fri, 18 Jul 2003 08:36:36 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Performance - storage.view() versus storage.getas() Reply-to: gmcm@hypernet.com CC: Metakit mailing list Message-ID: <3F17B194.11111.226DBDD9@localhost> Priority: normal In-reply-to: <1058495930.1548.2.camel@rebirth> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 18 Jul 2003 at 14:38, David McNab wrote: > I was thinking about opening my existing views with getas(), but am > worried about performance. > > Will performance suffer unduly if I do: > > vw = db.getas("people[first:S,last:S]") > > instead of: > > try: > vw = db.view('people') > except: > vw = db.getas('people[first:S,last:S]') - the latter way won't work unless you build from CVS - even then it will only detect that "people" isn't there - with a bit of twiddling, you can compare description strings All that said, the performance vs "getas" would probably only be noticable if you have large numbers of views. -- Gordon http://www.mcmillan-inc.com/ From david@rebirthing.co.nz Sun Jul 20 12:52:00 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6KHpvL03311 for ; Sun, 20 Jul 2003 12:52:00 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19eILc-0001Iz-00 for ; Mon, 21 Jul 2003 05:51:48 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058723454.1423.5.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 21 Jul 2003 05:50:54 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] A hack for Multi-User access Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I'm using the following hack to enable a crude multi-user ability for my Metakit database: db = metakitplus.storage() fd = open(dbfile, "rb+") fcntl.flock(fd, fcntl.LOCK_EX) db.load(fd) # do whatever (quick) processing is needed... fd.seek(0) fd.truncate(0) db.save(self.fd) fcntl.flock(fd, fcntl.LOCK_UN) fd.close() In my environment, there isn't much processing between opening and closing the database, so there won't be any processes 'clogging up the works'. My question is - is this the best way to make a metakit database safe for multi-user deployment, or is there something simpler/better/cleaner? -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From gmcm@hypernet.com Sun Jul 20 13:22:54 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6KIMsL05100 for ; Sun, 20 Jul 2003 13:22:54 -0500 Received: from PARANOIA (204.176.40.77) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Sun, 20 Jul 2003 14:24:41 -0400 From: "Gordon McMillan" To: David McNab , metakit@equi4.com Date: Sun, 20 Jul 2003 14:21:46 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] A hack for Multi-User access Reply-to: gmcm@hypernet.com Message-ID: <3F1AA57A.2095.2DF67675@localhost> Priority: normal In-reply-to: <1058723454.1423.5.camel@rebirth> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 21 Jul 2003 at 5:50, David McNab wrote: > I'm using the following hack to enable a crude multi-user > ability for my Metakit database: > > db = metakitplus.storage() > fd = open(dbfile, "rb+") > fcntl.flock(fd, fcntl.LOCK_EX) > db.load(fd) > > # do whatever (quick) processing is needed... > > fd.seek(0) > fd.truncate(0) > db.save(self.fd) > fcntl.flock(fd, fcntl.LOCK_UN) > fd.close() load / save are significantly more expensive than storate(fnm, 1) - they serialize all the data, rather than just mmap'ing the file. So I generally use a separate lock file. > In my environment, there isn't much processing between > opening and closing the database, so there won't be any > processes 'clogging up the works'. > > My question is - is this the best way to make a metakit > database safe for multi-user deployment, or is there something > simpler/better/cleaner? If you keep half an eye on the efficiency of your usage, the above will take you a long, long way. There are complex things you can do with commit-extend / aside, but I've never found the need. > leave this line intact so your email gets through my junk mail filter -- Gordon http://www.mcmillan-inc.com/ From david@rebirthing.co.nz Sun Jul 20 21:21:13 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6L2LBL28743 for ; Sun, 20 Jul 2003 21:21:13 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19eQIS-0003hy-00 for ; Mon, 21 Jul 2003 14:21:04 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1058754010.1213.20.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 21 Jul 2003 14:20:10 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Securing Metakit in web environment? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi again folks, (Thanks to Gordon and others who've been patiently answering my questions). Gordon, you're right - Metakit has really grown on me, and I *do* now wonder how I lived without it. :) But I'm a bit concerned about the security of Metakit in certain web environments. I'll state things as best I understand (please correct me if I'm wrong): * If an attacker doesn't have read access to the host filesystem, access passwords are good quality, and cookies contain only session IDs which are securely managed, then security need not be a worry * Public web hosts, which have n hosting accounts on each server, don't run their CGIs as the site owner's user. They tend to dispatch all CGIs of all hosting accounts under the one user, often 'www-data' or 'nobody'. * So if an attacker gets a 'neighbour' hosting account on the same server which s/he wants to attack, it'll be a simple matter to put up a script which delivers all desired files of the target account (including its raw Metakit database files, and any files containing web passwords for gaining admin access into the site). * Password hashing can prevent the attacker getting admin access into the site, but it can't prevent the attacker from lifting the databases (including credit card details, for example) * Such an attack will yield all data (except any data that's PGP or GPG-encrypted against a key whose private key is not present on the server). My questions: 1) Does this weakness also exist in sites on public hosts with MySQL databases? Or do public web hosts tend to block a site from accessing a MySQL database it doesn't 'own'? 2) Does anyone have any ideas for minimising the overall security risks with Metakit? -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From david@rebirthing.co.nz Mon Jul 21 04:42:00 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6L9fvL23582 for ; Mon, 21 Jul 2003 04:41:59 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19eXAy-0005jN-00 for ; Mon, 21 Jul 2003 21:41:48 +1200 Subject: Re: [Metakit] Securing Metakit in web environment? From: David McNab To: metakit@equi4.com In-Reply-To: <1058754010.1213.20.camel@rebirth> References: <1058754010.1213.20.camel@rebirth> Content-Type: text/plain Message-Id: <1058780454.1203.20.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 21 Jul 2003 21:40:55 +1200 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Mon, 2003-07-21 at 14:20, David McNab wrote: > 2) Does anyone have any ideas for minimising the overall security > risks with Metakit? The solution I came up with is to bury the Metakit data files into execute-only (mode 711) subdirectories with weird non-guessable names. The web framework is fired up by a very short python CGI script, a launcher which simply imports and runs the framework in a try...except. So I've embedded the launcher script into a 'system("python -c "launcherscriptcode")' call in a 3-liner C program, and compiled that C prog to 'index.cgi' and chmod'ed it to 111. That way, the whole world (via the web) can launch the framework, but there's no way for a "neighbour's" scripts to determine the pathname where the database files are located, because they can't read the binary - they can only execute it. To me, it feels that the end result is one there's now a level of security exceeding that of many conventional MySQL-based sites. The security is based on the fact that any program can read/write/modify the database files, but *only* if they know their full pathnames. So the directory names along the path act kinda like cryptographic keys. I'd welcome any comments or feedback on this, particularly if anyone can see any holes in this arrangement. -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From bkelley@wi.mit.edu Tue Jul 22 12:38:57 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6MHcuL08854 for ; Tue, 22 Jul 2003 12:38:56 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003072213385215823 for ; Tue, 22 Jul 2003 13:38:52 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HIFT0S00.87F for ; Tue, 22 Jul 2003 13:38:52 -0400 Message-ID: <3F1D770D.9090608@wi.mit.edu> Date: Tue, 22 Jul 2003 13:40:29 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Spanning Multiple Metakit Files Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I had been wondering if metakit supported having tables stored in different files. To my utter surprise, it did: import metakit db = metakit.storage("mk1",1) vw = db.getas("test[a,b,c]") vw.append(["this", "is", "a"]) vw.append(["test", "foo", "bar"]) vw.append(["test", "foo", "bar"]) db.commit() db2 = metakit.storage("mk2",1) vw2 = db2.getas("test2[a,e:I]") vw2.append(["this", 1]) vw2.append(["test", 2]) db2.commit() metakit.dump(vw.join(vw2, vw2.a)) Did I mention how incredibly amazing this is? I had spent a couple hours implementing a CombinedView class that implemented this functionality and here I discover that I need not have bothered. I guess I should have tried it first :) I think this particular "feature" should be trumpted a little louder. I'm planning on having a large "objects" database that is read only and centrally located. Each user then would have a "results" database that is occassionally joined to the "objects" database. This means that a user can hammer however they like on the "results" database while not touching the large central database. Client/Server heaven. Now my caveat, are there any cases where this might not work? -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From david@rebirthing.co.nz Wed Jul 23 08:20:42 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6NDKeL06164 for ; Wed, 23 Jul 2003 08:20:41 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19fJXX-0002Q9-00; Thu, 24 Jul 2003 01:20:19 +1200 Subject: Re: [Metakit] Securing Metakit in web environment? From: David McNab To: Lok Yek Soon Cc: metakit@equi4.com In-Reply-To: <5.2.0.9.2.20030723014424.028a7530@pop3.myrealbox.com> References: <5.2.0.9.2.20030723014424.028a7530@pop3.myrealbox.com> Content-Type: text/plain Message-Id: <1058966362.1240.17.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 24 Jul 2003 01:19:22 +1200 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Wed, 2003-07-23 at 05:49, Lok Yek Soon wrote: > At 02:20 PM 21-07-03 +1200, you wrote: > > > >My questions: > > > >1) Does this weakness also exist in sites on public hosts with MySQL > > databases? Or do public web hosts tend to block a site from > > accessing a MySQL database it doesn't 'own'? > > > > > > [Off my head] > One way is to host it on virtual private server (VPS) instead of shared > hosting. With VPS, each account gets their own partition. This will help > reduce some of the risk involved. Totally true. There are some great VPS hosts, using virtualisation layers like VMWare, so each client is root on his/her own virtual box, and can configure the box as s/he likes. Only one problem - such hosting accounts tend to be way more expensive than traditional shared vhosts. Anyway, I've got the solution down pat, even to the point of writing an intelligent installer with built-in FTP client, that uploads the site, and configures everything for maximum security. I uploaded to one of my clients' servers, then logged on with a shell-over-web script and tried my damndest to hack it. No way in. It's even more secure than many PHP setups (eg PHP-Nuke). Over the next couple of weeks, I'll make an announcement of a full Python based Content Management System. (Soon, the acronym LAMP will also mean: "Linux Apache Metakit Python"! ) Cheers David > > YekSoon > neuSteps Technologies Pte Ltd > http://neusteps.com > mobile: +65 9684 2308 > > "bringing efficiency into your work process" > > ______________________________________________________________________ > --- > Outgoing mail is certified Virus Free. > Checked by AVG anti-virus system (http://www.grisoft.com). > Version: 6.0.500 / Virus Database: 298 - Release Date: 10-Jul-03 -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From seanamckay@juno.com Wed Jul 23 17:09:48 2003 Received: from m10.nyc.untd.com (m10.nyc.untd.com [64.136.22.73]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6NM9mL28248 for ; Wed, 23 Jul 2003 17:09:48 -0500 Received: from cookie.untd.com by cookie.untd.com for <"qWseXT3rq5SaoRv93RsNRHHFxiy/EewObWfQDkyNLrEGjtz3WVuXxw==">; Wed, 23 Jul 2003 18:09:22 EDT Received: (from seanamckay@juno.com) by m10.nyc.untd.com (jqueuemail) id H5TAEH9P; Wed, 23 Jul 2003 18:09:22 EDT To: metakit@equi4.com Date: Wed, 23 Jul 2003 17:06:52 -0500 Message-ID: <20030723.170652.-3527161.0.seanamckay@juno.com> X-Mailer: Juno 4.0.11 MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Juno-Line-Breaks: 1-3 X-Juno-Att: 0 X-Juno-RefParts: 0 From: Sean A McKay Subject: [Metakit] Python2.3 compatible Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Does anyone have a Python2.3 compatible Mk4py.dll that they'd be willing to send me? TIA, Sean ________________________________________________________________ The best thing to hit the internet in years - Juno SpeedBand! Surf the web up to FIVE TIMES FASTER! Only $14.95/ month - visit www.juno.com to sign up today! From seanamckay@juno.com Mon Jul 28 13:38:20 2003 Received: from m10.nyc.untd.com (m10.nyc.untd.com [64.136.22.73]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h6SIcKL19693 for ; Mon, 28 Jul 2003 13:38:20 -0500 Received: from cookie.untd.com by cookie.untd.com for <"qWseXT3rq5SaoRv93RsNREN/tbLpKZJg8IeOq1vP72o3bCvOs0llOg==">; Mon, 28 Jul 2003 14:37:58 EDT Received: (from seanamckay@juno.com) by m10.nyc.untd.com (jqueuemail) id H57SAZ8G; Mon, 28 Jul 2003 14:37:58 EDT To: metakit@equi4.com Date: Mon, 28 Jul 2003 13:34:57 -0500 Message-ID: <20030728.133457.-3680709.1.seanamckay@juno.com> X-Mailer: Juno 4.0.11 MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Juno-Line-Breaks: 0-1,3-10,12-18,20-29 X-Juno-Att: 0 X-Juno-RefParts: 0 From: Sean A McKay Subject: [Metakit] Another MkSQL ? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hello, I'm encountering problems w/ the update statement. Perhaps I'm doing something stupid or 'slightly' wrong, and if so, please let me know! Thanks, Sean >>> from MkSQL.MkSQLDBI import * >>> conn = connect('mydb.db') >>> cur = conn.cursor() >>> cur.execute("create table person(person_id integer primary key, name varchar)") >>> cur.execute("insert into person values (0, 'sean')") >>> cur.execute("update person set name='joe' where name='sean'") Traceback (most recent call last): File "", line 1, in ? cur.execute("update person set name='joe' where name='sean'") File "F:\PYTHON22\lib\site-packages\MkSQL\MkSQLDBI.py", line 115, in execute self.rs = stmt.execute(self.conn.db, params) File "F:\PYTHON22\lib\site-packages\MkSQL\SQL.py", line 464, in execute return self.evaluator(db, params) File "F:\PYTHON22\lib\site-packages\MkSQL\SQL.py", line 451, in f bv.update(row.index, setter(bv[row.index], params)) File "F:\PYTHON22\lib\site-packages\MkSQL\RDB.py", line 165, in update kw = self._makekw(args) File "F:\PYTHON22\lib\site-packages\MkSQL\RDB.py", line 192, in _makekw kw[prop.name] = args[i] IndexError: tuple index out of range ________________________________________________________________ The best thing to hit the internet in years - Juno SpeedBand! Surf the web up to FIVE TIMES FASTER! Only $14.95/ month - visit www.juno.com to sign up today! From bkelley@wi.mit.edu Tue Jul 29 06:22:33 2003 Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6TBMWL16684 for ; Tue, 29 Jul 2003 06:22:33 -0500 Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.61.5.179](untrusted sender)) by attbi.com (rwcrmhc13) with SMTP id <2003072911222601500o7l9ke>; Tue, 29 Jul 2003 11:22:26 +0000 Message-ID: <3F2659EF.1050909@wi.mit.edu> Date: Tue, 29 Jul 2003 07:26:39 -0400 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Sean A McKay CC: metakit@equi4.com Subject: Re: [Metakit] Another MkSQL ? References: <20030728.133457.-3680709.1.seanamckay@juno.com> In-Reply-To: <20030728.133457.-3680709.1.seanamckay@juno.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: You need to use the "named" style of parameter passing as in: cur.execute( "select * from frequents where drinker>:name1 or drinker<:name2", {'name1':'norm', 'name2':'b'} ) ref: http://www.mcmillan-inc.com/mksqluser.html From gmcm@hypernet.com Tue Jul 29 07:15:15 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6TCFFL20711 for ; Tue, 29 Jul 2003 07:15:15 -0500 Received: from PARANOIA (204.176.40.50) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Tue, 29 Jul 2003 08:17:22 -0400 From: "Gordon McMillan" To: Sean A McKay , metakit@equi4.com Date: Tue, 29 Jul 2003 08:14:00 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Another MkSQL ? Reply-to: gmcm@hypernet.com Message-ID: <3F262CC8.6558.19C10745@localhost> Priority: normal In-reply-to: <20030728.133457.-3680709.1.seanamckay@juno.com> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 28 Jul 2003 at 13:34, Sean A McKay wrote: > Hello, > > I'm encountering problems w/ the update statement. Perhaps I'm doing > something stupid or 'slightly' wrong, and if so, please let me know! [...] > >>> cur.execute("update person set name='joe' where name='sean'") > Traceback (most recent call last): [...] > File "F:\PYTHON22\lib\site-packages\MkSQL\SQL.py", line 451, in f > bv.update(row.index, setter(bv[row.index], params)) > File "F:\PYTHON22\lib\site-packages\MkSQL\RDB.py", line 165, in update > kw = self._makekw(args) > File "F:\PYTHON22\lib\site-packages\MkSQL\RDB.py", line 192, in > _makekw > kw[prop.name] = args[i] > IndexError: tuple index out of range I don't think you're doing anything even slightly wrong. My guess is that I broke that in b1 - I unfortunately don't have any RDB test cases. I'll try to look at it more closely in the next few days, but from a quick glance, it sure resembles a brain fart :-(. -- Gordon http://www.mcmillan-inc.com/ From david@rebirthing.co.nz Wed Jul 30 20:00:04 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6V102L30991 for ; Wed, 30 Jul 2003 20:00:04 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19i1nO-0001Qb-00 for ; Thu, 31 Jul 2003 12:59:55 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1059613129.1192.3.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 31 Jul 2003 12:58:50 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Problem - sort() producing read-only rows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I'm having a bit of trouble. It seems that view methods like select() produce normal PyRowRef rows, but sort() produces PyRORowRef rows (readonly). Is there any reason why this should be the case, especially when a sort()ed view is isomorphic to the original one? Is there any possibility of metakit code being changed so this is no longer the case? Failing this, what's the most efficient way to set an attribute of a row in a sort()ed view, given only an index into that *sort()ed* view? -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From bkelley@wi.mit.edu Wed Jul 30 20:16:54 2003 Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6V1GsL31866 for ; Wed, 30 Jul 2003 20:16:54 -0500 Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.61.5.179](untrusted sender)) by attbi.com (rwcrmhc13) with SMTP id <2003073101164801500o7g2ie>; Thu, 31 Jul 2003 01:16:48 +0000 Message-ID: <3F286F04.9030008@wi.mit.edu> Date: Wed, 30 Jul 2003 21:21:08 -0400 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: David McNab CC: metakit@equi4.com Subject: Re: [Metakit] Problem - sort() producing read-only rows References: <1059613129.1192.3.camel@rebirth> In-Reply-To: <1059613129.1192.3.camel@rebirth> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: David McNab wrote: >Hi, > >I'm having a bit of trouble. > >It seems that view methods like select() produce normal PyRowRef rows, >but sort() produces PyRORowRef rows (readonly). > I have noticed this as well. You can sort rows in KitViewer but sometimes the sorted produces a read only view. Sometimes it doesn't though. >Failing this, what's the most efficient way to set an attribute of a row >in a sort()ed view, given only an index into that *sort()ed* view? > > I would like to know this as well. From david@rebirthing.co.nz Wed Jul 30 20:38:57 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6V1cuL00669 for ; Wed, 30 Jul 2003 20:38:57 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19i2P3-0001bK-00 for ; Thu, 31 Jul 2003 13:38:49 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1059615464.1192.8.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 31 Jul 2003 13:37:44 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] unique() messing up order Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, With a view, I'm doing select(), sort(), project() and unique(), in that order. After the project(), the order stays intact. But after the unique(), the order gets messed up. Could this be a bug? -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From david@rebirthing.co.nz Wed Jul 30 20:50:13 2003 Received: from freenet.org.nz (adsl75-31.world-net.co.nz [210.55.75.31]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6V1oBL01175 for ; Wed, 30 Jul 2003 20:50:12 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19i2Zq-0001dj-00; Thu, 31 Jul 2003 13:49:58 +1200 Subject: Re: [Metakit] Problem - sort() producing read-only rows From: David McNab To: Brian Kelley Cc: metakit@equi4.com In-Reply-To: <3F286F04.9030008@wi.mit.edu> References: <1059613129.1192.3.camel@rebirth> <3F286F04.9030008@wi.mit.edu> Content-Type: text/plain Message-Id: <1059616133.1192.14.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 31 Jul 2003 13:48:53 +1200 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Thu, 2003-07-31 at 13:21, Brian Kelley wrote: > >Failing this, what's the most efficient way to set an attribute of a row > >in a sort()ed view, given only an index into that *sort()ed* view? > > > I would like to know this as well. I don't think this is the most efficient way, but needs must: def setColOfRow(view, row, colName, value): v = view.select(col1=row.col1, col2=row.col2, ..., coln=row.coln) for r in v: setattr(r, colName, value) Maybe in my metakitplus module I should add a view method called 'matchRow()'. -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From gmcm@hypernet.com Thu Jul 31 07:25:36 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6VCPaL31973 for ; Thu, 31 Jul 2003 07:25:36 -0500 Received: from PARANOIA (204.176.40.69) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Thu, 31 Jul 2003 08:27:46 -0400 From: "Gordon McMillan" To: David McNab , metakit@equi4.com Date: Thu, 31 Jul 2003 08:24:22 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Problem - sort() producing read-only rows Reply-to: gmcm@hypernet.com Message-ID: <3F28D236.20270.24173EF1@localhost> Priority: normal In-reply-to: <1059613129.1192.3.camel@rebirth> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 31 Jul 2003 at 12:58, David McNab wrote: > I'm having a bit of trouble. > > It seems that view methods like select() produce normal > PyRowRef rows, but sort() produces PyRORowRef rows (readonly). Not necessarily. >>> v = db.getas('v[a:I,b:S]') >>> v.append(a=1,b='B') 0 >>> v.append(a=3,b='B') 1 >>> w = v.sort() >>> w[0] >>> w = v.select(b='B') # Note: back to v >>> w[0] >>> w = w.sort() >>> w[0] >>> You can select all you want, then sort (once). That's the limit, though. > Is there any reason why this should be the case, especially when a > sort()ed view is isomorphic to the original one? > > Is there any possibility of metakit code being changed so this is no > longer the case? The underlying C++ is somewhat, um, schizo :-). select & sort produce "notifiable" views (they are notified of changes to the base view). That approach doesn't scale real well, and was dropped in favor of viewers (r/o) and mapping views (modifiable). Mk4py makes select()ed and sort()ed views look (as much as it can) like mapping views (modifiable). It's not a direct capability of the C++ lib. I almost pulled out that code. Why? Because I *never* use it anymore. I still use select & sort, but only for reporting. If I'm doing modifications that depend on complex criteria, I use map, filter, remapwith, remove etc. I use ordered views extensively. If 2 different orderings are commonly required, I might even maintain an external index, (sort is rather expensive). -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Thu Jul 31 07:25:37 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h6VCPaL31977 for ; Thu, 31 Jul 2003 07:25:37 -0500 Received: from PARANOIA (204.176.40.69) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Thu, 31 Jul 2003 08:27:49 -0400 From: "Gordon McMillan" To: David McNab , metakit@equi4.com Date: Thu, 31 Jul 2003 08:24:22 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] unique() messing up order Reply-to: gmcm@hypernet.com Message-ID: <3F28D236.19799.24173E83@localhost> Priority: normal In-reply-to: <1059615464.1192.8.camel@rebirth> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 31 Jul 2003 at 13:37, David McNab wrote: > With a view, I'm doing select(), sort(), project() and > unique(), in that order. > > After the project(), the order stays intact. > But after the unique(), the order gets messed up. > > Could this be a bug? Nope. unique, minus and different will re-order the view(s). -- Gordon http://www.mcmillan-inc.com/ From garth@deadlybloodyserious.com Thu Jul 31 20:09:15 2003 Received: from thebe.your-site.com (thebe.your-site.com [140.186.45.26]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h7119FL13338 for ; Thu, 31 Jul 2003 20:09:15 -0500 Received: from gkiddxp2 (networkappliance-link.syd.comindico.com.au [203.220.64.134]) by thebe.your-site.com (Postfix) with ESMTP id 14266245DEB for ; Thu, 31 Jul 2003 21:09:20 -0400 (EDT) From: "Garth Kidd" To: Date: Fri, 1 Aug 2003 11:09:06 +1000 Organization: Deadly Bloody Serious Message-ID: <004201c357c9$82df4860$3e00800a@gkiddxp2> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Importance: Normal Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h7119FL13338 Subject: [Metakit] Building metakit with MSVC6 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I'm having trouble making this work. Can anyone help? D:\metakit-2.4.9.2\win\msvc60>NMAKE /f "mkpython.mak" Microsoft (R) Program Maintenance Utility Version 6.00.8168.0 Copyright (C) Microsoft Corp 1988-1998. All rights reserved. No configuration specified. Defaulting to mkpython - Win32 Debug. cl.exe /nologo /MDd /W3 /Gm /GX /Zi /Od /I "..\..\include" /I "..\..\python\scxx" /I "d:\python23\include" /D "_DEBUG" /D "W IN32" /D "_WINDOWS" /D "_MBCS" /D "_USRDLL" /D "MKPYTHON_EXPORTS" /FR".\..\..\builds\msvc60\mkpython\Debug\\" /Fo".\..\..\builds\msv c60\mkpython\Debug\\" /Fd".\..\..\builds\msvc60\mkpython\Debug\\" /FD /GZ /c ..\..\python\PyRowRef.cpp PyRowRef.cpp ..\..\python\scxx\PWONumber.h(19) : error C2629: unexpected 'class PWONumber (' ..\..\python\scxx\PWONumber.h(19) : error C2334: unexpected token(s) preceding ':'; skipping apparent function body ..\..\python\scxx\PWONumber.h(118) : error C2833: 'operator LONG_LONG' is not a recognized operator or type ..\..\python\scxx\PWONumber.h(118) : error C2059: syntax error : 'newline' ..\..\python\scxx\PWONumber.h(118) : error C2334: unexpected token(s) preceding '{'; skipping apparent function body ..\..\python\PyRowRef.cpp(147) : error C2440: 'type cast' : cannot convert from 'class PWONumber' to '__int64' Ambiguous user-defined-conversion NMAKE : fatal error U1077: 'cl.exe' : return code '0x2' Stop. Regards, Garth. From jyl@mod3.net Thu Jul 31 20:16:22 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h711GML13736 for ; Thu, 31 Jul 2003 20:16:22 -0500 Received: (qmail 28310 invoked by uid 48); 1 Aug 2003 01:13:18 -0000 Received: from 192.18.42.10 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 31 Jul 2003 18:13:18 -0700 (PDT) Message-ID: <59102.192.18.42.10.1059700398.squirrel@mod3.net> Date: Thu, 31 Jul 2003 18:13:18 -0700 (PDT) Subject: Re: [Metakit] Building metakit with MSVC6 From: "Jacob Levy" To: In-Reply-To: <004201c357c9$82df4860$3e00800a@gkiddxp2> References: <004201c357c9$82df4860$3e00800a@gkiddxp2> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Garth I dont use the command line approach. I successfully build MK 2.4.9.2 with VC6 all the time using the project file, in /win/msvc60/mksrc.dsw. To build the DLL choose the "mkdll Win32 Release" configuration, do a build all and you're set to go. --JYL > I'm having trouble making this work. Can anyone help? > > D:\metakit-2.4.9.2\win\msvc60>NMAKE /f "mkpython.mak" > > Microsoft (R) Program Maintenance Utility Version 6.00.8168.0 > Copyright (C) Microsoft Corp 1988-1998. All rights reserved. > > No configuration specified. Defaulting to mkpython - Win32 Debug. > cl.exe /nologo /MDd /W3 /Gm /GX /Zi /Od /I "..\..\include" /I > "..\..\python\scxx" /I "d:\python23\include" /D "_DEBUG" /D "W > IN32" /D "_WINDOWS" /D "_MBCS" /D "_USRDLL" /D "MKPYTHON_EXPORTS" > /FR".\..\..\builds\msvc60\mkpython\Debug\\" /Fo".\..\..\builds\msv > c60\mkpython\Debug\\" /Fd".\..\..\builds\msvc60\mkpython\Debug\\" /FD > /GZ /c ..\..\python\PyRowRef.cpp > PyRowRef.cpp > ..\..\python\scxx\PWONumber.h(19) : error C2629: unexpected 'class > PWONumber (' > ..\..\python\scxx\PWONumber.h(19) : error C2334: unexpected token(s) > preceding ':'; skipping apparent function body > ..\..\python\scxx\PWONumber.h(118) : error C2833: 'operator LONG_LONG' > is not a recognized operator or type > ..\..\python\scxx\PWONumber.h(118) : error C2059: syntax error : > 'newline' ..\..\python\scxx\PWONumber.h(118) : error C2334: unexpected > token(s) preceding '{'; skipping apparent function body > ..\..\python\PyRowRef.cpp(147) : error C2440: 'type cast' : cannot > convert from 'class PWONumber' to '__int64' > Ambiguous user-defined-conversion > NMAKE : fatal error U1077: 'cl.exe' : return code '0x2' > Stop. > > > > Regards, > Garth. > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From garth@deadlybloodyserious.com Thu Jul 31 22:40:44 2003 Received: from thebe.your-site.com (thebe.your-site.com [140.186.45.26]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h713ehL18544 for ; Thu, 31 Jul 2003 22:40:43 -0500 Received: from gkiddxp2 (networkappliance-link.syd.comindico.com.au [203.220.64.134]) by thebe.your-site.com (Postfix) with ESMTP id 5FA11244C14; Thu, 31 Jul 2003 23:40:48 -0400 (EDT) From: "Garth Kidd" To: Cc: Subject: RE: [Metakit] Building metakit with MSVC6 Date: Fri, 1 Aug 2003 13:40:34 +1000 Organization: Deadly Bloody Serious Message-ID: <006401c357de$ac166190$3e00800a@gkiddxp2> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 In-Reply-To: <59102.192.18.42.10.1059700398.squirrel@mod3.net> Importance: Normal Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h713ehL18544 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > I dont use the command line approach. I successfully build MK > 2.4.9.2 with VC6 all the time using the project file [...] I can do that, too, but I'm trying to rig a distutils bundle for Metakit, which means getting it building outside of Visual Studio itself. I've got everything compiling, now, but can't get the PYD to link. I'm not a C++ programmer, so have no idea what any of the following means. Any ideas? This is with either ``libraries=['mk4vc60s']`` or ``libraries=['mk4vc60']``. Creating library build\temp.win32-2.2\Release\Mk4py.lib and object build\temp .win32-2.2\Release\Mk4py.exp PyProperty.obj : error LNK2001: unresolved external symbol "protected: void __thiscall PWOBase::GrabRef(struct _object *)" (?GrabRef@PWOBase@@IAEXPAU_object@@@Z) PyRowRef.obj : error LNK2001: unresolved external symbol "protected: void __thiscall PWOBase::GrabRef(struct _object *)" (?GrabRef@PWOBase@@IAEXPAU_object@@@Z) PyStorage.obj : error LNK2001: unresolved external symbol "protected: void __thiscall PWOBase::GrabRef(struct _object *)" (?GrabRef@PWOBase@@IAEXPAU_object@@@Z) PyProperty.obj : error LNK2001: unresolved external symbol "void __cdecl Fail(struct _object *,char const *)" (?Fail@@YAXPAU_object@@PBD@Z) PyRowRef.obj : error LNK2001: unresolved external symbol "void __cdecl Fail(struct _object *,char const *)" (?Fail@@YAXPAU_object@@PBD@Z) PyStorage.obj : error LNK2001: unresolved external symbol "void __cdecl Fail(struct _object *,char const *)" (?Fail@@YAXPAU_object@@PBD@Z) PyRowRef.obj : error LNK2001: unresolved external symbol "public: __thiscall PyView::PyView(class c4_View const &,class PyView *,int)" (??0PyView@@QAE@ABVc4_View@@PAV0@H@Z) PyStorage.obj : error LNK2001: unresolved external symbol "public: __thiscall PyView::PyView(class c4_View const &,class PyView *,int)" (??0PyView@@QAE@ABVc4_View@@PAV0@H@Z) PyRowRef.obj : error LNK2001: unresolved external symbol "class PWDException const & const PWDPyException" (?PWDPyException@@3ABVPWDException@@B) PyRowRef.obj : error LNK2001: unresolved external symbol "public: int __thiscall PyView::setItem(int,struct _object *)" (?setItem@PyView@@QAEHHPAU_object@@@Z) PyRowRef.obj : error LNK2001: unresolved external symbol "struct _typeobject PyViewtype" (?PyViewtype@@3U_typeobject@@A) PyStorage.obj : error LNK2001: unresolved external symbol "struct _typeobject PyViewtype" (?PyViewtype@@3U_typeobject@@A) PyStorage.obj : error LNK2001: unresolved external symbol "struct _object * __cdecl PyView_new(struct _object *,struct _object *)" (?PyView_new@@YAPAU_object@@PAU1@0@Z) PyStorage.obj : error LNK2001: unresolved external symbol "struct _typeobject PyROViewertype" (?PyROViewertype@@3U_typeobject@@A) PyStorage.obj : error LNK2001: unresolved external symbol "struct _typeobject PyViewertype" (?PyViewertype@@3U_typeobject@@A) build\lib.win32-2.2\Mk4py.pyd : fatal error LNK1120: 9 unresolved externals error: command '"D:\Program Files\Microsoft Visual Studio\VC98\BIN\link.exe"' failed with exit status 1120 Regards, Garth. From garth@deadlybloodyserious.com Thu Jul 31 23:16:29 2003 Received: from thebe.your-site.com (thebe.your-site.com [140.186.45.26]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h714GTL19376 for ; Thu, 31 Jul 2003 23:16:29 -0500 Received: from gkiddxp2 (networkappliance-link.syd.comindico.com.au [203.220.64.134]) by thebe.your-site.com (Postfix) with ESMTP id 0DD5D244AC4; Fri, 1 Aug 2003 00:16:33 -0400 (EDT) From: "Garth Kidd" To: Cc: Subject: RE: [Metakit] Building metakit with MSVC6 Date: Fri, 1 Aug 2003 14:16:18 +1000 Organization: Deadly Bloody Serious Message-ID: <007801c357e3$aae9fd40$3e00800a@gkiddxp2> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 In-Reply-To: Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > I've got everything compiling, now, but can't get the PYD to > link. Belay that. I added PyView.cpp and scxx\PWOImp.cpp, and it worked -- at least, with Python 2.2. Here's setup.py, intended to be used from metakit-2.4.9.2/python, and tested only to the point of building cleanly: # ---------------------------------------------------------------------- from distutils.core import setup, Extension import os, os.path srcFiles = [] srcDir = os.path.join('..', 'src') for d in ['.', 'scxx', srcDir ]: srcFiles.extend([os.path.join(d, f) for f in os.listdir(d) if f.endswith('.cpp')]) setup(name="metakit", version="2.4.9.2", description="metakit database", author="Equi4 Software", author_email="jcw@equi4.com", maintainer="Garth T Kidd", maintainer_email="garth@deadlybloodyserious.com", url="http://www.equi4.com/metakit/", py_modules=['metakit'], ext_modules=[ Extension( "Mk4py", srcFiles, include_dirs=[ "scxx", srcDir, os.sep.join(['..','include']), os.sep.join(['..','win']) ], ), ] ) # ---------------------------------------------------------------------- With Python 2.3, I run into this: std.cpp D:\Program Files\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -UNDEBUG -UWIN32 -U_WINDOWS -U_MBCS -U_ USRDLL -UMKPYTHON_EXPORTS -Iscxx -I..\src -I..\include -I..\win -Id:\Python23\include -Id:\Python23\PC /TpPyRowRef.cpp /Fobuild\temp .win32-2.3\Release\PyRowRef.obj PyRowRef.cpp scxx\PWONumber.h(19) : error C2629: unexpected 'class PWONumber (' scxx\PWONumber.h(19) : error C2334: unexpected token(s) preceding ':'; skipping apparent function body scxx\PWONumber.h(118) : error C2833: 'operator LONG_LONG' is not a recognized operator or type scxx\PWONumber.h(118) : error C2059: syntax error : 'newline' scxx\PWONumber.h(118) : error C2334: unexpected token(s) preceding '{'; skipping apparent function body PyRowRef.cpp(147) : error C2440: 'type cast' : cannot convert from 'class PWONumber' to '__int64' Ambiguous user-defined-conversion error: command '"D:\Program Files\Microsoft Visual Studio\VC98\BIN\cl.exe"' failed with exit status 2 Any ideas? Regards, Garth. From garth@deadlybloodyserious.com Thu Jul 31 23:30:19 2003 Received: from thebe.your-site.com (thebe.your-site.com [140.186.45.26]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h714UJL19699 for ; Thu, 31 Jul 2003 23:30:19 -0500 Received: from gkiddxp2 (networkappliance-link.syd.comindico.com.au [203.220.64.134]) by thebe.your-site.com (Postfix) with ESMTP id D6A3E245F77 for ; Fri, 1 Aug 2003 00:30:23 -0400 (EDT) From: "Garth Kidd" To: Subject: RE: [Metakit] Building metakit with MSVC6 Date: Fri, 1 Aug 2003 14:30:10 +1000 Organization: Deadly Bloody Serious Message-ID: <007c01c357e5$99b96fe0$3e00800a@gkiddxp2> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 In-Reply-To: Importance: Normal Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h714UJL19699 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > tested only to the point of building cleanly metakit-2.4.9.2.win32-py2.2.exe turns out to be 112K, including metakit.py and Mk4py.pyd with Metakit compiled in with the Python extensions, as I can't find a way to have distutils package up a DLL. Yell if you'd like a copy for the download site -- it's about as easy as Metakit installation can get for Windows users. Regards, Garth. From david@rebirthing.co.nz Sat Aug 2 05:16:23 2003 Received: from freenet.org.nz (adsl75-112.world-net.co.nz [210.55.75.112]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h72AGLL31686 for ; Sat, 2 Aug 2003 05:16:22 -0500 Received: from palm ([192.168.2.2]) by freenet.org.nz with esmtp (Exim 3.36 #1 (Debian)) id 19itQr-0007eg-00 for ; Sat, 02 Aug 2003 22:16:13 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1059819304.1295.9.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 02 Aug 2003 22:15:05 +1200 Content-Transfer-Encoding: 7bit Subject: [Metakit] Question about large databases Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I've got a concern about what might happen if a database file starts getting *really* big - like say 20MB or more. When I first create the storage object: db = metakit.storage("my-big-db.mk", 1) Does the entire contents of the database get loaded into memory, or is database content pulled in as needed? If the whole database doesn't get loaded into memory lock, stock and barrel, what about views. When I load a view: vw = db.view("bigBloatedTable") do all the view's rows get loaded into memory? Or is this data just loaded as needed? Thanking you all muchly once again for your kind help, Cheers David From jyl@mod3.net Sat Aug 2 23:42:08 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h734g7L12039 for ; Sat, 2 Aug 2003 23:42:07 -0500 Received: (qmail 7275 invoked from network); 3 Aug 2003 04:38:54 -0000 Received: from adsl-209-233-24-218.dsl.snfc21.pacbell.net (HELO mod3.net) (jyl@209.233.24.218) by mod3.net with SMTP; 3 Aug 2003 04:38:54 -0000 Message-ID: <3F2C92AD.8090800@mod3.net> Date: Sat, 02 Aug 2003 21:42:21 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: David McNab CC: metakit@equi4.com Subject: Re: [Metakit] Question about large databases References: <1059819304.1295.9.camel@rebirth> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On operating systems on which Metakit finds enough support for using memory mappings, it will load the entire database into a memory mapped region of memory, and then the OS takes care of reading/writing the data from/to the actual storage as needed, based on memory accesses. This does mean that for those operating systems, the largest possible storage size is limited by the available virtual memory. --JYL David McNab wrote: >Hi, > >I've got a concern about what might happen if a database file starts >getting *really* big - like say 20MB or more. > >When I first create the storage object: > > db = metakit.storage("my-big-db.mk", 1) > >Does the entire contents of the database get loaded into memory, or is >database content pulled in as needed? > >If the whole database doesn't get loaded into memory lock, stock and >barrel, what about views. When I load a view: > > vw = db.view("bigBloatedTable") > >do all the view's rows get loaded into memory? Or is this data just >loaded as needed? > >Thanking you all muchly once again for your kind help, > >Cheers >David > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > > > From jyl@mod3.net Mon Aug 4 00:40:53 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h745erL20394 for ; Mon, 4 Aug 2003 00:40:53 -0500 Received: (qmail 31745 invoked by uid 48); 4 Aug 2003 05:37:36 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Sun, 3 Aug 2003 22:37:36 -0700 (PDT) Message-ID: <64870.209.233.24.218.1059975456.squirrel@mod3.net> Date: Sun, 3 Aug 2003 22:37:36 -0700 (PDT) From: "Jacob Levy" To: X-Priority: 3 Importance: Normal Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] [ANNOUNCE] e4Graph 1.0a8 released Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I am pleased to announce the 1.0a8 release of e4Graph, the eight Alpha release. WHAT IS IT: e4Graph is a C++ library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. The e4Graph package also provides bindings in several other languages, currently Tcl, Python and Java, and allows input/output of object graphs via an XML representation. The e4Graph package is built on top of Metakit 2.4.9.2, and optionally uses Tcl 8.4.4, Python 2.2.3, Java 1.1 or later, and Expat 1.95.5. WHERE TO GET: Downloads: http://sourceforge.net/projects/e4graph/ Homepage: http://www.e4graph.com/e4graph/ Changelog: http://www.e4graph.com/e4graph/changes.txt Installation: http://www.e4graph.com/e4graph/e4install.html WHAT IS NEW: Support for MacOS X 10.2 and several other platforms was added. Support for Python 2.2 and later releases was added. The Java binding was updated. The event mechanism was completely overhauled and support for user defined events was added. The vertex caching mechanism was moved from the Tcl binding to the core e4Graph library to enable other language bindings to benefit from the increased performance afforded by the cache. The interface between e4Graph and data base drivers was formalized in preparation for supporting new data base drivers in addition to Metakit. Additional work was done to support 64-bit systems and very large storages; more work is needed. Support for SWIG was added so new language bindings can be created with SWIG, in preparation for incorporating Perl, PHP and Ruby bindings in the next release. Many bugs were fixed. ACKNOWLEDGMENTS: Michael Krimerman contributed the Python binding. J.P. Fletcher helped me get started with SWIG support. Joe Mistachkin helped with porting to the FreeBSD platform. Daniel Steffen and Nicholas Brawn were instrumental helping with MacOS X support. I wish to acknowledge support from the HP test-drive facility and from the SourceForge compile farm. Many people (too many to mention) contributed bug reports and bug fixes. Geoff Shapiro and David Van Maren suggested many useful API changes. From mbloore@yahoo.com Tue Aug 5 10:00:45 2003 Received: from web14306.mail.yahoo.com (web14306.mail.yahoo.com [216.136.173.82]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h75F0jL16961 for ; Tue, 5 Aug 2003 10:00:45 -0500 Message-ID: <20030805150044.93337.qmail@web14306.mail.yahoo.com> Received: from [65.95.142.237] by web14306.mail.yahoo.com via HTTP; Tue, 05 Aug 2003 08:00:44 PDT Date: Tue, 5 Aug 2003 08:00:44 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] Question about large databases To: metakit@equi4.com In-Reply-To: <3F2C92AD.8090800@mod3.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: actually, if metakit can't memory-map the database file (eg because it is too big), it will revert to normal i/o. this means that you can open a huge file, but access will be slower. --- Jacob Levy wrote: > On operating systems on which Metakit finds enough support for using > memory mappings, it will load the entire database into a memory > mapped > region of memory, and then the OS takes care of reading/writing the > data > from/to the actual storage as needed, based on memory accesses. This > does mean that for those operating systems, the largest possible > storage > size is limited by the available virtual memory. ===== -- mARK bLOORE __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com From bkelley@wi.mit.edu Tue Aug 5 10:41:34 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h75FfYL20305 for ; Tue, 5 Aug 2003 10:41:34 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003080511413019025 for ; Tue, 05 Aug 2003 11:41:30 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HJ5KX600.LZP for ; Tue, 5 Aug 2003 11:41:30 -0400 Message-ID: <3F2FD07B.6090100@wi.mit.edu> Date: Tue, 05 Aug 2003 11:42:51 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Question about large databases References: <20030805150044.93337.qmail@web14306.mail.yahoo.com> In-Reply-To: <20030805150044.93337.qmail@web14306.mail.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: >actually, if metakit can't memory-map the database file (eg because it >is too big), it will revert to normal i/o. this means that you can >open a huge file, but access will be slower. > This has been my experience as well. I'm actually using databases of >500MB in size with very reasonable performance. Just make sure that your joins are 1 to 1 :) -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From tom.krehbiel@motorola.com Tue Aug 5 16:28:51 2003 Received: from motgate.mot.com (motgate.mot.com [129.188.136.100]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h75LSpL16105 for ; Tue, 5 Aug 2003 16:28:51 -0500 Received: from az33exr01.mot.com (az33exr01.mot.com [10.64.251.231]) by motgate.mot.com (Motorola/Motgate) with ESMTP id h75LSoSD021100 for ; Tue, 5 Aug 2003 14:28:50 -0700 (MST) Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exr01.mot.com (Motorola/az33exr01) with ESMTP id h75LSgrx003449 for ; Tue, 5 Aug 2003 16:28:42 -0500 Message-ID: <3F30218C.6090602@motorola.com> Date: Tue, 05 Aug 2003 14:28:44 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DMO User-Agent: Mozilla/5.0 (X11; U; HP-UX 9000/785; en-US; rv:1.2) Gecko/20021219 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] oomk Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I'm working with oomk but am having problems understanding how to work with a view. The documentation explains how to create a view but I can't figure out how to delete or modify a view. Does anyone know how to perform these operations in oomk? Tom K. From mbloore@yahoo.com Tue Aug 19 11:45:19 2003 Received: from web14301.mail.yahoo.com (web14301.mail.yahoo.com [216.136.173.77]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7JGjJD23215 for ; Tue, 19 Aug 2003 11:45:19 -0500 Message-ID: <20030819164518.56957.qmail@web14301.mail.yahoo.com> Received: from [65.95.141.100] by web14301.mail.yahoo.com via HTTP; Tue, 19 Aug 2003 09:45:18 PDT Date: Tue, 19 Aug 2003 09:45:18 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] Question about large databases To: metakit@equi4.com In-Reply-To: <20030805150044.93337.qmail@web14306.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --- mARK bLOORE wrote: > actually, if metakit can't memory-map the database file (eg because > it is too big), it will revert to normal i/o. this means that you can > open a huge file, but access will be slower. one odd wrinkle: the release version of metakit catches the error when it fails to map a file, and switches to normal i/o. the debug version reports the error and dies! ===== -- mARK bLOORE __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com From Michael_Scharf@gmx.de Sun Aug 24 22:18:08 2003 Received: from mail.gmx.net (pop.gmx.net [213.165.64.20]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7P3I8D09424 for ; Sun, 24 Aug 2003 22:18:08 -0500 Received: (qmail 4309 invoked by uid 65534); 25 Aug 2003 03:18:02 -0000 Received: from p50843F2D.dip.t-dialin.net (EHLO gmx.de) (80.132.63.45) by mail.gmx.net (mp021) with SMTP; 25 Aug 2003 05:18:02 +0200 Message-ID: <3F49801F.7000100@gmx.de> Date: Mon, 25 Aug 2003 05:18:55 +0200 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030529 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] [ANNOUNCE] e4Graph 1.0a8 released References: <64870.209.233.24.218.1059975456.squirrel@mod3.net> In-Reply-To: <64870.209.233.24.218.1059975456.squirrel@mod3.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi e4Graph seems pretty cool, but I could not find a mailing list or a forum where e4Graph is discussed.... Is e4Graph dead or am I missing something essential here.... Michael From jyl@mod3.net Sun Aug 24 22:48:15 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7P3mFD10781 for ; Sun, 24 Aug 2003 22:48:15 -0500 Received: (qmail 25049 invoked from network); 25 Aug 2003 03:43:49 -0000 Received: from adsl-209-233-24-218.dsl.snfc21.pacbell.net (HELO mod3.net) (jyl@209.233.24.218) by mod3.net with SMTP; 25 Aug 2003 03:43:49 -0000 Message-ID: <3F498719.90702@mod3.net> Date: Sun, 24 Aug 2003 20:48:41 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Michael Scharf CC: metakit@equi4.com Subject: Re: [Metakit] [ANNOUNCE] e4Graph 1.0a8 released References: <64870.209.233.24.218.1059975456.squirrel@mod3.net> <3F49801F.7000100@gmx.de> In-Reply-To: <3F49801F.7000100@gmx.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Michael There are several low volume email lists for e4graph described here: http://sourceforge.net/mail/?group_id=9803 I dont think e4graph is dead :) Look at the downloads and see there's been >3700 downloads so far, and there's three commercial products that use e4graph for their persistent storage needs, ranging from bio-informatics to personal PIM to a trading desk application. --JYL Michael Scharf wrote: > Hi > > e4Graph seems pretty cool, but I could not find a mailing list or > a forum where e4Graph is discussed.... > > Is e4Graph dead or am I missing something essential here.... > > > Michael > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From Michael_Scharf@gmx.de Mon Aug 25 08:25:07 2003 Received: from mail.gmx.net (pop.gmx.net [213.165.64.20]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7PDP6D06674 for ; Mon, 25 Aug 2003 08:25:07 -0500 Received: (qmail 16524 invoked by uid 65534); 25 Aug 2003 13:25:01 -0000 Received: from pD9E4B345.dip.t-dialin.net (EHLO gmx.de) (217.228.179.69) by mail.gmx.net (mp006) with SMTP; 25 Aug 2003 15:25:01 +0200 Message-ID: <3F4A0E66.1050100@gmx.de> Date: Mon, 25 Aug 2003 15:25:58 +0200 From: Michael Scharf User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030529 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] [ANNOUNCE] e4Graph 1.0a8 released References: <64870.209.233.24.218.1059975456.squirrel@mod3.net> <3F49801F.7000100@gmx.de> <3F498719.90702@mod3.net> In-Reply-To: <3F498719.90702@mod3.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > Michael > > There are several low volume email lists for e4graph described here: > > http://sourceforge.net/mail/?group_id=9803 That's what I found too. But that's *very* low volume. e.g. e4Graph-core 35 messages, the last one is 2.5 years old.... I am interested in e4Graph-java, but this list seems not to exist.... Michael From erikh2000@yahoo.com Mon Aug 25 12:33:37 2003 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7PHXbD23713 for ; Mon, 25 Aug 2003 12:33:37 -0500 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 25 Aug 2003 17:33:35 -0000 Message-Id: <5.2.0.9.0.20030825102618.00a68d98@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 25 Aug 2003 10:37:09 -0700 To: metakit@equi4.com From: Erik Hermansen Cc: mrimer@hotmail.com Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Store files in database? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I have an interest in storing media files of sizes ranging from 10k to 1mb in a Metakit database as "byte"-type field values. An alternative would be to store this data in files and keep an index to the files in the database with filepaths as "foreign keys". The first approach is a little simpler to write, and I like keeping my files neatly tucked away. But will it cause performance problems? I noticed the other thread about database size, and based on that I don't believe the total size of the database file will be an issue. To simplify the problem, I plan to use just one database file with one viewdef in it to contain file data. So each record of the table would store one file. -Erik From erikh2000@yahoo.com Mon Aug 25 12:43:17 2003 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7PHhGD24397 for ; Mon, 25 Aug 2003 12:43:16 -0500 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 25 Aug 2003 17:43:15 -0000 Message-Id: <5.2.0.9.0.20030825103731.02e734f0@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 25 Aug 2003 10:47:03 -0700 To: metakit@equi4.com From: Erik Hermansen Cc: mrimer@hotmail.com Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Using Unicode filepaths to open storage. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I would like to open a storage file at a Unicode filepath. My imagined solution is to have an additional c4_Storage constructor that accepts a Unicode filepath. Of course, not every O/S uses Unicode, but perhaps on these platforms the constructor code would compile so that it simply converts the Unicode filepath parameter to ASCII. Another programmer on my project wrote a workaround using LoadFrom() with a stream to open the file instead of constructing with a filepath parameter. It worked, but the time to open the database was much too long, so I had to abandon this approach. Maybe there is an easy way to solve my problem that I've overlooked. -Erik From jcw@equi4.com Mon Aug 25 13:26:58 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h7PIQwD27296; Mon, 25 Aug 2003 13:26:58 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id 8771483CC9; Mon, 25 Aug 2003 20:26:52 +0200 (CEST) Date: Mon, 25 Aug 2003 20:26:52 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list , Starkit list Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.552) Subject: [Metakit] paypal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, In response to occasional requests I get from people who wish to make a donation to support my OSS work (metakit, tclkit, starkits, catfish), I have added a button tied to the PayPal money transfer system, see: http://www.equi4.com/metakit/license.html Please don't jump to conclusions. This is not an initiative motivated by desperation, but simply me being tired of having to jump through hoops when people and companies get in touch and sincerely want to support my work without being able to afford the commercial Metakit Enterprise License. -jcw From jyl@mod3.net Tue Aug 26 22:21:59 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7R3LxD15113 for ; Tue, 26 Aug 2003 22:21:59 -0500 Received: (qmail 23025 invoked by uid 48); 27 Aug 2003 03:17:26 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Tue, 26 Aug 2003 20:17:26 -0700 (PDT) Message-ID: <64544.209.233.24.218.1061954246.squirrel@mod3.net> Date: Tue, 26 Aug 2003 20:17:26 -0700 (PDT) From: "Jacob Levy" To: X-Priority: 3 Importance: Normal Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] Metakit and multithreaded access? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Is Metakit multi-thread safe? What is and what isn't appropriate to do with a Metakit storage -- can I share one storage among multiple threads for example? A related question: I have an extension of Metakit (e4graph; see http://e4graph.com/e4graph) that I want to make thread-safe. If Metakit is indeed thread-safe and does this in a portable way, are there any hooks I can use to make my extension similarly thread-safe? --JYL From Garth.Lancaster@mbf.com.au Wed Aug 27 17:09:32 2003 Received: from fs1-smtp.mbf.com.au ([61.88.251.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h7RM9UD22166 for ; Wed, 27 Aug 2003 17:09:31 -0500 Received: from MBF-INT1-MTA by fs1-smtp.mbf.com.au with Novell_GroupWise; Thu, 28 Aug 2003 08:02:02 +1000 Message-Id: X-Mailer: Novell GroupWise Internet Agent 6.0.2 Date: Thu, 28 Aug 2003 08:12:44 +1000 From: "Garth Lancaster" To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Guinevere: 2.0.9 ; Subject: [Metakit] Re: Metakit and multithreaded access? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I beleive the answer to Jacob's question is no, from a c++ point of view - I have heard mutterings about tcl/python wrappers being better. However, John Taylor from Shift-Right posted a message a while back - Ive dredged it up to put here hth, Garth >>>>>> If anyone one is interested. In the course of using Metakit in our products at work, we have created a Multi-threaded C++ wrapper for Metakit. It is a client/server model using ITC (inter-thread-communications) to isolate all database actions for a given a database to a single thread. The wrapper is part of my company's open source C++ library. The library can be downloaded at http://sourceforge.net/projects/openrepo. After downloading, the example project for the Metakit wrapper is: projects//metakit/examples/mt John T. Taylor Shift-Right Technologies, LLC. Tel: 678-344-3115 Fax: 770-736-9276 email: jtaylor@shift-right.com url: www.shift-right.com >>> metakit-request@equi4.com 28/08/03 3:00:02 >>> Send metakit mailing list submissions to metakit@equi4.com To subscribe or unsubscribe via the World Wide Web, visit http://www.equi4.com/mailman/listinfo/metakit or, via email, send a message with subject or body 'help' to metakit-request@equi4.com You can reach the person managing the list at metakit-admin@equi4.com When replying, please edit your Subject line so it is more specific than "Re: Contents of metakit digest..." Today's Topics: 1. Metakit and multithreaded access? (Jacob Levy) --__--__-- Message: 1 Date: Tue, 26 Aug 2003 20:17:26 -0700 (PDT) From: "Jacob Levy" To: Reply-To: jyl@mod3.net Subject: [Metakit] Metakit and multithreaded access? Is Metakit multi-thread safe? What is and what isn't appropriate to do with a Metakit storage -- can I share one storage among multiple threads for example? A related question: I have an extension of Metakit (e4graph; see http://e4graph.com/e4graph) that I want to make thread-safe. If Metakit is indeed thread-safe and does this in a portable way, are there any hooks I can use to make my extension similarly thread-safe? --JYL --__--__-- _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit End of metakit Digest This email contains confidential information intended only for the addressee. If you received it in error, please inform us by reply email or by calling us from Australia on 131 137 or Internationally on 612 93239500. Please also delete this email and destroy any hard copy. You must not disclose or use the information in this email. From jeff@k2.com Wed Aug 27 19:54:38 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc2.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h7S0sbD02634 for ; Wed, 27 Aug 2003 19:54:37 -0500 Received: from k2.com (IDENT:jkay@kobalt.k2.com [192.168.0.2]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h7S0r2220024 for ; Wed, 27 Aug 2003 20:53:02 -0400 Date: Wed, 27 Aug 2003 20:54:39 -0400 Subject: Re: [Metakit] Re: Metakit and multithreaded access? Content-Type: multipart/alternative; boundary=Apple-Mail-2--26982552 Mime-Version: 1.0 (Apple Message framework v552) From: Jeffrey Kay To: metakit@equi4.com In-Reply-To: Message-Id: <33FFB2DC-D8F2-11D7-93F3-000A95842048@k2.com> X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2--26982552 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed From a c++ POV, there is no multi-threading. I've solved the problem by creating a couple of "locking" classes to ensure that I don't accidentally access a store from multiple threads. The first class opens up a db file. I've designed my code so that I have only one class managing a db file, so that class is derived from shDBManager. The second class retrieves a reference to a given store and sets a critical section to ensure that only one thread is working on the store. To protect myself, I usually make shStore a stack variable. If the code snippet below isn't helpful as shown, I can provide a more detailed example. Cheers -- jeff class shDBManager { public: shDBManager(const char *db); virtual ~shDBManager(void); friend class shStore; private: c4_Storage *m_store; //!< Database store wxCriticalSection m_dbcritsec; //!< Critical section for data store access. }; //! Constructor //! @param db fully qualified pathname of the database shDBManager::shDBManager(const char *db) { m_store = new c4_Storage(db,true); } //! Destructor. shDBManager::~shDBManager(void) { m_store->Commit(); delete m_store; } /** Autolocks and unlocks a store for a DBManager */ class shStore { public: shStore(shDBManager *parent) { m_parent = parent; m_parent->m_dbcritsec.Enter(); } virtual ~shStore() { m_parent->m_dbcritsec.Leave(); } c4_Storage& getStore(void) { return *m_parent->m_store; } private: shDBManager *m_parent; }; On Wednesday, August 27, 2003, at 06:12 PM, Garth Lancaster wrote: > I beleive the answer to Jacob's question is no, from a c++ point of > view > - I have heard mutterings about tcl/python wrappers being better. jeffrey kay weblog pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein --Apple-Mail-2--26982552 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII From a c++ POV, there is no multi-threading. I've solved the problem by creating a couple of "locking" classes to ensure that I don't accidentally access a store from multiple threads. The first class opens up a db file. I've designed my code so that I have only one class managing a db file, so that class is derived from shDBManager. The second class retrieves a reference to a given store and sets a critical section to ensure that only one thread is working on the store. To protect myself, I usually make shStore a stack variable. If the code snippet below isn't helpful as shown, I can provide a more detailed example. Cheers -- jeff 7676,0F0F,5050class shDBManager { 7676,0F0F,5050public: shDBManager(7676,0F0F,5050const 7676,0F0F,5050char *db); 7676,0F0F,5050virtual ~shDBManager(7676,0F0F,5050void); 7676,0F0F,5050friend 7676,0F0F,5050class shStore; 7676,0F0F,5050private: c4_Storage *m_store; 2323,6E6E,2525//!<< Database store wxCriticalSection m_dbcritsec; 2323,6E6E,2525//!<< Critical section for data store access. }; 2323,6E6E,2525//! Constructor 2323,6E6E,2525//! @param db fully qualified pathname of the database shDBManager::shDBManager(7676,0F0F,5050const 7676,0F0F,5050char *db) { m_store = 7676,0F0F,5050new c4_Storage(db,7676,0F0F,5050true); } 2323,6E6E,2525//! Destructor. shDBManager::~shDBManager(7676,0F0F,5050void) { m_store->Commit(); 7676,0F0F,5050delete m_store; } 2323,6E6E,2525/** Autolocks and unlocks a store for a DBManager */ 7676,0F0F,5050class shStore { 7676,0F0F,5050public: shStore(shDBManager *parent) { m_parent = parent; m_parent->m_dbcritsec.Enter(); } 7676,0F0F,5050virtual ~shStore() { m_parent->m_dbcritsec.Leave(); } c4_Storage& getStore(7676,0F0F,5050void) { 7676,0F0F,5050return *m_parent->m_store; } 7676,0F0F,5050private: shDBManager *m_parent; }; On Wednesday, August 27, 2003, at 06:12 PM, Garth Lancaster wrote: I beleive the answer to Jacob's question is no, from a c++ point of view - I have heard mutterings about tcl/python wrappers being better. jeffrey kay weblog < pgp key < aim < share files with me -- get shinkuro -- < "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein --Apple-Mail-2--26982552-- From mbloore@yahoo.com Wed Aug 27 20:04:00 2003 Received: from web14304.mail.yahoo.com (web14304.mail.yahoo.com [216.136.173.80]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7S13xD03366 for ; Wed, 27 Aug 2003 20:04:00 -0500 Message-ID: <20030828010359.6190.qmail@web14304.mail.yahoo.com> Received: from [64.231.194.102] by web14304.mail.yahoo.com via HTTP; Wed, 27 Aug 2003 18:03:59 PDT Date: Wed, 27 Aug 2003 18:03:59 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] Metakit and multithreaded access? To: metakit@equi4.com In-Reply-To: <64544.209.233.24.218.1061954246.squirrel@mod3.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: i use metakit in a server, which handles user requests in threads. my database use is pretty simple: most requests open the db file readonly. i have locking code to prevent two threads from opening the file for write at the same time. note that while many storage objects from different threads may open the same file (readonly), you can't share storage objects between threads. --- Jacob Levy wrote: > Is Metakit multi-thread safe? What is and what isn't appropriate to > do > with a Metakit storage -- can I share one storage among multiple > threads > for example? > > A related question: I have an extension of Metakit (e4graph; see > http://e4graph.com/e4graph) that I want to make thread-safe. If > Metakit is > indeed thread-safe and does this in a portable way, are there any > hooks I > can use to make my extension similarly thread-safe? > > --JYL > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit ===== -- mARK bLOORE __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com From jcw@equi4.com Thu Aug 28 12:42:15 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h7SHgED28705 for ; Thu, 28 Aug 2003 12:42:14 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id E242383CC9 for ; Thu, 28 Aug 2003 19:42:08 +0200 (CEST) Date: Thu, 28 Aug 2003 19:42:08 +0200 Subject: Re: [Metakit] Metakit and multithreaded access? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20030828010359.6190.qmail@web14304.mail.yahoo.com> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: > i use metakit in a server, which handles user requests in threads. my > database use is pretty simple: most requests open the db file > readonly. i have locking code to prevent two threads from opening the > file for write at the same time. > > note that while many storage objects from different threads may open > the same file (readonly), you can't share storage objects between > threads. > > --- Jacob Levy wrote: >> Is Metakit multi-thread safe? What is and what isn't appropriate to >> do with a Metakit storage -- can I share one storage among multiple >> threads for example? >> >> A related question: I have an extension of Metakit (e4graph; see >> http://e4graph.com/e4graph) that I want to make thread-safe. If >> Metakit is indeed thread-safe and does this in a portable way, are >> there any hooks I can use to make my extension similarly thread-safe? >> >> --JYL The core C++ library is thread safe, in a very specific (limited) way: different dataffiles in each thread. Don't share across threads, not even for reading (due to cache changes). The only thing guarded in MK is a global symbol table of properties. Code to mutex-protect is included for Win32 and for unix'es using the pthread library. In Python, MK's protection rides on Python, i.e. it is safe because of the global mutex lock around all of Python. So there, AFAIK, multi-threaded use is ok (since only on thread will be running in Python at any time). In Tcl, there is a global mutex to protect all calls into the Mk4tcl extension, so there too the library can be used across threads/interpreters (there's a "-shared" flag on open to make a db visible in all interps). As Garth Lancster pointed out, there is an MT wrapper for MK at the C++ level by John Taylor < jtaylor@shift-right.com> which ought to provide full thread-safety in all scenario's - let me reproduce his info again here: > The library can be downloaded at > http://sourceforge.net/projects/openrepo. After downloading, the > example project for the Metakit wrapper is: > projects//metakit/examples/mt (I haven't used this myself) Mark's approach above is a special trick to get around using the same datafile in different threads: don't share the storage object between threads, but re-open read-only in each thread. That gets around caching issues. It's not that expensive in terms of real memory use, because everything is mapped, but it'll consume memory mapping address space (of which there usually are a few Gb). If massive r/o sharing is needed, then one could probably implement a special strategy object which shares the memory map, so that this restriction is lifted and 100's of threads can have concurrent r/o access. The downside remains the fact that r/w is not possible (but commit-extend may be a way out). -jcw From mbloore@yahoo.com Thu Aug 28 13:14:08 2003 Received: from web14305.mail.yahoo.com (web14305.mail.yahoo.com [216.136.173.81]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h7SIE8D31429 for ; Thu, 28 Aug 2003 13:14:08 -0500 Message-ID: <20030828181407.46625.qmail@web14305.mail.yahoo.com> Received: from [64.231.194.102] by web14305.mail.yahoo.com via HTTP; Thu, 28 Aug 2003 11:14:07 PDT Date: Thu, 28 Aug 2003 11:14:07 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] Metakit and multithreaded access? To: metakit@equi4.com In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --- Jean-Claude Wippler wrote: > Mark's approach above is a special trick to get around using the same > datafile in different threads: don't share the storage object between > threads, but re-open read-only in each thread. That gets around > caching issues. It's not that expensive in terms of real memory use, > because everything is mapped, but it'll consume memory mapping address > space (of which there usually are a few Gb). do you mean that the file gets read to the same physical memory for each concurrent open, but is mapped to different virtual addresses? that could get to be a problem for me, as we scale up usage. > If massive r/o sharing is > needed, then one could probably implement a special strategy object > which shares the memory map, so that this restriction is lifted and > 100's of threads can have concurrent r/o access. The downside remains > the fact that r/w is not possible (but commit-extend may be a way out). commit-extend tends to make my data file grow very fast. commit-aside may help; the last time i experimented with it i failed to figure out how to roll the changes back into the main file. (i gather one has to serialize the whole thing to another file, then deserialize back to a new main file.) could you give me some hints on how to build this special strategy object you mentioned? ===== -- mARK bLOORE __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com From jcw@equi4.com Thu Aug 28 14:36:04 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h7SJa3D05547 for ; Thu, 28 Aug 2003 14:36:03 -0500 Received: from equi4.com (bookie.equi4.local [10.0.1.2]) by madder.xs4all.nl (Postfix) with ESMTP id B1C8983CC9 for ; Thu, 28 Aug 2003 21:35:57 +0200 (CEST) Date: Thu, 28 Aug 2003 21:35:56 +0200 Subject: Re: [Metakit] Metakit and multithreaded access? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20030828181407.46625.qmail@web14305.mail.yahoo.com> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: > do you mean that the file gets read to the same physical memory for > each concurrent open, but is mapped to different virtual addresses? > that could get to be a problem for me, as we scale up usage. Yes. The file is opened with different fd's, which are then mmap'ed. So blocks off the disk will be loaded once, but mapped as shared pages into multiple different areas of virtual memory. > commit-extend tends to make my data file grow very fast. commit-aside > may help; the last time i experimented with it i failed to figure out > how to roll the changes back into the main file. (i gather one has to > serialize the whole thing to another file, then deserialize back to a > new main file.) Yes, commit-extend can be expensive in file space. So is the current commit-aside implementation, because the planned smart diffs have not been implemented so far. The combination of the two is what I'm really hoping to do one day, it would lead to single-file commits, with little file overhead (storing diffs at the end), and massively scalable shared reading (no mutex locking at all). Rolling changes back into the main file is a matter of calling storage.Commit(true). At that point, the aside file contents are (logically) cleared, even though that file does not shrink. > could you give me some hints on how to build this special strategy > object you mentioned? Several things need to be tried IMO. Easiest would be to simply create a single c4_FileStrategy object and pass that same one to each open (taking care not to delete it on each close, but only after the last one). This will let you find out whether the mmap virtual pages are re-used when the same fd is used everywhere. Just do a couple of opens using the same c4_FileStrategy instance and you can find out whether the mappings are shared - look at the value of _mapStart in each instance, they should be non-zero and identical. I don't knowe what different OS flavors do with this. I'm quite sure you cannot safely share c4_Strategy objects between threads, so the above is just a quick test. In the end, you need to create an instance for each use, i.e. right before each datafile open. The c4_Storage class accepts c4_Strategy objects as alternative way of controlling all I/O. This is an extremely powerful option, when combined with custom c4_Strategy objects. So the trick is to derive a new custom class from c4_FileStrategy. The key member to override is either DataOpen (if a shared fd is good enough), or ResetFileMapping. In the latter case, the trick wil be to open and mmap the datafile once before using MK, and then special-casing ResetFileMapping in the new class to re-use that single mapping, i.e. copying the pointer + length instead of calling mmap/munmap. I haven't tried any of this myself, but did think it through at one point, just to have an option when this massive sharing would arise (it risked doing so in one project, but in the end I never had to solve that issue after all). Good luck - please let me know how things go! -jcw From Garth.Lancaster@mbf.com.au Fri Aug 29 18:14:25 2003 Received: from fs1-smtp.mbf.com.au ([61.88.251.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h7TNEOD12463 for ; Fri, 29 Aug 2003 18:14:25 -0500 Received: from MBF-INT1-MTA by fs1-smtp.mbf.com.au with Novell_GroupWise; Sat, 30 Aug 2003 09:07:24 +1000 Message-Id: X-Mailer: Novell GroupWise Internet Agent 6.0.2 Date: Sat, 30 Aug 2003 09:14:12 +1000 From: "Garth Lancaster" To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Guinevere: 2.0.9 ; Subject: [Metakit] C++ Metakit Locking/Threading & Concurrency Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi everyone .. thanks to the brilliant work done by John Taylor, and more recently Jeff Kay, Ive updated the bottom of the Metakit Wiki 'Threading and concurrency' discussion at http://www.equi4.com/metakit/wiki.cgi/13 to show two of the approaches to this issue. Thanks also to JCW for uploading Jeff's example code to his site as a central distribution point. There is nothing to stop anyone else following on and adding their ideas etc to that discussion, either. Garth This email contains confidential information intended only for the addressee. If you received it in error, please inform us by reply email or by calling us from Australia on 131 137 or Internationally on 612 93239500. Please also delete this email and destroy any hard copy. You must not disclose or use the information in this email. From david@rebirthing.co.nz Fri Aug 29 22:22:42 2003 Received: from freenet.org.nz (adsl75-84.world-net.co.nz [210.55.75.84]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h7U3MeD23918 for ; Fri, 29 Aug 2003 22:22:41 -0500 Received: from root (helo=192.168.2.2) by freenet.org.nz with local-smtp (Exim 3.36 #1 (Debian)) id 19swJt-0000TU-00 for ; Sat, 30 Aug 2003 15:22:33 +1200 From: David McNab To: metakit@equi4.com Content-Type: text/plain Message-Id: <1062213624.904.6.camel@rebirth> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.4 Date: Sat, 30 Aug 2003 15:20:24 +1200 Content-Transfer-Encoding: 7bit X-Spambayes-Classification: ham; 0.00 Subject: [Metakit] Feature Req: sort view on function Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I was looking through the python docs for metakit, and while there are view methods that select rows satisfying a function, there seems to be no way to sort a view with a function. I'd love some way to do: v = db.view('myview')... v1 = v.sortusing(lambda r1,r2: cmp(len(r1.field1), len(r2.field1))) Here, 'field1' is a string, and the above call would be sorting on the *length* of field1 (as opposed to its alphanumeric order). Any chance of something like this being added to Metakit, and implemented at a low enough level to get a speed gain? And preferably, the returned view being editable? -- Cheers David From joostkremers@fastmail.fm Tue Sep 2 15:26:19 2003 Received: from amsfep15-int.chello.nl (amsfep15-int.chello.nl [213.46.243.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h82KQID10362 for ; Tue, 2 Sep 2003 15:26:19 -0500 Received: from teuctli.localdomain ([62.194.202.111]) by amsfep15-int.chello.nl (InterMail vM.5.01.05.17 201-253-122-126-117-20021021) with ESMTP id <20030902202612.OXGH6169.amsfep15-int.chello.nl@teuctli.localdomain> for ; Tue, 2 Sep 2003 22:26:12 +0200 Received: from joost by teuctli.localdomain with local (Exim 4.10) id 19uHPA-0002ow-00 for metakit@equi4.com; Tue, 02 Sep 2003 22:05:32 +0200 Date: Tue, 2 Sep 2003 22:05:32 +0200 From: Joost Kremers To: metakit@equi4.com Message-ID: <20030902200532.GA10831@teuctli.arnhem.chello.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-Editor: Emacs of course! Subject: [Metakit] compile error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: hi all, i seem to have a problem compiling the python module for metakit. when i run configure with --with-python, make bails out with the following error: In file included from ../python/PyProperty.cpp:11: ../python/scxx/PWONumber.h:19: parse error before `i' ../python/scxx/PWONumber.h:21: ISO C++ forbids defining types within return type ../python/scxx/PWONumber.h:21: semicolon missing after declaration of `class PWONumber' ../python/scxx/PWONumber.h: In function `int PWONumber(double)': ../python/scxx/PWONumber.h:21: only constructors take base initializers ../python/scxx/PWONumber.h:21: confused by earlier errors, bailing out make: *** [PyProperty.o] Fout 1 this was while compiling 2.4.9.2, after having modified the Makefile to look for python2.3 (it was hard-coded to look for python2.2). i went and got the cvs, ran configure (now without --with-python as suggested in the archives of this list) and then ran 'python setup.py install' in the python subdir, but ran into the exact same error. my system is linux slackware 9.0 (x86) with gcc 3.2.2. anyone run into this before, or have any idea what's wrong and how it can be fixed? TIA -- Joost Kremers Life has its moments From njriley@shrug.csl.uiuc.edu Tue Sep 2 16:25:14 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h82LPED15864 for ; Tue, 2 Sep 2003 16:25:14 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.8/8.12.8) with ESMTP id h82LPDau038165; Tue, 2 Sep 2003 16:25:13 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.8/8.12.8/Submit) id h82LPBjb038103; Tue, 2 Sep 2003 16:25:11 -0500 (CDT) Date: Tue, 2 Sep 2003 16:25:11 -0500 From: Nicholas Riley To: Joost Kremers Cc: metakit@equi4.com Subject: Re: [Metakit] compile error Message-ID: <20030902212511.GA11371@uiuc.edu> Mail-Followup-To: Joost Kremers , metakit@equi4.com References: <20030902200532.GA10831@teuctli.arnhem.chello.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030902200532.GA10831@teuctli.arnhem.chello.nl> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Tue, Sep 02, 2003 at 10:05:32PM +0200, Joost Kremers wrote: > hi all, > > i seem to have a problem compiling the python module for metakit. when i > run configure with --with-python, make bails out with the following error: > > In file included from ../python/PyProperty.cpp:11: > ../python/scxx/PWONumber.h:19: parse error before `i' > ../python/scxx/PWONumber.h:21: ISO C++ forbids defining types within return > type > ../python/scxx/PWONumber.h:21: semicolon missing after declaration of `class > PWONumber' > ../python/scxx/PWONumber.h: In function `int PWONumber(double)': > ../python/scxx/PWONumber.h:21: only constructors take base initializers > ../python/scxx/PWONumber.h:21: confused by earlier errors, bailing out > make: *** [PyProperty.o] Fout 1 > > this was while compiling 2.4.9.2, after having modified the Makefile to > look for python2.3 (it was hard-coded to look for python2.2). i went and > got the cvs, ran configure (now without --with-python as suggested in the > archives of this list) and then ran 'python setup.py install' in the python > subdir, but ran into the exact same error. I've fixed this, I will send you a fixed version later today. I've been working on a patch to the Python bindings to fix lots of stuff, but it's not ready for incorporation yet; the version I'm currently using should work, however (except for the test suite being broken). -- =Nicholas Riley | From David.VanMaren@DakoCytomation.com Tue Sep 2 18:03:29 2003 Received: from cheetah.cytomation.com (cheetah.cytomation.com [63.74.101.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h82N3SD23146 for ; Tue, 2 Sep 2003 18:03:29 -0500 Received: by cheetah.cytomation.com with Internet Mail Service (5.5.2653.19) id ; Tue, 2 Sep 2003 17:02:25 -0600 Message-ID: <7751A746C53FE2439C4FCF28388738DB1A2394@cheetah.cytomation.com> From: David Van Maren To: "'metakit@equi4.com'" Date: Tue, 2 Sep 2003 17:02:19 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C371A6.42DC4EB0" Subject: [Metakit] Using head of file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C371A6.42DC4EB0 Content-Type: text/plain Hello. I'm using metakit (with e4graph) to store information associated with our software (see www.dakocytomation.com). We are very happy so far with both of them, and have been delighted with how few bugs we've experienced. However, we've recently observed some behavior that has raised some questions, which I'm hoping I can get answered here. I'm sorry if it's already been discussed or documented, but I've not been able to find anything on it so far. I noticed that constructing a (modifiable) storage from an existing non-metakit file succeeds, and allows metakit information to be written to it with no errors. Examining the file afterwards, I've seen that the metakit information is simply appended to the original file data, leaving it intact. I looked at some of the format documentation, and it indicated that there is both a header and a footer used by metakit. I'm guessing that the above behavior was by design, in order to allow users to use the head of the file for non-metakit information (such as their own magic number or similar information). But that's just a guess, so I've got a few questions: 1. Is this behavior intentional? 2. If not, shouldn't metakit fail construction of a Storage from an existing non-metakit file? 3a. If it is intentional, does metakit guarantee that it will leave the head of the file unchanged, even through a commit() which changes the metakit contents? 3b. Does metakit care if I subsequently modify the head of the file (after closing the Storage associated with it)? 3c. Does metakit care if the head of the file is grown or shrunk so long as the Storage is closed? We're wanting to mark each storage file with our own special magic number, and this looks like a very easy way to do it, if metakit supports it. Thankyou in advance, David Van Maren david.vanmaren@dakocytomation.com ------_=_NextPart_001_01C371A6.42DC4EB0 Content-Type: text/html Content-Transfer-Encoding: quoted-printable Using head of file

Hello.

I'm using metakit (with e4graph) to store information = associated with our software (see www.dakocytomation.com).  We are = very happy so far with both of them, and have been delighted with how = few bugs we've experienced.  However, we've recently observed some = behavior that has raised some questions, which I'm hoping I can get = answered here.  I'm sorry if it's already been discussed or = documented, but I've not been able to find anything on it so = far.

I noticed that constructing a (modifiable) storage = from an existing non-metakit file succeeds, and allows metakit = information to be written to it with no errors.  Examining the = file afterwards, I've seen that the metakit information is simply = appended to the original file data, leaving it intact.

I looked at some of the format documentation, and it = indicated that there is both a header and a footer used by = metakit.  I'm guessing that the above behavior was by design, in = order to allow users to use the head of the file for non-metakit = information (such as their own magic number or similar = information).  But that's just a guess, so I've got a few = questions:

1. Is this behavior intentional?

2. If not, shouldn't metakit fail construction of a = Storage from an existing non-metakit file?

3a. If it is intentional, does metakit guarantee that = it will leave the head of the file unchanged, even through a commit() = which changes the metakit contents?

3b. Does metakit care if I subsequently modify the = head of the file (after closing the Storage associated with it)?

3c. Does metakit care if the head of the file is = grown or shrunk so long as the Storage is closed?

We're wanting to mark each storage file with our own = special magic number, and this looks like a very easy way to do it, if = metakit supports it.

Thankyou in advance,

David Van Maren
david.vanmaren@dakocytomation.com

------_=_NextPart_001_01C371A6.42DC4EB0-- From jcw@equi4.com Tue Sep 2 18:29:48 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h82NTmD24764 for ; Tue, 2 Sep 2003 18:29:48 -0500 Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 6D84683CC9 for ; Wed, 3 Sep 2003 01:29:42 +0200 (CEST) Date: Wed, 3 Sep 2003 01:29:44 +0200 Subject: Re: [Metakit] Using head of file Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: "'metakit@equi4.com'" In-Reply-To: <7751A746C53FE2439C4FCF28388738DB1A2394@cheetah.cytomation.com> Message-Id: <5572E810-DD9D-11D7-B2D2-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h82NTmD24764 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: David Van Maren wrote: > I noticed that constructing a (modifiable) storage from an existing > non-metakit file succeeds, and allows metakit information to be > written to it with no errors.  Examining the file afterwards, I've > seen that the metakit information is simply appended to the original > file data, leaving it intact. > > I looked at some of the format documentation, and it indicated that > there is both a header and a footer used by metakit.  I'm guessing > that the above behavior was by design, in order to allow users to use > the head of the file for non-metakit information (such as their own > magic number or similar information).  But that's just a guess, so > I've got a few questions: > > 1. Is this behavior intentional? Yes. It's used in the Tcl scripting language to implement Starkits and Starpacks: scripts and executables which can be launched and contain a MK datafile, piggy-back style. In the case of Starkits, the header is a regular Tcl script (Tcl stops reading at a CTRL/Z in the file). > 2. If not, shouldn't metakit fail construction of a Storage from an > existing non-metakit file? I agree that this append behavior can confuse things. One way to check is to open read-only, and check the description string of the storage contents. I should come out as empty. > 3a. If it is intentional, does metakit guarantee that it will leave > the head of the file unchanged, even through a commit() which changes > the metakit contents? Yes. > 3b. Does metakit care if I subsequently modify the head of the file > (after closing the Storage associated with it)? No. No need to close - MK ignores header data. > 3c. Does metakit care if the head of the file is grown or shrunk so > long as the Storage is closed? No. The tail markers use relative sizes - here, "closed" is essential. > We're wanting to mark each storage file with our own special magic > number, and this looks like a very easy way to do it, if metakit > supports it. Yep. -jcw From jyl@mod3.net Tue Sep 2 19:18:36 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h830IaD27690 for ; Tue, 2 Sep 2003 19:18:36 -0500 Received: (qmail 17491 invoked by uid 48); 3 Sep 2003 00:18:34 -0000 Received: from 192.18.42.10 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Tue, 2 Sep 2003 17:18:34 -0700 (PDT) Message-ID: <59798.192.18.42.10.1062548314.squirrel@mod3.net> Date: Tue, 2 Sep 2003 17:18:34 -0700 (PDT) Subject: Re: [Metakit] Using head of file From: "Jacob Levy" To: X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: >I agree that this append behavior can confuse things. One way to check >is to open read-only, and check the description string of the storage >contents. I should come out as empty. You mean NULL or ""? --JYL From njriley@shrug.csl.uiuc.edu Wed Sep 3 01:41:02 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h836f2D14863 for ; Wed, 3 Sep 2003 01:41:02 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.8/8.12.8) with ESMTP id h836ewau025052; Wed, 3 Sep 2003 01:40:58 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.8/8.12.8/Submit) id h836ew14025051; Wed, 3 Sep 2003 01:40:58 -0500 (CDT) Date: Wed, 3 Sep 2003 01:40:58 -0500 From: Nicholas Riley To: Joost Kremers Cc: metakit@equi4.com Subject: Re: [Metakit] compile error Message-ID: <20030903064058.GA24814@uiuc.edu> Mail-Followup-To: Joost Kremers , metakit@equi4.com References: <20030902200532.GA10831@teuctli.arnhem.chello.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030902200532.GA10831@teuctli.arnhem.chello.nl> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Tue, Sep 02, 2003 at 10:05:32PM +0200, Joost Kremers wrote: > i seem to have a problem compiling the python module for metakit. when i > run configure with --with-python, make bails out with the following error: > > In file included from ../python/PyProperty.cpp:11: > ../python/scxx/PWONumber.h:19: parse error before `i' > ../python/scxx/PWONumber.h:21: ISO C++ forbids defining types within return > type > ../python/scxx/PWONumber.h:21: semicolon missing after declaration of `class > PWONumber' > ../python/scxx/PWONumber.h: In function `int PWONumber(double)': > ../python/scxx/PWONumber.h:21: only constructors take base initializers > ../python/scxx/PWONumber.h:21: confused by earlier errors, bailing out > make: *** [PyProperty.o] Fout 1 > > this was while compiling 2.4.9.2, after having modified the Makefile to > look for python2.3 (it was hard-coded to look for python2.2). i went and > got the cvs, ran configure (now without --with-python as suggested in the > archives of this list) and then ran 'python setup.py install' in the python > subdir, but ran into the exact same error. The problem here is that the build system assumes that where 'long long' is defined, 'LONG_LONG' is also defined. I've fixed that mistaken assumption, along with some miscellaneous other bugs in the Python binding in this version: I guess I wasn't reading the list carefully enough and didn't realize that a distutils based build system for the python bindings has been introduced. It still doesn't appear to be hooked up to the Makefile, so I have updated the make based build system for Python 2.3. A couple of things to watch out for in this version. - As I mentioned, the test scripts are totally broken. test.regrtest, which I used for the tests I contributed early this year, is unusable for external tests in Python 2.3. I'm in the process of submitting a patch to Python to fix this problem. Also, the ViewerTestCase, etc. tests which were written by someone else do not work with my 'all.py' contributed test script or Makefile, even though it's trivial to get unittest based tests to work with test.regrtest (as is done in the testing of Python itself). - I've made some changes, especially with regard to Unicode strings, that may not work properly on Python 2.2; I have only tested on 2.3. I didn't really want to release this patch at this point, but since other people are running into the same kinds of problems I did so anyway. Look for a cleaned-up, complete patch to be posted to the list in the next few days. -- =Nicholas Riley | From joostkremers@fastmail.fm Wed Sep 3 07:29:50 2003 Received: from amsfep15-int.chello.nl (amsfep15-int.chello.nl [213.46.243.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83CTnD07196 for ; Wed, 3 Sep 2003 07:29:50 -0500 Received: from teuctli.localdomain ([62.194.202.111]) by amsfep15-int.chello.nl (InterMail vM.5.01.05.17 201-253-122-126-117-20021021) with ESMTP id <20030903122943.HMGO6169.amsfep15-int.chello.nl@teuctli.localdomain> for ; Wed, 3 Sep 2003 14:29:43 +0200 Received: from joost by teuctli.localdomain with local (Exim 4.10) id 19uWRD-00070u-00 for metakit@equi4.com; Wed, 03 Sep 2003 14:08:39 +0200 Date: Wed, 3 Sep 2003 14:08:39 +0200 From: Joost Kremers To: metakit@equi4.com Subject: Re: [Metakit] compile error Message-ID: <20030903120839.GE10831@teuctli.arnhem.chello.nl> References: <20030902200532.GA10831@teuctli.arnhem.chello.nl> <20030903064058.GA24814@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030903064058.GA24814@uiuc.edu> User-Agent: Mutt/1.4.1i X-Editor: Emacs of course! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Wed, Sep 03, 2003 at 01:40:58AM -0500, Nicholas Riley wrote: > The problem here is that the build system assumes that where 'long > long' is defined, 'LONG_LONG' is also defined. I've fixed that > mistaken assumption, along with some miscellaneous other bugs in the > Python binding in this version: > > thanks! just tried it and it builds fine, using 'python setup.py install'. > - I've made some changes, especially with regard to Unicode strings, when i store a unicode string in metakit from python and then retrieve it again, it is no longer a unicode string. it seems to be utf-8. do you know if this is correct? (is there a way in python to determine the encoding of a string?) -- Joost Kremers Life has its moments From bkelley@wi.mit.edu Wed Sep 3 07:32:47 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h83CWlD07415 for ; Wed, 3 Sep 2003 07:32:47 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003090308324323801 for ; Wed, 03 Sep 2003 08:32:43 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HKN1IJ00.CM3; Wed, 3 Sep 2003 08:32:43 -0400 Message-ID: <3F55DF9D.4070805@wi.mit.edu> Date: Wed, 03 Sep 2003 08:33:33 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: David McNab , metakit@equi4.com Subject: Re: [Metakit] Feature Req: sort view on function References: <1062213624.904.6.camel@rebirth> In-Reply-To: <1062213624.904.6.camel@rebirth> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: David McNab wrote: >I'd love some way to do: > >v = db.view('myview')... > >v1 = v.sortusing(lambda r1,r2: cmp(len(r1.field1), len(r2.field1))) > > Using a function to sort will always be a little slow. I have a python version of what you are wanting that has reasonable performance (2ghz pentium 4), there appears to be a factor of 10 difference between sorting a python list with a function and sorting a metakit view with a function: loaded 50000 elements in 7.2009999752 seconds metakit sort with function 50000 elements in 12.0870000124 seconds python sort with function 50000 elements in 1.16199994087 seconds metakit sorted normally in 0.53100001812 seconds python sorted normally in 0.289999961853 seconds Although I expect you will get better performance by adding a temporary property (or properties) to your view, filling it in with values, and then sorting on that. If you want the function, I'll post it to the news group. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From David.VanMaren@DakoCytomation.com Wed Sep 3 09:26:12 2003 Received: from cheetah.cytomation.com (cheetah.cytomation.com [63.74.101.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83EQBD16019 for ; Wed, 3 Sep 2003 09:26:12 -0500 Received: by cheetah.cytomation.com with Internet Mail Service (5.5.2653.19) id ; Wed, 3 Sep 2003 08:25:08 -0600 Message-ID: <7751A746C53FE2439C4FCF28388738DB1A2395@cheetah.cytomation.com> From: David Van Maren To: "'metakit@equi4.com'" Subject: RE: [Metakit] Using head of file Date: Wed, 3 Sep 2003 08:25:07 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C37227.2CD36B90" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C37227.2CD36B90 Content-Type: text/plain JC: Thankyou for the clarification regarding using the head of a Storage. It sounds like I'll be able to use it quite easily. DVM david.vanmaren@dakocytomation.com ------_=_NextPart_001_01C37227.2CD36B90 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: [Metakit] Using head of file

JC:

Thankyou for the clarification regarding using the = head of a Storage.  It sounds like I'll be able to use it quite = easily.

DVM
david.vanmaren@dakocytomation.com

------_=_NextPart_001_01C37227.2CD36B90-- From njriley@shrug.csl.uiuc.edu Wed Sep 3 10:31:14 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83FVCD22175 for ; Wed, 3 Sep 2003 10:31:12 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.8/8.12.8) with ESMTP id h83FVBau042441; Wed, 3 Sep 2003 10:31:11 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.8/8.12.8/Submit) id h83FVBBZ042440; Wed, 3 Sep 2003 10:31:11 -0500 (CDT) Date: Wed, 3 Sep 2003 10:31:11 -0500 From: Nicholas Riley To: Joost Kremers Cc: metakit@equi4.com Subject: Re: [Metakit] compile error Message-ID: <20030903153110.GA42339@uiuc.edu> Mail-Followup-To: Joost Kremers , metakit@equi4.com References: <20030902200532.GA10831@teuctli.arnhem.chello.nl> <20030903064058.GA24814@uiuc.edu> <20030903120839.GE10831@teuctli.arnhem.chello.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030903120839.GE10831@teuctli.arnhem.chello.nl> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Wed, Sep 03, 2003 at 02:08:39PM +0200, Joost Kremers wrote: > On Wed, Sep 03, 2003 at 01:40:58AM -0500, Nicholas Riley wrote: > > The problem here is that the build system assumes that where 'long > > long' is defined, 'LONG_LONG' is also defined. I've fixed that > > mistaken assumption, along with some miscellaneous other bugs in the > > Python binding in this version: > > > > > > thanks! just tried it and it builds fine, using 'python setup.py install'. > > > - I've made some changes, especially with regard to Unicode strings, > > when i store a unicode string in metakit from python and then retrieve it > again, it is no longer a unicode string. it seems to be utf-8. do you know > if this is correct? (is there a way in python to determine the encoding of > a string?) According to the Metakit documentation, strings are defined to be UTF-8. When adding Unicode string support I was not sure whether it was worth it to always convert back to Unicode strings when retrieving from the database, as it's likely that a lot of fields will be ASCII only, and that's a bit of a waste of space and time. What do other people think? You can specify the encoding like this: x = unicode('sample', 'utf-8') unicode(string) uses the 'default encoding' which in most cases is UTF-8, but it's unclear whether this is always true. -- =Nicholas Riley | From joostkremers@fastmail.fm Wed Sep 3 11:08:44 2003 Received: from amsfep15-int.chello.nl (amsfep15-int.chello.nl [213.46.243.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83G8hD26226 for ; Wed, 3 Sep 2003 11:08:44 -0500 Received: from teuctli.localdomain ([62.194.202.111]) by amsfep15-int.chello.nl (InterMail vM.5.01.05.17 201-253-122-126-117-20021021) with ESMTP id <20030903160837.NPFR6169.amsfep15-int.chello.nl@teuctli.localdomain> for ; Wed, 3 Sep 2003 18:08:37 +0200 Received: from joost by teuctli.localdomain with local (Exim 4.10) id 19uZr0-0005HU-00; Wed, 03 Sep 2003 17:47:30 +0200 Date: Wed, 3 Sep 2003 17:47:30 +0200 From: Joost Kremers To: metakit@equi4.com Cc: Joost Kremers Subject: Re: [Metakit] compile error & unicode Message-ID: <20030903154730.GF10831@teuctli.arnhem.chello.nl> Mail-Followup-To: metakit@equi4.com References: <20030902200532.GA10831@teuctli.arnhem.chello.nl> <20030903064058.GA24814@uiuc.edu> <20030903120839.GE10831@teuctli.arnhem.chello.nl> <20030903153110.GA42339@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030903153110.GA42339@uiuc.edu> User-Agent: Mutt/1.4.1i X-Editor: Emacs of course! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Wed, Sep 03, 2003 at 10:31:11AM -0500, Nicholas Riley wrote: > > > > > > > thanks! just tried it and it builds fine, using 'python setup.py install'. btw, i forgot to mention in my previous message that i did get two warnings that you may or may not care about: PyView.cpp:642: warning: `char*relocrows__doc' defined but not used PyStorage.cpp: In function `PyObject* PyStorage_Description(PyStorage*, PyObject*)': PyStorage.cpp:197: warning: end of non-void function is reached without return value the second one i translated from dutch, so i'm not exactly sure what the original english warning says, but you'll know what it means. like i said, i don't know if you care about these. just thought i'd mention them. > According to the Metakit documentation, strings are defined to be > UTF-8. When adding Unicode string support I was not sure whether it > was worth it to always convert back to Unicode strings when retrieving > from the database, as it's likely that a lot of fields will be ASCII > only, and that's a bit of a waste of space and time. What do other > people think? > > You can specify the encoding like this: > > x = unicode('sample', 'utf-8') > > unicode(string) uses the 'default encoding' which in most cases is > UTF-8, but it's unclear whether this is always true. in my setup, this is what happens: >>> x = unicode('string') >>> x u'string' >>> x.encode('utf-8') 'string' >>> so (if i'm interpreting this correctly, i've only been experimenting with Python for a week or so), unicode('string') returns a Python unicode string, which is not the same as a utf-8 string. not translating it back to unicode works for me, because the strings i need to store are indeed mostly ascii. they only contain a few non-ascii characters for which i need unicode/utf-8. BTW, i can't tell you how happy i was to find metakit and Mk4Py. it is *exactly* the thing i need in the project i'm about to start with. :-) -- Joost Kremers Life has its moments From bkelley@wi.mit.edu Wed Sep 3 15:16:57 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h83KGvD18880 for ; Wed, 3 Sep 2003 15:16:57 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003090316165229948 for ; Wed, 03 Sep 2003 16:16:52 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HKNN0400.EU5 for ; Wed, 3 Sep 2003 16:16:53 -0400 Message-ID: <3F564C66.4070108@wi.mit.edu> Date: Wed, 03 Sep 2003 16:17:42 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "'metakit@equi4.com'" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] How long to wait before commiting? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I am loading a database with a bunch of new data that the user is allowed to validate. I have been using commit() and rollback() for these operations because it's easy :) The question I have is, what are the ramifications of loading a lot of data without commiting? Memory? Speed? Inquiring minds want to know. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From jcw@equi4.com Wed Sep 3 15:26:22 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83KQLD19935 for ; Wed, 3 Sep 2003 15:26:21 -0500 Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id D9C8183CC9 for ; Wed, 3 Sep 2003 22:26:15 +0200 (CEST) Date: Wed, 3 Sep 2003 22:26:19 +0200 Subject: Re: [Metakit] How long to wait before commiting? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: "'metakit@equi4.com'" Content-Transfer-Encoding: 7bit In-Reply-To: <3F564C66.4070108@wi.mit.edu> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > I am loading a database with a bunch of new data that the user is > allowed to validate. > > I have been using commit() and rollback() for these operations because > it's easy :) The question I have is, what are the ramifications of > loading a lot of data without commiting? Memory? Speed? Inquiring > minds want to know. Memory usage grows until the commit, as more and more view changes are buffered. For classical data entry, i.e. typing, I would assume that speed is never an issue, nor are the amounts of data, i.e. memory usage... -jcw From jyl@mod3.net Wed Sep 3 16:45:08 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h83Lj8D28886 for ; Wed, 3 Sep 2003 16:45:08 -0500 Received: (qmail 7071 invoked by uid 48); 3 Sep 2003 21:45:00 -0000 Received: from 192.18.42.11 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Wed, 3 Sep 2003 14:44:59 -0700 (PDT) Message-ID: <34033.192.18.42.11.1062625499.squirrel@mod3.net> Date: Wed, 3 Sep 2003 14:44:59 -0700 (PDT) Subject: Re: [Metakit] How long to wait before commiting? From: "Jacob Levy" To: In-Reply-To: References: <3F564C66.4070108@wi.mit.edu> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Is changed data buffered in in-core memory? I thought the new data was actually written to the storage and commit is just switching pointers? Or is the commit doing both the writing and pointer switching? --JYL > Brian Kelley wrote: > >> I am loading a database with a bunch of new data that the user is >> allowed to validate. >> >> I have been using commit() and rollback() for these operations because >> it's easy :) The question I have is, what are the ramifications of >> loading a lot of data without commiting? Memory? Speed? Inquiring >> minds want to know. > > Memory usage grows until the commit, as more and more view changes are > buffered. > > For classical data entry, i.e. typing, I would assume that speed is > never an issue, nor are the amounts of data, i.e. memory usage... > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Wed Sep 3 16:52:28 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83LqSD29701 for ; Wed, 3 Sep 2003 16:52:28 -0500 Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 4F9C683CC9 for ; Wed, 3 Sep 2003 23:52:22 +0200 (CEST) Date: Wed, 3 Sep 2003 23:52:25 +0200 Subject: Re: [Metakit] How long to wait before commiting? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <34033.192.18.42.11.1062625499.squirrel@mod3.net> Message-Id: X-Mailer: Apple Mail (2.552) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > Is changed data buffered in in-core memory? I thought the new data was > actually written to the storage and commit is just switching pointers? > Or > is the commit doing both the writing and pointer switching? Yes. No. Yes. :) -jcw From David.VanMaren@DakoCytomation.com Wed Sep 3 17:08:52 2003 Received: from cheetah.cytomation.com (cheetah.cytomation.com [63.74.101.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83M8pD31590 for ; Wed, 3 Sep 2003 17:08:52 -0500 Received: by cheetah.cytomation.com with Internet Mail Service (5.5.2653.19) id ; Wed, 3 Sep 2003 16:07:49 -0600 Message-ID: <7751A746C53FE2439C4FCF28388738DB1A239A@cheetah.cytomation.com> From: David Van Maren To: "'metakit@equi4.com'" Subject: RE: [Metakit] Using head of file Date: Wed, 3 Sep 2003 16:07:48 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C37267.CF6952D0" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C37267.CF6952D0 Content-Type: text/plain > > 2. If not, shouldn't metakit fail construction of a Storage from an > > existing non-metakit file? > > I agree that this append behavior can confuse things. One > way to check is to open read-only, and check the description > string of the storage contents. I should come out as empty. Your semantics make sense, especially considering commit-aside, serialize, etc. Thanks for the explanation. I'm having some trouble propagating this feature up to my users, however. You know how long your section of the file is by your footer and your header, both of which are at the end of the user's non-metakit data. I'd like to prepend my own information to that information, and to hide it from the user, providing him with all of the space in the file up to my information, just as Metakit does. The file will look like this: ------------------------------------------------------- | user data | my info | mk header | mk data | mk footer | ------------------------------------------------------- So long as I want to give my user free reign over the head of the file (as I do), I do not see a way to determine where my information is on an open(), in order to separate it properly from the user's data. I'd rather not complicate the user's life by having him keep track of his own length somehow (just as you've not required that your users do so). So I'm looking for a way to determine how long all of the metakit data is, which will allow me to find my information, all from the tail of the file. I know that I could directly decode your footer, but I'd rather go through your API (for obvious reasons). 1. Is there any way right now for me to determine how much space in a file is occupied by metakit? 2. If not, or if it's difficult to do, would you consider adding a method somewhere to get it? Thanks, DVM ------_=_NextPart_001_01C37267.CF6952D0 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: [Metakit] Using head of file

> > 2. If not, shouldn't metakit fail = construction of a Storage from an
> > existing non-metakit file?
>
> I agree that this append behavior can confuse = things.  One
> way to check is to open read-only, and check = the description
> string of the storage contents.  I should = come out as empty.

Your semantics make sense, especially considering = commit-aside, serialize, etc.  Thanks for the explanation.

I'm having some trouble propagating this feature up = to my users, however.  You know how long your section of the file = is by your footer and your header, both of which are at the end of the = user's non-metakit data.  I'd like to prepend my own information = to that information, and to hide it from the user, providing him with = all of the space in the file up to my information, just as Metakit = does.

The file will look like this:

 -------------------------------------------------------
| user data | my info | mk header | mk data | mk = footer |
 -------------------------------------------------------

So long as I want to give my user free reign over the = head of the file (as I do), I do not see a way to determine where my = information is on an open(), in order to separate it properly from the = user's data.  I'd rather not complicate the user's life by having = him keep track of his own length somehow (just as you've not required = that your users do so).  So I'm looking for a way to determine how = long all of the metakit data is, which will allow me to find my = information, all from the tail of the file.

I know that I could directly decode your footer, but = I'd rather go through your API (for obvious reasons).

1. Is there any way right now for me to determine how = much space in a file is occupied by metakit? 2. If not, or if it's = difficult to do, would you consider adding a method somewhere to get = it?

Thanks,
DVM

------_=_NextPart_001_01C37267.CF6952D0-- From jcw@equi4.com Wed Sep 3 17:22:34 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83MMXD00566 for ; Wed, 3 Sep 2003 17:22:33 -0500 Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 87AE383CC9 for ; Thu, 4 Sep 2003 00:22:27 +0200 (CEST) Date: Thu, 4 Sep 2003 00:22:30 +0200 Subject: Re: [Metakit] Using head of file Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: "'metakit@equi4.com'" In-Reply-To: <7751A746C53FE2439C4FCF28388738DB1A239A@cheetah.cytomation.com> Message-Id: <1BB10C86-DE5D-11D7-B2D2-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h83MMXD00566 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: David Van Maren wrote: > The file will look like this: > >  ------------------------------------------------------- > | user data | my info | mk header | mk data | mk footer | >  ------------------------------------------------------- Let me briefly describe MK's position logic: - look at end - determine location of header - may need to repeat (some tail markers are multi-step) - once header is know, we're done - never look before it > I know that I could directly decode your footer, but I'd rather go > through your API (for obvious reasons). > > 1. Is there any way right now for me to determine how much space in a > file is occupied by metakit? 2. If not, or if it's difficult to do, > would you consider adding a method somewhere to get it? storage.Strategy().EndOfData() -jcw From David.VanMaren@DakoCytomation.com Wed Sep 3 17:59:54 2003 Received: from cheetah.cytomation.com (cheetah.cytomation.com [63.74.101.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h83MxsD04383 for ; Wed, 3 Sep 2003 17:59:54 -0500 Received: by cheetah.cytomation.com with Internet Mail Service (5.5.2653.19) id ; Wed, 3 Sep 2003 16:58:52 -0600 Message-ID: <7751A746C53FE2439C4FCF28388738DB1A239C@cheetah.cytomation.com> From: David Van Maren To: "'metakit@equi4.com'" Subject: RE: [Metakit] Using head of file Date: Wed, 3 Sep 2003 16:58:50 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3726E.F0DB78B0" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C3726E.F0DB78B0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable JC: > > The file will look like this: > > > > =A0------------------------------------------------------- > > | user data | my info | mk header | mk data | mk footer | > > =A0------------------------------------------------------- >=20 > Let me briefly describe MK's position logic: > - look at end > - determine location of header > - may need to repeat (some tail markers are multi-step) > - once header is know, we're done > - never look before it OK. Sounds great. > > I know that I could directly decode your footer, but I'd rather go > > through your API (for obvious reasons). > > > > 1. Is there any way right now for me to determine how much=20 > space in a > > file is occupied by metakit? 2. If not, or if it's difficult to do, = > > would you consider adding a method somewhere to get it? >=20 > storage.Strategy().EndOfData() Perfect. This should get me right where I want to go. And I'm guessing that (storage.Strategy().EndOfData() > 0) is another indicator that the file has some metakit information in it. Am I = right? If so, is one to be preferred over the other? Thanks again, DVM ------_=_NextPart_001_01C3726E.F0DB78B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [Metakit] Using head of file

JC:

> > The file will look like this:
> >
> > = =A0-------------------------------------------------------
> > | user data | my info | mk header | mk = data | mk footer |
> > = =A0-------------------------------------------------------
>
> Let me briefly describe MK's position = logic:
>       - look at = end
>       - determine = location of header
>       - may need to = repeat (some tail markers are multi-step)
>       - once header is = know, we're done
>       - never look = before it

OK. Sounds great.

> > I know that I could directly decode your = footer, but I'd rather go
> > through your API (for obvious = reasons).
> >
> > 1. Is there any way right now for me to = determine how much
> space in a
> > file is occupied by metakit? 2. If not, or = if it's difficult to do,
> > would you consider adding a method = somewhere to get it?
>
> storage.Strategy().EndOfData()

Perfect.  This should get me right where I want = to go.

And I'm guessing that (storage.Strategy().EndOfData() = > 0) is another indicator that the file has some metakit information = in it.  Am I right?  If so, is one to be preferred over the = other?

Thanks again,
DVM

------_=_NextPart_001_01C3726E.F0DB78B0-- From David.VanMaren@DakoCytomation.com Thu Sep 4 17:34:36 2003 Received: from cheetah.cytomation.com (cheetah.cytomation.com [63.74.101.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h84MYZD15684 for ; Thu, 4 Sep 2003 17:34:36 -0500 Received: by cheetah.cytomation.com with Internet Mail Service (5.5.2653.19) id ; Thu, 4 Sep 2003 16:33:32 -0600 Message-ID: <7751A746C53FE2439C4FCF28388738DB1A239F@cheetah.cytomation.com> From: David Van Maren To: "'metakit@equi4.com'" Subject: RE: [Metakit] Using head of file Date: Thu, 4 Sep 2003 16:33:31 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C37334.91A5ED50" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C37334.91A5ED50 Content-Type: text/plain JC: > > 3b. Does metakit care if I subsequently modify the head of the file > > (after closing the Storage associated with it)? > > No. No need to close - MK ignores header data. If I want to offer write access to the head of the file while it's open as a Storage, would you recommend using fopen() directly, or obtaining a FILE * from you somehow? I would think that I could not use c4_FileStrategy.DataRead() and DataWrite(), as they only provide access the tail of the file. Am I right? Thanks, DVM ------_=_NextPart_001_01C37334.91A5ED50 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: [Metakit] Using head of file

JC:

> > 3b. Does metakit care if I subsequently = modify the head of the file
> > (after closing the Storage associated with = it)?
>
> No.  No need to close - MK ignores header = data.

If I want to offer write access to the head of the = file while it's open as a Storage, would you recommend using fopen() = directly, or obtaining a FILE * from you somehow?  I would think = that I could not use c4_FileStrategy.DataRead() and DataWrite(), as = they only provide access the tail of the file. Am I right?

Thanks,
DVM

------_=_NextPart_001_01C37334.91A5ED50-- From jyl@mod3.net Thu Sep 4 22:06:46 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8536kD02185 for ; Thu, 4 Sep 2003 22:06:46 -0500 Received: (qmail 28291 invoked by uid 48); 5 Sep 2003 03:06:45 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 4 Sep 2003 20:06:45 -0700 (PDT) Message-ID: <64975.209.233.24.218.1062731205.squirrel@mod3.net> Date: Thu, 4 Sep 2003 20:06:45 -0700 (PDT) Subject: RE: [Metakit] Using head of file From: "Jacob Levy" To: In-Reply-To: <7751A746C53FE2439C4FCF28388738DB1A239F@cheetah.cytomation.com> References: <7751A746C53FE2439C4FCF28388738DB1A239F@cheetah.cytomation.com> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: David There's a reason for having the Metakit abstraction layer, and you're attempting to break it :) I suggest you treat the joined-at-the-hip thing you're trying to create as a file for the purpose of writing header information, and as a Metakit storage when using it to do Metakit-ish things. You cannot get a FILE * from Metakit for an open storage because how Metakit actually accesses the storage (note: I did not say file) is its own business. On some systems Metakit does not use files but rather memory mappings, so there's no way to do what you are asking for. What I'd do: * Read/write your data using a FILE * created for that purpose. * Close it * Get a c4_Storage from Metakit on this file. Use that. I.e. for safety I wouldn't overlap the lifetime of the two abstractions. So I *think* (JC please correct me) that you need to be very careful to close the Metakit storage while you're writing the header data, especially if the amount of header data will change as a result of writing. Things would probably go *really* badly for you if held open a Metakit storage while the location of the start of Metakit data inside the file would change as a result of prepending header data. --JYL > JC: > >> > 3b. Does metakit care if I subsequently modify the head of the file >> (after closing the Storage associated with it)? >> >> No. No need to close - MK ignores header data. > > If I want to offer write access to the head of the file while it's open > as a Storage, would you recommend using fopen() directly, or obtaining a > FILE * from you somehow? I would think that I could not use > c4_FileStrategy.DataRead() and DataWrite(), as they only provide access > the tail of the file. Am I right? > > Thanks, > DVM From David.VanMaren@DakoCytomation.com Fri Sep 5 17:03:13 2003 Received: from cheetah.cytomation.com (cheetah.cytomation.com [63.74.101.5] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h85M3DD19030 for ; Fri, 5 Sep 2003 17:03:13 -0500 Received: by cheetah.cytomation.com with Internet Mail Service (5.5.2653.19) id ; Fri, 5 Sep 2003 16:02:09 -0600 Message-ID: <7751A746C53FE2439C4FCF28388738DB1A23A2@cheetah.cytomation.com> From: David Van Maren To: metakit@equi4.com Subject: RE: [Metakit] Using head of file Date: Fri, 5 Sep 2003 16:02:09 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C373F9.5A291A80" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C373F9.5A291A80 Content-Type: text/plain Jacob: > There's a reason for having the Metakit abstraction layer, > and you're attempting to break it :) Hmm. Perhaps it is a violation of the abstraction layer to ask Metakit for his FILE *. So, I would think that I need to do an fopen(storagename,"r+") to get it. > I suggest you treat the joined-at-the-hip thing you're trying > to create as a file for the purpose of writing header > information, and as a Metakit storage when using it to do > Metakit-ish things. You cannot get a FILE * from Metakit for > an open storage because how Metakit actually accesses the > storage (note: I did not say file) is its own business. On > some systems Metakit does not use files but rather memory > mappings, so there's no way to do what you are asking for. > > What I'd do: > > * Read/write your data using a FILE * created for that purpose. > * Close it > * Get a c4_Storage from Metakit on this file. Use that. > > I.e. for safety I wouldn't overlap the lifetime of the two > abstractions. So I *think* (JC please correct me) that you > need to be very careful to close the Metakit storage while > you're writing the header data, especially if the amount of > header data will change as a result of writing. Things would > probably go *really* badly for you if held open a Metakit > storage while the location of the start of Metakit data > inside the file would change as a result of prepending header data. It still seems clear to me from what JC wrote that: 1. If a file is not open as a Storage, a. I can fopen("r+"). b. I can shift any of the Storage information, so long as it stays at a fixed location relative to the end of the file. (This means that I can grow or shrink the head.) 2. If a file is open as a Storage, a. I can go ahead and do the fopen("r+") on it. b. I cannot shift any of the Storage information. c. I cannot overwrite any of the Storage information. Am I reading it wrongly? JC, would you care to comment? Thanks, DVM ------_=_NextPart_001_01C373F9.5A291A80 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: [Metakit] Using head of file

Jacob:
 
> There's a reason for having the Metakit = abstraction layer,
> and you're attempting to break it :)

Hmm.  Perhaps it is a violation of the = abstraction layer to ask Metakit for his FILE *.  So, I would = think that I need to do an fopen(storagename,"r+") to get = it.

> I suggest you treat the joined-at-the-hip thing = you're trying
> to create as a file for the purpose of writing = header
> information, and as a Metakit storage when = using it to do
> Metakit-ish things. You cannot get a FILE * = from Metakit for
> an open storage because how Metakit actually = accesses the
> storage (note: I did not say file) is its own = business. On
> some systems Metakit does not use files but = rather memory
> mappings, so there's no way to do what you are = asking for.
>
> What I'd do:
>
> * Read/write your data using a FILE * created = for that purpose.
> * Close it
> * Get a c4_Storage from Metakit on this file. = Use that.
>
> I.e. for safety I wouldn't overlap the lifetime = of the two
> abstractions. So I *think* (JC please correct = me) that you
> need to be very careful to close the Metakit = storage while
> you're writing the header data, especially if = the amount of
> header data will change as a result of writing. = Things would
> probably go *really* badly for you if held open = a Metakit
> storage while the location of the start of = Metakit data
> inside the file would change as a result of = prepending header data.

It still seems clear to me from what JC wrote = that:

1. If a file is not open as a Storage,
   a. I can fopen("r+").
   b. I can shift any of the Storage = information, so long as it
      stays at a fixed = location relative to the end of the file.
      (This means that I = can grow or shrink the head.)

2. If a file is open as a Storage,
   a. I can go ahead and do the = fopen("r+") on it.
   b. I cannot shift any of the Storage = information.
   c. I cannot overwrite any of the = Storage information.

Am I reading it wrongly?  JC, would you care to = comment?

Thanks,
DVM

------_=_NextPart_001_01C373F9.5A291A80-- From ian.fairclough@intelligent-ms.com Sat Sep 6 16:10:39 2003 Received: from scratchy.base.ims ([194.74.0.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h86LAcD23400 for ; Sat, 6 Sep 2003 16:10:38 -0500 Received: from grpware.intelligent-ms.com ([192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id h86Ktfl08034 for ; Sat, 6 Sep 2003 21:55:41 +0100 Received: from imsgsx6tvznbng ([192.168.200.180]) by grpware.intelligent-ms.com (8.11.2/8.11.2) with SMTP id h86KrIu31757 for ; Sat, 6 Sep 2003 21:53:18 +0100 Message-ID: <01b901c374bb$7d299790$802cfc3e@imsgsx6tvznbng> From: "Ian Fairclough" To: Date: Sat, 6 Sep 2003 22:11:46 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_01B6_01C374C3.DCE4E920" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Quick Write Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_01B6_01C374C3.DCE4E920 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable All, I have data represented as a sequence (a metakit view) of a sequence of = name / value pairs (where name and value are both strings) (a metakit = row). The name is the property name to be stored into the metakit row. = The row may contain nested views, in which case the length of the nested = view is also encoded in the sequence of name / value pairs. I have implemented a function that maps each data row into a metakit = row, and stores the row into a metakit view. However, the mapping = function seems to be rather slow and I'm looking for a performance = improvement. I have two questions, are the functions that I'm currently = using represent the optimal way to do this ? Is there an alternative = method of writing this data to a file, that can then be read as a = metakit view ? The sort of mapping functions I have used are as follows : c4_IntProp intProp (data_[offset].name.in()); intProp (row) =3D atoi(data_[offset].value.in()); where row is defined as (example) : dataView =3D db.GetAs (structure); dataView.SetSize (1); c4_RowRef row =3D dataView[0]; Any pointers for performance improvement would be much appreciated. Best regards, Ian Fairclough. ------=_NextPart_000_01B6_01C374C3.DCE4E920 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
All,
 
I have data represented as a sequence = (a metakit=20 view) of a sequence of name / value pairs (where name and value are both = strings) (a metakit row).  The name is the property name to be = stored into=20 the metakit row.  The row may contain nested views, in which = case the=20 length of the nested view is also encoded in the sequence of name / = value=20 pairs.
 
I have implemented a function that maps = each data=20 row into a metakit row, and stores the row into a metakit view.  = However,=20 the mapping function seems to be rather slow and I'm looking for a = performance=20 improvement.  I have two questions, are the functions that I'm = currently=20 using represent the optimal way to do this ?  Is there an = alternative=20 method of writing this data to a file, that can then be read as a = metakit view=20 ?
 
The sort of mapping functions I have = used are as=20 follows :
 
c4_IntProp intProp=20 (data_[offset].name.in());
intProp (row) =3D=20 atoi(data_[offset].value.in());
 
where row is defined as (example) = :
 
dataView =3D db.GetAs = (structure);
dataView.SetSize (1);
c4_RowRef row =3D = dataView[0];
 
Any pointers for performance = improvement would be=20 much appreciated.
 
Best regards,
Ian = Fairclough.
------=_NextPart_000_01B6_01C374C3.DCE4E920-- From jyl@mod3.net Mon Sep 8 00:07:20 2003 Received: from mod3.net (mod3.net [198.78.65.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8857KD08346 for ; Mon, 8 Sep 2003 00:07:20 -0500 Received: (qmail 2872 invoked from network); 8 Sep 2003 05:07:18 -0000 Received: from adsl-209-233-24-218.dsl.snfc21.pacbell.net (HELO mod3.net) (jyl@209.233.24.218) by mod3.net with SMTP; 8 Sep 2003 05:07:18 -0000 Message-ID: <3F5C0E95.9020405@mod3.net> Date: Sun, 07 Sep 2003 22:07:33 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] [ANNOUNCE] e4Graph 1.0a9 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I am pleased to announce the 1.0a9 release of e4Graph, the ninth Alpha release. WHAT IS IT: e4Graph is a C++ library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. The e4Graph package also provides bindings in several other languages, currently Tcl, Python and Java, and allows input/output of object graphs via an XML representation. The e4Graph package is built on top of Metakit 2.4.9.2, and optionally uses Tcl 8.4.4, Python 2.2.3, Java 1.1 or later, and Expat 1.95.5. WHERE TO GET: Downloads: http://sourceforge.net/projects/e4graph/ Homepage: http://www.e4graph.com/e4graph/ Changelog: http://www.e4graph.com/e4graph/changes.txt Installation: http://www.e4graph.com/e4graph/e4install.html WHAT IS NEW: This release is mainly a bug-fix release. A bug in vertex caching that caused incorrect rank information to be returned was fixed. A bug in the parsing and generation of BASE64 encoded strings was also fixed. An installer bug that prevented installation of the Tcl package index file for the tgraph library was fixed. New APIs were added to retrieve the version number for the e4Graph library running now. Each storage now also contains the version number for the e4Graph library used to write it, and new APIs were added to retrieve that information. From erikh2000@yahoo.com Mon Sep 8 15:30:46 2003 Received: from smtp013.mail.yahoo.com (smtp013.mail.yahoo.com [216.136.173.57]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h88KUjD19520 for ; Mon, 8 Sep 2003 15:30:45 -0500 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 8 Sep 2003 20:30:41 -0000 Message-Id: <5.2.0.9.0.20030908131414.00abb8d0@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 08 Sep 2003 13:34:50 -0700 To: metakit@equi4.com From: Erik Hermansen Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Checking for corruption. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I have a very difficult bug to find in my C++ Metakit-based application. The bug only happens intermittently, and my testers haven't yet found a way to reproduce it. I myself have never even seen the bug, despite my efforts to repro it. A .dat file that we are reading and writing from will end up with its structure corrupted, such that no data is retrievable from it. The records aren't just empty. In case anyone is interested, here is the corrupted file: http://www.drod.net/forum/getattachment.php?id=264 I don't know what causes the corruption. I suspect it has something to do with Metakit memory that holds changes before a commit getting overwritten by application code. Obviously, this out-of-bounds write would need to be fixed, but we've yet to narrow down any particular part of the application that might be responsible. And also, this is only my theory about what causes the corruption--it could be something else. I'm wondering if there are any functions that check the integrity of Metakit data that might be useful for troubleshooting. Are there any operations other than c4_Storage::Commit() that write to the .dat file? I'm not using streams. Could low-memory conditions account for this? If so, I'm wondering if I should check for low memory right before a Commit() call. Does anyone have any advice for tracking this problem down? Maybe you've had a similar problem. -Erik From jgodfrey@optinest.com Mon Sep 8 15:47:23 2003 Received: from linux2.optinest.com ([208.188.206.250]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h88KlND21250 for ; Mon, 8 Sep 2003 15:47:23 -0500 Received: from nt40ld ([208.188.206.131]) by linux2.optinest.com (8.11.6/8.11.3) with SMTP id h88KkRc25769 for ; Mon, 8 Sep 2003 15:46:27 -0500 From: "Jeff Godfrey" To: Subject: RE: [Metakit] Checking for corruption. Date: Mon, 8 Sep 2003 15:51:03 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal In-Reply-To: <5.2.0.9.0.20030908131414.00abb8d0@pop.mail.yahoo.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of Erik Hermansen > Sent: Monday, September 08, 2003 3:35 PM > To: metakit@equi4.com > Subject: [Metakit] Checking for corruption. > > > I have a very difficult bug to find in my C++ Metakit-based > application. The bug only happens intermittently, and my testers haven't > yet found a way to reproduce it. I myself have never even seen the bug, > despite my efforts to repro it. A .dat file that we are reading and > writing from will end up with its structure corrupted, such that > no data is > retrievable from it. The records aren't just empty. In case anyone is > interested, here is the corrupted file: Erik, I don't know if it's related at all or not, but I have experienced corrupted Metakit files under the following circumstance: The Metakit application was running under Windows NT and the Metakit storage file was being written to another system across a LAN (and I was using Metakit from Tcl, not C++). In my case, reproducing the problem was *very* touchy, at best. I posted more detailed info about my problems (and the methods I used to reproduce those problems) to this list sometime earlier this year. If any of the above sounds familiar, I can try to dig up my test cases. Ultimately, JC helped me devise a workaround, but as far as I know there is still an issue in Metakit with that specific scenario (Windows / saving storage file across a LAN). Hopefully, JC will have some additional input... Good luck. Jeff From erikh2000@yahoo.com Mon Sep 8 17:18:59 2003 Received: from smtp101.mail.sc5.yahoo.com (smtp101.mail.sc5.yahoo.com [216.136.174.139]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h88MIxD30297 for ; Mon, 8 Sep 2003 17:18:59 -0500 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 8 Sep 2003 22:18:58 -0000 Message-Id: <5.2.0.9.0.20030908150720.00abc5b0@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 08 Sep 2003 15:23:11 -0700 To: "Jeff Godfrey" , From: Erik Hermansen Subject: RE: [Metakit] Checking for corruption. Cc: mrimer@hotmail.com, schikore@nads-sc.uiowa.edu In-Reply-To: References: <5.2.0.9.0.20030908131414.00abb8d0@pop.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: At 03:51 PM 9/8/2003 -0500, Jeff Godfrey wrote: >Erik, > >I don't know if it's related at all or not, but I have experienced corrupted >Metakit files under the following circumstance: > >The Metakit application was running under Windows NT and the Metakit storage >file was being written to another system across a LAN (and I was using >Metakit from Tcl, not C++). > >In my case, reproducing the problem was *very* touchy, at best. I posted >more detailed info about my problems (and the methods I used to reproduce >those problems) to this list sometime earlier this year. > >If any of the above sounds familiar, I can try to dig up my test cases. Thanks Jeff, but it sounds like something different. No network connection is involved here. Here's more info in case anyone can help. This is what a good .dat file looks like before corruption: http://www.nads-sc.uiowa.edu/~schikore/drod/textGood.jpg ...and this is what a .dat file looks like after it gets corrupted: http://www.nads-sc.uiowa.edu/~schikore/drod/text.jpg If you look at the table viewdefs in the second version, there is some garbage towards the end like another viewdef partially appended. The difficulty here is reproducing the problem. If it could be reliably reproduced, then it should be short work to fix it. Insight into causes for this type of corruption would be very useful, because from that we can narrow our testing and code reviews. -Erik From joostkremers@fastmail.fm Tue Sep 9 18:57:52 2003 Received: from amsfep16-int.chello.nl (amsfep16-int.chello.nl [213.46.243.26]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h89NvqD18578 for ; Tue, 9 Sep 2003 18:57:52 -0500 Received: from teuctli.localdomain ([62.194.202.111]) by amsfep16-int.chello.nl (InterMail vM.5.01.05.17 201-253-122-126-117-20021021) with ESMTP id <20030909235745.RBQO24754.amsfep16-int.chello.nl@teuctli.localdomain> for ; Wed, 10 Sep 2003 01:57:45 +0200 Received: from joost by teuctli.localdomain with local (Exim 4.10) id 19wsKF-0007w7-00 for metakit@equi4.com; Wed, 10 Sep 2003 01:55:11 +0200 Date: Wed, 10 Sep 2003 01:55:11 +0200 From: Joost Kremers To: metakit@equi4.com Message-ID: <20030909235511.GT10831@teuctli.arnhem.chello.nl> Mail-Followup-To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-Editor: Emacs of course! Subject: [Metakit] Mk4Py bug? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: hi all, first off, let me know if this is not the right place for questions on Mk4Py. i couldn't find any other place, though. i'm running Mk4Py (from metakit-2.4.9.2+njr0.1) on linux x86, slackware 8.0, with python 2.3, and i'm running into something that appears to be to be a bug. i have this: >>> vw = db.view('entry') >>> db.description('entry') 'stem:S,category:S,id:I' then i try the following: >>> vw.append('h', 'c', 2) 1 >>> vw.append('he', 'c', 2) 2 >>> vw.append('hel', 'c', 2) Traceback (most recent call last): File "", line 1, in ? TypeError: not a number IOW when appending to the view without using keyword arguments, i can only insert strings of two characters or less. if a string contains more than two characters, i get this "TypeError: not a number". anyone know what could be causing this? -- Joost Kremers Life has its moments From njriley@shrug.csl.uiuc.edu Tue Sep 9 19:36:45 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8A0aiD22067 for ; Tue, 9 Sep 2003 19:36:44 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.8/8.12.8) with ESMTP id h8A0aiau088484 for ; Tue, 9 Sep 2003 19:36:44 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.8/8.12.8/Submit) id h8A0ahVV088483 for metakit@equi4.com; Tue, 9 Sep 2003 19:36:43 -0500 (CDT) Date: Tue, 9 Sep 2003 19:36:43 -0500 From: Nicholas Riley To: metakit@equi4.com Subject: Re: [Metakit] Mk4Py bug? Message-ID: <20030910003643.GA88341@uiuc.edu> Mail-Followup-To: metakit@equi4.com References: <20030909235511.GT10831@teuctli.arnhem.chello.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030909235511.GT10831@teuctli.arnhem.chello.nl> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Wed, Sep 10, 2003 at 01:55:11AM +0200, Joost Kremers wrote: > hi all, > > first off, let me know if this is not the right place for questions on > Mk4Py. i couldn't find any other place, though. This is as a good a place as any. > IOW when appending to the view without using keyword arguments, i can > only insert strings of two characters or less. if a string contains more > than two characters, i get this "TypeError: not a number". > > anyone know what could be causing this? This should help explain: >>> import metakit >>> db = metakit.storage() >>> vw = db.getas('entry[stem:S,category:S,id:I]') >>> vw.append('h', 'c', 2) 0 >>> vw.append('he', 'c', 2) 1 >>> metakit.dump(vw) stem category id ---- -------- -- h 0 h e 0 ---- -------- -- Total: 2 rows >>> vw.append('hel', 'c', 2) Traceback (most recent call last): File "", line 1, in ? TypeError: not a number >>> vw.append.__doc__ 'append(obj) -- coerce obj (or keyword args) to row and append, returns pos' >>> vw.append(('hel', 'c', 2)) 2 'append' is only supposed to have a single argument, or keyword arguments; not multiple positional arguments. So you need to pass it a tuple if you want to insert the way you're intending. The reason it works for shorter strings is that Mk4py looks at the first argument, notes it's a sequence (in Python, all strings are sequences), and proceeds to coerce each character of the string to fit into each column. Once the string has more characters than string fields. There are two ways I can think of fixing this (which was on my to-do list anyway, as I ran into the same bug a few weeks ago): 1. Raise an exception on multiple positional arguments. Otherwise, behave as we do now. 2. Treat multiple positional arguments as if they were a single sequence argument. This is a more Pythonic thing to do, I think, as a lot of Python builtins behave this way. What to do with mixes of positional and keyword arguments? Perhaps disallow them initially; I'm not sure I want to do that much work. :) Sorry I haven't submitted my patches; I've been going through a lot of upheaval in my life and haven't had time to work on Mk4py recently. I should have time later this week. -- =Nicholas Riley | From niki@vintech.bg Wed Sep 10 04:16:19 2003 Received: from intra.vintech.l (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8A9GHD32124 for ; Wed, 10 Sep 2003 04:16:18 -0500 Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id 3137F57222; Wed, 10 Sep 2003 12:25:32 +0300 (EEST) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id 9BBBD235D0; Wed, 10 Sep 2003 11:16:09 +0300 (EEST) Message-ID: <3F5EEB93.1080806@vintech.bg> Date: Wed, 10 Sep 2003 12:14:59 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: bg, en MIME-Version: 1.0 To: Nicholas Riley Cc: metakit@equi4.com Subject: Re: [Metakit] Mk4Py bug? References: <20030909235511.GT10831@teuctli.arnhem.chello.nl> <20030910003643.GA88341@uiuc.edu> In-Reply-To: <20030910003643.GA88341@uiuc.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Nicholas Riley wrote: [...] > 'append' is only supposed to have a single argument, or keyword > arguments; not multiple positional arguments. So you need to pass it > a tuple if you want to insert the way you're intending. The reason it > works for shorter strings is that Mk4py looks at the first argument, > notes it's a sequence (in Python, all strings are sequences), and > proceeds to coerce each character of the string to fit into each > column. Once the string has more characters than string fields. > > There are two ways I can think of fixing this (which was on my to-do > list anyway, as I ran into the same bug a few weeks ago): > > 1. Raise an exception on multiple positional arguments. Otherwise, > behave as we do now. > > 2. Treat multiple positional arguments as if they were a single > sequence argument. This is a more Pythonic thing to do, I think, as a > lot of Python builtins behave this way. What to do with mixes of > positional and keyword arguments? Perhaps disallow them initially; > I'm not sure I want to do that much work. :) > Option 1. is consistent with list.append(). Niki Spahiev From dgroth@gmx.de Wed Sep 10 06:39:10 2003 Received: from mail.gmx.net (mail.gmx.de [213.165.64.20]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8ABd9D11797 for ; Wed, 10 Sep 2003 06:39:09 -0500 Date: Wed, 10 Sep 2003 06:39:09 -0500 Message-Id: <200309101139.h8ABd9D11797@trixie.triqs.com> Received: (qmail 17934 invoked by uid 65534); 10 Sep 2003 11:39:04 -0000 Received: from mg01-44.molgen.mpg.de (EHLO MG01-44) (141.14.20.59) by mail.gmx.net (mp010) with SMTP; 10 Sep 2003 13:39:04 +0200 From: "Dr. Detlef Groth" To: metakit@equi4.com Subject: [Metakit] mk4tcl recursive mk::view info Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hello, I tried to walk the structure of a metakit-database: % package require Mk4tcl 2.4.9.2 (dev) 2 % mk::file open db d:/home/dgroth/data/go200304.mk db (dev) 3 % mk::view info db gonames:V godatabase:V goparents:V tree:V book:V (dev) 4 % mk::view info db.gonames go name (dev) 5 % mk::view info db.book name address phones:V date:I # How can I get information about phones ? (dev) 6 % mk::view info db.book.phones ;#this does not work (dev) 7 % mk::view info db.book!2.phones ;# crashes application any suggestions ? regards, Detlef -- Dr. Detlef Groth Max-Planck-Institut fuer Molekulare Genetik Ihnestr. 63/73 D-14195 Berlin Tel.: + 49 30 - 8413 1235g Email: dgroth@molgen.mpg.de Email: dgroth@gmx.de From joostkremers@fastmail.fm Wed Sep 10 07:52:42 2003 Received: from amsfep14-int.chello.nl (amsfep14-int.chello.nl [213.46.243.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8ACqfD18489 for ; Wed, 10 Sep 2003 07:52:41 -0500 Received: from teuctli.localdomain ([62.194.202.111]) by amsfep14-int.chello.nl (InterMail vM.5.01.05.17 201-253-122-126-117-20021021) with ESMTP id <20030910125235.GCXG5521.amsfep14-int.chello.nl@teuctli.localdomain> for ; Wed, 10 Sep 2003 14:52:35 +0200 Received: from joost by teuctli.localdomain with local (Exim 4.10) id 19x4Pu-00085N-00 for metakit@equi4.com; Wed, 10 Sep 2003 14:49:50 +0200 Date: Wed, 10 Sep 2003 14:49:50 +0200 From: Joost Kremers To: metakit@equi4.com Subject: Re: [Metakit] Mk4Py bug? Message-ID: <20030910124950.GU10831@teuctli.arnhem.chello.nl> Mail-Followup-To: metakit@equi4.com References: <20030909235511.GT10831@teuctli.arnhem.chello.nl> <20030910003643.GA88341@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030910003643.GA88341@uiuc.edu> User-Agent: Mutt/1.4.1i X-Editor: Emacs of course! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Tue, Sep 09, 2003 at 07:36:43PM -0500, Nicholas Riley wrote: > On Wed, Sep 10, 2003 at 01:55:11AM +0200, Joost Kremers wrote: > > hi all, > > > > first off, let me know if this is not the right place for questions on > > Mk4Py. i couldn't find any other place, though. > > This is as a good a place as any. ok. in that case, i have another question. (see my other message.) > 'append' is only supposed to have a single argument, or keyword > arguments; not multiple positional arguments. So you need to pass it > a tuple if you want to insert the way you're intending. The reason it > works for shorter strings is that Mk4py looks at the first argument, > notes it's a sequence (in Python, all strings are sequences), and > proceeds to coerce each character of the string to fit into each > column. Once the string has more characters than string fields. yes, it all makes sense now, thanks for the explanation. TBH i have no ideas on what would be the best way to handle multiple positional arguments, so i'll have to leave that up to you. to me, it doesn't really matter... > Sorry I haven't submitted my patches; I've been going through a lot of > upheaval in my life and haven't had time to work on Mk4py recently. I > should have time later this week. please don't feel obliged to apologise. i appreciate all the effort you put into the software. -- Joost Kremers Life has its moments From joostkremers@fastmail.fm Wed Sep 10 08:04:52 2003 Received: from amsfep13-int.chello.nl (amsfep13-int.chello.nl [213.46.243.24]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8AD4pD19871 for ; Wed, 10 Sep 2003 08:04:51 -0500 Received: from teuctli.localdomain ([62.194.202.111]) by amsfep13-int.chello.nl (InterMail vM.5.01.05.17 201-253-122-126-117-20021021) with ESMTP id <20030910130445.UXQR16676.amsfep13-int.chello.nl@teuctli.localdomain> for ; Wed, 10 Sep 2003 15:04:45 +0200 Received: from joost by teuctli.localdomain with local (Exim 4.10) id 19x4bh-00085a-00 for metakit@equi4.com; Wed, 10 Sep 2003 15:02:01 +0200 Date: Wed, 10 Sep 2003 15:02:01 +0200 From: Joost Kremers To: metakit@equi4.com Message-ID: <20030910130201.GV10831@teuctli.arnhem.chello.nl> Mail-Followup-To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-Editor: Emacs of course! Subject: [Metakit] Mk4Py: "dynamic" database Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: hi all, i put the word "dynamic" in quotes, as i'm not sure if it's the right term. i'm working on an application in Python that should allow the user to interactively define a database structure. i.e., i do not know beforehand what the database is going to look like. creating a database this way is not a problem: i can simply build a description string for all the views from the input of the user and then call getas with that: >>> descr = 'entry[stem:S,category:S,id:I]' >>> vw = db.getas(descr) and populate it like this: >>> vw.append(('tree', 'n', 1)) but the problem is that retrieving the information can AFAIK only be done with keywords: >>> print vw[0].stem 'tree' but this is problematic: i do not know beforehand which names the user will choose for the properties/columns. so i would like to be able to do something like this: >>> print vw[0]['stem'] 'tree' so that instead of the string 'stem' i can pass a variable holding a string to index the desired column. this, however, gives an error. i've been trying other ways, and i've been looking at the Mk4Py documentation, but i don't see anything that would make this possible. is there a way to do this? or is there a different way to achieve a "dynamic" database? TIA -- Joost Kremers Life has its moments From 1Jerry@usd1.com Wed Sep 10 09:38:51 2003 Received: from bfish.hurrah.com (root@bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8AEcpD29260 for ; Wed, 10 Sep 2003 09:38:51 -0500 Received: from localhost (12-211-33-9.client.attbi.com [12.211.33.9]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id h8AEcjb13332; Wed, 10 Sep 2003 07:38:45 -0700 Date: Wed, 10 Sep 2003 07:38:36 -0700 From: Jerry McRae <1Jerry@usd1.com> X-Mailer: The Bat! (v1.62r) Personal Reply-To: Jerry McRae X-Priority: 3 (Normal) Message-ID: <1093523298.20030910073836@urd1.com> To: Joost Kremers CC: Metakit Subject: Re: [Metakit] Mk4Py: "dynamic" database In-Reply-To: <20030910130201.GV10831@teuctli.arnhem.chello.nl> References: <20030910130201.GV10831@teuctli.arnhem.chello.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hello Joost, Wednesday, September 10, 2003, 6:02:01 AM, you wrote: JK> >>>> print vw[0]['stem'] JK> 'tree' JK> so that instead of the string 'stem' i can pass a variable holding a string JK> to index the desired column. JK> this, however, gives an error. i've been trying other ways, and i've been JK> looking at the Mk4Py documentation, but i don't see anything that would JK> make this possible. is there a way to do this? or is there a different way JK> to achieve a "dynamic" database? JK> TIA how 'bout... getattr(vw[0],'stem') -- Best regards, Jerry From joostkremers@fastmail.fm Wed Sep 10 10:03:29 2003 Received: from amsfep12-int.chello.nl (amsfep12-int.chello.nl [213.46.243.18]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8AF3SD31547 for ; Wed, 10 Sep 2003 10:03:28 -0500 Received: from teuctli.localdomain ([62.194.202.111]) by amsfep12-int.chello.nl (InterMail vM.5.01.05.17 201-253-122-126-117-20021021) with ESMTP id <20030910150323.KACZ2869.amsfep12-int.chello.nl@teuctli.localdomain> for ; Wed, 10 Sep 2003 17:03:23 +0200 Received: from joost by teuctli.localdomain with local (Exim 4.10) id 19x6C2-00008f-00 for metakit@equi4.com; Wed, 10 Sep 2003 16:43:38 +0200 Date: Wed, 10 Sep 2003 16:43:37 +0200 From: Joost Kremers To: Metakit Subject: Re: [Metakit] Mk4Py: "dynamic" database Message-ID: <20030910144337.GA389@teuctli.arnhem.chello.nl> Mail-Followup-To: Metakit References: <20030910130201.GV10831@teuctli.arnhem.chello.nl> <1093523298.20030910073836@urd1.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1093523298.20030910073836@urd1.com> User-Agent: Mutt/1.4.1i X-Editor: Emacs of course! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Wed, Sep 10, 2003 at 07:38:36AM -0700, Jerry McRae wrote: > >>>> print vw[0]['stem'] > JK> 'tree' > > JK> so that instead of the string 'stem' i can pass a variable holding a string > JK> to index the desired column. > > how 'bout... > getattr(vw[0],'stem') thanks jerry, that works perfectly! and i notice there's also a corresponding setattr, which really makes my day. ;-) (well, i guess the fact that i didn't know about getattr/setattr shows i still have a lot to learn about python... guess i'll just dive into the docs a bit more.) -- Joost Kremers Life has its moments From bkelley@wi.mit.edu Fri Sep 12 09:53:17 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8CErHD09953 for ; Fri, 12 Sep 2003 09:53:17 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003091210531319137 for ; Fri, 12 Sep 2003 10:53:13 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HL3W0P00.OEC; Fri, 12 Sep 2003 10:53:13 -0400 Message-ID: <3F61DE1A.4070908@wi.mit.edu> Date: Fri, 12 Sep 2003 10:54:18 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Joost Kremers CC: Metakit Subject: Re: [Metakit] Mk4Py: "dynamic" database References: <20030910130201.GV10831@teuctli.arnhem.chello.nl> <1093523298.20030910073836@urd1.com> <20030910144337.GA389@teuctli.arnhem.chello.nl> In-Reply-To: <20030910144337.GA389@teuctli.arnhem.chello.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: It gets even better though. class MyView: def __init__(self, view): self._view = view def keys(self): return [p.name for p in self._view.structure()] def has_key(self, key): return key in self.keys() def __getattr__(self, key): return getattr(self._view, key) def __setattr__(self, key, value): if key == "_view": self.__dict__[key] = value return setattr(self._view, key, value) def __len__(self): return len(self._view) def __getitem__(self, item): return self._view[item] import metakit storage = metakit.storage() descr = 'entry[stem:S,category:S,id:I]' view = storage.getas(descr) vw = MyView(view) print vw.keys() # -> ['stem', 'category', 'id'] vw.append(("asdf", "bar", 1)) print len(vw) # -> 1 print vw[0] assert vw.has_key("stem") Why is this useful? Well a user can actually have spaces inside a column name. You can actually retrieve the column name with getattr is as getattr(view, "my dog has fleas") but you couldn't do that with "row.my dog has fleas" for obvious reasons. -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From ryana@worlds-apart.com Mon Sep 15 11:56:38 2003 Received: from hrothgar.worlds-apart.com ([65.102.196.230]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8FGubD30208 for ; Mon, 15 Sep 2003 11:56:37 -0500 Received: from worlds-apart.com (desk4.worlds-apart.com [10.0.0.5]) by hrothgar.worlds-apart.com (8.12.3/8.12.3/Debian-6.4) with ESMTP id h8FGuWjs014502 for ; Mon, 15 Sep 2003 10:56:32 -0600 Message-ID: <3F65EF77.6090600@worlds-apart.com> Date: Mon, 15 Sep 2003 10:57:27 -0600 From: Ryan Anderson User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2.1) Gecko/20021130 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Do loaded rows hang around in memory? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I've just started using metakit, and so far am very happy with its performance and capablities. My question though is about the "load-as-needed" voodoo that it does when loading from a file.... If I load a view from a file and then a bunch of rows from the view, do those rows hang around in memory until I delete the storage object? If so is there anyway to tell metakit not to hold on to them, or to flush them out of memory and load them from disk again next time their needed? -Ryan Anderson Worlds Apart Productions, Inc. From jcw@equi4.com Mon Sep 15 13:20:12 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8FIKBD04523 for ; Mon, 15 Sep 2003 13:20:11 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 4A49383B2E; Mon, 15 Sep 2003 20:20:03 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id A46B483B2C for ; Mon, 15 Sep 2003 20:20:02 +0200 (CEST) Date: Mon, 15 Sep 2003 20:20:26 +0200 Subject: Re: [Metakit] Do loaded rows hang around in memory? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F65EF77.6090600@worlds-apart.com> Message-Id: <4784CFA4-E7A9-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ryan Anderson wrote: > My question though is about the "load-as-needed" voodoo that it does > when loading from a file.... If I load a view from a file and then a > bunch of rows from the view, do those rows hang around in memory until > I > delete the storage object? If so is there anyway to tell metakit not to > hold on to them, or to flush them out of memory and load them from disk > again next time their needed? I've taken this as an opportunity to write a little more about MMF's: http://www.equi4.com/mkmmf.html -jcw From erikh2000@yahoo.com Mon Sep 15 14:47:48 2003 Received: from smtp100.mail.sc5.yahoo.com (smtp100.mail.sc5.yahoo.com [216.136.174.138]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8FJllD11188 for ; Mon, 15 Sep 2003 14:47:48 -0500 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 15 Sep 2003 19:47:43 -0000 Message-Id: <5.2.0.9.0.20030915123251.00ac8960@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 15 Sep 2003 12:52:04 -0700 To: Metakit mailing list From: Erik Hermansen Subject: Re: [Metakit] Do loaded rows hang around in memory? Cc: mrimer@hotmail.com, schikore@nads-sc.uiowa.edu, trick@icculus.org In-Reply-To: <4784CFA4-E7A9-11D7-BDF8-000A9588127E@equi4.com> References: <3F65EF77.6090600@worlds-apart.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Thanks for the information in your article. I read it with interest. I've been trying for several weeks to find an elusive bug where the structure of my storage file is corrupted. I found your memory-mapping information useful, but it dispensed with some of my leads (my premises were "fiction"alized) and now I'm wondering what to check next. A major difficulty is that I put out a build with a new "fix" and takes about a week for one of the testers to report that the same bug is still present. No steps to reproduce have been generated, seemingly because the bug isn't much affected by what the user does in the application. The only pattern I can see is that the corruption usually occurs when the user exits the application. The commit buffers are also memory-mapped files? In which case they couldn't be corrupted by stray writes from my application code, right? Is there the possibility of exiting before writes performed in a commit are finished? The question sounds dumb to me, but I am grasping at straws because I've already tried so many things. The only other hint I have is that the bug was never reported until I split my database into three separate storage files. Another programmer is playing around with the order we perform commits and delete storage objects during the application exit, but each tweak takes about a week to confirm whether it did anything or not. This is the final release-stopping bug after a year of development. -Erik From jcw@equi4.com Mon Sep 15 15:23:33 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8FKNWD14021 for ; Mon, 15 Sep 2003 15:23:33 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 8741483B2E; Mon, 15 Sep 2003 22:23:24 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E70CC83B2C; Mon, 15 Sep 2003 22:23:23 +0200 (CEST) Date: Mon, 15 Sep 2003 22:23:48 +0200 Subject: Re: [Metakit] Do loaded rows hang around in memory? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: mrimer@hotmail.com, schikore@nads-sc.uiowa.edu, trick@icculus.org To: Metakit mailing list From: Jean-Claude Wippler In-Reply-To: <5.2.0.9.0.20030915123251.00ac8960@pop.mail.yahoo.com> Message-Id: <8361E5CC-E7BA-11D7-BDF8-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.5 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_01_02,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Erik Hermansen wrote: [trouble] > The only pattern I can see is that the corruption usually occurs when > the user exits the application. If you have AutoCommit() enabled, then that may be related - it'll commit before exit. Can you copy away the file > The commit buffers are also memory-mapped files? In which case they > couldn't be corrupted by stray writes from my application code, right? No - commit buffers are allocated. They, and administrative info, can still be corrupted. To rule this out requires running in a separate address space, i.e. process. > Is there the possibility of exiting before writes performed in a > commit are finished? The question sounds dumb to me, but I am > grasping at straws because I've already tried so many things. If writes do not finish, the last step of commit is not done, i.e. the file will not be adjusted to use the new state. Any premature exit/end/crash leaves original state intact. I'm saying this under the assumption that there is no bug in MK. If there is, I hope we can find it and resolve it ASAP! > The only other hint I have is that the bug was never reported until I > split my database into three separate storage files. Three different files, c4_Strategy objects, and c4_Storage objects? Should be no problem. There is no longer 100% consistency between the three, i.e. you may see one commit succeed and another fail (e.g. disk full). But none of this can damage datafiles IMO. > Another programmer is playing around with the order we perform commits > and delete storage objects during the application exit, but each tweak > takes about a week to confirm whether it did anything or not. This is > the final release-stopping bug after a year of development. Is it an idea to record & play back the changes to force the problem to the surface? A drastic method would be to instrument all modifying calls to write out a set of instructions, perhaps as a Python or Tcl script. Are you using threads? Are you 100% confident of the stability of the compiler and runtime library? I'm old enough to know not to point fingers at anyone but myself, but it doesn't hurt to rule out the obvious one more time... -jcw From erikh2000@yahoo.com Mon Sep 15 18:15:27 2003 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8FNFQD29579 for ; Mon, 15 Sep 2003 18:15:26 -0500 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 15 Sep 2003 23:15:14 -0000 Message-Id: <5.2.0.9.0.20030915154806.00a922d0@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 15 Sep 2003 16:19:31 -0700 To: Metakit mailing list From: Erik Hermansen Subject: Re: [Metakit] Do loaded rows hang around in memory? Cc: mrimer@hotmail.com, schikore@nads-sc.uiowa.edu, trick@icculus.org In-Reply-To: <8361E5CC-E7BA-11D7-BDF8-000A9588127E@equi4.com> References: <5.2.0.9.0.20030915123251.00ac8960@pop.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: At 10:23 PM 9/15/2003 +0200, Jean-Claude Wippler wrote: >Erik Hermansen wrote: > >[trouble] >>The only pattern I can see is that the corruption usually occurs when the >>user exits the application. > >If you have AutoCommit() enabled, then that may be related - it'll commit >before exit. Not using AutoCommit(). >Can you copy away the file I'm not sure what you were asking here--maybe you forgot to finish a sentence. If you would like to see what the file looks like after it is corrupted, here is a screenshot in a binary viewer of what might be the corrupted area of the storage file: http://www.nads-sc.uiowa.edu/~schikore/drod/text.jpg The repeating text "xt:B]" towards the end of the viewdefs looks suspicious, but on the other hand, it may not be data that is actually used--just some extra bytes that got written. This is a screenshot of what the storage file looked like before: http://www.nads-sc.uiowa.edu/~schikore/drod/textGood.jpg If you are interested I will find a corrupted file and make it available, but of course, you are a busy guy! >>The commit buffers are also memory-mapped files? In which case they >>couldn't be corrupted by stray writes from my application code, right? > >No - commit buffers are allocated. They, and administrative info, can >still be corrupted. To rule this out requires running in a separate >address space, i.e. process. Okay, that helps. It now seems possible to me that I could be corrupting the commit buffers some place with bad writes. Although, it will still be hard to find where, if that is the case. I suppose I could try committing after every write operation so that a failure will occur close to the time when the commit buffer is corrupted. (Assuming it is corrupted.) My app will often go hours without committing writes, because I am waiting to do it when it will not interrupt the user in an obvious way. ... >I'm saying this under the assumption that there is no bug in MK. If there >is, I hope we can find it and resolve it ASAP! Sure. My first guess is that the bug is in my code. >Is it an idea to record & play back the changes to force the problem to >the surface? A drastic method would be to instrument all modifying calls >to write out a set of instructions, perhaps as a Python or Tcl script. That is drastic, as you say, but I can see doing that if we run out of other ideas. >Are you using threads? Not for anything related to database access. We use a 3rd-party sound library which creates threads. >Are you 100% confident of the stability of the compiler and runtime >library? I'm old enough to know not to point fingers at anyone but >myself, but it doesn't hurt to rule out the obvious one more time... It's MSVC 6.0. There are other 3rd-party libraries being used, and no, I'm never 100% confident. Believe me, I don't even think of Metakit as the prime suspect. I am just trying to solve the problem wherever it lies. Thank you for your ideas. I appreciate the help greatly! -Erik From kris@cs.ucsb.edu Tue Sep 16 13:30:52 2003 Received: from letters.cs.ucsb.edu (letters.cs.ucsb.edu [128.111.41.13]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8GIUpD11462 for ; Tue, 16 Sep 2003 13:30:51 -0500 Received: from shawsheen.cs.ucsb.edu (shawsheen.cs.ucsb.edu [128.111.68.232]) by letters.cs.ucsb.edu (8.11.7+Sun/8.11.7) with ESMTP id h8GIUoe27435 for ; Tue, 16 Sep 2003 11:30:50 -0700 (PDT) From: "Kristian G. Kvilekval" To: metakit@equi4.com Content-Type: text/plain Message-Id: <1063737050.11613.1.camel@shawsheen.cs.ucsb.edu> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.4 Date: Tue, 16 Sep 2003 11:30:50 -0700 Content-Transfer-Encoding: 7bit Subject: [Metakit] hash table size in database Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I am using a hash view on a table of about 4K elements. Today I examined the database with the dump utility and noticed that the hash view sometimes has 4K elements and other times it has 8K.. What determines the size of the hash table? -- email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris From jcw@equi4.com Tue Sep 16 14:59:00 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8GJwxD21894 for ; Tue, 16 Sep 2003 14:58:59 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 6C5A883BE1; Tue, 16 Sep 2003 21:58:51 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 00D7483B2F for ; Tue, 16 Sep 2003 21:58:51 +0200 (CEST) Date: Tue, 16 Sep 2003 21:59:17 +0200 Subject: Re: [Metakit] hash table size in database Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <1063737050.11613.1.camel@shawsheen.cs.ucsb.edu> Message-Id: <40C78D80-E880-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.4 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Kristian G. Kvilekval wrote: > I am using a hash view on a table of about 4K elements. > > Today I examined the database with the dump utility > and noticed that the hash view sometimes has 4K elements and > other times it has 8K.. > > What determines the size of the hash table? It's a power of two, and it's always larger than the number of data rows. I've written a bit more about hashed and blocked views on this new page: http://www.equi4.com/mkmapping.html -jcw From kris@cs.ucsb.edu Tue Sep 16 15:14:47 2003 Received: from letters.cs.ucsb.edu (letters.cs.ucsb.edu [128.111.41.13]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8GKElD23729 for ; Tue, 16 Sep 2003 15:14:47 -0500 Received: from shawsheen.cs.ucsb.edu (shawsheen.cs.ucsb.edu [128.111.68.232]) by letters.cs.ucsb.edu (8.11.7+Sun/8.11.7) with ESMTP id h8GKEke08551 for ; Tue, 16 Sep 2003 13:14:46 -0700 (PDT) Subject: Re: [Metakit] hash table size in database From: "Kristian G. Kvilekval" To: metakit@equi4.com In-Reply-To: <40C78D80-E880-11D7-BDF8-000A9588127E@equi4.com> References: <40C78D80-E880-11D7-BDF8-000A9588127E@equi4.com> Content-Type: text/plain Message-Id: <1063743286.11611.62.camel@shawsheen.cs.ucsb.edu> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.4 Date: Tue, 16 Sep 2003 13:14:46 -0700 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Tue, 2003-09-16 at 12:59, Jean-Claude Wippler wrote: > Kristian G. Kvilekval wrote: > > What determines the size of the hash table? > > It's a power of two, and it's always larger than the number of data > rows. I've written a bit more about hashed and blocked views on this > new page: > http://www.equi4.com/mkmapping.html > > -jcw Hmm... that's exactly what prompted the question. I have a database with 4030 entries, but one machine it generates a hash with 4096 and the other with 8192.. Is it checking whether the database fits in memory? Just infinitely curious: ------------ Machine 1: 512KB ram Database Sz: 673725 mk4dump ~/.zinf/db/metadb | fgrep VIEW VIEW 1 rows = dbview:V dbview_H1:V VIEW 4030 rows = url:S type:S title:S artist:S album:S genre:S comment:S track:S year:S VIEW 4097 rows = _H:I _R:I ---------------------------------- Machine 2: 2GB ram Database sz : 689683 VIEW 1 rows = dbview:V dbview_H1:V VIEW 4030 rows = url:S type:S title:S artist:S album:S genre:S comment:S track:S year:S VIEW 8193 rows = _H:I _R:I -- email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris From jcw@equi4.com Tue Sep 16 15:24:03 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8GKO2D24836 for ; Tue, 16 Sep 2003 15:24:02 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id A6D1383BE1; Tue, 16 Sep 2003 22:23:54 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 123C583B2F for ; Tue, 16 Sep 2003 22:23:54 +0200 (CEST) Date: Tue, 16 Sep 2003 22:24:19 +0200 Subject: Re: [Metakit] hash table size in database Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <1063743286.11611.62.camel@shawsheen.cs.ucsb.edu> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.4 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Kristian G. Kvilekval wrote: > Hmm... that's exactly what prompted the question. I have a > database with 4030 entries, but one machine it generates > a hash with 4096 and the other with 8192.. Is it checking > whether the database fits in memory? > > Just infinitely curious: > > ------------ > Machine 1: 512KB ram > Database Sz: 673725 > > mk4dump ~/.zinf/db/metadb | fgrep VIEW > VIEW 1 rows = dbview:V dbview_H1:V > VIEW 4030 rows = url:S type:S title:S artist:S album:S genre:S > comment:S track:S year:S > VIEW 4097 rows = _H:I _R:I > > ---------------------------------- > Machine 2: 2GB ram > Database sz : 689683 > > VIEW 1 rows = dbview:V dbview_H1:V > VIEW 4030 rows = url:S type:S title:S artist:S album:S genre:S > comment:S track:S year:S > VIEW 8193 rows = _H:I _R:I No, "fits in memory" is not considered. What does matter is the order or adds/deletes. Space is reclaimed and re-used, when fill drops too low - there is hysteresis, i.e. the same number of rows can have a different hash table size depending on how entries were added and deleted. Same build order, different platform? (if so, there could be a bug) -jcw From kris@cs.ucsb.edu Tue Sep 16 15:52:27 2003 Received: from letters.cs.ucsb.edu (letters.cs.ucsb.edu [128.111.41.13]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8GKqRD28244 for ; Tue, 16 Sep 2003 15:52:27 -0500 Received: from shawsheen.cs.ucsb.edu (shawsheen.cs.ucsb.edu [128.111.68.232]) by letters.cs.ucsb.edu (8.11.7+Sun/8.11.7) with ESMTP id h8GKqPe13019 for ; Tue, 16 Sep 2003 13:52:25 -0700 (PDT) Subject: Re: [Metakit] hash table size in database From: "Kristian G. Kvilekval" To: Metakit mailing list In-Reply-To: References: Content-Type: text/plain Message-Id: <1063745545.11611.71.camel@shawsheen.cs.ucsb.edu> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.4 Date: Tue, 16 Sep 2003 13:52:25 -0700 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Tue, 2003-09-16 at 13:24, Jean-Claude Wippler wrote: > Kristian G. Kvilekval wrote: > > No, "fits in memory" is not considered. > > What does matter is the order or adds/deletes. Space is reclaimed and > re-used, when fill drops too low - there is hysteresis, i.e. the same > number of rows can have a different hash table size depending on how > entries were added and deleted. > > Same build order, different platform? (if so, there could be a bug) Both systems are: x86 linux 2.4.22 debian/unstable Entries should be unique: they are urls taken from a local directory hierarchy and the test directory is actually a mirror. No duplicates, no deletes, but insert order could be different. Still I notice that there was some discrepancy in the size of the database so that suggests something was different. -- email:kris@cs.ucsb.edu office:(805)893-4276 http://www.cs.ucsb.edu/~kris From jcw@equi4.com Tue Sep 16 16:02:07 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8GL26D29515 for ; Tue, 16 Sep 2003 16:02:06 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 3C3AD83BE1; Tue, 16 Sep 2003 23:01:58 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B900083B2F for ; Tue, 16 Sep 2003 23:01:57 +0200 (CEST) Date: Tue, 16 Sep 2003 23:02:23 +0200 Subject: Re: [Metakit] hash table size in database Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <1063745545.11611.71.camel@shawsheen.cs.ucsb.edu> Message-Id: <11C701A7-E889-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.5 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Kristian G. Kvilekval wrote: > No duplicates, no deletes, but insert order could be different. Then hash collisions can differ. Hence resizing decisions can differ. Therefore database sizes (and internal order of free space re-use) can differ. > Still I notice that there was some discrepancy in the size of the > database so that suggests something was different. This is normal with hashing, for the reasons described above. -jcw From njriley@shrug.csl.uiuc.edu Wed Sep 17 01:25:29 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8H6PSD26979 for ; Wed, 17 Sep 2003 01:25:28 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.8/8.12.8) with ESMTP id h8H6PSnL045368 for ; Wed, 17 Sep 2003 01:25:28 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.8/8.12.8/Submit) id h8H6PRHg045367 for metakit@equi4.com; Wed, 17 Sep 2003 01:25:27 -0500 (CDT) Date: Wed, 17 Sep 2003 01:25:27 -0500 From: Nicholas Riley To: metakit@equi4.com Message-ID: <20030917062527.GA45228@uiuc.edu> Mail-Followup-To: metakit@equi4.com References: <20030909235511.GT10831@teuctli.arnhem.chello.nl> <20030910003643.GA88341@uiuc.edu> <3F5EEB93.1080806@vintech.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3F5EEB93.1080806@vintech.bg> User-Agent: Mutt/1.4.1i Subject: [Metakit] Mk4py behavioral questions [was Re: Mk4Py bug?] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Wed, Sep 10, 2003 at 12:14:59PM +0300, Niki Spahiev wrote: > Nicholas Riley wrote: > >'append' is only supposed to have a single argument, or keyword > >arguments; not multiple positional arguments. So you need to pass it > >a tuple if you want to insert the way you're intending. The reason it > >works for shorter strings is that Mk4py looks at the first argument, > >notes it's a sequence (in Python, all strings are sequences), and > >proceeds to coerce each character of the string to fit into each > >column. Once the string has more characters than string fields. > > > >1. Raise an exception on multiple positional arguments. Otherwise, > >behave as we do now. > > Option 1. is consistent with list.append(). I agree. This now generates an error. >>> vw.append(1, 2) Traceback (most recent call last): File "", line 1, in ? TypeError: append() takes exactly one argument, or multiple keyword arguments As I've been working on this further over the past few days, I have another behavioral question: what to do with data that can't be obviously coerced into a row? >>> vw.append(1) 3 This generates no error because PyView::makeRow (which eventually gets called) is invoked with useDefaults=true, and so an essentially empty row is inserted. Here's another case: >>> vw = db.getas('ab[x:S,y:S]') >>> vw.append('abcdefg') 0 >>> mk.dump(vw) x y - - a b - - Total: 1 rows makeRow does the following, essentially: - If the object being inserted is a dictionary, or keyword arguments, map between property names and values. Currently, it will discard invalid keywords/dictionary keys, but I changed it so it generates an error. - Otherwise, iterate through the properties of the view. Populate each property in the new row as follows: - If the object is an instance, attempt to get the attribute whose name matches the property. Generate an error if the type doesn't match, but not if the attribute doesn't exist. - If the object is a sequence, get the property item corresponding to the index. If there are more elements in the sequence than properties, silently discard the remainder of the elements. (Should this change?) - If the object is neither, set the default value and generate no error. (Should this change?) I'm working under the assumption that, given a database, you'd prefer it to generate an error rather than discard data. The object instance mapping is a special case, one I probably wouldn't have implemented myself, but since it's there I don't want to break it. The cases above are most likely ones where the user does -not- want the data to be lost. But it seems large parts of Mk4py were designed under the assumption that any non-fitting data should be discarded silently. I don't particularly like this behavior, personally, but I'm not writing this software for myself alone :-). Given I don't want to keep "fixing" stuff that other people might not consider broken, I'd appreciate some feedback before I continue. Regardless of these issues, I've still got some more work to do before I'm finished with Mk4py; primarily I need to fix integer overflow handling in Python 2.3, and hack a bit on the build scripts if I have time. I hope to have a patch ready by the end of the week. -- =Nicholas Riley | From rcohen@dial.oleane.com Wed Sep 17 03:10:20 2003 Received: from bob.articque.com (free4.articque.com [195.25.225.74] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8H8AJD03182 for ; Wed, 17 Sep 2003 03:10:20 -0500 Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id 3FCEABF1F for ; Wed, 17 Sep 2003 09:19:10 +0200 (CEST) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 21765-5FB54BFB; Wed, 17 Sep 2003 09:19:10 +0200 Received: by bob.articque.com (Postfix, from userid 1003) id D6004BF23; Wed, 17 Sep 2003 09:19:09 +0200 (CEST) Received: from dial.oleane.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id D3D7CBF1F for ; Wed, 17 Sep 2003 09:19:06 +0200 (CEST) Message-ID: <3F681697.4040003@dial.oleane.com> Date: Wed, 17 Sep 2003 10:08:55 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com References: <20030909235511.GT10831@teuctli.arnhem.chello.nl> <20030910003643.GA88341@uiuc.edu> <3F5EEB93.1080806@vintech.bg> <20030917062527.GA45228@uiuc.edu> In-Reply-To: <20030917062527.GA45228@uiuc.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-1.8 required=4.0 tests=BAYES_30,IN_REP_TO,REFERENCES,SIGNATURE_LONG_SPARSE, USER_AGENT_MOZILLA_UA version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.21.0.1; VDF: 6.21.0.44; host: bob) Subject: [Metakit] storing all as byte array Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi I have several complexe structures to store (multiplateform path information, color rgb/hatching/dash, xyz vector, unicode 4bytes strings etc.) and also simple data (long, double etc.) I am building a class easy to maintain and easy to use, that store any of my data type as a byte array (c4_BytesProp). This is easyer for me to pack/unpack all kind of data in the same manner, and have always the same data type in my metakit. Especially I can easily manage endian, encoding etc.. for each part of the complex structures. I wonder if the c4_View.Select() function have good enough performances with bytes, compared to using simple data like c4_StringProp. I experience slow process in this function, and wonder if this is because of the c4_BytesProp type. This is the kind of actions that seems slow to me : c4_View view=storage->GetAs("mytable[vector:B,position:B]"); c4_Row row; c4_BytesProp prp("vector"); prp(row)=c4_Bytes(buf,(long)buflen,true);//make a copy c4_View selection=view.Select(row); After profiling my app, I found that 90% if the time is spent on c4_FilterSeq::Match() while it is only 20% of the work to do (after selecting rows, I do a lot of things). Time function + children (%): 99,2 art_geocod::make_it_all) (geocod.obj) 96,6 art_geocod::search() (geocod.obj) 96,5 art_db::select_start(class art_db_record const *,short) (adb.obj) 91,8 c4_View::Select(class c4_RowRef const &) (view.obj) 91,8 f4_CreateFilter(class c4_Sequence &,class c4_Cursor,class c4_Cursor) (derived.obj) 91,8 c4_FilterSeq::c4_FilterSeq(class c4_Sequence &,class c4_Cursor,class c4_Cursor) (derived.obj) 88,7 c4_FilterSeq::Match(int,class c4_Sequence &,int const *,int const *) (derived.obj) 38,4 c4_Handler::Compare(int,class c4_Bytes const &) (handler.obj) 34,2 c4_Handler::GetBytes(int,class c4_Bytes &,bool) (handler.obj) 31 c4_FormatB::Get(int,int &) (format.obj) 16,6 c4_Column::FetchBytes(long,int,class c4_Bytes &,bool) (column.obj) 15,5 f4_CompareFormat(char,class c4_Bytes const &,class c4_Bytes const &) (format.obj) 9,6 c4_ColIter::Next(void) (column.obj) 6,4 c4_Bytes::operator=(class c4_Bytes const &) (table.obj) 4,8 c4_Bytes::c4_Bytes(void const *,int,bool) (table.obj) etc..... Time function alone (%): 15,5 f4_CompareFormat(char,class c4_Bytes const &,class c4_Bytes const &) (format.obj) 12 c4_FilterSeq::Match(int,class c4_Sequence &,int const *,int const *) (derived.obj) 11,6 c4_FormatB::Get(int,int &) (format.obj) 9,6 c4_ColIter::Next(void) (column.obj) 7 c4_Column::FetchBytes(long,int,class c4_Bytes &,bool) (column.obj) 6,4 c4_Bytes::operator=(class c4_Bytes const &) (table.obj) 5,9 c4_Handler::Compare(int,class c4_Bytes const &) (handler.obj) 4,8 c4_Bytes::c4_Bytes(void const *,int,bool) (table.obj) 4,6 c4_Handler::GetBytes(int,class c4_Bytes &,bool) (handler.obj) 3,7 c4_HandlerSeq::NthHandler(int) (i) 2,8 c4_Sequence::Buffer(void) (viewx.obj) etc..... I suspect f4_CompareFormat, c4_FilterSeq::Match and c4_FormatB::Get to be particularly slow with BytesProp... Am I right ? or would it be the same with string ?? (PS: my bytes arrays are never bigger than 110 bytes) Thanks for any information -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From gmcm@hypernet.com Wed Sep 17 06:48:33 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8HBmWD23205 for ; Wed, 17 Sep 2003 06:48:32 -0500 Received: from PARANOIA (204.176.40.71) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Wed, 17 Sep 2003 07:49:18 -0400 From: "Gordon McMillan" To: Nicholas Riley , metakit@equi4.com Date: Wed, 17 Sep 2003 07:46:58 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Mk4py behavioral questions [was Re: Mk4Py bug?] Reply-to: gmcm@hypernet.com Message-ID: <3F681172.19708.181817EF@localhost> Priority: normal In-reply-to: <20030917062527.GA45228@uiuc.edu> References: <3F5EEB93.1080806@vintech.bg> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 17 Sep 2003 at 1:25, Nicholas Riley wrote: [...] > I'm working under the assumption that, given a database, you'd > prefer it to generate an error rather than discard data. Not at all. When working with GUI forms, I often use a form dict which may well have extra state I don't want persisted. You'll realize you're discarding data soon enough, and adding columns is incredibly easy. I, on the other hand, would have to add a bunch of code just to avoid a paternalistic error. If I liked doing that, I'd be using Java . BTW, doing type checking code that works across Python versions has gotten very difficult. Mk4py still works with Python 1.5.2, and at least a couple of us have virtual hosts still running 1.5.2. -- Gordon http://www.mcmillan-inc.com/ From jcw@equi4.com Wed Sep 17 07:19:58 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8HCJwD26617 for ; Wed, 17 Sep 2003 07:19:58 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 00CE283C02; Wed, 17 Sep 2003 14:19:50 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 65F8E83C00 for ; Wed, 17 Sep 2003 14:19:50 +0200 (CEST) Date: Wed, 17 Sep 2003 14:20:17 +0200 Subject: Re: [Metakit] Mk4py behavioral questions [was Re: Mk4Py bug?] Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F681172.19708.181817EF@localhost> Message-Id: <4C09580E-E909-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.5 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: > On 17 Sep 2003 at 1:25, Nicholas Riley wrote: > > [...] > >> I'm working under the assumption that, given a database, you'd >> prefer it to generate an error rather than discard data. > > Not at all. When working with GUI forms, I often use a form dict > which may well have extra state I don't want persisted. "Temporary properties", is what I usually call 'em... Yes, they are in fact extremely useful - you can have a view with props a, b, c - then open it with "getas a,b,d", then have c still linger around, copy c's to d's, say to convert, then commit. The result is a view with a,b,d. Properties which are restructured away like this, and properties not in the getas are temp props - they disappear on commit (and rollback). I think you can even add a prop, and do a getas after-the-fact to make it persist. Also, properties which do not persist offer a way to cache additional info for each row. -jcw From njriley@shrug.csl.uiuc.edu Wed Sep 17 08:42:51 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8HDgoD01727 for ; Wed, 17 Sep 2003 08:42:51 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.8/8.12.8) with ESMTP id h8HDgnnL046953; Wed, 17 Sep 2003 08:42:49 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.8/8.12.8/Submit) id h8HDgirv046952; Wed, 17 Sep 2003 08:42:44 -0500 (CDT) Date: Wed, 17 Sep 2003 08:42:44 -0500 From: Nicholas Riley To: Gordon McMillan Cc: metakit@equi4.com Subject: Re: [Metakit] Mk4py behavioral questions [was Re: Mk4Py bug?] Message-ID: <20030917134243.GA46876@uiuc.edu> Mail-Followup-To: Gordon McMillan , metakit@equi4.com References: <3F5EEB93.1080806@vintech.bg> <3F681172.19708.181817EF@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3F681172.19708.181817EF@localhost> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Wed, Sep 17, 2003 at 07:46:58AM -0400, Gordon McMillan wrote: > On 17 Sep 2003 at 1:25, Nicholas Riley wrote: > > [...] > > > I'm working under the assumption that, given a database, you'd > > prefer it to generate an error rather than discard data. > > Not at all. When working with GUI forms, I often use a form dict > which may well have extra state I don't want persisted. This addresses one of the three cases I mentioned (unfortunately the one I've implemented; oh well). What about the others - view.append(1) and view.append('abcdefg') (where you have fewer than 7 properties)? > You'll realize you're discarding data soon enough, and adding > columns is incredibly easy. I, on the other hand, would have to > add a bunch of code just to avoid a paternalistic error. If I liked > doing that, I'd be using Java . Understood about Java; that's why I'm using Python in the first place ;-) Still, I am trying to approximate the level of error checking of Python's builtins. ZODB, which I've used in the past, has a different mechanism for specifying which attributes of an object are persisted, beyond simply ignoring unspecified attributes. > BTW, doing type checking code that works across Python > versions has gotten very difficult. Mk4py still works with > Python 1.5.2, and at least a couple of us have virtual hosts > still running 1.5.2. I didn't realize I needed to test that far back. What a mess. It'd be nice if this minimum requirement were documented somewhere more obvious. What Python versions do you test with? I know I at least need to do 2.2 and 2.3 to test for varying Unicode and integer handling, but is 1.5.2 adequate in addition or do I need to test more intermediate versions such as 2.0, 2.1, etc.? -- =Nicholas Riley | From niki@vintech.bg Wed Sep 17 09:07:32 2003 Received: from intra.vintech.l (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8HE7VD04252 for ; Wed, 17 Sep 2003 09:07:31 -0500 Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id D902757222; Wed, 17 Sep 2003 17:16:51 +0300 (EEST) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id ACD94235D0; Wed, 17 Sep 2003 17:07:24 +0300 (EEST) Message-ID: <3F686A4B.7040507@vintech.bg> Date: Wed, 17 Sep 2003 17:06:03 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: bg, en MIME-Version: 1.0 To: Nicholas Riley Cc: metakit@equi4.com Subject: Re: [Metakit] Mk4py behavioral questions [was Re: Mk4Py bug?] References: <20030909235511.GT10831@teuctli.arnhem.chello.nl> <20030910003643.GA88341@uiuc.edu> <3F5EEB93.1080806@vintech.bg> <20030917062527.GA45228@uiuc.edu> In-Reply-To: <20030917062527.GA45228@uiuc.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Nicholas Riley wrote: > - If the object is a sequence, get the property item corresponding > to the index. If there are more elements in the sequence than > properties, silently discard the remainder of the elements. > (Should this change?) Currently this sequence check is broken for newstyle classes. I posted a patch about this. Niki Spahiev From gmcm@hypernet.com Wed Sep 17 12:59:41 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8HHxeD29418; Wed, 17 Sep 2003 12:59:41 -0500 Received: from PARANOIA (204.176.40.71) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Wed, 17 Sep 2003 14:00:28 -0400 From: "Gordon McMillan" To: Nicholas Riley , metakit@equi4.com Date: Wed, 17 Sep 2003 12:08:47 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Mk4py behavioral questions [was Re: Mk4Py bug?] Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3F684ECF.6653.1907C990@localhost> Priority: normal In-reply-to: <20030917134243.GA46876@uiuc.edu> References: <3F681172.19708.181817EF@localhost> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 17 Sep 2003 at 8:42, Nicholas Riley wrote: > ... What about the others - view.append(1) and > view.append('abcdefg') (where you have fewer than 7 properties)? I'm not happy that those act as they do. view.append() (that is, no arg or None) should definitely work (it's the common idiom for xbase people), and I agree the others should fail (unless you want to argue about the view-with-only-one-property case). But with the changes in Python's typing system, it's hard to find a common way to catch those. (I think it was 2.1 when Py_SequenceCheck starting returning true on instances, which required reordering the code.) [...] > Understood about Java; that's why I'm using Python in the > first place ;-) Still, I am trying to approximate the level of > error checking of Python's builtins. ZODB, which I've used in > the past, has a different mechanism for specifying which > attributes of an object are persisted, beyond simply ignoring > unspecified attributes. Let's say that creating that API (partly by design, partly by blunder ;-)) has made me deeply appreciate what Guido does. We've received patches from people who wanted array access (so row[i] was property i) and people who wanted hash access (view[key] as in dict access). We probably get more questions about why it's row.property and not row['property'] than anything else. More error checking is good (there's a lot more now than there used to be), but please be careful of semantics - what seems natural to one person may be an atrocity to another. > > BTW, doing type checking code that works across Python > > versions has gotten very difficult. Mk4py still works with > > Python 1.5.2, and at least a couple of us have virtual hosts > > still running 1.5.2. > > I didn't realize I needed to test that far back. What a mess. > It'd be nice if this minimum requirement were documented > somewhere more obvious. > > What Python versions do you test with? I know I at least > need to do 2.2 and 2.3 to test for varying Unicode and integer > handling, but is 1.5.2 adequate in addition or do I need to > test more intermediate versions such as 2.0, 2.1, etc.? 1.5.2 is a special case because of RH and all the hosts who don't upgrade. 2.0 (and 1.6) are non-issues - no one runs them. There are probably a few still maintaining 2.1 (I've still got a 2.1), but I wouldn't be too concerned. For 1.5.2, it's fine if new features / error checking are ifdef-ed out (if they don't work). I'd really just like to make sure they can bind to an up-to-date Metakit, and I hate maintaining branches. -- Gordon http://www.mcmillan-inc.com/ From bkelley@wi.mit.edu Wed Sep 17 14:55:28 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8HJtRD08123 for ; Wed, 17 Sep 2003 14:55:28 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003091715552032279 for ; Wed, 17 Sep 2003 15:55:20 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HLDJC900.GKJ; Wed, 17 Sep 2003 15:55:21 -0400 Message-ID: <3F68BC65.3050903@wi.mit.edu> Date: Wed, 17 Sep 2003 15:56:21 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gmcm@hypernet.com, metakit@equi4.com Subject: Re: [Metakit] Mk4py behavioral questions [was Re: Mk4Py bug?] References: <3F681172.19708.181817EF@localhost> <3F684ECF.6653.1907C990@localhost> In-Reply-To: <3F684ECF.6653.1907C990@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I'll just put in my two cents here, for cases where someone would like to make metakit on python to behave differently, it is relatively easy to create wrapper classes that behave like you want. Gordon McMillan wrote: >Let's say that creating that API (partly by design, partly by >blunder ;-)) has made me deeply appreciate what Guido does. >We've received patches from people who wanted array access >(so row[i] was property i) and people who wanted hash access >(view[key] as in dict access). We probably get more questions >about why it's row.property and not row['property'] than anything >else. > Believe it or not, metakit allows spaces in column names (one can also start with a letter) this means that row.property will not work for these columns. However, getattr(row, property) will. The solution, of course, is to now what the column names are which you can do with view.structure(). Of course, you could preparse the table format to look for errors. For instance I use a lot of foreign keys in form "othertable_id" which should only be populated with valid entries from the othertable view. I would like row.othertable_id = 27 to fail if 27 is not a valid othertable_id. Likewise, it is nice to have row.othertable return the othertable row corresponding to othertable_id. Naturally I don't expect metakit to supply this functionality, but it is actually easy to implement. Attached is code that you can use as a substitute to the metakit wrapper, you can hack away to your hearts content adding your own "append" functions, foreign key look-ups like I have described, parse the db.getas() calls so that all columns have valid python property names or what not. With a modicum of work you could implement all of Gordon's examples of what people want using this wrapper class and the rest will behave as normal. It behaves exactly like metakit with a few differences: print view -> outputs the metakit dump version of a view, useful for debugging view.columns() -> list the attributes (columns) of the view print row -> prints the contents of a row row.asDict() -> returns a dictionary of the row's values. The downside is that you can't have an attribute "asDict" in the row. I hereby enter metawrap into the public domain. Feel free to do whatever you want with this code. It is a bit like the old UserList and UserDict's of python past. Happy hacking. ==== metawrap Code Starts Here ===== """usage: import metawrap as metakit use just like metakit with the following changes: print view -> outputs the metakit dump version of a view, useful for debugging view.columns() -> list the attributes (columns) of the view print row -> prints the contents of a row row.asDict() -> returns a dictionary of the row's values. The downside is that you can't have an attribute "asDict" in the row. """ import metakit, sys, StringIO # Implementation notes: The hardest thing to understand here is the # _view class below. The trick is that sometimes a metakit function # returns a seperate view which we would also like to wrap up in our # handy view class. # # This is done by "intercepting" these functions (see VIEW_RETURNS below) # with a special class named _view # This class keeps track of the view associated with the function and # the name of the function called. # When the function is actually called, _view.__call__ is called # with the appropriate parameters. The result of applying the # function with these parameters is then wrapped up in a view class # and returned. *whew*! # # you can always add methods to replace these though, for # example you can add a view.append() method to replace the default # append. class MetaError(Exception): pass def dump(view): stdout = sys.stdout sys.stdout = StringIO.StringIO() metakit.dump(view) text = sys.stdout.getvalue() sys.stdout = stdout return text class storage: def __init__(self, filename=None, mode=None): if filename and mode: self.__db = metakit.storage(filename, mode) elif filename and not mode: raise MetaError("Need to supply both a filename and a mode") else: self.__db = metakit.storage() def __getattr__(self, attribute): return getattr(self.__db, attribute) def getas(self, table): return view(self.__db.getas(table)) def view(self, viewname): return view(self.__db.view(viewname)) # these functions return views as well VIEW_RETURNS = ["indices", "rview", "select", "project", "flatten", "join", "unique", "intersect", "different", "minus", "remapwith", "pair", "rename", "product", "groupby", "counts", "blocked", "hashed", "sort", "sortrev"] class _view: """This class wraps views returned by view method functions.""" def __init__(self, vw, st, funcname): self.vw = vw self.st = st self.func = func def __call__(self, *a, **kw): return view(vw=apply(getattr(self.vw, self.funcname), a, kw), st=self.st) class view: def __init__(self, vw=None, st=None): if vw is None: vw = metakit.view() self.__view = vw self.__storage = storage def __getattr__(self, attribute): if attribute in VIEW_RETURNS: return _view(self.__view, self.__storage, attribute) else: return getattr(self.__view, attribute) def __len__(self): return len(self.__view) def __getslice__(self, lo, hi): return view(self.__view[lo:hi]) def __delslice__(self, lo, hi): del self.__view[lo:hi] def __getitem__(self, index): return row(self.__view[index], self) def __str__(self): return dump(self) def columns(self): return [x.name for x in self.__view.structure()] class row: def __init__(self, rw, vw): self.__rw = rw self.__vw = vw def __repr__(self): res = ["Row with attributes:"] for prop in self.__vw.structure(): res.append("\t%s:\t%s"%(prop.name, getattr(self.__rw, prop.name))) return "\n".join(res) def __getattr__(self, column): if column[0:6] == "_row__": return self.__dict__[column] else: return getattr(self.__rw, column) def __setattr__(self, column, value): if column[0:6] == "_row__": self.__dict__[column] = value else: setattr(self.__rw, column, value) def asDict(self): res = {} for prop in self.__vw.structure(): res[prop.name] = getattr(self.__rw, prop.name) return res ===Code Ends Here=== -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From bkelley@wi.mit.edu Wed Sep 17 15:16:12 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8HKGCD10258 for ; Wed, 17 Sep 2003 15:16:12 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003091716160832540 for ; Wed, 17 Sep 2003 16:16:08 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HLDKAV00.2L1 for ; Wed, 17 Sep 2003 16:16:07 -0400 Message-ID: <3F68C144.5090304@wi.mit.edu> Date: Wed, 17 Sep 2003 16:17:08 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Mk4py behavioral questions [was Re: Mk4Py bug?] References: <3F681172.19708.181817EF@localhost> <3F684ECF.6653.1907C990@localhost> <3F68BC65.3050903@wi.mit.edu> In-Reply-To: <3F68BC65.3050903@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: It looks the formatting got a little confused: you can download the code from here: http://jura.wi.mit.edu/people/kelley/ or directly http://jura.wi.mit.edu/people/kelley/metawrap.py -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From rcohen@dial.oleane.com Thu Sep 18 04:38:52 2003 Received: from bob.articque.com (free4.articque.com [195.25.225.74] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8I9cpD09912 for ; Thu, 18 Sep 2003 04:38:51 -0500 Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id 2093BBF1F for ; Thu, 18 Sep 2003 10:47:41 +0200 (CEST) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 01515-7DC1C3CC; Thu, 18 Sep 2003 10:47:40 +0200 Received: by bob.articque.com (Postfix, from userid 1003) id A2114BF23; Thu, 18 Sep 2003 10:47:40 +0200 (CEST) Received: from dial.oleane.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id D4CAFBF1F for ; Thu, 18 Sep 2003 10:47:37 +0200 (CEST) Message-ID: <3F697CD7.4030409@dial.oleane.com> Date: Thu, 18 Sep 2003 11:37:27 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-1.4 required=4.0 tests=BAYES_30,SIGNATURE_LONG_SPARSE,USER_AGENT_MOZILLA_UA version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.21.0.1; VDF: 6.21.0.46; host: bob) Subject: [Metakit] beginners hash view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I built a view of 50k records that I need to access. With normal view it is too slow (180 ms), so I try with hash view that I never used, and the result is even slower !(250 ms) ! (I've just read the new page http://www.equi4.com/mkmapping.html) Here is what I've done : c4_View view=db.GetAs("table[key:S,val:S]"),selection; c4_View viewsec=db.GetAs("sec[_H:I,_R:I]"); c4_View viewhash=view.Hash(viewsec,1); c4_Row row,searchrow; c4_StringProp val("val"); c4_StringProp key("key"); for (idx=0;idx; Thu, 18 Sep 2003 04:48:41 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 4E89383C03; Thu, 18 Sep 2003 11:48:34 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id DDC6B83C01 for ; Thu, 18 Sep 2003 11:48:33 +0200 (CEST) Date: Thu, 18 Sep 2003 11:49:02 +0200 Subject: Re: [Metakit] beginners hash view Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F697CD7.4030409@dial.oleane.com> Message-Id: <55CBC021-E9BD-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.8 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_01_02,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > I built a view of 50k records that I need to access. With normal view > it is too slow (180 ms), so I try with hash view that I never used, > and the result is even slower !(250 ms) ! > (I've just read the new page http://www.equi4.com/mkmapping.html) > > Here is what I've done : > > c4_View view=db.GetAs("table[key:S,val:S]"),selection; > c4_View viewsec=db.GetAs("sec[_H:I,_R:I]"); > c4_View viewhash=view.Hash(viewsec,1); > c4_Row row,searchrow; > c4_StringProp val("val"); > c4_StringProp key("key"); > for (idx=0;idx { > sprintf(st1,"%d%d%d%d",idx,idx,idx,idx); > sprintf(st2,"%d%d%d%d",idx,idx,idx,idx); > key(row)=st1; > val(row)=st2; > view.Add(row); No! Do not touch view when there's a hash mapping on top. Use viewhash: viewhash.Add(row) > } > db.Commit(); > key(searchrow)="5555"; > selection=viewhash.Select(searchrow); > > what's wrong ?? is there any sample code ? What are you measuring - total time of the above code? Yes, that will be slower, it is now also setting up hashes along the way. But the select should be instant. One way to avoid the above confusion, it to rewrite your code a bit more and use: view=view.Hash(viewsec,1); at the top, replacing the "viewhash" declaration. In other words, hide the original view once you've set up a hash on it. -jcw From rcohen@dial.oleane.com Thu Sep 18 07:28:10 2003 Received: from bob.articque.com (free4.articque.com [195.25.225.74] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8ICS9D22056 for ; Thu, 18 Sep 2003 07:28:09 -0500 Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id AB224BF1F for ; Thu, 18 Sep 2003 13:36:58 +0200 (CEST) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 03267-0E606685; Thu, 18 Sep 2003 13:36:58 +0200 Received: by bob.articque.com (Postfix, from userid 1003) id 3EF38BF23; Thu, 18 Sep 2003 13:36:58 +0200 (CEST) Received: from dial.oleane.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id A49ADBF1F for ; Thu, 18 Sep 2003 13:36:54 +0200 (CEST) Message-ID: <3F69A482.1060704@dial.oleane.com> Date: Thu, 18 Sep 2003 14:26:42 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] beginners hash view References: <55CBC021-E9BD-11D7-BDF8-000A9588127E@equi4.com> In-Reply-To: <55CBC021-E9BD-11D7-BDF8-000A9588127E@equi4.com> Content-Type: multipart/mixed; boundary="------------060200070702030000060809" X-Spam-Status: No, hits=-2.2 required=4.0 tests=BAYES_30,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,REPLY_WITH_QUOTES,USER_AGENT_MOZILLA_UA autolearn=ham version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.21.0.1; VDF: 6.21.0.46; host: bob) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------060200070702030000060809 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Thanks for your quick answer. I did try to use only viewhash, for adding. But it did not change. Here is a sample code you can run (attached) 200ms=time spent on 1 Select() ! I use a pentium 500Mhz There must be something I dont understand... Jean-Claude Wippler wrote: > Riccardo Cohen wrote: > >> I built a view of 50k records that I need to access. With normal view >> it is too slow (180 ms), so I try with hash view that I never used, >> and the result is even slower !(250 ms) ! >> (I've just read the new page http://www.equi4.com/mkmapping.html) >> >> Here is what I've done : >> >> c4_View view=db.GetAs("table[key:S,val:S]"),selection; >> c4_View viewsec=db.GetAs("sec[_H:I,_R:I]"); >> c4_View viewhash=view.Hash(viewsec,1); >> c4_Row row,searchrow; >> c4_StringProp val("val"); >> c4_StringProp key("key"); >> for (idx=0;idx> { >> sprintf(st1,"%d%d%d%d",idx,idx,idx,idx); >> sprintf(st2,"%d%d%d%d",idx,idx,idx,idx); >> key(row)=st1; >> val(row)=st2; >> view.Add(row); > > > No! Do not touch view when there's a hash mapping on top. Use viewhash: > viewhash.Add(row) > >> } >> db.Commit(); >> key(searchrow)="5555"; >> selection=viewhash.Select(searchrow); >> >> what's wrong ?? is there any sample code ? > > > What are you measuring - total time of the above code? Yes, that will > be slower, it is now also setting up hashes along the way. But the > select should be instant. > > One way to avoid the above confusion, it to rewrite your code a bit more > and use: > view=view.Hash(viewsec,1); > at the top, replacing the "viewhash" declaration. > > In other words, hide the original view once you've set up a hash on it. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 --------------060200070702030000060809 Content-Type: text/plain; name="hash.cxx" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="hash.cxx" #include #include #include #include #include #include main() { struct _timeb tm1,tm2; unsigned long ms1,ms2; char st1[200],st2[200]; long index,idx,curtime,totaltime; remove("tests"); c4_Storage db("tests",1); c4_View view=db.GetAs("table[key:S,val:S]"),selection; c4_View viewsec=db.GetAs("sec[_H:I,_R:I]"); c4_Row row,searchrow; c4_StringProp val("val"); c4_StringProp key("key"); view=view.Hash(viewsec,1); printf("write\n"); for (idx=0;idx<50000;idx++) { sprintf(st1,"%dkey",idx); sprintf(st2,"%ddata",idx); key(row)=st1; val(row)=st2; view.Add(row); } db.Commit(); printf("select\n"); totaltime=0; for (idx=0;idx<100;idx++) { index=rand()%50000; sprintf(st1,"%dkey",idx); _ftime(&tm1); key(searchrow)=st1; selection=view.Select(searchrow); _ftime(&tm2); ms2=(unsigned long) (tm2.time*1000+tm2.millitm); ms1=(unsigned long) (tm1.time*1000+tm1.millitm); curtime=ms2-ms1; totaltime+=curtime; if (selection.GetSize()!=1) { printf("empty search\n"); break; } } totaltime=totaltime/100; printf("avg=%d ms/search\n",totaltime); return(0); } /* RESULT: select avg=194 ms/search */ --------------060200070702030000060809-- From jcw@equi4.com Thu Sep 18 08:06:56 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8ID6tD25711 for ; Thu, 18 Sep 2003 08:06:55 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 1810683C03; Thu, 18 Sep 2003 15:06:48 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id A23A883C01 for ; Thu, 18 Sep 2003 15:06:47 +0200 (CEST) Date: Thu, 18 Sep 2003 15:07:17 +0200 Subject: Re: [Metakit] beginners hash view Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F69A482.1060704@dial.oleane.com> Message-Id: <0756C0D1-E9D9-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.7 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_02_03,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > Thanks for your quick answer. > I did try to use only viewhash, for adding. But it did not change. Weird... try using Find() instead of Select()? -jcw From jcw@equi4.com Thu Sep 18 08:17:31 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8IDHUD26789 for ; Thu, 18 Sep 2003 08:17:30 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 6791B83C03; Thu, 18 Sep 2003 15:17:23 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E256183C01 for ; Thu, 18 Sep 2003 15:17:22 +0200 (CEST) Date: Thu, 18 Sep 2003 15:17:52 +0200 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F697CD7.4030409@dial.oleane.com> Message-Id: <8224C351-E9DA-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.9 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_01_02,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] Trends (was: Re: beginners hash view) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > c4_View view=db.GetAs("table[key:S,val:S]"),selection; > c4_View viewsec=db.GetAs("sec[_H:I,_R:I]"); > c4_View viewhash=view.Hash(viewsec,1); [...] > what's wrong ?? is there any sample code ? To follow up on this - the demo/ and examples/ subdirs in the MK source distribution have some sample code, for C++, Python, and Tcl. For C++, there are 140+ little self-contained tests in the tests/ regression test suite. They may not be perfect examples, but they are very small and self-contained, and definitely a good spot to look for uses of all of the different view operators. I don't want to discourage people. On the contrary. But I'm juggling time between a number of activities. I've been working on some Really Exciting Technology To Take Metakit Way Further (TM) for some time now. So my efforts to help and support and improve docs are going to be limited, while maintaining my long-term commitment to help resolve and fix bugs. As you may have seen, the www.equi4.com website has recently gotten an overhaul, in an attempt to make things easier to find. I've started writing up some more pages in response to questions on this list. And I've just finished a basic utility to display low-level stats and verify free-space integrity of MK datafiles, see http://www.equi4.com/mkstats.html A number of people have sent a donation lately (thank you!), and Apple Computer has recently rewarded the fact that MK is doing well for them in every release of MacOSX all the way to Panther by donating a 17" Powerbook (whee!), so I can't even start to tell you how motivated I am to take the revolution of Metakit further. I'm saying this to let you know that although a mailing list like this is usually about Q's and problems, there really are many things going pretty well these days. The one constraint seems to be my time (and a more fanatic focus). If you want to help, consider writing a small piece about some aspect of Metakit - as a good deed on some rainy day, perhaps. You can either do it all yourself, put it on your website and announce it so I can point to it, or enter it as a page in the MK wiki at http://www.equi4.com/metakit/wiki.cgi - or you can email me and I'll go out of my way to set up a new page and integrate it with what's on the website already (with full credits and acknowledgement). The other way to help, and it's really encouraging to see it happen more and more, is to participate and help out with questions on this mailing list. Happy coding, may Metakit serve everyone really well :) -jcw From rcohen@dial.oleane.com Thu Sep 18 08:35:06 2003 Received: from bob.articque.com (free4.articque.com [195.25.225.74] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8IDZ6D28768 for ; Thu, 18 Sep 2003 08:35:06 -0500 Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id 825D8BF1F for ; Thu, 18 Sep 2003 14:43:55 +0200 (CEST) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 04206-46034B89; Thu, 18 Sep 2003 14:43:55 +0200 Received: by bob.articque.com (Postfix, from userid 1003) id 0EB8ABF23; Thu, 18 Sep 2003 14:43:55 +0200 (CEST) Received: from dial.oleane.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id EDC8FBF1F for ; Thu, 18 Sep 2003 14:43:51 +0200 (CEST) Message-ID: <3F69B433.8080408@dial.oleane.com> Date: Thu, 18 Sep 2003 15:33:39 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 Cc: Metakit mailing list Subject: Re: [Metakit] beginners hash view References: <0756C0D1-E9D9-11D7-BDF8-000A9588127E@equi4.com> In-Reply-To: <0756C0D1-E9D9-11D7-BDF8-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-6.5 required=4.0 tests=BAYES_10,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,REPLY_WITH_QUOTES,SIGNATURE_LONG_SPARSE, USER_AGENT_MOZILLA_UA autolearn=ham version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.21.0.1; VDF: 6.21.0.46; host: bob) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: It worked fine with Find() ;) Then it comes two questions : 1) Is it normal that select does not use hash ? 2) If I do a SelectRange(), will it use hash ? Thanks Jean-Claude Wippler wrote: > Riccardo Cohen wrote: > >> Thanks for your quick answer. >> I did try to use only viewhash, for adding. But it did not change. > > > Weird... try using Find() instead of Select()? > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jcw@equi4.com Thu Sep 18 08:44:08 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8IDi8D29705 for ; Thu, 18 Sep 2003 08:44:08 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id B598D83C03; Thu, 18 Sep 2003 15:44:00 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 5945483C01 for ; Thu, 18 Sep 2003 15:44:00 +0200 (CEST) Date: Thu, 18 Sep 2003 15:44:29 +0200 Subject: Re: [Metakit] beginners hash view Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F69B433.8080408@dial.oleane.com> Message-Id: <3A2685FA-E9DE-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,SPAM_PHRASE_03_05, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > It worked fine with Find() ;) > > Then it comes two questions : > > 1) Is it normal that select does not use hash ? > 2) If I do a SelectRange(), will it use hash ? It seems you've just answered your own Q's. Think about it: hashing is by value, not sorted. So select, which uses selectrange to optimize, will not be able to use it - I haven't checked the source, but when you do I'm pretty sure that's what you'll find out. -jcw From rcohen@dial.oleane.com Thu Sep 18 11:50:42 2003 Received: from bob.articque.com (free4.articque.com [195.25.225.74] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8IGofD15137 for ; Thu, 18 Sep 2003 11:50:41 -0500 Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id B572DBF1F for ; Thu, 18 Sep 2003 17:59:30 +0200 (CEST) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 06602-7BD921A7; Thu, 18 Sep 2003 17:59:30 +0200 Received: by bob.articque.com (Postfix, from userid 1003) id 8C737BF1F; Thu, 18 Sep 2003 17:59:30 +0200 (CEST) Received: from dial.oleane.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id D083BBF1F for ; Thu, 18 Sep 2003 17:59:27 +0200 (CEST) Message-ID: <3F69E20A.2050407@dial.oleane.com> Date: Thu, 18 Sep 2003 18:49:14 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-5.7 required=4.0 tests=BAYES_00,SIGNATURE_LONG_SPARSE,USER_AGENT_MOZILLA_UA version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.21.0.1; VDF: 6.21.0.47; host: bob) Subject: [Metakit] tutorial Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, If you are confused with metakit syntax and structure, like I was at beginning, you may have a look at http://support.articque.com/metakit/metakit_tutorial.html It is available also from http://www.equi4.com/mkdocs.html for new users. Hope this helps. -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From bkelley@wi.mit.edu Thu Sep 18 15:38:38 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8IKccD04337 for ; Thu, 18 Sep 2003 15:38:38 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003091816383411916 for ; Thu, 18 Sep 2003 16:38:34 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HLFG0A00.0UL for ; Thu, 18 Sep 2003 16:38:34 -0400 Message-ID: <3F6A1800.1070909@wi.mit.edu> Date: Thu, 18 Sep 2003 16:39:28 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Ordered and Hash Views together! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I have a database table "table[id:I,name:S]" that I would like to find quickly using either id or name. Is it possible to have two hash views simultaneously on this table? I have come up with the following strategy. Take the view and make a hash for it. Additionally, reproject the view so that the name attribute comes first and make a hash for that derived view. Use the appropriate hash when necessary ala: st = metakit.storage("mytest.cmk", 1) vw = st.getas("table[id:I,name:S]") # reorder a new projection dvw = vw.project(vw.name, vw.id) _hashview = st.getas("table_hash[_H:I,_R:I]") _hashview2 = st.getas("table2_hash[_H:I,_R:I]") vw2 = vw.hash(_hashview, 1) dvw2 = dvw.hash(_hashview2, 1) I use vw2 to quickly find the "id" properties and dvw2 to quickly find the "name" property. It appears to work (the full test is below) which is pretty amazing. I can add data to vw2 and dvw2 is automagically updated. Is this safe and proper? ===full test code=== import metakit, time, os st = metakit.storage() vw = st.getas("table[id:I,name:S]") nvw = st.getas("table_normal[id:I,name:S]") # reorder a new projection dvw = vw.project(vw.name, vw.id) _hashview = st.getas("objects_hash_[_H:I,_R:I]") _hashview2 = st.getas("objects2_hash_[_H:I,_R:I]") vw2 = vw.hash(_hashview, 1) dvw2 = dvw.hash(_hashview2, 1) size = 1000 for i in range(size): vw2.append((i, str(i))) nvw.append((i, str(i))) assert len(vw2) == len(dvw2) t1 = time.time() for i in range(size): nvw.find(id=i) print "normal finding all 'id's", time.time() - t1 t1 = time.time() for i in range(size): nvw.find(name=str(i)) print "normal finding all 'name's", time.time() - t1 t1 = time.time() for i in range(size): vw2.find(id=i) print "hashed finding all 'id's", time.time() - t1 t1 = time.time() for i in range(size): dvw2.find(name=str(i)) print "hahsed finding all 'name's", time.time() - t1 -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From jcw@equi4.com Thu Sep 18 17:29:17 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8IMTHD18597 for ; Thu, 18 Sep 2003 17:29:17 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id DD3DC83C03; Fri, 19 Sep 2003 00:29:08 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 5512283C01 for ; Fri, 19 Sep 2003 00:29:08 +0200 (CEST) Date: Fri, 19 Sep 2003 00:29:37 +0200 Subject: Re: [Metakit] Ordered and Hash Views together! Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F6A1800.1070909@wi.mit.edu> Message-Id: <966BA6E6-EA27-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.8 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > I have a database table > "table[id:I,name:S]" > > that I would like to find quickly using either id or name. Is it > possible to have two hash views simultaneously on this table? This Q was bound to come up one day ;) > vw = st.getas("table[id:I,name:S]") > dvw = vw.project(vw.name, vw.id) > > _hashview = st.getas("table_hash[_H:I,_R:I]") > _hashview2 = st.getas("table2_hash[_H:I,_R:I]") > vw2 = vw.hash(_hashview, 1) > dvw2 = dvw.hash(_hashview2, 1) > > I use vw2 to quickly find the "id" properties and dvw2 to quickly find > the "name" property. It appears to work (the full test is below) > which is pretty amazing. I can add data to vw2 and dvw2 is > automagically updated. > > Is this safe and proper? I am not sure. I suspect "dvw2" being updated is not quite right - try changing en existing item in vw2 so its key stays the same, but the name is altered... (my hunch is that dvw2 triggers a full rehash upon seeing a size change in its underlying data view) The trick with hash views is that they must see all changes, so they can update the secondary info, while essentially passing through the request to the underlying data view. You might have to change the above so one hash is built on top of the other: vw = st.getas("table[id:I,name:S]") vw2 = vw.hash(_hashview, 1) dvw = vw2.project(vw.name, vw.id) dvw2 = dvw.hash(_hashview2, 1) And then always make changes *only* through dvw2, which will cascade changes through vw2. Do tread with caution - these are cases where I have not done any testing at all... there is definitely considerable room for optimization (internally) in this sort of stacked use which is *not* being done at all in the current implementation. -jcw From rcohen@dial.oleane.com Fri Sep 19 05:16:55 2003 Received: from bob.articque.com (free4.articque.com [195.25.225.74] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8JAGs512645 for ; Fri, 19 Sep 2003 05:16:54 -0500 Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id EF1FFBF1F for ; Fri, 19 Sep 2003 11:25:42 +0200 (CEST) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 12744-3FD397C3; Fri, 19 Sep 2003 11:25:42 +0200 Received: by bob.articque.com (Postfix, from userid 1003) id 5D5CABF1F; Fri, 19 Sep 2003 11:25:42 +0200 (CEST) Received: from dial.oleane.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id 973A4BF1F for ; Fri, 19 Sep 2003 11:25:39 +0200 (CEST) Message-ID: <3F6AD740.3010708@dial.oleane.com> Date: Fri, 19 Sep 2003 12:15:28 +0200 From: Riccardo Cohen User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-5.2 required=4.0 tests=BAYES_10,SIGNATURE_LONG_SPARSE,USER_AGENT_MOZILLA_UA version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.21.0.1; VDF: 6.21.0.47; host: bob) Subject: [Metakit] indexed view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: While looking at indexed view in view.cpp, I read the header of hash view, and noticed a small mistake : * Example of use: * @code * c4_View data = storage.GetAs("people[name:S,age:I]"); * c4_View datah = storage.GetAs("people_H1[name:S,age:I]"); * c4_View hash = raw.Hash(datah, 1); * ... hash.GetSize() ... * hash.Add(...) while the text above speaks about the secondary view [_H:I,_R:I] (Must be an old version) About indexed view, it is written : * This view is modifiable. Careful: when a row is changed in such a way * that its key is the same as in another row, that other row will be * deleted from the view. So this supposes that multiple key is supported for adding, but not for updating. Is that true ? By the way, there is no c++ example of indexed view. I cant find what to put in arguments const c4_View& map_ and const c4_View& props_ -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jcw@equi4.com Fri Sep 19 05:36:24 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8JAaN514227 for ; Fri, 19 Sep 2003 05:36:23 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 03C8683C03; Fri, 19 Sep 2003 12:36:13 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 6D9A583C01 for ; Fri, 19 Sep 2003 12:36:13 +0200 (CEST) Date: Fri, 19 Sep 2003 12:36:46 +0200 Subject: Re: [Metakit] indexed view Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F6AD740.3010708@dial.oleane.com> Message-Id: <2AFD4206-EA8D-11D7-BDF8-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.9 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_02_03,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > While looking at indexed view in view.cpp, I read the header of hash > view, and noticed a small mistake : [...] > * c4_View datah = storage.GetAs("people_H1[name:S,age:I]"); [...] > while the text above speaks about the secondary view [_H:I,_R:I] > (Must be an old version) Good catch, will be fixed in next checkin. Thx. > About indexed view, it is written : > * This view is modifiable. Careful: when a row is changed in such a > way > * that its key is the same as in another row, that other row will be > * deleted from the view. > > So this supposes that multiple key is supported for adding, but not > for updating. Is that true ? The catch is that modifying a key property is not unlike deleting a row and adding another one. That leads to some hairy details which I have not even thoguth through well enough... > By the way, there is no c++ example of indexed view. I cant find what > to put in arguments const c4_View& map_ and const c4_View& props_ I'm not sure. Indexed views are experimental at best, right now. I'm not too pleased with what's there, and would suggest staying away from it - it's not ready for real use IMO. Even ordered views have some blind spots - it's all due to an unfinished design of how implicit key ordering and explicit row# indexing should work together. And there's things like allowing duplicate keys or not, and most important of all: all ordering is going to be somewhat limited until MK supports custom comparisons (for UTF-8, case-sensitivity on/off, reverse ordering, etc). You're reaching limits of the current implementation. Some of this is simply unfinished, but some of it hinges on deeper issues which are taking a lot more time to understand and resolve than I originally assumed. The view model is very generic, the "inherent ordering" side of things needs more details about views to be managed before all operators can be done properly. Note that you can always take over, and derive a new custom viewer class for a certain purpose. What a custom viewer lets you do is intercept all access and changes, and manage extra details in secondary view, etc. I should probably document more of that generic (and *very* powerful) mechanism, rather than try to list all rough spots in indexed viewers and such. Custom viewers are the foundation for all the newer view operators in MK, including hashes, blocking, joins, groupby, remap. -jcw From pxritzel@comcast.net Thu Sep 18 20:15:59 2003 Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8J1FwD01326 for ; Thu, 18 Sep 2003 20:15:58 -0500 Received: from HOMEOFFICE (h000c6e4678c1.ne.client2.attbi.com[24.147.191.122]) by comcast.net (sccrmhc12) with SMTP id <2003091901155301200qs0v9e>; Fri, 19 Sep 2003 01:15:53 +0000 From: "Paul Ritzel" To: Date: Thu, 18 Sep 2003 21:15:44 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Mk4py error with Python2.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: After downloading the lastest verion of metakit for windows and then installing I get the following error when importing metakit: >>> import metakit Traceback (most recent call last): File "", line 1, in ? File "c:\python23\lib\site-packages\metakit\metakit.py", line 12, in ? from Mk4py import * ImportError: Module use of python22.dll conflicts with this version of Python. Is there an updated version that uses the python23.dll? How can I get metakit to work with python2.3? I down loaded binaries, don't have a c-complier to make from source. help! Thanks, Paul From jcw@equi4.com Fri Sep 19 18:00:21 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8JN0K515442 for ; Fri, 19 Sep 2003 18:00:20 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 4382883C03; Sat, 20 Sep 2003 01:00:09 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id D295183C01 for ; Sat, 20 Sep 2003 01:00:08 +0200 (CEST) Date: Sat, 20 Sep 2003 01:00:43 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: <18A2653A-EAF5-11D7-B749-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.3 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] Bugs, gaps, and suggestions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: There's a new page on the website which I'm going to use to collect issues which do not fit the bug tracking system well enough, as well as more open-ended ideas and suggestions for extending and improving Metakit. The new feature/to-do page is at http://www.equi4.com/mktodo.html The bug tracking system is as before at http://www.equi4.com/bugs The list is fresh and totally incomplete. Please send suggestions and reminders. I'd like to prevent good ones from falling through the cracks, and also maintain a good overview so important things can be done first. -jcw From pieterb@gewis.nl Sat Sep 20 08:23:53 2003 Received: from gewis.win.tue.nl (postfix@gewis.win.tue.nl [131.155.71.116]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8KDNrM02020 for ; Sat, 20 Sep 2003 08:23:53 -0500 Received: by gewis.win.tue.nl (Postfix, from userid 103) id 52152210FA; Sat, 20 Sep 2003 15:23:52 +0200 (CEST) Date: Sat, 20 Sep 2003 15:23:52 +0200 From: PieterB To: metakit@equi4.com Message-ID: <20030920152352.A51215@gewis.win.tue.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] metakit for python 2.3 on freebsd Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I'm trying to install metakit with python 2.3 on freebsd 5.1. When I run "make METAKIT_WITH_PYTHON=yes" from /usr/ports/databases/metakit (and changing python2.2 to python2.3 in the Makefile). I get the following error: /bin/sh ./libtool --mode=compile c++ -c -Dq4_INLINE -O -pipe -mcpu=pentiumpro -fpermissive -I./../unix/../include -I./../unix/../python/scxx -I/usr/local/include/python2.3 ./../unix/../python/PyProperty.cpp rm -f .libs/PyProperty.lo c++ -c -Dq4_INLINE -O -pipe -mcpu=pentiumpro -fpermissive -I./../unix/../include -I./../unix/../python/scxx -I/usr/local/include/python2.3 ./../unix/../python/PyProperty.cpp -fPIC -DPIC In file included from ../python/PyProperty.cpp:11: ../python/scxx/PWONumber.h:19: syntax error before `i' ../python/scxx/PWONumber.h:21: warning: ISO C++ forbids defining types within return type ../python/scxx/PWONumber.h:21: semicolon missing after declaration of `class PWONumber' ../python/scxx/PWONumber.h: In function `int PWONumber(double)': ../python/scxx/PWONumber.h:21: only constructors take base initializers ../python/scxx/PWONumber.h:21: confused by earlier errors, bailing out *** Error code 1 Stop in /usr/ports/databases/metakit/work/metakit-2.4.9.2/builds. *** Error code 1 ------------------------------------------------ I also tried compiling metakit-2.4.9.2+njr0.1.tar.gz , as described in Nicholas Riley's message at http://trixie.triqs.com/pipermail/metakit/2003-September/001330.html The command "cd builds ; ../unix/configure --with-python ; make" then gives the following error. ... /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../python/scxx -INONE/include/python2.3 ../unix/../python/PyProperty.cpp g++ -c -g -O2 -I../unix/../include -I../unix/../python/scxx -INONE/include/python2.3 ../unix/../python/PyProperty.cpp -fPIC -DPIC -o .libs/PyProperty.o In file included from ../python/PyProperty.h:12, from ../python/PyProperty.cpp:8: ../python/PyHead.h:11:20: Python.h: No such file or directory In file included from ../python/PyProperty.h:12, from ../python/PyProperty.cpp:8: ../python/PyHead.h:13: syntax error before `{' token ../python/PyHead.h:22: ISO C++ forbids declaration of `ob_type' with no type ../python/PyHead.h:22: `t' was not declared in this scope ../python/PyHead.h:23: syntax error before `}' token ../python/PyHead.h:8:1: unterminated #if In file included from ../python/PyProperty.cpp:8: ../python/PyProperty.h:8:1: unterminated #if *** Error code 1 Stop in /tmp/metakit/builds. ------------------------------------ Any clues how to fix this? Regards, Pieter -- http://zwiki.org/PieterB From jcw@equi4.com Sat Sep 20 08:53:09 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8KDr9M03277 for ; Sat, 20 Sep 2003 08:53:09 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 4A58C83C04; Sat, 20 Sep 2003 15:52:51 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id DC0E883C00 for ; Sat, 20 Sep 2003 15:52:50 +0200 (CEST) Date: Sat, 20 Sep 2003 15:53:28 +0200 Subject: Re: [Metakit] metakit for python 2.3 on freebsd Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20030920152352.A51215@gewis.win.tue.nl> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.9 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: PieterB wrote: > I'm trying to install metakit with python 2.3 on freebsd 5.1. When > I run "make METAKIT_WITH_PYTHON=yes" from /usr/ports/databases/metakit > (and changing python2.2 to python2.3 in the Makefile). Not sure how the FreeBSD ports system is setup with MK, so I can't comment on it. In general, make's for the python side of MK could use some tweaks, judging from a couple of recent posts. Here's perhaps an option for you: if you checkout from CVS, you'll find a new distutils solution added by Gordon McMillan: cd python && python setup.py build (or "install") Now that distutils is supported, being "the" Python way nowadays, should we perhaps move away from makes and all the autoconf/libtool complexities? When in Rome, etc... -jcw From pieterb@gewis.nl Sat Sep 20 09:15:22 2003 Received: from gewis.win.tue.nl (postfix@gewis.win.tue.nl [131.155.71.116]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8KEFMM04345 for ; Sat, 20 Sep 2003 09:15:22 -0500 Received: by gewis.win.tue.nl (Postfix, from userid 103) id 2FBAD210FA; Sat, 20 Sep 2003 16:15:21 +0200 (CEST) Date: Sat, 20 Sep 2003 16:15:21 +0200 From: PieterB To: metakit@equi4.com Subject: Re: [Metakit] metakit for python 2.3 on freebsd Message-ID: <20030920161521.A52984@gewis.win.tue.nl> References: <20030920152352.A51215@gewis.win.tue.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from jcw@equi4.com on Sat, Sep 20, 2003 at 03:53:28PM +0200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Sat, Sep 20, 2003 at 03:53:28PM +0200, Jean-Claude Wippler wrote: > Here's perhaps an option for you: if you checkout from CVS, you'll find > a new distutils solution added by Gordon McMillan: > cd python && python setup.py build (or "install") I've given it a try, but it gives the same error as listed in the "compile error", http://trixie.triqs.com/pipermail/metakit/2003-September/001325.html for which Nicholas Riley supplied some patches (but that don't work for me unfortuately) > Now that distutils is supported, being "the" Python way nowadays, > should we perhaps move away from makes and all the autoconf/libtool > complexities? When in Rome, etc... that looks much better than the autoconf/libtool stuff. I can have a look if I can create a metakit-python port for FreeBSD when I get the stuff up and running. Regards, Pieter -- http://zwiki.org/PieterB From jcw@equi4.com Sat Sep 20 10:59:48 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8KFxlM09744 for ; Sat, 20 Sep 2003 10:59:47 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 6993383C04; Sat, 20 Sep 2003 17:59:34 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E919883C00 for ; Sat, 20 Sep 2003 17:59:33 +0200 (CEST) Date: Sat, 20 Sep 2003 18:00:11 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: <83928F5D-EB83-11D7-B749-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.3 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] gcc 3.3 on mac Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: FYI, the MK build problems with gcc 3.3 on OS X are resolved by getting the latest gcc update from Apple (Aug 2003): This one is bonkers: $ gcc --version gcc (GCC) 3.3 20030304 (Apple Computer, Inc. build 1435) Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This one is sane: $ gcc --version gcc (GCC) 3.3 20030304 (Apple Computer, Inc. build 1493) Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. No changes to MK. Builds without warnings. Test suite passes cleanly. -jcw PS. Unrelated, but FYI: I've regenerated autoconf 2.57 / libtool 1.4.3, see CVS. From njriley@shrug.csl.uiuc.edu Sat Sep 20 11:08:56 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8KG8tM10266 for ; Sat, 20 Sep 2003 11:08:55 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.8/8.12.8) with ESMTP id h8KG8snL070398; Sat, 20 Sep 2003 11:08:54 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.8/8.12.8/Submit) id h8KG8suC070397; Sat, 20 Sep 2003 11:08:54 -0500 (CDT) Date: Sat, 20 Sep 2003 11:08:54 -0500 From: Nicholas Riley To: metakit@equi4.com Cc: PieterB Subject: Re: [Metakit] metakit for python 2.3 on freebsd Message-ID: <20030920160854.GA70198@uiuc.edu> Mail-Followup-To: metakit@equi4.com, PieterB References: <20030920152352.A51215@gewis.win.tue.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Sat, Sep 20, 2003 at 03:53:28PM +0200, Jean-Claude Wippler wrote: > PieterB wrote: > > >I'm trying to install metakit with python 2.3 on freebsd 5.1. When > >I run "make METAKIT_WITH_PYTHON=yes" from /usr/ports/databases/metakit > >(and changing python2.2 to python2.3 in the Makefile). > > Not sure how the FreeBSD ports system is setup with MK, so I can't > comment on it. In general, make's for the python side of MK could use > some tweaks, judging from a couple of recent posts. I successfully built my version of Metakit on FreeBSD 5.1. There are some changes from the +njr0.1 version that may make a difference, so here is a current snapshot: In case that still doesn't compile, the builds/ directory contains a built version of Metakit/Mk4py for FreeBSD; you should just need to make install from there. You may also want this, to run the Mk4py test suite: There's a bug in the test.regrtest package in Python 2.3; replace /usr/local/lib/python2.3/test/regrtest.py with that version and you should get no errors with 'make test-python'. > Here's perhaps an option for you: if you checkout from CVS, you'll find > a new distutils solution added by Gordon McMillan: > cd python && python setup.py build (or "install") > > Now that distutils is supported, being "the" Python way nowadays, > should we perhaps move away from makes and all the autoconf/libtool > complexities? When in Rome, etc... I plan to take a look at this after I finish everything else (hopefully later today). I don't think we can abandon the Makefile-based method because there were a number of bugs in earlier distutils versions. I am pretty sure Python 2.3's distutils should work fine everywhere, though. -- =Nicholas Riley | From aweber@comcast.net Mon Sep 22 11:24:24 2003 Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8MGOOM16655 for ; Mon, 22 Sep 2003 11:24:24 -0500 Received: from cellis (proxy.organoninc.com[12.24.98.2]) by comcast.net (rwcrmhc13) with SMTP id <2003092216241801500ft2jme> (Authid: aweber); Mon, 22 Sep 2003 16:24:18 +0000 Message-ID: <002e01c38125$f91daaf0$2474800a@documentum.com> Reply-To: "Aaron J Weber" From: "Aaron J Weber" To: Date: Mon, 22 Sep 2003 12:24:16 -0400 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_002B_01C38104.70DD4EF0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Coupla newbie Q's Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_002B_01C38104.70DD4EF0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Looked through the archives and didn't find anything addressing this, = specifically. Sorry if I missed it in a previous "post"... (All C++ stuff...) 1) Instantiating properties with names at run-time. Is it possible to do something like: c4_StringProperty *pProp1 =3D NULL; //I don't think this will work, bc = there is no appropriate/default constructor. ... CString sFirstPropName =3D "blah blah"; //user inputs value or it is = read from ini-type file... pProp1 =3D new c4_StringProperty(sFirstPropName); Or do I have to hard-code the properties ("columns") within the code? = Seems inflexible if that's the case. As a possible workaround, can I hardcode "placeholder" prop names and = just reference the columns (properties) that way? If I did so, would it = change the underlying view when it "committed"? 2) How do I "close" a view? 3) How do I open a view (and all it's derived views) for ReadOnly? = Really, I want to open the entire 'storage' for ReadOnly. 4) I'm not clear on how to appropriately "search" (not necessarily the = method) for a row. What are the criteria for choosing one method over = another? Also, how do I do a "wildcard search" -- something like a = starts-with letter "s" and return me the sub-view? The equivalent of = "select property1 from my_view where property1 like 's%';"? Thanks for any and all responses! -AJ ------=_NextPart_000_002B_01C38104.70DD4EF0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Looked through the archives and didn't = find=20 anything addressing this, specifically.  Sorry if I missed it in a = previous=20 "post"...
 
(All C++ stuff...)
 
1) Instantiating properties with names = at=20 run-time.
Is it possible to do something = like:
c4_StringProperty *pProp1 =3D = NULL;  //I don't=20 think this will work, bc there is no appropriate/default=20 constructor.
...
CString sFirstPropName =3D "blah blah"; = //user inputs=20 value or it is read from ini-type file...
 
pProp1 =3D new=20 c4_StringProperty(sFirstPropName);
 
Or do I have to hard-code the = properties=20 ("columns") within the code?  Seems inflexible if that's the=20 case.
As a possible workaround, can I = hardcode=20 "placeholder" prop names and just reference the columns (properties) = that=20 way?  If I did so, would it change the underlying view when it=20 "committed"?
 
2) How do I "close" a = view?
 
3) How do I open a view (and all it's = derived=20 views) for ReadOnly?  Really, I want to open the entire 'storage' = for=20 ReadOnly.
 
4) I'm not clear on how to = appropriately "search"=20 (not necessarily the method) for a row.  What are the criteria for = choosing=20 one method over another?  Also, how do I do a "wildcard search" --=20 something like a starts-with letter "s" and return me the = sub-view?  The=20 equivalent of "select property1 from my_view where property1 like=20 's%';"?
 
Thanks for any and all = responses!
 
-AJ
------=_NextPart_000_002B_01C38104.70DD4EF0-- From jeff@k2.com Mon Sep 22 22:08:30 2003 Received: from kobalt.k2.com (dsl092-155-098.wdc2.dsl.speakeasy.net [66.92.155.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8N38TM03601 for ; Mon, 22 Sep 2003 22:08:29 -0500 Received: from [192.168.0.24] (IDENT:jkay@kobalt.k2.com [192.168.0.2]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id h8N38Qf06795; Mon, 22 Sep 2003 23:08:26 -0400 User-Agent: Microsoft-Entourage/10.1.4.030702.0 Date: Mon, 22 Sep 2003 23:08:26 -0400 Subject: Re: [Metakit] Coupla newbie Q's From: Jeffrey Kay To: Aaron J Weber , Message-ID: In-Reply-To: <002e01c38125$f91daaf0$2474800a@documentum.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I didn't se a posted response, so here goes ... On 9/22/03 12:24 PM, "Aaron J Weber" wrote: > 1) Instantiating properties with names at run-time. > Is it possible to do something like: > c4_StringProperty *pProp1 = NULL; //I don't think this will work, bc there is > no appropriate/default constructor. > ... > CString sFirstPropName = "blah blah"; //user inputs value or it is read from > ini-type file... > > pProp1 = new c4_StringProperty(sFirstPropName); > > Or do I have to hard-code the properties ("columns") within the code? Seems > inflexible if that's the case. > As a possible workaround, can I hardcode "placeholder" prop names and just > reference the columns (properties) that way? If I did so, would it change the > underlying view when it "committed"? I don't believe that there's any problem using c4_StringProperty *p = NULL. p = new c4_StringProperty("myfield"); This doesn't require a default constructor, but c4_StringProperty p; does. If you are passing in the value, though, you should just be able to do something like char *t = "myfield"; c4_StringProp p(t); which would be less hassle later in the code. > 2) How do I "close" a view? Delete the view object. If you use a stack variable, then just surround the code in braces. > 3) How do I open a view (and all it's derived views) for ReadOnly? Really, I > want to open the entire 'storage' for ReadOnly. I believe that you open the storage c4_Storage("myname",0) to get it to run read-only. Some views are read-only by default because they are derived views. > 4) I'm not clear on how to appropriately "search" (not necessarily the method) > for a row. What are the criteria for choosing one method over another? Also, > how do I do a "wildcard search" -- something like a starts-with letter "s" and > return me the sub-view? The equivalent of "select property1 from my_view > where property1 like 's%';"? One of the things missing from the c++ api is a wildcard search. It would sure be nice to have one. find() locates a row with an exact match. select() creates a view with a subset that matches the selection criteria. locate() is also used to find a row, although I haven't used it in my code. Hope this helps ... jeffrey kay weblog pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein From gary.h.merrill@gsk.com Thu Sep 25 07:27:22 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8PCRMM01218 for ; Thu, 25 Sep 2003 07:27:22 -0500 Received: by usrtpnp1.glaxowellcome.com; id IAA15545; Thu, 25 Sep 2003 08:27:57 -0400 (EDT) From: Received: from keymaster.glaxo.com(192.58.204.209) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma007172; Thu, 25 Sep 03 08:20:01 -0400 Received: by usrtpnp2.glaxowellcome.com; id IAA28269; Thu, 25 Sep 2003 08:20:00 -0400 (EDT) Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma024278; Thu, 25 Sep 03 08:15:52 -0400 Received: by ussunt2.glaxo.com id IAA22366; Thu, 25 Sep 2003 08:15:59 -0400 (EDT) Received: from 166.71.134.118 by us9n55.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Thu, 25 Sep 2003 08:15:07 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h8PCF6m29974 for ; Thu, 25 Sep 2003 08:15:07 -0400 (EDT) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Thu, 25 Sep 2003 08:14:41 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 09/25/2003 08:15:12 AM MIME-Version: 1.0 X-WSS-ID: 136C03C12073168-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] S vs B datatypes in Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I gather that if I want to store an arbitrary Python object (say a class instance) as a property, I should use the B datatype. But I also gather (from a bit of experimentation) that the object needs to be pickled (into a string) before it is stored in this way. Given that, it's not obvious what the difference is from the Python point of view between using S or B. In the Python doc for Metakit both are documented as 'becomes a Python string' -- and I don't seem to be able to find any examples using B. Is there in fact, for Python, a difference between using S and using B? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From jcw@equi4.com Thu Sep 25 07:35:43 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8PCZhM02017 for ; Thu, 25 Sep 2003 07:35:43 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id C256D83EEA; Thu, 25 Sep 2003 14:35:35 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 1985B83EE7 for ; Thu, 25 Sep 2003 14:35:34 +0200 (CEST) Date: Thu, 25 Sep 2003 14:35:35 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.3 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] test - please ignore Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: (I'm fiddling with Mailman mailing list settings) -jcw From jcw@equi4.com Thu Sep 25 08:07:41 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8PD7eM04580 for ; Thu, 25 Sep 2003 08:07:41 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 7EEBA83EEA; Thu, 25 Sep 2003 15:07:33 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 1CD5B83EE7 for ; Thu, 25 Sep 2003 15:07:33 +0200 (CEST) Date: Thu, 25 Sep 2003 15:07:33 +0200 Subject: Re: [Metakit] S vs B datatypes in Python Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <3A433132-EF59-11D7-8DEE-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.1 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: wrote: > Is there in fact, for Python, a difference between using S and using B? There are some semantic differences between C/C++, Python, and Tcl - so there are always going to be some slight "impedance mismatches" between them for Metakit. In C, S-properties are zero-terminated strings, while B's are (sized) byte buffers. In Python and Tcl, the distinction is far smaller. The sort order of S's is done with stricmp (case insensitive textual comparison), while it is memcmp for B's. If your data may contain null bytes, you must use B's. If your data is text, use S's to get a decent sort order (Unicode and UTF-8 issues will also play a role for S's one day). -jcw From bkelley@wi.mit.edu Thu Sep 25 09:35:27 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8PEZRM14094 for ; Thu, 25 Sep 2003 09:35:27 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003092510352205470 for ; Thu, 25 Sep 2003 10:35:22 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HLRXUZ00.09Z; Thu, 25 Sep 2003 10:35:23 -0400 Message-ID: <3F72FD5E.3020800@wi.mit.edu> Date: Thu, 25 Sep 2003 10:36:14 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com, gary.h.merrill@gsk.com Subject: Re: [Metakit] S vs B datatypes in Python References: <3A433132-EF59-11D7-8DEE-000A9588127E@equi4.com> In-Reply-To: <3A433132-EF59-11D7-8DEE-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > wrote: > >> Is there in fact, for Python, a difference between using S and using B? > Yes. Pickle has two modes for working. The default mode stores everything basically in the S format with no ASCII values like null. The binary mode is slightly faster and more compact and will store values in binary format. This is not the default pickle format though, for example: pickle.dumps("binary string\000x", 1) will dump a string with null characters in it. The meat of the matter is as follows (pardon my formatting): pickle type: String metakit S: OK metakit B: OK pickle type: Binary metakit S: NO metakit B: OK So it is safest to always use the binary type for pickle stores. Then Python won't care what form the pickle is in. Also you have the side benefit that you can use the zlib library to compress the data in the database. This can save an awful lot of space in the db and actually make for faster retrieval in some cases. following is test code pickling a string with a null character in it. === import pickle, metakit data = "my dog has \000 fleas" db = metakit.storage() vw = db.getas("test[type:S,a:S,b:B]") vw.append(type="string pickle", a=pickle.dumps(data), b=pickle.dumps(data)) vw.append(type="binary pickle", a=pickle.dumps(data,1), b=pickle.dumps(data,1)) for row in vw: print row.type try: a = pickle.loads(row.a) print "\ta=",a except: print "\tCould not load a (string prop)" try: b = pickle.loads(row.b) print "\tb=",b except: print "\tcould not load b (binary prop)" -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From njriley@shrug.csl.uiuc.edu Fri Sep 26 01:41:23 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8Q6fMM08348 for ; Fri, 26 Sep 2003 01:41:22 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9) with ESMTP id h8Q6fMQZ012650 for ; Fri, 26 Sep 2003 01:41:22 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9/Submit) id h8Q6fLmV012649 for metakit@equi4.com; Fri, 26 Sep 2003 01:41:21 -0500 (CDT) (envelope-from njriley) Date: Fri, 26 Sep 2003 01:41:21 -0500 From: Nicholas Riley To: metakit@equi4.com Message-ID: <20030926064121.GA12542@uiuc.edu> Mail-Followup-To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Subject: [Metakit] Mk4py storage lifetime Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi again, I'm almost finished with my Mk4py work, finally. Down from about 20 items on my to-do list to two, at least. :-) Anyway, one more question. Should this work? >>> metakit.storage().getas('blah[x:S,y:I]').structure() [] Because this does: >>> s = metakit.storage() >>> s.getas('blah[x:S,y:I]').structure() [Property('S', 'x'), Property('I', 'y')] I haven't looked carefully enough to see if there's any potential for crashing in the case of a deallocated storage object, but does this behavior make sense? Should the storage stick around until all associated view are released? Should access to orphaned views throw an exception? Thanks, -- =Nicholas Riley | From jcw@equi4.com Fri Sep 26 04:19:07 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8Q9J6M20025 for ; Fri, 26 Sep 2003 04:19:07 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 7FAF283EEA; Fri, 26 Sep 2003 11:18:59 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B895F83EE7 for ; Fri, 26 Sep 2003 11:18:58 +0200 (CEST) Date: Fri, 26 Sep 2003 11:19:00 +0200 Subject: Re: [Metakit] Mk4py storage lifetime Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20030926064121.GA12542@uiuc.edu> Message-Id: <76BF6F19-F002-11D7-8DEE-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-3.0 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Nicholas Riley wrote: > I'm almost finished with my Mk4py work, finally. Down from about 20 > items on my to-do list to two, at least. :-) Wow, great! > Anyway, one more question. Should this work? > >>>> metakit.storage().getas('blah[x:S,y:I]').structure() > [] > > Because this does: > >>>> s = metakit.storage() >>>> s.getas('blah[x:S,y:I]').structure() > [Property('S', 'x'), Property('I', 'y')] This is correct behavior in the current design. Storages are not "kept open" by views. Their cleanup causes all views associated with them to become empty. > Should access to orphaned views throw an exception? That would indeed be another way to treat this (and probably useful to help detect incorrect use), but it would require some redesign of the C++ core. -jcw From njriley@shrug.csl.uiuc.edu Sat Sep 27 01:40:06 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8R6e5M15625 for ; Sat, 27 Sep 2003 01:40:05 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9) with ESMTP id h8R6e5QZ019878 for ; Sat, 27 Sep 2003 01:40:05 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9/Submit) id h8R6e49h019877 for metakit@equi4.com; Sat, 27 Sep 2003 01:40:04 -0500 (CDT) (envelope-from njriley) Date: Sat, 27 Sep 2003 01:40:04 -0500 From: Nicholas Riley To: metakit@equi4.com Message-ID: <20030927064004.GC19396@uiuc.edu> Mail-Followup-To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Subject: [Metakit] Mk4py changes Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, Only about a year late, I've fixed everything that's annoyed me about Mk4py. There's quite a bit here, as I often stumbled on one bug or brokenness as I was trying to fix another. Since I've also had to move and rename some files, I don't think it'd work well to provide a patch, so I've uploaded a new version here: In order to run the Python test cases with Python 2.3 or 2.3.1, you need to replace the shipped version of test/regrtest.py with this one: Python 2.2's regrtest works fine, this was a bug that was introduced in 2.3 and I didn't catch in time. (I've filed it and posted a patch, but it was too late to get into 2.3.1, apparently.) I've tested with: - Mac OS X 10.2, Python 2.3 - everything works. After all the garbage with using libtool, it does work to create a Mk4py library, but its composition is suboptimal (-flat_namespace, -bind_at_load are bad). Distutils works a lot better - and is easier to use and faster. ../unix/configure --with-python=/Library/Frameworks/Python.framework/Versions/2.3 - Debian unstable, Python 2.3 - everything works, make/distutils builds are equivalent. ../unix/configure --with-python=/usr - Debian stable, Python 2.2 - make, make test-python works. The version of distutils shipped with Python 2.2 is buggy; replacing it with a current version causes 'python setup.py ' to work fine. ../unix/configure --with-python=/usr/include/python2.2,/usr/lib/site-packages - Debian stable, Python 1.5.2 - make, make test-python works. Python 1.5.2 doesn't come with distutils; installing the current version works for build/install targets, but not for test (since some test code I didn't write already didn't work there, I didn't bother). In particular, Debian's 'python1.5-distutils' package does not work. - FreeBSD 5.1-CURRENT, Python 2.3 - everything works, limited testing. The changes: - examples/demo.py: import metakit, not Mk4py, better at the claimed purpose of "verify(ing) proper Mk4py installation". - python/metakit.py: - replaced comment at top of file with docstring so it shows up in pydoc and related things as "DESCRIPTION". - added metadata for use by pydoc; also referenced through setup.py for Python package managers (thanks, Bob Ippolito) - python/PyRowRef.cpp: - properly handle attempts to store long long-sized values into Metakit 'I' fields; throw an exception on overflow, instead of corrupting the value - properly handle the possibility of int subtypes in Metakit 'L' fields; reading the value is not guaranteed to succeed any more - convert Unicode objects to UTF-8 before storing in 'S' fields - don't silently discard all data after a null character in 'S' fields; throw an exception instead - python/PyStorage.cpp: - satisfy dumb compiler regarding guaranteed function exit from PyStorage_Description() - better type-check storage constructor, returning more intelligible errors and exceptions where appropriate - python/PyView.cpp: - check number of arguments in view.setsize() instead of discarding - clarify docstring for view.insert() - fix view.insert() to better check arguments, fixes all the cases I mentioned previously (attempting to insert a raw string or multiple positional arguments) - satisfy dumb compiler again in PyView_insert() - improve consistency in docstring for view.append() - check number of arguments in view.append() instead of discarding - clarify error message in PyView_getitem() - make clear that PyView::makeRowFromDict() is meant to discard keyword arguments/dictionary keys which don't have corresponding properties in the view - otherwise, it just looks like an oversight - fix PyView::makeRow to work with new-style classes, and still work back to *sob* Python 1.5.2; also, do the other part of the view.insert() changes above - properly check the number of positional arguments in view.append()/insert() - python/scxx/PWONumber.h: - don't assume LONG_LONG is defined if HAVE_LONG_LONG (it actually means 'long long'); use PY_LONG_LONG instead - fix PyNumber::operator long() to not assume that conversion to long will succeed, potentially corrupting data (Python longs are arbitrary-precision; they may not even fit in a long long) - python/scxx/PWOSequence.h: - consistency on PWOString type check failure exception string, since it's now more prevalent since I changed string handling on insert to use PWOString instead of raw PyObjects (needed to avoid the reference counting getting out of control) - added PWOString::size() for the aforementioned embedded null checking - python/setup.py: - added config command (not user-visible); since Mk4py doesn't import config.h and may be used on non-autoconf platforms, does a check for Python unicode support and passes it through to the build. This is the part that fails on pre-2.3 versions of distutils, ironically the versions where it'd be most useful. *sigh* At least it's possible to install the current distutils, which is kept backwards-compatible to 1.5.2. - added workaround for linker errors on several platforms. Tries in two ways to make the C++ libraries be included; first, tries to override $CC with $CXX (which was $CCC in Python 1.5.2) as defined at the time Python is compiled. Second, for older Python versions which may not have $CXX defined at all, or otherwise are broken, add "-lstdc++" on Linux. Other OSes can be added as needed, but at this point I gave up. This is really all a workaround for distutils' inadequate support for C++ extensions; it assumes the compiler/linker used to compile Python itself is acceptable for all extensions, when it isn't. - on Mac OS X with Python 2.3, removes the '-Wl,-x' linker option, which causes static constructors to break in a particularly nasty and confusing fashion. Took me three hours just to track down the problem to this linker option! I've discussed this on the pythonmac-sig list, it's pretty clear that it will be fixed in a forthcoming MacPython version, but... :) - added test command which runs tests with 'python setup.py test', much like 'make test-python', except even more intelligently - default to building in ../builds, for consistency with the rest of Metakit; you can still use build -b to override it - include package manager metadata (again thanks to Bob Ippolito) - python/test/all.py: - make sure we're testing the built modules instead of the installed modules; do some manipulation of sys.path to ensure this - fix a couple of occurrences of "MetaKit" -> "Metakit" :) - fix some comments including a partial sentence I sent in last time (oops; no idea what I meant to write) - python/test/leaktest.py: just a simple test I wrote to test Unicode strings, for which I was leaking memory - no more. May just be deleted I guess, since it isn't run automatically and is very Unix-specific. - python/test/mktestsupport.py: - moved some test support code here; e.g. testing for int/long handling and Unicode support - abstracted the insert/failure handling code from test_inttypes.py so tests such as test_inttypes.py and test_string.py appear cleaner and easier to understand. - python/test/test.py: moved to python/test/test_unittest.py; whoever created this file didn't bother to test that it broke my exsting test, and didn't notice the 'test-python' target didn't even reference it. If you want to run the tests standalone, use 'all.py', which I named as an analog to 'all.tcl' for the Tcl tests. - python/test/test_inttypes.py: - test int/long integration, where available - python/test/test_string.py: - test regular and Unicode string support, where available - python/test/test_unittest.py (formerly test.py): - test a lot more stuff - unix/aclocal.m4, configure, scripts: updated for Autoconf 2.57, GNU libtool 1.5 with Fink patches, - unix/configure.in: - prefer Python 2.3 - test for Python Unicode support (not used for distutils build) - unix/Makefile.in - fix 'clean' target to do a better job, removing libraries - unix/autogen.sh - regenerate from autoconf/automake/libtool/etc., since I always botch the magic command sequence (and doubt I am alone) That's it. I hope these changes will make it easier for people new to Mk4py to get up and running, instead of being mired in compilation and usage problems. -- =Nicholas Riley | From pieterb@gewis.nl Sat Sep 27 02:03:12 2003 Received: from gewis.win.tue.nl (postfix@gewis.win.tue.nl [131.155.71.116]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8R73BM16777 for ; Sat, 27 Sep 2003 02:03:12 -0500 Received: by gewis.win.tue.nl (Postfix, from userid 103) id 9539D21104; Sat, 27 Sep 2003 09:03:10 +0200 (CEST) Date: Sat, 27 Sep 2003 09:03:10 +0200 From: PieterB To: Nicholas Riley Cc: metakit@equi4.com Subject: Re: [Metakit] Mk4py changes Message-ID: <20030927090310.A36088@gewis.win.tue.nl> References: <20030927064004.GC19396@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20030927064004.GC19396@uiuc.edu>; from njriley@uiuc.edu on Sat, Sep 27, 2003 at 01:40:04AM -0500 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Sat, Sep 27, 2003 at 01:40:04AM -0500, Nicholas Riley wrote: > - FreeBSD 5.1-CURRENT, Python 2.3 - everything works, limited testing. I tried compiling it on FreeBSD 5.1 RELEASE using "cd builds ; ../unix/configure" and with "../unix/configure --with-python=/usr" but both fail during configure. Using "../unix/configure" (without params) also fails: ... checking for inline... inline checking for long... yes checking size of long... configure: error: cannot compute sizeof (long), 77 See `config.log' for more details. ... My config.log is online at http://gewis.nl/~pieterb/tmp/config.log.txt Do I need to upgrade autoconf, or should I change something? Thanks for all your effort! It would be great if the default metakit would work with Python out of the box. Pieter -- It takes longer to glue a vase together than to break one. From jcw@equi4.com Sat Sep 27 04:37:00 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8R9axM24479 for ; Sat, 27 Sep 2003 04:36:59 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 9DE3683EEA; Sat, 27 Sep 2003 11:36:51 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id BEEB583EE7 for ; Sat, 27 Sep 2003 11:36:50 +0200 (CEST) Date: Sat, 27 Sep 2003 11:36:52 +0200 Subject: Re: [Metakit] Mk4py changes Content-Type: multipart/mixed; boundary=Apple-Mail-9-448866243 Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list In-Reply-To: <20030927064004.GC19396@uiuc.edu> Message-Id: <1FFB6F42-F0CE-11D7-8DEE-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.2 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-9-448866243 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Nicholas, Thanks for this great contribution. I'll go through this to understand all the changes. In the meantime. I'm attaching a diff with the latest code in CVS, it wasn't so hard after all, and after editing out the differences reported for auto-generated stuff such as configure, it becomes a lot easier to read through all your changes. [...] > That's it. I hope these changes will make it easier for people new to > Mk4py to get up and running, instead of being mired in compilation and > usage problems. Thanks again - I *very* much share your concerns and appreciate your efforts to improve that side of things. Deployment hassles are the worst time-wasters ever IMO, it's worth spending all our time on (but only a few people, that is), to get better out-of-the-box solutions! -jcw --Apple-Mail-9-448866243 Content-Disposition: attachment; filename=njrdiffs.out.gz Content-Transfer-Encoding: base64 Content-Type: application/x-gzip; x-unix-mode=0644; name="njrdiffs.out.gz" H4sICOhXdT8AA25qcmRpZmZzLm91dADcPG172zaSn+NfgShtRSWULErymxL51nGc1ruJ7Sd2ku7l yemhSMhiTZFagrKs63p/+80MABKkKNlJ9sPd+WlrGZgZDOYdA6jnUbhkQcSmPHVvgvRF9EfSZ8ef LrfOVye2R/Mg9MX6eZ9P403TsbdxctudBesB+J07nYVc0feD8Zg1k5EGaXq3IgMhRlqzZSW+ntzq 2j2va+9svWLBdBYnKXt/05stbSaWAsamN2wgR7aazebWoQbC2ewPRZ+5AuBhuJLxIPLCuc/X7yyc u+snp+4Goc2W6SSONkhEAmxfLD/Eiw983PJmswoChfktp7NvO13Hk7+7IAv5E4yZdbE8jdLh8YR7 N5abpkmjkU0zZlleHImUeb0hQF0k8eyXBpvBrwazknjRAIFK/KPL4bvzs18lhZcZBR4KTqvgOHuK 0MOzOOL5Gn8qXWxiCIAO2ZMwjq5ZNJ+OeJIvK97BqF71MOMb6WjQAWs67JdfAOMkSYbnnjdPEu5b DWTn7OO7dw0Dj7G3bhACCyd33vCTG8454MSJzWpBlLI0jlnoJtccPjEQzC1PcJAdM+StVuDgEZKT DOZI9+vFBRJwuqC7XtuDf79LfyioH1DgxRIJlNWyqrsqKKe3A6zvefR7J2f//rvt5N5k7P+ZuaxV 1Dfay17HdvZ6nrMHQj8o28xlmgSrisoNB/h4vUy5YCmfzgwEMJPLqw+n2lBsAyWD+fXkanh5+p8n EoS9YI7Nxi6wCjsuG0wF44fsWTD2+Zj9dvTpZPjx7PT4/M3J+eu/nhxfDX9DzY7iOGSBGM6jwIt9 Ttr9KD+bG0JBPcE1ctAGo/VMhCPx8ertvuQ8x3uGkkV8qZXVFWk/EjKCMI2gF5/PJRkmStTKXOAQ ymv45uT4w8nbHBAnRPDffAhJCX7BOpJUC/+yJC1lJxM3Yc89OZ3BZYsR9lMaDnlkKbiGWnmd0Spq sELqBpFgHOzN97kPlheG4iVLkyWrv66zdDnjNcVN0U7UQrZkv6B5Z28P7HHfdw463+D2BWavYF3N 6yJB/0ZO2DhOmJQ8ug0ytjlxXqZx4l7z9ZnTANhyDnY9+BdcaI+Y8tzUmzCr1WqBrbKEp/MkYu2X EMu0aRch7mksh9t+zoSbBmK8BDlPZ0EIMef59lZ398Dr7rXt7l7P2DuK5unF8i1AKcsGSZyP/uBe +pw1ILSIL+2vK4F4Ix57XsAje6gICZXwEnqtSpIFm7mJCyKFPU0DIQId5Z5IVT+AD+vMpxwCZxRD /RWxeMYjNoY9IJFee9frOQ4I58k68yU0QrAZmESyvYYfKau1ZITUPkR8MA2wbBeKwhhcrsc0g+Jh G/sU8MV6A9OzW50dt7Nrd/aU7sgVQNotdFrKOE6ebtZITfBURgfFLhTFXgp1JbhRxjDyu9v2dtHz IDUJSETWLBZBGsQRyGr0R4M1m2CPPPE4/skskN8NXy7ixEciooF5C9IQqMVnkkBNC/LfRZCNOHgy Z5oOLLC7Y+/ue7t79r6uWUsSGpCEtMPEzUNJ6ii1IBqfUb60tPna7GZBS+vqhkxyIy4zkOmT87WQ wzDLw7gXz+HDgOWMvSwqVAEMWJv98585xiHrSB/cqGAl3rJ+00WcGyQZvKlxkquWpBsWTffQLB/y 9XE3Ku3k0gO5/MrTS7IwkELk3xWmtXxeFrI6Qr1ibRVg8K8XAyKdJagiBFN02zR/X1XhIMAh0TCo 5kQVfLG2MARPfvTEVDLgGxah8fKMZCB3qpFzi9jrePtgpAePzBD3358n9na9A/SFmjuD6Ohb3+pq Es1WCwk0kdyR/300tQPvd7yDA9tpt9e6b7vxQ/7U2NqHErfdMV06p1epTTCF/R3XafeArx0D8CE/ VNLZFGfJDafzMA1mITcFlvve/v6Bd9A+yOKOPGZcgovJolGuexr5/E4vTH+weJ6yeMwSN7rmtVIl /QgiqKygkpDTPth1HceB8sxx9o0qwggQ4KFgkLeQttjEFZCgwTjgWCRmMVS/UIfhQQU8Y9lnwXWE QRxM1QG1A91OF+h2eoouUPHcqJ6yP+ZUxUKtgQqagnADVc+9VMUsZF4gpGYE9kkuKH+yTqubc2mN 5rJgCHCigwrgbhIGHE4ekgmwD+p7ODt54wNLRgutLRiArwUQiiCeXchNBFyAwbHgxYtyYcXhBAau ClEgdSMPqgzB/zHn9ClCM7hns2VMVdOQCtMBDb8sBNgIRstryco/D1xPYxXjNpMrHYIlU6p+i3V1 BwLX/ApUe8QXTZEuwTy90BUCBq1MrB3YsR+TNFUgSpM56bKCF011DT+XSjRlfijJ8H8Mwa2v0wkd xWShOXxHIwCZn2JyuEMWZblCWvvbOJm6aZWp66XJVH/2gStO7vcyN2AYnWUaqNkGR6BKXbKmE3Sa i89vaA0+w5iWJagnqMgc7WWViLR5rIhIa6akDFtWvrprIeUuxalk9JsrjuCopNw8hs0Oh4Q6HNYa 2LWAql9m5rLc2Yptmgp8UvT1NRHwSBcXgdA1+qovoAf6gUd1RwIBAdwdYUdc5RNfpQ51MjCSvekG qw4aZS4JPo2xqn0Ajr2zWzouxcU+BlvrHgUg7eck7ZKgcknJxR0HA2UHF98vEDE6GlTqgTqC7FPJ 52UiYiX8jI3vICV5oyCOgtntFmjfF/7a6KfVgjGce0U+WvVSPp0OBXzkwVTOfUlR1PQwxqrOsLKB 9OfGs5YaKt8NrMxuodo8x+5gLfKMGWhQ6nxMgzBIsdTyZT8hnXCdbsAKeTJ2PWrjvQe0vwUpkfjp 1O8bdOxb5rR2WKfddrYdZ7sDH52+0+5399kC+GUndzP2EyFeTcCB4B84nVIqVkTRhyBopemsv729 WCxaINag14LyT29jm9BzGovEnQniNXEX8mpDV4sQ9u5SHonA3EFLosZsDqpUq1KYm19P4DdypUGx hTOdQX3jx33CYqUbEho8GuPZOp24wDzgY54OgxueSUWfpanIxXTu3kJocUdwRCd8CMULHob4e04q AItmPh8HEWxgxMN40WJyqxDjIPYIWouN4nRC+PiBTW9Qi1iJkgRaf/phaIv4HkMPxp0xVH5AbakU ikLQFW+tBv/9AVNAORx+nyEcVprBzePMgCLld5rAoWkAN48zgMNV9R/+kPIPf0j1h0rxueK+Qfta 7fDPcBi5Uz4cQjasKWI1Gr7lCYpNznRavdZBqyNnfC68JKBaQM4q0xgFVAkLtAzUhhau76buyAVp h8EogYQoqbhzQEokgV/hjAAE3nvvgzAEnrfZX7kbNY9Ddw4W+DmYwVkikWh8CgKUWH94i79khiFn 54ma22g7MjC2Juk0lGhh4IGRKCn8vv3+9IpRTUJ22N9siApXUVt/n7ktvLu7hy81EWo7ay60Juso mTBwfnExJffkxYE2Ievi73ShRf+RpZGeyYcRRY6ybJCZiGaDf0ueZOG0CSeatm7T0jVTsnKNBp+z PlPe6JejeZ4rI8vrqSEewM0Tr7qValBddjqm+td6oAWpxaTz+2Zh5lBbzg6k7p28C72mFjyjSlUZ v6xHy6fSNZiRiSkyTGfnwHV227az25HhRnWjoMCRVx66UaWOJqvXTYbc7l8+bIyFbbf+5YCX/2uz SGHh2Zo6Q89tUYkBwhsn8RTqYJFicBPgNBD/9HsDhLUhDajgrIT2DRg2O46nU4h5VWg00aKHHRqf /tjquL318CDjcXCtEeRfW113dwXDeDWhoHoudg2MxxSgyJ7dwUE6myhqw+mNJT81+kqVeMuXzCM4 6IXjbFBZPY61qEEwnHDX54lVV/dnsvpsTeq2gVH5o95pDP0gEYMvGceta54OpeKGAGI1vhYWxx8p voHkAveeBKM5Bv1WwoMIkpUb4j2dkp5VJ/gh5Nt6o4pSS0WZAatX3WrWDZxneOgWqcx205seOC4c VW8Dn8s0T40XL5W3o3nUusUrDLtARwR4HJMlgR9zgW2XZ0okrKZEMVGpUGpKbgMURR8KejJ2HVP6 E6tqe6bkBm4nHxaB2SCzcbLEDAxk3HkI1cI8CjmsJmbcC8YB5HvI5jxZBEYTTgtu3bIGIOlIck6Z FoTcaqmXTShZzBDdtt3rGLsBbTywF6IKxpnrWEqMFFzYMxSJIGesqKCIO37xIm/YWhMeQnWGcS6e cjYL3RSAp8LkXvlNLIwxz4NNxKLFo9sgAZsDe7Xqx7//XrdZ0YiVZ926iZxvNIpOhLHW88rGTeQ3 EDo+rjdgXypEQy3b6hTJrpLMmf0C+PWvsIDnFQWVuUirWvikKcft9eydXkG+izi5gfqR6rgwiG5A sOARUEZO2SKAMvC961V1BimILEVLSx2TaN13wcyiepn9NFmWhzIb0NpsybWHIoYYMI1vuVVrfg7t 5l2t7PKcukV9KOyF+KadvAui+Z2d9zk7VNeqfmbR0SAczIKUY2pIsXP60/Exc0fAFZgIhOkQ7VFo SnoLDwgnxOU7jxUOalIWtVC5t1R3vCZS33vxYrNIum13r2dj70bHnZSLdJjw6wQ/WCq5Ze74jJ0n wTUaDVOFOXoaHVck+iwWKfczaAg1r3kCO4dQ9FsMhwygFmFhniWxpgiu/6Iq4Ti5zgJDVttjLYy8 UMg14z8oLMBDYSy7QmFYy60cRJ5omwYCXwwZqAzRxAA1ADeuj4rZq0aRS0bPPGZaKmL2mURv5UTq jVrDrlhgNk84aAXXwJcUpUVK9PF4qy2gGfF5moCIzd2KNYsAzvctokK+95hVvmkF0ERRT1p2dCLj ASYYVmMvVglpkWHjsrC9MldoD01YtZqpnB3K3NkjGjQgsiQqAYLK/RJlZTjV1FUSAplAFaFNDDY2 I++JaYWWdqCC933N7fOxaXw1oSJDxRplwgWnQ70PCsU3QjHW2y61BOo5bp3a7QFoYSz9F8uUAiVL FQMQupIk8H0eof+i1NCdUCkeRD84OFUzqJW3wqMJI/W5BoZiD2gP6wb8XK8EyN26ROObior1ZaVM wqRkPMcoHyhUk3IzOSOWKWdbS10JxBiQuy+QQo+RhLJaTRMuh3rQGKwF7GEzlnZkK9RyjIeUAsDV eVRi00Ia3a4murJjKCwE7Gmopetb37kTUbFczkSBt0aj6DWVh5SCGZ9Gt/ENJ7tVqmNKneieMtLU GCpHJ+Vp7M/B8ovOEPAW4EHBXoeaAI7AspkkzSZNOF/xgkKFmttMgSqUK5xulDAhUtZ3odjAaxiO PTo4ZqbScxdu4psMxWg+Ll3TabQv/a9l8tRlzAmbHKqhlnrD0rYrXLfxeARpyY9B0E7dWKevj4IX gyadSgKlwnhjvbFavRcoFeVTmGpdXr25Orm8usTy4OtaqLPzqwJUxcnYDEorhg5CcZPr2y9O/6vx hnQWBqm1ggxxghViA97zlCn6PCxQrbAwiN8MCyJeYdgaXc18qasOSr0ggtUycwWPDrMFLFVYlhBV 5V12FryKnMOh2KqB20T04AAkIQr5sqiJKeRwCz9h3l+xrJIMpDNt8AM81SmXyvzI2PZKh724Of3V lME6WXY7dvfAc3b2bWcPr/+obWRhi3tQU/3oWv5i26h2B7qL/XpNF/vNazibyP51TkA2sQePbGDn ePMkHGzuURvAs+VQ7XRgbDWf96a+PAMM2J9GF4b1s14G/MnucwT4M6eYddSsGplWzS7dyK78fAFR qdvXJT7hBAzZvsuFXYAfZLcU+rIhL+x0cKkEzq4gcnjzfLICX7iYyHGwxTwsHmwMnNgzYaVC1zCv ryzK4PJSYgVcXVXk0KD1oiRNaLq6yGHR67B65kkF7CojObhiZiMj6s6impHsMiSHV8+4hAH/pa6v ddAUNWBuqgagYbIaLjPZnKAy3XputVn5htZcMObNXU9ZwvaLJ2ogpppX/bwZe58TMnwi47zCNR7o tqofEc8TD51r1VN6e3Zv33P2HdvZ7zzkacBU4g5ls1cMpjfwSTyA07C/ll4RfOsCj91k4+vWTtt1 9ndgJ/h6+9kz9i72oGD45CYBXnFSUoZR1Xppqhqtz2ozdfOhbxNkcdccsaxpWZOoJwC+6RoDNfvw nRpCbcPpeM3FhTGNdxfyVl5CN5vswzyikzWlyew2XL8cKF19KtE/Dv+mAr/rdX/0fYRm4Qfu1p2u 50g+6OIjb8bVBdMOpWqcbLlVyEIZocE7Pa/Ts7s9ov4mxoY8FCIkE3xh5EP5ramQxGz5TBJhstca OJ6t+96FghVPR6xeLB7q1GsEAnR5jpWOrh60n8PZnUgEkepx6fYFdRELTQwqlFCgMovgFXpF3R2r EaCEiU5XTfooZTK74PUEz/fEl+KHRJgddoybg3wPRAUZq6urEPxqXpSavMuGXRpM9SIguxZj1ucJ OCFtgojkGGgjSTwCl10yZNtXEdTGQ1FAdyau4hGMJ2ZUlz1jYhLPQx9n5YMJjrvCFm7C8TO9tlQ1 lFKoJKtINdbLEM6j3gLOuJnYvt1ObjI76ex73X3pUyh8F8qB+EZLKbOigpc+HHFC7t6QeUPIeBB4 SqBiPkPrrMAohyqZuRDwwaBGKQ7Ej8/QxKbwVgbMAt0KBQhZV+ivayIcRTBU+DU2SgkrE+JGWhXR rorW/44AuGMfdL0de0/fYpNCaHtKjfqggtvDS37u21jLjmL6Pk1+ofXKvBR+Jd+t/Y3zGZx5XO8G uRoDNgQEfOODJrlkE7pHsBm+6vbJ8P2A4wUaGuw4SIR8d4avrVXfMMBoMI/An7GDiKkX/P00Iz3U X7Vps8WERyqEyfAOkU1MOHhP4gZQEObbwSdiRfwBa8sdUJ8PHzbEM/kNEJvxJBlQxxYqihmEFe6r P10vnbsh/dHoq9LlOoRgExapq6lZgm8e6m+PTt/169iJkgsUZ+lzXR9TgjGurt/j4kJ9BQjDBpDk pAqufh1jsJMA+ncre1VsLqQ2V0lFT9ZzIdgm4+pzWajFv18wR5sJtfVAn+MwXpABYvh/xd4f/X56 dqVPwO6dpPz+9EyONtV8kwjBH3g1TtfjA9Z5/ny3q2dOz4yZJk1JhI8ljJ5m6MeT16vvS154jn8g URCLK68zaUF61SuTOR5ZZJyRj/Hw/Z1SFrMWeGU4SiCyyzSW1TKyVgE5b9NjJRWw8CT5H7gwdS+k boviBkDLGNEvfcueZtXRlCQXKNDOf0klYfIcYcK6dcPAN99fXvOkjtRkb4SdKyOh10WaFWr/kATU G0TjjJd9vRNmb6k7Jr/si31gIa9ivsA64/6pjTse9999rTe0ONU3FZN4IWR3lr5RgE1exaGbJO4S QOG3bN5lQYOesuCXmZ4/p29QaV5dGANQnAEkLafbfPRWj6Ef4/DTAU1rAtqvrLq8g63ruFQn5tQX KKaBmOIXzjJfHyANI2QhafM9tqFZ/aN8/ejDh6O/s+Pzs6uTs6vLfr0ERbdx8utGKAYdInCoBKlV 4s+nM+sWGP50evLZIGyyU7hAzodX7WkeZYFKfY8Cv+wkFUbqsrHCTdDaXlLFhMMyeaE7yze/PnAW hEIrXl6cke+VtFeS0q2+BycgYwIEUTWljDj7egno40G1KvVlX5IvsVC2s+JKuaT6ZSn+AEt1MZsn QTwX9RX2lPQSjqdsi2uqMr8MjFU2CzbAL3uule4apuaex7lfSEqDEgeNgvJ8HvKUW8HdI/UDHhkI /e0QiwqAIvWyB+EVmbS18gz+KP+6OLq87CuJcb2f9eDKwESeoF/9LA7Zz6LOfmYWN3I5K/yPFWSf v+S7VYJcJ7v/Q6ZovLAsHEh01nwuD1v/U8vR7rZtA3/vLQTXhZNUUkJJbmwMLprZzmLMa42mBjoM g+ep0ibEiFPbcdYO27OPdyQliiIpCWgN2LB9d+TxeDqS90FYD6A2H1iDhEPDYgBXQsB2kJsEQBjR NRkxRnlB69E3wTGROIpJ6AZEaYZIaGWI1zeC2K7HDMbtUQnMH0cpLVdqiN3b0QCbbbu8ErZu7DJc hZWGpQIrA6sgqEMDBM4uRVih9zupG2EtkW6gr/DpF+hiXwTK/IpbE95kaYdSsCF2RqUm6wmLnatE yArimHNJ1ahc8TwyVzUjh/XNIGUTp8cpuJL0Rb5Vo9Lmsi1Bvrd8wa58+AadSKcCDzqxPscoTiO0 bwOWRGrGkoUqqWjTETclKQv2m3WkCpfVKg7ikOps32QFA7MVhFNbWGdwApvBES2QgLgkGMbhwI1e qpxEc7NFDi/MwKjvXxiBAfGjJhaWPv1rPEZxm3Xagii3WY2IegHp9xoheiEZ+GTYCHl/2Bm4185a SdaVeStLWwUr8lbBisTrLL5N8A1ptfKXaOtmwIaqmQMLunkWSBi4JCJxNHSxlllrh4LLgdHwewEJ jcAhnRIjMLj0w0ZWmYkzcM5kB8NpW1LJllr0NafkImtP07KfHhWgHw57LXhaSh1YFyicN/PyhTNn BLO5s7Tthw0Nv2n+2hKXJGvRdsscNqZq3Vcxjy04k2eSRC/hdrwY/ZDVU2ITTwk6qCDcDN49dP2g J094pIRbxnWO3DnzJXsAlxP9Iz+npngeo6eg63VxORZQpsybs33y75LP+5PSwVbjKTqmUs7oEXtN TxWcdYpX/Gyffk1/qx6TAUpP/KnunAxhyez+MVFAz5wF3TBDMQMGyiATByqYszg7sLsnIP2FnpNZ RRSL4FU7Bg/pyTrF29fw+zGtHOMNoy6NTNBCWwa8mmHWDBdeIkFMT6z4yuD1DEIjTwnMjfNXsktc /MHu8siP9w6P02OyNjizMdCJ7sNKc8kxuWeKJkSscSmgSr3fVYbATvDP9/SkHid7dFikLtfSwjGZ Fi7Lkihr3ANfp3Omn1VXBy+uSisODLOLR3HvHHYZFR7zXKBrlD35lbiVaKbq6nz+MQ+g1bsyYXhy 08XqweNgkFddG85lYUCWUNok/Iv4j/fZwRYw3u8sF2IjMMXLfiCLCAp1w8D/T4t7iDfmhiiQMvXp MdtY7uZmSCKvRosB3qMa8Pn2zoxBpWErhQfwOUSb4CJLX0MrA+kO7iKmb6oi33XnDjV9Xuqc+Zs1 /djCl624v4VCPQ+UYhRvEvqwa1Cds9e3N/PZD6vb5fX17MNr+vvnt5PlfCr+sDDMssqocmk5lqGU ZbrnjPHTYvQc5/fF1fsbcdU61yrIrfleVF6xXCI05iIOl6nWR9fkJvujaO58nx0S72Ed363/TPY9 FjsGNAhzYtZ3UptSpuc0rOO0vkmZ07AJp0HoQo4RfIJKLH5Zzd6M58vJdDWZvRt1uv887Kjq/P1v Vawdjg/zr+Ba5NUR6tW8p7DD8Wt6CtWe0FGQuwvgVekU1iFe2K0fIacqupYpGoyzda842sa9VsYc Xa6jgdsnvJG388lqvFhcz69+vB11xTcOzAEdb9YtM+nkuIKdj/cbpTJcFIaLunCW7cT/pA+12t3V eDW+mY5/Wt1MrybTdydKcb4r8dOV+Xa8ia4EfkQ6EkmJQr8soWFh2RzcHP8PS4FyS/liAAA= --Apple-Mail-9-448866243-- From njriley@shrug.csl.uiuc.edu Mon Sep 29 01:10:27 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8T6ARM03983 for ; Mon, 29 Sep 2003 01:10:27 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9) with ESMTP id h8T6ANQZ057775; Mon, 29 Sep 2003 01:10:23 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9/Submit) id h8T6AMHW057774; Mon, 29 Sep 2003 01:10:22 -0500 (CDT) (envelope-from njriley) Date: Mon, 29 Sep 2003 01:10:22 -0500 From: Nicholas Riley To: PieterB Cc: metakit@equi4.com Subject: Re: [Metakit] Mk4py changes Message-ID: <20030929061022.GA57644@uiuc.edu> Mail-Followup-To: PieterB , metakit@equi4.com References: <20030927064004.GC19396@uiuc.edu> <20030927090310.A36088@gewis.win.tue.nl> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Dxnq1zWXvFF0Q93v" Content-Disposition: inline In-Reply-To: <20030927090310.A36088@gewis.win.tue.nl> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sat, Sep 27, 2003 at 09:03:10AM +0200, PieterB wrote: > Do I need to upgrade autoconf, or should I change something? > Thanks for all your effort! It would be great if the default metakit > would work with Python out of the box. No, actually, I think downgrading autoconf would have worked. There were some constructs in configure.in which were deprecated and no longer worked properly, and some macros were in a place where they didn't make any sense. In particular, AC_LANG_CXX was doing something strange - not setting the executable extension, for example. Try this one: I built with Python support on FreeBSD 5.1 just five minutes ago and it worked. So hopefully it will for you too. :) The changed files are attached. The autogen.sh change is because install-sh copying moved into automake; it complains now that you aren't using automake, but works anyway. -- =Nicholas Riley | --Dxnq1zWXvFF0Q93v Content-Type: application/x-sh Content-Disposition: attachment; filename="autogen.sh" Content-Transfer-Encoding: quoted-printable #!/bin/sh=0Arm -f aclocal.m4 && \=0Aaclocal && \=0Aautomake --foreign --add= -missing --copy; \=0Alibtoolize --copy --force && \=0Aautoheader && \=0Arm = -f configure && \=0Aautoconf && \=0Arm -rf autom4te*.cache=0A --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="configure.in" dnl Process this file with autoconf to produce a configure script. AC_PREREQ(2.5) AC_INIT(Metakit, 2.4.9.2) AC_CONFIG_SRCDIR(../include/mk4.h) AC_CONFIG_AUX_DIR(scripts) AC_CONFIG_HEADER(config.h) AC_LANG(C++) # Checks for programs. AC_PROG_CXX AC_PROG_INSTALL MK_TARGETS="core" MK_INSTALL="install-mk" AC_ARG_WITH(python, AC_HELP_STRING([--with-python=PATH], [build the Mk4py extension for Python with headers in `PATH/include/python2.3'; install the Mk4py modules in `PATH/lib/python2.3/site-packages'. If PATH is of the form `HEADER,LIB', search for header files in HEADER, and install Mk4py in LIB. If you omit the `=PATH' part, use the standard includedir and libdir.]), [with_python=${withval}], [with_python=no]) AC_MSG_CHECKING([for Python configuration]) case "$with_python" in "yes" ) PY_INCLUDE_DIR="${prefix}/include/python2.3" PY_LIB_DIR="${prefix}/lib/python2.3/site-packages" ;; "no" ) PY_INCLUDE_DIR="# no Python headers" ;; *","* ) PY_INCLUDE_DIR="`echo $with_python | sed -e 's/,.*$//'`" PY_LIB_DIR="`echo $with_python | sed -e 's/^.*,//'`" ;; * ) PY_INCLUDE_DIR="$with_python/include/python2.3" PY_LIB_DIR="$with_python/lib/python2.3/site-packages" ;; esac if test x"$with_python" = xno; then AC_MSG_RESULT(not enabled) else MK_TARGETS="$MK_TARGETS python" MK_INSTALL="$MK_INSTALL install-python" if test -f "${PY_INCLUDE_DIR}/Python.h" ; then MK_TARGETS="$MK_TARGETS python" MK_INSTALL="$MK_INSTALL install-python" AC_MSG_RESULT($PY_INCLUDE_DIR and $PY_LIB_DIR) OLD_CPPFLAGS=$CPPFLAGS CPPFLAGS="-I$PY_INCLUDE_DIR $CPPFLAGS" dnl Mk4py doesn't include config.h, so include in $CPPFLAGS AC_CHECK_HEADER(unicodeobject.h, CPPFLAGS="$OLD_CPPFLAGS -DHAVE_UNICODEOBJECT_H=1", CPPFLAGS=$OLD_CPPFLAGS) else AC_MSG_RESULT(can't find ${PY_INCLUDE_DIR}/Python.h) fi fi AC_ARG_WITH(tcl, AC_HELP_STRING([--with-tcl=PATH], [build the Mk4tcl extension for Tcl with tcl.h in `PATH/tcl.h'; install the Mk4tcl package in `PATH/../lib'. If PATH is of the form `HEADER,LIB', search for tcl.h in HEADER, and install Mk4tcl in LIB. If you omit the `=PATH' part, use the standard includedir and libdir.]), [with_tcl=${withval}], [with_tcl=no]) AC_MSG_CHECKING([for Tcl headers]) case "$with_tcl" in "yes" ) TCL_INCLUDE_DIR="${prefix}/include" TCL_LIB_DIR="${prefix}/lib/tcl8.4" ;; "no" ) TCL_INCLUDE_DIR="# no Tcl headers" ;; *","* ) TCL_INCLUDE_DIR="`echo $with_tcl | sed -e 's/,.*$//'`" TCL_LIB_DIR="`echo $with_tcl | sed -e 's/^.*,//'`" ;; * ) TCL_INCLUDE_DIR="$with_tcl" TCL_LIB_DIR="$with_tcl/../lib" ;; esac if test x"$with_tcl" = xno ; then AC_MSG_RESULT(not enabled) else MK_TARGETS="$MK_TARGETS tcl" MK_INSTALL="$MK_INSTALL install-tcl" if test -f "${TCL_INCLUDE_DIR}/tcl.h" ; then AC_MSG_RESULT(found ${TCL_INCLUDE_DIR}/tcl.h) MK_TARGETS="$MK_TARGETS tcl" MK_INSTALL="$MK_INSTALL install-tcl" AC_MSG_CHECKING(for Tcl package installation location) AC_MSG_RESULT(${TCL_LIB_DIR}) else AC_MSG_ERROR([${TCL_INCLUDE_DIR} directory doesn't contain tcl.h]) fi fi AC_ARG_ENABLE(threads, AC_HELP_STRING([--enable-threads], [build with threads]), [threads_ok=$enableval], [threads_ok=no]) AC_MSG_CHECKING([for building with threads]) if test "$threads_ok" = yes; then MK_THREADS="-Dq4_MULTI" AC_MSG_RESULT(yes) else MK_THREADS="" AC_MSG_RESULT([no (default)]) fi # Libtool support AC_LIBTOOL_WIN32_DLL AC_PROG_LIBTOOL # Checks for libraries. # Checks for header files. AC_HEADER_STDC # Checks for typedefs, structures, and compiler characteristics. AC_C_CONST AC_C_INLINE AC_CHECK_SIZEOF(long,4) AC_CHECK_TYPES(long long) # Checks for library functions. AC_CHECK_FUNCS(mmap memmove bcopy) # Deal with shared lib differences SHLIB_SUFFIX=".so" MODULE_SUFFIX=".so" STRIP_FLAGS= LIBTOOL_SHLIB_FLAGS= LIBTOOL_MODULE_FLAGS=-module case $build_os in darwin*) SHLIB_SUFFIX=".dylib" # MODULE_SUFFIX remains .so (Darwin doesn't care) STRIP_FLAGS=-S ;; hpux*) SHLIB_SUFFIX=".sl" MODULE_SUFFIX=".sl" if test "X$build_cpu" = "Xia64" ; then # libtool insists on .so files for HP-UX/Itanium :( SHLIB_SUFFIX=".sl" MODULE_SUFFIX=".sl" fi ;; cyg*) SHLIB_SUFFIX=".dll" MODULE_SUFFIX=".dll" ;; esac # --------------------------------------------------------- # Special hackery for hpux machines to allow usage of the # metakit C++ library from within Tcl (C application). # # The special configurations are: # # (1) CXX=aCC, CC=cc, build_os=hpux*, build_cpu=hppa # (1a) CXX=aCC, CC=cc, build_os=hpux*, build_cpu=hppa2.0w (true 64 bit) # (2) CXX=aCC, CC=cc, build_os=hpux*, build_cpu=ia64 # # [Ad 1] Link against -lCsup and insert cpprt0_stub.o # [Ad 1a] Link against -lCsup # [Ad 2] Link against -lCsup # # Variables to substitute # = LINK_SPECIAL_FLAGS # = LINK_SPECIAL_FILES LINK_SPECIAL_FLAGS="" LINK_SPECIAL_FILES="" case $build_os in hpux*) if test "X$build_cpu" != "Xia64" -a "$threads_ok" = yes; then # Handle special link requirements of hpux-parisc in threaded case. LINK_SPECIAL_FLAGS="$LINK_SPECIAL_FLAGS -lpthread" fi if test \ "`basename $CXX`x" = "aCCx" -a \ "Z`basename $CC`" = "Zcc" then # Native HP compilers if test "X$build_cpu" = "Xia64" ; then # Itanium: No -lstdc++ LINK_SPECIAL_FLAGS="$LINK_SPECIAL_FLAGS -lCsup" LINK_SPECIAL_FILES="" else # pa-risc: The exact architecture of the cpu is now important. # Actually: The exact nature of the code we generate is important. case $CXXFLAGS in *DA2.0W*) # True 64 bit, no stdc++, no stubs LINK_SPECIAL_FLAGS="$LINK_SPECIAL_FLAGS -lCsup" ;; *) # Everything else LINK_SPECIAL_FLAGS="$LINK_SPECIAL_FLAGS -lCsup -Wl,cpprt0_stubs.o" LINK_SPECIAL_FILES="cpprt0_stubs.o" LDFLAGS="${LDFLAGS} -lstdc++" ;; esac fi fi ;; solaris2.*) # Tested for solaris-2.8. if test "x`basename $CXX`" = "xCC" then # Native SUN C++ compiler # Depending on 32/64 mode choose between runtime libraries # Stupid which can insert warnings into the output # instad of using stderr. Bah. base=`which $CXX 2>/dev/null` ; # Full path of compiler. basea=`echo "$base" | sed -e 's/^Warning: ridiculously long PATH truncated//'` baseb=`dirname $basea` ; # Strip name of compiler basec=`dirname $baseb` ; # Strip bin directory CCRUNDIR="$basec/lib" case $CXXFLAGS in *xarch=v9*) CCRUNDIR="$CCRUNDIR/v9" ;; *) : ;; esac LDFLAGS="${LDFLAGS} -L${CCRUNDIR} -lCrun" else # Setup not using CC, use -lstdc++, like for case '*' below. LDFLAGS="${LDFLAGS} -lstdc++" fi ;; linux*) LDFLAGS="${LDFLAGS} -lstdc++" LINK_SPECIAL_FLAGS="-rpath ${libdir}" ;; *) LDFLAGS="${LDFLAGS} -lstdc++" ;; esac AC_SUBST(LINK_SPECIAL_FLAGS) AC_SUBST(LINK_SPECIAL_FILES) # --------------------------------------------------------- AC_SUBST(MK_TARGETS) AC_SUBST(MK_INSTALL) AC_SUBST(MK_THREADS) AC_SUBST(PY_INCLUDE_DIR) AC_SUBST(PY_LIB_DIR) AC_SUBST(TCL_INCLUDE_DIR) AC_SUBST(TCL_LIB_DIR) AC_SUBST(EXEEXT) AC_SUBST(CPPFLAGS) AC_SUBST(LDFLAGS) AC_SUBST(SHLIB_SUFFIX) AC_SUBST(MODULE_SUFFIX) AC_SUBST(LIBTOOL_SHLIB_FLAGS) AC_SUBST(LIBTOOL_MODULE_FLAGS) AC_SUBST(STRIP_FLAGS) AC_OUTPUT(Makefile) --Dxnq1zWXvFF0Q93v-- From njriley@shrug.csl.uiuc.edu Mon Sep 29 01:13:56 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8T6DuM04269 for ; Mon, 29 Sep 2003 01:13:56 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9) with ESMTP id h8T6DtQZ057881 for ; Mon, 29 Sep 2003 01:13:55 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9/Submit) id h8T6Dt1M057880 for metakit@equi4.com; Mon, 29 Sep 2003 01:13:55 -0500 (CDT) (envelope-from njriley) Date: Mon, 29 Sep 2003 01:13:55 -0500 From: Nicholas Riley To: metakit@equi4.com Subject: Re: [Metakit] Mk4py changes Message-ID: <20030929061355.GA57814@uiuc.edu> Mail-Followup-To: metakit@equi4.com References: <20030927064004.GC19396@uiuc.edu> <20030927090310.A36088@gewis.win.tue.nl> <20030929061022.GA57644@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030929061022.GA57644@uiuc.edu> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Mon, Sep 29, 2003 at 01:10:22AM -0500, Nicholas Riley wrote: > No, actually, I think downgrading autoconf would have worked. There > were some constructs in configure.in which were deprecated and no > longer worked properly, and some macros were in a place where they > didn't make any sense. In particular, AC_LANG_CXX was doing something > strange - not setting the executable extension, for example. Er, I meant AC_LANG_CPLUSPLUS here, not AC_LANG_CXX. Sorry. -- =Nicholas Riley | From rcohen@articque.com Mon Sep 29 11:13:06 2003 Received: from mwinf0401.wanadoo.fr (smtp5.wanadoo.fr [193.252.22.27]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8TGD5M23089 for ; Mon, 29 Sep 2003 11:13:05 -0500 Received: from articque.com (ATours-101-1-4-15.w80-11.abo.wanadoo.fr [80.11.89.15]) by mwinf0401.wanadoo.fr (SMTP Server) with ESMTP id 99C0B58000CA for ; Mon, 29 Sep 2003 18:12:58 +0200 (CEST) Message-ID: <3F785A06.1080109@articque.com> Date: Mon, 29 Sep 2003 18:12:54 +0200 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] find select and search, help once again please Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I'am puzzled.... I need to do a search, in a table of 15000 records. Below is a sample code and result output Summary: Search() ======== => many errors like : error search for key '836 my key 836' found idx 15000 this key is in the table, I see it with kitviewer, 3 records have this key. => when found, the value is sometimes not the good one : idx=498, key='195 my key 195', foundidx=5850, val='value for key 1950 [dum=5850]', avg=0.188377 it should be the value for key 195, not for key 1950 ! => If I increase the TOTAL to 10000 instead of 5000, then every search is found with no error ! => the search is very quick, but does not provide the result ! what does it do exactly ??? Find() ====== result ok, but quite slow. If I hash my table, the result is very quick, but I cant have multiple key ! (which is a problem for me) Select() ======== result ok, but even slower. and If I hash the table, then the result is better but still very slow. (3 times quicker) CONCLUSION: =========== - I can't do a quick search in a multiple key table - I can't do a quick search for a range (functin SelectRange uses the same process as Select) I dont need the performance of a Cray II running an Oracle Server, but 62ms per selection is too much for me (it takes 6 seconds for 100 searches !). Could anybody help me please ? Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 ========================================================== #ifdef ART_DEBUG #define TOTAL 1000 #define TOTALMODULO (TOTAL/10) #define TOTALSEARCH 100 #define TOTALSEARCHMODULO (TOTALSEARCH/10) #else #define TOTAL 5000 #define TOTALMODULO (TOTAL/3) #define TOTALSEARCH 500 #define TOTALSEARCHMODULO (TOTALSEARCH/3) #endif // search perfs void execute3() { remove("testperf.db"); c4_StringProp key("key"); c4_StringProp val("val"); c4_Row addrow,findrow; c4_Storage database("testperf.db", true); c4_View strings=database.GetAs("strings[key:S,val:S]"),selection; // TEST HASHING //c4_View viewsec=database.GetAs("sec[_H:I,_R:I]"); //strings=strings.Hash(viewsec,1); // char strkey[500],strval[500]; unsigned long idx,idx2,totaltime,counttime,time,randomnb; long foundidx; struct _timeb tm1,tm2; printf("add\n"); totaltime=0; counttime=0; for (idx=0;idx0) { printf("idx=%d\n",idx); } } printf("avg=%f ms/add\n",(double)totaltime/(double)counttime); _ftime(&tm1); database.Commit(); _ftime(&tm2); time=(tm2.time*1000+tm2.millitm)-(tm1.time*1000+tm1.millitm); printf("commit=%d ms/add\n",time); printf("search\n"); totaltime=0; counttime=0; for (idx=0;idx=strings.GetSize()) printf("error search for key '%s' found idx %d\n",strkey,foundidx); if (idx%TOTALSEARCHMODULO==0 && idx>0) { printf("idx=%d, key='%s', foundidx=%d, val='%s', avg=%f\n",idx,strkey,foundidx, (const char*)val(strings[foundidx]),(double)totaltime/(double)counttime); } } printf("avg=%f ms/search\n",(double)totaltime/(double)counttime); printf("find\n"); totaltime=0; counttime=0; for (idx=0;idx=strings.GetSize()) printf("error find for key '%s' found idx %d\n",strkey,foundidx); if (idx%TOTALSEARCHMODULO==0 && idx>0) { printf("idx=%d, key='%s', foundidx=%d, val='%s', avg=%f\n",idx,strkey,foundidx, (const char*)val(strings[foundidx]),(double)totaltime/(double)counttime); } while(foundidx>=0) { _ftime(&tm1); foundidx=strings.Find(findrow,foundidx+1); _ftime(&tm2); time=(tm2.time*1000+tm2.millitm)-(tm1.time*1000+tm1.millitm); totaltime+=time; counttime++; if (foundidx>=0 && idx%TOTALSEARCHMODULO==0 && idx>0) printf("idx=%d, key='%s', foundidx=%d, val='%s', avg=%f\n",idx,strkey,foundidx, (const char*)val(strings[foundidx]),(double)totaltime/(double)counttime); } } printf("avg=%f ms/find\n",(double)totaltime/(double)counttime); printf("select\n"); totaltime=0; counttime=0; for (idx=0;idx0) { for (idx2=0;(long)idx2; Mon, 29 Sep 2003 11:26:23 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 0A44E8399B; Mon, 29 Sep 2003 18:26:16 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 8CFA283999 for ; Mon, 29 Sep 2003 18:26:15 +0200 (CEST) Date: Mon, 29 Sep 2003 18:26:17 +0200 Subject: Re: [Metakit] find select and search, help once again please Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F785A06.1080109@articque.com> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-3.0 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > Search() > ======== > => many errors like : > error search for key '836 my key 836' found idx 15000 > > this key is in the table, I see it with kitviewer, 3 records have this > key. > > => when found, the value is sometimes not the good one : > idx=498, key='195 my key 195', foundidx=5850, val='value for key 1950 > [dum=5850]', avg=0.188377 > > it should be the value for key 195, not for key 1950 ! > > => If I increase the TOTAL to 10000 instead of 5000, then every search > is found with no error ! > > => the search is very quick, but does not provide the result ! > > what does it do exactly ??? Binary search. It can only work if the view is sorted on the key, and the key is the first property. > Find() > ====== > result ok, but quite slow. > If I hash my table, the result is very quick, but I cant have multiple > key ! (which is a problem for me) You could consider grouping first on the (non-unique) key, and then hashing the resulting view/subview structure? > I dont need the performance of a Cray II running an Oracle Server, but > 62ms per selection is too much for me (it takes 6 seconds for 100 > searches !). Could anybody help me please ? Have you tried a plain brute force loop? If it still isn't near the performance you'd expect, please post a (short) code example. There are a few things to avoid unnecessary copying. I don't think 62ms to go through 10k rows or so is accurate, there must be something else going on... -jcw From mss@mawhrin.net Mon Sep 29 12:35:38 2003 Received: from only.mawhrin.net (postfix@host4.transas.vianw.co.uk [194.119.138.213] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8THZcM30486 for ; Mon, 29 Sep 2003 12:35:38 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id 3A08513C46; Mon, 29 Sep 2003 18:35:35 +0100 (BST) Date: Mon, 29 Sep 2003 18:35:35 +0100 To: metakit@equi4.com Subject: Re: [Metakit] Mk4py changes Message-ID: <20030929173535.GA29009@mawhrin.net> Mail-Followup-To: metakit@equi4.com References: <20030927064004.GC19396@uiuc.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EVF5PPMfhYS0aIcm" Content-Disposition: inline In-Reply-To: <20030927064004.GC19396@uiuc.edu> User-Agent: Mutt/1.5.4i From: mss@mawhrin.net (Mikhail Sobolev) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 27, 2003 at 01:40:04AM -0500, Nicholas Riley wrote: > Only about a year late, I've fixed everything that's annoyed me about > Mk4py. There's quite a bit here, as I often stumbled on one bug or > brokenness as I was trying to fix another. >=20 > Since I've also had to move and rename some files, I don't think it'd > work well to provide a patch, so I've uploaded a new version here: Can you please be a bit more specific: did you change the files outside of mk4py? I mean, can I just take the tree that corresponds to the python interface and build it against the library installed in my system (debian unstable)? -- Misha --EVF5PPMfhYS0aIcm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (GNU/Linux) iD8DBQE/eG1nMYLUe7XufiwRAgF+AKDE/0AVFkbCYotaT6D7XO+dMbNLdwCaA5UG FBQybVwE+/5UkzM2zuPV9c0= =Vxrs -----END PGP SIGNATURE----- --EVF5PPMfhYS0aIcm-- From gary.h.merrill@gsk.com Mon Sep 29 13:21:25 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8TILPM01818 for ; Mon, 29 Sep 2003 13:21:25 -0500 Received: by usrtpnp1.glaxowellcome.com; id OAA22270; Mon, 29 Sep 2003 14:22:03 -0400 (EDT) From: Received: from keymaster.glaxo.com(192.58.204.209) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma021981; Mon, 29 Sep 03 13:50:12 -0400 Received: by usrtpnp2.glaxowellcome.com; id NAA15116; Mon, 29 Sep 2003 13:50:11 -0400 (EDT) Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma005230; Mon, 29 Sep 03 13:41:35 -0400 Received: by ussunt2.glaxo.com id NAA26568; Mon, 29 Sep 2003 13:41:43 -0400 (EDT) Received: from 166.71.134.118 by us9n52.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Mon, 29 Sep 2003 13:40:49 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h8THem406517 for ; Mon, 29 Sep 2003 13:40:49 -0400 (EDT) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Mon, 29 Sep 2003 13:39:44 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 09/29/2003 01:40:54 PM MIME-Version: 1.0 X-WSS-ID: 1366B12B738288-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Mysteriously growing Metakit db Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I'm just learning how to use this. So it's possible I've overlooked something. But this behavior is rather puzzling. I've created a MK database (using Python) with a rather simple view and a hash view on top of it. On entry to my application, the contents are checked and information is displayed. This works fine. Then, just as a test, I call a method in my application that gets each entry in the database, updates it, and puts it back -- at least that's what I think I'm doing. What I notice is that each time I run the application the database file gets larger (by 2K) -- even though my code is not adding any new rows. Why is the file size growing? What am I overlooking? Also, it occurred to me that in the code below I might be misusing 'getas' to access the existing view. But when I attempt to use 'view', it seems to return a valid Python object whether the named view exists or not!! So how can I tell whether the view is really there? (I guess I can attempt some kind of try/except access to the retirmed view, but that seems *really* kludgy.) Any help will be appreciated. The row format for the underlying view is 'Index[name:S,index:M]' The code for opening the database and setting up the view and hash view is: indexInfoFileName = join(indexDir, IndexInfoFileName) self.InfoDb = mk.storage(indexInfoFileName, 2) basicInfoView = self.InfoDb.getas(IndexInfoViewFmt) infoHashView = self.InfoDb.getas(IndexInfoHashFmt) self.InfoView = basicInfoView.hash(infoHashView, 1) self.InfoDb.commit() The code for updating looks like the following. As you can see, it attempts to modify existing rows and append new ones as necessary for infoInstance in self.itemsData: rowIdx = self.InfoView.find(name=infoInstance.indexName) binIndex = dumps(infoInstance, 1) if rowIdx >= 0: self.InfoView[rowIdx].index = binIndex else: self.InfoView.append(name=infoInstance.indexName, index=binIndex) self.InfoDb.commit() -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From njriley@shrug.csl.uiuc.edu Mon Sep 29 13:54:00 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8TIs0M04240 for ; Mon, 29 Sep 2003 13:54:00 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9) with ESMTP id h8TIrxQZ061939 for ; Mon, 29 Sep 2003 13:53:59 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9/Submit) id h8TIrwgm061938 for metakit@equi4.com; Mon, 29 Sep 2003 13:53:59 -0500 (CDT) (envelope-from njriley) Date: Mon, 29 Sep 2003 13:53:58 -0500 From: Nicholas Riley To: metakit@equi4.com Subject: Re: [Metakit] Mk4py changes Message-ID: <20030929185358.GA61747@uiuc.edu> Mail-Followup-To: metakit@equi4.com References: <20030927064004.GC19396@uiuc.edu> <20030929173535.GA29009@mawhrin.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030929173535.GA29009@mawhrin.net> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Mon, Sep 29, 2003 at 06:35:35PM +0100, Mikhail Sobolev wrote: > Can you please be a bit more specific: did you change the files outside > of mk4py? I mean, can I just take the tree that corresponds to the > python interface and build it against the library installed in my > system (debian unstable)? No, the Python interface actually requires it be built from a Metakit build tree - it links directly against the individual Metakit object files, instead of against the Metakit library as a whole. So Mk4py.{so,sl,dll} ends up containing the Metakit library as well as the Python interface. This was part of the setup.py script before I changed it, so I assumed there was some good reason for it and didn't change it. Perhaps I was wrong. :) Here's the build process I used on a Debian unstable system: ~/src% curl -O http://web.sabi.net/temp/metakit-2.4.9.2+njr2.tar.gz [...] ~/src% tar zxf metakit-2.4.9.2+njr2.tar.gz ~/src% cd metakit-2.4.9.2+njr2/builds ~/src/metakit-2.4.9.2+njr2/builds% python -V Python 2.3+ ~/src/metakit-2.4.9.2+njr2/builds% ../unix/configure --with-python=/usr [...] ~/src/metakit-2.4.9.2+njr2/builds% make [...] If you want to test: ~/src/metakit-2.4.9.2+njr2/builds% curl -O http://web.sabi.net/temp/regrtest.py ~/src/metakit-2.4.9.2+njr2/builds% sudo install regrtest.py /usr/lib/python2.3/test ~/src/metakit-2.4.9.2+njr2/builds% make test-python python ../unix/../python/test/all.py test_inttypes test_string test_unittest All 3 tests OK. Once you've run configure/make (with or without the --with-python option), you can also use Distutils to build. ~/src/metakit-2.4.9.2+njr2/builds% cd ../python ~/src/metakit-2.4.9.2+njr2/python% python setup.py build [...] ~/src/metakit-2.4.9.2+njr2/python% python setup.py test running test running build running build_py running build_ext running config gcc -pthread -E -I/usr/include/python2.3 -o _configtest.i _configtest.c removing: _configtest.c _configtest.i test_inttypes test_string test_unittest All 3 tests OK. Then use 'make install' and 'python setup.py install' (if you used Distutils) to install. -- =Nicholas Riley | From jyl@mod3.net Mon Sep 29 18:00:25 2003 Received: from mod3.net (mod3.net [69.55.237.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8TN0PM24247 for ; Mon, 29 Sep 2003 18:00:25 -0500 Received: (qmail 9094 invoked by uid 48); 29 Sep 2003 23:00:24 -0000 Received: from 192.18.42.10 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Mon, 29 Sep 2003 16:00:23 -0700 (PDT) Message-ID: <55666.192.18.42.10.1064876423.squirrel@mod3.net> Date: Mon, 29 Sep 2003 16:00:23 -0700 (PDT) Subject: Re: [Metakit] Mysteriously growing Metakit db From: "Jacob Levy" To: In-Reply-To: References: X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Gary Metakit uses the Commit model for making changes persistent. That means that until you call storage.Commit() the old data is the valid representation, and if you close the storage without committing, the next time you open it, none of your changes will be present. The key to your situation is in what you wrote: you read the data, check it, and *write it out* again. Since the old data is not overwritten (it needs to stay there till you commit, in case you decide to discard your changes) the storage grows. I don't know enough about the Python binding to answer the rest of your questions. --JYL > I'm just learning how to use this. So it's possible I've overlooked > something. But this behavior is rather puzzling. > > I've created a MK database (using Python) with a rather simple view and > a hash view on top of it. On entry to my application, the contents are > checked and information is displayed. This works fine. > > Then, just as a test, I call a method in my application that gets each > entry in the database, updates it, and puts it back -- at least that's > what I think I'm doing. > > What I notice is that each time I run the application the database file > gets larger (by 2K) -- even though my code is not adding any new rows. > Why is the file size growing? What am I overlooking? > > Also, it occurred to me that in the code below I might be misusing > 'getas' to access the existing view. But when I attempt to use 'view', > it seems to return a valid Python object whether the named view exists > or not!! So how can I tell whether the view is really there? (I guess > I can attempt some kind of try/except access to the retirmed view, but > that seems *really* kludgy.) > > Any help will be appreciated. > > The row format for the underlying view is 'Index[name:S,index:M]' > > The code for opening the database and setting up the view and hash view > is: > > indexInfoFileName = join(indexDir, IndexInfoFileName) > self.InfoDb = mk.storage(indexInfoFileName, 2) > basicInfoView = self.InfoDb.getas(IndexInfoViewFmt) > infoHashView = self.InfoDb.getas(IndexInfoHashFmt) > self.InfoView = basicInfoView.hash(infoHashView, 1) > self.InfoDb.commit() > > The code for updating looks like the following. As you can see, it > attempts to modify existing rows and append new ones as necessary > > for infoInstance in self.itemsData: > rowIdx = self.InfoView.find(name=infoInstance.indexName) > binIndex = dumps(infoInstance, 1) > if rowIdx >= 0: > self.InfoView[rowIdx].index = binIndex > else: > self.InfoView.append(name=infoInstance.indexName, > index=binIndex) > self.InfoDb.commit() > > -------------------------------------- > Gary H. Merrill > Director and Principal Scientist, New Applications > Data Exploration Sciences > GlaxoSmithKline Inc. > (919) 483-8456 > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From gary.h.merrill@gsk.com Mon Sep 29 22:18:31 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8U3IVM10342 for ; Mon, 29 Sep 2003 22:18:31 -0500 Received: by usrtpnp1.glaxowellcome.com; id XAA14137; Mon, 29 Sep 2003 23:19:10 -0400 (EDT) From: Received: from keymaster.glaxo.com(192.58.204.209) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma013949; Mon, 29 Sep 03 23:18:51 -0400 Received: by usrtpnp2.glaxowellcome.com; id XAA17393; Mon, 29 Sep 2003 23:18:51 -0400 (EDT) Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma017266; Mon, 29 Sep 03 23:18:13 -0400 Received: by ussunt3.glaxo.com id XAA03455; Mon, 29 Sep 2003 23:18:20 -0400 (EDT) Received: from 166.71.134.118 by us9n56.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Mon, 29 Sep 2003 23:17:23 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h8U3HM416806 for ; Mon, 29 Sep 2003 23:17:22 -0400 (EDT) Subject: Re: [Metakit] Mysteriously growing Metakit db To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Mon, 29 Sep 2003 23:18:02 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 09/29/2003 11:17:27 PM MIME-Version: 1.0 X-WSS-ID: 13662A492715917-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: With some additional experimentation I have discovered that: 1. I was creating the storage in mode 2 (commit-extend). Actually, I don't have a clue what this means, but I thought it was a kind of "update" mode. In that mode the file increases in size from 1K to 2K, 4K, 6K, ... 2. If I change to mode 1 (r/w mode), the file starts at 1K, grows to 4K and remains at that size. My hypothesis is that this has something to do with the hash view, though why this should be so is a mystery since it does not appear that anything needs rehashing (once the rows are originally added in the first invocation, they are only modifed thereafter, and the keys are never modified). I haven't really explored this hypothesis. I also don't know why I can't seem to duplicate this behavior in a simple test case. I suspect it may have something to do with loads/dumps of Python class instances together with the size of these, but this is nothing more than idle speculation. I am now fairly confident that my database will not grow without bound, consuming life as we know it on the east coast. But what is commit-extend mode anyway? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From jyl@mod3.net Mon Sep 29 22:36:45 2003 Received: from mod3.net (mod3.net [69.55.237.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8U3aiM11147 for ; Mon, 29 Sep 2003 22:36:45 -0500 Received: (qmail 31497 invoked from network); 30 Sep 2003 03:36:44 -0000 Received: from adsl-209-233-24-218.dsl.snfc21.pacbell.net (HELO mod3.net) (jyl@209.233.24.218) by mod3.net with SMTP; 30 Sep 2003 03:36:44 -0000 Message-ID: <3F78FA73.8020303@mod3.net> Date: Mon, 29 Sep 2003 20:37:23 -0700 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com CC: metakit@equi4.com Subject: Re: [Metakit] Mysteriously growing Metakit db References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: OK thats more useful information. Commit-extend means you can go back to a previous commit state and have everything there in the state it was when you committed. I think infinite backing up is supported. Of course that means that each commit adds a complete copy of the data, so the storage will grow. I don't really know how to actually go back to a previous commit state, for that, if you want to do that, ask on the metakit list. If you do not need that ability, I suggest you open the storage in regular commit mode which will limit the growth to about 2X of your data size, as I said before. Can you tell me a bit about what you're doing with Metakit? Just for my personal interest. Also, if you want a large example of an extension built with Metakit, check out my e4Graph package, a tool that allows you to represent directed graphs in persistent storages with Metakit: http://e4graph.com/e4graph. Cheers, --JYL gary.h.merrill@gsk.com wrote: >With some additional experimentation I have discovered that: > >1. I was creating the storage in mode 2 (commit-extend). Actually, I don't >have a clue what this means, but I thought it was a kind of "update" mode. >In that mode the file increases in size from 1K to 2K, 4K, 6K, ... >2. If I change to mode 1 (r/w mode), the file starts at 1K, grows to 4K and >remains at that size. > >My hypothesis is that this has something to do with the hash view, though >why this should be so is a mystery since it does not appear that anything >needs rehashing (once the rows are originally added in the first >invocation, they are only modifed thereafter, and the keys are never >modified). I haven't really explored this hypothesis. > >I also don't know why I can't seem to duplicate this behavior in a simple >test case. I suspect it may have something to do with loads/dumps of >Python class instances together with the size of these, but this is nothing >more than idle speculation. > >I am now fairly confident that my database will not grow without bound, >consuming life as we know it on the east coast. > >But what is commit-extend mode anyway? > >-------------------------------------- >Gary H. Merrill >Director and Principal Scientist, New Applications >Data Exploration Sciences >GlaxoSmithKline Inc. >(919) 483-8456 > > > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > > > From jcw@equi4.com Tue Sep 30 02:40:53 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8U7erM22724 for ; Tue, 30 Sep 2003 02:40:53 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id F0A788399B; Tue, 30 Sep 2003 09:40:45 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 2A97A83999 for ; Tue, 30 Sep 2003 09:40:45 +0200 (CEST) Date: Tue, 30 Sep 2003 09:40:50 +0200 Subject: Re: [Metakit] Mysteriously growing Metakit db Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <6998B4EE-F319-11D7-859C-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.2 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: wrote: [...] > I am now fairly confident that my database will not grow without bound, > consuming life as we know it on the east coast. Relieved :) > But what is commit-extend mode anyway? I've dug up information about this in the Metakit wiki and used it as basis for a new page on the website, see the last item of http://www.equi4.com/mkdocs.html -jcw From tschulze@physik.uni-wuerzburg.de Tue Sep 30 03:16:34 2003 Received: from we5x01.physik.uni-wuerzburg.de (root@we5x01.physik.uni-wuerzburg.de [132.187.18.1]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8U8GYM25562 for ; Tue, 30 Sep 2003 03:16:34 -0500 Received: from uhu.local (we5d85.physik.uni-wuerzburg.de [132.187.18.185]) by we5x01.physik.uni-wuerzburg.de (8.11.7/8.11.7) with ESMTP id h8U85Ci02647 for ; Tue, 30 Sep 2003 10:05:12 +0200 (METDST) From: Thomas Schulze To: metakit@equi4.com Content-Type: text/plain Organization: EP5 Message-Id: <1064909301.1983.11.camel@uhu.local> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.3 Date: 30 Sep 2003 10:08:21 +0200 Content-Transfer-Encoding: 7bit Subject: [Metakit] C++ API linker problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi all, i've got the following problem: compiling any c++-source-code (for example: >>g++ -o demo demo.cpp<< out of the metakit demos-directory) i get error messages of the form: /tmp/ccxal6DE.o(.text+0x4b): In function `main': : undefined reference to `c4_Storage::c4_Storage[in-charge](char const*, int)' I think, that it is a linker problem and tried several ways of linking, but i do not exactly know, what is missing and what to link. I had a lokk at the g++-manpage and a lot of websites, but i must have overseen the solution. Thank you for your help Thomas -- Thomas Schulze-Till From rcohen@articque.com Tue Sep 30 03:35:35 2003 Received: from mwinf0103.wanadoo.fr (smtp8.wanadoo.fr [193.252.22.30]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8U8ZYM27016 for ; Tue, 30 Sep 2003 03:35:34 -0500 Received: from articque.com (ATours-101-1-1-171.w193-252.abo.wanadoo.fr [193.252.49.171]) by mwinf0103.wanadoo.fr (SMTP Server) with ESMTP id 9EF901BFFFBB for ; Tue, 30 Sep 2003 10:35:28 +0200 (CEST) Message-ID: <3F79404C.2050808@articque.com> Date: Tue, 30 Sep 2003 10:35:24 +0200 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] C++ API linker problems References: <1064909301.1983.11.camel@uhu.local> In-Reply-To: <1064909301.1983.11.camel@uhu.local> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: maybe you should add the metakit library in g++ command, like -lmk4 (whatever the library name is) Thomas Schulze wrote: > Hi all, > > > i've got the following problem: > > compiling any c++-source-code (for example: >>g++ -o demo demo.cpp<< out > of the metakit demos-directory) i get error messages of the form: > > /tmp/ccxal6DE.o(.text+0x4b): In function `main': > : undefined reference to `c4_Storage::c4_Storage[in-charge](char const*, > int)' > > I think, that it is a linker problem and tried several ways of linking, > but i do not exactly know, what is missing and what to link. > > I had a lokk at the g++-manpage and a lot of websites, but i must have > overseen the solution. > > Thank you for your help > > Thomas > -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From henni@brainbot.com Tue Sep 30 04:28:01 2003 Received: from mail.brainbot.com ([145.253.163.18]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8U9S0M31256 for ; Tue, 30 Sep 2003 04:28:01 -0500 Received: (qmail 31632 invoked from network); 30 Sep 2003 09:27:55 -0000 Received: from bull.brainbot.com (HELO brainbot.com) (192.168.10.45) by heini.brainbot.com with SMTP; 30 Sep 2003 09:27:55 -0000 Message-ID: <3F794AE7.1010008@brainbot.com> Date: Tue, 30 Sep 2003 11:20:39 +0200 From: Thorsten Henninger Reply-To: henni@brainbot.com Organization: brainbot technologies AG User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Mk4Py for python2.3 on Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi all, I am trying to compile the Mk4py.dll (python bindings) for Python2.3 on Windows, but I did not succeed. I almost got it withthe mingw cross compiler on windows, but this one does not work as well! Does anybody out there have some Python2.3-Windowes-binaries for me ? Mk4py.dll Mk4tcl.dll mk4vc60.dll mk4vc60.lib Has anybody of you done this before? Thanks in advance, Henni -- brainbot technologies AG boppstrasse . 64 . 55118 mainz . germany fon +49 6131 211639-1 . fax +49 6131 211639-2 http://brainbot.com/ mailto:henni@brainbot.com From rcohen@articque.com Tue Sep 30 04:42:58 2003 Received: from mwinf0102.wanadoo.fr (smtp8.wanadoo.fr [193.252.22.30]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8U9gvM32314 for ; Tue, 30 Sep 2003 04:42:58 -0500 Received: from articque.com (ATours-101-1-1-171.w193-252.abo.wanadoo.fr [193.252.49.171]) by mwinf0102.wanadoo.fr (SMTP Server) with ESMTP id E82441BFD350 for ; Tue, 30 Sep 2003 11:42:51 +0200 (CEST) Message-ID: <3F795017.9070108@articque.com> Date: Tue, 30 Sep 2003 11:42:47 +0200 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] faq searching Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I have added some useful information at the end of my http://support.articque.com/metakit/metakit_tutorial.html about Find(), Select() and Search() This is the result of my own understanding, but I may be wrong. If your opinion differ, please let me know. -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jcw@equi4.com Tue Sep 30 05:06:29 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8UA6TM01250 for ; Tue, 30 Sep 2003 05:06:29 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 68B8E8399B; Tue, 30 Sep 2003 12:06:22 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 02BC583999 for ; Tue, 30 Sep 2003 12:06:21 +0200 (CEST) Date: Tue, 30 Sep 2003 12:06:27 +0200 Subject: Re: [Metakit] Mk4Py for python2.3 on Windows Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F794AE7.1010008@brainbot.com> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-3.0 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Thorsten Henninger wrote: > I am trying to compile the Mk4py.dll (python bindings) for Python2.3 > on Windows, but I did not succeed. I almost got it withthe mingw > cross compiler on windows, but this one does not work as well! There was an issue with 64 bit ints (PWONumber.h and PyRowRef.cpp, addressed by njr's patch), but after that change - and the inevitable adjustment of python22->python23 in the MSVC6 build project - it builds ok with MSVC6. > Mk4py.dll I've renamed the previous build to Mk4py22.dll, and have uploaded a new Mk4py.dll - enjoy :) -jcw From henni@brainbot.com Tue Sep 30 05:38:15 2003 Received: from mail.brainbot.com ([145.253.163.18]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h8UAcEM02929 for ; Tue, 30 Sep 2003 05:38:14 -0500 Received: (qmail 32580 invoked from network); 30 Sep 2003 10:38:08 -0000 Received: from bull.brainbot.com (HELO brainbot.com) (192.168.10.45) by heini.brainbot.com with SMTP; 30 Sep 2003 10:38:08 -0000 Message-ID: <3F795B5D.4030303@brainbot.com> Date: Tue, 30 Sep 2003 12:30:53 +0200 From: Thorsten Henninger Reply-To: henni@brainbot.com Organization: brainbot technologies AG User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Compiling metakit on Linux/FreeBsd with Python2.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I found out, that one has to insert the following line at the beginning of "PWONumber.h" in python/scxx: #define LONG_LONG long long in order to compile Metoakit with Python2.3. Then everything works just fine! reegards, Henni -- brainbot technologies AG boppstrasse . 64 . 55118 mainz . germany fon +49 6131 211639-1 . fax +49 6131 211639-2 http://brainbot.com/ mailto:henni@brainbot.com From mss@mawhrin.net Tue Sep 30 05:39:47 2003 Received: from only.mawhrin.net (postfix@[194.119.138.213]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8UAdkM03035 for ; Tue, 30 Sep 2003 05:39:47 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id 0EFE413968; Tue, 30 Sep 2003 11:39:44 +0100 (BST) Date: Tue, 30 Sep 2003 11:39:44 +0100 To: metakit@equi4.com Subject: Re: [Metakit] Mk4py changes Message-ID: <20030930103944.GA1074@mawhrin.net> Mail-Followup-To: metakit@equi4.com References: <20030929173535.GA29009@mawhrin.net> <20030929185358.GA61747@uiuc.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WYTEVAkct0FjGQmd" Content-Disposition: inline In-Reply-To: <20030929185358.GA61747@uiuc.edu> User-Agent: Mutt/1.5.4i From: mss@mawhrin.net (Mikhail Sobolev) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --WYTEVAkct0FjGQmd Content-Type: multipart/mixed; boundary="BXVAT5kNtrzKuDFl" Content-Disposition: inline --BXVAT5kNtrzKuDFl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 29, 2003 at 01:53:58PM -0500, Nicholas Riley wrote: > On Mon, Sep 29, 2003 at 06:35:35PM +0100, Mikhail Sobolev wrote: > > Can you please be a bit more specific: did you change the files outside > > of mk4py? I mean, can I just take the tree that corresponds to the > > python interface and build it against the library installed in my > > system (debian unstable)? >=20 > No, the Python interface actually requires it be built from a Metakit > build tree - it links directly against the individual Metakit object > files, instead of against the Metakit library as a whole. So > Mk4py.{so,sl,dll} ends up containing the Metakit library as well as > the Python interface. This was part of the setup.py script before I > changed it, so I assumed there was some good reason for it and didn't > change it. Perhaps I was wrong. :) This actually were two questions :)) The first one: did you change any files outside of mk4py distribution? And the answer seems to be yes. I tried to re-built the package by replacing _only_ python directory, and it seemed to be OK (one of the changes -- src/view.cpp -- was just a cosmetic one, as for the other one -- src/column.cpp -- I took the risk of not implementing it; also I tweaked with the configure stuff directly). I did want to test, so I installed python2.3-unit package (which was not installed on my system) and replaced the regrtest.py file with the one you offer on your website. Here comes my question: the python setup.py test gave no errors. However when I came to the test subdirectory and tried to run test_string.py file directly, it gave me the attached errors. I wonder whether it's serious at all? (Sorry I do not know a lot about the testing framework implemented in Python.) -- Misha --BXVAT5kNtrzKuDFl Content-Type: text/plain; charset=koi8-r Content-Disposition: attachment; filename=ts Content-Transfer-Encoding: quoted-printable PASS: rejected {'a': 3, 'b': ''} as expected Not a sequence PASS: rejected {'a': '\x00', 'b': 'hi\x00'} as expected string contains embedded nulls; t= ry 'B' type PASS: rejected {'a': 'abcdabcdabcd\x00hi', 'b': 'lo'} as expected string contains embedded nulls; t= ry 'B' type PASS: rejected {'a': '\x00\x00hi', 'b': 'lo'} as expected string contains embedded nulls; t= ry 'B' type FAIL: append {'a': u'hi there', 'b': 'hi'} got not a Python string exceptions.TypeError FAIL: append {'a': u'\u20ac', 'b': 'hi'} got not a Python string exceptions.TypeError FAIL: append {'a': u'Sample\u20acTesting', 'b': ''} got not a Python string exceptions.TypeError FAIL: append {'a': u'Sample\x00blahblah', 'b': 'yo'} got not a Python string exceptions.TypeError expected exceptions.Valu= eError VIEW CONTENTS: a b =20 ----------------- --------------- =20 asdasdfasdfasdfsa !@*%$#@#$%^&*() =E2=82=ACHi there =20 ----------------- --------------- Total: 3 rows PASS: retrieved {'a': '', 'b': ''} PASS: retrieved {'a': 'asdasdfasdfasdfsa', 'b': '!@*%$#@#$%^&*()'} PASS: retrieved {'a': '', 'b': '\xe2\x82\xacHi there'} Traceback (most recent call last): File "test_string.py", line 31, in ? v.finished() File "/var/home/mss/debian/libmetakit2.4.9.2-2.4.9.2/python/test/mktestsu= pport.py", line 147, in finished self.failure.assess() File "/var/home/mss/debian/libmetakit2.4.9.2-2.4.9.2/python/test/mktestsu= pport.py", line 51, in assess raise TestFailed( test.test_support.TestFailed: 4 failures; run in verbose mode for details --BXVAT5kNtrzKuDFl-- --WYTEVAkct0FjGQmd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/eV1wMYLUe7XufiwRAgHBAJ4uqCwV6JI9HNKrpMHZ2JWEl1TaOwCgu4R+ EJhXlPuK8wKpRyi+d8pvfgU= =Cb4R -----END PGP SIGNATURE----- --WYTEVAkct0FjGQmd-- From jcw@equi4.com Tue Sep 30 05:47:20 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8UAlJM03483 for ; Tue, 30 Sep 2003 05:47:19 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id D46558399B; Tue, 30 Sep 2003 12:47:12 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 6731C83999 for ; Tue, 30 Sep 2003 12:47:12 +0200 (CEST) Date: Tue, 30 Sep 2003 12:47:17 +0200 Subject: Re: [Metakit] Mk4py changes Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20030930103944.GA1074@mawhrin.net> Message-Id: <75D317F8-F333-11D7-859C-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.8 required=5.0 tests=AWL,IN_REP_TO,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Mikhail, > did you change any files outside of mk4py distribution? FYI, see http://trixie.triqs.com/pipermail/metakit/2003-September/001409.html - I posted a patch, i.e. all the differences in one file. You can see exactly what Nicholas did. -jcw From mss@mawhrin.net Tue Sep 30 06:04:29 2003 Received: from only.mawhrin.net (postfix@host4.transas.vianw.co.uk [194.119.138.213] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8UB4TM04406 for ; Tue, 30 Sep 2003 06:04:29 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id CB86F13968; Tue, 30 Sep 2003 12:04:27 +0100 (BST) Date: Tue, 30 Sep 2003 12:04:27 +0100 To: Metakit mailing list Subject: Re: [Metakit] Mk4py changes Message-ID: <20030930110427.GA1261@mawhrin.net> Mail-Followup-To: Metakit mailing list References: <20030930103944.GA1074@mawhrin.net> <75D317F8-F333-11D7-859C-000A9588127E@equi4.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="d6Gm4EdcadzBjdND" Content-Disposition: inline In-Reply-To: <75D317F8-F333-11D7-859C-000A9588127E@equi4.com> User-Agent: Mutt/1.5.4i From: mss@mawhrin.net (Mikhail Sobolev) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --d6Gm4EdcadzBjdND Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 30, 2003 at 12:47:17PM +0200, Jean-Claude Wippler wrote: > Mikhail, >=20 > >did you change any files outside of mk4py distribution? >=20 > FYI, see=20 > http://trixie.triqs.com/pipermail/metakit/2003-September/001409.html -=20 > I posted a patch, i.e. all the differences in one file. You can see=20 > exactly what Nicholas did. Thanks for the link. :) Are you going to release these changes any time soon? -- Misha --d6Gm4EdcadzBjdND Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/eWM7MYLUe7XufiwRAqcwAKCe0GrtMMZng64c5e5i5B0Vvx3mPQCeMvTS plJh+uHmPM+bn5lKuvIN9dI= =bciJ -----END PGP SIGNATURE----- --d6Gm4EdcadzBjdND-- From tschulze@physik.uni-wuerzburg.de Tue Sep 30 06:47:10 2003 Received: from we5x01.physik.uni-wuerzburg.de (root@we5x01.physik.uni-wuerzburg.de [132.187.18.1]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8UBlAM06729 for ; Tue, 30 Sep 2003 06:47:10 -0500 Received: from uhu.local (we5d85.physik.uni-wuerzburg.de [132.187.18.185]) by we5x01.physik.uni-wuerzburg.de (8.11.7/8.11.7) with ESMTP id h8UBJri03590 for ; Tue, 30 Sep 2003 13:19:53 +0200 (METDST) From: Thomas Schulze To: metakit@equi4.com Content-Type: text/plain Organization: EP5 Message-Id: <1064920982.1982.14.camel@uhu.local> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.3 Date: 30 Sep 2003 13:23:03 +0200 Content-Transfer-Encoding: 7bit Subject: [Metakit] C++ API linker problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi all, -lmk4 works. Thanks. Thomas -- Thomas Schulze-Till From njriley@shrug.csl.uiuc.edu Tue Sep 30 08:55:41 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8UDtfM15004 for ; Tue, 30 Sep 2003 08:55:41 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9) with ESMTP id h8UDtbQZ068640 for ; Tue, 30 Sep 2003 08:55:37 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9/Submit) id h8UDtbYi068639 for metakit@equi4.com; Tue, 30 Sep 2003 08:55:37 -0500 (CDT) (envelope-from njriley) Date: Tue, 30 Sep 2003 08:55:37 -0500 From: Nicholas Riley To: metakit@equi4.com Subject: Re: [Metakit] Mk4py changes Message-ID: <20030930135537.GA68081@uiuc.edu> Mail-Followup-To: metakit@equi4.com References: <20030929173535.GA29009@mawhrin.net> <20030929185358.GA61747@uiuc.edu> <20030930103944.GA1074@mawhrin.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030930103944.GA1074@mawhrin.net> User-Agent: Mutt/1.4.1i Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, My only changes outside the Python area were not source-related, they were just build system-related. The differences you saw were most likely because you compared against the release version, whereas I had updated to the latest CVS version. On Tue, Sep 30, 2003 at 11:39:44AM +0100, Mikhail Sobolev wrote: > Here comes my question: the python setup.py test gave no errors. > However when I came to the test subdirectory and tried to run > test_string.py file directly, it gave me the attached errors. I wonder > whether it's serious at all? (Sorry I do not know a lot about the > testing framework implemented in Python.) I think this may be linking against the wrong versions of some libraries; either run all.py or use make test-python or python setup.py test; don't run the test cases individually. Although the string messages you get don't look too good... -- =Nicholas Riley | From mss@mawhrin.net Tue Sep 30 11:20:16 2003 Received: from only.mawhrin.net (postfix@[194.119.138.213]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h8UGKFM23767 for ; Tue, 30 Sep 2003 11:20:15 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id E718813968; Tue, 30 Sep 2003 17:20:12 +0100 (BST) Date: Tue, 30 Sep 2003 17:20:12 +0100 To: metakit@equi4.com Subject: Re: [Metakit] Mk4py changes Message-ID: <20030930162012.GA2294@mawhrin.net> Mail-Followup-To: metakit@equi4.com References: <20030929173535.GA29009@mawhrin.net> <20030929185358.GA61747@uiuc.edu> <20030930103944.GA1074@mawhrin.net> <20030930135537.GA68081@uiuc.edu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0OAP2g/MAC+5xKAE" Content-Disposition: inline In-Reply-To: <20030930135537.GA68081@uiuc.edu> User-Agent: Mutt/1.5.4i From: mss@mawhrin.net (Mikhail Sobolev) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --0OAP2g/MAC+5xKAE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Sep 30, 2003 at 08:55:37AM -0500, Nicholas Riley wrote: > I think this may be linking against the wrong versions of some > libraries; either run all.py or use make test-python or python > setup.py test; don't run the test cases individually. Although the > string messages you get don't look too good... I am sorry I forgot to mention: "python setup.py test" said that everything's fine... So I am just a little bit confused. -- Misha --0OAP2g/MAC+5xKAE Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/ea08MYLUe7XufiwRAkqrAJ9XnzWyn/avuOHfVEWLBxWYT4uz9gCeLBOn In5d9C2oS/R+P1fEiXKkfwI= =gCeJ -----END PGP SIGNATURE----- --0OAP2g/MAC+5xKAE-- From mss@mawhrin.net Wed Oct 1 14:36:22 2003 Received: from only.mawhrin.net (postfix@[194.119.138.213]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h91JaLM31115 for ; Wed, 1 Oct 2003 14:36:22 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id A4E8A13968; Wed, 1 Oct 2003 20:36:19 +0100 (BST) Date: Wed, 1 Oct 2003 20:36:19 +0100 To: Metakit mailing list Subject: Re: [Metakit] Mk4py changes Message-ID: <20031001193619.GA6921@mawhrin.net> Mail-Followup-To: Metakit mailing list References: <20030927064004.GC19396@uiuc.edu> <1FFB6F42-F0CE-11D7-8DEE-000A9588127E@equi4.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="H+4ONPRPur6+Ovig" Content-Disposition: inline In-Reply-To: <1FFB6F42-F0CE-11D7-8DEE-000A9588127E@equi4.com> User-Agent: Mutt/1.5.4i From: mss@mawhrin.net (Mikhail Sobolev) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --H+4ONPRPur6+Ovig Content-Type: multipart/mixed; boundary="ReaqsoxgOBHFXBhH" Content-Disposition: inline --ReaqsoxgOBHFXBhH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline After clarification on distutils-sig, I suggest the following patch -- Misha --ReaqsoxgOBHFXBhH Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="setup.patch" --- setup.py.orig 2003-10-01 20:34:20.000000000 +0100 +++ setup.py 2003-10-01 20:35:21.000000000 +0100 @@ -162,8 +162,6 @@ author = metakit.__author__, author_email = metakit.__email__, url = metakit.__url__, - maintainer = metakit.__author__, - maintainer_email = metakit.__email__, license = metakit.__license__, keywords = ['database'], py_modules = ['metakit'], --ReaqsoxgOBHFXBhH-- --H+4ONPRPur6+Ovig Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/eyyyMYLUe7XufiwRAlUAAJ4sXKE20X+9vPZzlEVYVT4KuL78eQCgguzO BUvM/EObsKtn1d7JdN2ff2A= =F8La -----END PGP SIGNATURE----- --H+4ONPRPur6+Ovig-- From bkelley@wi.mit.edu Thu Oct 2 11:07:58 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h92G7vM22629 for ; Thu, 2 Oct 2003 11:07:57 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003100212075207218 for ; Thu, 02 Oct 2003 12:07:52 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HM50T500.5QB for ; Thu, 2 Oct 2003 12:07:53 -0400 Message-ID: <3F7C4D85.6080606@wi.mit.edu> Date: Thu, 02 Oct 2003 12:08:37 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0; stumbleupon.com) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Python Metakit Client/Server Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I scratched an itch and made a python server for metakit databases. The client uses the exact same syntax as metakit.py (with two exceptions). server usage: To serve up a metakit database named "test.cmk" import metaserve.server metaserve.server.serve("test.cmk", port=50008) client usage: Attach to the metakit database served up by locahost on port 50008 import metakit.client storage = metakit.client.storage(host='127.0.0.1', port=50008) ... use storage as normal. The main usage differences are that metakit.wrap is now a method of the storage object and storage.rollback() doesn't really make sense in a multiuser environment. Documentation: http://staffa.wi.mit.edu/people/kelley/Metaserve.html Download: http://staffa.wi.mit.edu/people/kelley/ Future: Every row access is a client-server operation. All row attributes could be bundled up and sent once across the wire. This has some side-effects such as not knowing when another user has updated the row but is considerably faster. Please send me bug reports! -- Brian Kelley bkelley@wi.mit.edu Whitehead Institute for Biomedical Research 617 258-6191 From jcw@equi4.com Thu Oct 2 11:40:02 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h92Ge2M26360 for ; Thu, 2 Oct 2003 11:40:02 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 8C1E68399B; Thu, 2 Oct 2003 18:39:54 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 32DA783999; Thu, 2 Oct 2003 18:39:54 +0200 (CEST) Date: Thu, 2 Oct 2003 18:40:02 +0200 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: Metakit mailing list To: "Brian Kelley" From: Jean-Claude Wippler In-Reply-To: <3F7C4D85.6080606@wi.mit.edu> Message-Id: <11DFD40D-F4F7-11D7-A2DC-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.9 required=5.0 tests=AWL,IN_REP_TO,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] On more comment about serialized MK Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian, Forgot to mention this... Serialized MK data has a size in the header, so one can efficiently do "read header, read the rest, process, repeat". All binary, but totally generic MK data. So while I used HTTP requests, MK serialization can also be used end-to-end on a socket connection which is kept open, with nu further "packetizing" needed. -jcw From jcw@equi4.com Thu Oct 2 11:43:28 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h92GhRM26863 for ; Thu, 2 Oct 2003 11:43:27 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 30CB78399B; Thu, 2 Oct 2003 18:43:20 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id A969383999 for ; Thu, 2 Oct 2003 18:43:19 +0200 (CEST) Date: Thu, 2 Oct 2003 18:43:28 +0200 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <11DFD40D-F4F7-11D7-A2DC-000A9588127E@equi4.com> Message-Id: <8C71D3F1-F4F7-11D7-A2DC-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.7 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] Re: On more comment about serialized MK Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Brian, > > Forgot to mention this... [...] Please ignore. This was meant as private email. I got carried away by Brian Kelley's cool new server, and things one could do with it. Sorry for the noise :) -jcw From dennis@illusions.com Thu Oct 2 19:33:17 2003 Received: from zeus.illusions.com ([69.28.129.146]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h930XGM04066 for ; Thu, 2 Oct 2003 19:33:16 -0500 Received: from zeus.illusions.com (localhost.localdomain [127.0.0.1]) by zeus.illusions.com (8.12.8/8.12.8) with ESMTP id h930cAZh023031 for ; Thu, 2 Oct 2003 17:38:10 -0700 Received: from localhost (dennis@localhost) by zeus.illusions.com (8.12.8/8.12.8/Submit) with ESMTP id h930cAxf023027 for ; Thu, 2 Oct 2003 17:38:10 -0700 X-Authentication-Warning: zeus.illusions.com: dennis owned process doing -bs Date: Thu, 2 Oct 2003 17:38:10 -0700 (MST) From: Dennis To: metakit@equi4.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] .net bindings Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, Has anyone thought of MONO/.NET bindings for Metakit? -- --- Dennis Sacks dennis@illusions.com From seanamckay@juno.com Fri Oct 3 10:39:41 2003 Received: from m10.nyc.untd.com (m10.nyc.untd.com [64.136.22.73]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h93FdfM03615 for ; Fri, 3 Oct 2003 10:39:41 -0500 Received: from cookie.untd.com by cookie.untd.com for <"qWseXT3rq5SaoRv93RsNRFXGIhx0hZnU+DANmYmA+F5iMJXeX/zb0w==">; Fri, 03 Oct 2003 08:39:26 PDT Received: (from seanamckay@juno.com) by m10.nyc.untd.com (jqueuemail) id JBKYQAKP; Fri, 03 Oct 2003 08:39:26 PDT To: metakit@equi4.com Date: Fri, 3 Oct 2003 10:40:07 -0500 Message-ID: <20031003.104007.-547661.0.seanamckay@juno.com> X-Mailer: Juno 4.0.11 MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Juno-Att: 0 X-Juno-RefParts: 0 From: Sean A McKay Subject: [Metakit] Re: Python Metakit Client/Server Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is cool! > > Message: 2 > Date: Thu, 02 Oct 2003 12:08:37 -0400 > From: "Brian Kelley" > To: Metakit mailing list > Subject: [Metakit] Python Metakit Client/Server ________________________________________________________________ The best thing to hit the internet in years - Juno SpeedBand! Surf the web up to FIVE TIMES FASTER! Only $14.95/ month - visit www.juno.com to sign up today! From erikh2000@yahoo.com Sun Oct 5 12:12:54 2003 Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h95HCsM26472 for ; Sun, 5 Oct 2003 12:12:54 -0500 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 5 Oct 2003 17:12:53 -0000 Message-Id: <5.2.0.9.0.20031005100235.02d63448@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Sun, 05 Oct 2003 10:18:34 -0700 To: Metakit mailing list From: Erik Hermansen Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Corrupted commit buffers. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I feel pretty lame to still be tracking down this bug, but here I am. In some weird, intermittent, unreproducable case, my storage files get corrupted and all data in them becomes inaccessible. My number one theory after weeks of troubleshooting is that something outside of Metakit is corrupting the commit buffers--either my application code or other 3rd-party libraries. Is the following troubleshooting technique plausible: - Checksum all the memory used for committing changes after each change to a property value or other write. This checksum will represent the data in a valid state. If I have to, I can update this checksum after every Metakit API call. - At some suitably frequent interval, checksum the same memory again and compare the checksum to the valid checksum. If they differ then I know that corrupting code has executed between this comparison and the last good comparison, and I can log some information about that. In this way, I narrow the corruption down to the offending code by reading field logs from testers that have seen the problem. If this sounds like overkill, consider that I've released several builds with "fixes" to my testers already and each time it takes about a week to confirm the bug still exists. We've yet to generate a reliable way to reproduce the bug. I think the above approach should work if the commit-relevant data is easy enough to checksum. Some advice on how to access that Metakit-internal data would be appreciated. -Erik From erikh2000@yahoo.com Sun Oct 5 18:41:21 2003 Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h95NfKM15852 for ; Sun, 5 Oct 2003 18:41:20 -0500 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 5 Oct 2003 23:41:19 -0000 Message-Id: <5.2.0.9.0.20031005164444.02d83c78@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Sun, 05 Oct 2003 16:47:00 -0700 To: Metakit mailing list From: Erik Hermansen Subject: Re: [Metakit] Corrupted commit buffers. In-Reply-To: <5.2.0.9.0.20031005100235.02d63448@pop.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: At 10:18 AM 10/5/2003 -0700, Erik Hermansen wrote: >In some weird, intermittent, unreproducable case, my storage files get >corrupted and all data in them becomes inaccessible. One other detail that may narrow this down quite a bit. In all cases where I have a corrupted storage file, c4_Storage::Description(0) will return an empty string instead of a description of views in the storage file. -Erik From pieterb@gewis.nl Thu Oct 9 03:34:39 2003 Received: from gewis.win.tue.nl (postfix@gewis.win.tue.nl [131.155.71.116]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h998YcX10423 for ; Thu, 9 Oct 2003 03:34:39 -0500 Received: by gewis.win.tue.nl (Postfix, from userid 103) id 2C3A521106; Thu, 9 Oct 2003 10:34:37 +0200 (CEST) Date: Thu, 9 Oct 2003 10:34:37 +0200 From: PieterB To: Nicholas Riley , metakit@equi4.com Message-ID: <20031009103437.A46688@gewis.win.tue.nl> References: <20030927064004.GC19396@uiuc.edu> <20030927090310.A36088@gewis.win.tue.nl> <20030929061022.GA57644@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20030929061022.GA57644@uiuc.edu>; from njriley@uiuc.edu on Mon, Sep 29, 2003 at 01:10:22AM -0500 Subject: [Metakit] Mk4py on FreeBSD 5.1 / Python 2.3 with Nicholas patches Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I finally got to get back to trying to run Metakit on my FreeBSD 5.1, but somehow it just doesn't work, and I can't figure out what's going wrong. I used Nicholas version (downloaded today): When I do: cd builds ; ../unix/configure --with-python=/usr or cd builds ; ../unix/configure --with-python it succesfully does the configure part now, but unfortuately the 'make' breaks with the error below: (BTW, I don't understand the --with-python syntax completely, and can't find information on that in the README or Python.html) 8<----------------------------------------------------------- /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../python/scxx -INONE/include/python2.3 ../unix/../python/PyProperty.cpp g++ -c -g -O2 -I../unix/../include -I../unix/../python/scxx -INONE/include/python2.3 ../unix/../python/PyProperty.cpp -fPIC -DPIC -o .libs/PyProperty.o In file included from ../python/PyProperty.h:12, from ../python/PyProperty.cpp:8: ../python/PyHead.h:11:20: Python.h: No such file or directory In file included from ../python/PyProperty.h:12, from ../python/PyProperty.cpp:8: ../python/PyHead.h:13: syntax error before `{' token ../python/PyHead.h:22: ISO C++ forbids declaration of `ob_type' with no type ../python/PyHead.h:22: `t' was not declared in this scope ../python/PyHead.h:23: syntax error before `}' token ../python/PyHead.h:8:1: unterminated #if In file included from ../python/PyProperty.cpp:8: ../python/PyProperty.h:8:1: unterminated #if *** Error code 1 8<----------------------------------------------------------- Can anybody tell me how if I need to install Python.h (how?) or how to fix the error above? Are the patches of Nicholes already integrated into CVS? Thanks in advance for helping me out, Pieter From njriley@shrug.csl.uiuc.edu Thu Oct 9 14:54:48 2003 Received: from shrug.csl.uiuc.edu (root@shrug.csl.uiuc.edu [130.126.137.135]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h99JsmX29813 for ; Thu, 9 Oct 2003 14:54:48 -0500 Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9) with ESMTP id h99JseQZ059497; Thu, 9 Oct 2003 14:54:40 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.9p1/8.12.9/Submit) id h99Jsak5059496; Thu, 9 Oct 2003 14:54:36 -0500 (CDT) (envelope-from njriley) Date: Thu, 9 Oct 2003 14:54:36 -0500 From: Nicholas Riley To: PieterB Cc: metakit@equi4.com Message-ID: <20031009195436.GA59404@uiuc.edu> Mail-Followup-To: PieterB , metakit@equi4.com References: <20030927064004.GC19396@uiuc.edu> <20030927090310.A36088@gewis.win.tue.nl> <20030929061022.GA57644@uiuc.edu> <20031009103437.A46688@gewis.win.tue.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20031009103437.A46688@gewis.win.tue.nl> User-Agent: Mutt/1.4.1i Subject: [Metakit] Re: Mk4py on FreeBSD 5.1 / Python 2.3 with Nicholas patches Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On Thu, Oct 09, 2003 at 10:34:37AM +0200, PieterB wrote: > I finally got to get back to trying to run Metakit on my FreeBSD > 5.1, but somehow it just doesn't work, and I can't figure out what's > going wrong. > > I used Nicholas version (downloaded today): > > > > When I do: > > cd builds ; ../unix/configure --with-python=/usr > or > cd builds ; ../unix/configure --with-python > > it succesfully does the configure part now, but unfortuately the > 'make' breaks with the error below: > > (BTW, I don't understand the --with-python syntax completely, > and can't find information on that in the README or Python.html) ../unix/configure --help should give you some hints. The -INONE thing is broken, I will take a look at that tomorrow - it indicates that the configure script wasn't able to find Python in /usr. Try --with-python=/usr/local. -- =Nicholas Riley | From jcw@equi4.com Fri Oct 10 07:10:15 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9ACAEX16389; Fri, 10 Oct 2003 07:10:14 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 5F21D8399B; Fri, 10 Oct 2003 14:10:07 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 9368883999; Fri, 10 Oct 2003 14:10:06 +0200 (CEST) Date: Fri, 10 Oct 2003 14:10:06 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list , Starkit list Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.5 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] bug when sorting results of groupby in Tcl Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Just to let you know that a bug has been found in the Mk4tcl extension. The problem shows up only when all of the following conditions apply: - you are using the Mk4too or Oomk packages, not just plain Mk4tcl - you are grouping views, i.e. the groupby operator - you are applying a selection or sort to the resulting subviews - the size of the subview is exactly one row The bug, with example code and output is further described here: http://www.equi4.com/BTS.cgi/BTS/editbugs?bugid=75 There is no problem in the C++ core, nor in the Python binding. As workaround, sorting can be skipped for 1-row subviews, e.g. in Oomk: if {[$v size] == 1]} { $v as v2 } else { [$v sort ...] as v2 } For selection, again the 1-row case will need to be dealt with separately. The bug is most likely in the "select" operator of Mk4tcl, which is a relatively complex combination of Select(), Sort(), and RemapWith() in the C++ core. -jcw From gary.h.merrill@gsk.com Fri Oct 10 08:59:08 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9ADx7X21794 for ; Fri, 10 Oct 2003 08:59:07 -0500 Received: by usrtpnp1.glaxowellcome.com; id JAA20961; Fri, 10 Oct 2003 09:59:54 -0400 (EDT) From: Received: from unknown(152.51.63.189) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma011388; Fri, 10 Oct 03 09:50:20 -0400 Received: by ussunt2.glaxo.com id JAA09309; Fri, 10 Oct 2003 09:50:27 -0400 (EDT) Received: from 166.71.134.118 by us9n53.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Fri, 10 Oct 2003 09:49:26 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h9ADnQm16564 for ; Fri, 10 Oct 2003 09:49:26 -0400 (EDT) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Fri, 10 Oct 2003 09:49:11 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 10/10/2003 09:49:31 AM MIME-Version: 1.0 X-WSS-ID: 1398676C2267853-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] BSDDB vs. Metakit performance? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: The application I am developing makes use of a number of databases for indexing terms, ontologies, document corpora, etc. At the moment, all of the ontology representations are kept in BSDDB databases of either the 'hash', 'recno', or (Pythonic) 'dbshelve' variety and I have been pretty happy with this although in certain ways its a bit cumbersome. When it became time to implement indexing of document corpora, it was apparent that Metakit provided utility that BSDDB does not. So I am using Metakit for that purpose. Now I am wondering if at some point I should go back and reimplement some of my other databases as Metakit rather than BSDDB. (Metakit is -- despite the currently unfortunate state of its documentation :-) -- somewhat easier to use than BSDDB.) There are, really, two issues: performance and scalability. I think that both BSDDB and Metakit will scale to what I need. So performance is the only genuine issue I'm concerned with. More particularly, this resolves to whether there is (or can be expected to be) a significant performance difference between a BSDDB hash/shelf database and a Metakit database using a hash view. Does anyone have any experience or thoughts on this? (Of course, if I get really squeezed for performance in certain areas, I'll abandon both and implement a minimal perfect hashing approach. But I am putting that off until absolutely necessary.) -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Fri Oct 10 09:56:12 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9AEuBX25586 for ; Fri, 10 Oct 2003 09:56:11 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003101010560716052 for ; Fri, 10 Oct 2003 10:56:07 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HMJQTJ00.D4K; Fri, 10 Oct 2003 10:56:07 -0400 Message-ID: <3F86C8C1.6010508@wi.mit.edu> Date: Fri, 10 Oct 2003 10:57:05 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com, metakit@equi4.com Subject: Re: [Metakit] BSDDB vs. Metakit performance? References: In-Reply-To: Content-Type: multipart/mixed; boundary="------------020009000504090004050609" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------020009000504090004050609 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit gary.h.merrill@gsk.com wrote: >...So performance is the only genuine issue I'm concerned with. > >More particularly, this resolves to whether there is (or can be expected to >be) a significant performance difference between a BSDDB hash/shelf >database and a Metakit database using a hash view. Does anyone have any >experience or thoughts on this? > It all depends. BSDDB only hashes on string or integer values and only stores data as pickled values. If you would like to have searchable content in a BSDDB database using anything but the desired key, you have to iterate through all the items and apply a query to each one. This means that you need to unpickle a whole lot of times. With metakit you can store annotations in the database along with the pickled values. Infact, you might even be able to let metakit serialize the object for you if it is made up of basic datatypes, strings, ints, floats even lists of ints and lists of floats. Now, BSDDB is more scalable in that you can stick a whole lot of data in it for archival purposes and have rapid access to the data. This begs the question, why not use both? Use metakit as a annotation schema where you can quickly find the index/key of object that you desire and then retrieve the object quickly from the BSDDB database. This is one of the techniques that I use. I store my compounds in the bsddb file (which is now 1.5+ GB of compounds) but I store compound fingerprints and annotations in the metakit database. I can quickly scan the metakit database, find the compounds that I want and then retrieve them from the bsddb data store. The downside is that there are two files to maintain. Now for some timings: Problem 1) load a package of python data, in this case the output from range(1000) 10000 times using a string key, namely "0" to "99999" 2) randomly retrieve the data 1000 times. 3) -> metakit only, retrieve base data type (in this case a binary string) 9.70299994946 seconds to load shelf 19.1780000925 seconds to load metakit 0.230000019073 seconds for 1000 random accesses to shelf 0.25100004673 seconds for 1000 random accesses to metakit 0.0599999427795 seconds for 1000 random accesses to metakit raw data Database sizes bsddb -> 80344K metakit -> 58044K So it looks as though bsddb wins on loading but they are pretty dead even for retrieval and the time spent is mostly caused by serialization of the data using pickle! Note that I used the highest (most efficient) pickle format i.e. "pickle.dumps(data, 2)" using "pickle.dumps(data)" is an order of magnitude slower. In short, if your data consists of basic datatypes like strings, floats, ints... it will be much faster to use metakit. Otherwise bsddb3 is marginally faster for retrieval and much faster for loading. Although, I trust it more with insanely large datasets like the 1.5GB+ I mentioned before. Another trick you can use is to use zlib to compress the data in the database and to uncompress before you use pickle.load... >(Of course, if I get really squeezed for performance in certain areas, I'll >abandon both and implement a minimal perfect hashing approach. But I am >putting that off until absolutely necessary.) > If you are using python, the best hashing scheme you could use would be python dictionaries. As long as you had a lot of memory :) Brian Kelley Whitehead Institute for Biomedical Research --------------020009000504090004050609 Content-Type: text/plain; name="bsddb_vs_metakit.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="bsddb_vs_metakit.py" ##import bsddb.dbshelve ##import metakit ##import time, random ##import cPickle as pickle ##db = bsddb.dbshelve.open("test.db") ##st = metakit.storage("test.mk", 1) ##map = st.getas("_hash[_H:I,_R:I]") ##view = st.getas("hash[key:S,data:B]") ##view = view.hash(map, 1) ##package = range(1000) ##t1 = time.time() ##for i in range(10000): ## db[str(i)] = package ##t2 = time.time() ##print t2-t1, "seconds to load shelf" ##dumps = pickle.dumps ##t1 = time.time() ##for i in range(10000): ## view.append((str(i), dumps(package, 2))) ## if i % 1000 == 1: st.commit() ##st.commit() ##t2 = time.time() ##print t2-t1, "seconds to load metakit" ##r = range(1000) ##choices = [random.choice(r) for x in r] ##t1 = time.time() ##for index in choices: ## data = db[str(index)] ##t2 = time.time() ##print t2-t1, "seconds for 1000 random accesses to shelf" loads = pickle.loads t1 = time.time() for index in choices: index = view.find(key=str(index)) data = loads(view[index].data) t2 = time.time() print t2-t1, "seconds for 1000 random accesses to metakit" t1 = time.time() for index in choices: index = view.find(key=str(index)) data = view[index].data t2 = time.time() print t2-t1, "seconds for 1000 random accesses to metakit raw data" --------------020009000504090004050609-- From jyl@mod3.net Fri Oct 10 10:46:04 2003 Received: from mod3.net (mod3.net [69.55.237.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9AFk3X30251 for ; Fri, 10 Oct 2003 10:46:04 -0500 Received: (qmail 11293 invoked by uid 48); 10 Oct 2003 15:46:03 -0000 Received: from 192.18.42.11 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Fri, 10 Oct 2003 08:46:03 -0700 (PDT) Message-ID: <55151.192.18.42.11.1065800763.squirrel@mod3.net> Date: Fri, 10 Oct 2003 08:46:03 -0700 (PDT) Subject: Re: [Metakit] BSDDB vs. Metakit performance? From: "Jacob Levy" To: In-Reply-To: <3F86C8C1.6010508@wi.mit.edu> References: <3F86C8C1.6010508@wi.mit.edu> X-Priority: 3 Importance: Normal Cc: , Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: OK, now try to iterate through the data and see Metakit really win. Metakit really hits a groove when you iterate over the data sequentially as that plays very well with the paging and caching on modern OSes and hardware. You didnt say which OS you used. Do you happen to know if Metakit uses memory mapped files on your OS? If it does, that's why loading is slower -- Metakit has to obtain committed address space pages from the OS to map all that data into the process's address space. --JYL > gary.h.merrill@gsk.com wrote: > >>...So performance is the only genuine issue I'm concerned with. >> >>More particularly, this resolves to whether there is (or can be >> expected to be) a significant performance difference between a BSDDB >> hash/shelf database and a Metakit database using a hash view. Does >> anyone have any experience or thoughts on this? >> > It all depends. BSDDB only hashes on string or integer values and only > stores data as pickled values. If you would like to have searchable > content in a BSDDB database using anything but the desired key, you have > to iterate through all the items and apply a query to each one. This > means that you need to unpickle a whole lot of times. With metakit you > can store annotations in the database along with the pickled values. > Infact, you might even be able to let metakit serialize the object for > you if it is made up of basic datatypes, strings, ints, floats even > lists of ints and lists of floats. > > Now, BSDDB is more scalable in that you can stick a whole lot of data in > it for archival purposes and have rapid access to the data. This begs > the question, why not use both? Use metakit as a annotation schema > where you can quickly find the index/key of object that you desire and > then retrieve the object quickly from the BSDDB database. > > This is one of the techniques that I use. I store my compounds in the > bsddb file (which is now 1.5+ GB of compounds) but I store compound > fingerprints and annotations in the metakit database. I can quickly > scan the metakit database, find the compounds that I want and then > retrieve them from the bsddb data store. The downside is that there are > two files to maintain. > > Now for some timings: > > Problem > > 1) load a package of python data, in this case the output from > range(1000) 10000 times using a string key, namely "0" to "99999" > 2) randomly retrieve the data 1000 times. > 3) -> metakit only, retrieve base data type (in this case a binary > string) > > 9.70299994946 seconds to load shelf > 19.1780000925 seconds to load metakit > 0.230000019073 seconds for 1000 random accesses to shelf > 0.25100004673 seconds for 1000 random accesses to metakit > 0.0599999427795 seconds for 1000 random accesses to metakit raw data > > Database sizes > bsddb -> 80344K > metakit -> 58044K > > So it looks as though bsddb wins on loading but they are pretty dead > even for retrieval and the time spent is mostly caused by serialization > of the data using pickle! Note that I used the highest (most efficient) > pickle format i.e. "pickle.dumps(data, 2)" using "pickle.dumps(data)" > is an order of magnitude slower. > > In short, if your data consists of basic datatypes like strings, floats, > ints... it will be much faster to use metakit. Otherwise bsddb3 is > marginally faster for retrieval and much faster for loading. Although, > I trust it more with insanely large datasets like the 1.5GB+ I mentioned > before. Another trick you can use is to use zlib to compress the data > in the database and to uncompress before you use pickle.load... > >>(Of course, if I get really squeezed for performance in certain areas, >> I'll abandon both and implement a minimal perfect hashing approach. >> But I am putting that off until absolutely necessary.) >> > If you are using python, the best hashing scheme you could use would be > python dictionaries. As long as you had a lot of memory :) > > Brian Kelley > Whitehead Institute for Biomedical Research From bkelley@wi.mit.edu Fri Oct 10 11:04:35 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9AG4ZX31438 for ; Fri, 10 Oct 2003 11:04:35 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003101012043116940 for ; Fri, 10 Oct 2003 12:04:31 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HMJTZJ00.5BE; Fri, 10 Oct 2003 12:04:31 -0400 Message-ID: <3F86D8CB.2060904@wi.mit.edu> Date: Fri, 10 Oct 2003 12:05:31 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: jyl@mod3.net, metakit@equi4.com Subject: Re: [Metakit] BSDDB vs. Metakit performance? References: <3F86C8C1.6010508@wi.mit.edu> <55151.192.18.42.11.1065800763.squirrel@mod3.net> In-Reply-To: <55151.192.18.42.11.1065800763.squirrel@mod3.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >OK, now try to iterate through the data and see Metakit really win. >Metakit really hits a groove when you iterate over the data sequentially >as that plays very well with the paging and caching on modern OSes and >hardware. > Let's do the test: 2.91400003433 seconds to iterate bsddb3 1.86299991608 seconds to iterate metakit So metakit is approximately 30% faster for linear access. Both are pretty good though. >You didnt say which OS you used. Do you happen to know if Metakit uses >memory mapped files on your OS? If it does, that's why loading is slower >-- Metakit has to obtain committed address space pages from the OS to map >all that data into the process's address space. > It does, but I'm assuming that most metakit installs are using memory mapped IO (mac OSX, linux, windows) so I think it is fair. I just know from experience that bsddb scales up to gigabyte files and metakit claims to have good performance to the several hundred megabyte region. I haven't found much of a problem in practice, some of my metakit files are 600MB+ Brian From gary.h.merrill@gsk.com Fri Oct 10 11:10:41 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9AGAfX31811 for ; Fri, 10 Oct 2003 11:10:41 -0500 Received: by usrtpnp1.glaxowellcome.com; id MAA29433; Fri, 10 Oct 2003 12:11:28 -0400 (EDT) From: Received: from unknown(152.51.63.189) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma005057; Fri, 10 Oct 03 11:40:45 -0400 Received: by ussunt2.glaxo.com id LAA22177; Fri, 10 Oct 2003 11:40:53 -0400 (EDT) Received: from 166.71.134.118 by us9n53.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Fri, 10 Oct 2003 11:39:45 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h9AFdim29338 for ; Fri, 10 Oct 2003 11:39:44 -0400 (EDT) Subject: Re: [Metakit] BSDDB vs. Metakit performance? To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Fri, 10 Oct 2003 11:39:29 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 10/10/2003 11:39:49 AM MIME-Version: 1.0 X-WSS-ID: 13980D4A2285504-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian: Thanks much for these observations. They are compatible with my own hunches at this point. And at the moment I'm using both BSDDB and Metakit in the way you describe: storing ontology structure information in BSDDB and document index information in Metakit (with references to ontology topic ids in the Metakit records). So document searching and information extraction will initially make use of the Metakit databases with ontology traversal and inferencing then using the BSDDB databases. From what you say, I should probably stick with that approach. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From jcw@equi4.com Fri Oct 10 11:45:19 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9AGjJX03328 for ; Fri, 10 Oct 2003 11:45:19 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 4313B8399B; Fri, 10 Oct 2003 18:45:11 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id AA05B83999 for ; Fri, 10 Oct 2003 18:45:10 +0200 (CEST) Date: Fri, 10 Oct 2003 18:45:09 +0200 Subject: Re: [Metakit] BSDDB vs. Metakit performance? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <3F86D8CB.2060904@wi.mit.edu> Message-Id: <1BEF5662-FB41-11D7-AEE3-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-3.1 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_02_03,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > Let's do the test: > 2.91400003433 seconds to iterate bsddb3 > 1.86299991608 seconds to iterate metakit > > So metakit is approximately 30% faster for linear access. Both are > pretty good though. As you know, statistics can be made to come out any way you like, i.e. your above figures could also be summarized as: bsddb3 needs 56% more time than MK. [jyl] >> You didnt say which OS you used. Do you happen to know if Metakit uses >> memory mapped files on your OS? If it does, that's why loading is >> slower >> -- Metakit has to obtain committed address space pages from the OS to >> map >> all that data into the process's address space. Keep in mind that this is a one-time mmap(). Pointer access and page faults do the rest. OS'es are pretty good at that, their entire code-loading and I/O designs are based on it, usually. Modern OS'es detect sequential accesses even and start pre-fetching. > I just know from experience that bsddb scales up to gigabyte files and > metakit claims to have good performance to the several hundred > megabyte region. I haven't found much of a problem in practice, some > of my metakit files are 600MB+ The hard limit is memory mapped address space, i.e. well under 2 Gb on 32-bit machines, in practice. I'd like to point out that storing blobs in MK is actually very efficient. Surprising at it may sound, above a certain size and number of rows, storing N items in a view will do just about the same as BDB does. Keep in mind that MK is columnar - the presence of a column, no matter how big or complex, does not affect traversal of the others. If it's all opaque binary data, and a substantial percentage is empty, then using a separate view should work out better. Personally, I think that unless file size is too constraining, you should just add the extra property to your view and let MK's adaptiveness figure out how to store things for that particular column (what MK does is switch to non-columnar storage if #bytes or #rows grows too far - there's a heuristic involved to find a decent trade-off). If you really want to get to the bottom om this, you should compare the mix of having data in MK for traversal, and either items in BDB for large storage or adding a single view with one S or B property, and storing items in that view. The logic of this should be similar. I suspect that MK will come out at least as fast (you're retrieving the N'th item from a view, vs. BDB doing an extra - albeit simple - hash lookup). You'd gain single-file convenience, and less installation dependencies. But you'll need to stay under say 1 to 1.5 Gb. Note that there is a downside in the current implementation: MK determines free space from a full traversal, so having millions of pieces in the file wil slow it down as it starts preparing for a commit. The file format has some unused features to greatly avoid such traversals, but MK 2.4.9.2 does not yet take advantage of that. It will definitely have to, before we can grow it to the terabyte range in 64-bit architectures. -jcw From gary.h.merrill@gsk.com Fri Oct 10 12:08:10 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9AH8AX06070 for ; Fri, 10 Oct 2003 12:08:10 -0500 Received: by usrtpnp1.glaxowellcome.com; id MAA21803; Fri, 10 Oct 2003 13:08:57 -0400 (EDT) From: Received: from unknown(152.51.63.189) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma026963; Fri, 10 Oct 03 12:31:12 -0400 Received: by ussunt2.glaxo.com id MAA08571; Fri, 10 Oct 2003 12:31:17 -0400 (EDT) Received: from 166.71.134.118 by us9n57.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Fri, 10 Oct 2003 12:30:09 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h9AGU8m25784 for ; Fri, 10 Oct 2003 12:30:09 -0400 (EDT) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Fri, 10 Oct 2003 12:29:05 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 10/10/2003 12:30:14 PM MIME-Version: 1.0 X-WSS-ID: 1398011B4204871-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Metakit for very large matrices Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: It strikes me that Metakit might provide a handy way of storing very large matrices. What I have in mind is something like a 2-dimensional matrice whose columns represented the categories of one ontology and whose rows represent the categories of another. By "very large" I mean in the 50,000x50,000 range (maybe this isn't *very* large?). But some questions: 1. Will Metakit in fact scale to this? 2. What is the best technique for creating and populating a view? Do I need to actually create a 50,000 element description string? This isn't a problem, but is it the way to do it? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From jcw@equi4.com Fri Oct 10 12:08:39 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9AH8cX06110 for ; Fri, 10 Oct 2003 12:08:38 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 70AF28399B; Fri, 10 Oct 2003 19:08:30 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E0CF683999 for ; Fri, 10 Oct 2003 19:08:29 +0200 (CEST) Date: Fri, 10 Oct 2003 19:08:29 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: <5E9E4EBA-FB44-11D7-AEE3-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.7 required=5.0 tests=AWL,SPAM_PHRASE_01_02,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] compression and encryption Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Two options not implemented in MK 2.4.9.2, are compression and encryption. Due to the column-wise design of MK, this may actually have substantial consequences. The idea, is that in a datafile with say layout "names[first:S,last:S,phones[type:S,number:S]]" it would be possible to designate some properties as being compressed, others as encrypted, and yet others as both. The compression would take place in a column-wise manner, i.e. all values of the designated property in all rows would be compressed. If there is major redundancy/repetition, then the storage size would be greatly reduced. On first access, such columns would be uncompressed (taking up some memory), and on commit, the data would be saved compressed again. For compression, it would seem that zlib is the de-facto standard to use. For encryption, a similar effect would be seen: on file, the entire column becomes encrypted, again for a specific property of all rows in that view. A complicating factor would be that encryption needs to be cusomizable, so in this case a callback through the c4_Strategy class seems the right way to do it. Perhaps some basic encryption such as David Wheeler's TEA could be included as default. When combining compression and encryption, compression would have to be done first, to have any effect. The encrypted result of that would be stored on file. On reading, the data must first be decrypted, then decompressed. This also requires a change to somehow specify the details in the description string, or in some other way. This will require more thought. I'm bringing this up because I am regularly compressing data before storing it (and Starkits in Tcl do it all the time), and because I think there is value in getting the encryption covered, especially since it could be done on a per-property basis. Encryption could be useful to "lock up" applications deployed as starkits/starpacks. The file format has hooks to allow this sort of thing, although such files will not be readable by current MK releases (they would not know how to skip over the extra admin info). Apart from "yes please, or no thanks", do you consider this a valuable option? Would you need it and use it right away? Desperate enough to fund it? (I had to ask...) Any ideas about how to encrypt? Or maybe only do compress? Are there any implications / trade-offs I'm forgetting about? -jcw, with a small marketing hat on... From jcw@equi4.com Fri Oct 10 12:21:03 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9AHL3X07690 for ; Fri, 10 Oct 2003 12:21:03 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 695E88399B; Fri, 10 Oct 2003 19:20:55 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id DB0EC83999 for ; Fri, 10 Oct 2003 19:20:54 +0200 (CEST) Date: Fri, 10 Oct 2003 19:20:54 +0200 Subject: Re: [Metakit] Metakit for very large matrices Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <1AAA614A-FB46-11D7-AEE3-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: wrote: > It strikes me that Metakit might provide a handy way of storing very > large > matrices. What I have in mind is something like a 2-dimensional > matrice > whose columns represented the categories of one ontology and whose rows > represent the categories of another. By "very large" I mean in the > 50,000x50,000 range (maybe this isn't *very* large?). > > But some questions: > > 1. Will Metakit in fact scale to this? You mean 2.5 million ints, floats, doubles? Easily. Strings? Yes, but not in a linear fashion. In 50,000 row units? Sure. > 2. What is the best technique for creating and populating a view? Do I > need to actually create a 50,000 element description string? This > isn't a > problem, but is it the way to do it? Wait. If the type of entries is the same across all columns, reconsider. Instead of: dim1[a0:I,a1:I,a2:I,...,a49999:I] Consider: dim1[dim2[a:I]] Then, you can index in two dimensions, like a real matrix. In Python: ... = v.dim1[123].dim2[456].a This should scale well beyond what you mention, it's simply 50,000 vectors of 50,000 ints each on file. -jcw From tom.krehbiel@motorola.com Fri Oct 10 12:34:21 2003 Received: from motgate3.mot.com (motgate3.mot.com [144.189.100.103]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9AHYLX09241 for ; Fri, 10 Oct 2003 12:34:21 -0500 Received: from az33exr01.mot.com (az33exr01.mot.com [10.64.251.231]) by motgate3.mot.com (Motorola/Motgate3) with ESMTP id h9AHYKoK025724; Fri, 10 Oct 2003 10:34:20 -0700 (MST) Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) by az33exr01.mot.com (Motorola/az33exr01) with ESMTP id h9AHXXnp015371; Fri, 10 Oct 2003 12:33:34 -0500 Message-ID: <3F86ED97.6040407@motorola.com> Date: Fri, 10 Oct 2003 10:34:15 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DMO User-Agent: Mozilla/5.0 (X11; U; HP-UX 9000/785; en-US; rv:1.2) Gecko/20021219 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com CC: metakit@equi4.com Subject: Re: [Metakit] Metakit for very large matrices References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: gary, I'm no metakit expert but I currently the size of a metakit is constrained by the file system. So if your OS or file system is limited to 2Gb you could have problem with a matix that is 50,000x50,000 since that 2.5G elements. Tom K. > It strikes me that Metakit might provide a handy way of storing very large > matrices. What I have in mind is something like a 2-dimensional matrice > whose columns represented the categories of one ontology and whose rows > represent the categories of another. By "very large" I mean in the > 50,000x50,000 range (maybe this isn't *very* large?). > > But some questions: > > 1. Will Metakit in fact scale to this? > 2. What is the best technique for creating and populating a view? Do I > need to actually create a 50,000 element description string? This isn't a > problem, but is it the way to do it? > > > -------------------------------------- > Gary H. Merrill From bkelley@wi.mit.edu Fri Oct 10 12:34:39 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9AHYcX09250; Fri, 10 Oct 2003 12:34:38 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003101013343417947 ; Fri, 10 Oct 2003 13:34:34 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HMJY5M00.KD4; Fri, 10 Oct 2003 13:34:34 -0400 Message-ID: <3F86EDE6.8050206@wi.mit.edu> Date: Fri, 10 Oct 2003 13:35:34 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Metakit mailing list Subject: Re: [Metakit] BSDDB vs. Metakit performance? References: <1BEF5662-FB41-11D7-AEE3-000A9588127E@equi4.com> In-Reply-To: <1BEF5662-FB41-11D7-AEE3-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > Personally, I think that unless file size is too constraining, you > should just add the extra property to your view and let MK's > adaptiveness figure out how to store things for that particular column > (what MK does is switch to non-columnar storage if #bytes or #rows > grows too far - there's a heuristic involved to find a decent trade-off). > > If you really want to get to the bottom om this, you should compare > the mix of having data in MK for traversal, and either items in BDB > for large storage or adding a single view with one S or B property, > and storing items in that view. The logic of this should be similar. > I suspect that MK will come out at least as fast (you're retrieving > the N'th item from a view, vs. BDB doing an extra - albeit simple - > hash lookup). You'd gain single-file convenience, and less > installation dependencies. But you'll need to stay under say 1 to 1.5 > Gb. That's good to know. With python bsddb is built-in, but you are write about the single file convenience. Except, I have actually stored my object blobs in another metakit database file entirely. It still boggles my mind you can do a join on views from two different metakit files. It hasn't blown up on me yet(TM). Brian From jcw@equi4.com Fri Oct 10 13:02:31 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9AI2UX11789 for ; Fri, 10 Oct 2003 13:02:30 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 3E4D58399B; Fri, 10 Oct 2003 20:02:22 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id A0C6983999 for ; Fri, 10 Oct 2003 20:02:20 +0200 (CEST) Date: Fri, 10 Oct 2003 20:02:20 +0200 Subject: Re: [Metakit] Metakit for very large matrices Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: metakit@equi4.com Content-Transfer-Encoding: 7bit In-Reply-To: <3F86ED97.6040407@motorola.com> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-3.1 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Tom Krehbiel wrote: > I'm no metakit expert but I currently the size of a metakit is > constrained > by the file system. So if your OS or file system is limited to 2Gb you > could > have problem with a matix that is 50,000x50,000 since that 2.5G > elements. Whoops! Yes, 2.5G elements x 4b = 10 Gb. Not possible today. -jcw, a maths major From jyl@mod3.net Fri Oct 10 13:07:43 2003 Received: from mod3.net (mod3.net [69.55.237.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9AI7hX12098 for ; Fri, 10 Oct 2003 13:07:43 -0500 Received: (qmail 12979 invoked by uid 48); 10 Oct 2003 18:07:42 -0000 Received: from 192.18.42.11 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Fri, 10 Oct 2003 11:07:42 -0700 (PDT) Message-ID: <65173.192.18.42.11.1065809262.squirrel@mod3.net> Date: Fri, 10 Oct 2003 11:07:42 -0700 (PDT) Subject: Re: [Metakit] Metakit for very large matrices From: "Jacob Levy" To: In-Reply-To: References: X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: The intersection of a row and column would represent what exactly? That there are entities that have both of these categories? Or some shared property of the two categories? Anyways, if the value is boolean, Metakit represents this as a one-bit relationship, so we're talking a total of O(50K^2) bits to represent the matrix. That's only 250,000,000 bits or about 32MB, peanuts. I suggest you just create properties on the fly as you need them. I don't know if Metakit does RLE for sparse columns, which would make sense if your matrix is very sparsely populated. In that case you might alternatively wish to consider a tree representation (http://e4graph.com/e4graph) but I'm not sure you'd win back any space savings. --JYL > It strikes me that Metakit might provide a handy way of storing very > large matrices. What I have in mind is something like a 2-dimensional > matrice whose columns represented the categories of one ontology and > whose rows represent the categories of another. By "very large" I mean > in the 50,000x50,000 range (maybe this isn't *very* large?). > > But some questions: > > 1. Will Metakit in fact scale to this? > 2. What is the best technique for creating and populating a view? Do I > need to actually create a 50,000 element description string? This isn't > a problem, but is it the way to do it? > > > -------------------------------------- > Gary H. Merrill > Director and Principal Scientist, New Applications > Data Exploration Sciences > GlaxoSmithKline Inc. > (919) 483-8456 > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jyl@mod3.net Fri Oct 10 13:22:51 2003 Received: from mod3.net (mod3.net [69.55.237.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9AIMpX13330 for ; Fri, 10 Oct 2003 13:22:51 -0500 Received: (qmail 20955 invoked by uid 48); 10 Oct 2003 18:22:51 -0000 Received: from 192.18.42.11 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Fri, 10 Oct 2003 11:22:50 -0700 (PDT) Message-ID: <52968.192.18.42.11.1065810170.squirrel@mod3.net> Date: Fri, 10 Oct 2003 11:22:50 -0700 (PDT) Subject: Re: [Metakit] compression and encryption From: "Jacob Levy" To: In-Reply-To: <5E9E4EBA-FB44-11D7-AEE3-000A9588127E@equi4.com> References: <5E9E4EBA-FB44-11D7-AEE3-000A9588127E@equi4.com> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I would use encryption for storages that contain Tcl programs that represent my IP, definitely. I would use compression for compressable data, e.g. if I stored GIF or JPEG images as blobs. Not too sure about the combination, I can't think of a scenario where I'd like to use compressed + encrypted. But if it can be done without complicating things overly, I'm sure someone will find a use for it. --JYL > Two options not implemented in MK 2.4.9.2, are compression and > encryption. > > Due to the column-wise design of MK, this may actually have substantial > consequences. The idea, is that in a datafile with say layout > "names[first:S,last:S,phones[type:S,number:S]]" it would be possible to > designate some properties as being compressed, others as encrypted, and > yet others as both. > > The compression would take place in a column-wise manner, i.e. all > values of the designated property in all rows would be compressed. If > there is major redundancy/repetition, then the storage size would be > greatly reduced. On first access, such columns would be uncompressed > (taking up some memory), and on commit, the data would be saved > compressed again. > > For compression, it would seem that zlib is the de-facto standard to > use. > > For encryption, a similar effect would be seen: on file, the entire > column becomes encrypted, again for a specific property of all rows in > that view. A complicating factor would be that encryption needs to be > cusomizable, so in this case a callback through the c4_Strategy class > seems the right way to do it. Perhaps some basic encryption such as > David Wheeler's TEA could be included as default. > > When combining compression and encryption, compression would have to be > done first, to have any effect. The encrypted result of that would be > stored on file. On reading, the data must first be decrypted, then > decompressed. > > This also requires a change to somehow specify the details in the > description string, or in some other way. This will require more > thought. > > I'm bringing this up because I am regularly compressing data before > storing it (and Starkits in Tcl do it all the time), and because I > think there is value in getting the encryption covered, especially > since it could be done on a per-property basis. Encryption could be > useful to "lock up" applications deployed as starkits/starpacks. > > The file format has hooks to allow this sort of thing, although such > files will not be readable by current MK releases (they would not know > how to skip over the extra admin info). > > Apart from "yes please, or no thanks", do you consider this a valuable > option? Would you need it and use it right away? Desperate enough to > fund it? (I had to ask...) Any ideas about how to encrypt? Or maybe > only do compress? Are there any implications / trade-offs I'm > forgetting about? > > -jcw, with a small marketing hat on... > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From bkelley@wi.mit.edu Fri Oct 10 14:28:44 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9AJSiX20363 for ; Fri, 10 Oct 2003 14:28:44 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003101015284019482 for ; Fri, 10 Oct 2003 15:28:40 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HMK3FS00.79I for ; Fri, 10 Oct 2003 15:28:40 -0400 Message-ID: <3F8708A4.2020504@wi.mit.edu> Date: Fri, 10 Oct 2003 15:29:40 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Metakit for very large matrices References: <65173.192.18.42.11.1065809262.squirrel@mod3.net> In-Reply-To: <65173.192.18.42.11.1065809262.squirrel@mod3.net> Content-Type: multipart/mixed; boundary="------------070908070406000504040408" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------070908070406000504040408 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Jacob Levy wrote: >The intersection of a row and column would represent what exactly? That >there are entities that have both of these categories? Or some shared >property of the two categories? > My compound fingerprinting scheme is pretty similar, except that I just store integers instead of the individual bits. They are much faster to retrieve. My matrix is something on the order of 1,000,000 x 1024 bits. Attached are the classes that I use (note some of the functionality has not been tested really thoroughly, but at least you'll start with some working code). It also includes code for intersections,unions and subsets (is row a totally contained in row b) --------------070908070406000504040408 Content-Type: text/plain; name="fingerprints.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="fingerprints.py" """Fingerprints stored in a metakit datbase. usage: class Fingerprint(storage, tablename, numInts=32) generate a fingerprint storage inside a database fingerprints = Fingerprint(storage, "compounds") assign data. assign a row: fingerprints[1000] = [...] retrieve an integer: print fingerprint[1000,30] retrieve a bit: print fingerprint.getBit(1000, 1024) set a bit: fingerprint.setBit(1000,30) the row must exist first clear a bit: fingerprint.setBit(1000,30,0) the row must exist first """ import metakit _fingerprintTable = "%s[row:I,%s]" def genFingerprintTable(tablename, numInts): """numInts -> generate the metakit table This table is of the form [row:I,fp0:I,fp1:I,fp2:I...]""" return _fingerprintTable%(tablename, ",".join(["fp%d:I"%x for x in range(0,numInts)])) class Fingerprint: def __init__(self, storage, tablename, numInts=32): self.storage = storage self.numInts = numInts table = self.storage.getas(genFingerprintTable(tablename, numInts)) hash = st.getas("_%s_hash[_H:I,_R:I]"%tablename) self.matrix = table.hash(hash, 1) def getBit(self, row, bit): integer = bit/32 offset = 31 - bit%32 val = self[row,integer] return val&(0x1<; Mon, 13 Oct 2003 08:39:14 -0500 Received: by usrtpnp1.glaxowellcome.com; id JAA08862; Mon, 13 Oct 2003 09:39:59 -0400 (EDT) From: Received: from unknown(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma003056; Mon, 13 Oct 03 09:34:42 -0400 Received: by ussunt3.glaxo.com id JAA06295; Mon, 13 Oct 2003 09:35:11 -0400 (EDT) Received: from 166.71.134.118 by us9n55.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Mon, 13 Oct 2003 09:33:48 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h9DDXlm09242 for ; Mon, 13 Oct 2003 09:33:48 -0400 (EDT) Subject: Re: [Metakit] Metakit for very large matrices To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Mon, 13 Oct 2003 09:34:56 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 10/13/2003 09:33:53 AM MIME-Version: 1.0 X-WSS-ID: 13947636218119-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Thanks for the example code. Since I'm dealing with some things that are potentially very long (documents of, at times, perhaps tens of thousands of words), I was a little concerned about scalability if I were to use ints to hold single bit values. Below is the BitVector class I am currently using. Note that it can be made more portable and perhaps a bit more efficient by first determining int size on the machine. It seems to be working okay in initial tests. There are probably some optimizations that can be made that I haven't gotten around to looking at yet. For various purposes related to my application, I may also be adding some additional methods to it. It might be better to reimplement this as a C extension, but I'm not convinced that the function call overhead wouldn't make it a wash. When I've done this kind of thing in pure C, I've made heavy use of C's macro capibilities. Alas, Python lacks such a feature, and even if this is used in a C extension, the only access to it is through method/function calls from the Python wrapper. """ This module implements bit vectors of any specified length: bv = BitVector(25) # a bit vector of 25 bits Methods are defined to test, set, and reset a bit at a given position. Methods are also defined to overload the operators '&', '|', '^', and '~'. Operations on vectors of different lengths are permitted. In the case of '&' the result is truncated to the length of the shorter operand. In the cases of 'or' and 'xor' the length of the result is the length of the longer operand. The bit vector is implemented as a sequence of integers (of 32 bits) and all operations are performed on units of that size. """ from copy import copy, deepcopy # Bit[n] is the (single bit) mask for the n-th bit in # an int-size word. # # NOTE: This could be made a bit (no pun intended) more # efficient by testing for the size of an int, and # then setting a HighBit mask and Bit sequence based # on that. HighBit_32 = 0x80000000 IntSize = 32 Bit = [0x01 << (31 - i) for i in range(32)] Bit = tuple(Bit) # It won't change. Make it a tuple for efficiency. class BitVector: """ self.length: number of bits in the vector self.numOfInts: number of ints needed for that many bits self.vector: the array of numOfBits ints holding the vector """ def __init__(self, numOfBits): self.length = numOfBits # How many 32-bit ints will it take to hold numOfBits? self.numOfInts = numOfBits / IntSize if numOfBits % IntSize: self.numOfInts += 1 # Create and initialize the vector self.vector = [0 for i in range(self.numOfInts)] def set(self, bitIndex): """ Set bit at positon bitIndex """ slot = bitIndex / IntSize offset = bitIndex % IntSize self.vector[slot] = self.vector[slot] | Bit[offset] def reset(self, bitIndex): """ reset bit at position bitIndex """ slot = bitIndex / IntSize offset = bitIndex % IntSize self.vector[slot] = self.vector[slot] ^ Bit[offset] def test(self, bitIndex): """ test bit at position bitIndex """ slot = bitIndex / IntSize offset = bitIndex % IntSize return self.vector[slot] & Bit[offset] == Bit[offset] def __and__(self, other): """ Bitwise AND of two vectors """ newBv = deepcopy(self) newBv.vector = map(self.andInts, newBv.vector, other.vector) # Truncate the result to the size of the smaller of the # two bit vectors. if self.length < other.length: newBv.length = self.length newBv.numOfInts = self.numOfInts else: newBv.length = other.length newBv.numOfInts = other.numOfInts newBv.vector = newBv.vector[:self.numOfInts] return newBv def __or__(self, other): """ Bitwise OR of two vectors """ newBv = deepcopy(self) newBv.vector = map(self.orInts, newBv.vector, other.vector) # The result will be the length of the longer of the two # bit vectors, as though the remaining slots of the shorter # vector were 0. if self.length > other.length: newBv.length = self.length newBv.numOfInts = self.numOfInts else: newBv.length = other.length newBv.numOfInts = other.numOfInts return newBv def __xor__(self, other): """ Bitwise XOR of two vectors """ newBv = deepcopy(self) newBv.vector = map(self.xorInts, newBv.vector, other.vector) # The result will be the length of the longer of the two # bit vectors, as though the remaining slots of the shorter # vector were 0. if self.length > other.length: newBv.length = self.length newBv.numOfInts = self.numOfInts else: newBv.length = other.length newBv.numOfInts = other.numOfInts return newBv def __invert__(self): """ Invert (negate) the vector """ newBv = deepcopy(self) newBv.vector = map(self.invertInt, newBv.vector) return newBv def andInts(self, firstInt, secondInt): try: result = firstInt & secondInt except: result = None return result def orInts(self, firstInt, secondInt): if firstInt is None: firstInt = 0 if secondInt is None: secondInt = 0 return firstInt | secondInt def xorInts(self, firstInt, secondInt): if firstInt is None: firstInt = 0 if secondInt is None: secondInt = 0 return firstInt ^ secondInt def invertInt(self, anInt): return ~anInt def isZero(self): for i in self.vector: if i != 0: return False return True -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From niki@vintech.bg Tue Oct 14 03:05:54 2003 Received: from intra.vintech.l (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9E85mX22854 for ; Tue, 14 Oct 2003 03:05:49 -0500 Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id 1B88457222; Tue, 14 Oct 2003 11:07:32 +0300 (EEST) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id AFF91239A7; Tue, 14 Oct 2003 11:05:36 +0300 (EEST) Message-ID: <3F8BADCF.4000507@vintech.bg> Date: Tue, 14 Oct 2003 11:03:27 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: bg, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com Cc: metakit@equi4.com Subject: Re: [Metakit] Metakit for very large matrices References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: gary.h.merrill@gsk.com wrote: > Thanks for the example code. Since I'm dealing with some things that are > potentially very long (documents of, at times, perhaps tens of thousands of > words), I was a little concerned about scalability if I were to use ints to > hold single bit values. Below is the BitVector class I am currently using. > Note that it can be made more portable and perhaps a bit more efficient by > first determining int size on the machine. It seems to be working okay in > initial tests. There are probably some optimizations that can be made that > I haven't gotten around to looking at yet. > > For various purposes related to my application, I may also be adding some > additional methods to it. > > It might be better to reimplement this as a C extension, but I'm not > convinced that the function call overhead wouldn't make it a wash. When > I've done this kind of thing in pure C, I've made heavy use of C's macro > capibilities. Alas, Python lacks such a feature, and even if this is used > in a C extension, the only access to it is through method/function calls > from the Python wrapper. Most of it is in C already - Python long HTH Niki Spahiev From rcohen@articque.com Tue Oct 14 06:41:04 2003 Received: from mwinf0802.wanadoo.fr (smtp8.wanadoo.fr [193.252.22.23]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9EBf3X05065 for ; Tue, 14 Oct 2003 06:41:03 -0500 Received: from articque.com (ATours-101-1-2-19.w217-128.abo.wanadoo.fr [217.128.52.19]) by mwinf0802.wanadoo.fr (SMTP Server) with ESMTP id 2B3D4180017F for ; Tue, 14 Oct 2003 13:40:57 +0200 (CEST) Message-ID: <3F8BE0BA.3090808@articque.com> Date: Tue, 14 Oct 2003 13:40:42 +0200 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] pclint Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, Is anyone using pclint? When testing something like : #include int main() { c4_StringProp region_id("region_id"); c4_Row reg2postrow; const char *str_regid="ok"; region_id(reg2postrow)=str_regid; return(0); } I have the following error : PC-lint for C/C++ (NT) Ver. 8.00e, Copyright Gimpel Software 1985-2001 --- Module: d:\zdrop\test.cpp region_id(reg2postrow)=str_regid; d:\zdrop\test.cpp(8): error 1058: (Error -- Initializing a non-const reference 'c4_StringRef &' with a non-lvalue) --- Global Wrap-up D:\develext\mk4\include\mk4.h(567): error 1512: (Warning -- destructor for base class 'c4_Property' (line 567, file D:\develext\mk4\include\mk4.h) is not virtual) D:\develext\mk4\include\mk4.h(934): error 1512: (Warning -- destructor for base class 'c4_Reference' (line 934, file D:\develext\mk4\include\mk4.h) is not virtual) D:\develext\mk4\include\mk4.h(221): error 1512: (Warning -- destructor for base class 'c4_View' (line 221, file D:\develext\mk4\include\mk4.h) is not virtual) error 900: (Note -- Successful completion, 4 messages produced) Tool returned code: 4 The error 1058 says that: 1058 Initializing a non-const reference 'Symbol' with a non-lvalue -- A reference is normally initialized with an lvalue. If you attempt to initialize a reference with a non-lvalue, a temporary is created to serve as a surrogate lvalue. However, modifications made to the temporary will be lost. This was legal at one time and is now illegal. Make the reference a const if you can. You may be initializing a reference without realizing it. A member function has an implicit parameter, which is taken to be a reference to its object. If this is the situation make the member const. That is, use void f(...) const; rather than void f(...); The error 1512 says that: 1512 destructor for base class 'Symbol' (Location) is not virtual -- In a final pass through all the classes, we have found a class (named in the message) that is the base class of a derivation and has a destructor but the destructor is not virtual. It is conventional for inherited classes to have virtual destructors so that it is safe to 'delete' a pointer to a base class. [19] i'm not afraid of the second error, even if I would rather not have it. But is seems that the first error on region_id(reg2postrow)=str_regid; is relevant of a real problem. They talk about temporary created and then not used... Do you think there is a problem ? Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From andreas.theDragon@gmx.net Wed Oct 15 06:58:51 2003 Received: from mail.gmx.net (mail.gmx.de [213.165.64.20]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9FBwnX26824 for ; Wed, 15 Oct 2003 06:58:50 -0500 Received: (qmail 5446 invoked by uid 65534); 15 Oct 2003 11:58:42 -0000 Received: from pC19EABBC.dip0.t-ipconnect.de (EHLO pC19EABBC.dip0.t-ipconnect.de) (193.158.171.188) by mail.gmx.net (mp015) with SMTP; 15 Oct 2003 13:58:42 +0200 X-Authenticated: #1987504 Date: Wed, 15 Oct 2003 13:53:54 +0200 From: Andreas Muegge X-Mailer: The Bat! (v2.00.6) Personal Reply-To: Andreas Muegge X-Priority: 3 (Normal) Message-ID: <636647926.20031015135354@gmx.de> To: metakit@equi4.com Subject: Re: [Metakit] compression and encryption In-Reply-To: <5E9E4EBA-FB44-11D7-AEE3-000A9588127E@equi4.com> References: <5E9E4EBA-FB44-11D7-AEE3-000A9588127E@equi4.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, the implementation of encryption is something I would like to see in Metakit. It would avoid another layer of complexity within the application using Metakit and thus the decision to use strong protection of data would be easier to made. I am not sure about compression. If we talk about "normal" strings I guess you must try it out. Decompression is usually rather fast and waiting 2 seconds at program start shouldn't bother the user. Would it still be possible to run Metakit on a readonly medium? For big data (1k and more) I have serious doubts. You would have to decompress several MBytes before the first access is allowed and this can be annoying for the user. In my application I have used the LZO library which doesn't compress as well as zlib but which is much faster. Of course I can only compress each record and not the whole column. Ciao, Andreas From ian.fairclough@intelligent-ms.com Wed Oct 15 14:10:24 2003 Received: from scratchy.base.ims ([194.74.0.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9FJANX19521 for ; Wed, 15 Oct 2003 14:10:23 -0500 Received: from grpware.intelligent-ms.com (grpware.base.ims [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id h9FIvol15624 for ; Wed, 15 Oct 2003 19:57:50 +0100 Received: from imsgsx6tvznbng ([192.168.200.144]) by grpware.intelligent-ms.com (8.11.2/8.11.2) with SMTP id h9FIpou15649 for ; Wed, 15 Oct 2003 19:51:50 +0100 Message-ID: <0cc001c39350$3f697b10$90c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Date: Wed, 15 Oct 2003 20:12:14 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0CBD_01C39358.9F9D7F10" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] c4_Storage new returns nil Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0CBD_01C39358.9F9D7F10 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable A quick question. I have the following code c4_Storage *db_ =3D new c4_Storage (filename, true); However, db_ is always 0 following the call. Is this behaviour expected or does it indicate something else is wrong ? Thanks in advance for any info you can give. Best regards, Ian Fairclough. ------=_NextPart_000_0CBD_01C39358.9F9D7F10 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
A quick question.
 
I have the following code
 
c4_Storage *db_ =3D new c4_Storage = (filename,=20 true);
 
However, db_ is always 0 following the=20 call.
 
Is this behaviour expected or does it = indicate=20 something else is wrong ?
 
Thanks in advance for any info you can=20 give.
 
Best regards,
Ian = Fairclough.
------=_NextPart_000_0CBD_01C39358.9F9D7F10-- From jcw@equi4.com Thu Oct 16 03:34:55 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9G8YsX25938 for ; Thu, 16 Oct 2003 03:34:55 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 25BF98399B; Thu, 16 Oct 2003 10:34:48 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 8BC3483999 for ; Thu, 16 Oct 2003 10:34:47 +0200 (CEST) Date: Thu, 16 Oct 2003 10:34:47 +0200 Subject: Re: [Metakit] c4_Storage new returns nil Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: In-Reply-To: <0cc001c39350$3f697b10$90c8a8c0@imsgsx6tvznbng> Message-Id: <9A0F1318-FFB3-11D7-AEE3-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.7 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h9G8YsX25938 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ian Fairclough wrote: > A quick question. >   > I have the following code >   > c4_Storage *db_ = new c4_Storage (filename, true); Storage objects are reference-counted smart pointers in C++. The above defeats it - though a null pointer return from "new" points to entirely different issues (memory full? heap corruption?). Use either of these: c4_Storage db (filename, true); or c4_Storage db = c4_Storage (filename, true); Same for c4_Views - c4_Storage is derived from c4_View. Just allocate on the stack, or as object members and pass around by copying. You'll be copying a single pointer, it's very efficient. The reward is fully automatic ref-counted memory management, which automatically cleans up on return, exceptions, etc. -jcw From jcw@equi4.com Thu Oct 16 03:45:20 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9G8jJX27159 for ; Thu, 16 Oct 2003 03:45:20 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 3EFEA8399B; Thu, 16 Oct 2003 10:45:13 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id BF7BD83999 for ; Thu, 16 Oct 2003 10:45:12 +0200 (CEST) Date: Thu, 16 Oct 2003 10:45:13 +0200 Subject: Re: [Metakit] compression and encryption Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <636647926.20031015135354@gmx.de> Message-Id: <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-3.1 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Andreas Muegge wrote: > the implementation of encryption is something I would like to see Ok, thanks for letting me know (surprising: just two responses so far). > I am not sure about compression. If we talk about "normal" strings I > guess > you must try it out. Decompression is usually rather fast and waiting 2 > seconds at program start shouldn't bother the user. Would it still be > possible to run Metakit on a readonly medium? Yes. Decompression would be an in-memory thing. > For big data (1k and more) I have serious doubts. You would have to > decompress several MBytes before the first access is allowed [...] No, the decompression would happen per column, on first access, and you can pick per column which one is stored compressed and which one isn't. For large strings, compression would not be per column even, but per item. The switchover point is hard to define, MK uses an adaptive heuristic to choose between ways to store strings. > Of course I can only compress each record and not the whole column. Yes, per-item compression (BTW, it's not record but item, i.e. property) is always possible of course, at the MK caller level. The encryption/compression I'm talking about would be column-wise, i.e. very effective with views in which a property has the same value across many rows. For an impression of the effectiveness on your data, create a datafile from scratch so it has no free space, and gzip it - the results could be normal (i.e. 10..30% reduction) or dramatic (i.e. 90% reduction), depending on the nature of your data. I once converted a 120 Mb database stored in another format, and ended up with 10 Mb in MK, which them went to 900 Kb when gzipped... -jcw From mss@mawhrin.net Thu Oct 16 04:15:27 2003 Received: from only.mawhrin.net (base.mawhrin.net [69.55.226.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9G9FRX30164 for ; Thu, 16 Oct 2003 04:15:27 -0500 Received: by only.mawhrin.net (Postfix, from userid 1000) id DAE8C67807C; Thu, 16 Oct 2003 13:15:24 +0400 (MSD) Date: Thu, 16 Oct 2003 13:15:24 +0400 To: Metakit mailing list Subject: Re: [Metakit] compression and encryption Message-ID: <20031016091524.GA12275@mawhrin.net> Mail-Followup-To: Metakit mailing list References: <636647926.20031015135354@gmx.de> <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tKW2IUtsqtDRztdT" Content-Disposition: inline In-Reply-To: <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> User-Agent: Mutt/1.5.4i From: mss@mawhrin.net (Mikhail Sobolev) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: --tKW2IUtsqtDRztdT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 16, 2003 at 10:45:13AM +0200, Jean-Claude Wippler wrote: > >the implementation of encryption is something I would like to see >=20 > Ok, thanks for letting me know (surprising: just two responses so far). Just for the record: ecnryption: +1 compression: -0 -- Misha --tKW2IUtsqtDRztdT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/jmGsMYLUe7XufiwRAtEAAJ4+J8jimyOzfDWkSpUnQnZn/39+jwCdEUFD zZvqbIlI9PFDO2PTDK8tisU= =91un -----END PGP SIGNATURE----- --tKW2IUtsqtDRztdT-- From angus@europeonline.net Thu Oct 16 05:31:32 2003 Received: from unicorn.europeonline.net (unicorn.europeonline.net [194.177.33.81]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9GAVVX05629 for ; Thu, 16 Oct 2003 05:31:32 -0500 Received: from groovinhigh (pppoe232-luxdsl-165.pt.lu [213.135.232.165]) (authenticated bits=0) by unicorn.europeonline.net (8.12.9/8.12.9) with ESMTP id h9GAVQuT098342 for ; Thu, 16 Oct 2003 12:31:28 +0200 (CEST) (envelope-from angus@europeonline.net) From: "Angus Lord" To: "'Metakit mailing list'" Subject: RE: [Metakit] compression and encryption Date: Thu, 16 Oct 2003 12:31:20 +0200 Message-ID: <001a01c393d0$a7c87a00$0900a8c0@groovinhigh> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 In-Reply-To: <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Importance: Normal X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham version=2.60 X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on spam.europeonline.net X-Scanned-By: MIMEDefang 2.36 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id h9GAVVX05629 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I can see that both would be interesting for my application. Just as a thought, would the lzo algorithm be better than using zlib? It has a far smaller footprint than zlib and is _really_ fast. Cheers Angus From jyl@mod3.net Thu Oct 16 09:25:54 2003 Received: from mod3.net (mod3.net [69.55.237.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9GEPrX24470 for ; Thu, 16 Oct 2003 09:25:53 -0500 Received: (qmail 32176 invoked by uid 48); 16 Oct 2003 14:25:51 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 16 Oct 2003 07:25:51 -0700 (PDT) Message-ID: <64698.209.233.24.218.1066314351.squirrel@mod3.net> Date: Thu, 16 Oct 2003 07:25:51 -0700 (PDT) Subject: Re: [Metakit] compression and encryption From: "Jacob Levy" To: In-Reply-To: <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> References: <636647926.20031015135354@gmx.de> <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Dont remember if I expressed my opinion already: * Compression: useful for large data, e.g. binary and string, less useful for other types. * Encryption: very useful. * Combination of the two: not sure where I'd use it. --JYL > Andreas Muegge wrote: > >> the implementation of encryption is something I would like to see > > Ok, thanks for letting me know (surprising: just two responses so far). > >> I am not sure about compression. If we talk about "normal" strings I >> guess >> you must try it out. Decompression is usually rather fast and waiting >> 2 seconds at program start shouldn't bother the user. Would it still >> be possible to run Metakit on a readonly medium? > > Yes. Decompression would be an in-memory thing. > >> For big data (1k and more) I have serious doubts. You would have to >> decompress several MBytes before the first access is allowed [...] > > No, the decompression would happen per column, on first access, and you > can pick per column which one is stored compressed and which one isn't. > For large strings, compression would not be per column even, but per > item. The switchover point is hard to define, MK uses an adaptive > heuristic to choose between ways to store strings. > >> Of course I can only compress each record and not the whole column. > > Yes, per-item compression (BTW, it's not record but item, i.e. > property) is always possible of course, at the MK caller level. > > The encryption/compression I'm talking about would be column-wise, i.e. > very effective with views in which a property has the same value across > many rows. For an impression of the effectiveness on your data, create > a datafile from scratch so it has no free space, and gzip it - the > results could be normal (i.e. 10..30% reduction) or dramatic (i.e. 90% > reduction), depending on the nature of your data. I once converted a > 120 Mb database stored in another format, and ended up with 10 Mb in > MK, which them went to 900 Kb when gzipped... > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From niki@vintech.bg Thu Oct 16 09:42:58 2003 Received: from intra.vintech.l (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9GEguX27600 for ; Thu, 16 Oct 2003 09:42:56 -0500 Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id D372957222 for ; Thu, 16 Oct 2003 17:45:31 +0300 (EEST) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id B4803239A7 for ; Thu, 16 Oct 2003 17:42:49 +0300 (EEST) Message-ID: <3F8EADE4.40801@vintech.bg> Date: Thu, 16 Oct 2003 17:40:36 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: bg, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] compression and encryption References: <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> In-Reply-To: <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I am using compression already. Python properties combined with sting encode('zlib') handle it transparently to the user. Niki Spahiev From bkelley@wi.mit.edu Thu Oct 16 10:50:32 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9GFoWX02622 for ; Thu, 16 Oct 2003 10:50:32 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003101611502702367 for ; Thu, 16 Oct 2003 11:50:27 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HMUXC300.KCI for ; Thu, 16 Oct 2003 11:50:27 -0400 Message-ID: <3F8EBE89.2010904@wi.mit.edu> Date: Thu, 16 Oct 2003 11:51:37 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] compression and encryption References: <0EA6A2D0-FFB5-11D7-AEE3-000A9588127E@equi4.com> <3F8EADE4.40801@vintech.bg> In-Reply-To: <3F8EADE4.40801@vintech.bg> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I suppose that this falls along the lines of I'll wait to see the implementation (TM). Since these are non-invasive changes to metakit, I'm always happy to see useful features. Here are my current comments with the proposed compression strategy: It seems that using the in-memory uncompression would make metakit unscalable in some cases. I target machines ranging from 64MB of memory to 1G and metakit scales(surprisingly) very nicely in these cases. I'm not certain if an in-memory decompression would scale as nicely to lower end machines. It seems that you could have a more scalable usage using temporary tables. compressed column -> uncompressed temporary (on-disk) column -> commits recompress temporary column on close the temporary column is removed from the storage. This seems to be the tricky part as I remember that metakit usage of free space in the database doesn't actually immediately save disk space when tables or columns are dropped which is sort of the point of compression :) Anyway, that's my two bits. I have been happy so far using row level compression with zlib as some other users have noted. Brian From schlenk@uni-oldenburg.de Thu Oct 16 11:07:19 2003 Received: from mailout11.sul.t-online.com (mailout11.sul.t-online.com [194.25.134.85]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9GG7IX04023; Thu, 16 Oct 2003 11:07:18 -0500 Received: from fwd02.aul.t-online.de by mailout11.sul.t-online.com with smtp id 1AAAej-0000rb-03; Thu, 16 Oct 2003 18:07:17 +0200 Received: from pD9526E6A.dip.t-dialin.net (XRfbw-ZloedRbPe2+850XdmyISh96PUvmgFbI-egp+y94WPuHEsBsK@[217.229.108.239]) by fmrl02.sul.t-online.com with esmtp id 1AAAeR-05bZom0; Thu, 16 Oct 2003 18:06:59 +0200 Received: from uni-oldenburg.de (unknown [192.168.2.2]) by pD9526E6A.dip.t-dialin.net (Postfix) with ESMTP id 747187BEF; Thu, 21 Jun 2001 03:02:00 +0000 (/etc/localtime) Message-ID: <3F8EC37B.5060506@uni-oldenburg.de> Date: Thu, 16 Oct 2003 18:12:43 +0200 From: Michael Schlenker Organization: =?ISO-8859-1?Q?Universit=E4t_Oldenburg?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: de-de, en-us MIME-Version: 1.0 To: Jean-Claude Wippler Cc: Metakit mailing list Subject: Re: [Metakit] compression and encryption References: <5E9E4EBA-FB44-11D7-AEE3-000A9588127E@equi4.com> In-Reply-To: <5E9E4EBA-FB44-11D7-AEE3-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Seen: false X-ID: XRfbw-ZloedRbPe2+850XdmyISh96PUvmgFbI-egp+y94WPuHEsBsK@t-dialin.net Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Two options not implemented in MK 2.4.9.2, are compression and > encryption. compression sounds good encryption done right sounds good too, but if i understand correctly it is only thought for the file stored, not an in memory encryption so data could end up unecrypted in a swap file? It's ok but encryption is only valuable against a certain class of attacks, as we all know.. For the encryption you really need a threat analysis, against what kind of attacks on a MK database shall it be effective? Per column compression sounds good, if the needed time and memory for compression/decompression on commit/access is reasonable. Zlib would probably ok, bzip2 could be an option for large databases. Michael From gary.h.merrill@gsk.com Fri Oct 17 15:01:28 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9HK1SX06084 for ; Fri, 17 Oct 2003 15:01:28 -0500 Received: by usrtpnp1.glaxowellcome.com; id QAA07873; Fri, 17 Oct 2003 16:02:15 -0400 (EDT) From: Received: from unknown(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma017348; Fri, 17 Oct 03 15:09:08 -0400 Received: by ussunt3.glaxo.com id PAA03906; Fri, 17 Oct 2003 15:08:16 -0400 (EDT) Received: from 166.71.134.118 by us9n57.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Fri, 17 Oct 2003 15:08:09 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h9HJ89m17329 for ; Fri, 17 Oct 2003 15:08:09 -0400 (EDT) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Fri, 17 Oct 2003 15:07:44 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 10/17/2003 03:08:10 PM MIME-Version: 1.0 X-WSS-ID: 138EE193748406-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Where have all the methods gone? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I'm just experimenting with manipulating some views and various of the methods seem to be missing. For example, 'delete'. Consider the following: >>> for r in current_term_view: print r.tokenId ... 51 >>> current_term_view.delete(0) Attribute not found: delete Traceback (innermost last): File "", line 1, in ? >>> This seems to indicate that current_term_view is a respectable view, and yet there is not 'delete' method for it. What am I missing here? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Fri Oct 17 15:42:14 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9HKgDX09132 for ; Fri, 17 Oct 2003 15:42:13 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003101716420817845 for ; Fri, 17 Oct 2003 16:42:08 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HMX5I900.QHH; Fri, 17 Oct 2003 16:42:09 -0400 Message-ID: <3F905468.5030802@wi.mit.edu> Date: Fri, 17 Oct 2003 16:43:20 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com CC: metakit@equi4.com Subject: Re: [Metakit] Where have all the methods gone? References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: gary.h.merrill@gsk.com wrote: >I'm just experimenting with manipulating some views and various of the >methods seem to be missing. For example, 'delete'. >Consider the following: > > If you think of views as sort of enhanced lists you should be all right. del current_term_view[0] or delete a bunch of rows del current_term_view[0:1] ==>deletes the 0th row del current_term_view[0:10] ==>deletes the first 10 rows >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > From gmcm@hypernet.com Sat Oct 18 07:46:23 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9ICkNX15156 for ; Sat, 18 Oct 2003 07:46:23 -0500 Received: from PARANOIA (204.176.40.69) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Sat, 18 Oct 2003 08:48:12 -0400 From: "Gordon McMillan" To: , metakit@equi4.com Date: Sat, 18 Oct 2003 08:44:35 -0400 MIME-Version: 1.0 Subject: Re: [Metakit] Where have all the methods gone? Reply-to: gmcm@hypernet.com Message-ID: <3F90FD73.10599.7B558B7@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 17 Oct 2003 at 15:07, gary.h.merrill@gsk.com wrote: > I'm just experimenting with manipulating some views and various of the > methods seem to be missing. For example, 'delete'. Consider the > following: > > >>> for r in current_term_view: print r.tokenId > ... > 51 > >>> current_term_view.delete(0) > Attribute not found: delete There are Views, Viewers and ROViewers. The 2nd will allow mutating rows, but not the container. The 3rd is read only. For example, joining 2 views will result in a ROViewer. -- Gordon http://www.mcmillan-inc.com/ From bkelley@wi.mit.edu Sun Oct 19 08:14:27 2003 Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9JDERX28122 for ; Sun, 19 Oct 2003 08:14:27 -0500 Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.147.123.161]) by comcast.net (rwcrmhc11) with SMTP id <2003101913142001300pa5nqe>; Sun, 19 Oct 2003 13:14:21 +0000 Message-ID: <3F928F73.1050707@wi.mit.edu> Date: Sun, 19 Oct 2003 09:19:47 -0400 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com CC: metakit@equi4.com Subject: Re: [Metakit] Where have all the methods gone? References: <3F90FD73.10599.7B558B7@localhost> In-Reply-To: <3F90FD73.10599.7B558B7@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: >There are Views, Viewers and ROViewers. The 2nd will allow >mutating rows, but not the container. The 3rd is read only. >For example, joining 2 views will result in a ROViewer. > Ahh, now I understand the question. I was also surprised when view.sort(...) returned a ROViewer. This *seems* like a bug to me but it might be part of the internal mechanics at work. The easiest way to find out what you have is: print view which will tell you what kind of view you are looking at. > >-- Gordon >http://www.mcmillan-inc.com/ > > From gary.h.merrill@gsk.com Mon Oct 20 06:44:10 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9KBi9X25456 for ; Mon, 20 Oct 2003 06:44:09 -0500 Received: by usrtpnp1.glaxowellcome.com; id HAA28193; Mon, 20 Oct 2003 07:44:55 -0400 (EDT) From: Received: from unknown(152.51.63.189) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma026954; Mon, 20 Oct 03 07:41:55 -0400 Received: by ussunt2.glaxo.com id HAA04780; Mon, 20 Oct 2003 07:41:08 -0400 (EDT) Received: from 166.71.134.118 by us9n52.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Mon, 20 Oct 2003 07:41:00 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h9KBf0m20886; Mon, 20 Oct 2003 07:41:00 -0400 (EDT) Subject: Re: [Metakit] Where have all the methods gone? To: bkelley@wi.mit.edu cc: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Mon, 20 Oct 2003 07:40:25 -0400 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 10/20/2003 07:41:01 AM MIME-Version: 1.0 X-WSS-ID: 138D164622730-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Seems like a bug to me as well. In addition, I don't think this distinction is even hinted at in the Python documentation. So I was really confused. One way or another this is a bug. It's either a functionality bug or a documentation bug. Take your pick :-). I suspected something like this was going on, but wondered if it was just a bug in the Python wrapper. I also noticed that if I did a copy of the sorted view, then I could delete stuff in *that*. But beyond that I don't yet know what to expect. My single biggest frustration in using Metakit is in figuring out how it works. I don't really want to pour over the C++ source code and then figure out how that maps to the Python wrapper, what various (at times quite unsuspected) constraints are, etc. At this point in the maturity of Metakit I would urge that substantial effort be devoted to documentation rather than enhancements. I know this is painful and no one likes to do it, but it's really an essential part of making something a *product* that can be used. Compare, for example, Metakit to wxWindows and wxPython (which I use heavily). The wxPython documentation is hardly a paradigm of completeness, but coupled with the extensive demos and the wxWindows documentation (well-organized in an online form that is browseable and searchable, and with embedded Python notes), it doesn't take much effort to use the product. I think this sort of thing is the single most important enhancement that Metakit now needs. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 "Brian Kelley" To: gary.h.merrill@gsk.com 19-Oct-2003 09:19 cc: metakit@equi4.com Subject: Re: [Metakit] Where have all the methods gone? >There are Views, Viewers and ROViewers. The 2nd will allow >mutating rows, but not the container. The 3rd is read only. >For example, joining 2 views will result in a ROViewer. > Ahh, now I understand the question. I was also surprised when view.sort(...) returned a ROViewer. This *seems* like a bug to me but it might be part of the internal mechanics at work. The easiest way to find out what you have is: print view which will tell you what kind of view you are looking at. > >-- Gordon >http://www.mcmillan-inc.com/ > > From mark@markroseman.com Mon Oct 20 09:12:23 2003 Received: from fep2.cogeco.net (smtp.cogeco.net [216.221.81.25]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9KECNX03339 for ; Mon, 20 Oct 2003 09:12:23 -0500 Received: from markroseman.com (d141-166-106.home.cgocable.net [24.141.166.106]) by fep2.cogeco.net (Postfix) with ESMTP id C09C01306 for ; Mon, 20 Oct 2003 10:12:15 -0400 (EDT) Date: Mon, 20 Oct 2003 10:15:56 -0400 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Mark Roseman To: metakit@equi4.com Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.552) Subject: [Metakit] documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > My single biggest frustration in using Metakit is in figuring out how > it > works. ...... At this point in the maturity of > Metakit I would urge that substantial effort be devoted to > documentation > rather than enhancements. I know this is painful and no one likes to > do > it, but it's really an essential part of making something a *product* > that > can be used. ..... I think this sort of thing is the single most > important > enhancement that Metakit now needs. While I would heartily agree that Metakit could use better documentation (and I suspect few would disagree on that point), I think the question is really how does it get there? That's certainly one of the drawbacks of Jean-Claude having released Metakit as open source; he doesn't have any particular incentives to spend significant time on the documentation: it doesn't scratch a personal itch for him, nor would I suggest (based on past history) its something that he enjoys nearly as much as coding and experimenting. That's not to say it shouldn't be done, it just makes it harder to get motivated to do so. I think it will probably come down to needing more involvement from the Metakit user community to advance the documentation situation, either in terms of people helping to actually do it, or putting down some money so that some other people could work on it. I've done a bit so far, and keep meaning to do a bit more, but its always hard to find the time and motivation. My opinions anyway. Mark From bkelley@wi.mit.edu Mon Oct 20 09:51:11 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9KEpBX05539 for ; Mon, 20 Oct 2003 09:51:11 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102010510700101 for ; Mon, 20 Oct 2003 10:51:07 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HN299700.URD for ; Mon, 20 Oct 2003 10:51:07 -0400 Message-ID: <3F93F6A0.8050203@wi.mit.edu> Date: Mon, 20 Oct 2003 10:52:16 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Where have all the methods gone? References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: The wxWindows comparison is apt. But I will note that the vast majority of the wxPython documentation is user written. There are plenty of issues that will cause a new user to stop in their tracks. For example, how to drag an object to a virtual grid. I added the documentation for this in the wiki because it took me more than ten minutes to solve. I am (slowly) working on new metakit python documentation but it really is not at the top of my list. FYI, this sorting issue came up a little while ago: http://www.mail-archive.com/metakit@equi4.com/msg00282.html ...Back in the mists of time Gordon wrote: >Mk4py makes select()ed and sort()ed views look (as much as it can) >like mapping views (modifiable). It's not a direct capability of the >C++ lib. >I almost pulled out that code. Why? Because I *never* use it anymore. >I still use select & sort, but only for reporting. If I'm doing >modifications that depend on complex criteria, I use map, filter, >remapwith, remove etc. I use ordered views extensively. If 2 different >orderings are commonly required, I might even maintain an external >index, (sort is rather expensive). I end up using hash and ordered views in these cases, in fact all my tables have a column "id" which is a unique for the view. In this fashion I can easily delete objects: index = view.find(id=object_id) if index != -1: delete view[index] Using the C++ code as a guide is not, in my opinion, very useful since Gordon rightfully took a pythonic approach to creating the python metakit wrappers and paradoxically Mk4py has at the same time *more* and *less* functionality than metakit. Bug reports are very welcome, of course. Gordon helped me fix some MkSQL issues a while ago. In this case it would be easier to A) fix the documentation (this is in progress) and B) fix the error messages. I.e. delete view[0] could return an error message like: AttributeError: Read only view Brian From baspey@yahoo.com Mon Oct 20 18:16:47 2003 Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9KNGlX09489 for ; Mon, 20 Oct 2003 18:16:47 -0500 Message-ID: <20031020231645.80641.qmail@web11501.mail.yahoo.com> Received: from [213.36.114.95] by web11501.mail.yahoo.com via HTTP; Mon, 20 Oct 2003 16:16:45 PDT Date: Mon, 20 Oct 2003 16:16:45 -0700 (PDT) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] KitBinder Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I'm interested in KitBinder features, but I can't find more than this page: http://www.equi4.com/metakit/api-old/doc_kbind.html I easily embed a metakit file into my app's resources, but I fail to open it from there (Visual C++). Can someone shed some light on this ? Thanks. __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From jcw@equi4.com Tue Oct 21 06:23:38 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9LBNcX15669 for ; Tue, 21 Oct 2003 06:23:38 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 41E3D83AEF; Tue, 21 Oct 2003 13:23:31 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id C0CFD83AE7 for ; Tue, 21 Oct 2003 13:23:30 +0200 (CEST) Date: Tue, 21 Oct 2003 13:23:30 +0200 Subject: Re: [Metakit] KitBinder Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <20031020231645.80641.qmail@web11501.mail.yahoo.com> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.6 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,SPAM_PHRASE_03_05, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Pascal Baspeyras wrote: > I'm interested in KitBinder features, but I can't find > more than this page: > http://www.equi4.com/metakit/api-old/doc_kbind.html > > I easily embed a metakit file into my app's resources, > but I fail to open it from there (Visual C++). Whoa ... that's 5 year old technology, it's truly ancient! ;) With today's MK, all you need is to append the MK datafile to the end of your executable. Then open the executable (read-only) and you'll have a datafile... this is cross-platform. The trick is to find out the path of the file to open. In Win32, use GetModuleFileName. -jcw From bkelley@wi.mit.edu Tue Oct 21 13:44:16 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LIiGX10806 for ; Tue, 21 Oct 2003 13:44:16 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102114441214374 for ; Tue, 21 Oct 2003 14:44:12 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HN4EPO00.K9I for ; Tue, 21 Oct 2003 14:44:12 -0400 Message-ID: <3F957EC0.1080001@wi.mit.edu> Date: Tue, 21 Oct 2003 14:45:20 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] New metakit python documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Consider this a trial version. We always seem to have documentation questions on the mailing list and, admittedly, there are some aspects of metakit which are confusing. In the vein of user supported documentation, I have extended the standard documents with some usage notes and source code. Bugs and comments are welcome. http://jura.wi.mit.edu/people/kelley/tutorial/python.html Enjoy. From baspey@yahoo.com Tue Oct 21 13:45:16 2003 Received: from web11503.mail.yahoo.com (web11503.mail.yahoo.com [216.136.172.35]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LIjFX10957 for ; Tue, 21 Oct 2003 13:45:16 -0500 Message-ID: <20031021184515.41522.qmail@web11503.mail.yahoo.com> Received: from [213.36.114.76] by web11503.mail.yahoo.com via HTTP; Tue, 21 Oct 2003 11:45:15 PDT Date: Tue, 21 Oct 2003 11:45:15 -0700 (PDT) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] KitBinder Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Thank you for the quick reply, Jean-Claude. I had indeed discovered this interesting possibility from a recent thread on this list. My problem is that the data I plan to embed into my application is necessary for it to work. So each time I'll modify my code, I'll have to recompile as usual, but also to append the database at the end of the generated binaries. With this solution, I think that my app would become a hell to develop and debug, which wouldn't be the case if the database was simply embedded as a resource. Am I missing something ? If only I could have a peek at KitBinder's code to see how it works... __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From bkelley@wi.mit.edu Tue Oct 21 13:55:42 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LItgX11949 for ; Tue, 21 Oct 2003 13:55:42 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102114553714562 for ; Tue, 21 Oct 2003 14:55:37 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HN4F8Q00.5AR for ; Tue, 21 Oct 2003 14:55:38 -0400 Message-ID: <3F95816D.4000703@wi.mit.edu> Date: Tue, 21 Oct 2003 14:56:45 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 CC: metakit@equi4.com Subject: Re: [Metakit] KitBinder References: <20031021184515.41522.qmail@web11503.mail.yahoo.com> In-Reply-To: <20031021184515.41522.qmail@web11503.mail.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > With this solution, I think that my app would become a > hell to develop and debug, which wouldn't be the case > if the database was simply embedded as a resource. > > Am I missing something ? > > Under unix: cat executable database > new_executable chmod +x new_executable Under windows copy /b executable+database new_executable Metakit takes care of the rest. From baspey@yahoo.com Tue Oct 21 14:16:45 2003 Received: from web11508.mail.yahoo.com (web11508.mail.yahoo.com [216.136.172.40]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LJGiX13574 for ; Tue, 21 Oct 2003 14:16:45 -0500 Message-ID: <20031021191643.37298.qmail@web11508.mail.yahoo.com> Received: from [213.36.116.158] by web11508.mail.yahoo.com via HTTP; Tue, 21 Oct 2003 12:16:43 PDT Date: Tue, 21 Oct 2003 12:16:43 -0700 (PDT) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] KitBinder Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: >Under unix: > >cat executable database > new_executable >chmod +x new_executable > >Under windows > >copy /b executable+database new_executable > >Metakit takes care of the rest. Thank you for the VERY quick reply, Brian. In my case, it's Windows (Visual Studio 6). Let's suppose I'm in debug mode, trying hard to fix a bug. I make various modifications, then hit the debug button... It seems to me that it becomes quite complicated with the appended database method, while it would be trivial if the database was embedded as a resource. Once again, I might be missing something. (who said brains ?) __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From bkelley@wi.mit.edu Tue Oct 21 14:34:25 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LJYPX14771 for ; Tue, 21 Oct 2003 14:34:25 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102115342015182 for ; Tue, 21 Oct 2003 15:34:20 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HN4H1800.06L; Tue, 21 Oct 2003 15:34:20 -0400 Message-ID: <3F958A80.8000508@wi.mit.edu> Date: Tue, 21 Oct 2003 15:35:28 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Pascal Baspeyras CC: metakit@equi4.com Subject: Re: [Metakit] KitBinder References: <20031021191643.37298.qmail@web11508.mail.yahoo.com> In-Reply-To: <20031021191643.37298.qmail@web11508.mail.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Pascal Baspeyras wrote: >Brian Kelley wrote: > > > >>Under unix: >> >>cat executable database > new_executable >>chmod +x new_executable >> >>Under windows >> >>copy /b executable+database new_executable >> >>Metakit takes care of the rest. >> >> > >Thank you for the VERY quick reply, Brian. > >In my case, it's Windows (Visual Studio 6). > >Let's suppose I'm in debug mode, trying hard to fix a >bug. I make various modifications, then hit the debug >button... > > You can add a post-build step to the project settings under visual studio 6. I would add something like copy /b executable+/path/to/metakit newexecutable and set the debugging executable to newexecutable Both are available under project->settings From baspey@yahoo.com Tue Oct 21 14:41:10 2003 Received: from web11508.mail.yahoo.com (web11508.mail.yahoo.com [216.136.172.40]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LJfAX15287 for ; Tue, 21 Oct 2003 14:41:10 -0500 Message-ID: <20031021194109.43625.qmail@web11508.mail.yahoo.com> Received: from [213.36.116.158] by web11508.mail.yahoo.com via HTTP; Tue, 21 Oct 2003 12:41:09 PDT Date: Tue, 21 Oct 2003 12:41:09 -0700 (PDT) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] New metakit python documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > In the vein of user supported documentation, I have > extended the standard documents with some usage notes > and source code. Woa ! I'm looking at it right now. Must have taken a great deal of time and effort! >Bugs and comments are welcome. I might have found one: >"people[name:S, name:F]" >will create two columns "name" and " name". I think that Metakit wouldn't accept the space after the coma ... Still reading ;-) __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From baspey@yahoo.com Tue Oct 21 14:44:40 2003 Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LJidX15605 for ; Tue, 21 Oct 2003 14:44:39 -0500 Message-ID: <20031021194439.10487.qmail@web11501.mail.yahoo.com> Received: from [213.36.116.158] by web11501.mail.yahoo.com via HTTP; Tue, 21 Oct 2003 12:44:39 PDT Date: Tue, 21 Oct 2003 12:44:39 -0700 (PDT) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] KitBinder Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: >You can add a post-build step to the project settings >under visual >studio 6. I would add something like > >copy /b executable+/path/to/metakit newexecutable > >and set the debugging executable to newexecutable > >Both are available under project->settings I didn't know it was possible. I will try it right now. Thank you very much ! __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From bkelley@wi.mit.edu Tue Oct 21 14:52:00 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LJq0X16127 for ; Tue, 21 Oct 2003 14:52:00 -0500 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102115515615444 for ; Tue, 21 Oct 2003 15:51:56 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HN4HUK00.U44; Tue, 21 Oct 2003 15:51:56 -0400 Message-ID: <3F958E9F.5050105@wi.mit.edu> Date: Tue, 21 Oct 2003 15:53:03 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Pascal Baspeyras , Metakit mailing list Subject: Re: [Metakit] New metakit python documentation References: <20031021194109.43625.qmail@web11508.mail.yahoo.com> In-Reply-To: <20031021194109.43625.qmail@web11508.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: >I think that Metakit wouldn't accept the space after >the coma ... > It will except just about anything besides a comma and a colon. >>> import metakit >>> vw = metakit.storage() >>> vw = st.getas("table[ salkfjasf*#$(#&$83490343857432349504:S]") >>> vw.properties() {' salkfjasf*#$(#&$83490343857432349504': Property('S', ' salkfjasf*#$(#&$83490343857432349504')} >>> From baspey@yahoo.com Tue Oct 21 14:52:48 2003 Received: from web11506.mail.yahoo.com (web11506.mail.yahoo.com [216.136.172.38]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LJqmX16167 for ; Tue, 21 Oct 2003 14:52:48 -0500 Message-ID: <20031021195247.65403.qmail@web11506.mail.yahoo.com> Received: from [213.36.116.158] by web11506.mail.yahoo.com via HTTP; Tue, 21 Oct 2003 12:52:47 PDT Date: Tue, 21 Oct 2003 12:52:47 -0700 (PDT) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] New metakit python documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: >Bugs and comments are welcome. The "shenanigans" link points toward your HD. The same for the "note" link just before it. __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From baspey@yahoo.com Tue Oct 21 15:10:56 2003 Received: from web11505.mail.yahoo.com (web11505.mail.yahoo.com [216.136.172.37]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LKAuX18748 for ; Tue, 21 Oct 2003 15:10:56 -0500 Message-ID: <20031021201055.15824.qmail@web11505.mail.yahoo.com> Received: from [213.36.116.158] by web11505.mail.yahoo.com via HTTP; Tue, 21 Oct 2003 13:10:55 PDT Date: Tue, 21 Oct 2003 13:10:55 -0700 (PDT) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] New metakit python documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: >>I think that Metakit wouldn't accept the space after >>the coma ... >> >It will except just about anything besides a comma >and a colon. Sorry. From my memories, I had had troubles when I just started using Metakit, because I wrote View = Storage.GetAs("View[Prop1:I, Prop2:I, Prop3:S]") and it didn't work. But that's old memories. I'm certainly mixing things up. __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From baspey@yahoo.com Tue Oct 21 15:53:07 2003 Received: from web11506.mail.yahoo.com (web11506.mail.yahoo.com [216.136.172.38]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9LKr7X21506 for ; Tue, 21 Oct 2003 15:53:07 -0500 Message-ID: <20031021205306.84077.qmail@web11506.mail.yahoo.com> Received: from [213.36.114.190] by web11506.mail.yahoo.com via HTTP; Tue, 21 Oct 2003 13:53:06 PDT Date: Tue, 21 Oct 2003 13:53:06 -0700 (PDT) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] New metakit python documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: >"name" and " name". Alright ... I did not read carefully enough ... Sorry for the noise. __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From jcw@equi4.com Thu Oct 23 03:43:18 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9N8hIX09488; Thu, 23 Oct 2003 03:43:18 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id 6C5FA83AEF; Thu, 23 Oct 2003 10:43:11 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E938183AE7; Thu, 23 Oct 2003 10:43:10 +0200 (CEST) Date: Thu, 23 Oct 2003 10:43:11 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list , Starkit list Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.5 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] Metakit@Web Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: FYI, I just added Stefan Vogel's new "Metakit@Web" script here: http://www.equi4.com/mklinks.html It's a new Tcl script which runs under the Tclhttpd web server to create a nice manager to list, browse, create, and edit MK datafiles. Very early, as Stefan tells me, but then that's what open source is about. It takes three files to try this out: mkweb.tcl, tclhttpd.kit, and whichever tclkit matches your platform. As this evolves, it could become a very nice way to manage MK-based data via the web, but also a simple way to try out things with MK: i.e. web forms as a better command-line... -jcw From bkelley@wi.mit.edu Fri Oct 24 17:23:45 2003 Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9OMNjX09802 for ; Fri, 24 Oct 2003 17:23:45 -0500 Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.147.123.161]) by comcast.net (sccrmhc13) with SMTP id <2003102422233601600qt96fe>; Fri, 24 Oct 2003 22:23:37 +0000 Message-ID: <3F99A7B1.9040801@wi.mit.edu> Date: Fri, 24 Oct 2003 18:29:05 -0400 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: j n CC: metakit@equi4.com References: <20031024221742.67529.qmail@web11507.mail.yahoo.com> In-Reply-To: <20031024221742.67529.qmail@web11507.mail.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Re: any_srv for metakit (similar purpose to metaserve) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: j n wrote: >This means the client for any_srv is 100% stupid. >And the server except for importing metakit, is also >100% stupid. The advantage is that if metakit changes, >the client/server code doesn't need to know about it. > > Cool. It's nice to have more than one implementation! >It also support iterators. Any iterable object >references are kept as that. So, the client can >iterate through the rows as it needs to. The entire >row result from a query is not sent back. > >Start the server with: >python any_srv.py > >Here is example client code: > mclient=any_server.Client() > #Create a database: > db = mclient.storage("datafile.mk",1) > > print 'Create a view' > vw = db.getas("people[first:S,last:S,bowling:I]") > > print 'Add a couple of rows' > vw.append(first='Fred',last='Flint',bowl=250) > vw.append(first='Barney',last='Rubble',bowl=150) > > print 'Commit the changes to file:' > db.commit() > > print 'Show a list of all people:' > for r in vw: print r.first(), r.last(),r.bowl() > > If you make these r.first, r.last, r.bowl it will be an easier transition from straight metakit to your server. Otherwise, good work. I'll be interested in investigating how you are doing your protocol passing. > > mclient.any_server_clean() > >2 things to think about: >Need to explicity kill the object references with >mclient.any_server_clean() > >Currently, attributes as well as methods end in (). > >The code is rough and needs to have more >features/documentation added but it seems to work >well. > >Any comments would be appreciated. > >john > > >__________________________________ >Do you Yahoo!? >The New Yahoo! Shopping - with improved product search >http://shopping.yahoo.com > > From pyguy2@yahoo.com Fri Oct 24 19:38:38 2003 Received: from web11702.mail.yahoo.com (web11702.mail.yahoo.com [216.136.172.68]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9P0ccX24294 for ; Fri, 24 Oct 2003 19:38:38 -0500 Message-ID: <20031025003837.6092.qmail@web11702.mail.yahoo.com> Received: from [24.26.151.86] by web11702.mail.yahoo.com via HTTP; Fri, 24 Oct 2003 17:38:37 PDT Date: Fri, 24 Oct 2003 17:38:37 -0700 (PDT) From: j n To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] metakit server any_srv (like metaserve) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hello there, It looks like everyone is stratching an itch nowadays. I've written 0.01 version of any_srv. It is a rpc client/server that takes the approach if knowing as little as possible to serve metakit (by its approach it could serve even something like the string library). It is at: http://savannah.nongnu.org/download/pytunnel/any_srv.py http://savannah.nongnu.org/download/pytunnel/meta_client.py It is an asynchat server (which means it is single threaded which is what metakit needs). My goal is to also be able stay as close as possible to the metakit library syntax. Rather than explicitly map everything with code, the client automatically forwards everything (via getattr). Anything that is not pickleable is returned as an object reference. This means the client for any_srv is 100% stupid. And the server except for importing metakit, is also 100% stupid. The advantage is that if metakit changes, the client/server code doesn't need to know about it. It also support iterators. Any iterable object references are kept as that. So, the client can iterate through the rows as it needs to. The entire row result from a query is not sent back. Start the server with: python any_srv.py Here is example client code: mclient=any_server.Client() #Create a database: db = mclient.storage("datafile.mk",1) print 'Create a view' vw = db.getas("people[first:S,last:S,bowling:I]") print 'Add a couple of rows' vw.append(first='Fred',last='Flint',bowl=250) vw.append(first='Barney',last='Rubble',bowl=150) print 'Commit the changes to file:' db.commit() print 'Show a list of all people:' for r in vw: print r.first(), r.last(),r.bowl() mclient.any_server_clean() 2 things to think about: Need to explicity kill the object references with mclient.any_server_clean() Currently, attributes as well as methods end in (). The code is rough and needs to have more features/documentation added but it seems to work well. Any comments would be appreciated. john __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From jcw@equi4.com Sat Oct 25 17:41:46 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9PMfjX25492 for ; Sat, 25 Oct 2003 17:41:46 -0500 Received: by madder.xs4all.nl (Postfix, from userid 101) id B383883AEF; Sun, 26 Oct 2003 00:41:37 +0200 (CEST) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 4610083AE7 for ; Sun, 26 Oct 2003 00:41:37 +0200 (CEST) Date: Sun, 26 Oct 2003 00:41:38 +0200 Mime-Version: 1.0 (Apple Message framework v552) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit Message-Id: <65434E79-073C-11D8-A572-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.5 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] make Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Can anyone name a reason why the MK core makefile should continue to include autoconf & libtool logic for building the Python extension (shared or static)? With Distutils being the standard way to build things in Pythonia nowadays, and since MK now has a python/setup.py script, I'm tempted to throw Mk4py out of the makefile. Finding the proper /usr/include/PythonX.Y/ directory has always been tricky anyway. I've not checked how setup.py actually builds and whether it includes things like -fPIC when compiling for linkage into a shared library. Nor whether Distutils is mature enough now to work with MSVC, mingw, and more esoteric platforms / compilers / C++ runtimes. Not sure about Mk4py cross-compilation either, nor whether it matters. I do know that "python setup.py build" is a lot simpler than configure/make when the proper ./configure --please-make=my/build:work/right incantations are not known or understood well enough... -jcw From bkelley@wi.mit.edu Sun Oct 26 18:14:08 2003 Received: from sccrmhc11.comcast.net (sccrmhc11.comcast.net [204.127.202.55]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9R0E8X05875; Sun, 26 Oct 2003 18:14:08 -0600 Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.147.123.161]) by comcast.net (sccrmhc11) with SMTP id <2003102700140101100c6ufje>; Mon, 27 Oct 2003 00:14:02 +0000 Message-ID: <3F9C6499.3090001@wi.mit.edu> Date: Sun, 26 Oct 2003 19:19:37 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Metakit mailing list Subject: Re: [Metakit] make Mk4py References: <65434E79-073C-11D8-A572-000A9588127E@equi4.com> In-Reply-To: <65434E79-073C-11D8-A572-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Can anyone name a reason why the MK core makefile should continue to > include autoconf & libtool logic for building the Python extension > (shared or static)? > > With Distutils being the standard way to build things in Pythonia > nowadays, and since MK now has a python/setup.py script, I'm tempted > to throw Mk4py out of the makefile. Finding the proper > /usr/include/PythonX.Y/ directory has always been tricky anyway. > > I've not checked how setup.py actually builds and whether it includes > things like -fPIC when compiling for linkage into a shared library. > Nor whether Distutils is mature enough now to work with MSVC, mingw, > and more esoteric platforms / compilers / C++ runtimes. You can check what machines you are running on and change compiler settings in the setup.py I deploy my chemoinformatics system using setup.py and extension creation that targets g++ on unix/linux and Mac and msvc or cygwin g++ on windows. Additionally, I believe there are targets for codewarrior and I don't think that mingw would be that difficult as well. I agree that mk4py should not be in the Makefile, since setup.py should be able to get the appopriate headers and build for the current version of python (2.2, 2.3 etc) altering the Makefile for this is a bit of a pain. > Not sure about Mk4py cross-compilation either, nor whether it matters. > > I do know that "python setup.py build" is a lot simpler than > configure/make when the proper ./configure > --please-make=my/build:work/right incantations are not known or > understood well enough... > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From erikh2000@yahoo.com Mon Oct 27 14:43:04 2003 Received: from smtp101.mail.sc5.yahoo.com (smtp101.mail.sc5.yahoo.com [216.136.174.139]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9RKh3X30269 for ; Mon, 27 Oct 2003 14:43:03 -0600 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Oct 2003 20:43:02 -0000 Message-Id: <5.2.0.9.0.20031027122750.00b10730@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 27 Oct 2003 12:43:16 -0800 To: Metakit mailing list From: Erik Hermansen Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Cause of data corruption bug found. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: A while back I posted about this horrible, intermittent bug that corrupted all the data in a storage file. After a lot of pain and suffering, one of our testers found a way to reproduce the bug and another programmer on the team fixed it. The problem was not with Metakit. We had a function that verified read-write access to a storage file before constructing a c4_Storage to open the file. Our verification function called fopen(filename,"w+") and closed without writing to make sure we had access. However, the Windows version of fopen() will write to the file in certain situations. If the last character in the file is 26 (ctrl-z), it will be stripped, causing other problems when Metakit is opened. The bug showed up intermittently, because the last byte was only occasionally set to 26. The programmer fixed this problem by changing the fopen call to fopen(filename "wb+"). So things are good! -Erik From jcw@equi4.com Mon Oct 27 15:00:55 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9RL0tX31415 for ; Mon, 27 Oct 2003 15:00:55 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 9CF3783AF3; Mon, 27 Oct 2003 22:00:46 +0100 (CET) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E95BD83AEF for ; Mon, 27 Oct 2003 22:00:45 +0100 (CET) Date: Mon, 27 Oct 2003 22:00:46 +0100 Subject: Re: [Metakit] Cause of data corruption bug found. Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <5.2.0.9.0.20031027122750.00b10730@pop.mail.yahoo.com> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,SPAM_PHRASE_02_03, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hello Erik, > A while back I posted about this horrible, intermittent bug that > corrupted all the data in a storage file. After a lot of pain and > suffering, one of our testers found a way to reproduce the bug and > another programmer on the team fixed it. Thank you very much for following up on this. Problems of this nature sometimes end up on the mailing list with the solution never reported - making it easy to jump to the conclusion that MK has some unresolved hard and intermittent bug... (MK may well have bugs hiding, of course - but thanks for explaining that this one wasn't) > fopen(filename "wb+"). Weeks - even months - of trouble, for a single "b" character. Ouch!!! Anyway... I'm really glad you found it at last :) -jcw From erikh2000@yahoo.com Mon Oct 27 15:19:52 2003 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9RLJqX32578 for ; Mon, 27 Oct 2003 15:19:52 -0600 Received: from 12-228-176-153.client.attbi.com (HELO HALPH.yahoo.com) (erikh2000@12.228.176.153 with login) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Oct 2003 21:19:51 -0000 Message-Id: <5.2.0.9.0.20031027131622.00afb008@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 27 Oct 2003 13:20:05 -0800 To: Jean-Claude Wippler , Metakit mailing list From: Erik Hermansen Subject: Re: [Metakit] Cause of data corruption bug found. In-Reply-To: References: <5.2.0.9.0.20031027122750.00b10730@pop.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: At 10:00 PM 10/27/2003 +0100, Jean-Claude Wippler wrote: >Thank you very much for following up on this. Problems of this nature >sometimes end up on the mailing list with the solution never reported - >making it easy to jump to the conclusion that MK has some unresolved hard >and intermittent bug... Sure. It is good to know the truth! >Anyway... I'm really glad you found it at last :) Thanks for your advice in response to my problem. It was useful in narrowing the possible causes down, and I learned some things about how Metakit works. -Erik From epo001@hotmail.com Tue Oct 28 03:33:16 2003 Received: from hotmail.com (law14-f79.law14.hotmail.com [64.4.21.79]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9S9XGX13254 for ; Tue, 28 Oct 2003 03:33:16 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Tue, 28 Oct 2003 01:33:11 -0800 Received: from 62.53.42.70 by lw14fd.law14.hotmail.msn.com with HTTP; Tue, 28 Oct 2003 09:33:10 GMT X-Originating-IP: [62.53.42.70] X-Originating-Email: [epo001@hotmail.com] From: "Ed ." To: metakit@equi4.com Date: Tue, 28 Oct 2003 09:33:10 +0000 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 28 Oct 2003 09:33:11.0042 (UTC) FILETIME=[80C0CE20:01C39D36] Subject: [Metakit] CVS and release: can't make python!!! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is RH9 with all patches up to date. Ignoring the fact that there is no install documentation worth reading this is a really poor first impression. It manifests itself on both the downloaded tar source file and on a CVS tree loaded yesterday. In a clean tree, I cd builds ../unix/configure --enable-python # N.B. no path flag so I use the default make # which seems to work make python # which produces: /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../python/scxx -I ../unix/../python/PyProperty.cpp rm -f .libs/PyProperty.lo g++ -c -g -O2 -I../unix/../include -I../unix/../python/scxx -I ../unix/../python/PyProperty.cpp -fPIC -DPIC g++: no input files make: *** [PyProperty.o] Error 1 Note the -I in front of the source file name I also tried all the above but with ../unix/configure --enable-python=/usr and got exactly the same result. Am I the only person to try using python, am I missing something obvious? Ed _________________________________________________________________ Tired of 56k? Get a FREE BT Broadband connection http://www.msn.co.uk/specials/btbroadband From epo001@hotmail.com Tue Oct 28 03:54:32 2003 Received: from hotmail.com (law14-f52.law14.hotmail.com [64.4.21.52]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9S9sWX14301 for ; Tue, 28 Oct 2003 03:54:32 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Tue, 28 Oct 2003 01:54:26 -0800 Received: from 62.53.42.15 by lw14fd.law14.hotmail.msn.com with HTTP; Tue, 28 Oct 2003 09:54:26 GMT X-Originating-IP: [62.53.42.15] X-Originating-Email: [epo001@hotmail.com] From: "Ed ." To: metakit@equi4.com Date: Tue, 28 Oct 2003 09:54:26 +0000 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 28 Oct 2003 09:54:26.0694 (UTC) FILETIME=[7919DA60:01C39D39] Subject: [Metakit] CVS and release: can't make python!!! -- yet more Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: After mailing this I tried entering the last compile line by hand but changing -I to -I/usr/include/python2.2 and it worked fine. I then did a make distclean and a ../unix/configure --enable-python=/usr//include/python2.2,/usr/lib/python2.2/site-packages and the first compile of make python still failed. I think this is all an interested user would attempt, if the build process is undocumented and the obvious build steps fail even when you specify everything explicitly you really have to look for alternatives. Ed _________________________________________________________________ Get Hotmail on your mobile phone http://www.msn.co.uk/msnmobile From bkelley@wi.mit.edu Tue Oct 28 08:07:35 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9SE7ZX32267 for ; Tue, 28 Oct 2003 08:07:35 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102809073113665 for ; Tue, 28 Oct 2003 09:07:31 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HNH0KJ00.SJF for ; Tue, 28 Oct 2003 09:07:31 -0500 Message-ID: <3F9E7860.3060506@wi.mit.edu> Date: Tue, 28 Oct 2003 09:08:32 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] CVS and release: can't make python!!! -- yet more References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ed . wrote: > After mailing this I tried entering the last compile line by hand but > changing -I to -I/usr/include/python2.2 and it worked fine. > > I then did a make distclean and a ../unix/configure > --enable-python=/usr//include/python2.2,/usr/lib/python2.2/site-packages > and the first compile of make python still failed. This doesn't look right here: --enable-python=/usr//include/python2.2,/usr/lib/python2.2/site-packages I think you want --enable-python --with-python=/usr Just like the ./configure --help states --with-python=PATH build the Mk4py extension for Python with headers in `PATH/include/python2.2'; install the Mk4py modules in `PATH/lib/python2.2/site-packages'. If PATH is of the form `HEADER,LIB', search for header files in HEADER, and install Mk4py in LIB. If you omit the `=PATH' part, use the standard includedir and libdir. http://www.equi4.com/pub/mk/mk-2.4.9-linux/ you will have to get the metakit.py file as well http://www.equi4.com/pub/mk/ > I think this is all an interested user would attempt, if the build > process is undocumented and the obvious build steps fail even when you > specify everything explicitly you really have to look for alternatives. Well, configure is supposed to take care of most of the self documentation, but it is a hairy mess. Most of the time the standard dance ./configure; make install works just fine but when it doesn't, ouch. Even worse would be trying to target the python 2.3 environment since they change some important header types. Anyway, metakit is migrating to the "python setup.py install" method which should be a bit easier and you could always download the binaries. I think they are RH9 compatible. On another note, documentation for projects of this type usually come from the user community. We are being given a tool to use for no charge and need to support it properly. I've written some new python documentation, the original while nice had some gotchas for new users but I need reviewers. I've only had one so far, not a great response for something that has been a common complaint. Would someone like to write up some better installation documentation? I'll certainly act as reviewer. > Ed From bkelley@wi.mit.edu Tue Oct 28 08:42:26 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9SEgQX02718 for ; Tue, 28 Oct 2003 08:42:26 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102809422214120 for ; Tue, 28 Oct 2003 09:42:22 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HNH26M00.HKZ; Tue, 28 Oct 2003 09:42:22 -0500 Message-ID: <3F9E808A.6080608@wi.mit.edu> Date: Tue, 28 Oct 2003 09:43:22 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Brian Kelley , metakit@equi4.com Subject: Re: [Metakit] CVS and release: can't make python!!! -- yet more References: <3F9E7860.3060506@wi.mit.edu> In-Reply-To: <3F9E7860.3060506@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: My post got a little out of order > Anyway, metakit is migrating to the "python setup.py install" method > which should be a bit easier and you could always download the > binaries. I think they are RH9 compatible. http://www.equi4.com/pub/mk/mk-2.4.9-linux/ you will have to get the metakit.py file as well http://www.equi4.com/pub/mk/ From epo001@hotmail.com Tue Oct 28 13:06:45 2003 Received: from hotmail.com (law14-f76.law14.hotmail.com [64.4.21.76]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9SJ6iX24947 for ; Tue, 28 Oct 2003 13:06:45 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Tue, 28 Oct 2003 11:06:39 -0800 Received: from 62.53.42.112 by lw14fd.law14.hotmail.msn.com with HTTP; Tue, 28 Oct 2003 19:06:39 GMT X-Originating-IP: [62.53.42.112] X-Originating-Email: [epo001@hotmail.com] From: "Ed ." To: metakit@equi4.com Subject: [Metakit] CVS and release: can't make python!!! -- yet more - Apologies Date: Tue, 28 Oct 2003 19:06:39 +0000 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 28 Oct 2003 19:06:39.0515 (UTC) FILETIME=[9DCD1EB0:01C39D86] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Firstly apologies for the lack of threading, haven't got a news reader configured yet. The problem was operator error, I hadn't read the output of configure --help properly. I can confirm that .../unix/configure --with-python=/usr make does indeed do what it is supposed to. Heaven only knows where I got the --enable-python flag from. I am grateful to Brian Kelley for his patience with my illiteracy. Cue Simpsons opening credits: I will double check everything before posting, I will ... Sorry for the distraction, Ed BTW If I do make test, it ends ... l06 - Bit field manipulations l07 - Huge description Done. diff ../unix/../tests/ok tests Common subdirectories: ../unix/../tests/ok/CVS and tests/CVS Only in tests: c22.txt make: *** [test] Error 1 _________________________________________________________________ Use MSN Messenger to send music and pics to your friends http://www.msn.co.uk/messenger From bkelley@wi.mit.edu Tue Oct 28 15:05:39 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9SL5dX06026 for ; Tue, 28 Oct 2003 15:05:39 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102816053519440 for ; Tue, 28 Oct 2003 16:05:35 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HNHJXB00.KNM; Tue, 28 Oct 2003 16:05:35 -0500 Message-ID: <3F9EDA5A.7030108@wi.mit.edu> Date: Tue, 28 Oct 2003 16:06:34 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Ed ." , metakit@equi4.com Subject: Re: [Metakit] CVS and release: can't make python!!! -- yet more - Apologies References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ed . wrote: > I am grateful to Brian Kelley for his patience with my illiteracy. Cue > Simpsons opening credits: I will double check everything before > posting, I will ... If I had a nickle for everytime I stuck my foot in my mouth, I'd be a rich rich man. Perhaps, the only good thing about being wrong in a public form is that you're a lot less likely to forget in the future. Ah the power of shame, for a good example just see some of my posts to comp.lang.python :) Brian From jcw@equi4.com Tue Oct 28 16:24:42 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9SMOfX13204 for ; Tue, 28 Oct 2003 16:24:41 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 67B1883AF3; Tue, 28 Oct 2003 23:24:33 +0100 (CET) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id DE07683AEF for ; Tue, 28 Oct 2003 23:24:32 +0100 (CET) Date: Tue, 28 Oct 2003 23:24:35 +0100 Subject: Re: [Metakit] CVS and release: can't make python!!! -- yet more - Apologies Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: <8269DEA8-0995-11D8-A5D4-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-1.6 required=5.0 tests=AWL,IN_REP_TO,PLING_PLING,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ed wrote: > .../unix/configure --with-python=/usr > make > > does indeed do what it is supposed to. Heaven only knows where I got > the --enable-python flag from. Glad it has been resolved. As to --enable-python... yes, it was mentioned in the unix/metakit.spec file, and I think things have been changed back and forth over the years. > BTW If I do make test, it ends ... > > l06 - Bit field manipulations > l07 - Huge description > Done. > diff ../unix/../tests/ok tests > Common subdirectories: ../unix/../tests/ok/CVS and tests/CVS > Only in tests: c22.txt > make: *** [test] Error 1 Ah - thank you! I see that I forgot to check this new test output file into CVS. So while test c22 ran, there was no regression data to compara against in tests/ok/. Both issues fixed in CVS. -jcw From bkelley@wi.mit.edu Tue Oct 28 16:34:41 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9SMYfX13996 for ; Tue, 28 Oct 2003 16:34:41 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003102817343720521 for ; Tue, 28 Oct 2003 17:34:37 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HNHO1P00.MLI for ; Tue, 28 Oct 2003 17:34:37 -0500 Message-ID: <3F9EEF39.5090909@wi.mit.edu> Date: Tue, 28 Oct 2003 17:35:37 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Strange hash error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I was using a hash view and ran into a problem where I could delete a row using the key: del hashview[hashview.find(id=x)] assert hashview.find(id=x) == -1 worked correctly but: hashview.select(id=x) return a row! Also for row in hashview: assert row.id=x would fail. I rebuilt the hashtables by dropping them and reassigning the hash view. Has anybody else seen this problem? I might have accidentally futzed with the underlying view instead of the hashview... Brian From ian.fairclough@intelligent-ms.com Thu Oct 30 13:14:12 2003 Received: from scratchy.base.ims ([194.74.0.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9UJEBX26151 for ; Thu, 30 Oct 2003 13:14:11 -0600 Received: from grpware.intelligent-ms.com (mail.intelligent-ms.com [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id h9UJ2Zl20833 for ; Thu, 30 Oct 2003 19:02:35 GMT Received: from imsgsx6tvznbng ([192.168.200.183]) by grpware.intelligent-ms.com (8.11.2/8.11.2) with SMTP id h9UIt9u08612 for ; Thu, 30 Oct 2003 18:55:09 GMT Message-ID: <00d101c39f1a$54683f30$b7c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Date: Thu, 30 Oct 2003 19:16:29 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Lock-up in view destructor Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Has anybody encountered a problem with destructing a view ? I have a situation where the destructor never terminates, and the process consumes all of the available processor resource. I am running under Red Hat Linux 7.3. Metakit was built with gcc version 2.96 20000731. Any suggestions for possible causes / solutions would be most welcome ! Ian Fairclough. From bkelley@wi.mit.edu Thu Oct 30 14:24:44 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9UKOiX31054 for ; Thu, 30 Oct 2003 14:24:44 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003103015244012039 for ; Thu, 30 Oct 2003 15:24:40 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HNL7D400.Q0D; Thu, 30 Oct 2003 15:24:40 -0500 Message-ID: <3FA173C2.8030809@wi.mit.edu> Date: Thu, 30 Oct 2003 15:25:38 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ian Fairclough CC: metakit@equi4.com Subject: Re: [Metakit] Lock-up in view destructor References: <00d101c39f1a$54683f30$b7c8a8c0@imsgsx6tvznbng> In-Reply-To: <00d101c39f1a$54683f30$b7c8a8c0@imsgsx6tvznbng> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ian Fairclough wrote: >Has anybody encountered a problem with destructing a view ? > > What do you mean by destroying a view? Are you dropping it like storage.GetAs("view") to remove it? Or are you closing down an application? I've seen something like this when you have a lot of data that hasn't been committed and you have autocommit turned on. >I have a situation where the destructor never terminates, and the process >consumes all of the available processor resource. > >I am running under Red Hat Linux 7.3. Metakit was built with gcc version >2.96 20000731. > >Any suggestions for possible causes / solutions would be most welcome ! > >Ian Fairclough. > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > From ian.fairclough@intelligent-ms.com Thu Oct 30 15:16:52 2003 Received: from scratchy.base.ims ([194.74.0.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9ULGqX03331 for ; Thu, 30 Oct 2003 15:16:52 -0600 Received: from grpware.intelligent-ms.com (grpware.base.ims [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id h9UL5Gl21156; Thu, 30 Oct 2003 21:05:16 GMT Received: from imsgsx6tvznbng ([192.168.200.183]) by grpware.intelligent-ms.com (8.11.2/8.11.2) with SMTP id h9UKvou09646; Thu, 30 Oct 2003 20:57:50 GMT Message-ID: <00d801c39f2b$77dc4400$b7c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: "Brian Kelley" Cc: References: <00d101c39f1a$54683f30$b7c8a8c0@imsgsx6tvznbng> <3FA173C2.8030809@wi.mit.edu> Subject: Re: [Metakit] Lock-up in view destructor Date: Thu, 30 Oct 2003 21:19:13 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > Ian Fairclough wrote: > > >Has anybody encountered a problem with destructing a view ? > > > > > What do you mean by destroying a view? Are you dropping it like > > storage.GetAs("view") Here's an example of my code : in someClass constructor : view = storage.GetAs(structure); void someClass::dummyFunction { c4_View selectedDataView = this->view.duplicate(); } My problem occurs when the "dummyFunction" is called and returns i.e. when the destructor for "selectedDataView" is called. I hope this is clear. Otherwise I will provide a full c++ class to illustrate my issue. > > to remove it? Or are you closing down an application? I've seen > something like this when you have a lot of data that hasn't been > committed and you have autocommit turned on. Is auto commit turned on by default ? > > >I have a situation where the destructor never terminates, and the process > >consumes all of the available processor resource. > > > >I am running under Red Hat Linux 7.3. Metakit was built with gcc version > >2.96 20000731. > > > >Any suggestions for possible causes / solutions would be most welcome ! > > > >Ian Fairclough. > > > > > >_______________________________________________ > >metakit mailing list - metakit@equi4.com > >http://www.equi4.com/mailman/listinfo/metakit > > > > > > From jcw@equi4.com Thu Oct 30 15:29:37 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9ULTaX04988 for ; Thu, 30 Oct 2003 15:29:36 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 82B2983B2D; Thu, 30 Oct 2003 22:29:28 +0100 (CET) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 0E71483AEF for ; Thu, 30 Oct 2003 22:29:27 +0100 (CET) Date: Thu, 30 Oct 2003 22:29:29 +0100 Subject: Re: [Metakit] Lock-up in view destructor Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <00d801c39f2b$77dc4400$b7c8a8c0@imsgsx6tvznbng> Message-Id: <24F8E493-0B20-11D8-AB3E-000A9588127E@equi4.com> X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-3.1 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ian Fairclough wrote: > in someClass constructor : > view = storage.GetAs(structure); > > void someClass::dummyFunction > { > c4_View selectedDataView = this->view.duplicate(); > } > > My problem occurs when the "dummyFunction" is called and returns i.e. > when > the destructor for "selectedDataView" is called. Weird. Does "view" contain subviews? > Is auto commit turned on by default ? Not in C++ and Python. >>> I am running under Red Hat Linux 7.3. Metakit was built with gcc >>> version >>> 2.96 20000731. What is the version of MK - see #define d4_MetaKitLibraryVersion in "mk4.h". You can then compare this with the current change log: http://www.equi4.com/metakit/viewcvs.cgi/metakit/CHANGES?rev=1.172 -jcw From gary.h.merrill@gsk.com Fri Oct 31 09:51:53 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9VFprX13968 for ; Fri, 31 Oct 2003 09:51:53 -0600 Received: by usrtpnp1.glaxowellcome.com; id KAA24633; Fri, 31 Oct 2003 10:52:42 -0500 (EST) From: Received: from unknown(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma014555; Fri, 31 Oct 03 10:44:24 -0500 Received: by ussunt3.glaxo.com id KAA03003; Fri, 31 Oct 2003 10:43:32 -0500 (EST) Received: from 166.71.134.118 by us9n52.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Fri, 31 Oct 2003 10:43:27 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id h9VFhQk24910 for ; Fri, 31 Oct 2003 10:43:26 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Fri, 31 Oct 2003 10:42:22 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 10/31/2003 10:43:30 AM MIME-Version: 1.0 X-WSS-ID: 13BC5C951261968-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Viewing a MK database Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a combined Metakit/wxPython question, but given my situation I think it resides a bit more on the Metakit side. Let's say I have a large Mk database that I want to display in a grid-like format (wxGrid, for example). What is the best way to approach this so that the display is very fast? Okay, we all know that kviewer and Brian's KitViewer do this, and that the proper approach is to use an underlying wxPyGridTableBase. But my problem is a bit more complicated than just displaying what is actually in the rows of the view. In this db, all of the porperties are numeric (ints, actually). But these *represent* other things and what I want to display is the string representations corresponding to (at least some of) the ints. This requires accessing other databases to retrieve or compute the appropriate string representation. It is then this resulting "string-ized" row that needs to be displayed in the grid. Currently (first attempt and being largely ignorant of the almost completely undocumented wxPyGridTableBase), I just loop through all the rows in the view, construct the appropiate corresponding sequence for each row (containing the strings instead of the ints), and add this to the 'data' sequence for the wxPyGridTableBase. It works, but it's not real fast. What is the best way to accomplish this? Should I, in Metakit, construct a new view that has the strings in place of their numeric representations and then somehow use this? But then don't I still need to loop through it to create the data sequence for the wxPyGridTableBase? Is there a way to quickly turn a view into a sequence of Python sequences: [ [prop1, prop2, ...], ...] which could then just be assigned as the wxPyGridTableBase data object? Should I convert to strings *before* I create the wxPyGridTableBase? Or should I use the numeric representations for the wxPyGridTableBase and handle the conversions at "display time" through something like GetValue()? Is all of this already done in some way within Brian's KitViewer code? I'm sure people must do this all the time. It's just not obvious to me what the best approach is. Thanks for any hints. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From jcw@equi4.com Fri Oct 31 10:31:51 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9VGVoX16324 for ; Fri, 31 Oct 2003 10:31:50 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 526CB83B2D; Fri, 31 Oct 2003 17:31:43 +0100 (CET) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id D080183AEF for ; Fri, 31 Oct 2003 17:31:42 +0100 (CET) Date: Fri, 31 Oct 2003 17:31:42 +0100 Subject: Re: [Metakit] Viewing a MK database Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.5 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,SPAM_PHRASE_01_02, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: wrote: > Let's say I have a large Mk database that I want to display in a > grid-like > format (wxGrid, for example). What is the best way to approach this so > that the display is very fast? [...] > But these *represent* other things and what I want to display > is the string representations corresponding to (at least some of) the > ints. > This requires accessing other databases to retrieve or compute the > appropriate string representation. It is then this resulting > "string-ized" > row that needs to be displayed in the grid. Quick (not well thought-through) response: This is where mk.wrap() can probably help. Define a view which wraps the MK view, and produces values in the way you need them. Then, access to items will go through your Python wrapper *on-demand*. The whole trick of KitView.exe (and I presume Brian's KitViewer) is "delayed rendering". Scrolling across an infinite number of rows can be instant. The issue is not doing more work than needed - and if you think about it, KitView does exactly the same as what you're after: take data out of MK, and present it in transformed way on the screen (namely visually). Back in the times when KitView.exe was built, I had only Borland C++ Builder's "datagrid" which was up to this task. Many simpler GUI approaches ask you to "fill" a matrix or listbox before using them. Nowadays, there are several widgets in Tk (TkTable as well as pure tcl ones such as Hugelist) which can play this on-demand delayed-rendering trick. I don't know wxPython (or wxWindows) unfortunately - but it can no doubt also do this. With Mk4py's wrap() you can create MK "views" which are virtual *and* which go through arbitrary Python code on each item access request. It's sort of the equivalent of MK's c4_CustomViewer class in C++, which is extremely powerful (many view operators are built on it). -jcw From jcw@equi4.com Fri Oct 31 10:35:17 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9VGZHX16577 for ; Fri, 31 Oct 2003 10:35:17 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id E845E83B2D; Fri, 31 Oct 2003 17:35:09 +0100 (CET) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 71E6E83AEF; Fri, 31 Oct 2003 17:35:09 +0100 (CET) Date: Fri, 31 Oct 2003 17:35:09 +0100 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: Metakit mailing list To: Frans Houweling From: Jean-Claude Wippler In-Reply-To: <3FA28394.8070301@tin.it> Message-Id: <317281E0-0BC0-11D8-AB3E-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.7 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] Re: Mk4tcl problem Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Frans Houweling wrote: > Hi! I am trying Mk4tcl on windows XP and tcl 8.4 > > package require Mk4tcl > mk::file open db "test.db" > mk::view layout db.table {token frequency:I} > mk::row append db.table token blurp frequency 1 > mk::file commit db > set res [mk::select -count 1 db.table token blurp] > # -----> ..breaks tclsh > #set res [mk::select db.table token blurp] > # ..does not Ouch, try: set res [mk::select db.table -count 1 token blurp] If that fixes it, then the bug in Mk4tcl is that it does not properly catch this incorrect call. I'd appreciate it if you have time to file a bug report so this does not get lost: http://www.equi4.com/bugs -jcw From bkelley@wi.mit.edu Fri Oct 31 11:34:28 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id h9VHYSX20398 for ; Fri, 31 Oct 2003 11:34:28 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003103112342422024 for ; Fri, 31 Oct 2003 12:34:24 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HNMU5C00.KBP for ; Fri, 31 Oct 2003 12:34:24 -0500 Message-ID: <3FA29D59.9020109@wi.mit.edu> Date: Fri, 31 Oct 2003 12:35:21 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Viewing a MK database References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: gary.h.merrill@gsk.com wrote: > Is all of this already done in some way within Brian's KitViewer > code? I'm > sure people must do this all the time. It's just not obvious to me what > the best approach is. Thanks for any hints. > > This is easier than it seems. Since you are using KitViewer I'll explain a possible solution that I use: def lookup(storage, attribute, data): view = storage.view(SPECIALVIEWS[attribute]) # somehow convert view and data into what you want to return class KitTable: def GetValue(self, row, col): attribute = self._headers[col] # get the attribute name data = self.table[row, col] # get the data if attribute in SPECIAL_COLUMN_NAMES: data = lookup(self.table.storage, attribute, data) return str(data) What this does is, when a piece of data is getting retrieved, see if it is a special attribute class. I tend to use attribute XXXX_id to mean look up this id in table XXXX so my lookup function is something like def lookup(storage, attribute, id): view = storage.view(attribute.split("_")[0]) index = view.find(id=id) if index == -1: return "" return view[index].name Of course, it is a little more complicated than that since I return different values depending on what the view being retrieved is and for speed issues you should make sure that the view is a hash view or ordered on the 'id' property. Another complication is how to you set these values inside the grid. I tend to generate a pull down menu from the currently acceptable values in the XXXX table. I'm writing a next generation KitViewer as we speak with internal support for these operations, but it won't be ready for a while. > -------------------------------------- > Gary H. Merrill > Director and Principal Scientist, New Applications > Data Exploration Sciences > GlaxoSmithKline Inc. > (919) 483-8456 > > Brian From ian.fairclough@intelligent-ms.com Fri Oct 31 13:19:58 2003 Received: from scratchy.base.ims ([194.74.0.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9VJJvX27078; Fri, 31 Oct 2003 13:19:57 -0600 Received: from grpware.intelligent-ms.com (grpware.base.ims [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id h9VJ8Ol27019; Fri, 31 Oct 2003 19:08:24 GMT Received: from imsgsx6tvznbng ([192.168.200.185]) by grpware.intelligent-ms.com (8.11.2/8.11.2) with SMTP id h9VJ0ru20131; Fri, 31 Oct 2003 19:00:53 GMT Message-ID: <017a01c39fe4$4b3c2aa0$b7c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Cc: Date: Fri, 31 Oct 2003 19:22:14 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Lock-up in view constructor Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: To answer the questions : The view does contain a number of subviews. The data file is 72M and the initial view contains 28104 rows. Metakit version is : #define d4_MetaKitLibraryVersion 249 // 2.4.9.2 release, Mar 17, 2003 I have built the "thread safe" version. Nothing jumps out at me from the release notes. I see a comment about a race condition in the C++ build (2003-03-05 Fixed two potential races in C++ threaded build) but it looks like that was included in 2.4.9.2. Any suggestions for things I might try would be much appreciated. Best regards, Ian Fairclough. > in someClass constructor : > view = storage.GetAs(structure); > > void someClass::dummyFunction > { > c4_View selectedDataView = this->view.duplicate(); > } > > My problem occurs when the "dummyFunction" is called and returns i.e. > when > the destructor for "selectedDataView" is called. Weird. Does "view" contain subviews? > Is auto commit turned on by default ? Not in C++ and Python. >>> I am running under Red Hat Linux 7.3. Metakit was built with gcc >>> version >>> 2.96 20000731. What is the version of MK - see #define d4_MetaKitLibraryVersion in "mk4.h". You can then compare this with the current change log: http://www.equi4.com/metakit/viewcvs.cgi/metakit/CHANGES?rev=1.172 From jcw@equi4.com Fri Oct 31 13:31:55 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id h9VJVsX27963 for ; Fri, 31 Oct 2003 13:31:54 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 27B6183B2D; Fri, 31 Oct 2003 20:31:46 +0100 (CET) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B6EB383AEF for ; Fri, 31 Oct 2003 20:31:44 +0100 (CET) Date: Fri, 31 Oct 2003 20:31:46 +0100 Subject: Re: [Metakit] Lock-up in view constructor Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <017a01c39fe4$4b3c2aa0$b7c8a8c0@imsgsx6tvznbng> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ian, > The view does contain a number of subviews. The data file is 72M and > the > initial view contains 28104 rows. The presence of subviews may be important. > #define d4_MetaKitLibraryVersion 249 // 2.4.9.2 release, Mar 17, > 2003 Good. > I have built the "thread safe" version. Can you test with a non-threaded build? I doubt that it matters, but if all else fails it will be important to verify this. > Any suggestions for things I might try would be much appreciated. Best thing would be a debug build and interrupting the hanging process to see where it is hanging exactly... i.e. a stack trace. A reproducible test case would be good as well, since then I can do such checks myself. I doubt that the problem is data related, so actual contents in such a test can probably easily be generated. -jcw From jtaylor@shift-right.com Sun Nov 2 19:16:29 2003 Received: from mactufus.net2atlanta.com (mactufus.net2atlanta.com [207.51.14.3]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA31GTX01241 for ; Sun, 2 Nov 2003 19:16:29 -0600 Received: from lefty (208.147.135.177 [208.147.135.177]) by mactufus.net2atlanta.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2657.72) id W1WYF5ZJ; Sun, 2 Nov 2003 20:12:43 -0500 From: "John Taylor" To: Date: Sun, 2 Nov 2003 20:16:31 -0500 Message-ID: <052701c3a1a8$2033cff0$6401a8c0@lefty> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Multi-threading in C++ - update Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Original Posting: ----------------- If anyone one is interested. In the course of using Metakit in our products at work, we have created a Multi-threaded C++ wrapper for Metakit. It is a client/server model using ITC (inter-thread-communications) to isolate all database actions for a given a database to a single thread. The wrapper is part of my company's open source C++ library. Update: ---------------- Over the last 18 months I have received numerous requests for help with the MT wrapper. The major complaint/defect is that the wrapper is part of much larger C++ library and is difficult to isolate and/or extract from the library. Well, enough requests came in and we figure out the logistics of maintaining a sub-set release, so now there is new subset or "compact" release that contains only the Metakit MT wrapper, its dependencies, and the example application. The compact-release (srt-compact-metakit-0.0.9d.tar.gz) can be downloaded at: http://sourceforge.net/project/showfiles.php?group_id=48957 I hope this helps. Feedback would be greatly appreciated. John T. Taylor Shift-Right Technologies, LLC. 433 Comanche Trl Lawrenceville, GA 30044 Tel: 678-344-3115 Fax: 770-736-9276 email: jtaylor@shift-right.com url: www.shift-right.com From ian.fairclough@intelligent-ms.com Mon Nov 3 10:39:56 2003 Received: from scratchy.base.ims ([194.74.0.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA3GdtX18178; Mon, 3 Nov 2003 10:39:55 -0600 Received: from grpware.intelligent-ms.com (grpware.base.ims [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id hA3GSXl12725; Mon, 3 Nov 2003 16:28:33 GMT Received: from imsgsx6tvznbng ([192.168.200.181]) by grpware.intelligent-ms.com (8.11.2/8.11.2) with SMTP id hA3GKju26834; Mon, 3 Nov 2003 16:20:45 GMT Message-ID: <033101c3a229$6d10eb10$b7c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Cc: Date: Mon, 3 Nov 2003 16:42:06 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Memory Leak / Lock-up in view destructor (view contains sub-views) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Thanks for coming back to me about this. I have spent the day tinkering with this. There does seem to be some problem when destructing a view which contains sub-views. It certainly seems like memory is leaked - have you seen that at all ? I think constructing / destructing a view (containing sub-views) would be a good start. Any pointers as to what might be wrong would be much appreciated. Best regards, Ian Fairclough. Ian, > The view does contain a number of subviews. The data file is 72M and > the > initial view contains 28104 rows. The presence of subviews may be important. > #define d4_MetaKitLibraryVersion 249 // 2.4.9.2 release, Mar 17, > 2003 Good. > I have built the "thread safe" version. Can you test with a non-threaded build? I doubt that it matters, but if all else fails it will be important to verify this. > Any suggestions for things I might try would be much appreciated. Best thing would be a debug build and interrupting the hanging process to see where it is hanging exactly... i.e. a stack trace. A reproducible test case would be good as well, since then I can do such checks myself. I doubt that the problem is data related, so actual contents in such a test can probably easily be generated. -jcw ----- Original Message ----- From: "Ian Fairclough" To: Cc: Sent: Friday, October 31, 2003 7:22 PM Subject: Lock-up in view constructor > To answer the questions : > > The view does contain a number of subviews. The data file is 72M and the > initial view contains 28104 rows. > > Metakit version is : > > #define d4_MetaKitLibraryVersion 249 // 2.4.9.2 release, Mar 17, 2003 > > I have built the "thread safe" version. > > Nothing jumps out at me from the release notes. I see a comment about a > race condition in the C++ build (2003-03-05 Fixed two potential races in > C++ threaded build) but it looks like that was included in 2.4.9.2. > > Any suggestions for things I might try would be much appreciated. > > Best regards, > Ian Fairclough. > > > in someClass constructor : > > view = storage.GetAs(structure); > > > > void someClass::dummyFunction > > { > > c4_View selectedDataView = this->view.duplicate(); > > } > > > > My problem occurs when the "dummyFunction" is called and returns i.e. > > when > > the destructor for "selectedDataView" is called. > > Weird. Does "view" contain subviews? > > > Is auto commit turned on by default ? > > Not in C++ and Python. > > >>> I am running under Red Hat Linux 7.3. Metakit was built with gcc > >>> version > >>> 2.96 20000731. > > What is the version of MK - see #define d4_MetaKitLibraryVersion in > "mk4.h". > > You can then compare this with the current change log: > http://www.equi4.com/metakit/viewcvs.cgi/metakit/CHANGES?rev=1.172 > From jcw@equi4.com Mon Nov 3 14:53:34 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA3KrXX30855 for ; Mon, 3 Nov 2003 14:53:33 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id DF2DD83AF3; Mon, 3 Nov 2003 21:53:25 +0100 (CET) Received: from equi4.com (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 4B49F83AEF for ; Mon, 3 Nov 2003 21:53:25 +0100 (CET) Date: Mon, 3 Nov 2003 21:53:26 +0100 Subject: Re: [Metakit] Memory Leak / Lock-up in view destructor (view contains sub-views) Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) From: Jean-Claude Wippler To: Metakit mailing list Content-Transfer-Encoding: 7bit In-Reply-To: <033101c3a229$6d10eb10$b7c8a8c0@imsgsx6tvznbng> Message-Id: X-Mailer: Apple Mail (2.552) X-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ian, > I have spent the day tinkering with this. There does seem to be some > problem when destructing a view which contains sub-views. It certainly > seems like memory is leaked - have you seen that at all ? I think > constructing / destructing a view (containing sub-views) would be a > good > start. Sorry to hear that this continues to not work properly. I'm going to have to insist on sample code to test with, because the problem you describe is so general that it's hard for me to even start looking anywhere. Views and subviews are fundamental to MK, top-level views are in fact also subviews (of a hidden row), and the regression tests have failed to detect this problem, so I really wonder where this comes from. Speaking of which: does MK's "make test" complete successfully on your system? The GCC you mention is not the latest, but then again it's not ancient either. I don't know of any issues with constructors/destructors/temp-opjects. Just to be safe, make sure you run a test with all optimizations disabled. -jcw, puzzled From JHu@CoreLab.com Mon Nov 3 17:54:04 2003 Received: from CLH-EXGW1.CORELAB.COM (pix.corelab.com [12.45.169.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA3Ns4X08684 for ; Mon, 3 Nov 2003 17:54:04 -0600 Received: from CLH-VSMAIL1.CORELAB.COM ([172.16.3.26]) by CLH-EXGW1.CORELAB.COM with Microsoft SMTPSVC(5.0.2195.6713); Mon, 3 Nov 2003 17:53:59 -0600 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3A265.C0170F1D" Date: Mon, 3 Nov 2003 17:53:59 -0600 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Please advise Thread-Index: AcOiZcAVq0HBvr8ZSp67pdPbMsa4Xw== From: "Jim Hu" To: X-OriginalArrivalTime: 03 Nov 2003 23:53:59.0558 (UTC) FILETIME=[C0256A60:01C3A265] Subject: [Metakit] Please advise Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------_=_NextPart_001_01C3A265.C0170F1D Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, All, Somehow I have to something like the diagram shown below. =20 W0 W1 W2 W3 W4 W5 .... WN . . . . . . . . . . . . . . . . . . . . . . . . . . . . .=20 V0 V1 V2 V3 V4 V5 .... VN W is one program running on different computer nodes from 0 to N and V is another program running on the same nodes from 0 to N. Beween W and V there may many other program. There is a way to pass data between W0 to Wn, V0 to Vn and We have some way to pass data from W to V (say MPI). Somehow I need some parameters passed upstream from V to W (ignore = any program in between). I have used Metakit to solve this problem but I have run = into trouble. W on some node cannot get data from V even V has already written down = the value (I used the sample dump program to check it out). Below is an example I used. void SetMigrateRunMode(int run_mode) { char fname[64]; int iw; MPI_Comm_rank( DaggerCommSubworld, &iw); sprintf(fname, "%s%1d%s", "Kmig_run", iw, ".db"); c4_StringProp pName ("name"); c4_IntProp iValue ("ivalue"); c4_Storage kdeck(fname, true); c4_View runModeView =3D kdeck.GetAs("runMode[name:S,ivalue:I]"); runModeView.Add(pName["run_mode"] + iValue[run_mode]); kdeck.Commit(); } int QueryMigrateRunMode(int *run_mode) { char fname[64]; int iw; MPI_Comm_rank( DaggerCommSubworld, &iw); sprintf(fname, "%s%1d%s", "Kmig_run", iw, ".db"); c4_Storage kdeck(fname, true); c4_StringProp pName ("name"); c4_IntProp iValue ("ivalue"); c4_View runModeView =3D kdeck.View("runMode"); if(runModeView.GetSize()=3D=3D0) return(false); *run_mode =3D iValue(runModeView[0]); return(true); } V calls SetMigrateRunMode to write down the data and W query the data like this while(!QueryMigrateRunMode(&kinput->run_mode)) { sleep(1); } Very often W on some node cannot get the data (may be still in the while = loop when the whole thing is down). Would you anyone kindly point out if I have done anything stupid? Many thanks. Jim Hu ------_=_NextPart_001_01C3A265.C0170F1D Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Please advise

Hi, All,

     Somehow I have = to something like the diagram shown below.

    
          = W0   W1   W2    W3  W4  W5 = .... WN

         . . . . = . . . . . . . . . . . . . . . . . . . . . . . . .

          = V0    V1    V2     = V3   V4   V5 .... VN


    W is one program = running on different computer nodes from 0 to N and
    V is another = program running on the same nodes from 0 to N.
    Beween W and V = there may many other program.

    There is a way to = pass data between W0 to Wn, V0 to Vn  and
We have some way to pass data from W = to V (say  MPI).

     Somehow I need = some parameters passed upstream from V to W (ignore any program
in between). I have used Metakit to = solve this problem but I have run into trouble.
W on some node cannot get data from V = even V has already written down the value
(I used the sample dump program to = check it out).

  Below is an example I = used.

void SetMigrateRunMode(int  = run_mode)
{
    char = fname[64];
    int iw;
    MPI_Comm_rank( = DaggerCommSubworld, &iw);
    sprintf(fname, = "%s%1d%s", "Kmig_run", iw, ".db");
    c4_StringProp pName = ("name");
    c4_IntProp iValue = ("ivalue");
    c4_Storage = kdeck(fname, true);
    c4_View runModeView = =3D kdeck.GetAs("runMode[name:S,ivalue:I]");
    = runModeView.Add(pName["run_mode"] + iValue[run_mode]);
    = kdeck.Commit();
}

int QueryMigrateRunMode(int = *run_mode)
{
    char = fname[64];
    int iw;
    MPI_Comm_rank( = DaggerCommSubworld, &iw);
    sprintf(fname, = "%s%1d%s", "Kmig_run", iw, ".db");
    c4_Storage = kdeck(fname, true);
    c4_StringProp pName = ("name");
    c4_IntProp iValue = ("ivalue");
    c4_View runModeView = =3D kdeck.View("runMode");
    = if(runModeView.GetSize()=3D=3D0) return(false);
    *run_mode =3D = iValue(runModeView[0]);
    = return(true);
}

V calls SetMigrateRunMode to write down = the data and
W query the data like this
while(!QueryMigrateRunMode(&kinput->run_mode)) = {
       = sleep(1);
}

Very often W on some node cannot get = the data (may be still in the while loop
when the whole thing is down).

   Would you anyone kindly = point out if I have done anything stupid?


Many thanks.

Jim Hu

------_=_NextPart_001_01C3A265.C0170F1D-- From bkelley@wi.mit.edu Thu Nov 6 09:04:21 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hA6F4LX04330 for ; Thu, 6 Nov 2003 09:04:21 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003110610041711093 for ; Thu, 06 Nov 2003 10:04:17 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HNXR7500.VA4 for ; Thu, 6 Nov 2003 10:04:17 -0500 Message-ID: <3FAA6335.1090402@wi.mit.edu> Date: Thu, 06 Nov 2003 10:05:25 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Metakit large versus small table question Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I have a couple of small tables that do not get modified very much and some tables that get modified a lot (mostly appends but a few deletes) I have noticed that the heavily modified tables take a while to load after a bit (I occasionally save the database to a new file to clean up some space) Is there any place I should place these tables to maximize performance? I.e. declare them last in the storage not first etc. I'll go ahead and do this and report back on the results, but if anybody else knows the answer, I'm all ears. Brian From rickbking@comcast.net Thu Nov 6 23:00:36 2003 Received: from sccrmhc11.comcast.net (sccrmhc11.comcast.net [204.127.202.55]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA750aX28188 for ; Thu, 6 Nov 2003 23:00:36 -0600 Received: from NMSXP1 (bgp01032946bgs.sothfd01.mi.comcast.net[68.41.238.130]) by comcast.net (sccrmhc11) with SMTP id <2003110705003001100j2b9ie>; Fri, 7 Nov 2003 05:00:30 +0000 Message-ID: <012601c3a4ec$11472c20$0200a8c0@NMSXP1> From: "Rick King" To: "Metakit Listserve" Date: Fri, 7 Nov 2003 00:00:27 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Problem flattening a view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi - I had a situation where flattening a view caused some fields to be corrupted. It seemed that data was being assigned to the wrong field. Once I removed duplicate field names in the view being flattened and the containing row the problem went away (or so it seems). Is this a known bug? Or am I missing something? Win XP Python 2.3 Metakit 2.4.9.2 -Rick King Southfield MI From jyl@mod3.net Fri Nov 7 00:49:43 2003 Received: from mod3.net (mod3.net [69.55.237.247]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hA76nhX32258 for ; Fri, 7 Nov 2003 00:49:43 -0600 Received: (qmail 25920 invoked by uid 48); 7 Nov 2003 06:49:42 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 6 Nov 2003 22:49:42 -0800 (PST) Message-ID: <65287.209.233.24.218.1068187782.squirrel@mod3.net> Date: Thu, 6 Nov 2003 22:49:42 -0800 (PST) Subject: Re: [Metakit] Problem flattening a view From: "Jacob Levy" To: In-Reply-To: <012601c3a4ec$11472c20$0200a8c0@NMSXP1> References: <012601c3a4ec$11472c20$0200a8c0@NMSXP1> X-Priority: 3 Importance: Normal Cc: Reply-To: jyl@mod3.net X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: So the flattened view was being created from a tree of views where a certain property name occurred in several of these views? I don't know for sure whether the Metakit view "parser" detects multiple properties with the same name, but in any case I believe that you cannot have multiple properties with the same name (even if it doesnt report an error) in one view. --JYL > Hi - I had a situation where flattening a view caused some fields to be > corrupted. It seemed that data was being assigned to the wrong field. > > Once I removed duplicate field names in the view being flattened and the > containing row the problem went away (or so it seems). > > Is this a known bug? Or am I missing something? > > Win XP > Python 2.3 > Metakit 2.4.9.2 > > -Rick King > Southfield MI > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From rickbking@comcast.net Fri Nov 7 08:45:53 2003 Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA7EjrX30353 for ; Fri, 7 Nov 2003 08:45:53 -0600 Received: from NMSXP1 (bgp01032946bgs.sothfd01.mi.comcast.net[68.41.238.130]) by comcast.net (sccrmhc13) with SMTP id <20031107144547016007hiuie>; Fri, 7 Nov 2003 14:45:47 +0000 Message-ID: <013a01c3a53d$d5037eb0$0200a8c0@NMSXP1> From: "Rick King" To: "Metakit Listserve" References: <012601c3a4ec$11472c20$0200a8c0@NMSXP1> <65287.209.233.24.218.1068187782.squirrel@mod3.net> Subject: Re: [Metakit] Problem flattening a view Date: Fri, 7 Nov 2003 09:45:44 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > So the flattened view was being created from a tree of views where a > certain property name occurred in several of these views? It's kind of difficult to simplify the problem in code so I'll try to explain it more fully. I know that when you join 2 views with common field names metakit uses the first one, e.g. 'Trip','Title' joined with 'Trip','Date','Title',.... => 'Trip','Title','Date',..... (where the result 'Title' is the trip title.) In my problem, I had something like this (where these are all fields): A, nestedview, Ba, Ca, X, Y, D, Ea (where X is set to some text and Y is set to 4.) The nestedview had some field names in common (B,C,E): F, Bb, Cb, G, H, I, Eb What happened when I flattened was: A, F, Bb, Cb, G, H, I, Eb, D, X, Y Most of this was ok except X, which had text in it, was now blank, and Y which was set to 4, was now 543647059. This decimal number in hex is 0x20676553 which looks a lot like part of an ascii string. -Rick King From gmcm@hypernet.com Fri Nov 7 09:16:15 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA7FGEX00818 for ; Fri, 7 Nov 2003 09:16:14 -0600 Received: from PARANOIA (204.176.40.64) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Fri, 7 Nov 2003 09:18:41 -0500 From: "Gordon McMillan" To: "Rick King" , "Metakit Listserve" Date: Fri, 07 Nov 2003 10:14:58 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Problem flattening a view Reply-to: gmcm@hypernet.com Message-ID: <3FAB70A2.9767.6F3E34E2@localhost> Priority: normal In-reply-to: <012601c3a4ec$11472c20$0200a8c0@NMSXP1> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 7 Nov 2003 at 0:00, Rick King wrote: > Hi - I had a situation where flattening a view caused some fields to be > corrupted. It seemed that data was being assigned to the wrong field. > > Once I removed duplicate field names in the view being flattened and the > containing row the problem went away (or so it seems). > > Is this a known bug? Or am I missing something? Mostly that's the way Metakit works, though the Python binding introduces a bit of a wrinkle. You can use view.rename(from, to). That's not an in-place operation. It's low overhead - the data isn't touched, it just tweaks a copy of the metadata. -- Gordon http://www.mcmillan-inc.com/ From rickbking@comcast.net Fri Nov 7 09:52:22 2003 Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA7FqLX06977 for ; Fri, 7 Nov 2003 09:52:22 -0600 Received: from NMSXP1 (bgp01032946bgs.sothfd01.mi.comcast.net[68.41.238.130]) by comcast.net (sccrmhc13) with SMTP id <20031107155216016007m069e>; Fri, 7 Nov 2003 15:52:16 +0000 Message-ID: <019e01c3a547$1e42c820$0200a8c0@NMSXP1> From: "Rick King" To: "Metakit Listserve" References: <3FAB70A2.9767.6F3E34E2@localhost> Subject: Re: [Metakit] Problem flattening a view Date: Fri, 7 Nov 2003 10:52:13 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > > > Hi - I had a situation where flattening a view caused some fields to be > > corrupted. It seemed that data was being assigned to the wrong field. > > > > Once I removed duplicate field names in the view being flattened and the > > containing row the problem went away (or so it seems). > > > > Is this a known bug? Or am I missing something? > > Mostly that's the way Metakit works, though the Python binding > introduces a bit of a wrinkle. It sounds like you're saying that Metakit is supposed to assign data to the wrong field. From gmcm@hypernet.com Fri Nov 7 12:06:36 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hA7I6ZX16476 for ; Fri, 7 Nov 2003 12:06:36 -0600 Received: from PARANOIA (204.176.40.42) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3) for ; Fri, 7 Nov 2003 12:09:02 -0500 From: "Gordon McMillan" To: "Metakit Listserve" Date: Fri, 07 Nov 2003 13:05:19 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Problem flattening a view Reply-to: gmcm@hypernet.com Message-ID: <3FAB988F.18029.6FDA2D96@localhost> Priority: normal In-reply-to: <019e01c3a547$1e42c820$0200a8c0@NMSXP1> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 7 Nov 2003 at 10:52, Rick King wrote: [flatten where subview has common column names with parent view] > > Mostly that's the way Metakit works, though the Python binding > > introduces a bit of a wrinkle. > > It sounds like you're saying that Metakit is supposed to assign data to > the wrong field. In Metakit, a property is a name (case insensitive) and type pair. In the context of a view, it's the column number. The Python binding uses attribute style access (though dictionary style access would have the same difficulty). Which means name only, with some hoops to jump through so the case insensitivity of the underlying code doesn't bite you. Depending on what you're doing, the code may make round trips through code that isn't exactly reversible. IOW, once you've broken the rules, odd things may happen. -- Gordon http://www.mcmillan-inc.com/ From heiko.hees@brainbot.com Sat Nov 8 12:54:23 2003 Received: from mail.brainbot.com ([145.253.163.18]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hA8IsMX15196 for ; Sat, 8 Nov 2003 12:54:22 -0600 Received: (qmail 16191 invoked from network); 8 Nov 2003 18:54:16 -0000 Received: from alf.brainbot.com (HELO brainbot.com) (heiko@192.168.10.213) by heini.brainbot.com with SMTP; 8 Nov 2003 18:54:16 -0000 Message-ID: <3FAD3BD8.7040704@brainbot.com> Date: Sat, 08 Nov 2003 19:54:16 +0100 From: Heiko Hees User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20031010 Debian/1.4-6 X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] join,intersect performance on sorted properties Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: hi, considering i have two views x,y both with a property z, which is sorted in both views, i would excpect joins or intersections on this property to be very fast. unfortunally in reality, the intersection of two sorted views with 1.000.000 integers takes several long seconds. are there any suggestions on how to improve performance for intersections of sorted properties? heiko -- brainbot technologies ag boppstrasse 64 . 55118 mainz . germany fon +49 6131 211639-1 . fax +49 6131 211639-2 http://brainbot.com/ mailto:hees@brainbot.com From gary.h.merrill@gsk.com Mon Nov 10 11:09:41 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAAH9fX26759 for ; Mon, 10 Nov 2003 11:09:41 -0600 Received: by usrtpnp1.glaxowellcome.com; id MAA05913; Mon, 10 Nov 2003 12:09:40 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma003842; Mon, 10 Nov 03 12:07:29 -0500 Received: by ussunt3.glaxo.com id MAA18329; Mon, 10 Nov 2003 12:07:27 -0500 (EST) Received: from 166.71.134.118 by us9n56.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Mon, 10 Nov 2003 12:07:19 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hAAH7Dk21134 for ; Mon, 10 Nov 2003 12:07:19 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Mon, 10 Nov 2003 12:08:08 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 11/10/2003 12:07:22 PM MIME-Version: 1.0 X-WSS-ID: 13B11A4D2612085-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Python internal error on sort Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I'll provide more details if necessary, but perhaps this is something pretty obvious to someone. I have a chunk of code that that unions several views together and then sorts the resulting view. It works fine. But I really need to do this kind of thing in a few places, so I thought I'd write a function to do so. The function does exactly what the inline code did in creating the union of the views and then it returns this view. When I attempt to sort this, I get an internal Python error. The union view looks fine. But the sort just won't work. If I leave the code in-line, it the sort works fine. Any ideas? (I suspect some goofy Python scoping thing is going on, but it seems pretty obscure. ) -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Mon Nov 10 11:31:17 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hAAHVHX28193 for ; Mon, 10 Nov 2003 11:31:17 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003111012311212159 for ; Mon, 10 Nov 2003 12:31:13 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HO5CO100.PZ0; Mon, 10 Nov 2003 12:31:13 -0500 Message-ID: <3FAFCBA0.5090707@wi.mit.edu> Date: Mon, 10 Nov 2003 12:32:16 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com CC: metakit@equi4.com Subject: Re: [Metakit] Python internal error on sort References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: gary.h.merrill@gsk.com wrote: >I'll provide more details if necessary, but perhaps this is something >pretty obvious to someone. > > I think you'll may need to post the smallest working example that you can. Internal python errors are pretty nasty to catch and you may be correct in assuming that a view is being deleted without you knowing about it. If you want to dig yourself, what may work in debugging is using the sys.getrefcount method call. a = [] sys.getrefcount(a) ==> 2 this returns two because a if referenced once by the binding of a=[] and twice by actually using sys.getrefcount(...) liberally sprinkling this through your function may help you diagnose the problem. If the code is small enough, go ahead and post it and I'll scan it for obvious errors. Brian Kelley Whitehead Institute for Biomedical Research From pyguy30@yahoo.com Mon Nov 10 12:56:13 2003 Received: from web11510.mail.yahoo.com (web11510.mail.yahoo.com [216.136.129.47]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hAAIuDX01910 for ; Mon, 10 Nov 2003 12:56:13 -0600 Message-ID: <20031110185612.89826.qmail@web11510.mail.yahoo.com> Received: from [192.131.99.117] by web11510.mail.yahoo.com via HTTP; Mon, 10 Nov 2003 10:56:12 PST Date: Mon, 10 Nov 2003 10:56:12 -0800 (PST) From: j n To: Metakit Listserve In-Reply-To: <3FAB70A2.9767.6F3E34E2@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] help on subtables Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hello there, I am trying to use subtables with metakit and python. Unfortunately, the docs are scarce about that. Let's say I want to create a subtable for every new day for a person. How do I add a new subtable later w/out having to redefine w/getas? Let's say I start with 2 subtables: db = metakit.storage("c:/tmp/metak/sub2.mk",1) vw=db.getas("barney[date1[location:S],date2[location:S]]") i=logons.append() vw[i].date1.append('home') vw[i].date1.append('bowling') vw[i].date1.append('home again') vw[i].date2.append('work') vw[i].date2.append('bowling') vw[i].date2.append('at the bar') This creates and adds data to 2 subtables. If I want to add a third date. Is there an easy way to just append the subtable? Thanks for any info, john __________________________________ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree From jcw@equi4.com Mon Nov 10 13:15:06 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAAJF6X03180 for ; Mon, 10 Nov 2003 13:15:06 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 43D7E83AF3; Mon, 10 Nov 2003 20:14:58 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 9A8848393A for ; Mon, 10 Nov 2003 20:14:57 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <20031110185612.89826.qmail@web11510.mail.yahoo.com> References: <20031110185612.89826.qmail@web11510.mail.yahoo.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <2B874982-13B2-11D8-A783-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] help on subtables Date: Mon, 10 Nov 2003 20:14:56 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-2.2 required=5.0 tests=AWL,IN_REP_TO,REFERENCES,SPAM_PHRASE_00_01, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: j n wrote: > I am trying to use subtables with metakit and python. > Unfortunately, the docs are scarce about that. > > Let's say I want to create a subtable for every new > day for a person. How do I add a new subtable later w/out > having to redefine w/getas? Can't be done. Changing structure goes through getas(). > vw=db.getas("barney[date1[location:S],date2[location:S]]") [...] > This creates and adds data to 2 subtables. If I want > to add a third date. Is there an easy way to just > append the subtable? Solution: think different. Instead of varying the structure according to your data needs, starts with a structure which varies in rows. Introduce an extra level: vw=db.getas("barney[dates[day:I,locations[name:S]]]]") Then you can do: vw[i].dates.setsize(30) vw[i].dates[1].locations.append('home') vw[i].dates[1].locations.append('bowling') vw[i].dates[1].locations.append('home again') vw[i].dates[2].locations.append('work') vw[i].dates[2].locations.append('bowling') vw[i].dates[2].locations.append('at the bar') This three-level structure is not necessarily very efficient (lots and lots of tiny subviews can have a bit of overhead in space and time). If that turns out to be an issue, consider this structure instead: vw=db.getas("barney[locations[day:I,name:S]]") Then add pairs/tuples such as (1,'home') instead. The general guideline in these suggestions is to freeze the horizontal dimension (data structures) at design time, and to vary the vertical dimension (row contents) at run time. -jcw From Murat_Berk@bmc.com Mon Nov 10 14:27:02 2003 Received: from creeper.bmc.com (fw-us-hou19.bmc.com [198.207.223.240]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAAKR1X07521 for ; Mon, 10 Nov 2003 14:27:01 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2-BMC/8.10.2) with ESMTP id hAAKQqA24071 for ; Mon, 10 Nov 2003 14:26:52 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2657.72) id ; Mon, 10 Nov 2003 14:26:33 -0600 Message-ID: <3DAC9CD31458D411BCE700D0B75D0A130B177C58@ES09-HOU.bmc.com> From: "Berk, Murat" To: "'metakit@equi4.com'" Date: Mon, 10 Nov 2003 14:26:30 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Question on how to remove top level views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: We have a database from the days where subviews are not as functional as now. We used to store a view name in one view (instead of marking cell as subview) so that we can do getAs(name) and use it. I do not want to change a lot of things, but when we try to delete the rows in the first view, the only thing we can do is to remove all elements of the second view but I cannot really delete it. view1 name name1 data... name2 data... __name1__ prop1 prop2 prop3 __name2__ prop1 prop2 prop3 When we delete name1 from the first view, I want to remove the whole view called __name1__. How can I do this.. Since everything is a view including database itself, in theory this should be possible. Murat From jcw@equi4.com Tue Nov 11 02:17:51 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAB8HpX09594 for ; Tue, 11 Nov 2003 02:17:51 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 8285783AF3; Tue, 11 Nov 2003 09:17:44 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 91C1D8393A for ; Tue, 11 Nov 2003 09:17:43 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <3DAC9CD31458D411BCE700D0B75D0A130B177C58@ES09-HOU.bmc.com> References: <3DAC9CD31458D411BCE700D0B75D0A130B177C58@ES09-HOU.bmc.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <8519F40F-141F-11D8-B582-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Question on how to remove top level views Date: Tue, 11 Nov 2003 09:17:41 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-3.4 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Berk, Murat wrote: > We used to store a view name in one view (instead of marking cell as > subview) so that we can do getAs(name) and use it. > > I do not want to change a lot of things, but when we try to delete the > rows in the first view, the only thing we can do is to remove all > elements of the second view but I cannot really delete it. > > view1 > name > name1 data... > name2 data... > > __name1__ > prop1 prop2 prop3 > > __name2__ > prop1 prop2 prop3 > > > When we delete name1 from the first view, I want to remove the whole > view > called __name1__. How can I do this.. storage.GetAs("__name1__") IOW, omit the usual "[...]" part. Am I missing something? > Since everything is a view including database itself, in theory this > should be possible. Yes. The one confusing part could be that the view won't go away until committed. All properties, including subviews, including therefore also top-level views, stay around afetr a restructure, and only truly vanish on commit (that makes it possible to re-structure, copy over, then commit). -jcw From jcw@equi4.com Tue Nov 11 02:31:57 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAB8VuX10259 for ; Tue, 11 Nov 2003 02:31:56 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 4A21583AF3; Tue, 11 Nov 2003 09:31:50 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id CB07E8393A for ; Tue, 11 Nov 2003 09:31:49 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <3FAD3BD8.7040704@brainbot.com> References: <3FAD3BD8.7040704@brainbot.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <7DBD2EBC-1421-11D8-B582-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] join,intersect performance on sorted properties Date: Tue, 11 Nov 2003 09:31:48 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-3.5 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,SPAM_PHRASE_01_02,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Heiko Hees wrote: > considering i have two views x,y both with a property z, which is > sorted in both views, > i would excpect joins or intersections on this property to be very > fast. unfortunally in reality, the intersection of two sorted views > with 1.000.000 integers takes several long seconds. MK does not know it's sorted and re-sorts both views when joining. Intersection is implemented by doing Counts(), then selecting items with count 2, then projecting away the added count field. Count in turn, uses a variant of Groupby, which sorts the view first - so as you can see it's quite general and I'm sure one could find better approaches for specific cases. > are there any suggestions on how to improve performance for > intersections of sorted properties? I can't think of any right now. You could look at the implementations (you don't mention which language you use) and see if you can come up with a special-cased version. There are no doubt ways to improve it, especially if the key is unique, i.e. the join is never to more than 1 row. Especially constructs using RemapWith() are worth looking into (and perhaps Concat or Pair). -jcw From bkelley@wi.mit.edu Tue Nov 11 07:05:51 2003 Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hABD5pX25374 for ; Tue, 11 Nov 2003 07:05:51 -0600 Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.147.123.161]) by comcast.net (rwcrmhc13) with SMTP id <2003111113054501500a7j1ue>; Tue, 11 Nov 2003 13:05:45 +0000 Message-ID: <3FB0E00B.9080605@wi.mit.edu> Date: Tue, 11 Nov 2003 08:11:39 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 CC: Metakit mailing list Subject: Re: [Metakit] join,intersect performance on sorted properties References: <3FAD3BD8.7040704@brainbot.com> <7DBD2EBC-1421-11D8-B582-000A9588127E@equi4.com> In-Reply-To: <7DBD2EBC-1421-11D8-B582-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Heiko Hees wrote: > >> considering i have two views x,y both with a property z, which is >> sorted in both views, >> i would excpect joins or intersections on this property to be very >> fast. unfortunally in reality, the intersection of two sorted views >> with 1.000.000 integers takes several long seconds. > > > MK does not know it's sorted and re-sorts both views when joining. I did a small test using python and got the time down a little bit. Here is what I did essentially (I'm sure that C++ would be faster) create a mapping object key=int value=index, you could also use (a bitvector now that I think about it, it will save some memory). create a temporary determine the smaller of the two views scan the z column of the smaller object, add integers to the mapping scan the z column of the larger object, if the integer is in the mapping, append the row to the temporary view. 1,000,000 integers took 5 seconds on a 2.0Ghz machine but the memory overhead was about 10M. Metakit is very fast for scanning through columns, so what I might do is just scan through both columns in parallel taking advantage of their sorted nature. In psuedo-code here while list1 and list2: z1 = list1.next() z2 = list2.next() while z1 > z2: list1.next() while z2 > z1: list2.next() if z1 == z2: add to view look ahead into the list1 and list2 columns to grab more matches is possible... The tricky part is the look ahead to make sure that you get all matches, but this code should really fly. From gary.h.merrill@gsk.com Tue Nov 11 07:48:52 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hABDmqX27269 for ; Tue, 11 Nov 2003 07:48:52 -0600 Received: by usrtpnp1.glaxowellcome.com; id IAA00021; Tue, 11 Nov 2003 08:48:47 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma028099; Tue, 11 Nov 03 08:47:09 -0500 Received: by ussunt3.glaxo.com id IAA00244; Tue, 11 Nov 2003 08:47:07 -0500 (EST) Received: from 166.71.134.118 by us9n52.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Tue, 11 Nov 2003 08:47:01 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hABDl0k02886 for ; Tue, 11 Nov 2003 08:47:00 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Tue, 11 Nov 2003 08:46:54 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 11/11/2003 08:47:03 AM MIME-Version: 1.0 X-WSS-ID: 13AE37DE3002273-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] More nested view questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: The recent questions around nested (or sub-) views come at an opportune time for me since I am about to generate a Metakit database for the "FactFinder" portion of my text mining platform. Each "fact" (and this is a simplistic concept at this point in time) will be represented as a set of (generally 3) "components". And each component will have the same structure, which will contain such information as its position in the source, the ontology topic it immediately falls under, etc.. The "facts view" will then consist of rows of facts. A flattened representation would look something like this: [sourceId:I,comp_1_start:I,comp_1_end:I,comp_1_topic:I,comp_2_start:I,comp_2_end:I,comp_2_topic:I,comp_3_start:I,comp_3_end:I,comp_3_topic:I] whereas a fairly natural and elegant nested approach would look like: [sourceId:I,comp_1:[start:I,end:I,topic:I],comp_2:[start:I,end:I,topic:I],comp_3:[start:I,end:I,topic:I]] But what are the tradeoffs here? What does the nested approach get me other than a more appealing-looking structure (which no one will ever see)? The nested structure is certainly a bit easier and natural to deal with, but functions and methods could be written to make the flat approach work about as well. Obviously, if I want or need to pass "components" around, then the nested approach is cleaner and easier. I'm very drawn to it, but I'm concerned about performance if, for example, I need to loop through the rows and do checks and comparisons on values within the nested views. (For example, this view itself will be generated by running an FSA (FST, actually) on a larger view of "topic occurrences", and a similar approach may need to be taken in order to extract further information out of the facts view.) Or should I keep the fundamental facts view in the nested form and then flatten it for various kinds of processing? What is the overhead involved in flattening? Any thoughts or insights will be appreciated. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From Murat_Berk@bmc.com Tue Nov 11 08:00:46 2003 Received: from creeper.bmc.com (fw-us-hou19.bmc.com [198.207.223.240]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hABE0kX27814; Tue, 11 Nov 2003 08:00:46 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2-BMC/8.10.2) with ESMTP id hABE0VR25689; Tue, 11 Nov 2003 08:00:32 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2657.72) id ; Tue, 11 Nov 2003 08:00:11 -0600 Message-ID: <3DAC9CD31458D411BCE700D0B75D0A130B177C7A@ES09-HOU.bmc.com> From: "Berk, Murat" To: "'Jean-Claude Wippler'" , Metakit mailing list Subject: RE: [Metakit] Question on how to remove top level views Date: Tue, 11 Nov 2003 08:00:08 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Thanks I did not know that GetAs("name") will delete the view :) Murat -----Original Message----- From: Jean-Claude Wippler [mailto:jcw@equi4.com] Sent: Tuesday, November 11, 2003 3:18 AM To: Metakit mailing list Subject: Re: [Metakit] Question on how to remove top level views Berk, Murat wrote: > We used to store a view name in one view (instead of marking cell as > subview) so that we can do getAs(name) and use it. > > I do not want to change a lot of things, but when we try to delete the > rows in the first view, the only thing we can do is to remove all > elements of the second view but I cannot really delete it. > > view1 > name > name1 data... > name2 data... > > __name1__ > prop1 prop2 prop3 > > __name2__ > prop1 prop2 prop3 > > > When we delete name1 from the first view, I want to remove the whole > view > called __name1__. How can I do this.. storage.GetAs("__name1__") IOW, omit the usual "[...]" part. Am I missing something? > Since everything is a view including database itself, in theory this > should be possible. Yes. The one confusing part could be that the view won't go away until committed. All properties, including subviews, including therefore also top-level views, stay around afetr a restructure, and only truly vanish on commit (that makes it possible to re-structure, copy over, then commit). -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From pyguy30@yahoo.com Tue Nov 11 16:03:20 2003 Received: from web11509.mail.yahoo.com (web11509.mail.yahoo.com [216.136.129.46]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hABM3KX27789 for ; Tue, 11 Nov 2003 16:03:20 -0600 Message-ID: <20031111220319.51745.qmail@web11509.mail.yahoo.com> Received: from [24.26.151.86] by web11509.mail.yahoo.com via HTTP; Tue, 11 Nov 2003 14:03:19 PST Date: Tue, 11 Nov 2003 14:03:19 -0800 (PST) From: j n Subject: Re: [Metakit] help on subtables To: Jean-Claude Wippler , Metakit mailing list In-Reply-To: <2B874982-13B2-11D8-A783-000A9588127E@equi4.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > The general guideline in these suggestions is to > freeze the horizontal > dimension (data structures) at design time, and to > vary the vertical > dimension (row contents) at run time. Thanks. I'll learn to think vertically. I was thinking subtables were functionally similar to tables. Top level tables can be added whenver you want, in effect horizonally. However, subtables function different and is best to approach vertically. Is there a limit to top level tables? 1000, 10,000? When does the overhead of managing that in one file become cumbersome? Again, thanks, john __________________________________ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree From gary.h.merrill@gsk.com Sat Nov 15 19:38:17 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAG1cGX18828 for ; Sat, 15 Nov 2003 19:38:17 -0600 Received: by usrtpnp1.glaxowellcome.com; id UAA19716; Sat, 15 Nov 2003 20:38:11 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma019439; Sat, 15 Nov 03 20:37:43 -0500 Received: by ussunt3.glaxo.com id UAA05392; Sat, 15 Nov 2003 20:37:41 -0500 (EST) Received: from 166.71.134.118 by us9n53.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Sat, 15 Nov 2003 20:37:37 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hAG1bZk10621 for ; Sat, 15 Nov 2003 20:37:36 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Sat, 15 Nov 2003 20:38:31 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 11/15/2003 08:37:39 PM MIME-Version: 1.0 X-WSS-ID: 13A80B6A3580128-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] unique() with count? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: It turns out that the unique() view operation is pretty useful to me. However, what would be even more useful is to have a count of each record which indicates the total number of records which were in its identity class. This would be extremely handy in certain kinds of datamining or inferencing operations. Is this information possibly available anywhere? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From jcw@equi4.com Sun Nov 16 04:45:57 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAGAjuX18364; Sun, 16 Nov 2003 04:45:57 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id B5C8483AF3; Sun, 16 Nov 2003 11:45:49 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id ED7C78393A; Sun, 16 Nov 2003 11:45:48 +0100 (CET) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <0A8CF550-1822-11D8-AF94-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] unique() with count? Date: Sun, 16 Nov 2003 11:45:49 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-2.6 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: wrote: > It turns out that the unique() view operation is pretty useful to me. > However, what would be even more useful is to have a count of each > record > which indicates the total number of records which were in its identity > class. apply(view.counts, view.structure() + ['frequency']) ? -jcw From kbachman@equis.com Tue Nov 18 14:50:06 2003 Received: from pierre.equis.com (pierre.equis.com [204.246.137.26]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hAIKo6X23484 for ; Tue, 18 Nov 2003 14:50:06 -0600 Received: from no.name.available by pierre.equis.com via smtpd (for equi4.com [216.110.35.178]) with SMTP; Tue, 18 Nov 2003 13:50:06 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3AE15.89F12F36" Date: Tue, 18 Nov 2003 13:50:02 -0700 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: On-the-fly data structures and nested views Thread-Index: AcOuFYisYKiGdZUYRly/Uj3vBWQrXw== From: "Kim Bachman" To: Subject: [Metakit] On-the-fly data structures and nested views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------_=_NextPart_001_01C3AE15.89F12F36 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello, =20 I'm evaluating the Metakit and have a few questions involving on-the-fly data structures and nested views. =20 =20 I am wondering what the best way to represent the data in Metakit would be. I have tried nested views, but I'm not sure they will be the most efficient. Plus, the on-the-fly data structures do not seem to be working -- at least for me. =20 I have a number of items. Each item can have zero to thousands of data records. The properties for the data records can differ for each item. For example, one item may have data like Date, Time, Style, Size, etc. while another item may have data like Date, Type, Length, Color, etc. =20 To make things worse, I need the ability to add/remove properties from the item data on a per item basis at run time. The item data could include one or more of hundreds of properties. =20 ItemView Item[Name:S,Description:S,Number:I...] ItemDataView ItemData[Date:I,Time:I,Style:I,Size:I...] ... ItemData[Date:I,Time:I,Style:I,Size:I...] =20 Item[Name:S,Description:S,Number:I...] ItemDataView ItemData[Date:I,Type:I,Length:I,Color:I...] ... ItemData[Date:I,Type:I,Length:I,Color:I...] =20 1. Is this possible in Metakit? 2. Any ideas on how to structure the data in Metakit? =20 Thank you, =20 Kim Bachman =20 =20 ------_=_NextPart_001_01C3AE15.89F12F36 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Message
Hello,
 
I'm evaluating the Metakit and have a few = questions=20 involving on-the-fly data structures and nested views. =20
 
I am wondering what the best way to represent = the data=20 in Metakit would be.  I have tried nested views, but I'm not=20 sure they will be the most=20 efficient.  Plus, the on-the-fly data structures do not seem to be=20 working -- at least for=20 me.
 
I have a number of items.  Each item can have zero to thousands = of data=20 records.  The properties for the data records can differ for=20 each item.  For = example,=20 one item may have data=20 like Date, Time, Style, Size,=20 etc. while another item may = have data=20 like Date, Type, Length, = Color,=20 etc.
 
To make = things worse,=20 I need the ability to add/remove properties from = the item data on a per item basis at run=20 time.  = The item data could include one or more of = hundreds=20 of properties.
 
ItemView
   =20 Item[Name:S,Description:S,Number:I...]
       =20 ItemDataView
=
            ItemData[Date:I,Time:I,Style:I,Size:I...]
          &nbs= p;=20 ...
            ItemData[Date:I,Time:I,Style:I,Size:I...]
 
   =20 Item[Name:S,Description:S,Number:I...]
<= /SPAN>
       =20 ItemDataView
=
      &nb= sp;     ItemData[Date:I,Type:I,Length:I,Color:I...]
          &nbs= p;=20 ...
      &nb= sp;     ItemData[Date:I,Type:I,Length:I,Color:I...]
   =20
1. Is this possible in = Metakit?
2. Any = ideas on how=20 to structure the data in=20 Metakit?
 
Thank you,
 
Kim=20 Bachman
 
 
------_=_NextPart_001_01C3AE15.89F12F36-- From bkelley@wi.mit.edu Tue Nov 18 15:46:02 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hAILk2X27199 for ; Tue, 18 Nov 2003 15:46:02 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003111816455731026 for ; Tue, 18 Nov 2003 16:45:58 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HOKHSL00.6OK; Tue, 18 Nov 2003 16:45:58 -0500 Message-ID: <3FBA935C.6010001@wi.mit.edu> Date: Tue, 18 Nov 2003 16:47:08 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kim Bachman , Metakit mailing list Subject: Re: [Metakit] On-the-fly data structures and nested views References: In-Reply-To: Content-Type: multipart/alternative; boundary="------------020602020208040801080709" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------020602020208040801080709 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Kim Bachman wrote: > > To make things worse, I need the ability to add/remove properties from > the item data on a per item basis at run time. The item data could > include one or more of hundreds of properties. > > ItemView > Item[Name:S,Description:S,Number:I...] > ItemDataView > ItemData[Date:I,Time:I,Style:I,Size:I...] > ... > ItemData[Date:I,Time:I,Style:I,Size:I...] > > Item[Name:S,Description:S,Number:I...] > ItemDataView > ItemData[Date:I,Type:I,Length:I,Color:I...] > ... > ItemData[Date:I,Type:I,Length:I,Color:I...] > > 1. Is this possible in Metakit? Yes. Get rid of ItemView and have Item in different tables. Let's assume that in your table "Name" is unique across all items. You will need some unique identifier for the following technique to work. If name as unique we can generate an "items" table which knows which underlying table a given item lives in. The underlying tables can have different properties and subviews but all should have an unique column Name. The idea is when looking up an object first find the name in the items table. From the items table find out the name of the subtable, retrieve this view and then look up Name in the subtable. You'll have to forgive my python style pseudo code here. itemtable = st.getas("items[Name:S,table:S]") itemtype1 = st.getas("itemtype1[Name:S.Description:S,Number:I...,data[Date:I,Time:I,Style:I,Size:I]") itemtype2 = st.getas("itemtype2[Name:S.Description:S,Number:I...,data[Date:I,Type:I,Length:I,Color:I...]") # we'll create two objects named "object1" and "object2" that live in the seperate tables itemtype1 itemtype2 itemtable.append(("object1", "itemtype1")) itemtable.append(("object2", "itemtype2")) # add some data to the objects index = itemtype1.append(("object1", ...)) # add some properties to the subview itemtype1[index].data.append(record1)) itemtype1[index].data.append(record2)) index = itemtype2.append(("object2", ...)) ... now the dispatch method to find the appropriate subtable The dispatcher finds a given item with name Name from itemtable and retrieve the appropriate row from the subtables #first find out which table belongs to the item index = itemtable.find(Name=Name) if index == -1: Error, cannot find item row = itemtable[index] #retrieve the proper table table = st.view(row.table) # the subtables can be restructured on the fly with new properties ect... # find the proper row index = table.find(Name=Name) if index == -1: Error, Name doesn't exist in the table! # get the row row = table[index] # play with the row :) for data in row.data: ... > --------------020602020208040801080709 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Kim Bachman wrote:
Message

To make things worse, I need the ability to add/remove properties from the item data on a per item basis at run time.  The item data could include one or more of hundreds of properties.
 
ItemView
    Item[Name:S,Description:S,Number:I...]
        ItemDataView
            ItemData[Date:I,Time:I,Style:I,Size:I...]
            ...
            ItemData[Date:I,Time:I,Style:I,Size:I...]
 
    Item[Name:S,Description:S,Number:I...]
        ItemDataView
            ItemData[Date:I,Type:I,Length:I,Color:I...]
            ...
            ItemData[Date:I,Type:I,Length:I,Color:I...]
   
1. Is this possible in Metakit?
Yes.  Get rid of ItemView and have Item in different tables.  Let's assume that in your table "Name" is unique across all items.  You will need some unique identifier for the following technique to work.  If name as unique we can generate an "items" table which knows which underlying table a given item lives in.  The underlying tables can have different properties and subviews but all should have an unique column Name.

The idea is when looking up an object first find the name in the items table.  From the items table find out the name of the subtable, retrieve this view and then look up Name in the subtable.

You'll have to forgive my python style pseudo code here. 

itemtable = st.getas("items[Name:S,table:S]")
itemtype1 = st.getas("itemtype1[Name:S.Description:S,Number:I...,data[Date:I,Time:I,Style:I,Size:I]")
itemtype2 = st.getas("itemtype2[Name:S.Description:S,Number:I...,data[Date:I,Type:I,Length:I,Color:I...]")

# we'll create two objects named "object1" and "object2" that live in the seperate tables itemtype1 itemtype2
itemtable.append(("object1", "itemtype1"))
itemtable.append(("object2", "itemtype2"))

# add some data to the objects
index = itemtype1.append(("object1", ...))
# add some properties to the subview
itemtype1[index].data.append(record1))
itemtype1[index].data.append(record2))

index = itemtype2.append(("object2", ...))
...

now the dispatch method to find the appropriate subtable

The dispatcher finds a given item with name Name from itemtable and retrieve the appropriate row from the subtables

#first find out which table belongs to the item
index = itemtable.find(Name=Name)
if index == -1: Error, cannot find item
row = itemtable[index]
#retrieve the proper table
table = st.view(row.table)
# the subtables can be restructured on the fly with new properties ect...
# find the proper row
index = table.find(Name=Name)
if index == -1: Error, Name doesn't exist in the table!
# get the row
row = table[index]
# play with the row :)
for data in row.data:
      ...
 

--------------020602020208040801080709-- From rcohen@articque.com Wed Nov 19 02:38:09 2003 Received: from mwinf0501.wanadoo.fr (smtp5.wanadoo.fr [193.252.22.26]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAJ8c8X29522 for ; Wed, 19 Nov 2003 02:38:08 -0600 Received: from articque.com (ATours-101-1-2-241.w217-128.abo.wanadoo.fr [217.128.52.241]) by mwinf0501.wanadoo.fr (SMTP Server) with ESMTP id 9BA6C40021A for ; Wed, 19 Nov 2003 09:38:01 +0100 (CET) Message-ID: <3FBB2BAE.5030004@articque.com> Date: Wed, 19 Nov 2003 09:37:02 +0100 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit mailing list References: <0A8CF550-1822-11D8-AF94-000A9588127E@equi4.com> In-Reply-To: <0A8CF550-1822-11D8-AF94-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] DigitalMars compiler on win32 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Hi, I tried compiling metakit with DigitalMars compiler : http://www.digitalmars.com/ . (I try to replace MSDEVstudio compiler, too expensive !) FIRST PROBLEM ============= It appears that opening a database for writing did not work because of the flag _O_NOINHERIT in : bool c4_FileStrategy::DataOpen(const char* fname_, int mode_) { d4_assert(!_file); #if q4_WIN32 && !q4_BORC && !q4_WINCE int flags = _O_BINARY | _O_NOINHERIT | (mode_ > 0 ? _O_RDWR : _O_RDONLY); On MSDEV5 documentation this flag does not exists for _open, but only for _sopen. I suppose _open ignore it. On DigitalMars documentation it does not exists at all, but it is defined as 128 (I tried to printf it) ! Removing it make the whole thing work correctly. First problem solved. SECOND PROBLEM ============== I think more and more that this IDDE is quite good, and the compiler produce quick result. for instance, on a 50 000 records database : MSDEV : adding takes 0.248 ms/record in avg commit (made every 3000 records) takes 125->1015 ms searching takes 23.81 ms/record in avg DIGITALMARS : adding takes 0.253 ms/record in avg commit (made every 3000 records) takes 125->1047 ms searching takes 33.93 ms/record in avg (a little bit worse). But i'm afraid there is a problem with memory management. At the end, digitalmars executable takes 38Mb of memory, and MSDEV executable takes 800 kb. I think the whole file is in memory. Trying with subviews and hashing, my digitalmars executable takes 1Gb of mem ! This is the modifications I made for DigitalMars : - fileio.cpp : I removed _O_NOINHERIT ifdef __DMC__ - string.cpp : I added __DMC__ for #define strcasecmp stricmp - win/config.h : I added __DMC__ with _MSC_VER >= 1000 for memory mapped file I'm sure there must be other modifications to make it work correctly ! What test should I try, what option should I set ? Thanks for any information. -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From Khorev@SoftLab.RU Wed Nov 19 03:11:37 2003 Received: from Hermes.SoftLab.RU (hermes.softlab.ru [217.174.99.210]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAJ9BbX31563 for ; Wed, 19 Nov 2003 03:11:37 -0600 Received: by HERMES with Internet Mail Service (5.5.2653.19) id ; Wed, 19 Nov 2003 12:11:50 +0300 Message-ID: From: Khorev Sergey To: "'Riccardo Cohen'" , Metakit mailing list Subject: RE: [Metakit] DigitalMars compiler on win32 Date: Wed, 19 Nov 2003 12:11:48 +0300 X-Mailer: Internet Mail Service (5.5.2653.19) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: > It appears that opening a database for writing did not work > because of the flag _O_NOINHERIT in :... > > On MSDEV5 documentation this flag does not exists for _open, > but only for _sopen. I suppose _open > ignore it. Look in the CRT sources. At least for VC++ 6 and 7.1 _open calls _sopen. From jcw@equi4.com Wed Nov 19 03:56:08 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAJ9u8X01768 for ; Wed, 19 Nov 2003 03:56:08 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 8DBE983EF2; Wed, 19 Nov 2003 10:56:01 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 219AA83BB0 for ; Wed, 19 Nov 2003 10:56:01 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <3FBB2BAE.5030004@articque.com> References: <0A8CF550-1822-11D8-AF94-000A9588127E@equi4.com> <3FBB2BAE.5030004@articque.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <95E9813C-1A76-11D8-AF63-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] DigitalMars compiler on win32 Date: Wed, 19 Nov 2003 10:56:03 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-3.4 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > I tried compiling metakit with DigitalMars compiler : > http://www.digitalmars.com/ . [very interesting results] > But i'm afraid there is a problem with memory management. At the end, > digitalmars executable takes 38Mb of memory, and MSDEV executable > takes 800 kb. I think the whole file is in memory. This indicates that memory-mapped files are not being used. Look at ResetFileMapping() in fileio.cpp and make sure the CreateFileMapping and MapViewOfFile calls return good values. If not, MK reverts to fully mem-resident reading of data, which grows to consume memory until the next Commit() or Rollback(). > This is the modifications I made for DigitalMars : > - fileio.cpp : I removed _O_NOINHERIT ifdef __DMC__ > - string.cpp : I added __DMC__ for #define strcasecmp stricmp > - win/config.h : I added __DMC__ with _MSC_VER >= 1000 for memory > mapped file Thanks, once you have the memory issue fixed, I'd like to integrate all changes. -jcw From rcohen@articque.com Wed Nov 19 09:17:41 2003 Received: from mwinf0501.wanadoo.fr (smtp5.wanadoo.fr [193.252.22.26]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAJFHeX19024 for ; Wed, 19 Nov 2003 09:17:40 -0600 Received: from articque.com (ATours-101-1-2-241.w217-128.abo.wanadoo.fr [217.128.52.241]) by mwinf0501.wanadoo.fr (SMTP Server) with ESMTP id 1EFC240022A for ; Wed, 19 Nov 2003 16:17:34 +0100 (CET) Message-ID: <3FBB8954.6080000@articque.com> Date: Wed, 19 Nov 2003 16:16:36 +0100 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 Cc: Metakit mailing list Subject: Re: [Metakit] DigitalMars compiler on win32 References: <0A8CF550-1822-11D8-AF94-000A9588127E@equi4.com> <3FBB2BAE.5030004@articque.com> <95E9813C-1A76-11D8-AF63-000A9588127E@equi4.com> In-Reply-To: <95E9813C-1A76-11D8-AF63-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: It appears that CreateFileMapping is called and return a non null handle, then MapViewOfFile is called and return a non null _mapStart After reading docs, all seems correct, but actually the file seems not to be mapped. I'll ask that to dm newsgroup, but do you have a 10 lines sample to check that ? Jean-Claude Wippler wrote: > Riccardo Cohen wrote: > >> I tried compiling metakit with DigitalMars compiler : >> http://www.digitalmars.com/ . > > [very interesting results] > >> But i'm afraid there is a problem with memory management. At the end, >> digitalmars executable takes 38Mb of memory, and MSDEV executable >> takes 800 kb. I think the whole file is in memory. > > > This indicates that memory-mapped files are not being used. > > Look at ResetFileMapping() in fileio.cpp and make sure the > CreateFileMapping and MapViewOfFile calls return good values. If not, > MK reverts to fully mem-resident reading of data, which grows to consume > memory until the next Commit() or Rollback(). > >> This is the modifications I made for DigitalMars : >> - fileio.cpp : I removed _O_NOINHERIT ifdef __DMC__ >> - string.cpp : I added __DMC__ for #define strcasecmp stricmp >> - win/config.h : I added __DMC__ with _MSC_VER >= 1000 for memory >> mapped file > > > Thanks, once you have the memory issue fixed, I'd like to integrate all > changes. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From rcohen@articque.com Thu Nov 20 03:32:00 2003 Received: from mwinf0101.wanadoo.fr (smtp1.wanadoo.fr [193.252.22.30]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAK9VxX25672 for ; Thu, 20 Nov 2003 03:32:00 -0600 Received: from articque.com (ATours-101-1-1-138.w193-252.abo.wanadoo.fr [193.252.49.138]) by mwinf0101.wanadoo.fr (SMTP Server) with ESMTP id 01A22C000133 for ; Thu, 20 Nov 2003 10:31:53 +0100 (CET) Message-ID: <3FBC89CE.8070302@articque.com> Date: Thu, 20 Nov 2003 10:30:54 +0100 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit mailing list Subject: Re: [Metakit] DigitalMars compiler on win32 References: <0A8CF550-1822-11D8-AF94-000A9588127E@equi4.com> <3FBB2BAE.5030004@articque.com> <95E9813C-1A76-11D8-AF63-000A9588127E@equi4.com> In-Reply-To: <95E9813C-1A76-11D8-AF63-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: about _O_NOINHERIT, this is the answer of digital mars newsgroup : -------- Original Message -------- Subject: Re: _O_NOINHERIT Date: Wed, 19 Nov 2003 23:48:45 +0100 From: Arjan Knepper Organization: Digital Mars Newsgroups: c++.idde References: OK you are just using the c-runtime '_open (..)'. Take a look at the source file which implements _open (..) in the compiler src directory. That way you can see how the _O_NOINHERIT is affecting the file open api call (which will probably use CreateFile). I would not use the "undocumented" _O_NOINHERIT. Has problaby something to do with the HANDLE returned from CreateFile not being inheritable by child processes or threads or DuplicateHandle. Arjan =====> I let the #ifndef __DMC__ I continue to investigate for memory mapped file, but thanks if you have a small sample I can submit. I know nothing about that topic! -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From gary.h.merrill@gsk.com Thu Nov 20 11:09:55 2003 Received: from usrtpnp2.glaxowellcome.com (firewall-user@firewall2.glaxowellcome.com [192.58.204.209]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAKH9tX22504 for ; Thu, 20 Nov 2003 11:09:55 -0600 Received: by usrtpnp2.glaxowellcome.com; id MAA04008; Thu, 20 Nov 2003 12:09:53 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma001851; Thu, 20 Nov 03 12:07:29 -0500 Received: by ussunt3.glaxo.com id MAA09572; Thu, 20 Nov 2003 12:07:27 -0500 (EST) Received: from 166.71.134.118 by us9n54.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Thu, 20 Nov 2003 12:07:26 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hAKH7PP27875 for ; Thu, 20 Nov 2003 12:07:26 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Thu, 20 Nov 2003 12:08:05 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 11/20/2003 12:07:29 PM MIME-Version: 1.0 X-WSS-ID: 13A22B444457586-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Passing views in Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I've now run into this difficulty a couple of times, but I'm still a bit puzzled about it. There are circumstances when I will pass a view as a parameter to a class constructor and things work fine. The callee gets the view and can access it without difficulty. There are other cases where I do (almost) the same thing, the callee gets the view, but a later access to it fails. On entry to the callee, the view is okay. But at some point apparently some "part" of the view is garbage collected, dereferenced, goes out of scope, or whatever and *some* properties of the view become inaccessible (determined by the dreaded internal Python interpreter error exception), though some *other* properties of the view are still accessible. For example, 'len(view)' works, but 'view[0].propertyName' does not. My original thought was that in the callee (constructor for the class being instantiated) all I would need to do was to create a self.view to hold a reference to the incoming view parameter. Then any of that class's methods should have a reliable reference to the view. But this is not sufficient. I have discovered that if instead I do 'self.view = view.copy()', then things go well. What I'm puzzled about is *when* I need to do the copy() and when I do not. I think there are some arcane Python scoping issues here, but I wonder if there are any guidelines (beyond, "If it doesn't work the way you think it should, make a copy."). I'm also puzzled because I don't seem to have the problem with other (i.e., non-view) objects in general. I also wonder if this is in fact what should reasonably be expected, or if there is some possibility of a subtle bug (or at least incompatibility) between the underlying C++ implementation and Python's approach to memory management and scoping. Any thoughts? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Thu Nov 20 13:04:44 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hAKJ4hX30553 for ; Thu, 20 Nov 2003 13:04:43 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003112014043920428 for ; Thu, 20 Nov 2003 14:04:39 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HONZNS00.P2G; Thu, 20 Nov 2003 14:04:40 -0500 Message-ID: <3FBD108C.7080602@wi.mit.edu> Date: Thu, 20 Nov 2003 14:05:48 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com, Metakit mailing list Subject: Re: [Metakit] Passing views in Python References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: My guess is the storage is going out of scope and being garbage collected and thereby closing your views without you knowing about it. You should keep the storage object in scope for your entire application run. Here is some proof of this: >>> storage = metakit.storage() >>> vw = storage.getas("test[I:I]") >>> vw.append(1) 0 >>> del storage >>> len(vw) 1 >>> vw[0].I Traceback (most recent call last): File "", line 1, in ? AttributeError: I >>> I'm of two minds whether I think this is a bug or not. I think it might be onerous for the python wrapper to keep track of all the views/subviews and like being created and used. It should still be in the documentation though :) I'll update my working docs with this tidbit. Brian From ian.fairclough@intelligent-ms.com Thu Nov 20 13:09:46 2003 Received: from scratchy.base.ims ([194.74.0.138]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAKJ9jX30878 for ; Thu, 20 Nov 2003 13:09:46 -0600 Received: from grpware.intelligent-ms.com (mail.intelligent-ms.com [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id hAKIxSl03965 for ; Thu, 20 Nov 2003 18:59:28 GMT Received: from imsgsx6tvznbng ([192.168.200.183]) by grpware.intelligent-ms.com (8.11.2/8.11.2) with SMTP id hAKIo4u14086 for ; Thu, 20 Nov 2003 18:50:04 GMT Message-ID: <05d401c3af9a$2d4d4950$a428fc3e@imsgsx6tvznbng> From: "Ian Fairclough" To: Date: Thu, 20 Nov 2003 19:11:57 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] Re-assigning a c4_View - a quickie Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Just a quick question, if you have the following code : c4_View viewB = viewA.Duplicate(); and then you want to re-assign viewB i.e. viewB = viewC.Duplicate(); What should you call prior to re-assigning viewB to ensure that the "first" viewB is properly destroyed. For example, would the following do it : viewB = viewA.Duplicate(); viewB.RemoveAll (); viewB = viewC.Duplicate(); Thanks in advance for any help anybody can give. Best regards, Ian Fairclough. From jcw@equi4.com Thu Nov 20 13:47:58 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAKJlvX00917 for ; Thu, 20 Nov 2003 13:47:58 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 290E583EF2; Thu, 20 Nov 2003 20:47:50 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id ACB0F83BB0 for ; Thu, 20 Nov 2003 20:47:49 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <3FBD108C.7080602@wi.mit.edu> References: <3FBD108C.7080602@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <6B89FBC9-1B92-11D8-AD3D-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Passing views in Python Date: Thu, 20 Nov 2003 20:47:49 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-3.2 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,SPAM_PHRASE_03_05,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Brian Kelley wrote: > My guess is the storage is going out of scope and being garbage > collected and thereby closing your views without you knowing about it. > You should keep the storage object in scope for your entire > application run. > > Here is some proof of this: > > >>> storage = metakit.storage() > >>> vw = storage.getas("test[I:I]") > >>> vw.append(1) > 0 > >>> del storage > >>> len(vw) > 1 > >>> vw[0].I > Traceback (most recent call last): > File "", line 1, in ? > AttributeError: I > >>> > > I'm of two minds whether I think this is a bug or not. I think it > might be onerous for the python wrapper to keep track of all the > views/subviews and like being created and used. Spot on, I think. When a storage goes out of scope, all data becomes unavailable. Keep in mind that MK uses memory-mapped files usually, so access to data actually goes *straight* to disk in most cases. Closing the file blocks off that access. But MK does not control where view objects are used, it merely tracks reference counts (in the same way as Python does for its own PyObject's). The C++ view objects themselves are not tracked or reachable from the storage object. So at some point, the only way out I could think of is to make views act as being empty. All "rows" continue to exist, they just don't have any properties anymore. Which does indeed make them pretty useless, but at least it leads to well-defined semantics. Views which do come from a storage are "unattached" views, these are therefore autonomous. That's what you get when you use copies. The flip side is that they eat up memory, and have to be allocated and copied in full. The way to avoid problems in Python, is to store the storage object in a variable which is guaranteed to stay around as long as its data needs to be accessed. -jcw From jcw@equi4.com Thu Nov 20 13:59:09 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAKJx8X02130 for ; Thu, 20 Nov 2003 13:59:08 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id B51D583EF2; Thu, 20 Nov 2003 20:59:00 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 453F283BB0 for ; Thu, 20 Nov 2003 20:59:00 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <05d401c3af9a$2d4d4950$a428fc3e@imsgsx6tvznbng> References: <05d401c3af9a$2d4d4950$a428fc3e@imsgsx6tvznbng> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Re-assigning a c4_View - a quickie Date: Thu, 20 Nov 2003 20:58:59 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-3.5 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,SPAM_PHRASE_01_02,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Ian Fairclough wrote: > Just a quick question, if you have the following code : > > c4_View viewB = viewA.Duplicate(); > > and then you want to re-assign viewB i.e. > > viewB = viewC.Duplicate(); > > What should you call prior to re-assigning viewB to ensure that the > "first" > viewB is properly destroyed. For example, would the following do it : > > viewB = viewA.Duplicate(); > viewB.RemoveAll (); > viewB = viewC.Duplicate(); There are two types of destruction. If the view is attached to a storage, the RemoveAll() will make sure all its rows are deleted (on file too, after commit). If all you care about is memory use and object clean-up, then you don't have to do anything. MK uses a technique called "smart pointers" in C++, which automatically manages all reference counts. The line viewB = viewC.Duplicate(); does a number of things: - it creates a new view with copies of what is in viewC - it increments the reference count of that new view - it decrements the refcount of whatever was in viewB - it makes viewB refer to the newly created copy If viewB previously referred to a copy of viewA, and if no other view object refers to it, then the decrement will cause that copy to be cleaned up. This is fully automatic in C++, as long as you stay away from pointers to c4_Views. There is no need whatsoever for these (same for c4_Storage, btw), not even performance-wise because c4_View objects are very very lightweight objects. Enjoy the magic of smart pointers! -jcw From bkelley@wi.mit.edu Thu Nov 20 14:08:44 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hAKK8iX03165; Thu, 20 Nov 2003 14:08:44 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003112015084021319 ; Thu, 20 Nov 2003 15:08:40 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HOO2MG00.U8B; Thu, 20 Nov 2003 15:08:40 -0500 Message-ID: <3FBD1F8C.5020806@wi.mit.edu> Date: Thu, 20 Nov 2003 15:09:48 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler CC: Metakit mailing list Subject: Re: [Metakit] Passing views in Python References: <3FBD108C.7080602@wi.mit.edu> <6B89FBC9-1B92-11D8-AD3D-000A9588127E@equi4.com> In-Reply-To: <6B89FBC9-1B92-11D8-AD3D-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > Brian Kelley wrote: > >> My guess is the storage is going out of scope and being garbage >> collected and thereby closing your views without you knowing about >> it. You should keep the storage object in scope for your entire >> application run. > > > Spot on, I think. > > < snip> > So at some point, the only way out I could think of is to make views > act as being empty. All "rows" continue to exist, they just don't > have any properties anymore. Which does indeed make them pretty > useless, but at least it leads to well-defined semantics. A possible solution to the python wrapper would be to have all views keep a reference to the initial storage or the storage from the view that created it. This would be a minor fix I think. > > The way to avoid problems in Python, is to store the storage object in > a variable which is guaranteed to stay around as long as its data > needs to be accessed. Yep. > > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From gary.h.merrill@gsk.com Fri Nov 21 07:53:50 2003 Received: from usrtpnp2.glaxowellcome.com (firewall-user@firewall2.glaxowellcome.com [192.58.204.209]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hALDroX06565 for ; Fri, 21 Nov 2003 07:53:50 -0600 Received: by usrtpnp2.glaxowellcome.com; id IAA07694; Fri, 21 Nov 2003 08:53:47 -0500 (EST) From: Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma006141; Fri, 21 Nov 03 08:52:25 -0500 Received: by ussunt2.glaxo.com id IAA27040; Fri, 21 Nov 2003 08:52:23 -0500 (EST) Received: from 166.71.134.118 by us9n53.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Fri, 21 Nov 2003 08:52:20 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hALDqJP27511 for ; Fri, 21 Nov 2003 08:52:19 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Fri, 21 Nov 2003 08:52:34 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 11/21/2003 08:52:22 AM MIME-Version: 1.0 X-WSS-ID: 13A0C71E4475319-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Storage persistence in Python (was: Passing views in Python) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Included here is my attempt a reconstructing our recent thread on this subject. Given that my mail system is Lotus Notes, I have no idea how this will appear to you. From my perspective a few workarounds have been suggested that I am happy with. I do also rather like JC's latest suggestion since it removes a burden from the programmer. However (perhaps for backward compatibility reasons and also perhaps for a bit more flexibility), it might be better to introduce a new storage creation method: openpersistentstorage(). I don't know, since I'm unsure of what all the consequences would be an whether it would be more confusing to have to "types" of storage. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 ----- Forwarded by Gary H Merrill/PharmRD/GSK on 21-Nov-2003 08:43 ----- ----- Forwarded by Gary H Merrill/PharmRD/GSK on 21-Nov-2003 08:44 ----- "Jean-Claude Wippler" To: "Brian Kelley" 21-Nov-2003 02:49 cc: gary.h.merrill@gsk.com Subject: Re: [Metakit] Passing views in Python Brian Kelley wrote: > There is a cheap way of keeping the storage alive for the length of > your program. > > make an empty python file like "globals.py" > > import metakit > import globals > storage = globals.storage = metakit.storage(...) > > now it will stay alive for the entire run of your program and be > closed at the end. Not pretty, but fairly effective. You can always > close the storage with > > globals.storage = None > > or > del globals.storage Couldn't the above lead to a trivial wrapper which could perhaps be added to metakit.py, which lets you do: storage = metakit.openstorage(...) ... storage.close() It really would be nice to have this discussion on metakit@equi4.com - for one, I'd like Gordon McMillan, Mk4py's author for all practical purposes, to see this discussion. It would allow me to understand what is best *and* how far we are from implementing that choice. The current behavior is the best in C++ - but of course that does not imply anything for Python. Gary, would you be willing to copy the relevant parts to the mailing list? -jcw "Brian Kelley" To: "Jean-Claude Wippler" 20-Nov-2003 22:41 cc: gary.h.merrill@gsk.com Subject: Re: [Metakit] Passing views in Python There is a cheap way of keeping the storage alive for the length of your program. make an empty python file like "globals.py" import metakit import globals storage = globals.storage = metakit.storage(...) now it will stay alive for the entire run of your program and be closed at the end. Not pretty, but fairly effective. You can always close the storage with globals.storage = None or del globals.storage Brian "Brian Kelley" To: "Jean-Claude Wippler" 20-Nov-2003 22:37 cc: gary.h.merrill@gsk.com Subject: Re: [Metakit] Passing views in Python Jean-Claude Wippler wrote: >> So as long as view.copy() will do the trick, I'm >> inclined to use that approach and live with whatever performance >> consequences there may be with it . If I have to do memory >> management, I >> may as well do it in C :-) and reap corresponding performance >> enhancements. >> But I'm happy with the view.copy() approach and understanding what the >> issues are. > I tend to think of this as resource management and not memory management, although they are related. It is fairly easy to write a proxy for the storage and views that will reference count correctly and have the behavior that you expect. I posted one a while back, I think it had one bug in it though. If you would like it, I'll send the fixed one. It can be used transparently with the metakit interface and has two nice extensions. 1) a view keeps a reference to its storage (and parent view) 2) all views are named, i.e. you can tell if a view was generated from a given table or combinations of tables. > Put the storage objects inside the objects which you want to hold the > storage open, and you'll be all set. No need for cheap shots at C > (which in this context is pretty far off, given how much C++ does > extra with reference counts and clean-up) ... :) I have to agree with Gary here that this isn't really the pythonic way. f = open("test") read = f.read del f f will not get garbage collected because we have a reference to it with the read function. The big difference here is that f has a close function f.close() while the storage object does not. It just turns out that python currently garbage collects and closes the storage object "soon" after it goes out of scope. The jython (JNI) binding to metakit would not. I'd feel much safer with more explicit control over closing a storage. If we could call storage.close() and then the views would get upset I'd be all for that. I actually don't like having to rely on the garbage collector to close the storage, it make me feel dirty inside :) Here is the question, if we had storage.close() would you care if open views kept the storage alive? > > > -jcw > ----- Forwarded by Gary H Merrill/PharmRD/GSK on 21-Nov-2003 08:41 ----- Gary H Merrill DES Centennial Campus Laboratory Venture Building II, 920 Main Campus Drive , Suite 120 20-Nov-2003 12:08 Raleigh, NC 27606 To: metakit@equi4.com cc: Subject: Passing views in Python I've now run into this difficulty a couple of times, but I'm still a bit puzzled about it. There are circumstances when I will pass a view as a parameter to a class constructor and things work fine. The callee gets the view and can access it without difficulty. There are other cases where I do (almost) the same thing, the callee gets the view, but a later access to it fails. On entry to the callee, the view is okay. But at some point apparently some "part" of the view is garbage collected, dereferenced, goes out of scope, or whatever and *some* properties of the view become inaccessible (determined by the dreaded internal Python interpreter error exception), though some *other* properties of the view are still accessible. For example, 'len(view)' works, but 'view[0].propertyName' does not. My original thought was that in the callee (constructor for the class being instantiated) all I would need to do was to create a self.view to hold a reference to the incoming view parameter. Then any of that class's methods should have a reliable reference to the view. But this is not sufficient. I have discovered that if instead I do 'self.view = view.copy()', then things go well. What I'm puzzled about is *when* I need to do the copy() and when I do not. I think there are some arcane Python scoping issues here, but I wonder if there are any guidelines (beyond, "If it doesn't work the way you think it should, make a copy."). I'm also puzzled because I don't seem to have the problem with other (i.e., non-view) objects in general. I also wonder if this is in fact what should reasonably be expected, or if there is some possibility of a subtle bug (or at least incompatibility) between the underlying C++ implementation and Python's approach to memory management and scoping. Any thoughts? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Fri Nov 21 08:44:48 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hALEilX10901 for ; Fri, 21 Nov 2003 08:44:47 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003112109444329061 for ; Fri, 21 Nov 2003 09:44:43 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HOPIAK00.RH2 for ; Fri, 21 Nov 2003 09:44:44 -0500 Message-ID: <3FBE251F.8050104@wi.mit.edu> Date: Fri, 21 Nov 2003 09:45:51 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: gary.h.merrill@gsk.com wrote: >Brian Kelley wrote: > > >>globals.storage = None >> >>or >>del globals.storage >> >> I should point out that using this method to explicitely close the storage is really exploiting python's garbage collector. While it works in Python 2.3 and below it may not work in python 2.4 and above. That's what I meant by a "cheap" solution. However, given the lack of storage.close() there are few alternatives. Brian From gmcm@hypernet.com Fri Nov 21 10:45:42 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hALGjfX24870 for ; Fri, 21 Nov 2003 10:45:41 -0600 Received: from PARANOIA (204.176.40.46) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3) for ; Fri, 21 Nov 2003 11:48:28 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Fri, 21 Nov 2003 11:44:49 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) Reply-to: gmcm@hypernet.com Message-ID: <3FBDFAB1.17153.B7A9830E@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 21 Nov 2003 at 8:52, gary.h.merrill@gsk.com wrote: [keeping storage alive for the length of the program...] I have to say I've never encountered this problem. I normally need storage.commit(), so I got in the habit early of keeping the storage as a module global somewhere. In fact, my typical usage pattern is opposite - I use v = storage.view(name) rather frequently (usually on any user interaction) rather than keep long-lived references to views. Those are temptations to keep long-lived references to rows, which gets you in trouble if routines can delete rows. I'll agree that lack of storage.close is kind of unusual, but you can do that with a simple proxy class. Heck, if you're on a platform that supports memory mapping and your view is large, reopening the storage & getting the view should be cheaper than view.copy(). IOW, in (blissful) ignorance of what you're actually doing, I suspect you're asking for trouble :-). -- Gordon From gmcm@hypernet.com Fri Nov 21 10:45:43 2003 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hALGjgX24873 for ; Fri, 21 Nov 2003 10:45:42 -0600 Received: from PARANOIA (204.176.40.46) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3) for ; Fri, 21 Nov 2003 11:48:29 -0500 From: "Gordon McMillan" To: Metakit mailing list Date: Fri, 21 Nov 2003 11:44:49 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) Reply-to: gmcm@hypernet.com Message-ID: <3FBDFAB1.2690.B7A982AA@localhost> Priority: normal In-reply-to: <3FBE251F.8050104@wi.mit.edu> References: X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 21 Nov 2003 at 9:45, Brian Kelley wrote: > >>globals.storage = None > I should point out that using this method to explicitely close the > storage is really exploiting python's garbage collector. While it works > in Python 2.3 and below it may not work in python 2.4 and above. That's > what I meant by a "cheap" solution. However, given the lack of > storage.close() there are few alternatives. No, that's not gc, that's just ref counting (but maybe that's what you meant). -- Gordon From gary.h.merrill@gsk.com Fri Nov 21 12:49:43 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hALInhX11934 for ; Fri, 21 Nov 2003 12:49:43 -0600 Received: by usrtpnp1.glaxowellcome.com; id NAA06483; Fri, 21 Nov 2003 13:49:41 -0500 (EST) From: Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma006288; Fri, 21 Nov 03 13:47:51 -0500 Received: by ussunt2.glaxo.com id NAA25157; Fri, 21 Nov 2003 13:47:49 -0500 (EST) Received: from 166.71.134.118 by us9n55.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Fri, 21 Nov 2003 13:47:46 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hALIljP02330 for ; Fri, 21 Nov 2003 13:47:45 -0500 (EST) Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Fri, 21 Nov 2003 13:48:43 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 11/21/2003 01:47:49 PM MIME-Version: 1.0 X-WSS-ID: 13A082584268714-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: I just noticed that apparently this was just sent to Gordon rather than the list. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 ----- Forwarded by Gary H Merrill/PharmRD/GSK on 21-Nov-2003 13:47 ----- Gary H Merrill DES Centennial Campus Laboratory Venture Building II, 920 Main Campus Drive , Suite 120 21-Nov-2003 12:42 Raleigh, NC 27606 To: gmcm@hypernet.com cc: Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) I don't think I understand what Gordon is suggesting -- and this is partly because there are still some things I don't understand about Python scoping, reference counting, etc. and partly because there are some things I don't understand about how Metakit does things. However, opening up potentially a fairly large number of storages which you stuff into what are basically global variables just so they don't disappear on you, and leaving those things open (and presumably using memory to at least some degree) doesn't seem to be an especially good practice. You wouldn't normally open a file when you need it and then just leave it open until you terminate the entire application (if, for example, the application manipulates a large number of files). You wouldn't normally do that with a database either. The 'open/use/close' model generally has benefits that the 'open and just leave open' model does not. And this feels like the same kind of thing to me. But perhaps I misunderstand. The particular situation confronting me is where I open an existing storage and get its view. That view (and the storage) are then never changed (think of it as read-only), but a number of other views are then created based on that one and passed to a variety of classes for further processing and display. Some of these views may then be "saved" at some point, but not in that original storage. But many will be transient and only for intermediate purposes. So I'm not really interested in doing a commit() on any of these. Intuitively, it seems as though a storage ought to remain in existence so long as you have a reference to a view that is in any way based on that storage. And the storage ought to go away when you no longer have any references to it (however indirectly). This just seems to be a description of the memory management (object management, reference management) model on which the language is based. If for some reason (having to do with the underlying implementation of some of the objects involved) this is not possible, then these objects just seem to violate that fundamental model. It seems as though in the current situation views and storages fall into this class. I can live with that. But ideally it seems as though it shouldn't be the case. On the other hand, it may be that I'm just not thinking along the right lines here. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Fri Nov 21 13:14:38 2003 Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id hALJEcX15812 for ; Fri, 21 Nov 2003 13:14:38 -0600 Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003112114143400381 for ; Fri, 21 Nov 2003 14:14:34 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HOPUSA00.CC3; Fri, 21 Nov 2003 14:14:34 -0500 Message-ID: <3FBE645D.1090704@wi.mit.edu> Date: Fri, 21 Nov 2003 14:15:41 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gmcm@hypernet.com, Metakit mailing list Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) References: <3FBDFAB1.2690.B7A982AA@localhost> In-Reply-To: <3FBDFAB1.2690.B7A982AA@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: >On 21 Nov 2003 at 9:45, Brian Kelley wrote: > > > >>>>globals.storage = None >>>> >>>> > > > >>I should point out that using this method to explicitely close the >>storage is really exploiting python's garbage collector. While it works >>in Python 2.3 and below it may not work in python 2.4 and above. That's >>what I meant by a "cheap" solution. However, given the lack of >>storage.close() there are few alternatives. >> >> > >No, that's not gc, that's just ref counting (but maybe that's what you >meant). >-- Gordon > > > I always get this particular terminology jumbled when it comes to python. There are two things going on, reference counting and garbage collection. Just to see if I have this straight: When a storage goes out of scope the current version of C Python finalizes the storage object relatively quickly. This is essentially calling the object's __del__ function. It is related to garbage collection in that when an object's reference count goes to zero, it can be finalized and also it can be garbage collected. The point is that python doesn't document when to finalize or garbage collect and one technically shouldn't make assumptions about when this is going to happen, just that it will. I've been raked over the coals on this particular subject :) The following pythonic idiom doesn't work on storage objects: storage = metakit.storage(...) try: # do some work on the storage finally: # ensure that the storage is closed storage.close() So one has to trust the implementation of python to handle the details. For all intents and purposes CPython 2.3 will close down the storage for you 'very soon' after the storage goes out of scope (or the refcount goes to 0). You mileage may very if you are using jython for instance. >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > From gmcm@hypernet.com Fri Nov 21 14:39:12 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hALKdCX27308 for ; Fri, 21 Nov 2003 14:39:12 -0600 Received: from PARANOIA (204.176.40.78) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3) for ; Fri, 21 Nov 2003 15:41:59 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Fri, 21 Nov 2003 15:38:20 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) Reply-to: gmcm@hypernet.com Message-ID: <3FBE316C.1864.B87F4C10@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 21 Nov 2003 at 13:48, gary.h.merrill@gsk.com wrote: > I don't think I understand what Gordon is suggesting -- and this is > partly because there are still some things I don't understand about > Python scoping, reference counting, etc. and partly because there are > some things I don't understand about how Metakit does things. > > However, opening up potentially a fairly large number of storages which > you stuff into what are basically global variables just so they don't > disappear on you, and leaving those things open (and presumably using > memory to at least some degree) doesn't seem to be an especially good > practice. You wouldn't normally open a file when you need it and then > just leave it open until you terminate the entire application (if, for > example, the application manipulates a large number of files). That depends on the size of the file, the number of times you need something from it, the relative expense of memory vs file handles. > You > wouldn't normally do that with a database either. Huh? In a typical RDB the expense of opening is someplace between large and huge. > The 'open/use/close' > model generally has benefits that the 'open and just leave open' model > does not. And this feels like the same kind of thing to me. But > perhaps I misunderstand. Memory mapped files typically use the OS's swap mechanism. Keeping it open just tells the OS to manage the "real" vs "virtual" memory usage - which it does more efficiently than a programmer can. > The particular situation confronting me is where I open an existing > storage and get its view. That view (and the storage) are then never > changed (think of it as read-only), but a number of other views are then > created based on that one and passed to a variety of classes for further > processing and display. Some of these views may then be "saved" at some > point, but not in that original storage. But many will be transient and > only for intermediate purposes. So I'm not really interested in doing a > commit() on any of these. > > Intuitively, it seems as though a storage ought to remain in existence > so long as you have a reference to a view that is in any way based on > that storage. And the storage ought to go away when you no longer have > any references to it (however indirectly). This just seems to be a > description of the memory management (object management, reference > management) model on which the language is based. If for some reason > (having to do with the underlying implementation of some of the objects > involved) this is not possible, then these objects just seem to violate > that fundamental model. It seems as though in the current situation > views and storages fall into this class. I can live with that. But > ideally it seems as though it shouldn't be the case. > > On the other hand, it may be that I'm just not thinking along the right > lines here. I can understand the surprise factor. But try this: If views held a reference to the storage, then storage = None wouldn't kill the storage. You'd have to have a storage.close() which should either close all views (so you'd have storage -> view references, ie, circular), or perhaps throw an exception if any view is still open, or just silently wait for GC (which would mean we'd have to make storage objects GC containers). You'd still have surprises. And other than making so you don't have to explicitly keep a reference to the storage, it would have no effect on memory or resource consumption except make it harder to directly control it. -- Gordon From gmcm@hypernet.com Fri Nov 21 14:39:13 2003 Received: from hypernet.com (hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hALKdCX27313 for ; Fri, 21 Nov 2003 14:39:13 -0600 Received: from PARANOIA (204.176.40.78) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3) for ; Fri, 21 Nov 2003 15:42:00 -0500 From: "Gordon McMillan" To: Metakit mailing list Date: Fri, 21 Nov 2003 15:38:20 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) Reply-to: gmcm@hypernet.com Message-ID: <3FBE316C.5651.B87F4B8E@localhost> Priority: normal In-reply-to: <3FBE645D.1090704@wi.mit.edu> References: <3FBDFAB1.2690.B7A982AA@localhost> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: On 21 Nov 2003 at 14:15, Brian Kelley wrote: [garbage collection vs ref counting] > I always get this particular terminology jumbled when it comes to > python. There are two things going on, reference counting and garbage > collection. Just to see if I have this straight: > > When a storage goes out of scope the current version of C Python > finalizes the storage object relatively quickly. Immediately. It happens inline. > ... It is related to garbage > collection in that when an object's reference count goes to zero, it can > be finalized and also it can be garbage collected. Not exactly. GC is for collecting unreachable objects which still have ref counts > 0. Usually that's a container where the contained reference the container, but the container has gone out of scope. > The point is that python doesn't document when to finalize or garbage > collect and one technically shouldn't make assumptions about when this > is going to happen, just that it will. I've been raked over the coals > on this particular subject :) Snort. If you rely on GC, you can't even rely on "it will". Personally, if I can't create cycle-less programs, I use weakrefs. > The following pythonic idiom doesn't work > on storage objects: > > storage = metakit.storage(...) > try: > # do some work on the storage > finally: > # ensure that the storage is closed > storage.close() > > So one has to trust the implementation of python to handle the details. > For all intents and purposes CPython 2.3 will close down the storage for > you 'very soon' after the storage goes out of scope (or the refcount > goes to 0). You mileage may very if you are using jython for instance. storage = None works fine there. GC will never be invoked in this case (there has to be a container involved). In jython you'd still need to say storage = None for GC to realize it was collectable. -- Gordon From gary.h.merrill@gsk.com Fri Nov 21 15:51:46 2003 Received: from usrtpnp1.glaxowellcome.com (firewall-user@firewall1.glaxowellcome.com [192.58.204.204]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hALLpjX04591 for ; Fri, 21 Nov 2003 15:51:45 -0600 Received: by usrtpnp1.glaxowellcome.com; id QAA12063; Fri, 21 Nov 2003 16:51:44 -0500 (EST) From: Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma010210; Fri, 21 Nov 03 16:50:12 -0500 Received: by ussunt2.glaxo.com id QAA05342; Fri, 21 Nov 2003 16:50:11 -0500 (EST) Received: from 166.71.134.118 by us9n57.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Fri, 21 Nov 2003 16:50:05 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hALLo5P18909 for ; Fri, 21 Nov 2003 16:50:05 -0500 (EST) Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Fri, 21 Nov 2003 16:49:37 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 11/21/2003 04:50:08 PM MIME-Version: 1.0 X-WSS-ID: 13A057073963797-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: It's *really* difficult to participate in this sort of exchange with a Lotus Notes mailer. Bear with me. Gordon says: "If views held a reference to the storage, then storage = None wouldn't kill the storage. You'd have to have a storage.close() which should either close all views (so you'd have storage -> view references, ie, circular), or perhaps throw an exception if any view is still open, or just silently wait for GC (which would mean we'd have to make storage objects GC containers). You'd still have surprises." Certainly a mechanism could be designed whereby storage.close() would close all views even while the storage has references to the views and the views have references to the storage. Or alternatively, throwing an exception together with a method for returning the list of all open views would be a workable approach. I don't think that in either case you would have what would be surprises since this behavior would be absolutely uniform, predictable, and documented. As it is, the documentation really needs to contain a caveat along the lines of "Warning: Storages and views don't behave as you would normally expect Python objects to.". Now the consequences of this *are* a surpise -- as evidence for which note the current discussion and the fact that at least some of its participants were surprised. A response may be along the lines that these things really do behave as Python objects do -- so long as you understand a great many of the details of reference counting, GC, etc. But this is in the tradition of arguments that Lotus developers and trainers give in arguing that Notes really does work in a coherent and reasonable manner -- once you grasp its design philosophy and inner beauty :-). To me, the bottom line is that (ideology aside) I seem to need to treat Metakit storages and views differently than I treat other "normal" Python objects. And I need to take special care in handling these in ways that I don't need to take special care in handling other Python objects. As I say, I can live with this. But I would prefer a situation that is uniform, predictable and can be documented so that the average programmer who just wants to use Metakit (rather than understand its implementation details) can use it with confidence and without surprises. Compare this to, for example, BSDDB. In many ways Metakit is much easier to use that BSDDB, but BSDDB does not seem to have such surprises. If at all possible, I would prefer a solution in Metakit that yielded similar results. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From jcw@equi4.com Fri Nov 21 16:29:11 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hALMTBX09884 for ; Fri, 21 Nov 2003 16:29:11 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id 63CE683EF6; Fri, 21 Nov 2003 23:29:03 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 0C8A383BB0 for ; Fri, 21 Nov 2003 23:29:03 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: References: Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <1C1C7F18-1C72-11D8-93C5-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Storage persistence in Python (was: Passing views in Python) Date: Fri, 21 Nov 2003 23:29:03 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-2.6 required=5.0 tests=AWL,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: wrote: > To me, the bottom line is that (ideology aside) I seem to need to treat > Metakit storages and views differently than I treat other "normal" > Python > objects. And I need to take special care in handling these [...] It does not really help to re-iterate the same over and over again :( I never encounter this issue. All application scenarios I use have a clear model of when data from a storage on file is and is not in use. Keeping lots of storages open sounds a bit unusual to me - like a database of databases. You've been given reasons, and workarounds. > Compare this to, for example, BSDDB. You can't. The issue is derived views, and views not holding on to their storage. It's not a BSDDB vs. MK issue, it's about views, which are not limited to data in MK files. As I said, I'm willing to deal with this and make changes. But I think it would be best for all of us to keep trying to make progress towards that goal - instead of all going for the trenches and digging in. At the C++ level, I don't think a reference to the storage can be added to each view attached to (i.e. produced from) it - it creates a cycle (in C++, not Python). So that options is out, AFAICT. What other options are there? -jcw From jcw@equi4.com Mon Nov 24 08:56:50 2003 Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id hAOEuoX23825; Mon, 24 Nov 2003 08:56:50 -0600 Received: by madder.xs4all.nl (Postfix, from userid 101) id BB88D83EF2; Mon, 24 Nov 2003 15:56:42 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 63EC083BB0; Mon, 24 Nov 2003 15:56:42 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: <69FD1D4F-1E8E-11D8-B51E-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Starkit list , Metakit mailing list From: Jean-Claude Wippler Date: Mon, 24 Nov 2003 15:56:41 +0100 X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-1.5 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] server changes Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.13 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: Just a brief note to let you know that the equi4.com domain is being migrated to another server. This mailing list may act funny or even lose messages for a day or two. I'll post another message when things should be back to normal. If you can't reach me, try jcw@madder.xs4all Better still, append ".nl" to the above ;) -jcw From jcw@equi4.com Tue Nov 25 22:32:01 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id C167DB3A86; Tue, 25 Nov 2003 22:32:01 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id F184583EF2; Tue, 25 Nov 2003 22:31:58 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 5ECD183BB0; Tue, 25 Nov 2003 22:31:58 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: Content-Type: text/plain; charset=US-ASCII; format=flowed To: Starkit list , Metakit mailing list From: Jean-Claude Wippler Date: Tue, 25 Nov 2003 22:31:58 +0100 X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-1.5 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] Re: server changes X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2003 21:32:02 -0000 The equi4.com domain migration to another server is done. Everything looks ok from this end. Please let me know if you see anything fishy: ping times, bad response times, website, email, mailing lists, wiki, anonymous cvs ... anything. -jcw From kbachman@equis.com Tue Nov 25 23:51:33 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from pierre.equis.com (pierre.equis.com [204.246.137.26]) by dizzie.triqs.com (Postfix) with SMTP id 4B8ECB3A8A for ; Tue, 25 Nov 2003 23:51:32 +0100 (CET) Received: from no.name.available by pierre.equis.com [80.190.244.176]) with SMTP; Tue, 25 Nov 2003 15:51:32 -0700 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3B3A6.A9A3C6ED" X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0 Date: Tue, 25 Nov 2003 15:51:28 -0700 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Memory Mapped File Limit Thread-Index: AcOzdKwlXV8SNZpXRAmeLShUiqE4eAAMe84g From: "Kim Bachman" To: Subject: [Metakit] Memory Mapped File Limit X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2003 22:51:33 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01C3B3A6.A9A3C6ED Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable >From some messages in the archive, I assume I'm running into the memory mapped file limit. I have a file that is about 1.1 GB. Is there any way to turn off memory mapped files? My database is not a quarter full and I'm hitting the limit. Is this size of database beyond the intended use of Metakit? Should I be looking elsewhere for my database needs? =20 Thanks, =20 Kim Bachman =20 ------_=_NextPart_001_01C3B3A6.A9A3C6ED Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Message
From some messages in the archive, I assume = I'm running=20 into the memory mapped file limit.  I have a file that is = about=20 1.1 GB.  Is there any way to turn off memory mapped files?  My = database is not a quarter full and I'm hitting the limit.  Is this = size of=20 database beyond the intended use of Metakit?  Should I be looking = elsewhere=20 for my database needs?
 
Thanks,
 
Kim=20 Bachman
 
------_=_NextPart_001_01C3B3A6.A9A3C6ED-- From jcw@equi4.com Wed Nov 26 10:52:47 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 26790B3A86 for ; Wed, 26 Nov 2003 10:52:47 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 0710583EF2; Wed, 26 Nov 2003 10:52:46 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 99CB683BB0 for ; Wed, 26 Nov 2003 10:52:45 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: References: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Message-Id: <4954831C-1FF6-11D8-B51E-000A9588127E@equi4.com> Content-Transfer-Encoding: quoted-printable From: Jean-Claude Wippler Subject: Re: [Metakit] Memory Mapped File Limit Date: Wed, 26 Nov 2003 10:52:46 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-3.5 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,SPAM_PHRASE_01_02,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Nov 2003 09:52:47 -0000 Kim Bachman wrote: > =46rom some messages in the archive, I assume I'm running into=A0the=20= > memory mapped file=A0limit.=A0 I have a file that is about 1.1 GB.=A0 = Is=20 > there any way to turn off memory mapped files?=A0 My database is not a=20= > quarter full and I'm hitting the limit.=A0 Is this size of database=20 > beyond the intended use of Metakit?=A0 Should I be looking elsewhere = for=20 > my database needs? You can compact your database with, in C++, something like: FILE* fp =3D open("newdat.mk", "wb"); storage.SaveTo(fp); close(fp); Then "mv newdat.mk dat.mk" (on Unix). You must be using views with very large numbers of rows to get such bad=20= free space re-use, perhaps with many small strings? Consider using=20 blocked views, they scale a lot better, and re-use free space *much*=20 better. Also, if you have indeed huge amounts of strings, and if there is a lot=20= of repetition in there, consider storing them in a separate view and=20 then store integer indices of the original view. It may sound awkward to have to fiddle with physical representations=20 like this, but I've seen numerous examples of this over the years,=20 occasionally taking up to three orders of magnitude out of the data,=20 and getting huge improvements in performance thrown in as a result. =20 With views, one can hide any amount of tricks beneath a layer,=20 especially when doing more advanced things with c4_CustomViewer. Don't turn off memory mapped files. You'd need a lot more memory, and=20= you will lose performance, in all but a few rare usage scenarios. A=20 250 Mb datafile is not beyond MK's capabilities, but on a 32-bit=20 machine, a 2.5 Gb file is. As I said, the trick is to get redundancy=20 out of the stored data - especially with ints, MK tends to create=20 extremely compact datasets (for example: a million rows with one int in=20= the range 0..10 take 500 Kb). -jcw From rickbking@comcast.net Wed Nov 26 16:57:51 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by dizzie.triqs.com (Postfix) with ESMTP id C8C67B3A86 for ; Wed, 26 Nov 2003 16:57:50 +0100 (CET) Received: from NMSXP1 (bgp01032946bgs.sothfd01.mi.comcast.net[68.41.238.130]) by comcast.net (rwcrmhc11) with SMTP id <2003112615574901300dqhfee>; Wed, 26 Nov 2003 15:57:49 +0000 Message-ID: <019601c3b436$0942eb00$0200a8c0@NMSXP1> From: "Rick King" To: "Metakit Listserve" Date: Wed, 26 Nov 2003 10:57:43 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: [Metakit] "Blocked" views X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Nov 2003 15:57:51 -0000 Hi everyone, Could someone give me a quick explanation of "blocked" views? The HTML help is way too sketchy for me. Or at least point me to another resource where I might get a better explanation? Thanks. -Rick King Southfield MI From jcw@equi4.com Wed Nov 26 17:15:27 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 751D5B3A86 for ; Wed, 26 Nov 2003 17:15:27 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 4748E83EF2; Wed, 26 Nov 2003 17:15:25 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id D109383BB0 for ; Wed, 26 Nov 2003 17:15:24 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <019601c3b436$0942eb00$0200a8c0@NMSXP1> References: <019601c3b436$0942eb00$0200a8c0@NMSXP1> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] "Blocked" views Date: Wed, 26 Nov 2003 17:15:22 +0100 To: Metakit Listserve X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-3.5 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,GAPPY_TEXT,IN_REP_TO, QUOTED_EMAIL_TEXT,REFERENCES,SPAM_PHRASE_01_02, USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Nov 2003 16:15:27 -0000 Rick King wrote: > Could someone give me a quick explanation of "blocked" views? The HTML > help > is way too sketchy for me. Or at least point me to another resource > where I > might get a better explanation? You want a Python example, presumably. I see that there are none in Python the examples/ dir in the source distro, and Brian Kelley's updated docs at http://jura.wi.mit.edu/people/kelley/tutorial/python.html hasn't covered it yet either. I'll use plain English, and let others come up with accurate Python: * instead of defining a view "blah[a:I,b:S,c:D]", define blah[_B[a:I,b:S,c:D]] * in other words, don't define a view of rows, but a view of views of rows * when you open the view, replace: view = storage.view("blah") with view = storage.view("blah").blocked() * or you can use getas, just make sure the structure is as above * in other words, don't just use the raw view but pass it through blocked() * that's it You cannot mix things. When blocked, never access the unblocked view. You cannot convert data as is, the only way to do so is to copy all data in. In C++ there is a call to insert one view into another (compatible) one, but I think in Python you'll have to copy row by row. -jcw From jyl@mod3.net Wed Nov 26 18:34:44 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id 00A08B3A86 for ; Wed, 26 Nov 2003 18:34:43 +0100 (CET) Received: (qmail 9890 invoked by uid 48); 26 Nov 2003 17:34:42 -0000 Received: from 192.18.42.11 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Wed, 26 Nov 2003 09:34:42 -0800 (PST) Message-ID: <64162.192.18.42.11.1069868082.squirrel@mod3.net> Date: Wed, 26 Nov 2003 09:34:42 -0800 (PST) Subject: Re: [Metakit] 'Blocked' views From: "Jacob Levy" To: In-Reply-To: References: <019601c3b436$0942eb00$0200a8c0@NMSXP1> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Nov 2003 17:34:45 -0000 JC Thanks for the example -- I'm sure I can construct the equivalent C++ code, and if not I'll look through the tests that I'm sure contain some examples. You mentioned that blocked views are advantageous for when you have lots of small strings. The advantages are better reuse of space and more compact storage? What other circumstances would benefit from using blocked views? Is there a (significant) performance penalty using blocked views? Specifically, does it make sense to convert these views to blocked (these come from my schema for e4Graph): #define MK4_GRAPHSTRINGS1_5 "e4GraphStrings[s:S,next:I,flags:I]" #define MK4_GRAPHNAMES1_5 "e4GraphNames[n:S,next:I,flags:I]" #define MK4_GRAPHBINARY1_5 "e4GraphBinary[b:B,next:I,flags:I]" If so, the equivalent blocked ones would be: #define MK4_GRAPHSTRINGS1_5 "e4GraphStrings[_B[s:S,next:I,flags:I]]" #define MK4_GRAPHNAMES1_5 "e4GraphNames[_B[n:S,next:I,flags:I]]" #define MK4_GRAPHBINARY1_5 "e4GraphBinary[_B[b:B,next:I,flags:I]]" and then instead of getting them from the storage with (note, this is C++ code): strings = storage->GetAs(MK4_GRAPHSTRINGS1_5); binary = storage->GetAs(MK4_GRAPHBINARY1_5); names = storage->GetAs(MK4_GRAPHNAMES1_5); I'd do: strings = storage->GetAs(MK4_GRAPHSTRINGS1_5).blocked(); binary = storage->GetAs(MK4_GRAPHBINARY1_5).blocked(); names = storage->GetAs(MK4_GRAPHNAMES1_5).blocked(); and use as before? --JYL > Rick King wrote: > >> Could someone give me a quick explanation of "blocked" views? The HTML >> help >> is way too sketchy for me. Or at least point me to another resource >> where I >> might get a better explanation? > > You want a Python example, presumably. > > I see that there are none in Python the examples/ dir in the source > distro, and Brian Kelley's updated docs at > http://jura.wi.mit.edu/people/kelley/tutorial/python.html hasn't > covered it yet either. > > I'll use plain English, and let others come up with accurate Python: > > * instead of defining a view "blah[a:I,b:S,c:D]", define > blah[_B[a:I,b:S,c:D]] > * in other words, don't define a view of rows, but a view of views of > rows > * when you open the view, replace: > view = storage.view("blah") > with > view = storage.view("blah").blocked() > * or you can use getas, just make sure the structure is as above > * in other words, don't just use the raw view but pass it through > blocked() > * that's it > > You cannot mix things. When blocked, never access the unblocked view. > > You cannot convert data as is, the only way to do so is to copy all > data in. In C++ there is a call to insert one view into another > (compatible) one, but I think in Python you'll have to copy row by row. > > -jcw > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Wed Nov 26 18:42:32 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 5EF2AB3A86 for ; Wed, 26 Nov 2003 18:42:32 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 6D63C83EF2; Wed, 26 Nov 2003 18:42:30 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 0955E83BB0 for ; Wed, 26 Nov 2003 18:42:30 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <64162.192.18.42.11.1069868082.squirrel@mod3.net> References: <019601c3b436$0942eb00$0200a8c0@NMSXP1> <64162.192.18.42.11.1069868082.squirrel@mod3.net> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] 'Blocked' views Date: Wed, 26 Nov 2003 18:42:29 +0100 To: Metakit mailing list X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-3.4 required=5.0 tests=AWL,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT, REFERENCES,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Nov 2003 17:42:32 -0000 Jacob Levy wrote: > Thanks for the example -- I'm sure I can construct the equivalent C++ > code, and if not I'll look through the tests that I'm sure contain some > examples. Look in examples/. It's all there, C++ and Tcl. > You mentioned that blocked views are advantageous for when you have > lots > of small strings. The advantages are better reuse of space and more > compact storage? What other circumstances would benefit from using > blocked > views? Is there a (significant) performance penalty using blocked > views? I'm not going to go into this - your best bet is to measure each case yourself. Look in examples - it has sample code, timing tests, scalability tests, etc. Look also at the link at the bottom of the MK docs page: http://www.equi4.com/mkdocs.html I forgot to mention that before. While MK may not have stellar documentation, we should at least try to make good use of what there is... right? Oh, and look in examples/ in the MK source distribution :) -jcw From jcw@equi4.com Fri Nov 28 00:55:10 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id D17A3B3A86; Fri, 28 Nov 2003 00:55:09 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 26CDB83EF2; Fri, 28 Nov 2003 00:55:07 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id AD90F83BB0; Fri, 28 Nov 2003 00:55:06 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: <2079D5F6-2135-11D8-B51E-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Starkit list , Metakit list From: Jean-Claude Wippler Date: Fri, 28 Nov 2003 00:55:06 +0100 X-Mailer: Apple Mail (2.606) X-Spam-Status: No, hits=-1.5 required=5.0 tests=AWL,SPAM_PHRASE_00_01,USER_AGENT_APPLEMAIL version=2.44 X-Spam-Level: Subject: [Metakit] started organizing some docs X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Nov 2003 23:55:10 -0000 FYI, I've started organizing some docs on the equi4.com website. Most prominent addition is a web conversion of Steve Landers' paper of Tclkit and Starkits. This is a new area for on-site documentation: http://www.equi4.com/papers/ Right now it's mostly scripting/Tcl, but I hope to improve this over time. Pointers to other documentation will, as before, be added to the relevant pages which already exist. Have been sprinkling a few links here and there in the past weeks. Please let me know of any missing links as well as links not present where you'd expect them. -jcw From rcohen@articque.com Fri Nov 28 13:11:33 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id CFED4B3A88 for ; Fri, 28 Nov 2003 13:11:33 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXkYikZt for ; Fri, 28 Nov 2003 13:11:32 +0100 (CET) Received: from bob.articque.com (unknown [195.25.225.74]) by dizzie.triqs.com (Postfix) with ESMTP id 9515BB3A90 for ; Fri, 28 Nov 2003 12:43:48 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id 8858FBF1F for ; Fri, 28 Nov 2003 12:44:22 +0100 (CET) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 07209-21EC12D3; Fri, 28 Nov 2003 12:44:22 +0100 Received: by bob.articque.com (Postfix, from userid 1003) id A77EDBF1F; Fri, 28 Nov 2003 12:44:21 +0100 (CET) Received: from articque.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id 54807BF20 for ; Fri, 28 Nov 2003 12:44:18 +0100 (CET) Message-ID: <3FC734A9.2070306@articque.com> Date: Fri, 28 Nov 2003 12:42:33 +0100 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit list Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.22.0.1; VDF: 6.22.0.52; host: bob) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] cannot hash subviews X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Nov 2003 12:11:34 -0000 Hi, I have a view with subviews : c4_View ax=db.GetAs("ax[px:I,ay[py:I,value:S],ayh[_H:I,_R:I]]"); I would like to hash every subview "ay" with view "ayh" Here is what I do : px(rowx)=idx; foundx=ax.Find(rowx); if (foundx<0) { ax.Add(rowx); foundx=ax.Find(rowx); } ay=ayprop(ax[foundx]); ayh=ayhprop(ax[foundx]); ay=ay.Hash(ayh,1); I found in kitviewer that the column "ayh" is empty. which means that the "ayh" property is not added, only px and ay. Is it normal ? Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jcw@equi4.com Fri Nov 28 13:42:13 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1D8B3B3A8A for ; Fri, 28 Nov 2003 13:42:13 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX15x0qq for ; Fri, 28 Nov 2003 13:42:11 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 07DE1B3A88 for ; Fri, 28 Nov 2003 13:42:11 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 6308383EF2; Fri, 28 Nov 2003 13:42:09 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E33C583BB0 for ; Fri, 28 Nov 2003 13:42:08 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <3FC734A9.2070306@articque.com> References: <3FC734A9.2070306@articque.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <47849564-21A0-11D8-B51E-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] cannot hash subviews Date: Fri, 28 Nov 2003 13:42:08 +0100 To: Metakit list X-Mailer: Apple Mail (2.606) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Nov 2003 12:42:13 -0000 Riccardo Cohen wrote: > c4_View > ax=db.GetAs("ax[px:I,ay[py:I,value:S],ayh[_H:I,_R:I]]"); > > I would like to hash every subview "ay" with view "ayh" Hm... never tried that. > Here is what I do : > > px(rowx)=idx; > foundx=ax.Find(rowx); > if (foundx<0) > { > ax.Add(rowx); > foundx=ax.Find(rowx); > } > ay=ayprop(ax[foundx]); > ayh=ayhprop(ax[foundx]); > ay=ay.Hash(ayh,1); > > I found in kitviewer that the column "ayh" is empty. which means that > the "ayh" property is not added, only px and ay. Looks ok to me. Weird - I can't see why this wouldn't work. But why does this? Why not: ax[px:I,py:I,value:S],axh[_H:I,_R:I] and then: ax = ax.Hash(axh,2) That way you get hashing for everything, not just ay? (if column sizes / row counts are an issue, consider combining blocked and hashed) -jcw From giva@bgnett.no Fri Nov 28 18:02:29 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A91DDB3A89 for ; Fri, 28 Nov 2003 18:02:29 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXNQj4TY for ; Fri, 28 Nov 2003 18:02:28 +0100 (CET) Received: from mail.broadpark.no (mail.broadpark.no [217.13.4.2]) by dizzie.triqs.com (Postfix) with ESMTP id 821AAB3A88 for ; Fri, 28 Nov 2003 18:02:28 +0100 (CET) Received: from pcgv (4.80-202-225.nextgentel.com [80.202.225.4]) by mail.broadpark.no (Postfix) with SMTP id 4BFD27893F for ; Fri, 28 Nov 2003 18:02:27 +0100 (MET) Message-ID: <0c3e01c3b5d1$6683c590$0600000a@broadpark.no> From: "Gisle Vanem" To: Date: Fri, 28 Nov 2003 18:02:26 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Python troubles X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Nov 2003 17:02:30 -0000 Hi, I'm a complete Python newbie. So this question is probably easy. I installed Mk4 in at g:\MingW32\src\metakit-2.4.9.2, put the mk*.dll files in \python and changed my $PYTHON_LIB_PATH to g:/ProgramFiler/Python.23/lib;g:/MingW32/src/metakit-2.4.9.2/python And cd'ed to 'examples'. But trying to run "python demo.py" failed to find Mk4py. I assume it's looking for Mk4py.dll which should AFAICS be found via my $PYTHON_LIB_PATH. But no luck. Anyone? Using Python 2.3 on Win-XP. Gisle V. # rm /bin/laden /bin/laden: Not found From jcw@equi4.com Sun Nov 30 14:33:49 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 8B70DB3A8A for ; Sun, 30 Nov 2003 14:33:49 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXpwUfsC for ; Sun, 30 Nov 2003 14:33:48 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 3D666B3A88 for ; Sun, 30 Nov 2003 14:33:48 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 564C183EF2; Sun, 30 Nov 2003 14:33:46 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B8B1283BB0 for ; Sun, 30 Nov 2003 14:33:44 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Sun, 30 Nov 2003 14:33:44 +0100 X-Mailer: Apple Mail (2.606) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] blocked view details X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2003 13:33:49 -0000 Added a page with some more details about how Metakit's blocked views work: http://www.equi4.com/mkblocked.html -jcw From bkelley@wi.mit.edu Mon Dec 1 15:02:55 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id AFD0AB3A8A for ; Mon, 1 Dec 2003 15:02:55 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXiSeYqk for ; Mon, 1 Dec 2003 15:02:54 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id ADCC3B3A88 for ; Mon, 1 Dec 2003 15:02:53 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003120109025209634 for ; Mon, 01 Dec 2003 09:02:52 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HP7Z0S00.RX1; Mon, 1 Dec 2003 09:02:52 -0500 Message-ID: <3FCB4A45.3070906@wi.mit.edu> Date: Mon, 01 Dec 2003 09:03:49 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Gisle Vanem Subject: Re: [Metakit] Python troubles References: <0c3e01c3b5d1$6683c590$0600000a@broadpark.no> In-Reply-To: <0c3e01c3b5d1$6683c590$0600000a@broadpark.no> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Dec 2003 14:02:56 -0000 Gisle Vanem wrote: >Hi, I'm a complete Python newbie. So this question is probably >easy. > >I installed Mk4 in at g:\MingW32\src\metakit-2.4.9.2, >put the mk*.dll files in \python and changed my >$PYTHON_LIB_PATH to >g:/ProgramFiler/Python.23/lib;g:/MingW32/src/metakit-2.4.9.2/python > >And cd'ed to 'examples'. But trying to run "python demo.py" failed to find >Mk4py. I assume it's looking for Mk4py.dll which should AFAICS be found >via my $PYTHON_LIB_PATH. But no luck. Anyone? > That should be $PYTHONPATH type python -h to get a full list of options. > >Using Python 2.3 on Win-XP. > >Gisle V. > ># rm /bin/laden >/bin/laden: Not found > >_____________________________________________ >Metakit mailing list - Metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > From bkelley@wi.mit.edu Mon Dec 1 17:06:14 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A515DB3A8A for ; Mon, 1 Dec 2003 17:06:14 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXwFXsGx for ; Mon, 1 Dec 2003 17:06:09 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id DFEB2B3A88 for ; Mon, 1 Dec 2003 17:06:08 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003120111060711577 for ; Mon, 01 Dec 2003 11:06:07 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HP84Q700.4TJ; Mon, 1 Dec 2003 11:06:07 -0500 Message-ID: <3FCB6728.7030801@wi.mit.edu> Date: Mon, 01 Dec 2003 11:07:04 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Tim Churches , metakit@equi4.com References: <006e01c3b321$e2fa1a00$a300a8c0@emilio> In-Reply-To: <006e01c3b321$e2fa1a00$a300a8c0@emilio> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Blocked view documentation update. X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Dec 2003 16:06:15 -0000 We just had a discussion on the mailing list about blocked views. I have just updated my documentation http://jura.wi.mit.edu/people/kelley/tutorial/python.html here is a small sample code loading 1,000,000 entries (just a integer and corresponding binary data) I hadn't really used blocked views before, but the results are very impressive. The example I give orders the view on the first integer key. This is metakit speak for the first key is an ordered property and lookups are nlogn on that key. First the output: 46.3960000277 seconds to load 1000000 entries 0.0600000619888 seconds to lookup up 1000 random entries or 6.00000619888e-005 seconds per lookup Now the code: import metakit, random, time st = metakit.storage("test.mk", 1) # create a blocked view and # order the view on the first integer value for lookup purposes vw = st.getas("large_view[_B[key:I,data:B]]").blocked().ordered(1) t1 = time.time() for i in range(1000000): vw.append((i, str(i))) if i % 10000 == 0: # commit every 10000 entries print i st.commit() t2 = time.time() st.commit() print (t2-t1), "seconds to load", len(vw), "entries" # now test lookup times lookup = [] size = len(vw) for i in range(1000): lookup.append(int(random.random()*size)) t1 = time.time() for i in lookup: vw.find(key=i) t2 = time.time() print (t2-t1), "seconds to lookup up", len(lookup), "random entries" print "or", (t2-t1)/len(lookup), "seconds per lookup" From rus20376@salemstate.edu Mon Dec 1 19:26:03 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 541CAB3A8A for ; Mon, 1 Dec 2003 19:26:03 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXi2bYoR for ; Mon, 1 Dec 2003 19:25:59 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id EBDB9B3A88 for ; Mon, 1 Dec 2003 19:25:58 +0100 (CET) Received: from salemstate.edu (PC502.bwh.harvard.edu [170.223.248.119]) hB1IQZT05136 for ; Mon, 1 Dec 2003 13:26:35 -0500 (EST) Message-ID: <3FCB87B7.6090808@salemstate.edu> Date: Mon, 01 Dec 2003 13:25:59 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] saving unions of 2 views? (python) X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Dec 2003 18:26:03 -0000 I am pretty new to metakit so hopefully this question isn't too naive. Suppose I have two views and I union them vw=vw.union(another_vw) How do I get the results of this union saved? That is, I don't have to recreate the 2 views and union them every time I want to see the union do I? Thanks!! From rickbking@comcast.net Mon Dec 1 23:40:26 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4EA4AB3A8A for ; Mon, 1 Dec 2003 23:40:26 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXZRFzhK for ; Mon, 1 Dec 2003 23:40:25 +0100 (CET) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by dizzie.triqs.com (Postfix) with ESMTP id D72E4B3A88 for ; Mon, 1 Dec 2003 23:40:24 +0100 (CET) Received: from NMSXP1 (bgp01032946bgs.sothfd01.mi.comcast.net[68.41.238.130]) by comcast.net (rwcrmhc13) with SMTP id <2003120122402301500po828e>; Mon, 1 Dec 2003 22:40:23 +0000 Message-ID: <024801c3b85c$19fda0a0$0200a8c0@NMSXP1> From: "Rick King" To: "Metakit Listserve" Date: Mon, 1 Dec 2003 17:40:16 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] vw.select( lo, hi ) not working X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Dec 2003 22:40:26 -0000 Hi - vw.select( lo, hi ) generates a traceback on my system. The following is copied from PyShell: PyShell 0.9.3 - The Flakiest Python Shell Sponsored by Orbtech - Your source for Python programming expertise. Python 2.3.1 (#47, Sep 23 2003, 23:47:32) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import metakit as mk >>> st=mk.storage() >>> vw=st.getas('test[a:I]') >>> for i in range(10): ... ix=vw.append(a=i) ... >>> mk.dump(vw) a - 0 1 2 3 4 5 6 7 8 9 - Total: 10 rows >>> a=vw.select(2,4) Traceback (most recent call last): File "", line 1, in ? ValueError: Object has no usable attributes >>> Python 2.3.1 Metakit version: 2.4.9.2 What am I missing? -Rick King Southfield MI From bkelley@wi.mit.edu Tue Dec 2 00:06:52 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 7F480B3A8B for ; Tue, 2 Dec 2003 00:06:52 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXO6z404 for ; Tue, 2 Dec 2003 00:06:49 +0100 (CET) Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by dizzie.triqs.com (Postfix) with ESMTP id 60F7BB3A88 for ; Tue, 2 Dec 2003 00:06:49 +0100 (CET) Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.128.15.128]) by comcast.net (sccrmhc12) with SMTP id <2003120123064701200i62jge>; Mon, 1 Dec 2003 23:06:48 +0000 Message-ID: <3FCBCAFF.1020403@wi.mit.edu> Date: Mon, 01 Dec 2003 18:13:03 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Rick King , Metakit mailing list Subject: Re: [Metakit] vw.select( lo, hi ) not working References: <024801c3b85c$19fda0a0$0200a8c0@NMSXP1> In-Reply-To: <024801c3b85c$19fda0a0$0200a8c0@NMSXP1> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Dec 2003 23:06:52 -0000 Rick King wrote: >>>>a=vw.select(2,4) >>>> I think you are using select incorrectly. Select is pretty similar to "find" but it returns a view object instead of an index. a = vw.select(a=2) What are you trying to do? You might be trying to do a = vw[2:4] which is a slice. if you want to return the rows with a between 2 and 4 a = vw.remapwith(vw.filter(lambda row: row >=2 and row < 4)) >>>> >>>> >Traceback (most recent call last): > File "", line 1, in ? >ValueError: Object has no usable attributes > > > >Python 2.3.1 >Metakit version: 2.4.9.2 > >What am I missing? > >-Rick King >Southfield MI > >_____________________________________________ >Metakit mailing list - Metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > From erebus@trendkill.co.uk Tue Dec 2 00:50:21 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 43F81B3A8B for ; Tue, 2 Dec 2003 00:50:21 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXyKZ6lL for ; Tue, 2 Dec 2003 00:50:18 +0100 (CET) Received: from www.sawboss.co.uk (unknown [212.158.206.239]) by dizzie.triqs.com (Postfix) with ESMTP id 6F9FDB3A88 for ; Tue, 2 Dec 2003 00:50:15 +0100 (CET) Received: from trendkill.co.uk (unknown [192.168.0.4]) by www.sawboss.co.uk (Postfix) with ESMTP id 2764312D2A0 for ; Tue, 2 Dec 2003 11:22:28 +0000 (GMT) Message-ID: <3FCBD3D6.2010801@trendkill.co.uk> Date: Mon, 01 Dec 2003 23:50:46 +0000 From: Andrew West User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031013 Thunderbird/0.3 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Add information to subviews X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Dec 2003 23:50:21 -0000 Hi, I'm following this tutorial on metakit, http://support.articque.com/metakit/metakit_tutorial.html but I'm confused about creating subviews (not sure if this is the right term for it) c4_View maps=database.GetAs("maps[mid:S,subview[mwidth:I,mheight:I,mpath:S]]"); c4_View viewsec=database.GetAs("sec[_H:I,_R:I]"); view=view.Hash(viewsec,1); Basically, exactly how do I add information to this view? From previous in the tutorial it gives this example for how to add information. c4_View datas=database.GetAs("datas[did:S,dmapid:S,dpath:S]"); c4_StringProp did("did"),dmapid("dmapid"),dpath("dpath"); c4_Row datarow; did(datarow)="1"; dmapid(datarow)="A"; dpath(datarow)="/home/A1.txt"; datas.Add(datarow); But how do I apply this to the subview? Sorry if this is a simple question or has already been answered before but I can't seem to find a way to search the mailing list archives and the documentation for metakit seems a bit lacking :( all the best, Andrew From rickbking@comcast.net Tue Dec 2 06:03:47 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 93EE5B3A8B for ; Tue, 2 Dec 2003 06:03:47 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXvtBxIe for ; Tue, 2 Dec 2003 06:03:46 +0100 (CET) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by dizzie.triqs.com (Postfix) with ESMTP id 59856B3A8A for ; Tue, 2 Dec 2003 06:03:46 +0100 (CET) Received: from NMSXP1 (bgp01032946bgs.sothfd01.mi.comcast.net[68.41.238.130]) by comcast.net (rwcrmhc12) with SMTP id <200312020503440140079ptge>; Tue, 2 Dec 2003 05:03:44 +0000 Message-ID: <026d01c3b891$a80e3d20$0200a8c0@NMSXP1> From: "Rick King" To: "Metakit Listserve" References: <024801c3b85c$19fda0a0$0200a8c0@NMSXP1> <3FCBCAFF.1020403@wi.mit.edu> Subject: Re: [Metakit] vw.select( lo, hi ) not working Date: Tue, 2 Dec 2003 00:03:38 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 05:03:47 -0000 > Rick King wrote: > > >>>>a=vw.select(2,4) > >>>> > I think you are using select incorrectly. Select is pretty similar to > "find" but it returns a view object instead of an index. > I am very confused now. What I was doing is reading your doc page at http://jura.wi.mit.edu/people/kelley/tutorial/python.html#view.filter and trying out the code samples in PyShell. I realize that select returns a view. The part that doesn't work is this (copied off of your page from the discussion about view.indices): """ The proper way to get this type of subset is: subset = view.select(10,19) indices = view.indices(subset) """ I created a simple view and tried this and got the traceback. All the docs (the Metakit docs, the PyShell completion tip) indicate that I should be able to call select like this, although it does seem a bit odd from a Python point of view because it is overloading the select method. I think I don't understand the docs. BTW, your page has been very helpful - I just wish I'd known about it sooner! Thanks for any help. -Rick From baspey@yahoo.com Tue Dec 2 08:12:18 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 57319B3A8A for ; Tue, 2 Dec 2003 08:12:18 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX8FC7fX for ; Tue, 2 Dec 2003 08:12:16 +0100 (CET) Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by dizzie.triqs.com (Postfix) with SMTP id 14CB1B3A88 for ; Tue, 2 Dec 2003 08:12:16 +0100 (CET) Message-ID: <20031202071215.73464.qmail@web11501.mail.yahoo.com> Received: from [213.36.116.42] by web11501.mail.yahoo.com via HTTP; Mon, 01 Dec 2003 23:12:15 PST Date: Mon, 1 Dec 2003 23:12:15 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] "" and empty strings X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 07:12:19 -0000 I observe a strange behaviour: it's like an empty string is different from "" when evaluated inside a conditionnal expression. snippet: c4_Row Row; c4_StringProp pString("String"); pString(Row) = ""; if ( pString(Row) != "" ) { printf("Problem..."); } Unless I missed something, it looks like a great pitfall for programmers :-( __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ From niki@vintech.bg Tue Dec 2 10:08:14 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id BA9C2B3A8B for ; Tue, 2 Dec 2003 10:08:14 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXpNJsur for ; Tue, 2 Dec 2003 10:08:13 +0100 (CET) Received: from intra.vintech.l (unknown [193.68.201.82]) by dizzie.triqs.com (Postfix) with ESMTP id B84A5B3A88 for ; Tue, 2 Dec 2003 10:08:12 +0100 (CET) Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id 9A14357278 for ; Tue, 2 Dec 2003 11:19:57 +0200 (EET) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id 07EDA23011 for ; Tue, 2 Dec 2003 11:08:09 +0200 (EET) Message-ID: <3FCC559A.1020003@vintech.bg> Date: Tue, 02 Dec 2003 11:04:26 +0200 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: bg, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] vw.select( lo, hi ) not working References: <024801c3b85c$19fda0a0$0200a8c0@NMSXP1> In-Reply-To: <024801c3b85c$19fda0a0$0200a8c0@NMSXP1> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 09:08:15 -0000 Rick King wrote: > Hi - vw.select( lo, hi ) generates a traceback on my system. The following > is copied from PyShell: > > PyShell 0.9.3 - The Flakiest Python Shell > Sponsored by Orbtech - Your source for Python programming expertise. > Python 2.3.1 (#47, Sep 23 2003, 23:47:32) [MSC v.1200 32 bit (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>>>import metakit as mk >>>>st=mk.storage() >>>>vw=st.getas('test[a:I]') >>>>for i in range(10): > > ... ix=vw.append(a=i) > ... > >>>>mk.dump(vw) > > a > - > 0 > 1 > 2 > 3 > 4 > 5 > 6 > 7 > 8 > 9 > - > Total: 10 rows > >>>>a=vw.select(2,4) > > Traceback (most recent call last): > File "", line 1, in ? > ValueError: Object has no usable attributes IIRC this must be vw.select( {'a':2}, {'a':4} ) Niki Spahiev From jcw@equi4.com Tue Dec 2 11:34:34 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 400A6B3A8B; Tue, 2 Dec 2003 11:34:34 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXkWYvjh; Tue, 2 Dec 2003 11:34:32 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id EB13FB3A88; Tue, 2 Dec 2003 11:34:31 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 0E8D883EF2; Tue, 2 Dec 2003 11:34:29 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 4D15983BB0; Tue, 2 Dec 2003 11:34:28 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: <1B016071-24B3-11D8-BAF8-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Starkit list , Metakit list From: Jean-Claude Wippler Date: Tue, 2 Dec 2003 11:34:27 +0100 X-Mailer: Apple Mail (2.606) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Views explained X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 10:34:34 -0000 In an attempt to better explain how MK deals with views and subviews, and how that works in MK's column-oriented design, I've set up two pages with lots of images. Start here: http://www.equi4.com/mkdocs.html These new pages are linked to from the MK documentation page, last section. -jcw From rcohen@articque.com Tue Dec 2 13:56:34 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id BCB4DB3A8B for ; Tue, 2 Dec 2003 13:56:34 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXncqcZs for ; Tue, 2 Dec 2003 13:56:33 +0100 (CET) Received: from bob.articque.com (unknown [195.25.225.74]) by dizzie.triqs.com (Postfix) with ESMTP id C7664B3A88 for ; Tue, 2 Dec 2003 13:56:32 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id 7C1A8BF40 for ; Tue, 2 Dec 2003 13:57:04 +0100 (CET) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 29365-2B69D6E9; Tue, 02 Dec 2003 13:57:04 +0100 Received: by bob.articque.com (Postfix, from userid 1003) id 571A3BF43; Tue, 2 Dec 2003 13:57:04 +0100 (CET) Received: from articque.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id E8DADBF40 for ; Tue, 2 Dec 2003 13:57:00 +0100 (CET) Message-ID: <3FCC8BB2.3020509@articque.com> Date: Tue, 02 Dec 2003 13:55:14 +0100 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Add information to subviews References: <3FCBD3D6.2010801@trendkill.co.uk> In-Reply-To: <3FCBD3D6.2010801@trendkill.co.uk> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.22.0.1; VDF: 6.22.0.54; host: bob) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 12:56:35 -0000 You have the c4_ViewProp which acts like any other prop : c4_ViewProp viewprop("subview"); c4_View aview=viewprop(datas[idx]); When you add the row datarow, the subview is created for you Andrew West wrote: > Hi, > > I'm following this tutorial on metakit, > > http://support.articque.com/metakit/metakit_tutorial.html > > but I'm confused about creating subviews (not sure if this is the right > term for it) > > c4_View > maps=database.GetAs("maps[mid:S,subview[mwidth:I,mheight:I,mpath:S]]"); > c4_View viewsec=database.GetAs("sec[_H:I,_R:I]"); > view=view.Hash(viewsec,1); > > Basically, exactly how do I add information to this view? > > From previous in the tutorial it gives this example for how to add > information. > > c4_View datas=database.GetAs("datas[did:S,dmapid:S,dpath:S]"); > c4_StringProp did("did"),dmapid("dmapid"),dpath("dpath"); > c4_Row datarow; > did(datarow)="1"; > dmapid(datarow)="A"; > dpath(datarow)="/home/A1.txt"; > datas.Add(datarow); > > But how do I apply this to the subview? > > Sorry if this is a simple question or has already been answered before > but I can't seem to find a way to search the mailing list archives and > the documentation for metakit seems a bit lacking :( > > all the best, > > Andrew > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From bkelley@wi.mit.edu Tue Dec 2 14:37:32 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 5D25BB3A8B for ; Tue, 2 Dec 2003 14:37:32 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX3cOynX for ; Tue, 2 Dec 2003 14:37:31 +0100 (CET) Received: from sccrmhc11.comcast.net (sccrmhc11.comcast.net [204.127.202.55]) by dizzie.triqs.com (Postfix) with ESMTP id 3A77AB3A88 for ; Tue, 2 Dec 2003 14:37:31 +0100 (CET) Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.128.15.128]) by comcast.net (sccrmhc11) with SMTP id <2003120213373001100nppq2e>; Tue, 2 Dec 2003 13:37:30 +0000 Message-ID: <3FCC9711.3040908@wi.mit.edu> Date: Tue, 02 Dec 2003 08:43:45 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Rick King Subject: Re: [Metakit] vw.select( lo, hi ) not working References: <024801c3b85c$19fda0a0$0200a8c0@NMSXP1> <3FCBCAFF.1020403@wi.mit.edu> <026d01c3b891$a80e3d20$0200a8c0@NMSXP1> In-Reply-To: <026d01c3b891$a80e3d20$0200a8c0@NMSXP1> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit Listserve X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 13:37:32 -0000 Rick King wrote: >""" >The proper way to get this type of subset is: > > subset = view.select(10,19) > indices = view.indices(subset) >""" > > I did some more testing and view.select seems horribly broken in this regard. I can only get it to work if the view has one column and select(2,4) returns all rows between 0 and 4 (that is the low value always appears to be set to 0). Interestingly, my version of your code works except for the low bug I just described. import metakit st = metakit.storage() vw = st.getas("test[a:I]") for i in range(10): vw.append(a=i) metakit.dump(vw) a=vw.select(3,6) metakit.dump(a) I suggest not using it this way given the bugs I've described :) I'm going to put a recommendation in the annotated docs not to use this form currently. Although there are some reasons why it would be nice if it worked. i.e. subset = vw[0:10] indices = vw.indices(subset) -> doesn't work >I created a simple view and tried this and got the traceback. All the docs >(the Metakit docs, the PyShell completion tip) indicate that I should be >able to call select like this, although it does seem a bit odd from a Python >point of view because it is overloading the select method. I think I don't >understand the docs. > >BTW, your page has been very helpful - I just wish I'd known about it >sooner! > They have only existed for a month or so. I'm always looking for updates and corrections by the way. Brian From gmcm@hypernet.com Tue Dec 2 15:30:14 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6DD0BB3A8B for ; Tue, 2 Dec 2003 15:30:14 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXtdPTf4 for ; Tue, 2 Dec 2003 15:30:12 +0100 (CET) Received: from hypernet.com (hypernet.com [204.176.40.2]) by dizzie.triqs.com (Postfix) with ESMTP id 7DA9AB3A88 for ; Tue, 2 Dec 2003 15:30:12 +0100 (CET) Received: from PARANOIA (204.176.40.65) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Tue, 2 Dec 2003 09:34:46 -0500 From: "Gordon McMillan" To: Brian Kelley Date: Tue, 02 Dec 2003 09:30:03 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] vw.select( lo, hi ) not working Message-ID: <3FCC5B9B.5416.F831C06@localhost> Priority: normal In-reply-to: <3FCC9711.3040908@wi.mit.edu> References: <026d01c3b891$a80e3d20$0200a8c0@NMSXP1> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit Listserve X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gmcm@hypernet.com List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 14:30:14 -0000 On 2 Dec 2003 at 8:43, Brian Kelley wrote: > Rick King wrote: > > >""" > >The proper way to get this type of subset is: > > > > subset = view.select(10,19) > > indices = view.indices(subset) > >""" There is no such form of select. >>> print v.select.__doc__ select(criteria) -- return virtual view with selected rows select(crit_lo, crit_hi) -- select rows in specified range (inclusive) criteria may be keyword args or dictionary >>> The second form requires dictionaries, as Niki said. Mk has to know what property you're attempting to select on. There's no exception for single-property views. -- Gordon http://www.mcmillan-inc.com/ > I did some more testing and view.select seems horribly broken in this > regard. I can only get it to work if the view has one column and > select(2,4) returns all rows between 0 and 4 (that is the low value > always appears to be set to 0). Interestingly, my version of your code > works except for the low bug I just described. > > import metakit > st = metakit.storage() > vw = st.getas("test[a:I]") > for i in range(10): > vw.append(a=i) > > metakit.dump(vw) > a=vw.select(3,6) > metakit.dump(a) > > > I suggest not using it this way given the bugs I've described :) I'm > going to put a recommendation in the annotated docs not to use this form > currently. Although there are some reasons why it would be nice if it > worked. > > i.e. > subset = vw[0:10] > indices = vw.indices(subset) -> doesn't work > > >I created a simple view and tried this and got the traceback. All the > >docs (the Metakit docs, the PyShell completion tip) indicate that I > >should be able to call select like this, although it does seem a bit > >odd from a Python point of view because it is overloading the select > >method. I think I don't understand the docs. > > > >BTW, your page has been very helpful - I just wish I'd known about it > >sooner! > > > They have only existed for a month or so. I'm always looking for > updates and corrections by the way. > > Brian > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From rcohen@articque.com Tue Dec 2 15:56:15 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3CBA7B3A8B for ; Tue, 2 Dec 2003 15:56:15 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXWs4ku6 for ; Tue, 2 Dec 2003 15:56:13 +0100 (CET) Received: from bob.articque.com (unknown [195.25.225.74]) by dizzie.triqs.com (Postfix) with ESMTP id 7AA55B3A88 for ; Tue, 2 Dec 2003 15:56:13 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by bob.articque.com (Postfix) with ESMTP id EA0C9BF24 for ; Tue, 2 Dec 2003 15:56:44 +0100 (CET) Received: from bob.articque.com (localhost [127.0.0.1]) by localhost (AvMailGate-2.0.1.10) id 31526-21FDE34B; Tue, 02 Dec 2003 15:56:44 +0100 Received: by bob.articque.com (Postfix, from userid 1003) id BEEECBF3E; Tue, 2 Dec 2003 15:56:44 +0100 (CET) Received: from articque.com (unknown [192.168.0.110]) by bob.articque.com (Postfix) with ESMTP id EACF9BF24 for ; Tue, 2 Dec 2003 15:56:41 +0100 (CET) Message-ID: <3FCCA7BA.7080502@articque.com> Date: Tue, 02 Dec 2003 15:54:50 +0100 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit list Subject: [Fwd: [Metakit] cannot hash subviews] Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-AntiVirus: checked by AntiVir MailGate (version: 2.0.1.10; AVE: 6.22.0.1; VDF: 6.22.0.54; host: bob) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 14:56:15 -0000 For those who followed this thread, it was my mistake : I set a bad name in one property. Subview is corretly hashed now. -------- Original Message -------- Subject: [Metakit] cannot hash subviews Date: Fri, 28 Nov 2003 12:42:33 +0100 From: Riccardo Cohen Organization: articque To: Metakit list Hi, I have a view with subviews : c4_View ax=db.GetAs("ax[px:I,ay[py:I,value:S],ayh[_H:I,_R:I]]"); I would like to hash every subview "ay" with view "ayh" Here is what I do : px(rowx)=idx; foundx=ax.Find(rowx); if (foundx<0) { ax.Add(rowx); foundx=ax.Find(rowx); } ay=ayprop(ax[foundx]); ayh=ayhprop(ax[foundx]); ay=ay.Hash(ayh,1); I found in kitviewer that the column "ayh" is empty. which means that the "ayh" property is not added, only px and ay. Is it normal ? Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 _____________________________________________ Metakit mailing list - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From bkelley@wi.mit.edu Tue Dec 2 17:45:25 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1F844B3A8B for ; Tue, 2 Dec 2003 17:45:25 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXFV8GDc for ; Tue, 2 Dec 2003 17:45:21 +0100 (CET) Received: from sccrmhc11.comcast.net (sccrmhc11.comcast.net [204.127.202.55]) by dizzie.triqs.com (Postfix) with ESMTP id 54EC2B3A88 for ; Tue, 2 Dec 2003 17:45:21 +0100 (CET) Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.128.15.128]) by comcast.net (sccrmhc11) with SMTP id <2003120216451901100nsdcke>; Tue, 2 Dec 2003 16:45:19 +0000 Message-ID: <3FCCC318.80601@wi.mit.edu> Date: Tue, 02 Dec 2003 11:51:36 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gmcm@hypernet.com Subject: Re: [Metakit] vw.select( lo, hi ) not working References: <026d01c3b891$a80e3d20$0200a8c0@NMSXP1> <3FCC5B9B.5416.F831C06@localhost> In-Reply-To: <3FCC5B9B.5416.F831C06@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit Listserve X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 16:45:25 -0000 Gordon McMillan wrote: >On 2 Dec 2003 at 8:43, Brian Kelley wrote: >There is no such form of select. > > > >>>>print v.select.__doc__ >>>> >>>> >select(criteria) -- return virtual view with selected rows >select(crit_lo, crit_hi) -- select rows in specified range (inclusive) > criteria may be keyword args or dictionary > > > >The second form requires dictionaries, as Niki said. Mk has to >know what property you're attempting to select on. There's no >exception for single-property views. > > That makes more sense. It wasn't very clear from the documentation. The problem was that view.select(0,10) works(tm) on one column views. When I say that it works, I mean it returns a sensible result in some cases. I'll fix my annotated docs. Thanks for the prompt update. Brian From rus20376@salemstate.edu Tue Dec 2 17:59:10 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D50CDB3A8B for ; Tue, 2 Dec 2003 17:59:09 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXno9PUq for ; Tue, 2 Dec 2003 17:59:08 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id A2943B3A88 for ; Tue, 2 Dec 2003 17:59:07 +0100 (CET) Received: from salemstate.edu (PC502.bwh.harvard.edu [170.223.248.119]) hB2GxgT27329 for ; Tue, 2 Dec 2003 11:59:42 -0500 (EST) Message-ID: <3FCCC4DA.5030501@salemstate.edu> Date: Tue, 02 Dec 2003 11:59:06 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit list References: <1B016071-24B3-11D8-BAF8-000A9588127E@equi4.com> In-Reply-To: <1B016071-24B3-11D8-BAF8-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] max db size X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 16:59:10 -0000 I guess this should be a FAQ question but I haven't seen an answer. Is their a practical limit to the size of a metakit db? What is the largest that anyone knows of? Any anecdotes about performance and large db size? Thanks!! From tom.krehbiel@motorola.com Tue Dec 2 18:46:36 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id EA365B3A8B for ; Tue, 2 Dec 2003 18:46:35 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXrZMTka for ; Tue, 2 Dec 2003 18:46:30 +0100 (CET) Received: from motgate5.mot.com (motgate5.mot.com [144.189.100.105]) by dizzie.triqs.com (Postfix) with ESMTP id 3CEDAB3A88 for ; Tue, 2 Dec 2003 18:46:29 +0100 (CET) Received: from az33exr01.mot.com (az33exr01.mot.com [10.64.251.231]) by motgate5.mot.com (Motorola/Motgate5) with ESMTP id hB2HkC6A000542; Tue, 2 Dec 2003 10:46:15 -0700 (MST) Received: from motorola.com (arbok.sps.mot.com [172.16.7.187]) hB2HeKGS001940; Tue, 2 Dec 2003 11:40:21 -0600 Message-ID: <3FCCCECC.4040901@motorola.com> Date: Tue, 02 Dec 2003 10:41:32 -0700 From: Tom Krehbiel Organization: MOTOROLA/SPS/T&M/DMO User-Agent: Mozilla/5.0 (X11; U; HP-UX 9000/785; en-US; rv:1.2) Gecko/20021219 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Adam Russell Subject: Re: [Metakit] max db size References: <1B016071-24B3-11D8-BAF8-000A9588127E@equi4.com> <3FCCC4DA.5030501@salemstate.edu> In-Reply-To: <3FCCC4DA.5030501@salemstate.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 17:46:36 -0000 Adam, Physical limit is that max file size on your file system. Practial limit is a personal preference :) -tjk > I guess this should be a FAQ question but I haven't seen an answer. > Is their a practical limit to the size of a metakit db? > What is the largest that anyone knows of? Any anecdotes about > performance and large db size? > Thanks!! From bkelley@wi.mit.edu Tue Dec 2 18:49:42 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 97327B3A8B for ; Tue, 2 Dec 2003 18:49:42 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXPTBish for ; Tue, 2 Dec 2003 18:49:40 +0100 (CET) Received: from sccrmhc11.comcast.net (sccrmhc11.comcast.net [204.127.202.55]) by dizzie.triqs.com (Postfix) with ESMTP id 8D4FFB3A88 for ; Tue, 2 Dec 2003 18:49:40 +0100 (CET) Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.128.15.128]) by comcast.net (sccrmhc11) with SMTP id <2003120217493701100nt57se>; Tue, 2 Dec 2003 17:49:38 +0000 Message-ID: <3FCCD229.6080904@wi.mit.edu> Date: Tue, 02 Dec 2003 12:55:53 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 Cc: Metakit Listserve Subject: Re: [Metakit] vw.select( lo, hi ) not working: docs updated References: <026d01c3b891$a80e3d20$0200a8c0@NMSXP1> <3FCC5B9B.5416.F831C06@localhost> <3FCCC318.80601@wi.mit.edu> In-Reply-To: <3FCCC318.80601@wi.mit.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 17:49:43 -0000 Okay, I have updated my annotated docs fixing my horribly broken understanding of view.select The pages are located at: http://jura.wi.mit.edu/people/kelley/tutorial/python.html as always, errors and corrections are appreciated. Brian From jcw@equi4.com Tue Dec 2 20:01:57 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 7F1D4B3A88 for ; Tue, 2 Dec 2003 20:01:55 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXQxz7wc for ; Tue, 2 Dec 2003 20:01:54 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id D4010B3A87 for ; Tue, 2 Dec 2003 20:01:53 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 4236483EF2; Tue, 2 Dec 2003 20:01:51 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id DE66783BB0 for ; Tue, 2 Dec 2003 20:01:30 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <3FCCCECC.4040901@motorola.com> References: <1B016071-24B3-11D8-BAF8-000A9588127E@equi4.com> <3FCCC4DA.5030501@salemstate.edu> <3FCCCECC.4040901@motorola.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] max db size Date: Tue, 2 Dec 2003 20:01:30 +0100 To: Metakit list X-Mailer: Apple Mail (2.606) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 19:01:58 -0000 Adam Russell asks: > Is their a practical limit to the size of a metakit db? Short answer: YMMV. Long answer: see below. Tom Krehbiel wrote: > Physical limit is that max file size on your file system. I'm afraid that is not quite right. Physical limit is what fits into the address space as memory mapped files - on 32-bit address machines, it may be up to 2 Gb, but a practical limit is probably closer to 1 Gb. On 64-bit machine, the limits go up - but nothing will get rid of the need to analyze the specs and carefully design with the strengths and weaknesses of MK's column-wise storage in mind. There are tricks to store unlimited amounts of data, by using MK *not* to store all data but only parts of it, and then have it *manage* data storage for the rest. A simplistic version of that would be: store really big items in separate files, and store the filename in MK. With such an approach, there are no real limits, evidently. But above all else: the key trick is to take redundancy out of the data and reduce data set sizes. That can have effects of an order of magnitude on database size, sometimes even more - especially when recoding repetitive strings as separate indices into string tables. I've been tempted a few times to write a wrapper view in MK which automates these tricks (both the external file storage and the "lookup table" ide), to hide them all behind a view layer. So far it hasn't happened, and no project has come up to let me justify working on this. Let me just finish by saying that I would not rule out MK for any amount of data - it obviously depends on requirements, usage scenario's, and acceptable development effort to make it happen. -jcw From gary.h.merrill@gsk.com Wed Dec 3 16:21:11 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 9A488B3A95 for ; Wed, 3 Dec 2003 16:21:11 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXwXeAl8 for ; Wed, 3 Dec 2003 16:21:00 +0100 (CET) Received: from usrtpnp1.glaxowellcome.com (firewall1.glaxowellcome.com [192.58.204.204]) by dizzie.triqs.com (Postfix) with ESMTP id 416C6B3A86 for ; Wed, 3 Dec 2003 16:21:00 +0100 (CET) Received: by usrtpnp1.glaxowellcome.com; id KAA11302; Wed, 3 Dec 2003 10:20:58 -0500 (EST) From: Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma010404; Wed, 3 Dec 03 10:19:45 -0500 Received: by ussunt2.glaxo.com id KAA09933; Wed, 3 Dec 2003 10:19:42 -0500 (EST) Received: from 166.71.134.118 by us9n54.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Wed, 03 Dec 2003 10:19:34 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hB3FJXP06290 for ; Wed, 3 Dec 2003 10:19:33 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Wed, 3 Dec 2003 10:20:27 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 12/03/2003 10:19:37 AM MIME-Version: 1.0 X-WSS-ID: 13D3208F6217415-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Clustering and metakit views X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2003 15:21:11 -0000 This isn't really a Metakit questioin, but I wonder if anyone on the list is running any clustering routines on Mk databases/views. I'm looking around for approaches to clustering and displaying (preferably in a graphical way) the results of my text and data mining, and if effective (and available) things of this kind have already been written it would same me some time. I can export my results and use external products (SAS, JMP, ...), but I'm wondering if something somewhat simpler and more direct (particularly in Python) is available. Any information will be appreciated. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Wed Dec 3 17:02:37 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 33024B3A95 for ; Wed, 3 Dec 2003 17:02:37 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX3l3avE for ; Wed, 3 Dec 2003 17:02:35 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id D40F6B3A86 for ; Wed, 3 Dec 2003 17:02:34 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003120311023304478 for ; Wed, 03 Dec 2003 11:02:33 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HPBTW900.SKM; Wed, 3 Dec 2003 11:02:33 -0500 Message-ID: <3FCE0950.9050609@wi.mit.edu> Date: Wed, 03 Dec 2003 11:03:28 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com Subject: Re: [Metakit] Clustering and metakit views References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2003 16:02:37 -0000 gary.h.merrill@gsk.com wrote: >This isn't really a Metakit questioin, but I wonder if anyone on the list >is running any clustering routines on Mk databases/views. I'm looking >around for approaches to clustering and displaying (preferably in a >graphical way) the results of my text and data mining, and if effective >(and available) things of this kind have already been written it would same >me some time. > >I can export my results and use external products (SAS, JMP, ...), but I'm >wondering if something somewhat simpler and more direct (particularly in >Python) is available. > You might be interested in the orange clustering libraries and some other links here http://ai.fri.uni-lj.si/~aleks/orng/ They have GUI components, but they all use Qt which I am not very fond of due to licensing issues. There is also a very robust python/R/S+ interface. S+ has a TON of datamining capabilities. Just google for python S+. I have a home-grown self organizing map that I use with metakit views. Brian Kelley From pyguy30@yahoo.com Wed Dec 3 17:32:39 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 7682FB3A96 for ; Wed, 3 Dec 2003 17:32:39 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXb0qv6m for ; Wed, 3 Dec 2003 17:32:38 +0100 (CET) Received: from web11508.mail.yahoo.com (web11508.mail.yahoo.com [216.136.172.40]) by dizzie.triqs.com (Postfix) with SMTP id 41022B3A86 for ; Wed, 3 Dec 2003 17:32:37 +0100 (CET) Message-ID: <20031203163236.88289.qmail@web11508.mail.yahoo.com> Received: from [192.131.99.117] by web11508.mail.yahoo.com via HTTP; Wed, 03 Dec 2003 08:32:36 PST Date: Wed, 3 Dec 2003 08:32:36 -0800 (PST) From: j n To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] meta_safe: a class for multiple safe readers/one writer +locking X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2003 16:32:39 -0000 Earlier there was discussion of using a combination of extend and aside to enable multiple readers and one writer. Here is a class that manages it for you: meta_safe. You access the storage object as an attribute with meta_safe and have meta_safe manage the extend and aisde isses. It also has a simple cross-platform method of 2-level locking based of of the assumption that mkdir is one successful once. If you want to manage locks yourself, you can eaisly turn off locking. The locking I call gentleman's locking. It is safe to call unlock even if you do not have the lock(it just won't do anything). And, you only keep your lock for a predermined time. If you want to keep your lock longer, you have to reassert it, before the time runs out. I called it meta_safe since with the combination of locking and extend and aside, it is safer to write to metakit. Here is an example of usage ##### import time,meta_safe dbfile='main.mk' #w means write #open for safe writing and create the view s=meta_safe.safe(dbfile,'w',create='a[i:S]') vw = s.db.view('a') vw.append(i=time.asctime()) s.close() #open for safe writing s=meta_safe.safe(dbfile,'w') vw = s.db.view('a') vw.append(i=time.asctime()) vw.append(i=time.asctime()) vw.append(i=time.asctime()) s.close() #open for reading s=meta_safe.safe(dbfile) vw=s.db.view('a') print len(vw) for item in vw: print item.i s.unsafe_close() #if you do not want locking set use_lock to 0 s=meta_safe.safe(dbfile,'w',create='a[i:S]',use_lock=0) ####meta_safe.py class#### import metakit,os,time #Version 0.1 #John Nielsen import metakit,os,time class lock: '''cross-platform locking. Locking will raise exceptions. Unlocking won't. So, unlock all you want''' def __init__(self,*args,**kwds): if not len(args): raise 'need a lock name' #detemine how long to wait for locks(min) if 'wait' in kwds: self.wait=kwds['wait']*60 else: self.wait=300 name=args[0] self.d=name+'_lock/' self.d2=name+'_lock2/' self.locked={} lock_successful=0 if self.locked.has_key(self.d2): try: #you got a lock, it is yours? if os.stat(self.d2)[8]==\ self.locked.get(self.d2): #try to extend lock before loss #ATOMIC operation, extend may #fail presence of self.d will #prevent loss of lock #just by the act of extending os.rmdir(self.d2) os.mkdir(self.d2) os.rmdir(self.d) os.mkdir(self.d) self.locked[self.d2]=\ os.stat(self.d2)[8] lock_successful=1 if self.locked.has_key(self.d2): del(self.locked[self.d2]) result='Fail: lost lock' except: if self.locked.has_key(self.d2): del(self.locked[self.d2]) result='Fail: lost lock' else: for t in range(0,self.wait): #try 10 times #not locked yet, try to lock it try: os.mkdir(self.d) #ATOMIC os.mkdir(self.d2) #ATOMIC self.locked[self.d2]=\ os.stat(self.d2)[8] lock_successful=1 break except Exception,error: result=error print 'locking??',error #mkdir probably failed #lock already there, #try to delete old lock m_dir2=0;m_dir=0 if os.path.exists(self.d): try: m_dir2=os.stat(self.d2)[8] except: pass try: m_dir=os.stat(self.d)[8] except: pass cur_tm=int(time.time()) #presence of either directory #can stop you from taking lock if cur_tm>m_dir+self.wait and\ cur_tm>m_dir2+self.wait: #delete old locks #ATOMIC here os.rmdir(self.d2) os.mkdir(self.d2) os.rmdir(self.d) os.mkdir(self.d) self.locked[self.d2]=\ os.stat(self.d2)[8] lock_successful=1 break time.sleep(1) #made it thru the loop, so we got no lock if not lock_successful: raise result def unlock(self): '''does not raise an exception, safe to unlock as often as you want it may just do nothing''' if self.locked.has_key(self.d2): #we're the ones that unlocked it, #if time matched if self.locked[self.d2]==\ os.stat(self.d2)[8]: try: del(self.locked[self.d2]) os.rmdir(self.d2) os.rmdir(self.d) return 1 except: return 0 else: del(self.locked[self.d2]) return 0 else: return 0 class safe: '''manage aside and extend automatically to allow multiple safe readers and one writer''' def __init__(self,*args,**kwds): self.use_lock=1 self.l=None#locking placeholders if 'lock' in kwds: self.use_lock=kwds['lock'] if 'store' in kwds: #must end in / or \ if kwds['store_dir'][-1] in ('/','\\'): self.store_dir=kwds['store_dir'] else: raise 'error: need / or \ to end store' else: self.store='' if len(args)==1: #read-only self.flag='r' self.dbfile=self.store+args[0] elif len(args)>1: self.dbfile=args[0] self.flag=args[1] if self.flag not in ('r','w'): raise 'error: use r or w' self.dbaside=self.dbfile+'a' if 'create' in kwds: if self.use_lock: l=lock(self.dbfile)##########LOCKING db = metakit.storage(self.dbfile, 1) vw=db.getas(kwds['create']) db.commit() del db if self.use_lock: l.unlock() ############UNLOCKING self.db=self.dba=None ## if self.flag=='r': self.db = metakit.storage(self.dbfile, 0) if os.path.exists(self.dbaside): self.dba =\ metakit.storage(self.dbaside, 0) elif self.flag=='w': if self.use_lock: self.l=lock(self.dbfile)######LOCKING self.db = metakit.storage(self.dbfile, 0) self.dba = metakit.storage(self.dbaside, 2) self.db.aside(self.dba) vw=self.db.view('a') def close(self): if self.flag=='w': self.db.commit() self.dba.commit() ############UNLOCKING if self.use_lock: if self.l: self.l.unlock() del self.db del self.dba def unsafe_close(self): l=lock(self.dbfile)##########LOCKING #synchronize aside and main db = metakit.storage(self.dbfile, 1) dba = metakit.storage(self.dbaside, 1) db.aside(dba) db.commit(1) del db del dba l.unlock() if __name__ == "__main__": print 'testing' dbfile='main.mk' #create is only needed if the view doesn't exist s=safe(dbfile,'w',create='a[i:S]',lock=1) vw = s.db.view('a') vw.append(i=time.asctime()) vw.append(i=time.asctime()) vw.append(i=time.asctime()) s.close() s=safe(dbfile) vw=s.db.view('a') print len(vw) for item in vw: print item.i s.unsafe_close() __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ From pyguy30@yahoo.com Wed Dec 3 20:20:03 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B4100B3A96 for ; Wed, 3 Dec 2003 20:20:03 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX0TFLf5 for ; Wed, 3 Dec 2003 20:20:02 +0100 (CET) Received: from web11506.mail.yahoo.com (web11506.mail.yahoo.com [216.136.172.38]) by dizzie.triqs.com (Postfix) with SMTP id 2EEECB3A86 for ; Wed, 3 Dec 2003 20:20:02 +0100 (CET) Message-ID: <20031203191955.31179.qmail@web11506.mail.yahoo.com> Received: from [192.131.99.117] by web11506.mail.yahoo.com via HTTP; Wed, 03 Dec 2003 11:19:55 PST Date: Wed, 3 Dec 2003 11:19:55 -0800 (PST) From: j n To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] when to use aside and extend? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2003 19:20:04 -0000 Earlier I implemented a class that does aside and extend. >From the growth of the aside file, it looks like the aside/extend method at this point is really only useful if you change the data a little bit. It seems that if you have a database that changes often, it may actually be more efficient to simply copy the file, prevent others from writing, and make all the changes there and copy it back (having some method to manage the copy part for the readers). Is that true? john __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ From jcw@equi4.com Thu Dec 4 12:36:16 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E9A1CB3A9B for ; Thu, 4 Dec 2003 12:36:15 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXc1eeat for ; Thu, 4 Dec 2003 12:36:14 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id D5443B3A86 for ; Thu, 4 Dec 2003 12:36:13 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 432D783EF2; Thu, 4 Dec 2003 12:36:12 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 6AEEE83BB0 for ; Thu, 4 Dec 2003 12:36:11 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <20031203191955.31179.qmail@web11506.mail.yahoo.com> References: <20031203191955.31179.qmail@web11506.mail.yahoo.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <0EEBCBAE-264E-11D8-BAF8-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] when to use aside and extend? Date: Thu, 4 Dec 2003 12:36:10 +0100 To: Metakit list X-Mailer: Apple Mail (2.606) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 11:36:16 -0000 j n wrote: > Earlier I implemented a class that does aside and > extend. Yes - fantastic, thanks for sharing it! >> From the growth of the aside file, it looks like the > aside/extend method at this point is really only > useful if you change the data a little bit. > > It seems that if you have a database that changes > often, it may actually be more efficient to simply > copy the file, prevent others from writing, and make > all the changes there and copy it back (having some > method to manage the copy part for the readers). The logic of commit-aside is only partly implemented. The idea behind commit-aside is that it would save only differences, which would make it a quick way to save small changes. Unfortunately, the project for which this work was done was cancelled, so I decided to bring out the mechanism, but stopped short of actually implementing binary diff logic. As a result, commit aside stores complete columns for now (the file format is ready for storing changes, it's just that each modified is saved as one "big change", i.e. in full. Commit extend does indeed grow the file rather quickly. This is a consequence of MK's column-wise storage. The goal of c-extend and c-aside is to be able to combine them. At that point, one can store changes with multiple readers and a single writer without any contention, hence no need for locking. This goal is still on my list - in fact I've made some progress in getting closer to it very recently - but it hasn't happened, and I can't predict when it will, at this stage. There is one case where the disk use is not as extreme: by using blocked views, changes tend to make changes in much smaller chunks. In this case, both c-extend and c-aside will be somewhat less disk-hungry. Having said that, commit extend has been in use in production code for some time now. The scenario where its disk usage is not too extreme is when there is just one writer and usually at most one reader. In that approach, once can compact the file regularly, and keep the total file size within reasonable limits. That does imply copying to a new file - so yes, your observation is accurate: with today's c-aside not taking advantage of the diff-storage design, copying to file and replacing the original is sensible. The one thing you gain with c-extend and c-aside is that the time when this is done can be picked independently (say every night, at times of low activity). I'm still quite keen on getting c-aside to work with diffs. The benefits are contention-free multi-user access and the ability to "change" read-only datafiles, e.g. those stored on CD-ROM. The use of a differential second file means one can ship one version of data, and then "apply" modifications by simply opening a second file. Lots of deployment options can be added (sending starkit updates in the Tcl world, for example). -jcw From jcw@equi4.com Thu Dec 4 14:43:08 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 03108B3A9B for ; Thu, 4 Dec 2003 14:43:08 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXkDogFe for ; Thu, 4 Dec 2003 14:43:05 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id B8996B3A86 for ; Thu, 4 Dec 2003 14:43:05 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 0E7C883EF2; Thu, 4 Dec 2003 14:43:04 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id A193283BB0 for ; Thu, 4 Dec 2003 14:43:03 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <0EEBCBAE-264E-11D8-BAF8-000A9588127E@equi4.com> References: <20031203191955.31179.qmail@web11506.mail.yahoo.com> <0EEBCBAE-264E-11D8-BAF8-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] when to use aside and extend? Date: Thu, 4 Dec 2003 14:43:03 +0100 To: Metakit list X-Mailer: Apple Mail (2.606) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 13:43:08 -0000 To follow up on my own email: [... info about commit-extend and commit-aside ...] I forgot to mention that there is a page on the web with more information: http://www.equi4.com/mkcommit.html -jcw From pyguy30@yahoo.com Thu Dec 4 17:52:12 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 5E321B3A9B for ; Thu, 4 Dec 2003 17:52:12 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXLPwj0y for ; Thu, 4 Dec 2003 17:52:10 +0100 (CET) Received: from web11505.mail.yahoo.com (web11505.mail.yahoo.com [216.136.172.37]) by dizzie.triqs.com (Postfix) with SMTP id 86B13B3A86 for ; Thu, 4 Dec 2003 17:52:09 +0100 (CET) Message-ID: <20031204165208.83733.qmail@web11505.mail.yahoo.com> Received: from [192.131.99.117] by web11505.mail.yahoo.com via HTTP; Thu, 04 Dec 2003 08:52:08 PST Date: Thu, 4 Dec 2003 08:52:08 -0800 (PST) From: j n To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] here is a simple extend/aside example X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 16:52:12 -0000 I put this permantently at: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/252494 As mentioned on the list earlier, the combination of extend aside gives you the ability to deside when _you_ want to make an unsafe write. If you do not want to, you just simply extend the aside file. import os,metakit,time #initalize db #storage options: # 0 = open read-only, most recently committed contents # 1 = open in exclusive read-write mode # 2 = open in "commit-extend" mode main='test.mk';aside='test.mka' #initalize if not os.path.exists(main): print 'initalizing' db = metakit.storage(main, 1) db.getas("a[i:S]") db.commit() dba = metakit.storage(aside, 1) db.aside(dba) del db del dba #commits w/extend do not #change old read old_read = metakit.storage(main, 0) old_reada = metakit.storage(aside, 0) old_read.aside(old_reada) vw = old_read.view("a") old_len=len(vw) print 'old length=',len(vw) #now do commits with extend on aside file #main is read-only aside is commit-extend db = metakit.storage(main, 0) dba = metakit.storage(aside, 2) db.aside(dba) vw = db.view("a") vw.append(i=str(time.asctime())) vw.append(i=str(time.asctime())) db.commit() dba.commit() del db del dba #a new read should see the updates #extended in the aside file new_read = metakit.storage(main, 0) new_reada = metakit.storage(aside, 0) new_read.aside(new_reada) vw = new_read.view('a') print "new length(should be %i)="%(old_len+2),len(vw) #the old view should not be changed vw = old_read.view('a') print "old length(should still be %i)="%(old_len),len(vw) #empty aside into main #this is unsafe what extend enables #you to do, is to control when #to do this operation empty_aside=1 if empty_aside: print 'Empty aside into main:', print 'not safe for readers!' db = metakit.storage(main, 1) dba = metakit.storage(aside, 1) db.aside(dba) db.commit(1) main_only = metakit.storage(main, 0) vw=main_only.view('a') print 'main now has',len(vw) del main_only __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ From pknight@elipsan.com Thu Dec 4 19:12:00 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6A5A0B3A9B for ; Thu, 4 Dec 2003 19:12:00 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXYdp8wa for ; Thu, 4 Dec 2003 19:11:58 +0100 (CET) Received: from hammer.elipsan.com (mailgate.elipsan.com [80.177.61.146]) by dizzie.triqs.com (Postfix) with ESMTP id 86B16B3A86 for ; Thu, 4 Dec 2003 19:11:58 +0100 (CET) Received: from budvar.elipsan.com ([192.168.7.63] helo=elipsan.com) by hammer.elipsan.com with esmtp (Exim 4.12) id 1ARxw0-0005XT-00 for metakit@equi4.com; Thu, 04 Dec 2003 18:10:40 +0000 Message-ID: <3FCF78ED.2000105@elipsan.com> Date: Thu, 04 Dec 2003 18:11:57 +0000 From: Pat Knight Organization: elipsan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: en, en-us MIME-Version: 1.0 To: metakit@equi4.com References: <20031204165208.83733.qmail@web11505.mail.yahoo.com> In-Reply-To: <20031204165208.83733.qmail@web11505.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] License X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 18:12:00 -0000 Hi, The Metakit license says I have to include the copyright notice and license text if my product contains "substantial" parts of the software. However, the precompiled DLLs for Windows don't contain the required text. Am I allowed to redistribute them, or do I have to build my own versions incorporating the text? -- Cheers, Pat Tel: 0117 930 9621 (Intl: +44-117-930-9621). From jcw@equi4.com Thu Dec 4 19:26:03 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 851A5B3AA2 for ; Thu, 4 Dec 2003 19:26:03 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXyzNarb for ; Thu, 4 Dec 2003 19:25:58 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id A4B8CB3A86 for ; Thu, 4 Dec 2003 19:25:58 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 646BA83EF2; Thu, 4 Dec 2003 19:25:56 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id EA6E583BB0 for ; Thu, 4 Dec 2003 19:25:55 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) In-Reply-To: <3FCF78ED.2000105@elipsan.com> References: <20031204165208.83733.qmail@web11505.mail.yahoo.com> <3FCF78ED.2000105@elipsan.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <4C9494DA-2687-11D8-BAF8-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] License Date: Thu, 4 Dec 2003 19:25:55 +0100 To: Metakit list X-Mailer: Apple Mail (2.606) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 18:26:03 -0000 Pat Knight wrote: > The Metakit license says I have to include the copyright notice and > license text if my product contains "substantial" parts of the > software. However, the precompiled DLLs for Windows don't contain the > required text. Am I allowed to redistribute them, or do I have to > build my own versions incorporating the text? Yes - feel free to redistribute them. To comply with the copyright/license, you can include the standard blurb in accompanying documentation. Perhaps also include a link to the MK homepage or license page. That way the origin of the software is clear - which is what the MIT license is all about (well, for me anyway). -jcw From pknight@elipsan.com Thu Dec 4 19:30:53 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 72CA2B3AA5 for ; Thu, 4 Dec 2003 19:30:53 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXjkWIck for ; Thu, 4 Dec 2003 19:30:52 +0100 (CET) Received: from hammer.elipsan.com (mailgate.elipsan.com [80.177.61.146]) by dizzie.triqs.com (Postfix) with ESMTP id ECBB1B3A86 for ; Thu, 4 Dec 2003 19:30:51 +0100 (CET) Received: from budvar.elipsan.com ([192.168.7.63] helo=elipsan.com) by hammer.elipsan.com with esmtp (Exim 4.12) id 1ARyEI-0005jT-00 for metakit@equi4.com; Thu, 04 Dec 2003 18:29:34 +0000 Message-ID: <3FCF7D5B.70005@elipsan.com> Date: Thu, 04 Dec 2003 18:30:51 +0000 From: Pat Knight Organization: elipsan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: en, en-us MIME-Version: 1.0 To: Metakit list Subject: Re: [Metakit] License References: <20031204165208.83733.qmail@web11505.mail.yahoo.com> <3FCF78ED.2000105@elipsan.com> <4C9494DA-2687-11D8-BAF8-000A9588127E@equi4.com> In-Reply-To: <4C9494DA-2687-11D8-BAF8-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 18:30:53 -0000 Jean-Claude Wippler wrote: > Pat Knight wrote: > << a query about including the license text>> > > Yes - feel free to redistribute them. To comply with the > copyright/license, you can include the standard blurb in accompanying > documentation. Perhaps also include a link to the MK homepage or > license page. That way the origin of the software is clear - which is > what the MIT license is all about (well, for me anyway). > Thanks, Jean-Claude. -- Cheers, Pat Tel: 0117 930 9621 (Intl: +44-117-930-9621). From mbloore@yahoo.com Fri Dec 5 16:40:54 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 32B69B3AF2 for ; Fri, 5 Dec 2003 16:40:54 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXcJjYr3 for ; Fri, 5 Dec 2003 16:40:43 +0100 (CET) Received: from web14310.mail.yahoo.com (web14310.mail.yahoo.com [216.136.224.60]) by dizzie.triqs.com (Postfix) with SMTP id 1C08DB3A89 for ; Fri, 5 Dec 2003 16:40:43 +0100 (CET) Message-ID: <20031205154041.95808.qmail@web14310.mail.yahoo.com> Received: from [65.95.141.207] by web14310.mail.yahoo.com via HTTP; Fri, 05 Dec 2003 07:40:41 PST Date: Fri, 5 Dec 2003 07:40:41 -0800 (PST) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] MMF when file gets big X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2003 15:40:54 -0000 i know that when a metakit data file is too large, it gets opened for normal i/o instead of being memory mapped. what happens when a file grows too large for memory mapping during a commit? i ask because i am getting hard-to-trace crashes at about the time the file is closed when i store a large amount of data. the data file is newly created, and grows to over 250MB (with a number of Commit calls, but the file stays open), but i don't know what the limit for MMF is on the test system. __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ From mbloore@yahoo.com Fri Dec 5 21:13:53 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 22C50B3AF2 for ; Fri, 5 Dec 2003 21:13:52 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX2DtOgg for ; Fri, 5 Dec 2003 21:13:49 +0100 (CET) Received: from web14302.mail.yahoo.com (web14302.mail.yahoo.com [216.136.173.78]) by dizzie.triqs.com (Postfix) with SMTP id 22AA1B3A89 for ; Fri, 5 Dec 2003 21:13:49 +0100 (CET) Message-ID: <20031205201347.69863.qmail@web14302.mail.yahoo.com> Received: from [65.95.141.207] by web14302.mail.yahoo.com via HTTP; Fri, 05 Dec 2003 12:13:47 PST Date: Fri, 5 Dec 2003 12:13:47 -0800 (PST) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] choosing blocked or not at run time X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2003 20:13:53 -0000 i have a database which is built a row at a time, and may be updated with random insertions. it can get large, so i want to use a blocked view. i gather that blocking is good for updates but may slow traversals of the whole database, which i do often. so, is it reasonable to do something like this: c4_View theView; c4_View raw = storage.GetAs ("stuff[_B[folder:S,files[name:S]]]"); // in reality there is more file data than this. if (readonly) theView = c4_ViewProp ("_B").Get (raw[0]); else theView = raw.Blocked(); and use theView from then on? must 'raw' live as long as theView? would it be possible or sensible to block the 'files' subview? the most extreme case i have encounted so far is 100,000 folders with three or four files each, and the number of folders will grow. hundreds or thousands of folders with thousands of files each is liable to be common. we might even get one folder with millions of files. ===== -- mARK bLOORE __________________________________ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/ From bkelley@wi.mit.edu Fri Dec 5 21:41:51 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 60F31B3AF5 for ; Fri, 5 Dec 2003 21:41:51 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXa5Adn1 for ; Fri, 5 Dec 2003 21:41:49 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 43D36B3A89 for ; Fri, 5 Dec 2003 21:41:49 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2003120515414702327 for ; Fri, 05 Dec 2003 15:41:47 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HPFW5N00.02V; Fri, 5 Dec 2003 15:41:47 -0500 Message-ID: <3FD0EDC0.2090809@wi.mit.edu> Date: Fri, 05 Dec 2003 15:42:40 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: mARK bLOORE , Metakit mailing list Subject: Re: [Metakit] choosing blocked or not at run time References: <20031205201347.69863.qmail@web14302.mail.yahoo.com> In-Reply-To: <20031205201347.69863.qmail@web14302.mail.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2003 20:41:51 -0000 mARK bLOORE wrote: >i have a database which is built a row at a time, and may be updated with random insertions. it >can get large, so i want to use a blocked view. i gather that blocking is good for updates but >may slow traversals of the whole database, which i do often. so, is it reasonable to do something >like this: > > > Warning, python code follows :) Why gather, just time the difference: Adding 100,000 folders to your database (using in-memory storage on windows2000 2ghz ) 0.731000065804 seconds to iterate through blocked view 0.53100001812 seconds to iterate through non-blocked view Same thing for 1000 folders 0.0210000276566 seconds to iterate through blocked view 0.00999999046326 seconds to iterate through non-blocked view I don't think this will really be an issue. However, you can use the following trick: vw = st.getas(view...) bvw = vw.blocked() if len(vw) == 1: iterate through vw[0]._B but remember, you always need to add items to bvw (the blocked view) You can also block subviews which, since you might have a million files might be important vw = st.getas("stuff[_B[folder:S,files[_B[name:S]]]]").blocked() And when you get a files subview use files = vw[0].files.blocked() Brian From pyguy30@yahoo.com Tue Dec 9 20:26:22 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 803D8B3A97 for ; Tue, 9 Dec 2003 20:26:19 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXKdcS5t for ; Tue, 9 Dec 2003 20:26:18 +0100 (CET) Received: from web11504.mail.yahoo.com (web11504.mail.yahoo.com [216.136.172.36]) by dizzie.triqs.com (Postfix) with SMTP id 48D8CB3A87 for ; Tue, 9 Dec 2003 20:26:17 +0100 (CET) Message-ID: <20031209192615.7443.qmail@web11504.mail.yahoo.com> Received: from [192.131.99.117] by web11504.mail.yahoo.com via HTTP; Tue, 09 Dec 2003 11:26:15 PST Date: Tue, 9 Dec 2003 11:26:15 -0800 (PST) From: j n To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] a simple metakit based python dictionary X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Dec 2003 19:26:22 -0000 If you have a big dictionary, you may not want to have to load the whole thing to get access to modify a key. Metakit provides an easy solution to this, if you use it to emulate a dictionary. import meta_dict file='test_dict.mk' a_dict={'a':'b',1:2} print 'starting with' for i in a_dict: print i,a_dict[i] #convert dictionary to a meta_kit based dictionary meta_dict.meta_save(a_dict,file) #write to meta_kit based dictionary m_dict=meta_dict.meta_load(file=file,write=1) #change a member m_dict[1]=5 #add a member m_dict['c']='d' #end writing m_dict.close() #read-only metakit based dictionary m2_dict=meta_dict.meta_load(file=file) print 'ending with' for i in m2_dict: print i,m2_dict[i] _________________ Gives you output: starting with a b 1 2 ending with a b 1 5 c d #meta_dict.py -- version 0.1 John Nielsen from __future__ import generators import metakit,marshal,time #in place update #make copy of most current data, lock it #update pieces class meta_load: def __init__(self,file='',tbl='',write=''): if not tbl: tbl='table0' if write: self.db = metakit.storage(file,1) self.tbl=self.db.getas('%s[k:S,v:B]'%(tbl)) else: self.db = metakit.storage(file,0) self.tbl=self.db.view(tbl) self._set_keys() def _set_keys(self): self.cur_dict={} #get "keys" loc=0 for i in self.tbl: self.cur_dict[i.k]=loc loc+=1 def close(self): self.db.commit() self.db = None def has_key(self, k): return k in self.cur_dict def keys(self): return self.cur_dict.keys() def get(self,key,default=None): try: return self.__getitem__(key) except: return default def items(self): return list(self.iteritems()) def values(self): return list(self.itervalues()) def __len__(self): return len(self.cur_dict.keys()) def __getitem__(self,k): '''import to define''' if k in self.cur_dict: #row_num=self.tbl.find(k=k) row_num=self.cur_dict[k] return marshal.loads(self.tbl[row_num].v) else: raise KeyError, k def iterkeys(self): for key in self.keys(): yield key def __iter__(self): return self.iterkeys() def iteritems(self): # yield children for k in self.iterkeys(): v=self.__getitem__(k) yield (k,v) def __repr__(self): return '{meta dict}' def __str__(self): return self.__repr__() def setdefault(self,key,default=None): try: return self.__getitem__(key) except: self.__setitem__(key) return default def update(self,d): for k,v in d.items(): self.__setitem__(k, v) def popitem(self): try: k, v = self.iteritems().next() del self[k] return k, v except StopIteration: raise KeyError, "meta_dict is empty" def __setitem__(self, k, v): if k in self.cur_dict: row_num=self.cur_dict[k] self.tbl[row_num].v=marshal.dumps(v) else: self.tbl.append(k=str(k), v=marshal.dumps(v)) #reload keys into memory self._set_keys() def __delitem__(self, key): raise NotImplementedError def __hash__(self): raise TypeError, "meta_dict is unhashable" def meta_save(data,file,tbl=''): if not tbl: tbl='table0' db = metakit.storage(file,1) tbl = db.getas('%s[k:S,v:B]'%(tbl)) for k in data: tbl.append(k=str(k),v=marshal.dumps(data[k])) db.commit() if __name__=='__main__': file='test_dict.mk' #convert dictionary to a meta_kit based dictionary a_dict={'a':'b',1:2} meta_save(a_dict,file) #write to meta_kit based dictionary m_dict=meta_load(file=file,write=1) #change a member m_dict[1]=5 #add a member m_dict['c']='d' m_dict.close() #read metakit based dictionary m2_dict=meta_load(file=file) for i in m2_dict: print i,m2_dict[i] __________________________________ Do you Yahoo!? New Yahoo! Photos - easier uploading and sharing. http://photos.yahoo.com/ From 4mklist@myd1.com Fri Dec 12 01:52:29 2003 Return-Path: <4mklist@myd1.com> X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 770BFB3A9D for ; Fri, 12 Dec 2003 01:52:29 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXBlo2tQ for ; Fri, 12 Dec 2003 01:52:24 +0100 (CET) Received: from bfish.hurrah.com (bfish.hurrah.com [208.151.247.36]) by dizzie.triqs.com (Postfix) with ESMTP id BFE7EB3A87 for ; Fri, 12 Dec 2003 01:52:22 +0100 (CET) Received: from [192.168.0.2] (12-211-49-15.client.attbi.com [12.211.49.15]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id hBC0qLK21077 for ; Thu, 11 Dec 2003 16:52:22 -0800 Date: Thu, 11 Dec 2003 16:52:05 -0800 From: Jerry <4mklist@myd1.com> X-Mailer: The Bat! (v2.01.3) Personal X-Priority: 3 (Normal) Message-ID: <175123811296.20031211165205@myd1.com> To: Metakit Comments: follow MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Py 2.3.2 & vw.append() X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2003 00:52:29 -0000 Hi, I recently upgraded my product from python 2.2 to 2.3.2 and did some minor clean up at the same time in preparation for the next added feature. I wasn't getting the correct test results and I have been tracking bugs for days, which I assumed were typos. Instead I found that Mk4py.dll for 2.3.2 no longer handles appending of PyRowRef's, which my complex data manipulation application makes heavy use of. In the past, matching attribute names would get their values added to the new row. Now it adds the row, with all attributes set to '' (or 0). Is this a bug? Can I do this at all with 2.3.2? Am I doing something wrong? mk.version 2.4.9.2 on both 2.3.2 mk4py.dll 229,376 bytes 2.2.1 mk4py.dll 225.280 bytes ===== working example ===== >>> import sys >>> sys.version '2.2.1 (#34, Apr 9 2002, 19:34:33) [MSC 32 bit (Intel)]' >>> mk.version '2.4.9.2' >>> db=mk.storage() >>> vw = db.getas('a[s,i:i]') >>> vw2 = db.getas('b[s,i:i]') >>> vw2.append(('test',999)) 0 >>> mk.dump(vw2) s i ---- --- test 999 ---- --- Total: 1 rows >>> vw.append(vw2[0]) 0 >>> mk.dump(vw) s i ---- --- test 999 ---- --- Total: 1 rows =========== non-working example ======= Python 2.3.2 (#49, Oct 2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] Type "help", "copyright", "credits" or "license" for more information. >>> import mk >>> db=mk.storage() >>> vw = db.getas('a[s,i:i]') >>> vw2 = db.getas('b[s,i:i]') >>> vw2.append(('test',999)) 0 >>> vw.append(vw2[0]) 0 >>> mk.dump(vw) s i - - 0 - - Total: 1 rows >>> mk.version '2.4.9.2' >>> -- Thanks, Jerry McRae MetaKit 2.4.9.2, Python 2.3.2, WinXP, 796M RAM From niki@vintech.bg Fri Dec 12 12:15:14 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2C4C3B3AA0 for ; Fri, 12 Dec 2003 12:15:14 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX6OsXWn for ; Fri, 12 Dec 2003 12:15:12 +0100 (CET) Received: from intra.vintech.l (unknown [193.68.201.82]) by dizzie.triqs.com (Postfix) with ESMTP id 931ACB3A87 for ; Fri, 12 Dec 2003 12:15:11 +0100 (CET) Received: from mail.vintech.bg (gw.vintech.bg [193.68.201.82]) by intra.vintech.l (Postfix) with ESMTP id F0F0657278; Fri, 12 Dec 2003 13:17:55 +0200 (EET) Received: from vintech.bg (sierra.vintech.l [10.0.0.7]) by mail.vintech.bg (Postfix) with ESMTP id 3008A23011; Fri, 12 Dec 2003 14:14:59 +0200 (EET) Message-ID: <3FD9A241.6010905@vintech.bg> Date: Fri, 12 Dec 2003 13:10:57 +0200 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: bg, en MIME-Version: 1.0 To: Jerry <4mklist@myd1.com> Subject: Re: [Metakit] Py 2.3.2 & vw.append() References: <175123811296.20031211165205@myd1.com> In-Reply-To: <175123811296.20031211165205@myd1.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2003 11:15:14 -0000 Jerry wrote: > Hi, > > I recently upgraded my product from python 2.2 to 2.3.2 and did some > minor clean up at the same time in preparation for the next added > feature. I wasn't getting the correct test results and I have been > tracking bugs for days, which I assumed were typos. Instead I found > that Mk4py.dll for 2.3.2 no longer handles appending of PyRowRef's, > which my complex data manipulation application makes heavy use of. In > the past, matching attribute names would get their values added to the > new row. Now it adds the row, with all attributes set to '' (or 0). > > Is this a bug? Can I do this at all with 2.3.2? Am I doing something > wrong? This can be related to PySequence_Check that is used by metakit when appending. HTH Niki Spahiev From jcw@equi4.com Fri Dec 12 12:31:23 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 18877B3AA0 for ; Fri, 12 Dec 2003 12:31:23 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXSqNBnD for ; Fri, 12 Dec 2003 12:31:21 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 28288B3A87 for ; Fri, 12 Dec 2003 12:31:21 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id CBE3E83C26; Fri, 12 Dec 2003 12:31:19 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 5C37883914 for ; Fri, 12 Dec 2003 12:31:13 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Fri, 12 Dec 2003 12:31:12 +0100 X-Mailer: Apple Mail (2.606) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Mk4py changes X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2003 11:31:23 -0000 At last, I've gone through the large set of changes contributed by Nicholas Riley (thank you!) a while back, see this post: http://www.equi4.com/pipermail/metakit/2003-September/001407.html All changes but one conflict in PyView.cpp have been checked into CVS. Unfortunately, it looks like I've botched the "setup.py" script - it throws an exception, right now. If someone has time to check things and help get this right, I'd be most grateful - then a new minor release could be brought out which ought to make it easier to build the Python binding with setup.py, which is so much more convenient than configure/make. -jcw PS. I'll try to also look into the Python 2.3.2 regression. The recent check-ins include a Python test suite, so that ought to help nail it down further. From 4mklist@myd1.com Fri Dec 12 16:12:14 2003 Return-Path: <4mklist@myd1.com> X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 94209B3AA0 for ; Fri, 12 Dec 2003 16:12:14 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXd4vd4r for ; Fri, 12 Dec 2003 16:12:13 +0100 (CET) Received: from bfish.hurrah.com (bfish.hurrah.com [208.151.247.36]) by dizzie.triqs.com (Postfix) with ESMTP id 91FA0B3A87 for ; Fri, 12 Dec 2003 16:12:09 +0100 (CET) Received: from [192.168.0.2] (12-211-49-15.client.attbi.com [12.211.49.15]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id hBCFBvK30408; Fri, 12 Dec 2003 07:11:57 -0800 Date: Fri, 12 Dec 2003 07:11:45 -0800 From: Jerry <4mklist@myd1.com> X-Mailer: The Bat! (v2.01.3) Personal X-Priority: 3 (Normal) Message-ID: <52175391625.20031212071145@myd1.com> To: Niki Spahiev Subject: Re: [Metakit] Py 2.3.2 & vw.append() In-Reply-To: <3FD9A241.6010905@vintech.bg> References: <175123811296.20031211165205@myd1.com> <3FD9A241.6010905@vintech.bg> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2003 15:12:15 -0000 Hi Niki, You wrote on Friday (12/12/2003 at 03:10) thus about [Metakit] Py 2.3.2 & vw.append(): NS> Jerry wrote: >> Hi, >> >> I recently upgraded my product from python 2.2 to 2.3.2 and did some >> minor clean up at the same time in preparation for the next added >> feature. I wasn't getting the correct test results and I have been >> tracking bugs for days, which I assumed were typos. Instead I found >> that Mk4py.dll for 2.3.2 no longer handles appending of PyRowRef's, >> which my complex data manipulation application makes heavy use of. In >> the past, matching attribute names would get their values added to the >> new row. Now it adds the row, with all attributes set to '' (or 0). >> >> Is this a bug? Can I do this at all with 2.3.2? Am I doing something >> wrong? NS> This can be related to PySequence_Check that is used by metakit when NS> appending. NS> HTH NS> Niki Spahiev Thanks for the reply. Unfortunately I'm not knowledgeable enough to know what that means. I use pre-built Mk4py.dll and have now C++ compiler. I guess if I am using the dll correctly, I would like to know: 1) Is this behavior shared by others? 2) If so, Is there a "fix" available? 3) If so, is this is "feature" permanent or temporary? 4) end if so's -- Jerry (Mailed safely with The Bat! 2.01.3) ___________________________ I used to be undecided, now I'm not sure! From gmcm@hypernet.com Fri Dec 12 19:32:32 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 01B3EB3AA0 for ; Fri, 12 Dec 2003 19:32:32 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXHxYUvd for ; Fri, 12 Dec 2003 19:32:30 +0100 (CET) Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by dizzie.triqs.com (Postfix) with ESMTP id B6150B3A87 for ; Fri, 12 Dec 2003 19:32:29 +0100 (CET) Received: from PARANOIA (204.176.40.53) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Fri, 12 Dec 2003 13:34:48 -0500 From: "Gordon McMillan" To: Jerry <4mklist@myd1.com> Date: Fri, 12 Dec 2003 13:31:02 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Py 2.3.2 & vw.append() Message-ID: <3FD9C316.27298.2A8657F5@localhost> Priority: normal In-reply-to: <52175391625.20031212071145@myd1.com> References: <3FD9A241.6010905@vintech.bg> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gmcm@hypernet.com List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Dec 2003 18:32:32 -0000 On 12 Dec 2003 at 7:11, Jerry wrote: > ... about [Metakit] Py 2.3.2 & vw.append() > 1) Is this behavior shared by others? Yes, it's in the CVS version of Mk4py (which is probably where your 2.3 dll came from - I don't think this is Python version related). > 2) If so, Is there a "fix" available? Not yet. > 3) If so, is this is "feature" permanent or temporary? It's definitely a bug. Alas, with the changes in the type system it has become very difficult to do runtime type checking in version portable C. The code giving you trouble was rearranged to fix a bug that bit a good deal more frequently. As a (temporary) workaround, a braindead proxy works: >>> class rowwrap: ... def __init__(self, o): ... self.o = o ... def __getattr__(self, nm): ... return getattr(self.o, nm) ... >>> vw.append(rowwrap(vw2[0])) 1 >>> mk.dump(vw) s i ---- --- 0 test 999 ---- --- Total: 2 rows -- Gordon http://www.mcmillan-inc.com/ From rcohen@articque.com Wed Dec 17 18:55:50 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A7203B3A8C for ; Wed, 17 Dec 2003 18:55:50 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXiVe20Z for ; Wed, 17 Dec 2003 11:55:49 +0100 (CET) Received: from mwinf0601.wanadoo.fr (smtp6.wanadoo.fr [193.252.22.25]) by dizzie.triqs.com (Postfix) with ESMTP id 934F8B3A89 for ; Wed, 17 Dec 2003 18:55:49 +0800 (WST) Received: from articque.com (ATours-101-1-3-121.w80-11.abo.wanadoo.fr [80.11.35.121]) by mwinf0601.wanadoo.fr (SMTP Server) with ESMTP id A4D9B3400105 for ; Wed, 17 Dec 2003 11:55:48 +0100 (CET) Message-ID: <3FE035DA.1040000@articque.com> Date: Wed, 17 Dec 2003 11:54:18 +0100 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] use views to return table of data X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Dec 2003 10:55:51 -0000 Hi, I would like to use storage and views to return a structured table of data instead of painful char** and double**. So I built a c4_Storage and return it. I'm not used to this syntax (I always use pointers). Is it safe to do that : c4_Storage execute0_sub() { printf("test mem only storage\n"); c4_Storage sto; c4_View view=sto.GetAs("myview[id:S,data:D]"); c4_Row row; c4_StringProp id("id"); c4_DoubleProp data("data"); id(row)="one"; data(row)=1; view.Add(row); id(row)="two"; data(row)=2; view.Add(row); return(sto); } void execute0() { printf("test mem only storage\n"); long idx; c4_Storage sto=execute0_sub(); c4_View view=sto.GetAs("myview[id:S,data:D]"); c4_StringProp id("id"); c4_DoubleProp data("data"); for (idx=0;idx X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 80B94B3A89 for ; Thu, 18 Dec 2003 00:39:24 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXwMbo7L for ; Wed, 17 Dec 2003 17:39:23 +0100 (CET) Received: from web14306.mail.yahoo.com (web14306.mail.yahoo.com [216.136.173.82]) by dizzie.triqs.com (Postfix) with SMTP id B0F24B3A8C for ; Thu, 18 Dec 2003 00:39:22 +0800 (WST) Message-ID: <20031217163921.13067.qmail@web14306.mail.yahoo.com> Received: from [64.231.195.202] by web14306.mail.yahoo.com via HTTP; Wed, 17 Dec 2003 08:39:21 PST Date: Wed, 17 Dec 2003 08:39:21 -0800 (PST) From: mARK bLOORE Subject: Re: [Metakit] use views to return table of data To: Riccardo Cohen , Metakit In-Reply-To: <3FE035DA.1040000@articque.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Dec 2003 16:39:24 -0000 returning a local object is OK, the caller will get a copy of it (so make sure the copy constructor is present and correct!). it is returning references or pointers to locals that is trouble. if all you want is temporary data storage, ie you don't need persistence in a database file, you might find something like this much easier: struct stuff { std::string id; double data; } typedef std::vector MyStuff; MyStuff execute0_sub() { ... shorter, more readable code, and faster to execute. --- Riccardo Cohen wrote: > Hi, > I would like to use storage and views to return a structured table of data instead of painful > char** > and double**. > So I built a c4_Storage and return it. I'm not used to this syntax (I always use pointers). Is > it > safe to do that : > > c4_Storage execute0_sub() > { > printf("test mem only storage\n"); > c4_Storage sto; > c4_View view=sto.GetAs("myview[id:S,data:D]"); > c4_Row row; > c4_StringProp id("id"); > c4_DoubleProp data("data"); > id(row)="one"; > data(row)=1; > view.Add(row); > id(row)="two"; > data(row)=2; > view.Add(row); > return(sto); > } > > void execute0() > { > printf("test mem only storage\n"); > long idx; > c4_Storage sto=execute0_sub(); > c4_View view=sto.GetAs("myview[id:S,data:D]"); > c4_StringProp id("id"); > c4_DoubleProp data("data"); > for (idx=0;idx printf("id='%s',data=%f\n",(const char*)id(view[idx]),(double)data(view[idx])); > } > > in particular returning an object that is local to the function ??? > It works but I need confirmation > Thanks > > -- > Riccardo Cohen > > Articque > Les Roches > 37230 Fondettes > France > web = http://www.articque.com > tel: +33 02 47 49 90 49 > fax: +33 02 47 49 91 49 > > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________ Do you Yahoo!? New Yahoo! Photos - easier uploading and sharing. http://photos.yahoo.com/ From rcohen@articque.com Thu Dec 18 02:31:27 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C3FC9B3A8C for ; Thu, 18 Dec 2003 02:31:26 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXDHQ0fH for ; Wed, 17 Dec 2003 19:31:23 +0100 (CET) Received: from mwinf0604.wanadoo.fr (smtp6.wanadoo.fr [193.252.22.25]) by dizzie.triqs.com (Postfix) with ESMTP id 3444FB3A89 for ; Thu, 18 Dec 2003 02:31:23 +0800 (WST) Received: from articque.com (ATours-101-1-3-79.w80-11.abo.wanadoo.fr [80.11.35.79]) by mwinf0604.wanadoo.fr (SMTP Server) with ESMTP id 756E828000B5 for ; Wed, 17 Dec 2003 19:31:22 +0100 (CET) Message-ID: <3FE0A0A0.6030601@articque.com> Date: Wed, 17 Dec 2003 19:29:52 +0100 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 Cc: Metakit Subject: Re: [Metakit] use views to return table of data References: <20031217163921.13067.qmail@web14306.mail.yahoo.com> In-Reply-To: <20031217163921.13067.qmail@web14306.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Dec 2003 18:31:27 -0000 Thanks a lot. I don't use template so I'll stick to metakit views mARK bLOORE wrote: > returning a local object is OK, the caller will get a copy of it (so make sure the copy > constructor is present and correct!). > it is returning references or pointers to locals that is trouble. > > if all you want is temporary data storage, ie you don't need persistence in a database file, you > might find something like this much easier: > > struct stuff > { > std::string id; > double data; > } > > typedef std::vector MyStuff; > > MyStuff execute0_sub() > { > ... > > shorter, more readable code, and faster to execute. > > > --- Riccardo Cohen wrote: > >>Hi, >>I would like to use storage and views to return a structured table of data instead of painful >>char** >>and double**. >>So I built a c4_Storage and return it. I'm not used to this syntax (I always use pointers). Is >>it >>safe to do that : >> >>c4_Storage execute0_sub() >>{ >> printf("test mem only storage\n"); >> c4_Storage sto; >> c4_View view=sto.GetAs("myview[id:S,data:D]"); >> c4_Row row; >> c4_StringProp id("id"); >> c4_DoubleProp data("data"); >> id(row)="one"; >> data(row)=1; >> view.Add(row); >> id(row)="two"; >> data(row)=2; >> view.Add(row); >> return(sto); >>} >> >>void execute0() >>{ >> printf("test mem only storage\n"); >> long idx; >> c4_Storage sto=execute0_sub(); >> c4_View view=sto.GetAs("myview[id:S,data:D]"); >> c4_StringProp id("id"); >> c4_DoubleProp data("data"); >> for (idx=0;idx> printf("id='%s',data=%f\n",(const char*)id(view[idx]),(double)data(view[idx])); >>} >> >>in particular returning an object that is local to the function ??? >>It works but I need confirmation >>Thanks >> >>-- >>Riccardo Cohen >> >>Articque >>Les Roches >>37230 Fondettes >>France >>web = http://www.articque.com >>tel: +33 02 47 49 90 49 >>fax: +33 02 47 49 91 49 >> >> >>_____________________________________________ >>Metakit mailing list - Metakit@equi4.com >>http://www.equi4.com/mailman/listinfo/metakit > > > > __________________________________ > Do you Yahoo!? > New Yahoo! Photos - easier uploading and sharing. > http://photos.yahoo.com/ > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jeff@k2.com Sat Dec 20 01:18:49 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B33A7B3A8E for ; Sat, 20 Dec 2003 01:18:49 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXTkicJe for ; Fri, 19 Dec 2003 18:18:48 +0100 (CET) Received: from kobalt.k2.com (dsl092-155-098.wdc2.dsl.speakeasy.net [66.92.155.98]) by dizzie.triqs.com (Postfix) with ESMTP id 37A3CB3A87 for ; Sat, 20 Dec 2003 01:18:47 +0800 (WST) Received: from [192.168.0.23] (IDENT:jkay@kobalt.k2.com [192.168.0.2]) by kobalt.k2.com (8.10.2/8.10.2) with ESMTP id hBJHIjk15978 for ; Fri, 19 Dec 2003 12:18:45 -0500 User-Agent: Microsoft-Entourage/10.1.4.030702.0 Date: Fri, 19 Dec 2003 12:18:45 -0500 From: Jeffrey Kay To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] MKStats X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Dec 2003 17:18:50 -0000 Is the source code for the mku portion of the mkstats program available? I thought that having the ability to compute the percentage of empty space in a db would be a helpful function to have in my code, specifically so I could decide when to compact the data. It appears that the c4_Strategy class has a FileSize() function, but that doesn't return the amount of bytes actually used in a file. How would I compute that value? Thanks and happy holidays everyone -- -- jeff jeffrey kay weblog pgp key aim share files with me -- get shinkuro -- "first get your facts, then you can distort them at your leisure" -- mark twain "if the person in the next lane at the stoplight rolls up the window and locks the door, support their view of life by snarling at them" -- a biker's guide to life "if A equals success, then the formula is A equals X plus Y plus Z. X is work. Y is play. Z is keep your mouth shut." -- albert einstein From jcw@equi4.com Sun Dec 21 05:38:41 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B5540B3A8F for ; Sun, 21 Dec 2003 05:38:41 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXKnqXnH for ; Sat, 20 Dec 2003 22:38:40 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 405C3B3A87 for ; Sun, 21 Dec 2003 05:38:40 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id D0AC383C27; Sat, 20 Dec 2003 22:38:37 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id A994883914 for ; Sat, 20 Dec 2003 22:38:36 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: References: Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] MKStats Date: Sat, 20 Dec 2003 22:38:36 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Dec 2003 21:38:42 -0000 Jeffrey Kay wrote: > Is the source code for the mku portion of the mkstats program > available? I > thought that having the ability to compute the percentage of empty > space in > a db would be a helpful function to have in my code, specifically so I > could > decide when to compact the data. It appears that the c4_Strategy > class has > a FileSize() function, but that doesn't return the amount of bytes > actually > used in a file. How would I compute that value? Sorry, I'm afraid not. Mkstats uses some new code which is part of a larger project. The "mku" utility is not based on the Metakit C++ core, and does a complete traversal on its own of the data on disk to generate the usage info. You can call it as an executable, and parse the output, as mkstats does. -jcw From jcw@equi4.com Sun Dec 21 05:50:08 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 89531B3A93 for ; Sun, 21 Dec 2003 05:50:08 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXHwV9QG for ; Sat, 20 Dec 2003 22:50:06 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 49A66B3A8F for ; Sun, 21 Dec 2003 05:50:06 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id ECD9383C27; Sat, 20 Dec 2003 22:50:03 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 027C083914; Sat, 20 Dec 2003 22:50:01 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <767BC525-3336-11D8-A82A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Date: Sat, 20 Dec 2003 22:50:01 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com cc: Lars Magne Ingebrigtsen Subject: [Metakit] Metakit list at gmane.org X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Dec 2003 21:50:08 -0000 If you prefer to read this mailing list over the web, then you may want to check out the new archive at http://news.gmane.org/gmane.comp.db.metakit/ - it's quite sophisticated in its support of keyboard navigation (for javascript-capable browsers). Click on the question mark in the top right corner for details. There's also an NNTP interface. Thanks to Lars Magne Ingebrigtsen - "Mr. Gmane", for making such a wonderful resource available, and for importing the entire Metakit mailing list archive. -jcw PS. FYI, Starkit list has also been on Gmane for some time now: http://news.gmane.org/gmane.comp.lang.tcl.starkit/ From yaryantua@yahoo.gr Tue Dec 23 14:14:25 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 46466B3A8C for ; Tue, 23 Dec 2003 14:14:25 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXEYX4nm for ; Tue, 23 Dec 2003 07:14:23 +0100 (CET) Received: from web60704.mail.yahoo.com (web60704.mail.yahoo.com [216.109.117.227]) by dizzie.triqs.com (Postfix) with SMTP id B5897B3A87 for ; Tue, 23 Dec 2003 14:14:22 +0800 (WST) Message-ID: <20031223061420.82018.qmail@web60704.mail.yahoo.com> Received: from [62.38.239.51] by web60704.mail.yahoo.com via HTTP; Tue, 23 Dec 2003 06:14:20 GMT Date: Tue, 23 Dec 2003 06:14:20 +0000 (GMT) From: =?iso-8859-7?q?andrian?= To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] metakit and UTF-8 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Dec 2003 06:14:25 -0000 I created a metakit database using "Mk4tcl 2.4.9.2". I have saved the script file with the data to populate the db in UTF-8. However, the stored data appear to be corrupted. I understand that according to Metakit's specification UTF-8 is supported. Furthermore, I use Wikit, where I have, succesfully, stored UTF-8 data. What can be the cause of this problem? Thanks for your attendance ____________________________________________________________ Do You Yahoo!? ÁðïêôÞóôå ôç äùñåÜí @yahoo.gr äéåýèõíóç óáò óôï http://www.otenet.gr From jcw@equi4.com Tue Dec 23 19:43:08 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 793D9B3AA4 for ; Tue, 23 Dec 2003 19:43:08 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXb3VxZm for ; Tue, 23 Dec 2003 12:43:05 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id E732DB3A87 for ; Tue, 23 Dec 2003 19:43:04 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 8CA8883C26; Tue, 23 Dec 2003 12:43:03 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id C3AAB83914 for ; Tue, 23 Dec 2003 12:43:01 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <20031223061420.82018.qmail@web60704.mail.yahoo.com> References: <20031223061420.82018.qmail@web60704.mail.yahoo.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <29A53AAC-353D-11D8-A82A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] metakit and UTF-8 Date: Tue, 23 Dec 2003 12:43:01 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Dec 2003 11:43:08 -0000 andrian wrote: > I created a metakit database using "Mk4tcl 2.4.9.2". > I have saved the script file with the data to populate > the db in UTF-8. However, the stored data appear to > be corrupted. > I understand that according to Metakit's specification > UTF-8 is supported. Furthermore, I use Wikit, where I > have, succesfully, stored UTF-8 data. > What can be the cause of this problem? UTF-8 can definitely be stored in MK (sorting is another matter). Can you create a test script which shows the problem? Without it, I have no way of helping or even reproducing the problem, I'm afraid. FWIW, I have not heard of a problem with UTF-8 before. If you use Mk4tcl, then you may want to consider posting to the Starkit mailing list, which has many more Tcl susbcribers than this list: http://www.equi4.com/mailman/listinfo/starkit Though I read and respond to both, of course :) -jcw From gary.h.merrill@gsk.com Sun Dec 28 05:31:16 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D585FB3A8B for ; Sun, 28 Dec 2003 05:31:15 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXTMnzNl for ; Sat, 27 Dec 2003 22:31:05 +0100 (CET) Received: from usrtpnp1.glaxowellcome.com (firewall1.glaxowellcome.com [192.58.204.204]) by dizzie.triqs.com (Postfix) with ESMTP id 9711CB3A87 for ; Sun, 28 Dec 2003 05:31:04 +0800 (WST) Received: by usrtpnp1.glaxowellcome.com; id QAA24204; Sat, 27 Dec 2003 16:30:56 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma024072; Sat, 27 Dec 03 16:30:30 -0500 Received: by ussunt3.glaxo.com id QAA16112; Sat, 27 Dec 2003 16:30:28 -0500 (EST) Received: from 166.71.134.118 by us9n57.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Sat, 27 Dec 2003 16:30:17 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hBRLUGc20573 for ; Sat, 27 Dec 2003 16:30:17 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Sat, 27 Dec 2003 16:30:00 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 12/27/2003 04:30:24 PM MIME-Version: 1.0 X-WSS-ID: 13F326631245671-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Closing storages (again) X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Dec 2003 21:31:16 -0000 Sorry to raise this touchy subject again, but I'm now trying to work around it in a somewhat different direction. There are circumstances in which I want to (a) create a storage based on a file and then (b) later delete this file. Of course I know that once the file is gone, then anything based on it (any views left hanging arount, etc.) are invalid. But that's okay. Now it turns out that I have a "viewer" module for such a storage. If I invoke this viewer (as a top-level wxPython frame) it instantiates some classes, one of which eventually opens the storage via 'metakit.storage(name, mode)'. Then I close the viewer window. Now I *thought* this would make the storage object go away -- but apparently not, since if I then try to delete the file the storage is based on I get an exception and investigation shows that the storage (or something pertaining to it) is holding the file open. I can duplicate this behavior exactly just by opening any file-based storage and then trying to delete the file. And that's how it *should* work, of course. What I *don't* understand is why the storage is still hanging around once I close the frame that created the class instance in which the storage is created. There's only the one reference to the storage (buried in that class instance) and the only instance of that class is created in the viewer, and the viewer is now gone. But obviously the storage is still hanging onto the file resource. This is hardly Metakit's problem. So I figured that I'd just circumvent all these storage lifetime problems I've been having by managing all storage references myself. And after some thought it looked as though a nice way to approach this would be to maintain only one storage object per file (or one storage object per file/mode pair), and when any part of the program wants to "create" a storage for a given file, it finds (or creates) that single storage object and returns a proxy (weak reference) to it. Then, if I ever need to clobber a storage object at least I know exactly where it is and that there is only *one* reference to it. I think this would work quite well except for one thing: "Cannot create weak reference to 'PyStorage' object". So now my question is this: Are weak references to PyStorage objects unsupported simply because the necessary stuff to support them was just never added? (It doesn't *look* as though it would take much to support this.) Or is there some more fundamental reason weak references to PyStorage objects can't be supported? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From gmcm@hypernet.com Mon Dec 29 21:48:20 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3B1F4B3A95 for ; Mon, 29 Dec 2003 21:48:20 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXW7ER1a for ; Mon, 29 Dec 2003 14:48:19 +0100 (CET) Received: from hypernet.com (hypernet.com [204.176.40.2]) by dizzie.triqs.com (Postfix) with ESMTP id 16965B3A87 for ; Mon, 29 Dec 2003 21:48:18 +0800 (WST) Received: from PARANOIA (208.229.12.6) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Mon, 29 Dec 2003 08:51:10 -0500 From: "Gordon McMillan" To: , metakit@equi4.com Date: Mon, 29 Dec 2003 08:47:15 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Closing storages (again) Message-ID: <3FEFEA13.23411.810EA1BB@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gmcm@hypernet.com List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Dec 2003 13:48:20 -0000 On 27 Dec 2003 at 16:30, gary.h.merrill@gsk.com wrote: [...] > So now my question is this: Are weak references to PyStorage > objects unsupported simply because the necessary stuff to > support them was just never added? (It doesn't *look* as > though it would take much to support this.) Or is there some > more fundamental reason weak references to PyStorage objects > can't be supported? Mk4py can still be compiled with Python 1.52. This has proved valuable to a number of people (myself included) whose hosting providers subscribe to the "if it aint broke don't fix it" philosophy. -- Gordon http://www.mcmillan-inc.com/ From gary.h.merrill@gsk.com Tue Dec 30 08:50:12 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3312BB3A99 for ; Tue, 30 Dec 2003 08:50:12 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXf75FoO for ; Tue, 30 Dec 2003 01:50:01 +0100 (CET) Received: from usrtpnp1.glaxowellcome.com (firewall1.glaxowellcome.com [192.58.204.204]) by dizzie.triqs.com (Postfix) with ESMTP id DDEB7B3A87 for ; Tue, 30 Dec 2003 08:50:00 +0800 (WST) Received: by usrtpnp1.glaxowellcome.com; id TAA12248; Mon, 29 Dec 2003 19:49:57 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma012117; Mon, 29 Dec 03 19:49:22 -0500 Received: by ussunt3.glaxo.com id TAA13305; Mon, 29 Dec 2003 19:49:21 -0500 (EST) Received: from 166.71.134.118 by us9n53.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Mon, 29 Dec 2003 19:49:15 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id hBU0nEc03001 for ; Mon, 29 Dec 2003 19:49:14 -0500 (EST) Subject: Re: [Metakit] Closing storages (again) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Mon, 29 Dec 2003 19:49:28 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 12/29/2003 07:49:21 PM MIME-Version: 1.0 X-WSS-ID: 13EE1400837615-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2003 00:50:12 -0000 Thanks. I understand the advantages of backward compatibility. And these kinds of things are difficult decisions to make. But I am *very* well aware of the reluctance of users not to do anything that they fear (correctly or not) would destabilize their critical application. There are, of course, at least two schools of thought on this sort of thing: (1) maintain only the lowest common denominator, or (2) maintain multiple releases. If you have an excellent source code control system or software engineering environment, (2) becomes reasonably straightforward. I'm not sure I'd be anxious to do it in Python. We used to do this with the SAS/C and C++ compilers and cross compilers -- using the Apollo precursor to Clearcase -- and we could easily maintian a wide variety of source bases for multiple hosts and multiple targets. But with anything less powerful than this, and without some kind of conditional compilation or conditional build capability, I would probably flee from it myself. Maintaining multiple versions is truly a pain if the right tools aren't available. And it really all depends on which market (and how wide a market) you're trying to serve. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 "Gordon McMillan" To: gary.h.merrill@gsk.com, metakit@equi4.com Sent by: metakit-bounces@e cc: qui4.com Subject: Re: [Metakit] Closing storages (again) 29-Dec-2003 08:47 Please respond to gmcm@hypernet.com On 27 Dec 2003 at 16:30, gary.h.merrill@gsk.com wrote: [...] > So now my question is this: Are weak references to PyStorage > objects unsupported simply because the necessary stuff to > support them was just never added? (It doesn't *look* as > though it would take much to support this.) Or is there some > more fundamental reason weak references to PyStorage objects > can't be supported? Mk4py can still be compiled with Python 1.52. This has proved valuable to a number of people (myself included) whose hosting providers subscribe to the "if it aint broke don't fix it" philosophy. -- Gordon http://www.mcmillan-inc.com/ _____________________________________________ Metakit mailing list - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From yaryantua@yahoo.gr Tue Dec 30 10:35:34 2003 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 79A40B3A9D for ; Tue, 30 Dec 2003 10:35:34 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXByDOJe for ; Tue, 30 Dec 2003 03:35:31 +0100 (CET) Received: from web60703.mail.yahoo.com (web60703.mail.yahoo.com [216.109.117.226]) by dizzie.triqs.com (Postfix) with SMTP id 9D76FB3A87 for ; Tue, 30 Dec 2003 10:35:30 +0800 (WST) Message-ID: <20031230023527.48366.qmail@web60703.mail.yahoo.com> Received: from [62.38.239.143] by web60703.mail.yahoo.com via HTTP; Tue, 30 Dec 2003 02:35:27 GMT Date: Tue, 30 Dec 2003 02:35:27 +0000 (GMT) From: =?iso-8859-7?q?andrian?= Subject: Re: [Metakit] metakit and UTF-8 To: mk , sk MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2003 02:35:34 -0000 This email is to undo my previous posting. Unicode support in 'Mk4tcl' is just a matter of setting the proper encoding using the 'fconfigure -encoding' Tcl command. There is a howto page with more details here: http://www.tcl.tk/doc/howto/i18n.html I cannot overemphasize the merits of building a multilingual database with such tools as tcl and metakit. Thanks and regards, andrian ____________________________________________________________ Do You Yahoo!? ÁðïêôÞóôå ôç äùñåÜí @yahoo.gr äéåýèõíóç óáò óôï http://www.otenet.gr From cmollis@objectlab.com Sat Jan 3 01:22:11 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0F119B3A9C for ; Sat, 3 Jan 2004 01:22:11 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXxJ3rIW for ; Fri, 2 Jan 2004 18:22:09 +0100 (CET) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by dizzie.triqs.com (Postfix) with ESMTP id 24E66B3A8B for ; Sat, 3 Jan 2004 01:22:09 +0800 (WST) Received: from d7cdkc01 (unknown[66.155.148.134]) by comcast.net (rwcrmhc13) with SMTP id <2004010217220701500801h2e> (Authid: cmollis83); Fri, 2 Jan 2004 17:22:07 +0000 Message-ID: <008e01c3d154$0ee8ed30$dd02a8c0@D7CDKC01> From: "chris mollis" To: Date: Fri, 2 Jan 2004 12:15:44 -0500 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_008B_01C3D12A.258DFDC0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Question X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2004 17:22:11 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_008B_01C3D12A.258DFDC0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I have a question about the best way to validate information on = reads/writes to the db. For example, I'd like to make sure that data = that is written out during a particular commit (by calculating a hash of = data written, perhaps) can be verified again when the database is = re-opened at a later date (possibly calculating the hash again and then = checking this against the previous hash). What do you recommend to be = the best way to do something like this? Should I override = DataWrite/DataRead methods of c4_FileStrategy to calculate hashes on = read and write operations? Any help would be greatly appreciated. Thanks Chris Mollis ------=_NextPart_000_008B_01C3D12A.258DFDC0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I have a question about the best way to = validate=20 information on reads/writes to the db.  For example, I'd like to = make sure=20 that data that is written out during a particular commit (by calculating = a hash=20 of data written, perhaps) can be verified again when the database is = re-opened=20 at a later date (possibly calculating the hash again and then checking = this=20 against the previous hash).  What do you recommend to be the best = way to do=20 something like this?  Should I override DataWrite/DataRead methods = of=20 c4_FileStrategy to calculate hashes on read and write = operations?

Any=20 help would be greatly appreciated.

Thanks

Chris=20 Mollis
------=_NextPart_000_008B_01C3D12A.258DFDC0-- From jcw@equi4.com Sat Jan 3 02:50:06 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 01765B3A9D for ; Sat, 3 Jan 2004 02:50:06 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXQ16urv for ; Fri, 2 Jan 2004 19:50:04 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 1DBA4B3A9C for ; Sat, 3 Jan 2004 02:50:04 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 01CBE83914; Fri, 2 Jan 2004 19:49:57 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 8109683914 for ; Fri, 2 Jan 2004 19:49:38 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: References: Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <6A41C347-3D54-11D8-9DB7-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Closing storages (again) Date: Fri, 2 Jan 2004 19:49:37 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2004 18:50:06 -0000 wrote: >> So now my question is this: Are weak references to PyStorage >> objects unsupported simply because the necessary stuff to >> support them was just never added? (It doesn't *look* as >> though it would take much to support this.) Or is there some >> more fundamental reason weak references to PyStorage objects >> can't be supported? > > Mk4py can still be compiled with Python 1.52. This has proved > valuable to a number of people (myself included) whose hosting > providers subscribe to the "if it aint broke don't fix it" philosophy. I don't know much about Python's weak references (simply because they were introduced after I was involved with Mk4py). I'd be willing to maintain a dual code-base, provided all differences are dealt with (one, I hope) #define's. So on my end, it's more a lack-of-time-not-high-enough-priority kind of issue than anything else. On a different, but related, note: I've been making good progress on integrating Nicholas Riley's changes to Mk4py. It now seems to be ok, other than that setup.py appears to be hitting distutils buglets with Python 2.2.3 (current default on my Gentoo Linux setup). For that combination, the answer will have to be: use make. Anyone using Mk4py: if you could download and verify the latest sources from CVS, that would be a big help - it's been holding up a new update of MK way too long already... -jcw From jcw@equi4.com Sat Jan 3 02:50:22 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id BEB79B3AA6 for ; Sat, 3 Jan 2004 02:50:22 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXujqpfz for ; Fri, 2 Jan 2004 19:50:20 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 182E7B3AA0 for ; Sat, 3 Jan 2004 02:50:20 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 3296883C27; Fri, 2 Jan 2004 19:50:16 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 1160883BE0 for ; Fri, 2 Jan 2004 19:49:41 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <008e01c3d154$0ee8ed30$dd02a8c0@D7CDKC01> References: <008e01c3d154$0ee8ed30$dd02a8c0@D7CDKC01> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Message-Id: <6DA4945F-3D54-11D8-9DB7-000A9588127E@equi4.com> Content-Transfer-Encoding: quoted-printable From: Jean-Claude Wippler Subject: Re: [Metakit] Question Date: Fri, 2 Jan 2004 19:49:43 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2004 18:50:23 -0000 chris mollis wrote: > I have a question about the best way to validate information on=20 > reads/writes to the db.=A0 For example, I'd like to make sure that = data=20 > that is written out during a particular commit (by calculating a hash=20= > of data written, perhaps) can be verified again when the database is=20= > re-opened at a later date (possibly calculating the hash again and=20 > then checking this against the previous hash).=A0 What do you = recommend=20 > to be the best way to do something like this?=A0 Should I override=20 > DataWrite/DataRead methods of c4_FileStrategy to calculate hashes on=20= > read and write operations? Good questions. There are several aspects to consider. The first one=20= is really what sort of validation you are after: if you need to verify=20= storage in general, then one could argue that there really is no other=20= option than full file checksums, and even then it'll depend on the sort=20= of validation as to when and how often you need to do it. Such=20 checksums could be done outside MK, i.e. after commits and before=20 opens. Another point to be made is that MK is a database: it does not read or=20= write all data each time the datafile is used. By validating writes on=20= commit, you'll be checking only what it changed, not the entire=20 datafile. Due to the way data is stored, the data written can be all=20 over the datafile, it's not necessarily contiguous (though individual=20 columns are). It gets worse: MK usually loads data by mapping a file into memory. =20 That means no "read" system calls take place at all in most cases: the=20= data is mapped to a range of addresses and paged in via O/S page faults=20= when "accessed", which is a matter of following pointers. If you really insist on doing this in some sort of fine-grained manner,=20= my suggestion would be to use a custom c4_Strategy class as you mention=20= yourself, in combination with a *second* MK datafile. The invariant is=20= that MK always writes entire columns - I suspect that it is possible to=20= detect the column boundaries written by intercepting DataWrite(). The=20= main call comes from column.cpp line 1532. Or it may be necessary to=20 introduce two extra strategy members which get called once in each call=20= to c4_Column::SaveNow(): - strategy_.DataInit(pos_) - unmodified while (iter...) loop - strategy_.DataDone(_size) The DataInit would reset a checksum field in the strategy object (and=20 remember pos_), the DataWrite calls would incrementally update the=20 checksum, and the DataDone call would save a triple in=20= the second MK datafile. It'll take some extra logic to make this work=20= across multiple commits, i.e. when space gets re-used, but that ought=20 to be doable. You may want to use hashed views for the secondary MK=20 file, to make it snappy. The most important problem to deal with is *when* to verify such saved=20= checksums. If it has to be done during access, then I can't think of=20 any other way than to disable memory mapping (by overriding=20 c4_Strategy::ResetFileMapping with a dummy which does nothing). That=20 makes MK slower and makes it use considerably more temp memory, however=20= - so you'll have to think hard whether that is really what you want. If you just want to checksum occasionally, then you could iterate=20 through all triples in the secondary MK file and verify each of the=20 ranges. Another idea would be to save checksums per fixed-size block, say 4 Kb.=20= That means DataWrite would track checksums, but it may need to read=20 some data of the disk to deal with writes which are not exactly on=20 block boundaries. This needs some thought to optimize, since most=20 DataWrite calls will not be aligned nicely. Then again, DataWrite does=20= get called in "mostly sequential" order, since it writes entire columns=20= most of the time. -jcw From bkelley@wi.mit.edu Sat Jan 3 03:55:23 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 057D5B3A9F for ; Sat, 3 Jan 2004 03:55:23 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXDGGadG for ; Fri, 2 Jan 2004 20:55:21 +0100 (CET) Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by dizzie.triqs.com (Postfix) with ESMTP id 05540B3A9D for ; Sat, 3 Jan 2004 03:55:21 +0800 (WST) Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.147.173.53]) by comcast.net (sccrmhc12) with SMTP id <2004010219551901200edsfte>; Fri, 2 Jan 2004 19:55:19 +0000 Message-ID: <3FF5CE48.1060602@wi.mit.edu> Date: Fri, 02 Jan 2004 15:02:16 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com, metakit@equi4.com Subject: Re: [Metakit] Closing storages (again) References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2004 19:55:23 -0000 gary.h.merrill@gsk.com wrote: You can solve this problem by creating a weakref proxy to this class which is a drop-in replacement to the storage class. import metakit class storage: def __init__(self, filename=None, mode=None): if filename and mode: self.__db = metakit.storage(filename, mode) elif filename and not mode: raise ValueError("Need to supply both a filename and a mode") else: self.__db = metakit.storage() def __getattr__(self, attribute): return getattr(self.__db, attribute) def getas(self, table): return self.__db.getas(table) def view(self, viewname): return self.__db.view(viewname) def close(self): del self.__db Which behaves for all intents and purposes like a metakit storage object. And additionally has a close method, just don't reference storage.__db from anywhere else and you should be okay. My full metakit wrapper is located here http://jura.wi.mit.edu/people/kelley/metawrap.py I did a bug release a couple of weeks ago. From ian.fairclough@intelligent-ms.com Mon Jan 5 23:39:08 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 15782B3A8A for ; Mon, 5 Jan 2004 23:39:08 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXY4Mnce for ; Mon, 5 Jan 2004 16:39:05 +0100 (CET) Received: from scratchy.base.ims (unknown [194.74.0.138]) by dizzie.triqs.com (Postfix) with ESMTP id A0AA4B3A87 for ; Mon, 5 Jan 2004 23:39:05 +0800 (WST) Received: from grpware.intelligent-ms.com (mail.intelligent-ms.com [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id i05FVjl20162 for ; Mon, 5 Jan 2004 15:31:45 GMT Received: from imsgsx6tvznbng ([192.168.200.181])i05FI1u24170 for ; Mon, 5 Jan 2004 15:18:01 GMT Message-ID: <063d01c3d3a2$78ebd530$b4c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Date: Mon, 5 Jan 2004 15:42:02 -0000 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_063A_01C3D3A2.76A5B5C0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Core Dump X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2004 15:39:08 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_063A_01C3D3A2.76A5B5C0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I'm getting a core dump which produces the following stack trace : #0 0x41513fa1 in kill () from /lib/libc.so.6 #1 0x41365fe6 in raise (sig=3D6) at signals.c:65 #2 0x41515264 in abort () from /lib/libc.so.6 #3 0x414b7e55 in __default_terminate () at ../../gcc/libgcc2.c:-1 #4 0x414b7e72 in __terminate () from /usr/lib/libstdc++-libc6.1-1.so.2 #5 0x4182fc2b in new_eh_context () at ../../gcc/libgcc2.c:3035 #6 0x4182ff0d in eh_context_specific () from = /usr/lib/libstdc++-libc6.2-2.so.3 #7 0x4182fe6e in eh_context_initialize () from = /usr/lib/libstdc++-libc6.2-2.so.3 #8 0x41830b4d in __rethrow (index=3D0x417faa44) from = /usr/lib/libstdc++-libc6.2-2.so.3 #9 0x417e0f88 in c4_HandlerSeq::c4_HandlerSeq () from = /home/portal/build_area/components/metakit/libmk4.so #10 0x417dfaf5 in c4_FormatV::At () from = /home/portal/build_area/components/metakit/libmk4.so #11 0x417dfbd8 in c4_FormatV::SetupAllSubviews () from = /home/portal/build_area/components/metakit/libmk4.so #12 0x417e0207 in c4_FormatV::Insert () from = /home/portal/build_area/components/metakit/libmk4.so #13 0x417efaae in c4_Sequence::InsertAt () from = /home/portal/build_area/components/metakit/libmk4.so #14 0x417ef793 in c4_Sequence::Resize () from = /home/portal/build_area/components/metakit/libmk4.so #15 0x41741418 in corbaMkTools::write () from = /home/portal/build_area/components/do_database_lib/libdoDatabase.so corbaMkTools::write is the last function in my application code that = calls "SetSize" on a data view. Does anybody have any idea what the = issue might be here and any suggestion for a workaround ? Thanks in advance for any help you can give. Best regards, Ian Fairclough. ------=_NextPart_000_063A_01C3D3A2.76A5B5C0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I'm getting a core dump which produces = the=20 following stack trace :
 
#0  0x41513fa1 in kill () from=20 /lib/libc.so.6
#1  0x41365fe6 in raise (sig=3D6) at=20 signals.c:65
#2  0x41515264 in abort () from = /lib/libc.so.6
#3 =20 0x414b7e55 in __default_terminate () at = ../../gcc/libgcc2.c:-1
#4 =20 0x414b7e72 in __terminate () from = /usr/lib/libstdc++-libc6.1-1.so.2
#5 =20 0x4182fc2b in new_eh_context () at ../../gcc/libgcc2.c:3035
#6 =20 0x4182ff0d in eh_context_specific () from=20 /usr/lib/libstdc++-libc6.2-2.so.3
#7  0x4182fe6e in=20 eh_context_initialize () from = /usr/lib/libstdc++-libc6.2-2.so.3
#8 =20 0x41830b4d in __rethrow (index=3D0x417faa44) from=20 /usr/lib/libstdc++-libc6.2-2.so.3
#9  0x417e0f88 in=20 c4_HandlerSeq::c4_HandlerSeq () from=20 /home/portal/build_area/components/metakit/libmk4.so
#10 0x417dfaf5 = in=20 c4_FormatV::At () from=20 /home/portal/build_area/components/metakit/libmk4.so
#11 0x417dfbd8 = in=20 c4_FormatV::SetupAllSubviews () from=20 /home/portal/build_area/components/metakit/libmk4.so
#12 0x417e0207 = in=20 c4_FormatV::Insert () from=20 /home/portal/build_area/components/metakit/libmk4.so
#13 0x417efaae = in=20 c4_Sequence::InsertAt () from=20 /home/portal/build_area/components/metakit/libmk4.so
#14 0x417ef793 = in=20 c4_Sequence::Resize () from=20 /home/portal/build_area/components/metakit/libmk4.so
#15 0x41741418 = in=20 corbaMkTools::write () from=20 /home/portal/build_area/components/do_database_lib/libdoDatabase.so
 
corbaMkTools::write is the last = function in my=20 application code that calls "SetSize" on a data view.  Does anybody = have=20 any idea what the issue might be here and any suggestion for a = workaround=20 ?
 
Thanks in advance for any help you can=20 give.
 
Best regards,
Ian = Fairclough.
------=_NextPart_000_063A_01C3D3A2.76A5B5C0-- From jcw@equi4.com Mon Jan 5 23:44:55 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C6351B3A8A for ; Mon, 5 Jan 2004 23:44:54 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXhgesMo for ; Mon, 5 Jan 2004 16:44:52 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 50D75B3A87 for ; Mon, 5 Jan 2004 23:44:51 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 4D20083C26; Mon, 5 Jan 2004 16:44:49 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 78C3483914 for ; Mon, 5 Jan 2004 16:44:47 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <063d01c3d3a2$78ebd530$b4c8a8c0@imsgsx6tvznbng> References: <063d01c3d3a2$78ebd530$b4c8a8c0@imsgsx6tvznbng> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Message-Id: <16F92880-3F96-11D8-9DB7-000A9588127E@equi4.com> Content-Transfer-Encoding: quoted-printable From: Jean-Claude Wippler Subject: Re: [Metakit] Core Dump Date: Mon, 5 Jan 2004 16:44:47 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2004 15:44:55 -0000 Ian Fairclough wrote: > I'm getting a core dump which produces the following stack trace : > =A0 > #0=A0 0x41513fa1 in kill () from /lib/libc.so.6 > #1=A0 0x41365fe6 in raise (sig=3D6) at signals.c:65 > #2=A0 0x41515264 in abort () from /lib/libc.so.6 > #3=A0 0x414b7e55 in __default_terminate () at ../../gcc/libgcc2.c:-1 > #4=A0 0x414b7e72 in __terminate () from = /usr/lib/libstdc++-libc6.1-1.so.2 > #5=A0 0x4182fc2b in new_eh_context () at ../../gcc/libgcc2.c:3035 > #6=A0 0x4182ff0d in eh_context_specific () from=20 > /usr/lib/libstdc++-libc6.2-2.so.3 > #7=A0 0x4182fe6e in eh_context_initialize () from=20 > /usr/lib/libstdc++-libc6.2-2.so.3 > #8=A0 0x41830b4d in __rethrow (index=3D0x417faa44) from=20 > /usr/lib/libstdc++-libc6.2-2.so.3 > #9=A0 0x417e0f88 in c4_HandlerSeq::c4_HandlerSeq () from=20 > /home/portal/build_area/components/metakit/libmk4.so > #10 0x417dfaf5 in c4_FormatV::At () from=20 > /home/portal/build_area/components/metakit/libmk4.so > #11 0x417dfbd8 in c4_FormatV::SetupAllSubviews () from=20 > /home/portal/build_area/components/metakit/libmk4.so > #12 0x417e0207 in c4_FormatV::Insert () from=20 > /home/portal/build_area/components/metakit/libmk4.so > #13 0x417efaae in c4_Sequence::InsertAt () from=20 > /home/portal/build_area/components/metakit/libmk4.so > #14 0x417ef793 in c4_Sequence::Resize () from=20 > /home/portal/build_area/components/metakit/libmk4.so > #15 0x41741418 in corbaMkTools::write () from=20 > /home/portal/build_area/components/do_database_lib/libdoDatabase.so > =A0 > corbaMkTools::write is the last function in my application code that=20= > calls "SetSize" on a data view.=A0 Does anybody have any idea what the=20= > issue might be here and any suggestion for a workaround ? The "__rethrow" looks suspicious to me: MK does not throw exceptions. You may want to rebuild MK with assertions enabled (-Dq4_CHECK) and=20 debug symbols (-g). -jcw From cmollis@objectlab.com Tue Jan 6 00:15:44 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 664FEB3A8A; Tue, 6 Jan 2004 00:15:44 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXay2SUb; Mon, 5 Jan 2004 17:15:41 +0100 (CET) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by dizzie.triqs.com (Postfix) with ESMTP id F2688B3A87; Tue, 6 Jan 2004 00:15:40 +0800 (WST) Received: from d7cdkc01 (unknown[66.155.148.134]) by comcast.net (rwcrmhc13) with SMTP id <2004010516153601500qbo31e> (Authid: cmollis83); Mon, 5 Jan 2004 16:15:37 +0000 Message-ID: <007501c3d3a6$426fa230$e102a8c0@D7CDKC01> From: "chris mollis" To: "Jean-Claude Wippler" , "Metakit list" References: <008e01c3d154$0ee8ed30$dd02a8c0@D7CDKC01> <6DA4945F-3D54-11D8-9DB7-000A9588127E@equi4.com> Subject: Re: [Metakit] Question Date: Mon, 5 Jan 2004 11:09:12 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2004 16:15:44 -0000 Thanks Jean-Claude.. a couple of questions (below): ----- Original Message ----- From: "Jean-Claude Wippler" To: "Metakit list" Sent: Friday, January 02, 2004 1:49 PM Subject: Re: [Metakit] Question chris mollis wrote: > I have a question about the best way to validate information on > reads/writes to the db. For example, I'd like to make sure that data > that is written out during a particular commit (by calculating a hash > of data written, perhaps) can be verified again when the database is > re-opened at a later date (possibly calculating the hash again and > then checking this against the previous hash). What do you recommend > to be the best way to do something like this? Should I override > DataWrite/DataRead methods of c4_FileStrategy to calculate hashes on > read and write operations? Good questions. There are several aspects to consider. The first one is really what sort of validation you are after: if you need to verify storage in general, then one could argue that there really is no other option than full file checksums, and even then it'll depend on the sort of validation as to when and how often you need to do it. Such checksums could be done outside MK, i.e. after commits and before opens. CM: agreed.. this was my first thought, but seemed a bit much if I just wanted to check "deltas" between reads and writes... Another point to be made is that MK is a database: it does not read or write all data each time the datafile is used. By validating writes on commit, you'll be checking only what it changed, not the entire datafile. Due to the way data is stored, the data written can be all over the datafile, it's not necessarily contiguous (though individual columns are). CM: right, this is what I thought.. I had a feeling it was going to be more difficult than I first thought :) It gets worse: MK usually loads data by mapping a file into memory. That means no "read" system calls take place at all in most cases: the data is mapped to a range of addresses and paged in via O/S page faults when "accessed", which is a matter of following pointers. CM: Ok.. I think I will need to disable memory-mapped IO (as you point out below).. If you really insist on doing this in some sort of fine-grained manner, my suggestion would be to use a custom c4_Strategy class as you mention yourself, in combination with a *second* MK datafile. The invariant is that MK always writes entire columns - I suspect that it is possible to detect the column boundaries written by intercepting DataWrite(). The main call comes from column.cpp line 1532. Or it may be necessary to introduce two extra strategy members which get called once in each call to c4_Column::SaveNow(): - strategy_.DataInit(pos_) - unmodified while (iter...) loop - strategy_.DataDone(_size) The DataInit would reset a checksum field in the strategy object (and remember pos_), the DataWrite calls would incrementally update the checksum, and the DataDone call would save a triple in the second MK datafile. It'll take some extra logic to make this work across multiple commits, i.e. when space gets re-used, but that ought to be doable. You may want to use hashed views for the secondary MK file, to make it snappy. CM: I'll have to work through an example of how this gets done and get back to you with questions.. I'm a bit confused about what actually gets written in the second file and how it's verified during access... (I'll have to test out some stuff before I can ask an intelligent question here :).. The most important problem to deal with is *when* to verify such saved checksums. If it has to be done during access, then I can't think of any other way than to disable memory mapping (by overriding c4_Strategy::ResetFileMapping with a dummy which does nothing). That makes MK slower and makes it use considerably more temp memory, however - so you'll have to think hard whether that is really what you want. If you just want to checksum occasionally, then you could iterate through all triples in the secondary MK file and verify each of the ranges. CM: again, I'm a bit confused as to what the "triples" are... I assume that these are just file markers (file position, size, along with the hash)... I'll try it out.. thanks for the advice! Another idea would be to save checksums per fixed-size block, say 4 Kb. That means DataWrite would track checksums, but it may need to read some data of the disk to deal with writes which are not exactly on block boundaries. This needs some thought to optimize, since most DataWrite calls will not be aligned nicely. Then again, DataWrite does get called in "mostly sequential" order, since it writes entire columns most of the time. -jcw _____________________________________________ Metakit mailing list - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From ian.fairclough@intelligent-ms.com Tue Jan 6 21:14:26 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C992DB3AE6; Tue, 6 Jan 2004 21:14:26 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XX4zBY3h; Tue, 6 Jan 2004 14:14:24 +0100 (CET) Received: from scratchy.base.ims (unknown [194.74.0.138]) by dizzie.triqs.com (Postfix) with ESMTP id AB31AB3A8E; Tue, 6 Jan 2004 21:14:23 +0800 (WST) Received: from grpware.intelligent-ms.com (mail.intelligent-ms.com [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id i06D77l26356; Tue, 6 Jan 2004 13:07:07 GMT Received: from imsgsx6tvznbng ([192.168.200.185])i06CrIu01816; Tue, 6 Jan 2004 12:53:18 GMT Message-ID: <077b01c3d457$6c6e5840$b4c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Subject: Re: [Metakit] Core Dump Date: Tue, 6 Jan 2004 13:17:23 -0000 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0778_01C3D457.6BBA2370" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 13:14:27 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0778_01C3D457.6BBA2370 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable JC, Thanks for the advice. I re-compiled as you suggested (with assertions = and debug symbols enabled), and I obtained a core dump with the = following stack trace : #0 0x41513fa1 in kill () from /lib/libc.so.6 #1 0x41365fe6 in raise (sig=3D6) at signals.c:65 #2 0x41515264 in abort () from /lib/libc.so.6 #3 0x414b7e55 in __default_terminate () at ../../gcc/libgcc2.c:-1 #4 0x414b7e72 in __terminate () from /usr/lib/libstdc++-libc6.1-1.so.2 #5 0x4183ac2b in new_eh_context () at ../../gcc/libgcc2.c:3035 #6 0x4183af0d in eh_context_specific () from = /usr/lib/libstdc++-libc6.2-2.so.3 #7 0x4183ae6e in eh_context_initialize () from = /usr/lib/libstdc++-libc6.2-2.so.3 #8 0x4183bb4d in __rethrow (index=3D0x41805a4c) from = /usr/lib/libstdc++-libc6.2-2.so.3 #9 0x417e3ee0 in c4_HandlerSeq::c4_HandlerSeq (this=3D0x3fffec00, = owner_=3D@0x2d93f440, handler_=3D0xf151f20) at = ../unix/../src/handler.cpp:102 #10 0x417e26dc in c4_FormatV::At (this=3D0xf151f20, index_=3D49699) at = ../unix/../src/format.cpp:44 #11 0x417e281a in c4_FormatV::SetupAllSubviews (this=3D0xf151f20) at = ../unix/../src/format.cpp:1027 #12 0x417e2b22 in c4_FormatV::Get (this=3D0xf151f20, index_=3D35525, = length_=3D@0xbce7a8d4) at ../unix/../src/format.cpp:1096 #13 0x417e3b9a in c4_Handler::GetBytes (this=3D0xf151f20, = index_=3D35525, buf_=3D@0xbce7a968, copySmall_=3Dfalse) at = ../unix/../src/handler.cpp:59 #14 0x417f4c87 in c4_Sequence::InsertAt (this=3D0x2e9923b8, index_=3D0, = newElem_=3D{_seq =3D 0x2d93f440, _index =3D 35525}, count_=3D1) at = ../unix/../src/viewx.cpp:210 #15 0x417f17e0 in c4_View::Add (this=3D0xbce7aa50, = newElem_=3D@0xbce7aa30) at ../include/mk4.inl:13 #16 0x4175495c in mkExt::eq () from = /home/portal/build_area/components/do_database_lib/libdoDatabase.so Any suggestions of what might be wrong would be much appreciated. Best regards, Ian Fairclough. ------=_NextPart_000_0778_01C3D457.6BBA2370 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
JC,
 
Thanks for the advice.  I = re-compiled as you=20 suggested (with assertions and debug symbols enabled), and I obtained a = core=20 dump with the following stack trace :
 
#0  0x41513fa1 in kill () from=20 /lib/libc.so.6
#1  0x41365fe6 in raise (sig=3D6) at=20 signals.c:65
#2  0x41515264 in abort () from = /lib/libc.so.6
#3 =20 0x414b7e55 in __default_terminate () at = ../../gcc/libgcc2.c:-1
#4 =20 0x414b7e72 in __terminate () from = /usr/lib/libstdc++-libc6.1-1.so.2
#5 =20 0x4183ac2b in new_eh_context () at ../../gcc/libgcc2.c:3035
#6 =20 0x4183af0d in eh_context_specific () from=20 /usr/lib/libstdc++-libc6.2-2.so.3
#7  0x4183ae6e in=20 eh_context_initialize () from = /usr/lib/libstdc++-libc6.2-2.so.3
#8 =20 0x4183bb4d in __rethrow (index=3D0x41805a4c) from=20 /usr/lib/libstdc++-libc6.2-2.so.3
#9  0x417e3ee0 in=20 c4_HandlerSeq::c4_HandlerSeq (this=3D0x3fffec00, owner_=3D@0x2d93f440, = handler_=3D0xf151f20) at=20 ../unix/../src/handler.cpp:102
#10 0x417e26dc in c4_FormatV::At=20 (this=3D0xf151f20, index_=3D49699) at = ../unix/../src/format.cpp:44
#11 0x417e281a=20 in c4_FormatV::SetupAllSubviews (this=3D0xf151f20) at=20 ../unix/../src/format.cpp:1027
#12 0x417e2b22 in c4_FormatV::Get=20 (this=3D0xf151f20, index_=3D35525, length_=3D@0xbce7a8d4) at=20 ../unix/../src/format.cpp:1096
#13 0x417e3b9a in c4_Handler::GetBytes = (this=3D0xf151f20, index_=3D35525, buf_=3D@0xbce7a968, = copySmall_=3Dfalse) at=20 ../unix/../src/handler.cpp:59
#14 0x417f4c87 in c4_Sequence::InsertAt = (this=3D0x2e9923b8, index_=3D0, newElem_=3D{_seq =3D 0x2d93f440, _index = =3D 35525},=20 count_=3D1) at ../unix/../src/viewx.cpp:210
#15 0x417f17e0 in = c4_View::Add=20 (this=3D0xbce7aa50, newElem_=3D@0xbce7aa30) at=20 ../include/mk4.inl:13
#16 0x4175495c in mkExt::eq () from=20 /home/portal/build_area/components/do_database_lib/libdoDatabase.so
 
Any suggestions of what might be wrong = would be=20 much appreciated.
 
Best regards,
Ian = Fairclough.
------=_NextPart_000_0778_01C3D457.6BBA2370-- From bkelley@wi.mit.edu Wed Jan 7 04:21:39 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3FEA5B3AE8 for ; Wed, 7 Jan 2004 04:21:39 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXFWxAVs for ; Tue, 6 Jan 2004 21:21:37 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id C95CFB3A8E for ; Wed, 7 Jan 2004 04:21:36 +0800 (WST) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004010615213208004 for ; Tue, 06 Jan 2004 15:21:32 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HR34JW00.OSO for ; Tue, 6 Jan 2004 15:21:32 -0500 Message-ID: <3FFB1939.3050202@wi.mit.edu> Date: Tue, 06 Jan 2004 15:23:21 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Potentially Stupid Question X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 20:21:39 -0000 Based on some previous posts to the metakit news group, I have learned that a metakit storage can be about 1.5 Gigabytes in storage before performance starts to decline. I.e. memory mapped access is no longer viable. What happens if you have two storages open? Can each be 1.5 Gigabytes or does memory mapping not really scale this way. Thanks Brian. From jcw@equi4.com Wed Jan 7 04:39:55 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0D8C3B3AE8 for ; Wed, 7 Jan 2004 04:39:55 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXpE2FZQ for ; Tue, 6 Jan 2004 21:39:51 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 6565FB3A8E for ; Wed, 7 Jan 2004 04:39:51 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 1D23683C26; Tue, 6 Jan 2004 21:39:49 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 74DE183914 for ; Tue, 6 Jan 2004 21:39:46 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <3FFB1939.3050202@wi.mit.edu> References: <3FFB1939.3050202@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <7721BF7C-4088-11D8-A456-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Potentially Stupid Question Date: Tue, 6 Jan 2004 21:39:46 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 20:39:55 -0000 Brian Kelley wrote: > Based on some previous posts to the metakit news group, I have learned > that a metakit storage can be about 1.5 Gigabytes in storage before > performance starts to decline. I.e. memory mapped access is no longer > viable. Good to know. > What happens if you have two storages open? Can each be 1.5 Gigabytes > or does memory mapping not really scale this way. Nope - address space is a per-process limitation. The real way out is 64-bit address space machines. You may be able to squeeze some slack with redundancy reduction, compression, etc - but it'll probably be hard and may not even offer much payback. If you have big data items, you can put them on file, seek/read as needed, and manage space in MK - but that too will take some work. -jcw PS. I disagree with the title! From Murat_Berk@bmc.com Wed Jan 7 04:49:36 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3C104B3AE8 for ; Wed, 7 Jan 2004 04:49:36 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXvTGjCr for ; Tue, 6 Jan 2004 21:49:34 +0100 (CET) Received: from creeper.bmc.com (fw-us-hou19.bmc.com [198.207.223.240]) by dizzie.triqs.com (Postfix) with ESMTP id 7BAD1B3A8E for ; Wed, 7 Jan 2004 04:49:33 +0800 (WST) Received: from EC02-HOU.BMC.COM (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2-BMC/8.10.2) with ESMTP id i06KnJI18328; Tue, 6 Jan 2004 14:49:19 -0600 (CST) Received: by ec02-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Tue, 6 Jan 2004 14:48:33 -0600 Message-ID: <3DAC9CD31458D411BCE700D0B75D0A130B7EF386@ES09-HOU.bmc.com> From: "Berk, Murat" To: "'Brian Kelley'" , metakit@equi4.com Subject: RE: [Metakit] Potentially Stupid Question Date: Tue, 6 Jan 2004 14:48:21 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 20:49:36 -0000 Unless you close and reopen the files on demand, this is not possible. We used files as large as 500 MB. We open upto 50 or more files concurrently in multi threaded application. But the total we could push w/o 64 application is 3 GB. (You need to run Windows 2000 advanced server or any W2003 server to do this. Linux already allows 3GB out of box. > -----Original Message----- > From: Brian Kelley [mailto:bkelley@wi.mit.edu] > Sent: Tuesday, January 06, 2004 3:23 PM > To: metakit@equi4.com > Subject: [Metakit] Potentially Stupid Question > > > Based on some previous posts to the metakit news group, I > have learned > that a metakit storage can be about 1.5 Gigabytes in storage before > performance starts to decline. I.e. memory mapped access is > no longer > viable. > > What happens if you have two storages open? Can each be 1.5 > Gigabytes > or does memory mapping not really scale this way. > > Thanks > Brian. > > > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From mbloore@yahoo.com Thu Jan 8 02:24:44 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 5605CB3A9A for ; Thu, 8 Jan 2004 02:24:44 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXI72t6J for ; Wed, 7 Jan 2004 19:24:43 +0100 (CET) Received: from web14311.mail.yahoo.com (web14311.mail.yahoo.com [216.136.224.61]) by dizzie.triqs.com (Postfix) with SMTP id B0FC5B3A8E for ; Thu, 8 Jan 2004 02:24:42 +0800 (WST) Message-ID: <20040107182441.73523.qmail@web14311.mail.yahoo.com> Received: from [65.95.141.171] by web14311.mail.yahoo.com via HTTP; Wed, 07 Jan 2004 10:24:41 PST Date: Wed, 7 Jan 2004 10:24:41 -0800 (PST) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Find() depends on order of properties X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 18:24:44 -0000 i have found that if i hash or order a view on the first two columns, and then use Find(), it only works if the row passed to Find was constructed in the same order that the properties were given in the expression passed to GetAs(). there is no problem if the view is not hashed or ordered. i have not tried any other kind of view. i am using MK 2.9.4.2 under WinXP. i tried both the MSVC6 and Intel8 compilers. here is some code which i added to tmapped.cpp to demonstrate the problem. there was no test of mapping more than one column already there: B (m08, Hash 2 cols and Find using operator +, 0) W (m08a); { c4_IntProp p1 ("p1"); c4_StringProp p2 ("p2"); c4_IntProp p3 ("p3"); c4_Storage s1 ("m08a", true); c4_View d1 = s1.GetAs ("d1[p1:I,p2:S,p3:I,p4:B]"); c4_View m1 = s1.GetAs ("m1[_H:I,_R:I]"); c4_View h1 = d1.Ordered (2);//.Hash (m1, 2); h1.Add (p1[1] + p2["one"] + p3[11]); h1.Add (p1[2] + p2["two"] + p3[22]); h1.Add (p1[3] + p2["three"] + p3[33]); int i = h1.Find (p1[2] + p2["two"]); A (i == 1); // THIS FAILS: int j = h1.Find (p2["two"] + p1[2]); A (j == 1); s1.Commit(); } D (m08a); R (m08a); E; B (m09, Hash 2 cols and Find using C4_Row, 0) W (m09a); { c4_IntProp p1 ("p1"); c4_StringProp p2 ("p2"); c4_IntProp p3 ("p3"); c4_Storage s1 ("m09a", true); c4_View d1 = s1.GetAs ("d1[p1:I,p2:S,p3:I,p4:B]"); c4_View m1 = s1.GetAs ("m1[_H:I,_R:I]"); c4_View h1 = d1.Ordered (2);//.Hash (m1, 2); h1.Add (p1[1] + p2["one"] + p3[11]); h1.Add (p1[2] + p2["two"] + p3[22]); h1.Add (p1[3] + p2["three"] + p3[33]); c4_Row forward; p1 (forward) = 2; p2 (forward) = "two"; int k = h1.Find (forward); A (k == 1); c4_Row backward; p2 (backward) = "two"; p1 (backward) = 2; // THIS FAILS: int l = h1.Find (backward); A (l == 1); s1.Commit(); } D (m09a); R (m09a); E; ===== -- mARK bLOORE __________________________________ Do you Yahoo!? Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus From rus20376@infolaunch.com Thu Jan 8 03:39:10 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1C895B3A9A for ; Thu, 8 Jan 2004 03:39:10 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXJGAwbF for ; Wed, 7 Jan 2004 20:39:09 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id BBA56B3A8E for ; Thu, 8 Jan 2004 03:39:08 +0800 (WST) Received: from infolaunch.com (PC502.bwh.harvard.edu [170.223.248.119]) i07Jd6L05824 for ; Wed, 7 Jan 2004 14:39:06 -0500 (EST) Message-ID: <3FFC6059.7060609@infolaunch.com> Date: Wed, 07 Jan 2004 14:39:05 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3b) Gecko/20030210 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] accessing multiple db storage files in succession X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 19:39:10 -0000 Currently, I have some code which acesses mutiple metakit storage files in succession. This is something I would like to speed up a bit. Suppose I have a loop which is something like this(simplified for clarity) for mkFile in files: db=metakit.storage(mkFile,1) #now do something. Maybe this is just a stupid question but is there a better, as in faster, way of accessing mutiple mk storage files? The answer cannot be having all the data in a single mk storage file however. Potentially, each one of these may reach a maximum size limit for memory mapped files. Just a thought, but would accessing each of the files in a seperate thread help much? Would that create other issues that I may not anticipate at first glance? From jyl@mod3.net Thu Jan 8 05:39:55 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 75E78B3A9A for ; Thu, 8 Jan 2004 05:39:55 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXLOIOf5 for ; Wed, 7 Jan 2004 22:39:53 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id 08745B3A8E for ; Thu, 8 Jan 2004 05:39:53 +0800 (WST) Received: (qmail 11984 invoked by uid 48); 7 Jan 2004 21:39:51 -0000 Received: from 192.18.42.10 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Wed, 7 Jan 2004 13:39:51 -0800 (PST) Message-ID: <62635.192.18.42.10.1073511591.squirrel@mod3.net> Date: Wed, 7 Jan 2004 13:39:51 -0800 (PST) Subject: Re: [Metakit] accessing multiple db storage files in succession From: "Jacob Levy" To: In-Reply-To: <3FFC6059.7060609@infolaunch.com> References: <3FFC6059.7060609@infolaunch.com> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 21:39:55 -0000 Since you're saying that each of these files may be as large as the virtual memory of the process will allow, it follows that you cannot have them open at the same time. Thus, putting the processing of each file into a separate thread will not help, and you're stuck iterating through them sequentially. If the files are very large, larger than the physical memory on the machine, it will be very beneficial to instead look at the code that processes each file. This is because if you're accessing the data in a random way, you'll cause massive page faulting, whereas if you access the data sequentially its likely that the OS will help and prefetch some of the pages, thereby avoiding the waits for page faults. So my advice is to carefully look at the processing-per-file code and not try to optimize the way files are processed (i.e. in what order or concurrency). --JYL > Currently, I have some code which acesses mutiple metakit storage files > in succession. This is something I would like to speed up a bit. > Suppose I have a loop which is something like this(simplified for > clarity) for mkFile in files: > db=metakit.storage(mkFile,1) > #now do something. > Maybe this is just a stupid question but is there a better, as in > faster, way of accessing mutiple mk storage files? The answer cannot be > having all the data in a single mk storage file however. Potentially, > each one of these may reach a maximum size limit for memory mapped > files. Just a thought, but would accessing each of the files in a > seperate thread help much? Would that create other issues that I may not > anticipate at first glance? > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From ian.fairclough@intelligent-ms.com Fri Jan 9 23:58:01 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 02A5EB3A95; Fri, 9 Jan 2004 23:58:01 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXt0tMuB; Fri, 9 Jan 2004 16:57:58 +0100 (CET) Received: from scratchy.base.ims (unknown [194.74.0.138]) by dizzie.triqs.com (Postfix) with ESMTP id 0E789B3A8D; Fri, 9 Jan 2004 23:57:58 +0800 (WST) Received: from grpware.intelligent-ms.com (grpware.base.ims [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id i09Forl21350; Fri, 9 Jan 2004 15:50:53 GMT Received: from imsgsx6tvznbng ([192.168.200.184])i09Faku06080; Fri, 9 Jan 2004 15:36:46 GMT Message-ID: <0aac01c3d6c9$bf6f2a50$b4c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Subject: Re: [Metakit] Core Dump - use of sub views ? Date: Fri, 9 Jan 2004 16:00:46 -0000 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0AA9_01C3D6C9.BE8D07B0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 15:58:01 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0AA9_01C3D6C9.BE8D07B0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Are there issues related to using sub-views ? I seem to be getting a variety of different core dumps (the trace of one = example appears below) when attempting to increase the size of a view = that has sub-views. Any information would be much appreciated. Best regards, Ian Fairclough. ----- Original Message -----=20 From: Ian Fairclough=20 To: jcw@equi4.com=20 Cc: metakit@equi4.com=20 Sent: Tuesday, January 06, 2004 1:17 PM Subject: Re: [Metakit] Core Dump JC, Thanks for the advice. I re-compiled as you suggested (with = assertions and debug symbols enabled), and I obtained a core dump with = the following stack trace : #0 0x41513fa1 in kill () from /lib/libc.so.6 #1 0x41365fe6 in raise (sig=3D6) at signals.c:65 #2 0x41515264 in abort () from /lib/libc.so.6 #3 0x414b7e55 in __default_terminate () at ../../gcc/libgcc2.c:-1 #4 0x414b7e72 in __terminate () from = /usr/lib/libstdc++-libc6.1-1.so.2 #5 0x4183ac2b in new_eh_context () at ../../gcc/libgcc2.c:3035 #6 0x4183af0d in eh_context_specific () from = /usr/lib/libstdc++-libc6.2-2.so.3 #7 0x4183ae6e in eh_context_initialize () from = /usr/lib/libstdc++-libc6.2-2.so.3 #8 0x4183bb4d in __rethrow (index=3D0x41805a4c) from = /usr/lib/libstdc++-libc6.2-2.so.3 #9 0x417e3ee0 in c4_HandlerSeq::c4_HandlerSeq (this=3D0x3fffec00, = owner_=3D@0x2d93f440, handler_=3D0xf151f20) at = ../unix/../src/handler.cpp:102 #10 0x417e26dc in c4_FormatV::At (this=3D0xf151f20, index_=3D49699) at = ../unix/../src/format.cpp:44 #11 0x417e281a in c4_FormatV::SetupAllSubviews (this=3D0xf151f20) at = ../unix/../src/format.cpp:1027 #12 0x417e2b22 in c4_FormatV::Get (this=3D0xf151f20, index_=3D35525, = length_=3D@0xbce7a8d4) at ../unix/../src/format.cpp:1096 #13 0x417e3b9a in c4_Handler::GetBytes (this=3D0xf151f20, = index_=3D35525, buf_=3D@0xbce7a968, copySmall_=3Dfalse) at = ../unix/../src/handler.cpp:59 #14 0x417f4c87 in c4_Sequence::InsertAt (this=3D0x2e9923b8, = index_=3D0, newElem_=3D{_seq =3D 0x2d93f440, _index =3D 35525}, = count_=3D1) at ../unix/../src/viewx.cpp:210 #15 0x417f17e0 in c4_View::Add (this=3D0xbce7aa50, = newElem_=3D@0xbce7aa30) at ../include/mk4.inl:13 #16 0x4175495c in mkExt::eq () from = /home/portal/build_area/components/do_database_lib/libdoDatabase.so Any suggestions of what might be wrong would be much appreciated. Best regards, Ian Fairclough. ------=_NextPart_000_0AA9_01C3D6C9.BE8D07B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Are there issues related to using = sub-views=20 ?
 
I seem to be getting a variety of = different core=20 dumps (the trace of one example appears below) when attempting to = increase the=20 size of a view that has sub-views.
 
Any information would be much=20 appreciated.
 
Best regards,
Ian Fairclough.
----- Original Message -----
From:=20 Ian Fairclough =
Sent: Tuesday, January 06, 2004 = 1:17=20 PM
Subject: Re: [Metakit] Core = Dump

JC,
 
Thanks for the advice.  I = re-compiled as you=20 suggested (with assertions and debug symbols enabled), and I obtained = a core=20 dump with the following stack trace :
 
#0  0x41513fa1 in kill () from=20 /lib/libc.so.6
#1  0x41365fe6 in raise (sig=3D6) at=20 signals.c:65
#2  0x41515264 in abort () from=20 /lib/libc.so.6
#3  0x414b7e55 in __default_terminate () at=20 ../../gcc/libgcc2.c:-1
#4  0x414b7e72 in __terminate () from=20 /usr/lib/libstdc++-libc6.1-1.so.2
#5  0x4183ac2b in = new_eh_context ()=20 at ../../gcc/libgcc2.c:3035
#6  0x4183af0d in = eh_context_specific ()=20 from /usr/lib/libstdc++-libc6.2-2.so.3
#7  0x4183ae6e in=20 eh_context_initialize () from = /usr/lib/libstdc++-libc6.2-2.so.3
#8 =20 0x4183bb4d in __rethrow (index=3D0x41805a4c) from=20 /usr/lib/libstdc++-libc6.2-2.so.3
#9  0x417e3ee0 in=20 c4_HandlerSeq::c4_HandlerSeq (this=3D0x3fffec00, owner_=3D@0x2d93f440, = handler_=3D0xf151f20)=20 at ../unix/../src/handler.cpp:102
#10 0x417e26dc in c4_FormatV::At=20 (this=3D0xf151f20, index_=3D49699) at = ../unix/../src/format.cpp:44
#11=20 0x417e281a in c4_FormatV::SetupAllSubviews (this=3D0xf151f20) at=20 ../unix/../src/format.cpp:1027
#12 0x417e2b22 in c4_FormatV::Get=20 (this=3D0xf151f20, index_=3D35525, length_=3D@0xbce7a8d4) at=20 ../unix/../src/format.cpp:1096
#13 0x417e3b9a in = c4_Handler::GetBytes=20 (this=3D0xf151f20, index_=3D35525, buf_=3D@0xbce7a968, = copySmall_=3Dfalse) at=20 ../unix/../src/handler.cpp:59
#14 0x417f4c87 in = c4_Sequence::InsertAt=20 (this=3D0x2e9923b8, index_=3D0, newElem_=3D{_seq =3D 0x2d93f440, = _index =3D 35525},=20 count_=3D1) at ../unix/../src/viewx.cpp:210
#15 0x417f17e0 in = c4_View::Add=20 (this=3D0xbce7aa50, newElem_=3D@0xbce7aa30) at=20 ../include/mk4.inl:13
#16 0x4175495c in mkExt::eq () from=20 = /home/portal/build_area/components/do_database_lib/libdoDatabase.so
 
Any suggestions of what might be = wrong would be=20 much appreciated.
 
Best regards,
Ian=20 Fairclough.
------=_NextPart_000_0AA9_01C3D6C9.BE8D07B0-- From mbloore@yahoo.com Sat Jan 10 02:02:08 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E93DAB3A95 for ; Sat, 10 Jan 2004 02:02:07 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXKbWibi for ; Fri, 9 Jan 2004 19:02:06 +0100 (CET) Received: from web14303.mail.yahoo.com (web14303.mail.yahoo.com [216.136.173.79]) by dizzie.triqs.com (Postfix) with SMTP id 15E7AB3A8D for ; Sat, 10 Jan 2004 02:02:06 +0800 (WST) Message-ID: <20040109180130.49722.qmail@web14303.mail.yahoo.com> Received: from [65.95.143.93] by web14303.mail.yahoo.com via HTTP; Fri, 09 Jan 2004 10:01:30 PST Date: Fri, 9 Jan 2004 10:01:30 -0800 (PST) From: mARK bLOORE Subject: Re: [Metakit] Core Dump - use of sub views ? To: metakit In-Reply-To: <0aac01c3d6c9$bf6f2a50$b4c8a8c0@imsgsx6tvznbng> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 18:02:08 -0000 i can't shed any light on this, except to say that it CAN work; i have an application that increases the size of subviews and the number of records with subviews without error. i use MK 2.4.9.2 with C++. actually, i have just changed my app to not use subviews, because if there are many records (tens of thousands) with one subview it can get very slow and memory hungry doing subview setup when the view is first accessed. --- Ian Fairclough wrote: > Are there issues related to using sub-views ? > > I seem to be getting a variety of different core dumps (the trace of one example appears below) > when attempting to increase the size of a view that has sub-views. > > Any information would be much appreciated. > > Best regards, > Ian Fairclough. > ----- Original Message ----- > From: Ian Fairclough > To: jcw@equi4.com > Cc: metakit@equi4.com > Sent: Tuesday, January 06, 2004 1:17 PM > Subject: Re: [Metakit] Core Dump > > > JC, > > Thanks for the advice. I re-compiled as you suggested (with assertions and debug symbols > enabled), and I obtained a core dump with the following stack trace : > > #0 0x41513fa1 in kill () from /lib/libc.so.6 > #1 0x41365fe6 in raise (sig=6) at signals.c:65 > #2 0x41515264 in abort () from /lib/libc.so.6 > #3 0x414b7e55 in __default_terminate () at ../../gcc/libgcc2.c:-1 > #4 0x414b7e72 in __terminate () from /usr/lib/libstdc++-libc6.1-1.so.2 > #5 0x4183ac2b in new_eh_context () at ../../gcc/libgcc2.c:3035 > #6 0x4183af0d in eh_context_specific () from /usr/lib/libstdc++-libc6.2-2.so.3 > #7 0x4183ae6e in eh_context_initialize () from /usr/lib/libstdc++-libc6.2-2.so.3 > #8 0x4183bb4d in __rethrow (index=0x41805a4c) from /usr/lib/libstdc++-libc6.2-2.so.3 > #9 0x417e3ee0 in c4_HandlerSeq::c4_HandlerSeq (this=0x3fffec00, owner_=@0x2d93f440, > handler_=0xf151f20) at ../unix/../src/handler.cpp:102 > #10 0x417e26dc in c4_FormatV::At (this=0xf151f20, index_=49699) at > ../unix/../src/format.cpp:44 > #11 0x417e281a in c4_FormatV::SetupAllSubviews (this=0xf151f20) at > ../unix/../src/format.cpp:1027 > #12 0x417e2b22 in c4_FormatV::Get (this=0xf151f20, index_=35525, length_=@0xbce7a8d4) at > ../unix/../src/format.cpp:1096 > #13 0x417e3b9a in c4_Handler::GetBytes (this=0xf151f20, index_=35525, buf_=@0xbce7a968, > copySmall_=false) at ../unix/../src/handler.cpp:59 > #14 0x417f4c87 in c4_Sequence::InsertAt (this=0x2e9923b8, index_=0, newElem_={_seq = > 0x2d93f440, _index = 35525}, count_=1) at ../unix/../src/viewx.cpp:210 > #15 0x417f17e0 in c4_View::Add (this=0xbce7aa50, newElem_=@0xbce7aa30) at > ../include/mk4.inl:13 > #16 0x4175495c in mkExt::eq () from > /home/portal/build_area/components/do_database_lib/libdoDatabase.so > > Any suggestions of what might be wrong would be much appreciated. > > Best regards, > Ian Fairclough.> _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > __________________________________ Do you Yahoo!? Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus From jcw@equi4.com Sat Jan 10 02:18:45 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id BBEFCB3A95 for ; Sat, 10 Jan 2004 02:18:45 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXaxgoFL for ; Fri, 9 Jan 2004 19:18:43 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 4B07AB3A8D for ; Sat, 10 Jan 2004 02:18:43 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 3B94783C26; Fri, 9 Jan 2004 19:18:41 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id D6FE983914 for ; Fri, 9 Jan 2004 19:18:39 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <0aac01c3d6c9$bf6f2a50$b4c8a8c0@imsgsx6tvznbng> References: <0aac01c3d6c9$bf6f2a50$b4c8a8c0@imsgsx6tvznbng> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <3F82B285-42D0-11D8-A456-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Core Dump - use of sub views ? Date: Fri, 9 Jan 2004 19:18:39 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 18:18:46 -0000 Ian Fairclough wrote: > Are there issues related to using sub-views ? None that I know of. There is no real distinction: top-level are subviews of a hidden row. Blocked views also resize and manipulate sub-views all the time. > I seem to be getting a variety of different core dumps (the trace of > one example appears below) when attempting to increase the size of a > view that has sub-views. Can you post or email the definition string you're using? -jcw From ian.fairclough@intelligent-ms.com Sun Jan 11 02:01:48 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0078CB3A97; Sun, 11 Jan 2004 02:01:48 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XX7ufWpQ; Sat, 10 Jan 2004 19:01:43 +0100 (CET) Received: from scratchy.base.ims (unknown [194.74.0.138]) by dizzie.triqs.com (Postfix) with ESMTP id 2F2CFB3A88; Sun, 11 Jan 2004 02:01:43 +0800 (WST) Received: from grpware.intelligent-ms.com (mail.intelligent-ms.com [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id i0AHsgl25568; Sat, 10 Jan 2004 17:54:42 GMT Received: from imsgsx6tvznbng ([192.168.200.185])i0AHeRu17370; Sat, 10 Jan 2004 17:40:27 GMT Message-ID: <0b4601c3d7a4$30ad54a0$b4c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Subject: Re: [Metakit] Core Dump - use of sub views ? Date: Sat, 10 Jan 2004 18:04:27 -0000 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0B43_01C3D7A4.30149710" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jan 2004 18:01:48 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0B43_01C3D7A4.30149710 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Following are the first couple of lines that are generated by the = metakit dump utility - they show the structure of the view that I am = using. The view structure is auto-generated dynamically from CORBA = type-codes (I am using metakit to cache data that has it's structure = defined in CORBA IDL). Hence I am using sub-views to store data that is = contained within nested sequences defined within CORBA data structures. Thanks again if you can provide any pointers for me. I dont know if it might be significant, but some core dumps I get are = generated with stack traces indicating a problem during commit. Is it = possible that I am experiencing some kind of memory corruption that is = showing itself when either trying to extend the size of a view or = commiting data to a file ? (see stack trace below) Best regards, Ian Fairclough. data/master_DO_WO_workOrderType.dat: 1 properties = DO_WO_workOrderType[dos.report.store:S,dos.report.woNum:S,dos.report.woTy= pe:S,dos.report.taskDesc:S,dos.report.requestDate.isNull:I,dos.report.req= uestDate.value.year:I, = dos.report.requestDate.value.month:I,dos.report.requestDate.value.date:I,= dos.report.requestTime.isNull:I,dos.report.requestTime.value.hour:I, = dos.report.requestTime.value.minute:I,dos.report.requestTime.value.second= :I,dos.report.schedStartDate.isNull:I,dos.report.schedStartDate.value.yea= r:I, = dos.report.schedStartDate.value.month:I,dos.report.schedStartDate.value.d= ate:I,dos.report.schedStartTime.isNull:I,dos.report.schedStartTime.value.= hour:I, = dos.report.schedStartTime.value.minute:I,dos.report.schedStartTime.value.= second:I,dos.report.completionDate.isNull:I,dos.report.completionDate.val= ue.year:I, = dos.report.completionDate.value.month:I,dos.report.completionDate.value.d= ate:I,dos.report.completionTime.isNull:I,dos.report.completionTime.value.= hour:I, = dos.report.completionTime.value.minute:I,dos.report.completionTime.value.= second:I,dos.report.priority:F,dos.report.priorityDescription:S,dos.repor= t.rTimeHrs.isNull:I, = dos.report.rTimeHrs.value:I,dos.report.contractorId:S,dos.report.contract= orName:S,dos.report.callAgentNotes:S,dos.report.equipment[dos.report.equi= pment.assetNumber:S,dos.report.equipment.description:S,dos.report.equipme= nt.department:S,dos.report.equipment.area:S, = dos.report.equipment.position:S],dos.report.visits[dos.report.visits.audi= tDate.isNull:I,dos.report.visits.auditDate.value.year:I,dos.report.visits= .auditDate.value.month:I,dos.report.visits.auditDate.value.date:I, = dos.report.visits.onTime.isNull:I,dos.report.visits.onTime.value.hour:I,d= os.report.visits.onTime.value.minute:I,dos.report.visits.onTime.value.sec= ond:I, = dos.report.visits.offTime.isNull:I,dos.report.visits.offTime.value.hour:I= ,dos.report.visits.offTime.value.minute:I,dos.report.visits.offTime.value= .second:I, = dos.report.visits.cnoNumber:S,dos.report.visits.ptwNumber:S,dos.report.vi= sits.employeeContractorId:S,dos.report.visits.employeeContractorName:S], = dos.updatestamp:I,details.woNum:S,details.userId:S,details.userName:S,det= ails.sbtId[details.sbtId:I],details.stopTime:I,details.woType:S,details.s= toreNumber:S,details.confirmationNum:S,details.completed:I,details.materi= alsUsed:I, = details.equipmentWorking:I,details.solutionCode:S,details.gasUsed:S,detai= ls.reason:S] VIEW 31 rows =3D DO_WO_workOrderType:V 0: subview 'DO_WO_workOrderType' VIEW 97690 rows =3D dos.report.store:S dos.report.woNum:S = dos.report.woType:S dos.report.taskDesc:S = dos.report.requestDate.isNull:I dos.report.requestDate.value.year:I = dos.report.requestDate.value.month:I dos.report.requestDate.value.date:I = dos.report.requestTime.isNull:I dos.report.requestTime.value.hour:I = dos.report.requestTime.value.minute:I = dos.report.requestTime.value.second:I dos.report.schedStartDate.isNull:I = dos.report.schedStartDate.value.year:I = dos.report.schedStartDate.value.month:I = dos.report.schedStartDate.value.date:I = dos.report.schedStartTime.isNull:I = dos.report.schedStartTime.value.hour:I = dos.report.schedStartTime.value.minute:I = dos.report.schedStartTime.value.second:I = dos.report.completionDate.isNull:I = dos.report.completionDate.value.year:I = dos.report.completionDate.value.month:I = dos.report.completionDate.value.date:I = dos.report.completionTime.isNull:I = dos.report.completionTime.value.hour:I = dos.report.completionTime.value.minute:I = dos.report.completionTime.value.second:I dos.report.priority:F = dos.report.priorityDescription:S dos.report.rTimeHrs.isNull:I = dos.report.rTimeHrs.value:I dos.report.contractorId:S = dos.report.contractorName:S dos.report.callAgentNotes:S = dos.report.equipment:V dos.report.visits:V dos.updatestamp:I = details.woNum:S details.userId:S details.userName:S details.sbtId:V = details.stopTime:I details.woType:S details.storeNumber:S = details.confirmationNum:S details.completed:I details.materialsUsed:I = details.equipmentWorking:I details.solutionCode:S details.gasUsed:S = details.reason:S #0 0x41513fa1 in kill () from /lib/libc.so.6 #1 0x41365fe6 in raise (sig=3D6) at signals.c:65 #2 0x41515264 in abort () from /lib/libc.so.6 #3 0x4150e05d in __assert_fail () from /lib/libc.so.6 #4 0x417d66a9 in c4_Column::Validate (this=3D0xbfe7a6f8) at = ../src/univ.inl:44 #5 0x417d6755 in c4_Column::~c4_Column (this=3D0xbfe7a6f8, = __in_chrg=3D2) at ../unix/../src/column.cpp:86 #6 0x417e44a4 in c4_FormatV::Commit (this=3D0x8562250, = ar_=3D@0xbfe7a968) at ../include/mk4.inl:403 #7 0x417e947f in c4_SaveContext::CommitSequence (this=3D0xbfe7a968, = seq_=3D@0x855d290, selfDesc_=3Dfalse) at ../unix/../src/persist.cpp:856 #8 0x417e42ba in c4_FormatV::Commit (this=3D0x85616c8, = ar_=3D@0xbfe7a968) at ../unix/../src/format.cpp:1251 #9 0x417e947f in c4_SaveContext::CommitSequence (this=3D0xbfe7a968, = seq_=3D@0x855cf30, selfDesc_=3Dtrue) at ../unix/../src/persist.cpp:856 #10 0x417e8cd1 in c4_SaveContext::SaveIt (this=3D0xbfe7a968, = root_=3D@0x855cf30, spacePtr_=3D0x81fab78, rootWalk_=3D@0x81fab88) at = ../unix/../src/persist.cpp:722 #11 0x417e992d in c4_Persist::Commit (this=3D0x81fab78, full_=3Dfalse) = at ../unix/../src/persist.cpp:969 #12 0x417f0ab1 in c4_Storage::Commit (this=3D0x81e53a0, full_=3Dfalse) = at ../include/mk4.inl:76 #13 0x417489b9 in corbaMkTools::commit () from = /home/portal/build_area/components/do_database_lib/libdoDatabase.so ----- Original Message -----=20 From: Ian Fairclough=20 To: jcw@equi4.com=20 Cc: metakit@equi4.com=20 Sent: Friday, January 09, 2004 4:00 PM Subject: Re: [Metakit] Core Dump - use of sub views ? Are there issues related to using sub-views ? I seem to be getting a variety of different core dumps (the trace of = one example appears below) when attempting to increase the size of a = view that has sub-views. Any information would be much appreciated. Best regards, Ian Fairclough. ----- Original Message -----=20 From: Ian Fairclough=20 To: jcw@equi4.com=20 Cc: metakit@equi4.com=20 Sent: Tuesday, January 06, 2004 1:17 PM Subject: Re: [Metakit] Core Dump JC, Thanks for the advice. I re-compiled as you suggested (with = assertions and debug symbols enabled), and I obtained a core dump with = the following stack trace : #0 0x41513fa1 in kill () from /lib/libc.so.6 #1 0x41365fe6 in raise (sig=3D6) at signals.c:65 #2 0x41515264 in abort () from /lib/libc.so.6 #3 0x414b7e55 in __default_terminate () at ../../gcc/libgcc2.c:-1 #4 0x414b7e72 in __terminate () from = /usr/lib/libstdc++-libc6.1-1.so.2 #5 0x4183ac2b in new_eh_context () at ../../gcc/libgcc2.c:3035 #6 0x4183af0d in eh_context_specific () from = /usr/lib/libstdc++-libc6.2-2.so.3 #7 0x4183ae6e in eh_context_initialize () from = /usr/lib/libstdc++-libc6.2-2.so.3 #8 0x4183bb4d in __rethrow (index=3D0x41805a4c) from = /usr/lib/libstdc++-libc6.2-2.so.3 #9 0x417e3ee0 in c4_HandlerSeq::c4_HandlerSeq (this=3D0x3fffec00, = owner_=3D@0x2d93f440, handler_=3D0xf151f20) at = ../unix/../src/handler.cpp:102 #10 0x417e26dc in c4_FormatV::At (this=3D0xf151f20, index_=3D49699) = at ../unix/../src/format.cpp:44 #11 0x417e281a in c4_FormatV::SetupAllSubviews (this=3D0xf151f20) at = ../unix/../src/format.cpp:1027 #12 0x417e2b22 in c4_FormatV::Get (this=3D0xf151f20, index_=3D35525, = length_=3D@0xbce7a8d4) at ../unix/../src/format.cpp:1096 #13 0x417e3b9a in c4_Handler::GetBytes (this=3D0xf151f20, = index_=3D35525, buf_=3D@0xbce7a968, copySmall_=3Dfalse) at = ../unix/../src/handler.cpp:59 #14 0x417f4c87 in c4_Sequence::InsertAt (this=3D0x2e9923b8, = index_=3D0, newElem_=3D{_seq =3D 0x2d93f440, _index =3D 35525}, = count_=3D1) at ../unix/../src/viewx.cpp:210 #15 0x417f17e0 in c4_View::Add (this=3D0xbce7aa50, = newElem_=3D@0xbce7aa30) at ../include/mk4.inl:13 #16 0x4175495c in mkExt::eq () from = /home/portal/build_area/components/do_database_lib/libdoDatabase.so Any suggestions of what might be wrong would be much appreciated. Best regards, Ian Fairclough. ------=_NextPart_000_0B43_01C3D7A4.30149710 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Following are the first couple of lines = that are=20 generated by the metakit dump utility - they show the structure of the = view that=20 I am using.  The view structure is auto-generated dynamically from = CORBA=20 type-codes (I am using metakit to cache data that has it's = structure=20 defined in CORBA IDL).  Hence I am using sub-views to store data = that is=20 contained within nested sequences defined within CORBA data=20 structures.
 
Thanks again if you can provide any = pointers for=20 me.
 
I dont know if it might be significant, = but some=20 core dumps I get are generated with stack traces indicating a problem = during=20 commit.  Is it possible that I am experiencing some kind of memory=20 corruption that is showing itself when either trying to extend the size = of a=20 view or commiting data to a file ? (see stack trace below)
 
Best regards,
Ian Fairclough.
 
 
data/master_DO_WO_workOrderType.dat: 1=20 properties
 =20 DO_WO_workOrderType[dos.report.store:S,dos.report.woNum:S,dos.report.woTy= pe:S,dos.report.taskDesc:S,dos.report.requestDate.isNull:I,dos.report.req= uestDate.value.year:I,
 dos.report.requestDate.value.month:I,dos.report.requestDat= e.value.date:I,dos.report.requestTime.isNull:I,dos.report.requestTime.val= ue.hour:I,
 dos.report.requestTime.value.minute:I,dos.report.requestTi= me.value.second:I,dos.report.schedStartDate.isNull:I,dos.report.schedStar= tDate.value.year:I,
 dos.report.schedStartDate.value.month:I,dos.report.schedSt= artDate.value.date:I,dos.report.schedStartTime.isNull:I,dos.report.schedS= tartTime.value.hour:I,
 dos.report.schedStartTime.value.minute:I,dos.report.schedS= tartTime.value.second:I,dos.report.completionDate.isNull:I,dos.report.com= pletionDate.value.year:I,
 dos.report.completionDate.value.month:I,dos.report.complet= ionDate.value.date:I,dos.report.completionTime.isNull:I,dos.report.comple= tionTime.value.hour:I,
 dos.report.completionTime.value.minute:I,dos.report.comple= tionTime.value.second:I,dos.report.priority:F,dos.report.priorityDescript= ion:S,dos.report.rTimeHrs.isNull:I,
 dos.report.rTimeHrs.value:I,dos.report.contractorId:S,dos.= report.contractorName:S,dos.report.callAgentNotes:S,dos.report.equipment[= dos.report.equipment.assetNumber:S,dos.report.equipment.description:S,dos= .report.equipment.department:S,dos.report.equipment.area:S,
 dos.report.equipment.position:S],dos.report.visits[dos.rep= ort.visits.auditDate.isNull:I,dos.report.visits.auditDate.value.year:I,do= s.report.visits.auditDate.value.month:I,dos.report.visits.auditDate.value= .date:I,
 dos.report.visits.onTime.isNull:I,dos.report.visits.onTime= .value.hour:I,dos.report.visits.onTime.value.minute:I,dos.report.visits.o= nTime.value.second:I,
 dos.report.visits.offTime.isNull:I,dos.report.visits.offTi= me.value.hour:I,dos.report.visits.offTime.value.minute:I,dos.report.visit= s.offTime.value.second:I,
 dos.report.visits.cnoNumber:S,dos.report.visits.ptwNumber:= S,dos.report.visits.employeeContractorId:S,dos.report.visits.employeeCont= ractorName:S],
 dos.updatestamp:I,details.woNum:S,details.userId:S,details= .userName:S,details.sbtId[details.sbtId:I],details.stopTime:I,details.woT= ype:S,details.storeNumber:S,details.confirmationNum:S,details.completed:I= ,details.materialsUsed:I,
 details.equipmentWorking:I,details.solutionCode:S,details.= gasUsed:S,details.reason:S]
 
 VIEW    31 rows = =3D=20 DO_WO_workOrderType:V
    0: subview=20 'DO_WO_workOrderType'
   VIEW 97690 rows =3D = dos.report.store:S=20 dos.report.woNum:S dos.report.woType:S dos.report.taskDesc:S=20 dos.report.requestDate.isNull:I dos.report.requestDate.value.year:I=20 dos.report.requestDate.value.month:I dos.report.requestDate.value.date:I = dos.report.requestTime.isNull:I dos.report.requestTime.value.hour:I=20 dos.report.requestTime.value.minute:I = dos.report.requestTime.value.second:I=20 dos.report.schedStartDate.isNull:I = dos.report.schedStartDate.value.year:I=20 dos.report.schedStartDate.value.month:I = dos.report.schedStartDate.value.date:I=20 dos.report.schedStartTime.isNull:I = dos.report.schedStartTime.value.hour:I=20 dos.report.schedStartTime.value.minute:I=20 dos.report.schedStartTime.value.second:I = dos.report.completionDate.isNull:I=20 dos.report.completionDate.value.year:I = dos.report.completionDate.value.month:I=20 dos.report.completionDate.value.date:I = dos.report.completionTime.isNull:I=20 dos.report.completionTime.value.hour:I = dos.report.completionTime.value.minute:I=20 dos.report.completionTime.value.second:I dos.report.priority:F=20 dos.report.priorityDescription:S dos.report.rTimeHrs.isNull:I=20 dos.report.rTimeHrs.value:I dos.report.contractorId:S=20 dos.report.contractorName:S dos.report.callAgentNotes:S = dos.report.equipment:V=20 dos.report.visits:V dos.updatestamp:I details.woNum:S details.userId:S=20 details.userName:S details.sbtId:V details.stopTime:I details.woType:S=20 details.storeNumber:S details.confirmationNum:S details.completed:I=20 details.materialsUsed:I details.equipmentWorking:I = details.solutionCode:S=20 details.gasUsed:S details.reason:S
 
#0  0x41513fa1 in kill () from=20 /lib/libc.so.6
#1  0x41365fe6 in raise (sig=3D6) at=20 signals.c:65
#2  0x41515264 in abort () from = /lib/libc.so.6
#3 =20 0x4150e05d in __assert_fail () from /lib/libc.so.6
#4  = 0x417d66a9 in=20 c4_Column::Validate (this=3D0xbfe7a6f8) at = ../src/univ.inl:44
#5 =20 0x417d6755 in c4_Column::~c4_Column (this=3D0xbfe7a6f8, __in_chrg=3D2) = at=20 ../unix/../src/column.cpp:86
#6  0x417e44a4 in = c4_FormatV::Commit=20 (this=3D0x8562250, ar_=3D@0xbfe7a968) at=20 ../include/mk4.inl:403
#7  0x417e947f in = c4_SaveContext::CommitSequence=20 (this=3D0xbfe7a968, seq_=3D@0x855d290,=20 selfDesc_=3Dfalse) at ../unix/../src/persist.cpp:856
#8  = 0x417e42ba in=20 c4_FormatV::Commit (this=3D0x85616c8, ar_=3D@0xbfe7a968) at=20 ../unix/../src/format.cpp:1251
#9  0x417e947f in=20 c4_SaveContext::CommitSequence (this=3D0xbfe7a968, seq_=3D@0x855cf30, = selfDesc_=3Dtrue) at=20 ../unix/../src/persist.cpp:856
#10 0x417e8cd1 in = c4_SaveContext::SaveIt=20 (this=3D0xbfe7a968, root_=3D@0x855cf30,=20 spacePtr_=3D0x81fab78, rootWalk_=3D@0x81fab88) at=20 ../unix/../src/persist.cpp:722
#11 0x417e992d in c4_Persist::Commit=20 (this=3D0x81fab78, full_=3Dfalse) at = ../unix/../src/persist.cpp:969
#12=20 0x417f0ab1 in c4_Storage::Commit (this=3D0x81e53a0, full_=3Dfalse) at=20 ../include/mk4.inl:76
#13 0x417489b9 in corbaMkTools::commit () from=20 /home/portal/build_area/components/do_database_lib/libdoDatabase.so
----- Original Message -----
From:=20 Ian Fairclough =
Sent: Friday, January 09, 2004 = 4:00=20 PM
Subject: Re: [Metakit] Core = Dump - use of=20 sub views ?

Are there issues related to using = sub-views=20 ?
 
I seem to be getting a variety of = different core=20 dumps (the trace of one example appears below) when attempting to = increase the=20 size of a view that has sub-views.
 
Any information would be much=20 appreciated.
 
Best regards,
Ian Fairclough.
----- Original Message -----
From:=20 Ian Fairclough =
Sent: Tuesday, January 06, = 2004 1:17=20 PM
Subject: Re: [Metakit] Core = Dump

JC,
 
Thanks for the advice.  I = re-compiled as=20 you suggested (with assertions and debug symbols enabled), and I = obtained a=20 core dump with the following stack trace :
 
#0  0x41513fa1 in kill () from = /lib/libc.so.6
#1  0x41365fe6 in raise (sig=3D6) at=20 signals.c:65
#2  0x41515264 in abort () from=20 /lib/libc.so.6
#3  0x414b7e55 in __default_terminate () at=20 ../../gcc/libgcc2.c:-1
#4  0x414b7e72 in __terminate () from = /usr/lib/libstdc++-libc6.1-1.so.2
#5  0x4183ac2b in = new_eh_context=20 () at ../../gcc/libgcc2.c:3035
#6  0x4183af0d in = eh_context_specific=20 () from /usr/lib/libstdc++-libc6.2-2.so.3
#7  0x4183ae6e in=20 eh_context_initialize () from = /usr/lib/libstdc++-libc6.2-2.so.3
#8 =20 0x4183bb4d in __rethrow (index=3D0x41805a4c) from=20 /usr/lib/libstdc++-libc6.2-2.so.3
#9  0x417e3ee0 in=20 c4_HandlerSeq::c4_HandlerSeq (this=3D0x3fffec00, owner_=3D@0x2d93f440, = handler_=3D0xf151f20)=20 at ../unix/../src/handler.cpp:102
#10 0x417e26dc in = c4_FormatV::At=20 (this=3D0xf151f20, index_=3D49699) at = ../unix/../src/format.cpp:44
#11=20 0x417e281a in c4_FormatV::SetupAllSubviews (this=3D0xf151f20) at=20 ../unix/../src/format.cpp:1027
#12 0x417e2b22 in c4_FormatV::Get=20 (this=3D0xf151f20, index_=3D35525, length_=3D@0xbce7a8d4) at=20 ../unix/../src/format.cpp:1096
#13 0x417e3b9a in = c4_Handler::GetBytes=20 (this=3D0xf151f20, index_=3D35525, buf_=3D@0xbce7a968, = copySmall_=3Dfalse) at=20 ../unix/../src/handler.cpp:59
#14 0x417f4c87 in = c4_Sequence::InsertAt=20 (this=3D0x2e9923b8, index_=3D0, newElem_=3D{_seq =3D 0x2d93f440, = _index =3D 35525},=20 count_=3D1) at ../unix/../src/viewx.cpp:210
#15 0x417f17e0 in = c4_View::Add=20 (this=3D0xbce7aa50, newElem_=3D@0xbce7aa30) = at=20 ../include/mk4.inl:13
#16 0x4175495c in mkExt::eq () from=20 = /home/portal/build_area/components/do_database_lib/libdoDatabase.so
 
Any suggestions of what might be = wrong would be=20 much appreciated.
 
Best regards,
Ian=20 Fairclough.
------=_NextPart_000_0B43_01C3D7A4.30149710-- From jcw@equi4.com Sun Jan 11 03:03:37 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 155CBB3A8A for ; Sun, 11 Jan 2004 03:03:37 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXC6SaEi for ; Sat, 10 Jan 2004 20:03:35 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id D98F3B3A88 for ; Sun, 11 Jan 2004 03:03:34 +0800 (WST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 9184983C26; Sat, 10 Jan 2004 20:03:32 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E634C83914 for ; Sat, 10 Jan 2004 20:03:28 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <0b4601c3d7a4$30ad54a0$b4c8a8c0@imsgsx6tvznbng> References: <0b4601c3d7a4$30ad54a0$b4c8a8c0@imsgsx6tvznbng> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Message-Id: Content-Transfer-Encoding: quoted-printable From: Jean-Claude Wippler Subject: Re: [Metakit] Core Dump - use of sub views ? Date: Sat, 10 Jan 2004 20:03:28 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jan 2004 19:03:37 -0000 Ian Fairclough wrote: > Following are the first couple of lines that are generated by the=20 > metakit dump utility - they show the structure of the view that I am=20= > using.=A0 The view structure is auto-generated dynamically from CORBA=20= > type-codes (I am using metakit to cache data that has=A0it's structure=20= > defined in CORBA IDL).=A0 Hence I am using sub-views to store data = that=20 > is contained within nested sequences defined within CORBA data=20 > structures. I took that definition and created a small Tcl script to add rows, also=20= to the subviews. The only thing I had to do is replace dots in names=20 with underscores ("a_b" i.s.o. "a.b"), but that is probably a Tcl issue=20= (I don't think dots in names are a problem in C++ or Python). Added a few rows, also to subviews, and committed changes. All seems=20 ok here. The number of properties, sizes of names, number of rows, etc - none of=20= these are larger than what some of the regression tests do. > #3=A0 0x4150e05d in __assert_fail () from /lib/libc.so.6 > #4=A0 0x417d66a9 in c4_Column::Validate (this=3D0xbfe7a6f8) at=20 > ../src/univ.inl:44 > #5=A0 0x417d6755 in c4_Column::~c4_Column (this=3D0xbfe7a6f8, = __in_chrg=3D2)=20 > at ../unix/../src/column.cpp:86 This looks suspicious. You have checking enabled, and on cleanup of a=20= column inside MK, it detects corruption. The fact that problems happen during commit says little, unfortunately.=20= Most of MK's work happens at that stage. If a stray pointer write corrupts memory, then commit is the most=20 likely place where the failure will happen. You may want to use one of the memory checking tools to check against=20 memory corruption (which could be anywhere in the program). I think=20 "valgrind" can do this, on Linux. I'm not too familar with this sort=20 of stuff, though. The conclusion so far is that I don't see any limits which could have=20 been exceeded. I don't see any other way out than to reduce and=20 simplify until more clues can be found. -jcw From bkelley@wi.mit.edu Sun Jan 11 03:26:35 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 564C6B3A8A; Sun, 11 Jan 2004 03:26:35 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXp810tg; Sat, 10 Jan 2004 20:26:33 +0100 (CET) Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by dizzie.triqs.com (Postfix) with ESMTP id 823F9B3A88; Sun, 11 Jan 2004 03:26:33 +0800 (WST) Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.147.173.53]) by comcast.net (sccrmhc12) with SMTP id <20040110192628012007p7i0e>; Sat, 10 Jan 2004 19:26:28 +0000 Message-ID: <4000538D.1040304@wi.mit.edu> Date: Sat, 10 Jan 2004 14:33:33 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler Subject: Re: [Metakit] Core Dump - use of sub views ? References: <0b4601c3d7a4$30ad54a0$b4c8a8c0@imsgsx6tvznbng> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jan 2004 19:26:35 -0000 Jean-Claude Wippler wrote: > If a stray pointer write corrupts memory, then commit is the most > likely place where the failure will happen. > > You may want to use one of the memory checking tools to check against > memory corruption (which could be anywhere in the program). I think > "valgrind" can do this, on Linux. I'm not too familar with this sort > of stuff, though. I have used electric fence with great success http://freshmeat.net/projects/efence/?topic_id=47 Brian From ian.fairclough@intelligent-ms.com Wed Jan 14 00:22:06 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3F05FB3AE9; Wed, 14 Jan 2004 00:22:06 +0800 (WST) Received: from by localhost (amavisd-new, port ) id XXwmeQDT; Tue, 13 Jan 2004 17:22:03 +0100 (CET) Received: from scratchy.base.ims (unknown [194.74.0.138]) by dizzie.triqs.com (Postfix) with ESMTP id 41875B3A8A; Wed, 14 Jan 2004 00:22:03 +0800 (WST) Received: from grpware.intelligent-ms.com (grpware.base.ims [192.168.200.3]) by scratchy.base.ims (8.11.2/8.11.2) with ESMTP id i0DGFDl18585; Tue, 13 Jan 2004 16:15:13 GMT Received: from imsgsx6tvznbng ([192.168.200.187])i0DG0hu25721; Tue, 13 Jan 2004 16:00:43 GMT Message-ID: <0d9101c3d9f1$e5a18760$b4c8a8c0@imsgsx6tvznbng> From: "Ian Fairclough" To: Subject: Re: [Metakit] Core Dump - use of sub views ? Date: Tue, 13 Jan 2004 16:25:43 -0000 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0D8D_01C3D9F1.E438F6B0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2004 16:22:07 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0D8D_01C3D9F1.E438F6B0 Content-Type: multipart/alternative; boundary="----=_NextPart_001_0D8E_01C3D9F1.E438F6B0" ------=_NextPart_001_0D8E_01C3D9F1.E438F6B0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Many thanks for your help with this. I have managed to re-create an isolated problem which might be = interesting. When I try to increase the size of the view for the attached data file = by 1, I get a core dump as follows : #0 0x41896ad1 in kill () from /lib/libc.so.6 #1 0x413630fb in raise () at /usr/include/g++-2/std/bastring.cc:177 #2 0x41898052 in abort () from /lib/libc.so.6 #3 0x418901f0 in __assert_fail () from /lib/libc.so.6 #4 0x41a70239 in c4_FormatB::InitOffsets (this=3D0x81419c8, = sizes_=3D@0x81419f4) at ../src/column.inl:39 #5 0x41a6f96e in c4_FormatB::Define (this=3D0x81419c8, = ptr_=3D0xbfffdba0) at ../unix/../src/format.cpp:439 #6 0x41a73ca0 in c4_HandlerSeq::Prepare (this=3D0x8141738, = ptr_=3D0xbfffdba0, selfDesc_=3Dfalse) at ../unix/../src/handler.cpp:358 #7 0x41a71823 in c4_FormatV::SetupAllSubviews (this=3D0x813c420) at = ../unix/../src/format.cpp:1027 #8 0x41a71fe3 in c4_FormatV::Insert (this=3D0x813c420, index_=3D1, = buf_=3D@0xbfffdc78, count_=3D1) at ../unix/../src/format.cpp:1184 #9 0x41a83e12 in c4_Sequence::InsertAt (this=3D0x81391d8, index_=3D1, = newElem_=3D{_seq =3D 0x813d608, _index =3D 0}, count_=3D1) at = ../unix/../src/viewx.cpp:243 #10 0x41a83a83 in c4_Sequence::Resize (this=3D0x81391d8, newSize_=3D2) = at ../include/mk4.inl:13 #11 0x413cf2de in corbaMkTools::write (this=3D0xbfffdfc4, = data=3D@0xbfffdfe8, commit_=3Dfalse) at = /usr/local/metakit-2.4.9.2/include/mk4.inl:65 #12 0x080677e4 in main (argc=3D7, argv=3D0xbfffe0b4) at client.cpp:221 #13 0x418845d7 in __libc_start_main () from /lib/libc.so.6 Any help would be much appreciated ! Best regards, Ian Fairclough ------=_NextPart_001_0D8E_01C3D9F1.E438F6B0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Many thanks for your help with = this.
 
I have managed to re-create an isolated = problem=20 which might be interesting.
 
When I try to increase the size of the = view for the=20 attached data file by 1, I get a core dump as follows :
 
#0  0x41896ad1 in kill () from=20 /lib/libc.so.6
#1  0x413630fb in raise () at=20 /usr/include/g++-2/std/bastring.cc:177
#2  0x41898052 in abort = () from=20 /lib/libc.so.6
#3  0x418901f0 in __assert_fail () from=20 /lib/libc.so.6
#4  0x41a70239 in c4_FormatB::InitOffsets=20 (this=3D0x81419c8, sizes_=3D@0x81419f4) at=20 ../src/column.inl:39
#5  0x41a6f96e in c4_FormatB::Define=20 (this=3D0x81419c8, ptr_=3D0xbfffdba0) at = ../unix/../src/format.cpp:439
#6 =20 0x41a73ca0 in c4_HandlerSeq::Prepare (this=3D0x8141738, = ptr_=3D0xbfffdba0,=20 selfDesc_=3Dfalse) at ../unix/../src/handler.cpp:358
#7  = 0x41a71823 in=20 c4_FormatV::SetupAllSubviews (this=3D0x813c420) at=20 ../unix/../src/format.cpp:1027
#8  0x41a71fe3 in = c4_FormatV::Insert=20 (this=3D0x813c420, index_=3D1, buf_=3D@0xbfffdc78, count_=3D1) = at=20 ../unix/../src/format.cpp:1184
#9  0x41a83e12 in = c4_Sequence::InsertAt=20 (this=3D0x81391d8, index_=3D1, newElem_=3D{_seq =3D 0x813d608, _index = =3D 0}, count_=3D1) at=20 ../unix/../src/viewx.cpp:243
#10 0x41a83a83 in c4_Sequence::Resize=20 (this=3D0x81391d8, newSize_=3D2) at ../include/mk4.inl:13
#11 = 0x413cf2de in=20 corbaMkTools::write (this=3D0xbfffdfc4, data=3D@0xbfffdfe8, = commit_=3Dfalse) at=20 /usr/local/metakit-2.4.9.2/include/mk4.inl:65
#12 0x080677e4 in main = (argc=3D7,=20 argv=3D0xbfffe0b4) at client.cpp:221
#13 0x418845d7 in = __libc_start_main ()=20 from /lib/libc.so.6
 
Any help would be much appreciated = !
 
Best regards,
Ian = Fairclough
------=_NextPart_001_0D8E_01C3D9F1.E438F6B0-- ------=_NextPart_000_0D8D_01C3D9F1.E438F6B0 Content-Type: application/octet-stream; name="master_edit_databaseAccess_PermitType.dat" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="master_edit_databaseAccess_PermitType.dat" JL=1A=00=00=00=10|=80=8079668531954=00=0C=00=00=00=00=00HUNTINGDON2=00=0C= =00=00=00=00=002694=00=05=00=00=00=00=00901138=00=07=00=00=00=00=00Andy = D Lord=00=0C=00=00=00=00=00Andy Lord - Catering Specialist - = 07966336273=00.7966853=00=08=00=00=00=00=00901138=00=07=00=00=00=00=00And= y D = Lord=00=0C=00=00=00=00=00LOW=00=04=00=00=00=00=00N/A=00=04=00=00=00=00=00= N/A=00=04=00=00=00=00=00901138=00=07=00=00=00=00=00alan j = carter=00=0E=00=00=00=00=00=F7=FB=03@=00=00HB=0A= =00=02=00=00=00=00:=00=02=00=00=00=00901138=00=07=00=00=00=00=00Andy D = Lord=00=0C=00=00=00=00=00h=FA=03@=88=05=04@replaced door switches=0A= =00=18=80=81=87=02=89=86=02=90=80=8C=02=96=86=02=A2=80=84=02=A8=84=02=AC=98= =02=B0=81=02=C8=80=80=80=80=80=80=80=80=80=80=80rest bite/micoaire = XE5=00=18n/a=00=04=00=00=00=00=00=80=81=98=02=F0=81=03=88=80=80=80=84=03=89= =86=03=8D=80=80=80=80=80=80=80=80=80=80=80Fault - Microwave = Ovens=00=18n/a=00=04=00=80=81=80=80=80=80=80=80=80=80=80=80=80=80isolatio= n=00=80=0A= =B9databaseAccess_PermitType[refNum:S,storeName:S,storeId:S,contractorId:= S,contractorName:S,contractorAddress:S,woNum:S,supervisorUserId:S,supervi= sorUserName:S,overallRiskHML:S,highRisksAuthUserId:S,highRisksAuthUserNam= e:S,highRisksAuthDate:I,medRisksAuthUserId:S,medRisksAuthUserName:S,medRi= sksAuthDate:I,lowRisksAuthUserId:S,lowRisksAuthUserName:S,lowRisksAuthDat= e:I,isolationReqd:I,smssShownToBranch:S,smsAvailable:S,authSmsAvaliable:S= ,ptwAuthUserId:S,ptwAuthUserName:S,ptwAuthDate:I,totalMileage:F,ptwComple= teUserId:S,ptwCompleteUserName:S,ptwCompleteDate:I,comments:S,ud1:S,ud2:S= ,ud3:S,ud4:S,ud5:S,ud6:S,ud7:S,ud8:S,ud9:S,ud10:S,users[users.userId:S,us= ers.userName:S,users.timeStarted:I,users.timeStopped:I,users.workCarriedO= ut:S,users.ud1:S,users.ud2:S,users.ud3:S,users.ud4:S,users.ud5:S],equipme= nt[equipment.description:S,equipment.num:S,equipment.assetNum:S,equipment= .ud1:S,equipment.ud2:S,equipment.ud3:S,equipment.ud4:S,equipment.ud5:S],r= isks[risks.taskdescription:S,risks.coshh:S,risks.riskLevel:S,risks.elimin= ation:S,risks.remainingRisk:S,risks.ud1:S,risks.ud2:S,risks.ud3:S,risks.u= d4:S,risks.ud5:S],asbestosRegister[asbestosRegister.details:S,asbestosReg= ister.ud1:S,asbestosRegister.ud2:S,asbestosRegister.ud3:S,asbestosRegiste= r.ud4:S,asbestosRegister.ud5:S],safetyMethodStatements[safetyMethodStatem= ents:S],accidentDetails:S]=81=82=88=0A= =00=00=00=00=00n/a=00=04=00=00=00=00=00=80=81=98=03=AF=81=03=C7=80=84=03=C8= =86=03=CC=80=84=03=D2=86=03=D6=80=8A=03=DC=86=0E=A5=80=84=0E=AB=86=0E=AF=80= =80=80=80=80=80=80=80=80=80=80n/a=00=04=00=00=00=00=00=80=81=84=0E=E4=86=0E= =E8=80=80=81=8C=8A=86=96=80=8C=9C=86=A8=80=85=AE=86=B3=80=87=B9=86=C0=80=8C= =C6=86=D2=80=AE=D8=81=01=86=80=88=01=87=86=01=8F=80=87=01=95=86=01=9C=80=8C= =01=A2=86=01=AE=80=84=01=B4=86=01=B8=80=80=80=80=80=80=80=80=80=80=80=80=80= =80=80=80=80=84=01=BE=86=01=C2=80=84=01=C8=86=01=CC=80=80=80=87=01=D2=86=01= =D9=80=8E=01=DF=86=01=ED=80=84=01=F3=84=01=F7=80=80=80=80=80=82=01=FB=85=01= =FD=80=82=02=82=85=02=84=80=80=80=80=80=80=80=80=80=80=80=80=80=80=80=80=80= =80=80=A7=02=C9=9C=03=93=AF=0E=B5=8E=03=CE=89=0E=EE=80=80=80=0A= =B9databaseAccess_PermitType[refNum:S,storeName:S,storeId:S,contractorId:= S,contractorName:S,contractorAddress:S,woNum:S,supervisorUserId:S,supervi= sorUserName:S,overallRiskHML:S,highRisksAuthUserId:S,highRisksAuthUserNam= e:S,highRisksAuthDate:I,medRisksAuthUserId:S,medRisksAuthUserName:S,medRi= sksAuthDate:I,lowRisksAuthUserId:S,lowRisksAuthUserName:S,lowRisksAuthDat= e:I,isolationReqd:I,smssShownToBranch:S,smsAvailable:S,authSmsAvaliable:S= ,ptwAuthUserId:S,ptwAuthUserName:S,ptwAuthDate:I,totalMileage:F,ptwComple= teUserId:S,ptwCompleteUserName:S,ptwCompleteDate:I,comments:S,ud1:S,ud2:S= ,ud3:S,ud4:S,ud5:S,ud6:S,ud7:S,ud8:S,ud9:S,ud10:S,users[users.userId:S,us= ers.userName:S,users.timeStarted:I,users.timeStopped:I,users.workCarriedO= ut:S,users.ud1:S,users.ud2:S,users.ud3:S,users.ud4:S,users.ud5:S],equipme= nt[equipment.description:S,equipment.num:S,equipment.assetNum:S,equipment= .ud1:S,equipment.ud2:S,equipment.ud3:S,equipment.ud4:S,equipment.ud5:S],r= isks[risks.taskdescription:S,risks.coshh:S,risks.riskLevel:S,risks.elimin= ation:S,risks.remainingRisk:S,risks.ud1:S,risks.ud2:S,risks.ud3:S,risks.u= d4:S,risks.ud5:S],asbestosRegister[asbestosRegister.details:S,asbestosReg= ister.ud1:S,asbestosRegister.ud2:S,asbestosRegister.ud3:S,asbestosRegiste= r.ud4:S,asbestosRegister.ud5:S],safetyMethodStatements[safetyMethodStatem= ents:S],accidentDetails:S]=81=01=A7=0E=F7mments:S,ud1:S,ud2:S,ud3:S,ud4:S= ,ud5:S,ud6:S,ud7:S,ud8:S,ud9:S,ud10:S,users[users.userId:S,users.userName= :S,users.timeStarted:I,users.timeStopped:I,users.workCarriedOut:S,users.u= d1:S,users.ud2:S,users.ud3:S,users.ud4:S,users.ud5:S],equipment[equipment= .description:S,equipment.num:S,equipment.assetNum:S,equipment.ud1:S,equip= ment.ud2:S,equipment.ud3:S,equipment.ud4:S,equipment.ud5:S],risks[risks.t= askdescription:S,risks.coshh:S,risks.riskLevel:S,risks.elimination:S,risk= s.remainingRisk:S,risks.ud1:S,risks.ud2:S,risks.ud3:S,risks.ud4:S,risks.u= d5:S],asbestosRegister[asbestosRegister.details:S,asbestosRegister.ud1:S,= asbestosRegister.ud2:S,asbestosRegister.ud3:S,asbestosRegister.ud4:S,asbe= stosRegister.ud5:S],safetyMethodStatements[safetyMethodStatements:S],acci= dentDetails:S]=81=01=9F=15=8C=80=00=00=00=00=00=10l=80=00=05A=00=00=08=1E ------=_NextPart_000_0D8D_01C3D9F1.E438F6B0-- From schlenk@uni-oldenburg.de Fri Jan 16 16:55:51 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 88F4BB3A97 for ; Fri, 16 Jan 2004 16:55:51 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXNQcAL2 for ; Fri, 16 Jan 2004 16:55:49 +0100 (CET) Received: from physnet.physik.uni-oldenburg.de (physnet.physik.uni-oldenburg.de [134.106.31.103]) by dizzie.triqs.com (Postfix) with ESMTP id 8BFD0B3A88 for ; Fri, 16 Jan 2004 16:55:49 +0100 (CET) Received: from uni-oldenburg.de (physnet.physik.uni-oldenburg.de [134.106.31.103])with ESMTP id i0GFtlH1006319; Fri, 16 Jan 2004 16:55:47 +0100 Message-ID: <40080B3A.3010607@uni-oldenburg.de> Date: Fri, 16 Jan 2004 17:03:06 +0100 From: Michael Schlenker Organization: =?ISO-8859-1?Q?Universit=E4t_Oldenburg?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: de-de, en-us MIME-Version: 1.0 To: Metakit mailing list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Metakit memory hungry? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2004 15:55:52 -0000 Hi all, i noticed a strange thing with a current project: I use the Mk4tcl binding from Tclkit 8.4.5 on Windows 2000, now i parse a textfile containing strings and an integer associated with the string. Something like: 10 AB_0057357 11 CF_8463800 11 CD_3648208 (about 270.000 records, 10 MB) Now i setup a simple in memory Metakit dataset with: mk:file open db datafile.mk mk::view layout db.map {key:S id:I} I load the small 10MB Datafile into memory with set fid [open $file] set lines [lrange [split [read $fid] \n] 0 end-1] Now i scan the data with a proc that is called from an [after 1] that reschedules itself with [after 1], it takes the current line number as argument, so i have something like: proc process_some_lines {db line} { global lines ;# in my actual script this is "my instvar data" using XOTcl set iteration 0 while {$iteration < 50} { if {[llength $lines] <= $line} {set ::done 1; return} set currentLine [lindex $lines $line] scan $line "%d %s" id key mk::row append $db.map key $key id $id incr line incr iteration } after 1 [list process_some_lines $db $line] } The data is commited, when ::done is set to 1, via a vwait and a mk::file commit. The strange thing i see is the memory hunger of this construction. It eats memory like nothing, the shell grows to 300 MB with size jumps in the range of up to 50MB in both directions, but the datafile produced on disk is only 4MB in size. If i process the data line by line i don't get this huge memory hunger. I'm quite sure that i don't copy anything. I'm not sure if Metakit or XOTcl or the normal Tcl is the thing that creates trouble here. With tclkit 8.4.1 it crashes actually with a can't realloc 1622777730 bytes dialog box popping up. I may recheck with a smaller Skript, as my current problem is deeply inside a multi thousand line XOTcl program and depends on lots of infrastrukture (that does not leak as i checked by using a different approach). Any idea what this could be? Michael From richardjones@optushome.com.au Sat Jan 17 06:00:29 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0316EB3AF1 for ; Sat, 17 Jan 2004 06:00:29 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX3KnvjE for ; Sat, 17 Jan 2004 06:00:28 +0100 (CET) Received: from mail015.syd.optusnet.com.au (mail015.syd.optusnet.com.au [211.29.132.161]) by dizzie.triqs.com (Postfix) with ESMTP id B4A48B3A88 for ; Sat, 17 Jan 2004 06:00:26 +0100 (CET) Received: from richardpc (c211-28-245-127.eburwd5.vic.optusnet.com.au [211.28.245.127])i0H50NL18402 for ; Sat, 17 Jan 2004 16:00:24 +1100 From: Richard Jones To: metakit@equi4.com Date: Sat, 17 Jan 2004 16:00:21 +1100 User-Agent: KMail/1.5.3 MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Boundary-02=_lFMCAxnAmJsW2TU"; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <200401171600.21830.richardjones@optushome.com.au> X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Roundup needs a metakit volunteer X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2004 05:00:29 -0000 --Boundary-02=_lFMCAxnAmJsW2TU Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Content-Description: signed data Content-Disposition: inline Some time ago a metakit backend was given to the Roundup Issue Tracker proj= ect=20 (http://roundup.sf.net/). Since then I've tried to keep the backend up to=20 date and fix bugs as they appear. The current most severe bug has metakit=20 segfaulting during Roundup unit testing. I'm considering removing support for metakit from Roundup as I'm not sure I= =20 have the time or energy to give it the attention it needs. I'd be more than willing to have a metakit-savvy volunteer have a look at t= he=20 code and look at the bits that need attention. Richard --Boundary-02=_lFMCAxnAmJsW2TU Content-Type: application/pgp-signature Content-Description: signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQBACMFlrGisBEHG6TARAtnSAJ45zzrIiYIcTJnCd1xIpvjQa0ykBwCfdHpb 19O6KT9FMbV4S9OJv0fmtAo= =AvBw -----END PGP SIGNATURE----- --Boundary-02=_lFMCAxnAmJsW2TU-- From jcw@equi4.com Mon Jan 19 00:21:03 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2EC63B3A8E for ; Mon, 19 Jan 2004 00:21:03 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXxjbWag for ; Mon, 19 Jan 2004 00:20:56 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 675D6B3A88 for ; Mon, 19 Jan 2004 00:20:54 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id C028483C26; Mon, 19 Jan 2004 00:20:51 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 7939E83914 for ; Mon, 19 Jan 2004 00:20:49 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <0d9101c3d9f1$e5a18760$b4c8a8c0@imsgsx6tvznbng> References: <0d9101c3d9f1$e5a18760$b4c8a8c0@imsgsx6tvznbng> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Message-Id: Content-Transfer-Encoding: quoted-printable From: Jean-Claude Wippler Subject: Re: [Metakit] Core Dump - use of sub views ? Date: Mon, 19 Jan 2004 00:20:49 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Jan 2004 23:21:03 -0000 Ian Fairclough wrote: > I have managed to re-create an isolated problem which might be=20 > interesting. > =A0 > When I try to increase the size of the view for the attached data file=20= > by 1, I get a core dump as follows : [...] It looks like this is a serious file-damaging bug in Metakit (at least=20= 2.4.9.2, probably a few before, this is tied to subview caching and=20 open/save optimizations). I've just checked in a change to Metakit which passes all tests,=20 including the new one which brought it all out in the open. The test=20 is frighteningly simple: { c4_Storage s1 ("t1.dat", true); c4_View v1 =3D s1.GetAs("v1[v2[p1:I]]"); v1.SetSize(1); s1.Commit(); } // start with a valid non-empty view with a subview { c4_Storage s1 ("t1.dat", true); c4_View v1 =3D s1.View("v1"); v1.SetSize(0); s1.Commit(); v1.SetSize(1); s1.Commit(); // at this point, the 2.4.9.2 file is corrupt! s1.SaveTo(c4_FileStream (fopen("t2.dat", "w"), true)); } // using this damaged datafile will then crash { c4_Storage s1 ("t1.dat", true); c4_View v1 =3D s1.View("v1"); v1.SetSize(2); } In plain English: you have to delete all rows of a view with subviews,=20= commit, resize to create a new row with an empty subview, and commit. =20= That produces a damaged MK datafile. leaving out the first commit=20 works fine. Closing and re-opening works fine. Only the sequence=20 above seems to have all the ingedients to trigger the bug. Blocked views use subviews and modify them constantly, but as far as I=20= can tell they can *never* lead to the above usage pattern (blocks don't=20= get resized to zero *and* re-used). So although blocked views are=20 probably the most common use for subviews, they will not hit this bug. I've checked in changes to CVS. I'll soon release a 2.4.9.3 update,=20 but would also like to ask you to give the current sources in CVS a=20 very good work-out. There are almost no changes to the core since the=20= MK 2.4.9.2 release in March 2003, so I don't expect any regression of=20 the code at all (bugs nor performance). If you doubt it, please run=20 some checks. I'd also like to very much thank Ian Fairclough for patiently digging=20 in, to get the issue to the surface and for ultimately coming up with a=20= simple stand-alone example I could use to reproduce the problem and=20 drill down into. It made *ALL* the difference in the world! Let's hope this fully resolves the issue. -jcw From jcw@equi4.com Mon Jan 19 12:48:39 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id CA628B3A9E for ; Mon, 19 Jan 2004 12:48:39 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXjGsMDk for ; Mon, 19 Jan 2004 12:48:38 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id E286BB3A88 for ; Mon, 19 Jan 2004 12:48:37 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 6D0E183C26; Mon, 19 Jan 2004 12:48:36 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 3FCFC83914 for ; Mon, 19 Jan 2004 12:48:34 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <40080B3A.3010607@uni-oldenburg.de> References: <40080B3A.3010607@uni-oldenburg.de> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <689310FB-4A75-11D8-BE82-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Metakit memory hungry? Date: Mon, 19 Jan 2004 12:48:33 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jan 2004 11:48:40 -0000 Michael Schlenker wrote: > i noticed a strange thing with a current project: > I use the Mk4tcl binding from Tclkit 8.4.5 on Windows 2000, now i > parse a textfile containing strings and an integer associated with the > string. > Something like: > > 10 AB_0057357 > 11 CF_8463800 > 11 CD_3648208 > (about 270.000 records, 10 MB) [...] > The strange thing i see is the memory hunger of this construction. > It eats memory like nothing, the shell grows to 300 MB with size jumps > in the range of up to 50MB in both directions, but the datafile > produced on disk is only 4MB in size. > If i process the data line by line i don't get this huge memory > hunger. I'm quite sure that i don't copy anything. This memory use is indeed extreme. The most memory-hungry scenario of MK is with lots and lots of subviews with lots and lots of properties (there are some cases of 4 Kb buffers, which leads to a lot of slack). Nothing as extreme as you describe, though. > I'm not sure if Metakit or XOTcl or the normal Tcl is the thing that > creates trouble here. With tclkit 8.4.1 it crashes actually with a > can't realloc 1622777730 bytes dialog box popping up. The cases I've seen this happen is when there is a 32/64-bit size mixup in stat, i.e. when Tcl and some extension were not built with matching 64-bit-ness for file offsets and such. -jcw From jcw@equi4.com Mon Jan 19 12:54:32 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 172FEB3B08 for ; Mon, 19 Jan 2004 12:54:32 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXKYDej1 for ; Mon, 19 Jan 2004 12:54:30 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 891F3B3A88 for ; Mon, 19 Jan 2004 12:54:30 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 3A56683C26; Mon, 19 Jan 2004 12:54:29 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 7757183914 for ; Mon, 19 Jan 2004 12:54:27 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <200401171600.21830.richardjones@optushome.com.au> References: <200401171600.21830.richardjones@optushome.com.au> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <3B363F1E-4A76-11D8-BE82-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Roundup needs a metakit volunteer Date: Mon, 19 Jan 2004 12:54:26 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jan 2004 11:54:32 -0000 Richard Jones wrote: > Some time ago a metakit backend was given to the Roundup Issue Tracker > project > (http://roundup.sf.net/). Since then I've tried to keep the backend up > to > date and fix bugs as they appear. The current most severe bug has > metakit > segfaulting during Roundup unit testing. > > I'm considering removing support for metakit from Roundup as I'm not > sure I > have the time or energy to give it the attention it needs. > > I'd be more than willing to have a metakit-savvy volunteer have a look > at the > code and look at the bits that need attention. Ouch, that would be most unfortunate. As I understand, MK is one of the faster bindings for Roundup. Is there a Python specialist in the room, willing to give this a check? I don't expect Roundup to be pushing any boundaries of MK that far... maybe it's some weak error checking in MK/Mk4py? (Richard, before you actually *drop* MK, please let me know - I'd hate to see this happening for what turns out to be some simple issue... the least we'll need to do is establish that the latest CVS code is indeed segfaulting) -jcw From richardjones@optushome.com.au Mon Jan 19 23:53:48 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 7AEA0B3B0B; Mon, 19 Jan 2004 23:53:47 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXkqKDVw; Mon, 19 Jan 2004 23:53:39 +0100 (CET) Received: from mail002.syd.optusnet.com.au (mail002.syd.optusnet.com.au [211.29.132.32]) by dizzie.triqs.com (Postfix) with ESMTP id 10156B3A88; Mon, 19 Jan 2004 23:53:36 +0100 (CET) Received: from richardpc (c211-28-245-127.eburwd5.vic.optusnet.com.au [211.28.245.127])i0JMrVV19140; Tue, 20 Jan 2004 09:53:32 +1100 From: Richard Jones To: Jean-Claude Wippler , Metakit list Subject: Re: [Metakit] Roundup needs a metakit volunteer Date: Tue, 20 Jan 2004 09:53:29 +1100 User-Agent: KMail/1.5.3 References: <200401171600.21830.richardjones@optushome.com.au> <3B363F1E-4A76-11D8-BE82-000A9588127E@equi4.com> In-Reply-To: <3B363F1E-4A76-11D8-BE82-000A9588127E@equi4.com> MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="Boundary-02=_p/FDA2E2t6qrc0z"; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200401200953.29886.richardjones@optushome.com.au> X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jan 2004 22:53:49 -0000 --Boundary-02=_p/FDA2E2t6qrc0z Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Description: signed data Content-Disposition: inline On Monday 19 January 2004 22:54, Jean-Claude Wippler wrote: > Richard Jones wrote: > > I'm considering removing support for metakit from Roundup > > Ouch, that would be most unfortunate. As I understand, MK is one of > the faster bindings for Roundup. It's actually the fastest. Unfortunately, it's also the least-supported. I= =20 honestly have little idea how it works. Also, metakit's lack of explicit nu= ll=20 has caused some difficultly, but I've managed some work-arounds for that -= =20 which in some cases means just skipping some unit tests :( > (Richard, before you actually *drop* MK, please let me know - I'd hate > to see this happening for what turns out to be some simple issue... the > least we'll need to do is establish that the latest CVS code is indeed > segfaulting) It's not going to be dropped overnight, since I still have some considerabl= e=20 work to do on the other new backends before the next release. Richard --Boundary-02=_p/FDA2E2t6qrc0z Content-Type: application/pgp-signature Content-Description: signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQBADF/prGisBEHG6TARAtgxAJ9PHQ5hA9UE0fEZxZhaOuV10jabfACcCBOq K0SmoVtZxh+TdMbB2Ats7zU= =IGtG -----END PGP SIGNATURE----- --Boundary-02=_p/FDA2E2t6qrc0z-- From bkelley@wi.mit.edu Tue Jan 20 18:04:45 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 48997B3A9E; Tue, 20 Jan 2004 18:04:45 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXA63m2K; Tue, 20 Jan 2004 18:04:43 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 01694B3A88; Tue, 20 Jan 2004 18:04:42 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004012012043801473 ; Tue, 20 Jan 2004 12:04:38 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HRSSRQ00.37W; Tue, 20 Jan 2004 12:04:38 -0500 Message-ID: <400D6002.2080607@wi.mit.edu> Date: Tue, 20 Jan 2004 12:06:10 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Richard Jones Subject: Re: [Metakit] Roundup needs a metakit volunteer References: <200401171600.21830.richardjones@optushome.com.au> <3B363F1E-4A76-11D8-BE82-000A9588127E@equi4.com> <200401200953.29886.richardjones@optushome.com.au> In-Reply-To: <200401200953.29886.richardjones@optushome.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list cc: Jean-Claude Wippler X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jan 2004 17:04:45 -0000 Richard Jones wrote: >On Monday 19 January 2004 22:54, Jean-Claude Wippler wrote: > > >>Richard Jones wrote: >> >> >>>I'm considering removing support for metakit from Roundup >>> >>> >>Ouch, that would be most unfortunate. As I understand, MK is one of >>the faster bindings for Roundup. >> >> > >It's actually the fastest. Unfortunately, it's also the least-supported. I >honestly have little idea how it works. Also, metakit's lack of explicit null >has caused some difficultly, but I've managed some work-arounds for that - >which in some cases means just skipping some unit tests :( > > > I'll be happy to take a look at the python code. I checked the most recent out of CVS played with it for a little while and have my first fix :) in back_metakit.py class _Database the __getattr__ statement should be def __getattr__(self, classname): if classname == 'transactions': return self.dirty # fall back on the classes try: return self.getclass(classname) except KeyError, msg: raise AttributeError(str(msg)) Otherwise admin.py will throw exceptions in Python 2.2, 2.3 when executing statements like "if self.db" Do you have any crashing scenerios that you could forward? Brian From gumbright@taltrade.com Thu Jan 22 20:25:43 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 085E9B3A8B for ; Thu, 22 Jan 2004 20:25:43 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXfMslJk for ; Thu, 22 Jan 2004 20:25:43 +0100 (CET) Received: from talchiexch03.taltrade.com (talchiexch03.taltrade.com [207.48.80.20]) by dizzie.triqs.com (Postfix) with ESMTP id 873B0B3A88 for ; Thu, 22 Jan 2004 20:25:42 +0100 (CET) Received: from mail pickup service by talchiexch03.taltrade.com with Microsoft SMTPSVC; Thu, 22 Jan 2004 13:25:41 -0600 x-gfisavedcharset: us-ascii Content-Type: text/plain; charset="us-ascii" Received: from talchiexch02.taltrade.com ([10.207.64.102]) by talchiexch03.taltrade.com with Microsoft SMTPSVC(5.0.2195.6713); Thu, 22 Jan 2004 13:25:38 -0600 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4927.1200 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Date: Thu, 22 Jan 2004 13:25:38 -0600 Message-ID: <45CE1FBFCDAD2E4DA84B49F2F866FC4F049A3501@talchiexch02.taltrade.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: 'Right' way to delete rows from a view in python? Thread-Index: AcPhHEp2YVddbMBITSiaYnoQyomLcg== From: "Guy Umbright" To: X-OriginalArrivalTime: 22 Jan 2004 19:25:38.0507 (UTC) FILETIME=[843801B0:01C3E11D] X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] 'Right' way to delete rows from a view in python? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2004 19:25:43 -0000 Given the following bit of python setup code: db =3D metakit.storage("mkdelete.mk",1)=20 mainView =3D db.getas("program[key:S,val1:I,val2:I]") aView =3D mainView.select(key=3D'aaa') what is the 'right' way to delete the records in aView from mainView (in python)? Banging my head on the desk on this.... =09 This electronic mail message and any attached files contain information = intended for the exclusive use of the individual or entity to whom it is = addressed and may contain information that is proprietary, privileged, = confidential and/or exempt from disclosure under applicable law. If you = are not the intended recipient, you are hereby notified that any = viewing, copying, disclosure or distribution of this information may be = subject to legal restriction or sanction. Please notify the sender, by = electronic mail or telephone, of any unintended recipients and delete = the original message without making any copies. From bkelley@wi.mit.edu Thu Jan 22 21:04:34 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E34DEB3A97 for ; Thu, 22 Jan 2004 21:04:33 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXkyRv3w for ; Thu, 22 Jan 2004 21:04:32 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id DC15FB3A88 for ; Thu, 22 Jan 2004 21:04:31 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004012215042628309 for ; Thu, 22 Jan 2004 15:04:26 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HRWQFF00.LP0; Thu, 22 Jan 2004 15:04:27 -0500 Message-ID: <40102D37.8080102@wi.mit.edu> Date: Thu, 22 Jan 2004 15:06:15 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Guy Umbright , Metakit mailing list Subject: Re: [Metakit] 'Right' way to delete rows from a view in python? References: <45CE1FBFCDAD2E4DA84B49F2F866FC4F049A3501@talchiexch02.taltrade.com> In-Reply-To: <45CE1FBFCDAD2E4DA84B49F2F866FC4F049A3501@talchiexch02.taltrade.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2004 20:04:34 -0000 Guy Umbright wrote: >Given the following bit of python setup code: > > >db = metakit.storage("mkdelete.mk",1) >mainView = db.getas("program[key:S,val1:I,val2:I]") > > >aView = mainView.select(key='aaa') > > >what is the 'right' way to delete the records in aView >from mainView (in python)? > >Banging my head on the desk on this.... > > > I think this is the way: indices = mainView.indices(aView) mainView.remove(indices) Here is a test script I just typed into the interpreter >>> import metakit >>> st = metakit.storage() >>> view = st.getas("test[a,b:I]") >>> for num, s in enumerate("abcdefg"): ... view.append((s, num)) ... >>> s = view.select(a='a') >>> indices = view.indices(s) >>> view.remove(indices) >>> metakit.dump(view) a b - - b 1 c 2 d 3 e 4 f 5 g 6 - - Total: 6 rows >>> From gumbright@taltrade.com Thu Jan 22 21:43:13 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 78BE3B3A90 for ; Thu, 22 Jan 2004 21:43:13 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXx3TtCg for ; Thu, 22 Jan 2004 21:43:10 +0100 (CET) Received: from talchiexch03.taltrade.com (talchiexch03.taltrade.com [207.48.80.20]) by dizzie.triqs.com (Postfix) with ESMTP id DAA0BB3A88 for ; Thu, 22 Jan 2004 21:43:07 +0100 (CET) Received: from mail pickup service by talchiexch03.taltrade.com with Microsoft SMTPSVC; Thu, 22 Jan 2004 14:43:05 -0600 x-gfisavedcharset: us-ascii Content-Type: text/plain; charset="us-ascii" Received: from talchiexch02.taltrade.com ([10.207.64.102]) by talchiexch03.taltrade.com with Microsoft SMTPSVC(5.0.2195.6713); Thu, 22 Jan 2004 14:43:02 -0600 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4927.1200 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Subject: RE: [Metakit] 'Right' way to delete rows from a view in python? Date: Thu, 22 Jan 2004 14:43:02 -0600 Message-ID: <45CE1FBFCDAD2E4DA84B49F2F866FC4F049A3502@talchiexch02.taltrade.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [Metakit] 'Right' way to delete rows from a view in python? Thread-Index: AcPhIvEEvMMU8m/VRH69NH+pA7qqzAABVprA From: "Guy Umbright" To: "Brian Kelley" , "Metakit mailing list" X-OriginalArrivalTime: 22 Jan 2004 20:43:02.0840 (UTC) FILETIME=[5474FB80:01C3E128] X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2004 20:43:13 -0000 Doh. Just what I needed. I kept reading past it in the doc. Thanks!=20 (my forehead appreciates it too.) -----Original Message----- From: Brian Kelley [mailto:bkelley@wi.mit.edu]=20 Sent: Thursday, January 22, 2004 2:06 PM To: Guy Umbright; Metakit mailing list Subject: Re: [Metakit] 'Right' way to delete rows from a view in python? Guy Umbright wrote: >Given the following bit of python setup code: > > >db =3D metakit.storage("mkdelete.mk",1)=20 >mainView =3D db.getas("program[key:S,val1:I,val2:I]") > > >aView =3D mainView.select(key=3D'aaa') > > >what is the 'right' way to delete the records in aView >from mainView (in python)? > >Banging my head on the desk on this.... > >=09 > I think this is the way: indices =3D mainView.indices(aView) mainView.remove(indices) Here is a test script I just typed into the interpreter >>> import metakit >>> st =3D metakit.storage() >>> view =3D st.getas("test[a,b:I]") >>> for num, s in enumerate("abcdefg"): ... view.append((s, num)) ... >>> s =3D view.select(a=3D'a') >>> indices =3D view.indices(s) >>> view.remove(indices) >>> metakit.dump(view) a b - - b 1 c 2 d 3 e 4 f 5 g 6 - - Total: 6 rows >>> This electronic mail message and any attached files contain information = intended for the exclusive use of the individual or entity to whom it is = addressed and may contain information that is proprietary, privileged, = confidential and/or exempt from disclosure under applicable law. If you = are not the intended recipient, you are hereby notified that any = viewing, copying, disclosure or distribution of this information may be = subject to legal restriction or sanction. Please notify the sender, by = electronic mail or telephone, of any unintended recipients and delete = the original message without making any copies. From jcw@equi4.com Thu Jan 22 22:01:03 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E6274B3A9F for ; Thu, 22 Jan 2004 22:00:59 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXYpyiy7 for ; Thu, 22 Jan 2004 22:00:57 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 8ABD6B3A90 for ; Thu, 22 Jan 2004 22:00:57 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 5EBF383C26; Thu, 22 Jan 2004 22:00:55 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id C2AA083914 for ; Thu, 22 Jan 2004 22:00:53 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) In-Reply-To: <40102D37.8080102@wi.mit.edu> References: <45CE1FBFCDAD2E4DA84B49F2F866FC4F049A3501@talchiexch02.taltrade.com> <40102D37.8080102@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <10C456B0-4D1E-11D8-BE82-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] 'Right' way to delete rows from a view in python? Date: Thu, 22 Jan 2004 22:00:53 +0100 To: Metakit list X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2004 21:01:03 -0000 Brian Kelley wrote: >> what is the 'right' way to delete the records in aView >> from mainView (in python)? >> >> Banging my head on the desk on this.... > > I think this is the way: > > indices = mainView.indices(aView) > mainView.remove(indices) Looks good. It's easy to miss, but there is a subtle issue here: row indices change as rows get removed, which is why the remove() call internally implements this removal in last-to-first order. -jcw From Bernhard.Nemec@jaeger-toennies.com Fri Jan 23 16:04:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 57CACB3A8B for ; Fri, 23 Jan 2004 16:04:38 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXqVfprq for ; Fri, 23 Jan 2004 16:04:37 +0100 (CET) Received: from mail.jaeger-toennies.com (unknown [62.180.118.137]) by dizzie.triqs.com (Postfix) with SMTP id 05550B3A88 for ; Fri, 23 Jan 2004 16:04:37 +0100 (CET) Received: FROM mail.jaeger-toennies.com BY mail.jaeger-toennies.com ; Fri Jan 23 16:04:36 2004 +0100 Received: by NTS-HOE04 with Internet Mail Service (5.5.2653.19) id ; Fri, 23 Jan 2004 16:04:35 +0100 Message-ID: <5FDDBF5436E7D61181BB0030482198AB0FE85A@nts-hoe32.jaeger-toennies.de> From: "Nemec, Bernhard" To: Metakit list Date: Fri, 23 Jan 2004 16:04:34 +0100 X-Mailer: Internet Mail Service (5.5.2653.19) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] RAM usage X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2004 15:04:38 -0000 Hello, I am evaluating Metakit and have looked at the memory usage under linux. I wrote a test program in C++ that adds batches of 5000 rows to a view and commits after each batch, repeating this every few seconds. I noticed that the RAM usage of my test program grows linearly as more rows are added. What can I do to limit the RAM usage of such a program? Thanks, Bernhard From tom.krehbiel@motorola.com Fri Jan 23 16:33:46 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0DD3EB3A8B for ; Fri, 23 Jan 2004 16:33:46 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXAvAaky for ; Fri, 23 Jan 2004 16:33:44 +0100 (CET) Received: from motgate3.mot.com (motgate3.mot.com [144.189.100.103]) by dizzie.triqs.com (Postfix) with ESMTP id A5914B3A88 for ; Fri, 23 Jan 2004 16:33:43 +0100 (CET) Received: from az33exr01.mot.com (az33exr01.mot.com [10.64.251.231]) by motgate3.mot.com (Motorola/Motgate3) with ESMTP id i0NFWa7U000152 for ; Fri, 23 Jan 2004 08:32:36 -0700 (MST) Received: from motorola.com ([172.16.104.118]) by az33exr01.mot.com (Motorola/az33exr01) with ESMTP id i0NFVMSR010720 for ; Fri, 23 Jan 2004 09:31:23 -0600 Message-ID: <40113ED2.3040006@motorola.com> Date: Fri, 23 Jan 2004 08:33:38 -0700 From: Tom Krehbiel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 Cc: Metakit list Subject: Re: [Metakit] RAM usage References: <5FDDBF5436E7D61181BB0030482198AB0FE85A@nts-hoe32.jaeger-toennies.de> In-Reply-To: <5FDDBF5436E7D61181BB0030482198AB0FE85A@nts-hoe32.jaeger-toennies.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2004 15:33:46 -0000 Nemec, Meta kit is an "imbeded" database that archives speed and simplicity by not dealing with page swapping like a more complex database. Thus, the database is in memory when you are using it. PS. I'm not an expert so I could be wrong :/ Tom K. > Hello, > > I am evaluating Metakit and have looked at the memory usage under linux. I > wrote a test program in C++ that adds batches of 5000 rows to a view and > commits after each batch, repeating this every few seconds. > > I noticed that the RAM usage of my test program grows linearly as more rows > are added. > What can I do to limit the RAM usage of such a program? > > Thanks, > Bernhard > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From mbloore@yahoo.com Fri Jan 23 17:52:12 2004 Return-Path: X-Original-To: Metakit@equi4.com Delivered-To: Metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 798E2B3A8B for ; Fri, 23 Jan 2004 17:52:12 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXhnPTdZ for ; Fri, 23 Jan 2004 17:52:11 +0100 (CET) Received: from web14306.mail.yahoo.com (web14306.mail.yahoo.com [216.136.173.82]) by dizzie.triqs.com (Postfix) with SMTP id E6783B3A88 for ; Fri, 23 Jan 2004 17:52:10 +0100 (CET) Message-ID: <20040123165206.39978.qmail@web14306.mail.yahoo.com> Received: from [65.95.141.67] by web14306.mail.yahoo.com via HTTP; Fri, 23 Jan 2004 08:52:06 PST Date: Fri, 23 Jan 2004 08:52:06 -0800 (PST) From: mARK bLOORE Subject: Re: [Metakit] RAM usage To: Metakit@equi4.com In-Reply-To: <5FDDBF5436E7D61181BB0030482198AB0FE85A@nts-hoe32.jaeger-toennies.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2004 16:52:13 -0000 MK saves up your added rows in ram until you call commit(), so you could commit more often. also, MK memory-maps the storage file, so it becomes part of your virtual address space. it then pages it in, rather than doing file i/o. it only gets those parts of the file that it is using, which in general means whole columns as you access them. blocked views can reduce the size of the data it pages in, by breaking up the columns tranparently to your code. --- "Nemec, Bernhard" wrote: > Hello, > > I am evaluating Metakit and have looked at the memory usage under linux. I > wrote a test program in C++ that adds batches of 5000 rows to a view and > commits after each batch, repeating this every few seconds. > > I noticed that the RAM usage of my test program grows linearly as more rows > are added. > What can I do to limit the RAM usage of such a program? > > Thanks, > Bernhard > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ From jyl@mod3.net Fri Jan 23 17:58:20 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id EB2A8B3A98 for ; Fri, 23 Jan 2004 17:58:19 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXnYFT85 for ; Fri, 23 Jan 2004 17:58:17 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id E141FB3A88 for ; Fri, 23 Jan 2004 17:58:16 +0100 (CET) Received: (qmail 3752 invoked by uid 48); 23 Jan 2004 16:58:14 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Fri, 23 Jan 2004 08:58:14 -0800 (PST) Message-ID: <65249.209.233.24.218.1074877094.squirrel@mod3.net> Date: Fri, 23 Jan 2004 08:58:14 -0800 (PST) Subject: Re: [Metakit] RAM usage From: "Jacob Levy" To: In-Reply-To: <40113ED2.3040006@motorola.com> References: <5FDDBF5436E7D61181BB0030482198AB0FE85A@nts-hoe32.jaeger-toennies.de> <40113ED2.3040006@motorola.com> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2004 16:58:20 -0000 This is basically correct, except for terminology -- what Metakit does is it uses memory mapped files. These files consume swap space as well as real space on the disk. Most modern OSes have two levels of memory mapping, committed and reserved: reserved means simply that an address range is reserved and nothing else can be mapped into it, but the space is not necessarily consumed yet -- that happens when the pages are committed. Metakit attempts to map the entire file of an open database into memory, so the reserved address range will grow with the size of the database. I am not sure whether Metakit plays smart tricks with committed versus reserved address ranges to optimize its use of address space. In any case, the committed address space shows up as swap space usage (and depending on the OS reserved ranges may also consume swap space, but I'm not sure about that), and when RAM is available, modern OSes will attempt to use that in preference to on-disk space. Metakit can memory-map files of sizes that range up to the limits of available address space, and access will cause page swapping at the OS level. This is generally a very fast operation if you use Metakit as intended, by accessing rows in increasing order -- that will make Metakit really "rip". Summary - Metakit relies on OSes to be smart, and use memory mapping and swap space, for speed. This achieves exactly the same effect as the page swapping algorithms in more complex databases, except that it suffers from the address space limitations of the 32-bit address space, whereas these other DBs can do address translation as well, so there this limitation would not hit. Metakit runs on 64-bit architectures and on these OSes the address space limits do not pose an issue with current disk sizes. --JYL P.S. I am also not an expert and will readily admit when wrong :/ > Nemec, > > Meta kit is an "imbeded" database that archives speed and > simplicity by not dealing with page swapping like a more complex > database. Thus, the database is in memory when you are using it. > > PS. I'm not an expert so I could be wrong :/ > Tom K. > >> Hello, >> >> I am evaluating Metakit and have looked at the memory usage under >> linux. I wrote a test program in C++ that adds batches of 5000 rows to >> a view and commits after each batch, repeating this every few seconds. >> >> I noticed that the RAM usage of my test program grows linearly as more >> rows are added. >> What can I do to limit the RAM usage of such a program? >> >> Thanks, >> Bernhard >> _____________________________________________ >> Metakit mailing list - Metakit@equi4.com >> http://www.equi4.com/mailman/listinfo/metakit > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Mon Jan 26 11:42:27 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id F1D93B3A8B for ; Mon, 26 Jan 2004 11:42:26 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXnmy0Hi for ; Mon, 26 Jan 2004 11:42:23 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id DAAC1B3A88 for ; Mon, 26 Jan 2004 11:42:22 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 49E7C83C26; Mon, 26 Jan 2004 11:42:21 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 1E5F383914 for ; Mon, 26 Jan 2004 11:42:17 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v609) Content-Transfer-Encoding: 7bit Message-Id: <4EF9E7D4-4FEC-11D8-9727-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Mon, 26 Jan 2004 11:42:16 +0100 X-Mailer: Apple Mail (2.609) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Metakit 2.4.9.3 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2004 10:42:27 -0000 This is to announce a new release of the Metakit embedded high-performance database library for C++, Python, and Tcl. This version consolidates bug fixes over the past 9 months since 2.4.9.2 came out. There should be no source code or binary incompatibilities, upgrading is recommended (but not urgent). An extract of the change log is appended, full details are available at: http://www.equi4.com/pub/mk/metakit-2.4.9.3.kit/CHANGES For details see the Metakit homepage at: http://www.equi4.com/metakit.html Sources and C++/Python/Tcl binaries for Windows, Mac OS X, and Linux are here: http://www.equi4.com/pub/mk/ Enjoy, Jean-Claude 2004-01-26 ############################################ MK 2.4.9.3 2004-01-22 Fixed refcount problem with temp rows in Mk4tcl 2004-01-21 Documentation updates 2004-01-20 Don't trip over duplicate property names 2004-01-18 Fixed rare but very serious subview resizing bug 2004-01-16 Gracefully deal with bad property type specifiers 2004-01-03 Fixed typo in PyView.cpp 2003-12-21 Fixed Mk4too sorting on subview of length 1 2003-12-13 Tweak to avoid two unisgned/signed compiler warnings 2003-12-11 Checked in numerous changes to Mk4py by Nicholas Riley 2003-11-23 Bumped to Python 2.3, doc tweaks, lots of name fixes 2003-10-28 Get rid of --enable-python, check in c22.txt 2003-10-16 Added note to Tcl docs 2003-10-10 Added c22 test 2003-10-01 Fixed bugs in Tcl test suite 2003-09-30 Python 2.3.1 & cleanup 2003-09-20 Autoconf and libtool rebuilds 2003-08-26 Documentation fix 2003-07-17 Fixes to Mk4py (Gordon) 2003-07-11 Fix for Linux not finding .lai file 2003-07-01 Fixed "Metakit" (preferred) vs "Metakit" (obsolete) 2003-06-06 Fix to Mk4py for case (in)sensitivity. 2003-05-15 Add distutils setup.py script (Gordon). 2003-05-08 Fixed array bound bug when not using mmap-ed files 2003-04-28 Sourceforge 2003-04-25 Autoconf/libtool update 2003-04-22 Fixes to Mk4py (Gordon). 2003-03-16 ############################################ MK 2.4.9.2 From rus20376@infolaunch.com Tue Jan 27 18:43:43 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 55BACB3A92 for ; Tue, 27 Jan 2004 18:43:43 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXoU6adA for ; Tue, 27 Jan 2004 18:43:42 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id B73EBB3A8F for ; Tue, 27 Jan 2004 18:43:41 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0RHhcW03140 for ; Tue, 27 Jan 2004 12:43:38 -0500 (EST) Message-ID: <4016A34B.5030108@infolaunch.com> Date: Tue, 27 Jan 2004 12:43:39 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] SystemError: NULL result without error in PyObject_Call X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 17:43:43 -0000 I am trying to do a simple listing of a hasged view and get the following error: SystemError: NULL result without error in PyObject_Call The code itself is very very simple import metakit db=metakit.storage("db/linelists"+"09_11_2003"+".mk",1) vw=db.getas("linelists[mid:S,field:S]") m=db.getas('map[_H:I,_R:I]') vwh=vw.hash(m,16) rows=[] for r in vwh: rows.append((r.mid,r.field)) db=None print rows What could cause such and error? this code actually resides in a much more complex class but that is the gist of it. Could something else be causing this to occur? From jcw@equi4.com Tue Jan 27 18:52:25 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 67F57B3A92 for ; Tue, 27 Jan 2004 18:52:25 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXr1IKFh for ; Tue, 27 Jan 2004 18:52:23 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id BE7A0B3A8F for ; Tue, 27 Jan 2004 18:52:23 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id E980D83C26; Tue, 27 Jan 2004 18:52:22 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 3B70283914 for ; Tue, 27 Jan 2004 18:52:01 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <4016A34B.5030108@infolaunch.com> References: <4016A34B.5030108@infolaunch.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <813BA4F2-50F1-11D8-8F8F-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] SystemError: NULL result without error in PyObject_Call Date: Tue, 27 Jan 2004 18:51:59 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 17:52:25 -0000 Adam Russell wrote: > I am trying to do a simple listing of a hasged view and get the > following error: > > SystemError: NULL result without error in PyObject_Call > > The code itself is very very simple > > import metakit > db=metakit.storage("db/linelists"+"09_11_2003"+".mk",1) > vw=db.getas("linelists[mid:S,field:S]") > m=db.getas('map[_H:I,_R:I]') > vwh=vw.hash(m,16) This doesn't look right - 16 keys? There are only two properties. Should probably be: vwh=vw.hash(m,16) > rows=[] > for r in vwh: > rows.append((r.mid,r.field)) > > db=None > print rows > > What could cause such and error? this code actually resides in a much > more complex class but that is the gist of it. Could something else be > causing this to occur? If this is the problem, then it's a bug in that MK does not catch the bad args. -jcw From rus20376@infolaunch.com Tue Jan 27 18:59:44 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 03691B3A92 for ; Tue, 27 Jan 2004 18:59:44 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXAmUQpC for ; Tue, 27 Jan 2004 18:59:41 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id D6BA4B3A8F for ; Tue, 27 Jan 2004 18:59:40 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0RHxeW05218 for ; Tue, 27 Jan 2004 12:59:40 -0500 (EST) Message-ID: <4016A70D.9040808@infolaunch.com> Date: Tue, 27 Jan 2004 12:59:41 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Re: SystemError: NULL result without error in PyObject_Call X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 17:59:44 -0000 Varying the number of keys make no difference. Admittedly, the 16 was amistake, however, just using 2 breaks as well with th exact same error. From: Jean-Claude Wippler equi4.com> Subject: Re: SystemError: NULL result without error in PyObject_Call Date: Tue, 27 Jan 2004 18:51:59 +0100 Adam Russell wrote: > I am trying to do a simple listing of a hasged view and get the > following error: > > SystemError: NULL result without error in PyObject_Call > > The code itself is very very simple > > import metakit > db=metakit.storage("db/linelists"+"09_11_2003"+".mk",1) > vw=db.getas("linelists[mid:S,field:S]") > m=db.getas('map[_H:I,_R:I]') > vwh=vw.hash(m,16) This doesn't look right - 16 keys? There are only two properties. Should probably be: vwh=vw.hash(m,16) > rows=[] > for r in vwh: > rows.append((r.mid,r.field)) > > db=None > print rows > > What could cause such and error? this code actually resides in a much > more complex class but that is the gist of it. Could something else be > causing this to occur? If this is the problem, then it's a bug in that MK does not catch the bad args. -jcw From bkelley@wi.mit.edu Tue Jan 27 19:10:30 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4DA7AB3A92 for ; Tue, 27 Jan 2004 19:10:30 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXLgCgm7 for ; Tue, 27 Jan 2004 19:10:29 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 4EBBDB3A8F for ; Tue, 27 Jan 2004 19:10:29 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004012713102409492 for ; Tue, 27 Jan 2004 13:10:24 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HS5UHC00.1OP; Tue, 27 Jan 2004 13:10:24 -0500 Message-ID: <4016A9F7.90908@wi.mit.edu> Date: Tue, 27 Jan 2004 13:12:07 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Adam Russell , metakit@equi4.com Subject: Re: [Metakit] Re: SystemError: NULL result without error in PyObject_Call References: <4016A70D.9040808@infolaunch.com> In-Reply-To: <4016A70D.9040808@infolaunch.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 18:10:30 -0000 Adam Russell wrote: > Varying the number of keys make no difference. Admittedly, the 16 was > amistake, however, just using 2 breaks as well with th exact same error. You might need to send the database along. Could you try this test code? It is the same code that you had, except that it creates a new database and puts in some values. If this fails, then something is wrong with your version of metakit. Otherwise your database might be corrupt. import metakit import random, string db=metakit.storage("foo.mk",1) vw=db.getas("linelists[mid:S,field:S]") m=db.getas('map[_H:I,_R:I]') vwh=vw.hash(m,2) # add data for i in range(100): vwh.append((random.choice(string.letters), random.choice(string.letters))) # end add rows=[] for r in vwh: rows.append((r.mid,r.field)) db=None print rows From rus20376@infolaunch.com Tue Jan 27 19:22:02 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 328A4B3A9F for ; Tue, 27 Jan 2004 19:22:02 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX5b9RLR for ; Tue, 27 Jan 2004 19:22:01 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id 14165B3A9E for ; Tue, 27 Jan 2004 19:22:01 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0RIM0W08536; Tue, 27 Jan 2004 13:22:00 -0500 (EST) Message-ID: <4016AC49.3040304@infolaunch.com> Date: Tue, 27 Jan 2004 13:22:01 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Brian Kelley , Metakit list Subject: Re: [Metakit] Re: SystemError: NULL result without error in PyObject_Call References: <4016A70D.9040808@infolaunch.com> <4016A9F7.90908@wi.mit.edu> In-Reply-To: <4016A9F7.90908@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 18:22:02 -0000 Maybe I am creating my hash all wrong. view looks like db=metakit.storage("foo.mk",1) vw=db.getas("linelists[mid:S,field1:S,field2:S,field3:S,field4:S,field5:S,field6:S,field7:S]") What should the mapped view look like? And if I need to re-open this later on can I use the same view which I can save somewhere in a dictionary? Brian Kelley wrote: > Adam Russell wrote: > >> Varying the number of keys make no difference. Admittedly, the 16 was >> amistake, however, just using 2 breaks as well with th exact same error. > > > You might need to send the database along. Could you try this test > code? It is the same code that you had, except that it creates a new > database and puts in some values. If this fails, then something is > wrong with your version of metakit. Otherwise your database might be > corrupt. > > import metakit > import random, string > db=metakit.storage("foo.mk",1) > vw=db.getas("linelists[mid:S,field:S]") > m=db.getas('map[_H:I,_R:I]') > vwh=vw.hash(m,2) > > # add data > for i in range(100): > vwh.append((random.choice(string.letters), > random.choice(string.letters))) > # end add > > rows=[] > for r in vwh: rows.append((r.mid,r.field)) > > db=None > print rows > From bkelley@wi.mit.edu Tue Jan 27 19:27:41 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 57901B3A9E for ; Tue, 27 Jan 2004 19:27:41 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXZywsea for ; Tue, 27 Jan 2004 19:27:41 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 952E0B3A94 for ; Tue, 27 Jan 2004 19:27:40 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004012713273509721 for ; Tue, 27 Jan 2004 13:27:35 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HS5V9Z00.PU5; Tue, 27 Jan 2004 13:27:35 -0500 Message-ID: <4016ADFE.8010906@wi.mit.edu> Date: Tue, 27 Jan 2004 13:29:18 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Adam Russell , metakit@equi4.com Subject: Re: [Metakit] Re: SystemError: NULL result without error in PyObject_Call References: <4016A70D.9040808@infolaunch.com> <4016A9F7.90908@wi.mit.edu> <4016AC49.3040304@infolaunch.com> In-Reply-To: <4016AC49.3040304@infolaunch.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 18:27:41 -0000 Adam Russell wrote: > Maybe I am creating my hash all wrong. > view looks like > db=metakit.storage("foo.mk",1) > vw=db.getas("linelists[mid:S,field1:S,field2:S,field3:S,field4:S,field5:S,field6:S,field7:S]") > Whenever you call getas the table is restructured. If in one script you call vw=db.getas("linelists[mid:S,field1:S,field2:S,field3:S,field4:S,field5:S,field6:S,field7:S]") unless you want the database to change, you should either call the same getas statement in another script or use vw = db.view("linelists") Your hash calls are correct. run the following script and send back the results (I'm just opening your db here): import metakit db=metakit.storage("db/linelists"+"09_11_2003"+".mk",1) print db.description() From rus20376@infolaunch.com Tue Jan 27 20:18:01 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 122D5B3A93 for ; Tue, 27 Jan 2004 20:18:01 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX4Ptw1v for ; Tue, 27 Jan 2004 20:18:00 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id AF913B3A92 for ; Tue, 27 Jan 2004 20:17:59 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0RJHwW15421; Tue, 27 Jan 2004 14:17:58 -0500 (EST) Message-ID: <4016B967.2030707@infolaunch.com> Date: Tue, 27 Jan 2004 14:17:59 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Brian Kelley Subject: Re: [Metakit] Re: SystemError: NULL result without error in PyObject_Call References: <4016A70D.9040808@infolaunch.com> <4016A9F7.90908@wi.mit.edu> <4016AC49.3040304@infolaunch.com> <4016ADFE.8010906@wi.mit.edu> In-Reply-To: <4016ADFE.8010906@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 19:18:01 -0000 I have narrowed the problem down to where I am actually touching the rows. Blows up as soon as that happens. import metakit db=metakit.storage("db/linelists"+"09_11_2003"+".mk",1)#OK vw=db.view("linelists")#OK m=db.getas("09_11_2003"+'_map[_H:I,_R:I]')#OK vw=vw.hash(m,1)#OK for r in vw:#OK print r.mid,r.syndrome,r.date_visit,r.age_group,r.gender,r.zip,r.event,r.disposition,r.temperature,r.dob,r.provider_id,r.mem_rec,r.event_id,r.icd9,r.tests,r.payer#This dies? Why would that be? I was assuming the error was in the hash business....... When just print r I get that there are actually things there i.e. Now I am really confused. :( Brian Kelley wrote: > Adam Russell wrote: > >> Maybe I am creating my hash all wrong. >> view looks like >> db=metakit.storage("foo.mk",1) >> vw=db.getas("linelists[mid:S,field1:S,field2:S,field3:S,field4:S,field5:S,field6:S,field7:S]") >> > > > Whenever you call getas the table is restructured. If in one script > you call > > vw=db.getas("linelists[mid:S,field1:S,field2:S,field3:S,field4:S,field5:S,field6:S,field7:S]") > > > unless you want the database to change, you should either call the > same getas statement in another script or use > > vw = db.view("linelists") > > Your hash calls are correct. > > run the following script and send back the results (I'm just opening > your db here): > > import metakit > db=metakit.storage("db/linelists"+"09_11_2003"+".mk",1) > print db.description() > From bkelley@wi.mit.edu Tue Jan 27 20:31:13 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 68D00B3A92 for ; Tue, 27 Jan 2004 20:31:13 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXRemcma for ; Tue, 27 Jan 2004 20:31:13 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 93743B3AE8 for ; Tue, 27 Jan 2004 20:31:12 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004012714310810748 for ; Tue, 27 Jan 2004 14:31:08 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HS5Y7W00.PV1; Tue, 27 Jan 2004 14:31:08 -0500 Message-ID: <4016BCE2.90006@wi.mit.edu> Date: Tue, 27 Jan 2004 14:32:50 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Adam Russell , Metakit mailing list Subject: Re: [Metakit] Re: SystemError: NULL result without error in PyObject_Call References: <4016A70D.9040808@infolaunch.com> <4016A9F7.90908@wi.mit.edu> <4016AC49.3040304@infolaunch.com> <4016ADFE.8010906@wi.mit.edu> <4016B967.2030707@infolaunch.com> In-Reply-To: <4016B967.2030707@infolaunch.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 19:31:13 -0000 Adam Russell wrote: > I have narrowed the problem down to where I am actually touching the > rows. Blows up as soon as that happens. > import metakit > db=metakit.storage("db/linelists"+"09_11_2003"+".mk",1)#OK > vw=db.view("linelists")#OK > m=db.getas("09_11_2003"+'_map[_H:I,_R:I]')#OK > vw=vw.hash(m,1)#OK > for r in vw:#OK > print > r.mid,r.syndrome,r.date_visit,r.age_group,r.gender,r.zip,r.event,r.disposition,r.temperature,r.dob,r.provider_id,r.mem_rec,r.event_id,r.icd9,r.tests,r.payer#This > dies? Just for grins try this m = db.view("_map") since you alread have a map in the database or the correct type. There is no need to create two hashes. Brian From rus20376@infolaunch.com Tue Jan 27 21:38:47 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id CF24DB3A92; Tue, 27 Jan 2004 21:38:46 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXsDStcq; Tue, 27 Jan 2004 21:38:46 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id 354DAB3A8D; Tue, 27 Jan 2004 21:38:45 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0RKciW25963; Tue, 27 Jan 2004 15:38:44 -0500 (EST) Message-ID: <4016CC54.8070108@infolaunch.com> Date: Tue, 27 Jan 2004 15:38:44 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler Subject: Re: [Metakit] Re: SystemError: NULL result without error in PyObject_Call References: <4016A70D.9040808@infolaunch.com> <4016A9F7.90908@wi.mit.edu> <4016AC49.3040304@infolaunch.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Brian Kelley cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 20:38:47 -0000 Problem Solved!! I turned off f-ing psyco and it just plain worked. I usually make it a habit of using it bt default for all my classes. I am now the biggest enemy of psyco you can imagine. Why psyco was breaking this is beyond me. While I know psyco will not make any improvement on the metakit stuff it would have made a difference in some other code which I'll now just break of into another class. Jean-Claude Wippler wrote: > Adam, > >> Maybe I am creating my hash all wrong. >> view looks like >> db=metakit.storage("foo.mk",1) >> vw=db.getas("linelists[mid:S,field1:S,field2:S,field3:S,field4: >> S,field5:S,field6:S,field7:S]") >> >> What should the mapped view look like? > > > The mapped view is always the same (2 ints): > m=db.getas('map[_H:I,_R:I]') > >> And if I need to re-open this later on can I use the same view which >> I can save somewhere in a dictionary? > > > I don't understand "save in a dictionary". > > You can re-open later with: > vw = db.view("linelists").hash(db.view("map"),1) > > You're not re-using the "map" view for other views as well, I hope? > It has to be managed by hash() and it should not be used anywhere > else but in the above setup. > > Can you construct an example which fails? FWIW, there are some > examples in the examples/ subdir of the MK source distribution, maybe > they can be of use. > > -jcw From bkelley@wi.mit.edu Tue Jan 27 22:37:11 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B9210B3A92; Tue, 27 Jan 2004 22:37:10 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXUVyTjq; Tue, 27 Jan 2004 22:37:10 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 8FE67B3A8D; Tue, 27 Jan 2004 22:37:09 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004012716370512746 ; Tue, 27 Jan 2004 16:37:05 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HS641T00.VU7; Tue, 27 Jan 2004 16:37:05 -0500 Message-ID: <4016DA68.7060300@wi.mit.edu> Date: Tue, 27 Jan 2004 16:38:48 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Adam Russell Subject: Re: [Metakit] Re: SystemError: NULL result without error in PyObject_Call References: <4016A70D.9040808@infolaunch.com> <4016A9F7.90908@wi.mit.edu> <4016AC49.3040304@infolaunch.com> <4016CC54.8070108@infolaunch.com> In-Reply-To: <4016CC54.8070108@infolaunch.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list cc: Jean-Claude Wippler X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 21:37:11 -0000 Adam Russell wrote: > Problem Solved!! > I turned off f-ing psyco and it just plain worked. > I usually make it a habit of using it bt default for all my classes. I > am now the biggest enemy of psyco you can imagine. That's interesting. I use psyco all the time in conjunction with metakit and haven't had these problems. I'll keep this in the back of my head if I run into any problems in the future. From rus20376@infolaunch.com Tue Jan 27 22:49:37 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0366AB3AF0; Tue, 27 Jan 2004 22:49:37 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXBykbts; Tue, 27 Jan 2004 22:49:36 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id 2B695B3A9F; Tue, 27 Jan 2004 22:49:31 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0RLnUW07279; Tue, 27 Jan 2004 16:49:30 -0500 (EST) Message-ID: <4016DCE9.90001@infolaunch.com> Date: Tue, 27 Jan 2004 16:49:29 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler Subject: Re: [Metakit] Re: SystemError: NULL result without error in PyObject_Call References: <4016A70D.9040808@infolaunch.com> <4016A9F7.90908@wi.mit.edu> <4016AC49.3040304@infolaunch.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jan 2004 21:49:37 -0000 Jean-Claude Wippler wrote: > Adam, > >> Maybe I am creating my hash all wrong. >> view looks like >> db=metakit.storage("foo.mk",1) >> vw=db.getas("linelists[mid:S,field1:S,field2:S,field3:S,field4: >> S,field5:S,field6:S,field7:S]") >> >> What should the mapped view look like? > > > The mapped view is always the same (2 ints): > m=db.getas('map[_H:I,_R:I]') > >> And if I need to re-open this later on can I use the same view which >> I can save somewhere in a dictionary? > > > I don't understand "save in a dictionary". By this I mean I was thinking about having a dictionary of views mk_views={} db=metakit.storage("foo.mk",1) vw=db.getas("linelists[mid:S,field1:S,field2:S,field3:S,field4: S,field5:S,field6:S,field7:S]") m=db.getas('map[_H:I,_R:I]') vw=vw.hash(m,2) mk_views[key]=vw and then to reuse I can just pull one out vw=mk_views[key] > > You can re-open later with: > vw = db.view("linelists").hash(db.view("map"),1) > > You're not re-using the "map" view for other views as well, I hope? > It has to be managed by hash() and it should not be used anywhere > else but in the above setup. > > Can you construct an example which fails? FWIW, there are some > examples in the examples/ subdir of the MK source distribution, maybe > they can be of use. > > -jcw From Bernhard.Nemec@jaeger-toennies.com Wed Jan 28 15:03:43 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 8D620B3A91 for ; Wed, 28 Jan 2004 15:03:43 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXeIo0lX for ; Wed, 28 Jan 2004 15:03:41 +0100 (CET) Received: from mail.jaeger-toennies.com (unknown [62.180.118.137]) by dizzie.triqs.com (Postfix) with SMTP id 8108AB3A8A for ; Wed, 28 Jan 2004 15:03:41 +0100 (CET) Received: FROM mail.jaeger-toennies.com BY mail.jaeger-toennies.com ; Wed Jan 28 15:03:39 2004 +0100 Received: by NTS-HOE04 with Internet Mail Service (5.5.2653.19) id ; Wed, 28 Jan 2004 15:03:39 +0100 Message-ID: <5FDDBF5436E7D61181BB0030482198AB0FE865@nts-hoe32.jaeger-toennies.de> From: "Nemec, Bernhard" To: metakit@equi4.com Subject: RE: [Metakit] RAM usage Date: Wed, 28 Jan 2004 15:03:39 +0100 X-Mailer: Internet Mail Service (5.5.2653.19) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 14:03:43 -0000 Thank you all for your explanations. However, my little test program keeps eating up more and more real, not only virtual, memory. Here is my test code for Linux: //------------------------------main.cpp----------- #include #include #include void ShowMemoryInfo(){ // shows some lines of /proc information pid_t id = getpid(); const long file_name_size = 256; char file_name[file_name_size]; snprintf( file_name, file_name_size, "/proc/%d/status", id ); FILE* fp = fopen( file_name, "r" ); if ( fp == NULL ) { std::cout << "cannot open file ='" << file_name << "'" << std::endl; return; } const long line_size = 1024; char line[line_size]; char* temp; while ( ! feof( fp ) ) { temp = fgets( line, line_size, fp ); if ( temp == NULL ) break; if ( strncasecmp( line, "vm", 2 ) == 0 ) std::cout << " " << line; } fclose( fp ); } int main(int, char**) { c4_Storage storage( "test.db", true ); c4_View testview = storage.GetAs("test[id:I,name:S]");; c4_IntProp id("id"); c4_StringProp name("name"); while (1) { ShowMemoryInfo(); int n; for (int i = 0; i < 10000; i++) { n = testview.Add( id[i] + name["Mr. Brown"] ); } std::cout << "last added row was no. " << n << std::endl; if ( !storage.Commit() ){ std::cout << "Error by Commit()" << std::endl; return 1; } } return 0; } //------------------------------------------------- I compile it using g++ -lmk4 main.cpp with gcc 3.2.2 and Metakit 2.4.9.3 on RedHat Linux 9. The output looks like this: VmSize: 2488 kB VmLck: 0 kB VmRSS: 968 kB VmData: 52 kB VmStk: 8 kB VmExe: 8 kB VmLib: 2364 kB last added row was no. 9999 VmSize: 2968 kB VmLck: 0 kB VmRSS: 1384 kB VmData: 408 kB VmStk: 8 kB VmExe: 8 kB VmLib: 2364 kB last added row was no. 19999 VmSize: 3472 kB VmLck: 0 kB VmRSS: 1644 kB VmData: 668 kB VmStk: 8 kB VmExe: 8 kB VmLib: 2364 kB last added row was no. 29999 VmSize: 3768 kB VmLck: 0 kB VmRSS: 1648 kB VmData: 672 kB VmStk: 8 kB VmExe: 8 kB VmLib: 2364 kB last added row was no. 39999 VmSize: 4284 kB VmLck: 0 kB VmRSS: 1772 kB VmData: 796 kB VmStk: 8 kB VmExe: 8 kB VmLib: 2364 kB last added row was no. 49999 VmSize: 4844 kB VmLck: 0 kB VmRSS: 1844 kB VmData: 868 kB VmStk: 8 kB VmExe: 8 kB VmLib: 2364 kB last added row was no. 59999 and goes on with increasing RSS and Data sizes forever. It looks like a memory leak to me. Am I doing anything wrong? Bernhard From jyl@mod3.net Wed Jan 28 17:13:31 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id AD606B3A8D for ; Wed, 28 Jan 2004 17:13:31 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXOj1Bbx for ; Wed, 28 Jan 2004 17:13:29 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id 8A80DB3A8A for ; Wed, 28 Jan 2004 17:13:28 +0100 (CET) Received: (qmail 1379 invoked by uid 48); 28 Jan 2004 16:13:26 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Wed, 28 Jan 2004 08:13:26 -0800 (PST) Message-ID: <65269.209.233.24.218.1075306406.squirrel@mod3.net> Date: Wed, 28 Jan 2004 08:13:26 -0800 (PST) Subject: RE: [Metakit] RAM usage From: "Jacob Levy" To: In-Reply-To: <5FDDBF5436E7D61181BB0030482198AB0FE865@nts-hoe32.jaeger-toennies.de> References: <5FDDBF5436E7D61181BB0030482198AB0FE865@nts-hoe32.jaeger-toennies.de> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 16:13:32 -0000 Bernard I'm going to assume that this test program is representative of what you're doing with Metakit -- otherwise my answer is wrong, and you should disregard it. No, you're not doing anything "wrong" here and neither is Metakit. It's just that your expectation of what Metakit should do is wrong. v.Add(..) will always add a new row, as the increasing number of rows printed by your program shows. Because the view keeps growing, so does the memory usage. Please read up on c4_View::Add() and what it does. I think you're expecting the id to somehow serve as a key and to overwrite existing rows with that id. This is not how Metakit works. --JYL > Thank you all for your explanations. > > However, my little test program keeps eating up more and more real, not > only virtual, memory. > Here is my test code for Linux: > > //------------------------------main.cpp----------- > #include > #include > #include > > void ShowMemoryInfo(){ // shows some lines of /proc information > pid_t id = getpid(); > > const long file_name_size = 256; > char file_name[file_name_size]; > snprintf( file_name, file_name_size, "/proc/%d/status", id ); > FILE* fp = fopen( file_name, "r" ); > if ( fp == NULL ) > { > std::cout << "cannot open file ='" << file_name << "'" << std::endl; > return; > } > > const long line_size = 1024; > char line[line_size]; > char* temp; > > while ( ! feof( fp ) ) > { > temp = fgets( line, line_size, fp ); > if ( temp == NULL ) > break; > > if ( strncasecmp( line, "vm", 2 ) == 0 ) > std::cout << " " << line; > } > > fclose( fp ); > } > > int main(int, char**) > { > c4_Storage storage( "test.db", true ); > c4_View testview = storage.GetAs("test[id:I,name:S]");; > > c4_IntProp id("id"); > c4_StringProp name("name"); > > > while (1) > { > ShowMemoryInfo(); > > int n; > > for (int i = 0; i < 10000; i++) > { > n = testview.Add( id[i] + name["Mr. Brown"] ); > } > > std::cout << "last added row was no. " << n << std::endl; > > if ( !storage.Commit() ){ > std::cout << "Error by Commit()" << std::endl; > return 1; > } > } > > return 0; > } > //------------------------------------------------- > > I compile it using > > g++ -lmk4 main.cpp > > with gcc 3.2.2 and Metakit 2.4.9.3 on RedHat Linux 9. > > The output looks like this: > > VmSize: 2488 kB > VmLck: 0 kB > VmRSS: 968 kB > VmData: 52 kB > VmStk: 8 kB > VmExe: 8 kB > VmLib: 2364 kB > last added row was no. 9999 > VmSize: 2968 kB > VmLck: 0 kB > VmRSS: 1384 kB > VmData: 408 kB > VmStk: 8 kB > VmExe: 8 kB > VmLib: 2364 kB > last added row was no. 19999 > VmSize: 3472 kB > VmLck: 0 kB > VmRSS: 1644 kB > VmData: 668 kB > VmStk: 8 kB > VmExe: 8 kB > VmLib: 2364 kB > last added row was no. 29999 > VmSize: 3768 kB > VmLck: 0 kB > VmRSS: 1648 kB > VmData: 672 kB > VmStk: 8 kB > VmExe: 8 kB > VmLib: 2364 kB > last added row was no. 39999 > VmSize: 4284 kB > VmLck: 0 kB > VmRSS: 1772 kB > VmData: 796 kB > VmStk: 8 kB > VmExe: 8 kB > VmLib: 2364 kB > last added row was no. 49999 > VmSize: 4844 kB > VmLck: 0 kB > VmRSS: 1844 kB > VmData: 868 kB > VmStk: 8 kB > VmExe: 8 kB > VmLib: 2364 kB > last added row was no. 59999 > > and goes on with increasing RSS and Data sizes forever. It looks like a > memory leak to me. > > Am I doing anything wrong? > > Bernhard > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From Bernhard.Nemec@jaeger-toennies.com Wed Jan 28 17:24:05 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 43052B3A9A for ; Wed, 28 Jan 2004 17:24:04 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXCZLQNg for ; Wed, 28 Jan 2004 17:24:02 +0100 (CET) Received: from mail.jaeger-toennies.com (unknown [62.180.118.137]) by dizzie.triqs.com (Postfix) with SMTP id D3717B3A8A for ; Wed, 28 Jan 2004 17:24:01 +0100 (CET) Received: FROM mail.jaeger-toennies.com BY mail.jaeger-toennies.com ; Wed Jan 28 17:23:59 2004 +0100 Received: by NTS-HOE04 with Internet Mail Service (5.5.2653.19) id ; Wed, 28 Jan 2004 17:23:59 +0100 Message-ID: <5FDDBF5436E7D61181BB0030482198AB0FE867@nts-hoe32.jaeger-toennies.de> From: "Nemec, Bernhard" To: metakit@equi4.com Subject: RE: [Metakit] RAM usage Date: Wed, 28 Jan 2004 17:23:58 +0100 X-Mailer: Internet Mail Service (5.5.2653.19) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 16:24:05 -0000 > No, you're not doing anything "wrong" here and neither is > Metakit. It's > just that your expectation of what Metakit should do is > wrong. v.Add(..) > will always add a new row, as the increasing number of rows printed by > your program shows. Because the view keeps growing, so does the memory > usage. But I call storage.Commit() after each batch of rows for the very purpose of flushing the new rows from memory to disk. How else can I tell Metakit to free up the space for these rows after writing them to disk? > > Please read up on c4_View::Add() and what it does. I think you're > expecting the id to somehow serve as a key and to overwrite > existing rows > with that id. This is not how Metakit works. No, I don't want to overwrite anything. I expect the data file to grow, and because of MMF, I also expect the size of allocated *virtual memory* to grow. But I don't want the *physical* RAM usage to grow endlessly. Bernhard From jyl@mod3.net Wed Jan 28 17:41:21 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D7EABB3A9F for ; Wed, 28 Jan 2004 17:41:20 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXvyvXS8 for ; Wed, 28 Jan 2004 17:41:18 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id 13FCCB3A8A for ; Wed, 28 Jan 2004 17:41:18 +0100 (CET) Received: (qmail 30400 invoked by uid 48); 28 Jan 2004 16:41:16 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Wed, 28 Jan 2004 08:41:16 -0800 (PST) Message-ID: <65168.209.233.24.218.1075308076.squirrel@mod3.net> Date: Wed, 28 Jan 2004 08:41:16 -0800 (PST) Subject: RE: [Metakit] RAM usage From: "Jacob Levy" To: In-Reply-To: <5FDDBF5436E7D61181BB0030482198AB0FE867@nts-hoe32.jaeger-toennies.de> References: <5FDDBF5436E7D61181BB0030482198AB0FE867@nts-hoe32.jaeger-toennies.de> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 16:41:21 -0000 Bernard Committing does not free the memory. As I said before, Metakit maps the *entire file* into memory. So if the file grows, so does the memory mapping requirement. Whether the physical RAM used by the program grows or not is a factor influenced by the OS, not by Metakit. --JYL >> No, you're not doing anything "wrong" here and neither is >> Metakit. It's >> just that your expectation of what Metakit should do is >> wrong. v.Add(..) >> will always add a new row, as the increasing number of rows printed by >> your program shows. Because the view keeps growing, so does the memory >> usage. > > But I call storage.Commit() after each batch of rows for the very > purpose of flushing the new rows from memory to disk. > How else can I tell Metakit to free up the space for these rows after > writing them to disk? > >> >> Please read up on c4_View::Add() and what it does. I think you're >> expecting the id to somehow serve as a key and to overwrite >> existing rows >> with that id. This is not how Metakit works. > > No, I don't want to overwrite anything. I expect the data file to grow, > and because of MMF, I also expect the size of allocated *virtual memory* > to grow. But I don't want the *physical* RAM usage to grow endlessly. > > Bernhard > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Wed Jan 28 18:35:45 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4A439B3A8D for ; Wed, 28 Jan 2004 18:35:45 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX5owL4G for ; Wed, 28 Jan 2004 18:35:42 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 21056B3A8A for ; Wed, 28 Jan 2004 18:35:42 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 33E4C83C26; Wed, 28 Jan 2004 18:35:40 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 56D0F83914 for ; Wed, 28 Jan 2004 18:35:36 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <5FDDBF5436E7D61181BB0030482198AB0FE867@nts-hoe32.jaeger-toennies.de> References: <5FDDBF5436E7D61181BB0030482198AB0FE867@nts-hoe32.jaeger-toennies.de> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <61C38F56-51B8-11D8-8F8F-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] RAM usage Date: Wed, 28 Jan 2004 18:35:36 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 17:35:45 -0000 Nemec, Bernhard wrote: (First of all, thanks for creating a simple test app) > No, I don't want to overwrite anything. I expect the data file to > grow, and > because of MMF, I also expect the size of allocated *virtual memory* to > grow. But I don't want the *physical* RAM usage to grow endlessly. Hmm... I'm not sure what is going on. I took your code, switched to blocked views, made an optimization so it would run faster, and had to take it *way* up to 95M rows to make it reach resident size which exceed the RAM on my Linux box. And indeed, it starts thrashing - I thought at first that Linux would simply use up memory because it was all free, but it does keep going and pushing stuff out to swap space: [...] last added row was no. 94399999 VmSize: 2083072 kB VmLck: 0 kB VmRSS: 729096 kB VmData: 794820 kB VmStk: 8 kB VmExe: 24 kB VmLib: 2368 kB last added row was no. 94499999 As you can see, I switched to blocked views: $ sdx mkinfo test.db test.db Metakit data starts at offset 0 and is stored as little-endian Views: 94401x test2 {{_B {id:I name}}} $ And swapping is obvious: $ vmstat 3 procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 1 0 140112 4312 7896 13448 0 2 27 23 24 18 1 0 95 3 1 0 141808 6092 6904 13028 961 1272 3161 2479 1727 1200 33 9 0 58 0 1 143680 6420 6008 14604 993 1357 3083 2543 1627 955 41 9 0 51 1 0 161000 4620 5384 10652 1673 5779 2205 6857 1351 395 43 5 0 52 0 3 162836 3352 5292 13092 1727 4520 3781 6171 1573 945 42 11 0 48 0 1 176228 4680 3496 15628 1795 1145 4045 2880 1690 1530 30 5 0 65 1 0 177392 5048 3364 14200 1432 984 2680 2804 1601 797 47 8 0 45 1 3 180332 3448 3328 11932 1143 1647 2699 3305 1640 834 41 10 0 49 1 0 199456 3592 3392 13484 1917 6375 2373 7743 1292 429 30 4 0 66 1 1 220064 5636 3356 14472 2647 7493 4469 7535 1875 1843 30 9 0 60 [...] Now compare this with a datafile which contains three 32-bit ints instead of your small string: last added row was no. 99199999 VmSize: 59220 kB VmLck: 0 kB VmRSS: 57400 kB VmData: 56760 kB VmStk: 8 kB VmExe: 24 kB VmLib: 2368 kB last added row was no. 99299999 $ sdx mkinfo test.db test.db Metakit data starts at offset 0 and is stored as little-endian Views: 99301x test2 {{_B {id:I n1:I n2:I n3:I}}} $ ls -l test.db -rw-r--r-- 1 jcw users 1536395782 Jan 28 18:10 test.db $ This is more like it: 57 Mb resident data for a 1.5 Gb datafile (and no swapping at all now). There are no memory leaks in MK as far as I know, so the only conclusion can be that MK must be holding on to a lot more temp data with string properties. Which is not totally surprising, variable-sized strings take quite some data management to track in columns - and there are 94,401 strings columns in that first test above (each subview in the blocked view). The math points to about 8.4 Kb resident per string column, which is in fact not surprising at all: internal buffers get malloc'ed in 4 Kb chuncks (the slack is for dealing with potential growth), and there are two vectors for each string column: byte data and string sizes. It looks like MK is keeping all subview string buffers allocated, not just mapped to file. I'm surprised. Note that all of the above uses blocked views. The problem just mentioned does not happen when using a single view (but the performance of commits slows down dramatically as the number of rows increase, I'd say at least two orders of magnitude for as far as I got). The good news though, is that the behavior you describe does *not* carry forward once the datafile grows way further. I do see some fluctuations: last added row was no. 12099999 VmSize: 473956 kB VmLck: 0 kB VmRSS: 1376 kB VmData: 400 kB VmStk: 8 kB VmExe: 24 kB VmLib: 2368 kB last added row was no. 12199999 VmSize: 474824 kB VmLck: 0 kB VmRSS: 2244 kB VmData: 1268 kB VmStk: 8 kB VmExe: 24 kB VmLib: 2368 kB last added row was no. 12299999 VmSize: 474316 kB VmLck: 0 kB VmRSS: 1736 kB VmData: 760 kB VmStk: 8 kB VmExe: 24 kB VmLib: 2368 kB last added row was no. 12399999 But memory usage really does not increase in the way you assume: last added row was no. 16099999 VmSize: 635836 kB VmLck: 0 kB VmRSS: 1536 kB VmData: 560 kB VmStk: 8 kB VmExe: 24 kB VmLib: 2368 kB last added row was no. 16199999 VmSize: 635840 kB VmLck: 0 kB VmRSS: 1540 kB VmData: 564 kB VmStk: 8 kB VmExe: 24 kB VmLib: 2368 kB last added row was no. 16299999 (The datafile was at 700 Mb at this point) -jcw PS. Below are the changes I made (it's better not to allocate temp property values inside a loop, use a row object defined outside it): c4_View testview = storage.GetAs("test2[_B[id:I,n1:I,n2:I,n3:I]]").Blocked(); //c4_View testview = storage.GetAs("test1[id:I,n1:I,n2:I,n3:I]"); c4_IntProp id("id"),n1("n1"),n2("n2"),n3("n3"); c4_StringProp name("name"); for (int j = 0; j < 1000; ++j) { ShowMemoryInfo(); c4_Row r; n1(r) = 1234567; n2(r) = 2345678; n3(r) = 3456789; int n; for (int i = 0; i < 100000; i++) { id(r) = i; n = testview.Add( r ); } [...] From rus20376@infolaunch.com Wed Jan 28 23:03:36 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 580A6B3A8D for ; Wed, 28 Jan 2004 23:03:36 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXI8fllT for ; Wed, 28 Jan 2004 23:03:35 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id B8E8CB3A8A for ; Wed, 28 Jan 2004 23:03:34 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0SM3XW04273 for ; Wed, 28 Jan 2004 17:03:33 -0500 (EST) Message-ID: <401831B5.20609@infolaunch.com> Date: Wed, 28 Jan 2004 17:03:33 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] keeping views in a dictionary? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 22:03:36 -0000 Below is a test of something I thought of. I create a hashed view and then store the view in a dictionary. I then try and use the vw from teh dict. Fails every time with a system error. Should this be able to work? Is my test code wrong somehow? import metakit class mk_test: stored_views={} def getVW(self,dt): if self.stored_views.has_key(dt): print "has view in dict" return self.stored_views[dt] else: db=metakit.storage("temp"+dt+".mk",1) db.autocommit() vw=db.getas("linelists[mid:S,field:S]") m=db.getas(dt+'_map[_H:I,_R:I]') vw=vw.hash(m,1) self.stored_views[dt]=vw return vw def load(self,dt): vw=self.getVW(dt) for i in range(9): vw.append(str(i),"ill") def getRows(self,dt): vw=self.getVW(dt) for r in vw: print r.mid,r.field if __name__=="__main__": dt="01_01_2004" mt=mk_test() mt.load(dt) mt.getRows(dt) From bkelley@wi.mit.edu Wed Jan 28 23:46:53 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id BBA80B3A8D for ; Wed, 28 Jan 2004 23:46:53 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXf6byZp for ; Wed, 28 Jan 2004 23:46:52 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id A41CFB3A8A for ; Wed, 28 Jan 2004 23:46:51 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004012817464727397 for ; Wed, 28 Jan 2004 17:46:47 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HS81XZ00.820; Wed, 28 Jan 2004 17:46:47 -0500 Message-ID: <40183C47.2050308@wi.mit.edu> Date: Wed, 28 Jan 2004 17:48:39 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Adam Russell , Metakit mailing list Subject: Re: [Metakit] keeping views in a dictionary? References: <401831B5.20609@infolaunch.com> In-Reply-To: <401831B5.20609@infolaunch.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 22:46:54 -0000 Adam Russell wrote: > Below is a test of something I thought of. > I create a hashed view and then store the view in a dictionary. > I then try and use the vw from teh dict. Fails every time with a > system error. > Should this be able to work? Is my test code wrong somehow? You need to keep a reference to the original storage (db in this case) when it gets garbage collected and the views won't work any more. Even if they are referenced in a dictionary. Your code didn't come across very well by the way, the indentation got messed up. It looks like you are keeping multiple hashes to the same single view. I don't think this will work in the long run. If you modify one hash, none of the other hashes will know about the update. Brian From rus20376@infolaunch.com Thu Jan 29 00:06:42 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4604AB3AEA for ; Thu, 29 Jan 2004 00:06:42 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXPafk41 for ; Thu, 29 Jan 2004 00:06:40 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id 1DB98B3A8A for ; Thu, 29 Jan 2004 00:06:40 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0SN6dW08874 for ; Wed, 28 Jan 2004 18:06:39 -0500 (EST) Message-ID: <4018407F.2020702@infolaunch.com> Date: Wed, 28 Jan 2004 18:06:39 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Subject: Re: [Metakit] keeping views in a dictionary? References: <401831B5.20609@infolaunch.com> <40183C47.2050308@wi.mit.edu> In-Reply-To: <40183C47.2050308@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 23:06:43 -0000 Thanks for the advice. Sorry about the messed up code but I didn't want to clutter anybody's inbox with an attachment and I guess the indents got hosed. The appearance of multiple hashes is not my intention. Just didn't pay too much attention to my test class. :( Brian Kelley wrote: > Adam Russell wrote: > >> Below is a test of something I thought of. >> I create a hashed view and then store the view in a dictionary. >> I then try and use the vw from teh dict. Fails every time with a >> system error. >> Should this be able to work? Is my test code wrong somehow? > > > You need to keep a reference to the original storage (db in this case) > when it gets garbage collected and the views won't work any more. > Even if they are referenced in a dictionary. Your code didn't come > across very well by the way, the indentation got messed up. > > It looks like you are keeping multiple hashes to the same single > view. I don't think this will work in the long run. If you modify > one hash, none of the other hashes will know about the update. > > Brian From andrew@pixymisa.com Wed Jan 28 17:36:59 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A9110B3A9F for ; Wed, 28 Jan 2004 17:36:58 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXfix6VS for ; Wed, 28 Jan 2004 17:36:55 +0100 (CET) Received: from ez3.ez-web-hosting.com (ez3.ez-web-hosting.com [209.151.89.253]) by dizzie.triqs.com (Postfix) with ESMTP id C4153B3A8A for ; Wed, 28 Jan 2004 17:36:54 +0100 (CET) Received: from adsl-3-64.swiftdsl.com.au ([218.214.3.64] helo=pixymisa.com) by ez3.ez-web-hosting.com with asmtp (Exim 4.24) id 1AlsgP-0001hr-NH for metakit@equi4.com; Wed, 28 Jan 2004 11:36:54 -0500 Message-ID: <4017E51F.5070904@pixymisa.com> Date: Thu, 29 Jan 2004 03:36:47 +1100 From: Andrew Maizels User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 Cc: metakit@equi4.com Subject: Re: [Metakit] RAM usage References: <5FDDBF5436E7D61181BB0030482198AB0FE867@nts-hoe32.jaeger-toennies.de> In-Reply-To: <5FDDBF5436E7D61181BB0030482198AB0FE867@nts-hoe32.jaeger-toennies.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner-Information: Please contact the ISP for more information X-MailScanner: Found to be clean X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - ez3.ez-web-hosting.com X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [47 12] X-AntiAbuse: Sender Address Domain - pixymisa.com X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Thu, 29 Jan 2004 11:42:45 +0100 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2004 16:36:59 -0000 Nemec, Bernhard wrote: > No, I don't want to overwrite anything. I expect the data file to grow, and > because of MMF, I also expect the size of allocated *virtual memory* to > grow. But I don't want the *physical* RAM usage to grow endlessly. As I understand it, the allocation of physical RAM is a function of the operating system, not of Metakit or the application. If you have enough physical RAM, your entire database will be mapped there. If not, some of it will be swapped out. That's just how Metakit works - it relies on the OS virtual memory system rather than building its own buffer pool on top of that. Andrew. From mbloore@yahoo.com Thu Jan 29 19:29:47 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2FC15B3AA1 for ; Thu, 29 Jan 2004 19:29:47 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXtiCvBE for ; Thu, 29 Jan 2004 19:29:46 +0100 (CET) Received: from web14304.mail.yahoo.com (web14304.mail.yahoo.com [216.136.173.80]) by dizzie.triqs.com (Postfix) with SMTP id 867CBB3A8D for ; Thu, 29 Jan 2004 19:29:45 +0100 (CET) Message-ID: <20040129182943.75051.qmail@web14304.mail.yahoo.com> Received: from [65.95.141.67] by web14304.mail.yahoo.com via HTTP; Thu, 29 Jan 2004 10:29:43 PST Date: Thu, 29 Jan 2004 10:29:43 -0800 (PST) From: mARK bLOORE Subject: RE: [Metakit] RAM usage To: metakit In-Reply-To: <65168.209.233.24.218.1075308076.squirrel@mod3.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2004 18:29:47 -0000 for clarification: MK saves modified columns of data on the heap. when you commit, it writes them to the file (via normal i/o or memory mapping, as appropriate), and then frees that heap space. so if the file is mapped (as it will be if it is not huge) the total space in use stays about the same over a commit. the virtual space will likely grow, for the first commit, as the memory allocator likely keeps the freed heap in the virual space. --- Jacob Levy wrote: > Bernard > > Committing does not free the memory. As I said before, Metakit maps the > *entire file* into memory. So if the file grows, so does the memory > mapping requirement. Whether the physical RAM used by the program grows or > not is a factor influenced by the OS, not by Metakit. > > --JYL __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ From jyl@mod3.net Thu Jan 29 19:56:16 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3C369B3AEA for ; Thu, 29 Jan 2004 19:56:16 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXiqgnwB for ; Thu, 29 Jan 2004 19:56:14 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id 60299B3AED for ; Thu, 29 Jan 2004 19:56:13 +0100 (CET) Received: (qmail 10497 invoked by uid 48); 29 Jan 2004 18:56:10 -0000 Received: from 192.18.42.10 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Thu, 29 Jan 2004 10:56:10 -0800 (PST) Message-ID: <38750.192.18.42.10.1075402570.squirrel@mod3.net> Date: Thu, 29 Jan 2004 10:56:10 -0800 (PST) Subject: RE: [Metakit] RAM usage From: "Jacob Levy" To: In-Reply-To: <20040129182943.75051.qmail@web14304.mail.yahoo.com> References: <65168.209.233.24.218.1075308076.squirrel@mod3.net> <20040129182943.75051.qmail@web14304.mail.yahoo.com> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2004 18:56:16 -0000 I've often wondered if a more sparse representation of modified data that *only* contains the modifications would save memory and also give acceptable lookup performance. Presumably after modification its likely that a commit will follow shortly, so then it might make sense to penalize lookups that happen between modifications and commits. Just a thought. --JYL > for clarification: > MK saves modified columns of data on the heap. when you commit, it > writes them to the file (via normal i/o or memory mapping, as > appropriate), and then frees that heap space. so if the file is mapped > (as it will be if it is not huge) the total space in use stays about the > same over a commit. the virtual space will likely grow, for the first > commit, as the memory allocator likely keeps the freed heap in the > virual space. > > --- Jacob Levy wrote: >> Bernard >> >> Committing does not free the memory. As I said before, Metakit maps >> the *entire file* into memory. So if the file grows, so does the >> memory mapping requirement. Whether the physical RAM used by the >> program grows or not is a factor influenced by the OS, not by Metakit. >> >> --JYL > > > __________________________________ > Do you Yahoo!? > Yahoo! SiteBuilder - Free web site building tool. Try it! > http://webhosting.yahoo.com/ps/sb/ > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From gary.h.merrill@gsk.com Thu Jan 29 20:21:22 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id CEF87B3AEA for ; Thu, 29 Jan 2004 20:21:21 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXgM3YOf for ; Thu, 29 Jan 2004 20:21:10 +0100 (CET) Received: from usrtpnp2.glaxowellcome.com (firewall2.glaxowellcome.com [192.58.204.209]) by dizzie.triqs.com (Postfix) with ESMTP id 560C3B3A91 for ; Thu, 29 Jan 2004 20:21:09 +0100 (CET) Received: by usrtpnp2.glaxowellcome.com; id OAA17884; Thu, 29 Jan 2004 14:21:08 -0500 (EST) From: Received: from smtphub.glaxo.com(152.51.63.189) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma017589; Thu, 29 Jan 04 14:20:46 -0500 Received: by ussunt2.glaxo.com id OAA05010; Thu, 29 Jan 2004 14:20:45 -0500 (EST) Received: from 166.71.134.118 by us9n52.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Thu, 29 Jan 2004 14:20:41 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i0TJKfB16209 for ; Thu, 29 Jan 2004 14:20:41 -0500 (EST) Subject: RE: [Metakit] RAM usage To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Thu, 29 Jan 2004 14:21:31 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 01/29/2004 02:20:50 PM MIME-Version: 1.0 X-WSS-ID: 6C078283504012-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2004 19:21:22 -0000 I think the assumption that a commit would follow shortly after modification is not in general valid. In the application I am currently working on, there is a point at which there will be a considerable number of access/modify actions prior to the next commit. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 "Jacob Levy" Sent by: To: mbloore@yahoo.com metakit-bounces@e qui4.com cc: metakit@equi4.com Subject: RE: [Metakit] RAM usage 29-Jan-2004 13:56 Please respond to jyl@mod3.net I've often wondered if a more sparse representation of modified data that *only* contains the modifications would save memory and also give acceptable lookup performance. Presumably after modification its likely that a commit will follow shortly, so then it might make sense to penalize lookups that happen between modifications and commits. Just a thought. --JYL > for clarification: > MK saves modified columns of data on the heap. when you commit, it > writes them to the file (via normal i/o or memory mapping, as > appropriate), and then frees that heap space. so if the file is mapped > (as it will be if it is not huge) the total space in use stays about the > same over a commit. the virtual space will likely grow, for the first > commit, as the memory allocator likely keeps the freed heap in the > virual space. > > --- Jacob Levy wrote: >> Bernard >> >> Committing does not free the memory. As I said before, Metakit maps >> the *entire file* into memory. So if the file grows, so does the >> memory mapping requirement. Whether the physical RAM used by the >> program grows or not is a factor influenced by the OS, not by Metakit. >> >> --JYL > > > __________________________________ > Do you Yahoo!? > Yahoo! SiteBuilder - Free web site building tool. Try it! > http://webhosting.yahoo.com/ps/sb/ > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit _____________________________________________ Metakit mailing list - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From tom.krehbiel@motorola.com Thu Jan 29 21:21:44 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B9F43B3AF2 for ; Thu, 29 Jan 2004 21:21:44 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXNHTX8A for ; Thu, 29 Jan 2004 21:21:42 +0100 (CET) Received: from motgate4.mot.com (motgate4.mot.com [144.189.100.102]) by dizzie.triqs.com (Postfix) with ESMTP id 8F487B3A8D for ; Thu, 29 Jan 2004 21:21:41 +0100 (CET) Received: from az33exr01.mot.com (az33exr01.mot.com [10.64.251.231]) by motgate4.mot.com (Motorola/Motgate4) with ESMTP id i0TKLQuP025721 for ; Thu, 29 Jan 2004 13:21:29 -0700 (MST) Received: from motorola.com ([172.16.104.118]) by az33exr01.mot.com (Motorola/az33exr01) with ESMTP id i0TKJ3l2002787 for ; Thu, 29 Jan 2004 14:19:04 -0600 Message-ID: <40196B48.40900@motorola.com> Date: Thu, 29 Jan 2004 13:21:28 -0700 From: Tom Krehbiel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 Cc: metakit@equi4.com Subject: Re: [Metakit] RAM usage References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2004 20:21:45 -0000 If the heap is being used to store record modification the it is possible that MetaKit is fragmenting the heap. It's been a long time since I looked at this issue but I'm pretty sure this will generate performance issues and I think it will also create RAM usage issues because of the minimum block size managed by the malloc command. Tom Krehbiel > I think the assumption that a commit would follow shortly after > modification is not in general valid. In the application I am currently > working on, there is a point at which there will be a considerable number > of access/modify actions prior to the next commit. > > -------------------------------------- > Gary H. Merrill > Director and Principal Scientist, New Applications > Data Exploration Sciences > GlaxoSmithKline Inc. > (919) 483-8456 > > "Jacob Levy" > > > Sent by: To: mbloore@yahoo.com > metakit-bounces@e > qui4.com cc: metakit@equi4.com > Subject: RE: [Metakit] RAM usage > > 29-Jan-2004 13:56 > Please respond to > jyl@mod3.net > > I've often wondered if a more sparse representation of modified data that > *only* contains the modifications would save memory and also give > acceptable lookup performance. Presumably after modification its likely > that a commit will follow shortly, so then it might make sense to penalize > lookups that happen between modifications and commits. Just a thought. > > --JYL > > >>for clarification: >>MK saves modified columns of data on the heap. when you commit, it >>writes them to the file (via normal i/o or memory mapping, as >>appropriate), and then frees that heap space. so if the file is mapped >>(as it will be if it is not huge) the total space in use stays about the >>same over a commit. the virtual space will likely grow, for the first >>commit, as the memory allocator likely keeps the freed heap in the >>virual space. >> >>--- Jacob Levy wrote: >> >>>Bernard >>> >>>Committing does not free the memory. As I said before, Metakit maps >>>the *entire file* into memory. So if the file grows, so does the >>>memory mapping requirement. Whether the physical RAM used by the >>>program grows or not is a factor influenced by the OS, not by Metakit. >>> >>>--JYL >> >> >>__________________________________ >>Do you Yahoo!? >>Yahoo! SiteBuilder - Free web site building tool. Try it! >>http://webhosting.yahoo.com/ps/sb/ >>_____________________________________________ >>Metakit mailing list - Metakit@equi4.com >>http://www.equi4.com/mailman/listinfo/metakit > > > > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > > > > > > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Thu Jan 29 21:28:37 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2A95DB3AF3 for ; Thu, 29 Jan 2004 21:28:37 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX79ymjx for ; Thu, 29 Jan 2004 21:28:35 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 0AA72B3A8D for ; Thu, 29 Jan 2004 21:28:35 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 79FE383C26; Thu, 29 Jan 2004 21:28:32 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id D21E783914 for ; Thu, 29 Jan 2004 21:28:14 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <38750.192.18.42.10.1075402570.squirrel@mod3.net> References: <65168.209.233.24.218.1075308076.squirrel@mod3.net> <20040129182943.75051.qmail@web14304.mail.yahoo.com> <38750.192.18.42.10.1075402570.squirrel@mod3.net> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] RAM usage Date: Thu, 29 Jan 2004 21:28:14 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2004 20:28:37 -0000 Jacob Levy wrote: > I've often wondered if a more sparse representation of modified data > that > *only* contains the modifications would save memory and also give > acceptable lookup performance. Presumably after modification its likely > that a commit will follow shortly, so then it might make sense to > penalize > lookups that happen between modifications and commits. Just a thought. That is actually already the case: a file is mapped, then when a change is made, MK allocates 4 Kb chunks *only* for those pieces which it needs to change, and continues to use mapped memory for the rest of the column. Two extra details: 1) insertions and deletions often cause more buffers to be allocated and copied than necessary, even though more could be done with simply moving and re-using the original mapping, and 2) all of this happens per column, i.e. only for those properties which are actually modified. The fly in the ointment is insertion/deletion of rows. There are lots of tricks one could play to take this further. Commit-aside was created in anticipation of that. The key is to allow fast access by byte offset into such a mix of original-mapped and modified/shifted regions. There is also another way to do this: introduce a layer at the view level which collects changes and "applies them virtually". This, btw, is where I'm planning to go in the future (yeah, promises, promises, I know...). I even made a web page for it not so long ago, though it was never made public: http://www.equi4.com/mkisolation.html - it contains a few inaccuracies and skims over a few details, but it ought to the get the basic idea across. -jcw From tom.krehbiel@motorola.com Thu Jan 29 22:30:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 25F3FB3AF3 for ; Thu, 29 Jan 2004 22:30:38 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXG5M6zh for ; Thu, 29 Jan 2004 22:30:36 +0100 (CET) Received: from motgate8.mot.com (motgate8.mot.com [129.188.136.8]) by dizzie.triqs.com (Postfix) with ESMTP id 86104B3A8D for ; Thu, 29 Jan 2004 22:30:35 +0100 (CET) Received: from az33exr01.mot.com (az33exr01.mot.com [10.64.251.231]) by motgate8.mot.com (Motorola/Motgate3) with ESMTP id i0TLUXqC009281 for ; Thu, 29 Jan 2004 14:30:34 -0700 (MST) Received: from motorola.com ([172.16.104.118]) by az33exr01.mot.com (Motorola/az33exr01) with ESMTP id i0TLIcl2016600 for ; Thu, 29 Jan 2004 15:18:38 -0600 Message-ID: <4019793F.9040708@motorola.com> Date: Thu, 29 Jan 2004 14:21:03 -0700 From: Tom Krehbiel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 Cc: Metakit list Subject: Re: [Metakit] RAM usage References: <65168.209.233.24.218.1075308076.squirrel@mod3.net> <20040129182943.75051.qmail@web14304.mail.yahoo.com> <38750.192.18.42.10.1075402570.squirrel@mod3.net> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jan 2004 21:30:38 -0000 I sounds like the heap is being managed in 4K chunks so fragmentation shouldn't be a problem. Tom K. > Jacob Levy wrote: > >> I've often wondered if a more sparse representation of modified data that >> *only* contains the modifications would save memory and also give >> acceptable lookup performance. Presumably after modification its likely >> that a commit will follow shortly, so then it might make sense to >> penalize >> lookups that happen between modifications and commits. Just a thought. > > That is actually already the case: a file is mapped, then when a change > is made, MK allocates 4 Kb chunks *only* for those pieces which it needs > to change, and continues to use mapped memory for the rest of the column. > > Two extra details: 1) insertions and deletions often cause more buffers > to be allocated and copied than necessary, even though more could be > done with simply moving and re-using the original mapping, and 2) all of > this happens per column, i.e. only for those properties which are > actually modified. The fly in the ointment is insertion/deletion of rows. > > There are lots of tricks one could play to take this further. > Commit-aside was created in anticipation of that. The key is to allow > fast access by byte offset into such a mix of original-mapped and > modified/shifted regions. > > There is also another way to do this: introduce a layer at the view > level which collects changes and "applies them virtually". This, btw, > is where I'm planning to go in the future (yeah, promises, promises, I > know...). I even made a web page for it not so long ago, though it was > never made public: http://www.equi4.com/mkisolation.html - it contains a > few inaccuracies and skims over a few details, but it ought to the get > the basic idea across. From rus20376@infolaunch.com Fri Jan 30 05:43:53 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6606CB3AEA for ; Fri, 30 Jan 2004 05:43:53 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXyfY7lL for ; Fri, 30 Jan 2004 05:43:52 +0100 (CET) Received: from stirling.bwh.harvard.edu (stirling.bwh.harvard.edu [134.174.249.23]) by dizzie.triqs.com (Postfix) with ESMTP id CA624B3A8D for ; Fri, 30 Jan 2004 05:43:50 +0100 (CET) Received: from infolaunch.com ([170.223.248.119])i0U4hnW27462 for ; Thu, 29 Jan 2004 23:43:49 -0500 (EST) Message-ID: <4019E105.5020300@infolaunch.com> Date: Thu, 29 Jan 2004 23:43:49 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit list Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] utf-8 string weirdness X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jan 2004 04:43:53 -0000 I am writing an application which usesmetakit to store lists of strings. Sometimes when I retrieve these strings I get these oddball utf-8 charcaters. wtf? It'll be workin a-ok then all of a sudden *boom* all these utf-8 strings. I'll then go back and re-red the same db file and it'll be ok. What would cause this? I am using mk4py. Any advice would be greatly appreciated!! ryngitis, Acute Nos; Dysphagia; Dysphagia', 'None', 'PR') ('33333334', 'Shock/Coma', '11/09/2003', '30 to 34', 'M', 'Minneapolis:55440:*No MemRec*', '', 'None', '0', '01/01/1970', 'BT, Test', '33333334', '33333334', 'Sh ock, Other, W/o Trauma', 'None', 'PR') read encounters, n= 15 cutoffdate= 20031110 ll= [' '] ('55555555', 'Fever', '1/10/2003', '/1970', 'e', 's:55440:*NoMemRec*', '', '/197 0', 'e', '/1/1970', 'T, Test', '2222222', '', 'itis, Acute Nos; Dysphagia; Dysph agia', 'PR', '\x01\x87\x1d\xbf\x88\n\xe4\x80\x03\xfa$\xea\x8f\n\xec\x80\xcb\r\xd 6\x88\x0e\xa1\x80\xad\x16\xa6\x84\x0e\xa9\x80PR') ('55555555', 'Respiratory', '/1/1970', '/1970', 'None', 's:55440:*NoMemRec*', '' , '1970', '\xaa\xaa\xaa*\x99\x99\x99\x99\x99\x99\x99\t11/10/2003', '\x05\xff\xff \xff?\xff\xff?\xff\xff?\xff?1970', 'T, Test', '8888888', '$\xea\x8f\n\xe6\x80\xc b\r\xce\x88\x0e\x99\x80\xad\x16\xa6\x84\x0e\xa1\x80\xd4\x80\x01\x87\x15\x9f\x88\ n\xdc\x80\x01\x87\x1d\xbf\x88\n\xe4\x80\x03\xfa$\xea\x8f\n\xec\x80\xcb\r\xd6\x88 \x0e\xa1\x80\xad\x16\xa6\x84\x0e\xa9\x80PR', 'bdominal Pain,generalize; Chest Pa in, Unspecified', 'ec*', 'PR') Exception in thread autoProcessor: From rus20376@infolaunch.com Fri Jan 30 18:10:04 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6042BB3A9A for ; Fri, 30 Jan 2004 18:10:04 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXsmM94Q for ; Fri, 30 Jan 2004 18:09:57 +0100 (CET) Received: from out012.verizon.net (out012pub.verizon.net [206.46.170.137]) by dizzie.triqs.com (Postfix) with ESMTP id D4D28B3A8D for ; Fri, 30 Jan 2004 18:09:56 +0100 (CET) Received: from infolaunch.com ([151.199.46.140]) by out012.verizon.net (InterMail vM.5.01.06.06 201-253-122-130-106-20030910) with ESMTP id <20040130170955.KYXV1590.out012.verizon.net@infolaunch.com> for ; Fri, 30 Jan 2004 11:09:55 -0600 Message-ID: <401A8FE3.2030105@infolaunch.com> Date: Fri, 30 Jan 2004 12:09:55 -0500 From: Adam Russell User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: RE: [Metakit] utf-8 string weirdness Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH at out012.verizon.net from [151.199.46.140] at Fri, 30 Jan 2004 11:09:55 -0600 X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jan 2004 17:10:04 -0000 This problem went away when I did the following. I had wrapped all my mk usage into one class whihc I had instantiated globally in a particular module which then made long heavy use of my cllass full of mk calls. That class of mk calls keeps a dictionary of the storages that have been opened(there may be quite a few) and re-uses them to recreate a view whihc is then used to select,append, etc. When I instead instatiated my mk interface class locally just in functions that needed it, and thus garbage collected often, the problem disappeared. I guess the problem is something about keeping the db references for a long time maybe? This was a really nutty problem, I'll try and replicate it with some simple test code. Message: 7 Date: Thu, 29 Jan 2004 23:43:49 -0500 From: Adam Russell Subject: [Metakit] utf-8 string weirdness To: Metakit list Message-ID: <4019E105.5020300@infolaunch.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed I am writing an application which usesmetakit to store lists of strings. Sometimes when I retrieve these strings I get these oddball utf-8 charcaters. wtf? It'll be workin a-ok then all of a sudden *boom* all these utf-8 strings. I'll then go back and re-red the same db file and it'll be ok. What would cause this? I am using mk4py. Any advice would be greatly appreciated!! ryngitis, Acute Nos; Dysphagia; Dysphagia', 'None', 'PR') ('33333334', 'Shock/Coma', '11/09/2003', '30 to 34', 'M', 'Minneapolis:55440:*No MemRec*', '', 'None', '0', '01/01/1970', 'BT, Test', '33333334', '33333334', 'Sh ock, Other, W/o Trauma', 'None', 'PR') read encounters, n= 15 cutoffdate= 20031110 ll= [' '] ('55555555', 'Fever', '1/10/2003', '/1970', 'e', 's:55440:*NoMemRec*', '', '/197 0', 'e', '/1/1970', 'T, Test', '2222222', '', 'itis, Acute Nos; Dysphagia; Dysph agia', 'PR', '\x01\x87\x1d\xbf\x88\n\xe4\x80\x03\xfa$\xea\x8f\n\xec\x80\xcb\r\xd 6\x88\x0e\xa1\x80\xad\x16\xa6\x84\x0e\xa9\x80PR') ('55555555', 'Respiratory', '/1/1970', '/1970', 'None', 's:55440:*NoMemRec*', '' , '1970', '\xaa\xaa\xaa*\x99\x99\x99\x99\x99\x99\x99\t11/10/2003', '\x05\xff\xff \xff?\xff\xff?\xff\xff?\xff?1970', 'T, Test', '8888888', '$\xea\x8f\n\xe6\x80\xc b\r\xce\x88\x0e\x99\x80\xad\x16\xa6\x84\x0e\xa1\x80\xd4\x80\x01\x87\x15\x9f\x88\ n\xdc\x80\x01\x87\x1d\xbf\x88\n\xe4\x80\x03\xfa$\xea\x8f\n\xec\x80\xcb\r\xd6\x88 \x0e\xa1\x80\xad\x16\xa6\x84\x0e\xa9\x80PR', 'bdominal Pain,generalize; Chest Pa in, Unspecified', 'ec*', 'PR') Exception in thread autoProcessor: From yarigin@protvino.systel.ru Mon Feb 9 06:58:21 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 90C5FB3A8E for ; Mon, 9 Feb 2004 06:58:21 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXAb2xqz for ; Mon, 9 Feb 2004 06:58:20 +0100 (CET) Received: from protvino.systel.ru (unknown [80.252.142.120]) by dizzie.triqs.com (Postfix) with ESMTP id D7E50B3A8D for ; Mon, 9 Feb 2004 06:58:19 +0100 (CET) Received: from protvino.systel.ru ([162.162.111.112]) by protvino.systel.ru (8.12.8p2/8.12.8) with ESMTP id i195wJRZ016057 for ; Mon, 9 Feb 2004 08:58:19 +0300 (MSK) (envelope-from yarigin@protvino.systel.ru) Message-ID: <4027217A.2070601@protvino.systel.ru> Date: Mon, 09 Feb 2004 08:58:18 +0300 From: Yarigin S Organization: SYSTEL-PROTVINO User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, ru, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Mon, 09 Feb 2004 11:01:08 +0100 Subject: [Metakit] New Python interface has unwanted "feature" X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 05:58:22 -0000 Hello, In the new version of python interface there is a problem: When python dictionary converted to list of properties, this properties list has the same order as python dict props list, which is not the same as native metakit properties order. It results in unpredictable behavour of binary search if combination of sort/locate/search is used from python. Better to return old behavour, or make apropriate changes in makeDictRow to preserv native order of properties. S. From bkelley@wi.mit.edu Mon Feb 9 15:29:16 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C6409B3A8E for ; Mon, 9 Feb 2004 15:29:16 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXHSriCu for ; Mon, 9 Feb 2004 15:29:15 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id D7AFAB3A8B for ; Mon, 9 Feb 2004 15:29:14 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004020909291011674 for ; Mon, 09 Feb 2004 09:29:10 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HSTMWM00.PKN; Mon, 9 Feb 2004 09:29:10 -0500 Message-ID: <4027999A.1020708@wi.mit.edu> Date: Mon, 09 Feb 2004 09:30:50 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Yarigin S Subject: Re: [Metakit] New Python interface has unwanted "feature" References: <4027217A.2070601@protvino.systel.ru> In-Reply-To: <4027217A.2070601@protvino.systel.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 14:29:17 -0000 Yarigin S wrote: > Hello, > In the new version of python interface there is a problem: > When python dictionary converted to list of properties, this > properties list has the same order as python dict props list, which is > not the same as native metakit properties order. It results in > unpredictable behavour of binary search if combination of > sort/locate/search is used from python. > Better to return old behavour, or make apropriate changes in > makeDictRow to preserv native order of properties. Could you give a small code example of this? I'm not clear on what you are saying. Here is what I think you are saying: >> st = metakit.storage() >> vw = st.getas("t[a,b,c:I,d:F]") >> print vw.properties() {'a': Property('S', 'a'), 'c': Property('I', 'c'), 'b': Property('S', 'b'), 'd': Property('F', 'd')} Note that there is no ordering to a dictionary, it is undefined in the python standard. If you want to get the properties in the table creation order use: >> print vw.structure() [Property('S', 'a'), Property('S', 'b'), Property('I', 'c'), Property('F', 'd')] so to extract a metakit table into a python list in native order: props = vw.structure() table = [[getattr(row, prop.name) for prop in props] for row in vw] # or more pedantically table = [] for row in vw: res = [] for prop in props: res.append(getattr(row, prop.name)) table.append(res) This might not be what you are talking about though. Brian > > S. > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Mon Feb 9 19:20:42 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 8AE69B3A9A for ; Mon, 9 Feb 2004 19:20:42 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX3el7Hn for ; Mon, 9 Feb 2004 19:20:41 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id EEA70B3A8B for ; Mon, 9 Feb 2004 19:20:40 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 0732883C27; Mon, 9 Feb 2004 19:20:38 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E5E5183914 for ; Mon, 9 Feb 2004 19:20:21 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) Content-Transfer-Encoding: 7bit Message-Id: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Mon, 9 Feb 2004 19:20:21 +0100 X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] multi-column sorting X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 18:20:42 -0000 Does anyone know how to do a multi-column sort, using column-wise permutations? I'm looking at ways to optimize sorting, based on the fact that MK has a column-wise data organization. The current sort does row-wise comparisons. Here's what I'm after: * take the first column, sort it, and produce a permutation vector for it * take the second column, sort it, and ... * etc... So you end up with N permutations vectors. Each sorts only on the specified column. Assume that the sorts are stable, i.e. identical entries are kep in input order. I'm looking for a way to combine these permutations so that the result is a permutation which represents full sort order of the entire view. Tried some ideas, but none of them seem to be right. I've googled on the web, but can't find much relevant info (or don't grasp the theoretical foundations enough to spot the essential tricks). It would seem related to radix sorting. D'you know what's involved or have tips on what terms to look for? -jcw From bkelley@wi.mit.edu Mon Feb 9 22:13:35 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 51B00B3A8F; Mon, 9 Feb 2004 22:13:35 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX7OSBwV; Mon, 9 Feb 2004 22:13:32 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 1E387B3A8B; Mon, 9 Feb 2004 22:13:31 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004020916132617355 ; Mon, 09 Feb 2004 16:13:26 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HSU5ME00.1JK; Mon, 9 Feb 2004 16:13:26 -0500 Message-ID: <4027F85A.30807@wi.mit.edu> Date: Mon, 09 Feb 2004 16:15:06 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler , Metakit mailing list Subject: Re: [Metakit] multi-column sorting References: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> In-Reply-To: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 21:13:36 -0000 Jean-Claude Wippler wrote: > Does anyone know how to do a multi-column sort, using column-wise > permutations? Is this the right approach? Thinking out loud here. A multi-column sort is really a precedence sort. You only need to sort on a secondary or tertiary key if the primary key has equivalent values. 1) sort on next property 2) if any more properties groupby property else goto 4 3) foreach groupby subview go to 1 4) reassemble final indices Result -> stable sort, I think :) Hopefully I'm being clear. Here is a stupid diagram in the form of a tree structure: view -> column1 -> groupby -> column 2 -> groupby -> sort -> Final indices sort column1 sort on subview A subview -> 1 2 B subview -> 7 8 This seems to be less work overall and might utilize metakit's tech a little better. I have a half-baked python implementation that requires an index column (mainly because the groupby method doesn't keep track of the row index) The best-time complexity should be less as well as the size of the sorts should decrease the further down the tree you go. Brian From bkelley@wi.mit.edu Mon Feb 9 22:51:53 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 96E05B3A8F; Mon, 9 Feb 2004 22:51:53 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXHCufmq; Mon, 9 Feb 2004 22:51:52 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 87E2DB3A8B; Mon, 9 Feb 2004 22:51:51 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004020916514717875 ; Mon, 09 Feb 2004 16:51:47 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HSU7EB00.5F0; Mon, 9 Feb 2004 16:51:47 -0500 Message-ID: <40280157.9020108@wi.mit.edu> Date: Mon, 09 Feb 2004 16:53:27 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Brian Kelley Subject: Re: [Metakit] multi-column sorting a real answer References: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> <4027F85A.30807@wi.mit.edu> In-Reply-To: <4027F85A.30807@wi.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit mailing list cc: Jean-Claude Wippler X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 21:51:53 -0000 How about another stupid answer. Suppose you have N perumation columns with M entries. Each permutation column contains the row's order if that column was the sort column. One caveat, you will need to have identical values have the same perumation order. In either case Form an integer for each of size N*int(log2(M)+1) bits in order of the ordered primary keys. Sort these using a radix-log2(M) sort or big-integer sort. Example: N1 N2 N3 => integer value 0 3 0 001100 0 1 1 000101 1 2 2 001010 2 2 3 101011 3 2 3 110101 and use a radix-2 sort on the integer values. This preperation step is O(N) but potentially memory intensive. Four keys on a million records using five keys would be 5 * int(log(1000000)+1) * 1000000 = 70,000,000 or ~70MB. Actually, that's not too bad. The problem comes when the number of bits to represent M gets larger and larger, but this grows logarithmically. Brian From jcw@equi4.com Mon Feb 9 22:52:56 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3D8BBB3AE8 for ; Mon, 9 Feb 2004 22:52:56 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXedS8WZ for ; Mon, 9 Feb 2004 22:52:51 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 797B4B3A8F for ; Mon, 9 Feb 2004 22:52:51 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 38B9583C27; Mon, 9 Feb 2004 22:52:49 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 003F683914 for ; Mon, 9 Feb 2004 22:52:46 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <4027F85A.30807@wi.mit.edu> References: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> <4027F85A.30807@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <4B837213-5B4A-11D8-9096-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] multi-column sorting Date: Mon, 9 Feb 2004 22:52:46 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 21:53:00 -0000 Brian Kelley wrote: > Jean-Claude Wippler wrote: > >> Does anyone know how to do a multi-column sort, using column-wise >> permutations? > > Is this the right approach? Thinking out loud here. > > A multi-column sort is really a precedence sort. You only need to > sort on a secondary or tertiary key if the primary key has equivalent > values. > > 1) sort on next property > 2) if any more properties groupby property else goto 4 > 3) foreach groupby subview go to 1 > 4) reassemble final indices > > Result -> stable sort, I think :) Yes, this is clear - and very much related, but quite what I mean. I'm looking for ways to use more efficient algorithms underneath MK, ie. as basis to do the above. I'm also looking for ways to do things lazily - i.e. defer some of the computations. This could have considerable implications when you sort a view and then as for a slice of it, i.e. only display a small section of it. > I have a half-baked python implementation that requires an index > column (mainly because the groupby method doesn't keep track of the > row index) The way this can be done is to add an extra column with row indices (sort of like APL's "iota") using the pair() operation, and then group. That way the result will carry original row indices with it. There are more such tricks waiting to be found & exposed. I'm currently trying to better understand what sort of core functions are needed to build the rest with. Hence the Q about per-column sorting and trying to find a way to combine permutations. To give an example - to sort on col 2, 4 reverse, and then 3 could be done using something like this: m2 = sortmap(col[2]) m3 = sortmap(col[3]) m4 = sortmap(col[4]) result = view.remap(m3.remap(reverse(m4)).remap(m2)) (with partial use, i.e. when fetching only a slice of the result, all sorts of neat tricks can be added, leading to behavior which I think resembles what you are describing above) Except that the above "permutation stacking" is not exactly right... alas. -jcw From bkelley@wi.mit.edu Mon Feb 9 23:01:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 7CEEAB3A8B for ; Mon, 9 Feb 2004 23:01:38 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXYoMK9j for ; Mon, 9 Feb 2004 23:01:36 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 938A0B3A8F for ; Mon, 9 Feb 2004 23:01:36 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004020917013518018 for ; Mon, 09 Feb 2004 17:01:35 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HSU7UN00.MGR for ; Mon, 9 Feb 2004 17:01:35 -0500 Message-ID: <402803A4.4010007@wi.mit.edu> Date: Mon, 09 Feb 2004 17:03:16 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Subject: [Fwd: Re: [Metakit] multi-column sorting] Content-Type: multipart/mixed; boundary="------------010104070807040408050201" X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 22:01:38 -0000 This is a multi-part message in MIME format. --------------010104070807040408050201 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit --------------010104070807040408050201 Content-Type: message/rfc822; name="Re: [Metakit] multi-column sorting" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="Re: [Metakit] multi-column sorting" Message-ID: <40280388.5010709@wi.mit.edu> Date: Mon, 09 Feb 2004 17:02:48 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler Subject: Re: [Metakit] multi-column sorting References: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> <4027F85A.30807@wi.mit.edu> <4B837213-5B4A-11D8-9096-000A9588127E@equi4.com> In-Reply-To: <4B837213-5B4A-11D8-9096-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Jean-Claude Wippler wrote: Ah, ignore my previous radix sort nonesense then. > To give an example - to sort on col 2, 4 reverse, and then 3 could be > done using something like this: > m2 = sortmap(col[2]) > m3 = sortmap(col[3]) > m4 = sortmap(col[4]) > result = view.remap(m3.remap(reverse(m4)).remap(m2)) > (with partial use, i.e. when fetching only a slice of the result, all > sorts of neat tricks can be added, leading to behavior which I think > resembles what you are describing above) > > Except that the above "permutation stacking" is not exactly right... > alas. It can't be can it? You should only remap m2 with m3 when m2 has equivalent values. You need a bucketing remap. view3.bucketremap(view2) -> remap the indices of view2 with the indices of view3 for equivalent values of m2. > > > -jcw > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit --------------010104070807040408050201-- From bkelley@wi.mit.edu Mon Feb 9 23:32:03 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C9BFAB3A8F; Mon, 9 Feb 2004 23:32:02 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX4Mn5dy; Mon, 9 Feb 2004 23:32:01 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id B57A0B3A8B; Mon, 9 Feb 2004 23:32:00 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004020917315618293 ; Mon, 09 Feb 2004 17:31:56 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HSU99800.GG8; Mon, 9 Feb 2004 17:31:56 -0500 Message-ID: <40280AC0.4050804@wi.mit.edu> Date: Mon, 09 Feb 2004 17:33:36 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler , Metakit mailing list Subject: Re: [Metakit] multi-column sorting References: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> <4027F85A.30807@wi.mit.edu> <4B837213-5B4A-11D8-9096-000A9588127E@equi4.com> In-Reply-To: <4B837213-5B4A-11D8-9096-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 22:32:03 -0000 > To give an example - to sort on col 2, 4 reverse, and then 3 could be > done using something like this: > m2 = sortmap(col[2]) > m3 = sortmap(col[3]) > m4 = sortmap(col[4]) > result = view.remap(m3.remap(reverse(m4)).remap(m2)) > (with partial use, i.e. when fetching only a slice of the result, all > sorts of neat tricks can be added, leading to behavior which I think > resembles what you are describing above) > > Except that the above "permutation stacking" is not exactly right... > alas. I think you need another piece of data. Imagine if your sortmap produced the following where order is the sort order and index is the index into the original view. Row is just the index in the view for my sanity. sortmap on column 1 Row # order index 0 0 7 1 0 5 2 1 4 3 2 3 4 3 2 5 4 11 6 4 18 7 4 1 sortmap on column 2 Row # order index 0 0 1 1 0 7 2 1 4 3 2 11 4 3 2 5 4 18 6 4 12 7 4 5 You can lazily evaluate secondary keys as follows. Imagine we are looking to find the index for sorted row #6. We look at the first sortmap and notice that there are multiple entries for order 4. This is a linear scan to lower rows and higher rows that assembles potential indices 11,18,1 where we want to find the second entry (since row 6 is the second 4) Now take the second sort map and isolate indices 11,18,1 row # order index 0 0 1 3 2 11 5 4 18 And the answer is original row 11! vw[6] -> machinations -> is original row 11 This could be nested indefinanetly. Pretty cool eh? > > > -jcw > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From tom.krehbiel@motorola.com Mon Feb 9 23:56:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 94CF3B3A8F for ; Mon, 9 Feb 2004 23:56:38 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXt5Xm1l for ; Mon, 9 Feb 2004 23:56:37 +0100 (CET) Received: from motgate8.mot.com (motgate8.mot.com [129.188.136.8]) by dizzie.triqs.com (Postfix) with ESMTP id B0A7CB3A8B for ; Mon, 9 Feb 2004 23:56:36 +0100 (CET) Received: from il06exr03.mot.com (il06exr03.mot.com [129.188.137.133]) by motgate8.mot.com (Motorola/Motgate3) with ESMTP id i19MuZAH008483 for ; Mon, 9 Feb 2004 15:56:35 -0700 (MST) Received: from az33exm31.corp.mot.com (az33exm31.corp.mot.com [10.64.65.145]) by il06exr03.mot.com (Motorola/il06exr03) with ESMTP id i19MuXlH004139 for ; Mon, 9 Feb 2004 16:56:33 -0600 Received: from motorola.com (RNA020-02 [172.16.104.118]) by az33exm31.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2657.2) id DJMAXQ7G; Mon, 9 Feb 2004 15:56:32 -0700 Message-ID: <40281020.1030900@motorola.com> Date: Mon, 09 Feb 2004 15:56:32 -0700 From: Tom Krehbiel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 Cc: Metakit list Subject: Re: [Metakit] multi-column sorting References: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> <4027F85A.30807@wi.mit.edu> <4B837213-5B4A-11D8-9096-000A9588127E@equi4.com> In-Reply-To: <4B837213-5B4A-11D8-9096-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2004 22:56:38 -0000 Jean-Claude, <...snip...> > To give an example - to sort on col 2, 4 reverse, and then 3 could be > done using something like this: > m2 = sortmap(col[2]) > m3 = sortmap(col[3]) > m4 = sortmap(col[4]) > result = view.remap(m3.remap(reverse(m4)).remap(m2)) > (with partial use, i.e. when fetching only a slice of the result, all > sorts of neat tricks can be added, leading to behavior which I think > resembles what you are describing above) > > Except that the above "permutation stacking" is not exactly right... alas. Are you looking for ideas on how to precondition or extend the column definition to improve the above calculation, or are you looking for effient methods of performing the above calculation without assuming any preconditioning or attributes on the col's? I'm not familiar with the method you used to perform the column wise store. I'm I correct in assuming that the values in a row are determined by equivelent offsets in the column stores? Tom K. From robert@mpe.mpg.de Tue Feb 10 10:43:05 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 256E6B3A8F for ; Tue, 10 Feb 2004 10:43:05 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXCWK5Vg for ; Tue, 10 Feb 2004 10:43:03 +0100 (CET) Received: from mpehp1.mpe-garching.mpg.de (mpehp1.mpe-garching.mpg.de [130.183.70.10]) by dizzie.triqs.com (Postfix) with ESMTP id B1C60B3A8A for ; Tue, 10 Feb 2004 10:43:02 +0100 (CET) Received: from robert2.mpe-garching.mpg.de (robert2.mpe-garching.mpg.de [130.183.136.59])with ESMTP id KAA04098 for ; Tue, 10 Feb 2004 10:43:00 +0100 (MET) Received: from robert2.mpe-garching.mpg.de (localhost.mpe-garching.mpg.de [127.0.0.1])i1A9hQxj090699 for ; Tue, 10 Feb 2004 10:43:26 +0100 (CET) (envelope-from krs@robert2.mpe-garching.mpg.de) Message-Id: <200402100943.i1A9hQxj090699@robert2.mpe-garching.mpg.de> To: metakit@equi4.com Subject: Re: [Metakit] multi-column sorting In-Reply-To: Your message of "Mon, 09 Feb 2004 15:56:32 MST." <40281020.1030900@motorola.com> Date: Tue, 10 Feb 2004 10:43:26 +0100 From: Klaus Robert Suetterlin X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: robert@mpe.mpg.de List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Feb 2004 09:43:05 -0000 I do not know if I understand the question correctly, but I'll try to help with the sorting stuff. I talk about radix sort, because JCW mentioned this in his initial post. Radix sort can also be used well on the individual columns. Just append virtual zeros to short values. If You want to sort several columns at once using radix sort You have to make the variable length values all the same length. I assume, that to each column an invisible sidecolumn is attached, that contains the sortposition of the column. I.e. for col[i] there is an asociated col*[i]. When col[i][j1] sorts to be first then col*[i][j1]=1. And the col[i][j10] that gets position 10 after a sort will have col*[i][j10]=10. Then just concatenate the bitpatterns of the col* in the order of Your columnwise sort and do a bitlevel radixsort. If You want to include one column in reverse sort orfer, just append its negated bitpattern. Example: col[1] col[2] col[3] www yahoo com cvs sourceforge net metakit equi4 com columnwise sorting will yield col*[1] col*[2] col*[3] 3 3 1 1 2 3 2 1 2 to sort col[1] col[2] col[3] use 11 11 01 01 10 11 10 01 10 radix sort will make this into 2 3 1 to sort col[2] col[3] col[1] use 11 01 11 10 11 01 01 10 10 radix sort will make this into 3 2 1 to sort col[3] reverse col[1] col[2] use 01 00 11 11 10 10 10 01 01 radix will result in 1 3 2 to sort reverse col[2] use 00 01 10 radix sort will produce 1 2 3 I hope this is an answer to the original question. -- Robert S. From tom.krehbiel@motorola.com Tue Feb 10 19:17:21 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id BDC09B3A94 for ; Tue, 10 Feb 2004 19:17:21 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXa2dfid for ; Tue, 10 Feb 2004 19:17:20 +0100 (CET) Received: from motgate8.mot.com (motgate8.mot.com [129.188.136.8]) by dizzie.triqs.com (Postfix) with ESMTP id EDFDDB3A8A for ; Tue, 10 Feb 2004 19:17:14 +0100 (CET) Received: from il06exr04.mot.com (il06exr04.mot.com [129.188.137.134]) by motgate8.mot.com (Motorola/Motgate3) with ESMTP id i1AIHDAH015984 for ; Tue, 10 Feb 2004 11:17:13 -0700 (MST) Received: from az33exm31.corp.mot.com (az33exm31.corp.mot.com [10.64.65.145]) by il06exr04.mot.com (Motorola/il06exr04) with ESMTP id i1AIHAsY013249 for ; Tue, 10 Feb 2004 12:17:11 -0600 Received: from motorola.com (RNA020-02 [172.16.104.118]) by az33exm31.corp.mot.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2657.2) id DJMAXSRV; Tue, 10 Feb 2004 11:17:10 -0700 Message-ID: <40292025.2060009@motorola.com> Date: Tue, 10 Feb 2004 11:17:09 -0700 From: Tom Krehbiel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit list Subject: Re: [Metakit] multi-column sorting References: <9F61D4AC-5B2C-11D8-9096-000A9588127E@equi4.com> <4027F85A.30807@wi.mit.edu> <4B837213-5B4A-11D8-9096-000A9588127E@equi4.com> <40281020.1030900@motorola.com> In-Reply-To: <40281020.1030900@motorola.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Feb 2004 18:17:22 -0000 Jean-Claude, An idea that may be useful in speed thing up is illustrated below. You probably are already aware of this method but here it is anyway. The method can clearly be improved but I will provide a simple example to illustrate the idea. Assumption: This example assumes that a single index can be used to access the same record contents for all columns. 1) A homogeneous index for a column is created by listing the first byte in each record. 2) An index is created for all columns of interest. 3) The column indexes are combined left to right in the order to be sorted followed by the row index value. At this point you have a homogeneous index for the columns of interest. 4) Sort the index using conventional methods. 5) The composite index now defines a set of ordered partitions on the data set. The individual partitions must be sorted on the complete record values but the scope of the sort has been reduced (which may be helpful on very large data sets). The partitions can also be used to speed up ordered record selection. Tom K. > Jean-Claude, > > <...snip...> > >> To give an example - to sort on col 2, 4 reverse, and then 3 could be >> done using something like this: >> m2 = sortmap(col[2]) >> m3 = sortmap(col[3]) >> m4 = sortmap(col[4]) >> result = view.remap(m3.remap(reverse(m4)).remap(m2)) >> (with partial use, i.e. when fetching only a slice of the result, all >> sorts of neat tricks can be added, leading to behavior which I think >> resembles what you are describing above) >> >> Except that the above "permutation stacking" is not exactly right... >> alas. > > > Are you looking for ideas on how to precondition or extend the column > definition to improve the above calculation, or are you looking for > effient methods of performing the above calculation without assuming any > preconditioning or attributes on the col's? > > I'm not familiar with the method you used to perform the column wise > store. I'm I correct in assuming that the values in a row are determined > by equivelent offsets in the column stores? > > Tom K. From moberle@fs.fed.us Wed Feb 11 01:25:43 2004 Return-Path: X-Original-To: Metakit@equi4.com Delivered-To: Metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 44AE4B3AE9 for ; Wed, 11 Feb 2004 01:25:43 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXc86LNc for ; Wed, 11 Feb 2004 01:25:41 +0100 (CET) Received: from rmaccsv2.rmacc.r2.fs.fed.us (rmaccsv2.rmacc.r2.fs.fed.us [166.2.208.11]) by dizzie.triqs.com (Postfix) with ESMTP id 05A55B3A94 for ; Wed, 11 Feb 2004 01:25:41 +0100 (CET) Received: from sv10.r2.fs.fed.us by rmaccsv2.rmacc.r2.fs.fed.us (USDA-FS/LakewoodCO/3-03-00) using ESMTP; id RAA53478 for ; Tue, 10 Feb 2004 17:25:39 -0700 To: Metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.7 March 21, 2001 Message-ID: From: Matt Oberle Date: Tue, 10 Feb 2004 17:24:05 -0700 X-MIMETrack: Serialize by Router on ENTR2B/E/USDAFS(Release 6.0.2CF2|July 23, 2003) at 02/10/2004 17:25:38 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Apparently-To: X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Win CE problem X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Feb 2004 00:25:43 -0000 I've been using Metakit for a while now with a Windows and Windows CE data entry application. It has been working well...fast and robust for some fairly heavy duty on-the-fly projections/selections and frequent commits. However, I recently noticed "leaky" behavior on a limited-memory Windows CE data recorder. I ran it through CodeSnitch (Entrek Software, Inc) which indicated that the app was riddled with small memory leaks. A majority of the reported leaks have a fairly long call stack, but all end up pointing to strdup in: void c4_StringArray::SetAt(int nIndex, const char* newElement) { char* s = (char*) _ptrs.GetAt(nIndex); if (s && *s) free(s); _ptrs.SetAt(nIndex, newElement && *newElement? _strdup(newElement) : ""); } The app RUNS, in fact it has been beta tested for a while now with no major problems (http://www.fs.fed.us/fmsc/measure/cruising/fstallymeter/). If I goofed anywhere with unicode/character conversions wouldn't the app bomb noticeably? Any ideas or clues to what I'm doing wrong? Anyone else using Metakit in Windows CE software? I'm using eVC++ 3.0 (MFC). ===================================== Matt Oberle From hgiese@ratiosoft.com Wed Feb 11 13:04:03 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 30665B3A92 for ; Wed, 11 Feb 2004 13:04:03 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXjXRekG for ; Wed, 11 Feb 2004 13:04:02 +0100 (CET) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.188]) by dizzie.triqs.com (Postfix) with ESMTP id 39D5BB3A88 for ; Wed, 11 Feb 2004 13:04:02 +0100 (CET) Received: from [212.227.126.206] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 1Aqt62-0005Hu-00 for metakit@equi4.com; Wed, 11 Feb 2004 13:04:02 +0100 Received: from [80.136.147.194] (helo=RATIOSOFTS.ratiosoft.com) by mrelayng.kundenserver.de with asmtp (TLSv1:RC4-SHA:128) (Exim 3.35 #1) id 1Aqt61-0000Hk-00 for metakit@equi4.com; Wed, 11 Feb 2004 13:04:01 +0100 Message-Id: <5.2.1.1.0.20040211125844.00a6ae30@pop.1und1.com> X-Sender: pt7397547-mbhg@pop.1und1.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.1 Date: Wed, 11 Feb 2004 13:04:01 +0100 To: metakit@equi4.com From: Helmut Giese In-Reply-To: <20040211110003.6A1AEB3AA4@dizzie.triqs.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Provags-ID: kundenserver.de abuse@kundenserver.de auth:78815929d143a437f91f152ae2fee55f X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Re: Win CE problem X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Feb 2004 12:04:03 -0000 > >void c4_StringArray::SetAt(int nIndex, const char* newElement) >{ > char* s = (char*) _ptrs.GetAt(nIndex); > if (s && *s) > free(s); > > _ptrs.SetAt(nIndex, newElement && *newElement? _strdup(newElement) : ""); >} Hi, to me the if (s && *s) looks suspicious: Imagine you have an empty string: char *p = ""; If you were to say char *q = strdup( p); I am rather certain, that *q == '\0' If this is true, the condition above will fail for any such string, 'free' wouldn't get called and you'd have a small leak. HTH Helmut Giese From jcw@equi4.com Wed Feb 11 13:18:40 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 79395B3A98 for ; Wed, 11 Feb 2004 13:18:40 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXeToC2g for ; Wed, 11 Feb 2004 13:18:38 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 95103B3A88 for ; Wed, 11 Feb 2004 13:18:38 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 3538C83C26; Wed, 11 Feb 2004 13:18:37 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 683AD83914 for ; Wed, 11 Feb 2004 13:18:34 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <5.2.1.1.0.20040211125844.00a6ae30@pop.1und1.com> References: <5.2.1.1.0.20040211125844.00a6ae30@pop.1und1.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <691608C6-5C8C-11D8-9096-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Re: Win CE problem Date: Wed, 11 Feb 2004 13:18:33 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Feb 2004 12:18:44 -0000 Helmut Giese wrote: >> void c4_StringArray::SetAt(int nIndex, const char* newElement) >> { >> char* s = (char*) _ptrs.GetAt(nIndex); >> if (s && *s) >> free(s); >> >> _ptrs.SetAt(nIndex, newElement && *newElement? _strdup(newElement) >> : ""); >> } > > Hi, > to me the > if (s && *s) > looks suspicious: > Imagine you have an empty string: char *p = ""; > If you were to say > char *q = strdup( p); > I am rather certain, that > *q == '\0' > > If this is true, the condition above will fail for any such string, > 'free' wouldn't get called and you'd have a small leak. Check the allocation in SetAt just below: if the string is an empty string, it does not go through strdup... Matt wrote: > I ran it through CodeSnitch (Entrek Software, Inc) which indicated > that the > app was riddled with small memory leaks. A majority of the reported > leaks > have a fairly long call stack, but all end up pointing to strdup in: I don't know what the memory leaks are, but note that you'd also see this if the entire StringArray never gets cleaned up. It just happens to consist entirely of strdups - so I'd look particularly for leaks of c4_StringArray allocations. -jcw From bkelley@wi.mit.edu Wed Feb 11 15:22:11 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C7614B3A94 for ; Wed, 11 Feb 2004 15:22:11 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXF4a8st for ; Wed, 11 Feb 2004 15:22:10 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id D6B93B3A88 for ; Wed, 11 Feb 2004 15:22:09 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004021109220502892 for ; Wed, 11 Feb 2004 09:22:05 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HSXBWT00.MUZ; Wed, 11 Feb 2004 09:22:05 -0500 Message-ID: <402A3B03.2040501@wi.mit.edu> Date: Wed, 11 Feb 2004 09:24:03 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Yarigin S , Metakit mailing list Subject: Re: [Metakit] New Python interface has unwanted "feature" References: <4027217A.2070601@protvino.systel.ru> <4027999A.1020708@wi.mit.edu> <4028A7CF.9040308@protvino.systel.ru> <40290410.1040501@wi.mit.edu> <4029F18F.6000107@protvino.systel.ru> In-Reply-To: <4029F18F.6000107@protvino.systel.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Feb 2004 14:22:12 -0000 I think you were getting lucky before. For locate to work correctly you need to have a view ordered on the key columns. In this case: vw=vw.ordered(3) will fix the code quite nicely. The downside is that the keys need to come first in the view definition from highest precedence to lowest. From my homegrown documentation: *search, locate- *You should probably never use these directly unless you are certain that the property you are searching is ordered. When using mapping views the fast binary searches will occur automatically. Brian From bkelley@wi.mit.edu Thu Feb 12 22:20:26 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C8167B3A8C; Thu, 12 Feb 2004 22:20:25 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXTGoyAk; Thu, 12 Feb 2004 22:20:24 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id E4577B3A88; Thu, 12 Feb 2004 22:20:22 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004021216201821520 ; Thu, 12 Feb 2004 16:20:18 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HSZPXU00.RAN; Thu, 12 Feb 2004 16:20:18 -0500 Message-ID: <402BEE87.40208@wi.mit.edu> Date: Thu, 12 Feb 2004 16:22:15 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list , support@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] MK4py Bizzare metakit behavior on ordered views X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Feb 2004 21:20:26 -0000 Ordered views are acting strangely. I'm using python with the latest metakit. view is as follows, two string columns A,B vw = st.getas("test[a:S,b:S]").ordered(1) vw.append(("JC4929", "a")) vw.append(("JC4929", "b")) yields (as expected) -> a b ------ - JC4929 a JC4929 b ------ - Total: 2 rows vw.append(("JC4929", "1")) obliterates the first row -> a b ------ - JC4929 1 JC4929 b ------ - Total: 2 rows What's worse is that it is order dependent: vw.append(("JC4929", "a")) and 'b' is obliterated. a b ------ - JC4929 1 JC4929 a ------ - Total: 2 rows Which just tells me that I don't know how the ordered column is supposed to work. It appears to except multiple key entries for the same orderd column and then checks the other columns for uniqueness, for example this would not add a row: vw.append(("JC4929", "a")) to the above column. One really bad case is: vw.append(("JC4929", '24227N')) vw.append(("JC4929", '498N')) a b ------ ------ JC4929 24227N JC4929 498N ------ ------ Total: 2 rows and now enter in the reverse order vw.append(("JC4929", '498N')) vw.append(("JC4929", '24227N')) a b ------ ------ JC4929 24227N ------ ------ Total: 1 rows attached is the sample code import metakit print "ordered columns with alphabetical strings" st = metakit.storage() vw = st.getas("test[a:B,b:B]").ordered(1) # two entries vw.append(("JC4929", "a")) vw.append(("JC4929", "b")) metakit.dump(vw) print "now add a string with a number at the front" print "this is not the same behavior :)" vw.append(("JC4929", "1")) metakit.dump(vw) vw.append(("JC4929", "a")) vw.append(("JC4929", "a")) metakit.dump(vw) print print "ordering is insertion order dependent" print "two pieces of data are added in different orders" st = metakit.storage() vw = st.getas("test[a,b]").ordered(1) print print "order 1, '22427N', '498N'" vw.append(("JC4929", '24227N')) vw.append(("JC4929", '498N')) metakit.dump(vw) st = metakit.storage() vw = st.getas("test[a,b]").ordered(1) print print "order 2 '498N', '22427N'" vw.append(("JC4929", '498N')) vw.append(("JC4929", '24227N')) metakit.dump(vw) From jcw@equi4.com Fri Feb 13 00:20:06 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 44D00B3A8C for ; Fri, 13 Feb 2004 00:20:06 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXra7Xbb for ; Fri, 13 Feb 2004 00:20:02 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 963FAB3A88 for ; Fri, 13 Feb 2004 00:20:01 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 1F6F383C26; Fri, 13 Feb 2004 00:19:59 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 6FF1083914 for ; Fri, 13 Feb 2004 00:19:57 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <402BEE87.40208@wi.mit.edu> References: <402BEE87.40208@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] MK4py Bizzare metakit behavior on ordered views Date: Fri, 13 Feb 2004 00:19:55 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Feb 2004 23:20:07 -0000 Brian Kelley wrote: > Ordered views are acting strangely. I'm using python with the latest > metakit. > > view is as follows, two string columns A,B > > vw = st.getas("test[a:S,b:S]").ordered(1) > > vw.append(("JC4929", "a")) > vw.append(("JC4929", "b")) > > yields (as expected) -> > > a b > ------ - > JC4929 a > JC4929 b > ------ - > Total: 2 rows > > vw.append(("JC4929", "1")) > > obliterates the first row -> > a b > ------ - > JC4929 1 > JC4929 b > ------ - > Total: 2 rows The ordered() operation cannot deal with duplicates. Unfortunately, it is not checking things properly, so the errors themselves are weird and wrong at times. I can see how it would make more sense to turn an add into a replace, and make the above behave like dictionaries, but for now it's not the case. Or am I'm missing the point - are you saying that ordered() worked better in 2.4.9.2? -jcw From frank@md-web.de Thu Feb 19 00:37:51 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 82BEFB3AE8 for ; Thu, 19 Feb 2004 00:37:51 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXwBTYY9 for ; Thu, 19 Feb 2004 00:37:50 +0100 (CET) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by dizzie.triqs.com (Postfix) with ESMTP id B65DFB3AE7 for ; Thu, 19 Feb 2004 00:37:49 +0100 (CET) Received: from [212.227.126.179] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 1AtbGH-0007bK-00 for metakit@equi4.com; Thu, 19 Feb 2004 00:37:49 +0100 Received: from [217.185.81.229] (helo=Paco) by mrelayng.kundenserver.de with asmtp (Exim 3.35 #1) id 1AtbGH-0005Qi-00 for metakit@equi4.com; Thu, 19 Feb 2004 00:37:49 +0100 Message-ID: <000d01c3f678$475d4870$e551b9d9@Paco> From: "Frank Meier-Doernberg" To: Date: Thu, 19 Feb 2004 00:38:09 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Provags-ID: kundenserver.de abuse@kundenserver.de auth:dcc3b1ef16f1c6f139ca9a99b649969c X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Fri, 20 Feb 2004 11:12:18 +0100 Subject: [Metakit] Why these nice store() and structure() methods gone ? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Frank Meier-Doernberg List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Feb 2004 23:37:52 -0000 A question in front of any own coding efforts: Why these nice store() and structure() methods gone ? Are they superseeded by something? What i'am missing here? Assume, I want to built a "wide" data-format-tree, representing many kinds of data-objects, so containing many different subviews in a moderate deep. And assume further, I want to do this separatly and independently from each other for each subview. What is the "MetaKit-Way" to do this type of "OO-Data" handling ? Compiling the "GetAs-Strings" together "by hand" seems to me less comfortable then doing this automagically by store() or structure(). Frank From jyl@mod3.net Mon Feb 23 16:04:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 07609B3AE9 for ; Mon, 23 Feb 2004 16:04:38 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXAC2KI3 for ; Mon, 23 Feb 2004 16:04:33 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id A16F2B3AE8 for ; Mon, 23 Feb 2004 16:04:32 +0100 (CET) Received: (qmail 5544 invoked by uid 48); 23 Feb 2004 15:04:30 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Mon, 23 Feb 2004 07:04:29 -0800 (PST) Message-ID: <65251.209.233.24.218.1077548669.squirrel@mod3.net> Date: Mon, 23 Feb 2004 07:04:29 -0800 (PST) From: "Jacob Levy" To: X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_20040223070429_83559" X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Problem building Metakit 2.4.9.3 on FreeBSD 4.9 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2004 15:04:38 -0000 ------=_20040223070429_83559 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Has anyone successfully built Metakit 2.4.9.3 on FreeBSD 4.9? If so, can you help me with this (probably) pilot error on my part? I'm attempting to convert e4graph over to metakit 2.4.9.3 and ran into some build problems. I'm probably doing something wrong.. Anyways I've tried to follow the instructions in README but it does not seem to build libmk4.so and I'm getting an error during install, see the attached log file. Thanks! --JYL jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ uname -a FreeBSD x86-freebsd1 4.9-PRERELEASE FreeBSD 4.9-PRERELEASE #23: Thu Sep 18 22:40:16 PDT 2003 root@usf-cf-x86-freebsd-1:/usr/obj/usr/src/sys/GENERIC i386 ------=_20040223070429_83559 Content-Type: text/plain; name="foo" Content-Disposition: attachment; filename="foo" Script started on Sun Feb 22 21:45:55 2004 bash: dircolors: command not found jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ ../unix/configue re --pre fix=`pwd`/../../install checking for Python configuration... not enabled checking for Tcl headers... not enabled checking for building with threads... no (default) checking for g++... g++ checking for C++ compiler default output... a.out checking whether the C++ compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking for a BSD-compatible install... /usr/bin/install -c checking build system type... i386-unknown-freebsd4.9 checking host system type... i386-unknown-freebsd4.9 checking for gcc... gcc checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ANSI C... none needed checking for ld used by GCC... /usr/libexec/elf/ld checking if the linker (/usr/libexec/elf/ld) is GNU ld... yes checking for /usr/libexec/elf/ld option to reload object files... -r checking for BSD-compatible nm... /usr/bin/nm -B checking for a sed that does not truncate output... /usr/bin/sed checking whether ln -s works... yes checking how to recognise dependent libraries... pass_all checking command to parse /usr/bin/nm -B output... ok checking how to run the C++ preprocessor... g++ -E checking for egrep... grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... no checking for unistd.h... yes checking dlfcn.h usability... yes checking dlfcn.h presence... yes checking for dlfcn.h... yes checking for ranlib... ranlib checking for strip... strip checking for objdir... .libs checking for gcc option to produce PIC... -fPIC checking if gcc PIC flag -fPIC works... yes checking if gcc static flag -static works... yes checking if gcc supports -c -o file.o... no checking if we can lock with hard links... yes checking if gcc supports -fno-rtti -fno-exceptions... yes checking whether the linker (/usr/libexec/elf/ld) supports shared libraries... yes checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking dynamic linker characteristics... freebsd4.9 ld.so checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking whether -lc should be explicitly linked in... yes creating libtool checking for ANSI C header files... (cached) yes checking for an ANSI C-conforming const... yes checking for inline... inline checking for long... yes checking size of long... 4 checking for long long... yes checking for mmap... yes checking for memmove... yes checking for bcopy... yes configure: creating ./config.status config.status: creating Makefile config.status: creating config.h config.status: config.h is unchanged jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ make install /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/column.cpp mkdir .libs g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/column.cpp -fPIC -DPIC mv -f column.o .libs/column.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/column.cpp >/dev/null 2>&1 mv -f .libs/column.lo column.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/custom.cpp rm -f .libs/custom.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/custom.cpp -fPIC -DPIC mv -f custom.o .libs/custom.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/custom.cpp >/dev/null 2>&1 mv -f .libs/custom.lo custom.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/derived.cpp rm -f .libs/derived.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/derived.cpp -fPIC -DPIC mv -f derived.o .libs/derived.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/derived.cpp >/dev/null 2>&1 mv -f .libs/derived.lo derived.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/fileio.cpp rm -f .libs/fileio.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/fileio.cpp -fPIC -DPIC mv -f fileio.o .libs/fileio.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/fileio.cpp >/dev/null 2>&1 mv -f .libs/fileio.lo fileio.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/field.cpp rm -f .libs/field.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/field.cpp -fPIC -DPIC mv -f field.o .libs/field.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/field.cpp >/dev/null 2>&1 mv -f .libs/field.lo field.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/format.cpp rm -f .libs/format.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/format.cpp -fPIC -DPIC mv -f format.o .libs/format.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/format.cpp >/dev/null 2>&1 mv -f .libs/format.lo format.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/handler.cpp rm -f .libs/handler.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/handler.cpp -fPIC -DPIC mv -f handler.o .libs/handler.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/handler.cpp >/dev/null 2>&1 mv -f .libs/handler.lo handler.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/persist.cpp rm -f .libs/persist.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/persist.cpp -fPIC -DPIC mv -f persist.o .libs/persist.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/persist.cpp >/dev/null 2>&1 mv -f .libs/persist.lo persist.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/remap.cpp rm -f .libs/remap.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/remap.cpp -fPIC -DPIC mv -f remap.o .libs/remap.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/remap.cpp >/dev/null 2>&1 mv -f .libs/remap.lo remap.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/std.cpp rm -f .libs/std.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/std.cpp -fPIC -DPIC mv -f std.o .libs/std.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/std.cpp >/dev/null 2>&1 mv -f .libs/std.lo std.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/store.cpp rm -f .libs/store.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/store.cpp -fPIC -DPIC mv -f store.o .libs/store.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/store.cpp >/dev/null 2>&1 mv -f .libs/store.lo store.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/string.cpp rm -f .libs/string.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/string.cpp -fPIC -DPIC mv -f string.o .libs/string.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/string.cpp >/dev/null 2>&1 mv -f .libs/string.lo string.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/table.cpp rm -f .libs/table.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/table.cpp -fPIC -DPIC mv -f table.o .libs/table.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/table.cpp >/dev/null 2>&1 mv -f .libs/table.lo table.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/univ.cpp rm -f .libs/univ.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/univ.cpp -fPIC -DPIC mv -f univ.o .libs/univ.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/univ.cpp >/dev/null 2>&1 mv -f .libs/univ.lo univ.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/view.cpp rm -f .libs/view.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/view.cpp -fPIC -DPIC mv -f view.o .libs/view.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/view.cpp >/dev/null 2>&1 mv -f .libs/view.lo view.lo /bin/sh ./libtool --mode=compile g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/viewx.cpp rm -f .libs/viewx.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/viewx.cpp -fPIC -DPIC mv -f viewx.o .libs/viewx.lo g++ -c -g -O2 -I../unix/../include -I../unix/../src -I. ../unix/../src/viewx.cpp >/dev/null 2>&1 mv -f .libs/viewx.lo viewx.lo /bin/sh ./libtool --mode=link g++ -o libmk4.la -g -O2 -I../unix/../include -I../unix/../src -I. -lstdc++ -avoid-version column.lo custom.lo derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo rm -fr .libs/libmk4.la .libs/libmk4.* .libs/libmk4.* ar cru .libs/libmk4.al column.lo custom.lo derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo ranlib .libs/libmk4.al creating libmk4.la (cd .libs && rm -f libmk4.la && ln -s ../libmk4.la libmk4.la) mkdir -p /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/include /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/lib /bin/sh ./libtool --mode=install /usr/bin/install -c -m 644 ../unix/../include/mk4.h ../unix/../include/mk4.inl ../unix/../include/mk4str.h ../unix/../include/mk4str.inl /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/include /usr/bin/install -c -m 644 ../unix/../include/mk4.h /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/include/mk4.h /usr/bin/install -c -m 644 ../unix/../include/mk4.inl /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/include/mk4.inl /usr/bin/install -c -m 644 ../unix/../include/mk4str.h /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/include/mk4str.h /usr/bin/install -c -m 644 ../unix/../include/mk4str.inl /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/include/mk4str.inl /bin/sh ./libtool --mode=install /usr/bin/install -c libmk4.la /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/lib /usr/bin/install -c .libs/libmk4.lai /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../install/lib/libmk4.la install: .libs/libmk4.lai: No such file or directory *** Error code 71 Stop in /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds. jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ exit exit Script done on Sun Feb 22 21:52:27 2004 ------=_20040223070429_83559-- From bkelley@wi.mit.edu Thu Feb 26 16:41:44 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1F6EAB3A8B for ; Thu, 26 Feb 2004 16:41:44 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXzyKG6T for ; Thu, 26 Feb 2004 16:41:42 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 3CE34B3A88 for ; Thu, 26 Feb 2004 16:41:42 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004022610413706534 for ; Thu, 26 Feb 2004 10:41:37 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HTP7LD00.RTM for ; Thu, 26 Feb 2004 10:41:37 -0500 Message-ID: <403E1415.5000609@wi.mit.edu> Date: Thu, 26 Feb 2004 10:43:17 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Saving views for recreation later X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 15:41:44 -0000 Suppose I create a view that is the result of many complicated operations. Is there anyway to save this view state so I don't have to repeat the operations in the future? If not, consider this a feature request :) It certainly sounds feasible since each generated view needs to have a mapping back to the original views. That would be really powerful. Currently all my views save their history so I can reconstruct them later, but the history is an operational history so the operations have to be reperformed. Brian From gary.h.merrill@gsk.com Thu Feb 26 16:52:50 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C7258B3A8B for ; Thu, 26 Feb 2004 16:52:49 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXWtRBml for ; Thu, 26 Feb 2004 16:52:39 +0100 (CET) Received: from usrtpnp1.glaxowellcome.com (firewall1.glaxowellcome.com [192.58.204.204]) by dizzie.triqs.com (Postfix) with ESMTP id 8A242B3A88 for ; Thu, 26 Feb 2004 16:52:38 +0100 (CET) Received: by usrtpnp1.glaxowellcome.com; id KAA13812; Thu, 26 Feb 2004 10:52:37 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma012245; Thu, 26 Feb 04 10:50:43 -0500 Received: by ussunt3.glaxo.com id KAA06529; Thu, 26 Feb 2004 10:50:41 -0500 (EST) Received: from 166.71.134.118 by us9n54.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Thu, 26 Feb 2004 10:50:38 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i1QFobB21235 for ; Thu, 26 Feb 2004 10:50:37 -0500 (EST) Subject: Re: [Metakit] Saving views for recreation later To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Thu, 26 Feb 2004 10:51:24 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 02/26/2004 10:50:41 AM MIME-Version: 1.0 X-WSS-ID: 6C20CA44402832-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 15:52:50 -0000 I strongly second Brian's request/suggestion!! -- I think. This would be an extraordinarily useful feature for using Metakit in a datamining environment where you want to keep a "trail" of results and refinements -- which is exactly what I'm involved in implementing. But if what you end up with is just a view, can't you simply save that now? I haven't yet got to the stage where I've actually tried to save the stages of an exploration like this. Or is Brian asking for some additional bookeeping to keep track of such views and make the process simpler? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Thu Feb 26 17:05:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A9347B3A8B for ; Thu, 26 Feb 2004 17:05:38 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXJw9dBb for ; Thu, 26 Feb 2004 17:05:37 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 8E171B3A88 for ; Thu, 26 Feb 2004 17:05:36 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004022611053206931 for ; Thu, 26 Feb 2004 11:05:32 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HTP8P800.4TU; Thu, 26 Feb 2004 11:05:32 -0500 Message-ID: <403E19B0.9010805@wi.mit.edu> Date: Thu, 26 Feb 2004 11:07:12 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com Subject: Re: [Metakit] Saving views for recreation later References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 16:05:39 -0000 gary.h.merrill@gsk.com wrote: >I strongly second Brian's request/suggestion!! -- I think. This would be >an extraordinarily useful feature for using Metakit in a datamining >environment where you want to keep a "trail" of results and refinements -- >which is exactly what I'm involved in implementing. > >But if what you end up with is just a view, can't you simply save that now? >I haven't yet got to the stage where I've actually tried to save the stages >of an exploration like this. > >Or is Brian asking for some additional bookeeping to keep track of such >views and make the process simpler? > I'm doing the bookkeeping myself now, each view contains the history of the operations that were performed on it. This includes joins and the like. vw.history() -> yields something like ['vw1 = st.view("table1")', 'vw2 = st.view("table2")', 'self = vw1.join(vw2, vw2.id)'] Which I can save so that the table I'm looking at could be regenerated. This turns out to be easy to implement using my wrapper classes. Actually, one thing that is missing in the python wrapper now that I think about it is the function that allows a view to be copied into a view with the same structure. I think this exists in C++. I wonder if it would be useful to have a clone function for a view such that: clone = vw.clone(name="temporary_table") where a new table "temporary_table" would be generated in the database and filled with visble rows/columns in vw. This would be a way of saving some form of state. Brian Brian Kelley From j.p.fletcher@aston.ac.uk Thu Feb 26 18:02:12 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2D49FB3A8B for ; Thu, 26 Feb 2004 18:02:12 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXsyXrcS for ; Thu, 26 Feb 2004 18:02:11 +0100 (CET) Received: from hermes.aston.ac.uk (hermes.aston.ac.uk [134.151.79.46]) by dizzie.triqs.com (Postfix) with ESMTP id 4DEE4B3A88 for ; Thu, 26 Feb 2004 18:02:11 +0100 (CET) Received: from [134.151.50.134] (helo=ceac134) by hermes.aston.ac.uk with esmtp (Exim 3.30 #1) id 1AwOtl-0006KP-00 for metakit@equi4.com; Thu, 26 Feb 2004 17:02:09 +0000 From: "John Fletcher" To: metakit@equi4.com Date: Thu, 26 Feb 2004 17:05:10 -0000 MIME-Version: 1.0 Message-ID: <403E2746.17681.19DA01E@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/html; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Metakit wiki X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 17:02:12 -0000
I cannot find on the metakit home page at http://www.equi4.com/metakit.html
any link to the metakit wiki at
http://www.equi4.com/metakit/wiki.cgi/0

I wondered if it was still there, and it is.  For some purposes the Email list is better, but for things which develop over a period of time the wiki can be a useful reference.

John Fletcher


-------------------------------------------------------------------
Dr John P. Fletcher          Tel: (44) 121 359 3611 ext 4625
Chemical Engineering and Applied Chemistry (CEAC),
School of Engineering and Applied Science (SEAS),
Aston University,            Fax: (44) 121 359 4094
Aston Triangle,              Email: J.P.Fletcher@aston.ac.uk
BIRMINGHAM B4 7ET  U.K.      CEAC Web site http://www.ceac.aston.ac.uk/

From j.p.fletcher@aston.ac.uk Thu Feb 26 18:12:00 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4F3EAB3A8B for ; Thu, 26 Feb 2004 18:12:00 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXIOCSG5 for ; Thu, 26 Feb 2004 18:11:59 +0100 (CET) Received: from hermes.aston.ac.uk (hermes.aston.ac.uk [134.151.79.46]) by dizzie.triqs.com (Postfix) with ESMTP id 7E6FEB3A88 for ; Thu, 26 Feb 2004 18:11:59 +0100 (CET) Received: from [134.151.50.134] (helo=ceac134) by hermes.aston.ac.uk with esmtp (Exim 3.30 #1) id 1AwP3F-0006fC-00 for metakit@equi4.com; Thu, 26 Feb 2004 17:11:57 +0000 From: "John Fletcher" To: metakit@equi4.com Date: Thu, 26 Feb 2004 17:14:59 -0000 MIME-Version: 1.0 Message-ID: <403E2993.26882.1A69B20@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/html; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] E4graph link X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 17:12:01 -0000
The e4graph link on page http://www.equi4.com/mklinks.html
should be changed to
http://e4graph.sourceforge.net/

John
-------------------------------------------------------------------
Dr John P. Fletcher          Tel: (44) 121 359 3611 ext 4625
Chemical Engineering and Applied Chemistry (CEAC),
School of Engineering and Applied Science (SEAS),
Aston University,            Fax: (44) 121 359 4094
Aston Triangle,              Email: J.P.Fletcher@aston.ac.uk
BIRMINGHAM B4 7ET  U.K.      CEAC Web site http://www.ceac.aston.ac.uk/

From 4mklist@myd1.com Thu Feb 26 20:03:24 2004 Return-Path: <4mklist@myd1.com> X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 5A910B3AEA for ; Thu, 26 Feb 2004 20:03:24 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXEjIuxU for ; Thu, 26 Feb 2004 20:03:22 +0100 (CET) Received: from bfish.hurrah.com (bfish.hurrah.com [208.151.247.36]) by dizzie.triqs.com (Postfix) with ESMTP id B12D4B3A88 for ; Thu, 26 Feb 2004 20:03:21 +0100 (CET) Received: from HP (c-24-21-43-223.client.comcast.net [24.21.43.223]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id i1QJ3Kv16122 for ; Thu, 26 Feb 2004 11:03:20 -0800 Date: Thu, 26 Feb 2004 11:03:16 -0800 From: Jerry <4mklist@myd1.com> X-Mailer: The Bat! (v2.02.3 CE) Personal X-Priority: 3 (Normal) Message-ID: <1132856872.20040226110316@myd1.com> To: Metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] data structure question X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 19:03:24 -0000 Hi, I have a data structure (that works well so far) with three similar sub-views that are accessed, set, and summarized at different points. Now I have a requirement to output a summary of all the detail with a label that identifies with of the three sets the data came from. The solution I came up with doesn't work, so I am thinking out loud to see if someone has an idea that involves the least amount of re-coding. The sub-views are accessed quite a bit in normal processing, and the summary only needs to be created once or twice a month. Simplized data format: >>> vw = db.getas('main[id:I,fname:S,lname:S,new[key:S,val:I],old[key:S,val:I],adj[key:S,val:I]') >>> jdb.dump(vw) id fname lname new old adjs ---- ------ ----- ------ ------ ------ 1 first1 last1 0 rows 0 rows 0 rows 2 bob last2 2 rows 0 rows 0 rows 4 first4 last4 2 rows 0 rows 4 rows Using a combination of flatten, union, and project I can get REAL CLOSE to what I want: >>> jdb.dump(vw31) id key val ---- ----- --- 2 val10 10 2 val20 20 4 val10 10 4 val10 10 4 val20 20 4 val20 20 4 val30 30 4 val40 40 What I need is to know WHICH type of value each row is: id key val type ---- ----- --- ---- 2 val10 10 new 2 val20 20 new 4 val10 10 new 4 val10 10 adj 4 val20 20 new 4 val20 20 adj 4 val30 30 adj 4 val40 40 adj The main problem being that one cannot successfully add a property to a PyROViewer object, which is the result of the union and flatten methods. It seems I either have to: 1) create a separate temporary view for each type and manually copy the flattened view into it, creating and setting 'type' appropriately. Then union off of these PyView objects. 2) or modify my system to always write the subview type into the subview. This means extra programming and run-time overhead, and extra strings in tens of thousands of records. 3) or some other creative idea you suggest. -- Thanks, Jerry McRae MetaKit 2.4.9.2, Python 2.3.2, WinXP, 796M RAM ___________________________ Conform and be dull --J. Frank Dobie From gary.h.merrill@gsk.com Thu Feb 26 21:01:08 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 5B6EBB3AF5 for ; Thu, 26 Feb 2004 21:01:08 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXLmnHcT for ; Thu, 26 Feb 2004 21:00:57 +0100 (CET) Received: from usrtpnp2.glaxowellcome.com (firewall2.glaxowellcome.com [192.58.204.209]) by dizzie.triqs.com (Postfix) with ESMTP id 18730B3A88 for ; Thu, 26 Feb 2004 21:00:57 +0100 (CET) Received: by usrtpnp2.glaxowellcome.com; id PAA25847; Thu, 26 Feb 2004 15:00:44 -0500 (EST) From: Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma025384; Thu, 26 Feb 04 14:59:40 -0500 Received: by ussunt2.glaxo.com id OAA02976; Thu, 26 Feb 2004 14:59:39 -0500 (EST) Received: from 166.71.134.118 by us9n57.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Thu, 26 Feb 2004 14:59:33 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i1QJxWB18841 for ; Thu, 26 Feb 2004 14:59:32 -0500 (EST) Subject: Re: [Metakit] Saving views for recreation later To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Thu, 26 Feb 2004 15:00:21 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 02/26/2004 02:59:36 PM MIME-Version: 1.0 X-WSS-ID: 6C208FAF452807-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 20:01:08 -0000 Yes, this is the sort of thing I had in mind. We seem to be thinking along the same lines in terms of using Metakit to support knowledge exploration. I have this concept of a "knowledge bank" that is closed under "knowledge operations", and a "knowledge pathway" that the user can navigate and modify. In terms of efficiency, I'm not sure whether it is better to actually keep the views or keep the operations (like an edit history). This depends partly, of course, on the expense of each; and perhaps the best approach is to support both. But I do believe that Metakit is a powerful tool to use in such a context. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 "Brian Kelley" To: gary.h.merrill@gsk.com 26-Feb-2004 11:07 cc: metakit@equi4.com Subject: Re: [Metakit] Saving views for recreation later gary.h.merrill@gsk.com wrote: >I strongly second Brian's request/suggestion!! -- I think. This would be >an extraordinarily useful feature for using Metakit in a datamining >environment where you want to keep a "trail" of results and refinements -- >which is exactly what I'm involved in implementing. > >But if what you end up with is just a view, can't you simply save that now? >I haven't yet got to the stage where I've actually tried to save the stages >of an exploration like this. > >Or is Brian asking for some additional bookeeping to keep track of such >views and make the process simpler? > I'm doing the bookkeeping myself now, each view contains the history of the operations that were performed on it. This includes joins and the like. vw.history() -> yields something like ['vw1 = st.view("table1")', 'vw2 = st.view("table2")', 'self = vw1.join(vw2, vw2.id)'] Which I can save so that the table I'm looking at could be regenerated. This turns out to be easy to implement using my wrapper classes. Actually, one thing that is missing in the python wrapper now that I think about it is the function that allows a view to be copied into a view with the same structure. I think this exists in C++. I wonder if it would be useful to have a clone function for a view such that: clone = vw.clone(name="temporary_table") where a new table "temporary_table" would be generated in the database and filled with visble rows/columns in vw. This would be a way of saving some form of state. Brian Brian Kelley From gmcm@hypernet.com Thu Feb 26 21:59:45 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D3C46B3A8B for ; Thu, 26 Feb 2004 21:59:44 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XX3kmwcG for ; Thu, 26 Feb 2004 21:59:43 +0100 (CET) Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by dizzie.triqs.com (Postfix) with ESMTP id 23039B3A88 for ; Thu, 26 Feb 2004 21:59:42 +0100 (CET) Received: from PARANOIA (204.176.40.236) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Thu, 26 Feb 2004 15:23:05 -0500 From: "Gordon McMillan" To: Jerry <4mklist@myd1.com>, Metakit Date: Thu, 26 Feb 2004 15:18:21 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] data structure question Message-ID: <403E0E3D.3734.1FEA7E7E@localhost> Priority: normal In-reply-to: <1132856872.20040226110316@myd1.com> X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gmcm@hypernet.com List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 20:59:45 -0000 On 26 Feb 2004 at 11:03, Jerry wrote: [...] > 1) create a separate temporary view for each type and manually copy > the flattened view into it, creating and setting 'type' > appropriately. Then union off of these PyView objects. > 2) or modify my system to always write the subview type into the > subview. This means extra programming and run-time overhead, and > extra strings in tens of thousands of records. > 3) or some other creative idea you suggest. I don't really grasp what you're trying to do, but 2 things come to mind: 1) view.rename("value", "newvalue") might help 2) tmp = vw[:] will turn a ROViewer into a (in memory) View, so you could add a column. I wouldn't worry too much about processing time - what you're doing is already pretty intensive. -- Gordon From gmcm@hypernet.com Thu Feb 26 21:59:49 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 96A46B3AF5 for ; Thu, 26 Feb 2004 21:59:49 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXFgCCbf for ; Thu, 26 Feb 2004 21:59:48 +0100 (CET) Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by dizzie.triqs.com (Postfix) with ESMTP id C1411B3A88 for ; Thu, 26 Feb 2004 21:59:45 +0100 (CET) Received: from PARANOIA (204.176.40.236) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.1.3); Thu, 26 Feb 2004 15:23:13 -0500 From: "Gordon McMillan" To: "Brian Kelley" Date: Thu, 26 Feb 2004 15:18:21 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Saving views for recreation later Message-ID: <403E0E3D.26477.1FEA7E10@localhost> Priority: normal In-reply-to: <403E19B0.9010805@wi.mit.edu> References: X-mailer: Pegasus Mail for Windows (v4.12a) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gmcm@hypernet.com List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 20:59:50 -0000 On 26 Feb 2004 at 11:07, Brian Kelley wrote: > Actually, one thing that is missing in the python wrapper now > that I think about it is the function that allows a view to be > copied into a view with the same structure. I think this > exists in C++. I wonder if it would be useful to have a > clone function for a view such that: > > clone = vw.clone(name="temporary_table") clone = vw[:] -- Gordon http://www.mcmillan-inc.com/ From bkelley@wi.mit.edu Thu Feb 26 22:24:23 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 8D5B3B3A8B for ; Thu, 26 Feb 2004 22:24:23 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXJqpkUh for ; Thu, 26 Feb 2004 22:24:22 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id AEFDEB3A88 for ; Thu, 26 Feb 2004 22:24:21 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004022616241711997 for ; Thu, 26 Feb 2004 16:24:17 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HTPNGH00.0X4; Thu, 26 Feb 2004 16:24:17 -0500 Message-ID: <403E6465.4000908@wi.mit.edu> Date: Thu, 26 Feb 2004 16:25:57 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gmcm@hypernet.com Subject: Re: [Metakit] Saving views for recreation later References: <403E0E3D.26477.1FEA7E10@localhost> In-Reply-To: <403E0E3D.26477.1FEA7E10@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 21:24:24 -0000 Gordon McMillan wrote: >On 26 Feb 2004 at 11:07, Brian Kelley wrote: > > > >>Actually, one thing that is missing in the python wrapper now >>that I think about it is the function that allows a view to be >>copied into a view with the same structure. I think this >>exists in C++. I wonder if it would be useful to have a >>clone function for a view such that: >> >>clone = vw.clone(name="temporary_table") >> >> > >clone = vw[:] > This doesn't save the data in the database though. I was thinking more about cloning into new permenant storage. My wording was not very clear though. There is also: clone = vw.copy() Brian From bkelley@wi.mit.edu Thu Feb 26 22:39:39 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0B6A5B3A8B for ; Thu, 26 Feb 2004 22:39:39 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXJQ1cLV for ; Thu, 26 Feb 2004 22:39:37 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 1B4D8B3A88 for ; Thu, 26 Feb 2004 22:39:37 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004022616393212206 for ; Thu, 26 Feb 2004 16:39:32 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HTPO5W00.VXX for ; Thu, 26 Feb 2004 16:39:32 -0500 Message-ID: <403E67F8.6000409@wi.mit.edu> Date: Thu, 26 Feb 2004 16:41:12 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Subject: Re: [Metakit] data structure question References: <403E0E3D.3734.1FEA7E7E@localhost> In-Reply-To: <403E0E3D.3734.1FEA7E7E@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 21:39:39 -0000 Gordon McMillan wrote: >I don't really grasp what you're trying to do, but 2 things come >to mind: >1) view.rename("value", "newvalue") might help >2) tmp = vw[:] will turn a ROViewer into a (in memory) View, so you >could add a column. > > Hmm, on metakit 2.4.9.3 this returns a Read only view. vw.copy() returns an in-memory view. As Gordon was saying, you could try this. prop = metakit.property("S", "type") t = vw.flatten(vw.new).copy() t.addproperty(prop) for row in t: row.type = "new" t2 = vw.flatten(vw.old).copy() t2.addproperty(prop) for row in t2: row.type = "old" metakit.dump(t.union(t2).project(t.id, t.key, t.val, t.type)) >I wouldn't worry too much about processing time - what you're >doing is already pretty intensive. > >-- Gordon > >_____________________________________________ >Metakit mailing list - Metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > From 4mklist@myd1.com Thu Feb 26 23:54:56 2004 Return-Path: <4mklist@myd1.com> X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1B26DB3A8B for ; Thu, 26 Feb 2004 23:54:56 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXXzGMMG for ; Thu, 26 Feb 2004 23:54:54 +0100 (CET) Received: from bfish.hurrah.com (bfish.hurrah.com [208.151.247.36]) by dizzie.triqs.com (Postfix) with ESMTP id 9D419B3A88 for ; Thu, 26 Feb 2004 23:54:53 +0100 (CET) Received: from HP (c-24-21-43-223.client.comcast.net [24.21.43.223]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id i1QMsrv02049 for ; Thu, 26 Feb 2004 14:54:53 -0800 Date: Thu, 26 Feb 2004 14:54:47 -0800 From: Jerry McRae <4mklist@myd1.com> X-Mailer: The Bat! (v2.02.3 CE) Personal X-Priority: 3 (Normal) Message-ID: <15510529385.20040226145447@myd1.com> To: Metakit Subject: Re: [Metakit] data structure question In-Reply-To: <403E0E3D.3734.1FEA7E7E@localhost> References: <1132856872.20040226110316@myd1.com> <403E0E3D.3734.1FEA7E7E@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2004 22:54:56 -0000 Hi Gordon, You wrote on Thursday (2/26/2004 at 12:18) thus about [Metakit] data structure question: >> 1) create a separate temporary view for each type and manually copy >> the flattened view into it, creating and setting 'type' >> appropriately. Then union off of these PyView objects. >> 2) or modify my system to always write the subview type into the >> subview. This means extra programming and run-time overhead, and >> extra strings in tens of thousands of records. >> 3) or some other creative idea you suggest. GM> I don't really grasp what you're trying to do, but 2 things come GM> to mind: GM> 1) view.rename("value", "newvalue") might help GM> 2) tmp = vw[:] will turn a ROViewer into a (in memory) View, so you GM> could add a column. GM> I wouldn't worry too much about processing time - what you're GM> doing is already pretty intensive. GM> -- Gordon Thanks for the reply. Sorry my explanation wasn't clear. Your #2 looked promising, but the resulting 'tmp' is still PyROViewer on my system (MK 2.4.9.2). So it looks like I am going to have to read the properties from my ROViewer, write a routine to add those properties onto a new view on a temporary storage, then copy my rows over each time I flatten a sub-view. As to #1, I really don't want to rename the rows. I want all the values in columns "key" and "val" from three sub-views to be in "key" and "val" in the new flattened view. I just want to know which sub-view they came from! -- Jerry (Mailed safely with The Bat! 2.02.3 CE) ___________________________ Let the people know the truth and the country is safe. --Abraham Lincoln From rajeev@accellion.com Wed Feb 25 12:41:27 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 51FF6B3AEA for ; Wed, 25 Feb 2004 12:41:27 +0100 (CET) Received: from by localhost (amavisd-new, port ) id XXojZG18 for ; Wed, 25 Feb 2004 12:41:25 +0100 (CET) Received: from mail01e.rapidsite.net (mail01e.rapidsite.net [208.55.43.102]) by dizzie.triqs.com (Postfix) with SMTP id 34A9CB3A93 for ; Wed, 25 Feb 2004 12:41:25 +0100 (CET) Received: from www.nyeq.com (131.103.204.244) by mail01e.rapidsite.net (RS ver 1.0.91vs) with SMTP id 4-0841999274 for ; Wed, 25 Feb 2004 06:41:19 -0500 (EST) Message-Id: <5.1.0.14.0.20040225171000.01c2e7f0@131.103.204.244> X-Sender: rajeev@131.103.204.244 X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Wed, 25 Feb 2004 17:10:46 +0530 To: metakit@equi4.com From: Rajeev Gupta Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Loop-Detect: 1 X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Fri, 27 Feb 2004 00:13:42 +0100 Subject: [Metakit] problems with hash view in 2.4.9.3 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2004 11:41:27 -0000 Hi, I have upgraded to 2.4.9.3 from 2.4.8 couple of days back. I am facing some problems with hash views now. Earlier if i tried to append a row with an existing 'key', it used to overwirte the old row; not any more. Below is a session snippet from python command line showing this: Could somebody point out what i am doing wrong. Thanks Rajeev s=metakit.storage(r'd:\del\5',1) >>> v=s.getas('DLE_URLS [url_hash:I,email_sender:S,ufi:S,url_flags:L,request_file_name:S,last_access_tim e:F,url:S,email_recipients:S,email_timestamp:F,subject:S]') >>> hv=s.getas('hv[_H:I,_R:I]') >>> v=v.hash(hv,1) >>> new_vals={'url_hash':1, 'url_flags':0, 'url':'adfsddddsdfdfsdfsdff', 'ufi':'fdfdsfdsfgfddfdfdfffddyddyy', 'email_timestamp':313, 'request_file_name':'abc123' } >>> v.append(new_vals) 0 >>> v.append(new_vals) 1 >>> v.append(new_vals) 2 >>> v.append(new_vals) 3 >>> len(v) 4 From gary.h.merrill@gsk.com Tue Mar 2 20:29:51 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6D387B3AE7 for ; Tue, 2 Mar 2004 20:29:51 +0100 (CET) Received: from usrtpnp2.glaxowellcome.com (firewall2.glaxowellcome.com [192.58.204.209]) by dizzie.triqs.com (Postfix) with ESMTP id 25D1EB3AE6 for ; Tue, 2 Mar 2004 20:29:38 +0100 (CET) Received: by usrtpnp2.glaxowellcome.com; id OAA01750; Tue, 2 Mar 2004 14:29:31 -0500 (EST) From: Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma001559; Tue, 2 Mar 04 14:28:49 -0500 Received: by ussunt2.glaxo.com id OAA03305; Tue, 2 Mar 2004 14:28:47 -0500 (EST) Received: from 166.71.134.118 by us9n53.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Tue, 02 Mar 2004 14:28:42 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i22JSgB14859 for ; Tue, 2 Mar 2004 14:28:42 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Tue, 2 Mar 2004 14:29:30 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 03/02/2004 02:28:46 PM MIME-Version: 1.0 X-WSS-ID: 6C5A3FE0880136-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Sharing Mk databases X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Mar 2004 19:29:51 -0000 I'm looking ahead to a time when I'll extend my application to be multi-user. This would mean multiple users accessing and updating the databases. Will Metakit support this? Or are the plans to support it in the future? If not, what is a reasonable workaround? Write a server that will handle all interactions with Metakit databases? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From gary.h.merrill@gsk.com Tue Mar 2 21:56:50 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 423D2B3AE6 for ; Tue, 2 Mar 2004 21:56:50 +0100 (CET) Received: from usrtpnp2.glaxowellcome.com (firewall2.glaxowellcome.com [192.58.204.209]) by dizzie.triqs.com (Postfix) with ESMTP id EC765B3A88 for ; Tue, 2 Mar 2004 21:56:38 +0100 (CET) Received: by usrtpnp2.glaxowellcome.com; id PAA11161; Tue, 2 Mar 2004 15:56:37 -0500 (EST) From: Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma010897; Tue, 2 Mar 04 15:56:19 -0500 Received: by ussunt2.glaxo.com id PAA29800; Tue, 2 Mar 2004 15:56:17 -0500 (EST) Received: from 166.71.134.118 by us9n55.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Tue, 02 Mar 2004 15:56:10 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i22Ku8B05333 for ; Tue, 2 Mar 2004 15:56:08 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Tue, 2 Mar 2004 15:56:56 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 03/02/2004 03:56:12 PM MIME-Version: 1.0 X-WSS-ID: 6C5A2B63884273-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] view() vs. getas() in Python X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Mar 2004 20:56:50 -0000 Can someone enlighten me about this distinction -- and in general the utility of view()? Supposedly view() is "The normal way to retrieve an existing view." But apparently even if the view doesn't exist, something gets retrieved -- though it isn't of much use. However, it does appear that you can append to this non-existent view since v = db.view('NonExistentView') v.append(foo=1) *appears* to work. But then, of course any attempt at referencing v[0].foo fails (though referencing v[0] does not). Shouldn't appending to a non-existent view raise an exception? If you look at db.properties() there again *appears* to be a view called 'NonExistentView' there. How can I tell that I've "retrieved" a non-existent view as opposed to, say, a merely empty one? If I try to use description() on a non-existent view I get a really ugly Python internal error. Given this, what is the point of view()? (No pun intended.) I guess that part of the utility would be that if I use view(), I need only the view name and not the description. And that if I do manage to "pseudo-retrieve" a non-existent view, then at some point something will go wrong in a fairly obvious (not to say dramatic) way. Whereas, if I make a mistake with getas(), a new view will get created and there may be other consequences to that. But why not raise an exception if a bogus view name is given to view()? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From david@rebirthing.co.nz Wed Mar 3 01:57:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 50453B3AE6 for ; Wed, 3 Mar 2004 01:57:38 +0100 (CET) Received: from dbmail-mx6.orcon.co.nz (unknown [219.88.242.4]) by dizzie.triqs.com (Postfix) with ESMTP id 64523B3A88 for ; Wed, 3 Mar 2004 01:57:35 +0100 (CET) Received: from rebirthing.co.nz (port-203-99-27-169.jet.net.nz [203.99.27.169])ESMTP id i230vCjA002045 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 3 Mar 2004 13:57:41 +1300 Message-ID: <40452D5B.9000102@rebirthing.co.nz> Date: Wed, 03 Mar 2004 13:56:59 +1300 From: David McNab User-Agent: Mozilla Thunderbird 0.5 (X11/20040208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com X-Enigmail-Version: 0.83.3.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.38 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Metakit auto** problem X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 00:57:38 -0000 Hi, I notice the build for 2.4.9.3 has been well-fixed with regard to the python extension. However, I have to question the hardwiring of 2.3 as the python version (requiring me to copy and hack the configure script for legacy pythons). At present, I'm working to enable building the python extension for legacy pythons going back to 1.5.2. Why? Because I'm building a web framework suitable for use on budget public vhosts with poor python support. Many of these budget hosts keep their PHP right up to date, but only have Python 1.5.2. -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From david@rebirthing.co.nz Wed Mar 3 02:36:51 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 169B9B3AE6 for ; Wed, 3 Mar 2004 02:36:51 +0100 (CET) Received: from dbmail-mx3.orcon.co.nz (unknown [219.88.242.4]) by dizzie.triqs.com (Postfix) with ESMTP id 18CD4B3A88 for ; Wed, 3 Mar 2004 02:36:48 +0100 (CET) Received: from rebirthing.co.nz (port-203-99-27-169.jet.net.nz [203.99.27.169])ESMTP id i231YWwS020212; Wed, 3 Mar 2004 14:34:33 +1300 Message-ID: <404536A9.2040308@rebirthing.co.nz> Date: Wed, 03 Mar 2004 14:36:41 +1300 From: David McNab User-Agent: Mozilla Thunderbird 0.5 (X11/20040208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com Subject: Re: [Metakit] Sharing Mk databases References: In-Reply-To: X-Enigmail-Version: 0.83.3.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.38 X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 01:36:51 -0000 Something I'm doing which works, albeit not exactly ideal in performance, is wrapping all the user-exposed classes, and implementing transparent file-locking. -- Kind regards David -- leave this line intact so your email gets through my junk mail filter gary.h.merrill@gsk.com wrote: > I'm looking ahead to a time when I'll extend my application to be > multi-user. This would mean multiple users accessing and updating the > databases. Will Metakit support this? Or are the plans to support it in > the future? > > If not, what is a reasonable workaround? Write a server that will handle > all interactions with Metakit databases? > > -------------------------------------- > Gary H. Merrill > Director and Principal Scientist, New Applications > Data Exploration Sciences > GlaxoSmithKline Inc. > (919) 483-8456 > > > > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From david@rebirthing.co.nz Wed Mar 3 03:21:18 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B0B5EB3AE6 for ; Wed, 3 Mar 2004 03:21:18 +0100 (CET) Received: from dbmail-mx1.orcon.co.nz (unknown [219.88.242.4]) by dizzie.triqs.com (Postfix) with ESMTP id B35F7B3A88 for ; Wed, 3 Mar 2004 03:21:15 +0100 (CET) Received: from rebirthing.co.nz (port-203-99-27-169.jet.net.nz [203.99.27.169])i232Jpie032111 for ; Wed, 3 Mar 2004 15:19:52 +1300 Message-ID: <40454115.70603@rebirthing.co.nz> Date: Wed, 03 Mar 2004 15:21:09 +1300 From: David McNab User-Agent: Mozilla Thunderbird 0.5 (X11/20040208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com X-Enigmail-Version: 0.83.3.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.38 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Instructions for Legacy Pythons X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 02:21:19 -0000 Hi, Below are instructions for people needing to build the python metakit wrapper for non-latest python versions. With the web app framework I'm building, I was using SQLObject for database storage. But given the fact that most public web hosts offering Python have version 2.1 or earlier, I've decided life is easiest for all if my framework can be made to work with python1.5.2 or later. These simple instructions walk people through the building of the Mk4py.so module for Python 1.5.2 or later, and can be used to help people compile a binary Mk4py.so locally, and drop it (with metakit.py) into their web hosting file tree. The need for this comes from the fact that, while most reasonable public web hosts support python, most of these have python1.5.2 or python2.1, and no Metakit. I hope the metakit devs might see fit to include these instructions in Metakit releases. Cheers David --------------------------- INSTRUCTIONS FOR BUILDING AND USING METAKIT ON EARLY PYTHON VERSIONS 1. PREPARATION cd into top-level Metakit directory, then: $ mkdir buildsx.x (here and below we use 'x.x' to mean the python version you want to build for, eg '1.5') 2. PRE-CONFIGURATION cd into the 'unix' directory, then: $ cp configure configurex.x Then, edit configurex.x, search for the 'case "$with_python in' line. Look down a bit, and change the PY_INCLUDE_DIR and PY_LIB_DIR lines, to reference the version of python you're building for 3. CONFIGURE AND BUILD cd into the buildsx.x directory, and type: $ ../pythonx.x/configurex.x --with-python=/usr (If your python doesn't live in /usr, change to /usr/local or wherever) $ make 4. INSTALLATION While still in your buildsx.x directory, type: $ cp Mk4py.so /usr/lib/pythonx.x/site-packages (you may need /usr/local/lib instead /usr/lib) Then, cd into the python directory: $ cd ../python Note that you can build Metakit Mk4py.so for Python 1.5.2, and use it quite happily in all later python versions. However, you will probably want to suppress the API warning. To do this, edit the metakit.py file, and add the following lines right after the docstring: import warnings warnings.filterwarnings('ignore', '.*Mk4py.*', RuntimeWarning) Then, just copy metakit.py into your python load path: $ cp metakit.py /usr/lib/pythonx.x/site-packages (change /usr/lib to /usr/local/lib or whatever if needed) 5. TESTING Start up your chosen python interpreter: $ pythonx.x And verify that you can load Metakit: >>> from metakit import * From njriley@shrug.csl.uiuc.edu Wed Mar 3 04:43:39 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id EF9A3B3AE6 for ; Wed, 3 Mar 2004 04:43:38 +0100 (CET) Received: from shrug.csl.uiuc.edu (shrug.csl.uiuc.edu [130.126.137.135]) by dizzie.triqs.com (Postfix) with ESMTP id 3D1E0B3A88 for ; Wed, 3 Mar 2004 04:43:37 +0100 (CET) Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.10/8.12.9) with ESMTP id i233hZAO071912; Tue, 2 Mar 2004 21:43:35 -0600 (CST) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.10/8.12.9/Submit) id i233hYNA071911; Tue, 2 Mar 2004 21:43:34 -0600 (CST) (envelope-from njriley) Date: Tue, 2 Mar 2004 21:43:34 -0600 From: Nicholas Riley To: David McNab Subject: Re: [Metakit] Metakit auto** problem Message-ID: <20040303034334.GA69285@uiuc.edu> Mail-Followup-To: David McNab , metakit@equi4.com References: <40452D5B.9000102@rebirthing.co.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <40452D5B.9000102@rebirthing.co.nz> User-Agent: Mutt/1.4.1i X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 03:43:39 -0000 On Wed, Mar 03, 2004 at 01:56:59PM +1300, David McNab wrote: > Hi, > > I notice the build for 2.4.9.3 has been well-fixed with regard to the > python extension. > > However, I have to question the hardwiring of 2.3 as the python version > (requiring me to copy and hack the configure script for legacy pythons). > > At present, I'm working to enable building the python extension for > legacy pythons going back to 1.5.2. > > Why? > > Because I'm building a web framework suitable for use on budget public > vhosts with poor python support. Many of these budget hosts keep their > PHP right up to date, but only have Python 1.5.2. Try using the distutils build method instead of ./configure --enable-python - it'll work back to 1.5.2 if you use the latest distutils (which is also guaranteed to work back to 1.5.2). -- =Nicholas Riley | From david@rebirthing.co.nz Wed Mar 3 05:15:37 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id EC363B3AE6 for ; Wed, 3 Mar 2004 05:15:36 +0100 (CET) Received: from dbmail-mx5.orcon.co.nz (unknown [219.88.242.4]) by dizzie.triqs.com (Postfix) with ESMTP id 0460CB3A88 for ; Wed, 3 Mar 2004 05:15:34 +0100 (CET) Received: from rebirthing.co.nz (port-203-99-27-169.jet.net.nz [203.99.27.169])ESMTP id i234Fgav023683; Wed, 3 Mar 2004 17:15:42 +1300 Message-ID: <40455BDF.3050302@rebirthing.co.nz> Date: Wed, 03 Mar 2004 17:15:27 +1300 From: David McNab User-Agent: Mozilla Thunderbird 0.5 (X11/20040208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Nicholas Riley Subject: Re: [Metakit] Metakit auto** problem References: <40452D5B.9000102@rebirthing.co.nz> <20040303034334.GA69285@uiuc.edu> In-Reply-To: <20040303034334.GA69285@uiuc.edu> X-Enigmail-Version: 0.83.3.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.38 X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 04:15:37 -0000 Nicholas Riley wrote: > Try using the distutils build method instead of ./configure > --enable-python - it'll work back to 1.5.2 if you use the latest > distutils (which is also guaranteed to work back to 1.5.2). Tried that. With metakit's setup.py, distutils doesn't work for pythons earlier than 2.3. -- Kind regards David -- leave this line intact so your email gets through my junk mail filter From jcw@equi4.com Wed Mar 3 13:21:54 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B2ABAB3AF5 for ; Wed, 3 Mar 2004 13:21:50 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id A5DBAB3A88 for ; Wed, 3 Mar 2004 13:21:46 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 3241883BE0; Wed, 3 Mar 2004 13:21:36 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id C130683BE0 for ; Wed, 3 Mar 2004 13:21:11 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <65251.209.233.24.218.1077548669.squirrel@mod3.net> References: <65251.209.233.24.218.1077548669.squirrel@mod3.net> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <42A4D4C2-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Problem building Metakit 2.4.9.3 on FreeBSD 4.9 Date: Wed, 3 Mar 2004 13:21:12 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:21:55 -0000 Jacob Levy wrote: > Has anyone successfully built Metakit 2.4.9.3 on FreeBSD 4.9? If so, > can > you help me with this (probably) pilot error on my part? [...] > /usr/bin/install -c .libs/libmk4.lai > /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../ > install/lib/libmk4.la > install: .libs/libmk4.lai: No such file or directory > *** Error code 71 > > Stop in /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds. Looks like libtool/autoconf madness. I suggest: make distclean cd ../unix aclocal libtoolize -f -c autoconf cd ../builds ../unix/configure make make install I'm sorely tempted to throw out everything related to autoconf, libtool, and even make. Older versions of Metakit used to have the following build instructions: gcc -O2 -shared -I../include ../src/*.cpp -o libmk4.so And I sure miss those times. Things use to be *simple* (does anyone remember that concept?). -jcw From jcw@equi4.com Wed Mar 3 13:21:55 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A28EFB3A88 for ; Wed, 3 Mar 2004 13:21:55 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id DD8EFB3AE7 for ; Wed, 3 Mar 2004 13:21:49 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id D142A83EF3; Wed, 3 Mar 2004 13:21:46 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 82FD383C27 for ; Wed, 3 Mar 2004 13:21:15 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <403E2746.17681.19DA01E@localhost> References: <403E2746.17681.19DA01E@localhost> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Message-Id: <44D5C21C-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: quoted-printable From: Jean-Claude Wippler Subject: Re: [Metakit] Metakit wiki Date: Wed, 3 Mar 2004 13:21:16 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:21:57 -0000 John Fletcher wrote: > I cannot find on the metakit home page at=20 > http://www.equi4.com/metakit.html > any link to the metakit wiki at > http://www.equi4.com/metakit/wiki.cgi/0 It's still there, at http://www.equi4.com/mkmailing.html > I wondered if it was still there, and it is.=A0 For some purposes the=20= > Email list is better, but for things which develop over a period of=20 > time the wiki can be a useful reference. I've been switching to ProjectForum as wiki for some other projects,=20 such as http://www.equi4.com/forum/rawiki/Home It offers more protection/authentification options, RSS feeds, CSS=20 themes, file attachments, and a lot more (and yes, it's all powered by=20= Metakit). Courtesy of Mark Roseman. Have been pondering for quite a while whether it would be feasible to=20 migrate the MK wiki pages to PF, and revitalize things a bit. I agree=20= that there is value in having information categorized, not just stored=20= in a timeline, as it is now: http://news.gmane.org/gmane.comp.db.metakit/ http://www.equi4.com/pipermail/metakit/ If there is sufficient interest to help fill a new area, I'd be happy=20 to set it up and participate in getting (some) old info transferred and=20= a setting up a good structure, perhaps with areas per language. -jcw From jcw@equi4.com Wed Mar 3 13:21:58 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id F2367B3AE7 for ; Wed, 3 Mar 2004 13:21:57 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 0FEF6B3AF3 for ; Wed, 3 Mar 2004 13:21:50 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 05B5283914; Wed, 3 Mar 2004 13:21:46 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 6938183914 for ; Wed, 3 Mar 2004 13:21:10 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <000d01c3f678$475d4870$e551b9d9@Paco> References: <000d01c3f678$475d4870$e551b9d9@Paco> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <40467C35-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Why these nice store() and structure() methods gone ? Date: Wed, 3 Mar 2004 13:21:08 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:21:59 -0000 Frank Meier-Doernberg wrote: > A question in front of any own coding efforts: > Why these nice store() and structure() methods gone ? > Are they superseeded by something? What i'am missing here? The Store() method was too buggy, and with no good way to fix it in sight, I had to drop it. > Compiling the "GetAs-Strings" together "by hand" seems to me less > comfortable then doing this automagically by store() or structure(). I agree that building GetAs() strings manually is "less comfortable" as you say, but I'm afraid that for now it's the only option in MK. And to be fair: it's not *that* hard, after all the description string uses a very regular format. One trouble spot is adding a view which has no rows, but *does* have column definitions. There is no good way to extract that info properly right now. Another issue is adding a view with subviews which do not all have exactly the same structure. These are artefacts of the original design, with attached vs. unattached views - and they mess things up badly, so instead of supporting what are really design flaws, I decided to drop them for now. -jcw From jcw@equi4.com Wed Mar 3 13:23:28 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 04737B3B0D for ; Wed, 3 Mar 2004 13:23:28 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 9E107B3AE7 for ; Wed, 3 Mar 2004 13:21:59 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 6843E83BE0; Wed, 3 Mar 2004 13:21:50 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id C292183C29 for ; Wed, 3 Mar 2004 13:21:19 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <403E2993.26882.1A69B20@localhost> References: <403E2993.26882.1A69B20@localhost> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <475B7086-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] E4graph link Date: Wed, 3 Mar 2004 13:21:20 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:23:28 -0000 John Fletcher wrote: > The e4graph link on page http://www.equi4.com/mklinks.html > should be changed to > http://e4graph.sourceforge.net/ Done, thx. -jcw From jcw@equi4.com Wed Mar 3 13:25:18 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 530CCB3AE6 for ; Wed, 3 Mar 2004 13:25:18 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id F10C3B3A88 for ; Wed, 3 Mar 2004 13:22:11 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id D7A2683BE0; Wed, 3 Mar 2004 13:22:04 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 9742883C26 for ; Wed, 3 Mar 2004 13:21:13 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <403E1415.5000609@wi.mit.edu> References: <403E1415.5000609@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <43BC7099-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Saving views for recreation later Date: Wed, 3 Mar 2004 13:21:14 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:25:18 -0000 Brian Kelley wrote: > Suppose I create a view that is the result of many complicated > operations. Is there anyway to save this view state so I don't have > to repeat the operations in the future? If not, consider this a > feature request :) > It certainly sounds feasible since each generated view needs to have a > mapping back to the original views. That would be really powerful. > Currently all my views save their history so I can reconstruct them > later, but the history is an operational history so the operations > have to be reperformed. What you're saying, I think is: you construct a tree of view derivations. You want to be able to save that construct, so it can be re-applied on next open, right? Yes, I've been thinking along those lines. It's a matter of the MK view operators not just doing the requested operation, but also keeping track of the request so it can be stored and repeated. It would not be hard: * keep a stack of definitions so far * for each new derivation request, add a new entry with refs into the stack * store the stack (or at least specific parts) * alternately: keep a tree, and serialize the tree uner some name * on open, reload and re-apply the requests for the requested name I'll go into a little hand-waving explanation of why it hasn't happened: * time, priorities, funding - or any suitable mix there-of * the desire to do this at the core level, not "just" Python, Tcl, whatever * the need for a higher-level approach to such things than C++ * me having new code which is not good enough yet to replace the current MK * waiting for that new code to get there, so I can code such ideas at a higher level * time, priorities, funding - or any suitable mix there-of It's really quite simple: if anyone wants *specific* things such as these to happen, you'll just have to reach out and fund me. Because as long as no-one does, I can only work on my own priorities, and these are: a next-generation Metakit design and contract work. Don't hold your breath on the next-gen stuff, btw. I'm not going to do more than mention it, since there is *no* time scale to put next to it. Don't get me wrong, btw. I'm not asking for one penny from anyone. There's plenty going on, and I'm in fact buried in work at the moment. The spare moments I do have left go fully into working on new stuff, and I'm having an absolutely terrific time with that. So if people want to change my to do list, they will have to do more than post requests. Having said that, I'm always willing to discuss - on-list or off-list - how to help you make MK do new things in ways which I can integrate and which leaves MK's multi-lingual perspective intact. -jcw From jcw@equi4.com Wed Mar 3 13:27:26 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 8BB30B3AF3 for ; Wed, 3 Mar 2004 13:27:26 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 3D653B3B09 for ; Wed, 3 Mar 2004 13:22:32 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 3BDC283C26; Wed, 3 Mar 2004 13:22:22 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 05C1583C2D for ; Wed, 3 Mar 2004 13:21:26 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <40452D5B.9000102@rebirthing.co.nz> References: <40452D5B.9000102@rebirthing.co.nz> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <4B241C44-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Metakit auto** problem Date: Wed, 3 Mar 2004 13:21:27 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:27:26 -0000 David McNab wrote: > I notice the build for 2.4.9.3 has been well-fixed with regard to the > python extension. > > However, I have to question the hardwiring of 2.3 as the python > version (requiring me to copy and hack the configure script for legacy > pythons). > > At present, I'm working to enable building the python extension for > legacy pythons going back to 1.5.2. > > Why? > > Because I'm building a web framework suitable for use on budget public > vhosts with poor python support. Many of these budget hosts keep their > PHP right up to date, but only have Python 1.5.2. It's *very* nice to hear that backward compatibility is working out. I hate the game of forcing everyone to hop along the version update game just for the heck of it. Why is 2.3 hardcoded in configure? Best explanation I can think of is that I always feel completely lost in autoconf, and so I've never dared go in and add more configurability myself. For C++ there is no issue, nor is there for Tcl (because Tcl's stubs make extension binaries built for A work on B>A in just about every case). If you know how, please do consider fixing unix/configure.in - it keeps coming back, as every new Python release forces me to edit the file again. -jcw From jcw@equi4.com Wed Mar 3 13:28:54 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id F184CB3B0D for ; Wed, 3 Mar 2004 13:28:53 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 18CADB3B08 for ; Wed, 3 Mar 2004 13:22:32 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id DF74983BE0; Wed, 3 Mar 2004 13:22:23 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B703983C2A for ; Wed, 3 Mar 2004 13:21:21 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <5.1.0.14.0.20040225171000.01c2e7f0@131.103.204.244> References: <5.1.0.14.0.20040225171000.01c2e7f0@131.103.204.244> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <4858BADA-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] problems with hash view in 2.4.9.3 Date: Wed, 3 Mar 2004 13:21:22 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:28:54 -0000 Rajeev Gupta wrote: > I have upgraded to 2.4.9.3 from 2.4.8 couple of days back. I am facing > some > problems with hash views now. Earlier if i tried to append a row with > an > existing 'key', it used to overwirte the old row; not any more. Below > is a > session snippet from python command line showing this: > > Could somebody point out what i am doing wrong. > Thanks > Rajeev > > > > s=metakit.storage(r'd:\del\5',1) > >>> v=s.getas('DLE_URLS > [url_hash:I,email_sender:S,ufi:S,url_flags:L,request_file_name: > S,last_access_tim > e:F,url:S,email_recipients:S,email_timestamp:F,subject:S]') > >>> hv=s.getas('hv[_H:I,_R:I]') > >>> v=v.hash(hv,1) > >>> new_vals={'url_hash':1, > 'url_flags':0, > 'url':'adfsddddsdfdfsdfsdff', > 'ufi':'fdfdsfdsfgfddfdfdfffddyddyy', > 'email_timestamp':313, > 'request_file_name':'abc123' > } > >>> v.append(new_vals) > 0 > >>> v.append(new_vals) > 1 > >>> v.append(new_vals) > 2 > >>> v.append(new_vals) > 3 > >>> len(v) > 4 Thanks for the example. I don't know what has changed, but is it an idea to check for existence before adding? -jcw From jcw@equi4.com Wed Mar 3 13:30:17 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 768D4B3AE6 for ; Wed, 3 Mar 2004 13:30:17 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 55AF6B3B0A for ; Wed, 3 Mar 2004 13:22:36 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id AC2A583BE0; Wed, 3 Mar 2004 13:22:30 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 61BFD83C28 for ; Wed, 3 Mar 2004 13:21:19 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <1132856872.20040226110316@myd1.com> References: <1132856872.20040226110316@myd1.com> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <45DBA85A-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] data structure question Date: Wed, 3 Mar 2004 13:21:18 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:30:17 -0000 Jerry wrote: > I have a data structure (that works well so far) with three similar > sub-views that are accessed, set, and summarized at different > points. Now I have a requirement to output a summary of all the > detail with a label that identifies with of the three sets the data > came from. The solution I came up with doesn't work, so I am > thinking out loud to see if someone has an idea that involves the > least amount of re-coding. The sub-views are accessed quite a bit > in normal processing, and the summary only needs to be created once > or twice a month. > > Simplized data format: >>>> vw = > > db.getas('main[id:I,fname:S,lname:S,new[key:S,val:I],old[key:S,val: > I],adj[key:S,val:I]') >>>> jdb.dump(vw) > id fname lname new old adjs > ---- ------ ----- ------ ------ ------ > 1 first1 last1 0 rows 0 rows 0 rows > 2 bob last2 2 rows 0 rows 0 rows > 4 first4 last4 2 rows 0 rows 4 rows > > Using a combination of flatten, union, and project I can get REAL > CLOSE to what I want: > >>>> jdb.dump(vw31) > id key val > ---- ----- --- > 2 val10 10 > 2 val20 20 > 4 val10 10 > 4 val10 10 > 4 val20 20 > 4 val20 20 > 4 val30 30 > 4 val40 40 > > What I need is to know WHICH type of value each row is: > id key val type > ---- ----- --- ---- > 2 val10 10 new > 2 val20 20 new > 4 val10 10 new > 4 val10 10 adj > 4 val20 20 new > 4 val20 20 adj > 4 val30 30 adj > 4 val40 40 adj > > The main problem being that one cannot successfully add a property to > a > PyROViewer object, which is the result of the union and flatten > methods. It seems I either have to: > > 1) create a separate temporary view for each type and manually copy > the flattened view into it, creating and setting 'type' > appropriately. Then union off of these PyView objects. > 2) or modify my system to always write the subview type into the > subview. This means extra programming and run-time overhead, and > extra strings in tens of thousands of records. > 3) or some other creative idea you suggest. This is exactly the sort of manipulation I hope to improve further, btw. It's becoming more and more clear that MK needs to offer full relational algebra + set operators. First of all, note that you can add properties on the fly to views, they will stay around until commit/rollback/close (but won't be saved if they are not part of a getas). So you could do: - for each row in "new" subview: set type property to "new" - etc for other subviews Then you'd be able to see ... ah, wait - I see your point now: Mk4py tracks R/O view status and forbids this (even though the C++ core would allow it). Ok, another idea: create a view with N copies of the string "new" (can be done virtually with wrap() in Mk4py). Use pair() to add that view "next" to the subview, i.e. horizontal concatenation. Does that take you closer to a solution? -jcw From jcw@equi4.com Wed Mar 3 13:32:16 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 515B3B3B18 for ; Wed, 3 Mar 2004 13:32:16 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id E6EC0B3B0B for ; Wed, 3 Mar 2004 13:22:41 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 1EFEB83C27; Wed, 3 Mar 2004 13:22:34 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E60B283C2B for ; Wed, 3 Mar 2004 13:21:23 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: References: Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <49597A80-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Sharing Mk databases Date: Wed, 3 Mar 2004 13:21:23 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:32:18 -0000 wrote: > I'm looking ahead to a time when I'll extend my application to be > multi-user. This would mean multiple users accessing and updating the > databases. Will Metakit support this? Or are the plans to support it > in > the future? > > If not, what is a reasonable workaround? Write a server that will > handle > all interactions with Metakit databases? There are many sides to multi-user: - single process (threads)? - multi-process, single file machine? - locking (not just contention), but application level "logical" interaction - isolation requirements (the "I" in "ACID"), nested commits Have a look at Brian Kelley's Metaserve. It's a Python-based server, with clients proxying all their requests through sockets to the server. Listed on the MK links page: http://www.equi4.com/mklinks.html I've used a similar (but simpler) approach in the past in Tcl, using a little server called "Tequila". Has been tremendously effective for me in two commercial projects, but evidently there are many parameters which will affect how well it can work - scalability, responsiveness, parallelism, etc. Yes, I do intend to support concurrent access. The commit-aside design (which is not finished - it uses huge amounts of disk space right now) is a way to provide contention-free concurrency. It can allow readers to proceed without ever locking. It should work well in multiple-reader/single-writer scenario's, and can be extended to serialize multiple writers. Concurrency in MK is not trivial, due to the very high level of caching which can be going on (and memory mapped files introduce yet more cache coherence headaches). It's very easy to open a file and read without problems, it's also trivial to commit *once* and not have it affect current readers. The trouble is the next commit, and the need to sync all readers - or offer them a way to keep looking at frozen state and provide that isolation level. I'm currently involved in writing a layer on top of MK which makes it usable as a multi-user system (but at the Tcl level, alas). The lessons from that will help design and implement proper mechanism at the core level. it's likely to go from where commit-aside stands today, and simply finish that job. But all of this is not going to help you further any day soon. If you need something operational, I'd definitely suggest looking at Kelley's Metaserve, and perhaps contributing to that so it becomes more capable. FWIW, I'll be following that closely, and along with my own work will look for ways to solve this at a core level. As is becoming clearer all the time, nobody seems to be very interested in solving things at any other level than the language they happen to use - but I am, more than ever. If I had the capability now, multi-user capability would be in MK today, and I would not have to solve it in "just" Tcl. At the risk of making a nuisance of myself... let me repeat: > I'll go into a little hand-waving explanation of why it hasn't > happened: > * time, priorities, funding - or any suitable mix there-of > * the desire to do this at the core level, not "just" Python, Tcl, > whatever > * the need for a higher-level approach to such things than C++ > * me having new code which is not good enough yet to replace the > current MK > * waiting for that new code to get there, so I can code such ideas at > a higher level > * time, priorities, funding - or any suitable mix there-of > > It's really quite simple: if anyone wants *specific* things such as > these to happen, you'll just have to reach out and fund me. Because > as long as no-one does, I can only work on my own priorities, and > these are: a next-generation Metakit design and contract work. Don't > hold your breath on the next-gen stuff, btw. I'm not going to do more > than mention it, since there is *no* time scale to put next to it. > > Don't get me wrong, btw. I'm not asking for one penny from anyone. > There's plenty going on, and I'm in fact buried in work at the moment. > The spare moments I do have left go fully into working on new stuff, > and I'm having an absolutely terrific time with that. So if people > want to change my to do list, they will have to do more than post > requests. > > Having said that, I'm always willing to discuss - on-list or off-list > - how to help you make MK do new things in ways which I can integrate > and which leaves MK's multi-lingual perspective intact. -jcw From jcw@equi4.com Wed Mar 3 13:33:31 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D1CECB3AF3 for ; Wed, 3 Mar 2004 13:33:31 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 67B93B3B0C for ; Wed, 3 Mar 2004 13:22:45 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 4037E83BE0; Wed, 3 Mar 2004 13:22:42 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 9D22783C2C for ; Wed, 3 Mar 2004 13:21:24 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: References: Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <4A45F843-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] view() vs. getas() in Python Date: Wed, 3 Mar 2004 13:21:25 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:33:32 -0000 wrote: > Supposedly view() is "The normal way to retrieve an existing view." > But > apparently even if the view doesn't exist, something gets retrieved -- > though it isn't of much use. However, it does appear that you can > append > to this non-existent view since > > v = db.view('NonExistentView') > v.append(foo=1) > > *appears* to work. But then, of course any attempt at referencing > v[0].foo > fails (though referencing v[0] does not). Shouldn't appending to a > non-existent view raise an exception? If you look at db.properties() > there > again *appears* to be a view called 'NonExistentView' there. > > How can I tell that I've "retrieved" a non-existent view as opposed to, > say, a merely empty one? If I try to use description() on a > non-existent > view I get a really ugly Python internal error. > > Given this, what is the point of view()? (No pun intended.) It's historical: getas used to be very expensive. And there used to be "storeas". Nowadays, like you I tend to use getas all the time. The app essentially says: get me a view of such-and-such shape. Just do it, make it that shape if need be. Extremely handy for adding properties over time. The future of this is going to be different still, btw. The plan is to treat view structure as a "meta view" itself. So you'll have a view, where each row describes a column. I started on that in the current MK design, but it really goes much deeper and benefits from a fundamental switch to this approach in the core. That will go as far as making a row add in the meta view be equivalent to defining a new column, and so on for renames and deletes. > But why not raise an exception if a bogus view name is given to view()? Good point. I think it would indeed help avoid time-wasting surprises. -jcw From jcw@equi4.com Wed Mar 3 13:35:18 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3B827B3B08 for ; Wed, 3 Mar 2004 13:35:18 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id F233AB3B0E for ; Wed, 3 Mar 2004 13:22:47 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 6ACC883BE0; Wed, 3 Mar 2004 13:22:46 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 1ABC483C2E for ; Wed, 3 Mar 2004 13:21:30 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <40454115.70603@rebirthing.co.nz> References: <40454115.70603@rebirthing.co.nz> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <4BFD4F75-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Instructions for Legacy Pythons Date: Wed, 3 Mar 2004 13:21:28 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 12:35:18 -0000 David McNab wrote: > Below are instructions for people needing to build the python metakit > wrapper for non-latest python versions. [...] > I hope the metakit devs might see fit to include these instructions in > Metakit releases. Great. I've checked this into CVS as python/README-dmn. Thanks! -jcw From jyl@mod3.net Wed Mar 3 16:47:11 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 5AB4EB3A88 for ; Wed, 3 Mar 2004 16:47:11 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id 5F3C1B3AE6 for ; Wed, 3 Mar 2004 16:47:06 +0100 (CET) Received: (qmail 21159 invoked by uid 48); 3 Mar 2004 15:46:53 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Wed, 3 Mar 2004 07:46:52 -0800 (PST) Message-ID: <65242.209.233.24.218.1078328812.squirrel@mod3.net> Date: Wed, 3 Mar 2004 07:46:52 -0800 (PST) Subject: Re: [Metakit] Problem building Metakit 2.4.9.3 on FreeBSD 4.9 From: "Jacob Levy" To: In-Reply-To: <42A4D4C2-6D0D-11D8-8A9A-000A9588127E@equi4.com> References: <65251.209.233.24.218.1077548669.squirrel@mod3.net> <42A4D4C2-6D0D-11D8-8A9A-000A9588127E@equi4.com> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 15:47:11 -0000 JC Sorry but no go: I tried on FreeBSD 4.9 and got this error during the .../unix/configure step: jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ cd ../unix jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ make distclean .... jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ aclocal jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/unix$ libtoolize -f -c Putting files in AC_CONFIG_AUX_DIR, `scripts'. jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/unix$ autoconf jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/unix$ cd ../builds/ jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ ../unix/configure --prefix=`pwd`/../../install checking for Python configuration... not enabled checking for Tcl headers... not enabled checking for building with threads... no (default) checking for g++... g++ checking for C++ compiler default output... a.out checking whether the C++ compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking for a BSD-compatible install... /usr/bin/install -c checking build system type... i386-unknown-freebsdelf4.9 checking host system type... i386-unknown-freebsdelf4.9 checking for ranlib... ranlib checking for gcc... gcc checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for ld used by GCC... /usr/libexec/elf/ld checking if the linker (/usr/libexec/elf/ld) is GNU ld... yes checking for BSD-compatible nm... /usr/bin/nm -B checking whether ln -s works... yes loading cache /dev/null within ltconfig ltconfig: you must specify a host type if you use `--no-verify' Try `ltconfig --help' for more information. configure: error: libtool configure failed jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ make make: no target to make. jyljyljyl@x86-freebsd1:~/compile/metakit-2.4.9.3/builds$ Really, is there a huge reason why you rely on this configure thing? What extra essential functionality is provided by libtool? --JYL > Jacob Levy wrote: > >> Has anyone successfully built Metakit 2.4.9.3 on FreeBSD 4.9? If so, >> can >> you help me with this (probably) pilot error on my part? > > [...] >> /usr/bin/install -c .libs/libmk4.lai >> /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds/../../ >> install/lib/libmk4.la >> install: .libs/libmk4.lai: No such file or directory >> *** Error code 71 >> >> Stop in /home/users/j/jy/jyljyljyl/compile/metakit-2.4.9.3/builds. > > Looks like libtool/autoconf madness. > > I suggest: > > make distclean > cd ../unix > aclocal > libtoolize -f -c > autoconf > cd ../builds > ../unix/configure > make > make install > > I'm sorely tempted to throw out everything related to autoconf, > libtool, and even make. Older versions of Metakit used to have the > following build instructions: > gcc -O2 -shared -I../include ../src/*.cpp -o libmk4.so > And I sure miss those times. Things use to be *simple* (does anyone > remember that concept?). > > -jcw > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jyl@mod3.net Wed Mar 3 17:35:50 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E613BB3AF5 for ; Wed, 3 Mar 2004 17:35:49 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id 3C036B3AF2 for ; Wed, 3 Mar 2004 17:35:39 +0100 (CET) Received: (qmail 15420 invoked by uid 48); 3 Mar 2004 16:35:37 -0000 Received: from 209.233.24.218 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Wed, 3 Mar 2004 08:35:37 -0800 (PST) Message-ID: <64610.209.233.24.218.1078331737.squirrel@mod3.net> Date: Wed, 3 Mar 2004 08:35:37 -0800 (PST) Subject: Re: [Metakit] Problem building Metakit 2.4.9.3 on FreeBSD 4.9 From: "Jacob Levy" To: In-Reply-To: <42A4D4C2-6D0D-11D8-8A9A-000A9588127E@equi4.com> References: <65251.209.233.24.218.1077548669.squirrel@mod3.net> <42A4D4C2-6D0D-11D8-8A9A-000A9588127E@equi4.com> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 16:35:51 -0000 Maybe we should start from here: > Older versions of Metakit used to have the > following build instructions: > gcc -O2 -shared -I../include ../src/*.cpp -o libmk4.so > And I sure miss those times. Things use to be *simple* (does anyone > remember that concept?). That looks simple :) Why do you need autoconf, configure, libtool and all that other cruft? FWIW, I tried several variations of the steps you suggested, leaving out aclocal, or leaving out autoconf. All lead to various errors in the configure or make steps as I reported earlier. --JYL From jcw@equi4.com Wed Mar 3 19:47:47 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 57398B3AE6 for ; Wed, 3 Mar 2004 19:47:47 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id A13F9B3A88 for ; Wed, 3 Mar 2004 19:47:45 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 93E0E83C26; Wed, 3 Mar 2004 19:47:41 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 51D1883914 for ; Wed, 3 Mar 2004 19:47:07 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) Content-Transfer-Encoding: 7bit Message-Id: <2A51B5BD-6D43-11D8-8A9A-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Wed, 3 Mar 2004 19:47:04 +0100 X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] libtool or not? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 18:47:47 -0000 If you are building Metakit on anything but the usual quad or so of most common platforms, any of C++, Python, or Tcl - could you please help decide what to do? YES OR NO: get rid of libtool in the MK build process? WHY: less fighting, drop dependency on libtool, which has changed over the years. WHY NOT: may require some work to build for special platforms (AIX? HPUX?) HOW: switch to "gcc -shared", with a few refinements to make it work on Mac OS X and such. These refinements can be added to the unix/configure.in logic, autoconf has sufficient capability to cover most cases, I think. WINDOWS: no change, when built with MSVC 6 (I just checked in a MSVC 7.0 version, btw). No change with mingw either, since "-shared" does the right thing nowadays. TCL: probably not affected, it has its own configure logic. PYTHON: probably not affected, it is moving to distutils. Your votes and opinions please... -jcw PS. In fact, I'd love to throw out all of make and autoconf, if I knew how to create an effective distro without them (Python is furthest along in that area, clearly, with its distutils). Make is a brilliant concept, but even that makes little sense when it's about deploying and compiling a tested distribution - once. IMO the only strong case for autoconf + make nowadays, is that everyone in OSS-land is used to the "configure; make; make install" salute. From jyl@mod3.net Wed Mar 3 20:44:09 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E90D5B3AFD for ; Wed, 3 Mar 2004 20:44:08 +0100 (CET) Received: from mod3.net (mod3.net [69.55.237.247]) by dizzie.triqs.com (Postfix) with SMTP id 6025EB3AF2 for ; Wed, 3 Mar 2004 20:44:06 +0100 (CET) Received: (qmail 17956 invoked by uid 48); 3 Mar 2004 19:44:02 -0000 Received: from 192.18.42.11 (SquirrelMail authenticated user jyl) by mod3.net with HTTP; Wed, 3 Mar 2004 11:44:02 -0800 (PST) Message-ID: <47420.192.18.42.11.1078343042.squirrel@mod3.net> Date: Wed, 3 Mar 2004 11:44:02 -0800 (PST) Subject: Re: [Metakit] libtool or not? From: "Jacob Levy" To: In-Reply-To: <2A51B5BD-6D43-11D8-8A9A-000A9588127E@equi4.com> References: <2A51B5BD-6D43-11D8-8A9A-000A9588127E@equi4.com> X-Priority: 3 Importance: Normal X-Mailer: SquirrelMail (version 1.2.10) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com cc: metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jyl@mod3.net List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 19:44:09 -0000 Anything that simplifies the build is goodness. My vote is for getting rid of libtool. --JYL > If you are building Metakit on anything but the usual quad or so of > most common platforms, any of C++, Python, or Tcl - could you please > help decide what to do? > > > > YES OR NO: get rid of libtool in the MK build process? > > WHY: less fighting, drop dependency on libtool, which has changed over > the years. > > WHY NOT: may require some work to build for special platforms (AIX? > HPUX?) > > HOW: switch to "gcc -shared", with a few refinements to make it work on > Mac OS X and such. These refinements can be added to the > unix/configure.in logic, autoconf has sufficient capability to cover > most cases, I think. > > WINDOWS: no change, when built with MSVC 6 (I just checked in a MSVC > 7.0 version, btw). No change with mingw either, since "-shared" does > the right thing nowadays. > > TCL: probably not affected, it has its own configure logic. > > PYTHON: probably not affected, it is moving to distutils. > > > > Your votes and opinions please... > > -jcw > > PS. In fact, I'd love to throw out all of make and autoconf, if I knew > how to create an effective distro without them (Python is furthest > along in that area, clearly, with its distutils). Make is a brilliant > concept, but even that makes little sense when it's about deploying and > compiling a tested distribution - once. IMO the only strong case for > autoconf + make nowadays, is that everyone in OSS-land is used to the > "configure; make; make install" salute. > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From pkropf@legato.com Wed Mar 3 21:47:34 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 947C0B3AF2 for ; Wed, 3 Mar 2004 21:47:34 +0100 (CET) Received: from smtp2.legato.com (smtp2.legato.com [137.69.100.7]) by dizzie.triqs.com (Postfix) with ESMTP id 5CAA0B3AE6 for ; Wed, 3 Mar 2004 21:47:31 +0100 (CET) Received: from mta1.legato.com (mta1.legato.com [137.69.1.14]) i23KlNXS007502 for ; Wed, 3 Mar 2004 15:47:28 -0500 (EST) Received: from pa-mail1.legato.com (pa-mail1.legato.com [137.69.23.23]) i23Kdq4I013898 for ; Wed, 3 Mar 2004 12:39:52 -0800 (PST) Received: by pa-mail1.legato.com with Internet Mail Service (5.5.2657.72) id <1617DFHB>; Wed, 3 Mar 2004 12:40:01 -0800 Message-ID: <96F3EC4BDF504540BB9715CBE9DACB8B0678B939@pa-mail1.legato.com> From: Peter Kropf To: metakit@equi4.com Subject: RE: [Metakit] libtool or not? Date: Wed, 3 Mar 2004 12:39:54 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C4015F.AF349D50" X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 20:47:34 -0000 This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C4015F.AF349D50 Content-Type: text/plain I tend to agree. Simplyfing the build process and removing the use of libtool would be good. - Peter > -----Original Message----- > From: Jacob Levy [mailto:jyl@mod3.net] > Sent: Wednesday, March 03, 2004 11:44 AM > To: jcw@equi4.com > Cc: metakit@equi4.com > Subject: Re: [Metakit] libtool or not? > > > Anything that simplifies the build is goodness. My vote is > for getting rid of libtool. > > --JYL > > > If you are building Metakit on anything but the usual quad or so of > > most common platforms, any of C++, Python, or Tcl - could > you please > > help decide what to do? > > > > > > > > YES OR NO: get rid of libtool in the MK build process? > > > > WHY: less fighting, drop dependency on libtool, which has > changed over > > the years. > > > > WHY NOT: may require some work to build for special platforms (AIX? > > HPUX?) > > > > HOW: switch to "gcc -shared", with a few refinements to > make it work > > on Mac OS X and such. These refinements can be added to the > > unix/configure.in logic, autoconf has sufficient capability > to cover > > most cases, I think. > > > > WINDOWS: no change, when built with MSVC 6 (I just checked > in a MSVC > > 7.0 version, btw). No change with mingw either, since > "-shared" does > > the right thing nowadays. > > > > TCL: probably not affected, it has its own configure logic. > > > > PYTHON: probably not affected, it is moving to distutils. > > > > > > > > Your votes and opinions please... > > > > -jcw > > > > PS. In fact, I'd love to throw out all of make and autoconf, if I > > knew how to create an effective distro without them (Python is > > furthest along in that area, clearly, with its distutils). > Make is a > > brilliant concept, but even that makes little sense when it's about > > deploying and compiling a tested distribution - once. IMO the only > > strong case for autoconf + make nowadays, is that everyone > in OSS-land > > is used to the "configure; make; make install" salute. > > > > _____________________________________________ > > Metakit mailing list - Metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > ------_=_NextPart_001_01C4015F.AF349D50 Content-Type: text/html RE: [Metakit] libtool or not?

I tend to agree. Simplyfing the build process and removing the use of libtool would be good.

- Peter


> -----Original Message-----
> From: Jacob Levy [mailto:jyl@mod3.net]
> Sent: Wednesday, March 03, 2004 11:44 AM
> To: jcw@equi4.com
> Cc: metakit@equi4.com
> Subject: Re: [Metakit] libtool or not?
>
>
> Anything that simplifies the build is goodness. My vote is
> for getting rid of libtool.
>
> --JYL
>
> > If you are building Metakit on anything but the usual quad or so of
> > most common platforms, any of C++, Python, or Tcl - could
> you please
> > help decide what to do?
> >
> >
> >
> > YES OR NO: get rid of libtool in the MK build process?
> >
> > WHY: less fighting, drop dependency on libtool, which has
> changed over
> > the years.
> >
> > WHY NOT: may require some work to build for special platforms (AIX?
> > HPUX?)
> >
> > HOW: switch to "gcc -shared", with a few refinements to
> make it work
> > on Mac OS X and such.  These refinements can be added to the
> > unix/configure.in logic, autoconf has sufficient capability
> to cover
> > most cases, I think.
> >
> > WINDOWS: no change, when built with MSVC 6 (I just checked
> in a MSVC
> > 7.0 version, btw).  No change with mingw either, since
> "-shared" does
> > the right thing nowadays.
> >
> > TCL: probably not affected, it has its own configure logic.
> >
> > PYTHON: probably not affected, it is moving to distutils.
> >
> >
> >
> > Your votes and opinions please...
> >
> > -jcw
> >
> > PS.  In fact, I'd love to throw out all of make and autoconf, if I
> > knew how to create an effective distro without them (Python is
> > furthest along in that area, clearly, with its distutils). 
> Make is a
> > brilliant concept, but even that makes little sense when it's about
> > deploying and compiling a tested distribution - once.  IMO the only
> > strong case for autoconf + make nowadays, is that everyone
> in OSS-land
> > is used to the "configure; make; make install" salute.
> >
> > _____________________________________________
> > Metakit mailing list  -  Metakit@equi4.com
> > http://www.equi4.com/mailman/listinfo/metakit
>
>
>
> _____________________________________________
> Metakit mailing list  -  Metakit@equi4.com
> http://www.equi4.com/mailman/listinfo/metakit
>

------_=_NextPart_001_01C4015F.AF349D50-- From david@rebirthing.co.nz Thu Mar 4 01:21:20 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D2003B3AF2 for ; Thu, 4 Mar 2004 01:21:19 +0100 (CET) Received: from dbmail-mx3.orcon.co.nz (unknown [219.88.242.4]) by dizzie.triqs.com (Postfix) with ESMTP id EA311B3AE6 for ; Thu, 4 Mar 2004 01:21:14 +0100 (CET) Received: from rebirthing.co.nz (port-203-99-27-169.jet.net.nz [203.99.27.169])ESMTP id i240IsSQ029973 for ; Thu, 4 Mar 2004 13:18:55 +1300 Message-ID: <40467672.60006@rebirthing.co.nz> Date: Thu, 04 Mar 2004 13:21:06 +1300 From: David McNab User-Agent: Mozilla Thunderbird 0.5 (X11/20040208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit list X-Enigmail-Version: 0.83.3.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.38 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Web deployment questions X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2004 00:21:20 -0000 Hi, I had been looking to use SQLObject as the database backend for an active web framework I'm building, but have had a fresh look at Metakit, and see a lot of positives. However, I'm unclear about some Metakit issues, and hope you might be able to enlighten me: Memory footprint: - does Metakit load the whole DB into memory upon calling storage()? - does Metakit hold each entire view in memory? Concurrency: - what are the better schemes for db locking, in cases where multiple clients (eg under Apache CGIs) need to write to a db? - if I'm using file locks to make it safe for multiple client writes, how severely will this impact the website performance under load (eg if being slashdotted) ? (worst case scenario - someone doing a site search) I look forward to your comments and insights Cheers David -- leave this line intact so your email gets through my junk mail filter From jcw@equi4.com Thu Mar 4 01:34:24 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 58ADBB3AF2 for ; Thu, 4 Mar 2004 01:34:24 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 2ECD6B3AE6 for ; Thu, 4 Mar 2004 01:34:22 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 84F4483C26; Thu, 4 Mar 2004 01:34:19 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id DAEAA83914 for ; Thu, 4 Mar 2004 01:34:13 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <40455BDF.3050302@rebirthing.co.nz> References: <40452D5B.9000102@rebirthing.co.nz> <20040303034334.GA69285@uiuc.edu> <40455BDF.3050302@rebirthing.co.nz> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Metakit auto** problem Date: Thu, 4 Mar 2004 01:34:14 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2004 00:34:24 -0000 David McNab wrote: > Nicholas Riley wrote: >> Try using the distutils build method instead of ./configure >> --enable-python - it'll work back to 1.5.2 if you use the latest >> distutils (which is also guaranteed to work back to 1.5.2). > > Tried that. > > With metakit's setup.py, distutils doesn't work for pythons earlier > than 2.3. Is there something simple we can do to fix it? -jcw From jcw@equi4.com Thu Mar 4 01:36:26 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2C1C6B3AF2 for ; Thu, 4 Mar 2004 01:36:26 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 475DBB3AE6 for ; Thu, 4 Mar 2004 01:36:24 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id B8BE683C26; Thu, 4 Mar 2004 01:36:21 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B9C2383914 for ; Thu, 4 Mar 2004 01:36:19 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <40467672.60006@rebirthing.co.nz> References: <40467672.60006@rebirthing.co.nz> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Web deployment questions Date: Thu, 4 Mar 2004 01:36:20 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2004 00:36:26 -0000 David McNab wrote: > Memory footprint: > - does Metakit load the whole DB into memory upon calling storage()? Nope. Memory mapped files. > - does Metakit hold each entire view in memory? If you create it: yes. Once committed: no. If you make changes: some pieces until commit. > Concurrency: > - what are the better schemes for db locking, in cases where > multiple clients (eg under Apache CGIs) need to write to a db? > - if I'm using file locks to make it safe for multiple client writes, > how severely will this impact the website performance under load > (eg if being slashdotted) ? > (worst case scenario - someone doing a site search) Metakit does not support concurrent access (other than the trivial all-read-only). Two ways to do it: server process or file locks. For wikit, a simple wiki based on MK, I use file locks + a static Apache page cache, triggered by error redirects when a static copy is not present. Works/scales well. For Python, have a look at Brian Kelley's Metaserve, it's on the links page: http://www.equi4.com/mklinks.html In the future, the design wil be commit-aside: saving changes to a second file, with "instant replay" for readers (again, mmap: no reading until used). A bit like journalling, but geared at MK's more specific properties. Will reduce contention to *only* the writers, i.e. just a file lock on the commit-aside file for writers. Semi-vaporware ("some condensation"). -jcw From david@rebirthing.co.nz Thu Mar 4 01:56:50 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2E7E8B3AF2; Thu, 4 Mar 2004 01:56:50 +0100 (CET) Received: from dbmail-mx2.orcon.co.nz (unknown [219.88.242.4]) by dizzie.triqs.com (Postfix) with ESMTP id 3EE03B3AE6; Thu, 4 Mar 2004 01:56:43 +0100 (CET) Received: from rebirthing.co.nz (port-203-99-27-169.jet.net.nz [203.99.27.169])i240sLVs025482; Thu, 4 Mar 2004 13:54:21 +1300 Message-ID: <40467EC3.7030305@rebirthing.co.nz> Date: Thu, 04 Mar 2004 13:56:35 +1300 From: David McNab User-Agent: Mozilla Thunderbird 0.5 (X11/20040208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler Subject: Re: [Metakit] Metakit auto** problem References: <40452D5B.9000102@rebirthing.co.nz> <20040303034334.GA69285@uiuc.edu> <40455BDF.3050302@rebirthing.co.nz> In-Reply-To: X-Enigmail-Version: 0.83.3.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.38 X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2004 00:56:50 -0000 Jean-Claude Wippler wrote: >> With metakit's setup.py, distutils doesn't work for pythons earlier >> than 2.3. > Is there something simple we can do to fix it? To be honest, I don't understand what's going in in metakit/python's setup.py file (I tend to write only the simplest setup.py files myself). I could study the metakit setup.py and try to fathom what's going on, but I suspect this is a task better suited to the setup.py author. Cheers David From njriley@shrug.csl.uiuc.edu Thu Mar 4 03:08:03 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 9FDA1B3AF2 for ; Thu, 4 Mar 2004 03:08:03 +0100 (CET) Received: from shrug.csl.uiuc.edu (shrug.csl.uiuc.edu [130.126.137.135]) by dizzie.triqs.com (Postfix) with ESMTP id 0F41CB3AE6 for ; Thu, 4 Mar 2004 03:08:02 +0100 (CET) Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.10/8.12.9) with ESMTP id i24280AO087708 for ; Wed, 3 Mar 2004 20:08:01 -0600 (CST) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.10/8.12.9/Submit) id i24280xa087707 for metakit@equi4.com; Wed, 3 Mar 2004 20:08:00 -0600 (CST) (envelope-from njriley) Resent-Message-Id: <200403040208.i24280xa087707@shrug.csl.uiuc.edu> Date: Wed, 3 Mar 2004 20:07:39 -0600 From: Nicholas Riley To: David McNab Subject: Re: [Metakit] Metakit auto** problem Message-ID: <20040304020739.GA86887@uiuc.edu> References: <40452D5B.9000102@rebirthing.co.nz> <20040303034334.GA69285@uiuc.edu> <40455BDF.3050302@rebirthing.co.nz> <40467EC3.7030305@rebirthing.co.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <40467EC3.7030305@rebirthing.co.nz> User-Agent: Mutt/1.4.1i Resent-From: Nicholas Riley Resent-Date: Wed, 3 Mar 2004 20:08:00 -0600 Resent-To: Metakit list X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2004 02:08:03 -0000 On Thu, Mar 04, 2004 at 01:56:35PM +1300, David McNab wrote: > Jean-Claude Wippler wrote: > >>With metakit's setup.py, distutils doesn't work for pythons earlier > >>than 2.3. > >Is there something simple we can do to fix it? > > To be honest, I don't understand what's going in in metakit/python's > setup.py file (I tend to write only the simplest setup.py files myself). > > I could study the metakit setup.py and try to fathom what's going on, > but I suspect this is a task better suited to the setup.py author. *pokes head out of sand* I'll take a look at this tonight. -- =Nicholas Riley | From rajeev@accellion.com Thu Mar 4 06:35:15 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6BA11B3AF2 for ; Thu, 4 Mar 2004 06:35:15 +0100 (CET) Received: from mail01g.rapidsite.net (mail01g.rapidsite.net [208.55.43.104]) by dizzie.triqs.com (Postfix) with SMTP id 6A195B3AE6 for ; Thu, 4 Mar 2004 06:35:13 +0100 (CET) Received: from www.nyeq.com (131.103.204.244) by mail01g.rapidsite.net (RS ver 1.0.91vs) with SMTP id 2-0903628502 for ; Thu, 4 Mar 2004 00:35:06 -0500 (EST) Message-Id: <5.1.0.14.0.20040304103611.01c25f88@131.103.204.244> X-Sender: rajeev@131.103.204.244 X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Thu, 04 Mar 2004 11:03:32 +0530 To: metakit@equi4.com From: Rajeev Gupta Subject: Re: [Metakit] problems with hash view in 2.4.9.3 In-Reply-To: <20040303123221.EE96EB3B1E@dizzie.triqs.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Loop-Detect: 1 X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2004 05:35:15 -0000 >Thanks for the example. > >I don't know what has changed, but is it an idea to check for existence >before adding? > >-jcw Thanks for the reply. The example was to show that it creates another row with the same key value in 2.4.9.3. Actually I have been using view.append to 'update' a row in a hash view. But it no longer works as the old row isn't overwritten. Now i am forced to shift to a 'native' view and do something like: new_index = view.append('changed_row') matching_indices=view.indices(view.select({'key':'row_to_be_updated'}) matching_indices.remove(matching_indices.select({'index':new_index})) view.remove(matching_indices) Shift to the native view is forced as after a 'remove' i need to recreate the hash view for it to recognise the changed row. I mean if i do a view.find({'key':'row_to_be_updated'}) after the above code, it returns a failure unless i do view=db.view('my_view') again. I checked this with 2.4.8 and it works fine there. I am not sure why nobody else has reported this so far. I have tried building 2.4.9.3 from code, with python 2.2 and 2.3 - all with the same results. Thx again for your time and effort. Rajeev From roseman@courseforum.com Wed Mar 3 21:12:45 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2E67FB3AF2 for ; Wed, 3 Mar 2004 21:12:45 +0100 (CET) Received: from fep3.cogeco.net (smtp.cogeco.net [216.221.81.25]) by dizzie.triqs.com (Postfix) with ESMTP id 4F293B3AE6 for ; Wed, 3 Mar 2004 21:12:44 +0100 (CET) Received: from [10.0.1.2] (d141-169-142.home.cgocable.net [24.141.169.142]) by fep3.cogeco.net (Postfix) with ESMTP id 3823A1231 for ; Wed, 3 Mar 2004 15:12:43 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <2A51B5BD-6D43-11D8-8A9A-000A9588127E@equi4.com> References: <2A51B5BD-6D43-11D8-8A9A-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Mark Roseman Subject: Re: [Metakit] libtool or not? Date: Wed, 3 Mar 2004 15:17:39 -0500 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Thu, 04 Mar 2004 10:29:10 +0100 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 20:12:45 -0000 Libtool has never been a source of happiness for me; usually everything works fine, but once in a while something goes wrong and its always libtool that is the cause of my grief and frustration. If it is not a lot of work for you to get rid of it, I think its worth doing at some convenient time. But I don't think its urgent. I think platform build support should be a non-issue; figure out a few platforms you're happy to 'support' (i.e. that you might smoke test a new release on). Its open source, so on other platforms (where I would suspect it will mostly just work) people can muddle along, submit fixes, or pay you for support. Beyond that, keep with configure/make/make install, which I think is still embedded deep in my brain. Mark From rjackson@nmcourts.com Thu Mar 4 00:52:34 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 838FCB3AF2 for ; Thu, 4 Mar 2004 00:52:34 +0100 (CET) Received: from jidmail.nmcourts.com (jidmail.nmcourts.com [198.59.128.143]) by dizzie.triqs.com (Postfix) with ESMTP id 55A1BB3AE6 for ; Thu, 4 Mar 2004 00:52:27 +0100 (CET) Received: from smtp.nmcourts.com ([198.59.128.248]) by jidmail.nmcourts.com (Post.Office MTA v3.5.4 release 224 ID# 0-59782U3000L300S0V35) with SMTP id com for ; Wed, 3 Mar 2004 16:52:25 -0700 Received: from nmcourts.com ([198.59.128.171]) by smtp.nmcourts.com (SAVSMTP 3.1.0.29) with SMTP id M2004030316530503292 for ; Wed, 03 Mar 2004 16:53:05 -0700 Message-ID: <40466F91.6020604@nmcourts.com> Date: Wed, 03 Mar 2004 16:51:45 -0700 From: rjackson@nmcourts.com (Robert E. Jackson) User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Thu, 04 Mar 2004 10:29:32 +0100 Subject: [Metakit] Memory Consumption X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 23:52:35 -0000 I am using Mk4tcl as a searchable data structure but not opening a disk file or saving to disk. Whenever possible I do a 'mk::view size WHATEVER 0'. However, the memory used by the process steadily increases. Is there any way to avoid this memory comsumption creep? Bob Jackson From jcw@equi4.com Sun Mar 7 13:30:36 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 8718FB3AED for ; Sun, 7 Mar 2004 13:30:36 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id A6784B3AEC for ; Sun, 7 Mar 2004 13:30:33 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 5936A83C26; Sun, 7 Mar 2004 13:30:31 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B978083914 for ; Sun, 7 Mar 2004 13:30:23 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) Content-Transfer-Encoding: 7bit Message-Id: <33D23BE8-7033-11D8-80D8-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Sun, 7 Mar 2004 13:30:22 +0100 X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] libtool has been removed X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2004 12:30:36 -0000 I've removed libtool from the Metakit build setup and checked in changes to CVS. The changes are very preliminary - this build is likely to work on less platforms than before. Use the 2.4.9.3 distribution if you are not prepared to deal with this. I'll be adjusting this further in the coming weeks based on feedback and through tests on the platforms I use myself, and am soliciting patches & suggestions on how to further improve things. Simplifications would be even better, especially drastic ones! -jcw From gary.h.merrill@gsk.com Sun Mar 7 18:35:29 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1FA79B3AEA for ; Sun, 7 Mar 2004 18:35:28 +0100 (CET) Received: from gate.ggr.co.uk (gate.ggr.co.uk [193.128.25.10]) by dizzie.triqs.com (Postfix) with ESMTP id EC8B1B3AEC for ; Sun, 7 Mar 2004 18:35:15 +0100 (CET) From: gary.h.merrill@gsk.com Received: (uucp@localhost) by gate.ggr.co.uk; Sun, 7 Mar 2004 17:39:34 GMT Received: from uk-stv-mhub1.ggr.co.uk(147.184.2.7) by gb-stv-fw2.ggr.co.uk via smap (V5.5) id xma008616; Sun, 7 Mar 04 17:39:00 GMT Received: from ukz1129.ggr.co.uk (ukz1129.ggr.co.uk [147.184.2.22]) by mailhub.ggr.co.uk; Sun, 7 Mar 2004 17:34:41 GMT Received: from 166.71.155.104 by ukz1128.ggr.co.uk with ESMTP ( Tumbleweed SMTP Relay); Sun, 07 Mar 2004 17:34:36 +0100 Received: from sbhsnsm1.sb.com (sbhsnsm1 [166.71.155.16]) by gbgms04.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i27HYZu19168 for ; Sun, 7 Mar 2004 17:34:35 GMT To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Sun, 7 Mar 2004 12:35:24 -0500 X-MIMETrack: Serialize by Router on SBHSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 07/03/2004 17:33:11 MIME-Version: 1.0 X-WSS-ID: 6C5582A6981194-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Storing views derived from multiple storages X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2004 17:35:29 -0000 I am now attempting to make better use of Metakit's capabilities and in doing so realized that I have really confused myself. At one point I take a set of views and union them. But each view is from a different storage! Apparently Metakit doesn't care about this, and that makes sense. But to which storage does the derived view belong? Well, I suppose it belongs to whichever one you use to do the final view operation. Fine. But suppose that I want to do such a complex operation on views drawn from multiple storages and then put that result into yet another storage. What is the best way to do this? I can create the new storage and the new view by using getas(). But how do I populate that new view with the result of my operations. That is, how do I "copy" my result into a view associated with the new storage? View.copy() won't work since it's view-relative. In general, how does one copy a view from one storage to another? I can see only two alternatives here: 1. Create the new storage and (empty) view first, and then its methods to create the derived view. 2. Do a row by row copy of the derived view to the new storage and view after you create them. The second alternative is not good in terms of performance. The first is at least a bit cumbersome since it requires that you do this for each such derived view, even though at some later point you might decide not to store the derived view. So you would then have to delete the storage you initially created. Is there an alternative that I am missing? If not, I'll go with 1. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From gary.h.merrill@gsk.com Sun Mar 7 18:40:02 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D3B91B3AEA for ; Sun, 7 Mar 2004 18:40:00 +0100 (CET) Received: from gate1.ggr.co.uk (gate1.ggr.co.uk [193.128.25.13]) by dizzie.triqs.com (Postfix) with ESMTP id 2B106B3AEC for ; Sun, 7 Mar 2004 18:39:49 +0100 (CET) From: gary.h.merrill@gsk.com Received: (uucp@localhost) by gate1.ggr.co.uk; Sun, 7 Mar 2004 17:39:35 GMT Received: from uk-stv-mhub1.ggr.co.uk(147.184.2.7) by gb-stv-fw3.ggr.co.uk via smap (V5.5) id xma026902; Sun, 7 Mar 04 17:38:48 GMT Received: from ukz1129.ggr.co.uk (ukz1129.ggr.co.uk [147.184.2.22]) by mailhub.ggr.co.uk; Sun, 7 Mar 2004 17:39:02 GMT Received: from 166.71.155.104 by ukz992.ggr.co.uk with ESMTP (Tumbleweed SMTP Relay); Sun, 07 Mar 2004 17:41:40 +0100 Received: from sbhsnsm1.sb.com (sbhsnsm1 [166.71.155.16]) by gbgms04.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i27Hctu20079 for ; Sun, 7 Mar 2004 17:38:55 GMT To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Sun, 7 Mar 2004 12:39:42 -0500 X-MIMETrack: Serialize by Router on SBHSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 07/03/2004 17:37:30 MIME-Version: 1.0 X-WSS-ID: 6C55815E964042-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Re: Storing views derived from multiple storages X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2004 17:40:02 -0000 Better? combinedView = ... result of operations on views from multiple storages ... newStorage = storage(filename, mode) newView = newStorage.getas( ... appropriate description...) newView.union(combinedView) newStorage.commit() -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From gary.h.merrill@gsk.com Mon Mar 8 01:13:05 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id CD6E6B3AF0 for ; Mon, 8 Mar 2004 01:13:03 +0100 (CET) Received: from usrtpnp1.glaxowellcome.com (firewall1.glaxowellcome.com [192.58.204.204]) by dizzie.triqs.com (Postfix) with ESMTP id 99E11B3AEC for ; Mon, 8 Mar 2004 01:12:45 +0100 (CET) Received: by usrtpnp1.glaxowellcome.com; id TAA26572; Sun, 7 Mar 2004 19:12:44 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma026464; Sun, 7 Mar 04 19:12:09 -0500 Received: by ussunt3.glaxo.com id TAA11907; Sun, 7 Mar 2004 19:12:07 -0500 (EST) Received: from 166.71.134.118 by us9n53.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Sun, 07 Mar 2004 19:12:06 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i280C5B25526 for ; Sun, 7 Mar 2004 19:12:05 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Sun, 7 Mar 2004 19:12:51 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 03/07/2004 07:12:15 PM MIME-Version: 1.0 X-WSS-ID: 6C5565DC1554883-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Product of views with same description X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2004 00:13:06 -0000 I am trying to optimize some of my search strategies and one way to do this involves creating a product of two or more views that have the same description. In this case, view.product() just creates the union. Not surprising. It's not actually a product, but what's it supposed to do? :-) The obvious way around this is to apply "isomorphic" but non-identical descriptions to the views and then apply view.product() to the results. Is there a quick way of creating a derived view that differs only in the property names? Currently I have a loop that just calls view.rename() for each property (apparently creating a new view each time), and this seems rather inefficient. Or is it not inefficient? Or is there a better way to do this? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From jcw@equi4.com Mon Mar 8 12:40:11 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 477A9B3AED for ; Mon, 8 Mar 2004 12:40:11 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 9E0C2B3AEC for ; Mon, 8 Mar 2004 12:40:08 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 3A15F83C26; Mon, 8 Mar 2004 12:40:07 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id AA9FF83914 for ; Mon, 8 Mar 2004 12:39:57 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <33D23BE8-7033-11D8-80D8-000A9588127E@equi4.com> References: <33D23BE8-7033-11D8-80D8-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <53070DFA-70F5-11D8-942A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] libtool has been removed Date: Mon, 8 Mar 2004 12:39:56 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2004 11:40:11 -0000 Me penned: > I've removed libtool from the Metakit build setup and checked in > changes to CVS. > > The changes are very preliminary - this build is likely to work on > less platforms than before. Use the 2.4.9.3 distribution if you are > not prepared to deal with this. Default shared libs now work on Linux, Mac OS X, and FreeBSD. The latter needs "LD_LIBRARY_PATH=. make test" to find the uninstalled shared library. My fingers are itching to throw out all the autoconf stuff as well, just leaving a simple hand-coded unix/configure script. No cross-compiles, and the Tcl/Python builds delegated to TEA and distutils, respectively. The following unix/configure would allow building in an arbitrary directory: d=`dirname $0` m=$d/makefile-${1-platform} if [ -f $m ] then echo topdir = $d/.. > Makefile cat $m >> Makefile echo "Makefile for '$1' installed. Type 'make'." else echo "$m: not found" echo "usage: $0 platform" exit 1 fi Then, anyone can submit a personal makefile and I just collect them for convenience. I'm pretty disillusioned by the autoconf/libtool approach. No doubt it can do magic in the right hands - but not in mine, and if these tools depend on volunteers to step forward for each OSS project on this planet, then let's not hold our breath... -jcw From bkelley@wi.mit.edu Tue Mar 9 16:53:25 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E7F68B3AF4 for ; Tue, 9 Mar 2004 16:53:24 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id A0021B3AF0 for ; Tue, 9 Mar 2004 16:53:21 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004030910531620994 for ; Tue, 09 Mar 2004 10:53:16 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HUBG4S00.HEO; Tue, 9 Mar 2004 10:53:16 -0500 Message-ID: <404DE8C3.6090706@wi.mit.edu> Date: Tue, 09 Mar 2004 10:54:43 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: David McNab , Metakit mailing list Subject: Re: [Metakit] Web deployment questions References: <40467672.60006@rebirthing.co.nz> In-Reply-To: <40467672.60006@rebirthing.co.nz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2004 15:53:25 -0000 David McNab wrote: > Hi, > > I had been looking to use SQLObject as the database backend for an > active web framework I'm building, but have had a fresh look at > Metakit, and see a lot of positives. I'm actually writing an SQLObject backend for metakit. About 90% of the work is done, by the way. > > > However, I'm unclear about some Metakit issues, and hope you might be > able to enlighten me: I also maintain the roundup metakit back (which is an issue tracker) end which I have been using here at the whitehead with good success. I suggest you down load the roundup sources and take a look at how they have implemented their database structures. I was really impressed with their hyper-db setup which handles all of the concerns that you have. You might even be able to easily create a roundup template for your website without too much trouble. They have trigger mechanisms, email-support and the like with support for multiple database back-ends. http://roundup.sourceforge.net/ Brian From bkelley@wi.mit.edu Tue Mar 9 17:02:51 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 37913B3AF4 for ; Tue, 9 Mar 2004 17:02:51 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id A179BB3AF0 for ; Tue, 9 Mar 2004 17:02:47 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004030911024121134 for ; Tue, 09 Mar 2004 11:02:41 -0500 Received: from wi.mit.edu (DNSSPOOFER [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HUBGKH00.JFQ; Tue, 9 Mar 2004 11:02:41 -0500 Message-ID: <404DEAF8.8010508@wi.mit.edu> Date: Tue, 09 Mar 2004 11:04:08 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com, Metakit mailing list Subject: Re: [Metakit] Product of views with same description References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2004 16:02:51 -0000 gary.h.merrill@gsk.com wrote: >I am trying to optimize some of my search strategies and one way to do this >involves creating a product of two or more views that have the same >description. In this case, view.product() just creates the union. Not >surprising. It's not actually a product, but what's it supposed to do? :-) > Well, it is still a product but if two views have the same property name and you can't have two columns with the same name then one will be dropped. What MKSql does is renames any column that is in both tables. MKSql keeps track of the original view names, but you could just append "A" for the first table and "B" for the second table. Here is an example: viewa = db.getas(...) viewb = db.getas(...) for prop in viewa.properties(): if prop in viewb.properties(): viewa = viewa.rename(prop, "A.%s"%prop) viewb = viewb.rename(prop, "B.%s"%prop) prod = viewa.product(viewb) of course, your queries will have to deconvolute the answers... Have you thought about actually using MKSql as the query language? Try downloading KitViewer and trying some sql statements. It is all run by MKSql so you can get a taste. One of the nice things about MKSql is that it runs on arbitrary metakit databases. It won't work with data stored in metakit subviews though i.e. v = db.getas("test[a:S,number:I,subview[name:S,last:S]]") You won't be able to query subview here. Download kitviewer here: http://jura.wi.mit.edu/people/kelley/ In anycase, browsing the source of mksql can be eye opening. Brian Kelley > >The obvious way around this is to apply "isomorphic" but non-identical >descriptions to the views and then apply view.product() to the results. Is >there a quick way of creating a derived view that differs only in the >property names? > >Currently I have a loop that just calls view.rename() for each property >(apparently creating a new view each time), and this seems rather >inefficient. Or is it not inefficient? > > >Or is there a better way to do this? > >-------------------------------------- >Gary H. Merrill >Director and Principal Scientist, New Applications >Data Exploration Sciences >GlaxoSmithKline Inc. >(919) 483-8456 > > > > >_____________________________________________ >Metakit mailing list - Metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > From bkelley@wi.mit.edu Tue Mar 9 17:15:50 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id DA985B3AF7; Tue, 9 Mar 2004 17:15:49 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 9C7C9B3AF0; Tue, 9 Mar 2004 17:15:46 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004030911154121338 ; Tue, 09 Mar 2004 11:15:42 -0500 Received: from wi.mit.edu (DNSSPOOFER [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HUBH6500.9FW; Tue, 9 Mar 2004 11:15:41 -0500 Message-ID: <404DEE04.1010702@wi.mit.edu> Date: Tue, 09 Mar 2004 11:17:08 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler Subject: Re: [Metakit] problems with hash view in 2.4.9.3 References: <5.1.0.14.0.20040225171000.01c2e7f0@131.103.204.244> <4858BADA-6D0D-11D8-8A9A-000A9588127E@equi4.com> In-Reply-To: <4858BADA-6D0D-11D8-8A9A-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2004 16:15:50 -0000 I don't know if this helps, but the error seems to be dependent on the column name 'email_sender' This is pretty weird... s=metakit.storage() fails = "t[url_hash:I,email_sender:S]" works = "t[url_hash:I,pizza_sender:S]" if 1: struc = fails else: struc = works v=s.getas(struc) hv=s.getas('hv[_H:I,_R:I]') v=v.hash(hv,1) new_vals={'url_hash':1, 'email_sender':'A', 't':'A',} print v.append(new_vals) print v.append(new_vals) print v.append(new_vals) print v.append(new_vals) print len(v) metakit.dump(v) From jcw@equi4.com Tue Mar 9 17:26:43 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 99FB0B3AFA for ; Tue, 9 Mar 2004 17:26:43 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 0316FB3AF7 for ; Tue, 9 Mar 2004 17:26:41 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 2E61683C26; Tue, 9 Mar 2004 17:26:39 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 1AE8883914 for ; Tue, 9 Mar 2004 17:26:27 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <404DEE04.1010702@wi.mit.edu> References: <5.1.0.14.0.20040225171000.01c2e7f0@131.103.204.244> <4858BADA-6D0D-11D8-8A9A-000A9588127E@equi4.com> <404DEE04.1010702@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <82A362F8-71E6-11D8-8E9A-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] problems with hash view in 2.4.9.3 Date: Tue, 9 Mar 2004 17:26:25 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2004 16:26:43 -0000 Brian Kelley wrote: > I don't know if this helps, but the error seems to be dependent on the > column name 'email_sender' This is pretty weird... > > s=metakit.storage() > fails = "t[url_hash:I,email_sender:S]" > works = "t[url_hash:I,pizza_sender:S]" > > if 1: > struc = fails field = 'email_sender' > else: > struc = works field = 'pizza_sender' > v=s.getas(struc) > hv=s.getas('hv[_H:I,_R:I]') > v=v.hash(hv,1) > new_vals={'url_hash':1, > 'email_sender':'A', field:'A', > 't':'A',} > print v.append(new_vals) > print v.append(new_vals) > print v.append(new_vals) > print v.append(new_vals) > print len(v) > metakit.dump(v) Uh, oh. Dict is used as sequence. Key order changes. To see it, add: for i in new_vals: print i -jcw From bkelley@wi.mit.edu Tue Mar 9 17:35:01 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id CE05AB3AF9; Tue, 9 Mar 2004 17:35:00 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id A3278B3AF0; Tue, 9 Mar 2004 17:34:57 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004030911345321619 ; Tue, 09 Mar 2004 11:34:53 -0500 Received: from wi.mit.edu (DNSSPOOFER [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HUBI2500.2GX; Tue, 9 Mar 2004 11:34:53 -0500 Message-ID: <404DF284.3000001@wi.mit.edu> Date: Tue, 09 Mar 2004 11:36:20 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler , Metakit mailing list Subject: Re: [Metakit] problems with hash view in 2.4.9.3 References: <5.1.0.14.0.20040225171000.01c2e7f0@131.103.204.244> <4858BADA-6D0D-11D8-8A9A-000A9588127E@equi4.com> <404DEE04.1010702@wi.mit.edu> <82A362F8-71E6-11D8-8E9A-000A9588127E@equi4.com> In-Reply-To: <82A362F8-71E6-11D8-8E9A-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2004 16:35:01 -0000 Jean-Claude Wippler wrote: > Uh, oh. Dict is used as sequence. Key order changes. Crap. this needs to be fixed ASAP. I'll download the code now and make a patch unless you want to contact Nicholas. I haven't run into this before because I tend to just use sequences for appending. I'll spend some time today making some critical-issue regression tests for metakit on the python side. I suggest pulling 2.4.9.3 for python until further notice. Brian. From jcw@equi4.com Tue Mar 9 18:30:53 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2D0AFB3AF4 for ; Tue, 9 Mar 2004 18:30:53 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 219BCB3AF0 for ; Tue, 9 Mar 2004 18:30:50 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 3A18A83C26; Tue, 9 Mar 2004 18:30:48 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 8DABD83914 for ; Tue, 9 Mar 2004 18:30:45 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) Content-Transfer-Encoding: 7bit Message-Id: <7EDBA687-71EF-11D8-8E9A-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Tue, 9 Mar 2004 18:30:44 +0100 X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] new Tcl build structure X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2004 17:30:53 -0000 While repairing the damage caused by removing libtool from the MK build process, I came up with what I think is a better way to deal with all the language bindings of Metakit. Have started implementing it for Tcl / Mk4tcl. The basic idea is to first build the core library in the builds/ directory, as before, possibly followed by running the regression test suite. Once that build is done, *leave* the object files there, and go to the respective language area to finish the job by building the extension there. This will re-use the object code generated from the initial core build, and link it all into the extension. Reasons for this unusual approach: - the core gets built first, and can be independently verified - extensions can adopt whatever the norm is for that language - no need to bring all the C++ config.h logic into extension builds - the result does not need a MK shared lib, since it includes it I've just checked new files into the tcl/ area of MK's CVS. It uses Tcl's standard "TEA" and is derived from Tcl's sample extension. The benefits so far is that the extension config logic is truly simple, all it does is link in a bunch of extra .o files from ${srcdir}/../builds/. Had to put "CC=g++" into the environment to make TEA work with C++. Also had to force using autoconf >= 2.5 on Gentoo (with "WANT_AUTOCONF_2_5=1", yuck). The result is a shared lib called "libMk4tcl2.4.9.3.so", and conveniences such as installing in the right place and with a suitably constructed pkgIndex.tcl file. The basic build logic should be: cd builds ../unix/configure make mkdir tcl cd tcl ../../tcl/configure --with-tcl=... make I hope to help do the same for Python / Mk4py and distutils. The one issue this approach introduces, is that the core library must be built first - with the same settings as the extension (shared vs. static, debug vs. non-debug, etc). It'll take a while to get these combinations right, and to document the new approach. The base configure scripts have not changed yet, but I think the libtool removal broke all scripting language bindings anyway. If you can't be bothered with any of this, use the 2.4.9.3 source distribution for now. -jcw From bkelley@wi.mit.edu Thu Mar 11 16:34:56 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D72BAB3AEB for ; Thu, 11 Mar 2004 16:34:55 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 2A424B3A8B for ; Thu, 11 Mar 2004 16:34:52 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004031110344615205 for ; Thu, 11 Mar 2004 10:34:46 -0500 Received: from wi.mit.edu (DNSSPOOFER [18.157.5.236]) byESMTP id HUF4LZ00.Q5Y for ; Thu, 11 Mar 2004 10:34:47 -0500 Message-ID: <4050876B.1080808@wi.mit.edu> Date: Thu, 11 Mar 2004 10:36:11 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] On making metakit more ACIDic X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2004 15:34:56 -0000 A part of this is just slight rambling, so bear with me. I'm looking for comments so feel free to tell me how you really feel :) One of the problems with metaserver.py is that each append/delete commit/rollback are applied globally which makes it slightly difficult to use in a client/server model. To solve a portion of this problem, I have been working on a layered view. Here is the basic idea: combined view ---------------- | server view | | -------------| ==> acts just like a metakit view | client view | ---------------- | deleted view | ---------------- New data gets appended to the client view. Mapping views check the client view first (for new updates) and then check the server view. For example, if there is a hash collision, it gets appended to the client view and subsequent hash lookups check the client view first then the server view. Deletes are a little trickier since metakit allows rows with identical values (some would claim that this violates relational algebra, but I think it's useful). My current approach is that deletes (not appends) lock the view so other users will not be able to delete until the first user does a commit or rollback. This allows me to simply mark server rows that will not be accessible in the combined view. During a commit, these rows are deleted and then the client view is appended. I will probably need some help with the logic of this one. The only new views that I have had to add is a row/column level locking table to control access. All the combined view is written in python and manages the two views acting as a single view. The code is fairly trivial and doesn't seem to affect speed too much. I currently do have a problem with operations between combined views, but operations between normal views and combined views work like a champ. I had a couple of thoughts based on this experiment. 1) Having a view level commit would be incredibly useful. storage.commit(viewname) or view.commit() would save a *lot* of headache. I have been slowly looking over the metakit code and speaking from a position of almost total ignorance I think this might be a little complicated. 2) The combined view is really a special form of a blocked view. I think (again) from a position of complete ignorance, that it would be fairly easy to generate one of these views using the blocked view code with a modicum of change. (i.e. appends go to the second view and lookups check the second view first then the first view) I've actually used Christian Tismer's code for inspiration for the combined view. I've mentioned this before, but I can even create a combined view using views from different storages! All in all, given the fact that I have been learning as I'm going, I think that metakit is really, really close to providing almost full ACID support. Am I kidding myself here? Brian From njriley@shrug.csl.uiuc.edu Fri Mar 12 02:14:06 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4126AB3AE9; Fri, 12 Mar 2004 02:14:06 +0100 (CET) Received: from shrug.csl.uiuc.edu (shrug.csl.uiuc.edu [130.126.137.135]) by dizzie.triqs.com (Postfix) with ESMTP id 9D285B3AE6; Fri, 12 Mar 2004 02:14:02 +0100 (CET) Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.10/8.12.9) with ESMTP id i2C1E1AO005769; Thu, 11 Mar 2004 19:14:01 -0600 (CST) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.10/8.12.9/Submit) id i2C1Dvqd005768; Thu, 11 Mar 2004 19:13:57 -0600 (CST) (envelope-from njriley) Date: Thu, 11 Mar 2004 19:13:57 -0600 From: Nicholas Riley To: Brian Kelley Subject: Re: [Metakit] problems with hash view in 2.4.9.3 Message-ID: <20040312011357.GA5107@uiuc.edu> Mail-Followup-To: Brian Kelley , Jean-Claude Wippler , Metakit mailing list References: <5.1.0.14.0.20040225171000.01c2e7f0@131.103.204.244> <4858BADA-6D0D-11D8-8A9A-000A9588127E@equi4.com> <404DEE04.1010702@wi.mit.edu> <82A362F8-71E6-11D8-8E9A-000A9588127E@equi4.com> <404DF284.3000001@wi.mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <404DF284.3000001@wi.mit.edu> User-Agent: Mutt/1.4.1i X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit mailing list cc: Jean-Claude Wippler X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2004 01:14:07 -0000 On Tue, Mar 09, 2004 at 11:36:20AM -0500, Brian Kelley wrote: > Jean-Claude Wippler wrote: > > >Uh, oh. Dict is used as sequence. Key order changes. This may be the case, but I can't see where this is a Python-specific bug then; once a hashed view is established, Mk4py doesn't treat it any different from a regular view. > Crap. this needs to be fixed ASAP. I'll download the code now and make > a patch unless you want to contact Nicholas. I haven't run into this > before because I tend to just use sequences for appending. I've adapted the test framework to handle hashed views and sequences vs. dictionaries (it previously used dictionaries for everything, but didn't test hashed views), and stuck your test in there. I've traced it through the Mk4py layer and can't see anything different happening. If the dictionary order was actually affecting anything, then the wrong elements would be inserted in the columns; yet this doesn't happen. The current state of my work is: 'python setup.py test' will run all the tests, or 'python setup.py test --test-options=-v --single-test=test_hash' will run just the hashed view test. To run the test script under Python 2.3, you need to apply the test.regrtest patch here: OT: While my patch still hasn't been applied, almost 6 months later, I do notice the underlying issue of absolute vs. relative imports is being addressed: -- =Nicholas Riley | From jcw@equi4.com Fri Mar 12 12:42:23 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0941FB3AE8 for ; Fri, 12 Mar 2004 12:42:23 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 9DC3EB3AE6 for ; Fri, 12 Mar 2004 12:42:20 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 60FF483C26; Fri, 12 Mar 2004 12:42:18 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B785983914 for ; Fri, 12 Mar 2004 12:42:13 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v612) In-Reply-To: <4050876B.1080808@wi.mit.edu> References: <4050876B.1080808@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <4E1D3244-741A-11D8-BBB8-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] On making metakit more ACIDic Date: Fri, 12 Mar 2004 12:42:13 +0100 To: Metakit list X-Mailer: Apple Mail (2.612) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2004 11:42:23 -0000 Brian Kelley wrote: > combined view > ---------------- > | server view | > | -------------| ==> acts just like a metakit view > | client view | > ---------------- > | deleted view | > ---------------- [...] > 1) Having a view level commit would be incredibly useful. Hard. I can only suggest: apply only changes to that view, then commit. I'm not sure what you're after - sometimes a consistent datafile will require changes in more than one view before getting committed (e.g. one view containing totals of another one). > 2) The combined view is really a special form of a blocked view. The whole mechanism of basic view and derived views is reaching its limits. There are *tons* of views one can come up with which could be defined in terms of basic ones. This is one of the areas where language boundaries bite: with a blocked view in C++, Mk4py cannot subclass it somehow, and adjust it for other purposes such as the above. > I've mentioned this before, but I can even create a combined view > using views from different storages! Yes. With modifiable combinations, things become more complex of course - but that's to be expected with updatable derived views (which are not fully supported right now). > All in all, given the fact that I have been learning as I'm going, I > think that metakit is really, really close to providing almost full > ACID support. Am I kidding myself here? It is, to a pretty large extent, I agree. Atomicity: yes, MK uses stable storage. Consistency: yes, same reason, as long as all views are in the same datafile. Isolation: this one needs work, it does not apply in single-user usage. Durability: yes, again stable storage. Isolation requires more than just collecting changes and being careful to apply them in ways which do not affect others too soon. It also needs to deal with coherence & contention, i.e. what if A adds $100 to a bank account, and B does the same: A: start: temp = balance; temp = temp + 100 B: start: temp = balance; temp = temp + 100; balance = temp; commit A: balance = temp; commit Either A's or B's changes have to be prevented or re-calculated. Note also that MK's commit-aside is another mechanism to start addressing these issues. The aside file essentially stores the difference between the old and the new state, in a transaction-safe way. This goes even further than the above cases, because commit-aside also properly covers all storage re-structuring changes. Commit aside works at the column level, with the potential to be particularly compact and efficient. The way I think everything can be made to work, is to store changes (similar to a journal) without actually committing (i.e. a "pseudo commit"), until all users are known to have an up to date view of things. For readers/writers which come to the party later, they can open the unmodified original, and each play back the pseudo-commits. IOW a bunch of layered views, as follows: original state | pseudo-commits | uncommitted changes In a client/server context, this can happen either locally or on the server (since there is exactly one unmodified state and one latest pseudo-committed state). All of this can be coded at the scripting level IMO. It looks like I will be able to explore exactly such an approach in an upcoming Tcl project. My agony continues because this means that solving it in one language will not make it usable with any other language binding! -jcw From bkelley@wi.mit.edu Fri Mar 12 16:39:39 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3207BB3AE6; Fri, 12 Mar 2004 16:39:39 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id DE45EB3AE5; Fri, 12 Mar 2004 16:39:35 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004031210393028349 ; Fri, 12 Mar 2004 10:39:30 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HUGZHV00.B0N; Fri, 12 Mar 2004 10:39:31 -0500 Message-ID: <4051DA06.3030501@wi.mit.edu> Date: Fri, 12 Mar 2004 10:40:54 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler Subject: Re: [Metakit] On making metakit more ACIDic References: <4050876B.1080808@wi.mit.edu> <4E1D3244-741A-11D8-BBB8-000A9588127E@equi4.com> In-Reply-To: <4E1D3244-741A-11D8-BBB8-000A9588127E@equi4.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2004 15:39:39 -0000 Jean-Claude Wippler wrote: >> 1) Having a view level commit would be incredibly useful. > > > Hard. I can only suggest: apply only changes to that view, then commit. > > I'm not sure what you're after - sometimes a consistent datafile will > require changes in more than one view before getting committed (e.g. > one view containing totals of another one). I suppose what I'm getting at is that if two users are working on different tables and one user commits, the changes the other user has should not be commited, i.e. they can't be rolled back. I'm second guessing this though. The combined view that I'm using already has this property so I'm not so sure that it is necessary anymore. >> 2) The combined view is really a special form of a blocked view. > > > The whole mechanism of basic view and derived views is reaching its > limits. There are *tons* of views one can come up with which could be > defined in terms of basic ones. This is one of the areas where > language boundaries bite: with a blocked view in C++, Mk4py cannot > subclass it somehow, and adjust it for other purposes such as the above. This is not insurmountable, this might just be a function of the Mk4py wrapper. The boost C++ wrappers, or even swig these days, allow python to subclass C++ classes although I see your point, a whole lot more of the meat would have to be exposed to be able to do this. >> I've mentioned this before, but I can even create a combined view >> using views from different storages! > > > Yes. With modifiable combinations, things become more complex of > course - but that's to be expected with updatable derived views (which > are not fully supported right now). My python wrapper takes care of this for now. >> All in all, given the fact that I have been learning as I'm going, I >> think that metakit is really, really close to providing almost full >> ACID support. Am I kidding myself here? > > > It is, to a pretty large extent, I agree. > > Atomicity: yes, MK uses stable storage. > Consistency: yes, same reason, as long as all views are in the same > datafile. > Isolation: this one needs work, it does not apply in single-user > usage. > Durability: yes, again stable storage. > > Isolation requires more than just collecting changes and being careful > to apply them in ways which do not affect others too soon. It also > needs to deal with coherence & contention, i.e. what if A adds $100 to > a bank account, and B does the same: > A: start: temp = balance; temp = temp + 100 > B: start: temp = balance; temp = temp + 100; balance = temp; commit > A: balance = temp; commit > Either A's or B's changes have to be prevented or re-calculated. I'm doing this now using row level locking. It's pretty easy in metakit and adds negligible overhead. I just add a lock column which is 0/1 Of course, I run into problems if the client crashes somehow and leaves rows locked, so I probably need a more sophisticated mechanism. On the plus side, these are really easy to implement in metakit. Already I have foreign keys, transaction implementations on views, triggers, internal functions and commit/rollback. MySQL took what, five years to do this :) > Note also that MK's commit-aside is another mechanism to start > addressing these issues. The aside file essentially stores the > difference between the old and the new state, in a transaction-safe > way. This goes even further than the above cases, because > commit-aside also properly covers all storage re-structuring changes. > Commit aside works at the column level, with the potential to be > particularly compact and efficient. > > The way I think everything can be made to work, is to store changes > (similar to a journal) without actually committing (i.e. a "pseudo > commit"), until all users are known to have an up to date view of > things. For readers/writers which come to the party later, they can > open the unmodified original, and each play back the pseudo-commits. > IOW a bunch of layered views, as follows: | original state | | | pseudo-commits | | | uncommitted changes This is pretty much the technique that I am using in the Combined View. The client provides a data-playground that keeps track of changes to the base views. This I see as sort of a multi-user commit-aside. During a commit, the data is sent to the server, checked for collisions and the like. The upside is that this is easily implemented, the downside is that data-changes are slower than normal. Of course this brings up the question of can metakit support multiple aside files, each from a different user? The answer, obviously is no, but intriguing. > All of this can be coded at the scripting level IMO. It looks like I > will be able to explore exactly such an approach in an upcoming Tcl > project. My agony continues because this means that solving it in one > language will not make it usable with any other language binding! The methodology will be in-place though, that is what rapid prototyping is for. Since I know what is around the corner, I expect this will be a short agony :) > > > -jcw > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From MAILER-DAEMON Sat Mar 13 12:00:40 2004 Return-Path: <> X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 59D3BB3AE5 for ; Sat, 13 Mar 2004 12:00:40 +0100 (CET) Received: from mail.improvision.com (unknown [62.232.123.113]) by dizzie.triqs.com (Postfix) with ESMTP id E29C8B3AE6 for ; Sat, 13 Mar 2004 12:00:10 +0100 (CET) Received: from ([80.190.241.133]) by mail.improvision.com (Merak 7.2.0) with SMTP id KJA74551 for ; Sat, 13 Mar 2004 11:00:10 -0000 Date: Sat, 13 Mar 2004 11:00:10 -0000 From: To: Message-Id: <812472325@mail.improvision.com> X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Re: Metakit Digest, Vol 5, Issue 9 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2004 11:00:40 -0000 I will be away on vacation from 12 March 2004 until 21 March 2004. I will be back in the office on Monday 22 March. If your enquiry is urgent, please contact Carol Robinson Cheers, Steve Stephen Baxter Software Development Manager Improvision steve@improvision.com From Jamshed.Qureshi@umit.com Mon Mar 15 08:15:20 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4A653B3AEC for ; Mon, 15 Mar 2004 08:15:20 +0100 (CET) Received: from umitdc.umit.com (unknown [212.15.17.4]) by dizzie.triqs.com (Postfix) with SMTP id D0522B3AEA for ; Mon, 15 Mar 2004 08:15:18 +0100 (CET) Received: from umitdc.umit.com ([10.1.1.5]) by umitdc.umit.com (SAVSMTP 3.1.2.35) with SMTP id M2004031509110111181 for ; Mon, 15 Mar 2004 09:11:01 +0200 Received: from umit.com ([10.1.1.31]) by umitdc.umit.com with Microsoft SMTPSVC(5.0.2195.6713); Mon, 15 Mar 2004 09:11:01 +0200 Message-ID: <40555919.8050701@umit.com> Date: Mon, 15 Mar 2004 09:19:53 +0200 From: Jamshed Qureshi User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit list References: <7EDBA687-71EF-11D8-8E9A-000A9588127E@equi4.com> In-Reply-To: <7EDBA687-71EF-11D8-8E9A-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 15 Mar 2004 07:11:01.0781 (UTC) FILETIME=[AC561C50:01C40A5C] X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Metakit used on millions of desktops X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 07:15:20 -0000 Hello everyone, I'm also a fan of metakit mostly is use Mk4Tcl. I would like to ask if anyone has any clue: On the metakit web site i see a comment that says: "This library is in active use in various commercial projects and products on millions of desktops." http://www.equi4.com/metakit.html millions of computers! does this mean some common product uses metakit on my machines and i dont know about....? Dont misunderstand i love metakit for its compact size and powerful. Best regards, Jamshed From njriley@shrug.csl.uiuc.edu Mon Mar 15 08:25:42 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 53A83B3AEC for ; Mon, 15 Mar 2004 08:25:42 +0100 (CET) Received: from shrug.csl.uiuc.edu (shrug.csl.uiuc.edu [130.126.137.135]) by dizzie.triqs.com (Postfix) with ESMTP id 2B2E5B3AEA for ; Mon, 15 Mar 2004 08:25:04 +0100 (CET) Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.10/8.12.9) with ESMTP id i2F7OqAO051189; Mon, 15 Mar 2004 01:24:52 -0600 (CST) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.10/8.12.9/Submit) id i2F7OpNU051188; Mon, 15 Mar 2004 01:24:51 -0600 (CST) (envelope-from njriley) Date: Mon, 15 Mar 2004 01:24:51 -0600 From: Nicholas Riley To: Jamshed Qureshi Subject: Re: [Metakit] Metakit used on millions of desktops Message-ID: <20040315072451.GA51047@uiuc.edu> Mail-Followup-To: Jamshed Qureshi , Metakit list References: <7EDBA687-71EF-11D8-8E9A-000A9588127E@equi4.com> <40555919.8050701@umit.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <40555919.8050701@umit.com> User-Agent: Mutt/1.4.1i X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 07:25:42 -0000 On Mon, Mar 15, 2004 at 09:19:53AM +0200, Jamshed Qureshi wrote: > Hello everyone, > I'm also a fan of metakit mostly is use Mk4Tcl. I would like to ask if=20 > anyone has any clue: > On the metakit web site i see a comment that says: >=20 > "This library is in active use in various commercial projects and=20 > products on millions of desktops." > http://www.equi4.com/metakit.html >=20 > millions of computers! does this mean some common product uses metakit=20 > on my machines and i dont know about....? Apple's Address Book on Mac OS X uses Metakit for storage, as you can see: [p2:1030] ~/Library/Application Support/AddressBook%ls -l total 1.2M -rw------- 1 nicholas staff 104k Oct 19 16:55 ABPerson.index -rw------- 1 nicholas staff 96k Mar 6 17:07 ABPerson.skIndex -rw------- 1 nicholas staff 268k Mar 6 17:36 AddressBook.data -rw------- 1 nicholas staff 267k Mar 6 17:36 AddressBook.data.be= foresave -rw------- 1 nicholas staff 405k Mar 6 17:00 AddressBook.data.pr= evious drwx------ 10 nicholas staff 340 Feb 1 16:06 Images/ [p2:1031] ~/Library/Application Support/AddressBook%python Python 2.3 (#1, Sep 13 2003, 00:49:11)=20 [GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import metakit >>> s =3D metakit.storage('AddressBook.data', 0) >>> d =3D s.description() >>> d 'ABDataType[UID:S,Property:S,_Property:I,Table:S,_Table:I,Type:I,_Type:I],_= ABDataType[_H:I,_R:I],ABInfo[UID:S,Me:S,_Me:I,Synced:I,_Synced:I,Version:I,= _Version:I,LastSavedVersion:S,_LastSavedVersion:I,CreatedVersion:S,_Created= Version:I],_ABInfo[_H:I,_R:I],ABGroup[UID:S,Creation:S,_Creation:I,Modifica= tion:S,_Modification:I,GroupName:S,_GroupName:I],_ABGroup[_H:I,_R:I],ABMail= Recent[UID:S,Last:S,_Last:I,PersonUID:S,_PersonUID:I,Email:S,_Email:I,First= :S,_First:I,MailLastDates:B,_MailLastDates:I],_ABMailRecent[_H:I,_R:I],ABGr= oupContents[UID:S,distribution:B,_distribution:I,MemberUID:S,_MemberUID:I,G= roupUID:S,_GroupUID:I],_ABGroupContents[_H:I,_R:I],ABPerson[UID:S,Organizat= ion:S,_Organization:I,AIMInstant:B,_AIMInstant:I,ICQInstant:B,_ICQInstant:I= ,MSNInstant:B,_MSNInstant:I,Email:B,_Email:I,HomePage:S,_HomePage:I,FirstPh= onetic:S,_FirstPhonetic:I,Middle:S,_Middle:I,MiddlePhonetic:S,_MiddlePhonet= ic:I,Last:S,_Last:I,MaidenName:S,_MaidenName:I,LastPhonetic:S,_LastPhonetic= :I,ABPersonFlags:I,_ABPersonFlags:I,Phone:B,_Phone:I,Modification:S,_Modifi= cation:I,YahooInstant:B,_YahooInstant:I,JobTitle:S,_JobTitle:I,Title:S,_Tit= le:I,Address:B,_Address:I,First:S,_First:I,Nickname:S,_Nickname:I,JabberIns= tant:B,_JabberInstant:I,Note:S,_Note:I,Suffix:S,_Suffix:I,StoredPictureName= :S,_StoredPictureName:I,Creation:S,_Creation:I,Birthday:S,_Birthday:I,ABRel= atedNames:B,_ABRelatedNames:I,ABDepartment:S,_ABDepartment:I,ABDate:B,_ABDa= te:I],_ABPerson[_H:I,_R:I],ABLog[UID:S,DeletedUID:S,_DeletedUID:I],_ABLog[_= H:I,_R:I]' Although since Apple provides an application and several APIs to access this information you never have to know... --=20 =3DNicholas Riley | From Jamshed.Qureshi@umit.com Mon Mar 15 08:31:32 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 90845B3AEC for ; Mon, 15 Mar 2004 08:31:32 +0100 (CET) Received: from umitdc.umit.com (unknown [212.15.17.4]) by dizzie.triqs.com (Postfix) with SMTP id 10228B3AEA for ; Mon, 15 Mar 2004 08:31:10 +0100 (CET) Received: from umitdc.umit.com ([10.1.1.5]) by umitdc.umit.com (SAVSMTP 3.1.2.35) with SMTP id M2004031509265211188 for ; Mon, 15 Mar 2004 09:26:52 +0200 Received: from umit.com ([10.1.1.31]) by umitdc.umit.com with Microsoft SMTPSVC(5.0.2195.6713); Mon, 15 Mar 2004 09:26:52 +0200 Message-ID: <40555CCF.2020807@umit.com> Date: Mon, 15 Mar 2004 09:35:43 +0200 From: Jamshed Qureshi User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Metakit used on millions of desktops References: <7EDBA687-71EF-11D8-8E9A-000A9588127E@equi4.com> <40555919.8050701@umit.com> <20040315072451.GA51047@uiuc.edu> In-Reply-To: <20040315072451.GA51047@uiuc.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable X-OriginalArrivalTime: 15 Mar 2004 07:26:52.0203 (UTC) FILETIME=[E2D4FFB0:01C40A5E] X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 07:31:32 -0000 Thanks Nicholas, Absolutely cool, that does clear my mind. Cheers! :) Nicholas Riley wrote: >On Mon, Mar 15, 2004 at 09:19:53AM +0200, Jamshed Qureshi wrote: > =20 > >>Hello everyone, >>I'm also a fan of metakit mostly is use Mk4Tcl. I would like to ask if = >>anyone has any clue: >>On the metakit web site i see a comment that says: >> >>"This library is in active use in various commercial projects and=20 >>products on millions of desktops." >>http://www.equi4.com/metakit.html >> >>millions of computers! does this mean some common product uses metakit = >>on my machines and i dont know about....? >> =20 >> > >Apple's Address Book on Mac OS X uses Metakit for storage, as you can >see: > >[p2:1030] ~/Library/Application Support/AddressBook%ls -l >total 1.2M >-rw------- 1 nicholas staff 104k Oct 19 16:55 ABPerson.index >-rw------- 1 nicholas staff 96k Mar 6 17:07 ABPerson.skIndex= >-rw------- 1 nicholas staff 268k Mar 6 17:36 AddressBook.data= >-rw------- 1 nicholas staff 267k Mar 6 17:36 AddressBook.data= =2Ebeforesave >-rw------- 1 nicholas staff 405k Mar 6 17:00 AddressBook.data= =2Eprevious >drwx------ 10 nicholas staff 340 Feb 1 16:06 Images/ >[p2:1031] ~/Library/Application Support/AddressBook%python >Python 2.3 (#1, Sep 13 2003, 00:49:11)=20 >[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin >Type "help", "copyright", "credits" or "license" for more information. > =20 > >>>>import metakit >>>>s =3D metakit.storage('AddressBook.data', 0) >>>>d =3D s.description() >>>>d >>>> =20 >>>> >'ABDataType[UID:S,Property:S,_Property:I,Table:S,_Table:I,Type:I,_Type:I= ],_ABDataType[_H:I,_R:I],ABInfo[UID:S,Me:S,_Me:I,Synced:I,_Synced:I,Versi= on:I,_Version:I,LastSavedVersion:S,_LastSavedVersion:I,CreatedVersion:S,_= CreatedVersion:I],_ABInfo[_H:I,_R:I],ABGroup[UID:S,Creation:S,_Creation:I= ,Modification:S,_Modification:I,GroupName:S,_GroupName:I],_ABGroup[_H:I,_= R:I],ABMailRecent[UID:S,Last:S,_Last:I,PersonUID:S,_PersonUID:I,Email:S,_= Email:I,First:S,_First:I,MailLastDates:B,_MailLastDates:I],_ABMailRecent[= _H:I,_R:I],ABGroupContents[UID:S,distribution:B,_distribution:I,MemberUID= :S,_MemberUID:I,GroupUID:S,_GroupUID:I],_ABGroupContents[_H:I,_R:I],ABPer= son[UID:S,Organization:S,_Organization:I,AIMInstant:B,_AIMInstant:I,ICQIn= stant:B,_ICQInstant:I,MSNInstant:B,_MSNInstant:I,Email:B,_Email:I,HomePag= e:S,_HomePage:I,FirstPhonetic:S,_FirstPhonetic:I,Middle:S,_Middle:I,Middl= ePhonetic:S,_MiddlePhonetic:I,Last:S,_Last:I,MaidenName:S,_MaidenName:I,L= astPhonetic:S,_LastPhonetic:I,ABPersonFlags:I,_ABPersonFlags:I,Phone:B,_P= hone:I,Modification:S,_Modification:I,YahooInstant:B,_YahooInstant:I,JobT= itle:S,_JobTitle:I,Title:S,_Title:I,Address:B,_Address:I,First:S,_First:I= ,Nickname:S,_Nickname:I,JabberInstant:B,_JabberInstant:I,Note:S,_Note:I,S= uffix:S,_Suffix:I,StoredPictureName:S,_StoredPictureName:I,Creation:S,_Cr= eation:I,Birthday:S,_Birthday:I,ABRelatedNames:B,_ABRelatedNames:I,ABDepa= rtment:S,_ABDepartment:I,ABDate:B,_ABDate:I],_ABPerson[_H:I,_R:I],ABLog[U= ID:S,DeletedUID:S,_DeletedUID:I],_ABLog[_H:I,_R:I]' > >Although since Apple provides an application and several APIs to >access this information you never have to know... > > =20 > From gary.h.merrill@gsk.com Mon Mar 15 20:40:39 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 24CD9B3AEA for ; Mon, 15 Mar 2004 20:40:39 +0100 (CET) Received: from usrtpnp2.glaxowellcome.com (firewall2.glaxowellcome.com [192.58.204.209]) by dizzie.triqs.com (Postfix) with ESMTP id 11F6AB3A93 for ; Mon, 15 Mar 2004 20:40:28 +0100 (CET) Received: by usrtpnp2.glaxowellcome.com; id OAA00366; Mon, 15 Mar 2004 14:40:26 -0500 (EST) From: Received: from ussunt3.glaxo.com(152.51.63.190) by usrtpnp2.glaxowellcome.com via smap (V5.5) id xma029838; Mon, 15 Mar 04 14:39:24 -0500 Received: by ussunt3.glaxo.com id OAA21710; Mon, 15 Mar 2004 14:39:23 -0500 (EST) Received: from 166.71.134.118 by us9n57.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Mon, 15 Mar 2004 14:39:13 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i2FJd7B09827 for ; Mon, 15 Mar 2004 14:39:07 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Mon, 15 Mar 2004 14:39:55 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 03/15/2004 02:39:19 PM MIME-Version: 1.0 X-WSS-ID: 6C48D9EB2540330-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Some view and property questions X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2004 19:40:39 -0000 I've been experimenting with these things (and looking at some other code, like KitViewer), but I still haven't managed to figure out the answers: 1) Given that I have a view, how can I get what the name of that view is? Certainly, I suppose, this question makes sense only for non-derived views, but it does seem to make sense for them and yet I can't see how to get the view's name from the view itself -- or how to "find" the view's name from anything else in the database together with the view itself. Is there a way to do this? 2) How do I get the list of properties belonging to a view in the order that these were specified in the description defining the view? I think that KitViewer and kviewer do this, but I can't see how they get the properties in this order, and when I try the same (or apparently similar) code, the list of properties I get is not in the order of the original description. -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From bkelley@wi.mit.edu Tue Mar 16 16:22:21 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 93763B3AEE for ; Tue, 16 Mar 2004 16:22:21 +0100 (CET) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id B6047B3A88 for ; Tue, 16 Mar 2004 16:22:18 +0100 (CET) Received: from sydney.wi.mit.edu ([18.157.0.220]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004031610221332237 for ; Tue, 16 Mar 2004 10:22:13 -0500 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HUODD100.NQI; Tue, 16 Mar 2004 10:22:13 -0500 Message-ID: <40571BF2.2020501@wi.mit.edu> Date: Tue, 16 Mar 2004 10:23:30 -0500 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com, Metakit mailing list Subject: Re: [Metakit] Some view and property questions References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 15:22:21 -0000 gary.h.merrill@gsk.com wrote: >I've been experimenting with these things (and looking at some other code, >like KitViewer), but I still haven't managed to figure out the answers: > >1) Given that I have a view, how can I get what the name of that view is? >Certainly, I suppose, this question makes sense only for non-derived views, >but it does seem to make sense for them and yet I can't see how to get the >view's name from the view itself -- or how to "find" the view's name from >anything else in the database together with the view itself. Is there a >way to do this? > The only way I have found is to hijack the getas and view functions using my metawrap metakit wrapper, see http://staffa.wi.mit.edu/people/kelley/metawrap.py. For example: def getas(self, description): name = description.split("[")[0] view = view(vw=self._db.getas(description), st=self) view.name = name return view def view(self, viewname): view = view(vw=self._db.view(viename), st=self) view.name = viewname return view I anyone has a better way, I'm all ears :) >2) How do I get the list of properties belonging to a view in the order >that these were specified in the description defining the view? I think >that KitViewer and kviewer do this, but I can't see how they get the >properties in this order, and when I try the same (or apparently similar) >code, the list of properties I get is not in the order of the original >description. > view.structure() -> returns the list in order of the view's properties. view.properties() returns a dictionary so this by definition is unordered. > >-------------------------------------- >Gary H. Merrill >Director and Principal Scientist, New Applications >Data Exploration Sciences >GlaxoSmithKline Inc. >(919) 483-8456 > > > > >_____________________________________________ >Metakit mailing list - Metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > > From bowmanjm@comcast.net Thu Mar 18 07:57:54 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B32CAB3A93 for ; Thu, 18 Mar 2004 07:57:54 +0100 (CET) Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by dizzie.triqs.com (Postfix) with ESMTP id 946A6B3A88 for ; Thu, 18 Mar 2004 07:57:52 +0100 (CET) Received: from griffon (h002078c644e3.ne.client2.attbi.com[65.96.178.255]) by comcast.net (sccrmhc13) with ESMTP id <2004031806574401600ldn22e>; Thu, 18 Mar 2004 06:57:44 +0000 From: Jeremy Bowman To: metakit@equi4.com Date: Thu, 18 Mar 2004 02:09:27 -0500 User-Agent: KMail/1.6.1 MIME-Version: 1.0 Content-Disposition: inline Content-Type: Multipart/Mixed; boundary="Boundary-00=_nsUWAARCoF+Tl6q" Message-Id: <200403180209.27869.bowmanjm@comcast.net> X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] New build process always generates shared library? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2004 06:57:55 -0000 --Boundary-00=_nsUWAARCoF+Tl6q Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline I tried building Metakit 2.4.9.3 on Mac OS X 10.3.3 recently and got the same error that Jacob Levy encountered on FreeBSD. So I grabbed the latest CVS to get the libtool-less configure script, and that does in fact compile correctly (with a few warnings, in the attached file). However, even if I run configure with "--disable-shared" or "--enable-shared=no" and get: "checking for building as shared lib... no, static" in the output, the generated Makefile produces a shared library. Tried on Linux also, and it has the same problem. Before I waste too much time trying to grok autoconf and such, could somebody more familiar with the build process fix it so that it's possible to make a static version of the library again? Or if I'm just missing something, please point out what it is? Thanks, Jeremy --Boundary-00=_nsUWAARCoF+Tl6q Content-Type: text/plain; charset="us-ascii"; name="warnings.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="warnings.txt" g++ -g -O2 -Dq4_MULTI -fPIC -I../unix/../include -I../unix/../src -I. -o demo \ ../unix/../demos/demo.cpp libmk4.dylib ld: warning prebinding not disabled because (__PAGEZERO segment (address = 0x0 size = 0x1000) of demo overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__TEXT segment (address = 0x1000 size = 0x9000) of demo overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__DATA segment (address = 0xa000 size = 0x1000) of demo overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__LINKEDIT segment (address = 0xb000 size = 0xf000) of demo overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib g++ -g -O2 -Dq4_MULTI -fPIC -I../unix/../include -I../unix/../src -I. -o dump \ ../unix/../demos/dump.cpp libmk4.dylib ld: warning prebinding not disabled because (__PAGEZERO segment (address = 0x0 size = 0x1000) of dump overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__TEXT segment (address = 0x1000 size = 0x9000) of dump overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__DATA segment (address = 0xa000 size = 0x1000) of dump overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__LINKEDIT segment (address = 0xb000 size = 0xf000) of dump overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib g++ -g -O2 -Dq4_MULTI -fPIC -I../unix/../include -I../unix/../src -I. -o myio \ ../unix/../demos/myio.cpp libmk4.dylib ld: warning prebinding not disabled because (__PAGEZERO segment (address = 0x0 size = 0x1000) of myio overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__TEXT segment (address = 0x1000 size = 0x9000) of myio overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__DATA segment (address = 0xa000 size = 0x1000) of myio overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib ld: warning prebinding not disabled because (__LINKEDIT segment (address = 0xb000 size = 0xf000) of myio overlaps with __TEXT segment (address = 0x0 size = 0x2c000) of libmk4.dylib --Boundary-00=_nsUWAARCoF+Tl6q-- From bkelley@wi.mit.edu Thu Mar 18 15:43:56 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D018FB3AE8; Thu, 18 Mar 2004 15:43:55 +0100 (CET) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by dizzie.triqs.com (Postfix) with ESMTP id 75A32B3A88; Thu, 18 Mar 2004 15:43:54 +0100 (CET) Received: from wi.mit.edu (h00080eddd7f1.ne.client2.attbi.com[24.147.173.53]) by comcast.net (rwcrmhc12) with SMTP id <2004031814435101400ja8lme>; Thu, 18 Mar 2004 14:43:51 +0000 Message-ID: <4059B79E.20804@wi.mit.edu> Date: Thu, 18 Mar 2004 09:52:14 -0500 From: Brian Kelley User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030507 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com, Jean-Claude Wippler Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] BUG: metakit blocked ordered views X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2004 14:43:56 -0000 Confused? Me too, this took me a while to figure out. If you have a blocked view with two integer props ordered on the first integer property everything works fine as long as the second column stays positive. As soon as it becomes negative vw.find no longer works. Although it does work on the non-ordered view so go figure. import metakit, random, time st = metakit.storage() blocked = st.getas("test[_B[id:I,v:I]]").blocked() ordered = blocked.ordered(1) data = ((1,1),(2,2), (3,3), (4,-1)) for id, val in data: ordered.append((id, val)) assert blocked.find(id=id) != -1 assert ordered.find(id=id) != -1, metakit.dump(ordered[-1:]) From ch.drube@electronic-thingks.de Thu Mar 18 23:51:54 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 38B03B3AE8 for ; Thu, 18 Mar 2004 23:51:54 +0100 (CET) Received: from ks3.hosteasy.de (unknown [213.131.252.202]) by dizzie.triqs.com (Postfix) with ESMTP id D7629B3AE4 for ; Thu, 18 Mar 2004 23:51:52 +0100 (CET) Received: from dsl-082-082-120-033.arcor-ip.net (dsl-082-082-120-033.arcor-ip.net [82.82.120.33]) (authenticated bits=0)i2IMqF92010468 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO) for ; Thu, 18 Mar 2004 23:52:17 +0100 From: Christoph Drube Organization: Electronic Thingks To: metakit@equi4.com Date: Thu, 18 Mar 2004 23:51:51 +0100 User-Agent: KMail/1.5.3 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200403182351.51359.ch.drube@electronic-thingks.de> X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Mk4tcl - SegFault when using cursors :-( X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2004 22:51:54 -0000 Hi folks, I have massive problems using MetaKit with Tcl (ActiveTcl 8.4.5). I have "connected" a tktable and a view so that all search and sort operations are done in the database and results are copied to tktable. Now I wanted to search a pattern in all "cells" and bring all hits to the top of the view: ---------------------------------------------------------------- # $v = view path # $proplist = list of properties in $v (all props are strings) # $s = search pattern # Searching property by property # Q: Is there a better way to search ? set hits {} foreach i $proplist \ { set hits [concat $hits [mk::select $v -first 2 -glob $i $s]] } # Eliminiate double hits and sort set hits [lsort -unique -integer $hits] # Now move all hits to top of view # To reach this, swap dest and nr and then increment dest # dest starts with row 2 because row0 keeps the row titles and # row1 the column width in pixels set dest 2 foreach nr $hits \ { # Save destination row content to temporary row eval "set cur1 \[mk::row create [mk::get $v!$dest]\]" # Copy the hit there mk::row replace $v!$dest $v!$nr # And copy now the "clipboard" to the old hit row mk::row replace $v!$nr $cur1 # Set the pointer to next row from top incr dest } # Write all values from db to table UpdateTable # And select search results $table selection set 1,0 [llength $hits],[$table cget -cols] ------------------------------------------------------------------ Well, when swapping the rows, this script crashes with seg fault. I had a look at the row contents and the search results - all is fine, but after the second or third iteration over nr it always crashes :-/ What I'm doing wrong? Have I misunderstood the mk::cursor command or their use? It really drives me to despair cause it's not the first seg fault with mk4tcl - isn't it possible to use property names with blanks? Any help would be great! Best regards, Christoph (frustrated) :-( From jcw@equi4.com Thu Mar 18 23:56:56 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 306EDB3AE8 for ; Thu, 18 Mar 2004 23:56:56 +0100 (CET) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 4B9A7B3AE4 for ; Thu, 18 Mar 2004 23:56:54 +0100 (CET) Received: by madder.xs4all.nl (Postfix, from userid 101) id 9D0ED83C26; Thu, 18 Mar 2004 23:56:51 +0100 (CET) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B339883914 for ; Thu, 18 Mar 2004 23:56:33 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v613) Content-Transfer-Encoding: 7bit Message-Id: <80EA6AB1-792F-11D8-AECA-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Thu, 18 Mar 2004 23:56:34 +0100 X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Fwd: [Starkit] Mk4tcl - SegFault when using cursors :-( X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2004 22:56:56 -0000 Begin forwarded message: > From: Jean-Claude Wippler > Date: March 18, 2004 23:54:31 CET > To: starkit list server > Subject: Re: [Starkit] Mk4tcl - SegFault when using cursors :-( > > Christoph Drube wrote: > >> I have massive problems using MetaKit with Tcl (ActiveTcl 8.4.5). > > Sorry to hear that. > >> # Searching property by property >> # Q: Is there a better way to search ? >> set hits {} >> foreach i $proplist \ >> { >> set hits [concat $hits [mk::select $v -first 2 -glob $i $s]] >> } > > mk::select $v -first 2 -glob $proplist $s > >> Well, when swapping the rows, this script crashes with seg fault. >> I had a look at the row contents and the search results - all is >> fine, but >> after the second or third iteration over nr it always crashes :-/ >> >> What I'm doing wrong? Have I misunderstood the mk::cursor command or >> their use? > > Make sure you use Mk4tcl 2.4.9.3 - from the change log: > > 2004-01-22 Fixed refcount problem with temp rows in Mk4tcl > > This was a long-standing bug: "mk::row create" did not work right > because the tracking of temporary rows was completely messed up. > Added test case for Tcl (mk6.8), fixes FB14, BTS#78, and BTS#29. > >> It really drives me to despair cause it's not the first seg fault >> with mk4tcl >> - isn't it possible to use property names with blanks? > > I'm not sure. I always avoid blanks in property names. Tcl has no > restrictions, but identifiers in C++ and Python are limited in the > same way. > >> Christoph (frustrated) :-( > > Ouch. > > -jcw > > _____________________________________________ > Starkit mailing list - Starkit@equi4.com > http://www.equi4.com/mailman/listinfo/starkit From rcohen@articque.com Mon Mar 29 18:32:26 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1500DB3AE3 for ; Mon, 29 Mar 2004 18:32:26 +0200 (CEST) Received: from mwinf0603.wanadoo.fr (smtp6.wanadoo.fr [193.252.22.25]) by dizzie.triqs.com (Postfix) with ESMTP id DFE99B3A8B for ; Mon, 29 Mar 2004 18:32:24 +0200 (CEST) Received: from articque.com (AOrleans-204-1-24-79.w81-250.abo.wanadoo.fr [81.250.166.79]) by mwinf0603.wanadoo.fr (SMTP Server) with ESMTP id 94721240009D for ; Mon, 29 Mar 2004 18:32:23 +0200 (CEST) Message-ID: <40684FFA.5020902@articque.com> Date: Mon, 29 Mar 2004 18:34:02 +0200 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: Metakit list Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] pb memory leak in 2.4.9.3 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Mar 2004 16:32:26 -0000 The following code shows a memory leak. Is this a bug ? (compiled with visual studio.net) Thanks #define CRTDBG_MAP_ALLOC #include #include #include #include void function() { c4_ViewProp subprop("sub"); } int main (int argc, char **argv) { _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );// check memory leak at exit function(); return(0); } -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jcw@equi4.com Mon Mar 29 19:36:53 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3A4ADB3AE6 for ; Mon, 29 Mar 2004 19:36:53 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 79EC6B3AE3 for ; Mon, 29 Mar 2004 19:36:47 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 7D09E83C26; Mon, 29 Mar 2004 19:36:44 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E069E83914 for ; Mon, 29 Mar 2004 19:36:26 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: <40684FFA.5020902@articque.com> References: <40684FFA.5020902@articque.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <9AB8CD1E-81A7-11D8-A0A3-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] pb memory leak in 2.4.9.3 Date: Mon, 29 Mar 2004 19:36:26 +0200 To: Metakit list X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Mar 2004 17:36:53 -0000 Riccardo Cohen wrote: > The following code shows a memory leak. Is this a bug ? > (compiled with visual studio.net) > > Thanks > > #define CRTDBG_MAP_ALLOC > #include > #include > #include > #include > > void function() > { > c4_ViewProp subprop("sub"); > } > > int main (int argc, char **argv) > { > _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );// > check memory leak at exit > function(); > return(0); > } There is a global symbol table for properties (2 allocs), and there is a 3-byte alloc for empty c4_String objects. The global symbol table is only cleaned up in debug mode, it's a once-only leak, so is the 3-byte empty string buffer. So yes, there are up to 3 allocations that may never be cleaned up. -jcw From rcohen@articque.com Tue Mar 30 09:22:10 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 9D746B3AE6 for ; Tue, 30 Mar 2004 09:22:10 +0200 (CEST) Received: from mwinf1002.wanadoo.fr (smtp10.wanadoo.fr [193.252.22.21]) by dizzie.triqs.com (Postfix) with ESMTP id BAB1CB3AE5 for ; Tue, 30 Mar 2004 09:22:09 +0200 (CEST) Received: from articque.com (AOrleans-204-1-2-118.w81-249.abo.wanadoo.fr [81.249.144.118]) by mwinf1002.wanadoo.fr (SMTP Server) with ESMTP id 7B0A01C000E1 for ; Tue, 30 Mar 2004 09:22:08 +0200 (CEST) Message-ID: <40692085.3010307@articque.com> Date: Tue, 30 Mar 2004 09:23:49 +0200 From: Riccardo Cohen Organization: articque User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en, fr MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] pb memory leak in 2.4.9.3 References: <40684FFA.5020902@articque.com> <9AB8CD1E-81A7-11D8-A0A3-000A9588127E@equi4.com> In-Reply-To: <9AB8CD1E-81A7-11D8-A0A3-000A9588127E@equi4.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2004 07:22:11 -0000 OK Thanks Jean-Claude Wippler wrote: > Riccardo Cohen wrote: > >> The following code shows a memory leak. Is this a bug ? >> (compiled with visual studio.net) >> >> Thanks >> >> #define CRTDBG_MAP_ALLOC >> #include >> #include >> #include >> #include >> >> void function() >> { >> c4_ViewProp subprop("sub"); >> } >> >> int main (int argc, char **argv) >> { >> _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );// >> check memory leak at exit >> function(); >> return(0); >> } > > > There is a global symbol table for properties (2 allocs), and there is a > 3-byte alloc for empty c4_String objects. The global symbol table is > only cleaned up in debug mode, it's a once-only leak, so is the 3-byte > empty string buffer. > > So yes, there are up to 3 allocations that may never be cleaned up. > > -jcw > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > -- Riccardo Cohen Articque Les Roches 37230 Fondettes France web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From chihung@mc-3.org Wed Mar 31 05:47:31 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 60E6EB3AE3 for ; Wed, 31 Mar 2004 05:47:31 +0200 (CEST) Received: from aspmail02 (unknown [202.79.221.153]) by dizzie.triqs.com (Postfix) with ESMTP id E899DB3A8B for ; Wed, 31 Mar 2004 05:47:29 +0200 (CEST) Received: from mc-3.org ([192.168.1.153]) by aspmail02.aspcentre.org (iPlanet Messaging Server 5.1 (built May 7 2001)) with SMTP id <0HVF00G5T8YBOS@aspmail02.aspcentre.org> for metakit@equi4.com; Wed, 31 Mar 2004 11:42:12 +0800 (SGT) Date: Wed, 31 Mar 2004 11:44:43 +0800 From: Chan Chi Hung To: Metakit list Message-id: <406A3EAB.4030400@mc-3.org> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] [Fwd: Tcl Wiki in CGI mode] X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2004 03:47:31 -0000 I am trying to run Tcl Wiki on my Redhat 8 and encounter "Internal Server Error" in Apache I am running Apache 1.3.29 with the lastest Tclkit/Wikit/Tcl wiki database. I followed the suggestion described in http://wiki.tcl.tk/24, however the script can only run in standalone GUI mode, but not as CGI via browser. Appreciate if anyone can provide me with more info. Cheers. --Chi Hung From chihung@mc-3.org Wed Mar 31 07:20:31 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4979EB3AE3 for ; Wed, 31 Mar 2004 07:20:31 +0200 (CEST) Received: from aspmail02 (unknown [202.79.221.153]) by dizzie.triqs.com (Postfix) with ESMTP id 7F328B3A8B for ; Wed, 31 Mar 2004 07:20:29 +0200 (CEST) Received: from mc-3.org ([192.168.1.153]) by aspmail02.aspcentre.org (iPlanet Messaging Server 5.1 (built May 7 2001)) with SMTP id <0HVF00H1KD9BFN@aspmail02.aspcentre.org> for metakit@equi4.com; Wed, 31 Mar 2004 13:15:11 +0800 (SGT) Date: Wed, 31 Mar 2004 13:17:41 +0800 From: Chan Chi Hung Subject: Re: [Metakit] [Fwd: Tcl Wiki in CGI mode] In-reply-to: <406A3EAB.4030400@mc-3.org> To: Metakit list Message-id: <406A5475.1050507@mc-3.org> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) References: <406A3EAB.4030400@mc-3.org> X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2004 05:20:31 -0000 I managed to get the wiki home page up ('cos the directory is not writable by 'nobody'), but all the links are referenced to http://.../tcl/wiki/600 which give 'http not found" error. May I know how to tel wikit (or apache) to get the wiki document instead of trying to retrieve a document named "600" Thanks. --Chi Hung Chan Chi Hung wrote: > I am trying to run Tcl Wiki on my Redhat 8 and encounter "Internal > Server Error" in Apache > > I am running Apache 1.3.29 with the lastest Tclkit/Wikit/Tcl wiki > database. I followed the suggestion described in http://wiki.tcl.tk/24, > however the script can only run in standalone GUI mode, but not as CGI > via browser. > > Appreciate if anyone can provide me with more info. Cheers. > --Chi Hung > > > _____________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- CHAN CHI HUNG Managed Computing Competency Centre NTU, MPE, Block N3, #01c-10 50 Nanyang Avenue, Singapore 639798 Tel: +65 6790 4420 (O) Fax: +65 6316 0042 Email: chihung@mc-3.org URL: www.mc-3.org From gary.h.merrill@gsk.com Wed Mar 31 23:18:20 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1EF75B3AE3 for ; Wed, 31 Mar 2004 23:18:20 +0200 (CEST) Received: from usrtpnp1.glaxowellcome.com (firewall1.glaxowellcome.com [192.58.204.204]) by dizzie.triqs.com (Postfix) with ESMTP id D7D9CB3A8B for ; Wed, 31 Mar 2004 23:18:08 +0200 (CEST) Received: by usrtpnp1.glaxowellcome.com; id QAA06726; Wed, 31 Mar 2004 16:18:02 -0500 (EST) From: Received: from ussunt2.glaxo.com(152.51.63.189) by usrtpnp1.glaxowellcome.com via smap (V5.5) id xma006439; Wed, 31 Mar 04 16:17:23 -0500 Received: by ussunt2.glaxo.com id QAA18313; Wed, 31 Mar 2004 16:17:23 -0500 (EST) Received: from 166.71.134.118 by us9n55.glaxo.com with ESMTP (Tumbleweed SMTP Relay); Wed, 31 Mar 2004 16:17:18 -0400 Received: from kopsnsm1.sb.com (kopsnsm1.sb.com [166.71.110.80]) by kopsasmtp01.sb.com (8.11.6+Sun/8.11.6) with ESMTP id i2VLHHB23954 for ; Wed, 31 Mar 2004 16:17:17 -0500 (EST) To: metakit@equi4.com X-Mailer: Lotus Notes Release 5.0.6a January 17, 2001 Message-ID: Date: Wed, 31 Mar 2004 16:18:01 -0500 X-MIMETrack: Serialize by Router on KOPSNSM1/SERVERS/SB_PLC(Release 5.0.10 |March 22, 2002) at 03/31/2004 04:17:28 PM MIME-Version: 1.0 X-WSS-ID: 6C75EAD44229979-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Naming a derived view? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Mar 2004 21:18:20 -0000 Is there a way to name a derived view? If I create a derived view and I later want to make use of it, do I have to keep a reference to it around? If I want to store the derived view, do I need to go through some sequence of steps to create a (named) empty view and then merge that with the derived one? -------------------------------------- Gary H. Merrill Director and Principal Scientist, New Applications Data Exploration Sciences GlaxoSmithKline Inc. (919) 483-8456 From hboeken@lucent.com Tue Apr 6 14:24:52 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id EB2E0B3AE7 for ; Tue, 6 Apr 2004 14:24:51 +0200 (CEST) Received: from auemail2.firewall.lucent.com (auemail2.lucent.com [192.11.223.163]) by dizzie.triqs.com (Postfix) with ESMTP id 86042B3AE5 for ; Tue, 6 Apr 2004 14:24:49 +0200 (CEST) Received: from nbgif1.de.lucent.com (h135-246-31-82.lucent.com [135.246.31.82])ESMTP id i36COi127059 for ; Tue, 6 Apr 2004 07:24:44 -0500 (CDT) Received: from slds1.de.lucent.com (slds1 [135.246.29.35]) i36COgc13295 for ; Tue, 6 Apr 2004 14:24:42 +0200 (MEST) Received: from lucent.com (GE1884PC9518 [135.246.14.139]) by slds1.de.lucent.com (8.11.7p1+Sun/8.8.5) with ESMTP id i36COgP06531 for ; Tue, 6 Apr 2004 14:24:42 +0200 (MEST) Message-ID: <4072A164.50404@lucent.com> Date: Tue, 06 Apr 2004 14:24:04 +0200 From: Hermann Boeken User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031205 Thunderbird/0.4 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Can't build Metakit 2.4.9.3 on SUN Solaris 5.8 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2004 12:24:52 -0000 Hello. I have problems building Metakit 2.4.9.3 on Sun Solaris 5.8. The final step (make install) fails with the following error message: ../unix/scripts/install-sh -c .libs/libmk4.lai /home/hboeken/ucsi.test/Tcl/8.4.4/shared/lib/libmk4.la install: .libs/libmk4.lai does not exist make: *** [install-mk] Error 1 And indeed, the file does not exist. I read Jacob Levy's thread and Jean-Claude's answer regarding similar build problems on FreeBSD and also tried some of Jean-Claude's suggestions, but didn't have any success. The result is always this error message. Has anyone successfully build Metakit 2.4.9.3 on Solaris? Greetings, Hermann From bkelley@wi.mit.edu Tue Apr 6 15:51:29 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2F124B3AE5 for ; Tue, 6 Apr 2004 15:51:29 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 6416AB3AE4 for ; Tue, 6 Apr 2004 15:51:26 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004040609511515795 for ; Tue, 06 Apr 2004 09:51:15 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HVR57T00.221; Tue, 6 Apr 2004 09:52:41 -0400 Message-ID: <4072B5A3.1020307@wi.mit.edu> Date: Tue, 06 Apr 2004 09:50:27 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: gary.h.merrill@gsk.com, Metakit mailing list Subject: Re: [Metakit] Naming a derived view? References: In-Reply-To: X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2004 13:51:29 -0000 gary.h.merrill@gsk.com wrote: > Is there a way to name a derived view? If I create a derived view > and I later want to make use of it, do I have to keep a reference > to it around? If I want to store the derived view, do I need to go > through some sequence of steps to create a (named) empty view and > then merge that with the derived one? > Pretty much. As long as your derived view doesn't have any subviews, the following code should work... def cloneView(storage, view, name, commitBlock=None): """storage, view, name, commitBlock=None -> create a new named view which is a copy of the contents of view. View cannot contain any subviews. commitBlock is optional, if it is not None then commit every commitBlock appends to the new view. This is useful when cloning large views""" structure = [] properties = view.properties().keys() # create the new derived view for p in view.structure(): if p.type == "V": raise ValueError("view contains subviews, cannot clone") structure.append("%s:%s"%(p.name, p.type)) vw = storage.getas("name[%s]"%",".join(structure)) # populate it with the data from view for i, row in enumerate(view): values = {} for p in properties: values[p] = getattr(row, p) vw.append(values) if commitBlock and i % commitBlock == 0: storage.commit() return vw if __name__ == "__main__": import metakit st = metakit.storage() vw = st.getas("test[id:I,a,b,c]") for i in range(100): vw.append((i, 'a','b','c')) vw2 = st.getas("test[id:i,a]") for i in range(100): vw2.append((i, 'a')) vw3 = vw.join(vw2, vw2.id) clone = cloneView(st, vw3, "derived") assert vw3.properties() == clone.properties() assert len(vw3) == len(clone) props = vw3.properties() for r1, r2 in zip(vw3, clone): for p in props: assert getattr(r1, p) == getattr(r2, p) From gparuchuri@oneil.com Tue Apr 6 18:09:56 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id AB899B3AE5 for ; Tue, 6 Apr 2004 18:09:55 +0200 (CEST) Received: from meridius.oneil.com (meridius.oneil.com [208.154.227.50]) by dizzie.triqs.com (Postfix) with ESMTP id 59ACAB3AE4 for ; Tue, 6 Apr 2004 18:09:52 +0200 (CEST) Received: from [192.168.1.20] ([192.168.1.20]:24070 "EHLO daytonnt20.oneilinc.net" ident: "NO-IDENT-SERVICE[2]") by meridius.oneil.com with ESMTP id ; Tue, 6 Apr 2004 12:02:43 -0400 Received: from DAYTONEX.oneilinc.net (unverified) by daytonnt20.oneilinc.net (Content Technologies SMTPRS 4.3.6) with ESMTP id for ; Tue, 6 Apr 2004 12:09:40 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5.6944.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C41BF1.90EFF087" Date: Tue, 6 Apr 2004 12:09:40 -0400 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Select view question. Thread-Index: AcQb8ZDHlAsuNBybS/qFDy0tThZQgw== From: "Goutam Paruchuri" To: X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Select view question. X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2004 16:09:56 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01C41BF1.90EFF087 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Hello ! I have a view called Engines in the database. I would like to define the view and reuse it in several places? Below is a code sample Sample 1 returns zero rows while Sample 2 returns rows.. I would like db.Engines to be a variable name (like tblEngines as defined). =20 mk::file open db mydata.db =20 # create a view within the datafile which describes what we'll store set tblEngines [mk::view layout db.Engines] Sample 1 foreach row [mk::select $tblEngines] { puts [mk::get $tblEngines!$row EngineCode] } =20 Sample 2 foreach row [mk::select db.Engines] { puts [mk::get db.Engines!$row EngineCode] } =20 Thanks ! Goutam =20 =20 ------------------------------------------------------------------------ ------------ "Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe." Sir Albert Einstein ------------------------------------------------------------------------ ------------ Goutam Paruchuri Database Consultant, O'NEIL & ASSOCIATES, INC. http://www.oneil.com =20 495 Byers Rd. Miamisburg, Ohio 45342-3662 Phone: (937) 865-0846 ext. 3051 Fax: (937) 865-5858 =20 =20 Confidentiality Notice The information contained in this e-mail is confidential and intended for u= se only by the person(s) or organization listed in the address. If you have= received this communication in error, please contact the sender at O'Neil = & Associates, Inc., immediately. Any copying, dissemination, or distributio= n of this communication, other than by the intended recipient, is strictly = prohibited. ------_=_NextPart_001_01C41BF1.90EFF087 Content-Type: text/html; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable
Hello=20 !
I have a = view called=20 Engines in the database. I would like to define the view and reuse it in se= veral=20 places? Below is a code sample
Sample 1 = returns=20 zero rows while Sample 2 returns rows..
I would l= ike=20 db.Engines to be a variable name (like tblEngines as=20 defined).
 
mk::file = open db=20 mydata.db
 
# create = a view=20 within the datafile which describes what we'll store
set tblEn= gines=20 [mk::view layout db.Engines]
S= ample=20 1
foreach r= ow=20 [mk::select $tblEngines] {
 puts [mk::get $tblEngines!$row=20 EngineCode]
}
 
Sample=20 2
foreach row [mk::select db.Engines] {
 puts=20 [mk::get db.Engines!$row=20 EngineCode]
}
 
Thanks=20 !
Goutam
 
 
------------------------------------------------------------------= ------------------
"Two things are infinite: the= universe=20 and human stupidity; and I'm not sure about the the universe."
Sir Albert Einstein
------------------------------------------------------------------= ------------------
Goutam=20 Paruchuri
Database= Consultant,
O'NEIL &a= mp;=20 ASSOCIATES, INC. http://www.oneil.com
495 Byers Rd.
Miamisbur= g, Ohio=20 45342-3662
Phone: (937) 865-0846 ext. 3051
Fax: (937) 865-5858=    
 


Confidentiality Notice
The information contained in this e-mail is confidential and intended for u= se only by the person(s) or organization listed in the address. If you have= received this communication in error, please contact the sender at O'Neil = & Associates, Inc., immediately. Any copying, dissemination, or distributio= n of this communication, other than by the intended recipient, is strictly = prohibited.
 ------_=_NextPart_001_01C41BF1.90EFF087-- From mosley@austin.rr.com Tue Apr 13 06:44:06 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id DDC04B3AE8 for ; Tue, 13 Apr 2004 06:44:05 +0200 (CEST) Received: from ms-smtp-02-eri0.texas.rr.com (ms-smtp-02.texas.rr.com [24.93.47.41]) by dizzie.triqs.com (Postfix) with ESMTP id 8DC08B3A8B for ; Tue, 13 Apr 2004 06:44:04 +0200 (CEST) Received: from shadrach (cs666877-133.austin.rr.com [66.68.77.133]) i3D4hxHw000474 for ; Mon, 12 Apr 2004 23:44:02 -0500 (CDT) Message-ID: <008e01c42110$394da640$8e00a8c0@austin.rr.com> From: "Ray Mosley" To: Date: Mon, 12 Apr 2004 23:31:43 -0500 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_008B_01C420E6.5023B1E0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: Symantec AntiVirus Scan Engine X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Tue, 13 Apr 2004 09:49:50 +0200 Subject: [Metakit] Questions X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Apr 2004 04:44:06 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_008B_01C420E6.5023B1E0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I am confused about the documentation in a couple of places.=20 First, the mk::file open command description says you can have any = number of readers OR one writer. Mark Roseman's documentation even says = only one user - period. The next sentence indicates the -extend option = can add data at the end of the database without affecting the readers. = Does this imply that you can have multiple readers and one writer using = the -extend option, or multiple writers using -extend?=20 Second, using -extend says readers can adjust to changes using rollback, = which is really a roll-forward. When you examine the mk::file rollback = command, it says it cancels all pending changes and reverts to what was = last stored on the file. How would this work? =20 Would a writer just do this?=20 mk::file open -extend=20 . . .=20 mk::row append=20 mk::file::commit=20 . . .=20 Then would subsequent requests by a reader locate the data by always = doing this?=20 mk::file open -extend=20 . . .=20 mk::file rollback=20 mk::get ....=20 In other words, ALWAYS do a mk::file rollback before any read operation? = In actual practice, I need a DB that can be changed by multiple = locations, and those other locations need to immediately "recognize" = that a change has occurred. Metakit in my tests seems wonderful, but I = can't get around the above requirement based upon my understanding of = the documentation. I have been investigating other options, also, but = none are as appealling as Metakit.=20 Thanks.=20 Ray Mosley mosley@austin.rr.com ------=_NextPart_000_008B_01C420E6.5023B1E0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I am confused about the = documentation=20 in a couple of places.
 
First, the mk::file open = command=20 description says you can have any number of readers OR one writer.  = Mark=20 Roseman's documentation even says only one user - period.  The = next=20 sentence indicates the -extend option can add data at the end of the = database=20 without affecting the readers.  Does this imply that you can have = multiple=20 readers and one writer using the -extend option, or multiple writers = using=20 -extend?
 
Second, using -extend says = readers can=20 adjust to changes using rollback, which is really a roll-forward.  = When you=20 examine the mk::file rollback command, it says it cancels all pending = changes=20 and reverts to what was last stored on the file.  How would this=20 work?  
Would a writer just do=20 this?
    mk::file open=20 -extend
    . .=20 .
    mk::row = append
    mk::file::commit
    . .=20 .
Then would =  subsequent requests=20 by a reader locate the data by always doing this? =
    mk::file open=20 -extend
    . .=20 .
    mk::file rollback =
    mk::get = ....
In other words, ALWAYS do = a mk::file=20 rollback before any read operation?
 
In actual practice, I need = a DB that=20 can be changed by multiple locations, and those other locations need to=20 immediately "recognize" that a change has occurred.  Metakit in my = tests=20 seems wonderful, but I can't get around the above requirement based upon = my=20 understanding of the documentation.  I have been investigating = other=20 options, also, but none are as appealling as Metakit.=20
 
Thanks.=20
Ray Mosley
mosley@austin.rr.com ------=_NextPart_000_008B_01C420E6.5023B1E0-- From bkelley@wi.mit.edu Tue Apr 13 16:10:52 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E8CA3B3AE8 for ; Tue, 13 Apr 2004 16:10:51 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 47197B3A8B for ; Tue, 13 Apr 2004 16:10:49 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004041310103505740 for ; Tue, 13 Apr 2004 10:10:35 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HW44RZ00.4O4; Tue, 13 Apr 2004 10:11:59 -0400 Message-ID: <407BF4A8.3020705@wi.mit.edu> Date: Tue, 13 Apr 2004 10:09:44 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ray Mosley , metakit@equi4.com Subject: Re: [Metakit] Questions References: <008e01c42110$394da640$8e00a8c0@austin.rr.com> In-Reply-To: <008e01c42110$394da640$8e00a8c0@austin.rr.com> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Apr 2004 14:10:52 -0000 There is a pretty good description of the extend/aside mode here (in python, but it should be easily transferable) http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/252494 You will still have the problem with one writer and simultaneous readers though. metaserver controls this by utilizing a metakit proxy that marshals all read/view/write operations through a central server. In practice it has been really fast although it does have a problem with transactions and rollbacks are kind of meaningless since any data no commited by any user will be rolled back. It is in python only though, Jean-Claude and I are working on a standaridized mkrpc but this is a little ways off. Again the prototype is in python. We are working on methods around this, in many ways the extend/aside mode is a good start except that it needs to be extended to support not just one aside file but many aside files (one for each user) but this becomes tricky when you consider the need for table locking or row locking so users don't clobber each other's data. So in short, multiple readers can access the data using extend/aside but you will have to marshall all data changes through the writer. In practice, this works very well for me because my frequency of data committing is small compared to the number of reads. I hope this helps. Brian Kelley From a.cappelli@tin.it Wed Apr 14 12:31:57 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id DA495B3AE7 for ; Wed, 14 Apr 2004 12:31:56 +0200 (CEST) Received: from fep02-svc.flexmail.it (fep02.tuttopmi.it [212.131.248.101]) by dizzie.triqs.com (Postfix) with ESMTP id 7075BB3A8B for ; Wed, 14 Apr 2004 12:31:53 +0200 (CEST) Received: from conan ([81.73.212.30]) by fep02-svc.flexmail.it with SMTP id <20040414103147.CGOK3871.fep02-svc.flexmail.it@conan> for ; Wed, 14 Apr 2004 12:31:47 +0200 Message-ID: <001301c4220b$af619890$1001a8c0@conan> From: "Andrea Cappelli" To: Date: Wed, 14 Apr 2004 12:31:02 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Metakit db file increasing X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Apr 2004 10:31:57 -0000 Hi, my name is Andrea and I'm a newbie of using Metakit. I'have written a simple application in c++ using Metakit and it works fine, only a problem: when I delete some rows from my view and I save back the view to the file, the older data remain in the file and the size of the file never decrease. When I open the file again in my application or with KitViewer I don't see the rows that I have deleted, but they are still in the file. I can see it if I open the file with Notepad, in this case I see the older data and the new data. Below the code I have written. Thanks for your help and excuse me for my bad english Andrea ######################################## // on launching application c4_Storage database ("data.db",true); c4_View table = database.GetAs("scadenzario[yyyy:I,mm:I,gg:I,titolo:S,note:S]"); // function that insert new record void insert(int yyyy, int mm, int dd, string note){ c4_IntProp anno("yyyy"), mese("mm"), giorno("gg"); c4_StringProp corpo("note"), title("titolo"); c4_Row nuova; anno(nuova)=yyyy; mese(nuova)=mm; giorno(nuova)=dd; const char* pippo = note.c_str(); c4_Row pluto; pluto=corpo.AsRow(pippo); nuova.ConcatRow(pluto); title(nuova)="Prova Scadenzario"; table.Add(nuova); } // function that delete n record from 0 int deleteDB(string nome, int number){ if (table.GetSize() < number) number = table.GetSize(); if (number>0) table.RemoveAt(0,number); return number; } // last statement in the main database.Commit(); ########################################## From pascal.baspeyras@coollector.com Thu Apr 15 12:44:23 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0BB18B3AE8 for ; Thu, 15 Apr 2004 12:44:23 +0200 (CEST) Received: from mail14.ha.ovh.net (b1.ovh.net [213.186.33.51]) by dizzie.triqs.com (Postfix) with ESMTP id C1D96B3AE2 for ; Thu, 15 Apr 2004 12:44:22 +0200 (CEST) Received: (qmail 27477 invoked by uid 503); 15 Apr 2004 10:44:38 -0000 Received: from dyn-83-155-57-229.ppp.tiscali.fr (HELO name) (83.155.57.229) by ns0.ovh.net with SMTP; 15 Apr 2004 10:44:38 -0000 Message-ID: <001c01c422d6$5b2ae4e0$e5399b53@name> From: "Pascal Baspeyras" To: Date: Thu, 15 Apr 2004 12:42:31 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Re: Metakit db file increasing X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2004 10:44:23 -0000 Hello Andrea, you'll have to use SaveTo() to get a compacted version of your database: c4_Storage MyDb("path_to_my_db", true); c4_FileStream MyDbCompacted(fopen("path_to_my_db_compacted"), true); MyDb.SaveTo(MyDbCompacted); HTH. From a.cappelli@tin.it Thu Apr 15 15:00:32 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 80AD8B3AE8 for ; Thu, 15 Apr 2004 15:00:32 +0200 (CEST) Received: from vsmtp4.tin.it (unknown [212.216.176.150]) by dizzie.triqs.com (Postfix) with ESMTP id B7CAAB3AE2 for ; Thu, 15 Apr 2004 15:00:30 +0200 (CEST) Received: from ims1e.cp.tin.it (192.168.70.101) by vsmtp4.tin.it (7.0.027) id 4061B1F7002822C6 for metakit@equi4.com; Thu, 15 Apr 2004 15:00:30 +0200 Received: from [192.168.70.229] by ims1e.cp.tin.it with HTTP; Thu, 15 Apr 2004 15:00:29 +0200 Date: Thu, 15 Apr 2004 15:00:29 +0200 Message-ID: <4043CF6C00042237@ims1e.cp.tin.it> From: a.cappelli@tin.it Subject: RE: [Metakit] Re: Metakit db file increasing To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-15" Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2004 13:00:34 -0000 Thanks for your help, but I still have a question. The class c4FileStream does not exist, there is only the class c4_Stream,= that is abstract. I have to define this class myself or someone has written it? >c4_Storage MyDb("path_to_my_db", true); >c4_FileStream MyDbCompacted(fopen("path_to_my_db_compacted"), true); >MyDb.SaveTo(MyDbCompacted); In this example, where are data? in path_to_my_db or in path_to_my_db_com= pacted? And in this case what is the function of the file path_to_my_db? Thanks again Andrea From pascal.baspeyras@coollector.com Thu Apr 15 17:10:32 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 805CEB3AE8 for ; Thu, 15 Apr 2004 17:10:32 +0200 (CEST) Received: from mail14.ha.ovh.net (b1.ovh.net [213.186.33.51]) by dizzie.triqs.com (Postfix) with ESMTP id E6A6FB3AE2 for ; Thu, 15 Apr 2004 17:10:29 +0200 (CEST) Received: (qmail 25921 invoked by uid 503); 15 Apr 2004 15:10:45 -0000 Received: from dyn-83-154-73-149.ppp.tiscali.fr (HELO name) (83.154.73.149) by ns0.ovh.net with SMTP; 15 Apr 2004 15:10:45 -0000 Message-ID: <001601c422fb$87a6ae80$95499a53@name> From: "Pascal Baspeyras" To: "metakit" Date: Thu, 15 Apr 2004 17:08:32 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Re: Metakit db file increasing X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2004 15:10:34 -0000 > The class c4FileStream does not exist, there is only the class c4_Stream, > that is abstract. > I have to define this class myself or someone has written it? You just need to include this: #include "mk4io.h" > >c4_Storage MyDb("path_to_my_db", true); > >c4_FileStream MyDbCompacted(fopen("path_to_my_db_compacted"), true); > >MyDb.SaveTo(MyDbCompacted); > > In this example, where are data? in path_to_my_db or in > path_to_my_db_compacted? > And in this case what is the function of the file path_to_my_db? Example: path_to_my_db => c:\MyDatabase.dat (your data is within MyDatabase.dat) path_to_my_db_compacted => c:\MyDatabaseCompact.dat you will get a new file, MyDatabaseCompact.dat, that contains the same data as MyDatabase.dat, but stripped from any unnecessary space (compact version). :)) From postmaster@coollector.com Thu Apr 15 12:34:40 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 35167B3AE8 for ; Thu, 15 Apr 2004 12:34:40 +0200 (CEST) Received: from mail14.ha.ovh.net (b1.ovh.net [213.186.33.51]) by dizzie.triqs.com (Postfix) with ESMTP id EB144B3AE2 for ; Thu, 15 Apr 2004 12:34:38 +0200 (CEST) Received: (qmail 7687 invoked by uid 503); 15 Apr 2004 10:34:54 -0000 Received: from dyn-83-155-57-229.ppp.tiscali.fr (HELO name) (83.155.57.229) by ns0.ovh.net with SMTP; 15 Apr 2004 10:34:54 -0000 Message-ID: <000701c422d4$ff1fc5e0$e5399b53@name> From: "Pascal Baspeyras" To: Date: Thu, 15 Apr 2004 12:32:47 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Fri, 16 Apr 2004 11:40:54 +0200 Subject: [Metakit] Re: Metakit db file increasing X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2004 10:34:40 -0000 Hello Andrea, you'll have to use SaveTo() to get a compacted version of your database: c4_Storage MyDb("path_to_my_db", true); c4_FileStream MyDbCompacted(fopen("path_to_my_db_compacted"), true); MyDb.SaveTo(MyDbCompacted); HTH. From mosley@austin.rr.com Sun Apr 18 01:21:47 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4C17EB3AE3 for ; Sun, 18 Apr 2004 01:21:47 +0200 (CEST) Received: from ms-smtp-05-eri0.domain.rr.com (ms-smtp-05.texas.rr.com [24.93.47.44]) by dizzie.triqs.com (Postfix) with ESMTP id F2A20B3AE2 for ; Sun, 18 Apr 2004 01:21:45 +0200 (CEST) Received: from shadrach (cs666877-133.austin.rr.com [66.68.77.133]) i3HNLdDj024964 for ; Sat, 17 Apr 2004 18:21:44 -0500 (CDT) Message-ID: <00f701c424d0$c2c367c0$8e00a8c0@austin.rr.com> From: "Ray Mosley" To: References: <20040414100003.C776EB3AE9@dizzie.triqs.com> Date: Sat, 17 Apr 2004 18:07:30 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Re: Metakit Digest, Vol 6, Issue 4 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Apr 2004 23:21:49 -0000 Okay, so multiple write locations is not presently possible. Assuming I limit my application to one writer, what will happen in these two scenarios? Are locks protecting the data in these events? 1) What happens when a reader is active and a write request is made? 2) What happens to a reader if the writer is active? ----- Original Message ----- From: To: Sent: Wednesday, April 14, 2004 5:00 AM Subject: Metakit Digest, Vol 6, Issue 4 > Send Metakit mailing list submissions to > metakit@equi4.com > > To subscribe or unsubscribe via the World Wide Web, visit > http://www.equi4.com/mailman/listinfo/metakit > or, via email, send a message with subject or body 'help' to > metakit-request@equi4.com > > You can reach the person managing the list at > metakit-owner@equi4.com > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Metakit digest..." > > > Today's Topics: > > 1. Re: Questions (Brian Kelley) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 13 Apr 2004 10:09:44 -0400 > From: "Brian Kelley" > Subject: Re: [Metakit] Questions > To: Ray Mosley , metakit@equi4.com > Message-ID: <407BF4A8.3020705@wi.mit.edu> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > There is a pretty good description of the extend/aside mode here (in > python, but it should be easily transferable) > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/252494 > > You will still have the problem with one writer and simultaneous > readers though. > > metaserver controls this by utilizing a metakit proxy that marshals > all read/view/write operations through a central server. In practice > it has been really fast although it does have a problem with > transactions and rollbacks are kind of meaningless since any data no > commited by any user will be rolled back. It is in python only > though, Jean-Claude and I are working on a standaridized mkrpc but > this is a little ways off. Again the prototype is in python. > > We are working on methods around this, in many ways the extend/aside > mode is a good start except that it needs to be extended to support > not just one aside file but many aside files (one for each user) but > this becomes tricky when you consider the need for table locking or > row locking so users don't clobber each other's data. > > So in short, multiple readers can access the data using extend/aside > but you will have to marshall all data changes through the writer. In > practice, this works very well for me because my frequency of data > committing is small compared to the number of reads. > > I hope this helps. > > Brian Kelley > > > ------------------------------ > > _______________________________________________ > Metakit mailing list - Metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > > End of Metakit Digest, Vol 6, Issue 4 > ************************************* > From gb@murphy.bofh.ms Tue Apr 20 19:08:13 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 1761EB3AFC for ; Tue, 20 Apr 2004 19:08:13 +0200 (CEST) Received: from simon.bofh.ms (simon.bofh.ms [213.133.111.3]) by dizzie.triqs.com (Postfix) with ESMTP id 71B1CB3AE2 for ; Tue, 20 Apr 2004 19:08:12 +0200 (CEST) Received: from pd950c3bc.dip.t-dialin.net ([217.80.195.188] helo=firewall.bofh.ms ident=Debian-exim) by simon.bofh.ms with asmtp (Exim 4.20 #1 (Debian)) id 1BFyoG-0006xL-Oo for ; Tue, 20 Apr 2004 19:13:24 +0200 Received: from tex.bofh.ms ([10.0.0.145] helo=murphy.bofh.ms) by firewall.bofh.ms with esmtp (Exim 4.31) id 1BFyjD-0007pE-L2 for metakit@equi4.com; Tue, 20 Apr 2004 19:08:13 +0200 Date: Tue, 20 Apr 2004 19:08:08 +0200 Mime-Version: 1.0 (Apple Message framework v553) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Georg Bauer To: metakit@equi4.com Content-Transfer-Encoding: 7bit Message-Id: <4B9BE4E1-92ED-11D8-A564-000A9573A72A@murphy.bofh.ms> X-Mailer: Apple Mail (2.553) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Fwd: [PyCS-devel] Re: [Pyds-users] Maybe problems with Metakit 2.4.9.3 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Apr 2004 17:08:13 -0000 Hi! A PyDS user reported the following problem with Metakit 2.4.9.3. Any ideas on what might go wrong? bye, Georg > Von: Yasushi Iwata > Datum: Di, 20. Apr 2004 05:08:51 Europe/Berlin > An: pyds-users@muenster.westfalen.de > Cc: pycs-devel@lists.sourceforge.net > Betreff: [PyCS-devel] Re: [Pyds-users] Maybe problems with Metakit > 2.4.9.3 > > Hi, > > I found this code causes segmentation fault with Metakit 2.4.9.3. But > no problem with 2.4.9.2. > > -- > import metakit > > db = metakit.storage('foo.dat', 1) > > comments = db.getas( > 'comments[user:S, paragraph:S, link:S, notes[name:S, email:S, > url:S, comment:S, date:S]]').ordered( 2 ) > > user = '0000001' > paragraph = 'P1' > link = 'http://foobar.com/weblog/2004/04/20.html#P1' > > print 'Now, appending...' > comments.append({ > 'user': user, > 'paragraph': paragraph, > 'link': link, > 'notes': metakit.view() > }) > print 'Appended.' > db.commit() > print 'committed.' > From bkelley@wi.mit.edu Tue Apr 20 19:22:54 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3F862B3AFC for ; Tue, 20 Apr 2004 19:22:54 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id D9A6BB3AE2 for ; Tue, 20 Apr 2004 19:22:52 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004042013223500072 for ; Tue, 20 Apr 2004 13:22:35 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HWHCBW00.J9S; Tue, 20 Apr 2004 13:23:56 -0400 Message-ID: <40855C30.3080803@wi.mit.edu> Date: Tue, 20 Apr 2004 13:21:52 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Georg Bauer , Metakit mailing list Subject: Re: [Metakit] Fwd: [PyCS-devel] Re: [Pyds-users] Maybe problems with Metakit 2.4.9.3 References: <4B9BE4E1-92ED-11D8-A564-000A9573A72A@murphy.bofh.ms> In-Reply-To: <4B9BE4E1-92ED-11D8-A564-000A9573A72A@murphy.bofh.ms> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Apr 2004 17:22:54 -0000 Georg Bauer wrote: > Hi! > > A PyDS user reported the following problem with Metakit 2.4.9.3. > Any ideas on what might go wrong? > > bye, Georg > >> comments = db.getas( 'comments[user:S, paragraph:S, link:S, >> notes[name:S, email:S, url:S, comment:S, date:S]]').ordered( 2 ) Most likely it is the spaces in your getas statement here. If you remove them, it works just fine. From gb@murphy.bofh.ms Tue Apr 20 19:28:45 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6AE58B3B05 for ; Tue, 20 Apr 2004 19:28:45 +0200 (CEST) Received: from simon.bofh.ms (simon.bofh.ms [213.133.111.3]) by dizzie.triqs.com (Postfix) with ESMTP id E28ABB3AE2 for ; Tue, 20 Apr 2004 19:28:44 +0200 (CEST) Received: from pd950c3bc.dip.t-dialin.net ([217.80.195.188] helo=firewall.bofh.ms ident=Debian-exim) by simon.bofh.ms with asmtp (Exim 4.20 #1 (Debian)) id 1BFz88-0007At-WF; Tue, 20 Apr 2004 19:33:57 +0200 Received: from tex.bofh.ms ([10.0.0.145] helo=murphy.bofh.ms) by firewall.bofh.ms with esmtp (Exim 4.31) id 1BFz35-0008Ab-TV; Tue, 20 Apr 2004 19:28:45 +0200 Date: Tue, 20 Apr 2004 19:28:40 +0200 Subject: Re: [Metakit] Fwd: [PyCS-devel] Re: [Pyds-users] Maybe problems with Metakit 2.4.9.3 Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) To: "Brian Kelley" From: Georg Bauer In-Reply-To: <40855C30.3080803@wi.mit.edu> Message-Id: <2A2B412A-92F0-11D8-A564-000A9573A72A@murphy.bofh.ms> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit mailing list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Apr 2004 17:28:45 -0000 Hi! > >> comments = db.getas( 'comments[user:S, paragraph:S, link:S, > >> notes[name:S, email:S, url:S, comment:S, date:S]]').ordered( 2 ) > > Most likely it is the spaces in your getas statement here. If you > remove them, it works just fine. Damn. I remember I stumbled over this specific problem already some time ago. I think I even asked on this list about it :-/ But what still is a bit weird: why does it work with 2.4.9.2? bye, Georg From yasusii@lowlife.jp Wed Apr 21 10:22:52 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 61F88B3B0B for ; Wed, 21 Apr 2004 10:22:52 +0200 (CEST) Received: from ns1.radio1.jp (ns1.radio1.jp [61.198.22.2]) by dizzie.triqs.com (Postfix) with SMTP id 5D253B3AE2 for ; Wed, 21 Apr 2004 10:22:48 +0200 (CEST) Received: (qmail 30669 invoked from network); 21 Apr 2004 08:22:44 -0000 Received: from unknown (HELO localhost) (221.191.63.97) by ns1.radio1.jp with SMTP; 21 Apr 2004 08:22:44 -0000 Date: Wed, 21 Apr 2004 17:22:12 +0900 (JST) Message-Id: <20040421.172212.41628338.yasusii@lowlife.jp> To: metakit@equi4.com From: Yasushi Iwata X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Re: Maybe problems with Metakit 2.4.9.3 X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Apr 2004 08:22:52 -0000 Hi, I removed spaces from getas statement. But following code still causes segmentation fault with 2.4.9.3. Am I doing something wrong? -- import metakit db = metakit.storage('foo.dat', 1) comments = db.getas('comments[user:S,paragraph:S,link:S,notes[name:S,email:S,url:S,comment:S,date:S]]').ordered(2) for i in range(1, 30): print 'Appending %d...' % i comments.append({ 'user': '00000001', 'paragraph': 'P%d' % i, 'link': 'http://foo.com/', 'notes': metakit.view() }) print '%d OK' % i db.commit() From bkelley@wi.mit.edu Wed Apr 21 15:42:46 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6F599B3B0D for ; Wed, 21 Apr 2004 15:42:46 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 39FBFB3B0B for ; Wed, 21 Apr 2004 15:42:44 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004042109422512353 for ; Wed, 21 Apr 2004 09:42:25 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HWIWSZ00.GHW; Wed, 21 Apr 2004 09:43:47 -0400 Message-ID: <40867A17.3000704@wi.mit.edu> Date: Wed, 21 Apr 2004 09:41:43 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Yasushi Iwata , Metakit mailing list Subject: Re: [Metakit] Re: Maybe problems with Metakit 2.4.9.3 References: <20040421.172212.41628338.yasusii@lowlife.jp> In-Reply-To: <20040421.172212.41628338.yasusii@lowlife.jp> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Apr 2004 13:42:46 -0000 Yasushi Iwata wrote: > Hi, > > I removed spaces from getas statement. But following code still > causes segmentation fault with 2.4.9.3. Am I doing something wrong? > > > -- > > import metakit > > db = metakit.storage('foo.dat', 1) > > comments = > db.getas('comments[user:S,paragraph:S,link:S,notes[name:S,email:S,url:S,comment:S,date:S]]').ordered(2) > > > for i in range(1, 30): print 'Appending %d...' % i > comments.append({ 'user': '00000001', 'paragraph': 'P%d' % i, > 'link': 'http://foo.com/', 'notes': metakit.view() }) print '%d OK' > % i db.commit() I might have made the change without telling you, but you don't need the "metakit.view()" entry for the 'notes', I'm not sure why this crashes, but by default the notes view will be created empty. The spaces were certainly wrong, but they probably shouldn't have caused a crash, so maybe the metakit.view() in the append statement is broken. To be safe you could use 'notes': None which will insert the default object, in this case the view. To access the view you will need to find the right row, such as index = comments.find(user='00000001', paragraph='P1') if index != -1: notes = comments[index].notes notes.append(...) Brian > _____________________________________________ Metakit mailing list > - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From yasusii@lowlife.jp Thu Apr 22 06:00:41 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2BDAEB3AED for ; Thu, 22 Apr 2004 06:00:41 +0200 (CEST) Received: from ns1.radio1.jp (ns1.radio1.jp [61.198.22.2]) by dizzie.triqs.com (Postfix) with SMTP id 62A9EB3AE6 for ; Thu, 22 Apr 2004 06:00:37 +0200 (CEST) Received: (qmail 2363 invoked from network); 22 Apr 2004 04:00:33 -0000 Received: from unknown (HELO localhost) (221.191.63.97) by ns1.radio1.jp with SMTP; 22 Apr 2004 04:00:33 -0000 Date: Thu, 22 Apr 2004 13:00:01 +0900 (JST) Message-Id: <20040422.130001.108738679.yasusii@lowlife.jp> To: metakit@equi4.com Subject: Re: [Metakit] Re: Maybe problems with Metakit 2.4.9.3 From: Yasushi Iwata In-Reply-To: <40867A17.3000704@wi.mit.edu> References: <20040421.172212.41628338.yasusii@lowlife.jp> <40867A17.3000704@wi.mit.edu> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Apr 2004 04:00:41 -0000 Hi, On Wed, 21 Apr 2004 09:41:43 -0400 Brian Kelley wrote: > The spaces were certainly wrong, but they probably shouldn't have > caused a crash, so maybe the metakit.view() in the append statement is > broken. To be safe you could use 'notes': None which will insert the > default object, in this case the view. I tried to use 'notes': None but TypeError raised. $ python2.3 foo.py Appending 1... Traceback (most recent call last): File "test4.py", line 11, in ? comments.append({ TypeError: Not a sequence > > To access the view you will need to find the right row, such as > > index = comments.find(user='00000001', paragraph='P1') > if index != -1: > notes = comments[index].notes > notes.append(...) > > Brian > > > _____________________________________________ Metakit mailing list > > - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit > > > From yasusii@lowlife.jp Thu Apr 22 09:23:17 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B2BC2B3AED for ; Thu, 22 Apr 2004 09:23:17 +0200 (CEST) Received: from ns1.radio1.jp (ns1.radio1.jp [61.198.22.2]) by dizzie.triqs.com (Postfix) with SMTP id 6BAD3B3AE6 for ; Thu, 22 Apr 2004 09:23:13 +0200 (CEST) Received: (qmail 2940 invoked from network); 22 Apr 2004 07:23:10 -0000 Received: from unknown (HELO localhost) (221.191.63.97) by ns1.radio1.jp with SMTP; 22 Apr 2004 07:23:10 -0000 Date: Thu, 22 Apr 2004 16:22:37 +0900 (JST) Message-Id: <20040422.162237.104028246.yasusii@lowlife.jp> To: metakit@equi4.com Subject: Re: [Metakit] Re: Maybe problems with Metakit 2.4.9.3 From: Yasushi Iwata In-Reply-To: <40867A17.3000704@wi.mit.edu> References: <20040421.172212.41628338.yasusii@lowlife.jp> <40867A17.3000704@wi.mit.edu> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Apr 2004 07:23:18 -0000 Hi, I found another problem. Following code dose not work as expected. -- db = metakit.storage('foo.dat', 1) comments = db.getas( 'comments[user:S,paragraph:S,link:S,notes[name:S,email:S,url:S,comment:S,date:S]]' ).ordered(2) for i in range(1, 3): print 'Appending %d...' % i comments.append({ 'user': '00000001', 'paragraph': 'P%d' % i, 'link': 'http://foo.com/', }) print '%d OK' % i db.commit() for i in range(1, 3): idx = comments.find(user='00000001', paragraph='P%d' % i) if idx == -1: print 'P%d not found' % i else: print comments[idx].paragraph, 'found' -- $ python2.3 foo.py Appending 1... 1 OK Appending 2... 2 OK P1 not found P2 not found But if you remove ordered(2) from getas(), it works as expected. I also removed ordered(2) from example code that I posted yesterday, it worked fine. There must be something wrong with ordered(). From jack@performancedrivers.com Fri Apr 23 22:10:27 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 21384B3AEB for ; Fri, 23 Apr 2004 22:10:27 +0200 (CEST) Received: from lsh099.siteprotect.com (lsh099.siteprotect.com [66.113.134.251]) by dizzie.triqs.com (Postfix) with ESMTP id 15427B3AE3 for ; Fri, 23 Apr 2004 22:10:25 +0200 (CEST) Received: from wopr (zundlk.ne.client2.attbi.com [65.96.181.124]) by lsh099.siteprotect.com (8.11.6/8.11.6) with ESMTP id i3NKAOj01692 for ; Fri, 23 Apr 2004 15:10:24 -0500 Received: from jack by wopr with local (Exim 3.36 #1 (Debian)) id 1BH70A-00076T-00 for ; Fri, 23 Apr 2004 16:10:22 -0400 Date: Fri, 23 Apr 2004 16:10:22 -0400 From: Jack Diederich To: metakit@equi4.com Message-ID: <20040423201022.GD25755@performancedrivers.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.5.1+cvs20040105i X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] python --with-pydebug build segfaults metakit X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Apr 2004 20:10:27 -0000 I emailed JCW directly earlier (his address is in the README, I didn't find this mailing list until later). If you build metakit using a --with-python that ponits to a debug build of python it will segfault. I'm fairly certain this is due to the HEAD struct being different than what is expected. Specifically I think the _ob_next and _ob_prev members (which are only in debug builds) are corrupted or not set. ob_refcnt's seem to be correct, the segfault comes when python is trying to assert references during object teardown. C++ classes that are also valid PyObject pointers are tricky, the initialization has to be just right. I don't personally know enough to fix it. # crashme.py import metakit db = metakit.storage('test.mk', 1) metakit 2.4.93, python 2.3.3, gcc 2.95.4 (Debian prerelease) TIA, -jack From jcw@equi4.com Sat Apr 24 12:08:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id EBC7BB3AE4 for ; Sat, 24 Apr 2004 12:08:37 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id CA42EB3AE3 for ; Sat, 24 Apr 2004 12:08:35 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 1D6CE83C26; Sat, 24 Apr 2004 12:08:34 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id B2D998393A for ; Sat, 24 Apr 2004 12:08:30 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: <20040423201022.GD25755@performancedrivers.com> References: <20040423201022.GD25755@performancedrivers.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <561AB526-95D7-11D8-94AE-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] python --with-pydebug build segfaults metakit Date: Sat, 24 Apr 2004 12:08:30 +0200 To: Metakit list X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Apr 2004 10:08:38 -0000 Jack Diederich wrote: > If you build metakit using a --with-python that ponits to a > debug build of python it will segfault. I'm fairly certain this > is due to the HEAD struct being different than what is expected. > Specifically I think the _ob_next and _ob_prev members (which are > only in debug builds) are corrupted or not set. ob_refcnt's > seem to be correct, the segfault comes when python is trying > to assert references during object teardown. > > C++ classes that are also valid PyObject pointers are tricky, > the initialization has to be just right. I don't personally > know enough to fix it. Good catch, but I can't seem to find the reason either. The logic in MK is as follows: Python does "manual inheritance", by putting the same struct fields at the start in every object (see #define PyObject_HEAD in object.h). MK is C++, so at the time I decided to take advantage of that and define a PyHead *class* with exactly the same structure, and then have all other objects (PyProperty, PyRowRef, PyView) defined as C++ derived classes. It adds the ability to pass pointers to any of those objects in places where a PyObject* is expected without having to cast them. This sounds more ominous than it is, really. All it took, was a minute header file called "metakit/python/PyHead.h" to make things work that way. In there is logic which initializes the two fields, and an #ifdef which initializes two more when this is a debug build. Looking at object.h, I can see no difference in declaration & initialization, for release nor debug builds. Something must have changed, but I'm at a loss as to what that might be. MK used to work fine with debug builds. If anyone can shed light on this, please tell... -jcw From jcw@equi4.com Sat Apr 24 12:14:33 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 087DCB3AE4 for ; Sat, 24 Apr 2004 12:14:33 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 6FD60B3AE3 for ; Sat, 24 Apr 2004 12:14:31 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 1119D83C26; Sat, 24 Apr 2004 12:14:29 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E759B8393A for ; Sat, 24 Apr 2004 12:14:27 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: <20040422.162237.104028246.yasusii@lowlife.jp> References: <20040421.172212.41628338.yasusii@lowlife.jp> <40867A17.3000704@wi.mit.edu> <20040422.162237.104028246.yasusii@lowlife.jp> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <2B486B8E-95D8-11D8-94AE-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Re: Maybe problems with Metakit 2.4.9.3 Date: Sat, 24 Apr 2004 12:14:27 +0200 To: Metakit list X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Apr 2004 10:14:33 -0000 Yasushi Iwata wrote: > I found another problem. Following code dose not work as expected. [...] > But if you remove ordered(2) from getas(), it works as expected. I > also removed ordered(2) from example code that I posted yesterday, it > worked fine. There must be something wrong with ordered(). Thanks for diagnosing this. Yes, I suspect ordered() has troubles - perhaps it's with more than 1 key field. There are some complex interactions between the view model of indexed access, i.e. control over where things go, and ordered - which tries to decide on its own where to put things (and hash() has no such issues, since it maintains order in MK). -jcw From jcw@equi4.com Sat Apr 24 13:42:54 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C8BCBB3AE4 for ; Sat, 24 Apr 2004 13:42:54 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 89D02B3AE3 for ; Sat, 24 Apr 2004 13:42:52 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 05C4483C26; Sat, 24 Apr 2004 13:42:50 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 296068393A for ; Sat, 24 Apr 2004 13:42:48 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) Content-Transfer-Encoding: 7bit Message-Id: <827ECE3A-95E4-11D8-94AE-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Sat, 24 Apr 2004 13:42:48 +0200 X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Mk4py build & test Q X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Apr 2004 11:42:55 -0000 I have a question for Python experts, w.r.t. distutils: I'd like to try and get setup.py working on its own. Here's what I get right now (cvs HEAD, build dir wiped): $ python setup.py build running build running build_py creating ../builds/lib.linux-i686-2.3 copying metakit.py -> ../builds/lib.linux-i686-2.3 running build_ext running config gcc -E -I/usr/include/python2.3 -o _configtest.i _configtest.c removing: _configtest.c _configtest.i building 'Mk4py' extension creating ../builds/temp.linux-i686-2.3 creating ../builds/temp.linux-i686-2.3/scxx g++ -fno-strict-aliasing -DNDEBUG -fPIC -DHAVE_UNICODEOBJECT_H=1 -Iscxx -I../include -I/usr/include/python2.3 -c PyView.cpp -o ../builds/temp.linux-i686-2.3/PyView.o [...] g++ -pthread -shared ../builds/temp.linux-i686-2.3/PyProperty.o ../builds/temp.linux-i686-2.3/PyRowRef.o ../builds/temp.linux-i686-2.3/PyStorage.o ../builds/temp.linux-i686-2.3/PyView.o ../builds/temp.linux-i686-2.3/scxx/PWOImp.o ../builds/column.o ../builds/custom.o ../builds/derived.o ../builds/fileio.o ../builds/field.o ../builds/format.o ../builds/handler.o ../builds/persist.o ../builds/remap.o ../builds/std.o ../builds/store.o ../builds/string.o ../builds/table.o ../builds/univ.o ../builds/view.o ../builds/viewx.o -lstdc++ -o ../builds/lib.linux-i686-2.3/Mk4py.so g++: ../builds/column.o: No such file or directory [...] g++: ../builds/viewx.o: No such file or directory error: command 'g++' failed with exit status 1 $ Is there a simple way to resolve this? The workaround is to first do: cd ../builds; ../unix/configure; make The other issue I ran into is testing: $ python setup.py test running test running build running build_py running build_ext running config gcc -E -I/usr/include/python2.3 -o _configtest.i _configtest.c removing: _configtest.c _configtest.i Traceback (most recent call last): File "setup.py", line 184, in ? extra_objects=mkobjs, File "/usr/lib/python2.3/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/lib/python2.3/distutils/dist.py", line 907, in run_commands self.run_command(cmd) File "/usr/lib/python2.3/distutils/dist.py", line 927, in run_command cmd_obj.run() File "setup.py", line 133, in run import test.regrtest ImportError: No module named regrtest $ (Am using 2.3.3 on Linux, btw) It went away when I disable the line in setup.py: #sys.path.insert(0, self.test_dir) But then it seems to get lost in finding other stuff: $ python setup.py test running test running build running build_py running build_ext running config gcc -E -I/usr/include/python2.3 -o _configtest.i _configtest.c removing: _configtest.c _configtest.i test_inttypes test_inttypes skipped -- No module named test_inttypes 1 test skipped: test_inttypes 1 skip unexpected on linux2: test_inttypes $ -jcw From jack@performancedrivers.com Sat Apr 24 23:09:38 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2E268B3B1F for ; Sat, 24 Apr 2004 23:09:38 +0200 (CEST) Received: from lsh099.siteprotect.com (lsh099.siteprotect.com [66.113.134.251]) by dizzie.triqs.com (Postfix) with ESMTP id B4634B3AE3 for ; Sat, 24 Apr 2004 23:09:35 +0200 (CEST) Received: from wopr (zundlk.ne.client2.attbi.com [65.96.181.124]) by lsh099.siteprotect.com (8.11.6/8.11.6) with ESMTP id i3OL9YO23334 for ; Sat, 24 Apr 2004 16:09:34 -0500 Received: from jack by wopr with local (Exim 3.36 #1 (Debian)) id 1BHUOy-00010c-00 for ; Sat, 24 Apr 2004 17:09:32 -0400 Date: Sat, 24 Apr 2004 17:09:32 -0400 From: Jack Diederich To: Metakit list Subject: Re: [Metakit] Mk4py build & test Q Message-ID: <20040424210932.GA2492@performancedrivers.com> Mail-Followup-To: Metakit list References: <827ECE3A-95E4-11D8-94AE-000A9588127E@equi4.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <827ECE3A-95E4-11D8-94AE-000A9588127E@equi4.com> User-Agent: Mutt/1.5.5.1+cvs20040105i X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Apr 2004 21:09:38 -0000 On Sat, Apr 24, 2004 at 01:42:48PM +0200, Jean-Claude Wippler wrote: > I have a question for Python experts, w.r.t. distutils: > > The other issue I ran into is testing: > > $ python setup.py test > running test > running build > running build_py > running build_ext > running config > gcc -E -I/usr/include/python2.3 -o _configtest.i _configtest.c > removing: _configtest.c _configtest.i > Traceback (most recent call last): > File "setup.py", line 184, in ? > extra_objects=mkobjs, > File "/usr/lib/python2.3/distutils/core.py", line 149, in setup > dist.run_commands() > File "/usr/lib/python2.3/distutils/dist.py", line 907, in run_commands > self.run_command(cmd) > File "/usr/lib/python2.3/distutils/dist.py", line 927, in run_command > cmd_obj.run() > File "setup.py", line 133, in run > import test.regrtest > ImportError: No module named regrtest > $ There is a naming conflict between the stdlib test module and your test.py # python Python 2.3.3 (#2, Feb 24 2004, 09:29:20) Type "help", "copyright", "credits" or "license" for more information. >>> import test >>> test # python Python 2.3.3 (#2, Feb 24 2004, 09:29:20) Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path.insert(0, './test/') >>> import test >>> test If you rename test.py to mktest.py you should be able to use both of them. -jackdied From jack@performancedrivers.com Mon Apr 26 05:24:19 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B9E36B3AFB for ; Mon, 26 Apr 2004 05:24:19 +0200 (CEST) Received: from lsh099.siteprotect.com (lsh099.siteprotect.com [66.113.134.251]) by dizzie.triqs.com (Postfix) with ESMTP id A0FBCB3AF0 for ; Mon, 26 Apr 2004 05:24:17 +0200 (CEST) Received: from wopr (zundlk.ne.client2.attbi.com [65.96.181.124]) by lsh099.siteprotect.com (8.11.6/8.11.6) with ESMTP id i3Q3OGm07284 for ; Sun, 25 Apr 2004 22:24:16 -0500 Received: from jack by wopr with local (Exim 3.36 #1 (Debian)) id 1BHwj8-00052N-00 for ; Sun, 25 Apr 2004 23:24:14 -0400 Date: Sun, 25 Apr 2004 23:24:14 -0400 From: Jack Diederich To: Metakit list Subject: Re: [Metakit] Mk4py build & test Q Message-ID: <20040426032414.GB2492@performancedrivers.com> Mail-Followup-To: Metakit list References: <827ECE3A-95E4-11D8-94AE-000A9588127E@equi4.com> <20040424210932.GA2492@performancedrivers.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040424210932.GA2492@performancedrivers.com> User-Agent: Mutt/1.5.5.1+cvs20040105i X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 03:24:20 -0000 On Sat, Apr 24, 2004 at 05:09:32PM -0400, Jack Diederich wrote: > On Sat, Apr 24, 2004 at 01:42:48PM +0200, Jean-Claude Wippler wrote: > > I have a question for Python experts, w.r.t. distutils: > > If you rename test.py to mktest.py you should be able to use both of > them. > I saw the mktest.py rename in CVS, and it almost works for me. I get the 'freebsd4' suite of tests (on debian linux) which tries to include a stdlib test module that only applies to freebsd. I haven't looked at it any closer, but I would guess something in CVS has a hard definition for freebsd. -jackdied From jcw@equi4.com Mon Apr 26 12:22:57 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 4FC65B3AF0 for ; Mon, 26 Apr 2004 12:22:57 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id AFAB6B3AE2 for ; Mon, 26 Apr 2004 12:22:55 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 8901283C26; Mon, 26 Apr 2004 12:22:51 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 27AC18393A for ; Mon, 26 Apr 2004 12:22:49 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: <20040426032414.GB2492@performancedrivers.com> References: <827ECE3A-95E4-11D8-94AE-000A9588127E@equi4.com> <20040424210932.GA2492@performancedrivers.com> <20040426032414.GB2492@performancedrivers.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Mk4py build & test Q Date: Mon, 26 Apr 2004 12:22:51 +0200 To: Metakit list X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 10:22:57 -0000 Hello Jack, (thanks for your help on test.py vs. mktest.py) >> If you rename test.py to mktest.py you should be able to use both of >> them. > > I saw the mktest.py rename in CVS, and it almost works for me. > I get the 'freebsd4' suite of tests (on debian linux) which tries > to include a stdlib test module that only applies to freebsd. > > I haven't looked at it any closer, but I would guess something > in CVS has a hard definition for freebsd. I've just checked in some more changes and a few files I missed for Mk4py testing. The tests now seem to work on Linux. I've not found anything specific for FreeBSD so far. It may be caused by something which is Mac OS X specific, which is also *BSD-ish. -jcw From bkelley@wi.mit.edu Mon Apr 26 15:25:39 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 55063B3AFC; Mon, 26 Apr 2004 15:25:39 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id D8E5CB3AE2; Mon, 26 Apr 2004 15:25:36 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004042609251508953 ; Mon, 26 Apr 2004 09:25:15 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HWS5CC00.AIT; Mon, 26 Apr 2004 09:26:36 -0400 Message-ID: <408D0D9B.1070506@wi.mit.edu> Date: Mon, 26 Apr 2004 09:24:43 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler , Metakit mailing list Subject: Re: [Metakit] Re: Maybe problems with Metakit 2.4.9.3 References: <20040421.172212.41628338.yasusii@lowlife.jp> <40867A17.3000704@wi.mit.edu> <20040422.162237.104028246.yasusii@lowlife.jp> <2B486B8E-95D8-11D8-94AE-000A9588127E@equi4.com> In-Reply-To: <2B486B8E-95D8-11D8-94AE-000A9588127E@equi4.com> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 13:25:39 -0000 I have some test more test cases for blocked ordered views that fail. Here is a nice juicy one suitable for debugging. Only the blocked.ordered() views fail. As a further mystery, in some cases find fails for a blocked ordered view for two integer columns if the second one is negative. import metakit st = metakit.storage() vw = st.getas("test[_B[a:I,b:I]]").blocked().ordered() vw.append((1,1)) vw.append((2,-1)) vw.append((3,2)) # positive second columns are okay assert vw.find(a=3, b=2) != -1 assert vw.search(a=3) != -1 assert vw.find(a=2, b=-1) != -1 assert vw.search(a=2) != -1 # find now fails - negative second column is bad :( assert vw.find(a=2) != -1 From mbloore@yahoo.com Mon Apr 26 17:38:19 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B0E24B3B0E for ; Mon, 26 Apr 2004 17:38:18 +0200 (CEST) Received: from web50201.mail.yahoo.com (web50201.mail.yahoo.com [206.190.38.42]) by dizzie.triqs.com (Postfix) with SMTP id 98CB6B3AE2 for ; Mon, 26 Apr 2004 17:38:16 +0200 (CEST) Message-ID: <20040426153801.84310.qmail@web50201.mail.yahoo.com> Received: from [65.95.156.152] by web50201.mail.yahoo.com via HTTP; Mon, 26 Apr 2004 08:38:01 PDT Date: Mon, 26 Apr 2004 08:38:01 -0700 (PDT) From: mARK bLOORE Subject: Re: [Metakit] Re: Maybe problems with Metakit 2.4.9.3 To: metakit In-Reply-To: <20040422.162237.104028246.yasusii@lowlife.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 15:38:19 -0000 i don't know if this has anything to do with it, but i noticed (and posted here) some time ago that if one does a find (using C++, in my case) with a row that was constructed in a different order than the fields are given in the getas call, then the find fails. this python API is rather different, in particular find seems to take two separate args, rather than one row that was built with two fields, but i wonder if maybe the python find constructs a row behind the scenes, in reverse order? --- Yasushi Iwata wrote: > Hi, > > I found another problem. Following code dose not work as expected. > > -- > > db = metakit.storage('foo.dat', 1) > > comments = db.getas( > 'comments[user:S,paragraph:S,link:S,notes[name:S,email:S,url:S,comment:S,date:S]]' > ).ordered(2) > > for i in range(1, 3): > print 'Appending %d...' % i > comments.append({ > 'user': '00000001', > 'paragraph': 'P%d' % i, > 'link': 'http://foo.com/', > }) > print '%d OK' % i > db.commit() > > for i in range(1, 3): > idx = comments.find(user='00000001', paragraph='P%d' % i) > > if idx == -1: > print 'P%d not found' % i > else: > print comments[idx].paragraph, 'found' > > -- > > $ python2.3 foo.py > Appending 1... > 1 OK > Appending 2... > 2 OK > P1 not found > P2 not found > > But if you remove ordered(2) from getas(), it works as expected. I > also removed ordered(2) from example code that I posted yesterday, it > worked fine. There must be something wrong with ordered(). ===== -- mARK bLOORE __________________________________ Do you Yahoo!? Yahoo! Photos: High-quality 4x6 digital prints for 25¢ http://photos.yahoo.com/ph/print_splash From jcw@equi4.com Wed Apr 28 11:15:23 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id CCEC4B3AE6 for ; Wed, 28 Apr 2004 11:15:23 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 13959B3AE5 for ; Wed, 28 Apr 2004 11:15:22 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id BD11D83C26; Wed, 28 Apr 2004 11:15:20 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 757148393A for ; Wed, 28 Apr 2004 11:15:18 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) Content-Transfer-Encoding: 7bit Message-Id: <90E5D0AE-98F4-11D8-AF11-000A9588127E@equi4.com> Content-Type: text/plain; charset=US-ASCII; format=flowed To: Metakit list From: Jean-Claude Wippler Date: Wed, 28 Apr 2004 11:15:17 +0200 X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] blocked views X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Apr 2004 09:15:24 -0000 There is a faster implementation of blocked views in CVS now. It evolved from a change submitted by M. Berk (thank you!) and appears to have a considerable effect on performance. The trick is to cache the last used subview. If you use blocked views and check out the latest code from CVS, you will see. If you don't, let me just say that blocked views are now a good option for very large views. Performance benefits are particularly good for views with many properties, and when traversing them sequentially. To switch to using blocked views, change code which looks like: vw = store.getas("vw[...]") to vm = store.getas("vw[_B[...]]").blocked() You'll also need to reload data, this change won't convert it for you. With a somewhat lower raw access performance, you'll get much more scalable views (millions of rows and more), faster commits, and smaller datafiles. There's no need to switch over every view - it's still a trade-off. If your views are rarely modified, or contain no strings, or are always accessed in random order (hash maps), then flat is often still better. But it's there if you want it. -jcw From ralf@brainbot.com Wed Apr 28 14:22:37 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 76C84B3AE8 for ; Wed, 28 Apr 2004 14:22:37 +0200 (CEST) Received: from mail.brainbot.com (unknown [145.253.163.18]) by dizzie.triqs.com (Postfix) with SMTP id A139DB3AE5 for ; Wed, 28 Apr 2004 14:22:34 +0200 (CEST) Received: (qmail 4396 invoked from network); 28 Apr 2004 12:22:34 -0000 Received: from stronzo.brainbot.com (192.168.10.43) by heini.brainbot.com with SMTP; 28 Apr 2004 12:22:34 -0000 Received: from stronzo.brainbot.com (localhost.brainbot.com [127.0.0.1]) i3SCN8oq091975; Wed, 28 Apr 2004 14:23:08 +0200 (CEST) (envelope-from ralf@brainbot.com) Received: (from ralf@localhost) by stronzo.brainbot.com (8.12.11/8.12.11/Submit) id i3SCN8l2091974; Wed, 28 Apr 2004 14:23:08 +0200 (CEST) (envelope-from ralf@brainbot.com) X-Authentication-Warning: stronzo.brainbot.com: ralf set sender to ralf@brainbot.com using -f To: Jean-Claude Wippler Subject: Re: [Metakit] Mk4py build & test Q References: <827ECE3A-95E4-11D8-94AE-000A9588127E@equi4.com> From: Ralf Schmitt Date: Wed, 28 Apr 2004 14:23:07 +0200 In-Reply-To: <827ECE3A-95E4-11D8-94AE-000A9588127E@equi4.com> (Jean-Claude Wippler's message of "Sat, 24 Apr 2004 13:42:48 +0200") Message-ID: <86isfki95g.fsf@stronzo.brainbot.com> User-Agent: Gnus/5.1006 (Gnus v5.10.6) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Apr 2004 12:22:38 -0000 --=-=-= Jean-Claude Wippler writes: > I have a question for Python experts, w.r.t. distutils: > > I'd like to try and get setup.py working on its own. Here's what I > get right now (cvs HEAD, build dir wiped): > > $ python setup.py build > running build > running build_py > creating ../builds/lib.linux-i686-2.3 > copying metakit.py -> ../builds/lib.linux-i686-2.3 > running build_ext > running config > gcc -E -I/usr/include/python2.3 -o _configtest.i _configtest.c > removing: _configtest.c _configtest.i > building 'Mk4py' extension > creating ../builds/temp.linux-i686-2.3 > creating ../builds/temp.linux-i686-2.3/scxx > g++ -fno-strict-aliasing -DNDEBUG -fPIC -DHAVE_UNICODEOBJECT_H=1 > -Iscxx -I../include -I/usr/include/python2.3 -c PyView.cpp -o > ../builds/temp.linux-i686-2.3/PyView.o > [...] > g++ -pthread -shared ../builds/temp.linux-i686-2.3/PyProperty.o > ../builds/temp.linux-i686-2.3/PyRowRef.o > ../builds/temp.linux-i686-2.3/PyStorage.o > ../builds/temp.linux-i686-2.3/PyView.o > ../builds/temp.linux-i686-2.3/scxx/PWOImp.o ../builds/column.o > ../builds/custom.o ../builds/derived.o ../builds/fileio.o > ../builds/field.o ../builds/format.o ../builds/handler.o > ../builds/persist.o ../builds/remap.o ../builds/std.o > ../builds/store.o ../builds/string.o ../builds/table.o > ../builds/univ.o ../builds/view.o ../builds/viewx.o -lstdc++ -o > ../builds/lib.linux-i686-2.3/Mk4py.so > g++: ../builds/column.o: No such file or directory > [...] > g++: ../builds/viewx.o: No such file or directory > error: command 'g++' failed with exit status 1 > $ > > Is there a simple way to resolve this? The workaround is to first do: > cd ../builds; ../unix/configure; make Attached is a setup.py which calls configure itself and uses python distutils to compile the source files. --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=setup.py Content-Transfer-Encoding: base64 IyEgL3Vzci9iaW4vZW52IHB5dGhvbgoKZnJvbSBkaXN0dXRpbHMuY29yZSBpbXBvcnQgc2V0dXAs IEV4dGVuc2lvbiwgQ29tbWFuZApmcm9tIGRpc3R1dGlscy5jb21tYW5kLmJ1aWxkIGltcG9ydCBi dWlsZApmcm9tIGRpc3R1dGlscy5jb21tYW5kLmJ1aWxkX2V4dCBpbXBvcnQgYnVpbGRfZXh0CmZy b20gZGlzdHV0aWxzLmNvbW1hbmQuY29uZmlnIGltcG9ydCBjb25maWcKZnJvbSBkaXN0dXRpbHMu bXN2Y2NvbXBpbGVyIGltcG9ydCBNU1ZDQ29tcGlsZXIKZnJvbSBkaXN0dXRpbHMgaW1wb3J0IHN5 c2NvbmZpZwppbXBvcnQgc3RyaW5nCmltcG9ydCBzeXMKaW1wb3J0IG9zCgpta29ianMgPSBbJ2Nv bHVtbicsICdjdXN0b20nLCAnZGVyaXZlZCcsICdmaWxlaW8nLCAnZmllbGQnLAogICAgICAgICAg J2Zvcm1hdCcsICdoYW5kbGVyJywgJ3BlcnNpc3QnLCAncmVtYXAnLCAnc3RkJywKICAgICAgICAg ICdzdG9yZScsICdzdHJpbmcnLCAndGFibGUnLCAndW5pdicsICd2aWV3JywgJ3ZpZXd4J10KCm1r c3JjID0gW29zLnBhdGguam9pbigiLi4iLCAic3JjIiwgIiVzLmNwcCIgJSB4KSBmb3IgeCBpbiBt a29ianNdCgpjbGFzcyBjb25maWd1cmUoQ29tbWFuZCk6CiAgICBkZWYgaW5pdGlhbGl6ZV9vcHRp b25zIChzZWxmKToKICAgICAgICBzZWxmLmNvbmZpZ3VyZV9wYXRoPW9zLnBhdGguYWJzcGF0aCgi Li4vdW5peC9jb25maWd1cmUiKQogICAgICAgIHNlbGYuYnVpbGRfdGVtcCA9IE5vbmUKICAgICAg ICAKICAgIGRlZiBmaW5hbGl6ZV9vcHRpb25zIChzZWxmKToKICAgICAgICBpZiBzZWxmLmJ1aWxk X3RlbXAgaXMgTm9uZToKICAgICAgICAgICAgc2VsZi5idWlsZF90ZW1wID0gc2VsZi5nZXRfZmlu YWxpemVkX2NvbW1hbmQoJ2J1aWxkJykuYnVpbGRfdGVtcAogICAgICAgICAgICBiYyA9IHNlbGYu Z2V0X2ZpbmFsaXplZF9jb21tYW5kKCdidWlsZF9leHQnKQogICAgICAgICAgICBpZiBzeXMucGxh dGZvcm09PSd3aW4zMic6CiAgICAgICAgICAgICAgICAjIGluc3RlYWQgb2YgcnVubmluZyBjb25m aWd1cmUgdXNlIC4uL3dpbi9jb25maWcuaAogICAgICAgICAgICAgICAgYmMuaW5jbHVkZV9kaXJz LmFwcGVuZCgnLi4vd2luJykKIyMgICAgICAgICAgICAgICAgIGlmIGJjLmNvbXBpbGVyPT0nbWlu Z3czMic6CiMjICAgICAgICAgICAgICAgICAgICAgICBiYy5saWJyYXJpZXMuYXBwZW5kKCdzdGRj KysnKQogICAgICAgICAgICBlbHNlOgogICAgICAgICAgICAgICAgYmMuaW5jbHVkZV9kaXJzLmFw cGVuZChzZWxmLmJ1aWxkX3RlbXApCgogICAgICAgICAgCiAgICBkZWYgcnVuIChzZWxmKToKICAg ICAgICBpZiBzeXMucGxhdGZvcm09PSd3aW4zMic6CiAgICAgICAgICAgIHJldHVybgogICAgICAg IAogICAgICAgIGlmIG9zLnBhdGguZXhpc3RzKG9zLnBhdGguam9pbihzZWxmLmJ1aWxkX3RlbXAs ICJjb25maWcuaCIpKToKICAgICAgICAgICAgcmV0dXJuCgogICAgICAgIHNlbGYubWtwYXRoKHNl bGYuYnVpbGRfdGVtcCkKICAgICAgICBjd2QgPSBvcy5nZXRjd2QoKQogICAgICAgIAogICAgICAg IHRyeToKICAgICAgICAgICAgb3MuY2hkaXIoc2VsZi5idWlsZF90ZW1wKQogICAgICAgICAgICBy ZXM9b3Muc3lzdGVtKHNlbGYuY29uZmlndXJlX3BhdGgpCiAgICAgICAgICAgIGlmIHJlcyE9MDoK ICAgICAgICAgICAgICAgIHJhaXNlIFN5c3RlbUV4aXQoImNvbmZpZ3VyZSBmYWlsZWQiKQogICAg ICAgIGZpbmFsbHk6CiAgICAgICAgICAgIG9zLmNoZGlyKGN3ZCkKCmNsYXNzIGJ1aWxkX21rKGJ1 aWxkKToKICAgIGRlZiBpbml0aWFsaXplX29wdGlvbnMoc2VsZik6CiAgICAgICAgIyBidWlsZCBp biBidWlsZHMgZGlyZWN0b3J5IGJ5IGRlZmF1bHQsIHVubGVzcyBzcGVjaWZpZWQgb3RoZXJ3aXNl CiAgICAgICAgYnVpbGQuaW5pdGlhbGl6ZV9vcHRpb25zKHNlbGYpCiAgICAgICAgc2VsZi5idWls ZF9iYXNlID0gJy4uL2J1aWxkcycKCmNsYXNzIGJ1aWxkX21rZXh0KGJ1aWxkX2V4dCk6CiAgICBk ZWYgcnVuKHNlbGYpOgogICAgICAgIHNlbGYucnVuX2NvbW1hbmQoJ2NvbmZpZ3VyZScpCiAgICAg ICAgYnVpbGRfZXh0LnJ1bihzZWxmKQogICAgICAgIAogICAgZGVmIGZpbmFsaXplX29wdGlvbnMo c2VsZik6CiAgICAgICAgIyBmb3JjZSB1c2Ugb2YgQysrIGNvbXBpbGVyIChoZWxwcyBvbiBzb21l IHBsYXRmb3JtcykKICAgICAgICBpbXBvcnQgb3MKICAgICAgICBjYyA9IG9zLmVudmlyb24uZ2V0 KCdDWFgnLCBzeXNjb25maWcuZ2V0X2NvbmZpZ192YXIoJ0NYWCcpKQogICAgICAgIGlmIG5vdCBj YzoKICAgICAgICAgICAgY2MgPSBzeXNjb25maWcuZ2V0X2NvbmZpZ192YXIoJ0NDQycpICMgUHl0 aG9uIDEuNS4yCiAgICAgICAgaWYgY2M6CiAgICAgICAgICAgIG9zLmVudmlyb25bJ0NDJ10gPSBj YwoKICAgICAgICBidWlsZF9leHQuZmluYWxpemVfb3B0aW9ucyhzZWxmKQoKICAgIGRlZiBidWls ZF9leHRlbnNpb24oc2VsZiwgZXh0KToKICAgICAgICAjIHdvcmsgYXJvdW5kIGxpbmtlciBwcm9i bGVtIHdpdGggTWFjUHl0aG9uIDIuMwogICAgICAgIGlmIHN5cy5wbGF0Zm9ybSA9PSAnZGFyd2lu JzoKICAgICAgICAgICAgdHJ5OgogICAgICAgICAgICAgICAgc2VsZi5jb21waWxlci5saW5rZXJf c28ucmVtb3ZlKCItV2wsLXgiKQogICAgICAgICAgICBleGNlcHQ6IHBhc3MKICAgICAgICAjIHdv cmsgYXJvdW5kIGxpbmtlciBwcm9ibGVtIHdpdGggTGludXgsIFB5dGhvbiAyLjIgYW5kIGVhcmxp ZXI6CiAgICAgICAgIyBkZXNwaXRlIHNldHRpbmcgJENDIGFib3ZlLCBzdGlsbCB1c2VzIFB5dGhv biBjb21waWxlcgogICAgICAgIGlmIHN5cy5wbGF0Zm9ybSA9PSAnbGludXgyJzoKICAgICAgICAg ICAgdHJ5OgogICAgICAgICAgICAgICAgZXh0LmxpYnJhcmllcy5hcHBlbmQoInN0ZGMrKyIpCiAg ICAgICAgICAgIGV4Y2VwdDogcGFzcwogICAgICAgIGlmIGV4dC5uYW1lID09ICJNazRweSI6CiAg ICAgICAgICAgIGlmIGlzaW5zdGFuY2Uoc2VsZi5jb21waWxlciwgTVNWQ0NvbXBpbGVyKToKICAg ICAgICAgICAgICAgIHN1ZmZpeCA9ICcub2JqJwogICAgICAgICAgICAgICAgaWYgc2VsZi5kZWJ1 ZzoKICAgICAgICAgICAgICAgICAgICBwcmVmaXggPSAnLi4vYnVpbGRzL21zdmM2MC9ta2xpYi9E ZWJ1Zy8nCiAgICAgICAgICAgICAgICBlbHNlOgogICAgICAgICAgICAgICAgICAgIHByZWZpeCA9 ICcuLi9idWlsZHMvbXN2YzYwL21rbGliL1JlbGVhc2UvJwogICAgICAgICAgICBlbHNlOgogICAg ICAgICAgICAgICAgc3VmZml4ID0gJy5vJwogICAgICAgICAgICAgICAgcHJlZml4ID0gJy4uL2J1 aWxkcy8nCiAgICAgICAgICAgIGZvciBpIGluIHJhbmdlKGxlbihleHQuZXh0cmFfb2JqZWN0cykp OgogICAgICAgICAgICAgICAgbm0gPSBleHQuZXh0cmFfb2JqZWN0c1tpXQogICAgICAgICAgICAg ICAgaWYgbm0gaW4gbWtvYmpzOgogICAgICAgICAgICAgICAgICAgIGlmIHN0cmluZy5maW5kKG5t LCAnLicpID09IC0xOgogICAgICAgICAgICAgICAgICAgICAgICBubSA9IG5tICsgc3VmZml4CiAg ICAgICAgICAgICAgICAgICAgbm0gPSBwcmVmaXggKyBubQogICAgICAgICAgICAgICAgICAgIGV4 dC5leHRyYV9vYmplY3RzW2ldID0gbm0KICAgICAgICBidWlsZF9leHQuYnVpbGRfZXh0ZW5zaW9u KHNlbGYsIGV4dCkKICAgIApjbGFzcyB0ZXN0X3JlZ3J0ZXN0KENvbW1hbmQpOgogICAgIyBPcmln aW5hbCB2ZXJzaW9uIG9mIHRoaXMgY2xhc3MgcG9zdGVkCiAgICAjIGJ5IEJlcnRob2xkIEhvZWxs bWFubiB0byBkaXN0dXRpbHMtc2lnQHB5dGhvbi5vcmcKICAgIGRlc2NyaXB0aW9uID0gInRlc3Qg dGhlIGRpc3RyaWJ1dGlvbiBwcmlvciB0byBpbnN0YWxsIgoKICAgIHVzZXJfb3B0aW9ucyA9IFsK ICAgICAgICAoJ2J1aWxkLWJhc2U9JywgJ2InLAogICAgICAgICAiYmFzZSBidWlsZCBkaXJlY3Rv cnkgKGRlZmF1bHQ6ICdidWlsZC5idWlsZC1iYXNlJykiKSwKICAgICAgICAoJ2J1aWxkLXB1cmVs aWI9JywgTm9uZSwKICAgICAgICAgImJ1aWxkIGRpcmVjdG9yeSBmb3IgcGxhdGZvcm0tbmV1dHJh bCBkaXN0cmlidXRpb25zIiksCiAgICAgICAgKCdidWlsZC1wbGF0bGliPScsIE5vbmUsCiAgICAg ICAgICJidWlsZCBkaXJlY3RvcnkgZm9yIHBsYXRmb3JtLXNwZWNpZmljIGRpc3RyaWJ1dGlvbnMi KSwKICAgICAgICAoJ2J1aWxkLWxpYj0nLCBOb25lLAogICAgICAgICAiYnVpbGQgZGlyZWN0b3J5 IGZvciBhbGwgZGlzdHJpYnV0aW9uIChkZWZhdWx0cyB0byBlaXRoZXIgIiArCiAgICAgICAgICJi dWlsZC1wdXJlbGliIG9yIGJ1aWxkLXBsYXRsaWIiKSwKICAgICAgICAoJ3Rlc3QtZGlyPScsIE5v bmUsCiAgICAgICAgICJkaXJlY3RvcnkgdGhhdCBjb250YWlucyB0aGUgdGVzdCBkZWZpbml0aW9u cyIpLAogICAgICAgICgndGVzdC1vcHRpb25zPScsIE5vbmUsCiAgICAgICAgICJjb21tYW5kLWxp bmUgb3B0aW9ucyB0byBwYXNzIHRvIHRlc3QucmVncnRlc3QiKQogICAgICAgIF0KCiAgICBkZWYg aW5pdGlhbGl6ZV9vcHRpb25zKHNlbGYpOgogICAgICAgIHNlbGYuYnVpbGRfYmFzZSA9IE5vbmUK ICAgICAgICAjIHRoZXNlIGFyZSBkZWNpZGVkIG9ubHkgYWZ0ZXIgJ2J1aWxkX2Jhc2UnIGhhcyBp dHMgZmluYWwgdmFsdWUKICAgICAgICAjICh1bmxlc3Mgb3ZlcnJpZGRlbiBieSB0aGUgdXNlciBv ciBjbGllbnQpCiAgICAgICAgc2VsZi5idWlsZF9wdXJlbGliID0gTm9uZQogICAgICAgIHNlbGYu YnVpbGRfcGxhdGxpYiA9IE5vbmUKICAgICAgICBzZWxmLnRlc3RfZGlyID0gJ3Rlc3QnCiAgICAg ICAgc2VsZi50ZXN0X29wdGlvbnMgPSBOb25lCiAgICAgICAgCiAgICBkZWYgZmluYWxpemVfb3B0 aW9ucyhzZWxmKToKICAgICAgICBidWlsZCA9IHNlbGYuZGlzdHJpYnV0aW9uLmdldF9jb21tYW5k X29iaignYnVpbGQnKQogICAgICAgIGJ1aWxkX29wdGlvbnMgPSAoJ2J1aWxkX2Jhc2UnLCAnYnVp bGRfcHVyZWxpYicsICdidWlsZF9wbGF0bGliJykKICAgICAgICBmb3Igb3B0aW9uIGluIGJ1aWxk X29wdGlvbnM6CiAgICAgICAgICAgIHZhbCA9IGdldGF0dHIoc2VsZiwgb3B0aW9uKQogICAgICAg ICAgICBpZiB2YWw6CiAgICAgICAgICAgICAgICBzZXRhdHRyKGJ1aWxkLCBvcHRpb24sIGdldGF0 dHIoc2VsZiwgb3B0aW9uKSkKICAgICAgICBidWlsZC5lbnN1cmVfZmluYWxpemVkKCkKICAgICAg ICBmb3Igb3B0aW9uIGluIGJ1aWxkX29wdGlvbnM6CiAgICAgICAgICAgIHNldGF0dHIoc2VsZiwg b3B0aW9uLCBnZXRhdHRyKGJ1aWxkLCBvcHRpb24pKQoKICAgIGRlZiBydW4oc2VsZik6CiAgICAg ICAgCiAgICAgICAgIyBJbnZva2UgdGhlICdidWlsZCcgY29tbWFuZCB0byAiYnVpbGQiIHB1cmUg UHl0aG9uIG1vZHVsZXMKICAgICAgICAjIChpZS4gY29weSAnZW0gaW50byB0aGUgYnVpbGQgdHJl ZSkKICAgICAgICBzZWxmLnJ1bl9jb21tYW5kKCdidWlsZCcpCgogICAgICAgICMgcmVtZW1iZXIg b2xkIHN5cy5wYXRoIHRvIHJlc3RvcmUgaXQgYWZ0ZXJ3YXJkcwogICAgICAgIG9sZF9wYXRoID0g c3lzLnBhdGhbOl0KCiAgICAgICAgIyBleHRlbmQgc3lzLnBhdGgKICAgICAgICBzeXMucGF0aC5p bnNlcnQoMCwgc2VsZi5idWlsZF9wdXJlbGliKQogICAgICAgIHN5cy5wYXRoLmluc2VydCgwLCBz ZWxmLmJ1aWxkX3BsYXRsaWIpCiAgICAgICAgc3lzLnBhdGguaW5zZXJ0KDAsIHNlbGYudGVzdF9k aXIpCiAgICAgICAgCiAgICAgICAgIyBVc2UgdGVzdC5yZWdydGVzdCwgdW5saWtlIHRoZSBvcmln aW5hbCB2ZXJzaW9uIG9mIHRoaXMgY2xhc3MKICAgICAgICBpbXBvcnQgdGVzdC5yZWdydGVzdAoK CSMgamN3IDIwMDQtMDQtMjYgLSB3aHkgZG8gSSBuZWVkIHRvIGFkZCB0aGVzZSBoZXJlIHRvIGZp bmQgdGhlIHRlc3RzPwoJI2ltcG9ydCBsZWFrdGVzdCAtIG5vdCB2ZXJ5IHBvcnRhYmxlCglpbXBv cnQgdGVzdF9pbnR0eXBlcwoJaW1wb3J0IHRlc3Rfc3RyaW5ndHlwZQoJI2ltcG9ydCB0ZXN0X2hh c2ggLSBkb2Vzbid0IHdvcmsKCSMgamN3IGVuZAoKICAgICAgICB0ZXN0LnJlZ3J0ZXN0LlNURFRF U1RTID0gW10KICAgICAgICB0ZXN0LnJlZ3J0ZXN0Lk5PVFRFU1RTID0gW10KCiAgICAgICAgaWYg c2VsZi50ZXN0X29wdGlvbnM6CiAgICAgICAgICAgIHN5cy5hcmd2WzE6XSA9IHN0cmluZy5zcGxp dChzZWxmLnRlc3Rfb3B0aW9ucywgJyAnKQogICAgICAgIGVsc2U6CiAgICAgICAgICAgIGRlbCBz eXMuYXJndlsxOl0KCiAgICAgICAgIyByZW1vdmUgc3RhbGUgbW9kdWxlcwogICAgICAgIGRlbCBz eXMubW9kdWxlc1snbWV0YWtpdCddCiAgICAgICAgdHJ5OgogICAgICAgICAgICBkZWwgc3lzLm1v ZHVsZXNbJ01rNHB5J10KICAgICAgICBleGNlcHQ6CiAgICAgICAgICAgIHBhc3MKCiAgICAgICAg c2VsZi5hbm5vdW5jZSgicnVubmluZyB0ZXN0cyIpCiAgICAgICAgdGVzdC5yZWdydGVzdC5tYWlu KHRlc3RkaXI9c2VsZi50ZXN0X2RpcikKCiAgICAgICAgIyByZXN0b3JlIHN5cy5wYXRoCiAgICAg ICAgc3lzLnBhdGggPSBvbGRfcGF0aFs6XQoKdHJ5OgogICAgaW1wb3J0IG1ldGFraXQKZXhjZXB0 OgogICAgbWV0YWtpdCA9IHN5cy5tb2R1bGVzWydtZXRha2l0J10KCnNldHVwKG5hbWUgICAgICAg ICAgICAgPSBtZXRha2l0Ll9fbmFtZV9fLAogICAgICB2ZXJzaW9uICAgICAgICAgID0gbWV0YWtp dC5fX3ZlcnNpb25fXywKICAgICAgZGVzY3JpcHRpb24gICAgICA9IG1ldGFraXQuX19kZXNjcmlw dGlvbl9fLAogICAgICBsb25nX2Rlc2NyaXB0aW9uID0gbWV0YWtpdC5fX2RvY19fLAogICAgICBh dXRob3IgICAgICAgICAgID0gbWV0YWtpdC5fX2F1dGhvcl9fLAogICAgICBhdXRob3JfZW1haWwg ICAgID0gbWV0YWtpdC5fX2VtYWlsX18sCiAgICAgIHVybCAgICAgICAgICAgICAgPSBtZXRha2l0 Ll9fdXJsX18sCiAgICAgIG1haW50YWluZXIgICAgICAgPSBtZXRha2l0Ll9fYXV0aG9yX18sCiAg ICAgIG1haW50YWluZXJfZW1haWwgPSBtZXRha2l0Ll9fZW1haWxfXywKICAgICAgbGljZW5zZSAg ICAgICAgICA9IG1ldGFraXQuX19saWNlbnNlX18sCiAgICAgIGtleXdvcmRzICAgICAgICAgPSBb J2RhdGFiYXNlJ10sCiAgICAgIHB5X21vZHVsZXMgICAgICAgPSBbJ21ldGFraXQnXSwKICAgICAg Y21kY2xhc3MgICAgICAgICA9IHsnYnVpbGQnOiBidWlsZF9taywgJ2J1aWxkX2V4dCc6IGJ1aWxk X21rZXh0LAogICAgICAgICAgICAgICAgICAgICAgICAgICd0ZXN0JzogdGVzdF9yZWdydGVzdCwg Iydjb25maWcnOiBjb25maWdfbWssCiAgICAgICAgICAgICAgICAgICAgICAgICAgJ2NvbmZpZ3Vy ZSc6IGNvbmZpZ3VyZX0sCiAgICAgIGV4dF9tb2R1bGVzICAgICAgPSBbRXh0ZW5zaW9uKCJNazRw eSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNvdXJjZXM9WyJQeVByb3Bl cnR5LmNwcCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJQ eVJvd1JlZi5jcHAiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAiUHlTdG9yYWdlLmNwcCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICJQeVZpZXcuY3BwIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgInNjeHgvUFdPSW1wLmNwcCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIF0rbWtzcmMsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIGluY2x1ZGVfZGlycz1bInNjeHgiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICIuLi9pbmNsdWRlIl0sCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIGRlZmluZV9tYWNyb3M9WygiSEFWRV9DT05GSUdfSCIsIDEpLCAoInE0X0NI RUNLIiwgMSldLAoKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgI2V4dHJhX29i amVjdHM9bWtvYmpzLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICApXQogICAg ICApCgojIyBMb2NhbCBWYXJpYWJsZXM6CiMjIGNvbXBpbGUtY29tbWFuZDogInB5dGhvbiBzZXR1 cC5weSBidWlsZCAtYiAuLi9idWlsZHMiCiMjIEVuZDoK --=-=-= -- brainbot technologies ag boppstrasse 64 . 55118 mainz . germany fon +49 6131 211639-1 . fax +49 6131 211639-2 http://brainbot.com/ mailto:ralf@brainbot.com --=-=-=-- From bkelley@wi.mit.edu Wed Apr 28 16:43:50 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C3FECB3AE8; Wed, 28 Apr 2004 16:43:50 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 24CDCB3AE5; Wed, 28 Apr 2004 16:43:48 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004042810432609218 ; Wed, 28 Apr 2004 10:43:26 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HWVYAO00.K2L; Wed, 28 Apr 2004 10:44:48 -0400 Message-ID: <408FC2EE.3070709@wi.mit.edu> Date: Wed, 28 Apr 2004 10:42:54 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler , Metakit mailing list Subject: Re: [Metakit] blocked views References: <90E5D0AE-98F4-11D8-AF11-000A9588127E@equi4.com> In-Reply-To: <90E5D0AE-98F4-11D8-AF11-000A9588127E@equi4.com> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Apr 2004 14:43:51 -0000 Jean-Claude Wippler wrote: > With a somewhat lower raw access performance, you'll get much more > scalable views (millions of rows and more), faster commits, and > smaller datafiles. > > There's no need to switch over every view - it's still a trade-off. > If your views are rarely modified, or contain no strings, or are > always accessed in random order (hash maps), then flat is often > still better. But it's there if you want it. I'll just throw in my two cents. I switched my application over to blocked views a couple of months ago and have never looked back. I began to have a problem with random-access and hash maps around 100,000 objects in a particular table. Eventually the database would refuse to commit data anymore. The solution was to convert to blocked views and use .blocked().ordered() for random access of key values. In practice, there has been little noticable difference between the application using hashes and the application using .blocked().ordered() One caveat is that when searching the blocked and ordered tables you should use "search" and not "find" for a couple of reasons described on the mailing list. Currently my database is over 1 gig and the main table has 1+ million entries. One to one joins are still incredibly fast on such tables and I couldn't be happier. Suffice it to say, that I'm all for any improvements in blocked table performance. Brian > > -jcw > > _____________________________________________ Metakit mailing list > - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From Murat_Berk@bmc.com Thu Apr 29 20:05:19 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 290CEB3AEA for ; Thu, 29 Apr 2004 20:05:18 +0200 (CEST) Received: from mangrove.bmc.com (fw-us-hou19.bmc.com [198.207.223.240]) by dizzie.triqs.com (Postfix) with ESMTP id 9309EB3AE4 for ; Thu, 29 Apr 2004 20:05:13 +0200 (CEST) Received: from mangrove.bmc.com (localhost [127.0.0.1]) by localhost.bmc.com (Postfix) with ESMTP id BDB7146C5 for ; Thu, 29 Apr 2004 13:07:02 -0500 (CDT) Received: from EC02-HOU.BMC.COM (ec02-hou.bmc.com [172.17.0.151]) by mangrove.bmc.com (Postfix) with ESMTP id 91F6846B0 for ; Thu, 29 Apr 2004 13:07:02 -0500 (CDT) Received: by ec02-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 29 Apr 2004 13:05:09 -0500 Message-ID: <3DAC9CD31458D411BCE700D0B75D0A130CAA204D@ES09-HOU.bmc.com> From: "Berk, Murat" To: "'metakit@equi4.com'" Date: Thu, 29 Apr 2004 13:02:59 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] metakit tests are failing on RHAS 3.0 IA64 when inlining enabled and optimization is on. X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Apr 2004 18:05:20 -0000 When compiling for debug, inline option does not matter. When we compile for production, unit tests are failing and test program is crashing.. Did anybody else observed this? Compiler is standard RH compiler gcc-c++-3.2.3-24 Flags used: g++ -fPIC -Wall -ansi -Wno-unused -Wno-ctor-dtor-privacy -fno-implicit-inline-templates -Wno-non-template-friend -Wno-deprecated -g -O2 -Dos_linux=24 -Darch_ia64 -DBMC_ARCHBITS=64 -Dos_type_unix -Dcompiler_gxx=323 -Dlibrary_type_shared -Dthread_type_pthread Copying platform specific data file... cp ../unix/reversed . Starting test program... b00 - Should fail Failed: A(false) b01 - Should succeed b02 - Int property b03 - Float property b04 - String property b05 - View property b06 - View construction b07 - Row manipulation b08 - Row expressions b09 - View manipulation b10 - View sorting b11 - View selection b12 - Add after remove b13 - Clear view entry b14 - Empty view outlives temp storage b15 - View outlives temp storage b16 - View outlives cleared temp storage b17 - Double property b18 - SetAtGrow usage b19 - Bytes property b20 - Search sorted view Failed: A(p1 (v2[1]) == 123) Failed: A(p1 (v2[2]) == 222) Failed: A(p1 (v2[3]) == 234) Failed: A(p1 (v2[4]) == 333) Failed: A(p1 (v2[5]) == 345) Failed: A(p1 (v3[1]) == 345) Failed: A(p1 (v3[2]) == 111) Failed: A(p1 (v3[3]) == 123) Failed: A(p1 (v3[4]) == 333) Failed: A(p1 (v3[5]) == 222) Failed: A(p1 (v4[1]) == 333) Failed: A(p1 (v4[2]) == 123) Failed: A(p1 (v4[3]) == 111) Failed: A(p1 (v4[4]) == 345) Failed: A(p1 (v4[5]) == 234) b21 - Memo property b22 - Stored view references b23 - Sort comparison fix b24 - Custom view comparisons b25 - Copy row from derived b26 - Partial memo field access b27 - Copy value to another row n01 - Add to selection Failed: A(p1 (v2[1]) == 333) Failed: A(p1 (v2[2]) == 234) n02 - Remove from selection n03 - Modify into selection n04 - Modify out of selection n05 - Add to sorted n06 - Remove from sorted n07 - New property through sort n08 - Nested project and select n09 - Multiple dependencies Failed: A(p1 (v2[1]) == 333) Failed: A(p1 (v2[2]) == 234) n10 - Modify sorted duplicates n11 - Resize compound derived view n12 - Alter multiply derived view n13 - Project without n14 - Insert in non-mapped position c01 - Slice forward c02 - Slice backward c03 - Slice reverse c04 - Cartesian product Failed: A(p2 (v3[1]) == 222) Failed: A(p1 (v3[2]) == 234) Failed: A(p1 (v3[3]) == 234) Failed: A(p2 (v3[3]) == 222) Failed: A(p1 (v3[4]) == 345) Failed: A(p1 (v3[5]) == 345) Failed: A(p2 (v3[5]) == 222) c05 - Remapping c06 - Pairwise combination Failed: A(p1 (v3[1]) == 234) Failed: A(p2 (v3[1]) == 222) Failed: A(p1 (v3[2]) == 345) Failed: A(p2 (v3[2]) == 333) c07 - Concatenate views c08 - Rename property c09 - GroupBy operation Failed: A(p2 (v3[0]) == 1) Failed: A(v3.GetSize() == 2) Failed: A(p2 (v3[0]) == 1) Failed: A(p2 (v3[1]) == 2) Failed: A(v3.GetSize() == 3) Failed: A(p2 (v3[1]) == 2) Failed: A(p2 (v3[2]) == 3) c10 - Counts operation c22 - Groupby with selection Failed: A(p3 (v3[1]) == 20) Failed: A(p3 (v4[0]) == 30) make: *** [post_target] Segmentation fault From gb@murphy.bofh.ms Tue May 4 15:51:44 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A509AB3AF0 for ; Tue, 4 May 2004 15:51:44 +0200 (CEST) Received: from timmy.gws-muenster.de (timmy.gws-online.de [80.146.170.26]) by dizzie.triqs.com (Postfix) with ESMTP id 0339CB3AE5 for ; Tue, 4 May 2004 15:51:42 +0200 (CEST) Received: from pd950c38a.dip.t-dialin.net ([217.80.195.138] helo=firewall.bofh.ms ident=Debian-exim) by timmy.gws-muenster.de with asmtp (Exim 3.36 #1 (Debian)) id 1BL0Kj-00032J-00 for ; Tue, 04 May 2004 15:51:41 +0200 Received: from localhost ([127.0.0.1] helo=goggle.bofh.ms ident=www-data) by firewall.bofh.ms with esmtp (Exim 4.31) id 1BL0Kh-0008Ub-QI for metakit@equi4.com; Tue, 04 May 2004 15:51:42 +0200 Received: from 192.168.167.1 (SquirrelMail authenticated user gb); by goggle.bofh.ms with HTTP; Tue, 4 May 2004 15:51:39 +0200 (CEST) Message-ID: <54521.192.168.167.1.1083678699.squirrel@192.168.167.1> Date: Tue, 4 May 2004 15:51:39 +0200 (CEST) From: "Georg Bauer" To: metakit@equi4.com User-Agent: SquirrelMail/1.5.0 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Problem with metakit file X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2004 13:51:45 -0000 Hi! I have a metakit file that hangs on opening. I have this short Python script: db = metakit.storage('settings.dat') v = db.view('users') this already hangs. Anyone has any tools that try to fix integrity problems with metakit files? This file is rather important (it's the configuration of my weblog server muensterland.org), so I would really like to get the data back :-) bye, Georg From bkelley@wi.mit.edu Tue May 4 16:24:49 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id BB8B0B3AEF for ; Tue, 4 May 2004 16:24:49 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 8D059B3AE5 for ; Tue, 4 May 2004 16:24:47 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004050410242118895 for ; Tue, 04 May 2004 10:24:21 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HX71EV00.JCA; Tue, 4 May 2004 10:25:43 -0400 Message-ID: <4097A782.6000901@wi.mit.edu> Date: Tue, 04 May 2004 10:24:02 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Georg Bauer , Metakit mailing list Subject: Re: [Metakit] Problem with metakit file References: <54521.192.168.167.1.1083678699.squirrel@192.168.167.1> In-Reply-To: <54521.192.168.167.1.1083678699.squirrel@192.168.167.1> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2004 14:24:50 -0000 Georg Bauer wrote: > Hi! > > I have a metakit file that hangs on opening. I have this short > Python script: > > db = metakit.storage('settings.dat') v = db.view('users') > > this already hangs. Anyone has any tools that try to fix integrity > problems with metakit files? This file is rather important (it's > the configuration of my weblog server muensterland.org), so I would > really like to get the data back :-) It might just be a typo, but you never know... The script you wrote shouldn't work. You need to open the storage using a mode: db = metakit.storage('settings.dat', 0) v = db.view('users') Brian Kelley > > bye, Georg > > > > _____________________________________________ Metakit mailing list > - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From gb@murphy.bofh.ms Tue May 4 17:57:51 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 787A8B3AEF for ; Tue, 4 May 2004 17:57:51 +0200 (CEST) Received: from timmy.gws-muenster.de (timmy.gws-online.de [80.146.170.26]) by dizzie.triqs.com (Postfix) with ESMTP id D9FC6B3AE5 for ; Tue, 4 May 2004 17:57:48 +0200 (CEST) Received: from pd950c38a.dip.t-dialin.net ([217.80.195.138] helo=firewall.bofh.ms ident=Debian-exim) by timmy.gws-muenster.de with asmtp (Exim 3.36 #1 (Debian)) id 1BL2Im-0004bT-00; Tue, 04 May 2004 17:57:48 +0200 Received: from tex.bofh.ms ([10.0.0.145] helo=murphy.bofh.ms) by firewall.bofh.ms with esmtp (Exim 4.31) id 1BL2Im-0001cl-Hu; Tue, 04 May 2004 17:57:49 +0200 Date: Tue, 4 May 2004 17:57:45 +0200 Subject: Re: [Metakit] Problem with metakit file Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) To: "Brian Kelley" From: Georg Bauer In-Reply-To: <4097A782.6000901@wi.mit.edu> Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit mailing list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2004 15:57:51 -0000 Hi! > It might just be a typo, but you never know... The script you wrote > shouldn't work. You need to open the storage using a mode: Yep, was a typo. I have it running again, but with an older version of the file. So the problem isn't too pressing at the moment, but to know what to do in future situations like this it would be interesting to know _why_ exactly it hangs. The "users" view is just a view with a subview per row. Nothing complicated. But up to now I never had a problem with metakit files being broken. But this file can't be opened by metakit. bye, Georg From jcw@equi4.com Tue May 4 18:04:58 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 81E97B3AEF for ; Tue, 4 May 2004 18:04:58 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 0E2E3B3AE5 for ; Tue, 4 May 2004 18:04:57 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 1D9FC83C26; Tue, 4 May 2004 18:04:55 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 78A098393A for ; Tue, 4 May 2004 18:04:48 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: References: Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Problem with metakit file Date: Tue, 4 May 2004 18:04:47 +0200 To: Metakit list X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2004 16:04:58 -0000 Georg Bauer wrote: > But this file can't be opened by metakit. Try the mkstats utility mentioned at the end of: http://www.equi4.com/metakit.html If errors are reported, then the file is definitely damaged. One cause for this is having a MK file open more than once in r/w mode - if things are mixed up, writes over each other will lead to a damaged datafile. For example, this can happen when quitting an app and launching it again before the first instance has really exited and flushed its I/O. -jcw From bkelley@wi.mit.edu Tue May 4 20:30:52 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id B9677B3AEF for ; Tue, 4 May 2004 20:30:52 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 7801AB3AE5 for ; Tue, 4 May 2004 20:30:49 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004050414302323436 for ; Tue, 04 May 2004 14:30:23 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HX7CSX00.NF6 for ; Tue, 4 May 2004 14:31:45 -0400 Message-ID: <4097E12B.6040904@wi.mit.edu> Date: Tue, 04 May 2004 14:30:03 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list Subject: Re: [Metakit] Problem with metakit file References: In-Reply-To: X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: multipart/mixed; boundary="------------050603000709060909050005" X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2004 18:30:53 -0000 This is a multi-part message in MIME format. --------------050603000709060909050005 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Jean-Claude Wippler wrote: > Georg Bauer wrote: > >> But this file can't be opened by metakit. > > > Try the mkstats utility mentioned at the end of: > http://www.equi4.com/metakit.html > > If errors are reported, then the file is definitely damaged. > > One cause for this is having a MK file open more than once in r/w > mode - if things are mixed up, writes over each other will lead to > a damaged datafile. For example, this can happen when quitting an > app and launching it again before the first instance has really > exited and flushed its I/O. > There is a pretty simple python solution for detecting multiple open files. I have a common database that multiple users can open so I needed a mechanism to prevent multiple people from opening the same database. Metakit allows for a super header that is appended before the metakit database. This can be any binary data that doesn't contain the metakit magic string. Here is a simple python recipe that you can call to see if a database is opened or not. What it does is it transparently wraps a storage and puts in a 32 byte header to store various flags. One of which is "is this database open" when the storage closes down and is deleted, it resets the superheader flags. You can see the test cases at the end... Other wise the storage can be used as a normal storage. My metakit server actually stores the ip address and socket of the host machine for the database and automagically connects to the server to use the database :) I stole this from FileMaker I think. Anyway, enjoy... --------------050603000709060909050005 Content-Type: text/plain; name="test_mk_info.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="test_mk_info.py" import metakit, array, os OPEN_STORAGE = 0x01 CLOSED_STORAGE = 0x00 OPEN_STORAGE_STR = array.array('i', [0x01]).tostring() CLOSED_STORAGE_STR = array.array('i', [0]).tostring() class storage: def __init__(self, filename, mode): self.filename = filename self.mode = mode self.st = None if mode == 0: self.st = metakit.storage(filename, mode) else: if os.path.exists(filename): # check the superheader file = open(filename, "rb") buffer = array.array("i", file.read(32)) file.close() if buffer[0] & OPEN_STORAGE: raise IOError("Database already open") file = open(filename, 'r+b') file.write(OPEN_STORAGE_STR) file.close() self.st = metakit.storage(filename, mode) else: a = array.array("i", [OPEN_STORAGE,0,0,0]) file = open(filename, 'wb') file.write(a.tostring()) file.close() self.st = metakit.storage(filename, mode) def __del__(self): self.close(True) def close(self,noerrors=False): if not noerrors and not self.st: raise IOError("Database already closed") del self.st self.st = None if self.mode and os.path.exists(self.filename): file = open(self.filename, 'r+b') file.write(CLOSED_STORAGE_STR) file.close() del file def __getattr__(self, attr): return getattr(self.st, attr) def forceopen(filename): """Force a storage to open, the storage must exist""" file = open(self.filename, 'r+b') file.write(CLOSED_STORAGE_STR) file.close() def test(): s2 = storage("foo8", 1) try: s2 = storage("foo8", 1) except IOError: print "caught error, database is open" del s2 s2 = storage("foo8", 1) s2.close() os.remove("foo8") --------------050603000709060909050005-- From wkvmk@netshark.com Tue May 4 21:38:55 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6D4BCB3AEF for ; Tue, 4 May 2004 21:38:55 +0200 (CEST) Received: from chrysalis.netshark.com (63-227-180-195.clsp.qwest.net [63.227.180.195]) by dizzie.triqs.com (Postfix) with ESMTP id 20FF2B3AE5 for ; Tue, 4 May 2004 21:38:54 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by chrysalis.netshark.com (8.12.8/8.12.8) with ESMTP id i44JcqOl031443 for ; Tue, 4 May 2004 13:38:52 -0600 From: "William K. Volkman" To: metakit@equi4.com Content-Type: text/plain Message-Id: <1083699532.30580.42.camel@chrysalis.netshark.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.5 (1.4.5-2) Date: Tue, 04 May 2004 13:38:52 -0600 Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] revisit 'Right' way to delete rows from a view in python? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2004 19:38:55 -0000 I'm pondering the list archive however haven't yet spotted something that explains this. Given the below code, is there anyway to delete just the rows selected by the filter? db = metakit.storage('/var/tmp/test.mk',1) vw = db.getas('test[f1:I,f2:s,seq:I,state:s]') for i in xrange(5): vw.append((i,'%d'%i,99,'F')) num = random.randint(0,4) if num > 0: for s in range(num): vw.append((i,'%d'%i,s,'%d'%s)) db.commit() metakit.dump(vw) pickon = random.randint(0,4) print "******** Entry to pick on",pickon subvw = vw.select(f1=pickon) rs = subvw.filter(lambda row: row.state == 'F') subsubvw = subvw.remapwith(rs) metakit.dump(subsubvw) # ## Now I want remove the filter rows, how can I back track? # ## If I use ##vw.remove(rs) ## It only works if order is perfect # ## The delete and remove methods are not available on the derived views ##subvw.remove(rs) ##subsubvw.remove(rs) # vw.remove(vw.indices(subvw)) # Deletes all 'pickon' records, not just the ones in state 'F' Like the poster in January, I'm banging my head on my desk. Metakit 2.4.9.3 on linux RH9 (with python 2.2) From gb@murphy.bofh.ms Tue May 4 22:52:26 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 6D7CBB3AEA; Tue, 4 May 2004 22:52:26 +0200 (CEST) Received: from timmy.gws-muenster.de (timmy.gws-online.de [80.146.170.26]) by dizzie.triqs.com (Postfix) with ESMTP id C47DBB3AE5; Tue, 4 May 2004 22:52:23 +0200 (CEST) Received: from pd950c38a.dip.t-dialin.net ([217.80.195.138] helo=firewall.bofh.ms ident=Debian-exim) by timmy.gws-muenster.de with asmtp (Exim 3.36 #1 (Debian)) id 1BL6tr-0008KY-00; Tue, 04 May 2004 22:52:23 +0200 Received: from tex.bofh.ms ([10.0.0.145] helo=murphy.bofh.ms) by firewall.bofh.ms with esmtp (Exim 4.31) id 1BL6ts-0004pU-4Z; Tue, 04 May 2004 22:52:25 +0200 Date: Tue, 4 May 2004 22:52:19 +0200 Subject: Re: [Metakit] Problem with metakit file Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) To: Jean-Claude Wippler From: Georg Bauer In-Reply-To: Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2004 20:52:26 -0000 Hi! > Try the mkstats utility mentioned at the end of: > http://www.equi4.com/metakit.html I get the following: simon:~# ./tclkit mkstats.kit settings.dat-broken settings.dat-broken: /root/mku.linux: /lib/libc.so.6: version `GLIBC_2.3' not found (required by /root/mku.linux) Weird, especially since it is a static executable ... (at least I downloaded the static tclkit and ldd says it's a static executable). > One cause for this is having a MK file open more than once in r/w mode > - if things are mixed up, writes over each other will lead to a > damaged datafile. For example, this can happen when quitting an app > and launching it again before the first instance has really exited and > flushed its I/O. In my case it was a classic hard disc crash, so it might be that the settings file was already damaged when the last backup was made. bye, Georg From gb@bofh.ms Tue May 4 13:48:09 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E71EBB3AEF for ; Tue, 4 May 2004 13:48:08 +0200 (CEST) Received: from timmy.gws-muenster.de (timmy.gws-online.de [80.146.170.26]) by dizzie.triqs.com (Postfix) with ESMTP id 5A3F4B3AE5 for ; Tue, 4 May 2004 13:48:06 +0200 (CEST) Received: from pd950c38a.dip.t-dialin.net ([217.80.195.138] helo=firewall.bofh.ms ident=Debian-exim) by timmy.gws-muenster.de with asmtp (Exim 3.36 #1 (Debian)) id 1BKyP7-0001Tt-00 for ; Tue, 04 May 2004 13:48:05 +0200 Received: from localhost ([127.0.0.1] helo=goggle.bofh.ms ident=www-data) by firewall.bofh.ms with esmtp (Exim 4.31) id 1BKyP5-00074t-H6 for metakit@equi4.com; Tue, 04 May 2004 13:48:06 +0200 Received: from 192.168.167.1 (SquirrelMail authenticated user gb); by goggle.bofh.ms with HTTP; Tue, 4 May 2004 13:48:03 +0200 (CEST) Message-ID: <55573.192.168.167.1.1083671283.squirrel@192.168.167.1> Date: Tue, 4 May 2004 13:48:03 +0200 (CEST) From: "Georg Bauer" To: metakit@equi4.com User-Agent: SquirrelMail/1.5.0 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 Importance: Normal X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Wed, 05 May 2004 11:56:07 +0200 Subject: [Metakit] Problem with metakit file X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2004 11:48:09 -0000 Hi! I have a metakit file that hangs on opening. I have this short Python script: db = metakit.storage('settings.dat') v = db.view('users') this already hangs. Anyone has any tools that try to fix integrity problems with metakit files? This file is rather important (it's the configuration of my weblog server muensterland.org), so I would really like to get the data back :-) bye, Georg From gmcm@hypernet.com Wed May 5 14:24:39 2004 Return-Path: X-Original-To: Metakit@equi4.com Delivered-To: Metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id DED87B3AEC for ; Wed, 5 May 2004 14:24:38 +0200 (CEST) Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by dizzie.triqs.com (Postfix) with ESMTP id BCB45B3AE4 for ; Wed, 5 May 2004 14:24:37 +0200 (CEST) Received: from eddie.twinoaks.net (208.229.12.2) by hypernet.com with ESMTP (Eudora Internet Mail Server X 3.2.3) for ; Wed, 5 May 2004 08:24:35 -0400 From: Gordon McMillan To: Metakit@equi4.com Subject: Re: [Metakit] revisit 'Right' way to delete rows from a view in python? Date: Wed, 5 May 2004 08:28:19 -0400 User-Agent: KMail/1.6.1 References: <1083699532.30580.42.camel@chrysalis.netshark.com> In-Reply-To: <1083699532.30580.42.camel@chrysalis.netshark.com> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <200405050828.19620.gmcm@hypernet.com> X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: gmcm@hypernet.com List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2004 12:24:39 -0000 On Tuesday 04 May 2004 03:38 pm, William K. Volkman wrote: > db =3D metakit.storage('/var/tmp/test.mk',1) > vw =3D db.getas('test[f1:I,f2:s,seq:I,state:s]') > for i in xrange(5): > =A0 =A0 vw.append((i,'%d'%i,99,'F')) > =A0 =A0 num =3D random.randint(0,4) > =A0 =A0 if num > 0: > =A0 =A0 =A0 =A0 for s in range(num): > =A0 =A0 =A0 =A0 =A0 =A0 vw.append((i,'%d'%i,s,'%d'%s)) > =A0 =A0 db.commit() > metakit.dump(vw) > > pickon =3D random.randint(0,4) > print "******** Entry to pick on",pickon > > subvw =3D vw.select(f1=3Dpickon) > rs =3D subvw.filter(lambda row: row.state =3D=3D 'F') Why filter the selected view? Now you've got the indices in the selected vi= ew,=20 and getting the true indices in the original view takes something like this: iv1 =3D vw.indices(subvw) iv2 =3D iv1.remapwith(rs) vw.remove(iv2)=20 Why not dead =3D vw.filter(lambda row: row.f1=3D=3Dpickon and row.state =3D=3D 'F') vw.remove(dead) > subsubvw =3D subvw.remapwith(rs) > metakit.dump(subsubvw) > # > ## Now I want remove the filter rows, how can I back track? > # > ## If I use > ##vw.remove(rs) > ## It only works if order is perfect > # > ## The delete and remove methods are not available on the derived views > ##subvw.remove(rs) > ##subsubvw.remove(rs) > # > vw.remove(vw.indices(subvw)) > # Deletes all 'pickon' records, not just the ones in state 'F' =2D- Gordon From wkvmk@netshark.com Thu May 6 02:52:22 2004 Return-Path: X-Original-To: Metakit@equi4.com Delivered-To: Metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 2B890B3AF2 for ; Thu, 6 May 2004 02:52:22 +0200 (CEST) Received: from chrysalis.netshark.com (63-227-180-195.clsp.qwest.net [63.227.180.195]) by dizzie.triqs.com (Postfix) with ESMTP id 47C71B3AEF for ; Thu, 6 May 2004 02:52:18 +0200 (CEST) Received: from localhost (localhost [127.0.0.1])i460qGOl015827; Wed, 5 May 2004 18:52:16 -0600 Subject: Re: [Metakit] revisit 'Right' way to delete rows from a view in python? From: "William K. Volkman" To: gmcm@hypernet.com In-Reply-To: <200405050828.19620.gmcm@hypernet.com> References: <1083699532.30580.42.camel@chrysalis.netshark.com> <200405050828.19620.gmcm@hypernet.com> Content-Type: text/plain Message-Id: <1083804735.11072.86.camel@chrysalis.netshark.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.5 (1.4.5-2) Date: Wed, 05 May 2004 18:52:16 -0600 Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2004 00:52:22 -0000 The problem of course is that the small example doesn't really convey the complexity of the application. On Wed, 2004-05-05 at 06:28, Gordon McMillan wrote: > Why filter the selected view? Now you've got the indices in the selected view, > and getting the true indices in the original view takes something like this: > iv1 = vw.indices(subvw) > iv2 = iv1.remapwith(rs) > vw.remove(iv2) The reason for the select was because I needed my results today ;-) Using just filter it takes around 1.7 seconds to examine one subset of data. Performing a select first and then filtering for records of interest took about 1.1 seconds. Given that I may need to perform this operation two to twenty times per selected area the incremental cost per filter operation on the selected data turned out to be about .05 second. So in short, 1.1 + 3*0.05 = 1.25 vs. 4*1.7=6.8 for each dataset. Now there are 20000 to 52000 of these data sets collected each day so the rough math shows: 52000*1.25/60/60 = 18 hrs vs. 52000*6.8/60/60 = 98 hrs filtering (and before any one asks, the data sets are financial not physics related). Initially tried your above suggestion however mistakenly I used the wholly filtered view: f1 f2 seq state -- -- --- ----- 3 3 99 F -- -- --- ----- Total: 1 rows Segmentation fault (core dumped) A little unfriendly for a python environment, I don't have time to dig in however below is the stack dump if anyone is curious. A metakit.dump of the iv1 returned: index ----- -1 -1 -1 ----- Total: 3 rows Note this was after three attempts to run the program. IMHO it would have been better to throw an exception instead of returning data that could cause catastrophic error or data loss, -1 would mean the last row if interpreted by the python code, this is probably already on someones plate waiting for cycles to get to it so please consider this in the spirit of a data point and not a criticism. Switching back to the select first, then filtered view, your suggestion worked. Thank you very much, I had not considered applying the filter result to an indices view from the select. In keeping the concept of data views separate this would not have occurred to me. I will keep this in mind for the future. Even with it taking 1.25 seconds this was close to intolerable, so using BFI on the sorted data (reading the whole data set once keeping track of the state of everything) it's down to 0.65 seconds which is still higher than I would like, I may have to investigate other alternatives. gdb /usr/bin/python2.2 core.15440 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...(no debugging symbols found)... warning: core file may not match specified executable file. Core was generated by `python wkv.py'. Program terminated with signal 11, Segmentation fault. Loaded symbols for /usr/lib/python2.2/lib-dynload/strop.so #0 c4_ColOfInts::Get_8i(int) (this=0x8178da4, index_=3) at column.inl:55 55 column.inl: No such file or directory. in column.inl (gdb) Current language: auto; currently c++ (gdb) info stack #0 c4_ColOfInts::Get_8i(int) (this=0x8178da4, index_=3) at column.inl:55 #1 0x4009c983 in c4_ColOfInts::Get(int, int&) (this=0x1, index_=135766136, length_=@0x817a078) at ../src/column.cpp:1264 #2 0x400a6490 in c4_FormatX::Get(int, int&) (this=0x8178d98, index_=135766136, length_=@0x817a078) at ../src/format.cpp:127 #3 0x400aa093 in c4_Handler::GetBytes(int, c4_Bytes&, bool) (this=0x1, index_=135766136, buf_=@0x817a078, copySmall_=false) at ../src/handler.cpp:58 #4 0x400bb15b in c4_Sequence::Get(int, int, c4_Bytes&) (this=0x817a020, index_=1, propId_=135766136, buf_=@0x1) at ../src/viewx.cpp:356 #5 0x400b78a3 in c4_View::GetItem(int, int, c4_Bytes&) const (this=0x817a020, row_=1, col_=135766136, buf_=@0x817a078) at mk4.inl:28 #6 0x4009e86e in c4_RemapWithViewer::GetItem(int, int, c4_Bytes&) (this=0x817a020, row_=135761316, col_=135766136, buf_=@0x817a078) at ../src/custom.cpp:425 #7 0x4009da9c in c4_CustomSeq::DoGet(int, int, c4_Bytes&) const (this=0x1, row_=135766136, col_=135766136, buf_=@0x817a078) at ../src/custom.cpp:170 #8 0x4009d557 in c4_CustomHandler::Get(int, int&) (this=0x81797d8, index_=135766136, length_=@0x1) at ../src/custom.cpp:65 #9 0x400aa093 in c4_Handler::GetBytes(int, c4_Bytes&, bool) (this=0x1, index_=135766136, buf_=@0x817a078, copySmall_=true) at ../src/handler.cpp:58 #10 0x400a3707 in c4_SortSeq::LessThan(long, long) (this=0x8189670, a=1, b=0) at ../src/derived.cpp:509 #11 0x400a38f9 in c4_SortSeq::MergeSortThis(long*, int, long*) (this=0x8189670, ar=0x81390d8, size=1, scratch=0x817b168) at ../src/derived.cpp:558 #12 0x400a3b1e in c4_SortSeq::MergeSort(long*, int) (this=0x817a078, ar=0x81390d8, size=2) at ../src/derived.cpp:641 #13 0x400a3f25 in c4_SortSeq (this=0x81390e0, seq_=@0x817a078, down_=0x0) at univ.inl:105 #14 0x400a4c4e in f4_CreateSort(c4_Sequence&, c4_Sequence*) (seq_=@0x817a078, down_=0x817a078) at ../src/derived.cpp:994 #15 0x400b806f in c4_View::Sort() const (this=0x817a078) at ../src/view.cpp:448 #16 0x40098589 in PyView::remove(PyView const&) (this=0x8177e08, indices=@0x8189670) at ../python/PyView.cpp:1319 #17 0x40095958 in PyView_remove (o=0x817a078, _args=0x817a078) at ../python/PyView.cpp:760 #18 0x080d1a79 in PyCFunction_Call () #19 0x0807b1c0 in PyEval_EvalCode () #20 0x0807bd7e in PyEval_EvalCodeEx () > > Why not > dead = vw.filter(lambda row: row.f1==pickon and row.state == 'F') > vw.remove(dead) See above. > > > subsubvw = subvw.remapwith(rs) > > metakit.dump(subsubvw) > > # > > ## Now I want remove the filter rows, how can I back track? > > # > > ## If I use > > ##vw.remove(rs) > > ## It only works if order is perfect > > # > > ## The delete and remove methods are not available on the derived views > > ##subvw.remove(rs) > > ##subsubvw.remove(rs) > > # > > vw.remove(vw.indices(subvw)) > > # Deletes all 'pickon' records, not just the ones in state 'F' From bkelley@wi.mit.edu Thu May 6 19:26:20 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id E86E1B3AE5 for ; Thu, 6 May 2004 19:26:19 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id BF042B3AE4 for ; Thu, 6 May 2004 19:26:16 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004050613255021508 for ; Thu, 06 May 2004 13:25:50 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HXAZ5900.CTN; Thu, 6 May 2004 13:27:09 -0400 Message-ID: <409A7507.4070406@wi.mit.edu> Date: Thu, 06 May 2004 13:25:27 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "William K. Volkman" , Metakit mailing list Subject: Re: [Metakit] revisit 'Right' way to delete rows from a view in python? References: <1083699532.30580.42.camel@chrysalis.netshark.com> <200405050828.19620.gmcm@hypernet.com> <1083804735.11072.86.camel@chrysalis.netshark.com> In-Reply-To: <1083804735.11072.86.camel@chrysalis.netshark.com> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2004 17:26:20 -0000 I've taken a look at your data and was surprised to come to the same conclusion as yourself that for you dataset, a select followed by a filter is the fastest solution. You might not know that you can select on two attribute, so you could have written: subvw = vw.select(f1=pickon, state=state) but the following is faster: subvw = vw.select(f1=pickon) rs = subvw.filter(lambda row: row.state == state) Now you mention that it takes 1.7 seconds to examine one set of data. This seems slow to me which indicates that A) the problem is different from the one you describe since on records of 100,000 generated through the mechanism you gave take only .2 seconds to search using the select followed with the filter. Now, I think a better solution to your problem lies in how you store the data. You are in desperate need of subviews. db = metakit.storage(DB,1) vw = db.getas('test[f1:I,f2:s,data[seq:I,state:s]]').ordered(1) I am setting up a view ordered on f1:I here for quick lookups. Each row has a subtable data which holds your state information. This subtable is key now, it dramatically reduces the size of the outer view. Now the lookups on my 100000 row table are now take 8.79299998283e-005 seconds which is several orders of magnitude better than what we were finding before. Additionally, the filtering time for the data subviews is almost inconsequential. If the number of keys (f1 in this case) is small, say around 100000 rows or so, you can get much better performance out of using the hash tables. And if it is much larger, say 200000+, you can use blocked and ordered views with a minor speed hit. Of course, you will take longer to load the data but this might be overcome by the new found quickness of your searches. One caveat is that when you are adding rows to a hashed or ordered view, the index returned is not the actual index in the ordered table so you can't just blindly use the index that append returns. I'm hoping to fix this bug in metakit at some point, until that day if you need the index you will have to search for it using view.find. Here is some test code import metakit, random, os, time DB = "foo.mk" if os.path.exists(DB): os.remove(DB) db = metakit.storage(DB,1) vw = db.getas('test[f1:I,f2:s,data[seq:I,state:s]]').ordered(1) #, [(s,'%d'%s)]) num = 10000 for i in xrange(num): if i % 1000 == 0: if i % 10000 == 0: print i db.commit() f1 = random.randint(0, num) index = vw.find(f1=f1) if index == -1: badindex = vw.append((f1,'%d'%f1)) # we need to search for the correct index # since append returns the index from the # original view, not the mapping view index = vw.find(f1=f1) data = vw[index].data seq = random.randint(0,4) if seq > 0: for s in range(seq): data.append((s,'%d'%s)) t1 = time.time() for i in range(10000): s = random.randint(0, num) index = vw.find(f1=s) t2 = time.time() print (t2-t1)/num, "seconds per search" From cybersamurai@terra.com.br Wed May 12 01:25:21 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id CECD1B3AE6 for ; Wed, 12 May 2004 01:25:21 +0200 (CEST) Received: from leticia.terra.com.br (leticia.terra.com.br [200.154.55.226]) by dizzie.triqs.com (Postfix) with ESMTP id 87A28B3AE2 for ; Wed, 12 May 2004 01:25:19 +0200 (CEST) Received: from cuenca.terra.com.br (cuenca.terra.com.br [200.154.55.130]) by leticia.terra.com.br (Postfix) with ESMTP id C648D3C28E for ; Tue, 11 May 2004 20:25:17 -0300 (BRT) Received: from terra.com.br (unknown [200.97.48.121]) (authenticated user cybersamurai) by cuenca.terra.com.br (Postfix) with ESMTP id E453E3C036 for ; Tue, 11 May 2004 20:25:16 -0300 (BRT) Message-ID: <40A16286.9020706@terra.com.br> Date: Tue, 11 May 2004 20:32:22 -0300 From: Luiz Siqueira User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.5) Gecko/20031013 Thunderbird/0.3 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] remove a view completely X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 May 2004 23:25:25 -0000 How can I remove a view completely? From bkelley@wi.mit.edu Wed May 12 15:26:26 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 8B168B3AE6 for ; Wed, 12 May 2004 15:26:26 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 7EBF5B3AE2 for ; Wed, 12 May 2004 15:26:24 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004051209255602182 for ; Wed, 12 May 2004 09:25:56 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HXLS1E00.KDC; Wed, 12 May 2004 09:27:14 -0400 Message-ID: <40A225CA.2020409@wi.mit.edu> Date: Wed, 12 May 2004 09:25:30 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Luiz Siqueira , Metakit mailing list Subject: Re: [Metakit] remove a view completely References: <40A16286.9020706@terra.com.br> In-Reply-To: <40A16286.9020706@terra.com.br> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2004 13:26:26 -0000 Luiz Siqueira wrote: > How can I remove a view completely? > to remove a view named table storage.getas("table") storage.commit() translate to C++ as appropriate. > _____________________________________________ Metakit mailing list > - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From bkelley@wi.mit.edu Wed May 12 19:55:51 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0CD4EB3AE8 for ; Wed, 12 May 2004 19:55:51 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 08C6FB3AE6 for ; Wed, 12 May 2004 19:55:47 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004051213551906625 for ; Wed, 12 May 2004 13:55:19 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HXM4ID00.NCX for ; Wed, 12 May 2004 13:56:37 -0400 Message-ID: <40A264ED.7050305@wi.mit.edu> Date: Wed, 12 May 2004 13:54:53 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Metakit mailing list X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] First metakit failure, database grew to 2+ gigabytes X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2004 17:55:51 -0000 This is just a FYI: I had my first metakit failure today that corrupted the database. I have a process that deletes objects from tables by finding the object id, searching for it in the table and then deleting it wholesale. It wasn't intelligent enough to find table "spans" to remove blocks of objects at a time, so it was deleting a row at a time and commiting the data occasionally. The database started at 363MB and grew to 2,097,153KB before the views got corrupted. Before that, it was as happy as a clam and chugging away. So I guess that I now know the size limit of mk databases. The take home message is deleting rows can rapidly grow memory. Is there an alternate strategy that I can use in this case for deleting rows from a database? Anyway, I can keep better track of database size know that I know the limitations a little better. Brian From Murat_Berk@bmc.com Wed May 12 19:59:24 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A990CB3AE8 for ; Wed, 12 May 2004 19:59:24 +0200 (CEST) Received: from mangrove.bmc.com (fw-us-hou19.bmc.com [198.207.223.240]) by dizzie.triqs.com (Postfix) with ESMTP id 6E798B3AE6 for ; Wed, 12 May 2004 19:59:20 +0200 (CEST) Received: from mangrove.bmc.com (localhost [127.0.0.1]) by localhost.bmc.com (Postfix) with ESMTP id 21F298ACF; Wed, 12 May 2004 13:01:23 -0500 (CDT) Received: from EC02-HOU.BMC.COM (ec02-hou.bmc.com [172.17.0.151]) by mangrove.bmc.com (Postfix) with ESMTP id D46E08AA8; Wed, 12 May 2004 13:01:22 -0500 (CDT) Received: by ec02-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Wed, 12 May 2004 12:59:18 -0500 Message-ID: <3DAC9CD31458D411BCE700D0B75D0A130CAA2184@ES09-HOU.bmc.com> From: "Berk, Murat" To: 'Brian Kelley' , Metakit mailing list Subject: RE: [Metakit] First metakit failure, database grew to 2+ gigabyte s Date: Wed, 12 May 2004 12:57:06 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2004 17:59:25 -0000 We use 'spans' and remove them in one operation and also do not commmit anything until we finish a pass over all rows. But main trick is blocked views, which uses smaller footprint on commits. Murat -----Original Message----- From: Brian Kelley [mailto:bkelley@wi.mit.edu] Sent: Wednesday, May 12, 2004 1:55 PM To: Metakit mailing list Subject: [Metakit] First metakit failure, database grew to 2+ gigabytes This is just a FYI: I had my first metakit failure today that corrupted the database. I have a process that deletes objects from tables by finding the object id, searching for it in the table and then deleting it wholesale. It wasn't intelligent enough to find table "spans" to remove blocks of objects at a time, so it was deleting a row at a time and commiting the data occasionally. The database started at 363MB and grew to 2,097,153KB before the views got corrupted. Before that, it was as happy as a clam and chugging away. So I guess that I now know the size limit of mk databases. The take home message is deleting rows can rapidly grow memory. Is there an alternate strategy that I can use in this case for deleting rows from a database? Anyway, I can keep better track of database size know that I know the limitations a little better. Brian _____________________________________________ Metakit mailing list - Metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From bkelley@wi.mit.edu Wed May 12 20:29:39 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 856C6B3AE8 for ; Wed, 12 May 2004 20:29:39 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 81C3EB3AE6 for ; Wed, 12 May 2004 20:29:34 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004051214290607276 for ; Wed, 12 May 2004 14:29:06 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HXM62O00.LGY; Wed, 12 May 2004 14:30:24 -0400 Message-ID: <40A26CD8.6080800@wi.mit.edu> Date: Wed, 12 May 2004 14:28:40 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Berk Murat" Subject: Re: [Metakit] First metakit failure, database grew to 2+ gigabyte s References: <3DAC9CD31458D411BCE700D0B75D0A130CAA2184@ES09-HOU.bmc.com> In-Reply-To: <3DAC9CD31458D411BCE700D0B75D0A130CAA2184@ES09-HOU.bmc.com> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit mailing list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2004 18:29:39 -0000 Berk, Murat wrote: > We use 'spans' and remove them in one operation and also do not > commmit anything until we finish a pass over all rows. > > But main trick is blocked views, which uses smaller footprint on > commits. Murat Yeah, I am using blocked views as well, but after checking the code, I was commiting after every delete! Ouch! I'm switching over to deleting spans so it should work a lot better. Brian From jcw@equi4.com Wed May 12 22:11:42 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id ACA38B3AEA for ; Wed, 12 May 2004 22:11:40 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 207ECB3AE8 for ; Wed, 12 May 2004 22:11:38 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 8A1A983C26; Wed, 12 May 2004 22:11:35 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 956238393A for ; Wed, 12 May 2004 22:11:29 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: <40A26CD8.6080800@wi.mit.edu> References: <3DAC9CD31458D411BCE700D0B75D0A130CAA2184@ES09-HOU.bmc.com> <40A26CD8.6080800@wi.mit.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <8E5FDD76-A450-11D8-9A35-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] First metakit failure, database grew to 2+ gigabyte s Date: Wed, 12 May 2004 22:11:30 +0200 To: Metakit list X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2004 20:11:42 -0000 Brian Kelley wrote: > Berk, Murat wrote: > > > We use 'spans' and remove them in one operation and also do not > > commmit anything until we finish a pass over all rows. > > > > But main trick is blocked views, which uses smaller footprint on > > commits. Murat > > Yeah, I am using blocked views as well, but after checking the code, I > was commiting after every delete! Ouch! I'm switching over to > deleting spans so it should work a lot better. The memory usage of individual deletes, especially across blocked views, is most probably due to MK allocating 4 Kb buffer chunks in every column a change is made (and sometimes much more to hold modified copies of ranges of data). With blocked views, I suspect that memory usage could indeed rise to a multiple of the dataset. A blocked view with say 5 columns and 40000 rows, could have 5 x 40 = 200 blocks, i.e. 800 Kb of sparsely filled buffers pending until flushed by a commit or rollback. The fix for this would be to track the total set of buffers, and start coalescing some in-memory data buffers to free some of that up (and to do so well before actual commits). I'm surprised that memory usage stays high across commits though, and even more by what looks like a 32-bit sign overflow in file positions getting through undetected and messing up a datafile. The 2 Gb limit should lead to commit failures, not file damage! -jcw From cybersamurai@terra.com.br Thu May 13 03:43:58 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3B2C6B3AE8 for ; Thu, 13 May 2004 03:43:58 +0200 (CEST) Received: from chiapa.terra.com.br (chiapa.terra.com.br [200.154.55.224]) by dizzie.triqs.com (Postfix) with ESMTP id 44B4EB3AE6 for ; Thu, 13 May 2004 03:43:55 +0200 (CEST) Received: from talara.terra.com.br (talara.terra.com.br [200.154.55.136]) by chiapa.terra.com.br (Postfix) with ESMTP id 7D602EC326 for ; Wed, 12 May 2004 22:43:53 -0300 (BRT) Received: from terra.com.br (unknown [200.97.49.41]) (authenticated user cybersamurai) by talara.terra.com.br (Postfix) with ESMTP id 409A03C06F for ; Wed, 12 May 2004 22:43:52 -0300 (BRT) Message-ID: <40A2D483.3000900@terra.com.br> Date: Wed, 12 May 2004 22:50:59 -0300 From: Luiz Siqueira User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.5) Gecko/20031013 Thunderbird/0.3 X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] I need get a view list from db. X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 May 2004 01:43:58 -0000 Some one know a easy way to make a introspection in a metakit database. I need get a view list from db. From erikh2000@yahoo.com Thu May 13 04:41:18 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 41600B3AE8 for ; Thu, 13 May 2004 04:41:18 +0200 (CEST) Received: from smtp104.mail.sc5.yahoo.com (smtp104.mail.sc5.yahoo.com [66.163.169.223]) by dizzie.triqs.com (Postfix) with SMTP id 0ACD4B3AE6 for ; Thu, 13 May 2004 04:41:17 +0200 (CEST) Received: from unknown (HELO HALPH.yahoo.com) (erikh2000@24.18.222.0 with login) by smtp104.mail.sc5.yahoo.com with SMTP; 13 May 2004 02:00:36 -0000 Message-Id: <5.2.0.9.0.20040512190743.080a23b0@pop.mail.yahoo.com> X-Sender: erikh2000@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 12 May 2004 19:08:51 -0700 To: Luiz Siqueira , metakit@equi4.com From: Erik Hermansen Subject: Re: [Metakit] I need get a view list from db. In-Reply-To: <40A2D483.3000900@terra.com.br> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 May 2004 02:41:18 -0000 At 10:50 PM 5/12/2004 -0300, Luiz Siqueira wrote: >Some one know a easy way to make a introspection in a metakit database. > >I need get a view list from db. You could call c4_Structure::Description() and parse the returned string. -Erik From bkelley@wi.mit.edu Thu May 13 17:22:06 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 594ACB3AE6 for ; Thu, 13 May 2004 17:22:06 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 4C344B3AE3 for ; Thu, 13 May 2004 17:22:04 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004051311213419644 for ; Thu, 13 May 2004 11:21:34 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HXNS2400.6IZ; Thu, 13 May 2004 11:22:52 -0400 Message-ID: <40A39265.3000408@wi.mit.edu> Date: Thu, 13 May 2004 11:21:09 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: cybersamurai , Metakit mailing list Subject: Re: [Metakit] remove a view completely References: In-Reply-To: X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 May 2004 15:22:06 -0000 cybersamurai wrote: > ---------- Cabeçalho inicial ----------- > > But why I can see in database file the table name after execute > this steps? > I don't know what you mean by see in the database file, but the table will not appear in the storage.contents() view. Metakit will reclaim the space used by the table over time, or you can save the storage to a new file and reopen. Brian From bkelley@wi.mit.edu Thu May 13 17:26:57 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 641AAB3AEA; Thu, 13 May 2004 17:26:57 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 52EFCB3AE6; Thu, 13 May 2004 17:26:54 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004051311262419740 ; Thu, 13 May 2004 11:26:24 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HXNSA700.JJK; Thu, 13 May 2004 11:27:43 -0400 Message-ID: <40A39386.7020705@wi.mit.edu> Date: Thu, 13 May 2004 11:25:58 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jean-Claude Wippler Subject: Re: [Metakit] First metakit failure, database grew to 2+ gigabyte s References: <3DAC9CD31458D411BCE700D0B75D0A130CAA2184@ES09-HOU.bmc.com> <40A26CD8.6080800@wi.mit.edu> <8E5FDD76-A450-11D8-9A35-000A9588127E@equi4.com> In-Reply-To: <8E5FDD76-A450-11D8-9A35-000A9588127E@equi4.com> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 May 2004 15:26:57 -0000 Jean-Claude Wippler wrote: > Brian Kelley wrote: > > The memory usage of individual deletes, especially across blocked > views, is most probably due to MK allocating 4 Kb buffer chunks in > every column a change is made (and sometimes much more to hold > modified copies of ranges of data). With blocked views, I suspect > that memory usage could indeed rise to a multiple of the dataset. > A blocked view with say 5 columns and 40000 rows, could have 5 x 40 > = 200 blocks, i.e. 800 Kb of sparsely filled buffers pending until > flushed by a commit or rollback. > > The fix for this would be to track the total set of buffers, and > start coalescing some in-memory data buffers to free some of that > up (and to do so well before actual commits). > > I'm surprised that memory usage stays high across commits though, > and even more by what looks like a 32-bit sign overflow in file > positions getting through undetected and messing up a datafile. > The 2 Gb limit should lead to commit failures, not file damage! I'm not sure that this was a memory issue, the metakit file itself was growing to 2+ gig, so it seems that in the way I was deleting from the blocked views rapidly grew the database. I'll spend some time today making an example database and doing the individual commits in case you want to test the issue. Brian From njriley@shrug.csl.uiuc.edu Fri May 14 10:28:33 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 25986B3AE8 for ; Fri, 14 May 2004 10:28:33 +0200 (CEST) Received: from shrug.csl.uiuc.edu (shrug.csl.uiuc.edu [130.126.137.135]) by dizzie.triqs.com (Postfix) with ESMTP id 073B6B3AE4 for ; Fri, 14 May 2004 10:28:28 +0200 (CEST) Received: from shrug.csl.uiuc.edu (njriley@localhost [127.0.0.1]) by shrug.csl.uiuc.edu (8.12.10/8.12.9) with ESMTP id i4E8SRJq043376; Fri, 14 May 2004 03:28:27 -0500 (CDT) (envelope-from njriley@shrug.csl.uiuc.edu) Received: (from njriley@localhost) by shrug.csl.uiuc.edu (8.12.10/8.12.9/Submit) id i4E8SR6E043375; Fri, 14 May 2004 03:28:27 -0500 (CDT) (envelope-from njriley) Date: Fri, 14 May 2004 03:28:27 -0500 From: Nicholas Riley To: PyDS List Message-ID: <20040514082827.GA43297@uiuc.edu> Mail-Followup-To: PyDS List , Metakit mailing list References: <20040514063202.GE38318@uiuc.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040514063202.GE38318@uiuc.edu> User-Agent: Mutt/1.4.1i X-UID: 694 X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit mailing list Subject: [Metakit] PyDS 0.7.2 database corruption X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 May 2004 08:28:33 -0000 After a few bounces (why doesn't PyDS respond to ^c and stop itself?) trying to fix the calendar layout, I found that the upstreaming and events databases were corrupted. Trying to access the upstreaming tool caused 100% CPU and then a crash; trying to access events gave me gibberish for the page; you could see portions of the view specifications instead of the expected data. Removing the databases fixed the problem. I am not sure whether this is PyDS threading issues or Metakit bugs. In any case, Metakit should not crash while attempting to read data from a database! I tried updating to core Metakit CVS (+ my Python bindings) from 2.4.9.2, but that did not have any effect. The corruption occured under stock 2.4.9.2. If anyone (Jean-Claude?) wants to see one/more of the databases, I can send them. -- =Nicholas Riley | From jcw@equi4.com Fri May 14 10:41:45 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 78176B3AE8 for ; Fri, 14 May 2004 10:41:45 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id D43D3B3AE4 for ; Fri, 14 May 2004 10:41:43 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id B75C083C26; Fri, 14 May 2004 10:41:41 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id E2B688393A; Fri, 14 May 2004 10:41:33 +0200 (CEST) In-Reply-To: <20040514082827.GA43297@uiuc.edu> References: <20040514063202.GE38318@uiuc.edu> <20040514082827.GA43297@uiuc.edu> Mime-Version: 1.0 (Apple Message framework v613) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <80754F56-A582-11D8-9A35-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] PyDS 0.7.2 database corruption Date: Fri, 14 May 2004 10:41:32 +0200 To: Nicholas Riley X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 May 2004 08:41:45 -0000 Nicholas Riley wrote: > I am not sure whether this is PyDS threading issues or Metakit bugs. > In any case, Metakit should not crash while attempting to read data > from a database! Agree. But stray pointer writes can damage things. I'm not saying this is the case here, just pointing out that software bugs in the same address space can damage a MK datafile despite its failsafe logic. > If anyone (Jean-Claude?) wants to see one/more of the databases, I can > send them. Please do. -jcw From jcw@equi4.com Fri May 14 21:26:49 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C6163B3AE8 for ; Fri, 14 May 2004 21:26:48 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id E99BCB3AE3 for ; Fri, 14 May 2004 21:26:46 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id 434FA83C26; Fri, 14 May 2004 21:26:44 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 7B4468393A for ; Fri, 14 May 2004 21:26:39 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v613) In-Reply-To: <20040514082827.GA43297@uiuc.edu> References: <20040514063202.GE38318@uiuc.edu> <20040514082827.GA43297@uiuc.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] PyDS 0.7.2 database corruption Date: Fri, 14 May 2004 21:26:40 +0200 To: Metakit list X-Mailer: Apple Mail (2.613) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 May 2004 19:26:49 -0000 Nicholas Riley wrote: > After a few bounces (why doesn't PyDS respond to ^c and stop itself?) > trying to fix the calendar layout, I found that the upstreaming and > events databases were corrupted. Trying to access the upstreaming > tool caused 100% CPU and then a crash; trying to access events gave me > gibberish for the page; you could see portions of the view > specifications instead of the expected data. Removing the databases > fixed the problem. I suspect that space characters in the description string could be the problem. They will restructure the datafile, causing data loss (but not really damage - the loss is because properties will be dropped). As for the 100% CPU - the only case where I have seen this before is with (sub)views containing no properties. The example on this PyDS page worked here, once I removed the spaces: http://pyds.muensterland.org/weblog/2004/04/20.html#P227 > I am not sure whether this is PyDS threading issues or Metakit bugs. > In any case, Metakit should not crash while attempting to read data > from a database! I agree. And I'm not sure how to reproduce that - the files you sent appear to open properly, and the mkstats utility reports no file damage problems. -jcw From bauer@gws.ms Fri May 14 10:58:55 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D19C9B3AE8 for ; Fri, 14 May 2004 10:58:54 +0200 (CEST) Received: from kenny.gws-muenster.de (kenny.gws-online.de [195.145.186.100]) by dizzie.triqs.com (Postfix) with ESMTP id 0E06FB3AE4 for ; Fri, 14 May 2004 10:58:53 +0200 (CEST) Received: from kyle.gws-muenster.de ([217.6.136.6] helo=jack.gws-muenster.de ident=balance) by kenny.gws-muenster.de with esmtp (Exim 3.35 #1 (Debian)) id 1BOYWs-0005vd-00 for ; Fri, 14 May 2004 10:58:54 +0200 Received: from amavis by jack.gws-muenster.de with scanned-ok (Exim 3.12 #1 (Debian)) id 1BOYWq-0007RT-00 for ; Fri, 14 May 2004 10:58:52 +0200 Received: from inside.gws-muenster.de ([195.145.186.81] helo=pelota.gws-muenster.de ident=balance) by jack.gws-muenster.de with esmtp (Exim 3.12 #1 (Debian)) id 1BOYWU-0007Qo-00; Fri, 14 May 2004 10:58:30 +0200 Received: from nt-gws12.gws-muenster.de (nt_gws12.gws-muenster.de) [10.64.65.12] by pelota.gws-muenster.de with esmtp (Exim 3.35 #1) id 1BOYWU-0005wK-00 (Debian); Fri, 14 May 2004 10:58:30 +0200 Received: by nt-gws12.gws-muenster.de with Internet Mail Service (5.5.2653.19) id ; Fri, 14 May 2004 11:14:25 +0200 Message-ID: From: "Bauer, Georg" To: 'Nicholas Riley' , PyDS List Date: Fri, 14 May 2004 10:55:38 +0200 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" X-Virus-Scanned: virengescanned bei Maildurchlauf GWS (jack) X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Sat, 15 May 2004 11:07:02 +0200 cc: Metakit mailing list Subject: [Metakit] RE: [Pyds-dev] PyDS 0.7.2 database corruption X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 May 2004 08:58:55 -0000 Hi! > After a few bounces (why doesn't PyDS respond to ^c and stop itself?) The 0.7.2 still doesn't hook up signal handlers when running in the foreground. So you should stop it with pyds-stop in another terminal window. > trying to fix the calendar layout, I found that the upstreaming and > events databases were corrupted. Trying to access the upstreaming The upstreaming database can be deleted, but you will need to reregister with the Python community server, as that data is stored in the upstream database. Use the same email address and password you used with your current installation and you just get your old usernumber back on registration. The events database can just be deleted. > I am not sure whether this is PyDS threading issues or Metakit bugs. > In any case, Metakit should not crash while attempting to read data > from a database! PyDS threading shouldn't have any influence on Metakit, as it only uses the Python metakit interface that itself says it's python-threading-aware. And all write access to Metakit files is secured by locks in PyDS anyway, so only one thread at a time accesses the metakit database. > I tried updating to core Metakit CVS (+ my Python bindings) from > 2.4.9.2, but that did not have any effect. The corruption occured > under stock 2.4.9.2. Weird. There _are_ some situations where people reported corrupted metakit databases, but I was never able to reproduce them, except one situation where my PyCS settings.dat file was broken. It showed the same symptoms like your database corruption showed, so it might be the same problem. In my case a dying hard disc was the reason for the corruption, though. I never had corruptions with Metakit 2.4.9.2 during normal use, even though I quite often need to kill my PyDS installations with kill -9 due to my development versions I am running. So I think it must be rather weird and special situations that lead to this corruption issue. > If anyone (Jean-Claude?) wants to see one/more of the databases, I can > send them. Yep, that would be good, as the database file I have for PyCS is very big - over 25 MB - and so not really usefull for debugging. bye, Georg From gb@murphy.bofh.ms Sat May 15 14:59:18 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id A8CF1B3AE6; Sat, 15 May 2004 14:59:18 +0200 (CEST) Received: from timmy.gws-muenster.de (timmy.gws-online.de [80.146.170.26]) by dizzie.triqs.com (Postfix) with ESMTP id C8374B3AE3; Sat, 15 May 2004 14:59:17 +0200 (CEST) Received: from p3ee24740.dip.t-dialin.net ([62.226.71.64] helo=firewall.bofh.ms ident=Debian-exim) by timmy.gws-muenster.de with asmtp (Exim 3.36 #1 (Debian)) id 1BOyl3-0000ld-00; Sat, 15 May 2004 14:59:17 +0200 Received: from tex.bofh.ms ([10.0.0.145] helo=murphy.bofh.ms) by firewall.bofh.ms with esmtp (Exim 4.31) id 1BOyl2-0004GH-TR; Sat, 15 May 2004 14:59:18 +0200 Date: Sat, 15 May 2004 14:59:14 +0200 Subject: Re: [Metakit] PyDS 0.7.2 database corruption Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v553) To: Jean-Claude Wippler From: Georg Bauer In-Reply-To: Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.553) X-Virus-Scanned: by amavisd-new at triqs.com cc: Metakit list X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 May 2004 12:59:19 -0000 Hi! Am Freitag, 14.05.04 um 21:26 Uhr schrieb Jean-Claude Wippler: > I suspect that space characters in the description string could be the > problem. They will restructure the datafile, causing data loss (but > not really damage - the loss is because properties will be dropped). > As for the 100% CPU - the only case where I have seen this before is > with (sub)views containing no properties. PyDS doesn't have any spaces in getas description strings. The sample was from a different context and a different project (PyCS - and even that has no spaces, the spaces where introduced by Yasushi Iwata when trying stuff). So Nicholas problems are not due to spaces. And my broken PyCS database isn't broken due to spaces, too. The problem with the PyCS database is that it's too large and to much private content as that I could upload it somewhere - the file includes all users settings, passwords, private data etc. The symptoms are allways the same: the program grabs all CPU resources and the process itself hangs on opening a view. Opening the database file itself works fine. Just the view opening hangs. Both projects (PyDS and PyCS) are written in Python. They still usually use Metakit 2.4.9.2, as Yasushi had bigger problems with crashes when trying to use Metakit 2.4.9.3 (and I myself use Debian packages that still are at 2.4.9.2). The mkstats didn't work for me as reported in one of my earlier messages, there were some problems with missing symbols in glibc - don't actually know why, as mkstats is a statically linked executable and shouldn't have dependencies on shared libraries on my system ... bye, Georg From catcher@linuxmail.org Thu May 27 18:43:04 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 85014B3AEA for ; Thu, 27 May 2004 18:43:04 +0200 (CEST) Received: from webmail-outgoing.us4.outblaze.com (webmail-outgoing.us4.outblaze.com [205.158.62.67]) by dizzie.triqs.com (Postfix) with ESMTP id DF338B3AE8 for ; Thu, 27 May 2004 18:43:00 +0200 (CEST) Received: from wfilter.us4.outblaze.com (wfilter.us4.outblaze.com [205.158.62.180])366C218008D2 for ; Thu, 27 May 2004 16:43:00 +0000 (GMT) X-OB-Received: from unknown (205.158.62.148) by wfilter.us4.outblaze.com; 27 May 2004 16:42:30 -0000 Received: by ws5-6.us4.outblaze.com (Postfix, from userid 1001) id 0919421AFF9; Thu, 27 May 2004 16:43:00 +0000 (GMT) Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Mailer: MIME-tools 5.41 (Entity 5.404) Received: from [152.121.100.100] by ws5-6.us4.outblaze.com with http for catcher@linuxmail.org; Fri, 28 May 2004 00:42:58 +0800 From: "Bob X" To: metakit@equi4.com Date: Fri, 28 May 2004 00:42:58 +0800 X-Originating-Ip: 152.121.100.100 X-Originating-Server: ws5-6.us4.outblaze.com Message-Id: <20040527164300.0919421AFF9@ws5-6.us4.outblaze.com> X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Metakit and Tcl (and maybe others) X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2004 16:43:05 -0000 I am using 2.4.9.3 on Windows XP with ActiveTcl. I am creating a simple ticket tracker and I defined my view: set view [mk::view layout db.tracker "username:S ticket:S recieved:I closed:I problem:S notes:S status:I"] I then append into the view: mk::row append $view username "Jeff Walsh" ticket "000001081" recieved "20040419" closed "20040419" problem "Password locked" notes "Reset password to Ellipse" status "0" I then get errors: expected integer but got "000001058" (looks like invalid octal number) while executing "mk::row append $view username "Don Lang" ticket "000001058" recieved "20040409" closed "20040409" problem "Application is hanging" notes "Network prob..." (file "initial_loader.tcl" line 18) I could change it to a String (works that way) but I would like to leave it an Integer. Are the leading zero's causing the problem? I have to have those as the program spitting the data out uses those. Robert -- ______________________________________________ Check out the latest SMS services @ http://www.linuxmail.org This allows you to send and receive SMS through your mailbox. Powered by Outblaze From jcw@equi4.com Thu May 27 19:06:46 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0C01DB3AEA for ; Thu, 27 May 2004 19:06:46 +0200 (CEST) Received: from madder.xs4all.nl (madder.xs4all.nl [80.126.24.9]) by dizzie.triqs.com (Postfix) with ESMTP id 12A95B3AE8 for ; Thu, 27 May 2004 19:06:44 +0200 (CEST) Received: by madder.xs4all.nl (Postfix, from userid 101) id F141083C26; Thu, 27 May 2004 19:06:41 +0200 (CEST) Received: from [10.0.1.4] (biggie.equi4.local [10.0.1.4]) by madder.xs4all.nl (Postfix) with ESMTP id 116ED8393A for ; Thu, 27 May 2004 19:06:39 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v618) In-Reply-To: <20040527164300.0919421AFF9@ws5-6.us4.outblaze.com> References: <20040527164300.0919421AFF9@ws5-6.us4.outblaze.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <3780FA6B-B000-11D8-8255-000A9588127E@equi4.com> Content-Transfer-Encoding: 7bit From: Jean-Claude Wippler Subject: Re: [Metakit] Metakit and Tcl (and maybe others) Date: Thu, 27 May 2004 19:06:38 +0200 To: Metakit list X-Mailer: Apple Mail (2.618) X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2004 17:06:46 -0000 Bob X wrote: > I am using 2.4.9.3 on Windows XP with ActiveTcl. > > I am creating a simple ticket tracker and I defined my view: > > set view [mk::view layout db.tracker "username:S ticket:S recieved:I > closed:I problem:S notes:S status:I"] > > I then append into the view: > > mk::row append $view username "Jeff Walsh" ticket "000001081" recieved > "20040419" closed "20040419" problem "Password locked" notes "Reset > password to Ellipse" status "0" > > I then get errors: > > > expected integer but got "000001058" (looks like invalid octal number) > while executing > "mk::row append $view username "Don Lang" ticket "000001058" recieved > "20040409" closed "20040409" problem "Application is hanging" notes > "Network prob..." > (file "initial_loader.tcl" line 18) > Yes, leading zero's bite when treating a Tcl string as an integer. I'm assuming the "ticket:S" is actually a "ticket:I" in the example you gave - then it would fail. The leading zero defaulting to octal mode is a painful idiosyncrasy of Tcl, see http://mini.net/tcl/498 http://www.tcl.tk/cgi-bin/tct/tip/114.html > I could change it to a String (works that way) but I would like to > leave it an Integer. Are the leading zero's causing the problem? I > have to have those as the program spitting the data out uses those. You can't have your cake and eat it in cases like these :) - either you treat values as integers (which have no knowledge of representation, such as leading zero's) or you stick to a string, which is slower and takes up more space. I tend to use either of two tricks for this: - add 1000000000 to the value and store that (then strip 1st char again on extract to make sure the 0's stay) - convert to int via ... ticket [scan %d "000001058"] ... (and convert back as needed with: "puts [format %9d $value]") -jcw From root@spammotel.com Fri May 28 12:07:18 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id C651BB3AED for ; Fri, 28 May 2004 12:07:17 +0200 (CEST) Received: from spammotel.spammotel.com (unknown [206.222.31.7]) by dizzie.triqs.com (Postfix) with ESMTP id A02E2B3AE8 for ; Fri, 28 May 2004 12:07:16 +0200 (CEST) Received: from spammotel.spammotel.com (localhost [127.0.0.1]) i4SA5mva026691 for ; Fri, 28 May 2004 06:05:48 -0400 Received: (from root@localhost) by spammotel.spammotel.com (8.12.11/8.12.11/Submit) id i4SA5ltk026677 for metakit@equi4.com; Fri, 28 May 2004 06:05:47 -0400 Date: Fri, 28 May 2004 06:04:00 -0400 MIME-Version: 1.0 Content-Type: multipart/report; report-type=delivery-status; boundary="i4SA40R8014976.1085738640/spammotel.spammotel.com" Auto-Submitted: auto-generated (failure) X-Sender: peodfyehrsbg@spammotel.com Received: from spammotel.com [206.222.31.7] From: peodfyehrsbg@spammotel.com Message-ID: <1085738747.peodfyehrsbg@spammotel.com> Sender: peodfyehrsbg@spammotel.com To: metakit@equi4.com X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Returned mail: see transcript for details X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: peodfyehrsbg@spammotel.com List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2004 10:07:18 -0000 This is a MIME-encapsulated message --i4SA40R8014976.1085738640/spammotel.spammotel.com The original message was received at Fri, 28 May 2004 06:03:57 -0400 from localhost [127.0.0.1] ----- The following addresses had permanent fatal errors ----- (reason: 550 : User unknown) ----- Transcript of session follows ----- ... while talking to joinme-com.mr.outblaze.com.: >>> DATA <<< 550 : User unknown 550 5.1.1 ... User unknown <<< 554 Error: no valid recipients 451 4.0.0 Cannot open hash database /etc/mail/aliases.db: Invalid argument --i4SA40R8014976.1085738640/spammotel.spammotel.com Content-Type: message/delivery-status Reporting-MTA: dns; spammotel.spammotel.com Received-From-MTA: DNS; localhost Arrival-Date: Fri, 28 May 2004 06:03:57 -0400 Final-Recipient: RFC822; peodfyehrsbg@spammotel.com Action: failed Status: 5.1.1 Remote-MTA: DNS; joinme-com.mr.outblaze.com Diagnostic-Code: SMTP; 550 : User unknown Last-Attempt-Date: Fri, 28 May 2004 06:03:59 -0400 --i4SA40R8014976.1085738640/spammotel.spammotel.com Content-Type: text/rfc822-headers Return-Path: Received: from spammotel.spammotel.com (localhost [127.0.0.1]) by spammotel.spammotel.com (8.12.11/8.12.11) with ESMTP id i4SA3vR8014967 for ; Fri, 28 May 2004 06:03:57 -0400 Received: (from root@localhost) by spammotel.spammotel.com (8.12.11/8.12.11/Submit) id i4SA3vQU014959 for peodfyehrsbg@spammotel.com; Fri, 28 May 2004 06:03:57 -0400 Old-Return-Path: Received: from dizzie.triqs.com (ipx20065.ipxserver.de [80.190.241.133]) by spammotel.spammotel.com (8.12.11/8.12.11) with ESMTP id i4S9wshL029670 for ; Fri, 28 May 2004 05:58:55 -0400 Received: from dizzie.triqs.com (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 54158B3B10; Fri, 28 May 2004 12:00:05 +0200 (CEST) From: metakit-request@equi4.com Old-Subject: Metakit Digest, Vol 7, Issue 9 To: metakit@equi4.com Old-Reply-To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: metakit-bounces@equi4.com Errors-To: metakit-bounces@equi4.com Message-Id: <20040528100005.54158B3B10@dizzie.triqs.com> Date: Fri, 28 May 2004 12:00:05 +0200 (CEST) Subject: **metakit: Metakit Digest, Vol 7, Issue 9 Reply-To: --i4SA40R8014976.1085738640/spammotel.spammotel.com-- From catcher@linuxmail.org Fri May 28 15:49:24 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id D03A1B3AED for ; Fri, 28 May 2004 15:49:24 +0200 (CEST) Received: from webmail-outgoing.us4.outblaze.com (webmail-outgoing.us4.outblaze.com [205.158.62.67]) by dizzie.triqs.com (Postfix) with ESMTP id 3E51DB3AEA for ; Fri, 28 May 2004 15:49:23 +0200 (CEST) Received: from wfilter.us4.outblaze.com (wfilter.us4.outblaze.com [205.158.62.180])B8F0F1801477 for ; Fri, 28 May 2004 13:49:20 +0000 (GMT) X-OB-Received: from unknown (205.158.62.133) by wfilter.us4.outblaze.com; 28 May 2004 13:48:48 -0000 Received: by ws5-3.us4.outblaze.com (Postfix, from userid 1001) id 4FC5E23BFD; Fri, 28 May 2004 13:49:20 +0000 (GMT) Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Mailer: MIME-tools 5.41 (Entity 5.404) Received: from [152.121.100.100] by ws5-3.us4.outblaze.com with http for catcher@linuxmail.org; Fri, 28 May 2004 21:49:20 +0800 From: "Robert X" To: metakit@equi4.com Date: Fri, 28 May 2004 21:49:20 +0800 X-Originating-Ip: 152.121.100.100 X-Originating-Server: ws5-3.us4.outblaze.com Message-Id: <20040528134920.4FC5E23BFD@ws5-3.us4.outblaze.com> X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Does MK have a server mode? X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2004 13:49:25 -0000 Can Metakit be used in "server" mode or just a local embedded one? If it can, can you point me to where I can bone up on it? Robert -- ______________________________________________ Check out the latest SMS services @ http://www.linuxmail.org This allows you to send and receive SMS through your mailbox. Powered by Outblaze From bkelley@wi.mit.edu Fri May 28 16:08:50 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 01185B3AEE for ; Fri, 28 May 2004 16:08:50 +0200 (CEST) Received: from parramatta.wi.mit.edu (parramatta.wi.mit.edu [18.157.0.216]) by dizzie.triqs.com (Postfix) with SMTP id 14F35B3AEA for ; Fri, 28 May 2004 16:08:47 +0200 (CEST) Received: from sydney.wi.mit.edu ([18.157.0.58]) by parramatta.wi.mit.edu (SAVSMTP 3.1.0.29) with SMTP id M2004052810081129605 for ; Fri, 28 May 2004 10:08:11 -0400 Received: from wi.mit.edu (stockwell16.wi.mit.edu [18.157.5.236]) by sydney.wi.mit.edu (Netscape Messaging Server 4.15) with ESMTP id HYFGNQ00.E3J; Fri, 28 May 2004 10:09:26 -0400 Message-ID: <40B747C4.6040000@wi.mit.edu> Date: Fri, 28 May 2004 10:08:04 -0400 From: "Brian Kelley" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert X , Metakit mailing list Subject: Re: [Metakit] Does MK have a server mode? References: <20040528134920.4FC5E23BFD@ws5-3.us4.outblaze.com> In-Reply-To: <20040528134920.4FC5E23BFD@ws5-3.us4.outblaze.com> X-Enigmail-Version: 0.83.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2004 14:08:50 -0000 Robert X wrote: >Can Metakit be used in "server" mode or just a local >embedded one? If it can, can you point me to where I can bone up on it? > >Robert > > > There are a couple of server modes available on the python side which are fairly python specific. I remember hearing about a TCL server mode as well. Jean-Claude and I have been working on a platform independent server mode for a little bit, the prototype is working fairly well on the python side and should be easily ported to C++ by an interested party. You can find the initial server mode for python here http://jura.wi.mit.edu/people/kelley/ look for Metaserve. If you are interested in helping to make a C++ client for the metakit server, we should probably talk about that. I will be moving in the near future to a new position, so if anyone wants to reach me, here is my temporary address which I will be able to check sporadically for the next few weeks. archaeopteryx@spymac.com Brian From root@spammotel.com Sat May 29 12:11:12 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 3C21BB3AEB for ; Sat, 29 May 2004 12:11:12 +0200 (CEST) Received: from spammotel.spammotel.com (unknown [206.222.31.7]) by dizzie.triqs.com (Postfix) with ESMTP id A3A79B3AE6 for ; Sat, 29 May 2004 12:11:10 +0200 (CEST) Received: from spammotel.spammotel.com (localhost [127.0.0.1]) i4TA9fHY024529 for ; Sat, 29 May 2004 06:09:41 -0400 Received: (from root@localhost) by spammotel.spammotel.com (8.12.11/8.12.11/Submit) id i4TA9fvB024471 for metakit@equi4.com; Sat, 29 May 2004 06:09:41 -0400 Date: Sat, 29 May 2004 06:05:04 -0400 MIME-Version: 1.0 Content-Type: multipart/report; report-type=delivery-status; boundary="i4TA54J5005466.1085825104/spammotel.spammotel.com" Auto-Submitted: auto-generated (failure) X-Sender: peodfyehrsbg@spammotel.com Received: from spammotel.com [206.222.31.7] From: peodfyehrsbg@spammotel.com Message-ID: <1085825381.peodfyehrsbg@spammotel.com> Sender: peodfyehrsbg@spammotel.com To: metakit@equi4.com X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Returned mail: see transcript for details X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list Reply-To: peodfyehrsbg@spammotel.com List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 May 2004 10:11:12 -0000 This is a MIME-encapsulated message --i4TA54J5005466.1085825104/spammotel.spammotel.com The original message was received at Sat, 29 May 2004 06:05:01 -0400 from localhost [127.0.0.1] ----- The following addresses had permanent fatal errors ----- (reason: 550 : User unknown) ----- Transcript of session follows ----- ... while talking to joinme-com.mr.outblaze.com.: >>> DATA <<< 550 : User unknown 550 5.1.1 ... User unknown <<< 554 Error: no valid recipients 451 4.0.0 Cannot open hash database /etc/mail/aliases.db: Invalid argument --i4TA54J5005466.1085825104/spammotel.spammotel.com Content-Type: message/delivery-status Reporting-MTA: dns; spammotel.spammotel.com Received-From-MTA: DNS; localhost Arrival-Date: Sat, 29 May 2004 06:05:01 -0400 Final-Recipient: RFC822; peodfyehrsbg@spammotel.com Action: failed Status: 5.1.1 Remote-MTA: DNS; joinme-com.mr.outblaze.com Diagnostic-Code: SMTP; 550 : User unknown Last-Attempt-Date: Sat, 29 May 2004 06:05:03 -0400 --i4TA54J5005466.1085825104/spammotel.spammotel.com Content-Type: text/rfc822-headers Return-Path: Received: from spammotel.spammotel.com (localhost [127.0.0.1]) by spammotel.spammotel.com (8.12.11/8.12.11) with ESMTP id i4TA51J5005463 for ; Sat, 29 May 2004 06:05:01 -0400 Received: (from root@localhost) by spammotel.spammotel.com (8.12.11/8.12.11/Submit) id i4TA51Ai005436 for peodfyehrsbg@spammotel.com; Sat, 29 May 2004 06:05:01 -0400 Old-Return-Path: Received: from dizzie.triqs.com (ipx20065.ipxserver.de [80.190.241.133]) by spammotel.spammotel.com (8.12.11/8.12.11) with ESMTP id i4T9wneN010859 for ; Sat, 29 May 2004 05:58:53 -0400 Received: from dizzie.triqs.com (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 569A3B3AF1; Sat, 29 May 2004 12:00:04 +0200 (CEST) From: metakit-request@equi4.com Old-Subject: Metakit Digest, Vol 7, Issue 10 To: metakit@equi4.com Old-Reply-To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: metakit-bounces@equi4.com Errors-To: metakit-bounces@equi4.com Message-Id: <20040529100004.569A3B3AF1@dizzie.triqs.com> Date: Sat, 29 May 2004 12:00:04 +0200 (CEST) Subject: **metakit: Metakit Digest, Vol 7, Issue 10 Reply-To: --i4TA54J5005466.1085825104/spammotel.spammotel.com-- From anotherguynamedalex@yahoo.com Wed Jun 9 23:30:47 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 9661EB3AED for ; Wed, 9 Jun 2004 23:30:47 +0200 (CEST) Received: from web53209.mail.yahoo.com (web53209.mail.yahoo.com [206.190.39.225]) by dizzie.triqs.com (Postfix) with SMTP id DD8F3B3AEA for ; Wed, 9 Jun 2004 23:30:44 +0200 (CEST) Message-ID: <20040609213044.8419.qmail@web53209.mail.yahoo.com> Received: from [12.104.244.6] by web53209.mail.yahoo.com via HTTP; Wed, 09 Jun 2004 14:30:44 PDT Date: Wed, 9 Jun 2004 14:30:44 -0700 (PDT) From: Alex Sauer-Budge To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com Subject: [Metakit] Of Default Values and Ordered Views X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2004 21:30:47 -0000 Metakit Illuminati, I would be pleased if you could share some of your hard earned knowledge with me by answering some of my questions about Metakit. But first, some context. I am working with a logically three-dimensional data set (observations,objects,measurements). At each of 400-1200 observations, there is a list of ~5000 objects each of which has 1-25 measurements recorded for that observation. The tricky part is that the list of objects varies from observation to observation and the measurements taken on each object varies from observation to observation. For the most part, I will be building the database once and then reading it many times. An example: observation1: object1: measurement1, measurement2 object2: measurement3 observation2: object1: measurement1, measurement2 object3: measurement1 object4: measurement2, measurement3 The typical access pattern is to move from observation to observation, visiting a subset of objects and measurements at each observation. One possible Metakit schema would be: View: observation1, subview1 . . observation1200, subview1200 SubView: object1, measurement1, measurement2, ..., measurement25 . . object5000, measurement1, measurement2, ..., measurement25 Where unavailable measurements are indicated by a special value or there is a boolean flag for each measurement. Question 1: Is there a better schema? It seems that using a special value would reduce the storage requirements and would be more computationally efficient as a result of fewer property look ups. Question 2: Is there a way to override the default value of a property? Can I safely replace c4_Handler::ClearBytes(c4_Bytes& buf_) with my own custom version? In order to quickly search a given observation for a specified object, I would like the subviews to be ordered, but it is not clear to me how to create and maintain order subviews. c4_View c4_View::Ordered(int) returns a derived view that provides a way of accessing the underlying view such that it appears to be ordered. I would rather that the underlying view itself be maintained (and stored) ordered so that I do not need to keep creating ordered derived views. Question 3: Is there a way to directly maintain a view in order? If not, how can I store the ordered view (so I don't need to recreate it every time) and will it be a resource hog? Much thanks for any thoughts you may have on these questions! Thanks, Alex __________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ From anotherguynamedalex@yahoo.com Thu Jun 10 00:54:57 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 0ED7BB3AED for ; Thu, 10 Jun 2004 00:54:57 +0200 (CEST) Received: from web53207.mail.yahoo.com (web53207.mail.yahoo.com [206.190.39.223]) by dizzie.triqs.com (Postfix) with SMTP id EE883B3AEA for ; Thu, 10 Jun 2004 00:54:52 +0200 (CEST) Message-ID: <20040609225451.27505.qmail@web53207.mail.yahoo.com> Received: from [12.104.244.6] by web53207.mail.yahoo.com via HTTP; Wed, 09 Jun 2004 15:54:51 PDT Date: Wed, 9 Jun 2004 15:54:51 -0700 (PDT) From: Alex Sauer-Budge Subject: Re: [Metakit] Of Default Values and Ordered Views To: metakit@equi4.com In-Reply-To: <20040609213044.8419.qmail@web53209.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at triqs.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the Metakit database library. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2004 22:54:58 -0000 --- Alex Sauer-Budge wrote: > Question 3: Is there a way to directly maintain a > view in order? If not, how can I store the ordered > view (so I don't need to recreate it every time) > and will it be a resource hog? It seems the answer to my question is that creating a new view using: c4_View = db.GetAs("data[obs:I,objs[id:I,val:D]]").Ordered(); and c4_ViewProp pSubView("val"); c4_View subView = pSubView(row).Ordered(); and adding rows as normal does indeed result in the underlying view being ordered in storage. Is this correct? When I later reopen my database for read-only access, do I need to again create the ordered identity view in order to benefit from the faster find, or is this stored in the file along with the view? I think the answer is no, it is not stored and I must recreate the ordered identity view. If this is so, then if I use Search instead of Find will it use the faster search method even if it isn't an Ordered view (that is, I didn't say .Ordered())? The reason I am concerned is that my subviews are randomly accessed and I don't want to repeatedly and unecessarily create ordered identity views whenever they are queried. Metakit is really quite beautiful, thanks for sharing it! Thanks! Alex __________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ From jlawhead@nvisionsolutions.com Fri Jun 18 21:39:30 2004 Return-Path: X-Original-To: metakit@equi4.com Delivered-To: metakit@equi4.com Received: from localhost (localhost [127.0.0.1]) by dizzie.triqs.com (Postfix) with ESMTP id 53BEEB3AF2 for ; Fri, 18 Jun 2004 21:39:29 +0200 (CEST) Received: from mail.nvisionsolutions.com (unknown [208.62.99.190]) by dizzie.triqs.com (Postfix) with ESMTP id D3814B3AE9 for ; Fri, 18 Jun 2004 21:39:22 +0200 (CEST) Received: from [208.62.99.195] by mail.nvisionsolutions.com (ArGoSoft Mail Server Pro for WinNT/2000/XP, Version 1.8 (1.8.3.2)); Fri, 18 Jun 2004 14:39:21 -0500 Message-ID: <40D344E5.4070904@nvisionsolutions.com> Date: Fri, 18 Jun 2004 14:39:17 -0500 From: Joel Lawhead Organization: NVision Solutions Inc. User-Agent: Mozilla Thunderbird 0.6 (Windows/20040502) X-Accept-Language: en-us, en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at triqs.com X-Mailman-Approved-At: Sat, 19 Jun 2004 11:06:02 +0200 Subject: [Metakit] Python, Metakit, Sorting tracking data X-BeenThere: metakit@equi4.com X-Mailman-Version: