From jcw@equi4.com Thu Mar 29 04:12:18 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA04181 for ; Thu, 29 Mar 2001 04:12:18 -0600 From: Jean-Claude Wippler To: metakit Date: Thu, 29 Mar 2001 02:07:58 -0800 Message-Id: <20010329100758.23554@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Welcome! Sender: metakit-admin@www.equi4.com Errors-To: metakit-admin@www.equi4.com X-BeenThere: metakit@www.equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Welcome to the new mailing list. There are still some quirks in this setup but it ought to work. I will soon move over all users of the old metakit-core and metakit-scripting mailing lists - currently hosted by SourceForge. Once that works out, the metakit- announce list (which holds several hundred subscribers) will also be relocated and announced. Reasons for these changes were: - my SF setup has not been fixed after *5* months of pleading - the split between -core and -scripting turns out to be confusing - the new setup lets me to manage good archives of all messages Regards, Jean-Claude From jcw@equi4.com Thu Mar 29 13:03:27 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA23881 for ; Thu, 29 Mar 2001 13:03:24 -0600 From: Jean-Claude Wippler To: metakit Date: Thu, 29 Mar 2001 10:59:12 -0800 Message-Id: <20010329185912.6172@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] mailing lists moved Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The metakit-core@equi4.com and metakit-scripting@equi4.com mailing lists have been merged into one and moved away from SourceForge to the equi4.com server. You are receiving this email because you were subscribed to either of the above lists. This change has several unfortunate side-effects: - you need to re-enable daily digests again, if you want them - you need to alter your password if you don't want the new one - all mailing archives are lost (but I'll try to restore them) As before, all list administration can be done through the web, because the new list is again based on Mailman. Just go to: http://www.equi4.com/mailman/listinfo/metakit See also the new "announce" mailing list, which replaces the previous "metakit-announce" list. It too is now located on the equi4.com server: http://www.equi4.com/mailman/listinfo/announce Please let me know if the relocated list does not work for you. Allow me to add that because of this move, we will at last have properly working archives for all new messages, and that problems with mail not being sent to everyone should now be fully resolved: these were the two main reasons for this change. Regards, Jean-Claude From chris@pinebush.com Thu Mar 29 16:40:48 2001 Received: from pinebush.com (host-64-65-203-122.choiceone.net [64.65.203.122]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA31350 for ; Thu, 29 Mar 2001 16:40:48 -0600 Received: from pinebush.com (winntsrv1.pinebush.com [10.0.0.17]) by pinebush.com (8.9.3/8.9.3) with ESMTP id RAA14855 for ; Thu, 29 Mar 2001 17:36:27 -0500 (EST) Message-ID: <3AC3B8EB.54620796@pinebush.com> Date: Thu, 29 Mar 2001 17:36:27 -0500 From: Chris Nelson Organization: Pinebush Technologies, Inc. (http://www.pinebush.com) X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Problem with open files on 'Windows Me Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I realize this is vague and wide open but it's all I've got right now and I'm hoping it rings a bell with someone. We have a Tcl-based 'Windows application that uses MetaKit v2.0. On NT, 95, and 98, all is well but on Me, the application appears to leave it's MetaKit file open such that after the application is exited, the file can't be deleted because the system thinks it's in use. Same binary, same code paths, etc. The only variable seems to be the OS. Any insight would be appreciated. Chris -- Rens-se-LEER is a county. RENS-se-ler is a city. R-P-I is a school! From jcw@equi4.com Thu Mar 29 18:49:43 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA02278; Thu, 29 Mar 2001 18:49:37 -0600 From: Jean-Claude Wippler To: , Date: Thu, 29 Mar 2001 16:45:25 -0800 Message-Id: <20010330004525.17056@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.3.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There's a new 2.3.4 release of MetaKit. It consolidates and fixes a few more issues since 2.3.3 was released many months ago. This release now includes the code needed to build TclKit, an installation-free runtime combining MetaKit with Tcl and Tk. The new code passes all tests on Win32, Linux, and Solaris (except for one difference in test result "s33c", which can be ignored for now). The next release will be called "final" (really!), as soon as some more platform builds have been verified. As before, a detailed change log is online at: http://www.equi4.com/metakit/CHANGES The source download (including a WHATSNEW summary) is at: http://www.equi4.com/pub/mk/metakit-2.3.4-29.tar.gz Binary releases will be added to the web site shortly. Two other changes need to be mentioned here: - As you can see from this email, MK's mailing lists have changed. Please post to metakit@equi4.com from now on. - The CVS repository has been relocated to my "equi4.com" server. For details, see http://www.equi4.com/metakit/ Regards, Jean-Claude From akao347@hotmail.com Thu Mar 29 20:33:25 2001 Received: from hotmail.com (oe44.law4.hotmail.com [216.33.148.81]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA03842 for ; Thu, 29 Mar 2001 20:33:24 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 29 Mar 2001 18:28:33 -0800 X-Originating-IP: [18.245.5.30] From: "A K" To: References: <20010314062842.C2EE124889@mail.digital-smarties.com> <20010327075252.3689@smtp.telus.net> Date: Thu, 29 Mar 2001 21:26:28 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 30 Mar 2001 02:28:33.0805 (UTC) FILETIME=[1E0273D0:01C0B8C1] Subject: [Metakit] Re: [MK-Core] Demo crashes alternatively Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, This problem still occurs in the 2.3.4 release. It only happen on debug builds. It doesn't matter if you try to use STD or MFC, they all have the same behavior. It is something related to committing after retrieving views differently from how it is already stored. I tried to reconstruct the exact situation to cause the crash, but I was unable to. There seems to be some sort of bad karma around mkdemo. Kao. ----- Original Message ----- From: "Jean-Claude Wippler" To: Sent: Tuesday, March 27, 2001 2:52 AM Subject: Re: [MK-Core] Demo crashes alternatively > Geoffrey Zhu wrote: > > >I just downloaded the metakit database. It is really a great product. But > >here is a problem: the mkdemo included in the sample project will crash the > >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, > >especially when I am tracing the code line by line in debug mode. Everything > >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I run > >the compiled program outside the IDE. > > > >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows NT > >4.0 with SP6. > > > >Anybody has any clue? > > Could you please fetch the latest release from the new CVS repository on > equi4.com and try things there? The CVS commands (on Unix) are: > cvs -d:pserver:anonymous@equi4.com:/home/cvs login > > cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit > > I seem to recall something like this happening before, but AFAIK it has > been resolved. Or were you perhaps using MK 2.0? > > -jcw > > > _______________________________________________ > metakit-core@equi4.com - http://www.equi4.com/metakit/ > http://lists.sourceforge.net/lists/listinfo/metakit-core > From readonly@getsoft.com Thu Mar 29 20:46:29 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA04098 for ; Thu, 29 Mar 2001 20:46:28 -0600 Received: from NEVILLE550 (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id TAA15706; Thu, 29 Mar 2001 19:42:04 -0700 (MST) Message-ID: <200103301242030770.00CE00C2@getsoft.com> In-Reply-To: References: <20010314062842.C2EE124889@mail.digital-smarties.com> <20010327075252.3689@smtp.telus.net> X-Mailer: Calypso Version 3.10.03.02 (3) Date: Fri, 30 Mar 2001 12:42:03 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "A K" , metakit-core@equi4.com Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id UAA04098 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Kao, FYI I've had memory corruptions and other exceptions using 2.3.3 with VC++ / MFC. *********** REPLY SEPARATOR *********** On 29 Mar 2001 at 21:26 A K wrote: >Hi, > >This problem still occurs in the 2.3.4 release. It only happen on debug >builds. It doesn't matter if you try to use STD or MFC, they all have the >same behavior. It is something related to committing after retrieving views >differently from how it is already stored. > >I tried to reconstruct the exact situation to cause the crash, but I was >unable to. There seems to be some sort of bad karma around mkdemo. > >Kao. > > >----- Original Message ----- >From: "Jean-Claude Wippler" >To: >Sent: Tuesday, March 27, 2001 2:52 AM >Subject: Re: [MK-Core] Demo crashes alternatively > > >> Geoffrey Zhu wrote: >> >> >I just downloaded the metakit database. It is really a great product. But >> >here is a problem: the mkdemo included in the sample project will crash >the >> >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, >> >especially when I am tracing the code line by line in debug mode. >Everything >> >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I >run >> >the compiled program outside the IDE. >> > >> >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows >NT >> >4.0 with SP6. >> > >> >Anybody has any clue? >> >> Could you please fetch the latest release from the new CVS repository on >> equi4.com and try things there? The CVS commands (on Unix) are: >> cvs -d:pserver:anonymous@equi4.com:/home/cvs login >> >> cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit >> >> I seem to recall something like this happening before, but AFAIK it has >> been resolved. Or were you perhaps using MK 2.0? >> >> -jcw >> >> >> _______________________________________________ >> metakit-core@equi4.com - http://www.equi4.com/metakit/ >> http://lists.sourceforge.net/lists/listinfo/metakit-core >> >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find lots of information on ED and while you're there you can click on 'Stay Informed' to join our mailing list and 'Future Plans' to read about Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From gzhu@peak6.com Thu Mar 29 20:54:05 2001 Received: from p6chifs1.peak6.com (node-cffbbe42.powerinter.net [207.251.190.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id UAA04273 for ; Thu, 29 Mar 2001 20:54:05 -0600 Received: by P6CHIFS1 with Internet Mail Service (5.5.2653.19) id ; Thu, 29 Mar 2001 20:49:39 -0600 Message-ID: From: Geoffrey Zhu To: "'metakit-core@equi4.com'" Subject: RE: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 20:49:37 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From the fact that it crashes every other time, I figured that it may be the problem of the data file. I deleted the .DAT file that comes with the package and mkdemo seems to be fine. If I replace it with the old .DAT file, mkdemo will crash every other time. -----Original Message----- From: A K [mailto:akao347@hotmail.com] Sent: Thursday, March 29, 2001 8:26 PM To: metakit-core@equi4.com Subject: [Metakit] Re: [MK-Core] Demo crashes alternatively Hi, This problem still occurs in the 2.3.4 release. It only happen on debug builds. It doesn't matter if you try to use STD or MFC, they all have the same behavior. It is something related to committing after retrieving views differently from how it is already stored. I tried to reconstruct the exact situation to cause the crash, but I was unable to. There seems to be some sort of bad karma around mkdemo. Kao. ----- Original Message ----- From: "Jean-Claude Wippler" To: Sent: Tuesday, March 27, 2001 2:52 AM Subject: Re: [MK-Core] Demo crashes alternatively > Geoffrey Zhu wrote: > > >I just downloaded the metakit database. It is really a great product. But > >here is a problem: the mkdemo included in the sample project will crash the > >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, > >especially when I am tracing the code line by line in debug mode. Everything > >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I run > >the compiled program outside the IDE. > > > >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows NT > >4.0 with SP6. > > > >Anybody has any clue? > > Could you please fetch the latest release from the new CVS repository on > equi4.com and try things there? The CVS commands (on Unix) are: > cvs -d:pserver:anonymous@equi4.com:/home/cvs login > > cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit > > I seem to recall something like this happening before, but AFAIK it has > been resolved. Or were you perhaps using MK 2.0? > > -jcw > > > _______________________________________________ > metakit-core@equi4.com - http://www.equi4.com/metakit/ > http://lists.sourceforge.net/lists/listinfo/metakit-core > _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From akao347@hotmail.com Thu Mar 29 21:16:39 2001 Received: from hotmail.com (oe69.law4.hotmail.com [216.33.148.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA04558 for ; Thu, 29 Mar 2001 21:16:39 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 29 Mar 2001 19:11:45 -0800 X-Originating-IP: [18.245.5.30] From: "A K" To: References: Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 22:09:49 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 30 Mar 2001 03:11:45.0955 (UTC) FILETIME=[270D2730:01C0B8C7] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Yes, it is related to the .dat file. Let me try to explain a bit more. So the first time you run the program, it creates the database with the following structure: address[name:S,country:S] then it adds a third field, so the structure becomes: address[name:S,country:S,age:I] The second time you run the program, it tries to conver the structure from address[name:S,country:S,age:I] to address[name:S,country:S], which causes a crash on commit. It appears to succed in commit, but it still crashes. So the next time you run the program, the database has the structure address[name:S,country:S], which is exactly what the program initially wants. I don't seem to be very good at explaining. Brendan. ----- Original Message ----- From: "Geoffrey Zhu" To: Sent: Thursday, March 29, 2001 9:49 PM Subject: RE: [Metakit] Re: [MK-Core] Demo crashes alternatively > From the fact that it crashes every other time, I figured that it may be the > problem of the data file. I deleted the .DAT file that comes with the > package and mkdemo seems to be fine. If I replace it with the old .DAT file, > mkdemo will crash every other time. > > > > > > -----Original Message----- > From: A K [mailto:akao347@hotmail.com] > Sent: Thursday, March 29, 2001 8:26 PM > To: metakit-core@equi4.com > Subject: [Metakit] Re: [MK-Core] Demo crashes alternatively > > > Hi, > > This problem still occurs in the 2.3.4 release. It only happen on debug > builds. It doesn't matter if you try to use STD or MFC, they all have the > same behavior. It is something related to committing after retrieving views > differently from how it is already stored. > > I tried to reconstruct the exact situation to cause the crash, but I was > unable to. There seems to be some sort of bad karma around mkdemo. > > Kao. > > > ----- Original Message ----- > From: "Jean-Claude Wippler" > To: > Sent: Tuesday, March 27, 2001 2:52 AM > Subject: Re: [MK-Core] Demo crashes alternatively > > > > Geoffrey Zhu wrote: > > > > >I just downloaded the metakit database. It is really a great product. But > > >here is a problem: the mkdemo included in the sample project will crash > the > > >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, > > >especially when I am tracing the code line by line in debug mode. > Everything > > >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I > run > > >the compiled program outside the IDE. > > > > > >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows > NT > > >4.0 with SP6. > > > > > >Anybody has any clue? > > > > Could you please fetch the latest release from the new CVS repository on > > equi4.com and try things there? The CVS commands (on Unix) are: > > cvs -d:pserver:anonymous@equi4.com:/home/cvs login > > > > cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit > > > > I seem to recall something like this happening before, but AFAIK it has > > been resolved. Or were you perhaps using MK 2.0? > > > > -jcw > > > > > > _______________________________________________ > > metakit-core@equi4.com - http://www.equi4.com/metakit/ > > http://lists.sourceforge.net/lists/listinfo/metakit-core > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From akao347@hotmail.com Thu Mar 29 21:17:44 2001 Received: from hotmail.com (oe25.law4.hotmail.com [216.33.148.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA04581 for ; Thu, 29 Mar 2001 21:17:43 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 29 Mar 2001 19:12:52 -0800 X-Originating-IP: [18.245.5.30] From: "A K" To: References: <20010314062842.C2EE124889@mail.digital-smarties.com> <20010327075252.3689@smtp.telus.net> <200103301242030770.00CE00C2@getsoft.com> Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 22:10:54 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 30 Mar 2001 03:12:52.0784 (UTC) FILETIME=[4EE27300:01C0B8C7] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Can you give some examples? I'm not using the MFC, but I am worried about any possible memory corruptions / exceptions. Kao. ----- Original Message ----- From: "Neville Franks" To: "A K" ; Sent: Thursday, March 29, 2001 9:42 PM Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Kao, FYI I've had memory corruptions and other exceptions using 2.3.3 with VC++ / MFC. *********** REPLY SEPARATOR *********** On 29 Mar 2001 at 21:26 A K wrote: >Hi, > >This problem still occurs in the 2.3.4 release. It only happen on debug >builds. It doesn't matter if you try to use STD or MFC, they all have the >same behavior. It is something related to committing after retrieving views >differently from how it is already stored. > >I tried to reconstruct the exact situation to cause the crash, but I was >unable to. There seems to be some sort of bad karma around mkdemo. > >Kao. > > >----- Original Message ----- >From: "Jean-Claude Wippler" >To: >Sent: Tuesday, March 27, 2001 2:52 AM >Subject: Re: [MK-Core] Demo crashes alternatively > > >> Geoffrey Zhu wrote: >> >> >I just downloaded the metakit database. It is really a great product. But >> >here is a problem: the mkdemo included in the sample project will crash >the >> >2nd, 4th, 6th, 8th... time when I run it in Visual C++ 6.0 environment, >> >especially when I am tracing the code line by line in debug mode. >Everything >> >is perfect in the 1st, 3rt, 5th, 7th... run. Everything is okay when I >run >> >the compiled program outside the IDE. >> > >> >I am using Visual C++ 6.0 with SP4 and the latest Platform SDK, Windows >NT >> >4.0 with SP6. >> > >> >Anybody has any clue? >> >> Could you please fetch the latest release from the new CVS repository on >> equi4.com and try things there? The CVS commands (on Unix) are: >> cvs -d:pserver:anonymous@equi4.com:/home/cvs login >> >> cvs -d:pserver:anonymous@equi4.com:/home/cvs -z6 co metakit >> >> I seem to recall something like this happening before, but AFAIK it has >> been resolved. Or were you perhaps using MK 2.0? >> >> -jcw >> >> >> _______________________________________________ >> metakit-core@equi4.com - http://www.equi4.com/metakit/ >> http://lists.sourceforge.net/lists/listinfo/metakit-core >> >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find lots of information on ED and while you're there you can click on 'Stay Informed' to join our mailing list and 'Future Plans' to read about Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From jcw@equi4.com Thu Mar 29 21:55:06 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA05017 for ; Thu, 29 Mar 2001 21:55:01 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 19:50:06 -0800 Message-Id: <20010330035006.20918@smtp.telus.net> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A K wrote: >Can you give some examples? I'm not using the MFC, but I am worried about >any possible memory corruptions / exceptions. I've been able to reproduce the problem. It does not seem related to MFC. It crashes when run from MSVC6, but *not* when run from the command line. Every other time, as reported. Strange. I will investigate. Luckily it's a debug build, so that does make things easier! -jcw P.S. Please post to metakit@equi4.com instead of metakit-core@equi4.com / metakit-scripting@equi4.com from now on, those latter two mailing lists are about to be shut down (all subscriptions have been transferred). From akao347@hotmail.com Thu Mar 29 22:38:30 2001 Received: from hotmail.com (oe11.law4.hotmail.com [216.33.148.115]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA05492 for ; Thu, 29 Mar 2001 22:38:30 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 29 Mar 2001 20:33:38 -0800 X-Originating-IP: [18.245.5.30] From: "A K" To: References: <20010330035006.20918@smtp.telus.net> Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively Date: Thu, 29 Mar 2001 23:31:41 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 30 Mar 2001 04:33:38.0871 (UTC) FILETIME=[9760A470:01C0B8D2] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Really? The program still crashes when I run from the commandline? Are you running mkdemo.exe or mkdemo_d.exe (using your build scripts). Kao. ----- Original Message ----- From: "Jean-Claude Wippler" To: Sent: Thursday, March 29, 2001 10:50 PM Subject: Re: [Metakit] Re: [MK-Core] Demo crashes alternatively > A K wrote: > > >Can you give some examples? I'm not using the MFC, but I am worried about > >any possible memory corruptions / exceptions. > > I've been able to reproduce the problem. It does not seem related to > MFC. It crashes when run from MSVC6, but *not* when run from the command > line. Every other time, as reported. > > Strange. I will investigate. Luckily it's a debug build, so that does > make things easier! > > -jcw > > P.S. Please post to metakit@equi4.com instead of metakit-core@equi4.com > / metakit-scripting@equi4.com from now on, those latter two mailing lists > are about to be shut down (all subscriptions have been transferred). > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From klisa@online.fr Fri Mar 30 01:18:57 2001 Received: from mark.labs.trema.com (mark.labs.trema.com [194.103.215.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA06804 for ; Fri, 30 Mar 2001 01:18:56 -0600 Received: from dellvladimir (pclabs189.labs.trema.com [194.103.215.189]) by mark.labs.trema.com (8.9.3/8.9.3) with SMTP id JAA10733 for ; Fri, 30 Mar 2001 09:14:08 +0200 (MET DST) Message-ID: <003a01c0b8e9$0f679030$bdd767c2@labs.trema.com> From: "Vladimir Klisic" To: References: <20010323163724.7AEAC25558@mail.digital-smarties.com> <20010327075023.4571@smtp.telus.net> Date: Fri, 30 Mar 2001 09:14:21 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Re: [MK-Core] Metakit performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Even if you do not know the number of rows in advance, pre-allocating > say chunks of 1000 rows whenever you run out, and cleaning up at > the end, can make all the difference. It makes all the difference in the world, indeed. Thanks. What I'm getting right now is almost constant row preparation and insertion time. Insertion time is now roughly between 1 and 2 seconds for 1000 rows in debug build. It will likely be much faster in release build, with or without Metakit being inlined (unfortunately, I cannot test it right now). > The difference between C++ and Python surprises me. My only > explanation is that one was compiled with q4_INLINE and > the other without. It could be true. Python test program uses Metakit DLL which was downloaded from equi4 web site, and I was building Metakit without q4_INLINE defined. > How many ints/floats/strings do you have *each* per row? It's hard to say. It depends on each particular stored procedure that is used as primary data source. On average, string columns should comprise 1/3 of all columns. > The speed of MK is directly proportional to the number of fields. Also, > strings are variable-sized and a lot slower to deal with than ints/floats. Some basic profiling showed that - 45% of the time was spent in RtlAllocateHeap (just another name for HeapAlloc on Win32). Called roughly 14 million times for 40000 rows. - 22% of the time was spent RtlFreeHeap. Called roughly 14 million times. - 5% of the time was spent in RtlReallocateHeap. Called 10 million times. Next five functions on this list are - c4_DoubleRef::operator= (4.5%). Called 2 million times. - c4_View::Add (3.3%). Called 40000 times :). - c4_IntRef::operator= (3%). Called 1.4 million times. - c4_StringRef::operator= (2%). Called 720000 times. - c4_Row::~c4_Row (2%). Same as c4_View, called 40000. Results above are obtained before I used row-preallocation code. I expected to see more direct way to move data from c4_Row to physical file on the disk (without that much memory management calls), but I guess it's the trade-off between simplicity and flexibility. And it still works quite fast if row-preallocation technique is used. Since there is significant number of heap allocations, it may be a good idea to use separate heaps for some or all Metakit types? Having only objects of the same size on the heap could help heap manager perform better (not to mention less memory fragmentation). > what performance are you after? and what type of operations > are dominant in your app? It goes like this: 1. Data is fetched from the stored procedure. Basically, it's a loop in which rows get stored one-by-one in Metakit file. No additional actions are taken in between, just inserts. Needs to be as fast as possible (or even faster). 2. Data is processed. Main loop goes through all the rows, creating several _new_ columns (existing columns are not modified). For each column, its value gets calculated out of values from other columns. (This is by far the most problematic part, and I did not even come to it because I was stuck with insertion performance.) What happens here is that a) Type of new column is not known in advance. It is determined once the value is calculated for the very first time. b) Once the very first row is processed, types of all additional columns are known and Metakit structure needs to be updated on the fly. Content of those additional columns is then inserted to the same Metakit file from step 1, repeating this for every row from then on. 3. Data is retrieved by some other application. Now the loop goes through all the rows, just reading the content. Needs to be as fast as possible. As you can see, looks like I'm using only core Metakit functionality. It's highly unlikely that I will ever need to add something more to it, so the question is: "what can I do to speed up everything from step 1 to step 3?" Thanks, Vladimir From jcw@equi4.com Fri Mar 30 01:49:30 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA07438 for ; Fri, 30 Mar 2001 01:49:28 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Re: [MK-Core] Metakit performance Date: Thu, 29 Mar 2001 23:45:20 -0800 Message-Id: <20010330074520.26913@smtp.telus.net> In-Reply-To: <003a01c0b8e9$0f679030$bdd767c2@labs.trema.com> References: <003a01c0b8e9$0f679030$bdd767c2@labs.trema.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Vladimir Klisic wrote: [...] >Some basic profiling showed that >- 45% of the time was spent in RtlAllocateHeap (just another name for >HeapAlloc on Win32). Called roughly 14 million times for 40000 rows. I.e. 35 times per row (1/3rd of the 100 props are strings, you say?). >- 22% of the time was spent RtlFreeHeap. Called roughly 14 million times. >- 5% of the time was spent in RtlReallocateHeap. Called 10 million times. Maybe it's possible to avoid some string allocations altogether. Can you determine whether most mallocs take places inside MK? Are you using c4_String? MK is relatively careful to not do too many allocs (c4_Bytes are used internally, which avoid mallocs for strings under 16 bytes). If we can determine where the mallocs are, that would help a lot. Can you interrupt a few random times and get stack traces? It's important to find out what *calls* the RtlAllocateHeap code. >question is: "what can I do to speed up everything from step 1 to step 3?" I think tracing the cause of the mallocs is a useful exercise. Only as last resort would I consider making the mallocs faster. Based on yout stats, even instant mallocs can "only" give you a 72% speed increase (i.e. less than an order of magnitude). -jcw From vladimir.klisic@trema.com Fri Mar 30 03:14:22 2001 Received: from mark.labs.trema.com (mark.labs.trema.com [194.103.215.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA09238 for ; Fri, 30 Mar 2001 03:14:21 -0600 Received: from dellvladimir (pclabs189.labs.trema.com [194.103.215.189]) by mark.labs.trema.com (8.9.3/8.9.3) with SMTP id LAA10914 for ; Fri, 30 Mar 2001 11:09:38 +0200 (MET DST) Message-ID: <00ba01c0b8f9$31966040$bdd767c2@labs.trema.com> From: "Vladimir Klisic" To: Subject: Re: [Metakit] Re: [MK-Core] Metakit performance Date: Fri, 30 Mar 2001 11:09:54 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00B5_01C0B909.F28D3AC0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_00B5_01C0B909.F28D3AC0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable > how do you do profiling on win32? Performance tuning feature in VC++ is availabe only in MSVC Enterprise editions. Assuming that you have one, step-by-step explanation about how = to do it can be found in MSDN Library. Just follow Visual C++ documentation = -> Using Visual C++ -> Visual C++ Programmer's Guide -> Performance Tuning. = You may also wish to check MS online Knowledge Base. There is also another way to do it: using NuMega TrueTime profiler, part = of excellent suite called NuMega DevPartner Studio (www.numega.com). = TrueTime integrates into MSVC development environment, making profiling as easy = as 'Build with TrueTime', 'Run with TrueTime'. Last time I was checking it, they were offering 15 or 30-day demo at their site (it was not recently, though). Highly recommended (just as their BoundsChecker, btw). In my case, due to some particularities of build process I wasn't able = to profile the application I'm working on directly from MSVC IDE. Instead, = I've extracted all Metakit-related code out of it and created small test application (with exactly the same code constructions) which was easier = to profile. In your case it should be much easier. ------=_NextPart_000_00B5_01C0B909.F28D3AC0 Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
> how do you do profiling on=20 win32?

Performance tuning feature in VC++ is availabe only in = MSVC=20 Enterprise
editions. Assuming that you have one, step-by-step = explanation=20 about how to
do it can be found in MSDN Library. Just follow Visual = C++=20 documentation ->
Using Visual C++ -> Visual C++ Programmer's = Guide=20 -> Performance Tuning. You
may also wish to check MS online = Knowledge=20 Base.

There is also another way to do it: using NuMega TrueTime = profiler,=20 part of
excellent suite called NuMega DevPartner Studio (www.numega.com). TrueTime
integrates into MSVC development environment, = making=20 profiling as easy as
'Build with TrueTime', 'Run with TrueTime'. Last = time I=20 was checking it,
they were offering 15 or 30-day demo at their site = (it was=20 not recently,
though). Highly recommended (just as their = BoundsChecker,=20 btw).

In my case, due to some particularities of build process I = wasn't=20 able to
profile the application I'm working on directly from MSVC = IDE.=20 Instead, I've
extracted all Metakit-related code out of it and = created small=20 test
application (with exactly the same code constructions) which was = easier=20 to
profile. In your case it should be much=20 easier.

------=_NextPart_000_00B5_01C0B909.F28D3AC0-- From readonly@getsoft.com Fri Mar 30 05:32:22 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA15137 for ; Fri, 30 Mar 2001 05:32:21 -0600 Received: from NEVILLE550 (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id EAA21412; Fri, 30 Mar 2001 04:27:51 -0700 (MST) Message-ID: <200103302127510650.00998EBC@getsoft.com> In-Reply-To: <00ba01c0b8f9$31966040$bdd767c2@labs.trema.com> References: <00ba01c0b8f9$31966040$bdd767c2@labs.trema.com> X-Mailer: Calypso Version 3.10.03.02 (3) Date: Fri, 30 Mar 2001 21:27:51 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "Vladimir Klisic" , metakit@equi4.com Subject: Re: [Metakit] Re: [MK-Core] Metakit performance Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=====_9859516716334=_" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=====_9859516716334=_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable An very good Windows Profiler can be found at http://www.glowcode.com/ *********** REPLY SEPARATOR *********** On 30 Mar 2001 at 11:09 Vladimir Klisic wrote: > how do you do profiling on win32? Performance tuning feature in VC++ is availabe only in MSVC Enterprise editions. Assuming that you have one, step-by-step explanation about how to do it can be found in MSDN Library. Just follow Visual C++ documentation -> Using Visual C++ -> Visual C++ Programmer's Guide -> Performance Tuning.= You may also wish to check MS online Knowledge Base. There is also another way to do it: using NuMega TrueTime profiler, part of excellent suite called NuMega DevPartner Studio (www.numega.com). TrueTime integrates into MSVC development environment, making profiling as easy as 'Build with TrueTime', 'Run with TrueTime'. Last time I was checking it, they were offering 15 or 30-day demo at their site (it was not recently, though). Highly recommended (just as their BoundsChecker, btw). In my case, due to some particularities of build process I wasn't able to profile the application I'm working on directly from MSVC IDE. Instead,= I've extracted all Metakit-related code out of it and created small test application (with exactly the same code constructions) which was easier to profile. In your case it should be much easier. Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find= lots of information on ED and while you're there you can click on 'Stay= Informed' to join our mailing list and 'Future Plans' to read about= Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- --=====_9859516716334=_ Content-Type: text/html; charset="us-ascii"
An very good Windows Profiler can be found at http://www.glowcode.com/
 

*********** REPLY SEPARATOR ***********

On 30 Mar 2001 at 11:09 Vladimir Klisic wrote:
> how do you do profiling on win32?

Performance tuning feature in VC++ is availabe only in MSVC Enterprise
editions. Assuming that you have one, step-by-step explanation about how to
do it can be found in MSDN Library. Just follow Visual C++ documentation ->
Using Visual C++ -> Visual C++ Programmer's Guide -> Performance Tuning. You
may also wish to check MS online Knowledge Base.

There is also another way to do it: using NuMega TrueTime profiler, part of
excellent suite called NuMega DevPartner Studio (www.numega.com). TrueTime
integrates into MSVC development environment, making profiling as easy as
'Build with TrueTime', 'Run with TrueTime'. Last time I was checking it,
they were offering 15 or 30-day demo at their site (it was not recently,
though). Highly recommended (just as their BoundsChecker, btw).

In my case, due to some particularities of build process I wasn't able to
profile the application I'm working on directly from MSVC IDE. Instead, I've
extracted all Metakit-related code out of it and created small test
application (with exactly the same code constructions) which was easier to
profile. In your case it should be much easier.

Best Regards,
    Neville Franks (Author of ED for Windows)

PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find=
 lots of information on ED and while you're there you can click on 'Stay=
 Informed' to join our mailing list and 'Future Plans' to read about=
 Version 4.

--------------------------------------------------------------
Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia
            Developers and publishers of ED for Windows
Ph  +61 3 9885 4445          WWW  http://www.getsoft.com
Fax +61 3 9885 4444         Email sales@getsoft.com
---------------------------------------------------------------
--=====_9859516716334=_-- From lvirden@cas.org Fri Mar 30 08:24:07 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA19201 for ; Fri, 30 Mar 2001 08:24:06 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id JAA16734 for ; Fri, 30 Mar 2001 09:19:11 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.9.3+Sun/m4_8.9.3/CAS_CLIENT-1.17) id JAA05529; Fri, 30 Mar 2001 09:19:09 -0500 (EST) Date: Fri, 30 Mar 2001 09:19:09 -0500 (EST) From: "Larry W. Virden" Message-Id: <0103300919.AA5475@cas.org> To: Subject: [Metakit] Forwarded: Returned mail: User unknown, Wikit change needed - preventing wikit from trying to create pages with titles that cause problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I think that the wikit should modify titles that contain characters or strings that cause problems. For instance, pages that end in .tcl only get their title boldfaced and resized. The rest of the text doesn't get processed. The wikit should just not create files by that name (at least by default) - instead, perhaps the user should be asked 'are you sure'? Also, if the user included punctuation that would cause problems, then it should be removed before the title is created... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From bernardl@acumenassociates.com Fri Mar 30 12:36:40 2001 Received: from amadeus.acumenassociates.com ([205.219.35.70]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA27894 for ; Fri, 30 Mar 2001 12:36:39 -0600 Received: from c1524207a ([65.12.238.16]) by amadeus.acumenassociates.com (Post.Office MTA v3.5.3 release 223 ID# 0-59667U100L2S100V35) with SMTP id com for ; Fri, 30 Mar 2001 10:43:19 -0800 Message-ID: <00ab01c0b947$7d12bb10$6b01a8c0@bllvu1.wa.home.com> From: "Bernard Littau" To: Date: Fri, 30 Mar 2001 10:30:25 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Subject: [Metakit] More optimization Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been following the discussion on optimizing loading, and have used some of the suggestions to good effect. My database, however, has a view within a row. For example: c4_View db = rage.GetAs( "db[a:I,b:I,c:S,d:I,e:I," "foo[x:I,y:S,z:I]," "bar[x:I,y:S,z:I]," "baz[x:I,y:S]]" ); If I run the insert with the foo, bar and baz inserts commented out, the inserts are very fast. I am preallocating rows, and using a RowRef to do the data inserts. If I add in even just foo, and foo is just one row, the process starts to slow as more rows get added to db. What strategies can be used to optimize the load given one has views within a row? Thanks, Bernard Littau From jcw@equi4.com Fri Mar 30 12:45:30 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA28342 for ; Fri, 30 Mar 2001 12:45:29 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] More optimization Date: Fri, 30 Mar 2001 10:41:30 -0800 Message-Id: <20010330184130.26531@smtp.telus.net> In-Reply-To: <00ab01c0b947$7d12bb10$6b01a8c0@bllvu1.wa.home.com> References: <00ab01c0b947$7d12bb10$6b01a8c0@bllvu1.wa.home.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bernard Littau wrote: > c4_View db = >rage.GetAs( > "db[a:I,b:I,c:S,d:I,e:I," > "foo[x:I,y:S,z:I]," > "bar[x:I,y:S,z:I]," > "baz[x:I,y:S]]" ); > >If I run the insert with the foo, bar and baz inserts commented out, the >inserts are very fast. I am preallocating rows, and using a RowRef to do >the data inserts. > >If I add in even just foo, and foo is just one row, the process starts to >slow as more rows get added to db. > >What strategies can be used to optimize the load given one has views >within a row? How are you setting subviews? Constructing rows is slow (you are correct in using rowref's as much as possible). Could you show some code on how you create and set foo? MK might be copying subviews too often, with code examples I can suggest things to try. Hm. Or maybe you could load column-wise? Define the view without subviews, and load the a..e columns. Then restructure to include foo, and in a loop set all foo values to what you need, then bar, etc (or maybe all of them at once). -jcw From jcw@equi4.com Fri Mar 30 14:45:54 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA00320 for ; Fri, 30 Mar 2001 14:45:53 -0600 From: Jean-Claude Wippler To: Date: Fri, 30 Mar 2001 12:41:38 -0800 Message-Id: <20010330204138.18631@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] How to unsubscribe Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: With the renewed traffic on the relocated metakit@equi4.com list, I've received a few requests from people who wish to unsubscribe, so here's the info: - go to http://www.equi4.com/mailman/listinfo/metakit - go to your option page as follows: enter your email address at the *bottom*, press "Edit Options" - all passwords were changed, so you need to find out yours press "Email My Password To Me" - when you have it, enter that password and press "Unsubscribe" Alternately, you could choose to batch messages into daily digests: - Choose "On" in the "Set Digest Mode" field - Enter your password at the bottom of the page - Press "Submit My Changes" All of the above use the Mailman web interface. The other way to do things (but I've never tried it), is to send a message to "metakit- request@equi4.com" with "help" in the body of your email msg. Note that a reminder with your (new) password is emailed to you once a month, the first one should happen within in a day or so. My apologies for the password changes and for losing all "send digest" preferences, I saw no way to maintain these in the new list setup. -jcw From bernardl@acumenassociates.com Fri Mar 30 16:09:32 2001 Received: from amadeus.acumenassociates.com ([205.219.35.70]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA02825 for ; Fri, 30 Mar 2001 16:09:32 -0600 Received: from c1524207a ([65.12.238.16]) by amadeus.acumenassociates.com (Post.Office MTA v3.5.3 release 223 ID# 0-59667U100L2S100V35) with SMTP id com for ; Fri, 30 Mar 2001 14:16:23 -0800 Message-ID: <001d01c0b965$40944c30$6b01a8c0@bllvu1.wa.home.com> From: "Bernard Littau" To: Date: Fri, 30 Mar 2001 14:03:28 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Subject: [Metakit] hash use... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am a bit confused as to how to add a hash to a view. From some of the MetaKit documentation: Example of use: c4_View data = storage.GetAs("people[name:S,age:I]"); c4_View datah = storage.GetAs("people_H1[name:S,age:I]"); c4_View hash = raw.Hash(datah, 1); ... hash.GetSize() ... hash.Add(...) Can I assume that "raw" in the above should be "data"? It looks to me that the hash view is not persistent, but that datah and data are. Is this correct? When data and datah are loaded from storage (new program execution) does one always need to execute the third line above that returns the hash view? I take it additions are made to the hash view, and they propagate to data and datah? In the same documentation: The map_ view must be empty the first time this hash view is used, so that MetaKit can fill it based on whatever rows are already present in the underlying view. After that, neither the underlying view nor the map view may be modified other than through this hash mapping layer. The defined structure of the map view must be "_H:I,_R:I". What is meant by the last line? datah above does not have this structure. If I just wanted to hash based on name, would datah become: c4_View datah = storage.GetAs("people_H1[name:S]"); How do I actually do a lookup using the hash? Do I Find on the hash view? Thanks, Bernard From jcw@equi4.com Fri Mar 30 16:25:29 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA03322 for ; Fri, 30 Mar 2001 16:25:28 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] hash use... Date: Fri, 30 Mar 2001 14:20:57 -0800 Message-Id: <20010330222057.2448@smtp.telus.net> In-Reply-To: <001d01c0b965$40944c30$6b01a8c0@bllvu1.wa.home.com> References: <001d01c0b965$40944c30$6b01a8c0@bllvu1.wa.home.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bernard Littau wrote: > >I am a bit confused as to how to add a hash to a view. > >>From some of the MetaKit documentation: > >Example of use: > > c4_View data = storage.GetAs("people[name:S,age:I]"); > c4_View datah = storage.GetAs("people_H1[name:S,age:I]"); > c4_View hash = raw.Hash(datah, 1); > ... hash.GetSize() ... > hash.Add(...) That example is wrong. The "datah" view must have the form: "people_H1[_H:I,_R:I]" >Can I assume that "raw" in the above should be "data"? Yes, another error: c4_View hash = data.Hash(datah, 1); >It looks to me that the hash view is not persistent, but that datah and data >are. Is this correct? Yes. So on open, you have to reconstruct hash again: c4_View hash = data.Hash(datah, 1); That is instant. >When data and datah are loaded from storage (new program execution) does one >always need to execute the third line above that returns the hash view? Yes. >I take it additions are made to the hash view, and they propagate to data >and datah? It's slightly different: the "hash" view is a custom viewer which intercepts changes. It then "turns around" and applies the appropriate changes to data and datah. >In the same documentation: >The map_ view must be empty the first time this hash view is used, so that >MetaKit can fill it based on whatever rows are already present in the >underlying view. After that, neither the underlying view nor the map view >may be modified other than through this hash mapping layer. The defined >structure of the map view must be "_H:I,_R:I". > >What is meant by the last line? datah above does not have this structure. I hope the above corrections clear this up... >If I just wanted to hash based on name, would datah become: > c4_View datah = storage.GetAs("people_H1[name:S]"); No. The "1" arg to Hash says how many of the first props comprise the key. The datah view does not hold data, it's just hash codes and mapping stuff, all handled internally. Always use the following, regardless of what data is: c4_View datah = storage.GetAs("people_H1[_H:I,_R:I]"); >How do I actually do a lookup using the hash? Do I Find on the hash view? Yes. Give Find a row with just a "name" prop. It'll see that the request matches the first field of the hash and will switch to hashing. If Tcl doesn't confuse you too much, see examples/mapped.tcl - likewise, examples/find.py has useful sample code (plus timing results). -jcw From bernard@blama.com Fri Mar 30 16:44:05 2001 Received: from bandit.telemarker.com ([205.219.35.71]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA03953 for ; Fri, 30 Mar 2001 16:44:05 -0600 Received: from c1524207a ([65.12.238.16]) by bandit.telemarker.com (Post.Office MTA v3.5.3 release 223 ID# 0-69692U200L2S100V35) with SMTP id com for ; Fri, 30 Mar 2001 14:23:32 -0800 Message-ID: <006101c0b96a$142d5100$6b01a8c0@bllvu1.wa.home.com> From: bernard@blama.com (Bernard Littau) To: References: <001d01c0b965$40944c30$6b01a8c0@bllvu1.wa.home.com> <20010330222057.2448@smtp.telus.net> Subject: Re: [Metakit] hash use... Date: Fri, 30 Mar 2001 14:38:01 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thank You, that helps a bunch! > >Example of use: > > > > c4_View data = storage.GetAs("people[name:S,age:I]"); > > c4_View datah = storage.GetAs("people_H1[name:S,age:I]"); > > c4_View hash = raw.Hash(datah, 1); > > ... hash.GetSize() ... > > hash.Add(...) > > That example is wrong. The "datah" view must have the form: > "people_H1[_H:I,_R:I]" > > >Can I assume that "raw" in the above should be "data"? > > Yes, another error: c4_View hash = data.Hash(datah, 1); > [snip] From bernard@blama.com Fri Mar 30 17:22:14 2001 Received: from bandit.telemarker.com ([205.219.35.71]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA05091; Fri, 30 Mar 2001 17:22:13 -0600 Received: from c1524207a ([65.12.238.16]) by bandit.telemarker.com (Post.Office MTA v3.5.3 release 223 ID# 0-69692U200L2S100V35) with SMTP id com; Fri, 30 Mar 2001 15:01:40 -0800 Message-ID: <006901c0b96f$687dfed0$6b01a8c0@bllvu1.wa.home.com> From: bernard@blama.com (Bernard Littau) To: "Jean-Claude Wippler" Cc: References: <00e701c0b94b$4ef070c0$6b01a8c0@bllvu1.wa.home.com> <20010330221207.10060@smtp.telus.net> Subject: Re: [Metakit] More optimization Date: Fri, 30 Mar 2001 15:16:10 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Please don't expect me to fully go into your specific app, I'm just > glancing through to see how I can help. Your concern is to fill a DB > quickly. You have many rows in the main view, and a couple of small-or- > empty subviews in each row, right? Correct, many rows at the top level, each with 5 subviews, and each of those subviews having none to eight rows. > > Are you filling rows incrementally, including the subviews? Or do > subviews get extended in a random order? Creating empty subviews should > not take much time. Th efirst row added will have the overhead of > setting up the subview. After that you get O(N) performance, but you may > not have to preallocate subviews if it's mostly about a few rows. Then > again, albums have a dozen or so songs, right? You could preallocate > subviews to size 10, say. And track all sizes in use. Then, at the end > go through all rows and truncate all subviews to the right size. I figured since the number of rows in each subview was small, that preallocation would not gain much. The problem here is curious, as the time to insert just one row into one subview starts to creep up as more and more rows are added to the top level view. I did a test with 1000 top level rows, each with many columns and one subview. If I do not fill the subview at all, each row is added in zero ms by GetTickCount(), basically, the debug output showing the timing for each row scrolls too fast to read. If I then poulate the one subview with one row, the behavior changes. At first top level rows are added fast, 0 ms using GetTickCount(). By the time a few hundred top level rows are added, the time to insert one such top level row starts to climb, 10 ms, 20 ms and so on. And that is with just one subview with just one row in it. The full blown insert gets glacial, in computer terms :-) Here is my quick table of insert speed against number of rows added at the top level (this is for the full blown dataset), using the first set of code below for each of five subviews. 1 <10 ms 500 130 ms 1000 260 ms 1500 390 ms 2000 490 ms 2500 610 ms The problem is not one of insert speed per se, as the insert speed is quite acceptable, initially. It's just that adding more and more top level rows with a subview row causes the insert speed to slow down. This was unexpected. The original, slow, code to insert rows into a subview looked like this: Data *pM = metaData.pM; while (pM) { pId( m_row ) = atol(pM->id); pDegree( m_row ) = pM->degree; if ( pM->name ) pName( m_row ) = pM->name; m_view.Add( m_row ); pM = pM->pNextM; } pMView( s_row ) = m_view.SortOn( pId ); m_view.RemoveAll(); The code that does not slow down significantly as more and more rows are added looks like this: Data *pM = metaData.pM; c4_View m_view = pMView( s_row ); // do one pass through the data to get a count of rows needed int m_count = 0; while( pM ) { m_count++; pM = pM->pNextM; } // preallocate the rows mood_view.SetSize( m_count ); // reset back to the beginning of the data, and go though again adding to the subview pM = metaData.pM; int m_iter = 0; while (pM) { c4_RowRef m_row = m_view[ m_iter++ ]; pId( m_row ) = atol(pM->id); pDegree( m_row ) = pM->degree; if ( pM->name ) pName( m_row ) = pM->name; // m_view.Add( m_row ); pM = pM->pNextM; } // pMView( s_row ) = m_view.SortOn( pId ); // m_view.RemoveAll(); I am not sure why this helps, but it sure does. The second code example allows adds of top level rows at 10 ms or less for thousands of top level rows. I'd be curious as to why this behavior happens, but for now I am happy to have the data being inserted at an acceptable rate. I hope this helps someone else, too. Best, Bernard Littau From jcw@equi4.com Sat Mar 31 16:14:58 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA20363 for ; Sat, 31 Mar 2001 16:14:57 -0600 From: Jean-Claude Wippler To: Date: Sat, 31 Mar 2001 14:10:58 -0800 Message-Id: <20010331221058.22623@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit bug fix and binaries Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Just days after the 2.3.4 release, an important bug has been resolved (new in the 2.3.x code). This caused the crash on every 2nd call of mkdemo (debug build, Windows) - and may have been the cause of other intermittent crashes during commit and exit. The source code fix has been checked into CVS at equi4.com, updated binaries for Linux and Windows (also of Mk4tcl and Mk4py) are at: http://www.equi4.com/pub/mk/ (in the *-2.3.4-29-* dirs) Also, TclKit has been rebuilt, with binaries for Windows and Linux: http://www.equi4.com/pub/tk/ (in the *-8.4-21-* dirs) For details abotu this one-line change, see: http://www.equi4.com/previews/thebug.txt If you try things out, please let me know if there are any problems! Builds / tweaks for other platforms are most welcome. -jcw From resnikb@EUnet.yu Sun Apr 1 12:45:33 2001 Received: from smtp.EUnet.yu (smtp.EUnet.yu [194.247.192.50]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA27614 for ; Sun, 1 Apr 2001 12:45:32 -0500 Received: from P-1.114.EUnet.yu (P-1.114.EUnet.yu [213.240.1.114]) by smtp.EUnet.yu (8.11.2/8.11.2) with ESMTP id f31HeuH14781 for ; Sun, 1 Apr 2001 19:40:56 +0200 (MEST) Date: Sun, 1 Apr 2001 19:37:24 +0200 From: Bojan Resnik X-Mailer: The Bat! (v1.51) Personal Reply-To: Bojan Resnik X-Priority: 3 (Normal) Message-ID: <1444246228.20010401193724@eunet.yu> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Saving changes to a file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I created a simple application, just as a starting point in learning MetaKit API, but it doesn't quite work as it should. I'm using the MetaKit 2.3.4.29 with the latest bug fixed, compiled under VC++ 6.0 on Windows 98. This is just a simple test program: //------------ Begin -------------------- c4_StringProp p_name("name"); c4_IntProp p_age("age"); c4_Storage data_file( "MatPod.mk", true ); c4_View v_people = data_file.GetAs("person[name:S,age:I]"); v_people.Add( p_name["Bojan"] + p_age[22] ); v_people.Add( p_name["Tamara"] + p_age[28] ); data_file.Commit(); const int v_size = v_people.GetSize(); std::ostringstream s; for ( int p_index = 0; p_index < v_size; ++p_index ) { c4_RowRef r = v_people[p_index]; s << "Name: " << (const char *)(p_name(r)) << '\n' << "Age: " << t4_i32(p_age(r)) << '\n' << std::endl; } //------------ End -------------------- When this executes, the result displayed is Name: Age: 0 Name: Age: 0 And nothing is stored in the file. If I change the argument to c4_Storage constructor to false, disallowing changes to the file, it magically displays the correct data, but again nothing is stored in the output file. I suppose this is a bug in MetaKit, but I may very well be doing something wrong. Regards, Bojan Resnik From jcw@equi4.com Tue Apr 10 14:24:50 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA14160; Tue, 10 Apr 2001 14:24:49 -0500 From: Jean-Claude Wippler To: Cc: Date: Tue, 10 Apr 2001 10:36:58 -0700 Message-Id: <20010410173658.5022@smtp.telus.net> In-Reply-To: <00256A29.005BEF37.00@gblutmg01.gmio.com> References: <00256A29.005BEF37.00@gblutmg01.gmio.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Re: Metakit pkgindex.tcl Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Binayak, >I tried to download the latest windows version of mk4tcl. I got a bunch of >dlls, but no pkgindex.tcl. >The examples appear to assume the existence of one, as they start with a >'package require'. I also >looked in the source code tcl directory, but couldn't find it there >either. Am >I missing something? No - you are right, it's not properly set up. I always use TclKit nowadays, so I don't use Mk4tcl as a separate shared library anymore: http://www.equi4.com/tclkit/ But you can create the file "pkgIndex.tcl" as follows: package ifneeded Mk4tcl 2.3.4 \ [list load [file join [set dir] mk4tcl[info sharedlibextension]]] Place both of them together in a directory called "mk4tcl-2.3.4", and place that directory in a place where Tcl looks for extensions (or keep it in "." and insert "lappend auto_path ." at the top of your script). -jcw From bernard@blama.com Wed Apr 11 17:24:18 2001 Received: from bandit.telemarker.com ([205.219.35.71]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA26091 for ; Wed, 11 Apr 2001 17:24:18 -0500 Received: from c1524207a ([205.219.35.36]) by bandit.telemarker.com (Post.Office MTA v3.5.3 release 223 ID# 0-69692U200L2S100V35) with SMTP id com for ; Wed, 11 Apr 2001 15:02:21 -0700 Message-ID: <007501c0c2d5$5e277280$2423dbcd@bllvu1.wa.home.com> From: bernard@blama.com (Bernard Littau) To: Date: Wed, 11 Apr 2001 15:18:42 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Subject: [Metakit] Clean up issues... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I am seeing a fair bit of unreleased memory from Metakit 2.3.3 when my Windows application is exited. I have a pointer to the Storage and that returns a View when I do the GetAs pMetadb = new c4_Storage( "metadb", true ); MetaView = pMetadb->GetAs( ... ); pId = new c4_IntProp("a_id"); pTitle = new c4_StringProp( "a_title" ); I tried doing delete pMetadb, but that crashes from time to time. No idea how to clean up the view. What is the correct way to clean up the Metakit objects? Thanks, Bernard Littau Woodinville, WA From jcw@equi4.com Wed Apr 11 17:51:36 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.165]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA26613 for ; Wed, 11 Apr 2001 17:51:36 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Clean up issues... Date: Wed, 11 Apr 2001 15:46:08 -0700 Message-Id: <20010411224608.69@smtp.telus.net> In-Reply-To: <007501c0c2d5$5e277280$2423dbcd@bllvu1.wa.home.com> References: <007501c0c2d5$5e277280$2423dbcd@bllvu1.wa.home.com> X-Mailer: CTM PowerMail 3.0.8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bernard Littau wrote: >I am seeing a fair bit of unreleased memory from Metakit 2.3.3 when my >Windows application is exited. That's bad. The last tests I did with 2.3.4 seemed ok, but if I missed any leaks, I definitely would like to plug them. >I have a pointer to the Storage and that returns a View when I do the GetAs > >pMetadb = new c4_Storage( "metadb", true ); > MetaView = pMetadb->GetAs( ... ); > > pId = new c4_IntProp("a_id"); > pTitle = new c4_StringProp( "a_title" ); > >I tried doing delete pMetadb, but that crashes from time to time. No idea >how to clean up the view. You can get rid of a view with: MetaView = c4_View (); (Well, no more data, still an empty view object, until MetaView goes out of scope) >What is the correct way to clean up the Metakit objects? As usual: delete if allocated with new on the heap, and leaving scope if allocated on the stack. My hunch is that 2.3.3 is the problem. FYI, just after 2.3.4 a bug was fixed which may well explain your crashes, see: http://www.equi4.com/metakit/CHANGES and http://www.equi4.com/previews/thebug.txt The best version is in CVS, see: http://www.equi4.com/metakit/download.html -jcw From steven@haryan.to Sun Apr 15 07:48:02 2001 Received: from bdg.centrin.net.id (IDENT:root@bdg.centrin.net.id [202.146.253.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA00314 for ; Sun, 15 Apr 2001 07:47:59 -0500 Received: from yola.haryan.to ([202.146.244.232]) by bdg.centrin.net.id (8.10.2/8.10.2) with ESMTP id f3FCgEt14316 for ; Sun, 15 Apr 2001 19:42:16 +0700 Message-Id: <4.3.2.7.2.20010415170420.01b651b8@202.53.255.51> X-Sender: steven@202.53.255.51 (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Sun, 15 Apr 2001 19:42:10 +0700 To: metakit@equi4.com From: Steven Haryanto Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] failed test report (metakit-2.3.4-29) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Metakit version: ---------------- metakit-2.3.4-29 Relevant lines: --------------- diff ../unix/../tests/ok tests diff ../unix/../tests/ok/s33c.txt tests/s33c.txt 1,6c1 < VIEW 1 rows = a:V < 0: subview 'a' < VIEW 3 rows = p1:B < 0: (2b) < 1: (4b) < 2: (5b) --- > VIEW 1 rows = make: *** [test] Error 1 Also, since I am completely ignorant of libtool, this is a warning I get when I make the 'install' and 'install-python' targets: libtool: install: warning: remember to run `libtool --finish /usr/local/lib' 'install-python' did not copy Mk4py.so & metakit.py to my python site-packages directory. Not a big deal, though. My platform: ------------ [root@server builds]# uname -a Linux server 2.4.2 #4 SMP Mon Apr 9 19:38:48 JAVT 2001 i586 unknown [root@server builds]# gcc --version egcs-2.91.66 [root@server builds]# python Python 1.5.2 (#1, Feb 1 2000, 16:32:16) [GCC egcs-2.91.66 19990314/Linux (egcs- on linux-i386 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> Cheers, -- sh From steven@haryan.to Sun Apr 15 07:48:03 2001 Received: from bdg.centrin.net.id (IDENT:root@bdg.centrin.net.id [202.146.253.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA00318 for ; Sun, 15 Apr 2001 07:48:01 -0500 Received: from yola.haryan.to ([202.146.244.232]) by bdg.centrin.net.id (8.10.2/8.10.2) with ESMTP id f3FCgIt14321 for ; Sun, 15 Apr 2001 19:42:19 +0700 Message-Id: <4.3.2.7.2.20010415194240.01cf9658@202.53.255.51> X-Sender: steven@202.53.255.51 (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Sun, 15 Apr 2001 19:46:31 +0700 To: metakit@equi4.com From: Steven Haryanto Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] sortrev() syntax Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have started playing with Metakit for Python again after a while. vw.sortrev() syntax seems a little awkward to me. Is there a reason why it is so? Regards, Steve From jyl@best.com Fri Apr 20 12:19:03 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA18353 for ; Fri, 20 Apr 2001 12:19:03 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id KAA04341; Fri, 20 Apr 2001 10:12:21 -0700 (PDT) Message-ID: <3AE06F62.B7B4C2D5@best.com> Date: Fri, 20 Apr 2001 10:18:27 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com References: <1444246228.20010401193724@eunet.yu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] [ANNOUNCE] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Home page: http://e4graph.sourceforge.net e4Graph is a C++ library based on Metakit that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. e4Graph support input of arbitrary data graphs from XML using the expat parser, and generates XML output from stored data graphs. Changes: Improved C++ binding; improved efficiency and reliability; improved Tcl binding; eliminated many memory leaks; new Java binding; new support for MacOS X. From konrad@minus1.de Sun Apr 22 07:08:14 2001 Received: from sun2.ruf.uni-freiburg.de (sun2.ruf.uni-freiburg.de [132.230.1.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA04123 for ; Sun, 22 Apr 2001 07:08:13 -0500 Received: from fohnhab.minus1.de (gw.sw-stusie.uni-freiburg.de [132.230.131.225]) by sun2.ruf.uni-freiburg.de (8.9.3+Sun/8.9.1) with ESMTP id OAA00235 for ; Sun, 22 Apr 2001 14:01:57 +0200 (MET DST) Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 14rIc1-0001M6-00 for metakit@equi4.com; Sun, 22 Apr 2001 14:05:09 +0200 Date: Sun, 22 Apr 2001 14:05:09 +0200 From: Konrad Anton To: metakit@equi4.com Message-ID: <20010422140509.A5124@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] (Python) problem with commit-aside for read-write opened files Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all... The commit-aside feature looks very promising, but I cannot make it work the way I think it is supposed to work... I open a storage 'sto' with opening mode 1 (read-write), open another read-write storage, attach the second to the first with sto.aside(sto2). If I make changes to views in sto and call sto.commit(1) (and, for general safety, sto2.commit()), I expect the main storage file to contain the newest changes (even when I reopen it in read-only without commit-aside buffer), at least that's what I conclude from python.html. (See code below) But when I reopen it read-only, my view looks like it did before I opened it in commit-aside mode. Is this correct? ---SNIP SNIP--------------- #!/usr/bin/python import metakit def printView(vw): for row in vw: print row.x, print import os try: os.remove("TEMP.mk") os.remove("TEMP.mka") except OSError: pass print "opening RW, adding 111..." sto = metakit.storage("TEMP.mk", 1) vw = sto.getas("wuff[x:I]") vw.append(x=111) sto.commit() del sto, vw print "reopening commit-aside-RW" sto = metakit.storage("TEMP.mk", 1) sto2 = metakit.storage("TEMP.mka", 1) sto.aside(sto2) vw = sto.view("wuff") print "contents: " printView(vw) print "removing 111, appending 222, full commit" del vw[0] vw.append(x=222) sto.commit(1) # commit as commit can sto2.commit(1) print "contents:" printView(vw) del sto, sto2, vw sto = metakit.storage("TEMP.mk", 0) vw = sto.view("wuff") print "reopening base file, contents should be 222:" printView(vw) ---/SNIP SNIP-------------- output is: opening RW, adding 111... reopening commit-aside-RW contents: 111 removing 111, appending 222, full commit contents: 222 reopening base file, contents should be 222: 111 Any help would be appreciated. Konrad. -- Konrad Anton gpg-keyid: 0x22954D8A gunkl|dadadodo sagt: Es dramaturgisch macht aber sich mir wird und er hat sich her? From piet@cs.uu.nl Sun Apr 22 10:59:07 2001 Received: from pietoost (stol-104-182.uu.studentennet.nl [145.98.104.182]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA05393 for ; Sun, 22 Apr 2001 10:59:06 -0500 Received: from [127.0.0.1] by pietoost (ArGoSoft Mail Server, Version 1.61 (1.6.1.2)); Sun, 22 Apr 2001 17:52:54 +0200 X-Mailer: emacs 20.7.1 (via feedmail 8 I); VM 6.75 under Emacs 20.7.1 Message-ID: <15074.65104.500000.897747@gargle.gargle.HOWL> Date: Sun, 22 Apr 2001 17:52:48 +0200 (W. Europe Daylight Time) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: metakit@equi4.com From: piet@cs.uu.nl (Piet van Oostrum) Subject: [Metakit] exclusive access to metakit database Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I need exclusive access to a metakit file for a short period (to do an update from a web page). It is for a reservation system and I don't want multiple reservations to be made for the same slot. I am using it from Python and I thought if I use db = metakit.storage(dbfile, 1) it would open it for exclusive access (at least no other process should be able to open it in the same way). However I tried this on Windows 95 and Windows 2000 and I can do the open in two seperate processes without error. I can then make two updates and of course the first one gets lost. As far as I can see using commit-extend mode will not solve the problem as it also would cause lost updates. How can this be solved? -- Piet van Oostrum Dillenburg 6 3738 VN Maartensdijk From guenther.fischer@hrz.tu-chemnitz.de Tue Apr 24 22:15:03 2001 Received: from obelix.hrz.tu-chemnitz.de (obelix.hrz.tu-chemnitz.de [134.109.132.55]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA13001 for ; Tue, 24 Apr 2001 22:15:02 -0500 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by obelix.hrz.tu-chemnitz.de with esmtp (Exim 3.22 #4) id 14sLHp-0002A5-00 for metakit@equi4.com; Wed, 25 Apr 2001 11:08:37 +0200 Date: Wed, 25 Apr 2001 11:08:36 +0200 (MEST) From: Guenther Fischer To: In-Reply-To: <15074.65104.500000.897747@gargle.gargle.HOWL> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] tkhtml Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I use the tkhtml.[so|dll] from the mk4doc and the hv.tcl as a start script to include a helping system to my application. It works fine, but for special help, I want to jump into a html-file like filename.htm#about If I use the and the reference it works - if I start with the URL/Filename filename.htm#about it shows only the begin of the document ... Any idea? -- ~Guenther Fischer From sbornuz@yahoo.com Sat Apr 28 14:48:14 2001 Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA28180 for ; Sat, 28 Apr 2001 14:48:13 -0500 Received: from 11dial29.xnet.ro (HELO jumpy) (217.10.213.29) by smtp.mail.vip.sc5.yahoo.com with SMTP; 28 Apr 2001 19:49:15 -0000 X-Apparently-From: From: "Stefan-Mircea BORNUZ" To: "metakit4 list" Date: Sat, 28 Apr 2001 22:49:21 +0300 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Building kviewer with BCBuilder4 Standard Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I would like to use mk4 with BCBuilder4 Standard. While compiling the sample kviewer CBuilder project I get [C++ Error] kview.cpp(33): E2034 Cannot convert 'c4_Storage' to 'c4_View'. [C++ Warning] kview.cpp(45): W8004 'root' is assigned a value that is never used. [C++ Error] kview.cpp(90): E2285 Could not find a match for 'c4_View::operator =(c4_Storage)'. in //------------------------------------------------------------------------ void __fastcall TMainForm::FormActivate(TObject *Sender) { c4_View root = *_storage; <----------------------------HERE !!! TTreeNode* top = StructTree->TopItem; _path.SetSize(0); const char* desc = _storage->Description(); SetupTree(top, desc); StructTree->FullExpand(); StructTree->Selected = StructTree->Items->Item[0]; SetupData(); } //------------------------------------------------------------------------- Any ideas ? Many thanks, Stefan _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From sbornuz@yahoo.com Sun Apr 29 04:10:37 2001 Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id EAA00585 for ; Sun, 29 Apr 2001 04:10:37 -0500 Received: from 76dial200.xnet.ro (HELO jumpy) (213.233.76.200) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Apr 2001 09:11:49 -0000 X-Apparently-From: From: "Stefan-Mircea BORNUZ" To: "metakit4 list" Subject: RE: [Metakit] Building kviewer with BCBuilder4 Standard Date: Sun, 29 Apr 2001 12:12:08 +0300 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ooops! Sorry! I have messed v2.01 headers with v2.3.4-29 libs! S. > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of Stefan-Mircea BORNUZ > Sent: 28 aprilie 2001 22:49 > To: metakit4 list > Subject: [Metakit] Building kviewer with BCBuilder4 Standard > > > Hello, > > I would like to use mk4 with BCBuilder4 Standard. > > While compiling the sample kviewer CBuilder project I get > > [C++ Error] kview.cpp(33): E2034 Cannot convert 'c4_Storage' to 'c4_View'. > [C++ Warning] kview.cpp(45): W8004 'root' is assigned a value > that is never > used. > [C++ Error] kview.cpp(90): E2285 Could not find a match for > 'c4_View::operator =(c4_Storage)'. > > in > > //------------------------------------------------------------------------ > void __fastcall TMainForm::FormActivate(TObject *Sender) > { > c4_View root = *_storage; <----------------------------HERE !!! > TTreeNode* top = StructTree->TopItem; > > _path.SetSize(0); > > const char* desc = _storage->Description(); > SetupTree(top, desc); > > StructTree->FullExpand(); > StructTree->Selected = StructTree->Items->Item[0]; > > SetupData(); > } > //---------------------------------------------------------------- > --------- > > > Any ideas ? > > Many thanks, > > Stefan > > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From fgeiger@datec.at Tue May 1 02:37:00 2001 Received: from datec.at (datec.at [213.208.128.21]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA28380 for ; Tue, 1 May 2001 02:36:59 -0500 Received: from nb0204 [194.183.151.149] by datec.at (SMTPD32-6.04) id A7035220272; Tue, 01 May 2001 09:34:27 +0200 Reply-To: From: "F. GEIGER" To: Date: Tue, 1 May 2001 09:35:56 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] New to Mk4py - Where are the samples? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi everybody, encountered Mk4py and find it kind of pretty cool. What really is a pitty is the lack of samples. The page www.equi4.com/metakit/python.html is too less - at least for me. Does anybody have some samples? In concrete: I wanted to fake a SQL-SELECT-statement using LIKE and tried to do it by using view.map() and function of mine. This returns a RowRef array and that's where I got stuck already. Another one: The page mentioned above tells me to import metakit. All I could import was Mk4py. Whta's about metakit? Where is it? Platform: ActivePython 2.0 on Windows NT4. Best regards Franz GEIGER From gmcm@hypernet.com Tue May 1 18:12:40 2001 Received: from hypernet.com ([204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA23903 for ; Tue, 1 May 2001 18:12:39 -0500 Received: from me (204.176.40.129) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Tue, 1 May 2001 19:12:24 -0400 From: "Gordon McMillan" To: , Date: Tue, 1 May 2001 19:12:23 -0400 Subject: Re: [Metakit] New to Mk4py - Where are the samples? Reply-to: gmcm@hypernet.com Message-ID: <3AEF0A97.22657.7C8A1DFA@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Franz GEIGER wrote: > > encountered Mk4py and find it kind of pretty cool. What really is > a pitty is the lack of samples. The page > www.equi4.com/metakit/python.html is too less - at least for me. > Does anybody have some samples? > > In concrete: I wanted to fake a SQL-SELECT-statement using LIKE > and tried to do it by using view.map() and function of mine. This > returns a RowRef array and that's where I got stuck already. I think you mean filter, not map? (map is for mutating the rows in the view, and returns None, at least in my version.) filter returns a new view. The view has a single column, named _index. It's a number - the index in the original view of a row satisfying your filter function. So it might go something like this: view = db.view('customers') myfilter = lambda(row: row.lname == 'Smi') indices = view.filter(myfilter) for indexrec in indices: # do something with view[indexrec._index] You could also sort on the column you're interested in, then select using lo and hi: copy = view.sort(view.lname) v2 = copy.select({'lname':'Smi'}, {'lname':'Smi\377'}) In this case you won't be able to mutate anything. > Another one: The page mentioned above tells me to import metakit. > All I could import was Mk4py. Whta's about metakit? Where is it? Mk4py is all you need (it's statically linked to the C++ metakit library). > Platform: ActivePython 2.0 on Windows NT4. > > Best regards > Franz GEIGER > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit - Gordon From gmcm@hypernet.com Tue May 1 22:03:16 2001 Received: from hypernet.com ([204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA26322 for ; Tue, 1 May 2001 22:03:16 -0500 Received: from me (204.176.40.74) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Tue, 1 May 2001 23:02:59 -0400 From: "Gordon McMillan" To: , Date: Tue, 1 May 2001 23:02:59 -0400 Subject: Re: [Metakit] New to Mk4py - Where are the samples? Reply-to: gmcm@hypernet.com Message-ID: <3AEF40A3.14240.7D5D3B8A@localhost> Priority: normal In-reply-to: <3AEF0A97.22657.7C8A1DFA@localhost> References: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Oops, hopefully obviously I meant: myfilter = lambda(row: row.lname[:3] == 'Smi') - Gordon > Franz GEIGER wrote: > > > > encountered Mk4py and find it kind of pretty cool. What really > > is a pitty is the lack of samples. The page > > www.equi4.com/metakit/python.html is too less - at least for > > me. Does anybody have some samples? > > > > In concrete: I wanted to fake a SQL-SELECT-statement using LIKE > > and tried to do it by using view.map() and function of mine. > > This returns a RowRef array and that's where I got stuck > > already. > > I think you mean filter, not map? (map is for mutating the rows > in the view, and returns None, at least in my version.) > > filter returns a new view. The view has a single column, named > _index. It's a number - the index in the original view of a row > satisfying your filter function. So it might go something like > this: > > view = db.view('customers') > myfilter = lambda(row: row.lname == 'Smi') > indices = view.filter(myfilter) > for indexrec in indices: > # do something with view[indexrec._index] > > You could also sort on the column you're interested in, then > select using lo and hi: copy = view.sort(view.lname) v2 = > copy.select({'lname':'Smi'}, {'lname':'Smi\377'}) > > In this case you won't be able to mutate anything. > > > > Another one: The page mentioned above tells me to import > > metakit. All I could import was Mk4py. Whta's about metakit? > > Where is it? > > Mk4py is all you need (it's statically linked to the C++ metakit > library). > > > Platform: ActivePython 2.0 on Windows NT4. > > > > Best regards > > Franz GEIGER > > > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > > - Gordon > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From cer@transelec.cl Wed May 2 09:14:59 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA12133 for ; Wed, 2 May 2001 09:14:57 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id KAA22324; Wed, 2 May 2001 10:13:14 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Wed, 2 May 2001 10:19:03 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C03F2E330@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: gmcm@hypernet.com Cc: metakit@equi4.com Subject: RE: [Metakit] New to Mk4py - Where are the samples? Date: Wed, 2 May 2001 10:19:01 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi About the last release, I got Mk4py, but I can't find MkSQL Anybody knows where is it? Cristian Echeverria From cer@transelec.cl Thu May 3 14:33:58 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA28636 for ; Thu, 3 May 2001 14:33:57 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id PAA12475; Thu, 3 May 2001 15:32:03 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Thu, 3 May 2001 15:37:57 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C03FEB6EB@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: gmcm@hypernet.com Cc: metakit@equi4.com Subject: FW: [Metakit] New to Mk4py - Where are the samples? Date: Thu, 3 May 2001 15:37:54 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id OAA28636 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Please, I need an answer Where can I find MkSQL? Is this working with Mk4py 2.3.4-29? > -----Original Message----- > From: Echeverría, Cristian On Behalf Of Echeverria Rabi, Cristian > Rodrigo, TRANSELEC S.A. > Sent: Wednesday, May 02, 2001 10:11 AM > To: 'gmcm@hypernet.com' > Cc: 'metakit@equi4.com' > Subject: RE: [Metakit] New to Mk4py - Where are the samples? > Importance: High > > Hi > About the last release, I got Mk4py, but I can't find MkSQL > Anybody knows where is it? > > Cristian Echeverria From jcw@equi4.com Thu May 3 14:41:45 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA28897 for ; Thu, 3 May 2001 14:41:45 -0500 From: Jean-Claude Wippler To: Subject: Re: FW: [Metakit] New to Mk4py - Where are the samples? Date: Thu, 3 May 2001 12:41:57 -0700 Message-Id: <20010503194157.17172@smtp.telus.net> In-Reply-To: <16D7A2FE3592D41184A200508BDE826C03FEB6EB@SYN201> References: <16D7A2FE3592D41184A200508BDE826C03FEB6EB@SYN201> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. wrote: >Please, I need an answer >Where can I find MkSQL? I'm afraid that the answer may not help you much furter: MkSQL is not finished, nor supported. I've added some comments about this on the MetaKit Wiki: http://www.equi4.com/metakit/wiki.cgi/126.html -jcw From jcw@equi4.com Thu May 3 15:10:40 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29901 for ; Thu, 3 May 2001 15:10:29 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] tkhtml Date: Thu, 3 May 2001 13:04:21 -0700 Message-Id: <20010503200421.29161@smtp.telus.net> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: >I use the tkhtml.[so|dll] from the mk4doc and the hv.tcl as a start script >to include a helping system to my application. > >It works fine, but for special help, I want to jump into a html-file >like filename.htm#about > >If I use the and the reference it works >- if I start with the URL/Filename filename.htm#about it shows only the >begin of the document ... > >Any idea? Yes, I've come across the same issue before. This seems to be an issue which the current hv.tcl or tkhtml.tcl do not address - I suggest contacting the author of tkhtml about this, Richard Hipp . -jcw From jcw@equi4.com Thu May 3 15:10:47 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29909 for ; Thu, 3 May 2001 15:10:40 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] (Python) problem with commit-aside for read-write opened files Date: Thu, 3 May 2001 13:06:50 -0700 Message-Id: <20010503200650.20101@smtp.telus.net> In-Reply-To: <20010422140509.A5124@minus1.de> References: <20010422140509.A5124@minus1.de> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Konrad Anton wrote: >The commit-aside feature looks very promising, but I cannot make it >work the way I think it is supposed to work... Have you checked the examples/*.py files in the source distribution? There should be a commit-aside example in there. Notre that the new commit modes are not fully tested. I do not expect to have them working 100% in time for the next release - there are some tricky issues which are as yet unresolved. -jcw From jcw@equi4.com Thu May 3 15:10:52 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29912 for ; Thu, 3 May 2001 15:10:47 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] exclusive access to metakit database Date: Thu, 3 May 2001 13:10:29 -0700 Message-Id: <20010503201029.32271@smtp.telus.net> In-Reply-To: <15074.65104.500000.897747@gargle.gargle.HOWL> References: <15074.65104.500000.897747@gargle.gargle.HOWL> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Piet van Oostrum wrote: >I need exclusive access to a metakit file for a short period (to do an >update from a web page). It is for a reservation system and I don't >want multiple reservations to be made for the same slot. > >I am using it from Python and I thought if I use > db = metakit.storage(dbfile, 1) >it would open it for exclusive access (at least no other process >should be able to open it in the same way). However I tried this on >Windows 95 and Windows 2000 and I can do the open in two seperate >processes without error. I can then make two updates and of course the >first one gets lost. > >As far as I can see using commit-extend mode will not solve the >problem as it also would cause lost updates. > >How can this be solved? The MK 2.01 release used to include locking, but because it turned out to be too hard (for me anyway, but all help is welcome) to support this reliably across all patforms (and especially with NFS), I decided to drop this instead of leaving an unreliable solution. My suggestion would be to implement a locking scheme *around* MK from Python, using the flock etc primitives it provides. Not locking will lead to trouble. The moment a commit is done, *all* other open versions of the data file are at risk. Well, technically speaking, read-only versions would only fail on a second commit - but it's easier to just consider a write as invalidating all open files. -jcw From jcw@equi4.com Thu May 3 15:14:03 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA30070 for ; Thu, 3 May 2001 15:14:03 -0500 From: Jean-Claude Wippler To: metakit Date: Thu, 3 May 2001 13:14:04 -0700 Message-Id: <20010503201404.16814@smtp.telus.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Mailing list - reply to? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For some reason, after a recent HD replacement and full system re- install, Mailman 2.0.3 seems to want to set the reply address of every post to this list to "metakit-admin@equi4.com" instead of leaving the reply to sender. I have no idea why. If anyone has a suggestion, please let me know. Until it is resolved, please manually adjust the reply-to destination - metakit-admin@equi4.com is definitely not right! -jcw From nathan@vividworks.com Wed May 9 13:01:32 2001 Received: from mail.frontlogic.com (h139-142-46-3.gtcust.grouptelecom.net [139.142.46.3]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id NAA02652 for ; Wed, 9 May 2001 13:01:30 -0500 Message-Id: <200105091801.NAA02652@trixie.triqs.com> Received: (qmail 18204 invoked from network); 9 May 2001 17:51:49 -0000 Received: from unknown (HELO localhost) (139.142.46.131) by mail.frontlogic.com with SMTP; 9 May 2001 17:51:49 -0000 Date: Wed, 9 May 2001 11:59:58 -0500 Content-Type: text/plain; format=flowed; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v388) From: Nathan Heagy To: metakit@equi4.com X-Mailer: Apple Mail (2.388) Content-Transfer-Encoding: 7bit Subject: [Metakit] How to compile Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I can't figure out how to compile Mk4py. According to the very vague and complex build instructions for MetaKit it would seem that the python module gets built automatically: * Linux / gcc 2.95.2 Builds with gcc 2.95.2 work out of the box. The Python extension ends up as Mk4py.so (ignore libmk4py.*), the Tcl extension is called Mk4tcl.so (ignore libmk4tcl.*). Other than this there is *no* mention of how to build the python module. In addition to updating the documentation to be clear, can anyone tell me how to do this? thanks, nathan From lvirden@cas.org Thu May 17 07:11:58 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA21371 for ; Thu, 17 May 2001 07:11:57 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id IAA26742 for ; Thu, 17 May 2001 08:11:16 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f4HCBEI03901; Thu, 17 May 2001 08:11:14 -0400 (EDT) Date: Thu, 17 May 2001 08:11:14 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010517081114.AAB3879@cas.org> To: Subject: [Metakit] Help sought getting tclkit/metakit/wiki to work as CGI Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Platform: SPARC Solaris Web server: Netscape Enterprise I have the wikit.cgi file set up to point to the tclkit and the wikit.tkd. I can run the cgi script from the command line and it works. When I try to run it as a CGI , the server reports this error: [17/May/2001:07:57:20] failure (13910): for host 134.243.55.72 trying to GET /~lwv26/cgi-bin/wikit.cgi, cgieng_scan_headers reports: the CGI program /home/lwv26/castest_html/cgi-bin/wikit.cgi did not produce a valid header (name without value: got line "file open failed") Anyone have any ideas? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From lvirden@cas.org Thu May 17 07:34:02 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA21899 for ; Thu, 17 May 2001 07:34:02 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id IAA03794 for ; Thu, 17 May 2001 08:33:18 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f4HCXIm04211; Thu, 17 May 2001 08:33:18 -0400 (EDT) Date: Thu, 17 May 2001 08:33:18 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010517083318.AAB4202@cas.org> Subject: Re: [Metakit] Help sought getting tclkit/metakit/wiki to work as CGI In-Reply-To: <20010517081114.AAB3879@cas.org> of Thu, 17 May 2001 08:11:14 -0400 (EDT) To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The good news is that I found what my problem was - there was a permission problem with tclkit causing that error msg. Sorry for the bother. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From rmenegon@bigpond.net.au Thu May 17 03:55:47 2001 Received: from mailin2.email.bigpond.com (juicer14.bigpond.com [139.134.6.23]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA13655 for ; Thu, 17 May 2001 03:55:46 -0500 Received: from unspecified.host ([144.135.24.78]) by mailin2.email.bigpond.com (Netscape Messaging Server 4.15) with SMTP id GDH2DA00.D8E for ; Thu, 17 May 2001 19:00:46 +1000 Received: from CPE-144-132-187-24.nsw.bigpond.net.au ([144.132.187.24]) by bwmam04.mailsvc.email.bigpond.com(MailRouter V2.9d 8329/575000); 17 May 2001 18:59:13 Received: from 192.168.11.13 ([192.168.11.13]) by 192.168.11.1 (WinRoute Pro 4.1) with SMTP; Thu, 17 May 2001 18:55:52 +1000 Received: by localhost with Microsoft MAPI; Thu, 17 May 2001 19:00:06 +1000 Message-ID: <01C0DF03.964034A0.rmenegon@bigpond.net.au> From: Rob Menegon Reply-To: "rmenegon@bigpond.net.au" To: "'metakit@equi4.com'" Date: Thu, 17 May 2001 19:00:05 +1000 Organization: MenMac X-Mailer: Microsoft Internet E-mail/MAPI - 8.0.0.4211 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] Getting started Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Some background I have recently embarked on a project to store and analyse statistics associated with particulat sport that my kids are involved in. The basic structure goes something like this: Meeting Code / unique identifier Name Event Code / unique identifier Name Competitor Code / unique identifier Name and so on Competitor Code / unique identifier Name and so on Event Code / unique identifier Name Competitor Code / unique identifier Name and so on Competitor Code / unique identifier Name and so on In summary there can be multiple Meetings each containing some local information as well as many events which in turn have local information and contain many competitor information bits ... much like a directory listing in structure. Help I have been beating myself up on how I should store this so that I can retrieve the information easily and quickly. In my investigations I came across your product. I have described my structure outlined above by the following: c4_View vMeeting = storage.GetAs("meeting[mcode:S,mname:S," "event[ecode:S,rname:S," "competitor[ccode:S,cname:S]]]"); My question is relating to how I update (access) the event and competitor information and retrieve this information later using this structure. pMeetingCode (row) = "AAAA"; pMeetingName (row) = "FirstMeet"; OR perhaps I am way of the mark and need to structure it differently i.e. 3 views, Meetings, Events, Competitors Is someone able to get me started? Rob _______________________________________ Rob Menegon 349 Somerville Road Hornsby Heights NSW 2077 Sydney, Australia (Mobile) +61 (0)416 271 348 ( Voice) +61 (0)2 9477 1348 ( Fax) +61 (0)2 8570 2031 rmenegon@bigpond.net.au _______________________________________ This email contains information intended for the addressee only. This information is confidential and may be privileged. If you are not the intended recipient, you must not read, use, copy or distribute this email or its attachments. If you have received this in error, please notify me immediately by return email or telephone, and delete and destroy this email. Thank you. _______________________________________ From g.lancaster@mbf.com.au Thu May 17 18:19:36 2001 Received: from mbfdns ([203.101.28.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA03371 for ; Thu, 17 May 2001 18:19:34 -0500 Received: from MBF-INT1-Message_Server by mbfdns with Novell_GroupWise; Fri, 18 May 2001 09:05:41 +1000 Message-Id: X-Mailer: Novell GroupWise 5.5.2 Date: Fri, 18 May 2001 09:16:25 +1000 From: "Garth Lancaster" To: , Subject: Re: [Metakit] Getting started Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline X-Guinevere: 1.0.13 ; Medical Benefits Fun Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id SAA03371 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Rob, I think you would be better off using a recursive 'graph-node' structure to do this (ie represent your data) - it makes traversal etc easier - Jacob Levy has doen this nicely in a package called e4graph, at http://e4graph.sourceforge.net/ - this sits on top of MetaKit, and says good things about the extensibility/usefulness of MetaKit itself (I have no commercial interest in anything I describe here, in case you're wondering - I just come across a lot of usefull pieces in my 'travels', and like to share knowledge) If you wanted to make this 'transportable', you could :- traverse your graph structure and build an XML data-set, or unravel the graph to a flat file, or maybe read the MetaKit representation of the Graph directly ... I can send you the URL for a 'lightweight' XML package (not free, but cheap), that I found, if you like, if you want/need to do the first option (of course, there are other ways to skin a cat, maybe someone else out there has better ideas) have fun, regards, Garth Garth Lancaster MBF InfoTech (Infrastructure) Integration Engineer g.lancaster@mbf.com.au Ph : +61 2 9323 9534 Fax : +61 2 9267 7359 From Slick@battleop.net Thu May 17 23:29:37 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA05497 for ; Thu, 17 May 2001 23:29:36 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Thu, 17 May 2001 21:35:42 -0700 Message-ID: <004901c0df53$7b6b8ee0$0200a8c0@pacbell.net> From: "Slick" To: Date: Thu, 17 May 2001 21:32:00 -0700 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0046_01C0DF18.CEEB2520" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Getting started with the latest revision v2.3.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0046_01C0DF18.CEEB2520 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I have been reading the web site with some interest. However, some = links go to older versions for example v2.0. Is there 1 zip file for Windows that contains all needed files? I need = instructions on how to set it up and where to get the latest header = files etc so that I can experiment with it. Any links/instructions are appreciated. I saw the directory with the = latest *.lib and *.dll files but where are the related header files? Thanks (I'll continue to dig around the web site also) ------=_NextPart_000_0046_01C0DF18.CEEB2520 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I have been reading the web site with = some=20 interest.  However, some links go to older versions for example=20 v2.0.
 
Is there 1 zip file for Windows that = contains all=20 needed files?  I need instructions on how to set it up and where to = get the=20 latest header files etc so that I can experiment with it.
 
Any links/instructions are = appreciated.  I saw=20 the directory with the latest *.lib and *.dll files but where are the = related=20 header files?
 
Thanks (I'll continue to dig around the = web site=20 also)
------=_NextPart_000_0046_01C0DF18.CEEB2520-- From jcw@equi4.com Fri May 18 00:37:01 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id AAA05944 for ; Fri, 18 May 2001 00:37:00 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Getting started with the latest revision v2.3.4 Date: Thu, 17 May 2001 22:36:50 -0700 Message-Id: <20010518053650.27829@smtp.telus.net> In-Reply-To: <004901c0df53$7b6b8ee0$0200a8c0@pacbell.net> References: <004901c0df53$7b6b8ee0$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: >Is there 1 zip file for Windows that contains all needed files? I need >instructions on how to set it up and where to get the latest header files >etc so that I can experiment with it. > >Any links/instructions are appreciated. I saw the directory with the >latest *.lib and *.dll files but where are the related header files? The sources and build files (also for MSVC6, Windows) are in: http://www.equi4.com/pub/mk/metakit.tar.gz This currently points to: http://www.equi4.com/pub/mk/metakit-2.3.4-29.tar.gz The public C++ headers are all in the include/ subdirectory. For Python or Tcl, you need no headers - just mk4py.dll/mk4tcl.dll ... -jcw From Slick@battleop.net Fri May 18 20:20:46 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id UAA06208 for ; Fri, 18 May 2001 20:20:44 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Fri, 18 May 2001 18:26:46 -0700 Message-ID: <001301c0e002$43e34c60$0200a8c0@pacbell.net> From: "Slick" To: Date: Fri, 18 May 2001 18:23:09 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I put just this one line of code in: c4_StringProp pName ("name"); I have the following includes: #include "mk4.h" #include "mk4str.h" I get the following errors: error LNK2001: unresolved external symbol "public: __thiscall c4_Property::c4_Property(char,char const *)" (??0c4_Property@@QAE@DPBD@Z) error LNK2001: unresolved external symbol "public: __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) I built all the metakit builds that I could (static, debug etc). I have the relevant paths set up to the metakit directories. What could be happening here? Thanks From jcw@equi4.com Fri May 18 20:36:50 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA06337; Fri, 18 May 2001 20:36:47 -0500 From: Jean-Claude Wippler To: , Subject: Re: [Metakit] Linking error Date: Fri, 18 May 2001 18:36:40 -0700 Message-Id: <20010519013640.17413@smtp.telus.net> In-Reply-To: <001301c0e002$43e34c60$0200a8c0@pacbell.net> References: <001301c0e002$43e34c60$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: >I put just this one line of code in: > >c4_StringProp pName ("name"); > >I have the following includes: > >#include "mk4.h" >#include "mk4str.h" > >I get the following errors: > >error LNK2001: unresolved external symbol "public: __thiscall >c4_Property::c4_Property(char,char const *)" (??0c4_Property@@QAE@DPBD@Z) >error LNK2001: unresolved external symbol "public: __thiscall >c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) > >I built all the metakit builds that I could (static, debug etc). > >I have the relevant paths set up to the metakit directories. > >What could be happening here? Try adding the define "q4_INLINE" (either in the compiler, or as "#define a4_INLINE 1" *before* the mk4 includes). Does this fix things? If so, I'll mention it on the Metakit wiki. -jcw From jerry@usd1.com Fri May 18 20:59:24 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA06499 for ; Fri, 18 May 2001 20:59:23 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f4J1wji24114 for ; Fri, 18 May 2001 18:58:46 -0700 (PDT) Date: Fri, 18 May 2001 18:40:17 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae <1jerry@usd1.com> Organization: usd1 X-Priority: 3 (Normal) Message-ID: <6917523888.20010518184017@usd1.com> To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Python 2.1 & Mk4py.dll Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello metakit, Just installed Python 2.1. The version of Mk4py.dll I have been using is dated 11/23/00 and is 221k. Python didn't like it loading python20.dll, so I downloaded the newest Mk4py today. (Oddly, it is MUCH smaller at ~70k!). It still does not work. Gives the following message: >>> import Mk4py Fatal Python error: Interpreter not initialized (version mismatch?) abnormal program termination I have tried it with the Mk4py.dll in the same directory, just to make sure I'm using the correct one. Any help? -- Best regards, 1Jerry From jcw@equi4.com Fri May 18 21:19:56 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA06644 for ; Fri, 18 May 2001 21:19:55 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Python 2.1 & Mk4py.dll Date: Fri, 18 May 2001 19:19:43 -0700 Message-Id: <20010519021943.6085@smtp.telus.net> In-Reply-To: <6917523888.20010518184017@usd1.com> References: <6917523888.20010518184017@usd1.com> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry McRae wrote: >Just installed Python 2.1. The version of Mk4py.dll I have been using >is dated 11/23/00 and is 221k. Python didn't like it loading >python20.dll, so I downloaded the newest Mk4py today. (Oddly, it is >MUCH smaller at ~70k!). It still does not work. Gives the following >message: > >>>> import Mk4py >Fatal Python error: Interpreter not initialized (version mismatch?) > >abnormal program termination > >I have tried it with the Mk4py.dll in the same directory, just to make >sure I'm using the correct one. The "newest" was also linked against 2.0 AFAIK. If anyone has newer releases, I'd be happy to accept 2.1 builds for Windows, Linux, etc - and put them next to the others on the web. (Tcl has "stubs", it never needs a recompile, mumble, mumble...) The 70 Kb size is due to UPX compression, see http://wildsau.idv.uni-linz.ac.at/mfx/upx.html -jcw From jyl@best.com Fri May 18 23:00:15 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA07171 for ; Fri, 18 May 2001 23:00:15 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id UAA11380; Fri, 18 May 2001 20:58:10 -0700 (PDT) Message-ID: <3B05F0D1.5B3D35B0@best.com> Date: Fri, 18 May 2001 21:04:33 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: "rmenegon@bigpond.net.au" CC: "'metakit@equi4.com'" Subject: Re: [Metakit] Getting started References: <01C0DF03.964034A0.rmenegon@bigpond.net.au> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This looks like an ideal application for e4Graph. Check out http://e4graph.sourceforge.net/ The e4Graph software uses Metakit as a storage mechanism. --JYL Rob Menegon wrote: > Some background > > I have recently embarked on a project to store and analyse statistics > associated with particulat sport that my kids are involved in. The basic > structure goes something like this: > > Meeting > Code / unique identifier > Name > Event > Code / unique identifier > Name > Competitor > Code / unique identifier > Name > and so on > Competitor > Code / unique identifier > Name > and so on > Event > Code / unique identifier > Name > Competitor > Code / unique identifier > Name > and so on > Competitor > Code / unique identifier > Name > and so on > > In summary there can be multiple Meetings each containing some local > information as well as many events which in turn have local information and > contain many competitor information bits ... much like a directory listing > in structure. > > Help > > I have been beating myself up on how I should store this so that I can > retrieve the information easily and quickly. In my investigations I came > across your product. I have described my structure outlined above by the > following: > > c4_View vMeeting = storage.GetAs("meeting[mcode:S,mname:S," > "event[ecode:S,rname:S," > "competitor[ccode:S,cname:S]]]"); > > My question is relating to how I update (access) the event and competitor > information and retrieve this information later using this structure. > > pMeetingCode (row) = "AAAA"; > pMeetingName (row) = "FirstMeet"; > > OR perhaps I am way of the mark and need to structure it differently i.e. 3 > views, Meetings, Events, Competitors > > Is someone able to get me started? > > Rob > _______________________________________ > Rob Menegon > > 349 Somerville Road > Hornsby Heights > NSW 2077 > Sydney, Australia > (Mobile) +61 (0)416 271 348 > ( Voice) +61 (0)2 9477 1348 > ( Fax) +61 (0)2 8570 2031 > rmenegon@bigpond.net.au > _______________________________________ > This email contains information intended for the addressee only. This > information is confidential and may be privileged. If you are not the > intended recipient, you must not read, use, copy or distribute this email > or > its attachments. If you have received this in error, please notify me > immediately by return email or telephone, and delete and destroy this > email. > Thank you. > _______________________________________ > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jyl@best.com Fri May 18 23:02:23 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA07210 for ; Fri, 18 May 2001 23:02:20 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id VAA16422; Fri, 18 May 2001 21:00:27 -0700 (PDT) Message-ID: <3B05F160.D51B1427@best.com> Date: Fri, 18 May 2001 21:06:56 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Garth Lancaster CC: rmenegon@bigpond.net.au, metakit@equi4.com Subject: Re: [Metakit] Getting started References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks Garth for the mention! I should add that e4Graph already supports XML input/output so your problem is nearly completely solved. You just have to design the tree structure you want and serialize/deserialize it and you should be close to done. --JYL Garth Lancaster wrote: > Rob, I think you would be better off using a recursive 'graph-node' structure to do this (ie represent your data) - it makes traversal etc easier - Jacob Levy has doen this nicely in a package called e4graph, at http://e4graph.sourceforge.net/ - this sits on top of MetaKit, and says good things about the extensibility/usefulness of MetaKit itself > > (I have no commercial interest in anything I describe here, in case you're wondering - I just come across a lot of usefull pieces in my 'travels', and like to share knowledge) > > If you wanted to make this 'transportable', you could :- traverse your graph structure and build an XML data-set, or unravel the graph to a flat file, or maybe read the MetaKit representation of the Graph directly ... I can send you the URL for a 'lightweight' XML package (not free, but cheap), that I found, if you like, if you want/need to do the first option > > (of course, there are other ways to skin a cat, maybe someone else out there has better ideas) > > have fun, regards, Garth > > Garth Lancaster > MBF InfoTech (Infrastructure) Integration Engineer > g.lancaster@mbf.com.au > Ph : +61 2 9323 9534 > Fax : +61 2 9267 7359 > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From Slick@battleop.net Sat May 19 01:04:59 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA08687 for ; Sat, 19 May 2001 01:04:57 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Fri, 18 May 2001 23:10:56 -0700 Message-ID: <000b01c0e029$f5f18120$0200a8c0@pacbell.net> From: "Slick" To: Date: Fri, 18 May 2001 23:07:18 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I tried adding the following immediately before the includes in my .cpp file: #define q4_inline 1 also tried #define a4_inline 1 since you mentioned both (not sure which is correct) When I choose build|compile I don't get any errors. It is when they are linked. Normally I just choose build (F7) when I'm working since no reason for 2 steps. I am not familiar with the #define q4_inline 1 command. Is it possible that I don't have all of the necessary metakit .dll or .libs built? I don't know what the minimum config is under VC6 in a MFC app. Unfortunate that I can't get it to work after only 1 line of code :) In the directory config I have the following settings: executable files \builds include files \include library files \builds source files \src From jcw@equi4.com Sat May 19 01:48:35 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA09604 for ; Sat, 19 May 2001 01:48:35 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Linking error Date: Fri, 18 May 2001 23:48:23 -0700 Message-Id: <20010519064823.9621@smtp.telus.net> In-Reply-To: <000b01c0e029$f5f18120$0200a8c0@pacbell.net> References: <000b01c0e029$f5f18120$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: >I tried adding the following immediately before the includes in my .cpp >file: > >#define q4_inline 1 #define q4_INLINE 1 (case matters in C++) >#define a4_inline 1 >since you mentioned both (not sure which is correct) Oops, sorry. It's "q". -jcw From Slick@battleop.net Sat May 19 12:16:29 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA22952 for ; Sat, 19 May 2001 12:16:27 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Sat, 19 May 2001 10:22:27 -0700 Message-ID: <001901c0e087$c5c36e60$0200a8c0@pacbell.net> From: "Slick" To: Date: Sat, 19 May 2001 10:18:50 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Still does not work after doing the following: #define q4_INLINE 1 //mine #include "mk4.h" //mine #include "mk4str.h" //mine I do not know what the inline means and was supposed to accomplish. My guess is that I am missing a file - could it be that I get these errors because on of the dll's or lib's didn't build when I set up metakit? fyi I have built mk_test_d.exe and it appears to work by running through all of the tests ok. They scroll quickly but it looks like they all work. From jyl@best.com Sat May 19 12:27:33 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA23054 for ; Sat, 19 May 2001 12:27:33 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id KAA13638; Sat, 19 May 2001 10:25:25 -0700 (PDT) Message-ID: <3B06AE0A.37F39BC0@best.com> Date: Sat, 19 May 2001 10:31:54 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Slick CC: metakit@equi4.com Subject: Re: [Metakit] Linking error References: <001901c0e087$c5c36e60$0200a8c0@pacbell.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I could take a look to try and reproduce your problem if you could send all needed files. Zip format preferred. --JYL Slick wrote: > Still does not work after doing the following: > > #define q4_INLINE 1 //mine > #include "mk4.h" //mine > #include "mk4str.h" //mine > > I do not know what the inline means and was supposed to accomplish. > > My guess is that I am missing a file - could it be that I get these errors > because on of the dll's or lib's didn't build when I set up metakit? > > fyi I have built mk_test_d.exe and it appears to work by running through all > of the tests ok. They scroll quickly but it looks like they all work. > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From Slick@battleop.net Mon May 21 21:37:54 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA15384 for ; Mon, 21 May 2001 21:37:52 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Mon, 21 May 2001 19:43:40 -0700 Message-ID: <000b01c0e268$84704400$0200a8c0@pacbell.net> From: "Slick" To: Date: Mon, 21 May 2001 19:40:08 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking error Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I still can't get metakit to work. The same error even in a new MFC app: Linking... Demo1Doc.obj : error LNK2001: unresolved external symbol "public: __thiscall c4_Property::c4_Property(char,char const *)" (??0c4_Property@@QAE@DPBD@Z) Demo1Doc.obj : error LNK2001: unresolved external symbol "public: __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) Debug/Demo1.exe : fatal error LNK1120: 2 unresolved externals Error executing link.exe. Demo1.exe - 3 error(s), 0 warning(s) Any further ideas? What can I check for? From j.p.fletcher@aston.ac.uk Tue May 22 09:37:12 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA01147; Tue, 22 May 2001 09:37:10 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.16 #1) id 152DHY-0001Gh-00; Tue, 22 May 2001 15:37:08 +0100 From: "John Fletcher" To: Date: Tue, 22 May 2001 15:34:57 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT CC: Jean-Claude Wippler Priority: normal In-reply-to: <19341204142430.7498@216.110.36.111> References: <20010109183103.7EBCA24D57@mail> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Subject: [Metakit] Re: [MK-Core] Metakit for Mingw32 (Dev-C++) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler To: metakit-core Subject: Re: [MK-Core] Metakit for Mingw32 (Dev-C++) Date sent: Tue, 9 Jan 2001 12:52:46 -0800 > >From: Darel Finkbeiner > [...] > >I've compiled the library with the Mingw32 compiler that comes with Dev-C++ > >(gcc-2.95.2 on windows). I used what was in the 2.3.3-28 and it seems to > >work ok so far. > > > >How should I make this available for people to use? Or who should I send > >it to to let the project put it up on their site? > > > >I also can include written instructions for integrated with the Dev-C++ IDE. > > > >Darel Finkbeiner > > Hello Darel, > > I'll be happy to add such a download to the http://www.equi4.com/mk/ > tree, binary as well as source+instructions if you like. Alternately, > you can put it somewhere else and I could list it on the http:// > www.equi4.com/metakit/ homepage. Whichever is most convenient for you. > > Thanks for letting us know... > > -jcw > > _______________________________________________ > metakit-core@equi4.com - http://www.equi4.com/metakit/ > http://lists.sourceforge.net/mailman/listinfo/metakit-core Hello Did the mingw32 version get loaded up somewhere? If so, where please. John Fletcher > ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From cer@transelec.cl Tue May 22 20:12:40 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA17852; Tue, 22 May 2001 20:12:39 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id VAA14865; Tue, 22 May 2001 21:08:02 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Tue, 22 May 2001 21:15:19 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: Jean-Claude Wippler , metakit@equi4.com Subject: RE: [Metakit] Python 2.1 & Mk4py.dll Date: Tue, 22 May 2001 08:53:32 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi >The 70 Kb size is due to UPX compression, see > http://wildsau.idv.uni-linz.ac.at/mfx/upx.html I'am Mk4py user and I think metakit is great, but I have a problem with exe or dll compressors like UPX. Jordan Russell, the author of Inno Setup Installer, write the next about exe compressors, and I like to know if you agree or not: (http://www.jrsoftware.org/striprlc.htm) Why not use an EXE compressor? Some have asked why I made StripReloc when there are EXE compression programs such as ASPack and UPX that will trim more bytes off of executables than StripReloc ever could. The reason is there are downsides to using EXE compressors. Most notably: * Upon startup of a compressed EXE/DLL, all of the code is decompressed from the disk image into memory in one pass, which can cause disk thrashing if the system is low on memory and is forced to access the swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates memory for code pages on demand (i.e. when they are executed). * Multiple instances of a compressed EXE/DLL create multiple instances of the code in memory. (So if your EXE is 1 MB and the user starts 5 instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the code is only stored in memory once and shared between instances. * Some older virus scanners flag compressed EXE/DLLs as being virus-infected. (This is this reason I stopped compressing Inno Setup's EXEs.) From Slick@battleop.net Wed May 23 00:23:25 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id AAA19415 for ; Wed, 23 May 2001 00:23:24 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Tue, 22 May 2001 22:29:11 -0700 Message-ID: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> From: "Slick" To: Date: Tue, 22 May 2001 22:25:40 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Building DLL and LIB files Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm still trying to work out why I can't get Metakit to work. In some of the documentation I read : _______ The MS Visual C++ 6.0 project is "win/msvc60/mksrc.dsw", with subprojects for the C++ demo (mkdemo), building dll's (mkdll), static libs (mklib), regression tests (mktest), as well as Tcl (mktcl) and Python (mkpython). It has been set up to place all intermediate files and final results in the "builds/" subdirectory, even though you'll launch it from "win/". To build with STL containers and strings under MSVC, define "q4_STD". To build with MFC containers and strings under MSVC, define "q4_MFC". _______ If I am using MFC where do I put the define command before building the dll and libs? I tried in the config.h file at the top with various combinations such as : #define q4_MFC but i get errors like this: C:\Metakit\metakit-2.3.4-29\src\header.h(45) : fatal error C1017: invalid integer constant expression custom.cpp C:\Metakit\metakit-2.3.4-29\src\header.h(45) : fatal error C1017: invalid integer constant expression derived.cpp Does anyone have v2.3.4 working well and could simply explain how they got the source to build dll's and lib's that actually work? Thanks for the help. I may have to abandon trying to get it to work since I am not a guru and can't work this out. From Slick@battleop.net Wed May 23 01:48:52 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA20017 for ; Wed, 23 May 2001 01:48:51 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Tue, 22 May 2001 23:54:38 -0700 Message-ID: <004301c0e354$bdea97c0$0200a8c0@pacbell.net> From: "Slick" To: Date: Tue, 22 May 2001 23:51:06 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] For others potentially having problems v2.3.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I don't know if I am the only one who has had problems getting Metakit up and running. Jean_Claude I can build and run both mkdemo and mktest. Mkdemo says the country of paco pena is spain and mktest runs through a whole series of tests that appear to work with the exception of e05 it says failed a(s == "hi!hello"). Most of them scroll too quickly but i think they worked. Then if I create a new MFC sdi app to keep it simple and add : c4_StringProp pNew ("Hey"); plus #include "mk4.h" //mine I get errors: --------------------Configuration: mktest2 - Win32 Debug-------------------- Compiling resources... Compiling... StdAfx.cpp Compiling... mktest2.cpp MainFrm.cpp mktest2Doc.cpp mktest2View.cpp Generating Code... Linking... mktest2View.obj : error LNK2001: unresolved external symbol "public: __thiscall c4_Property::c4_Property(char,char const *)" (??0c4_Property@@QAE@DPBD@Z) mktest2View.obj : error LNK2001: unresolved external symbol "public: __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) Debug/mktest2.exe : fatal error LNK1120: 2 unresolved externals Error executing link.exe. mktest2.exe - 3 error(s), 0 warning(s) Oh well... From spahievi@vega.bg Wed May 23 02:17:10 2001 Received: from moria.vega.bg (moria.vega.bg [217.197.128.1]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA20430 for ; Wed, 23 May 2001 02:17:08 -0500 Received: from b1-dial-pool172.vega.bg (b1-dial-pool172.vega.bg [217.197.129.172]) by moria.vega.bg (8.9.1a/8.9.1) with ESMTP id JAA01459 for ; Wed, 23 May 2001 09:17:24 +0300 Date: Wed, 23 May 2001 09:45:57 +0300 From: Niki Spahiev X-Mailer: The Bat! (v1.52) UNREG / FE6CDE745D9DC872 Reply-To: spahievi X-Priority: 3 (Normal) Message-ID: <482270334.20010523094557@vega.bg> To: metakit@equi4.com Subject: Re: [Metakit] Building DLL and LIB files In-Reply-To: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> References: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Slick, Wednesday, May 23, 2001, 8:25:40 AM, you wrote: [...] S> #define q4_MFC #define q4_MFC 1 -- regards, Niki Spahiev From spahievi@vega.bg Wed May 23 02:17:13 2001 Received: from moria.vega.bg (moria.vega.bg [217.197.128.1]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA20434 for ; Wed, 23 May 2001 02:17:11 -0500 Received: from b1-dial-pool172.vega.bg (b1-dial-pool172.vega.bg [217.197.129.172]) by moria.vega.bg (8.9.1a/8.9.1) with ESMTP id JAA01486 for ; Wed, 23 May 2001 09:17:29 +0300 Date: Wed, 23 May 2001 09:46:14 +0300 From: Niki Spahiev X-Mailer: The Bat! (v1.52) UNREG / FE6CDE745D9DC872 Reply-To: spahievi X-Priority: 3 (Normal) Message-ID: <772287551.20010523094614@vega.bg> To: metakit@equi4.com Subject: Re[2]: [Metakit] Python 2.1 & Mk4py.dll In-Reply-To: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> References: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Echeverria, Tuesday, May 22, 2001, 3:53:32 PM, you wrote: ERCRTSA> Hi >>The 70 Kb size is due to UPX compression, see >> http://wildsau.idv.uni-linz.ac.at/mfx/upx.html ERCRTSA> I'am Mk4py user and I think metakit is great, but I have a problem with exe ERCRTSA> or dll compressors like UPX. ERCRTSA> Jordan Russell, the author of Inno Setup Installer, write the next about exe ERCRTSA> compressors, and I like to know if you agree or not: ERCRTSA> (http://www.jrsoftware.org/striprlc.htm) ERCRTSA> Why not use an EXE compressor? ERCRTSA> Some have asked why I made StripReloc when there are EXE compression ERCRTSA> programs such as ASPack and UPX that will trim more bytes off of executables ERCRTSA> than StripReloc ever could. ERCRTSA> The reason is there are downsides to using EXE compressors. Most notably: ERCRTSA> * Upon startup of a compressed EXE/DLL, all of the code is ERCRTSA> decompressed from the disk image into memory in one pass, which can cause ERCRTSA> disk thrashing if the system is low on memory and is forced to access the ERCRTSA> swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates memory ERCRTSA> for code pages on demand (i.e. when they are executed). ERCRTSA> * Multiple instances of a compressed EXE/DLL create multiple instances ERCRTSA> of the code in memory. (So if your EXE is 1 MB and the user starts 5 ERCRTSA> instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the ERCRTSA> code is only stored in memory once and shared between instances. ERCRTSA> * Some older virus scanners flag compressed EXE/DLLs as being ERCRTSA> virus-infected. (This is this reason I stopped compressing Inno Setup's ERCRTSA> EXEs.) A agree with this. Also it's impossible to extract dependency info from compressed DLLs. -- regards, Niki Spahiev From cer@transelec.cl Wed May 23 07:55:51 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA27756 for ; Wed, 23 May 2001 07:55:50 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id IAA26237; Wed, 23 May 2001 08:50:29 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Wed, 23 May 2001 08:57:47 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C045C9A11@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: spahievi Cc: metakit@equi4.com Subject: RE: Re[2]: [Metakit] Python 2.1 & Mk4py.dll Date: Wed, 23 May 2001 08:57:25 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It must be said that the same UPX compressor can be used to uncompress exes and dlls compressed with it. (This is the main reason I have it on my PC) Cristian Echeverria > -----Original Message----- > From: Niki Spahiev [SMTP:spahievi@vega.bg] > Sent: Wednesday, May 23, 2001 2:46 AM > To: metakit@equi4.com > Subject: Re[2]: [Metakit] Python 2.1 & Mk4py.dll > > Hello Echeverria, > > Tuesday, May 22, 2001, 3:53:32 PM, you wrote: > > ERCRTSA> Hi > > >>The 70 Kb size is due to UPX compression, see > >> http://wildsau.idv.uni-linz.ac.at/mfx/upx.html > > ERCRTSA> I'am Mk4py user and I think metakit is great, but I have a > problem with exe > ERCRTSA> or dll compressors like UPX. > ERCRTSA> Jordan Russell, the author of Inno Setup Installer, write the > next about exe > ERCRTSA> compressors, and I like to know if you agree or not: > ERCRTSA> (http://www.jrsoftware.org/striprlc.htm) > > > ERCRTSA> Why not use an EXE compressor? > ERCRTSA> Some have asked why I made StripReloc when there are EXE > compression > ERCRTSA> programs such as ASPack and UPX that will trim more bytes off of > executables > ERCRTSA> than StripReloc ever could. > ERCRTSA> The reason is there are downsides to using EXE compressors. Most > notably: > ERCRTSA> * Upon startup of a compressed EXE/DLL, all of the code is > ERCRTSA> decompressed from the disk image into memory in one pass, which > can cause > ERCRTSA> disk thrashing if the system is low on memory and is forced to > access the > ERCRTSA> swap file. In contrast, with uncompressed EXE/DLLs, the OS > allocates memory > ERCRTSA> for code pages on demand (i.e. when they are executed). > ERCRTSA> * Multiple instances of a compressed EXE/DLL create > multiple instances > ERCRTSA> of the code in memory. (So if your EXE is 1 MB and the user > starts 5 > ERCRTSA> instances of it, that adds up to 5+ MB.) With uncompressed > EXE/DLLs, the > ERCRTSA> code is only stored in memory once and shared between instances. > ERCRTSA> * Some older virus scanners flag compressed EXE/DLLs as > being > ERCRTSA> virus-infected. (This is this reason I stopped compressing Inno > Setup's > ERCRTSA> EXEs.) > > A agree with this. Also it's impossible to extract dependency info from > compressed DLLs. > > -- > regards, > Niki Spahiev From msackett@yin.interaccess.com Wed May 23 09:39:24 2001 Received: from postal.interaccess.com (from.interaccess.com [207.208.131.20]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA31296 for ; Wed, 23 May 2001 09:39:24 -0500 Received: from yin.interaccess.com (yin.interaccess.com [207.208.45.69]) by postal.interaccess.com (8.10.2/8.10.2) with SMTP id f4NEcFm13404 for ; Wed, 23 May 2001 09:38:15 -0500 (CDT) Received: (qmail 11193 invoked by uid 28828); 23 May 2001 14:38:03 -0000 Date: Wed, 23 May 2001 09:38:03 -0500 (CDT) From: Mike Sackett To: Slick cc: metakit@equi4.com Subject: Re: [Metakit] For others potentially having problems v2.3.4 In-Reply-To: <004301c0e354$bdea97c0$0200a8c0@pacbell.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I don't know enough about Metakit to be able to comment on the specific issue you're running into, but I do have a general suggestion: Since you appear to be running Visual C++, you can use the dumpbin utility to get a dump of the symbols defined by a particular module or library. In the case of the error you describe, it's apparent that mktest2View.obj is attempting to import variations of the c4_Property constructor which are not being provided by any module available to the linker. This often means, as I'm sure you're aware, that the build options supplied when the library was built differ in some way from those used to build the referencing module. At this point, it's often helpful to use your compiler's command line options to get a listing of the preprocessed source. For example, if you use the /E or /P options when compiling mktest2View, you can see how *it* is interpreting Metakit's headers. You can then look at the headers themselves and try to figure out the variations and what affects them. It's painful and slow, but in the end you have all the tools you need on your machine to figure out what's going on. I think one of the lessons most people learn after a while is to be very careful about build flags, preprocessor flags, etc., when combining libraries. I tend to look at this kind of debugging as being somewhat of a fixed cost that I have to pay every time I add a new, complex library to an existing build. ..Mike On Tue, 22 May 2001, Slick wrote: > I don't know if I am the only one who has had problems getting Metakit up > and running. > > Jean_Claude I can build and run both mkdemo and mktest. Mkdemo says the > country of paco pena is spain and mktest runs through a whole series of > tests that appear to work with the exception of e05 it says failed a(s == > "hi!hello"). Most of them scroll too quickly but i think they worked. > > Then if I create a new MFC sdi app to keep it simple and add : > > c4_StringProp pNew ("Hey"); > > plus > > #include "mk4.h" //mine > > I get errors: > > --------------------Configuration: mktest2 - Win32 Debug-------------------- > Compiling resources... > Compiling... > StdAfx.cpp > Compiling... > mktest2.cpp > MainFrm.cpp > mktest2Doc.cpp > mktest2View.cpp > Generating Code... > Linking... > mktest2View.obj : error LNK2001: unresolved external symbol "public: > __thiscall c4_Property::c4_Property(char,char const *)" > (??0c4_Property@@QAE@DPBD@Z) > mktest2View.obj : error LNK2001: unresolved external symbol "public: > __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) > Debug/mktest2.exe : fatal error LNK1120: 2 unresolved externals > Error executing link.exe. > > mktest2.exe - 3 error(s), 0 warning(s) > > Oh well... > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jcw@equi4.com Wed May 23 17:11:33 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA14700 for ; Wed, 23 May 2001 17:11:33 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Building DLL and LIB files Date: Wed, 23 May 2001 15:01:15 -0700 Message-Id: <20010523220115.18616@smtp.telus.net> In-Reply-To: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> References: <003101c0e348$ce6eeb20$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: >I'm still trying to work out why I can't get Metakit to work. I'm sorry that this is causing you so much trouble. MSVC can be quite fussy about things w.r.t. mixing pre-compiled code. I've set up a sample project with a complete snapshot and information on what was needed to make it build and run properly. It's at: http://www.equi4.com/pub/mk/metakit-2.3.4-29-win/ -jcw From jcw@equi4.com Wed May 23 17:12:51 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA14778 for ; Wed, 23 May 2001 17:12:50 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Python 2.1 & Mk4py.dll Date: Wed, 23 May 2001 15:12:41 -0700 Message-Id: <20010523221241.1770@smtp.telus.net> In-Reply-To: <6917523888.20010518184017@usd1.com> References: <6917523888.20010518184017@usd1.com> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry McRae wrote: >Just installed Python 2.1. The version of Mk4py.dll I have been using >is dated 11/23/00 and is 221k. Python didn't like it loading >python20.dll, so I downloaded the newest Mk4py today. I've added a build for use with Python 2.1, see: http://www.equi4.com/pub/mk/metakit-2.3.4-29-win/py21/ -jcw From jcw@equi4.com Wed May 23 17:16:53 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA14957 for ; Wed, 23 May 2001 17:16:52 -0500 From: Jean-Claude Wippler To: Subject: RE: [Metakit] Python 2.1 & Mk4py.dll Date: Wed, 23 May 2001 15:16:41 -0700 Message-Id: <20010523221641.21016@smtp.telus.net> In-Reply-To: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> References: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. wrote: >>The 70 Kb size is due to UPX compression, see >> http://wildsau.idv.uni-linz.ac.at/mfx/upx.html First off, please allow me to (respectfully) disagree on every issue... >* Upon startup of a compressed EXE/DLL, all of the code is >decompressed from the disk image into memory in one pass, which can cause >disk thrashing if the system is low on memory and is forced to access the >swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates memory >for code pages on demand (i.e. when they are executed). That should not matter with a tiny package such as MetaKit. >* Multiple instances of a compressed EXE/DLL create multiple instances >of the code in memory. (So if your EXE is 1 MB and the user starts 5 >instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the >code is only stored in memory once and shared between instances. I'm no expert on this, but I'd assume that this not the case if the exe is shared as well (which is standard on Win32, AFAIK). >* Some older virus scanners flag compressed EXE/DLLs as being >virus-infected. (This is this reason I stopped compressing Inno Setup's >EXEs.) By now, I would hope that these older scanners have been updated? Anyway, as you noted, the compression is reversible. And in rare burst of trying to be maximally helpful, I've rebuilt and replaced "mk4vc60.dll" :) -jcw From cer@transelec.cl Wed May 23 17:22:29 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA15221 for ; Wed, 23 May 2001 17:22:28 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id SAA08433; Wed, 23 May 2001 18:17:13 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Wed, 23 May 2001 18:24:33 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C04679B11@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: spahievi@vega.bg Cc: metakit@equi4.com Subject: FW: [Metakit] Python 2.1 & Mk4py.dll Date: Wed, 23 May 2001 18:24:30 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I agree with you on point 1 and 3, this is not relevant, but point 2 is really important and Niki Spahiev has pointed other problems. Niki can you help with this? > -----Original Message----- > From: Jean-Claude Wippler [SMTP:jcw@equi4.com] > Sent: Wednesday, May 23, 2001 6:10 PM > To: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. > Subject: RE: [Metakit] Python 2.1 & Mk4py.dll > > Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. > wrote: > > >>The 70 Kb size is due to UPX compression, see > >> http://wildsau.idv.uni-linz.ac.at/mfx/upx.html > > First off, please allow me to (respectfully) disagree on every issue... > > >* Upon startup of a compressed EXE/DLL, all of the code is > >decompressed from the disk image into memory in one pass, which can cause > >disk thrashing if the system is low on memory and is forced to access the > >swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates > memory > >for code pages on demand (i.e. when they are executed). > > That should not matter with a tiny package such as MetaKit. > > >* Multiple instances of a compressed EXE/DLL create multiple instances > >of the code in memory. (So if your EXE is 1 MB and the user starts 5 > >instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the > >code is only stored in memory once and shared between instances. > > I'm no expert on this, but I'd assume that this not the case if the exe > is shared as well (which is standard on Win32, AFAIK). > > >* Some older virus scanners flag compressed EXE/DLLs as being > >virus-infected. (This is this reason I stopped compressing Inno Setup's > >EXEs.) > > By now, I would hope that these older scanners have been updated? > > Anyway, as you noted, the compression is reversible. And in rare burst > of trying to be maximally helpful, I've rebuilt and replaced "mk4vc60.dll" > :) > > -jcw From Slick@battleop.net Wed May 23 21:26:21 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA19181 for ; Wed, 23 May 2001 21:26:20 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Wed, 23 May 2001 19:32:04 -0700 Message-ID: <002701c0e3f9$3cf624e0$0200a8c0@pacbell.net> From: "Slick" To: Date: Wed, 23 May 2001 19:28:37 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking errors Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I downloaded mksample and I can get it to work. That is the good news. However, most of the time I use debug builds to check things out. I tried also copying the mk4vc60_d.dll and lib files that I built from the 2.3.4 release but I get the same linking errors that I was getting before. Do you not post the debug libs and dlls to your download site? From spahievi@vega.bg Thu May 24 03:56:28 2001 Received: from moria.vega.bg (moria.vega.bg [217.197.128.1]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA21503 for ; Thu, 24 May 2001 03:56:26 -0500 Received: from b1-dial-pool27.vega.bg (b1-dial-pool27.vega.bg [217.197.129.27]) by moria.vega.bg (8.9.1a/8.9.1) with ESMTP id KAA02171 for ; Thu, 24 May 2001 10:56:37 +0300 Date: Thu, 24 May 2001 11:11:02 +0300 From: Niki Spahiev X-Mailer: The Bat! (v1.52) UNREG / FE6CDE745D9DC872 Reply-To: spahievi X-Priority: 3 (Normal) Message-ID: <23276972.20010524111102@vega.bg> To: metakit@equi4.com Subject: Re[2]: [Metakit] Python 2.1 & Mk4py.dll In-Reply-To: <20010523221641.21016@smtp.telus.net> References: <16D7A2FE3592D41184A200508BDE826C04450719@SYN201> <20010523221641.21016@smtp.telus.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >>* Multiple instances of a compressed EXE/DLL create multiple instances >>of the code in memory. (So if your EXE is 1 MB and the user starts 5 >>instances of it, that adds up to 5+ MB.) With uncompressed EXE/DLLs, the >>code is only stored in memory once and shared between instances. JCW> I'm no expert on this, but I'd assume that this not the case if the exe JCW> is shared as well (which is standard on Win32, AFAIK). Sharing uses memory mapped files which is impossible with packed DLLs. -- regards, Niki Spahiev From lvirden@cas.org Thu May 24 14:20:07 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA07624 for ; Thu, 24 May 2001 14:20:07 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id PAA13512 for ; Thu, 24 May 2001 15:18:50 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f4OJIn318598; Thu, 24 May 2001 15:18:49 -0400 (EDT) Date: Thu, 24 May 2001 15:18:49 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010524151849.AAB18578@cas.org> To: metakit@equi4.com Subject: [Metakit] White papers/technical papers/etc. on Tclkit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Besides the 'ancient' seminal paper JC presented at the last Tcl Conference, and an upcoming conference paper by Steve Landers, I am unaware of other papers written concerning TclKit. I know that one esteemed journalist has written a column or two about it; however I was wondering if anyone knew of other papers detailing their work with the idea. If there is a better mailing list on which to ask this question, please let me know. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From rmenegon@bigpond.net.au Fri May 25 05:31:55 2001 Received: from mailin1.bigpond.com (juicer13.bigpond.com [139.134.6.21]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA22045 for ; Fri, 25 May 2001 05:31:54 -0500 Received: from unspecified.host ([139.134.4.58]) by mailin1.bigpond.com (Netscape Messaging Server 4.15) with SMTP id GDW04S00.IM1 for ; Fri, 25 May 2001 20:36:28 +1000 Received: from CPE-144-132-187-24.nsw.bigpond.net.au ([144.132.187.24]) by mail1.bigpond.com (Claudes-Perennial-MailRouter V2.9c 1/8606054); 25 May 2001 20:30:50 Received: from 192.168.11.13 ([192.168.11.13]) by 192.168.11.1 (WinRoute Pro 4.1) with SMTP; Fri, 25 May 2001 20:32:19 +1000 Received: by localhost with Microsoft MAPI; Fri, 25 May 2001 20:31:26 +1000 Message-ID: <01C0E559.ABFF76F0.rmenegon@bigpond.net.au> From: Rob Menegon Reply-To: "rmenegon@bigpond.net.au" To: "'metakit@equi4.com'" Date: Fri, 25 May 2001 20:31:25 +1000 Organization: MenMac X-Mailer: Microsoft Internet E-mail/MAPI - 8.0.0.4211 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] Synchronisation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Does anyone have any advice on file sharing mechanisms? I have a scenario in which multiple sources need to update the same metakit database. At present I am leaning towards all the process posting the updates to a queue and giving another process the responsibility of clearing the queue and updating the metakit database file. Any thought on the matter wold be greatly received. Rob _______________________________________ Rob Menegon 349 Somerville Road Hornsby Heights NSW 2077 Sydney, Australia (Mobile) +61 (0)416 271 348 ( Voice) +61 (0)2 9477 1348 ( Fax) +61 (0)2 8570 2031 rmenegon@bigpond.net.au _______________________________________ This email contains information intended for the addressee only. This information is confidential and may be privileged. If you are not the intended recipient, you must not read, use, copy or distribute this email or its attachments. If you have received this in error, please notify me immediately by return email or telephone, and delete and destroy this email. Thank you. _______________________________________ From claird@starbase.neosoft.com Fri May 25 07:42:24 2001 Received: from mail3.iadfw.net (mail3.iadfw.net [209.196.123.3]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id HAA24427 for ; Fri, 25 May 2001 07:42:24 -0500 Received: from starbase.neosoft.com from [206.109.1.32] by mail3.iadfw.net (/\##/\ Smail3.1.30.16 #30.25) with esmtp for sender: id ; Fri, 25 May 2001 07:42:52 -0500 (CDT) Received: (from claird@localhost) by starbase.neosoft.com (8.9.3/8.9.3) id HAA20822; Fri, 25 May 2001 07:54:24 -0500 (CDT) (envelope-from claird) Date: Fri, 25 May 2001 07:54:24 -0500 (CDT) From: Cameron Laird Message-Id: <200105251254.HAA20822@starbase.neosoft.com> To: metakit@equi4.com, rmenegon@bigpond.net.au Subject: Re: [Metakit] Synchronisation In-Reply-To: <01C0E559.ABFF76F0.rmenegon@bigpond.net.au> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >From metakit-admin@equi4.com Fri May 25 05:48:13 2001 > . > . > . >Does anyone have any advice on file sharing mechanisms? I have a scenario >in which multiple sources need to update the same metakit database. At >present I am leaning towards all the process posting the updates to a queue >and giving another process the responsibility of clearing the queue and >updating the metakit database file. > . > . > . Tequila might fit your needs perfectly. Tequila is a TclKit "add-on" Jean-Claude wrote last year. From g.lancaster@mbf.com.au Fri May 25 18:18:59 2001 Received: from fs1-smtp (fs1-smtp.mbf.com.au [203.101.28.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA08806 for ; Fri, 25 May 2001 18:18:56 -0500 Received: from MBF-INT1-Message_Server by fs1-smtp with Novell_GroupWise; Sat, 26 May 2001 09:04:08 +1000 Message-Id: X-Mailer: Novell GroupWise 5.5.2 Date: Sat, 26 May 2001 09:14:47 +1000 From: "Garth Lancaster" To: , Subject: Re: [Metakit] Synchronisation Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline X-Guinevere: 1.0.13 ; Medical Benefits Fun Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id SAA08806 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I agree with the theory of what Rob said, "all the process posting the updates to a queue and giving another process the responsibility of clearing the queue and updating the metakit database file." and Tequila might be ok Cameron, except Im not a Tcl kinda guy and I dont know if Rob is either... (but hey, thanks, and each to his own poison :-) ) When I asked about this a while back , JCW suggested that I need a central Metakit Server handling all the requests, locking etc, from possibly multiple clients, which sounded pretty plausible/do-able.. so its likely that such a server would be queue based - he also pointed me to the 'FAQ'/Discussion pages at the MetaKit Wiki :- http://www.equi4.com/metakit/wiki.cgi/ which had some notes on locking etc ... Cameron - you know more about Tequila than I do - is it possible to 'simulate' its 'mode of operation' from C++ do you think, or a different approach required ? (I'll start looking at it in my free time - thats usually between 10pm and 5am !! ) If anyone (Rob included) actually has implementation ideas, why dont we discuss them on the Metakit Wiki, so we can share them ? Garth Garth Lancaster MBF InfoTech (Infrastructure) Integration Engineer g.lancaster@mbf.com.au Ph : +61 2 9323 9534 Fax : +61 2 9267 7359 From g.lancaster@mbf.com.au Fri May 25 18:23:20 2001 Received: from fs1-smtp (fs1-smtp.mbf.com.au [203.101.28.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA08954 for ; Fri, 25 May 2001 18:23:18 -0500 Received: from MBF-INT1-Message_Server by fs1-smtp with Novell_GroupWise; Sat, 26 May 2001 09:08:31 +1000 Message-Id: X-Mailer: Novell GroupWise 5.5.2 Date: Sat, 26 May 2001 09:19:12 +1000 From: "Garth Lancaster" To: , , Subject: Re: [Metakit] Synchronisation Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline X-Guinevere: 1.0.13 ; Medical Benefits Fun Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id SAA08954 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The link to Tequila on the Metakit Wiki is broken - but I see at http://www.equi4.com/tequila/ "Tequila is now part of the MetaKit open source distribution." 'G' From rvcoulman@acm.org Fri May 25 23:51:41 2001 Received: from mail.dlcwest.com (backup.dlcwest.com [204.83.37.9]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA13315 for ; Fri, 25 May 2001 23:51:40 -0500 Received: (qmail 20669 invoked by uid 0); 26 May 2001 04:20:02 -0000 Received: from unknown (HELO acm.org) (204.83.37.45) by backup.dlcwest.com with SMTP; 26 May 2001 04:20:02 -0000 Message-ID: <3B0F3733.A8058F5F@acm.org> Date: Fri, 25 May 2001 22:55:15 -0600 From: Randy & Val Coulman X-Mailer: Mozilla 4.7 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Slick CC: metakit@equi4.com Subject: Re: [Metakit] For others potentially having problems v2.3.4 References: <004301c0e354$bdea97c0$0200a8c0@pacbell.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: > > I don't know if I am the only one who has had problems getting Metakit up > and running. > > > --------------------Configuration: mktest2 - Win32 Debug-------------------- > Compiling resources... > Compiling... > StdAfx.cpp > Compiling... > mktest2.cpp > MainFrm.cpp > mktest2Doc.cpp > mktest2View.cpp > Generating Code... > Linking... > mktest2View.obj : error LNK2001: unresolved external symbol "public: > __thiscall c4_Property::c4_Property(char,char const *)" > (??0c4_Property@@QAE@DPBD@Z) > mktest2View.obj : error LNK2001: unresolved external symbol "public: > __thiscall c4_Property::~c4_Property(void)" (??1c4_Property@@QAE@XZ) > Debug/mktest2.exe : fatal error LNK1120: 2 unresolved externals > Error executing link.exe. > > mktest2.exe - 3 error(s), 0 warning(s) > This suggestion is so simple and obvious that I can't believe no one has suggested it yet, but are you sure you're linking your application with the Metakit library? You say you've built a whole bunch of versions of the library, and that you have all of your paths set up, but are you actually telling the linker to use it? If not, this would be your problem. Just go on the Link tab in the Project Settings and add the Metakit library you want to the Object/library modules edit box. Randy -- Randy and Val Coulman rvcoulman@acm.org From Slick@battleop.net Sat May 26 01:19:35 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA14691 for ; Sat, 26 May 2001 01:19:34 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Fri, 25 May 2001 23:26:09 -0700 Message-ID: <001001c0e5ac$212e6c40$0200a8c0@pacbell.net> From: "Slick" To: Date: Fri, 25 May 2001 23:21:41 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Linking errors Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thank you all for your help. I now have it working. I am somewhat of a newbie in some areas and better in others. I had previously added another 3rd party class just by specifying the lib path under the tools | options menu. I assumed that metakit would work the same way. Probably a newbie mistake but it wouldn't have hurt to specify you have to type the lib name in also in another area. Maybe I've learned something and that is how you are supposed to do it. Anyway, I have it working (including different builds :)). My next problem is good examples and docs. For example I need a code snippet to show me how to find a row in a view based upon a prop value. I am trying things like: int x = vServer.Find(pServer "Two"); but I can't work it out. Are there any samples other than what is posted on the metakit site? I have achieved loading, saving and deleting rows. Also, out of curiosity when I get to the point of concurrent access is it easiest to do a batch to write out to the database? I assume you can't manipulate the database in different threads at the same time (my ultimate goal but a way off yet). From cristi_clot@hotmail.com Sat May 26 09:06:36 2001 Received: from hotmail.com (f49.law8.hotmail.com [216.33.241.49]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA22069 for ; Sat, 26 May 2001 09:06:26 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sat, 26 May 2001 06:52:48 -0700 Received: from 217.10.194.6 by lw8fd.law8.hotmail.msn.com with HTTP; Sat, 26 May 2001 13:52:48 GMT X-Originating-IP: [217.10.194.6] From: "Clot Cristian" To: metakit@equi4.com Date: Sat, 26 May 2001 13:52:48 -0000 Mime-Version: 1.0 Content-Type: text/html Message-ID: X-OriginalArrivalTime: 26 May 2001 13:52:48.0441 (UTC) FILETIME=[26068290:01C0E5EB] Subject: [Metakit] Newcomer at metakit. Hello to all of you! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive:
Hi, I'm a new member of your community, and I found MetaKit a very very good engine.
I am from Bucharest, Romania.
I'm using MetaKit for 2 months and I have 2 questions for you, if you are so kind to
answer me.
 
1. I tried to import some .dbf IV files of about 2 - 3 milion records in MetaKit, but (depending on size of the table, number of fields, etc.), the engine crashes when it reach  aprox. the 1.200.000'th record. I also use Jean-Claude's program for importing dbf files, but the same thing happens. Why ? How can I get over it ?
2. When I make a new sorted view on a Character field, and I go "Search" command, the found record is not the real one, but 3 or 4 records bellow or above. When I create the view, I  create it dinamically and access it through a pointer, and when I destroy it, the engine crashes. Why ? When I try to read records from this view, I also get a crush. This happens only on Character fields. can you tell me how can I make it work, please ?
 
Thank you very much,
Cristi CLOT
 
 
P.S. Jean-Claude, you're the best and I really, really apreciate what you're doing for internet and programmers community. Thanks.

Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

From cristi_clot@hotmail.com Sat May 26 09:12:34 2001 Received: from hotmail.com (f153.law8.hotmail.com [216.33.241.153]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA22132 for ; Sat, 26 May 2001 09:12:34 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sat, 26 May 2001 06:59:25 -0700 Received: from 217.10.194.6 by lw8fd.law8.hotmail.msn.com with HTTP; Sat, 26 May 2001 13:59:25 GMT X-Originating-IP: [217.10.194.6] From: "Clot Cristian" To: metakit@equi4.com Date: Sat, 26 May 2001 13:59:25 -0000 Mime-Version: 1.0 Content-Type: text/html Message-ID: X-OriginalArrivalTime: 26 May 2001 13:59:25.0886 (UTC) FILETIME=[12EBCDE0:01C0E5EC] Subject: [Metakit] Re about file sharing mechanism to Rob MENEGON Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive:
Did you try a client-server architecture ? I use oneright now and it works very good.

Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.

From nicolasb@maich.gr Mon May 28 03:53:14 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id DAA15977 for ; Mon, 28 May 2001 03:53:12 -0500 Received: FROM maich.gr BY mail-server ; Mon May 28 11:58:48 2001 +0300 Message-ID: <3B1214D2.4DEF5E41@maich.gr> Date: Mon, 28 May 2001 12:05:22 +0300 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: Clot Cristian CC: metakit@equi4.com Subject: Re: [Metakit] Re about file sharing mechanism to Rob MENEGON References: Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, What exactly are you doing? Nicolas Boretos Clot Cristian wrote: > > Did you try a client-server architecture ? I use oneright now and it > works very good. > > ---------------------------------------------------------------------- > Get Your Private, Free E-mail from MSN Hotmail at > http://www.hotmail.com. > > _______________________________________________ metakit mailing list - > metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From rmenegon@bigpond.net.au Mon May 28 13:48:09 2001 Received: from mailin2.email.bigpond.com (juicer14.bigpond.com [139.134.6.23]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA08570 for ; Mon, 28 May 2001 13:48:07 -0500 Received: from unspecified.host ([144.135.24.75]) by mailin2.email.bigpond.com (Netscape Messaging Server 4.15) with SMTP id GE2L0900.G9L; Tue, 29 May 2001 09:52:57 +1000 Received: from CPE-144-132-187-24.nsw.bigpond.net.au ([144.132.187.24]) by bwmam03.mailsvc.email.bigpond.com(MailRouter V2.9d 8323/935193); 29 May 2001 09:52:56 Received: from 192.168.11.202 ([192.168.11.202]) by 192.168.11.1 (WinRoute Pro 4.1) with SMTP; Tue, 29 May 2001 09:48:22 +1000 Received: by localhost with Microsoft MAPI; Tue, 29 May 2001 09:48:15 +1000 Message-ID: <01C0E824.7B842B60.rmenegon@bigpond.net.au> From: Rob Menegon Reply-To: "rmenegon@bigpond.net.au" To: "'nicolasb@maich.gr'" , "'cristi_clot@hotmail.com'" Cc: "'metakit@equi4.com'" Date: Tue, 29 May 2001 09:48:14 +1000 Organization: MenMac X-Mailer: Microsoft Internet E-mail/MAPI - 8.0.0.4211 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] about file sharing mechanism to Rob MENEGON (Nicolas Boretos) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas, I am reading data from various sources ... last count this was 6 serial lines but that the sources could be disk files as well. While the data is related, no particular source has the entire picture (it has its own unique piece of the puzzle), however it has enough information to uniquely identify its destination (there is come commonality) ... I need to aggregate the data in order to build the picture. The data is streaming in continuously over an 8 hour period. I am using MetaKit to provide the data aggregation and my original model was one that involved involved each of the readers (1 to 6 in this case) updating the database. This posed the access / sharing issue that I raised earlier. I have sinced modified my stance and introduced a writer process. The readers are responsible for reading and posting data to a queue and the writer is responsible for reading from the queue and updating the database. It give me other real estate and configuration flexibility advantages. By the way the environment is NT and C++ While I have not tested the theory I think that I could have also used the mutex kernel object for mutual exclusion of serveral threads. I am making the assumption that the only time MetaKit does any writing to disk is on the Commit call. This would allow me to permit a thread at a time into this section of code. That is my thinking at least. Rob _______________________________________ Rob Menegon 349 Somerville Road Hornsby Heights NSW 2077 Sydney, Australia (Mobile) +61 (0)416 271 348 ( Voice) +61 (0)2 9477 1348 ( Fax) +61 (0)2 8570 2031 rmenegon@bigpond.net.au _______________________________________ This email contains information intended for the addressee only. This information is confidential and may be privileged. If you are not the intended recipient, you must not read, use, copy or distribute this email or its attachments. If you have received this in error, please notify me immediately by return email or telephone, and delete and destroy this email. Thank you. _______________________________________ From ahmad@u.washington.edu Wed May 30 17:57:20 2001 Received: from mxout2.cac.washington.edu (mxout2.cac.washington.edu [140.142.33.4]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA10210 for ; Wed, 30 May 2001 17:57:20 -0500 Received: from mailhost2.u.washington.edu (mailhost2.u.washington.edu [140.142.33.2]) by mxout2.cac.washington.edu (8.11.2+UW01.01/8.11.2+UW01.04) with ESMTP id f4V3v3o17893 for ; Wed, 30 May 2001 20:57:03 -0700 Received: from cae0 (g-turkiyyah-2.ce.washington.edu [128.95.204.230]) by mailhost2.u.washington.edu (8.11.2+UW01.01/8.11.2+UW01.04) with SMTP id f4V3v2C30341 for ; Wed, 30 May 2001 20:57:02 -0700 Message-ID: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> From: "Ahmad El Husseini" To: Subject: [Metakit] Eficiency problem Date: Wed, 30 May 2001 21:01:31 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello,... I am working on a Structural Modeling software that has Tcl as its main interface and below Tcl there are bunch of solvers that do Dynamic Structural analysis (for earthquakes). Response quantities of objects (displacements, forces, etc) are generated at the end of every timestep. I am using Metakit as my database system where I write data from the C++ classes and extract data from the Tcl Interface. A typical project contains 4000 object and every object has an average of 4 double properties. analysis is done around 1000 times. I tried the following data format alternatives: * every object is a view and at the end of every timestep a row is added to the view. This was EXTREMELY slow. Object(property1, property2, property3, ...) 4000 such views with 1000 row each * I grouped objects into two categories and every category is a view with every object as a subview (formatted the data into columns) and at the end of every timestep a row is added. Category1(c1-Obj1, c1-Obj2,....) c1-Obj1(prop1, prop2, ...) ... Category2(c2-Obj1, c2-Obj2,....) c2-Obj1(prop1, prop2, ...) ... The second alternative proved to be way more efficient than the first one but still extremely slow. I am wondering: first, is Metakit the right tool to use for such a project? second, What is slowing things this much? (it takes minutes to write 0.5 Mb of data, I never exceeded 50 timesteps with only 1000 objects, it simply takes forever) Note: The solvers are very fast (relative to similar analysis programs) I expect to write data out to disk once or twice a second. Any help is greatly appreciated thanks, Ahmad From jcw@equi4.com Wed May 30 18:39:57 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA10505 for ; Wed, 30 May 2001 18:39:55 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Eficiency problem Date: Wed, 30 May 2001 21:39:40 -0700 Message-Id: <20010531043940.32689@smtp.telus.net> In-Reply-To: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> References: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ahmad El Husseini wrote: [...] >* every object is a view and at the end of every timestep a row is added to >the view. This was EXTREMELY slow. >Object(property1, property2, property3, ...) >4000 such views with 1000 row each It will be much faster to preallocate rows (and perhaps truncate when done). Do not append rows in a loop. In itself, 4000 views of 1000 rows of 4 doubles, i.e. 64 Mb, should be fine. But you're probably better off storing things the other way around first: one view per analysis run, with 4000 rows - one per object. You could then later re-open the file and re-organize the other way around. The time-consuming factor is touching all of the 4000 views on each cycle. Metakit is column-wise, so adding even just one row means the entire view gets written on commit. >first, is Metakit the right tool to use for such a project? If you are willing to rethink into columns and try not to commit every single time, yes - I think so. >second, What is slowing things this much? (it takes minutes to write 0.5 Mb >of data, I never exceeded 50 timesteps with only 1000 objects, it simply >takes forever) You're basically rewriting the entire db on each commit, because all views have been altered. >Note: The solvers are very fast (relative to similar analysis programs) I >expect to write data out to disk once or twice a second. -jcw From jyl@best.com Wed May 30 19:20:07 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA10836 for ; Wed, 30 May 2001 19:20:06 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id WAA16879; Wed, 30 May 2001 22:18:35 -0700 (PDT) Message-ID: <3B15D5B9.68FD70DF@best.com> Date: Wed, 30 May 2001 22:25:13 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Ahmad El Husseini CC: metakit@equi4.com Subject: Re: [Metakit] Eficiency problem References: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ahmad Another option is to store things in e4Graph which is built on top of Metakit. Each object could be a node with four "double" vertices. Check out http://e4graph.sourceforge.net. --JYL Ahmad El Husseini wrote: > Hello,... > > I am working on a Structural Modeling software that has Tcl as its main > interface and below Tcl there are bunch of solvers that do Dynamic > Structural analysis (for earthquakes). Response quantities of objects > (displacements, forces, etc) are generated at the end of every timestep. I > am using Metakit as my database system where I write data from the C++ > classes and extract data from the Tcl Interface. > > A typical project contains 4000 object and every object has an average of 4 > double properties. analysis is done around 1000 times. > > I tried the following data format alternatives: > > * every object is a view and at the end of every timestep a row is added to > the view. This was EXTREMELY slow. > Object(property1, property2, property3, ...) > 4000 such views with 1000 row each > > * I grouped objects into two categories and every category is a view with > every object as a subview (formatted the data into columns) and at the end > of every timestep a row is added. > Category1(c1-Obj1, c1-Obj2,....) > c1-Obj1(prop1, prop2, ...) > ... > Category2(c2-Obj1, c2-Obj2,....) > c2-Obj1(prop1, prop2, ...) > ... > The second alternative proved to be way more efficient than the first one > but still extremely slow. > > I am wondering: > first, is Metakit the right tool to use for such a project? > second, What is slowing things this much? (it takes minutes to write 0.5 Mb > of data, I never exceeded 50 timesteps with only 1000 objects, it simply > takes forever) > Note: The solvers are very fast (relative to similar analysis programs) I > expect to write data out to disk once or twice a second. > > Any help is greatly appreciated > thanks, > Ahmad > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From ahmad@u.washington.edu Thu May 31 10:09:12 2001 Received: from mxout1.cac.washington.edu (mxout1.cac.washington.edu [140.142.32.5]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA03464 for ; Thu, 31 May 2001 10:09:10 -0500 Received: from mailhost2.u.washington.edu (mailhost2.u.washington.edu [140.142.33.2]) by mxout1.cac.washington.edu (8.11.2+UW01.01/8.11.2+UW01.04) with ESMTP id f4VK8n115503 for ; Thu, 31 May 2001 13:08:49 -0700 Received: from cae0 (g-turkiyyah-2.ce.washington.edu [128.95.204.230]) by mailhost2.u.washington.edu (8.11.2+UW01.01/8.11.2+UW01.04) with SMTP id f4VK8nC32375 for ; Thu, 31 May 2001 13:08:49 -0700 Message-ID: <00b101c0ea0e$3a6e6910$e6cc5f80@ce.washington.edu> From: "Ahmad El Husseini" To: References: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> <20010531043940.32689@smtp.telus.net> Subject: Re: [Metakit] Eficiency problem Date: Thu, 31 May 2001 13:13:59 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Jean-Claude Wippler" [...] > But you're probably better off storing things the other way around first: > one view per analysis run, with 4000 rows - one per object. Is it more efficient to write 4000 columns, one per object, every object will be a nested view (with the whole view containing one row only)? because as you mentioned before adding a row causes metakit to write the whole view again. From: "Jacob Levy" > Another option is to store things in e4Graph ... Does e4Graph optimize the way data is formatted? Thanks very much for your time, Ahmad From claird@starbase.neosoft.com Thu May 31 10:24:41 2001 Received: from mail3.iadfw.net (mail3.iadfw.net [209.196.123.3]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA03909 for ; Thu, 31 May 2001 10:24:41 -0500 Received: from starbase.neosoft.com from [206.109.1.32] by mail3.iadfw.net (/\##/\ Smail3.1.30.16 #30.25) with esmtp for sender: id ; Thu, 31 May 2001 15:25:39 -0500 (CDT) Received: (from claird@localhost) by starbase.neosoft.com (8.9.3/8.9.3) id PAA44525; Thu, 31 May 2001 15:38:36 -0500 (CDT) (envelope-from claird) Date: Thu, 31 May 2001 15:38:36 -0500 (CDT) From: Cameron Laird Message-Id: <200105312038.PAA44525@starbase.neosoft.com> To: g.lancaster@mbf.com.au, metakit@equi4.com, rmenegon@bigpond.net.au Subject: Re: [Metakit] Synchronisation In-Reply-To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >From metakit-admin@equi4.com Fri May 25 18:33:17 2001 > . > . > . >When I asked about this a while back , JCW suggested that I need a central Metakit Server handling all the requests, locking etc, from possibly multiple clients, which sounded pretty plausible/do-able.. so its likely that such a server would be queue based - he also pointed me to the 'FAQ'/Discussion pages at the MetaKit Wiki :- >http://www.equi4.com/metakit/wiki.cgi/ >which had some notes on locking etc ... >Cameron - you know more about Tequila than I do - is it possible to 'simulate' its 'mode of operation' from C++ do you think, or a different approach required ? (I'll start looking at it in my free time - thats usually between 10pm and 5am !! ) > . > . > . Yes. But if you don't have a background in Tcl, I think it's probably best just to start from Jean-Claude's suggestion and comments, as you refer to them above. From cer@transelec.cl Thu May 31 10:56:56 2001 Received: from proxyinverso.enersis.cl (proxyinverso.enersis.cl [206.49.219.35]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA04867 for ; Thu, 31 May 2001 10:56:46 -0500 Received: from syn201.enersis.cl ([10.33.1.2]) by proxyinverso.enersis.cl (8.9.3/8.9.3) with ESMTP id QAA27983 for ; Thu, 31 May 2001 16:51:53 -0400 Received: by SYN201 with Internet Mail Service (5.5.2650.21) id ; Thu, 31 May 2001 16:59:48 -0400 Message-ID: <16D7A2FE3592D41184A200508BDE826C048B55B0@SYN201> From: "Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A." To: metakit@equi4.com Date: Thu, 31 May 2001 16:59:46 -0400 Importance: high X-Priority: 1 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain Subject: [Metakit] Mk4py: Closing a metakit file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Python 2.0, Mk4py 2.3.4-29 Wich is the right way to close a metakit file? I have an aplication that compare a metakit database into the aplication directory with a posible new database in other directory (a public directory on a network). Then the aplication copy the new file inside the aplication directory replacing the old one. Then the aplication continue using the updated database The only way I could do the copy step, without craching my aplication, was deleting the python objects asociated to the metakit databases. I could't find any way to close the metakit file than this one. Is this the right way to do this, or there is a close() method somewhere? My python code is something like this: ############################### file_one = mk.storage(CURREN_DB,0) #read only file_two = mk.storage(NEW_DB,0) #read only ..... compare(file_one, file_two) ..... del file_one # <---- If I delete this lines del file_two # <---- the aplications crach shutil.copyfile(NEW_DB, CURRENT_DB) final_file = mk.storage(CURREN_DB,0) .... work_with_the_file() .... ################################# Thanks Cristian Echeverria From rcohen@dial.oleane.com Thu May 31 23:53:34 2001 Received: from caroubier.wanadoo.fr (smtp-rt-6.wanadoo.fr [193.252.19.160]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA18226 for ; Thu, 31 May 2001 23:53:32 -0500 Received: from mahonia.wanadoo.fr (193.252.19.58) by caroubier.wanadoo.fr; 1 Jun 2001 11:52:38 +0200 Received: from dial.oleane.com (217.128.52.70) by mahonia.wanadoo.fr; 1 Jun 2001 11:52:11 +0200 Message-ID: <3B1773CD.82744536@dial.oleane.com> Date: Fri, 01 Jun 2001 11:51:57 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] join Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I have just began to use metakit, and I cannot do a join ... c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); c4_View table2=database.GetAs("table2[index2:I,val2:D]"); // fill tables c4_View join; c4_IntProp index("index"); c4_IntProp index2("index2"); join.AddProperty(index2); join.AddProperty(index); c4_View result=table2.Join(join,table1); the result view has no data ! Thanks for any help -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From Slick@battleop.net Fri Jun 1 15:29:28 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id PAA11605 for ; Fri, 1 Jun 2001 15:29:27 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Fri, 1 Jun 2001 18:36:35 -0700 Message-ID: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> From: "Slick" To: Date: Fri, 1 Jun 2001 18:32:27 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Avoiding using global variables Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have a dialog class that accesses a metakit database. I have it working well. However, I had the variables declared globally. I decided that I wanted to have the variables just local to the class because: (1) I don't want a lot of global variables, and (2) I am worried that the database remains open using resources even after the dialog is closed. This may not be true but I can't find much actual documentation on metakit usage ( i like the small discussion on getting started on the metakit site but beyond this there isn't much). I tried simply adding member variables but when building they are misinterpreted as function declarations because of the (). For example public: c4_StringProp pServer ("server"); gives error C2660: 'pServer' : function does not take 1 parameters The only way I have been able to do it is to declare the variables as static within the class. I don't think this is a good solution. All comments etc appreciated. From readonly@getsoft.com Fri Jun 1 15:45:25 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA11752 for ; Fri, 1 Jun 2001 15:45:24 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id TAA18543; Fri, 1 Jun 2001 19:44:57 -0600 (MDT) Message-ID: <200106021145070534.0A8383A4@getsoft.com> In-Reply-To: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> References: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Sat, 02 Jun 2001 11:45:07 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "Slick" , metakit@equi4.com Subject: Re: [Metakit] Avoiding using global variables Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id PAA11752 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick, This has nothing to do with MK and is basic C++ 101 stuff. You can't assign a value to a class member in the declaration. This should most likely be done in the class constructor. It has been a while since I've used MK but you most likely need to use a pointer to c4_StringProp. eg. c4_StringProp* m_pServer; In the constructor: m_pServer = new c4_StringProp("server"); and don't forget to delete it in the destructor. And yes, statics and globals are BAD. *********** REPLY SEPARATOR *********** On 1 Jun 2001 at 18:32 Slick wrote: >I have a dialog class that accesses a metakit database. I have it working >well. However, I had the variables declared globally. I decided that I >wanted to have the variables just local to the class because: > >(1) I don't want a lot of global variables, and >(2) I am worried that the database remains open using resources even after >the dialog is closed. This may not be true but I can't find much actual >documentation on metakit usage ( i like the small discussion on getting >started on the metakit site but beyond this there isn't much). > >I tried simply adding member variables but when building they are >misinterpreted as function declarations because of the (). For example > >public: > c4_StringProp pServer ("server"); >gives >error C2660: 'pServer' : function does not take 1 parameters > >The only way I have been able to do it is to declare the variables as static >within the class. I don't think this is a good solution. > >All comments etc appreciated. > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find lots of information on ED and while you're there you can click on 'Stay Informed' to join our mailing list and 'Future Plans' to read about Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From jcw@equi4.com Fri Jun 1 23:08:57 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19092 for ; Fri, 1 Jun 2001 23:08:56 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Avoiding using global variables Date: Sat, 2 Jun 2001 00:47:35 -0700 Message-Id: <20010602074735.29040@smtp.telus.net> In-Reply-To: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> References: <000b01c0eb03$e220bf00$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: [...] >public: > c4_StringProp pServer ("server"); >gives >error C2660: 'pServer' : function does not take 1 parameters That's not proper C++. Constructors of members are specified in the constructor of the encapsulating class, not in the declaration of the member: class Classy { public: Classy () : pServer ("server") { } c4_StringProp pServer; }; -jcw From jcw@equi4.com Fri Jun 1 23:08:58 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19095 for ; Fri, 1 Jun 2001 23:08:58 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Mk4py: Closing a metakit file Date: Sat, 2 Jun 2001 01:16:20 -0700 Message-Id: <20010602081620.6923@smtp.telus.net> In-Reply-To: <16D7A2FE3592D41184A200508BDE826C048B55B0@SYN201> References: <16D7A2FE3592D41184A200508BDE826C048B55B0@SYN201> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. wrote: >Python 2.0, Mk4py 2.3.4-29 > >Wich is the right way to close a metakit file? There is no other way to close than to delete all references to the storage object, I'm afraid. This, as you discovered, works: db = metakit.storage(...) ... del db You could also use: db = None The key is that every reference to the storage object must be gone. Once the internal reference count drops to zero, the datafile will be closed. -jcw From jcw@equi4.com Fri Jun 1 23:09:03 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19102 for ; Fri, 1 Jun 2001 23:09:02 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] join Date: Sat, 2 Jun 2001 02:09:03 -0700 Message-Id: <20010602090903.15357@smtp.telus.net> In-Reply-To: <3B1773CD.82744536@dial.oleane.com> References: <3B1773CD.82744536@dial.oleane.com> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: > c4_View join; > c4_IntProp index("index"); > c4_IntProp index2("index2"); > join.AddProperty(index2); > join.AddProperty(index); > c4_View result=table2.Join(join,table1); Which can also be written as: c4_View result=table2.Join((index2,index),table1); >the result view has no data ! Without knowing what data table1 and table2 contain, it's hard to tell whether it should. Try a simpler join first: c4_View result=table2.Join(index2,table1); Look at tests/tcusto2.cpp, test "c17" for a simple example. If you cannot solve the problem, please send me a simple test case so I can investigate furher. -jcw From jcw@equi4.com Fri Jun 1 23:09:02 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19098; Fri, 1 Jun 2001 23:08:58 -0500 From: Jean-Claude Wippler To: , , Subject: Re: [Metakit] Synchronisation Date: Sat, 2 Jun 2001 01:58:00 -0700 Message-Id: <20010602085800.16605@smtp.telus.net> In-Reply-To: <200105312038.PAA44525@starbase.neosoft.com> References: <200105312038.PAA44525@starbase.neosoft.com> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Garth Lancaster wrote: >> [... Tequila ...] is it possible to >'simulate' its 'mode of operation' from C++ do you think, or a different >approach required ? Tequila is a tiny network server written in Tcl. It doesn't do that much, just accept requests and propagate change requests to all attached clients - Tcl makes such things trivial. Note that the current Tequila design "talks" Tcl over the wire. You'd be having more work to construct and decode Tcl-style commands and lists in C++, than is probably worth it. Back to the original issue... Rob Menegon wrote: >Does anyone have any advice on file sharing mechanisms? I have a scenario >in which multiple sources need to update the same metakit database. At >present I am leaning towards all the process posting the updates to a queue >and giving another process the responsibility of clearing the queue and >updating the metakit database file. > >Any thought on the matter wold be greatly received. You'll have to coordinate reading as well, MK does a considerable amount of caching, so a change (even if it's a single writer), will not make readers aware of the changes. One surprising part will be that readers will appear to work properly at first, because a single commit does not invalidate the previous one. So readers will simply continue to see the previous state. The trouble is that the *next* commit can clobber that previous state, at which point readers can crash and burn any time, i.e.: reader opens -> write commits twice -> reader is in trouble. To put it differently: as long as there are readers, the system must avoid altering state underneath them. A second issue is that at some point, you'll probably want the readers to see the new state, not the original. This is the reason why I introduced esoteric things such as "commit- extend" and "commit-aside" (both still experimental). Commit-extend just keeps appending, thus guaranteeing that readers continue to see the original state at all times (but files will grow very very quickly). Commit-aside stores differences into a separate file, again letting readers continue to see a consistent (old) state. One approach which seems worth pursueing, is the combination of both: - file A holds a dataset - file A' stores changes, in a growing manner That means readers can open A, for the original state, or even A + A' for the latest state. A single writer keeps on appending differences to A'. Then, at some point everyone has to close their datasets, the changes from A' are folded back into A, and A' is deleted. There are several trouble-spots with this today: - commit-extend/commit-aside have not been tested enough - the combination has not yet been tried at all - commit-aside is still dumb, it stores all data, not differences Not of these are show stoppers, but they have to be fixed/finished first. It may all seem a bit strange to have to resort to such unconventional tactics, but you have to keep in mind that MK stores its data column- wise. This means that normally, a change to one value in a view of 100,000 rows, causes an entire *column* of values to be saved on commit. Commit-aside was designed to avoid this, and store a byte-oriented "difference set". Commit-extend solves the issue of a writer never modifying things underneath readers. Both can work with locking, and hence without contention - i.e. at top speed. With a bit of extra machinery (semaphores, probably), one could then signal changes and make readers adjust themselves to new commits efficiently. I think. All in all, there's a lot of potential in this IMO, but it's not there today. If you need something soon, I continue to suggest writing a very simple server (or better still, adopting something that exists), and pick a protocol (perhaps a standard one), then make all I/O go through the server, i.e. both reading *and* writing. In a way, that turns clients into "dumb" participants, but it also gets around a lot of problems. Locking can be added at a higher level, i.e. through a few server primitives. I'd be happy to discuss this further (the wiki is a good idea), but have to add that I cannot spend time on this before August. To close the circle of this email: I have recently started to alter Tequila so it uses an even simpler, and less language-specific protocol over the wire. It might be effective to code the server in Tcl or Python, yet do it in such a way that it fits easily into a C/C++ client (or any other language). -jcw From Slick@battleop.net Sat Jun 2 06:37:08 2001 Received: from slickserver (adsl-207-105-204-47.dsl.lsan03.pacbell.net [207.105.204.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA23701 for ; Sat, 2 Jun 2001 06:37:07 -0500 Received: from slk (192.168.0.2) by slickserver with MERCUR-SMTP/POP3/IMAP4-Server (v3.20.01 FC-8388608) for ; Sat, 2 Jun 2001 09:44:12 -0700 Message-ID: <008101c0eb82$ac9fbf00$0200a8c0@pacbell.net> From: "Slick" To: Date: Sat, 2 Jun 2001 09:40:03 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.3018.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Avoiding using global variables Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: public: > c4_StringProp pServer ("server"); >gives >error C2660: 'pServer' : function does not take 1 parameters That's not proper C++. Constructors of members are specified in the constructor of the encapsulating class, not in the declaration of the member: class Classy { public: Classy () : pServer ("server") { } c4_StringProp pServer; }; -jcw ______________________________ All I am trying to do is define the variables within my dialog class so that I can use them in different subs within the dialog class. I just need them to show as member variables. I tried your example but I can't get it to work. Here is one of the tries: class SettingsDlg : public CDialog { // Construction public: SettingsDlg(CWnd* pParent = NULL); // standard constructor //c4_StringProp pServer ("server"); SettingsDlg(): pServer ("server") {} c4_StringProp pServer; CString m_curServerName; BOOL m_bNewitem; The dlg constructor is already set up. Putting in SettingsDlg(): doesn't make sense to me. I have about 8 metakit variables that I want to create as member variables. pServer is just one of them. Like I mentioned before I have it working with them as either global or static but there has to be a better way. This is probably my lack of skill. From jyl@best.com Sat Jun 2 07:14:12 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA24017 for ; Sat, 2 Jun 2001 07:14:12 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id KAA28241; Sat, 2 Jun 2001 10:13:12 -0700 (PDT) Message-ID: <3B192036.58CF8BA0@best.com> Date: Sat, 02 Jun 2001 10:19:50 -0700 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Ahmad El Husseini CC: metakit@equi4.com Subject: Re: [Metakit] Eficiency problem References: <002001c0e986$604d44e0$e6cc5f80@ce.washington.edu> <20010531043940.32689@smtp.telus.net> <00b101c0ea0e$3a6e6910$e6cc5f80@ce.washington.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ahmad Apologies for not replying earlier -- I was away. The (*THE*) best way to understand how e4Graph stores its data is to read the definition of its storage layout in core/mkdriver/mkstorage.h at the head of the file. This is (on purpose) not documented so I can change it if there's a need. With that caveat, here is some explanation. e4Graph is designed to store arbitrary graph-like data in a table format in a relational or column-oriented database like Metakit. I intend to write (one day far in the future :) drivers for other database systems such as mySQL, GDBM, etc. What follows (and what you'll see in mkstorage.h) is Metakit specific. The details will be vastly different when you use any other database drivers, without impacting in any way the program you write to the e4Graph API. The Metakit database driver for e4Graph stores its date in one storage. That storage contains exactly these views: #define E4_GRAPHMARKERS "e4GraphMarkers[nameID:I,next:I,root:I,used:I]" #define E4_GRAPHNODES \ "e4GraphNodes[firstVertex:I,lastVertex:I,vertexCount:I,parentID:I," \ "used:I,refCount:I,nodeMarkers:I]" #define E4_GRAPHVERTICES \ "e4GraphVertices[nameID:I,nodeID:I,vertexType:I,row:I,next:I,prev:I,used:I]" #define E4_GRAPHFLOATS "e4GraphFloats[f:F,used:I]" #define E4_GRAPHSTRINGS "e4GraphStrings[s:S,used:I]" #define E4_GRAPHNAMES "e4GraphNames[n:S,used:I]" #define E4_GRAPHBINARY "e4GraphBinary[b:B,used:I]" #define E4_GRAPHUNUSED "e4GraphUnused[first:I]" #define E4_GRAPHPARENTS "e4GraphParents[nodeID:I,count:I,used:I]" So, after examining this you'll see that this is highly optimized to store graph-like structures with the ability to efficiently traverse, update, modify and commit. For further information about how to use e4Graph in your program, please check the web site at http://e4graph.sourceforge.net. Also, if you need more sample code, check out the e4XML directory, the tcl directory and the test directory in the source code distributions. Finally, if you have more questions, I'll be happy to answer them on this mailing list or in private, as you prefer. One final note: e4Graph is scheduled to be in Beta by mid-June and the final 1.0 release is in September or October 2001. It's based on Metakit 2.3.4-29, which I hope will be final (Metakit 2.4.0) by that time. --JYL Ahmad El Husseini wrote: > From: "Jean-Claude Wippler" > [...] > > But you're probably better off storing things the other way around first: > > one view per analysis run, with 4000 rows - one per object. > > Is it more efficient to write 4000 columns, one per object, every object > will be a nested view (with the whole view containing one row only)? because > as you mentioned before adding a row causes metakit to write the whole view > again. > > From: "Jacob Levy" > > Another option is to store things in e4Graph ... > > Does e4Graph optimize the way data is formatted? > > Thanks very much for your time, > Ahmad > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Sat Jun 2 07:50:02 2001 Received: from [10.0.1.2] (ajgg45s3y2ph.bc.hsia.telus.net [216.232.137.79]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA24497 for ; Sat, 2 Jun 2001 07:50:01 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Avoiding using global variables Date: Sat, 2 Jun 2001 10:50:08 -0700 Message-Id: <20010602175008.6571@smtp.telus.net> In-Reply-To: <008101c0eb82$ac9fbf00$0200a8c0@pacbell.net> References: <008101c0eb82$ac9fbf00$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Slick wrote: [...] >class SettingsDlg : public CDialog >{ >// Construction > >public: > SettingsDlg(CWnd* pParent = NULL); // standard constructor > //c4_StringProp pServer ("server"); > SettingsDlg(): pServer ("server") {} > c4_StringProp pServer; > > CString m_curServerName; > BOOL m_bNewitem; > >The dlg constructor is already set up. Putting in SettingsDlg(): doesn't >make sense to me. I have about 8 metakit variables that I want to create as >member variables. pServer is just one of them. > >Like I mentioned before I have it working with them as either global or >static but there has to be a better way. This is probably my lack of skill. This is a C++ issue and has nothing to do with MK. In your code, you have: SettingsDlg::SettingsDlg(CWnd* pParent) { ... Change it to: SettingsDlg::SettingsDlg(CWnd* pParent) : pServer ("server") { ... That's as far as I can go with answering general C++ Q's on this list, I'm afraid. -jcw From jerry@usd1.com Tue Jun 12 14:47:38 2001 Received: from asteroid.pacifier.com (asteroid.pacifier.com [199.2.117.154]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA26838 for ; Tue, 12 Jun 2001 14:47:37 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id f5D0jtM23476 for ; Tue, 12 Jun 2001 17:45:55 -0700 (PDT) Date: Tue, 12 Jun 2001 17:45:19 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <12131613106.20010612174519@usd1.com> To: Metakit Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] [python] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I have been writing a batch processing system in Python based on MetaKit that will take a number of records (10K to 200k) from an outside source (flat, SQL, CSV), process it, then export selected fields to a different outside source. I have created dictionary driven classes for Records and DataSets. These classes view and modify the data in the tables directly, handling the multiple levels (sub-views). Question 1) I have read that it is more efficient to pre-allocate the row. How is that done with Python? Question 2) I have been struggling with creating a less direct method of creating / changing fields in Mk through Python. When creating or modifying a row from an import routine or a screen, I need to verify the integrity of the row before I 'commit' it to the table (view). Option A: Use .commit() & .rollback(). I rejected this immediately because of the disk I/O overhead when writing 100,000 records. Option B: copy the contents of a row into a Python structure (I wish there was a Mk function to do this -- it is a bit tedious); modify / verify the changes; copy back to row (append if new). Two MAJOR problems: 1) creating new... Using view 'test[id:I,name,sales[region,value:I]]'.. newrec = {'id':1,'name':'bob','sales':[{'region':'N','value':100}]} vw.append(newrec) works, appends 'bob', with id 1 and one row in 'sales', but all fields are empty. problem 2), updating existing rows. I found the following statement to be very bad for the health of the program: vw[n] = newrec Thanks for reading this far! I hope this makes sense. -- Thanks, Jerry McRae MetaKit 2.3.4, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From jerry@usd1.com Tue Jun 12 17:10:48 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA29464 for ; Tue, 12 Jun 2001 17:10:48 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f5D395i02922 for ; Tue, 12 Jun 2001 20:09:06 -0700 (PDT) Date: Tue, 12 Jun 2001 20:08:34 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <15540209510.20010612200834@usd1.com> To: metakit Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] [python] more tries Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, (addendum to last message) Spending concerted effort today to making a 'buffer' for data entry and importing, I have tried the following ideas. C) reserving row zero for the I/O buffer, since it has exactly the same properties, then copy or append if it verifies. Downside: I can figure out no easy way of clearing the row's contents before using. Something like vw.__dict__ = {} would be real nice. D) add a new row to the end to use as the buffer, then delete it. Downside: the .delete() function crashes for me. using interactive prompt: Python 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> import mk >>> vw=mk.storage('com.mk',1).view('com') >>> len(vw) 1000 >>> vw.delete(999) >>> len(vw) SystemError: null argument to internal routine >>> vw >>> vw[0].fname # is a valid property Traceback (most recent call last): File "", line 1, in ? AttributeError: fname >>> vw >>> vw.structure() [] #-------------------------------- using PyWin: >>> vw.append(vw[1000]) #copy last row 1001 >>> vw.delete(1001) #delete last row Firing event 'ProcessEnter' failed. Traceback (most recent call last): File "f:\python21\pythonwin\pywin\scintilla\bindings.py", line 141, in fire rc = apply(binding.handler, args) File "f:\python21\pythonwin\pywin\framework\interact.py", line 476, in ProcessEnterEvent self.history.history_store(source) File "f:\python21\pythonwin\pywin\idle\IdleHistory.py", line 75, in history_store self.history.remove(source) SystemError: null argument to internal routine >>> len(vw) # it did work, though 999 #------------------------------- It is odd that id does not say anything about MetaKit in the errors. E) create a separate view for a buffer. I think this will work. I.E. vwb = vw[0:0] recbuffer = vwb.append() ...set all the fields -- (which still involves multiple loops for complex structures) vw.append(vwb[0]) -or- vw[index] = vwb[0] One idea I had was that it would be nice to have a mini-rollback option... to un-do the append, or the changes just to the last record. [Another thing I think would be nice is the equivalent of the .filter() that actually returns a new view. This would save several steps!] Other options? How do others work with rows and balance the commits/rollbacks and the direct row access? -- Thanks again!, Jerry McRae MetaKit 2.3.4, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From barnard@aragorn.stat.harvard.edu Wed Jun 13 03:34:44 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA09083 for ; Wed, 13 Jun 2001 03:34:43 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id JAA07550; Wed, 13 Jun 2001 09:43:42 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.28162.951393.336937@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 09:43:30 -0400 (EDT) To: Jerry McRae Cc: Metakit Subject: [Metakit] [python] In-Reply-To: <12131613106.20010612174519@usd1.com> References: <12131613106.20010612174519@usd1.com> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, Jerry McRae writes: > Hi, > > I have been writing a batch processing system in Python based on > MetaKit that will take a number of records (10K to 200k) from an > outside source (flat, SQL, CSV), process it, then export selected > fields to a different outside source. I have created dictionary driven > classes for Records and DataSets. These classes view and modify the > data in the tables directly, handling the multiple levels (sub-views). > > Question 1) I have read that it is more efficient to pre-allocate the > row. How is that done with Python? You cannot do this with Mk4py, however, it's easy to add the necessary code to add the functionality to the Python interface: (1) Add the following to PyView.cpp: static char *setsize__doc = "setsize(nrows)\n"; static PyObject *PyView_setsize(PyView *o, PyObject *_args) { try { PWOSequence args(_args); PWONumber nrows = PWONumber(args[0]); o->SetSize((int) nrows); return nrows.disOwn(); } catch (PWException e) { return e.toPython(); } } (2) Add the following to the ViewMethods array in PyView.cpp: {"setsize", (PyCFunction)PyView_setsize, METH_VARARGS, setsize__doc}, Of course you'll need to rebuild the Mk4py library. > Question 2) I have been struggling with creating a less direct method > of creating / changing fields in Mk through Python. When creating or > modifying a row from an import routine or a screen, I need to verify > the integrity of the row before I 'commit' it to the table (view). > > Option A: Use .commit() & .rollback(). I rejected this immediately > because of the disk I/O overhead when writing 100,000 records. > > Option B: copy the contents of a row into a Python structure (I wish > there was a Mk function to do this -- it is a bit tedious); modify > / verify the changes; copy back to row (append if new). Two MAJOR > problems: 1) creating new... > Using view 'test[id:I,name,sales[region,value:I]]'.. > newrec = {'id':1,'name':'bob','sales':[{'region':'N','value':100}]} > vw.append(newrec) > > works, appends 'bob', with id 1 and one row in 'sales', but all fields > are empty. > > problem 2), updating existing rows. I found the following statement > to be very bad for the health of the program: > vw[n] = newrec > > Thanks for reading this far! I hope this makes sense. I cannot offer any advice on this. I've made a Python wrapper class around PyView so that I can completely customize setting, getting, etc. of my views. It's been a lot of work, but I'm now extremely happy with the efforts. Best, John Barnard -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From barnard@aragorn.stat.harvard.edu Wed Jun 13 04:24:30 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA10806 for ; Wed, 13 Jun 2001 04:24:30 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id KAA07636; Wed, 13 Jun 2001 10:33:27 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.31156.462747.59373@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 10:33:24 -0400 (EDT) To: Jerry McRae Cc: metakit Subject: [Metakit] [python] more tries In-Reply-To: <15540209510.20010612200834@usd1.com> References: <15540209510.20010612200834@usd1.com> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry McRae writes: > D) add a new row to the end to use as the buffer, then delete it. > Downside: the .delete() function crashes for me. > > using interactive prompt: > Python 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32 > Type "copyright", "credits" or "license" for more information. > >>> import mk > >>> vw=mk.storage('com.mk',1).view('com') > >>> len(vw) > 1000 > >>> vw.delete(999) > >>> len(vw) > SystemError: null argument to internal routine I also get this message after deleting a row regardless of the command given after the deletion. I'm running Python 2.1, Metakit 2.3.4, and Windows 2000 SP1. If I make a change in the code for delete in PyView.cpp from static PyObject* PyView_delete(PyView *o, PyObject* _args) { try { PWOSequence args(_args); int ndx = PWONumber (args[0]); //o->RemoveAt(ndx); PWOSequence seq; o->setSlice(ndx, ndx+1, seq); Py_INCREF(Py_None); return Py_None; } catch(PWException e) { return e.toPython(); } } to static PyObject* PyView_delete(PyView *o, PyObject* _args) { try { PWOSequence args(_args); int ndx = PWONumber (args[0]); o->RemoveAt(ndx); // PWOSequence seq; // o->setSlice(ndx, ndx+1, seq); Py_INCREF(Py_None); return Py_None; } catch(PWException e) { return e.toPython(); } } i.e., uncomment the RemoveAt line and comment out the slice lines, I no longer get the error message. I haven't had the time to look closely at setSlice, which eventually calls RemoveAt and does some range checking. The raw RemoveAt might be dangerous without some range checking, but at least it appears to work. -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From barnard@aragorn.stat.harvard.edu Wed Jun 13 05:12:12 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA12303 for ; Wed, 13 Jun 2001 05:12:12 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id LAA07728; Wed, 13 Jun 2001 11:21:02 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.34014.171893.297213@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 11:21:02 -0400 (EDT) To: metakit@equi4.com X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Subject: [Metakit] Deleting rows and Obtaining Views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've observed, as demonstrated in the example below, that under Mk4py after deleting a row from a view, deleting the view, and reloading the view from storage that the reloaded view still has the row deletion, i.e., isn't the original view as I expected. Am I misunderstanding how to retrieve the original view? If I delete the storage object containing the view, then recreate the storage and view I get the original view. Why do I need to delete and recreate the storage? Python 2.1 (#15, Apr 16 2001, 18:25:49) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> import metakit as mk >>> test2 = mk.storage("junk", 1) >>> v2 = test2.view("v2") >>> mk.dump(v2) var1 var2 ---- ---- 0.0 0.0 0.0 1.0 0.0 2.0 0.0 3.0 0.0 4.0 ---- ---- Total: 5 rows >>> v2.delete(3) >>> mk.dump(v2) var1 var2 ---- ---- 0.0 0.0 0.0 1.0 0.0 2.0 0.0 4.0 ---- ---- Total: 4 rows >>> del v2 >>> v2 = test2.view("v2") >>> mk.dump(v2) var1 var2 ---- ---- 0.0 0.0 0.0 1.0 0.0 2.0 0.0 4.0 ---- ---- Total: 4 rows -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From gmcm@hypernet.com Wed Jun 13 05:54:54 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA13463 for ; Wed, 13 Jun 2001 05:54:53 -0500 Received: from me (204.176.40.67) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Wed, 13 Jun 2001 11:52:18 -0400 From: "Gordon McMillan" To: Jerry McRae , Metakit Date: Wed, 13 Jun 2001 11:52:01 -0400 Subject: Re: [Metakit] [python] Reply-to: gmcm@hypernet.com Message-ID: <3B2753E1.3600.3E49835B@localhost> Priority: normal In-reply-to: <12131613106.20010612174519@usd1.com> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, I tend to use MK for persistence, not batch, so I tend to commit on every little change. But I can help with some of these questions. > I have been writing a batch processing system in Python based > on MetaKit that will take a number of records (10K to 200k) > from an outside source (flat, SQL, CSV), process it, then > export selected fields to a different outside source. I have > created dictionary driven classes for Records and DataSets. > These classes view and modify the data in the tables directly, > handling the multiple levels (sub-views). > > Question 1) I have read that it is more efficient to > pre-allocate the row. How is that done with Python? Not as efficient as the underlying C++, but you can build a large view fairly efficiently by using repeated concatenation for i in range(10): v = v + v > Question 2) I have been struggling with creating a less direct > method of creating / changing fields in Mk through Python. > When creating or modifying a row from an import routine or a > screen, I need to verify the integrity of the row before I > 'commit' it to the table (view). > > Option A: Use .commit() & .rollback(). I rejected this > immediately because of the disk I/O overhead when writing > 100,000 records. > > Option B: copy the contents of a row into a Python structure (I > wish there was a Mk function to do this -- it is a bit > tedious); modify / verify the changes; copy back to row (append > if new). Two MAJOR problems: 1) creating new... Using view > 'test[id:I,name,sales[region,value:I]]'.. newrec = > {'id':1,'name':'bob','sales':[{'region':'N','value':100}]} > vw.append(newrec) > > works, appends 'bob', with id 1 and one row in 'sales', but all > fields are empty. I'm afraid there's no way around the fact that subviews are created empty. > problem 2), updating existing rows. I found the following > statement to be very bad for the health of the program: vw[n] = > newrec You might do something like this for a start: class RowProxy: def __init__(self, row): self._row = row def __getattr__(self, nm): return getattr(self._row, nm) def commit(self): for k,v in self.__dict__.items(): if hasattr(self._row, k): setattr(self._row, k, v) With this proxy, changing an attribute will change it in the proxy's __dict__. row.shoesize will always return the current value of shoesize (which may be the original value - you get the original value with row._row.shoesize). row.commit will update the underlying row in the view. From your second message, I have no idea what's going wrong with view.delete. It certainly works for me, but I haven't tried MK with Py 2.1 yet (I wonder if it has something to do with gc). OTOH, I almost always use a isdel attribute on rows. That's because: r = v[5] r.name = 'fred' v.delete(0) r.name # no longer 'fred', because he's moved to v[4] I'm not sure what you're getting at with trying to create a buffer space in MK. If I have to intercept the changes to a row, I use a proxy like that above. For inserting new rows, I either build values in a dict, use an instance of a dummy class (basically sytactic sugar around the instance __dict__), or append an empty row and modify it. Note that you can proxy a view to return proxied rows with: class Wrapper: def __init__(self, view, klass): self.__obj = view self.__klass = klass def __getattr__(self, nm): return getattr(self.__obj, nm) def __getitem__(self, i): return self.__klass(self.__obj[i]) def __len__(self): return len(self.__obj) The reference semantics of MK rows can be hard to get used to, but once you do, it can be very powerful. - Gordon From barnard@aragorn.stat.harvard.edu Wed Jun 13 07:34:31 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA16490 for ; Wed, 13 Jun 2001 07:34:31 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id NAA07969; Wed, 13 Jun 2001 13:43:31 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.42562.868686.728229@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 13:43:30 -0400 (EDT) To: metakit@equi4.com Subject: RE: [Metakit] Deleting rows and Obtaining Views X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. writes: > you need to do a commit() to save the changes until deleting the storage > object > in you case: > > test2.commit() That makes the row deletion permanent. I was trying to retrieve the original view (without the row deletion) without destroying the storage object and then recreating it. -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From akao347@hotmail.com Wed Jun 13 08:45:41 2001 Received: from hotmail.com (oe76.law4.hotmail.com [216.33.148.172]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA19974 for ; Wed, 13 Jun 2001 08:45:40 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 13 Jun 2001 11:43:48 -0700 X-Originating-IP: [151.203.34.140] From: "A K" To: References: <15143.42562.868686.728229@aragorn.stat.harvard.edu> Subject: Re: [Metakit] Deleting rows and Obtaining Views Date: Wed, 13 Jun 2001 14:44:43 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 13 Jun 2001 18:43:48.0945 (UTC) FILETIME=[C8BB7C10:01C0F438] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, This might not be the answer you want, but try calling Rollback. It makes sense that when you destroy the view, that the data in the Storage is still modified since you are suppose to detach your views before destroyng the Storage. Brendan. ----- Original Message ----- From: "John Barnard" To: Sent: Wednesday, June 13, 2001 1:43 PM Subject: RE: [Metakit] Deleting rows and Obtaining Views > Echeverria Rabi, Cristian Rodrigo, TRANSELEC S.A. writes: > > you need to do a commit() to save the changes until deleting the storage > > object > > in you case: > > > > test2.commit() > > That makes the row deletion permanent. I was trying to retrieve the > original view (without the row deletion) without destroying the > storage object and then recreating it. > > -- > John Barnard > Senior Research Statistician, deCODE Genetics > 124 Mount Auburn Str., Suite 200N > Cambridge, MA 02138 > Phone: (617) 520-6612 > Fax: (617) 576-5874 > Email: barnard@stat.harvard.edu > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From barnard@aragorn.stat.harvard.edu Wed Jun 13 09:02:09 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA20901 for ; Wed, 13 Jun 2001 09:02:09 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id PAA08092; Wed, 13 Jun 2001 15:10:59 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15143.47811.702754.104680@aragorn.stat.harvard.edu> Date: Wed, 13 Jun 2001 15:10:59 -0400 (EDT) To: "A K" Cc: Subject: Re: [Metakit] Deleting rows and Obtaining Views In-Reply-To: References: <15143.42562.868686.728229@aragorn.stat.harvard.edu> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Brendan, A K writes: > Hi, > > This might not be the answer you want, but try calling Rollback. > > It makes sense that when you destroy the view, that the data in the Storage > is still modified since you are suppose to detach your views before > destroyng the Storage. Thanks. That's what I wanted. Not sure how I overlooked it! John From akao347@hotmail.com Wed Jun 13 10:39:31 2001 Received: from hotmail.com (oe41.law4.hotmail.com [216.33.148.78]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA25470 for ; Wed, 13 Jun 2001 10:39:31 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 13 Jun 2001 13:37:40 -0700 X-Originating-IP: [151.203.38.197] From: "A K" To: "Metakit" References: <3B2753E1.3600.3E49835B@localhost> Date: Wed, 13 Jun 2001 16:38:47 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 13 Jun 2001 20:37:40.0458 (UTC) FILETIME=[B0A1A4A0:01C0F448] Subject: [Metakit] CustomViewer versus DerivedSequence Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am in the process of writing an application where it is necessary to notify changes to the "derived" views. It seems that there are two primary back-ends to a derived view. One is based off the CustomViewer and the other is based off DerivedSequence. It also seems that derived views based off a DerivedSequence will receive notification changes, whereas CustomViewer will not. Are these statements true? What are the reasons for doing one or the other? Would it be good if I ported the CustomViewers to DerivedSequences? Thank you, Brendan Kao. From akao347@hotmail.com Wed Jun 13 10:57:54 2001 Received: from hotmail.com (oe4.law4.hotmail.com [216.33.148.108]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA26477 for ; Wed, 13 Jun 2001 10:57:54 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 13 Jun 2001 13:56:03 -0700 X-Originating-IP: [151.203.38.197] From: "A K" To: "Metakit" References: <3B2753E1.3600.3E49835B@localhost> Subject: Re: [Metakit] CustomViewer versus DerivedSequence Date: Wed, 13 Jun 2001 16:57:10 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 13 Jun 2001 20:56:03.0453 (UTC) FILETIME=[421152D0:01C0F44B] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Oops, Correction to what I said. I mean that derived views based off DerivedSequence derived views will receive notification messages, where as if it were based off a CustomViewer derived view, it would not. Brendan ----- Original Message ----- From: "A K" To: "Metakit" Sent: Wednesday, June 13, 2001 4:38 PM Subject: [Metakit] CustomViewer versus DerivedSequence > Hi, > > I am in the process of writing an application where it is necessary to > notify changes to the "derived" views. It seems that there are two primary > back-ends to a derived view. One is based off the CustomViewer and the > other is based off DerivedSequence. It also seems that derived views based > off a DerivedSequence will receive notification changes, whereas > CustomViewer will not. Are these statements true? > > What are the reasons for doing one or the other? Would it be good if I > ported the CustomViewers to DerivedSequences? > > Thank you, > Brendan Kao. > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jerry@usd1.com Wed Jun 13 11:27:47 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA27817 for ; Wed, 13 Jun 2001 11:27:46 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f5DLPui18591; Wed, 13 Jun 2001 14:25:57 -0700 (PDT) Date: Wed, 13 Jun 2001 14:25:25 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <11719078322.20010613142525@usd1.com> To: "Gordon McMillan" CC: Metakit Subject: Re[2]: [Metakit] [python] In-reply-To: <3B2753E1.3600.3E49835B@localhost> References: <3B2753E1.3600.3E49835B@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Gordon, Wednesday, June 13, 2001, 8:52:01 AM, you wrote: GM> Jerry, GM> I tend to use MK for persistence, not batch, so I tend to GM> commit on every little change. But I can help with some of GM> these questions. >> I have been writing a batch processing system in Python based >> on MetaKit that will take a number of records (10K to 200k) >> from an outside source (flat, SQL, CSV), process it, then >> export selected fields to a different outside source. I have >> created dictionary driven classes for Records and DataSets. >> These classes view and modify the data in the tables directly, >> handling the multiple levels (sub-views). >> >> Question 1) I have read that it is more efficient to >> pre-allocate the row. How is that done with Python? GM> Not as efficient as the underlying C++, but you can build a GM> large view fairly efficiently by using repeated concatenation GM> for i in range(10): GM> v = v + v This looks like a real easy way to do this. Unfortunately, while it worked very fast when I tried it, the new view does not have new rows, but has 'pseudo rows', that all point to the same row: ## Example ---------->> >>> vw=dbt.getas('test[name,id,val:I]') >>> len(vw) 1 >>> vw=vw+vw >>> vw[1] >>> vw[1].name='bob' >>> dbt.commit() >>> len(vw) 2 >>> vw=vw+vw >>> vw.structure() [, ... >>> mk.dump(vw) name id val ---- -- --- bob 0 bob 0 bob 0 bob 0 ---- -- --- Total: 4 rows ### hmmm - I did not assign [0] to bob!? >>> vw[2].name='fred' >>> mk.dump(vw) name id val ---- -- --- fred 0 fred 0 fred 0 fred 0 ---- -- --- Total: 4 rows #-------------<< Likewise, if I try vw = vw * 8, I get eight rows, but without any properties. The del vw[x] command works great. Didn't think to try that. Thanks, Niki Thanks John for the cpp code. I don't have a C++ compiler nor the experience to use one. I use something like RowProxy and Wrapper, except I am doing it a bit differently. I might be able to modify it to use the commit() logic. The challenge I have is with nested rows and other complex interfaces. I will go into a little more detail. Keeping track of adding a new row to a nested view so it appears transparent, then doing a commit() to commit the 'deepcopy' of the structure would be more of a challenge. I keep a separate record layout structure that allows me to use specific classes on each and every column. This allows a very complex, dictionary driven interface. One of my goals to to allow the user to modify and/or create fields as he/she wishes using scripting. r=dfile[1] # both r and dfile are wrapper classes print r.earn # prints 10050 directly from the underlying PyRow print r._earn # prints $100.50, using a row class wrapper r.earn += 1 # sets the internal value directly, now 10051 r._earn += 1 # sets the value to 10151. The class knows scale of 2 print r._totalEarn # would print a total based on a virtual field. Basically just an eval of a string I store in the dictionary. Another dictionary type is a 'code' type, where I assign strings to numbers, but only store the number. The PyProperty will be an "I". Example r.service # 2 r._service # 'good' r._service = 'excellent' r.service # 4 r._service > 'good' # true, compares internal values r._service +=1 r._service # 'outstanding' Now the more complicated dictionary interfaces -- handling nested views. Using the getas string fragment 'pay[code,val:I]' I allow one dictionary entry to handle this. (Actually codeF('pay',1) in the dictionary layout handles it, the '1' says the value portion will be a money class). So now we can do... r.pay #PyView object at .... len(r.pay) # 0 r._pay # r._pay['E']=100 # adds new row, code='E', val=100 r._pay._NW=2 # adds new row, code="NW", val=200 r._pay._NW+=1 # adds 1 to row with code="NW", now val = 300 r._pay('NW').val # 300 r._pay('NW')._val # r._pay.sum() # 400 - with class interfaces I can get the sum, first, last, next (for iterating), keys, values, items, and clearit to clear whole nested structure. r._pay.keys() # ['E','NW'] Since the fields and sub-fields can be just about anything, storing all the changed values somewhere else until committed is another big challenge. [snip snip] GM> - Gordon Q1) in the above example, on PyWin the properties list as: [, ... at the interactive prompt, the same command ( vw.structure() ) I get [Property('I', 'iid'), Property( ... Does anyone know why? Q2) the primary reason for choosing MetaKet was the nested views. When I am adding 100,000 records, it seems from the mailing list that it will be faster to create a view without the nested views, add to that, then re-structure. Will this save me some time? Wish 1) method to import / export Python structure direct to/from a PyRow. Maybe by exposing a __dict__ interface, including the dictionary methods. Wish 3) have vw = vw * 1000 create 1000 new rows in current view -- Best regards, Jerry From gmcm@hypernet.com Wed Jun 13 14:15:43 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA00560 for ; Wed, 13 Jun 2001 14:15:42 -0500 Received: from me (204.176.40.132) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Wed, 13 Jun 2001 20:13:07 -0400 From: "Gordon McMillan" To: Jerry McRae Date: Wed, 13 Jun 2001 20:13:03 -0400 Subject: Re: Re[2]: [Metakit] [python] Reply-to: gmcm@hypernet.com CC: Metakit Message-ID: <3B27C94F.23905.13D31FF@localhost> Priority: normal In-reply-to: <11719078322.20010613142525@usd1.com> References: <3B2753E1.3600.3E49835B@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, > GM> Not as efficient as the underlying C++, but you can build a > GM> large view fairly efficiently by using repeated concatenation > GM> for i in range(10): GM> v = v + v > > This looks like a real easy way to do this. Unfortunately, while > it worked very fast when I tried it, the new view does not have > new rows, but has 'pseudo rows', that all point to the same row: Not exactly. They're all copies of the same row. >>> v = s.getas('test[name:S,shoesize:I]') >>> v.append({}) 0 >>> for i in range(3): v= v + v ... >>> len(v) 8 >>> v[3].name='fred' >>> v[7].name '' > Likewise, if I try vw = vw * 8, I get eight rows, but without > any properties. Yeah. I don't know what's wrong there. I guess I'm to blame, because that looks like my original code. My excuse is that I hadn't used the C++ interface to MetaKit when I wrote the wrapper, so I really didn't know what the calls did when I wrapped them. > I use something like RowProxy and Wrapper, except I am doing it > a bit differently. I might be able to modify it to use the > commit() logic. The challenge I have is with nested rows and > other complex interfaces. I struggled for a time with trying to make it so you could pass a Python object with an attribute that was a list, and have the list populate the corresponding subview. But the C++ API doesn't work that way, and I gave up. In using Mk4py (extensively), I really haven't missed it much. There are a lot of habits from RDB (and ODB) that don't work with MK. Fairly often I have to poke around before I figure out what will work with MK. OTOH, the lack of clutter (and the speed, and the compactness) make it all worthwhile. I know Jean-Claude is working (off & on) on a revised engine that should allow writing a cleaner and more efficient Python interface. ... > GM> - Gordon > > Q1) in the above example, on PyWin the properties list as: > [, ... > at the interactive prompt, the same command ( vw.structure() > ) I get > [Property('I', 'iid'), Property( ... > Does anyone know why? I think PythonWin uses repr where the interpreter calls the print slot (or maybe it's vice versa). > Q2) the primary reason for choosing MetaKet was the nested views. > When > I am adding 100,000 records, it seems from the mailing list > that it will be faster to create a view without the nested > views, add to that, then re-structure. Will this save me > some time? Sorry, I can't answer that. The only MK app I have that has counts approaching that doesn't use subviews. > Wish 1) method to import / export Python structure direct to/from > a PyRow. > Maybe by exposing a __dict__ interface, including the > dictionary methods. I don't know why you're asking for that. A row has attributes, gotten through the standard getattr mechanism. Setting a row from a Python object will see if the object is a dictionary, then it will try list (not recommended, since it's not very safe), and then will try sucking attributes from whatever it is. The only thing I see here that's non-Python is that you have to ask the view what the properties of the row are. > Wish 3) have vw = vw * 1000 create 1000 new rows in current view Unfortunately, I still don't understand the C++ API well enough to know why the properties get lost. - Gordon From jerry@usd1.com Wed Jun 13 18:52:33 2001 Received: from asteroid.pacifier.com (asteroid.pacifier.com [199.2.117.154]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA04146 for ; Wed, 13 Jun 2001 18:52:32 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id f5E4ohM17297; Wed, 13 Jun 2001 21:50:43 -0700 (PDT) Date: Wed, 13 Jun 2001 21:50:09 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <445766549.20010613215009@usd1.com> To: "Gordon McMillan" CC: Metakit Subject: Re[4]: [Metakit] [python] In-reply-To: <3B27C94F.23905.13D31FF@localhost> References: <3B2753E1.3600.3E49835B@localhost> <3B27C94F.23905.13D31FF@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Gordon, Thanks for the reply. The append with __add__ still doesn't work for me. See below. Wednesday, June 13, 2001, 5:13:03 PM, you wrote: GM> Jerry, >> GM> Not as efficient as the underlying C++, but you can build a >> GM> large view fairly efficiently by using repeated concatenation >> GM> for i in range(10): GM> v = v + v >> >> This looks like a real easy way to do this. Unfortunately, while >> it worked very fast when I tried it, the new view does not have >> new rows, but has 'pseudo rows', that all point to the same row: GM> Not exactly. They're all copies of the same row. >>>> v = s.getas('test[name:S,shoesize:I]') >>>> v.append({}) GM> 0 >>>> for i in range(3): v= v + v GM> ... >>>> len(v) GM> 8 >>>> v[3].name='fred' >>>> v[7].name GM> '' *** I tried this exact code both on PyWin and Python (version 2.1 (#15 April 16 2001)) and I get the results I stated earlier, any change to any row changes them all. I tried with the v.append({}) and the way I usually do it, v.append(). >> Likewise, if I try vw = vw * 8, I get eight rows, but without >> any properties. GM> Yeah. I don't know what's wrong there. I guess I'm to blame, GM> because that looks like my original code. My excuse is that I GM> hadn't used the C++ interface to MetaKit when I wrote the GM> wrapper, so I really didn't know what the calls did when I GM> wrapped them. >> I use something like RowProxy and Wrapper, except I am doing it >> a bit differently. I might be able to modify it to use the >> commit() logic. The challenge I have is with nested rows and >> other complex interfaces. GM> I struggled for a time with trying to make it so you could pass GM> a Python object with an attribute that was a list, and have the GM> list populate the corresponding subview. But the C++ API GM> doesn't work that way, and I gave up. In using Mk4py GM> (extensively), I really haven't missed it much. GM> GM> There are a lot of habits from RDB (and ODB) that don't work GM> with MK. Fairly often I have to poke around before I figure out GM> what will work with MK. OTOH, the lack of clutter (and the GM> speed, and the compactness) make it all worthwhile. GM> GM> I know Jean-Claude is working (off & on) on a revised engine GM> that should allow writing a cleaner and more efficient Python GM> interface. GM> ... >> GM> - Gordon >> >> Q1) in the above example, on PyWin the properties list as: >> [, ... >> at the interactive prompt, the same command ( vw.structure() >> ) I get >> [Property('I', 'iid'), Property( ... >> Does anyone know why? GM> I think PythonWin uses repr where the interpreter calls the GM> print slot (or maybe it's vice versa). *** 'tis an odd an curious thing. Both interpreters operate as expected on most classes and builtins. In PyWin (repr or str) it quotes "pyProperty object at", and Python command line (repr or str) it always displays the more useful "Property('I',...". Things that make ya go hmmmm. >> Q2) the primary reason for choosing MetaKet was the nested views. >> When >> I am adding 100,000 records, it seems from the mailing list >> that it will be faster to create a view without the nested >> views, add to that, then re-structure. Will this save me >> some time? GM> Sorry, I can't answer that. The only MK app I have that has GM> counts approaching that doesn't use subviews. >> Wish 1) method to import / export Python structure direct to/from >> a PyRow. >> Maybe by exposing a __dict__ interface, including the >> dictionary methods. GM> I don't know why you're asking for that. A row has attributes, GM> gotten through the standard getattr mechanism. Setting a row GM> from a Python object will see if the object is a dictionary, then GM> it will try list (not recommended, since it's not very safe), and GM> then will try sucking attributes from whatever it is. *** access to dictionary methods keys, items, values, has_attr would be useful. Right now if I pas a PyRowRef to anything, I ALWAYS have to pass more information, because it loses all context. There is no way, AFAIK, to know what attribute names are available, valid, or even how many of them there are. Nor do I know what view it is a member of or the index. I end up always passing the view and a dictionary to all my wrapper classes so I can verify the field (row) names, since the whole point to my project is that the user can create all but a small subset of rows on their own. GM> The only thing I see here that's non-Python is that you have to GM> ask the view what the properties of the row are. >> Wish 3) have vw = vw * 1000 create 1000 new rows in current view GM> Unfortunately, I still don't understand the C++ API well enough GM> to know why the properties get lost. GM> - Gordon -- Best regards, Jerry From pvision@planet-d.net Wed Jun 13 22:39:01 2001 Received: from relais-int8.globalintranet.net (mailgate.globalintranet.net [194.206.181.244]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA06458 for ; Wed, 13 Jun 2001 22:39:00 -0500 Received: from STARGATE ([10.255.7.37]) by relais-int8.globalintranet.net (Netscape Messaging Server 4.15) with ESMTP id GEWVXK00.V2Q for ; Thu, 14 Jun 2001 10:36:56 +0200 Received: from m2881 ([10.228.8.190]) by STARGATE (Lotus Domino Release 5.07a) with SMTP id 2001061410375650:1027 ; Thu, 14 Jun 2001 10:37:56 +0200 Message-ID: <000801c0f4ae$1a328820$be08e40a@m2881> From: "Olivier Brunet" To: Date: Thu, 14 Jun 2001 10:43:27 +0200 MIME-Version: 1.0 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-MIMETrack: Itemize by SMTP Server on stargate/DMZ(Release 5.07a |May 14, 2001) at 14/06/2001 10:37:56, Serialize by Router on stargate/DMZ(Release 5.07a |May 14, 2001) at 14/06/2001 10:37:57, Serialize complete at 14/06/2001 10:37:57 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_01C0F4BE.D849C250" Subject: [Metakit] Threading Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: C'est un message de format MIME en plusieurs parties. ------=_NextPart_000_0005_01C0F4BE.D849C250 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" =20 After reading the Wiki, I'm still not clear about some threading issues. = Using Metakit C++ 2.3.4 on Windows (and after on Linux), I have N = threads who needs to access and update 2 views in one storage. Is this = supported right out of the box ? Or if not what should be added ? And if = yes how ? Currently I have tried 2 ways: 15 threads, opening N times same storage (N<=3D15), getting views and = updating and commiting, seemed to work (no crashes) but seemed to = "forget" some updates on view 15 threads, all sharing the same storage objects, views (global = variables), and doing update and commits =3D> random crashes. Enlighgt me please. ------=_NextPart_000_0005_01C0F4BE.D849C250 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="iso-8859-1"
   
After reading the Wiki, I'm still not = clear about=20 some threading issues. Using Metakit C++ 2.3.4 on Windows (and after on = Linux),=20 I have N threads who needs to access and update 2 views in one storage. = Is this=20 supported right out of the box ? Or if not what should be added ? And if = yes how=20 ?
 
Currently I have tried 2 = ways:
 
15 threads, opening N times same = storage=20 (N<=3D15), getting views and updating and commiting, seemed to work = (no=20 crashes) but seemed to "forget" some updates on view
 
15 threads, all sharing the same = storage objects,=20 views (global variables), and doing update and commits =3D> random=20 crashes.
 
Enlighgt me = please.
------=_NextPart_000_0005_01C0F4BE.D849C250-- From akao347@hotmail.com Thu Jun 14 02:54:24 2001 Received: from hotmail.com (oe35.law4.hotmail.com [216.33.148.28]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA10811 for ; Thu, 14 Jun 2001 02:54:24 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Thu, 14 Jun 2001 05:52:31 -0700 X-Originating-IP: [151.203.38.197] From: "A K" To: References: <000801c0f4ae$1a328820$be08e40a@m2881> Subject: Re: [Metakit] Threading Date: Thu, 14 Jun 2001 08:53:35 -0400 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_005B_01C0F4AF.7F399C30" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 14 Jun 2001 12:52:31.0078 (UTC) FILETIME=[DFC1F860:01C0F4D0] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_005B_01C0F4AF.7F399C30 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, Well all the emails this week and the wiki web site seem to say that the = proper way to do this is to give only one thread access to the database, = and have all the threads post to this one thread for data. Brendan Kao ----- Original Message -----=20 From: Olivier Brunet=20 To: metakit@equi4.com=20 Sent: Thursday, June 14, 2001 4:43 AM Subject: [Metakit] Threading =20 After reading the Wiki, I'm still not clear about some threading = issues. Using Metakit C++ 2.3.4 on Windows (and after on Linux), I have = N threads who needs to access and update 2 views in one storage. Is this = supported right out of the box ? Or if not what should be added ? And if = yes how ? Currently I have tried 2 ways: =20 15 threads, opening N times same storage (N<=3D15), getting views and = updating and commiting, seemed to work (no crashes) but seemed to = "forget" some updates on view 15 threads, all sharing the same storage objects, views (global = variables), and doing update and commits =3D> random crashes. Enlighgt me please. ------=_NextPart_000_005B_01C0F4AF.7F399C30 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi,
 
Well all the emails this week and the = wiki web site=20 seem to say that the proper way to do this is to give only one thread = access to=20 the database, and have all the threads post to this one thread for=20 data.
 
 
Brendan Kao
----- Original Message -----
From:=20 Olivier=20 Brunet
Sent: Thursday, June 14, 2001 = 4:43=20 AM
Subject: [Metakit] = Threading

   
After reading the Wiki, I'm still not = clear about=20 some threading issues. Using Metakit C++ 2.3.4 on Windows (and after = on=20 Linux), I have N threads who needs to access and update 2 views in one = storage. Is this supported right out of the box ? Or if not what = should be=20 added ? And if yes how ?
 
Currently I have tried 2 = ways:
 
15 threads, opening N times same = storage=20 (N<=3D15), getting views and updating and commiting, seemed to work = (no=20 crashes) but seemed to "forget" some updates on view
 
15 threads, all sharing the same = storage objects,=20 views (global variables), and doing update and commits =3D> random=20 crashes.
 
Enlighgt me=20 please.
------=_NextPart_000_005B_01C0F4AF.7F399C30-- From gmcm@hypernet.com Thu Jun 14 05:14:06 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA14054 for ; Thu, 14 Jun 2001 05:14:05 -0500 Received: from me (204.176.40.87) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 14 Jun 2001 11:11:26 -0400 From: "Gordon McMillan" To: Jerry McRae Date: Thu, 14 Jun 2001 11:11:23 -0400 Subject: Re: Re[4]: [Metakit] [python] Reply-to: gmcm@hypernet.com CC: Metakit Message-ID: <3B289BDB.6377.473A746@localhost> Priority: normal In-reply-to: <445766549.20010613215009@usd1.com> References: <3B27C94F.23905.13D31FF@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, > Thanks for the reply. The append with __add__ still doesn't work > for me. See below. Then it looks like a bug crept in. I'll try to have a look at CVS and see if I can track it down. [snip] > >> Wish 1) method to import / export Python structure direct > >> to/from a PyRow. > >> Maybe by exposing a __dict__ interface, including the > >> dictionary methods. There are a bunch of issues here, some of them MetaKit's, some Python's. The current row.attr syntax is both very nice and very efficient. It's efficient because it's lazy. In my MK usage, I do a lot of RDB-ish things (searching etc.), normally involving only a few columns. MK shines in that respect, because it's column oriented. Mk4py would be dog slow if it didn't maintain that laziness. So a real Python dict is out. There's also a namespace issue: in a Python type object, methods and attributes live in the same namespace. If a PyRowRef had methods, they'd conflict with column names (actually column names would win). Conceivably, a PyRowRef could implement either the sequence interface (so row[i] would access the ith property) or the mapping interface (so row[i] would access the property named "i"). Python makes doing both interfaces (together) very, very difficult. And the mapping interface is not complete (you need to implement certain methods to make the object really look like a dict - the slot only contains the equivalent of __len__, __getitem__ and __setitem__). Both "items" and "values" are certainly not outlandish names for columns (I've used "values" for a subview), so I think the mapping interface is out. The seq interface is more self contained, but not that good a match. What is row[3]? The value of the 3rd property? The 3rd property (as in PyProperty object)? A tuple (property, value)? You can only assign to the value so only the first choice makes sense. Many seq methods make no sense (slicing? concat? repeat?). So even if implemented, it wouldn't help you. That leaves (unless I've missed something) a (lazily created) special attribute named "__dict__" that does implement the mapping protocol and methods. Maybe I'm being lazy, but that sounds like a lot of work :-). Even the issue of what view does a row belong to (and what's it's index) is not simple. Consider: v = db.view('customers') v2 = v.select(shoesize=48) # European sizes, of course r = v2[0] Now r is a ref to v2[0], but v2 is not a "real" view. It's a wrapper around v, where r has some comletely different index. If you're just reading, you probably care about v2, but if you're trying to update, you're interested in r's position in v. MetaKit is opaque on these issues, but even if it weren't, I don't know how to do a sane API that handles these issues (this one is one-deep; but you can chain this stuff up to arbitrary depths). Jean-Claude has a proof-of-concept of a new engine, which should let Mk4py reach a little deeper into the workings. He's too busy right now to reply even in monosyllabic grunts, (and typically, when he's not busy, I am), so don't take that as anything more than a glimmer of hope :-). - Gordon From barnard@aragorn.stat.harvard.edu Thu Jun 14 06:37:04 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA16896 for ; Thu, 14 Jun 2001 06:37:04 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id MAA12609; Thu, 14 Jun 2001 12:46:02 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15144.59973.685666.665537@aragorn.stat.harvard.edu> Date: Thu, 14 Jun 2001 12:45:57 -0400 (EDT) To: gmcm@hypernet.com Cc: Jerry McRae , Metakit Subject: Re: Re[4]: [Metakit] [python] In-Reply-To: <3B289BDB.6377.473A746@localhost> References: <3B27C94F.23905.13D31FF@localhost> <3B289BDB.6377.473A746@localhost> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gordon McMillan writes: > > >> Wish 1) method to import / export Python structure direct > > >> to/from a PyRow. > > >> Maybe by exposing a __dict__ interface, including the > > >> dictionary methods. > > There are a bunch of issues here, some of them MetaKit's, > some Python's. The current row.attr syntax is both > very nice and very efficient. > It's efficient because it's lazy. In my MK usage, I do a lot of > RDB-ish things (searching etc.), normally involving only a few > columns. MK shines in that respect, because it's column > oriented. Mk4py would be dog slow if it didn't maintain that > laziness. So a real Python dict is out. Agreed. > There's also a namespace issue: in a Python type object, > methods and attributes live in the same namespace. If a > PyRowRef had methods, they'd conflict with column names > (actually column names would win). > > Conceivably, a PyRowRef could implement either the > sequence interface (so row[i] would access the ith property) or > the mapping interface (so row[i] would access the property > named "i"). Python makes doing both interfaces (together) > very, very difficult. And the mapping interface is not complete > (you need to implement certain methods to make the object > really look like a dict - the slot only contains the equivalent of > __len__, __getitem__ and __setitem__). I'm missing something. It would be relatively easy to provide a C function for getitem in the mapping protocol that retrieves the ith element of a row when i is an integer, the element corresponding to the property name i when i is of type string, and the element corresponding to the property i when i is of type PyProperty. I'll provide such a function tomorrow. You're essentially implementing the get/set part of the mapping protocol with special behavior depending on the type of the key -- the sequence protocol only allows you to pass integers to get/set. You could still use the row.propname way to get and set values in addition to the new ways. > Both "items" and "values" are certainly not outlandish names for > columns (I've used "values" for a subview), so I think the mapping > interface is out. Again, I'm missing something. If I add a __getitem__ and __setitem__ in C, I'm not using any names like "items" or "values". You can name the properties anything you want. > The seq interface is more self contained, but not that good a > match. What is row[3]? The value of the 3rd property? The 3rd > property (as in PyProperty object)? A tuple (property, value)? > You can only assign to the value so only the first choice > makes sense. Many seq methods make no sense (slicing? > concat? repeat?). So even if implemented, it wouldn't help you. Slicing makes sense -- row[2:4] gives me a row with elements 2 and 3. Granted concat and repeat probably don't, but the beauty of Python is you don't have to implement them. If all we want is to be able to do row[i] with i a number, then we can skip the rest of the sequence protocol. > That leaves (unless I've missed something) a (lazily created) > special attribute named "__dict__" that does implement the > mapping protocol and methods. Maybe I'm being lazy, but that > sounds like a lot of work :-). > > Even the issue of what view does a row belong to (and what's > it's index) is not simple. Consider: > > v = db.view('customers') > v2 = v.select(shoesize=48) # European sizes, of course > r = v2[0] > > Now r is a ref to v2[0], but v2 is not a "real" view. It's a wrapper > around v, where r has some comletely different index. If you're > just reading, you probably care about v2, but if you're trying to > update, you're interested in r's position in v. MetaKit is opaque > on these issues, but even if it weren't, I don't know how to do a > sane API that handles these issues (this one is one-deep; but > you can chain this stuff up to arbitrary depths). Agreed, however, this linkage to arbitrary depths is very powerful. I've been trying to handle some these issues by writing Python class wrappers around PyView and PyRowRef so that I can manage the lifetimes of "child" (derived/linked) views. For example, if I insert a new row into a "master" view, all of its children views are invalidated. It took me awhile to get the wrapper classes the way I wanted, but that's because I kept changing the design! John -- John Barnard Senior Research Statistician, deCODE Genetics 124 Mount Auburn Str., Suite 200N Cambridge, MA 02138 Phone: (617) 520-6612 Fax: (617) 576-5874 Email: barnard@stat.harvard.edu From gmcm@hypernet.com Thu Jun 14 08:49:53 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA20350 for ; Thu, 14 Jun 2001 08:49:52 -0500 Received: from me (204.176.40.91) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 14 Jun 2001 14:47:11 -0400 From: "Gordon McMillan" To: John Barnard Date: Thu, 14 Jun 2001 14:47:09 -0400 Subject: Re: Re[4]: [Metakit] [python] Reply-to: gmcm@hypernet.com CC: Jerry McRae , Metakit Message-ID: <3B28CE6D.7885.5392E5C@localhost> Priority: normal In-reply-to: <15144.59973.685666.665537@aragorn.stat.harvard.edu> References: <3B289BDB.6377.473A746@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John, [me (Gordon)] > > There's also a namespace issue: in a Python type object, > > methods and attributes live in the same namespace. If a > > PyRowRef had methods, they'd conflict with column names > > (actually column names would win). > > Conceivably, a PyRowRef > could implement either the > sequence interface (so row[i] would > access the ith property) or > the mapping interface (so row[i] > would access the property > named "i"). Python makes doing both > interfaces (together) > very, very difficult. And the mapping > interface is not complete > (you need to implement certain > methods to make the object > really look like a dict - the slot > only contains the equivalent of > __len__, __getitem__ and > __setitem__). > > I'm missing something. It would be relatively easy to provide a C > function for getitem in the mapping protocol that retrieves the > ith element of a row when i is an integer, the element > corresponding to the property name i when i is of type string, > and the element corresponding to the property i when i is of type > PyProperty. I'll provide such a function tomorrow. You're > essentially implementing the get/set part of the mapping protocol > with special behavior depending on the type of the key -- the > sequence protocol only allows you to pass integers to get/set. > You could still use the row.propname way to get and set values in > addition to the new ways. Hmm. I think you can get row[3] to work, but you can't get for prop in row: to work (that requires the seq protocol). > > Both "items" and "values" are certainly not outlandish names > for > columns (I've used "values" for a subview), so I think the > mapping > interface is out. > > Again, I'm missing something. If I add a __getitem__ and > __setitem__ in C, I'm not using any names like "items" or > "values". You can name the properties anything you want. But what Jerry *wants* is row.items(), or row.keys(). Those are methods, not slots. He wants to be able to get to the property names through the row instead of the view. Aside from the messy implementation details, the properties belong to the view, not the row, so there are semantic issues as well. > > The seq interface is more self contained, but not that good a > > match. What is row[3]? The value of the 3rd property? The 3rd > > property (as in PyProperty object)? A tuple (property, value)? > > You can only assign to the value so only the first choice > > makes sense. Many seq methods make no sense (slicing? > concat? > repeat?). So even if implemented, it wouldn't help you. > > Slicing makes sense -- row[2:4] gives me a row with elements 2 > and 3. Or maybe it returns a list [prop2, prop3]. If it returned a row, you'd really be getting a rowref to the only row in a temporary view (with data copied into it). So maybe a list [value2, value3] makes more sense. At times like these, I'm glad Guido is the language desinger, not me :-). > Granted concat and repeat probably don't, but the beauty > of Python is you don't have to implement them. If all we want is > to be able to do row[i] with i a number, then we can skip the > rest of the sequence protocol. > > > That leaves (unless I've missed something) a (lazily created) > > special attribute named "__dict__" that does implement the > > mapping protocol and methods. Maybe I'm being lazy, but that > > sounds like a lot of work :-). > > Even the issue of what view > does a row belong to (and what's > it's index) is not simple. > Consider: > > v = db.view('customers') > v2 = > v.select(shoesize=48) # European sizes, of course > r = v2[0] > > > Now r is a ref to v2[0], but v2 is not a "real" view. It's a > wrapper > around v, where r has some comletely different index. > If you're > just reading, you probably care about v2, but if > you're trying to > update, you're interested in r's position in > v. MetaKit is opaque > on these issues, but even if it weren't, > I don't know how to do a > sane API that handles these issues > (this one is one-deep; but > you can chain this stuff up to > arbitrary depths). > > Agreed, however, this linkage to arbitrary depths is very > powerful. I've been trying to handle some these issues by writing > Python class wrappers around PyView and PyRowRef so that I can > manage the lifetimes of "child" (derived/linked) views. For > example, if I insert a new row into a "master" view, all of its > children views are invalidated. It took me awhile to get the > wrapper classes the way I wanted, but that's because I kept > changing the design! If those wrappers point the way to a sane API, I'd love to see them! - Gordon From barnard@aragorn.stat.harvard.edu Fri Jun 15 09:36:51 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA15552 for ; Fri, 15 Jun 2001 09:36:51 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id PAA17654; Fri, 15 Jun 2001 15:45:51 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="mMALseFBLG" Content-Transfer-Encoding: 7bit Message-ID: <15146.26068.846358.125463@aragorn.stat.harvard.edu> Date: Fri, 15 Jun 2001 15:45:24 -0400 (EDT) To: gmcm@hypernet.com Cc: John Barnard , Jerry McRae , Metakit Subject: Re: Re[4]: [Metakit] [python] In-Reply-To: <3B28CE6D.7885.5392E5C@localhost> References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --mMALseFBLG Content-Type: text/plain; charset=us-ascii Content-Description: message body text Content-Transfer-Encoding: 7bit Inspired by the recent discussions, I've made several additions to PyRowRef.cpp. The modified version is attached below. With this modification you now do the following (row is of type PyRowRef): row["var1"] # equivalent to row.var1 row[i] # returns ith element of row, where i is an integer row[i:j] # returns a list containing the elements i through j, where i # and j are integers row[:] # returns all elements of row in a list # can use the usual slice notation, e.g., -4:-2, etc to get # corresponding elements in a list row[i:j:k] # returns elements of the row corresponding to the slice # i:j:k, where k is the step size (extended slice notation) # in a list (i or j or k can be missing) Using same indices as above but with ,1 appended returns the corresponding properties instead of the elements. E.g., row[:,1] returns a list of all of the properties "for" the row, and row["var1",1] returns the property for var1. In fact, it doesn't have to ,1 it can be ,anything (e.g. row[:,10.2]) -- I use the presence of a second index as a flag to return PyProperties instead of elements. It's not as elegant as row.keys(), but it avoids any name conflicts. You can also set elements of rows using the indices: row["var1"] = 10 row[2:4] = ['a', 'b'] # must be a sequence with length >= 2 row[2:6:2] = (1,4) # must be a sequence with length >= 2 row[:] = x # x is a sequence with length >= length of row Note that row[:,1] = x raises as error since :,1 is a passed to the code as a tuple and a tuple index is not supported for setting. If people find this useful, I suggest it be added to the next release of metakit. Best, John --mMALseFBLG Content-Type: text/plain Content-Description: PyRowRef.cpp Content-Disposition: inline; filename="PyRowRef.cpp" Content-Transfer-Encoding: base64 Ly8gIENvcHlyaWdodCAxOTk5IE1jTWlsbGFuIEVudGVycHJpc2VzLCBJbmMuIC0tIHd3dy5t Y21pbGxhbi1pbmMuY29tDQ0KLy8gIENvcHlyaWdodCAoQykgMTk5OS0yMDAwIEplYW4tQ2xh dWRlIFdpcHBsZXIgPGpjd0BlcXVpNC5jb20+DQ0KLy8NDQovLyAgUm93UmVmIGNsYXNzIGlt cGxlbWVudGF0aW9uDQ0KLy8gIE1vZGlmaWVkIGJ5IEouIEJhcm5hcmQgDQ0KDQ0KI2luY2x1 ZGUgPFBXT1NlcXVlbmNlLmg+DQ0KI2luY2x1ZGUgPFBXT051bWJlci5oPg0NCiNpbmNsdWRl ICJQeVJvd1JlZi5oIg0NCiNpbmNsdWRlICJQeVZpZXcuaCINDQojaW5jbHVkZSAiUHlQcm9w ZXJ0eS5oIg0NCg0NCnN0YXRpYyBQeU1ldGhvZERlZiBSb3dSZWZNZXRob2RzW10gPSB7DQ0K ICAgIHswLCAwLCAwLCAwfQ0NCn07DQ0KDQ0Kc3RhdGljIHZvaWQgUHlSb3dSZWZfZGVhbGxv YyhQeVJvd1JlZiAqbykgew0NCiAgICAvL28tPn5QeVJvd1JlZigpOw0NCiAgICBkZWxldGUg bzsNDQp9DQ0KDQ0Kc3RhdGljIGludCBQeVJvd1JlZl9wcmludChQeVJvd1JlZiAqbywgRklM RSAqZiwgaW50KSB7DQ0KICAgIGZwcmludGYoZiwgIjxQeVJvd1JlZiBvYmplY3QgYXQgJXg+ IiwgKGludClvKTsNDQogICAgcmV0dXJuIDA7DQ0KfQ0NCg0NCnN0YXRpYyBQeU9iamVjdCog UHlSb3dSZWZfZ2V0YXR0cihQeVJvd1JlZiAqbywgY2hhciAqbm0pIHsNDQogICAgdHJ5IHsN DQogICAgICAgIFB5T2JqZWN0KiBhdHRyID0gby0+Z2V0UHJvcGVydHlWYWx1ZShubSk7DQ0K ICAgICAgICBpZiAoYXR0cikNDQogICAgICAgICAgICByZXR1cm4gYXR0cjsNDQogICAgfQ0N CiAgICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgICAgICByZXR1cm4gZS50b1B5dGhv bigpOw0NCiAgICB9DQ0KICAgIHJldHVybiBQeV9GaW5kTWV0aG9kKFJvd1JlZk1ldGhvZHMs IChQeU9iamVjdCogKW8sIG5tKTsNDQp9DQ0KDQ0Kc3RhdGljIGludCBQeVJvd1JlZl9zZXRh dHRyKFB5Um93UmVmICpvLCBjaGFyICpubSwgUHlPYmplY3QqIHYpIHsNDQogICAgdHJ5IHsN DQogICAgICAgIFB5UHJvcGVydHkgKnAgPSBvLT5nZXRQcm9wZXJ0eShubSk7DQ0KICAgICAg ICBpZiAocCkgew0NCiAgICAgICAgICAgIGlmICh2KQ0NCiAgICAgICAgICAgICAgICBQeVJv d1JlZjo6c2V0RnJvbVB5dGhvbigqbywgKnAsIHYpOw0NCiAgICAgICAgICAgIGVsc2UgDQ0K ICAgICAgICAgICAgICAgIFB5Um93UmVmOjpzZXREZWZhdWx0KCpvLCAqcCk7DQ0KICAgICAg ICAgICAgUHlfREVDUkVGKHApOw0NCiAgICAgICAgICAgIHJldHVybiAwOw0NCiAgICAgICAg fQ0NCiAgICAgICAgUHlFcnJfU2V0U3RyaW5nKFB5RXhjX0F0dHJpYnV0ZUVycm9yLCAiZGVs ZXRlIG9mIG5vbi1leGlzdGluZyBhdHRyaWJ1dGUiKTsNDQogICAgICAgIHJldHVybiAtMTsN DQogICAgfQ0NCiAgICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgICAgICBlLnRvUHl0 aG9uKCk7DQ0KICAgICAgICByZXR1cm4gLTE7DQ0KICAgIH0NDQp9DQ0KDQ0Kc3RhdGljIGlu dCBQeVJvd19sZW5ndGgoUHlSb3dSZWYgKm8pIHsNDQogIHJldHVybiBvLT5Db250YWluZXIo KS5OdW1Qcm9wZXJ0aWVzKCk7DQ0KfQ0NCg0NCnN0YXRpYyBQeU9iamVjdCogUHlSb3dfZ2V0 aXRlbShQeVJvd1JlZiAqbywgaW50IG4pIHsNDQogICAgdHJ5IHsNDQogICAgICBpZiAobj49 MCAmJiBuIDwgby0+Q29udGFpbmVyKCkuTnVtUHJvcGVydGllcygpKSB7DQ0KCXJldHVybiBv LT5hc1B5dGhvbihvLT5Db250YWluZXIoKS5OdGhQcm9wZXJ0eShuKSk7DQ0KICAgICAgfQ0N CiAgICAgIHRocm93IFBXRXhjZXB0aW9uKFB5RXhjX0luZGV4RXJyb3IsICJJbmRleCBvdXQg b2YgcmFuZ2UiKTsNDQogICAgfQ0NCiAgICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAg ICAgICByZXR1cm4gZS50b1B5dGhvbigpOw0NCiAgICB9DQ0KfQ0NCg0NCnN0YXRpYyBpbnQg UHlSb3dfc2V0aXRlbShQeVJvd1JlZiAqbywgaW50IG4sIFB5T2JqZWN0KiB2KSB7DQ0KICAg IHRyeSB7DQ0KICAgICAgaWYgKG4+PTAgJiYgbiA8IG8tPkNvbnRhaW5lcigpLk51bVByb3Bl cnRpZXMoKSkgew0NCgljb25zdCBjNF9Qcm9wZXJ0eSBwID0gby0+Q29udGFpbmVyKCkuTnRo UHJvcGVydHkobik7DQ0KCWlmICh2KQ0NCgkgIFB5Um93UmVmOjpzZXRGcm9tUHl0aG9uKCpv LCBwLCB2KTsNDQoJZWxzZSANDQoJICBQeVJvd1JlZjo6c2V0RGVmYXVsdCgqbywgcCk7DQ0K CXJldHVybiAwOw0NCiAgICAgIH0NDQogICAgICBQeUVycl9TZXRTdHJpbmcoUHlFeGNfSW5k ZXhFcnJvciwgIkluZGV4IG91dCBvZiByYW5nZSIpOw0NCiAgICAgIHJldHVybiAtMTsNDQog ICAgfQ0NCiAgICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgICAgICBlLnRvUHl0aG9u KCk7DQ0KICAgICAgICByZXR1cm4gLTE7DQ0KICAgIH0NDQp9DQ0KDQ0Kc3RhdGljIFB5T2Jq ZWN0KiBQeVJvd19nZXRzbGljZShQeVJvd1JlZiAqbywgaW50IGlsb3csIGludCBpaGlnaCkg ew0NCiAgdHJ5IHsNDQogICAgaW50IGwgPSBvLT5Db250YWluZXIoKS5OdW1Qcm9wZXJ0aWVz KCk7DQ0KICAgIGlmIChpaGlnaCA8IDApIGloaWdoICs9IGw7DQ0KICAgIGlmIChpbG93ICA8 IDApIGlsb3cgICs9IGw7DQ0KICAgIGlmIChpbG93IDwgMCkgaWxvdyA9IDA7DQ0KICAgIGVs c2UgaWYgKGlsb3cgPiBsKSBpbG93ID0gbDsNDQogICAgaWYgKGloaWdoIDwgMCkgaWhpZ2gg PSAwOw0NCiAgICBlbHNlIGlmIChpaGlnaCA+IGwpIGloaWdoID0gbDsNDQogICAgaWYgKGlo aWdoIDwgaWxvdykgaWhpZ2ggPSBpbG93Ow0NCiAgICANDQogICAgUHlPYmplY3QgKm91dCA9 IFB5TGlzdF9OZXcoaWhpZ2gtaWxvdyk7DQ0KICAgIGludCBpLCByaTsNDQogICAgZm9yKHJp PWlsb3csaT0wOyByaSA8IGloaWdoOyByaSsrLGkrKykgew0NCiAgICAgIFB5TGlzdF9TZXRJ dGVtKG91dCwgaSwgUHlSb3dfZ2V0aXRlbShvLCByaSkpOw0NCiAgICB9DQ0KICAgIHJldHVy biBvdXQ7DQ0KICB9DQ0KICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgIHJldHVybiBl LnRvUHl0aG9uKCk7DQ0KICB9DQ0KfQ0NCg0NCnN0YXRpYyBpbnQgUHlSb3dfc2V0c2xpY2Uo UHlSb3dSZWYgKm8sIGludCBpbG93LCBpbnQgaWhpZ2gsIFB5T2JqZWN0ICp2YWx1ZSkgew0N CiAgdHJ5IHsNDQogICAgaW50IGwgPSBvLT5Db250YWluZXIoKS5OdW1Qcm9wZXJ0aWVzKCk7 DQ0KICAgIGlmIChpaGlnaCA8IDApIGloaWdoICs9IGw7DQ0KICAgIGlmIChpbG93ICA8IDAp IGlsb3cgICs9IGw7DQ0KICAgIGlmIChpbG93IDwgMCkgaWxvdyA9IDA7DQ0KICAgIGVsc2Ug aWYgKGlsb3cgPiBsKSBpbG93ID0gbDsNDQogICAgaWYgKGloaWdoIDwgMCkgaWhpZ2ggPSAw Ow0NCiAgICBlbHNlIGlmIChpaGlnaCA+IGwpIGloaWdoID0gbDsNDQogICAgaWYgKGloaWdo IDwgaWxvdykgaWhpZ2ggPSBpbG93Ow0NCiAgICBpbnQgc2xlbiA9IGloaWdoLWlsb3c7DQ0K DQ0KICAgIGlmIChQeVNlcXVlbmNlX0NoZWNrKHZhbHVlKSAmJiAoUHlPYmplY3RfTGVuZ3Ro KHZhbHVlKSA+PSBzbGVuKSkgew0NCiAgICAgIGludCBpLCByaTsNDQogICAgICBmb3Iocmk9 aWxvdyxpPTA7IHJpIDwgaWhpZ2g7IHJpKyssaSsrKSB7DQ0KCVB5Um93X3NldGl0ZW0obywg cmksIFB5U2VxdWVuY2VfR2V0SXRlbSh2YWx1ZSwgaSkpOw0NCiAgICAgIH0NDQogICAgICBy ZXR1cm4gMDsNDQogICAgfQ0NCiAgICB0aHJvdyBQV0V4Y2VwdGlvbihQeUV4Y19UeXBlRXJy b3IsICJWYWx1ZSBtdXN0IGJlIGEgc2VxdWVuY2Ugd2l0aCBsZW5ndGggPj0gbGVuZ3RoIG9m IHNsaWNlZCByb3ciKTsNDQogIH0NDQogIGNhdGNoKFBXRXhjZXB0aW9uIGUpIHsNDQogICAg ZS50b1B5dGhvbigpOw0NCiAgICByZXR1cm4gLTE7DQ0KICB9DQ0KfQ0NCg0NCnN0YXRpYyBp bnQgc2xpY2VfR2V0SW5kaWNlcyhQeVNsaWNlT2JqZWN0ICpyLCBpbnQgbGVuZ3RoLCANDQog ICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50ICpzdGFydCwgaW50ICpzdG9wLCBpbnQg KnN0ZXApDQ0Kew0NCiAgICBpZiAoci0+c3RlcCA9PSBQeV9Ob25lKSB7DQ0KCSpzdGVwID0g MTsNDQogICAgfSBlbHNlIHsNDQoJaWYgKCFQeUludF9DaGVjayhyLT5zdGVwKSkgcmV0dXJu IC0xOw0NCgkqc3RlcCA9IFB5SW50X0FzTG9uZyhyLT5zdGVwKTsNDQogICAgfQ0NCiAgICBp ZiAoci0+c3RhcnQgPT0gUHlfTm9uZSkgew0NCgkqc3RhcnQgPSAqc3RlcCA8IDAgPyBsZW5n dGgtMSA6IDA7DQ0KICAgIH0gZWxzZSB7DQ0KCWlmICghUHlJbnRfQ2hlY2soci0+c3RhcnQp KSByZXR1cm4gLTE7DQ0KCSpzdGFydCA9IFB5SW50X0FzTG9uZyhyLT5zdGFydCk7DQ0KCWlm ICgqc3RhcnQgPCAwKSAqc3RhcnQgKz0gbGVuZ3RoOw0NCiAgICB9DQ0KICAgIGlmIChyLT5z dG9wID09IFB5X05vbmUpIHsNDQoJKnN0b3AgPSAqc3RlcCA8IDAgPyAtMSA6IGxlbmd0aDsN DQogICAgfSBlbHNlIHsNDQoJaWYgKCFQeUludF9DaGVjayhyLT5zdG9wKSkgcmV0dXJuIC0x Ow0NCgkqc3RvcCA9IFB5SW50X0FzTG9uZyhyLT5zdG9wKTsNDQoJaWYgKCpzdG9wIDwgMCkg KnN0b3AgKz0gbGVuZ3RoOw0NCiAgICB9DQ0KDQ0KICAgIGlmICgqc3RhcnQgPiAobGVuZ3Ro LTEpKSAqc3RhcnQgPSBsZW5ndGg7DQ0KICAgIGlmICgqc3RhcnQgPCAwKSAqc3RhcnQgPSAw Ow0NCiAgICBpZiAoKnN0b3AgPCAtMSkgKnN0b3AgPSAtMTsNDQogICAgZWxzZSBpZiAoKnN0 b3AgPiBsZW5ndGgpICpzdG9wID0gbGVuZ3RoOw0NCiAgICByZXR1cm4gMDsNDQp9DQ0KDQ0K DQ0Kc3RhdGljIGludCBnZXRfc2xpY2UoUHlPYmplY3QgKm9wLCBpbnQgbWF4LCBpbnQgKm5w LCBpbnQgKnNwKSB7DQ0KICAgIGludCBzdGFydCwgc3RvcCwgc3RlcDsNDQoJDQ0KICAgIGlm IChQeVNsaWNlX0NoZWNrKG9wKSkgew0NCglpZiAoc2xpY2VfR2V0SW5kaWNlcygoUHlTbGlj ZU9iamVjdCAqKW9wLCBtYXgsIA0NCgkJCSAgICAgJnN0YXJ0LCAmc3RvcCwgJnN0ZXApID09 IC0xKSByZXR1cm4gLTE7DQ0KCQkNDQoJaWYgKHN0ZXAgIT0gMCkgew0NCgkgICAgaWYgKHN0 ZXAgPCAwKSAqbnAgPSAoc3RvcC1zdGFydCsxK3N0ZXApL3N0ZXA7DQ0KCSAgICBlbHNlICpu cCA9IChzdG9wLXN0YXJ0LTErc3RlcCkvc3RlcDsNDQoJfSBlbHNlIHsNDQoJICAgIGlmIChz dG9wID09IHN0YXJ0KSB7DQ0KCQkqbnAgPSAwOyBzdGVwID0gMTsNDQoJICAgIH0NDQoJICAg IGVsc2UgcmV0dXJuIC0xOw0NCgl9DQ0KCWlmICgqbnAgPCAwKSAqbnAgPSAwOw0NCgkqc3Ag PSBzdGVwOw0NCglyZXR1cm4gc3RhcnQ7DQ0KICAgIH0gIA0NCiAgICByZXR1cm4gLTE7DQ0K fQ0NCg0NCnN0YXRpYyBQeU9iamVjdCogUHlSb3dfZ2V0cHlwcm9waXRlbShQeVJvd1JlZiAq c2VsZiwgUHlPYmplY3QgKm9wKSB7DQ0KICBpbnQgcHJvcGluZGV4Ow0NCiAgdHJ5IHsNDQog ICAgaWYgKFB5SW50X0NoZWNrKG9wKSkgew0NCiAgICAgIGludCBuID0gUHlJbnRfQXNMb25n KG9wKTsNDQogICAgICBpZiAobiA+PSAwICYmIG4gPCBzZWxmLT5Db250YWluZXIoKS5OdW1Q cm9wZXJ0aWVzKCkpIHsNDQoJcmV0dXJuIG5ldyBQeVByb3BlcnR5KHNlbGYtPkNvbnRhaW5l cigpLk50aFByb3BlcnR5KG4pKTsNDQogICAgICB9DQ0KICAgICAgdGhyb3cgUFdFeGNlcHRp b24oUHlFeGNfSW5kZXhFcnJvciwgIkluZGV4IG91dCBvZiByYW5nZSIpOw0NCiAgICB9DQ0K ICAgIGVsc2UgaWYgKFB5U3RyaW5nX0NoZWNrKG9wKSkgew0NCiAgICAgIHByb3BpbmRleCA9 IHNlbGYtPkNvbnRhaW5lcigpLkZpbmRQcm9wSW5kZXhCeU5hbWUoUHlTdHJpbmdfQXNTdHJp bmcob3ApKTsNDQogICAgICBpZiAocHJvcGluZGV4ICE9IC0xKSB7DQ0KCXJldHVybiBuZXcg UHlQcm9wZXJ0eShzZWxmLT5Db250YWluZXIoKS5OdGhQcm9wZXJ0eShwcm9waW5kZXgpKTsN DQogICAgICB9DQ0KICAgICAgdGhyb3cgUFdFeGNlcHRpb24oUHlFeGNfQXR0cmlidXRlRXJy b3IsICJObyBwcm9wZXJ0eSBjb3VsZCBiZSBmb3VuZCB3aXRoIHRoYXQgbmFtZSIpOw0NCiAg ICB9DQ0KICAgIGVsc2UgaWYgKFB5UHJvcGVydHlfQ2hlY2sob3ApKSB7DQ0KICAgICAgcHJv cGluZGV4ID0gc2VsZi0+Q29udGFpbmVyKCkuRmluZFByb3BlcnR5KCgoUHlQcm9wZXJ0eSop IG9wKS0+R2V0SWQoKSk7DQ0KICAgICAgaWYgKHByb3BpbmRleCAhPSAtMSkgew0NCglyZXR1 cm4gbmV3IFB5UHJvcGVydHkoc2VsZi0+Q29udGFpbmVyKCkuTnRoUHJvcGVydHkocHJvcGlu ZGV4KSk7DQ0KICAgICAgfQ0NCiAgICAgIHRocm93IFBXRXhjZXB0aW9uKFB5RXhjX0F0dHJp YnV0ZUVycm9yLCAiR2l2ZW4gcHJvcGVydHkgZG9lcyBub3QgZXhpc3QgZm9yIHRoaXMgcm93 Iik7DQ0KICAgIH0NDQogICAgZWxzZSBpZiAoUHlTbGljZV9DaGVjayhvcCkpIHsNDQogICAg ICBpbnQgaSwgcmksIG5fc3RlcHMsIHN0ZXBfc2l6ZTsNDQogICAgICBpZiAoKHJpID0gZ2V0 X3NsaWNlKG9wLCBzZWxmLT5Db250YWluZXIoKS5OdW1Qcm9wZXJ0aWVzKCksIA0NCgkJCSAg Jm5fc3RlcHMsICZzdGVwX3NpemUpKSA+PSAwKSB7DQ0KCVB5T2JqZWN0ICpvdXQgPSBQeUxp c3RfTmV3KG5fc3RlcHMpOw0NCglmb3IoaT0wOyBpIDwgbl9zdGVwczsgaSsrLCByaSArPSBz dGVwX3NpemUpIHsNDQoJICANDQoJICBQeUxpc3RfU2V0SXRlbShvdXQsIGksIA0NCgkJCSBu ZXcgUHlQcm9wZXJ0eShzZWxmLT5Db250YWluZXIoKS5OdGhQcm9wZXJ0eShyaSkpKTsNDQoJ fQ0NCglyZXR1cm4gb3V0Ow0NCiAgICAgIH0NDQogICAgICB0aHJvdyBQV0V4Y2VwdGlvbihQ eUV4Y19JbmRleEVycm9yLCAiaW52YWxpZCBzbGljZSIpOw0NCiAgICB9DQ0KICB0aHJvdyBQ V0V4Y2VwdGlvbihQeUV4Y19UeXBlRXJyb3IsICJUeXBlIG9mIGluZGV4IG5vdCBzdXBwb3J0 ZWQ6IG11c3QgYmUgb2YgdHlwZSBpbnRlZ2VyLCBzdHJpbmcsIFB5UHJvcGVydHksIG9yIHNs aWNlIik7DQ0KICB9DQ0KICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgIHJldHVybiBl LnRvUHl0aG9uKCk7DQ0KICB9DQ0KfQ0NCg0NCnN0YXRpYyBQeU9iamVjdCogUHlSb3dfZ2V0 cHlpdGVtKFB5Um93UmVmICpzZWxmLCBQeU9iamVjdCAqb3ApIHsNDQogIHRyeSB7DQ0KICAg IGlmIFB5VHVwbGVfQ2hlY2sob3ApIHsNDQogICAgICBpZiAoUHlUdXBsZV9TaXplKG9wKSA+ IDEpIHsNDQoJcmV0dXJuIFB5Um93X2dldHB5cHJvcGl0ZW0oc2VsZiwgUHlUdXBsZV9HZXRJ dGVtKG9wLCAwKSk7DQ0KICAgICAgfQ0NCiAgICB9DQ0KICAgIGVsc2UgaWYgKFB5SW50X0No ZWNrKG9wKSkgew0NCiAgICAgIGludCBuID0gUHlJbnRfQXNMb25nKG9wKTsNDQogICAgICBp ZiAobj49MCAmIG4gPCBzZWxmLT5Db250YWluZXIoKS5OdW1Qcm9wZXJ0aWVzKCkpIHsNDQoJ cmV0dXJuIHNlbGYtPmFzUHl0aG9uKHNlbGYtPkNvbnRhaW5lcigpLk50aFByb3BlcnR5KG4p KTsNDQogICAgICB9DQ0KICAgICAgdGhyb3cgUFdFeGNlcHRpb24oUHlFeGNfSW5kZXhFcnJv ciwgIkluZGV4IG91dCBvZiByYW5nZSIpOw0NCiAgICB9DQ0KICAgIGVsc2UgaWYgKFB5U3Ry aW5nX0NoZWNrKG9wKSkgew0NCiAgICAgIHJldHVybiBQeVJvd1JlZl9nZXRhdHRyKHNlbGYs IFB5U3RyaW5nX0FzU3RyaW5nKG9wKSk7DQ0KICAgIH0NDQogICAgZWxzZSBpZiAoUHlQcm9w ZXJ0eV9DaGVjayhvcCkpIHsNDQogICAgICByZXR1cm4gUHlSb3dSZWZfZ2V0YXR0cihzZWxm LCAoY2hhciopICgoKFB5UHJvcGVydHkqKW9wKS0+TmFtZSgpKSk7DQ0KICAgIH0NDQogICAg ZWxzZSBpZiAoUHlTbGljZV9DaGVjayhvcCkpIHsNDQogICAgICBpbnQgaSwgcmksIG5fc3Rl cHMsIHN0ZXBfc2l6ZTsNDQogICAgICBpZiAoKHJpID0gZ2V0X3NsaWNlKG9wLCBzZWxmLT5D b250YWluZXIoKS5OdW1Qcm9wZXJ0aWVzKCksIA0NCgkJCSAgJm5fc3RlcHMsICZzdGVwX3Np emUpKSA+PSAwKSB7DQ0KCVB5T2JqZWN0ICpvdXQgPSBQeUxpc3RfTmV3KG5fc3RlcHMpOw0N Cglmb3IoaT0wOyBpIDwgbl9zdGVwczsgaSsrLCByaSArPSBzdGVwX3NpemUpIHsNDQoJICBQ eUxpc3RfU2V0SXRlbShvdXQsIGksIFB5Um93X2dldGl0ZW0oc2VsZiwgcmkpKTsNDQoJfQ0N CglyZXR1cm4gb3V0Ow0NCiAgICAgIH0NDQogICAgICB0aHJvdyBQV0V4Y2VwdGlvbihQeUV4 Y19JbmRleEVycm9yLCAiaW52YWxpZCBzbGljZSIpOw0NCiAgICB9DQ0KICB0aHJvdyBQV0V4 Y2VwdGlvbihQeUV4Y19UeXBlRXJyb3IsICJUeXBlIG9mIGluZGV4IG5vdCBzdXBwb3J0ZWQ6 IG11c3QgYmUgb2YgdHlwZSBpbnRlZ2VyLCBzdHJpbmcsIFB5UHJvcGVydHksIG9yIHNsaWNl Iik7DQ0KICB9DQ0KICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAgIHJldHVybiBlLnRv UHl0aG9uKCk7DQ0KICB9DQ0KfQ0NCg0NCg0NCnN0YXRpYyBpbnQgUHlSb3dfc2V0cHlpdGVt KFB5Um93UmVmICpzZWxmLCBQeU9iamVjdCAqaW5kZXgsIFB5T2JqZWN0ICp2YWx1ZSkgew0N CiAgdHJ5IHsNDQogICAgaWYgKFB5SW50X0NoZWNrKGluZGV4KSkgew0NCiAgICAgIHJldHVy biBQeVJvd19zZXRpdGVtKHNlbGYsIFB5SW50X0FzTG9uZyhpbmRleCksIHZhbHVlKTsNDQog ICAgfQ0NCiAgICBlbHNlIGlmIChQeVN0cmluZ19DaGVjayhpbmRleCkpIHsNDQogICAgICBy ZXR1cm4gUHlSb3dSZWZfc2V0YXR0cihzZWxmLCBQeVN0cmluZ19Bc1N0cmluZyhpbmRleCks IHZhbHVlKTsNDQogICAgfQ0NCiAgICBlbHNlIGlmIChQeVByb3BlcnR5X0NoZWNrKGluZGV4 KSkgew0NCiAgICAgIHJldHVybiBQeVJvd1JlZl9zZXRhdHRyKHNlbGYsIChjaGFyKikgKCgo UHlQcm9wZXJ0eSopaW5kZXgpLT5OYW1lKCkpLA0NCgkJCSAgICAgIHZhbHVlKTsNDQogICAg fQ0NCiAgICBlbHNlIGlmIChQeVNsaWNlX0NoZWNrKGluZGV4KSkgew0NCiAgICAgIGludCBp LCByaSwgbl9zdGVwcywgc3RlcF9zaXplOw0NCiAgICAgIGlmICgocmkgPSBnZXRfc2xpY2Uo aW5kZXgsIHNlbGYtPkNvbnRhaW5lcigpLk51bVByb3BlcnRpZXMoKSwgDQ0KCQkJICAmbl9z dGVwcywgJnN0ZXBfc2l6ZSkpID49IDApIHsNDQoJaWYgKFB5U2VxdWVuY2VfQ2hlY2sodmFs dWUpICYmIChQeU9iamVjdF9MZW5ndGgodmFsdWUpID49IG5fc3RlcHMpKSB7DQ0KCSAgZm9y KGk9MDsgaSA8IG5fc3RlcHM7IGkrKywgcmkgKz0gc3RlcF9zaXplKSB7DQ0KCSAgICBQeVJv d19zZXRpdGVtKHNlbGYsIHJpLCBQeVNlcXVlbmNlX0dldEl0ZW0odmFsdWUsIGkpKTsNDQoJ ICB9DQ0KCSAgcmV0dXJuIDA7DQ0KCX0NDQoJdGhyb3cgUFdFeGNlcHRpb24oUHlFeGNfVHlw ZUVycm9yLCAiVmFsdWUgbXVzdCBiZSBhIHNlcXVlbmNlIHdpdGggbGVuZ3RoID49IGxlbmd0 aCBvZiBzbGljZWQgcm93Iik7DQ0KICAgICAgfQ0NCiAgICAgIHRocm93IFBXRXhjZXB0aW9u KFB5RXhjX0luZGV4RXJyb3IsICJpbnZhbGlkIHNsaWNlIik7CSAgICANDQogICAgfQ0NCiAg ICB0aHJvdyBQV0V4Y2VwdGlvbihQeUV4Y19UeXBlRXJyb3IsICJUeXBlIG9mIGluZGV4IG5v dCBzdXBwb3J0ZWQ6IG11c3QgYmUgb2YgdHlwZSBpbnRlZ2VyLCBzdHJpbmcsIFB5UHJvcGVy dHksIG9yIHNsaWNlIik7DQ0KICB9DQ0KICBjYXRjaChQV0V4Y2VwdGlvbiBlKSB7DQ0KICAg IGUudG9QeXRob24oKTsNDQogICAgcmV0dXJuIC0xOw0NCiAgfQ0NCn0NDQoNDQpzdGF0aWMg UHlNYXBwaW5nTWV0aG9kcyBSb3dBc01hcHBpbmcgPSB7DQ0KICAgIChpbnF1aXJ5KVB5Um93 X2xlbmd0aCwJCS8qbXBfbGVuZ3RoKi8NDQogICAgKGJpbmFyeWZ1bmMpUHlSb3dfZ2V0cHlp dGVtLCAgIC8qbXBfc3Vic2NyaXB0Ki8NDQogICAgKG9iam9iamFyZ3Byb2MpUHlSb3dfc2V0 cHlpdGVtLAkvKm1wX2Fzc19zdWJzY3JpcHQqLw0NCn07DQ0KDQ0Kc3RhdGljIFB5U2VxdWVu Y2VNZXRob2RzIFJvd0FzU2VxID0gew0NCiAgICAoaW5xdWlyeSlQeVJvd19sZW5ndGgsIC8v c3FfbGVuZ3RoDQ0KICAgIDAsIC8vc3FfY29uY2F0DQ0KICAgIDAsIC8vc3FfcmVwZWF0DQ0K ICAgIChpbnRhcmdmdW5jKVB5Um93X2dldGl0ZW0sIC8vc3FfaXRlbQ0NCiAgICAoaW50aW50 YXJnZnVuYylQeVJvd19nZXRzbGljZSwgLy9zcV9zbGljZQ0NCiAgICAoaW50b2JqYXJncHJv YylQeVJvd19zZXRpdGVtLCAvL3NxX2Fzc19pdGVtDQ0KICAgIChpbnRpbnRvYmphcmdwcm9j KVB5Um93X3NldHNsaWNlLCAvL3NxX2Fzc19zbGljZQ0NCn07DQ0KDQ0KDQ0KLy8gICAgDQ0K DQ0KUHlUeXBlT2JqZWN0IFB5Um93UmVmdHlwZSA9IHsNDQogICAgUHlPYmplY3RfSEVBRF9J TklUKCZQeVR5cGVfVHlwZSkNDQogICAgMCwNDQogICAgIlB5Um93UmVmIiwNDQogICAgc2l6 ZW9mKFB5Um93UmVmKSwNDQogICAgMCwNDQogICAgKGRlc3RydWN0b3IpUHlSb3dSZWZfZGVh bGxvYywgLyp0cF9kZWFsbG9jKi8NDQogICAgKHByaW50ZnVuYylQeVJvd1JlZl9wcmludCwg Lyp0cF9wcmludCovDQ0KICAgIChnZXRhdHRyZnVuYylQeVJvd1JlZl9nZXRhdHRyLCAvKnRw X2dldGF0dHIqLw0NCiAgICAoc2V0YXR0cmZ1bmMpUHlSb3dSZWZfc2V0YXR0ciwgICAgICAv KnRwX3NldGF0dHIqLw0NCiAgICAoY21wZnVuYykwLCAvKnRwX2NvbXBhcmUqLw0NCiAgICAo cmVwcmZ1bmMpMCwgLyp0cF9yZXByKi8NDQogICAgMCwgICAgICAvKnRwX2FzX251bWJlciov DQ0KICAgICZSb3dBc1NlcSwgIC8qdHBfYXNfc2VxdWVuY2UqLw0NCiAgICAmUm93QXNNYXBw aW5nLCAgICAgIC8qdHBfYXNfbWFwcGluZyovDQ0KfTsNDQoNDQpQeVJvd1JlZjo6UHlSb3dS ZWYoY29uc3QgYzRfUm93UmVmJiBvLCBQeVZpZXcgKm93bmVyKQ0NCiAgOiBQeUhlYWQgKFB5 Um93UmVmdHlwZSksIGM0X1Jvd1JlZihvKSB7DQ0KfQ0NCg0NCiAgICAvLyB3aXRoIHRoYW5r cyB0byBOaWtpIFNwYWhpZXYgZm9yIGltcHJvdmluZyBjb252ZXJzaW9ucyBhbmQgZXJyb3Ig Y2hlY2tzDQ0Kdm9pZCBQeVJvd1JlZjo6c2V0RnJvbVB5dGhvbihjb25zdCBjNF9Sb3dSZWYm IHJvdywgY29uc3QgYzRfUHJvcGVydHkmIHByb3AsIFB5T2JqZWN0KiBhdHRyKSB7DQ0KICAg IHN3aXRjaCAocHJvcC5UeXBlKCkpIHsNDQogICAgICAgIGNhc2UgJ0knOiAgIA0NCiAgICAg ICAgICAgIGlmIChQeUludF9DaGVjayhhdHRyKSkNDQogICAgICAgICAgICAgICAgKChjb25z dCBjNF9JbnRQcm9wJikgcHJvcCkgKHJvdykgPSBQeUludF9BU19MT05HKGF0dHIpOw0NCgkg ICAgZWxzZSBpZiAoYXR0ciAhPSBQeV9Ob25lKQ0NCgkgICAgew0NCgkJUFdPTnVtYmVyIG51 bWJlciAoYXR0cik7DQ0KICAgICAgICAgICAgICAgICgoY29uc3QgYzRfSW50UHJvcCYpIHBy b3ApIChyb3cpID0gKGxvbmcpIG51bWJlcjsNDQoJICAgIH0NDQogICAgICAgICAgICBicmVh azsNDQogICAgICAgIGNhc2UgJ0wnOiAgIA0NCgkgIGlmIChQeUludF9DaGVjayhhdHRyKSkN DQoJICAgICgoY29uc3QgYzRfTG9uZ1Byb3AmKSBwcm9wKSAocm93KSA9IFB5SW50X0FTX0xP TkcoYXR0cik7DQ0KCSAgZWxzZSBpZiAoUHlMb25nX0NoZWNrKGF0dHIpKQ0NCgkgICAgKChj b25zdCBjNF9Mb25nUHJvcCYpIHByb3ApIChyb3cpID0gUHlMb25nX0FzTG9uZyhhdHRyKTsN DQoJICBlbHNlIGlmIChhdHRyICE9IFB5X05vbmUpDQ0KCSAgICB7DQ0KCSAgICAgIFBXT051 bWJlciBudW1iZXIgKGF0dHIpOw0NCgkgICAgICAoKGNvbnN0IGM0X0xvbmdQcm9wJikgcHJv cCkgKHJvdykgPSAobG9uZykgbnVtYmVyOw0NCgkgICAgfQ0NCgkgIGJyZWFrOw0NCiAgICAg ICAgY2FzZSAnRic6ICAgDQ0KICAgICAgICAgICAgaWYgKFB5RmxvYXRfQ2hlY2soYXR0cikp DQ0KICAgICAgICAgICAgICAgICgoY29uc3QgYzRfRmxvYXRQcm9wJikgcHJvcCkgKHJvdykg PSBQeUZsb2F0X0FTX0RPVUJMRShhdHRyKTsNDQoJICAgIGVsc2UgaWYgKGF0dHIgIT0gUHlf Tm9uZSkNDQoJICAgIHsNDQoJCVBXT051bWJlciBudW1iZXIgKGF0dHIpOw0NCiAgICAgICAg ICAgICAgICAoKGNvbnN0IGM0X0Zsb2F0UHJvcCYpIHByb3ApIChyb3cpID0gKGRvdWJsZSkg bnVtYmVyOw0NCgkgICAgfQ0NCiAgICAgICAgICAgIGJyZWFrOw0NCiAgICAgICAgY2FzZSAn RCc6ICAgDQ0KICAgICAgICAgICAgaWYgKFB5RmxvYXRfQ2hlY2soYXR0cikpDQ0KICAgICAg ICAgICAgICAgICgoY29uc3QgYzRfRG91YmxlUHJvcCYpIHByb3ApIChyb3cpID0gUHlGbG9h dF9BU19ET1VCTEUoYXR0cik7DQ0KCSAgICBlbHNlIGlmIChhdHRyICE9IFB5X05vbmUpDQ0K CSAgICB7DQ0KCQlQV09OdW1iZXIgbnVtYmVyIChhdHRyKTsNDQogICAgICAgICAgICAgICAg KChjb25zdCBjNF9Eb3VibGVQcm9wJikgcHJvcCkgKHJvdykgPSAoZG91YmxlKSBudW1iZXI7 DQ0KCSAgICB9DQ0KICAgICAgICAgICAgYnJlYWs7DQ0KICAgICAgICBjYXNlICdTJzogDQ0K ICAgICAgICAgICAgaWYgKFB5U3RyaW5nX0NoZWNrKGF0dHIpKSB7DQ0KICAgICAgICAgICAg ICAgIGM0X0J5dGVzIHRlbXAgKFB5U3RyaW5nX0FTX1NUUklORyhhdHRyKSwNDQoJCQkJUHlT dHJpbmdfR0VUX1NJWkUoYXR0cikgKyAxLCBmYWxzZSk7DQ0KICAgICAgICAgICAgICAgIHBy b3AgKHJvdykuU2V0RGF0YSh0ZW1wKTsNDQogICAgICAgICAgICB9DQ0KCSAgICBlbHNlIGlm IChhdHRyICE9IFB5X05vbmUpDQ0KCQl0aHJvdyBQV0V4Y2VwdGlvbihQeUV4Y19UeXBlRXJy b3IsICJ3cm9uZyB0eXBlIGZvciBTdHJpbmdQcm9wIik7DQ0KICAgICAgICAgICAgYnJlYWs7 DQ0KICAgICAgICBjYXNlICdWJzogICANDQogICAgICAgICAgICBpZiAoUHlWaWV3X0NoZWNr KGF0dHIpKSB7DQ0KICAgICAgICAgICAgICAgIFB5VmlldyAqb2JqID0gKFB5VmlldyopYXR0 cjsNDQogICAgICAgICAgICAgICAgKChjb25zdCBjNF9WaWV3UHJvcCYpIHByb3ApKHJvdykg PSAqb2JqOw0NCiAgICAgICAgICAgIH0NDQogICAgICAgICAgICBlbHNlIHsgIA0NCiAgICAg ICAgICAgICAgICAoKGNvbnN0IGM0X1ZpZXdQcm9wJikgcHJvcCkgKHJvdykgPSBjNF9WaWV3 ICgpOw0NCiAgICAgICAgICAgICAgICBQeVZpZXcgdG1wKCgoY29uc3QgYzRfVmlld1Byb3Am KSBwcm9wKShyb3cpKTsNDQogICAgICAgICAgICAgICAgUFdPU2VxdWVuY2UgbHN0KGF0dHIp Ow0NCiAgICAgICAgICAgICAgICB0bXAuU2V0U2l6ZShsc3QubGVuKCkpOw0NCiAgICAgICAg ICAgICAgICBmb3IgKGludCBpPTA7IGk8bHN0LmxlbigpOyBpKyspDQ0KICAgICAgICAgICAg ICAgIHsNDQogICAgICAgICAgICAgICAgICAgIFB5T2JqZWN0KiBlbnRyeSA9IGxzdFtpXTsN DQogICAgICAgICAgICAgICAgICAgIHRtcC5zZXRJdGVtKGksIGVudHJ5KTsNDQogICAgICAg ICAgICAgICAgICAgIFB5X0RFQ1JFRihlbnRyeSk7DQ0KICAgICAgICAgICAgICAgIH0NDQog ICAgICAgICAgICB9DQ0KICAgICAgICAgICAgYnJlYWs7DQ0KICAgICAgICBjYXNlICdCJzoN DQogICAgICAgIGNhc2UgJ00nOiAgIA0NCiAgICAgICAgICAgIGlmIChQeVN0cmluZ19DaGVj ayhhdHRyKSkgew0NCiAgICAgICAgICAgICAgICBjNF9CeXRlcyB0ZW1wIChQeVN0cmluZ19B U19TVFJJTkcoYXR0ciksDQ0KCQkJCVB5U3RyaW5nX0dFVF9TSVpFKGF0dHIpLCBmYWxzZSk7 DQ0KICAgICAgICAgICAgICAgIHByb3AgKHJvdykuU2V0RGF0YSh0ZW1wKTsNDQogICAgICAg ICAgICB9DQ0KCSAgICBlbHNlIGlmIChhdHRyICE9IFB5X05vbmUpDQ0KCQl0aHJvdyBQV0V4 Y2VwdGlvbihQeUV4Y19UeXBlRXJyb3IsICJ3cm9uZyB0eXBlIGZvciBCeXRlUHJvcCIpOw0N CiAgICB9DQ0KfQ0NCg0NCnZvaWQgUHlSb3dSZWY6OnNldERlZmF1bHQoY29uc3QgYzRfUm93 UmVmJiByb3csIGNvbnN0IGM0X1Byb3BlcnR5JiBwcm9wKSB7DQ0KICAgIHN3aXRjaCAocHJv cC5UeXBlKCkpIHsNDQogICAgICAgIGNhc2UgJ0knOg0NCiAgICAgICAgICAgICgoY29uc3Qg YzRfSW50UHJvcCYpIHByb3ApIChyb3cpID0gMDsNDQogICAgICAgICAgICBicmVhazsNDQog ICAgICAgIGNhc2UgJ0wnOg0NCiAJICAgICgoY29uc3QgYzRfTG9uZ1Byb3AmKSBwcm9wKSAo cm93KSA9IDA7DQ0KCSAgICBicmVhazsNDQogICAgICAgIGNhc2UgJ0YnOiANDQogICAgICAg ICAgICAoKGNvbnN0IGM0X0Zsb2F0UHJvcCYpIHByb3ApIChyb3cpID0gMC4wOw0NCiAgICAg ICAgICAgIGJyZWFrOw0NCiAgICAgICAgIGNhc2UgJ0QnOg0NCiAgICAgICAgICAgICAoKGNv bnN0IGM0X0RvdWJsZVByb3AmKSBwcm9wKSAocm93KSA9IDAuMDsNDQogICAgICAgICAgICAg YnJlYWs7DQ0KICAgICAgICBjYXNlICdTJzoNDQogICAgICAgICAgICAoKGNvbnN0IGM0X1N0 cmluZ1Byb3AmKSBwcm9wKSAocm93KSA9ICIiOw0NCiAgICAgICAgICAgIGJyZWFrOw0NCiAg ICAgICAgY2FzZSAnVic6ICAgDQ0KICAgICAgICAgICAgKChjb25zdCBjNF9WaWV3UHJvcCYp IHByb3ApIChyb3cpID0gYzRfVmlldyAoKTsNDQogICAgICAgICAgICBicmVhazsNDQogICAg ICAgIGNhc2UgJ0InOg0NCiAgICAgICAgY2FzZSAnTSc6DQ0KICAgICAgICAgICAgew0NCiAg ICAgICAgICAgICAgICBjNF9CeXRlcyB0ZW1wOw0NCiAgICAgICAgICAgICAgICBwcm9wIChy b3cpLlNldERhdGEodGVtcCk7DQ0KICAgICAgICAgICAgfQ0NCiAgICB9DQ0KfQ0NCg0NClB5 T2JqZWN0KiBQeVJvd1JlZjo6YXNQeXRob24oY29uc3QgYzRfUHJvcGVydHkmIHByb3ApIHsN DQogICAgc3dpdGNoIChwcm9wLlR5cGUoKSkgew0NCiAgICBjYXNlICdJJzogew0NCiAgICAg ICAgUFdPTnVtYmVyIHJzbHQoKChjb25zdCBjNF9JbnRQcm9wJilwcm9wKSgqdGhpcykpOw0N CiAgICAgICAgcmV0dXJuIHJzbHQuZGlzT3duKCk7DQ0KICAgICAgICAgICAgICB9DQ0KICAg IGNhc2UgJ0wnOiB7DQ0KICAgICAgcmV0dXJuIFB5TG9uZ19Gcm9tTG9uZygoKGNvbnN0IGM0 X0xvbmdQcm9wJilwcm9wKSgqdGhpcykpOw0NCiAgICAgICAgICAgICAgfQ0NCiAgICBjYXNl ICdGJzogew0NCiAgICAgICAgUFdPTnVtYmVyIHJzbHQoKChjb25zdCBjNF9GbG9hdFByb3Am KXByb3ApKCp0aGlzKSk7DQ0KICAgICAgICByZXR1cm4gcnNsdC5kaXNPd24oKTsNDQogICAg ICAgICAgICAgIH0NDQogICAgY2FzZSAnRCc6IHsNDQogICAgICAgIFBXT051bWJlciByc2x0 KCgoY29uc3QgYzRfRG91YmxlUHJvcCYpcHJvcCkoKnRoaXMpKTsNDQogICAgICAgIHJldHVy biByc2x0LmRpc093bigpOw0NCiAgICAgICAgICAgICAgfQ0NCiAgICBjYXNlICdTJzogew0N CiAgICAgICAgLy9jNF9TdHJpbmcgdG1wID0gKChjNF9TdHJpbmdQcm9wJilwcm9wKS5HZXQo KnRoaXMpOw0NCiAgICAgICAgUFdPU3RyaW5nIHJzbHQoKChjNF9TdHJpbmdQcm9wJilwcm9w KS5HZXQoKnRoaXMpKTsNDQogICAgICAgIHJldHVybiByc2x0LmRpc093bigpOw0NCiAgICAg ICAgICAgICAgfQ0NCiAgICBjYXNlICdWJzogew0NCiAgICAgICAgcmV0dXJuIG5ldyBQeVZp ZXcoKChjb25zdCBjNF9WaWV3UHJvcCYpcHJvcCkoKnRoaXMpKTsNDQogICAgICAgICAgICAg IH0NDQogICAgY2FzZSAnQic6DQ0KICAgIGNhc2UgJ00nOiB7DQ0KICAgICAgICBjNF9CeXRl cyB0ZW1wOw0NCiAgICAgICAgcHJvcCAoKnRoaXMpLkdldERhdGEodGVtcCk7DQ0KICAgICAg ICBQV09TdHJpbmcgcnNsdCgoY29uc3QgY2hhciopdGVtcC5Db250ZW50cygpLCB0ZW1wLlNp emUoKSk7DQ0KICAgICAgICByZXR1cm4gcnNsdC5kaXNPd24oKTsNDQogICAgICAgICAgICAg IH0NDQogICAgfQ0NCiAgICByZXR1cm4gMDsNDQp9DQ== --mMALseFBLG-- From jerry@usd1.com Fri Jun 15 10:21:54 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA16986 for ; Fri, 15 Jun 2001 10:21:53 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f5FKJui00232; Fri, 15 Jun 2001 13:19:57 -0700 (PDT) Date: Fri, 15 Jun 2001 13:19:24 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <11218023581.20010615131924@usd1.com> To: John Barnard CC: gmcm@hypernet.com, Metakit Subject: Re[6]: [Metakit] [python] In-reply-To: <15146.26068.846358.125463@aragorn.stat.harvard.edu> References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi John, This sounds very useful. General Question: >MyList = row[:] >NewList = DoSomethingWith(MyList) >row[:]= NewList As long as NewList has the same surface level element types and number as MyList, this will work? Even with nested views? Specific Question: Can someone make an updated Mk4Py.dll for me, for handle snakes but I do not C. -- Thanks!, Jerry Friday, June 15, 2001, 12:45:24 PM, you wrote: JB> Inspired by the recent discussions, I've made several additions to JB> PyRowRef.cpp. The modified version is attached below. With this JB> modification you now do the following (row is of type PyRowRef): JB> row["var1"] # equivalent to row.var1 JB> row[i] # returns ith element of row, where i is an integer JB> row[i:j] # returns a list containing the elements i through j, where i JB> # and j are integers JB> row[:] # returns all elements of row in a list JB> # can use the usual slice notation, e.g., -4:-2, etc to get JB> # corresponding elements in a list JB> row[i:j:k] # returns elements of the row corresponding to the slice JB> # i:j:k, where k is the step size (extended slice notation) JB> # in a list (i or j or k can be missing) JB> Using same indices as above but with ,1 appended returns the JB> corresponding properties instead of the elements. E.g., row[:,1] JB> returns a list of all of the properties "for" the row, and JB> row["var1",1] returns the property for var1. In fact, it doesn't have JB> to ,1 it can be ,anything (e.g. row[:,10.2]) -- I use the presence of JB> a second index as a flag to return PyProperties instead of JB> elements. It's not as elegant as row.keys(), but it avoids any name JB> conflicts. JB> You can also set elements of rows using the indices: JB> row["var1"] = 10 JB> row[2:4] = ['a', 'b'] # must be a sequence with length >= 2 JB> row[2:6:2] = (1,4) # must be a sequence with length >= 2 JB> row[:] = x # x is a sequence with length >= length of row JB> Note that row[:,1] = x raises as error since :,1 is a passed to the JB> code as a tuple and a tuple index is not supported for setting. JB> If people find this useful, I suggest it be added to the next release JB> of metakit. JB> Best, JB> John From barnard@aragorn.stat.harvard.edu Fri Jun 15 10:29:51 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA17247 for ; Fri, 15 Jun 2001 10:29:50 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id QAA17731; Fri, 15 Jun 2001 16:38:50 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15146.29273.140550.390713@aragorn.stat.harvard.edu> Date: Fri, 15 Jun 2001 16:38:49 -0400 (EDT) To: Jerry McRae Cc: John Barnard , gmcm@hypernet.com, Metakit Subject: Re[6]: [Metakit] [python] In-Reply-To: <11218023581.20010615131924@usd1.com> References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> <11218023581.20010615131924@usd1.com> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry, Jerry McRae writes: > This sounds very useful. Great. > General Question: > >MyList = row[:] > >NewList = DoSomethingWith(MyList) > >row[:]= NewList > > As long as NewList has the same surface level element types and number > as MyList, this will work? Even with nested views? Yes, I believe it will, although I haven't yet tested it with nested views. E.g., this works as expected row[:] = [x**2 for x in row[:]] i.e., replaces the elements of the row with their squared values. As long as the values you're assigning to a row position are accepted types for that property type, it should work (including subviews). > Specific Question: Can someone make an updated Mk4Py.dll for me, for > handle snakes but I do not C. What version of Python are you using. I have one for Python 2.1. John From barnard@aragorn.stat.harvard.edu Fri Jun 15 10:40:20 2001 Received: from aragorn.stat.harvard.edu (aragorn.stat.harvard.edu [140.247.29.210]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA17653 for ; Fri, 15 Jun 2001 10:40:20 -0500 Received: (from barnard@localhost) by aragorn.stat.harvard.edu (8.9.3/8.9.3) id QAA17747; Fri, 15 Jun 2001 16:49:18 -0400 From: John Barnard MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15146.29902.477687.839249@aragorn.stat.harvard.edu> Date: Fri, 15 Jun 2001 16:49:18 -0400 (EDT) To: John Barnard Cc: Jerry McRae , gmcm@hypernet.com, Metakit Subject: Re[6]: [Metakit] [python] In-Reply-To: <15146.29273.140550.390713@aragorn.stat.harvard.edu> References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> <11218023581.20010615131924@usd1.com> <15146.29273.140550.390713@aragorn.stat.harvard.edu> X-Mailer: VM 6.75 under 21.1 (patch 8) "Bryce Canyon" XEmacs Lucid Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John Barnard writes: > Jerry, > Yes, I believe it will, although I haven't yet tested it with nested > views. E.g., this works as expected > > row[:] = [x**2 for x in row[:]] A simpler and more efficient (saves on creating a list of the row elements) but equivalent form is row[:] = [x**2 for x in row] John From rcohen@dial.oleane.com Fri Jun 15 10:54:40 2001 Received: from apicra.wanadoo.fr (smtp-rt-3.wanadoo.fr [193.252.19.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA18258 for ; Fri, 15 Jun 2001 10:54:40 -0500 Received: from andira.wanadoo.fr (193.252.19.152) by apicra.wanadoo.fr; 15 Jun 2001 22:52:40 +0200 Received: from dial.oleane.com (217.128.52.6) by andira.wanadoo.fr; 15 Jun 2001 22:52:31 +0200 Message-ID: <3B2A8387.33367DEB@dial.oleane.com> Date: Fri, 15 Jun 2001 22:52:07 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> <11218023581.20010615131924@usd1.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] MacOs crash Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I made a little program that create 10000 rows of very simple data and store it in the file with metakit. (see at end of mail) It works perfectly on Windows and Solaris. But on MacOs 8 it crashes. If I reduce MAX to 9000 it runs ok. Even with 50Mb allocated to the app, it still crashes for 10000 insertions and not for 9000. After about 8500 times view.Add() is called, a huge bloc of memory is requested to the system (I saw that on the system panel "about your computer" where memory is displayed). then another, and the app that was 2 or 3 megas is now about 30 Mb. (at 9000) I beleive at 10000 it goes to more than 50Mb ! I dont understand why it happens. I tried to call Commit() every 1000 Add, but it did not change anything Is there any way to solve that ? I would like it to work on MacOs 8 and then not use the mapped memory that would be available on MacOs 9.1 (anyway I would not know how to set it). Thanks for any Idea. I cannot use metakit if a 10000 rows insert is crashing with no warn...gulp -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 ====================================================================== #define MAX 10000 long idx; c4_StringProp id ("id"); c4_IntProp index("index"); c4_DoubleProp val("val"); c4_BytesProp blob("blob"); c4_Storage database("test1.vxb", true); c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); c4_Row row; for (idx=0;idx; Sun, 17 Jun 2001 03:33:01 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sun, 17 Jun 2001 06:18:25 -0700 X-Originating-IP: [151.203.35.48] From: "A K" To: "Mailing list Metakit" References: <3B289BDB.6377.473A746@localhost> <3B28CE6D.7885.5392E5C@localhost> <15146.26068.846358.125463@aragorn.stat.harvard.edu> <11218023581.20010615131924@usd1.com> <3B2A8387.33367DEB@dial.oleane.com> Subject: Re: [Metakit] MacOs crash Date: Sun, 17 Jun 2001 09:19:28 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Message-ID: X-OriginalArrivalTime: 17 Jun 2001 13:18:25.0382 (UTC) FILETIME=[FD6F3060:01C0F72F] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, or does it also work on the old Mac OS'es as well? Thank you, Brendan Kao ----- Original Message ----- From: "Riccardo Cohen" To: "Mailing list Metakit" Sent: Friday, June 15, 2001 5:52 PM Subject: [Metakit] MacOs crash > Hello, > > I made a little program that create 10000 rows of very simple data and store it in the file with metakit. > (see at end of mail) > > It works perfectly on Windows and Solaris. But on MacOs 8 it crashes. If I reduce MAX to 9000 it runs ok. > > Even with 50Mb allocated to the app, it still crashes for 10000 insertions and not for 9000. > > After about 8500 times view.Add() is called, a huge bloc of memory is requested to the system (I > saw that on the system panel "about your computer" where memory is displayed). then another, and > the app that was 2 or 3 megas is now about 30 Mb. (at 9000) > I beleive at 10000 it goes to more than 50Mb ! > > I dont understand why it happens. > > I tried to call Commit() every 1000 Add, but it did not change anything > > > Is there any way to solve that ? I would like it to work on MacOs 8 and then not use the mapped > memory that would be available on MacOs 9.1 (anyway I would not know how to set it). > > > Thanks for any Idea. I cannot use metakit if a 10000 rows insert is crashing with no warn...gulp > -- > Riccardo Cohen > > Articque > Les Roches > 37230 Fondettes > France > email = rcohen@dial.oleane.com > web = http://www.articque.com > tel: +33 02 47 49 90 49 > fax: +33 02 47 49 91 49 > > ====================================================================== > #define MAX 10000 > long idx; > c4_StringProp id ("id"); > c4_IntProp index("index"); > c4_DoubleProp val("val"); > c4_BytesProp blob("blob"); > c4_Storage database("test1.vxb", true); > c4_View table1=database.GetAs("table1[id:S,index:I,val:D,blob:B]"); > c4_Row row; > for (idx=0;idx { > sprintf(tmp,"id%ld",idx); > id(row)=tmp; > index(row)=idx; > val(row)=idx*195.0465; > blob(row)=c4_Bytes(tmp,strlen(tmp)+1); > table1.Add(row); > } > database.Commit(); > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From berdahl@intelligentparadigm.com Mon Jun 18 08:41:49 2001 Received: from intelligentparadigm.com (sdsl-64-139-36-165.dsl.sjc.megapath.net [64.139.36.165]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA09710 for ; Mon, 18 Jun 2001 08:41:47 -0500 Received: from [10.0.1.17] ([10.0.1.17] verified) by intelligentparadigm.com (Stalker SMTP Server 1.8b8) with ESMTP id S.0000214425 for ; Mon, 18 Jun 2001 11:39:18 -0700 From: Eric Berdahl To: Mailing list Metakit Subject: Re: [Metakit] MacOs crash Date: Mon, 18 Jun 2001 11:38:30 -0700 Message-Id: <20010618183830.27984@mail.intelligentparadigm.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A K wrote: >I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, >or does it also work on the old Mac OS'es as well? I don't know if MetaKit runs on MacOS X, but I would expect to work fairly well there. I'm using it right now on MacOS 9.x in a couple applications. Works first rate. Hope this helps, Eric ----------------------------------------------------------------------- Eric Berdahl "Humor is like a frog -- you can find out Man Behind the Curtain how it works by dissection, but the frog Intelligent Paradigm tends to die in the process." berdahl@serendipity.org - Unknown berdahl@intelligentparadigm.com From rcohen@dial.oleane.com Tue Jun 19 00:48:46 2001 Received: from magnolia.wanadoo.fr (smtp-rt-11.wanadoo.fr [193.252.19.62]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id AAA25111 for ; Tue, 19 Jun 2001 00:48:45 -0500 Received: from citronier.wanadoo.fr (193.252.19.222) by magnolia.wanadoo.fr; 19 Jun 2001 12:46:28 +0200 Received: from dial.oleane.com (193.252.49.172) by citronier.wanadoo.fr; 19 Jun 2001 12:46:28 +0200 Message-ID: <3B2F3B78.39C9B918@dial.oleane.com> Date: Tue, 19 Jun 2001 12:46:00 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Subject: Re: [Metakit] MacOs crash References: <20010618183830.27984@mail.intelligentparadigm.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Eric Berdahl wrote: > > A K wrote: > > >I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, > >or does it also work on the old Mac OS'es as well? > > I don't know if MetaKit runs on MacOS X, but I would expect to work > fairly well there. > > I'm using it right now on MacOS 9.x in a couple applications. Works first > rate. > > Hope this helps, > Eric > Did you have any crash like mine ? -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From berdahl@intelligentparadigm.com Tue Jun 19 06:16:11 2001 Received: from intelligentparadigm.com (sdsl-64-139-36-165.dsl.sjc.megapath.net [64.139.36.165]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA02127 for ; Tue, 19 Jun 2001 06:16:11 -0500 Received: from [216.15.61.23] (HELO [10.0.1.28]) by intelligentparadigm.com (Stalker SMTP Server 1.8b8) with ESMTP id S.0000214685 for ; Tue, 19 Jun 2001 09:13:35 -0700 From: Eric Berdahl To: Mailing list Metakit Subject: Re: [Metakit] MacOs crash Date: Tue, 19 Jun 2001 09:12:15 -0700 Message-Id: <20010619161215.26238@mail.intelligentparadigm.com> In-Reply-To: <3B2F3B78.39C9B918@dial.oleane.com> References: <3B2F3B78.39C9B918@dial.oleane.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Riccardo Cohen wrote: >Eric Berdahl wrote: >> >> I'm using it right now on MacOS 9.x in a couple applications. Works first >> rate. >> >> Hope this helps, >> Eric >> > >Did you have any crash like mine ? Nope. Works first rate for me. Hope this helps, Eric ----------------------------------------------------------------------- Eric Berdahl "Macintosh -- we might not get everything Man Behind the Curtain right, but at least we knew the century Intelligent Paradigm was going to end." berdahl@serendipity.org - Douglas Adams on the Y2K problem berdahl@intelligentparadigm.com From brunobl@ifrance.com Tue Jun 19 08:22:30 2001 Received: from postfix2-2.free.fr (postfix@postfix2-2.free.fr [213.228.0.140]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA05994; Tue, 19 Jun 2001 08:22:29 -0500 Received: from [212.27.37.198] (dijon-37-198.dial.proxad.net [212.27.37.198]) by postfix2-2.free.fr (Postfix) with ESMTP id 61B696B801; Tue, 19 Jun 2001 20:20:40 +0200 (CEST) User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Tue, 19 Jun 2001 20:20:13 +0200 Subject: Re: [Metakit] MacOs crash From: Bruno Blondeau To: , Message-ID: In-Reply-To: <200106171701.MAA22639@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, > or does it also work on the old Mac OS'es as well? It's running fine under the old MacOS as well. I tested it successfuly under MacOS 8, 9 and X. Bruno -- Bruno Blondeau http://www.brunoblondeau.com You can praise them, disagree with them, quote them, disbelieve them, glorify or vilify them. About the only thing you can't do is ignore them. From bryan@tapestrysolutions.com Fri Jun 22 10:24:59 2001 Received: from tapestrysolutions.com (tapestry-2.customer.nethere.net [209.132.102.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA18633 for ; Fri, 22 Jun 2001 10:24:58 -0500 Received: from tapestrysolutions.com (toscana [204.94.74.81]) by tapestrysolutions.com (8.8.7/8.8.7) with ESMTP id OAA12673 for ; Fri, 22 Jun 2001 14:06:13 -0700 Message-ID: <3B33A8EE.D1260786@tapestrysolutions.com> Date: Fri, 22 Jun 2001 13:22:06 -0700 From: Me X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] MacOs crash Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > > I didn't even know that MetaKit runs on Mac OS. Is this the new Mac OS X, > > or does it also work on the old Mac OS'es as well? > > It's running fine under the old MacOS as well. I tested it successfuly under > MacOS 8, 9 and X. > > > Bruno Does anyone have a version of Mk4py.slb that will work with MacPython 2.1? From jimdesu@pacbell.net Fri Jun 22 12:40:40 2001 Received: from JMITCHE27100D (unk-17-26-112.peoplesoft.com [209.17.26.112]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA27035 for ; Fri, 22 Jun 2001 12:40:40 -0500 Received: from [127.0.0.1] by JMITCHE27100D (ArGoSoft Mail Server Plus, Version 1.6 (1.6.0.0)); Fri, 22 Jun 2001 15:38:09 -0700 Message-Id: <5.1.0.14.0.20010622153639.00a72120@postoffice.pacbell.net> X-Sender: jimdesu@postoffice.pacbell.net X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 22 Jun 2001 15:38:07 -0700 To: metakit@equi4.com From: James Mitchell Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] newbie Q: sizing? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I've recently read the explanation & sales-blurb on the metakit web page and am impressed. It mentions that it's not for 1M+ records at the moment, but the blurb is specified to be out of date. Can anyone say what the currently supported volumes are? Hoping to use it, James From readonly@getsoft.com Fri Jun 22 13:32:50 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA27781 for ; Fri, 22 Jun 2001 13:32:49 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id RAA04929; Fri, 22 Jun 2001 17:30:38 -0600 (MDT) Message-ID: <200106230930380193.0422B297@getsoft.com> In-Reply-To: <5.1.0.14.0.20010622153639.00a72120@postoffice.pacbell.net> References: <5.1.0.14.0.20010622153639.00a72120@postoffice.pacbell.net> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Sat, 23 Jun 2001 09:30:38 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "James Mitchell" , metakit@equi4.com Subject: Re: [Metakit] newbie Q: sizing? Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id NAA27781 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: James, In my experience I didn't have much joy using > 10,000 rows. When it got to 100K - 300K I had to give up. There were basic things like sorting, inserting, deleting which would take too long. You can optimize the adds by padding with blank dummy rows, but the code gets convaluted. Further you can't apply this trick to deletes. I also had regular unexplained crashes with large'ish databases. And found problems with sorting, where the sort order wasn't always 100% right. If you need indexed lookup (BTrees) you wan't get any help in MK. This observations are from 2.3.3.28 If the database isn't too dynamic or performance isn't a big issue it may be workable for you. I'm sure other peoples mileage will vary on this. *********** REPLY SEPARATOR *********** On 22 Jun 2001 at 15:38 James Mitchell wrote: >Hello, > >I've recently read the explanation & sales-blurb on the metakit web page >and am impressed. It mentions that it's not for 1M+ records at the moment, >but the blurb is specified to be out of date. Can anyone say what the >currently supported volumes are? > >Hoping to use it, > >James > > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit Best Regards, Neville Franks (Author of ED for Windows) PS. Don't forget to visit our Web site www.getsoft.com soon. You'll find lots of information on ED and while you're there you can click on 'Stay Informed' to join our mailing list and 'Future Plans' to read about Version 4. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From jcw@equi4.com Mon Jun 25 16:09:48 2001 Received: from lizzie (bjgl45sdy2xh.bc.hsia.telus.net [216.232.138.208]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA08274 for ; Mon, 25 Jun 2001 16:09:47 -0500 From: Jean-Claude Wippler To: metakit@equi4.com Date: Mon, 25 Jun 2001 19:10:41 -0700 X-Mailer: KMail [version 1.1.99] Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Message-Id: <0106251910410B.14952@lizzie> Content-Transfer-Encoding: 8bit Subject: [Metakit] Upcoming release 2.4.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It's been a while coming... the *final* 2.4.0 release of MetaKit is imminent. The latest changes have just been checked into CVS, info on fetching it is at: http://www.equi4.com/metakit/download.html The change log list all recent details (not really that many since 2.3.4): http://www.equi4.com/metakit/CHANGES Please let me know if I botched anything, forgot to incorporate a patch/fix, or in any way need to make adjustments before freezing 2.4.0 for real. There are still a few days to make minor mods. Please try and exercise it - but do note that a switch from < 2.3.x causes an irreversible file format conversion. Regards, Jean-Claude From rcohen@dial.oleane.com Wed Jun 27 12:00:51 2001 Received: from areca.wanadoo.fr (smtp-rt-4.wanadoo.fr [193.252.19.156]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA07803 for ; Wed, 27 Jun 2001 12:00:50 -0500 Received: from mahonia.wanadoo.fr (193.252.19.58) by areca.wanadoo.fr; 27 Jun 2001 23:57:54 +0200 Received: from dial.oleane.com (217.128.52.54) by mahonia.wanadoo.fr; 27 Jun 2001 23:57:43 +0200 Message-ID: <3B3A64C2.744FEA00@dial.oleane.com> Date: Wed, 27 Jun 2001 23:57:06 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] multiple or single file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I need an information before designing the database : If I have 10 or 20 tables, and some of them are about 10k rows. Will it be faster to store them in separate files or is it all right in the same file ? (I dont need to join, information is independant) Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From jerry@usd1.com Thu Jun 28 11:49:10 2001 Received: from comet.pacifier.com (comet.pacifier.com [199.2.117.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA08790 for ; Thu, 28 Jun 2001 11:49:10 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by comet.pacifier.com (8.11.2/8.11.1) with ESMTP id f5SLkCi16883 for ; Thu, 28 Jun 2001 14:46:12 -0700 (PDT) Date: Thu, 28 Jun 2001 14:17:42 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.47 Halloween Edition) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <19516621092.20010628141742@usd1.com> To: metakit Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] python: compare to type of PyProperty Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Can I compare to Mk types without creating or accessing existing objects? From generic modules I can compare a passed object to a string with "obj == type('')" or "import types; obj == types.StringType". The only way I know how to compare to Mk types is to either point to a known variable of that object: "obj == type(vw[0].name)", which is way to specific for the module to know. Or (2) I import mk4py and create dummy copies of all my objects to compare against. Or (3) specific hacks for each type, like all PyProperty items will have an attribute of 'name', so I can USUALLY get a match with: try: getattr(obj,'name') except AttributeError: # not a PyProperty Any other ideas? -- Thanks, Jerry McRae MetaKit 2.3.4, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From jcw@equi4.com Thu Jun 28 23:56:08 2001 Received: from lizzie (bjgl45sdy2xh.bc.hsia.telus.net [216.232.138.208]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA18036; Thu, 28 Jun 2001 23:56:07 -0500 From: Jean-Claude Wippler To: announce@equi4.com, metakit@equi4.com Date: Fri, 29 Jun 2001 02:57:09 -0700 X-Mailer: KMail [version 1.1.99] Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Message-Id: <01062902570906.00686@lizzie> Content-Transfer-Encoding: 8bit Subject: [Metakit] [ANN] MetaKit 2.4.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The latest MetaKit 2.4.0 code has been released. This version is intended for use as stable production release, and now supercedes all previous ones. - This release, as well the previous 2.3.4, has been in production use in several projects in the past months. I advise using 2.4.0 for every new project, and converting all older projects - there have been some changes in the API (such as dropping c4_Storage::Store - use GetAs from now on). - Changes since 2.3.4 are relatively minor, see http://www.equi4.com/metakit/CHANGES - There's an older but still valid summary of changes since 2.01 at: http://www.equi4.com/previews/WHATSNEW - The source archive of this new release is at: http://www.equi4.com/pub/mk/metakit-2.4.0-30.tar.gz - So far, there are binaries for Windows (MSVC 6) and for Linux (gcc 3.0): http://www.equi4.com/pub/mk/ - Be aware that 2.4.0 converts older datafiles during open, and saves the file in a new format on commit. The file size overhead has not really increased, but this conversion may lead to unused gaps (reused later). - Future major releases of MetaKit will only support the new file format, so if you have old datafiles (in use or archived), be sure to use this release whenever you want to convert them. - MetaKit docs continue to be weak and lacking (as well as lagging). - TclKit build 8.4-25 (based on MetaKit 2.4.0) is available from these areas: http://www.equi4.com/pub/tk/tclkit-8.4-25-linux/ http://www.equi4.com/pub/tk/tclkit-8.4-25-win/ The sources for building TclKit are in the MetaKit archive (see tcl/kit/). - The homepages of MetaKit and TclKit are as before, respectively: http://www.equi4.com/metakit/ and http://www.equi4.com/tclkit/ Regards, Jean-Claude P.S. As of tomorrow, I will be off-line until August. Have a nice summer, (or for the growing group of Australian users down-under: winter!) From dherity@iolfree.ie Sat Jun 30 04:12:12 2001 Received: from fargo.iolfree.ie (fargo.iolfree.ie [194.145.128.32]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA25126 for ; Sat, 30 Jun 2001 04:12:12 -0500 Received: from dervish (b-airlock236.esatclear.ie [194.145.131.236]) by fargo.iolfree.ie (8.9.3/8.9.3) with SMTP id PAA18396; Sat, 30 Jun 2001 15:09:02 +0100 Message-ID: <03f001c1016c$a1fa3670$d9a2a5c2@s3group.com> From: "Dominic Herity" To: Cc: Date: Sat, 30 Jun 2001 14:57:41 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_03EB_01C10175.03105D10" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Subject: [Metakit] Metakit Application - Reality Check Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_03EB_01C10175.03105D10 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi, I wonder if Metakit users could comment on this possible application. Platform: Windows 98/NT4 or later Compiler: C++ Builder 5 or 6 Database Size: 100K Records, 50MB Medium: CD-ROM. The database and software are both on a CDROM. The software runs off the CD without need for installation. The database is encrypted. (Seems like a use for a Strategy class.) The startup time must be short. My first look at Metakit suggests that its suitable for the purpose, but = I wonder if people familiar with it will see a problem (like scalability, for = example). Comments appreciated. Best Regards Dominic Herity ------=_NextPart_000_03EB_01C10175.03105D10 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi,
 
I wonder if Metakit users could comment = on this=20 possible application.
 
Platform: Windows 98/NT4 or = later
Compiler: C++ Builder 5 or = 6
Database Size: 100K Records, = 50MB
Medium: CD-ROM.
 
The database and software are both on a = CDROM.
The software runs off the CD = without need for=20 installation.
The database is encrypted. (Seems = like a use=20 for a Strategy class.)
The startup time must be = short.
 
My first look at Metakit suggests that = its suitable=20 for the purpose, but I wonder if
people familiar with it will see a = problem (like=20 scalability, for example).
 
Comments appreciated.
 
Best Regards
Dominic Herity
 
------=_NextPart_000_03EB_01C10175.03105D10-- From nrogers@softhome.net Wed Jul 4 08:28:55 2001 Received: from tango.SoftHome.net (tango.SoftHome.net [204.144.231.49]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA10697 for ; Wed, 4 Jul 2001 08:28:55 -0500 Received: (qmail 12145 invoked by uid 417); 4 Jul 2001 18:30:29 -0000 Received: from cx292990-b.lbbck1.tx.home.com (HELO cx292990-b) (24.15.152.161) by i-softhome-tango with SMTP; 4 Jul 2001 18:30:29 -0000 Date: Wed, 4 Jul 2001 13:25:58 -0500 From: Nathan Rogers To: metakit@equi4.com Message-Id: <20010704132558.66b31426.nrogers@softhome.net> X-Mailer: Sylpheed version 0.4.64 (GTK+ 1.2.10; i386-debian-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] hashing bug Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm running MetaKit 2.4.0 on Linux. Most of the time it works great, but I ran into a case where Find() on a hashed view goes into an infinite loop. Here is a simplified test case that hangs every time I run it. #include #include void main() { c4_Storage db; c4_View test_data, test_hash, test; c4_StringProp pKey ("key"); test_data = db.GetAs("test_data[key:S]"); test_hash = db.GetAs("test_hash[_H:I,_R:I]"); test = test_data.Hash(test_hash, 1); test.Add(pKey["b93655249726e5ef4c68e45033c2e0850570e1e07"]); test.Add(pKey["2ab03fba463d214f854a71ab5c951cea096887adf"]); test.Add(pKey["2e196eecb91b02c16c23360d8e1b205f0b3e3fa3d"]); test.Find(pKey["7c0734c9187133f34588517fb5b39294076f22ba3"]); } From ASeifert@XSystemsInc.com Thu Jul 5 07:50:25 2001 Received: from dom01.xsystemsinc.com (hidden-user@xsystems5.vtcrc.net [12.5.16.43] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA15315 for ; Thu, 5 Jul 2001 07:50:25 -0500 Received: from demo2 ([64.204.192.3]) by dom01.xsystemsinc.com (Lotus Domino Release 5.0.7) with SMTP id 2001070513472314:879 ; Thu, 5 Jul 2001 13:47:23 -0400 Message-ID: <008001c1057a$0c9c6690$0300a8c0@xsystemsinc.com> Reply-To: "Allen Seifert" From: "Allen Seifert" To: Date: Thu, 5 Jul 2001 13:43:49 -0400 Organization: X.Systems, Inc. MIME-Version: 1.0 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 X-MIMETrack: Itemize by SMTP Server on dom01/xsystems(Release 5.0.7 |March 21, 2001) at 07/05/2001 01:47:23 PM, Serialize by Router on dom01/xsystems(Release 5.0.7 |March 21, 2001) at 07/05/2001 01:47:24 PM, Serialize complete at 07/05/2001 01:47:24 PM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] where is the latest Mk4Tcl download Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I would like to get the latest Mk4Tcl binary download for Windows - presumably 2.4. All I can find on the Equi4 web site is the 2.0 distribution. Can someone point me to the correct URL for this binary? Thanks, Allen From lvirden@cas.org Thu Jul 5 08:19:36 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA16642 for ; Thu, 5 Jul 2001 08:19:33 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id OAA13412 for ; Thu, 5 Jul 2001 14:15:57 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f65IFuM08480; Thu, 5 Jul 2001 14:15:56 -0400 (EDT) Date: Thu, 5 Jul 2001 14:15:56 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010705141556.AAB8457@cas.org> Subject: Re: [Metakit] where is the latest Mk4Tcl download In-Reply-To: Your message of Thu, 5 Jul 2001 13:43:49 -0400 To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It looks to me as if http://www.equi4.com/pub/mk/metakit-2.4.0-30-win/ is what you want - but I'm confused because it was pretty simple to find it, so I'm uncertain what you might have found. It is the directory you find looking at http://www.equi4.com/metakit/ which is the Metakit web page... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From remy@shockfish.com Tue Jul 10 08:29:03 2001 Received: from berau.travel.ch (postfix@www.abp.ch [193.135.59.43] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA15163 for ; Tue, 10 Jul 2001 08:28:56 -0500 Received: from pc2.shockfish.com (shockfishpc1.epfl.ch [128.179.48.2]) by berau.travel.ch (Postfix) with ESMTP id 94DD512F015 for ; Tue, 10 Jul 2001 20:23:06 +0200 (MEST) Message-Id: <5.0.2.1.0.20010710201542.00a80898@mail1.travel.ch> X-Sender: remymail@mail1.travel.ch X-Mailer: QUALCOMM Windows Eudora Version 5.0.2 Date: Tue, 10 Jul 2001 20:31:48 +0200 To: From: Remy Blank - Shockfish Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Strange implementation of c4_FormatS::DoCompare Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, While studying the implementation of the various format handlers within Metakit, I came across a strange implementation detail in c4_FormatS. This is the function: int c4_FormatS::DoCompare(const c4_Bytes& b1_, const c4_Bytes& b2_) { c4_String v1 ((const char*) b1_.Contents(), b1_.Size()); c4_String v2 ((const char*) b2_.Contents(), b2_.Size()); return v1.CompareNoCase(v2); } While this is totally correct, the construction of v1 and v2 from the contents of b1_ and b2_ means one memory allocation and one full copy each, even *before* they can be compared, and then the release of that memory at the end of the function. c4_String::CompareNoCase() is quite simple, too: int c4_String::CompareNoCase(const char* str) const { return Data() == str ? 0 : stricmp(Data(), str); } Wouldn't it be much more efficient to just inline CompareNoCase() "manually" into c4_FormatS::DoCompare(), working directly on the pointers? This would improve row comparisons, for example when sorting views. I imagine something like: int c4_FormatS::DoCompare(const c4_Bytes& b1_, const c4_Bytes& b2_) { char const* s1 = (const char*) b1_.Contents(); char const* s2 = (const char*) b2_.Contents(); return s1 == s2 ? 0 : stricmp(s1, s2); } Anyway, I'll run a few tests and see what pops out. --Remy From lvirden@cas.org Tue Jul 10 16:29:34 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA22278 for ; Tue, 10 Jul 2001 16:29:33 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.9.3+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id WAA04721 for ; Tue, 10 Jul 2001 22:25:37 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f6B2PYs05947; Tue, 10 Jul 2001 22:25:34 -0400 (EDT) Date: Tue, 10 Jul 2001 22:25:34 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010710222534.AAB5921@cas.org> To: metakit@equi4.com Subject: [Metakit] Wikit help sought Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What are the commands that I have available when interacting with a wikit so that I can look at the history of wikit pages. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From weyhan@my-deja.com Fri Jul 13 05:12:53 2001 Received: from mail22.bigmailbox.com (mail22.bigmailbox.com [209.132.220.199] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA01669 for ; Fri, 13 Jul 2001 05:12:53 -0500 Received: by mail22.bigmailbox.com (8.8.7/8.8.7) id IAA30506; Fri, 13 Jul 2001 08:08:44 -0700 Date: Fri, 13 Jul 2001 08:08:44 -0700 Message-Id: <200107131508.IAA30506@mail22.bigmailbox.com> Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: binary X-Mailer: MIME-tools 4.104 (Entity 4.116) Mime-Version: 1.0 X-Originating-Ip: [203.126.136.170] From: "Wey-Han Ng" To: metakit@equi4.com Subject: [Metakit] s33c: test fail Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I have downloaded and compile Metakit 2.4.0 on Solaris 2.5. When I try to run "make test" I got the following error: ... l07 - Huge description Done. diff ../unix/../tests/ok tests diff ../unix/../tests/ok/s33c.txt tests/s33c.txt 1,6c1 < VIEW 1 rows = a:V < 0: subview 'a' < VIEW 3 rows = p1:B < 0: (2b) < 1: (4b) < 2: (5b) --- > VIEW 1 rows = make: *** [test] Error 1 Is there a problem with my build? ------------------------------------------------------------ --== Sent via Deja.com ==-- http://www.deja.com/ From bryan@tapestrysolutions.com Tue Jul 17 15:33:03 2001 Received: from tapestrysolutions.com (tapestry-2.customer.nethere.net [209.132.102.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA07401 for ; Tue, 17 Jul 2001 15:33:02 -0500 Received: from tapestrysolutions.com (toscana [204.94.74.81]) by tapestrysolutions.com (8.8.7/8.8.7) with ESMTP id TAA13470 for ; Tue, 17 Jul 2001 19:14:16 -0700 Message-ID: <3B54E5E1.21CB677D@tapestrysolutions.com> Date: Tue, 17 Jul 2001 18:26:57 -0700 From: Me X-Mailer: Mozilla 4.5 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com References: <200107051701.MAA24376@trixie.triqs.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Where is the latest Mk4py download Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Allen Seifert wrote: > It looks to me as if http://www.equi4.com/pub/mk/metakit-2.4.0-30-win/ is what you > want Does anyone know where I can get the latest Mk4py for the Mac. The only version I can find (http://www.equi4.com/pub/mk/metakit-2.3.3-29-mac/) doesn't work with MacPython 2.1. From guenther.fischer@hrz.tu-chemnitz.de Thu Jun 21 20:42:11 2001 Received: from asterix.hrz.tu-chemnitz.de (asterix.hrz.tu-chemnitz.de [134.109.132.84]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA27209 for ; Thu, 21 Jun 2001 20:42:11 -0500 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by asterix.hrz.tu-chemnitz.de with esmtp (Exim 3.22 #4) id 15N7Gf-0002Ok-00 for metakit@equi4.com; Thu, 19 Jul 2001 08:26:37 +0200 Date: Thu, 19 Jul 2001 08:26:36 +0200 (MEST) From: Guenther Fischer To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] python and mk4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I will done a side step from tcl to python. My first try works wrong: - in my current dir is Mk4py.so and metakit py Type: python Python 1.5.2 (#1, Dec 21 2000, 15:29:08) [GCC egcs-2.91.66 19990314/Linux (egcs- on linux-i386 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import metakit >>> db = metakit.storage("wein.dat",r) Traceback (innermost last): File "", line 1, in ? AttributeError: storage >>> the same error with your Millions.py example. Shurly a silly error, but please any help... -- ~Guenther Fischer From guenther.fischer@hrz.tu-chemnitz.de Mon Jul 23 07:53:35 2001 Received: from obelix.hrz.tu-chemnitz.de (obelix.hrz.tu-chemnitz.de [134.109.132.55]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA17569 for ; Mon, 23 Jul 2001 07:53:34 -0500 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by obelix.hrz.tu-chemnitz.de with esmtp (Exim 3.22 #4) id 15OfE4-0005Pp-00 for metakit@equi4.com; Mon, 23 Jul 2001 14:54:20 +0200 Date: Mon, 23 Jul 2001 14:54:20 +0200 (MEST) From: Guenther Fischer To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Unicode und Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I will read my mk4-database written with Mk4tcl with Python/mk4py - all works fine, but the german umlaut and other european character will fail. As I remember such characters are coded in unicode - I think this happens automatically with mk4tcl, but with mk4py not. Is this true - what can I do? -- ~Guenther Fischer From niki@vintech.bg Mon Jul 23 08:24:23 2001 Received: from www.vintech.bg (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA18439 for ; Mon, 23 Jul 2001 08:24:19 -0500 Received: from vintech.bg (sierra [10.0.0.7]) by www.vintech.bg (8.11.2/8.8.7) with ESMTP id f6NDRZO11871; Mon, 23 Jul 2001 16:27:35 +0300 Message-ID: <3B5C2628.80508@vintech.bg> Date: Mon, 23 Jul 2001 16:27:04 +0300 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.2) Gecko/20010628 X-Accept-Language: bg, en MIME-Version: 1.0 To: Guenther Fischer CC: metakit@equi4.com Subject: Re: [Metakit] Unicode und Mk4py References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: > Hi, > I will read my mk4-database written with Mk4tcl with Python/mk4py - > all works fine, but the german umlaut and other european character > will fail. As I remember such characters are coded in unicode - > I think this happens automatically with mk4tcl, but with mk4py not. > > Is this true - what can I do? > > AFAIK metakit has no unicode support. If you use Python >= 2.0 try utf-1 encoding or decoding with string from Mk4py. HTH Niki Spahiev From jw2@scriptthing.com Tue Jul 24 22:51:06 2001 Received: from compuvar04.compuvar.com (HERMES.COMPUVAR.COM [209.101.164.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id WAA17046 for ; Tue, 24 Jul 2001 22:51:04 -0500 Received: from [208.57.85.201] by compuvar04.compuvar.com (NTMail 5.06.0016/NT0699.00.e2308d62) with ESMTP id nznubbaa for metakit@equi4.com; Tue, 24 Jul 2001 20:51:12 -0700 Mime-Version: 1.0 X-Sender: jw2@mail.scriptthing.com Message-Id: Date: Tue, 24 Jul 2001 20:51:29 -0700 To: metakit@equi4.com From: "James W. Walker" Content-Type: text/plain; charset="us-ascii" ; format="flowed" Subject: [Metakit] MK 2.4.0 crashing Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Today I tried updating from MK 2.0.1 to 2.4.0, and it crashes when trying to open files created by the older MK. I don't have a huge amount of time for debugging this before I have to ship a product, so I'll probably have to revert to 2.0.1. But in case anyone has any ideas or relevant experience, here's what I discovered so far: Soon after c4_Persist::LoadAll starts, c4_Strategy::EndOfData sets _rootPos to 0 and _rootLen to -1. Comments in the code say that a _rootLen value of -1 flags an "old-style header", so I guess that's correct, but I don't know what _rootPos is supposed to be. The consequence, though, is that LoadAll sets _oldSeek to 0. Then c4_Persist::FetchOldValue reads the very start of the file, and c4_Column::PullValue tries to interpret the initial 'LJ' etc. as a number. -- James W. Walker, ScriptPerfection Enterprises, Inc. From titoffen@pacbell.net Sat Jul 28 23:18:49 2001 Received: from snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19192 for ; Sat, 28 Jul 2001 23:18:48 -0500 Message-Id: <200107290418.XAA19192@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta5.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GH7003EXVZZHT@mta5.snfc21.pbi.net> for metakit@equi4.com; Sat, 28 Jul 2001 21:19:12 -0700 (PDT) Date: Sat, 28 Jul 2001 21:20:03 -0700 From: Tito Ciuro To: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-3-893334421 Subject: [Metakit] Search() and Locate() are broken! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-3-893334421 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hello, To my surprise, I found out that both, Search() and Locate() are broken. If I'm not mistaken, v2.3.4r29 and v2.4.0 contain the same code for both functions. Assumptions: ------------ The data file contains a specific record with ID 604 at index 601. The file does not contain any negative numbers. There are 640 rows in total. Testing Search(): -------------- index = dataFileView.Search(pUID[604]); ---> index = 601 OK! index = dataFileView.Search(pUID[999]); ---> index = 640 (last + 1) OK! index = dataFileView.Search(pUID[-1]); ---> index = 0 WRONG! Comments on Search(): -------------------- Searching ID 604 works fine. Searching ID 999 works fine, but perhaps it would be better to obtain index = -1 (such as in Find(), for example.) Searching ID -1, which doesn't exist, returns index 0, which is wrong (index 0 contains ID 1 in my case). Testing Search(): -------------- count = dataFileView.Locate(pUID[604], &index); ---> count = 0; index = 601 WRONG! count = dataFileView.Locate(pUID[999], &index); ---> count = 0; index = 640 (last + 1) OK! count = dataFileView.Locate(pUID[-1], &index); ---> count = 0; index = 0 WRONG! Comments on Locate(): -------------------- Locating ID 604 doesn't work. It should be count = 1; index = 601. Locating ID 999 works fine, but perhaps it would be better to obtain index = -1 (such as in Find(), for example.) Locating ID -1, which doesn't exist, returns index 0, which is wrong (index 0 contains ID 1 in my case). Final comment: -------------- I can solve this issue by using Find(), but its linear search isn't as optimized as Search() or Locate(). It really shows when profiling the code... :( Has anyone seen this before? Thanks, -- Tito --Apple-Mail-3-893334421 Content-Transfer-Encoding: quoted-printable Content-Type: text/enriched; charset=us-ascii CourierHello, To my surprise, I found out that both, Search() and Locate() are broken. If I'm not mistaken, v2.3.4r29 and v2.4.0 contain the same code for both functions. Assumptions: ------------ The data file contains a specific record with ID 604 at index 601. The file does not contain any negative numbers. There are 640 rows in total. Testing Search(): -------------- index =3D dataFileView.Search(pUID[604]); ---> index =3D 601 OK! index =3D dataFileView.Search(pUID[999]); ---> index =3D 640 (last + 1) = OK! index =3D dataFileView.Search(pUID[-1]); ---> index =3D 0 WRONG! Comments on Search(): -------------------- Searching ID 604 works fine. Searching ID 999 works fine, but perhaps it would be better to obtain index =3D -1 (such as in Find(), for example.) Searching ID -1, which doesn't exist, returns index 0, which is wrong (index 0 contains ID 1 in my case). Testing Search(): -------------- count =3D dataFileView.Locate(pUID[604], &index); ---> count =3D 0; = index =3D 601 WRONG! count =3D dataFileView.Locate(pUID[999], &index); ---> count =3D 0; = index =3D 640 (last + 1) OK! count =3D dataFileView.Locate(pUID[-1], &index); ---> count =3D 0; index = =3D 0 WRONG! Comments on Locate(): -------------------- Locating ID 604 doesn't work. It should be count =3D 1; index =3D 601. Locating ID 999 works fine, but perhaps it would be better to obtain index =3D -1 (such as in Find(), for example.) Locating ID -1, which doesn't exist, returns index 0, which is wrong (index 0 contains ID 1 in my case). Final comment: -------------- I can solve this issue by using Find(), but its linear search isn't as optimized as Search() or Locate(). It really shows when profiling the code... :( Has anyone seen this before? Thanks, -- Tito= --Apple-Mail-3-893334421-- From lvirden@cas.org Sun Jul 29 04:49:03 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA29121 for ; Sun, 29 Jul 2001 04:49:03 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id f6T9mrA02589 for ; Sun, 29 Jul 2001 05:48:53 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f6T9mre25023; Sun, 29 Jul 2001 05:48:53 -0400 (EDT) Date: Sun, 29 Jul 2001 05:48:53 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010729054853.AAB25010@cas.org> To: metakit@equi4.com Subject: [Metakit] Wikit question - anyone looked into the caching code? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've an idea for what seems like a simple improvement to the wikit's caching code and wondered if anyone was familar with this part of the code? I'd suggest that as the wikit display code fetchs in the cache, that it check the state of any link marked within the cached'd page as uncreated, and if that state is no longer true, that it update the page and the cache appropriately. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From readonly@getsoft.com Sun Jul 29 04:56:57 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA29470 for ; Sun, 29 Jul 2001 04:56:57 -0500 Received: from 192.168.101.11 (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id DAA23624; Sun, 29 Jul 2001 03:57:05 -0600 (MDT) Date: Sun, 29 Jul 2001 19:56:43 +1000 From: Neville Franks X-Mailer: The Bat! (v1.53d) UNREG / CD5BF9353B3B7091 Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <143105308669.20010729195643@getsoft.com> To: Tito Ciuro CC: metakit@equi4.com Subject: Re: [Metakit] Search() and Locate() are broken! In-Reply-To: <200107290418.XAA19192@trixie.triqs.com> References: <200107290418.XAA19192@trixie.triqs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Tito, Sunday, July 29, 2001, 2:20:03 PM, you wrote: TC> Hello, TC> To my surprise, I found out that both, Search() and Locate() are broken. TC> If I'm not mistaken, v2.3.4r29 and v2.4.0 contain the same code for both TC> functions. ..... TC> Has anyone seen this before? I haven't used Metakit for some time, however I did find the sort code was buggy. You should find an earlier post from me about this. I'm assuming Search() and Locate() work on a sorted view so if this is right then the sort may be the cause of the problem. -- Best regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From gmcm@hypernet.com Sun Jul 29 06:30:25 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA00802 for ; Sun, 29 Jul 2001 06:30:24 -0500 Received: from me (204.176.40.43) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sun, 29 Jul 2001 07:24:48 -0400 From: "Gordon McMillan" To: Tito Ciuro , metakit@equi4.com Date: Sun, 29 Jul 2001 07:29:19 -0400 Subject: Re: [Metakit] Search() and Locate() are broken! Reply-to: gmcm@hypernet.com Message-ID: <3B63BB4F.5013.70E6B97@localhost> Priority: normal In-reply-to: <200107290418.XAA19192@trixie.triqs.com> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, > index = dataFileView.Search(pUID[604]); ---> index = 601 OK! > index = dataFileView.Search(pUID[999]); ---> index = 640 (last + > 1) OK! index = dataFileView.Search(pUID[-1]); ---> index = 0 > WRONG! No, these are correct. Search tells you where the value is or where it belongs (where it should be inserted to preserve order). It does not tell if it exists. > count = dataFileView.Locate(pUID[604], &index); ---> count = 0; > index = 601 WRONG! count = dataFileView.Locate(pUID[999], > &index); ---> count = 0; index = 640 (last + 1) OK! count = > dataFileView.Locate(pUID[-1], &index); ---> count = 0; index = 0 > WRONG! I'm not familiar with Locate, but my guess is that the last test is OK (like Search, telling you where to insert). Looks like you did find a bug with the first test. - Gordon From titoffen@pacbell.net Sun Jul 29 11:54:45 2001 Received: from snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA03888 for ; Sun, 29 Jul 2001 11:54:44 -0500 Message-Id: <200107291654.LAA03888@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta5.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GH80052JUZU7B@mta5.snfc21.pbi.net> for metakit@equi4.com; Sun, 29 Jul 2001 09:55:07 -0700 (PDT) Date: Sun, 29 Jul 2001 09:55:58 -0700 From: Tito Ciuro Subject: Re: [Metakit] Search() and Locate() are broken! In-reply-to: <3B63BB4F.5013.70E6B97@localhost> To: gmcm@hypernet.com Cc: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-2-938689125 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2-938689125 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hi Gordon, On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: > Tito, > >> index = dataFileView.Search(pUID[604]); ---> index = 601 OK! >> index = dataFileView.Search(pUID[999]); ---> index = 640 (last + >> 1) OK! >> index = dataFileView.Search(pUID[-1]); ---> index = 0 >> WRONG! > > No, these are correct. Search tells you where the value is or > where it belongs (where it should be inserted to preserve > order). It does not tell if it exists. Huh?? I would expect something like PositionWhenInserted() to do that, but certainly not Search(). Shouldn't Search() tell me where the match (that is, the index) exists? (either -1 or an out-of-bounds value if it doesn't would be great) In my first post above, how good is Search() if it can't distinguish between 0 being a "this is where it should go" kind of index and 0 being a "it doesn't exist" kind of index? In other words, if Search() returns an index, how do we know if it found (or didn't find) a match? >> count = dataFileView.Locate(pUID[604], &index); ---> count = 0; >> index = 601 WRONG! >> count = dataFileView.Locate(pUID[999], >> &index); ---> count = 0; index = 640 (last + 1) OK! >> count = dataFileView.Locate(pUID[-1], &index); ---> count = 0; >> index = 0 >> WRONG! > > I'm not familiar with Locate, but my guess is that the last test > is OK (like Search, telling you where to insert). Looks like you > did find a bug with the first test. Locate() is defined in c4_View in the following manner: > /// Return number of matching keys, and pos of first one as arg > int c4_View::Locate(const c4_RowRef& crit_, int* pos_) const It seems a pretty straightforward call... Thanks for replying Gordon! -- Tito --Apple-Mail-2-938689125 Content-Transfer-Encoding: quoted-printable Content-Type: text/enriched; charset=us-ascii Hi Gordon, On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: Tito, index =3D dataFileView.Search(pUID[604]); ---> index =3D 601 = OK! index =3D dataFileView.Search(pUID[999]); ---> index =3D 640 (last + 1) OK! index =3D dataFileView.Search(pUID[-1]); ---> index =3D 0 WRONG! No, these are correct. Search tells you where the value is or=20 where it belongs (where it should be inserted to preserve=20 order). It does not tell if it exists. Huh?? I would expect something like PositionWhenInserted() to do that, but certainly not Search(). Shouldn't Search() tell me where the match (that is, the index) exists? (either -1 or an out-of-bounds value if it doesn't would be great) In my first post above, how good is Search() if it can't distinguish between 0 being a "this is where it should go" kind of index and 0 being a "it doesn't exist" kind of index? In other words, if Search() returns an index, how do we know if it found (or didn't find) a match?=20 0000,0000,DEB7 count =3D dataFileView.Locate(pUID[604], &index); ---> count =3D 0; index =3D 601 WRONG! count =3D dataFileView.Locate(pUID[999], &index); ---> count =3D 0; index =3D 640 (last + 1) = OK! count =3D dataFileView.Locate(pUID[-1], &index); ---> count =3D 0; index =3D 0 WRONG! I'm not familiar with Locate, but my guess is that the last test=20 is OK (like Search, telling you where to insert). Looks like you=20 did find a bug with the first test. Locate() is defined in c4_View in the following manner: /// Return number of matching keys, and pos of first one as arg int c4_View::Locate(const c4_RowRef& crit_, int* pos_) = const0000,6363,1212 It seems a pretty straightforward call... Thanks for replying Gordon! -- Tito= --Apple-Mail-2-938689125-- From gmcm@hypernet.com Sun Jul 29 12:49:06 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA04407 for ; Sun, 29 Jul 2001 12:49:06 -0500 Received: from me (204.176.40.129) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sun, 29 Jul 2001 13:43:28 -0400 From: "Gordon McMillan" To: Tito Ciuro Date: Sun, 29 Jul 2001 13:47:59 -0400 Subject: Re: [Metakit] Search() and Locate() are broken! Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3B64140F.24405.869181A@localhost> Priority: normal References: <3B63BB4F.5013.70E6B97@localhost> In-reply-to: <1215697061-57448175@hypernet.com> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, I'm not defending the API, just pointing out how it works. It looks like Locate was an attempt to overcome the awkward API of Search, but I haven't really adjusted to newer releases of MetaKit yet. All of my (old) code tests the index for out-of- bounds, and then the value of the property after the Search. You should probably post the full text of your Locate test so Jean-Claude can look at it when he gets home next week. (And I'll upgrade my code to use Locate, since it does what I want!) - Gordon > On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: > > > Tito, > > > >> index = dataFileView.Search(pUID[604]); ---> index = 601 OK! > >> index = dataFileView.Search(pUID[999]); ---> index = 640 (last > >> + 1) OK! index = dataFileView.Search(pUID[-1]); ---> index = 0 > >> WRONG! > > > > No, these are correct. Search tells you where the value is or > > where it belongs (where it should be inserted to preserve > > order). It does not tell if it exists. > > Huh?? I would expect something like PositionWhenInserted() to do > that, but certainly not Search(). Shouldn't Search() tell me > where the match (that is, the index) exists? (either -1 or an > out-of-bounds value if it doesn't would be great) > > In my first post above, how good is Search() if it can't > distinguish between 0 being a "this is where it should go" kind > of index and 0 being a "it doesn't exist" kind of index? > > In other words, if Search() returns an index, how do we know if > it found (or didn't find) a match? > > >> count = dataFileView.Locate(pUID[604], &index); ---> count = > >> 0; index = 601 WRONG! count = dataFileView.Locate(pUID[999], > >> &index); ---> count = 0; index = 640 (last + 1) OK! count = > >> dataFileView.Locate(pUID[-1], &index); ---> count = 0; index = > >> 0 WRONG! > > > > I'm not familiar with Locate, but my guess is that the last > > test is OK (like Search, telling you where to insert). Looks > > like you did find a bug with the first test. > > Locate() is defined in c4_View in the following manner: > > > /// Return number of matching keys, and pos of first one as arg > > int c4_View::Locate(const c4_RowRef& crit_, int* pos_) const > > It seems a pretty straightforward call... > > Thanks for replying Gordon! > > -- Tito > From titoffen@pacbell.net Sun Jul 29 18:20:39 2001 Received: from pltn13.pbi.net (mta7.pltn13.pbi.net [64.164.98.8]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA08709 for ; Sun, 29 Jul 2001 18:20:38 -0500 Message-Id: <200107292320.SAA08709@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta7.pltn13.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GH90009JCUZPY@mta7.pltn13.pbi.net> for metakit@equi4.com; Sun, 29 Jul 2001 16:20:59 -0700 (PDT) Date: Sun, 29 Jul 2001 16:21:34 -0700 From: Tito Ciuro Subject: Re: [Metakit] Search() and Locate() are broken! In-reply-to: <3B64140F.24405.869181A@localhost> To: gmcm@hypernet.com Cc: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-1-961825626 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-961825626 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hello Gordon, On Sunday, July 29, 2001, at 10:47 AM, Gordon McMillan wrote: > Tito, > > I'm not defending the API, just pointing out how it works. I'm sorry I gave you the impression I was upset. I have been working all night long trying to optimize my code, only to find out that the *most* critical part of the code is broken (or so it seems!) If Search() or Locate() worked (remember, I need to find out whether negative values exist), the app would run zippy. Instead, when the file is somewhat large (2 MB/700 rows), the time it takes to perform a Find() is just too long. An binary search would kick some serious butt. Given that with both, Search() and Locate(), there isn't a way to know whether it found a match or not, I'm stuck with the less optimized Find(). I have nothing against the API though!... So, I want to extend my thanks to you and Neville Franks for taking the time to answer my questions.concerns. :-) > It looks like Locate was an attempt to overcome the awkward > API of Search, but I haven't really adjusted to newer releases > of MetaKit yet. All of my (old) code tests the index for out-of- > bounds, and then the value of the property after the Search. If you look for a negative number that doesn't exist in your data file, what index do you get back? > You should probably post the full text of your Locate test so > Jean-Claude can look at it when he gets home next week. > (And I'll upgrade my code to use Locate, since it does what I > want!) > - Gordon What do you mean by the full text of the Locate() test? The test is very easy: try to execute a query as stated above. Thank you very much, -- Tito >> On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: >> >>> Tito, >>> >>>> index = dataFileView.Search(pUID[604]); ---> index = 601 OK! >>>> index = dataFileView.Search(pUID[999]); ---> index = 640 (last >>>> + 1) OK! index = dataFileView.Search(pUID[-1]); ---> index = 0 >>>> WRONG! >>> >>> No, these are correct. Search tells you where the value is or >>> where it belongs (where it should be inserted to preserve >>> order). It does not tell if it exists. >> >> Huh?? I would expect something like PositionWhenInserted() to do >> that, but certainly not Search(). Shouldn't Search() tell me >> where the match (that is, the index) exists? (either -1 or an >> out-of-bounds value if it doesn't would be great) >> >> In my first post above, how good is Search() if it can't >> distinguish between 0 being a "this is where it should go" kind >> of index and 0 being a "it doesn't exist" kind of index? >> >> In other words, if Search() returns an index, how do we know if >> it found (or didn't find) a match? >> >>>> count = dataFileView.Locate(pUID[604], &index); ---> count = >>>> 0; index = 601 WRONG! count = dataFileView.Locate(pUID[999], >>>> &index); ---> count = 0; index = 640 (last + 1) OK! count = >>>> dataFileView.Locate(pUID[-1], &index); ---> count = 0; index = >>>> 0 WRONG! >>> >>> I'm not familiar with Locate, but my guess is that the last >>> test is OK (like Search, telling you where to insert). Looks >>> like you did find a bug with the first test. >> >> Locate() is defined in c4_View in the following manner: >> >>> /// Return number of matching keys, and pos of first one as arg >>> int c4_View::Locate(const c4_RowRef& crit_, int* pos_) const >> >> It seems a pretty straightforward call... >> >> Thanks for replying Gordon! >> >> -- Tito >> > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > ____________________________________________ "The pen is mightier than the sword, and considerably easier to write with." Marty Feldman --Apple-Mail-1-961825626 Content-Transfer-Encoding: quoted-printable Content-Type: text/enriched; charset=us-ascii Hello Gordon, On Sunday, July 29, 2001, at 10:47 AM, Gordon McMillan wrote: Tito, I'm not defending the API, just pointing out how it works. I'm sorry I gave you the impression I was upset. I have been working all night long trying to optimize my code, only to find out that the *most* critical part of the code is broken (or so it seems!) If Search() or Locate() worked (remember, I need to find out whether negative values exist), the app would run zippy. Instead, when the file is somewhat large (2 MB/700 rows), the time it takes to perform a Find() is just too long. An binary search would kick some serious butt. Given that with both, Search() and Locate(), there isn't a way to know whether it found a match or not, I'm stuck with the less optimized Find(). I have nothing against the API though!... So, I want to extend my thanks to you and Neville Franks for taking the time to answer my questions.concerns. :-) It looks like Locate was an attempt to overcome the awkward=20 API of Search, but I haven't really adjusted to newer releases=20 of MetaKit yet. All of my (old) code tests the index for out-of- bounds, and then the value of the property after the Search. If you look for a negative number that doesn't exist in your data file, what index do you get back? 0000,0000,DEB7 You should probably post the full text of your Locate test so=20 Jean-Claude can look at it when he gets home next week.=20 (And I'll upgrade my code to use Locate, since it does what I=20 want!)0000,0000,DEB7 0000,0000,DEDE- Gordon What do you mean by the full text of the Locate() test? The test is very easy: try to execute a query as stated above. Thank you very much, -- Tito 0000,0000,DEB7 On Sunday, July 29, 2001, at 04:29 AM, Gordon McMillan wrote: Tito, index =3D dataFileView.Search(pUID[604]); ---> index =3D 601 = OK! index =3D dataFileView.Search(pUID[999]); ---> index =3D 640 (last + 1) OK! index =3D dataFileView.Search(pUID[-1]); ---> index =3D 0 WRONG! No, these are correct. Search tells you where the value is or where it belongs (where it should be inserted to preserve order). It does not tell if it exists. Huh?? I would expect something like PositionWhenInserted() to do that, but certainly not Search(). Shouldn't Search() tell me where the match (that is, the index) exists? (either -1 or an out-of-bounds value if it doesn't would be great) In my first post above, how good is Search() if it can't distinguish between 0 being a "this is where it should go" kind of index and 0 being a "it doesn't exist" kind of index? In other words, if Search() returns an index, how do we know if it found (or didn't find) a match? count =3D dataFileView.Locate(pUID[604], &index); ---> count =3D 0; index =3D 601 WRONG! count =3D dataFileView.Locate(pUID[999], &index); ---> count =3D 0; index =3D 640 (last + 1) OK! count =3D dataFileView.Locate(pUID[-1], &index); ---> count =3D 0; index =3D 0 WRONG! I'm not familiar with Locate, but my guess is that the last test is OK (like Search, telling you where to insert). Looks like you did find a bug with the first test. Locate() is defined in c4_View in the following manner: /// Return number of matching keys, and pos of first one as arg int c4_View::Locate(const c4_RowRef& crit_, int* pos_) const It seems a pretty straightforward call... Thanks for replying Gordon! -- Tito _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit = Verdana________________________________= ____________ 0000,0000,FFFF"The pen is mightier than the sword, and considerably easier to write with." 0000,6666,3333 Marty Feldman= --Apple-Mail-1-961825626-- From titoffen@pacbell.net Sun Jul 29 18:30:23 2001 Received: from snfc21.pbi.net (mta6.snfc21.pbi.net [206.13.28.240]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA08805 for ; Sun, 29 Jul 2001 18:30:23 -0500 Message-Id: <200107292330.SAA08805@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta6.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GH900G4ZDB706@mta6.snfc21.pbi.net> for metakit@equi4.com; Sun, 29 Jul 2001 16:30:43 -0700 (PDT) Date: Sun, 29 Jul 2001 16:31:35 -0700 From: Tito Ciuro Subject: Re: [Metakit] Search() and Locate() are broken! In-reply-to: <200107300916450047.1A29DC88@getsoft.com> To: readonly@getsoft.com Cc: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-2-962427212 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2-962427212 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hello Neville! On Sunday, July 29, 2001, at 04:16 PM, Neville Franks wrote: > If the column you want to search on is not in a sorted order then you > can create a sorted (derived) view to Search on. Because the sort > didn't always sort correctly, search would fail or return the wrong > thing. Humm... interesting. Let's see: the first column in my view contains the UIDs. These are all sorted physically in the file (executing Dump verifies that). Could it be that when I obtain a View via GetAs() the sort is not honored? Should I obtain a sorted view instead? I see some sort-related functions... how could I specify a sorted view based on a specific column? Another question: I wrote a getRowForUID() function, which basically returns a c4_RowRef. I use this function a lot, and this is where Search() would save *a lot* of time. So wouldn't be too costly to get a sorted view each time I needed to search for a particular UID? > I gave up on trying to use MK because of problems like this and others, > which was a pity as it is nice in many ways. That's too bad. I have looked a several other solutions and I must say that I *really like* MK. It sure rocks! :-) > Hope that helps. Thanks so much! -- Tito --Apple-Mail-2-962427212 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=us-ascii Hello Neville! On Sunday, July 29, 2001, at 04:16 PM, Neville Franks wrote: 0000,0000,DEDEIf the column you want to search on is not in a sorted order then you can create a sorted (derived) view to Search on. Because the sort didn't always sort correctly, search would fail or return the wrong thing. Humm... interesting. Let's see: the first column in my view contains the UIDs. These are all sorted physically in the file (executing Dump verifies that). Could it be that when I obtain a View via GetAs() the sort is not honored? Should I obtain a sorted view instead? I see some sort-related functions... how could I specify a sorted view based on a specific column? Another question: I wrote a getRowForUID() function, which basically returns a c4_RowRef. I use this function a lot, and this is where Search() would save *a lot* of time. So wouldn't be too costly to get a sorted view each time I needed to search for a particular UID? 0000,0000,DEDE I gave up on trying to use MK because of problems like this and others, which was a pity as it is nice in many ways. That's too bad. I have looked a several other solutions and I must say that I *really like* MK. It sure rocks! :-) 0000,0000,DEDE Hope that helps. Thanks so much! -- Tito --Apple-Mail-2-962427212-- From readonly@getsoft.com Sun Jul 29 19:43:27 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA09497 for ; Sun, 29 Jul 2001 19:43:27 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id SAA12410; Sun, 29 Jul 2001 18:43:23 -0600 (MDT) Message-ID: <200107301042480221.1A78A52E@getsoft.com> In-Reply-To: <200107292330.SAA08805@trixie.triqs.com> References: <200107292330.SAA08805@trixie.triqs.com> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Mon, 30 Jul 2001 10:42:48 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "Tito Ciuro" Cc: metakit@equi4.com Subject: Re: [Metakit] Search() and Locate() are broken! Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id TAA09497 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Tito, On 29 Jul 2001 at 16:31 Tito Ciuro wrote: >Hello Neville! > >On Sunday, July 29, 2001, at 04:16 PM, Neville Franks wrote: > >> If the column you want to search on is not in a sorted order then you >> can create a sorted (derived) view to Search on. Because the sort >> didn't always sort correctly, search would fail or return the wrong >> thing. > >Humm... interesting. Let's see: the first column in my view contains the >UIDs. These are all sorted physically in the file (executing Dump >verifies that). Rows are either added to the end of a view or inserted, depending on what you are doing. So the result of these operations is the natural order. >Could it be that when I obtain a View via GetAs() the sort is not >honored? Should I obtain a sorted view instead? I see some sort-related >functions... how could I specify a sorted view based on a specific >column? GetAs() doesn't do any sorting. >Another question: I wrote a getRowForUID() function, which basically >returns a c4_RowRef. I use this function a lot, and this is where >Search() would save *a lot* of time. So wouldn't be too costly to get a >sorted view each time I needed to search for a particular UID? A sorted view is a map of the underlying real view. When items are added etc. to the real view they appear in the sorted view so you don't need to create new sorted views. >> I gave up on trying to use MK because of problems like this and others, >> which was a pity as it is nice in many ways. > >That's too bad. I have looked a several other solutions and I must say >that I *really like* MK. It sure rocks! :-) You mentioned in your other post that you have 700 rows. I would have thought Find would be fast enough as this realy isn't many rows. I ran into problems when I got over 10,000 rows. If your DB is going to scale up to this sort of size I suggest you start working with it at these larger sizes now. Best Regards, Neville Franks (Author of ED for Windows) PS. Did you know ED4W Version V4 has been released? This is a major new release with lots of great new features and enhancements. See our What's New page http://www.getsoft.com/whats_new.html for details. And while you're there you can click on 'Stay Informed' to join our mailing list. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From gmcm@hypernet.com Sun Jul 29 20:23:49 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA09829 for ; Sun, 29 Jul 2001 20:23:45 -0500 Received: from me (204.176.40.53) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sun, 29 Jul 2001 21:17:59 -0400 From: "Gordon McMillan" To: Tito Ciuro Date: Sun, 29 Jul 2001 21:22:27 -0400 Subject: Re: [Metakit] Search() and Locate() are broken! Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3B647E93.21055.A092A47@localhost> Priority: normal In-reply-to: <200107292320.SAA08709@trixie.triqs.com> References: <3B64140F.24405.869181A@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, > ... I have been > working all night long trying to optimize my code, only to find > out that the *most* critical part of the code is broken (or so it > seems!) If Search() or Locate() worked (remember, I need to find > out whether negative values exist), the app would run zippy. > Instead, when the file is somewhat large (2 MB/700 rows), the > time it takes to perform a Find() is just too long. An binary > search would kick some serious butt. > > Given that with both, Search() and Locate(), there isn't a way to > know whether it found a match or not, I'm stuck with the less > optimized Find(). I have nothing against the API though!... Locate() *should* fill your needs. Search() does, it's just awkward (you need to check the row at the returned index to see if the UID matches what you Search()ed on). > If you look for a negative number that doesn't exist in your data > file, what index do you get back? I believe in your example, all UIDs were positive, so any negative UID would return 0. > What do you mean by the full text of the Locate() test? The test > is very easy: try to execute a query as stated above. Well, what I was actually getting at was "is the view sorted correctly", since Search() & Locate() both only work on views sorted on the property you're searching on. From your discussion with Neville: > Humm... interesting. Let's see: the first column in my view > contains the UIDs. These are all sorted physically in the > file (executing Dump verifies that). > Could it be that when I obtain a View via GetAs() the sort is > not honored? No. While that *could* affect the structure, I'm pretty sure it won't affect the order. > Should I obtain a sorted view instead? I see > some sort-related functions... how could I specify a sorted > view based on a specific column? To be safe, I'd sort it when first opened. For inserts, I'd use InsertAt(), using the index returned by Search() or Locate(). That should be enough (if you're sure that's the only way rows get added, you only need to sort once). If UID is your first property, then view->Sort() will do it. > Another question: I wrote a getRowForUID() function, which > basically returns a c4_RowRef. I use this function a lot, and > this is where Search() would save *a lot* of time. So wouldn't > be too costly to get sorted view each time I needed to > search for a particular UID? It would be costly (although it's only indexes getting shuffled, not data), but it shouldn't be necessary. I've never observed Neville's problems with sorted views, but I've never used MetaKit beyond a few 10Ks of rows. I have been surprised at MetaKit being case-insensitive, but that's never actually hurt me. - Gordon From readonly@getsoft.com Sun Jul 29 21:09:48 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA10286 for ; Sun, 29 Jul 2001 21:09:47 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id UAA21517; Sun, 29 Jul 2001 20:09:48 -0600 (MDT) Message-ID: <200107301209130247.1AC7C330@getsoft.com> In-Reply-To: <3B647E93.21055.A092A47@localhost> References: <3B64140F.24405.869181A@localhost> <3B647E93.21055.A092A47@localhost> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Mon, 30 Jul 2001 12:09:13 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: gmcm@hypernet.com, "Tito Ciuro" Cc: metakit@equi4.com Subject: Re: [Metakit] Search() and Locate() are broken! Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id VAA10286 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Gordon, >It would be costly (although it's only indexes getting shuffled, >not data), but it shouldn't be necessary. I've never observed >Neville's problems with sorted views, but I've never used >MetaKit beyond a few 10Ks of rows. If you scan the entire View and compare the row.sort clm value with the previous row using the sort comparision function you may find the sort order is wrong. I rracked this down to a problem with the MK Merge Sort code but wasn't able to fix it. >I have been surprised at MetaKit being case-insensitive, but >that's never actually hurt me. I sent an update to JC which let you use your own sort comparison function, but I don't know if it has ever been incorporated into MK. Best Regards, Neville Franks (Author of ED for Windows) PS. Did you know ED4W Version V4 has been released? This is a major new release with lots of great new features and enhancements. See our What's New page http://www.getsoft.com/whats_new.html for details. And while you're there you can click on 'Stay Informed' to join our mailing list. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From readonly@getsoft.com Mon Jul 30 01:46:34 2001 Received: from getsoft.com (getsoft.com [207.159.132.217]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA13587 for ; Mon, 30 Jul 2001 01:46:34 -0500 Received: from soft-p3-nf (co3005849-a.eburwd1.vic.optushome.com.au [203.164.66.44]) by getsoft.com (8.9.3/8.9.3) with ESMTP id AAA23347; Mon, 30 Jul 2001 00:46:43 -0600 (MDT) Message-ID: <200107301646090510.1BC54E77@getsoft.com> In-Reply-To: <200107300622.AAA19906@getsoft.com> References: <200107300622.AAA19906@getsoft.com> X-Mailer: Calypso Version 3.10.03.02 (4) Date: Mon, 30 Jul 2001 16:46:09 +1000 Reply-To: readonly@getsoft.com From: "Neville Franks" To: "Tito Ciuro" Cc: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id BAA13587 Subject: [Metakit] Re: Still not working... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 29 Jul 2001 at 23:22 Tito Ciuro wrote: >On Sunday, July 29, 2001, at 10:57 PM, Neville Franks wrote: > >> Hi Tito, >> I'm relying just on my memory here as I can't find my MK code but I >> thought you created you view as normal, and then created a sorted view >> mapped on top of the real view. For insert/add you work with the >> underlying real view and just use the sorted view for lookup purposes. >> >> Hopefully there is an example somewhere. > >Hello guys, > >I guess I didn't explain myself properly before. One of my questions was >whether the sort view would be always be kept in sync with the real view. > >Obviously, If I start the app my mapping the sort view on top of the >real view, both are considered to hold the same information; they're in >sync. > >However, as I add/edit/delete information from the real view, would the >sort view "notice" this changes? Or do I have to call Sort() prior to >performing a query? (my guess is that this methodology would be very >expensive) > >If on the other hand, both views are kept in sync automatically, then >great! Then there's little more to add to the code to make it work. My understanding is that the sorted view is kept in sync with the real view. The sort view is in fact just pointers into the real view. The only cost is the time to do the sort in the first place. Also because the sorted view doesn't persist you have to do this each time you open the database. For a large database this can be an expensive operation. This is an area where persistant indexing, such as a btree would be very usefull. >I'll have to continue tomorrow. It's 11:20 PM and I'm exhausted! :-) > >Have a great Monday and again, a million more thanks, > >-- Tito Best Regards, Neville Franks (Author of ED for Windows) PS. Did you know ED4W Version V4 has been released? This is a major new release with lots of great new features and enhancements. See our What's New page http://www.getsoft.com/whats_new.html for details. And while you're there you can click on 'Stay Informed' to join our mailing list. -------------------------------------------------------------- Soft As It Gets P/L, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Developers and publishers of ED for Windows Ph +61 3 9885 4445 WWW http://www.getsoft.com Fax +61 3 9885 4444 Email sales@getsoft.com --------------------------------------------------------------- From guenther.fischer@hrz.tu-chemnitz.de Tue Jul 31 05:04:01 2001 Received: from obelix.hrz.tu-chemnitz.de (obelix.hrz.tu-chemnitz.de [134.109.132.55]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA31431 for ; Tue, 31 Jul 2001 05:04:00 -0500 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by obelix.hrz.tu-chemnitz.de with esmtp (Exim 3.22 #4) id 15RWNq-0001ti-00 for metakit@equi4.com; Tue, 31 Jul 2001 12:04:14 +0200 Date: Tue, 31 Jul 2001 12:04:14 +0200 (MEST) From: Guenther Fischer To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The Mk4py-dok is a little bit small ... I have a database application with Mk4. I can now read it in python and show all the rows, but how to update? def SaveEntry(f, id): ix = vw.find(ID=id) # vw is the view as global if f.NAME.IsModified(): # was the presented entry modified? vw.insert(ix,NAME=f.NAME.GetValue()) # NAME is a property name but nothing will happens - no error too -- ~Guenther Fischer From gmcm@hypernet.com Tue Jul 31 09:05:54 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA08374 for ; Tue, 31 Jul 2001 09:05:53 -0500 Received: from me (204.176.40.60) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Tue, 31 Jul 2001 10:00:07 -0400 From: "Gordon McMillan" To: Guenther Fischer , Date: Tue, 31 Jul 2001 10:04:34 -0400 Subject: Re: [Metakit] Mk4py Reply-to: gmcm@hypernet.com Message-ID: <3B6682B2.17552.11E94412@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: > > I have a database application with Mk4. I can now read it in > python and show all the rows, but how to update? > > def SaveEntry(f, id): > ix = vw.find(ID=id) # vw is the view as global if > f.NAME.IsModified(): # was the presented entry > modified? > vw.insert(ix,NAME=f.NAME.GetValue()) # NAME is a property > name > > but nothing will happens - no error too First make sure the storage is open for update. You'll also need a commit at some point to make changes permanent. What your code is doing is adding a second row with only one property filled in. You probably want something more like: if f.NAME.IsModified(): ix = vw.find(ID=id) if ix > -1: vw[ix].NAME=f.NAME.GetValue() db.commit() # can be done later else: # error - ID not found - Gordon From acarter@covad.net Tue Jul 31 19:43:03 2001 Received: from smtp6.covad.net (psmtp6.array3.laserlink.net [63.65.123.56]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA27683 for ; Tue, 31 Jul 2001 19:43:02 -0500 Received: from acarter1 (tide79.microsoft.com [131.107.3.79]) by smtp6.covad.net (8.9.3/8.9.3) with ESMTP id UAA11581 for ; Tue, 31 Jul 2001 20:43:14 -0400 (EDT) From: "Andrew Carter" To: Date: Tue, 31 Jul 2001 17:43:13 -0700 Message-ID: <87C80E823E824644BCB711D6348262B50181A5FB@red-msg-05.redmond.corp.microsoft.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0008_01C119E8.46C95030" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2526.0000 Importance: Normal Subject: [Metakit] Primary key generation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0008_01C119E8.46C95030 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit I'm just getting started with MetaKit. I like what I've seen so far. I will probably want to maintain mostly relational-style data in my data files. As such, I would like to generate primary key values for rows of data. Is there an easy way to do this? One way would be to use a MAX() type function - I couldn't find this built-into the tool set. Also, is there any way to enforce constraints like unique properties? If not, this would be a useful future feature. Thanks, Andrew ------=_NextPart_000_0008_01C119E8.46C95030 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I’m just getting started with MetaKit.  I = like what I’ve seen so far.  I will probably want to maintain mostly = relational-style data in my data files.  As such, I would like to generate primary key = values for rows of data.  Is there an easy way to do this?  One way would be = to use a MAX() type function – I couldn’t find this built-into the tool = set.  Also, is there any way to enforce constraints like unique properties?  If = not, this would be a useful future feature.


Thanks,

Andrew

------=_NextPart_000_0008_01C119E8.46C95030-- From titoffen@pacbell.net Fri Aug 3 03:02:25 2001 Received: from snfc21.pbi.net (mta6.snfc21.pbi.net [206.13.28.240]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA17740 for ; Fri, 3 Aug 2001 03:02:25 -0500 Message-Id: <200108030802.DAA17740@trixie.triqs.com> Received: from localhost ([209.233.25.139]) by mta6.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GHH00KUAFNXYC@mta6.snfc21.pbi.net> for metakit@equi4.com; Fri, 03 Aug 2001 01:02:22 -0700 (PDT) Date: Fri, 03 Aug 2001 01:03:20 -0700 From: Tito Ciuro To: metakit@equi4.com MIME-version: 1.0 (Apple Message framework v404.1) X-Mailer: Apple Mail (2.404.1) Content-type: multipart/alternative; boundary=Apple-Mail-2--808751601 Subject: [Metakit] MK Transaction Modes Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2--808751601 Content-Transfer-Encoding: 7bit Content-Type: text/plain; format=flowed; charset=us-ascii Hello! After testing and watching MK closely, I realized what is slowing down my queries. A little background: I wrote an API to access the data from multiple processes (Unix). The file is open in "commit-extend" mode, which makes the file grow enormously when saving changes. Just to give you a quick example, saving a single string (17 chars) increases the file size by about 16KB. Yikes! The documentation is very clear stating that saving in commit-extend mode makes the file grow very quickly. Browsing MK home page, I just came across the following link: http://www.equi4.com/metakit/wiki.cgi/82.html In particular, I believe that a model that will work better for me is the following one (I have selected the important bits in bold): > Combined modes > > The features described above open a range of options for dealing with > multi-user / multi-tasking scenario's, even though they do not yet > handle all possible cases. > > First of all, commit-extend offers extreme performance in the case when > lots of readers require access to a consistent state, but only few > changes are made. Readers will see the state at time of opening the > datafile (they can close and re-open any time to see new committed > changes). This mode is efficient because readers never wait, not even > while the extender is committing changes. > > The drawback of commti extend is the potentially large disk space > consumption, and hence the need to reclaim and clean up periodically. > This requires exclusive access. > > Commit-aside can be very effective for single-task situations with lots > of commits, since commit performance is relatively high, and disk space > consuption is limited. But again, at some point, changes will need to > be consolidated, and again enclusive access mode is needed to fold all > changes back into the main datafile and to clear the aside file. > > Finally, and interesting option is to use commit aside with a read-only > main datafile, and commit-extend for the secondary aside file. Since > only changes are saved, the aside file will not grow that quickly, and > since both files support concurrency, multiple readers can get at the > latest consistent state without delay. The only contention is that if > there are multiple writers, these will have to find a way to take turns > in making changes. > > Yet more scenarios will become available later, once the main datafile > can also be used as the aside file. This is currently not allowed. Has anyone tried this "commit aside" option. I've tried, but without success: 1) Open a c4_Storage in ~/Library/Test/mainFile. I set the mode to read only (I tried both, "read-only, original contents" and "read-only, most recently committed contents") 2) Open the second c4_Storage in ~/Library/Test/mainFile_aside. I set the mode to commit-extend. 3) Called mainFile->storage->SetAside(mainFile_aside). 4) Perform all operations via mainFile. Questions: 1) assuming the code works with one c4_Storage/commit-extend mode, why is this commit-extend-aside mode not working? 2) should I deal with the aside file directly, or is MK handling everything behind the scenes? 3) With multiple readers, I assume I have to open *both files* in read only, right? Then on save, reopen in exclusive write and perform the save. The doubt I have is whether I have to reopen only the aside file or also the main file? It's a little confusing because the doc doesn't mention much. Also, the FAQ doesn't contain an entry which deals with this issue. Last, but not least, I was wondering if there's a simple way to compact the data (in this commit-extend-aside model). Thanks a lot in advance, -- Tito --Apple-Mail-2--808751601 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=us-ascii Hello! After testing and watching MK closely, I realized what is slowing down my queries. A little background: I wrote an API to access the data from multiple processes (Unix). The file is open in "commit-extend" mode, which makes the file grow enormously when saving changes. Just to give you a quick example, saving a single string (17 chars) increases the file size by about 16KB. Yikes! The documentation is very clear stating that saving in commit-extend mode makes the file grow very quickly. Browsing MK home page, I just came across the following link: http://www.equi4.com/metakit/wiki.cgi/82.html In particular, I believe that a model that will work better for me is the following one (I have selected the important bits in bold): Combined modes The features described above open a range of options for dealing with multi-user / multi-tasking scenario's, even though they do not yet handle all possible cases. First of all, commit-extend offers extreme performance in the case when lots of readers require access to a consistent state, but only few changes are made. Readers will see the state at time of opening the datafile (they can close and re-open any time to see new committed changes). This mode is efficient because readers never wait, not even while the extender is committing changes. The drawback of commti extend is the potentially large disk space consumption, and hence the need to reclaim and clean up periodically. This requires exclusive access. Commit-aside can be very effective for single-task situations with lots of commits, since commit performance is relatively high, and disk space consuption is limited. But again, at some point, changes will need to be consolidated, and again enclusive access mode is needed to fold all changes back into the main datafile and to clear the aside file. Finally, and interesting option is to use commit aside with a read-only main datafile, and commit-extend for the secondary aside file. Since only changes are saved, the aside file will not grow that quickly, and since both files support concurrency, multiple readers can get at the latest consistent state without delay. The only contention is that if there are multiple writers, these will have to find a way to take turns in making changes. Yet more scenarios will become available later, once the main datafile can also be used as the aside file. This is currently not allowed. Has anyone tried this "commit aside" option. I've tried, but without success: 1) Open a c4_Storage in ~/Library/Test/mainFile. I set the mode to read only (I tried both, "read-only, original contents" and "read-only, most recently committed contents") 2) Open the second c4_Storage in ~/Library/Test/mainFile_aside. I set the mode to commit-extend. 3) Called mainFile->storage->SetAside(mainFile_aside). 4) Perform all operations via mainFile. Questions: 1) assuming the code works with one c4_Storage/commit-extend mode, why is this commit-extend-aside mode not working? 2) should I deal with the aside file directly, or is MK handling everything behind the scenes? 3) With multiple readers, I assume I have to open *both files* in read only, right? Then on save, reopen in exclusive write and perform the save. The doubt I have is whether I have to reopen only the aside file or also the main file? It's a little confusing because the doc doesn't mention much. Also, the FAQ doesn't contain an entry which deals with this issue. Last, but not least, I was wondering if there's a simple way to compact the data (in this commit-extend-aside model). Thanks a lot in advance, -- Tito --Apple-Mail-2--808751601-- From tciuro@yahoo.com Fri Aug 10 18:10:36 2001 Received: from web4002.mail.yahoo.com (web4002.mail.yahoo.com [216.115.104.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA01481 for ; Fri, 10 Aug 2001 18:10:36 -0500 Message-ID: <20010810231009.20315.qmail@web4002.mail.yahoo.com> Received: from [17.202.22.175] by web4002.mail.yahoo.com via HTTP; Fri, 10 Aug 2001 16:10:09 PDT Date: Fri, 10 Aug 2001 16:10:09 -0700 (PDT) From: Tito Ciuro To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] Compacting a MetaKit file? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I was trying to compact the MK file by doing this: #include "mk4.h" #include "mk4str.h" int main() { // Create the property I'll use c4_StringProp pName ("Name"); // Open the storage file in exclusive read/write c4_Storage storage ("test.dat", 1); // Obtain the view c4_View vAddress = storage.GetAs("address[Name:S]"); // Edit a property pName(vAddress[0]) = "John Doe"; // Save the changes storage.Commit(); return 0; } The page in the following link: http://www.equi4.com/metakit/wiki.cgi/82.html states: -----> begin quote [...] The way to reclaim free space, is to re-open the datafile in exclusive mode (1), and then commit normally. There are a number of ways to actully bring the file back to its minimal size (this won't happen right away, since space will only become re-usable after the commit completes succesfully). <----- end quote So it seems that I'm doing the right thing, no? The last sentence puzzles me... When will the file size be reduced? After calling commit(), I would expect the file size to be reduced significantly, but this isn't the case. By the way... the edit goes through and gets saved properly. Regarding the file size, I've seen that 1000 records added and commited once (at the end), results in 96KB. If I commit after each save (commit-extend mode), the file size if 45.5MB. With this file, searching and editing takes forever, so I would like to compact the file, hopefully getting close to the 96KB mark. Has anyone accomplished this? Thanks, -- Tito __________________________________________________ Do You Yahoo!? Send instant messages & get email alerts with Yahoo! Messenger. http://im.yahoo.com/ From tciuro@yahoo.com Fri Aug 10 22:38:07 2001 Received: from web4001.mail.yahoo.com (web4001.mail.yahoo.com [216.115.104.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id WAA06079 for ; Fri, 10 Aug 2001 22:38:06 -0500 Message-ID: <20010811033738.3188.qmail@web4001.mail.yahoo.com> Received: from [17.202.22.175] by web4001.mail.yahoo.com via HTTP; Fri, 10 Aug 2001 20:37:38 PDT Date: Fri, 10 Aug 2001 20:37:38 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Compacting a MetaKit file? To: Tito Ciuro , metakit@equi4.com In-Reply-To: <20010810231009.20315.qmail@web4002.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi guys! OK... I solved the puzzle, so here's the code that does it: FILE *myOldFile = fopen (kBigDataFileName, "r"); c4_FileStream myOldStream (myOldFile, true); FILE *myNewFile = fopen (kSmallDataFileName, "w"); c4_FileStream myNewStream (myNewFile, true); storage.LoadFrom( myOldStream ); storage.SaveTo( myNewStream ); fclose (myOldFile); fclose (myNewFile); If you think there's a better way, by all means, let me know! :) I created 2 data files by opening in Commit-Extend mode, then call commit() after each save. This has created huge files (as expected) which is great to test compacting times. Care about the results? Here we go then (sorry if it looks out of whack): Results: ---------- Number of records 1,000 5,000 original size 45.5MB 1.1GB compacted size 96KB 464KB time to compact (local) 110 ms 530 ms time to compact (NFS) 580 ms 2970 ms As you can see, most users will use data files locally, so compacting can almost be done on each save! :) One thing I'd love to see if some way to calculate what the wasted space ratio is for a given data file. (Jean-Claude?) That would give us a pointer as to how often we should trigger a custom Save & Compact method. I hope this helps! -- Tito --- Tito Ciuro wrote: > Hello, > > I was trying to compact the MK file by doing this: > > #include "mk4.h" > #include "mk4str.h" > > int main() > { > // Create the property I'll use > c4_StringProp pName ("Name"); > > // Open the storage file in exclusive read/write > c4_Storage storage ("test.dat", 1); > > // Obtain the view > c4_View vAddress = > storage.GetAs("address[Name:S]"); > > // Edit a property > pName(vAddress[0]) = "John Doe"; > > // Save the changes > storage.Commit(); > > return 0; > } > > The page in the following link: > > http://www.equi4.com/metakit/wiki.cgi/82.html > > states: > > -----> begin quote > [...] The way to reclaim free space, is to re-open > the > datafile in exclusive mode (1), and then commit > normally. There are a number of ways to actully > bring > the file back to its minimal size (this won't happen > right away, since space will only become re-usable > after the commit completes succesfully). > <----- end quote > > So it seems that I'm doing the right thing, no? The > last sentence puzzles me... When will the file size > be > reduced? After calling commit(), I would expect the > file size to be reduced significantly, but this > isn't > the case. By the way... the edit goes through and > gets > saved properly. > > Regarding the file size, I've seen that 1000 records > added and commited once (at the end), results in > 96KB. > If I commit after each save (commit-extend mode), > the > file size if 45.5MB. With this file, searching and > editing takes forever, so I would like to compact > the > file, hopefully getting close to the 96KB mark. > > Has anyone accomplished this? > > Thanks, > > -- Tito > > __________________________________________________ > Do You Yahoo!? > Send instant messages & get email alerts with Yahoo! > Messenger. > http://im.yahoo.com/ > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Send instant messages & get email alerts with Yahoo! Messenger. http://im.yahoo.com/ From jyl@best.com Sat Aug 11 16:23:07 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA13374; Sat, 11 Aug 2001 16:23:07 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id OAA26406; Sat, 11 Aug 2001 14:21:49 -0700 (PDT) Message-ID: <3B75A430.E6D9E858@best.com> Date: Sat, 11 Aug 2001 14:31:28 -0700 From: Jacob Levy X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com, jcw@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] How to "rewrite" a view? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm trying to change the format of a view from #define OLDFORMAT "e4GraphMarkers[nameID:I,next:I,root:I,used:I,userData:I]" to: #define NEWFORMAT "e4GraphMarkers[nameID:I,next:I,root:I,flags:I,userData:I]" I believe I've nearly got it, with the following code: c4_IntProp pUsed("used"); c4_IntProp pFlags("flags"); c4_Storage s = .... c4_View oldview = s.GetAs(OLDFORMAT); c4_View newview = oldview.Rename(pUsed, pFlags); s.Store("e4GraphMarkers", newview); Ooops. It seems that MK 2.4.0 doesn't support Store anymore -- what should I use instead? --JYL From titoffen@pacbell.net Sat Aug 11 17:13:59 2001 Received: from pltn13.pbi.net (mta7.pltn13.pbi.net [64.164.98.8]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA13679 for ; Sat, 11 Aug 2001 17:13:58 -0500 Received: from monkey2000 ([209.233.25.139]) by mta7.pltn13.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with SMTP id <0GHX0001JCEFDV@mta7.pltn13.pbi.net> for metakit@equi4.com; Sat, 11 Aug 2001 15:13:28 -0700 (PDT) Date: Sat, 11 Aug 2001 15:14:29 -0700 From: Tito Ciuro Subject: Re: [Metakit] How to "rewrite" a view? To: Jacob Levy , metakit@equi4.com Message-id: <000c01c122b2$fd6f2fd0$6601a8c0@monkey2000> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 X-Mailer: Microsoft Outlook Express 5.50.4522.1200 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7bit X-Priority: 3 X-MSMail-priority: Normal References: <3B75A430.E6D9E858@best.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jacob, Given that MK can change views on the fly, why not simply calling s.Commit()? My understanding is that if you do the following: > c4_Storage s = .... > > c4_View oldview = s.GetAs(NEWFORMAT); > s.Commit(); Then the column "used" gets dropped and "flags" gets added. Of course, the values held by "used" are lost. However, it seems to me that if you perform the steps you've specified and call s.Commit() instead of Store(), the column "used" will be renamed to "flags" while keeping its values. Let me know what happens, OK? Cheers, -- Tito > #define OLDFORMAT > "e4GraphMarkers[nameID:I,next:I,root:I,used:I,userData:I]" > > to: > > #define NEWFORMAT > "e4GraphMarkers[nameID:I,next:I,root:I,flags:I,userData:I]" > > I believe I've nearly got it, with the following code: > > c4_IntProp pUsed("used"); > c4_IntProp pFlags("flags"); > > c4_Storage s = .... > > c4_View oldview = s.GetAs(OLDFORMAT); > c4_View newview = oldview.Rename(pUsed, pFlags); > > s.Store("e4GraphMarkers", newview); > > Ooops. It seems that MK 2.4.0 doesn't support Store anymore -- what > should I use instead? > > --JYL > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From jyl@best.com Sat Aug 11 17:45:15 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA13889; Sat, 11 Aug 2001 17:45:14 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id PAA17577; Sat, 11 Aug 2001 15:42:54 -0700 (PDT) Message-ID: <3B75B73E.97BE388D@best.com> Date: Sat, 11 Aug 2001 15:52:46 -0700 From: Jacob Levy X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Tito Ciuro CC: metakit@equi4.com, jcw@equi4.com Subject: Re: [Metakit] How to "rewrite" a view? References: <3B75A430.E6D9E858@best.com> <000c01c122b2$fd6f2fd0$6601a8c0@monkey2000> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: OK I didnt give as much information as I should have. I want to keep the old values of "used" and just rename the column to "flags". In the meantime I've redone the whole thing as follows: c4_View oldview = s.GetAs(OLDFORMAT); oldview.AddProperty(pFlags); for (i = 0, l = oldview.GetSize(); i < l; i++) { if (pUsed(oldview[i]) == 1) { pFlags(oldview[i]) = E4_INUSE; } } /* * Now get the old view in the new format (which should drop the "used" property): */ oldview = s.GetAs(NEWFORMAT); /* * We're done, save everything. */ s.Commit(); } This works except I'm not sure that the "used" column really is gone. Oh well.. Good enough. --JYL Tito Ciuro wrote: > Hi Jacob, > > Given that MK can change views on the fly, why not simply calling > s.Commit()? My understanding is that if you do the following: > > > c4_Storage s = .... > > > > c4_View oldview = s.GetAs(NEWFORMAT); > > s.Commit(); > > Then the column "used" gets dropped and "flags" gets added. Of course, the > values held by "used" are lost. > > However, it seems to me that if you perform the steps you've specified and > call s.Commit() instead of Store(), the column "used" will be renamed to > "flags" while keeping its values. > > Let me know what happens, OK? > > Cheers, > > -- Tito > > > #define OLDFORMAT > > "e4GraphMarkers[nameID:I,next:I,root:I,used:I,userData:I]" > > > > to: > > > > #define NEWFORMAT > > "e4GraphMarkers[nameID:I,next:I,root:I,flags:I,userData:I]" > > > > I believe I've nearly got it, with the following code: > > > > c4_IntProp pUsed("used"); > > c4_IntProp pFlags("flags"); > > > > c4_Storage s = .... > > > > c4_View oldview = s.GetAs(OLDFORMAT); > > c4_View newview = oldview.Rename(pUsed, pFlags); > > > > s.Store("e4GraphMarkers", newview); > > > > Ooops. It seems that MK 2.4.0 doesn't support Store anymore -- what > > should I use instead? > > > > --JYL > > > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > From rcohen@dial.oleane.com Mon Aug 13 02:25:21 2001 Received: from sun2.articque.com (sun2.articque.com [195.25.225.73]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA29134 for ; Mon, 13 Aug 2001 02:25:20 -0500 Received: from dial.oleane.com ([195.25.225.86]) by sun2.articque.com (8.10.1/8.10.0) with ESMTP id f7D8Jr323973 for ; Mon, 13 Aug 2001 09:19:54 +0100 (GMT+0100) Message-ID: <3B778E73.3F5A9F91@dial.oleane.com> Date: Mon, 13 Aug 2001 09:23:15 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Subject: Re: [Metakit] Compacting a MetaKit file? References: <20010811033738.3188.qmail@web4001.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > I created 2 data files by opening in Commit-Extend > mode, then call commit() after each save. This has > created huge files (as expected) which is great to > test compacting times. Care about the results? Here we > go then (sorry if it looks out of whack): I was quite interested in this information because I could not understand in which case a 96kb data could be stored in a 45Mb file ? Could you tell me what to do and why it does it ? Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From tciuro@yahoo.com Mon Aug 13 03:02:50 2001 Received: from web4002.mail.yahoo.com (web4002.mail.yahoo.com [216.115.104.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id DAA29434 for ; Mon, 13 Aug 2001 03:02:49 -0500 Message-ID: <20010813080210.7521.qmail@web4002.mail.yahoo.com> Received: from [209.233.25.139] by web4002.mail.yahoo.com via HTTP; Mon, 13 Aug 2001 01:02:10 PDT Date: Mon, 13 Aug 2001 01:02:10 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Compacting a MetaKit file? To: Riccardo Cohen , Mailing list Metakit In-Reply-To: <3B778E73.3F5A9F91@dial.oleane.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Riccardo, During Commit-Extend mode, each save forces MK to rewrite the *entire* column. This is why data grows so quickly and slows down queries. Therefore, once in a while, it's necessary to compact the file. I sent an email with the code that will allow you to compact the file. Just i case, here it is: FILE *myOldFile = fopen ("myOldFile.dat", "r"); c4_FileStream myOldStream (myOldFile, true); FILE *myNewFile = fopen ("myNewFile.dat", "w"); c4_FileStream myNewStream (myNewFile, true); storage.LoadFrom( myOldStream ); storage.SaveTo( myNewStream ); fclose (myOldFile); fclose (myNewFile); I hope this helps, -- Tito --- Riccardo Cohen wrote: > > I created 2 data files by opening in Commit-Extend > > mode, then call commit() after each save. This has > > created huge files (as expected) which is great to > > test compacting times. Care about the results? > Here we > > go then (sorry if it looks out of whack): > > I was quite interested in this information because I > could not understand in which case a 96kb data could > be stored in a 45Mb file ? Could you tell me what to > do and why it does it ? > > Thanks > > -- > Riccardo Cohen > > Articque > Les Roches > 37230 Fondettes > France > email = rcohen@dial.oleane.com > web = http://www.articque.com > tel: +33 02 47 49 90 49 > fax: +33 02 47 49 91 49 > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Send instant messages & get email alerts with Yahoo! Messenger. http://im.yahoo.com/ From rcohen@dial.oleane.com Mon Aug 13 06:54:44 2001 Received: from sun2.articque.com (sun2.articque.com [195.25.225.73]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA32005 for ; Mon, 13 Aug 2001 06:54:43 -0500 Received: from dial.oleane.com ([195.25.225.86]) by sun2.articque.com (8.10.1/8.10.0) with ESMTP id f7DCn5324397; Mon, 13 Aug 2001 13:49:09 +0100 (GMT+0100) Message-ID: <3B77CD8D.BFF8964@dial.oleane.com> Date: Mon, 13 Aug 2001 13:52:29 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: fr,en-US,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Galaxy , Mailing list Metakit Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] strstr Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Does anyone know where I could find the source code of standard strstr() function ? (I would like to know how they made it so quick !!) Thanks -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From vv@iware.com Mon Aug 13 07:12:08 2001 Received: from sdrc.com (heimdall.sdrc.com [146.122.132.195]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA32452 for ; Mon, 13 Aug 2001 07:12:08 -0500 Received: from tyr.sdrc.com (mailhub-cvg.sdrc.com [146.122.142.31]) by sdrc.com (8.9.1/8.9.1) with ESMTP id IAA03605; Mon, 13 Aug 2001 08:11:29 -0400 (EDT) Received: from boojum.iware.com (boojum [209.57.178.2]) by tyr.sdrc.com (8.8.6 (PHNE_17190)/8.8.5) with ESMTP id IAA15498; Mon, 13 Aug 2001 08:11:27 -0400 (EDT) Received: from bane.iware.com (bane.iware.com [209.57.178.105]) by boojum.iware.com (8.9.3/8.9.0) with ESMTP id IAA21453; Mon, 13 Aug 2001 08:11:28 -0400 Message-Id: <4.3.2.7.2.20010813081053.03b28ee8@mailhost.iware.com> X-Sender: vv@mailhost.iware.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Mon, 13 Aug 2001 08:11:27 -0400 To: gum@yahoogroups.com, Mailing list Galaxy , Mailing list Metakit From: "Victor R. Volkman" In-Reply-To: <3B77CD8D.BFF8964@dial.oleane.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Re: [gum] strstr Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Assuming you have VC++ and you bothered to install the C Run Time (CRT) source code it would be staring you in the face: Directory of C:\VS\VC98\CRT\SRC 06/17/1998 12:00a 1,382 STRSTR.C 1 File(s) 1,382 bytes At 01:52 PM 8/13/2001 +0100, Riccardo Cohen wrote: >Hi, >Does anyone know where I could find the source code of standard strstr() >function ? >(I would like to know how they made it so quick !!) > >Thanks > >-- >Riccardo Cohen > >Articque >Les Roches >37230 Fondettes >France >email = rcohen@dial.oleane.com >web = http://www.articque.com >tel: +33 02 47 49 90 49 >fax: +33 02 47 49 91 49 > > > >Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ From art_stine@mindspring.com Mon Aug 13 14:22:54 2001 Received: from smtp5-cm.mail.eni.net (smtp5-cm.mail.eni.net [216.133.226.138]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA14658 for ; Mon, 13 Aug 2001 14:22:54 -0500 Received: from DUALIE.mindspring.com (node-d8e9522.powerinter.net [216.233.82.2]) by smtp5-cm.mail.eni.net (8.9.3/8.9.3) with ESMTP id MAA02091; Mon, 13 Aug 2001 12:22:15 -0700 Message-Id: <5.1.0.14.0.20010813121957.0d331a88@pop3.norton.antivirus> X-Sender: art_stine/pop.mindspring.com@pop3.norton.antivirus X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 13 Aug 2001 12:22:06 -0700 To: gum@yahoogroups.com, Mailing list Galaxy , Mailing list Metakit From: Art Stine In-Reply-To: <3B77CD8D.BFF8964@dial.oleane.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Re: [gum] strstr Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: At 01:52 PM 8/13/2001 +0100, Riccardo Cohen wrote: >Hi, >Does anyone know where I could find the source code of standard strstr() >function ? >(I would like to know how they made it so quick !!) > >Thanks Go to www.freebsd.org and look thru the sources on line in their cvsweb interface (http://www.FreeBSD.org/cgi/cvsweb.cgi/) ... for i386, strstr() is not asm itself, but strncmp() is (which is where the bulk of the work is done). -art From nobody@www.prosapia.com Mon Aug 13 18:57:48 2001 Received: from www.prosapia.com ([64.40.109.174]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA23898 for ; Mon, 13 Aug 2001 18:57:48 -0500 Date: Mon, 13 Aug 2001 16:57:08 -0700 Message-Id: <200108132357.QAA10239@www.prosapia.com> To: metakit@equi4.com From: pr@grokster.com Subject: [Metakit] Grokster: Next Generation File Sharing Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sirs! We would like you to consider using/reviewing the Next Generation File Sharing tool. It is called Grokster and can be seen at www.Grokster.com Grokster accesses over 400,000+ online users through the FastTrack P2P Stack. Grokster is the latest generation of file sharing software. The Grokster generation of file sharing presents these exciting features: * Peer-to-Peer: True person to person file searches and transfers. * Fast Searches: Unlike Gnutella type programs, Grokster's SuperNode technology provides for very fast searches. * Detailed Searches: The Grokster technology tracks detailed file info and allows detailed searches on it. * No Incomplete/Failed Downloads: Grokster automatically removes incomplete downloads. * Auto Resume: Grokster will automatically seek out the same file from multiple sources and continue downloading until complete. * Fastest Downloads: Grokster will download the file from multiple sources with the fastest connections as fast as you can download, automatically. * File Preview: Preview files as you start downloading them. * All File Types: Share any type of file. * Full Bandwidth Controls: Grokster allows for full contol uploads/downloads and total bandwidth. This is a one time email. You will receive no other emails from us. Thank you for your time. Cheers relations@grokster.com http://www.grokster.com Grokster: Next Generation File Sharing From emailsebek@home.com Mon Aug 20 17:14:14 2001 Received: from femail25.sdc1.sfba.home.com (femail25.sdc1.sfba.home.com [24.254.60.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA27774 for ; Mon, 20 Aug 2001 17:14:04 -0500 Received: from home.com ([24.20.193.155]) by femail25.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20010820221242.WPGO21066.femail25.sdc1.sfba.home.com@home.com> for ; Mon, 20 Aug 2001 15:12:42 -0700 Message-ID: <3B818C51.89E04DEF@home.com> Date: Mon, 20 Aug 2001 15:16:49 -0700 From: Daniel Tracy X-Mailer: Mozilla 4.51 [en] (Win95; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Metakit: Multi-language storage access Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm developing an application in which I need to have access to a single storage file in two different languages: Python and C++. I clearly can't open it in each language using the .storage member function because I'd have two in memory instances. What I need is a simple way of sharing a storage with the two languages, but I haven't found one. Do I have to initiate the storage in Python and access an underlying C++ pointer, passing it to an initializing function in C++? I'd rather load the file in C++ given my program design. It would be most ideal if Mk4py could hook up to an existing, open C++ metakit class... Daniel From andreas.muegge@gmx.de Tue Aug 21 06:39:46 2001 Received: from mx0.gmx.net (mx0.gmx.net [213.165.64.100]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA07811 for ; Tue, 21 Aug 2001 06:39:45 -0500 Received: (qmail 18459 invoked by uid 0); 21 Aug 2001 11:38:05 -0000 Date: Tue, 21 Aug 2001 13:38:05 +0200 (MEST) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: metakit@equi4.com Cc: Riccardo Cohen MIME-Version: 1.0 References: <3B77CD8D.BFF8964@dial.oleane.com> Subject: Re: [Metakit] strstr X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [62.180.31.3] Message-ID: <32630.998393885@www36.gmx.net> X-Mailer: WWW-Mail 1.5 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Does anyone know where I could find the source code of standard strstr() > function ? > (I would like to know how they made it so quick !!) I found out that writing your own function based on the Boyer-Moore Sunday algorithm gives even better results than asm optimized but stupid strstr(). If someone is interested in the code he can send me an email - I have written a small class which uses the algo. Ciao, Andreas -- GMX - Die Kommunikationsplattform im Internet. http://www.gmx.net From j.p.fletcher@aston.ac.uk Tue Aug 21 07:04:11 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA08552 for ; Tue, 21 Aug 2001 07:04:06 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 15ZAGM-0000GV-00 for metakit@equi4.com; Tue, 21 Aug 2001 13:04:06 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Tue, 21 Aug 2001 13:01:05 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Priority: normal X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Subject: [Metakit] Re: Metakit: Multi-language storage access Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sorry, I meant to send this to the list. John ------- Forwarded Message Follows ------- From: Self To: Daniel Tracy Subject: Re: [Metakit] Metakit: Multi-language storage access Date sent: Tue, 21 Aug 2001 10:05:25 +0100 From: Daniel Tracy To: metakit@equi4.com Subject: [Metakit] Metakit: Multi-language storage access Date sent: Mon, 20 Aug 2001 15:16:49 -0700 > I'm developing an application in which I need to have access to a single storage file in > two different languages: Python and C++. I clearly can't open it in each language using > the .storage member function because I'd have two in memory instances. What I need is a > simple way of sharing a storage with the two languages, but I haven't found one. Do I > have to initiate the storage in Python and access an underlying C++ pointer, passing it to > an initializing function in C++? I'd rather load the file in C++ given my program > design. It would be most ideal if Mk4py could hook up to an existing, open C++ metakit > class... > > Daniel > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > Have you looked at SWIG, http://www.swig.org/ which would enable you to access your C++ code from python? I have only used SWIG for TCL and C++ myself. John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From jyl@best.com Sat Aug 25 14:07:58 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA00699; Sat, 25 Aug 2001 14:07:52 -0500 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id MAA06454; Sat, 25 Aug 2001 12:05:49 -0700 (PDT) Message-ID: <3B87F99D.68031A2C@best.com> Date: Sat, 25 Aug 2001 12:16:45 -0700 From: Jacob Levy X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: metakit@equi4.com, jcw@equi4.com CC: jyl@best.com Subject: [Metakit] e4Graph 1.0a4 released References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: [JCW: Could you please post this on Metakit-announce? Thanks, --JYL] I'm pleased to announce the release of e4Graph 1.0a4. Get it from here: http://sourceforge.net/projects/e4graph/ WHAT IS E4GRAPH: e4Graph is a C++ library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. The e4Graph library is built on top of Metakit 2.4.0. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. Here are the CHANGE NOTES: Release: 1.0a4, 08/24/2001 This is the fourth release, the fourth Alpha BUG FIXES: * Re-imlemented deletion mechanism completely from scratch to fix severe performance issues for deletion of cyclical graphs (JYL). * Fixed a memory leak in the e4Graph Metakit storage driver (JYL). * Fixed several memory leaks in the Tcl binding of e4Graph (JYL). * Fixed e4_Node::SetVertex and e4_Node::SetVertexByRank; the bug was that when the previous value of the vertex was a node, the vertex would disappear instead of being set to a new value (DVM). * Fixed bug with XML parsing of character data sections; the bug was that a contiguous character data section could result in a series of "__data__" vertices instead of one "__data__" vertex containing the entire text of the section (JPF). NEW FUNCTIONALITY: * Implemented a framework for automatically updating the format of storages written by an older release to the format used by the new release. This should be transparent to users. Storage files written by e4Graph 1.0a3 should be readable by e4Graph 1.0a4 and onwards. Also implemented a mechanism to detect when a storage written by a newer release is read in an older release. In the future this will detect e.g. when a storage written by e4Graph 1.0b1 is read in a client using e4Graph 1.0a4. * New method e4_Node::DeleteFirstVertexWithNode to delete the first vertex of this node that has the argument node as its value (JYL, DVM). * XML parsing and generation of XML comments, XML CDATA sections, XML processing instructions, XML declarations, and document type declarations (JYL, JPF). STORAGE FORMAT CHANGE: The storage format used by e4Graph 1.0a4 is different from that used by e4Graph 1.0a3. However, e4Graph 1.0a4 is able to read storages written by e4Graph 1.0a3 and update their format on the fly to that used by e4Graph 1.0a4. e4Graph 1.0a3 can not read storages written by e4Graph 1.0a4 and later. E4GRAPH DIRECTORY STRUCTURE REORGANIZED: The 'core' directory has been renamed to 'main' (DVM). E4GRAPH BUILD STRUCTURE REORGANIZED: All build output goes to builds/* by default (depending on the configuration options given on Unix systems; this is the default on win32) (JYL, DVM). The win32 builds leave their product in builds\win32 instead of in individual sub-target directories (JYL, DVM). E4GRAPH NOW SUPPORTS STATIC LIBRARY BUILDS ON WIN32: * Static libraries and static linking of executables with e4Graph lead to faster execution at runtime. For example, teste4lib, the statically linked API tester, is about 14% faster than teste4dll, the dynamically linked API tester (DVM, JYL). * E4_STATIC must be defined for all those who want to compile from the e4graph headers and then link to the static library. I wish this were not the case, but there was no way around it. If it is not defined, it will only link with the dll library (DVM). * The 'core' project has been removed, and replaced by two new ones: - e4dll which is the dll-generating version - e4lib which is a statically-linked version In order to track this, the 'testcore' project has also been removed, and replaced by two new ones: - teste4dll which is the dll-requiring test program - teste4lib which is a statically-linked stand-alone program Note that running teste4dll requires that the e4graph and metakit dynamic libraries be installed at runtime in order to work. Teste4lib, however, has statically linked in the e4graph and metakit libraries, so that it does not depend on metakit or e4graph libraries to be installed at runtime (DVM). GENERAL BUILD CHANGES: * This release depends on Metakit 2.4.0 instead of on Metakit 2.3.4-29 (JYL). * This release depends on Expat 1.95.2 instead of on Expat 1.95.1 (JYL). * The current project settings and Makefiles require that the Metakit source code root directory be located as a peer to e4graph's root directory, and that it be called 'metakit-2.4.0'. This may be easily accomplished by downloading the zip or tar version of that release and unpacking it in the parent of where the e4graph code is rooted. Alternatively, the CVS version of Metakit may be used if its root directory name is changed from 'mk' to 'metakit-2.4.0'. If this is followed, the project files and Makefiles in e4graph may be used without alteration (JYL, DVM). BUILD CHANGES ON WIN32: * All compile output (including .pch, .obj, etc.) now goes to the 'builds\win32' directory. This means that you can delete everything there, and be sure of a clean build, but also that there is NO mess left behind in the other directories. Note that it now uses the post-link step ONLY to copy the header files (DVM). a. all .lib and .dll files are now in 'builds\win32\lib' b. all .exe files are now in 'builds\win32\bin' c. all .h files are still in 'builds\win32\include' d. all .obj files are now in 'builds\msvc\\Debug' or 'builds\msvc\\Release' * Since the debug/release and static/dynamic versions of the files are now in the same directories, they have different names to distinguish them. The naming convention is stolen from metakit. The following is representative of the convention, covering the dynamic/static and debug/release combinations for the e4graph and teste4graph code. The others follow similar conventions (DVM). a. Static Release in 'builds\win32\lib\e4graphs.lib' b. Static Debug in 'builds\win32\lib\e4graphs_d.lib' c. Dynamic Release in 'builds\win32\lib\e4graph.lib/dll' d. Dynamic Debug in 'builds\win32\lib\e4graph_d.lib/dll' e. Dynamic Test Release in 'builds\win32\bin\teste4dll.exe' f. Dynamic Test Debug in 'builds\win32\bin\teste4dll_d.exe' g. Static Test Release in 'builds\win32\bin\teste4lib.exe' h. Static Test Debug in 'builds\win32\bin\teste4lib_d.exe' --JYL From jerry@usd1.com Mon Aug 27 15:03:58 2001 Received: from asteroid.pacifier.com ([199.2.117.154]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA07244 for ; Mon, 27 Aug 2001 15:03:54 -0500 Received: from c1138921-b.vncvr1.wa.home.com (c1138921-b.vncvr1.wa.home.com [65.12.183.41]) by asteroid.pacifier.com (8.11.2/8.11.1) with ESMTP id f7RK1gM25804 for ; Mon, 27 Aug 2001 13:01:43 -0700 (PDT) Date: Mon, 27 Aug 2001 13:01:07 -0700 From: Jerry McRae X-Mailer: The Bat! (v1.53d) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <6515374064.20010827130107@usd1.com> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Python & row copies on new view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am having a problem the FIRST time a row is copied in a view. Any string fields that are set get converted to garbage the first time. I make a lot of use of copying rows, and I would like not to have to put special logic in to check if it is the first copy done for each view! The second and subsequent times it works perfectly. Example: >>> db=mk.storage() >>> t2 = db.getas('t2[name]') >>> t2.append() >>> t2.append() >>> t2[1].name = 'fred' >>> t2[0].name '' >>> t2[0] = t2[1] >>> t2[0].name '\xd0\xb26\x01\x18\xff8\x01' >>> t2[0] = t2[1] >>> t2[0].name 'fred' I get the same results if there are multiple string columns, if multiple rows were added, or if I copy to a row other than zero. Anyone have solutions or suggestions? -- Thanks, Jerry McRae MetaKit 2.4.0, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From lupoja@qwest.net Fri Aug 31 20:49:28 2001 Received: from dnvrpop3.dnvr.uswest.net (dnvrpop3.dnvr.uswest.net [206.196.128.5]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id UAA01154 for ; Fri, 31 Aug 2001 20:49:27 -0500 Received: (qmail 455 invoked by uid 0); 1 Sep 2001 01:47:34 -0000 Received: from ndialup193.dnvr.uswest.net (209.180.244.193) by dnvrpop3.dnvr.uswest.net with SMTP; 1 Sep 2001 01:47:34 -0000 Received: from egor.linux.home (localhost [127.0.0.1]) by localhost (8.12.0.Beta19/8.12.0.Beta19/Debian 8.12.0.Beta19) with ESMTP id f811lWIG006796 for ; Fri, 31 Aug 2001 19:47:32 -0600 Received: (from lupoja@localhost) by egor.linux.home (8.12.0.Beta19/8.12.0.Beta19/Debian 8.12.0.Beta19) id f811kbLL006721; Fri, 31 Aug 2001 19:46:37 -0600 Date: Fri, 31 Aug 2001 19:46:37 -0600 Message-Id: <200109010146.f811kbLL006721@egor.linux.home> From: "James A. Lupo" To: metakit@equi4.com Subject: [Metakit] I/O Error on Commit After Load Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm using Mk4tcl-2.4.0 on Linux and MS Windows-9x. Any time I attempt an "mk::file commit" after doing an "mk::file load" I get an I/O error. The data appears to have been set correctly following the load, but there is no way to save the data to a file. Here's a toy script that demonstrates the problem, stored in a file name "t.tcl" and sourced in tclsh (make sure t.dat does not exist before trying script): package require Mk4tcl mk::file open db t.dat mk::view layout db.dummy {f1 f2} mk::row append db.dummy f1 "field one" f2 "field two" mk::file commit db array set result [mk::get db.dummy!0] puts "Original, saved settings" puts "f1 = $result(f1)" puts "f2 = $result(f2)" set f [open t.sav w] mk::file save db $f close $f mk::set db.dummy!0 f1 "one" f2 "two" mk::file commit db array set result [mk::get db.dummy!0] puts "Modified settings" puts "f1 = $result(f1)" puts "f2 = $result(f2)" set f [open t.sav r] mk::file load db $f close $f array set result [mk::get db.dummy!0] puts "Reloaded settings." puts "f1 = $result(f1)" puts "f2 = $result(f2)" mk::file commit db mk::file close db Here is the results of running the script: % source t.tcl Original, saved settings f1 = field one f2 = field two Modified settings f1 = one f2 = two Reloaded settings. f1 = field one f2 = field two I/O error during commit This sort of thing use to work before. -- Jim ------------------- Optional Methods ----------------------- Dr James A. Lupo (303) 423-2652 9667 Independence Dr. lupoja@qwest.net Westminster CO 80021-6845 jalupo@regis.edu From jcw@equi4.com Wed Sep 5 12:32:46 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA18241 for ; Wed, 5 Sep 2001 12:32:45 -0500 Received: from takkie.equi4.nl ([195.121.235.162]) by smtp04.wxs.nl (Netscape Messaging Server 4.05) with ESMTP id GJ79YO00.VNC for ; Wed, 5 Sep 2001 19:30:24 +0200 Received: from [10.0.1.4] (airsie [195.108.246.51]) by takkie.equi4.nl (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f85HTdf15812 for ; Wed, 5 Sep 2001 19:29:39 +0200 From: Jean-Claude Wippler To: Date: Wed, 5 Sep 2001 19:30:26 +0200 Message-Id: <20010905173026.32078@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Fixed several bugs Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: After a (too) long absence, I've just fixed a number of bugs and checked them into CVS (see http://www.equi4.com/metakit/download.html for details): Hashing bug fixed: failed to terminate when looking for a missing key after a hash collision. The fill count was not being tracked properly to enforce that there is always at least one unused slot. Fixed bug when setting a string from a higher row: the data gets trashed, because too little copying was done while creating a gap. This bug may have led to some other cases of "damaged" datafiles. Solved by copying input data in c4_FormatB::SetOne(). Fixed long-standing bug with commit-after-load, by now doing a full view/subview copy. This is inefficient, but it avoids all sorts of mapping/strategy problems. This also fixes test s33 (at last!). Added regression tests b27, c21, m01, and s36 to check above fixes. Many many thanks to James Lupo, Jerry McRae, and Nathan Rogers for reporting these problems along with such great code samples - these have each been a big help in finding and fixing the problems. If any further problems remain, please let me know. Regards, Jean-Claude From j.p.fletcher@aston.ac.uk Fri Sep 7 04:58:09 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA10359 for ; Fri, 7 Sep 2001 04:58:09 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 15fIMY-0005T1-00 for metakit@equi4.com; Fri, 07 Sep 2001 10:55:50 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Fri, 7 Sep 2001 10:54:03 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Priority: normal X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Subject: [Metakit] Metakit 2.4.0 with mingw32 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am compiling the metakit 2.4.0 distribution with gcc 2.95.2 under mingw32. This combination defines __GCC__ and _WIN32 and this requires a patch to HEADER.H as follows: Comparing files header.h and f:header.h ****** header.h #define q4_DOS 1 #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || defined(_AIX) ****** f:header.h #define q4_DOS 1 #elif defined (_WIN32) && defined (__GNUC__) #define d4_OS_H "win.h" #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || defined(_AIX) ****** A further patch is needed in FILEIO.CPP Comparing files fileio.cpp and f:fileio.cpp ****** fileio.cpp void c4_FileStrategy::ResetFileMapping() { #if q4_WIN32 if (_mapStart != 0) ****** f:fileio.cpp void c4_FileStrategy::ResetFileMapping() { #if q4_WIN32 && !defined(__GNUC__) if (_mapStart != 0) ****** although this leaves the function ResetFileMapping() empty. Further testing is in progress. John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From j.p.fletcher@aston.ac.uk Fri Sep 7 05:51:44 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA11458 for ; Fri, 7 Sep 2001 05:51:43 -0500 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 15fJCO-0002tp-00 for metakit@equi4.com; Fri, 07 Sep 2001 11:49:24 +0100 From: "John Fletcher" To: metakit@equi4.com Date: Fri, 7 Sep 2001 11:47:37 +0100 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] Metakit 2.4.0 with mingw32 - correction Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > I am compiling the metakit 2.4.0 distribution with gcc 2.95.2 under > mingw32. This combination defines __GCC__ and _WIN32 and __GCC__ should read __GNUC__ - sorry, John > this requires a patch to HEADER.H as follows: > > Comparing files header.h and f:header.h > ****** header.h > #define q4_DOS 1 > #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || > defined(_AIX) > ****** f:header.h > #define q4_DOS 1 > #elif defined (_WIN32) && defined (__GNUC__) > #define d4_OS_H "win.h" > #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || > defined(_AIX) > ****** > > A further patch is needed in FILEIO.CPP > > Comparing files fileio.cpp and f:fileio.cpp > > > > ****** fileio.cpp > void c4_FileStrategy::ResetFileMapping() > { > #if q4_WIN32 > if (_mapStart != 0) > ****** f:fileio.cpp > void c4_FileStrategy::ResetFileMapping() > { > #if q4_WIN32 && !defined(__GNUC__) > if (_mapStart != 0) > ****** > > although this leaves the function ResetFileMapping() empty. > > Further testing is in progress. > > John Fletcher > ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From pdominioni@rsr.it Mon Sep 10 10:35:28 2001 Received: from mono.mmservice.com ([217.56.215.2]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA01105 for ; Mon, 10 Sep 2001 10:35:23 -0500 Received: (qmail 24573 invoked from network); 10 Sep 2001 21:37:08 -0000 Received: from ppp-144-1.24-151.libero.it (HELO server) (151.24.1.144) by 0 with SMTP; 10 Sep 2001 21:37:08 -0000 Message-ID: <008701c13a0e$5d8eb030$7e7e7e7e@server> From: "RSR - Piero Dominioni" To: Date: Mon, 10 Sep 2001 17:28:35 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Subject: [Metakit] How to cross-compile for TQM823L? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi everybody. I downloaded Metakit-2.4.0.tar.gz and followed the instructions to build it under Unix (I've a Slckware 7.2 Linux-box on my Pentium II PC). Everything seem to go fine: I'm not familiar yet with Metakit, but the command: ./demo gives this result: The country of Paco Pena is: Spain So I guess the demo works and Metakit works too... My first need, however, is to embed metakit on a PowerPC board, TQM823L, with Linux installed on it. I have got the cross-compilation tools (powerpc-linux-gcc, powerpc-linux-ld, etc.), I've looked in the mailing list archives (without appreciable results) and I've tryed to hand- modify the templates for configure and Makefile. My attempt lead to errors, because I really don't know which are the correct settings to put in these files. Does anyone tell me how to modify the auto-configuration files or how to write a suitable Makefile by hand in order to cross-compile for my Motorola PowerPC 823 platform? Is there some specific documentation about this matter? Thanks in advance for any kind of help. Best regards. Piero Dominioni R.S.R. srl P.S.: If some information is missing to give a complete answer, please, let me know! From remy@shockfish.com Tue Sep 11 01:57:25 2001 Received: from berau.travel.ch (postfix@www.abp.ch [193.135.59.43] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA06067 for ; Tue, 11 Sep 2001 01:57:16 -0500 Received: from pc2.shockfish.com (shockfishpc1.epfl.ch [128.179.67.16]) by berau.travel.ch (Postfix) with ESMTP id 4F0EC12F015; Tue, 11 Sep 2001 08:50:38 +0200 (MEST) Message-Id: <5.0.2.1.0.20010911082955.00a7fea8@mail1.travel.ch> X-Sender: remymail@mail1.travel.ch (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 5.0.2 Date: Tue, 11 Sep 2001 08:53:31 +0200 To: metakit@equi4.com From: Remy Blank - Shockfish Cc: pdominioni@rsr.it Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] Re: How to cross-compile for TQM823L? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Piero, I cross-compiled Metakit for ARM almost without changes to the makefiles or configure scripts. That's how I do it: - Go to "build" directory - Set environment variables for all build tools: AR=arm-linux-ar AS=arm-linux-as CC=arm-linux-gcc CXX=arm-linux-gcc NM=arm-linux-nm OBJCOPY=arm-linux-objcopy OBJDUMP=arm-linux-objdump RANLIB=arm-linux-ranlib STRIP=arm-linux-strip - Execute "../unix/configure --host=i386-pc-linux-gnu" (With other configuration option like "--prefix" if you need them) - Execute "make core" This builds the library. - Finally "make install-mk" This installs the library (I don't know if you need this step. You'll probably just want to copy the files in ".libs" to your target system.) Obviously, you'll want to substitute "arm-linux" with your particular target. IIRC, I had to make a few minor changes to "unix/configure" to make it work. There was one error in a test case where the script tries to compile a program with a main() without return value specification, and this stopped the compiler with an error. I can't remember the other changes. All this was with metakit-2.3.3. I don't know if it's still necessary with the newest version. Good luck! --Remy Blank PS: For the configure script, try this patch: --- configure 2001/02/15 08:47:16 1.1.1.1 +++ configure 2001/02/15 08:50:29 1.2 @@ -1925,7 +1925,7 @@ int main() { /* Ultrix mips cc rejects this. */ -typedef int charset[2]; const charset x; +//typedef int charset[2]; const charset x; /* SunOS 4.1.1 cc rejects this. */ char const *const *ccp; char **p; @@ -2000,7 +2000,7 @@ #include "confdefs.h" int main() { -} $ac_kw foo() { +} $ac_kw int foo() { ; return 0; } EOF if { (eval echo configure:2007: \"$ac_compile\") 1>&5; (eval $ac_compile) 2>&5; }; then From lvirden@cas.org Thu Sep 13 06:11:28 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA14307 for ; Thu, 13 Sep 2001 06:11:27 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id f8DB8F620279 for ; Thu, 13 Sep 2001 07:08:15 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f8DB8Dv24063; Thu, 13 Sep 2001 07:08:13 -0400 (EDT) Date: Thu, 13 Sep 2001 07:08:13 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20010913070813.AAB24044@cas.org> To: metakit@equi4.com Subject: [Metakit] Wikit 'database' questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have a wikit that I would like to learn to manipulate. 1. How can I determine the titles of each of the pages in the wikit? 2. How can I extract the latest versions of all the pages in the wikit? 3. How can I take these extracted pages and insert them into a new, empty, wikit (say one that has been built with updated code, etc.) 4. How can I access the history of changes to any specific page on the wikit? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From horst@freedict.de Sun Sep 16 11:26:42 2001 Received: from mail.netbeat.de (mail.netbeat.de [212.6.214.38]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id LAA20097 for ; Sun, 16 Sep 2001 11:26:35 -0500 From: horst@freedict.de Received: (qmail 19883 invoked from network); 16 Sep 2001 16:36:20 -0000 Received: from dialin-145-254-116-141.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.116.141) by mail.netbeat.de with SMTP; 16 Sep 2001 16:36:20 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id f8GFrke01995 for ; Sun, 16 Sep 2001 17:53:47 +0200 Message-Id: <200109161553.f8GFrke01995@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Sun, 16 Sep 2001 17:53:43 +0200 (CEST) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] metakit & python - how to hash and sort Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I am using metakit in a python application, and want to (efficiently) access the elements (dictionary entries) in a sorted manner - that is the n'th element should be the n'th position when printed in a dictionary. what funktions would I use to do this? - I tried hash and ordered, but could not use both of them together Thanks, Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From klas@flum.net Tue Sep 18 12:00:20 2001 Received: from naver.egenweb.net (naver.egenweb.net [66.33.25.87]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA17573 for ; Tue, 18 Sep 2001 12:00:20 -0500 Received: from localhost (flum@localhost) by naver.egenweb.net (8.9.3/8.9.3) with ESMTP id TAA24825 for ; Tue, 18 Sep 2001 19:10:33 +0200 Date: Tue, 18 Sep 2001 19:10:32 +0200 (CEST) From: Klas Axelsson X-Sender: flum@naver.egenweb.net To: metakit@equi4.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Corrupt file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Have been using MetaKit for a short time and just had a problem with a corrupt database. I'm afraid that I haven't examined this very much myself yet, I intend to do a few tests and such, but I'll give a short description right away in case I never get around to it. I'm using Mk4py.so version 2.4.0 for linux. I find it very likely that the problem was caused by a full disk, the file could be rescued by removing the end of the file. If anyone would like to see the corrupt data file (about 9k) send me a private email. Regards Klas From tciuro@yahoo.com Wed Sep 19 14:33:03 2001 Received: from web20607.mail.yahoo.com (web20607.mail.yahoo.com [216.136.226.165]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA02563 for ; Wed, 19 Sep 2001 14:33:03 -0500 Message-ID: <20010919192956.16826.qmail@web20607.mail.yahoo.com> Received: from [17.202.22.175] by web20607.mail.yahoo.com via HTTP; Wed, 19 Sep 2001 12:29:56 PDT Date: Wed, 19 Sep 2001 12:29:56 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Corrupt file To: Klas Axelsson , metakit@equi4.com In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I think we have a real corruption bug in our hands. Klas sent me an example of a MetaKit data file which had been corrupted. He believes that it was caused by a lack of hard disk space. I ran the MetaKit's dump utility to check the contents and here's the output: board.backup: 0 properties VIEW 1 rows = Opening the file with a text editor reveals that the schema has been destroyed. However, the contents of the file are there. I have seen *exactly* the same problem, where 1MB, 5.3MB, and 10MB files have been corrupted. However, I disagree with Klas in one respect: hard disk space has nothing to do with it. I know that because I have 20GB of free disk space. I've seen this a few times, and at first I thought that it could be that I sent a corrupted schema to MetaKit, and the confusion caused MK to corrupt the data file. I still have no evidence that this is the case, but seeing Klas having the same problem, I wonder whether we have a serious bug in MetaKit. The version I use is 2.3.4r29/C++. Does anyone have more info on this issue? Jean-Claude, would you happen to know what could cause this? If you need a tangible proof, I can send you and example data file that has been corrupted. So, heads up everybody. Take care, -- Tito --- Klas Axelsson wrote: > Have been using MetaKit for a short time and just > had a problem with a > corrupt database. > > I'm afraid that I haven't examined this very much > myself yet, I intend to > do a few tests and such, but I'll give a short > description right away in > case I never get around to it. > > I'm using Mk4py.so version 2.4.0 for linux. > > I find it very likely that the problem was caused by > a full disk, the file > could be rescued by removing the end of the file. > > If anyone would like to see the corrupt data file > (about 9k) send me a > private email. > > Regards > Klas > > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Terrorist Attacks on U.S. - How can you help? Donate cash, emergency relief information http://dailynews.yahoo.com/fc/US/Emergency_Information/ From gmcm@hypernet.com Wed Sep 19 15:05:04 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA03990 for ; Wed, 19 Sep 2001 15:05:01 -0500 Received: from me (204.176.40.99) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Wed, 19 Sep 2001 16:00:42 -0400 From: "Gordon McMillan" To: metakit@equi4.com Date: Wed, 19 Sep 2001 16:00:40 -0400 Subject: Re: [Metakit] Corrupt file Reply-to: gmcm@hypernet.com Message-ID: <3BA8C128.7977.166E73DE@localhost> Priority: normal In-reply-to: <20010919192956.16826.qmail@web20607.mail.yahoo.com> References: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito wrote: > > I think we have a real corruption bug in our hands. I've had more than one corrupted MetaKit file, but in every case, it turned out that *I* had corrupted it. It's easy to do with threads (or multiple processes). You can do it single-threaded, too, by not thinking through callbacks / event driven code, or GUI-driven idle-tasks. In one app, it was an event-driven bug that slipped past months of QA testing and showed up in the field. One of QA's favorite tests was running out of disk space. I never saw my db get corrupted from that. (OTOH, the app did commits frequently, and slung files around like mad, so had lots of out- of-disk-space checks to protect the integrity of the files.) I was never able to learn squat from a corrupted DB. It was only in figuring out how to re-create the problem that it became obvious what I was doing. - Gordon ----rest of message-------------- > Klas sent me an example of a MetaKit data file which > had been corrupted. He believes that it was caused by > a lack of hard disk space. I ran the MetaKit's dump > utility to check the contents and here's the output: > > board.backup: 0 properties > > > VIEW 1 rows = > > Opening the file with a text editor reveals that the > schema has been destroyed. However, the contents of > the file are there. > > I have seen *exactly* the same problem, where 1MB, > 5.3MB, and 10MB files have been corrupted. However, I > disagree with Klas in one respect: hard disk space has > nothing to do with it. I know that because I have 20GB > of free disk space. > > I've seen this a few times, and at first I thought > that it could be that I sent a corrupted schema to > MetaKit, and the confusion caused MK to corrupt the > data file. I still have no evidence that this is the > case, but seeing Klas having the same problem, I > wonder whether we have a serious bug in MetaKit. > > The version I use is 2.3.4r29/C++. > > Does anyone have more info on this issue? Jean-Claude, > would you happen to know what could cause this? If you > need a tangible proof, I can send you and example data > file that has been corrupted. > > So, heads up everybody. > > Take care, > > -- Tito > > > --- Klas Axelsson wrote: > > Have been using MetaKit for a short time and just > > had a problem with a > > corrupt database. > > > > I'm afraid that I haven't examined this very much > > myself yet, I intend to > > do a few tests and such, but I'll give a short > > description right away in > > case I never get around to it. > > > > I'm using Mk4py.so version 2.4.0 for linux. > > > > I find it very likely that the problem was caused by > > a full disk, the file > > could be rescued by removing the end of the file. > > > > If anyone would like to see the corrupt data file > > (about 9k) send me a > > private email. > > > > Regards > > Klas > > > > > > > > > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > __________________________________________________ > Terrorist Attacks on U.S. - How can you help? > Donate cash, emergency relief information > http://dailynews.yahoo.com/fc/US/Emergency_Information/ > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Wed Sep 19 15:38:54 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA05441 for ; Wed, 19 Sep 2001 15:38:53 -0500 Received: from takkie.equi4.local ([195.121.237.245]) by smtp05.wxs.nl (Netscape Messaging Server 4.05) with ESMTP id GJXFVL00.Z92; Wed, 19 Sep 2001 22:35:45 +0200 Received: from [10.0.1.41] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f8JKZ3v15326; Wed, 19 Sep 2001 22:35:08 +0200 From: Jean-Claude Wippler To: Klas Axelsson , metakit Subject: Re: [Metakit] Corrupt file Date: Wed, 19 Sep 2001 22:36:20 +0200 Message-Id: <20010919203620.10776@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Klas, >I'm using Mk4py.so version 2.4.0 for linux. > >I find it very likely that the problem was caused by a full disk, the file >could be rescued by removing the end of the file. > >If anyone would like to see the corrupt data file (about 9k) send me a >private email. Please do send me a copy. This sounds ominous. I have another report of damage - using MK > 2.01, i.e. with the new file format implementation. The fact that removing the end "fixes" the file is a sign that you caught the problem very early on: MK commits by calculating the new file size, putting a dummy marker there, saving changes without altering the last- committed data (but not necessarily writing just at the end: it may write in unused space), and then as very last step changing the end marker to a "commit-ok" one. The last changes (Sep 5) fixed bugs which can corrupt files. I am currently chasing another problem, which seems partly just in Mk4tcl, but has also uncovered a deeper one (though it does not seem to corrupt - just fail). As soon as that has been resolved, I'll put new binaries on the web. As well as all changes in CVS of course. The key question now is: can the problem be repeated? It will be especially important once there is a new build - to make sure everything is fixed. -jcw From PSchlegel@moodlogic.com Wed Sep 26 14:52:55 2001 Received: from hqmx02.hq.moodlogic.com (nobody@dsl081-060-070.sfo1.dsl.speakeasy.net [64.81.60.70]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA23928 for ; Wed, 26 Sep 2001 14:52:38 -0500 Received: by hqmx02.hq.moodlogic.com with Internet Mail Service (5.5.2653.19) id ; Wed, 26 Sep 2001 12:05:18 -0700 Message-ID: <04BE866D1B25C348B9C40E043B216D4836A9B9@hqmx02.hq.moodlogic.com> From: Paul Schlegel To: "'metakit@equi4.com'" Date: Wed, 26 Sep 2001 12:05:17 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Threading problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I'm working on a Win32 project that uses multiple MetaKit databases accessed in different threads. I've seen several notes that MetaKit isn't thread safe, but for some time it seemed that I was getting away with using thread locks around code that accesses these databases until reports of corrupted databases started coming in. After some investigation, a partner of mine noticed that in the file "fileio.cpp" around line 248 there is a block a Win32 code that uses a static buffer (which would be shared by all threads) for writing files. This strongly appears to be a culprit in the corruption that we are seeing. So, my questions are... 1) What are the repercussions of making this static buffer a variable in the c4_FileStrategy class, or perhaps even a stack variable? 2) My project is intended for Win98 and above. There is a comment in the block of code in question that says it is a hack for network drives on Win95. Could I possible remove this block of code (and the static buffer) altogether? 3) Are there other parts of the code that make threaded use of MetaKit impossible, i.e. use of global or static buffers? What parts of the code are they? For clarification, I'm using one thread lock object per database (as opposed to one for all MetaKit accesses). I suppose that if only one lock was used for all database interactions, then any contention with this static buffer would be avoided. Thanks in advance. From horst@freedict.de Thu Sep 27 23:49:08 2001 Received: from mail.netbeat.de ([62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA11566 for ; Thu, 27 Sep 2001 23:49:07 -0500 From: horst@freedict.de Received: (qmail 22678 invoked from network); 28 Sep 2001 04:29:37 -0000 Received: from dialin-145-254-113-076.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.113.76) by mail.netbeat.de with SMTP; 28 Sep 2001 04:29:37 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id f8RL9Is00856 for ; Thu, 27 Sep 2001 23:09:19 +0200 Message-Id: <200109272109.f8RL9Is00856@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Thu, 27 Sep 2001 23:09:15 +0200 (CEST) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] MetaKit usage for Dictionary application Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I asked the question already on this newsgroup, but as I did not receive an answer, I ask again: I am writing a dictionary application in (wx)Python, which stores the data in utf-8. Actually the translation is stored in XML, but this is not of importance, as each translation is stored in a separate database row. I expect to have 100k .. 1M Words (definitions) in the dictionary (for each language), and the translations are about 200bytes summing up to 20MBytes of translation text. The database also should store extra information, as the Author and Date last changed. Two things appear important at the moment: a) the application (obviously) needs to access the translation for a given word quite fast. - there is no need for case sensitivity, or similar string operations I am not sure which of the views I should apply (assuming that doing nothing special will not give efficient access) b) For a wordlist, I want to display the words in a dictionary sorted way, according to the language beeing used (buzzword collate). - MetaKit uses utf-8 to store data, but it does not provide a way of sorting in the required way, does it? Luckily Python does provide a function to generate a index which can be used in a "normal" sort (locale.strxfrm). Would it be a good idea to store the index in a "B" field, and sort it? How would I connect the two requirenments? v1 = view("B:collate,S:word,S:author,S:date") v2 = v1.sort() v3 = v2. ... I use v1 to insert data, v2 to display the wordlist and v3 to access the translation from the "Word" Thank you for your help Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From horst@freedict.de Sat Sep 29 02:40:38 2001 Received: from mail.netbeat.de (mail.netbeat.de [212.6.214.38]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id CAA22518 for ; Sat, 29 Sep 2001 02:40:37 -0500 From: horst@freedict.de Received: (qmail 11461 invoked from network); 29 Sep 2001 07:51:23 -0000 Received: from dialin-145-254-118-200.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.118.200) by mail.netbeat.de with SMTP; 29 Sep 2001 07:51:23 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id f8T7DJI01418 for ; Sat, 29 Sep 2001 09:13:20 +0200 Message-Id: <200109290713.f8T7DJI01418@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Sat, 29 Sep 2001 09:13:17 +0200 (CEST) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] metakit documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I think some of the documentation from "the MetaKit database format" actually should go into the MetaKit documentation. - Otherwise it is difficult to understand some of the mecanics behind MetaKit Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From horst@freedict.de Sat Sep 29 02:40:40 2001 Received: from mail.netbeat.de (mail.netbeat.de [212.6.214.38]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id CAA22523 for ; Sat, 29 Sep 2001 02:40:39 -0500 From: horst@freedict.de Received: (qmail 11482 invoked from network); 29 Sep 2001 07:51:26 -0000 Received: from dialin-145-254-118-200.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.118.200) by mail.netbeat.de with SMTP; 29 Sep 2001 07:51:26 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id f8T7INI01449 for ; Sat, 29 Sep 2001 09:18:24 +0200 Message-Id: <200109290718.f8T7INI01449@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Sat, 29 Sep 2001 09:18:20 +0200 (CEST) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] Compressed memo fields? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, yet another idea: Reading through "the MetaKit data format", I wondered, if (combined with the blocked view) a compressed memo (or whatever) would be feasible. I do have data, which compresses very well (a factor of approx. 10). So compression propably would be a good idea. -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From jrichards@starband.net Sun Sep 30 11:38:31 2001 Received: from c0mailgw08.prontomail.com (mailgw.prontomail.com [216.163.180.10]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA13614 for ; Sun, 30 Sep 2001 11:38:31 -0500 Received: by c0mailgw08.prontomail.com (NPlex 5.5.029) id 3B9FF675004F29F4 for metakit@equi4.com; Sun, 30 Sep 2001 09:27:26 -0700 Received: from 148.75.36.102 by SmtpServer for ; Sun, 30 Sep 2001 16:27:23 +0000 X-Sender: 14195.starband.net From: "Jim Richards" To: Date: Sun, 30 Sep 2001 12:24:29 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Subject: [Metakit] Groupby and Counts Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I am looking for some guidance on using groupby and counts. I am new to python and don not fully understand the documention supplied. I am hoping to use these two functions to further analyze my firewall logs. Thanks Jim Richards From mlh@idi.ntnu.no Sun Sep 30 12:53:22 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA14562 for ; Sun, 30 Sep 2001 12:53:21 -0500 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id TAA27503 for ; Sun, 30 Sep 2001 19:49:29 +0200 (MEST) Message-ID: <011401c149d8$4118c330$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: Date: Sun, 30 Sep 2001 19:49:29 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Time series? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've been thinking about using metakit to store/process time series data, but I'm not quite sure how to store it... Simply put, I have a number of sequences, where each sequence element has a timestamp. The value of the element might be a number, or a vector, or anything, really. (But I can live with only having a number, I guess.) How is this best stored? One possibility is certainly to just stuff each sequence in one binary field... Another is to have two columns per sequence, but when I have hundreds or thousands of sequences, something tells me this isn't a good idea... :) A simple solution would be to use timestamp, sequence ID, and value as three fields in one table... But if I need to access the elements of a sequence sequentially, can this be done efficiently somehow? (I.e. can I be certain that they are stored that way or something?) Perhaps MetaKit isn't the right solution for this, I don't know. -- Magnus Lie Hetland http://www.hetland.org "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick From gmcm@hypernet.com Sun Sep 30 12:57:39 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA14650 for ; Sun, 30 Sep 2001 12:57:39 -0500 Received: from me (204.176.40.88) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Sun, 30 Sep 2001 13:47:53 -0400 From: "Gordon McMillan" To: "Jim Richards" , Date: Sun, 30 Sep 2001 13:53:17 -0400 Subject: Re: [Metakit] Groupby and Counts Reply-to: gmcm@hypernet.com Message-ID: <3BB723CD.1059.4E9FC9FA@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jim Richards wrote: > I am looking for some guidance on using groupby and counts. I am > new to python and don not fully understand the documention > supplied. I am hoping to use these two functions to further > analyze my firewall logs. That's an open-ended question. Let's assume you've shoved your logs into a view log[ipfrom:S,ipto:S,tmstmp:D,devfrom:S,devto:S...] Given: logv = db.view('log') You'd use: v2 = logv.groupby(logv.ipfrom, 'details') or v3 = logv.counts(logv.ipfrom, 'detailcounts') (or any other column(s) in place of ipfrom...) and note that len(v2[i].details) == v3[i].detailcounts Also note that v2[i].details is a view, so can be iterated, groupby-ed, counts-ed or anything else. Does that get you started? - Gordon From jcw@equi4.com Mon Oct 1 04:35:02 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA27687 for ; Mon, 1 Oct 2001 04:35:01 -0500 Received: from takkie.equi4.local ([195.121.232.180]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKIT3U00.NZ7 for ; Mon, 1 Oct 2001 11:31:06 +0200 Received: from [10.0.1.19] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f919TuF16688 for ; Mon, 1 Oct 2001 11:29:57 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Threading problems Date: Mon, 1 Oct 2001 11:31:53 +0200 Message-Id: <20011001093153.23954@mail.planet.nl> In-Reply-To: <04BE866D1B25C348B9C40E043B216D4836A9B9@hqmx02.hq.moodlogic.com> References: <04BE866D1B25C348B9C40E043B216D4836A9B9@hqmx02.hq.moodlogic.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Paul Schlegel wrote: [...] >After some investigation, a partner of mine noticed that in the file >"fileio.cpp" around line 248 there is a block a Win32 code that uses a >static buffer (which would be shared by all threads) for writing files. >This strongly appears to be a culprit in the corruption that we are seeing. > >So, my questions are... > >1) What are the repercussions of making this static buffer a variable in the >c4_FileStrategy class, or perhaps even a stack variable? Either way sounds like a good idea. >2) My project is intended for Win98 and above. There is a comment in the >block of code in question that says it is a hack for network drives on >Win95. Could I possible remove this block of code (and the static buffer) >altogether? No, I doubt it. Memory mapped files in Win9x seem to be somewhat "hacked-on" to the DOS core. The problem is that one cannot write to file without affecting the seek pointer which the memory-mapped read operation requires. Then again, it's quite easy to test: use a network drive and try it. Make local changes in a sufficiently large view, then commit. A few attempts will probably trigger the problem. >3) Are there other parts of the code that make threaded use of MetaKit >impossible, i.e. use of global or static buffers? What parts of the code >are they? No, this is AFAIK the only issue. MK used to be ok, until this particular case had to be worked around. >For clarification, I'm using one thread lock object per database (as opposed >to one for all MetaKit accesses). I suppose that if only one lock was used >for all database interactions, then any contention with this static buffer >would be avoided. Yes, but again, if you fix the above buffer contention, you should be fine as long as each datafile is accessed only within the thread thatr opens it. -jcw From jcw@equi4.com Mon Oct 1 04:35:18 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA27696 for ; Mon, 1 Oct 2001 04:35:18 -0500 Received: from takkie.equi4.local ([195.121.232.180]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKIT4801.8WE for ; Mon, 1 Oct 2001 11:31:20 +0200 Received: from [10.0.1.19] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f919UAF16720 for ; Mon, 1 Oct 2001 11:30:10 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] MetaKit usage for Dictionary application Date: Mon, 1 Oct 2001 11:32:07 +0200 Message-Id: <20011001093207.2361@mail.planet.nl> In-Reply-To: <200109272109.f8RL9Is00856@eaglesnest.mceggman> References: <200109272109.f8RL9Is00856@eaglesnest.mceggman> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: [dictionary app] >Two things appear important at the moment: >a) the application (obviously) needs to access the translation for a >given word quite fast. - there is no need for case sensitivity, or >similar string operations > >I am not sure which of the views I should apply (assuming that doing >nothing special will not give efficient access) I'd look at hashing, and once things get larger also blocking. A hashed mapping provides hashed access as add-on to an existing view. A blocked mapping maintains the illusion of a single view, but internally breaks things up into subviews, which is likely to be more efficient for say >100,000 rows. >b) For a wordlist, I want to display the words in a dictionary sorted >way, according to the language beeing used (buzzword collate). - MetaKit >uses utf-8 to store data, but it does not provide a way of sorting in >the required way, does it? Not directly, but if you create a view with a single int as contents, referring to the index in the big data view, then you can "remap" them, i.e. apply one as a permutation to the other. The result is a view which has been reshuffled: import metakit db=metakit.storage() v1=db.getas('v1[text:S]') v2=db.getas('v2[index:I]') for s in "hello there": v1.append(s) for i in range(len(v1),0,-1): v2.append(i-1) v3=v1.remapwith(v2) metakit.dump(v1,'v1') metakit.dump(v2,'v2') metakit.dump(v3,'v3') The result is that v3 has the contents of v1, but with all rows reversed. It's all virtual, no copying is done. -jcw From jcw@equi4.com Mon Oct 1 04:35:56 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA27716 for ; Mon, 1 Oct 2001 04:35:56 -0500 Received: from takkie.equi4.local ([195.121.232.180]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKIT4J01.61B for ; Mon, 1 Oct 2001 11:31:31 +0200 Received: from [10.0.1.19] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f919ULF16724 for ; Mon, 1 Oct 2001 11:30:21 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Compressed memo fields? Date: Mon, 1 Oct 2001 11:32:18 +0200 Message-Id: <20011001093218.13651@mail.planet.nl> In-Reply-To: <200109290718.f8T7INI01449@eaglesnest.mceggman> References: <200109290718.f8T7INI01449@eaglesnest.mceggman> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: >Reading through "the MetaKit data format", I wondered, if (combined with >the blocked view) a compressed memo (or whatever) would be feasible. > >I do have data, which compresses very well (a factor of approx. 10). So >compression propably would be a good idea. Good point, thanks for bringing up the issue again. There are two aspects to this: compression on an item-by-item basis, and compression over entire columns. The first is easy to add on top of MK, and is in fact what TclKit does by default in the Virtual File System it implements. The second is more involved. There are ways to implement them, similarly to what the "myio" does with encryption, but it means writing a new derived "c4_Strategy" class. I've refrained from implementing this as part of MK, because it would need some sort of new interface to be able to specify what compression engine to use, and because ideally compression should not be done data-file wide but only on specific columns. OTOH, it might be simpler to stick to compress-everything and then simply split things into two datafiles after all. -jcw From jcw@equi4.com Mon Oct 1 04:36:06 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA27735 for ; Mon, 1 Oct 2001 04:36:06 -0500 Received: from takkie.equi4.local ([195.121.232.180]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKIT4T00.H9N for ; Mon, 1 Oct 2001 11:31:41 +0200 Received: from [10.0.1.19] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f919UVF16728 for ; Mon, 1 Oct 2001 11:30:32 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Time series? Date: Mon, 1 Oct 2001 11:32:28 +0200 Message-Id: <20011001093228.28634@mail.planet.nl> In-Reply-To: <011401c149d8$4118c330$156ff181@idi.ntnu.no> References: <011401c149d8$4118c330$156ff181@idi.ntnu.no> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Magnus Lie Hetland wrote: >I've been thinking about using metakit to store/process >time series data, but I'm not quite sure how to store >it... > >Simply put, I have a number of sequences, where each >sequence element has a timestamp. The value of the >element might be a number, or a vector, or anything, >really. (But I can live with only having a number, >I guess.) > >How is this best stored? One possibility is certainly >to just stuff each sequence in one binary field... >Another is to have two columns per sequence, but when >I have hundreds or thousands of sequences, something >tells me this isn't a good idea... :) > >A simple solution would be to use timestamp, sequence >ID, and value as three fields in one table... But if >I need to access the elements of a sequence sequentially, >can this be done efficiently somehow? (I.e. can I be >certain that they are stored that way or something?) > >Perhaps MetaKit isn't the right solution for this, I >don't know. It's hard to tell. The main aspect where you may expect good performance as well as compact representation when using MK, is when the data type is homogenous in "some" direction. MK's data storage is columnar - you need to keep this in mind to get to grips with the implications for type. If sequences are typed, but different sequences can have different types, then you might want to consider a data structure of the form: seq[id:I,type:S,index:I] ints[vec[i:I]] strings[vec[s:S]] doubles[vec[d:D]] What this does is create a sequence of sequence desriptions, each description has an id (well, whatever you need to identify it), a type (being "ints", "strings", "doubles", i.e. the name of the view where the data actually resides. Then, each of the remaining views is a strongly- typed collection of sequences. If a sequence has several fields, then you could specify them instead as: seq[id:I,fields[name:S,type:S,index:I]] The datastructures created this way are essentially vectors: a sequence is one (or more) vectors, accessed by indexing as a subview in the set which has the proper datatype. Hm, maybe I've lost you while describing this... perhaps a brief sample, or a description of a few operations you need to implement, would make it easier to make suggestions or mention trade-offs. For an elaborate system which uses similar design tricks to store generalized graphs in MK, see also Jacob Levy's "e4graph", it's at: http://sf.net/projects/e4graph -jcw From mlh@idi.ntnu.no Mon Oct 1 18:39:33 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA28614 for ; Mon, 1 Oct 2001 18:39:32 -0500 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id BAA28234 for ; Tue, 2 Oct 2001 01:35:35 +0200 (MEST) Message-ID: <036501c14ad1$c507d650$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: Date: Tue, 2 Oct 2001 01:35:35 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] A Question about Blocked and Hashed Views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: (Sorry if this is a very obvious question...) In metakit for Python, it seems that view.hash() and view.blocked() can be called without any arguments, which seems reasonable... But the documentation gives the impression that you must supply a mapview and a blockview, respectively. (And, it seems to be able to specify numkeys for hash you have to supply a mapview). My question is basically, what would be the point of supplying such a view, if its structure is set, and you can't edit it? (Or is that only the case for hash? I haven't really understood what "blockview" is...) And... If these are optional arguments (as they seem to be), perhaps that should be indicated in the docs? Just a thought. -- Magnus Lie Hetland http://www.hetland.org "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick From mlh@idi.ntnu.no Mon Oct 1 18:58:07 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA28914 for ; Mon, 1 Oct 2001 18:58:06 -0500 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id BAA29063 for ; Tue, 2 Oct 2001 01:54:10 +0200 (MEST) Message-ID: <036b01c14ad4$5d631020$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: References: <011401c149d8$4118c330$156ff181@idi.ntnu.no> <20011001093228.28634@mail.planet.nl> Subject: Re: [Metakit] Time series? Date: Tue, 2 Oct 2001 01:54:10 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Jean-Claude Wippler" > > Magnus Lie Hetland wrote: [snip] > It's hard to tell. The main aspect where you may expect good performance > as well as compact representation when using MK, is when the data type is > homogenous in "some" direction. MK's data storage is columnar - you need > to keep this in mind to get to grips with the implications for type. Yes, I've understood that. > If sequences are typed, but different sequences can have different types, > then you might want to consider a data structure of the form: > seq[id:I,type:S,index:I] > ints[vec[i:I]] > strings[vec[s:S]] > doubles[vec[d:D]] Ah... That's clever :) [snip] > If a sequence has several fields, then you could specify them instead as: > seq[id:I,fields[name:S,type:S,index:I]] Even more clever :)) (I'm gradually getting used to the mk way of doing things -- and I like it a lot.) > The datastructures created this way are essentially vectors: a sequence > is one (or more) vectors, accessed by indexing as a subview in the set > which has the proper datatype. Right. > Hm, maybe I've lost you while describing this... perhaps a brief sample, > or a description of a few operations you need to implement, would make it > easier to make suggestions or mention trade-offs. Ok... Small example (based on your suggestions). I'm working on (among other things) comparing time sequences. Let's say you want to compare melodies. The sequence would have two fields: Time and pitch (for each note -- we'll use only the "note on" timestamp). Then I could use something like this? tune[notes[time:D,pitch:D]] (Assuming that the tune id is simply its index.) I guess one of the things that would have to be efficient is to iterate over the notes of a given tune, which would be quite efficient, right? I think the main reason why I couldn't see how to do this sort of thing is that I didn't think of using subviews. (Stuck in a simpler table terminology, I guess ;) > For an elaborate system which uses similar design tricks to store > generalized graphs in MK, see also Jacob Levy's "e4graph", it's at: > http://sf.net/projects/e4graph I've seen it, although I haven't looked at the details. Perhaps I should. > -jcw - Magnus, who is very happy about using MetaKit in Python :)) -- Magnus Lie Hetland http://www.hetland.org "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick From lvirden@cas.org Tue Oct 2 09:50:37 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA17839 for ; Tue, 2 Oct 2001 09:50:37 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id f92Ek8709866 for ; Tue, 2 Oct 2001 10:46:08 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f92Ek7A22432; Tue, 2 Oct 2001 10:46:07 -0400 (EDT) Date: Tue, 2 Oct 2001 10:46:07 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20011002104607.AAB22397@cas.org> To: Subject: [Metakit] Help sought for dealing with the Tcler's Wiki scripted document Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What are the commands needed to interact with the scripts and data that comprise the Tcler's Wiki as it exists today? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Wed Oct 3 07:03:34 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA06235 for ; Wed, 3 Oct 2001 07:03:32 -0500 Received: from takkie.equi4.local ([195.121.234.226]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GKMPAB00.209 for ; Wed, 3 Oct 2001 13:58:59 +0200 Received: from [10.0.1.21] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f93Bw6F00880 for ; Wed, 3 Oct 2001 13:58:06 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Help sought for dealing with the Tcler's Wiki scripted document Date: Wed, 3 Oct 2001 13:59:24 +0200 Message-Id: <20011003115924.11317@mail.planet.nl> In-Reply-To: <20011002104607.AAB22397@cas.org> References: <20011002104607.AAB22397@cas.org> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >What are the commands needed to interact with the scripts and data that >comprise the Tcler's Wiki as it exists today? As with every scripted document: it's all in the scripts for you to inspect and adjust as you like. The complicating factor with the current version of the Tcl'ers Wiki is that it uses pre-VFS style scripted documents. The only way to access *was* through a somewhat tricky "command line interface". But there is good news... here is a script which lets you list and extract the contents of all pages stored in the wiki, it's at: http://www.equi4.com/wikit/wikitool Do a chmod +x and run it without args to get info about usage. This self-contained script should run with any recent Tcl installation, because it includes a brand-new Metakit datafile reader coded entirely in Tcl. Do you wish to extract the wikit's own scripting code as well? -jcw From lvirden@cas.org Wed Oct 3 07:09:16 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA06612 for ; Wed, 3 Oct 2001 07:09:16 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id f93C4i720404 for ; Wed, 3 Oct 2001 08:04:44 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.9.3/CAS_CLIENT-1.17) id f93C4gd04554; Wed, 3 Oct 2001 08:04:42 -0400 (EDT) Date: Wed, 3 Oct 2001 08:04:42 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20011003080442.AAB4543@cas.org> Subject: Re: [Metakit] Help sought for dealing with the Tcler's Wiki In-Reply-To: <20011003115924.11317@mail.planet.nl> of Wed, 3 Oct 2001 13:59:24 +0200 To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler > Do you wish to extract the wikit's own scripting code as well? Yes, I sometimes have a need/desire to do this as well. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From nicolasb@maich.gr Wed Oct 3 09:00:26 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA11401 for ; Wed, 3 Oct 2001 09:00:18 -0500 Received: FROM maich.gr BY mail-server ; Wed Oct 03 17:08:17 2001 +0300 Message-ID: <3BBB1CD8.17CFA553@maich.gr> Date: Wed, 03 Oct 2001 17:12:40 +0300 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Subject: [Metakit] Copying/Saving Views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All, I wonder how one could simply copy a view or a slice of a view from a main data file, and save it to file in Tclkit. I want basically to keep track of record(s) deleted from a view and stick them in a deleted.dat file, which maybe opened for inspection in the future... set main_view [mk::view open main.view] set deleted_view [mk::view open deleted.view] Now what? Currently looping through records and it's not much fun, esp. when a datafile contains~20 views... Would appreciate any help... tia, nicolas boretos From nrogers@softhome.net Sat Oct 6 12:04:14 2001 Received: from softhome.net (jive.SoftHome.net [66.54.152.27]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA16111 for ; Sat, 6 Oct 2001 12:04:14 -0500 Received: from cx292990-b ([24.254.252.167]) by softhome.net with esmtp; Sat, 06 Oct 2001 10:56:57 -0600 Date: Sat, 6 Oct 2001 12:00:21 -0500 From: Nathan Rogers To: metakit@equi4.com Message-Id: <20011006120021.290da3de.nrogers@softhome.net> X-Mailer: Sylpheed version 0.6.2 (GTK+ 1.2.10; i386-debian-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] cvs hash performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've been trying out the CVS version of Metakit lately, and I noticed that the time it takes to append a row to a hashed view has increased dramatically since 2.4. With a few thousand rows, inserts take about 100 times longer than they used to. Adding blocking to the data and/or map views doesn't seem to help. Any ideas on what might be causing this or what can be done about it? Also, I'm having trouble getting Ordered views to work in C++. I've been trying code like this. c4_Storage db; c4_IntProp pKey ("key"); c4_View vw = db.GetAs("test[key:I]"); c4_View ov = vw.Ordered(); ov.Add(pKey[3]); But it always seems to crash. The same thing seems to work fine in python and tcl. Nathan Rogers From tciuro@mac.com Sat Oct 6 13:39:30 2001 Received: from smtp-out.mac.com ([204.179.120.86]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA16798 for ; Sat, 6 Oct 2001 13:39:30 -0500 Received: from smtp-relay01.mac.com (smtp-relay01-qfe3 [10.13.10.224]) by smtp-out.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f96IXOa10691 for ; Sat, 6 Oct 2001 11:33:25 -0700 (PDT) Received: from asmtp01.mac.com ([10.13.10.65]) by smtp-relay01.mac.com (Netscape Messaging Server 4.15 relay01 Jun 21 2001 23:53:48) with ESMTP id GKSRMM00.3SA for ; Sat, 6 Oct 2001 11:35:10 -0700 Received: from localhost ([209.233.25.139]) by asmtp01.mac.com (Netscape Messaging Server 4.15 asmtp01 Jun 21 2001 23:53:48) with ESMTP id GKSRML00.VD0 for ; Sat, 6 Oct 2001 11:35:09 -0700 Date: Sat, 6 Oct 2001 11:35:10 -0700 Mime-Version: 1.0 (Apple Message framework v500) Content-Type: multipart/alternative; boundary=Apple-Mail-1-463790074 From: Tito Ciuro To: metakit@equi4.com Message-Id: X-Mailer: Apple Mail (2.500) Subject: [Metakit] Why is View.Search() not working? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-463790074 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hi, I posted a similar email a while ago and nobody came up with an answer. Let's try again! :) I'm trying to use the Search() method because it's supposed to be faster than Find(). The code I use to try it out is this: > void QuickTest(c4_View *cacheView) > { > c4_View aView = cacheView->Sort(); > int i, num, value; > > { > c4_IntProp pUID ("UID"); > num = aView.GetSize(); > > printf("Number of records: %d\n", num); > for (i = 0; i < num; i++) { > value = pUID(aView[i]); > printf(" ID %d is in row %d\n", value, i); > } > > printf(" Search for -1: is in row %d\n", > aView.Search(pUID[-1])); > printf(" Search for 1: is in row %d\n", > aView.Search(pUID[1])); > printf(" Search for 3: is in row %d\n", > aView.Search(pUID[3])); > } > } This the output I get: > Number of records: 4 > ID 1 is in row 0 > ID 2 is in row 1 > ID 3 is in row 2 > ID 4 is in row 3 > Search for -1: is in row 0 > Search for 1: is in row 0 > Search for 3: is in row 2 How come that searching for a non-existing value returns row zero? (row zero is occupied by UID #1) If Search() returned -1 for non-found values, it'll be a lot better! Is this a known bug? Thanks, -- Tito --Apple-Mail-1-463790074 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hi, I posted a similar email a while ago and nobody came up with an answer. Let's try again! :) I'm trying to use the Search() method because it's supposed to be faster than Find(). The code I use to try it out is this: void QuickTest(c4_View *cacheView) { c4_View aView = cacheView->Sort(); int i, num, value; { c4_IntProp pUID ("UID"); num = aView.GetSize(); printf("Number of records: %d\n", num); for (i = 0; i << num; i++) { value = pUID(aView[i]); printf(" ID %d is in row %d\n", value, i); } printf(" Search for -1: is in row %d\n", aView.Search(pUID[-1])); printf(" Search for 1: is in row %d\n", aView.Search(pUID[1])); printf(" Search for 3: is in row %d\n", aView.Search(pUID[3])); } } This the output I get: Number of records: 4 ID 1 is in row 0 ID 2 is in row 1 ID 3 is in row 2 ID 4 is in row 3 Search for -1: is in row 0 Search for 1: is in row 0 Search for 3: is in row 2 How come that searching for a non-existing value returns row zero? (row zero is occupied by UID #1) If Search() returned -1 for non-found values, it'll be a lot better! Is this a known bug? Thanks, -- Tito --Apple-Mail-1-463790074-- From nrogers@softhome.net Sat Oct 6 14:39:05 2001 Received: from softhome.net (jive.SoftHome.net [66.54.152.27]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA17204 for ; Sat, 6 Oct 2001 14:39:05 -0500 Received: from cx292990-b ([24.254.252.167]) by softhome.net with esmtp; Sat, 06 Oct 2001 13:31:50 -0600 Date: Sat, 6 Oct 2001 14:35:18 -0500 From: Nathan Rogers To: metakit@equi4.com Subject: Re: [Metakit] Why is View.Search() not working? Message-Id: <20011006143518.4445d80e.nrogers@softhome.net> In-Reply-To: References: X-Mailer: Sylpheed version 0.6.2 (GTK+ 1.2.10; i386-debian-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It may not be terribly intuitive, but Search is doing the right thing. Also, the behavior you noticed is documented. Suppose you want to insert a row if the key doesn't exist. If Search returned -1 you would have to binary search again to know where the row should be inserted. So having Search defined the way it is speeds up this common situation. Before you can use the value returned by Search, you need to check the row to see if it really matches or not. I suppose it would be nice for Search to check this for you. Maybe it could take an optional bool* to return this information in. Something like this should work. int c4_View::Search(const c4_RowRef& crit_, bool *found_=0) const { int l = -1, u = GetSize(); while (l + 1 != u) { const int m = (l + u) >> 1; if (_seq->Compare(m, &crit_) < 0) l = m; else u = m; } if(found_ != NULL) { if(u < GetSize() && _seq->Compare(u, &crit_) == 0) *found_ = true; else *found_ = false; } } return u; } Nathan Rogers From tciuro@yahoo.com Sat Oct 6 16:09:10 2001 Received: from web20602.mail.yahoo.com (web20602.mail.yahoo.com [216.136.226.158]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA18117 for ; Sat, 6 Oct 2001 16:09:10 -0500 Message-ID: <20011006210454.95512.qmail@web20602.mail.yahoo.com> Received: from [63.202.175.206] by web20602.mail.yahoo.com via HTTP; Sat, 06 Oct 2001 14:04:54 PDT Date: Sat, 6 Oct 2001 14:04:54 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Why is View.Search() not working? To: Nathan Rogers , metakit@equi4.com In-Reply-To: <20011006143518.4445d80e.nrogers@softhome.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Now it makes more sense... and it's indeed logical the way it works. But like you suggested, perhaps it would be more complete if Search() also returned whether a match was found. Thanks a lot! Best regards, -- Tito __________________________________________________ Do You Yahoo!? NEW from Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. http://geocities.yahoo.com/ps/info1 From jrichards@starband.net Wed Oct 10 19:11:20 2001 Received: from c0mailgw13.prontomail.com (mailgw.prontomail.com [216.163.180.10]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA05301 for ; Wed, 10 Oct 2001 19:11:20 -0500 Received: from C6SERVICE11.prontomail.com (216.163.178.10) by c0mailgw13.prontomail.com (NPlex 5.5.029) id 3BAF80D5003B1A99 for metakit@equi4.com; Wed, 10 Oct 2001 16:59:28 -0700 Received: from gateway (148.75.36.102) by C6SERVICE11.prontomail.com (NPlex 5.5.029) id 3BC2C77E000098A3 for metakit@equi4.com; Wed, 10 Oct 2001 16:59:28 -0700 From: "Jim Richards" To: Date: Wed, 10 Oct 2001 19:56:18 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Subject: [Metakit] Thank You Gordon McMillan!!! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been working with MetaKit for Python for approx. three weeks now. It was suggested that I use it for analyzing my firewall data. Well today I hit a milestone and felt it appropriate to let you and others know that MetaKit is real. I imported 238Mb of data which consisted of 2,315,000 packets or rows of data. The import took approx 45 minutes and the analysis which consists of various counts and groupings took about 15 minutes. I am impressed with MetaKits ease of use and would suggest that more Python programmers use it for similar tasks. Thank you for your effort and I hope to use MetaKit for other projects as well. It truly is a solid product. Jim Richards Unix / NT Administrator Opticom, Inc. 200 Brickstone Square Andover, MA 01810 From gmcm@hypernet.com Wed Oct 10 21:18:03 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA08545 for ; Wed, 10 Oct 2001 21:18:03 -0500 Received: from me (204.176.40.47) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Wed, 10 Oct 2001 22:07:38 -0400 From: "Gordon McMillan" To: "Jim Richards" Date: Wed, 10 Oct 2001 22:13:01 -0400 Subject: Re: [Metakit] Thank You Gordon McMillan!!! Reply-to: gmcm@hypernet.com CC: Message-ID: <3BC4C7ED.26669.83E8E773@localhost> Priority: normal In-reply-to: X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jim, If you find the Python bindings for MetaKit to be pleasant to work with, I am very, very grateful. However, most of your praise should be directed to Jean-Claude Wippler who is the sole author (and genius) behind MetaKit. (Nearly every project I do these days involves MetaKit somehow - a whole bunch of stuff I used to think of as drudgery is now fun.) - Gordon > I have been working with MetaKit for Python for approx. three > weeks now. It was suggested that I use it for analyzing my > firewall data. Well today I hit a milestone and felt it > appropriate to let you and others know that MetaKit is real. I > imported 238Mb of data which consisted of 2,315,000 packets or > rows of data. The import took approx 45 minutes and the analysis > which consists of various counts and groupings took about 15 > minutes. I am impressed with MetaKits ease of use and would > suggest that more Python programmers use it for similar tasks. > > Thank you for your effort and I hope to use MetaKit for other > projects as well. It truly is a solid product. > > Jim Richards > Unix / NT Administrator > Opticom, Inc. > 200 Brickstone Square > Andover, MA 01810 > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit - Gordon From jcw@equi4.com Fri Oct 12 09:21:45 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA05482 for ; Fri, 12 Oct 2001 09:21:45 -0500 Received: from takkie.equi4.local ([195.121.232.221]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL3IEW00.0IG for ; Fri, 12 Oct 2001 15:49:44 +0200 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9CDlrl32485 for ; Fri, 12 Oct 2001 15:48:10 +0200 From: Jean-Claude Wippler To: metakit Date: Fri, 12 Oct 2001 15:49:13 +0200 Message-Id: <20011012134913.28579@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Custom-extended TclKit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There's a README and tar preview on the web, containing adjusted versions of sdx (called nsdx for now), and tclkit (Windows and Linux) - more or less the same as the 8.4-25 distribution with upgraded Tcl/Tk/MK, and a small patch: http://www.equi4.com/previews/nsdx.README http://www.equi4.com/previews/nsdx.tar.gz What these provide, is the option to create a single-file application based on TclKit, and including scripts, libraries, and extensions. I'm making this available for those who need this *now* or want to have some advance info on what is coming. It's rough, you may have to dive in and try out things to make it fly - but it should work. Comments welcome. Patches and improvements too! -jcw From jcw@equi4.com Sun Oct 14 08:54:21 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA03327 for ; Sun, 14 Oct 2001 08:54:21 -0500 Received: from takkie.equi4.local ([195.121.233.233]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL77QD00.OO2 for ; Sun, 14 Oct 2001 15:49:25 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EDlFB01009 for ; Sun, 14 Oct 2001 15:47:15 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Copying/Saving Views Date: Sun, 14 Oct 2001 15:47:54 +0200 Message-Id: <20011014134754.30060@mail.planet.nl> In-Reply-To: <3BBB1CD8.17CFA553@maich.gr> References: <3BBB1CD8.17CFA553@maich.gr> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas Boretos wrote: >I wonder how one could simply copy a view or a slice of a view from a >main data file, and save it to file in Tclkit. I want basically to keep >track of record(s) deleted from a view and stick them in a deleted.dat >file, which maybe opened for inspection in the future... > >set main_view [mk::view open main.view] > >set deleted_view [mk::view open deleted.view] > >Now what? The Tcl objectified interface is not quite as complete as the C++/Python ones. My suggestion would be to try one of the following instead: - Open read-only or nocommit, then delete what you don't want, and "serialize" the remainder, using something like: set fd [open saved.mk w] mk::file save db $fd close $fd This produces a MK datafile with just what is there at the time of saving, and by not committing these changes back in the original datafile, you are essentially create a "filtered" backup. - Open a fresh db, define the structure properly, and then copy views from the original into the new datafile. This is not possible with the base Mk4tcl interface, but with the objectified one, you might get away with it: just copy the rows you want to save. IIRC, the subviews will automatically get copied along with it *if* the strcuture of the new db has the proper structure. >Currently looping through records and it's not much fun, esp. when a >datafile contains~20 views... I don't understand... it's code running *for* you, i.e. the computer is doing the work, right? Is it slow? What am I missing? -jcw From jcw@equi4.com Sun Oct 14 08:54:42 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA03332; Sun, 14 Oct 2001 08:54:41 -0500 Received: from takkie.equi4.local ([195.121.233.233]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL77QC00.QFH; Sun, 14 Oct 2001 15:49:24 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EDlEB01006; Sun, 14 Oct 2001 15:47:14 +0200 From: Jean-Claude Wippler To: announce , metakit Date: Sun, 14 Oct 2001 15:36:29 +0200 Message-Id: <20011014133629.18740@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.1 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There are new distributions of MetaKit and TclKit on the website. These are "maintenance releases" - wrapping up recent fixes and minor changes: http://www.equi4.com/metakit/ http://www.equi4.com/tclkit/ Builds for Linux and Windows have also been uploaded, see: http://www.equi4.com/pub/mk/ http://www.equi4.com/pub/tk/ These include static libs, and the extensions Mk4py and Mk4tcl. The CHANGELOG can be found at http://www.equi4.com/metakit/CHANGES Btw, I am looking for people to see if we can come up with an API and wrapper for Perl - and would appreciate any suggestions or references! -jcw From jcw@equi4.com Sun Oct 14 08:54:43 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA03335 for ; Sun, 14 Oct 2001 08:54:42 -0500 Received: from takkie.equi4.local ([195.121.233.233]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL77QD00.JI7 for ; Sun, 14 Oct 2001 15:49:25 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EDlGB01012 for ; Sun, 14 Oct 2001 15:47:16 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] cvs hash performance Date: Sun, 14 Oct 2001 15:49:56 +0200 Message-Id: <20011014134956.3678@mail.planet.nl> In-Reply-To: <20011006120021.290da3de.nrogers@softhome.net> References: <20011006120021.290da3de.nrogers@softhome.net> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nathan Rogers wrote: >I've been trying out the CVS version of Metakit lately, and I noticed that >the time it takes to append a row to a hashed view has increased >dramatically since 2.4. With a few thousand rows, inserts take about 100 >times longer than they used to. Adding blocking to the data and/or map >views doesn't seem to help. Any ideas on what might be causing this or >what can be done about it? Hm, it loks like I've not really solved the issue right. The hashing logic is probably wrong, causing the code to loop looking for a free slot perhaps. >Also, I'm having trouble getting Ordered views to work in C++. I've been >trying code like this. > >c4_Storage db; >c4_IntProp pKey ("key"); >c4_View vw = db.GetAs("test[key:I]"); >c4_View ov = vw.Ordered(); >ov.Add(pKey[3]); > >But it always seems to crash. The same thing seems to work fine in python >and tcl. I will look into this (debug mode does not provide any clues, I presume?). Thanks for the sample code. -jcw From mlh@idi.ntnu.no Sun Oct 14 09:59:54 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA04358; Sun, 14 Oct 2001 09:59:53 -0500 Received: from vier.idi.ntnu.no (IDENT:22955@vier.idi.ntnu.no [129.241.103.4]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with ESMTP id QAA01533; Sun, 14 Oct 2001 16:55:06 +0200 (MEST) Date: Sun, 14 Oct 2001 16:55:06 +0200 (MEST) From: Magnus Lie Hetland To: Jean-Claude Wippler cc: metakit Subject: Re: [Metakit] MetaKit 2.4.1 In-Reply-To: <20011014133629.18740@mail.planet.nl> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Sun, 14 Oct 2001, Jean-Claude Wippler wrote: > Btw, I am looking for people to see if we can come up with an API and > wrapper for Perl - and would appreciate any suggestions or references! A thought: How about jni-wrapper for Java? I know I would use it... Not sure if I could write it, though :) > -jcw -- Magnus Lie Hetland http://www.hetland.org "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick From jcw@equi4.com Sun Oct 14 14:22:27 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA08149 for ; Sun, 14 Oct 2001 14:22:27 -0500 Received: from takkie.equi4.local ([195.121.232.127]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL7MWG01.1UJ for ; Sun, 14 Oct 2001 21:17:04 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EJEtB02633 for ; Sun, 14 Oct 2001 21:14:55 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] MetaKit 2.4.1 Date: Sun, 14 Oct 2001 21:11:51 +0200 Message-Id: <20011014191151.1131@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Magnus Lie Hetland wrote: >A thought: How about jni-wrapper for Java? I know I would use it... >Not sure if I could write it, though :) Yes, I would be interested in that too (and there have been requests in the past, though less recent). I would need help with this as well, to find out how to set things up (development-wise), and to have some feedback on usability and testing. In itself, MK's C++ design ought to map relatively straight to Java, I would hope. -jcw From mpwilson@earthlink.net Sun Oct 14 15:01:06 2001 Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA08812 for ; Sun, 14 Oct 2001 15:01:05 -0500 Received: from BRIGADOON (1Cust24.VR1.NYC4.broadband.uu.net [63.13.131.24]) by pintail.mail.pas.earthlink.net (EL-8_9_3_3/8.9.3) with ESMTP id MAA07085; Sun, 14 Oct 2001 12:56:12 -0700 (PDT) Date: Sun, 14 Oct 2001 16:05:20 -0400 From: "Michael P. Wilson" X-Mailer: The Bat! (v1.53bis) Personal Reply-To: "Michael P. Wilson" X-Priority: 3 (Normal) Message-ID: <121656456717.20011014160520@earthlink.net> To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] VLDB performance? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hya, I've been lurking around looking for a relatively low-level library for maintaining an extremely large (ok, to me anyway) database of fairly large objects (about 3k of large-ish text columns plus blobs of up to 5 meg.) The semantic needs are very thin. I only ever need append and read-only access, including fairly basic selection and filtering across the set. Are there any immediate "gotchas" in MetaKit that I should know about before giving it a shot? I don't need complex data types, advanced 'select' management, record updating, etc. However, all fields are variable length including the blob field. Oh, btw, when I say "extremely large" I mean something in excess of a 1.2Terabytes (and growing at about 100-200 meg a day.) SQL databases are just too heavyweight. I'd write it myself if I can't find something that readily serves the need. Any comments? (no, not from this crowd ;) Thanks - Mike P.S. Regardless of whether or not metakit fits my particular, fairly peculiar needs, I've got to say it seems like a serious butt kicker. From jcw@equi4.com Sun Oct 14 15:33:05 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA09286 for ; Sun, 14 Oct 2001 15:33:04 -0500 Received: from takkie.equi4.local ([195.121.232.88]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL7Q6A01.H8A for ; Sun, 14 Oct 2001 22:27:46 +0200 Received: from [10.0.1.4] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9EKPdB03064 for ; Sun, 14 Oct 2001 22:25:39 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] VLDB performance? Date: Sun, 14 Oct 2001 22:28:43 +0200 Message-Id: <20011014202843.17133@mail.planet.nl> In-Reply-To: <121656456717.20011014160520@earthlink.net> References: <121656456717.20011014160520@earthlink.net> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Michael P. Wilson wrote: >I've been lurking around looking for a relatively low-level library for >maintaining an extremely large (ok, to me anyway) database of fairly >large objects (about 3k of large-ish text columns plus blobs of up to 5 >meg.) > >The semantic needs are very thin. I only ever need append and >read-only access, including fairly basic selection and filtering across >the set. > >Are there any immediate "gotchas" in MetaKit that I should know about >before giving it a shot? I don't need complex data types, advanced >'select' management, record updating, etc. However, all fields are >variable length including the blob field. > >Oh, btw, when I say "extremely large" I mean something in excess of a >1.2Terabytes (and growing at about 100-200 meg a day.) Most things you say are within what I'd consider tough but still feasible. Large objects do not in themselves cause much trouble, because they are stored individually. Access to a row when the large object field is not used, is as fast as if the large object had not been present, due to the column-wise data organization. But the 1.2 Tb kills things. MK is based on memory mapped files and (currently) requires that the entire file be mapped (R/O). On a 32-bit machine, don't even *think* about datasets over 2 Gb. There is only one way to make this fly: use a 64-bit architecture which supports mapping the entire DBMS. I'd be interested to see where bottlenecks turn up, if any, and then try to resolve it. There are numerous tricks to apply, and the view mechanism on top of MK allows for a wide range of "special-purpose cleverness". Read-only data in itself makes the issue feasible IMO. The "append-only" mode you need is trickier, because - again due to the column-wise design - appends, even of a single item, normally have the effect of copying an entire column. One solution is to segment the data, and then recombine it virtually - which the various custom views can IMO be made to do just fine. But let's not forget the reality that 1 Tb is way, way, over what MK has been used for today - and ever considered for AFAIK. It will take deep knowledge of the access/use/modification patterns and of the MK design to have any chance at success with this. Just a thought: why not save the data serially to file(s), and have MK *manage* it, i.e. store offsets and some key navigational data, for example? Transaction safety can surely be obtained, and the amount of data residing in MK could go down dramatically. Again, keep in mind that custom views can hide this physical design choice. -jcw From ctrueman@dial.pipex.com Mon Oct 15 02:07:55 2001 Received: from gadolinium.btinternet.com (gadolinium.btinternet.com [194.73.73.111]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA17813 for ; Mon, 15 Oct 2001 02:07:55 -0500 Received: from [213.123.154.112] (helo=iclassfactory) by gadolinium.btinternet.com with smtp (Exim 3.22 #6) id 15t1mE-0004dC-00 for metakit@equi4.com; Mon, 15 Oct 2001 08:03:06 +0100 Reply-To: From: "Chris Trueman" To: Date: Mon, 15 Oct 2001 08:00:55 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Subject: [Metakit] Multi-process access to MetaKit repository Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've only glanced at the introduction and some of the sample code and my question is: Can MetaKit be used as a central repository where multiple programs read and write data from/to it? I guess my concerns are about the concurrency features of the engine. Any help would be appreciated. Thanks. Chris. From rcohen@dial.oleane.com Mon Oct 15 02:15:33 2001 Received: from apicra.wanadoo.fr (smtp-rt-3.wanadoo.fr [193.252.19.155]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA17966 for ; Mon, 15 Oct 2001 02:15:32 -0500 Received: from mel-rta8.wanadoo.fr (193.252.19.79) by apicra.wanadoo.fr; 15 Oct 2001 09:10:01 +0200 Received: from dial.oleane.com (80.11.89.224) by mel-rta8.wanadoo.fr; 15 Oct 2001 09:09:56 +0200 Message-ID: <3BCA99F0.80A92AA0@dial.oleane.com> Date: Mon, 15 Oct 2001 09:10:24 +0100 From: Riccardo Cohen Organization: articque X-Mailer: Mozilla 4.77 [en] (WinNT; U) X-Accept-Language: en-US,fr,af,sq,en,en-GB,fr-CA,de-CH,fi MIME-Version: 1.0 To: Mailing list Metakit Subject: Re: [Metakit] MetaKit 2.4.1 References: <20011014191151.1131@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been wrapping several little apis from C to Java. It it really easy with C functions but not so easy with C++ classes (never tried!). I am not a java specialist, but if there is nobody else, I can help. I will not have much time to write or test for the moment but can give examples and information. -- Riccardo Cohen Articque Les Roches 37230 Fondettes France email = rcohen@dial.oleane.com web = http://www.articque.com tel: +33 02 47 49 90 49 fax: +33 02 47 49 91 49 From ctrueman@dial.pipex.com Mon Oct 15 06:30:29 2001 Received: from gadolinium.btinternet.com (gadolinium.btinternet.com [194.73.73.111]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA27543 for ; Mon, 15 Oct 2001 06:30:28 -0500 Received: from [213.123.154.112] (helo=iclassfactory) by gadolinium.btinternet.com with smtp (Exim 3.22 #6) id 15t5sI-0001xU-00 for metakit@equi4.com; Mon, 15 Oct 2001 12:25:38 +0100 Reply-To: From: "Chris Trueman" To: Date: Mon, 15 Oct 2001 12:23:28 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal Subject: [Metakit] Performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I was trialling the performance of the storage engine using code similiar to: // Declare a bunch of string and int properties c4_StringProp name("Name"); c4_IntProp age("Age"); c4_View test = db.GetAt("test[Name:S, Age:I]"); for(int i = 0; i < 1000000; ++i) { c4_Row row; name(row) = age(row) = rnd(65); test.Add(row); db.Commit(); } With a view that contains more properties that what I gave abouve (around 22), the performance is not that great. It writes about 3 or 4 rows per second. I assume that it my lack of knowledge of programming against MetaKit that is causing this poor rate of insertion. Any ideas? Chris. From jcw@equi4.com Mon Oct 15 08:16:41 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA00831 for ; Mon, 15 Oct 2001 08:16:41 -0500 Received: from takkie.equi4.local ([195.121.234.230]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GL90NN00.DUE for ; Mon, 15 Oct 2001 15:11:47 +0200 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9FD9jB08082 for ; Mon, 15 Oct 2001 15:09:45 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Performance Date: Mon, 15 Oct 2001 15:11:37 +0200 Message-Id: <20011015131137.17738@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Chris Trueman wrote: >for(int i = 0; i < 1000000; ++i) >{ > c4_Row row; ^^^^^^^^^^ This one will slow you down a tad, move it up - but you won't see the effect until you first fix the big bottleneck below. Row constructors/ destructors are not top-speed, it's best to reuse them if you can. > name(row) = Strings are considerably slower than ints/floats, since they are variable-size. Clearly there is not much you can do, if you need strings... > age(row) = rnd(65); > > test.Add(row); This is slow, preallocate before entering the loop, then use "test[i] = row;". Or at least resize in chunks, instead of growing the view on each iteration (just as you would with C arrays in general). > db.Commit(); ^^^^^^^^^^^^ That kills performance, it's inside the loop. You need to move it down. Or at least call it every N cycles. >} Have a look at the MetaKit wiki FAQs http://www.equi4.com/metakit/ wiki.cgi for more info. It's not abundant, but it's there. See also: http://www.equi4.com/metakit/api/tips.html#faster Now that you have a baseline, please feel free to report how you progress :) -jcw From claird@starbase.neosoft.com Mon Oct 15 15:19:30 2001 Received: from mx1.airmail.net (mx1.airmail.net [209.196.77.98]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29153 for ; Mon, 15 Oct 2001 15:19:30 -0500 Received: from mail3.iadfw.net ([209.196.123.3]) by mx1.airmail.net with smtp (Exim 3.16 #10) id 15tE8E-0004uB-00; Mon, 15 Oct 2001 15:14:39 -0500 Received: from starbase.neosoft.com from [206.109.1.32] by mail3.iadfw.net (/\##/\ Smail3.1.30.16 #30.61) with esmtp for sender: id ; Mon, 15 Oct 2001 15:17:19 -0500 (CDT) Received: (from claird@localhost) by starbase.neosoft.com (8.9.3/8.9.3) id PAA37455; Mon, 15 Oct 2001 15:19:31 -0500 (CDT) (envelope-from claird) Date: Mon, 15 Oct 2001 15:19:31 -0500 (CDT) From: Cameron Laird Message-Id: <200110152019.PAA37455@starbase.neosoft.com> To: ctrueman@dial.pipex.com, metakit@equi4.com Subject: Re: [Metakit] Multi-process access to MetaKit repository In-Reply-To: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > From metakit-admin@equi4.com Mon Oct 15 02:12:21 2001 > Reply-To: > From: "Chris Trueman" > To: > . > . > . > I've only glanced at the introduction and some of the sample code and my > question is: > Can MetaKit be used as a central repository where multiple programs read and > write data from/to it? > I guess my concerns are about the concurrency features of the engine. > . > . > . Yes and no. Officially, MetaKit "does not offer true concurrent access". That's what the home page says. HOWEVER, plenty of people use MetaKit in multi-user applications. A lovely mechanism for this is Tequila, part of the MetaKit family [http://www.equi4.com/metakit/wiki.cgi/19.html]. From tciuro@mac.com Mon Oct 15 16:55:16 2001 Received: from smtp-out.mac.com ([204.179.120.85]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA02440 for ; Mon, 15 Oct 2001 16:55:16 -0500 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtp-out.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9FLaWJ20454 for ; Mon, 15 Oct 2001 14:36:32 -0700 (PDT) Received: from asmtp01.mac.com ([10.13.10.65]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GL9OKY00.TZR for ; Mon, 15 Oct 2001 14:48:34 -0700 Received: from ciurti ([17.202.22.175]) by asmtp01.mac.com (Netscape Messaging Server 4.15 asmtp01 Jun 21 2001 23:53:48) with ESMTP id GL9OKX00.TIT for ; Mon, 15 Oct 2001 14:48:33 -0700 Date: Mon, 15 Oct 2001 14:48:33 -0700 Mime-Version: 1.0 (Apple Message framework v500) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Tito Ciuro To: metakit@equi4.com Content-Transfer-Encoding: 7bit Message-Id: <612F38BD-C1B6-11D5-8F25-00039310A0E8@mac.com> X-Mailer: Apple Mail (2.500) Subject: [Metakit] How Can I Add/Remove Views in MetaKit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, It seems that the documentation is a bit out of date. I see a "Store" method in c4_Storage, which "[...]Attach a view using specified name in this storage object.", but the the mk4.h file states that it has been dropped. So the question is: is there a way to add and remove views on demand? BTW, I'm using MetaKit/C++. Thanks x 1000, -- Tito From ctrueman@dial.pipex.com Mon Oct 15 17:03:48 2001 Received: from carbon.btinternet.com (carbon.btinternet.com [194.73.73.92]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA02897 for ; Mon, 15 Oct 2001 17:03:47 -0500 Received: from [213.123.154.10] (helo=iclassfactory) by carbon.btinternet.com with smtp (Exim 3.22 #6) id 15tFl8-0006qy-00; Mon, 15 Oct 2001 22:58:55 +0100 Reply-To: From: "Chris Trueman" To: "Tito Ciuro" , Subject: RE: [Metakit] How Can I Add/Remove Views in MetaKit? Date: Mon, 15 Oct 2001 22:56:43 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-Reply-To: <612F38BD-C1B6-11D5-8F25-00039310A0E8@mac.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: GetAs instead? Chris. > It seems that the documentation is a bit out of date. I see a "Store" > method in c4_Storage, which "[...]Attach a view using specified name in > this storage object.", but the the mk4.h file states that it has been > dropped. > > So the question is: is there a way to add and remove views on demand? From tciuro@mac.com Mon Oct 15 17:08:15 2001 Received: from smtp-out.mac.com ([204.179.120.85]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA03166 for ; Mon, 15 Oct 2001 17:08:15 -0500 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtp-out.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9FLnuJ22279 for ; Mon, 15 Oct 2001 14:49:56 -0700 (PDT) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GL9P7B00.90K for ; Mon, 15 Oct 2001 15:01:59 -0700 Received: from ciurti ([17.202.22.175]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GL9P7A00.18J; Mon, 15 Oct 2001 15:01:58 -0700 Date: Mon, 15 Oct 2001 15:01:58 -0700 Subject: Re: [Metakit] How Can I Add/Remove Views in MetaKit? Content-Type: multipart/alternative; boundary=Apple-Mail-4--893685258 Mime-Version: 1.0 (Apple Message framework v500) Cc: To: From: Tito Ciuro In-Reply-To: Message-Id: <40F217A6-C1B8-11D5-8F25-00039310A0E8@mac.com> X-Mailer: Apple Mail (2.500) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-4--893685258 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hi Chris, If I understand correctly, GetAs() creates a "virtual" view which becomes "physical" after Commit(), correct? Once the view has been stored in the data file, how do I remove it? Thanks, -- Tito On Monday, October 15, 2001, at 02:56 PM, Chris Trueman wrote: > GetAs instead? > > > Chris. > >> It seems that the documentation is a bit out of date. I see a "Store" >> method in c4_Storage, which "[...]Attach a view using specified name in >> this storage object.", but the the mk4.h file states that it has been >> dropped. >> >> So the question is: is there a way to add and remove views on demand? > > ____________________________________________ "I'm desperately trying to figure out why kamikaze pilots wore helmets." Dave Edison --Apple-Mail-4--893685258 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hi Chris, If I understand correctly, GetAs() creates a "virtual" view which becomes "physical" after Commit(), correct? Once the view has been stored in the data file, how do I remove it? Thanks, -- Tito On Monday, October 15, 2001, at 02:56 PM, Chris Trueman wrote: GetAs instead? Chris. It seems that the documentation is a bit out of date. I see a "Store" method in c4_Storage, which "[...]Attach a view using specified name in this storage object.", but the the mk4.h file states that it has been dropped. So the question is: is there a way to add and remove views on demand? Lucida Grande ____________________________________________Lucida Grande 0000,0000,FFFF"I'm desperately trying to figure out why kamikaze pilots wore helmets." 0000,6666,3333 Dave Edison --Apple-Mail-4--893685258-- From ctrueman@dial.pipex.com Mon Oct 15 17:15:29 2001 Received: from rhenium (rhenium.btinternet.com [194.73.73.93]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA03637 for ; Mon, 15 Oct 2001 17:15:24 -0500 Received: from [213.123.154.10] (helo=iclassfactory) by rhenium with smtp (Exim 3.22 #6) id 15tFwI-0000Ok-00; Mon, 15 Oct 2001 23:10:26 +0100 Reply-To: From: "Chris Trueman" To: "Tito Ciuro" Cc: Subject: RE: [Metakit] How Can I Add/Remove Views in MetaKit? Date: Mon, 15 Oct 2001 23:08:13 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Importance: Normal In-Reply-To: <40F217A6-C1B8-11D5-8F25-00039310A0E8@mac.com> Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: What about setting the view size to 0 and committing the results? Chris. If I understand correctly, GetAs() creates a "virtual" view which becomes "physical" after Commit(), correct? Once the view has been stored in the data file, how do I remove it? From tciuro@mac.com Mon Oct 15 17:27:09 2001 Received: from smtp-out.mac.com ([204.179.120.85]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA04310 for ; Mon, 15 Oct 2001 17:27:08 -0500 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtp-out.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9FM97J25126 for ; Mon, 15 Oct 2001 15:09:07 -0700 (PDT) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GL9Q3G00.C43 for ; Mon, 15 Oct 2001 15:21:16 -0700 Received: from ciurti ([17.202.22.175]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GL9Q3G00.TA7; Mon, 15 Oct 2001 15:21:16 -0700 Date: Mon, 15 Oct 2001 15:21:15 -0700 Subject: Re: [Metakit] How Can I Add/Remove Views in MetaKit? Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v500) Cc: To: From: Tito Ciuro In-Reply-To: Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.500) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The doc states: "[...]void SetSize (int newSize_, int growBy_ =-1); Since views act like dynamic arrays, you can quickly change their size. Increasing the size will append rows with zero/empty values, while decreasing it will delete the last rows. The growBy_ parameter is currently unused." So it seems that setting the size of a view to zero will remove all rows, leaving the view there. I'd like to remove the view + rows, the whole thing. Thanks, -- Tito On Monday, October 15, 2001, at 03:08 PM, Chris Trueman wrote: > What about setting the view size to 0 and committing the results? > > > Chris. > > If I understand correctly, GetAs() creates a "virtual" view which > becomes > "physical" after Commit(), correct? Once the view has been stored in the > data file, how do I remove it? From jcw@equi4.com Thu Oct 18 05:43:12 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA07956 for ; Thu, 18 Oct 2001 05:43:11 -0500 Received: from takkie.equi4.local ([195.121.233.208]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLEDJK00.EI8 for ; Thu, 18 Oct 2001 12:38:08 +0200 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9IAaTB28821 for ; Thu, 18 Oct 2001 12:36:30 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] cvs hash performance Date: Thu, 18 Oct 2001 12:39:05 +0200 Message-Id: <20011018103905.25823@mail.planet.nl> In-Reply-To: <20011006120021.290da3de.nrogers@softhome.net> References: <20011006120021.290da3de.nrogers@softhome.net> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nathan Rogers wrote: >I've been trying out the CVS version of Metakit lately, and I noticed that >the time it takes to append a row to a hashed view has increased >dramatically since 2.4. With a few thousand rows, inserts take about 100 >times longer than they used to. Adding blocking to the data and/or map >views doesn't seem to help. Any ideas on what might be causing this or >what can be done about it? This was a glaring bug, introduced recently and causing the hash map to be reconstructed on every insertion... >Also, I'm having trouble getting Ordered views to work in C++. I've been >trying code like this. > >c4_Storage db; >c4_IntProp pKey ("key"); >c4_View vw = db.GetAs("test[key:I]"); >c4_View ov = vw.Ordered(); >ov.Add(pKey[3]); > >But it always seems to crash. The same thing seems to work fine in python >and tcl. This was a problem in interaction between inserts (which are positional) and ordered views (which ignore position, and implicitly derive one). The Add implementation was changed to use InsertAt instead of SetAtGrow (which does not work with ordered views). The code in CVS should be ok now for both issues. Thanks for reporting both problems, and for the sample code which helped figure out this second problem. -jcw From jcw@equi4.com Thu Oct 18 07:15:14 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA13210 for ; Thu, 18 Oct 2001 07:15:13 -0500 Received: from takkie.equi4.local ([195.121.234.220]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLEHSV01.0MB for ; Thu, 18 Oct 2001 14:10:07 +0200 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9IC8TB29245 for ; Thu, 18 Oct 2001 14:08:29 +0200 From: Jean-Claude Wippler To: metakit Date: Thu, 18 Oct 2001 14:03:16 +0200 Message-Id: <20011018120316.2001@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Mapped view performance overview Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There's now a page on the MetaKit wiki, exploring the various aspects of performance w.r.t. adding, commits, and searching. This is the output of a Tcl script which is in the MK 2.4.x distributions - annotated to explain some of the behavior: http://www.equi4.com/metakit/wiki.cgi/129.html As with all performance issues - please add salt to suit your taste... :) -jcw From jcw@equi4.com Thu Oct 18 07:50:29 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA15539 for ; Thu, 18 Oct 2001 07:50:28 -0500 Received: from takkie.equi4.local ([195.121.234.111]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLEJER00.GCC for ; Thu, 18 Oct 2001 14:44:51 +0200 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9ICh0B29438 for ; Thu, 18 Oct 2001 14:43:02 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] How Can I Add/Remove Views in MetaKit? Date: Thu, 18 Oct 2001 14:45:21 +0200 Message-Id: <20011018124521.7856@mail.planet.nl> In-Reply-To: <612F38BD-C1B6-11D5-8F25-00039310A0E8@mac.com> References: <612F38BD-C1B6-11D5-8F25-00039310A0E8@mac.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito Ciuro wrote: >So the question is: is there a way to add and remove views on demand? Yes, use GetAs with a view name but no structure, i.e. to define v1: c4_View v = storage.GetAs("v1[...]"); To remove it (just ignore the result value from this call): storage.GetAs("v1"); You may still see the view as temporary property, but after commit it's gone. The name "GetAs" is a bit misleading, now that "Store" is gone. One reason for getting rid of Store, was to encourage you to set up the view with GetAs *before* filling it. You can still do it after the fact, but it's less efficient: c4_View vdata; <... fill vdata here ...> c4_View v = storage.GetAs("v1[...]"); v.InsertAt(0, vdata); This will do a full copy (and consume twice as much memory). -jcw From rjn103s@mail.mgr3.k12.mo.us Fri Oct 19 14:07:31 2001 Received: from mail.mgr3.k12.mo.us (mail.mgr3.k12.mo.us [204.184.227.130]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA15167 for ; Fri, 19 Oct 2001 14:07:30 -0500 Received: (from Unknown UID 65534@localhost) by mail.mgr3.k12.mo.us (8.11.6/8.11.4) id f9JIw4R05141; Fri, 19 Oct 2001 13:58:04 -0500 (CDT) (envelope-from rjn103s@mail.mgr3.k12.mo.us) Date: Fri, 19 Oct 2001 13:58:04 -0500 (CDT) Message-Id: <200110191858.f9JIw4R05141@mail.mgr3.k12.mo.us> X-Authentication-Warning: mail.mgr3.k12.mo.us: Unknown UID 65534 set sender to rjn103s@mail.mgr3.k12.mo.us using -f From: "Richard Nelson" To: metakit@equi4.com Reply-To: rjn103s@mail.mgr3.k12.mo.us X-Mailer: NeoMail 1.25 X-IPAddress: 172.16.0.5 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] FreeBSD4.4 complie help needed for MetaKit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, using Tcl-8.3.3 / Tk-8.3 ============Make tcl output======================== make tcl /bin/sh ./libtool --mode=compile c++ -c -O2 -fomit-frame-pointer -I../unix/../include -I../unix/../../include -I/usr/local/lib/tcl8.3/../generic -DUSE_TCL_STUBS ../unix/../tcl/mk4tcl.cpp rm -f .libs/mk4tcl.lo c++ -c -O2 -fomit-frame-pointer -I../unix/../include -I../unix/../../include -I/usr/local/lib/tcl8.3/../generic -DUSE_TCL_STUBS ../unix/../tcl/mk4tcl.cpp -fPIC -DPIC -o .libs/mk4tcl.lo In file included from ../unix/../tcl/mk4tcl.cpp:5: ../unix/../tcl/mk4tcl.h:8: tcl.h: No such file or directory *** Error code 1 Any ideas? Many thanks! -- Panther - Webmail Nelson MG From stug@pacbell.net Sun Oct 21 15:56:55 2001 Received: from snfc21.pbi.net (mta6.snfc21.pbi.net [206.13.28.240]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA29080 for ; Sun, 21 Oct 2001 15:56:55 -0500 Received: from slk ([207.105.204.47]) by mta6.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with SMTP id <0GLK00GMVPY381@mta6.snfc21.pbi.net> for metakit@equi4.com; Sun, 21 Oct 2001 13:51:39 -0700 (PDT) Date: Sun, 21 Oct 2001 13:57:54 -0700 From: Stu To: metakit@equi4.com Message-id: <000901c15a73$0e525460$0200a8c0@pacbell.net> MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-Mailer: Microsoft Outlook Express 6.00.2600.0000 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7bit X-Priority: 3 X-MSMail-priority: Normal Subject: [Metakit] How do I do a static build with Metakit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I know this isn't Metakit specific but I can't seem to get a build that doesn't require the metakit dll. I am also linking statically to mfc. Thanks From OOO@vmts.ru Sun Oct 21 23:29:23 2001 Received: from mail2.vmts.ru (mail.vmts.ru [212.122.1.133]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA05869 for ; Sun, 21 Oct 2001 23:29:21 -0500 Received: by mail2.vmts with Internet Mail Service (5.5.2653.19) id ; Mon, 22 Oct 2001 15:19:12 +1100 Message-ID: <09DE0F07AB09D4119D390050DA394B84A7CE78@mail2.vmts> From: "Oleg O. Orlov" To: metakit@equi4.com Date: Mon, 22 Oct 2001 15:19:12 +1100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C15AB0.B3D095E0" Subject: [Metakit] more than 3 hash keys Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C15AB0.B3D095E0 Content-Type: text/plain; charset="koi8-r" what's wrong in following sources ???: create some properties: c4_StringProp pSourceIp ("source_ip"); c4_IntProp pSourcePort ("source_port"); c4_StringProp pDestIp ("dest_ip"); c4_IntProp pDestPort ("dest_port"); c4_IntProp pPkts ("pkts"); link it with storage c4_Storage storage ("connections.dat", true); create base view c4_View vNote = storage.GetAs( "vNote[source_ip:S,source_port:I,dest_ip:S,dest_port:I,pkts:I"); create view for hash c4_View hNote = storage.GetAs("hNote[_H:I,_R:I]"); create hash with 4 keys stored as special view c4_View hash = vNote.Hash(hNote,4); _____________________________________________________________ All works perfect up to 3 first key of base view An 4-th key causes infinite loop in program ... _________________ Oleg Orlov ------_=_NextPart_001_01C15AB0.B3D095E0 Content-Type: text/html; charset="koi8-r"
what's wrong in following sources ???:
 
    create some properties:
 
c4_StringProp pSourceIp ("source_ip");
c4_IntProp pSourcePort ("source_port");
c4_StringProp pDestIp ("dest_ip");    
c4_IntProp pDestPort ("dest_port");   
c4_IntProp pPkts ("pkts");            
 
    link it with storage
 
c4_Storage storage ("connections.dat", true);
 
    create base view
 
c4_View vNote = storage.GetAs(
    "vNote[source_ip:S,source_port:I,dest_ip:S,dest_port:I,pkts:I");
 
    create view for hash
 
c4_View hNote = storage.GetAs("hNote[_H:I,_R:I]");
 
    create hash with 4 keys stored as special view
 
c4_View hash = vNote.Hash(hNote,4);
 
_____________________________________________________________
 
 
All works perfect up to 3 first key of base view
An 4-th key causes infinite loop in program
 
 
...
 
 
_________________
Oleg Orlov
------_=_NextPart_001_01C15AB0.B3D095E0-- From rjn103s@mail.mgr3.k12.mo.us Tue Oct 23 09:19:51 2001 Received: from mail.mgr3.k12.mo.us (mail.mgr3.k12.mo.us [204.184.227.130]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA24331 for ; Tue, 23 Oct 2001 09:19:50 -0500 Received: (from Unknown UID 65534@localhost) by mail.mgr3.k12.mo.us (8.11.6/8.11.4) id f9NE9gD82886; Tue, 23 Oct 2001 09:09:42 -0500 (CDT) (envelope-from rjn103s@mail.mgr3.k12.mo.us) Date: Tue, 23 Oct 2001 09:09:42 -0500 (CDT) Message-Id: <200110231409.f9NE9gD82886@mail.mgr3.k12.mo.us> X-Authentication-Warning: mail.mgr3.k12.mo.us: Unknown UID 65534 set sender to rjn103s@mail.mgr3.k12.mo.us using -f From: "Richard Nelson" To: metakit@equi4.com Reply-To: rjn103s@mail.mgr3.k12.mo.us X-Mailer: NeoMail 1.25 X-IPAddress: 172.16.0.1 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] FreeBSD now compiles but fails test-tcl (assistance needed) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, first let me thanks Dirk Meyer for his work in providing me a FreeBSD port for metakit-2.4.1 that compiles. I am testing so it can be added to the FreeBSD ports tree. The problem is that the port will compile but fails the test-tcl option. Below is, I hope, the critical output from this make option to provide insight: c++ -c -I./../unix/../include ./../unix/../tests/tbasics.cpp -o tbasics.o c++ -c -I./../unix/../include ./../unix/../tests/tcusto1.cpp -o tcusto1.o c++ -c -I./../unix/../include ./../unix/../tests/tcusto2.cpp -o tcusto2.o c++ -c -I./../unix/../include ./../unix/../tests/tdiffer.cpp -o tdiffer.o c++ -c -I./../unix/../include ./../unix/../tests/textend.cpp -o textend.o c++ -c -I./../unix/../include ./../unix/../tests/tformat.cpp -o tformat.o c++ -c -I./../unix/../include ./../unix/../tests/tlimits.cpp -o tlimits.o c++ -c -I./../unix/../include ./../unix/../tests/tmapped.cpp -o tmapped.o c++ -c -I./../unix/../include ./../unix/../tests/tnotify.cpp -o tnotify.o c++ -c -I./../unix/../include ./../unix/../tests/tresize.cpp -o tresize.o c++ -c -I./../unix/../include ./../unix/../tests/tstore1.cpp -o tstore1.o c++ -c -I./../unix/../include ./../unix/../tests/tstore2.cpp -o tstore2.o c++ -c -I./../unix/../include ./../unix/../tests/tstore3.cpp -o tstore3.o c++ -c -I./../unix/../include ./../unix/../tests/tstore4.cpp -o tstore4.o /bin/sh ./libtool --mode=link c++ -static -o regress regress.o tbasics.o tcusto1.o tcusto2.o tdiffer.o textend.o tformat.o tlimits.o tmapped.o tnotify.o tresize.o tstore1.o tstore2.o tstore3.o tstore4.o libmk4.a c++ -o regress regress.o tbasics.o tcusto1.o tcusto2.o tdiffer.o textend.o tformat.o tlimits.o tmapped.o tnotify.o tresize.o tstore1.o tstore2.o tstore3.o tstore4.o libmk4.a regress.o: In function `ViewDisplay(c4_View const &, __sFILE *, int)': regress.o(.text+0x201): undefined reference to `c4_String::c4_String(char, int)' regress.o: In function `c4_String::c4_String(void)': regress.o(.gnu.linkonce.t.__9c4_String+0x13): undefined reference to `c4_String::Init(void const *, int)' regress.o: In function `c4_String::operator+=(c4_String const &)': regress.o(.gnu.linkonce.t.__apl__9c4_StringRC9c4_String+0x26): undefined reference to `operator+(c4_String const &, c4_String const &)' regress.o(.gnu.linkonce.t.__apl__9c4_StringRC9c4_String+0x55): undefined reference to `c4_String::operator=(c4_String const &)' regress.o: In function `c4_String::GetLength(void) const': regress.o(.gnu.linkonce.t.GetLength__C9c4_String+0x21): undefined reference to `c4_String::FullLength(void) const' tbasics.o: In function `TestBasics(void)': tbasics.o(.text+0x5cc): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0xfce): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0x1212): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0x13be): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0xa9a0): undefined reference to `c4_String::c4_String(char const *)' tbasics.o(.text+0xaa89): more undefined references to `c4_String::c4_String(char const *)' follow tbasics.o: In function `operator+(c4_String const &, char const *)': tbasics.o(.gnu.linkonce.t.__pl__FRC9c4_StringPCc+0x5e): undefined reference to `operator+(c4_String const &, c4_String const &)' tbasics.o: In function `operator==(char const *, c4_String const &)': tbasics.o(.gnu.linkonce.t.__eq__FPCcRC9c4_String+0x14): undefined reference to `c4_String::Compare(char const *) const' tbasics.o: In function `operator==(c4_String const &, char const *)': tbasics.o(.gnu.linkonce.t.__eq__FRC9c4_StringPCc+0x14): undefined reference to `c4_String::Compare(char const *) const' tcusto1.o: In function `TestCustom1(void)': tcusto1.o(.text+0x47e0): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x48cc): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x49b8): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x4aa4): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x5f09): undefined reference to `c4_String::c4_String(char const *)' tcusto1.o(.text+0x5ff1): more undefined references to `c4_String::c4_String(char const *)' follow textend.o: In function `c4_String::c4_String(void const *, int)': textend.o(.gnu.linkonce.t.__9c4_StringPCvi+0x19): undefined reference to `c4_String::Init(void const *, int)' tlimits.o: In function `TestLimits(void)': tlimits.o(.text+0x16c): undefined reference to `c4_String::operator=(c4_String const &)' tlimits.o(.text+0xac1): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0xc1c): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0xd81): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0x231c): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0x24f0): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.text+0x2614): more undefined references to `c4_String::c4_String(char const *)' follow tlimits.o: In function `TestLimits(void)': tlimits.o(.text+0x307f): undefined reference to `c4_String::operator=(c4_String const &)' tlimits.o: In function `c4_String::operator+=(char const *)': tlimits.o(.gnu.linkonce.t.__apl__9c4_StringPCc+0x25): undefined reference to `c4_String::c4_String(char const *)' tlimits.o: In function `operator+(char const *, c4_String const &)': tlimits.o(.gnu.linkonce.t.__pl__FPCcRC9c4_String+0x29): undefined reference to `c4_String::c4_String(char const *)' tlimits.o(.gnu.linkonce.t.__pl__FPCcRC9c4_String+0x5e): undefined reference to `operator+(c4_String const &, c4_String const &)' tnotify.o: In function `TestNotify(void)': tnotify.o(.text+0x84dc): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x85d0): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x86c4): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x87b8): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x88ac): undefined reference to `c4_String::c4_String(char const *)' tnotify.o(.text+0x89a0): more undefined references to `c4_String::c4_String(char const *)' follow tresize.o: In function `TestResize(void)': tresize.o(.text+0x1a1e): undefined reference to `c4_String::c4_String(char, int)' tresize.o(.text+0x1b02): undefined reference to `c4_String::c4_String(char, int)' tresize.o(.text+0x1bee): undefined reference to `c4_String::c4_String(char, int)' tresize.o(.text+0x1cc2): undefined reference to `c4_String::c4_String(char, int)' tstore3.o: In function `TestStores3(void)': tstore3.o(.text+0x6b4): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0x7a4): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0x894): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0x984): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0xab4): undefined reference to `c4_String::c4_String(char const *)' tstore3.o(.text+0xba4): more undefined references to `c4_String::c4_String(char const *)' follow *** Error code 1 Stop in /usr/home/rjent/work2/metakit/work/metakit-2.4.1/builds. *** Error code 1 Stop in /usr/home/rjent/work2/metakit. *** Error code 1 Stop in /usr/home/rjent/work2/metakit. *** Error code 1 Stop in /usr/home/rjent/work2/metakit. Any assistance on correcting this matter will be greatly appreciated! Again thanks to Dirk Meyer and a special thanks to JCW for all his work! -- Panther - Webmail Nelson MG From jcw@equi4.com Tue Oct 23 11:43:33 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA01445 for ; Tue, 23 Oct 2001 11:43:33 -0500 Received: from takkie.equi4.local ([195.121.235.177]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLO3IR01.8AS for ; Tue, 23 Oct 2001 18:37:39 +0200 Received: from [10.0.1.14] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9NGahB24663 for ; Tue, 23 Oct 2001 18:36:43 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] FreeBSD now compiles but fails test-tcl (assistance needed) Date: Tue, 23 Oct 2001 18:37:59 +0200 Message-Id: <20011023163759.3257@mail.planet.nl> In-Reply-To: <200110231409.f9NE9gD82886@mail.mgr3.k12.mo.us> References: <200110231409.f9NE9gD82886@mail.mgr3.k12.mo.us> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Richard Nelson wrote: >first let me thanks Dirk Meyer for his work in providing me a FreeBSD >port for metakit-2.4.1 that compiles. I am testing so it can be added >to the FreeBSD ports tree. The problem is that the port will compile but >fails the test-tcl option. Below is, I hope, the critical output from >this make option to provide insight: > >c++ -c -I./../unix/../include ./../unix/../tests/tbasics.cpp -o tbasics.o [...] >c++ -o regress regress.o tbasics.o tcusto1.o tcusto2.o tdiffer.o >textend.o tformat.o tlimits.o tmapped.o tnotify.o tresize.o tstore1.o >tstore2.o tstore3.o tstore4.o libmk4.a >regress.o: In function `ViewDisplay(c4_View const &, __sFILE *, int)': >regress.o(.text+0x201): undefined reference to >`c4_String::c4_String(char, int)' >regress.o: In function `c4_String::c4_String(void)': >regress.o(.gnu.linkonce.t.__9c4_String+0x13): undefined reference to >`c4_String::Init(void const *, int)' [...] This definition appears to be in mk4str.inl. If the main code was compiled with option "-Dq4_STD" then the same setting should be given to compile regress.cpp etc. Either that, or there is a mixup with inline enabled/disabled. Try adding "-Dq4_INLINE" to the compilation of regress.cpp etc. Would it be possible to send me info on how to improve the build process for FreeBSD? Or alternately, could you add a note on the MetaKit Wiki on how what needs to be adjusted to build MK 2.4.1 on FreeBSD? The page to adjust is: http://www.equi4.com/metakit/wiki.cgi/35.html As for Tcl: be sure to specify the "--with-tcl=" option to configure, with being the path of the unix/ dir of a compiled (recent) Tcl distribution. -jcw From jcw@equi4.com Tue Oct 23 12:01:10 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA02536 for ; Tue, 23 Oct 2001 12:01:10 -0500 Received: from takkie.equi4.local ([195.121.235.177]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLO3VG00.JU4 for ; Tue, 23 Oct 2001 18:45:16 +0200 Received: from [10.0.1.14] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9NGiEB24685 for ; Tue, 23 Oct 2001 18:44:14 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] How do I do a static build with Metakit? Date: Tue, 23 Oct 2001 18:45:06 +0200 Message-Id: <20011023164506.4861@mail.planet.nl> In-Reply-To: <000901c15a73$0e525460$0200a8c0@pacbell.net> References: <000901c15a73$0e525460$0200a8c0@pacbell.net> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stu wrote: >I know this isn't Metakit specific but I can't seem to get a build that >doesn't require the metakit dll. I am also linking statically to mfc. This sort of stuff can be quite tricky to figure and get right. If you are using MSVC, then you should build the "mklib" target, not "mkdll". The MSVC project is in "win/msvc60/mksrc.dsw". BTW, there is a static build of MK 2.4.1 for Windows over here: http://www.equi4.com/pub/mk/metakit-2.4.1-31-win/mk4vc60s.lib -jcw From rjn103s@mail.mgr3.k12.mo.us Tue Oct 23 13:23:39 2001 Received: from mail.mgr3.k12.mo.us (mail.mgr3.k12.mo.us [204.184.227.130]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA07897 for ; Tue, 23 Oct 2001 13:23:39 -0500 Received: (from Unknown UID 65534@localhost) by mail.mgr3.k12.mo.us (8.11.6/8.11.4) id f9NIDSD02232; Tue, 23 Oct 2001 13:13:28 -0500 (CDT) (envelope-from rjn103s@mail.mgr3.k12.mo.us) Date: Tue, 23 Oct 2001 13:13:28 -0500 (CDT) Message-Id: <200110231813.f9NIDSD02232@mail.mgr3.k12.mo.us> X-Authentication-Warning: mail.mgr3.k12.mo.us: Unknown UID 65534 set sender to rjn103s@mail.mgr3.k12.mo.us using -f From: "Richard Nelson" To: metakit@equi4.com Reply-To: rjn103s@mail.mgr3.k12.mo.us Subject: Re: [Metakit] FreeBSD now compiles but fails test-tcl (Success!) X-Mailer: NeoMail 1.25 X-IPAddress: 172.16.0.1 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, Greetings, > Either that, or there is a mixup with inline enabled/disabled. Try > adding "-Dq4_INLINE" to the compilation of regress.cpp etc. > yes this seemed to do the trick!! All runs well and so does the test! > Would it be possible to send me info on how to improve the build process > for FreeBSD? Or alternately, could you add a note on the MetaKit Wiki on > how what needs to be adjusted to build MK 2.4.1 on FreeBSD? The page to > adjust is: > http://www.equi4.com/metakit/wiki.cgi/35.html First all credit must go to Dirk Meyer who provided me with a port for metakit. I believe that the plan is to put this in the ports tree and keep the port up to date. If a FreeBSD user wants MetaKit they will only have to cd /usr/ports/databases/metakit and do a make. There will be some settings that they need to adjust in their /etc/make.conf for specific building of metakit and these can be documented after the final port is established and submitted to the ports tree;) I will try to assist with this:) I have forwarded you messge to Dirk and he may be able to provide ways to improve the compile under FreeBSD. As for now a freshly rolled port seems to address much:) Perhaps after the new port is submitted a make option for the TclKit could be added;) > > As for Tcl: be sure to specify the "--with-tcl=" option to > configure, with being the path of the unix/ dir of a compiled > (recent) Tcl distribution. I think Dirk did magic in the port Makefile to address the dir setting for FreeBSD4.4. > > -jcw > Thanks so much for your reply and I will attempt to do some documentation on the wikit after final port is rolled and submitted. > -- Panther - Webmail Nelson MG From rjn103s@mail.mgr3.k12.mo.us Wed Oct 24 09:51:05 2001 Received: from mail.mgr3.k12.mo.us (mail.mgr3.k12.mo.us [204.184.227.130]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA04052 for ; Wed, 24 Oct 2001 09:51:05 -0500 Received: (from Unknown UID 65534@localhost) by mail.mgr3.k12.mo.us (8.11.6/8.11.4) id f9OEejl87684; Wed, 24 Oct 2001 09:40:45 -0500 (CDT) (envelope-from rjn103s@mail.mgr3.k12.mo.us) Date: Wed, 24 Oct 2001 09:40:45 -0500 (CDT) Message-Id: <200110241440.f9OEejl87684@mail.mgr3.k12.mo.us> X-Authentication-Warning: mail.mgr3.k12.mo.us: Unknown UID 65534 set sender to rjn103s@mail.mgr3.k12.mo.us using -f From: "Richard Nelson" To: metakit@equi4.com Reply-To: rjn103s@mail.mgr3.k12.mo.us X-Mailer: NeoMail 1.25 X-IPAddress: 172.16.0.5 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] FreeBSD4.4 status (some errors during make testing) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, Dirk has kindly provided a new port to me that seems close to production ready, however, now that testing has been integrated into the port some feedback from the tests suggest that there may still be some issues. Below is a snippet from where the testing gives feedback of problems: diff ./../unix/../tests/ok tests cd ./../unix/../tcl/test && tclsh8.3 all.tcl Processing 9 scripts...  mk1basic    FAILED: mk1basic.6 - open same file again  mk2chan  mk3struct  mk4commit  mk5object    FAILED: mk5object.7 - blocked hash view  mk6fixed  mk7limit  mk8fail  mk9crash Failed 2 of 32 tests: *** mk1basic.6 - open same file again *** C db2 {file open failed} *** mk5object.7 - blocked hash view *** C 17 9 Thoughts welcome! -- Panther - Webmail Nelson MG From john_matthews27@hotmail.com Fri Oct 26 02:52:35 2001 Received: from hotmail.com (f119.law14.hotmail.com [64.4.21.119]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA05209 for ; Fri, 26 Oct 2001 02:52:25 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Fri, 26 Oct 2001 00:46:22 -0700 Received: from 193.120.90.34 by lw14fd.law14.hotmail.msn.com with HTTP; Fri, 26 Oct 2001 07:46:22 GMT X-Originating-IP: [193.120.90.34] From: "John Matthews" To: metakit@equi4.com Date: Fri, 26 Oct 2001 07:46:22 +0000 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 26 Oct 2001 07:46:22.0615 (UTC) FILETIME=[4EA77270:01C15DF2] Subject: [Metakit] Metakit Memory Consmption Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm evaluating Metakit 2.4.0 as the engine for a Windows-based product involving browsing a CDROM database. I'm using Borland C++ Builder 5. The database is encrypted using a Strategy class. It consists of a single table with 76K rows and 28 columns. Most columns are strings and about three have strings of highly variable length (mostly short, but a few exceeding 1KB). The total database size is 39MB. Tests browsing the database show very high memory consumption at 36MB (up from 5MB initially), even though only one row was accessed at a time (one c4_View, one c4-Row, ...). Could the memory consumption be caused by Metakit? If so, is there anything I can do to mitigate it? Thanks in Advance JM _________________________________________________________________ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp From jcw@equi4.com Fri Oct 26 04:57:25 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA10543 for ; Fri, 26 Oct 2001 04:57:24 -0500 Received: from takkie.equi4.local ([195.121.232.87]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLT4Q401.1S8 for ; Fri, 26 Oct 2001 11:51:40 +0200 Received: from [10.0.1.15] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9Q9p5B06912 for ; Fri, 26 Oct 2001 11:51:06 +0200 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Metakit Memory Consmption Date: Fri, 26 Oct 2001 11:47:14 +0200 Message-Id: <20011026094714.16044@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: John Matthews wrote: >[...] The database is encrypted using a Strategy class. [...] >Tests browsing the database show very high memory consumption at 36MB (up >from 5MB initially), even though only one row was accessed at a time (one >c4_View, one c4-Row, ...). > >Could the memory consumption be caused by Metakit? >If so, is there anything I can do to mitigate it? Yes, the strategy class disables the use of memory mapped files, or more precisely: it creates a copy in memory of all data. You can get rid of all buffered data by closing and re-opening, or by calling c4_Storage::Rollback, which is more convenient. In both cases, you have to "re-attach" the view: your view object becomes empty, so you have to call c4_Storage::View or c4_Storage::GetAs again. This should be relatively quick in MK 2.4, because the time to open a datafile is no longer proportional to the complexity of the stored data structure, but more or less constant. Then again, calculating file offsets for a column with 76K strings does take a few cycles. You can reduce the latency on first access (also after rollback), by considering a "Blocked" view, i.e. one which segments a view into a bunch of smaller subviews. This requires a change in your data structure. I couldn't find a C++ example to illustrate, not even a Python one, but there is one in Tcl, see examples/mapped.tcl, line 23. The idea that instead of a view "v[a:S,b:S,c:S]", you define the view as "_B[v[a:S,b:S,c:S]]". Then, instead of c4_View v = storage.View("v"); use: c4_View v = storage.View("_B"); v = v.Blocked(); The rest of your code does not need to change. It is essential that you do the same while filling the view with data. What this does is to transparently create an extra layer of indirection, storing the 76K or so rows in subviews with each some 8.5K rows. Access for a full traversal will be slower, but access to a single item will avoid the (one-time) 76K string offset calculations needed when a column is accessed. You'll need to compare to find out whether such a change is an improvement. Feel free to post comparisons and timing/memory results to this mailing list :) Regards, Jean-Claude From jcw@equi4.com Fri Oct 26 04:57:46 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA10559 for ; Fri, 26 Oct 2001 04:57:45 -0500 Received: from takkie.equi4.local ([195.121.232.87]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GLT4Q500.N5A for ; Fri, 26 Oct 2001 11:51:41 +0200 Received: from [10.0.1.15] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9Q9p6B06915 for ; Fri, 26 Oct 2001 11:51:07 +0200 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] FreeBSD4.4 status (some errors during make testing) Date: Fri, 26 Oct 2001 11:51:57 +0200 Message-Id: <20011026095157.28662@mail.planet.nl> In-Reply-To: <200110241440.f9OEejl87684@mail.mgr3.k12.mo.us> References: <200110241440.f9OEejl87684@mail.mgr3.k12.mo.us> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id EAA10559 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Richard Nelson wrote: >  mk1basic >    FAILED: mk1basic.6 - open same file again This is a bug in the test suite, you can ignore it. It is a remnant from the time when MK did locking on the file (which is a hairy topic, cross- platform). I'll fix it - thanks! >  mk5object >    FAILED: mk5object.7 - blocked hash view This is probably fixed in the latest revision in CVS, release 2.4.1 had a bug in the hashing logic, see the change log at: http://www.equi4.com/metakit/CHANGES -jcw From mlh@idi.ntnu.no Sat Oct 27 13:31:06 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA22242 for ; Sat, 27 Oct 2001 13:31:05 -0500 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id UAA07196 for ; Sat, 27 Oct 2001 20:25:25 +0200 (MEST) Message-ID: <006b01c15f14$bf889af0$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: Date: Sat, 27 Oct 2001 20:25:25 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Another possible binding... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've come to think of another binding that would be great to have for MetaKit: PHP. Whenever I use PHP I keep using flat text files (comma-separated or the like), because I usually don't want the hassle of SQL databases, and dbm-style bases aren't flexible enough... MetaKit would be just wonderful to have there... (Perhaps it could even be included in the PHP distribution? They don't seem adverse to include lots of stuff... ;) Just a thought. I have no idea of how to implement such bindings for PHP myself. -- Magnus Lie Hetland http://hetland.org From nathan.thompson-amato@dnamerican.com Mon Oct 29 15:01:32 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id PAA01143 for ; Mon, 29 Oct 2001 15:01:31 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id qa054382 for ; Mon, 29 Oct 2001 16:06:52 -0500 Message-ID: <3BDDC23E.9040304@dnamerican.com> Date: Mon, 29 Oct 2001 15:55:26 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com References: <006b01c15f14$bf889af0$156ff181@idi.ntnu.no> Content-Type: multipart/mixed; boundary="------------050302070103040506000906" Subject: [Metakit] Hashed lookups in C++ Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------050302070103040506000906 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, all. I'm new to MetaKit, and I'm having a bit of trouble using hashed lookups on views combined with Commit() calls; as far as I can tell, my Commit()s aren't actually writing anything to the data file except the key for each record. If anybody here can point me at a snippet of working code that uses hashed lookups, I'd appreciate it. Also, I've attached a short segment of code that demonstrates the problem I'm having; the first time it's run, it works as expected, but once the data.dat file is created, "age" is always zero. Thanks, Nathan --------------050302070103040506000906 Content-Type: text/plain; name="mktest.cc" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="mktest.cc" #include #include int main(void) { c4_StringProp name ("name"); c4_IntProp age ("age"); c4_Storage storage ("data.dat", 1); c4_View view = storage.GetAs("data[name:S,age:I]"); // Create a mapping view for fast lookups by name c4_View view_h = storage.GetAs("data_H1[_H:I,_R:I]"); // Build the hash table c4_View hash_view = view.Hash(view_h, 1); c4_Row row; name (row) = "Bob"; age (row) = 20; hash_view.Add(row); storage.Commit(); // Look for "Bob" int index = hash_view.Find(name ["Bob"]); cout << "Bob is at index " << index << endl; row = hash_view[index]; cout << "Info for Bob:" << (const char *)(name(row)) << ";" << (t4_i32)(age(row)) << endl; return 0; } --------------050302070103040506000906-- From tciuro@mac.com Mon Oct 29 20:20:38 2001 Received: from smtpout.mac.com ([204.179.120.85]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA14324 for ; Mon, 29 Oct 2001 20:20:38 -0600 Received: from smtp-relay01.mac.com (smtp-relay01-qfe3 [10.13.10.224]) by smtpout.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9U2Eda29509 for ; Mon, 29 Oct 2001 18:14:39 -0800 (PST) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay01.mac.com (Netscape Messaging Server 4.15 relay01 Jun 21 2001 23:53:48) with ESMTP id GLZY8L00.162 for ; Mon, 29 Oct 2001 18:14:45 -0800 Received: from ciurti.apple.com ([17.202.22.175]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GLZY8K00.86I for ; Mon, 29 Oct 2001 18:14:44 -0800 Date: Mon, 29 Oct 2001 18:14:39 -0800 Mime-Version: 1.0 (Apple Message framework v501) Content-Type: multipart/alternative; boundary=Apple-Mail-1-331075676 From: Tito Ciuro To: metakit@equi4.com Message-Id: X-Mailer: Apple Mail (2.501) Subject: [Metakit] Problems with View.Search()? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-331075676 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hello, I have the following method: SInt32 quickFindRow (DataFileInfo* info, SInt32 uid) { if (info == NULL || info->mainViewStructure == NULL) return - kRecordNotFound; c4_View aView = *info->viewCache; if (aView.GetSize() == 0) return kRecordNotFound; // Find where the UID is (or should be) c4_IntProp pUID (k_AF_UIDProperty); SInt32 rowNumber = aView.Search(pUID[uid]); // Get the UID of the selected row and see if it matches // with what we're looking for SInt32 uidInRow = pUID(aView[rowNumber]); if (uidInRow == uid) // Yes, we found the match return rowNumber; else // We have to do this because sometimes MK seems to go one row above the match (???) if (rowNumber > 0) { rowNumber--; uidInRow = pUID(aView[rowNumber]); if (uidInRow == uid) // Yes, we found the match return rowNumber; } // No, the UID doesn't exist return kRecordNotFound; } There is a problem with Search(): I'm looking for ID 284 in the datafile, which I know it's located in row #266 (dump confirms that) but row 267 is returned instead (ID 285). Why is that? This is why I decrement the row number in the above snippet, then compare the row at that position and return the proper result (the row number if found; kRecordNotFound (-1) otherwise). This is a hack, of course!... Why is Search() behaving like that? Is Search() returning the next row where the ID should go? Is there something I'm missing? Some notes/assumptions: - c4_View aView = *info->viewCache is a view that I keep open while the datafile is open. I do that because calling GetAs() turns out to be an expensive operation. All ops (adding, editing, removing) are performed in this view. I also keep it sorted (I think!) by calling: _info->viewCache = sortView(_info->viewCache); c4_View* sortView (c4_View* aView) { c4_View sortedView = *aView; sortedView.Sort(); return aView; } - I tried to use Locate() instead: int rowNumber = kRecordNotFound; SInt32 numMatches = aView.Locate(pUID[uid], &rowNumber); It returns zero matches and row #267. I have no idea why... Questions: 1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I sorting the view properly? 2) Why is Search() and Locate() failing? Thanks so much, -- Tito --Apple-Mail-1-331075676 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hello, I have the following method: 0000,0000,FFFFSInt32 quickFindRow (DataFileInfo* info, SInt32 uid) { if (info == NULL || info->mainViewStructure == NULL) return - kRecordNotFound; c4_View aView = *info->viewCache; if (aView.GetSize() == 0) return kRecordNotFound; // Find where the UID is (or should be) c4_IntProp pUID (k_AF_UIDProperty); SInt32 rowNumber = aView.Search(pUID[uid]); // Get the UID of the selected row and see if it matches // with what we're looking for SInt32 uidInRow = pUID(aView[rowNumber]); if (uidInRow == uid) // Yes, we found the match return rowNumber; else // We have to do this because sometimes MK seems to go one row above the match (???) if (rowNumber > 0) { rowNumber--; uidInRow = pUID(aView[rowNumber]); if (uidInRow == uid) // Yes, we found the match return rowNumber; } // No, the UID doesn't exist return kRecordNotFound; } There is a problem with Search(): I'm looking for ID 284 in the datafile, which I know it's located in row #266 (dump confirms that) but row 267 is returned instead (ID 285). Why is that? This is why I decrement the row number in the above snippet, then compare the row at that position and return the proper result (the row number if found; kRecordNotFound (-1) otherwise). This is a hack, of course!... Why is Search() behaving like that? Is Search() returning the next row where the ID should go? Is there something I'm missing? Some notes/assumptions: - c4_View aView = *info->viewCache is a view that I keep open while the datafile is open. I do that because calling GetAs() turns out to be an expensive operation. All ops (adding, editing, removing) are performed in this view. I also keep it sorted (I think!) by calling: 0000,0000,FFFF_info->viewCache = sortView(_info->viewCache); c4_View* sortView (c4_View* aView) { c4_View sortedView = *aView; sortedView.Sort(); return aView; } - I tried to use Locate() instead: 0000,0000,FFFFint rowNumber = kRecordNotFound; SInt32 numMatches = aView.Locate(pUID[uid], &rowNumber); It returns zero matches and row #267. I have no idea why... Questions: 1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I sorting the view properly? 2) Why is Search() and Locate() failing? Thanks so much, -- Tito --Apple-Mail-1-331075676-- From jcw@equi4.com Tue Oct 30 00:57:51 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id AAA18725 for ; Tue, 30 Oct 2001 00:57:51 -0600 Received: from takkie.equi4.local ([195.121.234.25]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM0B2M00.NA3 for ; Tue, 30 Oct 2001 07:51:58 +0100 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9U6nYI08063 for ; Tue, 30 Oct 2001 07:49:44 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Problems with View.Search()? Date: Tue, 30 Oct 2001 07:49:01 +0100 Message-Id: <20011030064901.1028@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito Ciuro wrote: >Why is Search() behaving like that? Is Search() returning the next row >where the ID should go? Is there something I'm missing? Search returns the position of the row where it is. If absent, it returns the position where it should be inserted to maintain sort order. Search only works if the view is sorted, and uses binary search. >_info->viewCache = sortView(_info->viewCache); > >c4_View* sortView (c4_View* aView) >{ > c4_View sortedView = *aView; > sortedView.Sort(); > return aView; >} This does little more than eat CPU cycles. You're throwing away the sorted result. This call is a no-op. >1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I >sorting the view properly? Nope, I'm afraid not. >2) Why is Search() and Locate() failing? I'm pretty sure that if you were to check the view, you'd find it not to have all rows in sorted order. But you have to be careful how you fix this. Doing "v = v.Sort()" will produce a sorted view, but it also creates (and then hides) an extra level of indirection at each call. The result of a sort is a virtual view, which consists of a pointer to the original plus a permutation vector. Access will become slower each time you stack a sort on top of the previous one, and modification will very quickly become unusable (wrong in fact, due to the current limitations on stacked sort view updates). There are a few ways out: - keep the view sorted at all times and keep it stored that way - or sort once on open, then (copy it and) keep the volatile copy sorted - do not sort, but hash instead, if this used for exact-key access - write your own sort/ordering logic, optionally hide it in a custom view - stick with brute force, i.e. linear scan It could be helpful to write a little checking routine which traverses the entire view and verifies that the ID of each successive row is larger than its predecessor. It would no doubt have caught the unsorted status of your view. -jcw From jcw@equi4.com Tue Oct 30 01:22:50 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA19181 for ; Tue, 30 Oct 2001 01:22:49 -0600 Received: from takkie.equi4.local ([195.121.232.92]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM0C8800.MLD for ; Tue, 30 Oct 2001 08:16:56 +0100 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9U7EgI08276 for ; Tue, 30 Oct 2001 08:14:42 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Hashed lookups in C++ Date: Tue, 30 Oct 2001 08:18:00 +0100 Message-Id: <20011030071800.23880@mail.planet.nl> In-Reply-To: <3BDDC23E.9040304@dnamerican.com> References: <3BDDC23E.9040304@dnamerican.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nathan Thompson-Amato wrote: >Hi, all. I'm new to MetaKit, and I'm having a bit of trouble using hashed >lookups on views combined with Commit() calls; as far as I can tell, my >Commit()s aren't actually writing anything to the data file except the key >for each record. > >If anybody here can point me at a snippet of working code that uses hashed >lookups, I'd appreciate it. Also, I've attached a short segment of code >that demonstrates the problem I'm having; the first time it's run, it works >as expected, but once the data.dat file is created, "age" is always zero. [...] >int main(void) >{ [...] > name (row) = "Bob"; > age (row) = 20; > hash_view.Add(row); > > storage.Commit(); [...] >} This works for me - in the sense that the view will not grow and keep on returning the same info (age 20), because hashing assumes keys are unique, so "adding" in this case becomes a replace internally. I suspect that you are not using the very latest revision. A bug in hashing was fixed *after* the 2.4.1 release, see: http://www.equi4.com/metakit/CHANGES You're best off getting the latest from CVS, see: http://www.equi4.com/metakit/download.html -jcw From nathan.thompson-amato@dnamerican.com Tue Oct 30 08:20:15 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA04097 for ; Tue, 30 Oct 2001 08:20:14 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ha054425 for ; Tue, 30 Oct 2001 09:24:39 -0500 Message-ID: <3BDEB577.40107@dnamerican.com> Date: Tue, 30 Oct 2001 09:13:11 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Hashed lookups in C++ References: <3BDDC23E.9040304@dnamerican.com> <20011030071800.23880@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > >> name (row) = "Bob"; >> age (row) = 20; >> hash_view.Add(row); >> >> storage.Commit(); >> > > This works for me - in the sense that the view will not grow and keep on > returning the same info (age 20), because hashing assumes keys are > unique, so "adding" in this case becomes a replace internally. Using MetaKit 2.4.1, this code returned age 20 the first time and age 0 every subsequent time. (I should've mentioned that in my first post.) I've just tried the same code with the latest CVS version, and it seems to work the way I expect (always returning 20). Thanks very much for your help, Nathan From j.p.fletcher@aston.ac.uk Tue Oct 30 08:25:33 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA04486 for ; Tue, 30 Oct 2001 08:25:32 -0600 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 15yZk3-0007Ub-00 for metakit@equi4.com; Tue, 30 Oct 2001 14:19:47 +0000 From: "John Fletcher" To: Date: Tue, 30 Oct 2001 14:17:43 -0000 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] Hashed lookups in C++ Priority: normal In-reply-to: <20011030071800.23880@mail.planet.nl> References: <3BDDC23E.9040304@dnamerican.com> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler To: Subject: Re: [Metakit] Hashed lookups in C++ Date sent: Tue, 30 Oct 2001 08:18:00 +0100 > Nathan Thompson-Amato wrote: > > >Hi, all. I'm new to MetaKit, and I'm having a bit of trouble using hashed > >lookups on views combined with Commit() calls; as far as I can tell, my > >Commit()s aren't actually writing anything to the data file except the key > >for each record. > > > >If anybody here can point me at a snippet of working code that uses hashed > >lookups, I'd appreciate it. Also, I've attached a short segment of code > >that demonstrates the problem I'm having; the first time it's run, it works > >as expected, but once the data.dat file is created, "age" is always zero. > [...] > >int main(void) > >{ > [...] > > name (row) = "Bob"; > > age (row) = 20; > > hash_view.Add(row); > > > > storage.Commit(); > [...] > >} > > This works for me - in the sense that the view will not grow and keep on > returning the same info (age 20), because hashing assumes keys are > unique, so "adding" in this case becomes a replace internally. > > I suspect that you are not using the very latest revision. A bug in > hashing was fixed *after* the 2.4.1 release, see: > http://www.equi4.com/metakit/CHANGES > You're best off getting the latest from CVS, see: > http://www.equi4.com/metakit/download.html > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > I have just encountered the same sort of problem with hashed tables with duplicate keys. Is it possible to get access to the patched code other than via cvs, which I do not have on my PC based system. Thanks John ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From jcw@equi4.com Tue Oct 30 09:15:56 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA07731 for ; Tue, 30 Oct 2001 09:15:55 -0600 Received: from takkie.equi4.local ([195.121.235.100]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM0Y4S01.MH1 for ; Tue, 30 Oct 2001 16:10:04 +0100 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9UF7rI11048 for ; Tue, 30 Oct 2001 16:07:53 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Hashed lookups in C++ Date: Tue, 30 Oct 2001 16:09:39 +0100 Message-Id: <20011030150939.390@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Tue, Oct 30, 2001, John Fletcher wrote: [...] >Is it possible to get access to the patched code other than via cvs, >which I do not have on my PC based system. Isn't there wincvs? Anyway, I've placed a current CVS snapshot over here: http://www.equi4.com/previews/metakit.tar.gz -jcw From tciuro@mac.com Tue Oct 30 09:34:31 2001 Received: from smtpout.mac.com (smtpout.mac.com [204.179.120.89]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA09056 for ; Tue, 30 Oct 2001 09:34:30 -0600 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtpout.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9UFSX221767 for ; Tue, 30 Oct 2001 07:28:33 -0800 (PST) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GM0YZN00.E8J for ; Tue, 30 Oct 2001 07:28:35 -0800 Received: from localhost ([209.233.25.139]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GM0YZM00.DGW for ; Tue, 30 Oct 2001 07:28:34 -0800 Date: Tue, 30 Oct 2001 07:28:34 -0800 Subject: Re: [Metakit] Problems with View.Search()? Content-Type: multipart/alternative; boundary=Apple-Mail-1-378710705 Mime-Version: 1.0 (Apple Message framework v501) From: Tito Ciuro To: metakit In-Reply-To: <20011030064901.1028@mail.planet.nl> Message-Id: X-Mailer: Apple Mail (2.501) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-378710705 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hello, Thanks Jean-Claude for your prompt answer. > - do not sort, but hash instead, if this used for exact-key access I liked your suggestion, so I implemented it and seems to work fine (searching/editing/adding/removing). I based my implementation on the code used in the regression test as supplied with MetaKit: > c4_View v1 = s1.GetAs("v1[p1:S]"); > c4_View v2 = s1.GetAs("v2[_H:I,_R:I]"); > c4_View v3 = v1.Hash(v2, 1); What I noticed though is that v2 is also saved on file. Is v2 needed on disk? If possible, how can avoid having MetaKit save the hashed view on disk each time I call Commit()? Thanks in advance, -- Tito On Monday, October 29, 2001, at 10:49 PM, Jean-Claude Wippler wrote: > Tito Ciuro wrote: > >> Why is Search() behaving like that? Is Search() returning the next row >> where the ID should go? Is there something I'm missing? > > Search returns the position of the row where it is. If absent, it > returns the position where it should be inserted to maintain sort order. > Search only works if the view is sorted, and uses binary search. > >> _info->viewCache = sortView(_info->viewCache); >> >> c4_View* sortView (c4_View* aView) >> { >> c4_View sortedView = *aView; >> sortedView.Sort(); >> return aView; >> } > > This does little more than eat CPU cycles. You're throwing away the > sorted result. This call is a no-op. > >> 1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I >> sorting the view properly? > > Nope, I'm afraid not. > >> 2) Why is Search() and Locate() failing? > > I'm pretty sure that if you were to check the view, you'd find it not to > have all rows in sorted order. > > But you have to be careful how you fix this. Doing "v = v.Sort()" will > produce a sorted view, but it also creates (and then hides) an extra > level of indirection at each call. The result of a sort is a virtual > view, which consists of a pointer to the original plus a permutation > vector. Access will become slower each time you stack a sort on top of > the previous one, and modification will very quickly become unusable > (wrong in fact, due to the current limitations on stacked sort view > updates). > > There are a few ways out: > - keep the view sorted at all times and keep it stored that way > - or sort once on open, then (copy it and) keep the volatile copy > sorted > - do not sort, but hash instead, if this used for exact-key access > - write your own sort/ordering logic, optionally hide it in a custom > view > - stick with brute force, i.e. linear scan > > It could be helpful to write a little checking routine which traverses > the entire view and verifies that the ID of each successive row is > larger > than its predecessor. It would no doubt have caught the unsorted status > of your view. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > --Apple-Mail-1-378710705 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hello, Thanks Jean-Claude for your prompt answer. 0000,0000,DEDE - do not sort, but hash instead, if this used for exact-key access 0000,0000,DEDE 0000,0000,0000I liked your suggestion, so I implemented it and seems to work fine (searching/editing/adding/removing). I based my implementation on the code used in the regression test as supplied with MetaKit: c4_View v1 = s1.GetAs("v1[p1:S]"); c4_View v2 = s1.GetAs("v2[_H:I,_R:I]"); c4_View v3 = v1.Hash(v2, 1); What I noticed though is that v2 is also saved on file. Is v2 needed on disk? If possible, how can avoid having MetaKit save the hashed view on disk each time I call Commit()? Thanks in advance, -- Tito 0000,0000,DEDE On Monday, October 29, 2001, at 10:49 PM, Jean-Claude Wippler wrote: Tito Ciuro < wrote: Why is Search() behaving like that? Is Search() returning the next row where the ID should go? Is there something I'm missing? Search returns the position of the row where it is. If absent, it returns the position where it should be inserted to maintain sort order. Search only works if the view is sorted, and uses binary search. _info->viewCache = sortView(_info->viewCache); c4_View* sortView (c4_View* aView) { c4_View sortedView = *aView; sortedView.Sort(); return aView; } This does little more than eat CPU cycles. You're throwing away the sorted result. This call is a no-op. 1) Assuming the view is defined as main[ID:I;First:S;Last:S], am I sorting the view properly? Nope, I'm afraid not. 2) Why is Search() and Locate() failing? I'm pretty sure that if you were to check the view, you'd find it not to have all rows in sorted order. But you have to be careful how you fix this. Doing "v = v.Sort()" will produce a sorted view, but it also creates (and then hides) an extra level of indirection at each call. The result of a sort is a virtual view, which consists of a pointer to the original plus a permutation vector. Access will become slower each time you stack a sort on top of the previous one, and modification will very quickly become unusable (wrong in fact, due to the current limitations on stacked sort view updates). There are a few ways out: - keep the view sorted at all times and keep it stored that way - or sort once on open, then (copy it and) keep the volatile copy sorted - do not sort, but hash instead, if this used for exact-key access - write your own sort/ordering logic, optionally hide it in a custom view - stick with brute force, i.e. linear scan It could be helpful to write a little checking routine which traverses the entire view and verifies that the ID of each successive row is larger than its predecessor. It would no doubt have caught the unsorted status of your view. -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit --Apple-Mail-1-378710705-- From jcw@equi4.com Tue Oct 30 09:40:41 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA09504 for ; Tue, 30 Oct 2001 09:40:40 -0600 Received: from takkie.equi4.local ([195.121.235.236]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM0Z9Y00.UYP for ; Tue, 30 Oct 2001 16:34:46 +0100 Received: from [10.0.1.2] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9UFWZI11359 for ; Tue, 30 Oct 2001 16:32:35 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Problems with View.Search()? Date: Tue, 30 Oct 2001 16:34:20 +0100 Message-Id: <20011030153420.3530@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito Ciuro wrote: >> - do not sort, but hash instead, if this used for exact-key access > >I liked your suggestion, so I implemented it and seems to work fine >(searching/editing/adding/removing). I based my implementation on the >code used in the regression test as supplied with MetaKit: > >> c4_View v1 = s1.GetAs("v1[p1:S]"); >> c4_View v2 = s1.GetAs("v2[_H:I,_R:I]"); >> c4_View v3 = v1.Hash(v2, 1); > >What I noticed though is that v2 is also saved on file. Is v2 needed on >disk? If possible, how can avoid having MetaKit save the hashed view on >disk each time I call Commit()? Doesn't the following work as expected? c4_View v1 = s1.GetAs("v1[p1:S]"); c4_View v2; c4_View v3 = v1.Hash(v2, 1); -jcw From tciuro@mac.com Tue Oct 30 10:03:15 2001 Received: from smtpout.mac.com (smtpout.mac.com [204.179.120.89]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA11397; Tue, 30 Oct 2001 10:03:15 -0600 Received: from smtp-relay02.mac.com (smtp-relay02-qfe3 [10.13.10.225]) by smtpout.mac.com (8.10.2/8.10.2/1.0) with ESMTP id f9UFvD201176; Tue, 30 Oct 2001 07:57:13 -0800 (PST) Received: from asmtp01.mac.com ([10.13.10.65]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GM10BI00.GH6; Tue, 30 Oct 2001 07:57:18 -0800 Received: from localhost ([209.233.25.139]) by asmtp01.mac.com (Netscape Messaging Server 4.15 asmtp01 Jun 21 2001 23:53:48) with ESMTP id GM10BH00.K0L; Tue, 30 Oct 2001 07:57:17 -0800 Date: Tue, 30 Oct 2001 07:57:17 -0800 Subject: Re: [Metakit] Hashed lookups in C++ Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v501) Cc: metakit@equi4.com To: Jean-Claude Wippler From: Tito Ciuro In-Reply-To: <20011030071800.23880@mail.planet.nl> Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.501) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, > Doesn't the following work as expected? > > c4_View v1 = s1.GetAs("v1[p1:S]"); > c4_View v2; > c4_View v3 = v1.Hash(v2, 1); Thanks Jean-Claude once again for your help. It works as expected. I guess the confusion was in this comment found in the view.cpp file: > * The defined structure of the map view must be "_H:I,_R:I". Also, the code in the regression test (file tmapped.cpp) seemed to confirmed that: > c4_StringProp p1 ("p1"); > > c4_Storage s1; > c4_View v1 = s1.GetAs("v1[p1:S]"); > c4_View v2 = s1.GetAs("v2[_H:I,_R:I]"); > c4_View v3 = v1.Hash(v2, 1); So this is why I thought it was required to have this type of view. In any case, thank you! :) -- Tito On Monday, October 29, 2001, at 11:18 PM, Jean-Claude Wippler wrote: > Nathan Thompson-Amato wrote: > >> Hi, all. I'm new to MetaKit, and I'm having a bit of trouble using >> hashed >> lookups on views combined with Commit() calls; as far as I can tell, my >> Commit()s aren't actually writing anything to the data file except the >> key >> for each record. >> >> If anybody here can point me at a snippet of working code that uses >> hashed >> lookups, I'd appreciate it. Also, I've attached a short segment of >> code >> that demonstrates the problem I'm having; the first time it's run, it >> works >> as expected, but once the data.dat file is created, "age" is always >> zero. > [...] >> int main(void) >> { > [...] >> name (row) = "Bob"; >> age (row) = 20; >> hash_view.Add(row); >> >> storage.Commit(); > [...] >> } > > This works for me - in the sense that the view will not grow and keep on > returning the same info (age 20), because hashing assumes keys are > unique, so "adding" in this case becomes a replace internally. > > I suspect that you are not using the very latest revision. A bug in > hashing was fixed *after* the 2.4.1 release, see: > http://www.equi4.com/metakit/CHANGES > You're best off getting the latest from CVS, see: > http://www.equi4.com/metakit/download.html > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > From steve@improvision.com Tue Oct 30 18:38:20 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA19753 for ; Tue, 30 Oct 2001 18:38:19 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id AAA08064 for ; Wed, 31 Oct 2001 00:30:37 GMT Received: from [10.0.1.2] ([213.105.144.33]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971 for ; Wed, 31 Oct 2001 00:32:25 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Wed, 31 Oct 2001 00:32:22 +0000 From: Steve Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Problem with partial access of byte properties in a hashed view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, New to Metakit and not sure whether I am doing something wrong or have found a bug! I have a view that has a bytes (B) property and is hashed on the first column (a unique integer key to the view). I find that I cannot partially access the bytes field using Access() or Modify() through the hash view - the c4_Bytes object returned is empty. I can reproduce this by changing test b26 as follows: B(b26, Partial memo field access, 0) { c4_BytesProp p1 ("p1"); c4_Storage s1; c4_View raw = s1.GetAs("raw[p1:B]"); c4_View hash = s1.GetAs("hash[_H:I,_R:I]"); c4_View v1 = raw.Hash(hash, 1); v1.Add(p1 [c4_Bytes ("12345", 5)]); A(v1.GetSize() == 1); c4_Bytes buf = p1 (v1[0]); A(buf.Size() == 5); A(buf == c4_Bytes ("12345", 5)); buf = p1(v1[0]).Access(1,3); A(buf == c4_Bytes ("234", 3)); p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); buf = p1 (v1[0]); A(buf == c4_Bytes ("12ab5", 5)); p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); buf = p1 (v1[0]); A(buf == c4_Bytes ("1ABCab5", 7)); p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); buf = p1 (v1[0]); A(buf == c4_Bytes ("1Axyz", 5)); p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); buf = p1 (v1[0]); A(buf == c4_Bytes ("1Axy3456", 8)); } E; Anyone know if I am using the hash view in the wrong way, or is this a bug? Cheers, Steve. P.S. Very, very impressed with Metakit - on only 2 days I have totally rewritten the database engine for one of our products, it's just so neat! From jcw@equi4.com Wed Oct 31 02:07:04 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA00593 for ; Wed, 31 Oct 2001 02:07:03 -0600 Received: from takkie.equi4.local ([195.121.233.32]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM28X200.RZU for ; Wed, 31 Oct 2001 09:00:38 +0100 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9V7wXI15574 for ; Wed, 31 Oct 2001 08:58:33 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Problem with partial access of byte properties in a hashed view Date: Wed, 31 Oct 2001 09:01:46 +0100 Message-Id: <20011031080146.16916@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Steve Baxter wrote: >New to Metakit and not sure whether I am doing something wrong or have found >a bug! You have found a bug. Thanks for the example, which made understanding this one easy. Unfortunately, the problem is a fundamental one which I cannot easily fix (more precisely: I could fix Access, but I can't fix Modify without losing its performance advantage for partial access). > B(b26, Partial memo field access, 0) > { > c4_BytesProp p1 ("p1"); > c4_Storage s1; > c4_View raw = s1.GetAs("raw[p1:B]"); > c4_View hash = s1.GetAs("hash[_H:I,_R:I]"); > c4_View v1 = raw.Hash(hash, 1); > > v1.Add(p1 [c4_Bytes ("12345", 5)]); > A(v1.GetSize() == 1); > c4_Bytes buf = p1 (v1[0]); > A(buf.Size() == 5); > A(buf == c4_Bytes ("12345", 5)); > buf = p1(v1[0]).Access(1,3); > A(buf == c4_Bytes ("234", 3)); > p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); > buf = p1 (v1[0]); > A(buf == c4_Bytes ("12ab5", 5)); > p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1ABCab5", 7)); > p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1Axyz", 5)); > p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1Axy3456", 8)); > } E; The above will work properly, if you replace all Access/Modify calls to operate on the underlying "raw" view instead of "v1". The situation can be summarized as: the new mapped views (hash, ordered, blocked, and indexed) are fine for normal get/set access, but the illusion they present breaks down with Access/Modify for byte properties. In all cases *except* blocked, this will work properly when you use the underlying "raw" view. This is due to the fact that Access/Modify do not insert/delete/move rows, but care must be taken so a key is never modified. Thanks for reporting this. I've added a page on the MetaKit to document this issue: http://www.equi4.com/metakit/wiki.cgi/132.html -jcw From steve@improvision.com Wed Oct 31 04:37:18 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA05170; Wed, 31 Oct 2001 04:37:17 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id KAA07465; Wed, 31 Oct 2001 10:29:32 GMT Received: from [192.168.1.78] ([192.168.1.78]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Wed, 31 Oct 2001 10:31:20 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Wed, 31 Oct 2001 10:31:20 +0000 Subject: Re: [Metakit] Problem with partial access of byte properties in a hashed view From: Stephen Baxter To: Jean-Claude Wippler , Message-ID: In-Reply-To: <20011031080146.16916@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Cool - I'm not going mad after all. One more question - I am going to use the hash search to find the record to update or read, then update or read it using Modify() and Access(). Can I assume that records in raw and v1 correspond? What I mean is can I assume that row 5 in v1 is the same record as row 5 in raw? Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Wed, 31 Oct 2001 09:01:46 +0100 > To: > Subject: Re: [Metakit] Problem with partial access of byte properties in a > hashed view > > Steve Baxter wrote: > >> New to Metakit and not sure whether I am doing something wrong or have found >> a bug! > > You have found a bug. Thanks for the example, which made understanding > this one easy. Unfortunately, the problem is a fundamental one which I > cannot easily fix (more precisely: I could fix Access, but I can't fix > Modify without losing its performance advantage for partial access). > >> B(b26, Partial memo field access, 0) >> { >> c4_BytesProp p1 ("p1"); >> c4_Storage s1; >> c4_View raw = s1.GetAs("raw[p1:B]"); >> c4_View hash = s1.GetAs("hash[_H:I,_R:I]"); >> c4_View v1 = raw.Hash(hash, 1); >> >> v1.Add(p1 [c4_Bytes ("12345", 5)]); >> A(v1.GetSize() == 1); >> c4_Bytes buf = p1 (v1[0]); >> A(buf.Size() == 5); >> A(buf == c4_Bytes ("12345", 5)); >> buf = p1(v1[0]).Access(1,3); >> A(buf == c4_Bytes ("234", 3)); >> p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); >> buf = p1 (v1[0]); >> A(buf == c4_Bytes ("12ab5", 5)); >> p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); >> buf = p1 (v1[0]); >> A(buf == c4_Bytes ("1ABCab5", 7)); >> p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); >> buf = p1 (v1[0]); >> A(buf == c4_Bytes ("1Axyz", 5)); >> p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); >> buf = p1 (v1[0]); >> A(buf == c4_Bytes ("1Axy3456", 8)); >> } E; > > The above will work properly, if you replace all Access/Modify calls to > operate on the underlying "raw" view instead of "v1". > > The situation can be summarized as: the new mapped views (hash, ordered, > blocked, and indexed) are fine for normal get/set access, but the > illusion they present breaks down with Access/Modify for byte properties. > In all cases *except* blocked, this will work properly when you use the > underlying "raw" view. This is due to the fact that Access/Modify do not > insert/delete/move rows, but care must be taken so a key is never modified. > > Thanks for reporting this. I've added a page on the MetaKit to document > this issue: http://www.equi4.com/metakit/wiki.cgi/132.html > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From terabaap@users.sourceforge.net Wed Oct 31 05:14:59 2001 Received: from femail40.sdc1.sfba.home.com (femail40.sdc1.sfba.home.com [24.254.60.34]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA06540 for ; Wed, 31 Oct 2001 05:14:59 -0600 Received: from galadriel ([24.183.20.14]) by femail40.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011031110900.QSBS2143.femail40.sdc1.sfba.home.com@galadriel> for ; Wed, 31 Oct 2001 03:09:00 -0800 Date: Wed, 31 Oct 2001 05:09:35 -0600 From: Yumpee X-Mailer: The Bat! (v1.49) UNREG / CD5BF9353B3B7091 Reply-To: Yumpee X-Priority: 3 (Normal) Message-ID: <55213009958.20011031050935@users.sourceforge.net> To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Mk4py: Internal Python error with simple test program Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I've been playing around with Mk4py and encountered some strange behavior with metakit-2.4.1 built from source on Win98 with MSVC6 and Python-2.0: Consider the following program which creates a view, adds two records and then tries to dump the view: -------------------------------- import metakit db = metakit.storage() v = db.getas('t[f:S]') i1 = v.append(f="FOO") i2 = v.append(f="BAR") metakit.dump(v, "orig") # -- dump is a view pretty-printer v.delete(i1) # -- defined in metakit.py metakit.dump(v,"after delete") -------------------------------- Running this gives me this output and traceback: =================================================== orig f --- FOO BAR --- Total: 2 rows Traceback (most recent call last): File "test.py", line 10, in ? metakit.dump(v,"after delete") File "metakit.py", line 27, in dump if prop.type in ('I','F','D','V'): SystemError: null argument to internal routine ==================================================== I tried putting in some prints in the dump() routine defined in metakit.py and there seems to be nothing wrong with "prop" or "prop.type" but Python barfs when it comes to testing prop.type in that tuple of characters. Now comes the weird part. I put in a print statement in the dump() routine just before the "if prop.type check": ---------------------------------------- print type(prop.type) ---------------------------------------- And now the test program runs just fine!! Usually if this happens in a C program (printf statement preventing core dump), it means that memory is getting corrupted somewhere and the printf is preventing the corruption. Wondering if that is the case with the Mk4py extension also ... Y. From steve@improvision.com Wed Oct 31 06:04:46 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA08546 for ; Wed, 31 Oct 2001 06:04:44 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id LAA22809 for ; Wed, 31 Oct 2001 11:56:58 GMT Received: from [10.0.1.10] ([192.168.1.2]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971 for ; Wed, 31 Oct 2001 11:58:45 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Wed, 31 Oct 2001 11:58:44 +0000 From: Steve Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Another problem with byte properties Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I think I may have found a second problem with storing byte properties. I commit my storage after every "transaction" in my database. I am finding though that changes to byte properties are not correctly committed. I can reproduce this by changing test b26 again to have file-based storage and to commit after each change to the string. After a couple of changes, the test fails. Changes to other properties seem to be stored correctly, it's just the byte properties that are corrupted. Here is the modified test: B(b26, Partial memo field access, 0) { c4_BytesProp p1 ("p1"); c4_Storage s1( "test.dat", true ); c4_View v1 = s1.GetAs("v1[key:I,p1:B]"); v1.Add(p1 [c4_Bytes ("12345", 5)]); A(v1.GetSize() == 1); s1.Commit(); c4_Bytes buf = p1 (v1[0]); A(buf.Size() == 5); A(buf == c4_Bytes ("12345", 5)); buf = p1(v1[0]).Access(1,3); A(buf == c4_Bytes ("234", 3)); p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); s1.Commit(); buf = p1 (v1[0]); A(buf == c4_Bytes ("12ab5", 5)); p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); s1.Commit(); buf = p1 (v1[0]); A(buf == c4_Bytes ("1ABCab5", 7)); p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); s1.Commit(); buf = p1 (v1[0]); A(buf == c4_Bytes ("1Axyz", 5)); p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); s1.Commit(); buf = p1 (v1[0]); A(buf == c4_Bytes ("1Axy3456", 8)); } E; You need to remember to delete test.dat before running the test each time or A(v1.GetSize() == 1) will fail. Again, have I misunderstood the use of Commit() or is this a bug? Cheers, Steve. From jcw@equi4.com Wed Oct 31 06:20:57 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA09400 for ; Wed, 31 Oct 2001 06:20:56 -0600 Received: from takkie.equi4.local ([195.121.233.224]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM2KO700.T3V for ; Wed, 31 Oct 2001 13:14:31 +0100 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9VCCRI17373 for ; Wed, 31 Oct 2001 13:12:27 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Problem with partial access of byte properties in a hashed view Date: Wed, 31 Oct 2001 12:50:35 +0100 Message-Id: <20011031115035.14692@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: [hash] >Can I assume that records in raw and v1 correspond? What I mean is can I >assume that row 5 in v1 is the same record as row 5 in raw? Yes. -jcw From gmcm@hypernet.com Wed Oct 31 07:42:17 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA12676 for ; Wed, 31 Oct 2001 07:42:15 -0600 Received: from me (204.176.40.87) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Wed, 31 Oct 2001 08:30:19 -0500 From: "Gordon McMillan" To: Yumpee , metakit@equi4.com Date: Wed, 31 Oct 2001 08:35:49 -0500 Subject: Re: [Metakit] Mk4py: Internal Python error with simple test program Reply-to: gmcm@hypernet.com Message-ID: <3BDFB7E5.11460.42FBEB2D@localhost> Priority: normal In-reply-to: <55213009958.20011031050935@users.sourceforge.net> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Seems to be an error leftover from the delete that hasn't been cleared. So, for example "v2 = v.sort()" also gets the error, but succeeds on a 2nd try. Need a PyErr_Clear() someplace in delete. > I've been playing around with Mk4py > and encountered some strange behavior > with metakit-2.4.1 built from source > on Win98 with MSVC6 and Python-2.0: > > Consider the following program > which creates a view, adds two > records and then tries to dump > the view: > -------------------------------- > import metakit > > db = metakit.storage() > v = db.getas('t[f:S]') > i1 = v.append(f="FOO") > i2 = v.append(f="BAR") > metakit.dump(v, "orig") # -- dump is a view pretty-printer > v.delete(i1) # -- defined in metakit.py > metakit.dump(v,"after delete") -------------------------------- > > Running this gives me this > output and traceback: > =================================================== > orig > f > --- > FOO > BAR > --- > Total: 2 rows > Traceback (most recent call last): > File "test.py", line 10, in ? > metakit.dump(v,"after delete") > File "metakit.py", line 27, in dump > if prop.type in ('I','F','D','V'): > SystemError: null argument to internal routine > ==================================================== > > I tried putting in some prints in the > dump() routine defined in metakit.py > and there seems to be nothing wrong > with "prop" or "prop.type" but Python > barfs when it comes to testing prop.type > in that tuple of characters. > > Now comes the weird part. I put in > a print statement in the dump() > routine just before the "if prop.type > check": > ---------------------------------------- > print type(prop.type) > ---------------------------------------- > > And now the test program runs just fine!! > > Usually if this happens in a C program > (printf statement preventing core dump), > it means that memory is getting corrupted > somewhere and the printf is preventing > the corruption. Wondering if that is > the case with the Mk4py extension also ... > > Y. > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit - Gordon From jcw@equi4.com Wed Oct 31 17:32:11 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA07733 for ; Wed, 31 Oct 2001 17:32:11 -0600 Received: from takkie.equi4.local ([195.121.235.139]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM3FRL00.CEB for ; Thu, 1 Nov 2001 00:26:09 +0100 Received: from [10.0.1.6] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id f9VNO8I21565 for ; Thu, 1 Nov 2001 00:24:09 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Mk4py: Internal Python error with simple test program Date: Thu, 1 Nov 2001 00:27:29 +0100 Message-Id: <20011031232729.15988@mail.planet.nl> In-Reply-To: <3BDFB7E5.11460.42FBEB2D@localhost> References: <3BDFB7E5.11460.42FBEB2D@localhost> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gordon McMillan wrote: >Seems to be an error leftover from the delete that hasn't been >cleared. So, for example "v2 = v.sort()" also gets the error, but >succeeds on a 2nd try. Need a PyErr_Clear() someplace in delete. Bingo, but a pretty tricky little bugger, this one. The solution: diff -r1.4 PyView.cpp 86c86 < PWOSequence seq; --- > PWOTuple seq; 903c903 < PWOSequence seq; --- > PWOTuple seq; [...] >> v.delete(i1) # -- defined in metakit.py >> metakit.dump(v,"after delete") -------------------------------- [...] >> Traceback (most recent call last): >> File "test.py", line 10, in ? >> metakit.dump(v,"after delete") >> File "metakit.py", line 27, in dump >> if prop.type in ('I','F','D','V'): >> SystemError: null argument to internal routine I've updated CVS, and http://www.equi4.com/metakit/CHANGES Thanks, Yumpee , for reporting this. -jcw From jcw@equi4.com Thu Nov 1 04:25:36 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA20677; Thu, 1 Nov 2001 04:25:35 -0600 Received: from takkie.equi4.local ([195.121.234.119]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM49ZV00.NCX; Thu, 1 Nov 2001 11:19:07 +0100 Received: from [10.0.1.7] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA1AH0I23933; Thu, 1 Nov 2001 11:17:10 +0100 From: Jean-Claude Wippler To: announce , metakit Date: Thu, 1 Nov 2001 11:20:16 +0100 Message-Id: <20011101102016.27063@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MkSQL Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a brief announcement to draw your attention to a (very slick, IMO) Python project called "MkSQL" which implements an amazingly complete SQL engine on top of the MetaKit database. Full details can be found here: http://www.mcmillan-inc.com/mksqlintro.html This second incarnation of MkSQL includes a Python DBI v2 interface, and adds support for referential integrity constraints. Like MetaKit itself, MkSQL is not multi-user, but it could no doubt be turned into such a solution by setting it up as a server (and addressing the issues of concurrent commits and isolation, if multi-user modification is required). MkSQL is the brainchild of Gordon McMillan , who also created the Mk4py binding which makes the C++ Metakit core fit so well to the Python world (see http://www.equi4.com/metakit/ for further info). Regards, Jean-Claude From steve@improvision.com Thu Nov 1 12:13:44 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA06250 for ; Thu, 1 Nov 2001 12:13:44 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA21573 for ; Thu, 1 Nov 2001 18:05:55 GMT Received: from [192.168.1.78] ([192.168.1.78]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Thu, 01 Nov 2001 18:07:43 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Thu, 01 Nov 2001 18:07:43 +0000 Subject: Re: [Metakit] Another problem with byte properties From: Stephen Baxter To: Steve Baxter , Message-ID: In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm still completely stuck on this. If I leave my commit until the file is closed, I get no problems. If I commit at any other time, after a few commits I get corrupted data written to the file. Are there limitations on when c4_Storage::Commit() can be called? Cheers, Steve. > From: Steve Baxter > Date: Wed, 31 Oct 2001 11:58:44 +0000 > To: > Subject: [Metakit] Another problem with byte properties > > I think I may have found a second problem with storing byte properties. > > I commit my storage after every "transaction" in my database. I am finding > though that changes to byte properties are not correctly committed. > > I can reproduce this by changing test b26 again to have file-based storage > and to commit after each change to the string. After a couple of changes, > the test fails. Changes to other properties seem to be stored correctly, > it's just the byte properties that are corrupted. > > Here is the modified test: > > B(b26, Partial memo field access, 0) > { > c4_BytesProp p1 ("p1"); > c4_Storage s1( "test.dat", true ); > c4_View v1 = s1.GetAs("v1[key:I,p1:B]"); > > v1.Add(p1 [c4_Bytes ("12345", 5)]); > A(v1.GetSize() == 1); > s1.Commit(); > > c4_Bytes buf = p1 (v1[0]); > A(buf.Size() == 5); > A(buf == c4_Bytes ("12345", 5)); > buf = p1(v1[0]).Access(1,3); > A(buf == c4_Bytes ("234", 3)); > p1 (v1[0]).Modify(c4_Bytes ("ab", 2), 2, 0); > s1.Commit(); > > buf = p1 (v1[0]); > A(buf == c4_Bytes ("12ab5", 5)); > p1 (v1[0]).Modify(c4_Bytes ("ABC", 3), 1, 2); > s1.Commit(); > > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1ABCab5", 7)); > p1 (v1[0]).Modify(c4_Bytes ("xyz", 3), 2, -2); > s1.Commit(); > > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1Axyz", 5)); > p1 (v1[0]).Modify(c4_Bytes ("3456", 4), 4, 0); > s1.Commit(); > > buf = p1 (v1[0]); > A(buf == c4_Bytes ("1Axy3456", 8)); > } E; > > You need to remember to delete test.dat before running the test each time or > A(v1.GetSize() == 1) will fail. > > Again, have I misunderstood the use of Commit() or is this a bug? > > Cheers, > > Steve. > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Fri Nov 2 09:45:23 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA29844 for ; Fri, 2 Nov 2001 09:45:22 -0600 Received: from takkie.equi4.local ([195.121.233.129]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM6JGP01.4RY for ; Fri, 2 Nov 2001 16:38:49 +0100 Received: from [10.0.1.8] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA2Fb3I01713 for ; Fri, 2 Nov 2001 16:37:03 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Another problem with byte properties Date: Fri, 2 Nov 2001 16:38:53 +0100 Message-Id: <20011102153853.23501@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >I'm still completely stuck on this. If I leave my commit until the file is >closed, I get no problems. If I commit at any other time, after a few >commits I get corrupted data written to the file. > >Are there limitations on when c4_Storage::Commit() can be called? No. This was a bug in the new 2.3/2.4 storage format code, which only shows when you change a small item using partial access, i.e. Modify. The commit works, but it leaves a bad pointer so subsequent use in the same program will dereference a stale pointer. I've commited the one-line fix to CVS. Thanks for your patience, Steve! -jcw From nathan.thompson-amato@dnamerican.com Fri Nov 2 14:49:22 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA16063 for ; Fri, 2 Nov 2001 14:49:22 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ca054758 for ; Fri, 2 Nov 2001 15:54:35 -0500 Message-ID: <3BE3054D.4000201@dnamerican.com> Date: Fri, 02 Nov 2001 15:42:53 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit References: <20011030064901.1028@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Statically linking libmk4.a fails; library is stripped Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, I can't seem to statically link against Metakit (either 2.4.1 or the CVS version). Apparently, libmk4.a is being stripped by the 'make install' rule, and the resulting archive exports no symbols (according to 'nm'). When I statically link against the unstripped library, the link succeeds and my program runs. Perhaps the 'make install' rule needs changing? Or am I missing something? Thanks, Nathan From jcw@equi4.com Fri Nov 2 15:26:32 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA18366 for ; Fri, 2 Nov 2001 15:26:31 -0600 Received: from takkie.equi4.local ([195.121.233.211]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GM6ZA000.7FY for ; Fri, 2 Nov 2001 22:20:24 +0100 Received: from [10.0.1.8] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA2LIdI04295 for ; Fri, 2 Nov 2001 22:18:39 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Statically linking libmk4.a fails; library is stripped Date: Fri, 2 Nov 2001 22:21:03 +0100 Message-Id: <20011102212103.2499@mail.planet.nl> In-Reply-To: <3BE3054D.4000201@dnamerican.com> References: <3BE3054D.4000201@dnamerican.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nathan Thompson-Amato wrote: >Perhaps the 'make install' rule needs changing? Or am I missing something? Oops. Fixed. In CVS. Blush. Thanks. -jcw From terabaap@users.sourceforge.net Fri Nov 2 21:03:15 2001 Received: from femail24.sdc1.sfba.home.com (femail24.sdc1.sfba.home.com [24.0.95.149]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA30404 for ; Fri, 2 Nov 2001 21:03:15 -0600 Received: from galadriel ([24.183.20.14]) by femail24.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011103025704.XOAV18561.femail24.sdc1.sfba.home.com@galadriel> for ; Fri, 2 Nov 2001 18:57:04 -0800 Date: Fri, 2 Nov 2001 20:57:06 -0600 From: Manoj Plakal X-Mailer: The Bat! (v1.49) UNREG / CD5BF9353B3B7091 Reply-To: Manoj Plakal X-Priority: 3 (Normal) Message-ID: <129442696434.20011102205706@users.sourceforge.net> To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Derived join views don't stay updated? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is there any documentation on when derived views stay updated and when they don't? I was testing deriving views by joining two views and it looked like the derived view wouldn't get updated if the join column values were changed in the underlying views. Sample Python code below. Manoj ---------------------------------------------- import metakit db = metakit.storage() v = db.getas('t[f:S,f2:S]') v.append(f="FOO",f2="U2") v.append(f="BAR",f2="Prodigy") v.append(f="BAZ",f2="NIN") metakit.dump(v, "orig") v2 = db.getas('t2[f:S]') v2.append(f="BAZ") v2.append(f="FOO") metakit.dump(v2, "to join") v3 = v.join(v2, metakit.property( 'S', 'f' ) ) metakit.dump(v3, "joined" ) v[0].f2="Dylan" # -- Changes U2 to Dylan in v3 metakit.dump(v3, "updated join" ) v2[1].f="BAR" # -- Does not change v3, shouldn't it? metakit.dump(v3, "updated join" ) ---------------------------------------------- From steve@improvision.com Mon Nov 5 03:44:20 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA01736; Mon, 5 Nov 2001 03:44:19 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id JAA02076; Mon, 5 Nov 2001 09:36:15 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 05 Nov 2001 09:38:01 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 09:38:06 +0000 Subject: Re: [Metakit] Another problem with byte properties From: Stephen Baxter To: Jean-Claude Wippler , Message-ID: In-Reply-To: <20011102153853.23501@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: That fixes the problem - thanks once again! Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Fri, 2 Nov 2001 16:38:53 +0100 > To: > Subject: Re: [Metakit] Another problem with byte properties > > Stephen Baxter wrote: > >> I'm still completely stuck on this. If I leave my commit until the file is >> closed, I get no problems. If I commit at any other time, after a few >> commits I get corrupted data written to the file. >> >> Are there limitations on when c4_Storage::Commit() can be called? > > No. > > This was a bug in the new 2.3/2.4 storage format code, which only shows > when you change a small item using partial access, i.e. Modify. The > commit works, but it leaves a bad pointer so subsequent use in the same > program will dereference a stale pointer. > > I've commited the one-line fix to CVS. Thanks for your patience, Steve! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From steve@improvision.com Mon Nov 5 06:24:16 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA08286 for ; Mon, 5 Nov 2001 06:24:15 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id MAA19826 for ; Mon, 5 Nov 2001 12:16:09 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971 for ; Mon, 05 Nov 2001 12:18:02 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 12:18:01 +0000 From: Stephen Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] c4_BytesRef::Access() reads all data Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Another problem with accessing binary data I'm afraid. I'm using a Mac (therefore no memory mapped files). I'm finding that if I read from a byte property using Access(), *all* the data in the byte property is read. Since my byte properties can each be many MB in size this takes forever! It is critical to my application that I can read partial streams. The problem seems to be that c4_Column::SetupSegments() runs through and reads the entire set of segments even if only one segment is needed. I would guess that this problem does not occur with memory-mapped files as the data is not actually read. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ From jcw@equi4.com Mon Nov 5 08:37:07 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA14868 for ; Mon, 5 Nov 2001 08:37:07 -0600 Received: from takkie.equi4.local ([195.121.234.204]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMC0AL01.D1M for ; Mon, 5 Nov 2001 15:30:21 +0100 Received: from [10.0.1.12] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA5ESmI21870 for ; Mon, 5 Nov 2001 15:28:58 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] c4_BytesRef::Access() reads all data Date: Mon, 5 Nov 2001 15:07:02 +0100 Message-Id: <20011105140702.4152@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >I'm using a Mac (therefore no memory mapped files). I'm finding that if I >read from a byte property using Access(), *all* the data in the byte >property is read. Since my byte properties can each be many MB in size this >takes forever! It is critical to my application that I can read partial >streams. > >The problem seems to be that c4_Column::SetupSegments() runs through and >reads the entire set of segments even if only one segment is needed. I >would guess that this problem does not occur with memory-mapped files as the >data is not actually read. Correct. In Mac OS 9., support was added for memory mapped files. Unfortunately, the docs say that mapping may only be used if the file is not open for any other use (if I remember correctly). That rules out the current mechanism MK uses: a R/O mapping, with R/W file access. Commits write to file (accessing the R/O map when copying parts which have not changed, but never writing to parts which are accessed through the R/O map so there are never problems with VM page coherence, no matter how limited mmap support is). Technically speaking it *could* be fixed for this case - by copying necessary data from the mapping, then unmapping, then opening the file for write, then writing from in-mem data only, then closing, then re- mapping R/O. But that would be a limited solution - for anything < OS 9, this would still not work. On the Mac, MK has always had high memory use (up to having all data in mem). On Mac OS X, normal memory-mapped file mechanism work just fine, btw. Better still, on Mac OS X 10.1, I'm told MK is included - used by AddressServices :) I'm afraid that there is no quick way out. You could slice up your data into smaller chunks (and perhaps use blocked views to slice up the resulting columns, though I'd try simple chunking first). Or store this particular part of your data in a separate datafile and have MK *manage* such "blobs" instead of actually storing them. I.e. store offset/length values in MK, and use a seek/read for actual access (it'll take more work for writing, i.e. managing free space, but it's definitely feasible). -jcw From steve@improvision.com Mon Nov 5 08:58:02 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA16081; Mon, 5 Nov 2001 08:58:02 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id OAA24068; Mon, 5 Nov 2001 14:49:56 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 05 Nov 2001 14:51:48 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 14:51:47 +0000 Subject: Re: [Metakit] c4_BytesRef::Access() reads all data From: Stephen Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011105140702.4152@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hmm, nasty - I see what you mean. I did just try implementing lazy reading within Metakit. Basically, I changed c4_Column so that the _segments array can be filled sparsely. When it actually needs the data it goes to fetch it from disk. The problem here seems to be that it slows down non-bytes columns. Do you think there is any use in me pursuing this? I have actually already implemented the "metakit-managing-many-files" approach - we have to do this because our databases can be 10s of GB in size (lots of data though not necessarily lots of records). I would also like a "single file" solution for users who have smaller data requirements, but if this is not possible then so be it. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Mon, 5 Nov 2001 15:07:02 +0100 > To: metakit > Subject: Re: [Metakit] c4_BytesRef::Access() reads all data > > Stephen Baxter wrote: > >> I'm using a Mac (therefore no memory mapped files). I'm finding that if I >> read from a byte property using Access(), *all* the data in the byte >> property is read. Since my byte properties can each be many MB in size this >> takes forever! It is critical to my application that I can read partial >> streams. >> >> The problem seems to be that c4_Column::SetupSegments() runs through and >> reads the entire set of segments even if only one segment is needed. I >> would guess that this problem does not occur with memory-mapped files as the >> data is not actually read. > > Correct. > > In Mac OS 9., support was added for memory mapped files. > > Unfortunately, the docs say that mapping may only be used if the file is > not open for any other use (if I remember correctly). That rules out the > current mechanism MK uses: a R/O mapping, with R/W file access. Commits > write to file (accessing the R/O map when copying parts which have not > changed, but never writing to parts which are accessed through the R/O > map so there are never problems with VM page coherence, no matter how > limited mmap support is). > > Technically speaking it *could* be fixed for this case - by copying > necessary data from the mapping, then unmapping, then opening the file > for write, then writing from in-mem data only, then closing, then re- > mapping R/O. But that would be a limited solution - for anything < OS 9, > this would still not work. > > On the Mac, MK has always had high memory use (up to having all data in > mem). On Mac OS X, normal memory-mapped file mechanism work just fine, > btw. Better still, on Mac OS X 10.1, I'm told MK is included - used by > AddressServices :) > > I'm afraid that there is no quick way out. You could slice up your data > into smaller chunks (and perhaps use blocked views to slice up the > resulting columns, though I'd try simple chunking first). Or store this > particular part of your data in a separate datafile and have MK *manage* > such "blobs" instead of actually storing them. I.e. store offset/length > values in MK, and use a seek/read for actual access (it'll take more work > for writing, i.e. managing free space, but it's definitely feasible). > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From cherry314159@yahoo.com Mon Nov 5 10:35:26 2001 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA22320 for ; Mon, 5 Nov 2001 10:35:26 -0600 Received: from pool-138-89-71-221.mad.east.verizon.net (HELO localhost.localdomain) (138.89.71.221) by smtp.mail.vip.sc5.yahoo.com with SMTP; 5 Nov 2001 16:29:12 -0000 X-Apparently-From: From: Jose Vasconcellos To: metakit@equi4.com Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/0.15 (Preview Release) Date: 05 Nov 2001 11:20:24 -0500 Message-Id: <1004977225.23523.15.camel@localhost.localdomain> Mime-Version: 1.0 Subject: [Metakit] Catfish file format Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I've written a Python version of catfish so I could browse the database from Linux. It's a bit alpha at the moment but it does work. I've also added an option to create or update a database. My problem is that I'm not clear on the definition of the 'info' view. Can anyone describe the 8 items that make up this view? Jose Vasconcellos _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From nathan.thompson-amato@dnamerican.com Mon Nov 5 14:12:10 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA04230 for ; Mon, 5 Nov 2001 14:11:55 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ea054838 for ; Mon, 5 Nov 2001 15:17:02 -0500 Message-ID: <3BE6F0FA.90700@dnamerican.com> Date: Mon, 05 Nov 2001 15:05:14 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: multipart/mixed; boundary="------------060102060208090303040307" Subject: [Metakit] Forgotten change after Find() on hash view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------060102060208090303040307 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit I'm having a bit of trouble updating existing rows when using a hash table view: after updating a row and doing a Find() on the hash view, the new values for the row are forgotten. Perhaps somebody can tell me where I'm going wrong... I've attached a code snippet to illustrate. Thanks, Nathan --------------060102060208090303040307 Content-Type: text/plain; name="mktest.cc" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="mktest.cc" #include #include int main(void) { c4_StringProp name ("name"); c4_IntProp age ("age"); c4_Storage storage ("data.dat", 1); c4_View view = storage.GetAs("data[name:S,age:I]"); // Create a mapping view for fast lookups by name c4_View view_h = storage.GetAs("data_H1[_H:I,_R:I]"); // Build the hash table c4_View hash_view = view.Hash(view_h, 1); c4_Row row; // Add Bob name (row) = "Bob"; age (row) = 20; hash_view.Add(row); // Look for Bob int index = hash_view.Find(name ["Bob"]); cout << "Bob is at index " << index << endl; row = hash_view[index]; cout << "Info for Bob: " << (const char *)(name(row)) << ";" << (t4_i32)(age(row)) << endl; // Change Bob age(row) = 22; cout << "Changed info for Bob: " << (const char *)(name(row)) << ";" << (t4_i32)(age(row)) << endl; // Find Bob again index = hash_view.Find(name ["Bob"]); cout << "Bob is at index " << index << endl; row = hash_view[index]; // Bob's age is 20 again! cout << "Changed info for Bob after a new Find(): " << (const char *)(name(row)) << ";" << (t4_i32)(age(row)) << endl; return 0; } --------------060102060208090303040307-- From steve@improvision.com Mon Nov 5 15:29:59 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA09487 for ; Mon, 5 Nov 2001 15:29:58 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id VAA21580 for ; Mon, 5 Nov 2001 21:21:52 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 05 Nov 2001 21:23:43 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 21:23:42 +0000 Subject: Re: [Metakit] Forgotten change after Find() on hash view From: Stephen Baxter To: Nathan Thompson-Amato , Message-ID: In-Reply-To: <3BE6F0FA.90700@dnamerican.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is it because a c4_Row is being used when you really wanted a c4_RowRef? I would guess that the row is a copy of the data from the view, not the real data. When you update the row you are updating the copy, not the real row. Try using a c4_Row to add the data, but a c4_RowRef when you get it in; row = hash_view[index]; Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Nathan Thompson-Amato > Organization: D.N. American > Date: Mon, 05 Nov 2001 15:05:14 -0500 > To: metakit@equi4.com > Subject: [Metakit] Forgotten change after Find() on hash view > > I'm having a bit of trouble updating existing rows when using a hash table > view: after updating a row and doing a Find() on the hash view, the new > values for the row are forgotten. Perhaps somebody can tell me where I'm > going wrong... > > I've attached a code snippet to illustrate. > > Thanks, > > Nathan > > > #include > #include > > int main(void) > { > c4_StringProp name ("name"); > c4_IntProp age ("age"); > > c4_Storage storage ("data.dat", 1); > > c4_View view = storage.GetAs("data[name:S,age:I]"); > > // Create a mapping view for fast lookups by name > c4_View view_h = storage.GetAs("data_H1[_H:I,_R:I]"); > > // Build the hash table > c4_View hash_view = view.Hash(view_h, 1); > > c4_Row row; > > // Add Bob > name (row) = "Bob"; > age (row) = 20; > hash_view.Add(row); > > // Look for Bob > int index = hash_view.Find(name ["Bob"]); > > cout << "Bob is at index " << index << endl; > > row = hash_view[index]; > > cout << "Info for Bob: " > << (const char *)(name(row)) << ";" > << (t4_i32)(age(row)) << endl; > > // Change Bob > age(row) = 22; > > cout << "Changed info for Bob: " > << (const char *)(name(row)) << ";" > << (t4_i32)(age(row)) << endl; > > // Find Bob again > index = hash_view.Find(name ["Bob"]); > > cout << "Bob is at index " << index << endl; > > row = hash_view[index]; > > // Bob's age is 20 again! > cout << "Changed info for Bob after a new Find(): " > << (const char *)(name(row)) << ";" > << (t4_i32)(age(row)) << endl; > > return 0; > } > From nathan.thompson-amato@dnamerican.com Mon Nov 5 16:17:06 2001 Received: from montecarlo.DNAMERICAN.COM ([206.156.169.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA13020 for ; Mon, 5 Nov 2001 16:16:55 -0600 Received: from [206.156.169.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ta054853 for ; Mon, 5 Nov 2001 17:21:56 -0500 Message-ID: <3BE70E3F.3050307@dnamerican.com> Date: Mon, 05 Nov 2001 17:10:07 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.5) Gecko/20011012 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Forgotten change after Find() on hash view References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: > Is it because a c4_Row is being used when you really wanted a c4_RowRef? I > would guess that the row is a copy of the data from the view, not the real > data. When you update the row you are updating the copy, not the real row. > > Try using a c4_Row to add the data, but a c4_RowRef when you get it in; > > row = hash_view[index]; > Good call! At least at first glance, this seems to work perfectly. I'd better stare at the docs a bit and make sure I'm understanding just what Refs are supposed to be for. Many thanks, Nathan From steve@improvision.com Mon Nov 5 16:28:52 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA14008 for ; Mon, 5 Nov 2001 16:28:52 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id WAA23334 for ; Mon, 5 Nov 2001 22:20:45 GMT Received: from [192.168.1.87] ([192.168.1.87]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 05 Nov 2001 22:22:37 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 05 Nov 2001 22:22:34 +0000 Subject: Re: [Metakit] Forgotten change after Find() on hash view From: Stephen Baxter To: Nathan Thompson-Amato , Message-ID: In-Reply-To: <3BE70E3F.3050307@dnamerican.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: If I've got this right (and I've only been using Metakit for a week), a c4_Row is an independent view with one row, whereas a c4_RowRef is like a cursor within a view. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Nathan Thompson-Amato > Organization: D.N. American > Date: Mon, 05 Nov 2001 17:10:07 -0500 > To: metakit@equi4.com > Subject: Re: [Metakit] Forgotten change after Find() on hash view > > Stephen Baxter wrote: > >> Is it because a c4_Row is being used when you really wanted a c4_RowRef? I >> would guess that the row is a copy of the data from the view, not the real >> data. When you update the row you are updating the copy, not the real row. >> >> Try using a c4_Row to add the data, but a c4_RowRef when you get it in; >> >> row = hash_view[index]; >> > > > Good call! At least at first glance, this seems to work perfectly. I'd > better stare at the docs a bit and make sure I'm understanding just what > Refs are supposed to be for. > > Many thanks, > > Nathan > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Mon Nov 5 17:36:38 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA19371 for ; Mon, 5 Nov 2001 17:36:38 -0600 Received: from takkie.equi4.local ([195.121.236.56]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMCPAE01.52A; Tue, 6 Nov 2001 00:30:14 +0100 Received: from [10.0.1.12] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA5NSoI25925; Tue, 6 Nov 2001 00:28:51 +0100 From: Jean-Claude Wippler To: Cc: Subject: Re: [Metakit] Forgotten change after Find() on hash view Date: Tue, 6 Nov 2001 00:31:09 +0100 Message-Id: <20011105233109.25893@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >If I've got this right (and I've only been using Metakit for a week), a >c4_Row is an independent view with one row, whereas a c4_RowRef is like a >cursor within a view. Correct. In C++ terms, it's like "int a = b" versus "int& a = b". I just checked in a small change to allow "const c4_RowRef&" as arg in two places where a "c4_Cursor" was being used (while leaving the old API intact). It looks like the c4_Cursor is not really needed, one can do just about the same with a "const c4_RowRef&". It may be a bit confusing though: that's a reference to a C++ object which is represents a *reference* to a MK row... Longer term, I would like to further simplify the API. There's not terribly much left to simplify, but while trying to use "SWIG" to wrap MK, it became apparent that a number of classes can be left out of the API without limiting the functionality. Speaking of SWIG: I've done a very brief test with 1.3.9, and it *looks* like SWIG might now be up to the task of wrapping MK. There's still the issue of operator overloading (most of these are already redundant), and of function overloading (which is also not excessively frequent in the MK API). Well... a bit of heavy hacking led to (ta-dah!): >>> a basic test working in Java, Perl, Ruby, Python, and Tcl Just proof of concept for now, but if anyone wants to have a go at this, you can unpack the following in "/swig/", and try it (Makefile is for Linux): http://www.equi4.com/previews/test1.tar.gz I'm not sure if there is a SWIG interface for PHP. There is one for Lua, IIRC. The task of binding MK to new languages will take more effort than such a quick trial, and requires people with experience in those languages IMO to come up with bindings which work as naturally as the Tcl and Python ones do. Please let me know if you can help with any of this, or know others who might be interested. -jcw From beazley@cs.uchicago.edu Mon Nov 5 17:52:48 2001 Received: from laime.cs.uchicago.edu (laime.cs.uchicago.edu [128.135.11.244]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA20538; Mon, 5 Nov 2001 17:52:48 -0600 Received: from gargoyle.cs.uchicago.edu (gargoyle.cs.uchicago.edu [128.135.11.238]) by laime.cs.uchicago.edu (8.10.2/8.9.3) with ESMTP id fA5NkVs07111; Mon, 5 Nov 2001 17:46:32 -0600 (CST) Received: by gargoyle.cs.uchicago.edu (Postfix, from userid 32587) id 94F1731D55; Mon, 5 Nov 2001 17:46:30 -0600 (CST) From: David Beazley MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15335.9430.350668.215048@gargoyle.cs.uchicago.edu> Date: Mon, 5 Nov 2001 17:46:30 -0600 (CST) To: Jean-Claude Wippler Cc: , Subject: Re: [Metakit] Forgotten change after Find() on hash view In-Reply-To: <20011105233109.25893@mail.planet.nl> References: <20011105233109.25893@mail.planet.nl> X-Mailer: VM 6.71 under 21.1 "20 Minutes to Nikko" XEmacs Lucid (patch 2) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler writes: > > Speaking of SWIG: I've done a very brief test with 1.3.9, and it *looks* > like SWIG might now be up to the task of wrapping MK. There's still the > issue of operator overloading (most of these are already redundant), and > of function overloading (which is also not excessively frequent in the MK > API). Well... a bit of heavy hacking led to (ta-dah!): FYI, SWIG-1.3.10 (in development) allows overloaded operators to be wrapped (with the understanding that certain operators might not map cleanly onto the target language). If there are other C++ features that need to be addressed, I'd certainly be interested in that. Cheers, Dave From jcw@equi4.com Mon Nov 5 19:17:14 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA26122 for ; Mon, 5 Nov 2001 19:17:08 -0600 Received: from takkie.equi4.local ([195.121.234.186]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMCTXZ01.M72 for ; Tue, 6 Nov 2001 02:10:47 +0100 Received: from [10.0.1.12] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA619II26951 for ; Tue, 6 Nov 2001 02:09:28 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Catfish file format Date: Tue, 6 Nov 2001 02:11:24 +0100 Message-Id: <20011106011124.9829@mail.planet.nl> In-Reply-To: <1004977225.23523.15.camel@localhost.localdomain> References: <1004977225.23523.15.camel@localhost.localdomain> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jose Vasconcellos wrote: >I've written a Python version of catfish so I could browse >the database from Linux. It's a bit alpha at the moment but >it does work. I've also added an option to create or update >a database. My problem is that I'm not clear on the definition >of the 'info' view. Can anyone describe the 8 items that >make up this view? The source code is in the MK source distribution (in win/catfish/), does that help? It's been a while, I'd need to dig up the details. Essentially, these fields store some info about the disk (cluster size, capacity, etc), so that CF can report usage, type of medium, and such things. Not everything is used in the current CF utility, btw. -jcw From jcw@equi4.com Tue Nov 6 04:03:27 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA16083 for ; Tue, 6 Nov 2001 04:03:20 -0600 Received: from takkie.equi4.local ([195.121.234.96]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMDIA200.W0R; Tue, 6 Nov 2001 10:56:26 +0100 Received: from [10.0.1.12] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA69sxI28852; Tue, 6 Nov 2001 10:55:10 +0100 From: Jean-Claude Wippler To: metakit Cc: David Beazley Date: Tue, 6 Nov 2001 10:57:42 +0100 Message-Id: <20011106095742.6558@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Correction, SWIG Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: To follow up on the last post... SWIG 1.3.9 is definitely *much* closer to getting MK wrapped for use in Java, Perl, and Ruby. Contrary to my previous comments, which were partly based on the version that comes standard with SuSE Linux 7.1 (1.3a5, IIRC), very few changes are need to get a basic test working. I've made a diff showing the changes at http://www.equi4.com/previews/ test2.diff - in summary: - anonymous union (private, commented out) - operator casting to a type ("operator int()" etc) - "long long" cause compiles to choke (at least the Java wrap) - a few other changes to exclude some MK defs from wrapping Operators are properly skipped, default args appear to be ok, overloaded member appear to be skipped ok, and friends no longer cause errors. This second test has regressed w.r.t. the Ruby wrapper, which no longer compiles, I've not checked this further for now. Finally, two fixes had to be made (they were also in test1): - I'm inserting a call to Tcl_InitStubs in the Tcl wrapper, so that this thing compiles with stub support (-DUSE_TCL_STUBS), for things like binary Tcl version independence. The sed script applied to the generated C++ wrapper is: /interp == 0/i \ if (Tcl_InitStubs(interp, "8.1", 0) == NULL) return TCL_ERROR; - In Ruby 1.6.5, VALUE() needs to have the "..." elipsis removed, or it won't compile. The sed script for this is the highly sophisticated (2 changes): /\.\.\./s///g -jcw From terabaap@yumpee.org Tue Nov 6 04:53:39 2001 Received: from femail23.sdc1.sfba.home.com (femail23.sdc1.sfba.home.com [24.0.95.148]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA18732 for ; Tue, 6 Nov 2001 04:53:38 -0600 Received: from yumpee.org ([24.183.20.14]) by femail23.sdc1.sfba.home.com (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20011106104714.VFSW23889.femail23.sdc1.sfba.home.com@yumpee.org> for ; Tue, 6 Nov 2001 02:47:14 -0800 Message-ID: <3BE7BFEC.2080800@yumpee.org> Date: Tue, 06 Nov 2001 04:48:12 -0600 From: Manoj Plakal User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.5) Gecko/20011011 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Derived join views don't stay updated\ References: <675614530.20011106044250@users.sourceforge.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Any comments on whether this is a bug or expected behavior? Manoj Manoj Plakal wrote: > Is there any documentation on when derived views > stay updated and when they don't? > > I was testing deriving views by joining two views > and it looked like the derived view wouldn't > get updated if the join column values were changed > in the underlying views. > > Sample Python code below. > > Manoj > > ---------------------------------------------- > import metakit > > db = metakit.storage() > > v = db.getas('t[f:S,f2:S]') > v.append(f="FOO",f2="U2") > v.append(f="BAR",f2="Prodigy") > v.append(f="BAZ",f2="NIN") > metakit.dump(v, "orig") > > v2 = db.getas('t2[f:S]') > v2.append(f="BAZ") > v2.append(f="FOO") > metakit.dump(v2, "to join") > > v3 = v.join(v2, metakit.property( 'S', 'f' ) ) > metakit.dump(v3, "joined" ) > > v[0].f2="Dylan" # -- Changes U2 to Dylan in v3 > metakit.dump(v3, "updated join" ) > > v2[1].f="BAR" # -- Does not change v3, shouldn't it? > metakit.dump(v3, "updated join" ) > ---------------------------------------------- From mlh@idi.ntnu.no Tue Nov 6 05:36:03 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA21531 for ; Tue, 6 Nov 2001 05:36:02 -0600 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id MAA25192 for ; Tue, 6 Nov 2001 12:29:44 +0100 (MET) Message-ID: <01f301c166b6$556c5a90$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: References: <20011105233109.25893@mail.planet.nl> Subject: Re: [Metakit] Forgotten change after Find() on hash view Date: Tue, 6 Nov 2001 12:29:44 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Jean-Claude Wippler" [...] > Speaking of SWIG: I've done a very brief test with 1.3.9, and it *looks* > like SWIG might now be up to the task of wrapping MK. There's still the > issue of operator overloading (most of these are already redundant), and > of function overloading (which is also not excessively frequent in the MK > API). Well... a bit of heavy hacking led to (ta-dah!): > > >>> a basic test working in Java, Perl, Ruby, Python, and Tcl Wow! That's great :) > I'm not sure if there is a SWIG interface for PHP. It doesn't seem like it (although I can't get up swig.org right now). Pity. > There is one for Lua, IIRC. Another nice language... :) > -jcw -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.sf.net From jcw@equi4.com Wed Nov 7 17:07:12 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA02973 for ; Wed, 7 Nov 2001 17:07:12 -0600 Received: from takkie.equi4.local ([195.121.238.84]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMGD8H01.4OM; Thu, 8 Nov 2001 00:00:17 +0100 Received: from [10.0.1.13] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA7MxDI07896; Wed, 7 Nov 2001 23:59:13 +0100 From: Jean-Claude Wippler To: metakit Cc: Date: Thu, 8 Nov 2001 00:00:52 +0100 Message-Id: <20011107230052.3585@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] XML Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: If anyone is interested, here's a new trial to load XML data into MetaKit: http://www.equi4.com/previews/xml2.README http://www.equi4.com/previews/xml2.tar.gz The "mk4xml" tool in there is not optimized, and should not be taken as an indication of how efficient/inefficient things can be made. My first goal is to get more experience with having such data in MK and finding ways to effectively deal with it. Things like DOM would be nice (but take a much larger effort). Transferring data XML <-> MK <-> XML is a secondary issue for now, and IMO feasible efficiently in several ways. -jcw From baspey@yahoo.com Wed Nov 7 18:14:46 2001 Received: from web11505.mail.yahoo.com (web11505.mail.yahoo.com [216.136.172.37]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA06341 for ; Wed, 7 Nov 2001 18:14:45 -0600 Message-ID: <20011108000823.25007.qmail@web11505.mail.yahoo.com> Received: from [212.27.49.251] by web11505.mail.yahoo.com via HTTP; Wed, 07 Nov 2001 16:08:23 PST Date: Wed, 7 Nov 2001 16:08:23 -0800 (PST) From: popeye sailorman To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1502978244-1005178103=:23354" Subject: [Metakit] 3 bytes memory leak when creating c4_Storage on the heap Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1502978244-1005178103=:23354 Content-Type: text/plain; charset=us-ascii Hi, the following code is a simplification of what JCW wrote on the mailing list subject "Using a storage as member in an object": c4_Storage * MyStorage = new c4_Storage ("test.dat", true); delete MyStorage; Whenever I place this code anywhere in my program, I get a 3 bytes memory leak when execution is finished. I am using Visual C++, with the wxWindows framework, under Windows ME. When executing in Debug mode, wxWindows offers the facility of reporting memory leaks. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1502978244-1005178103=:23354 Content-Type: text/html; charset=us-ascii

Hi,

the following code is a simplification of what JCW wrote on the mailing list subject "Using a storage as member in an object":

c4_Storage * MyStorage = new c4_Storage ("test.dat", true);

delete MyStorage;

Whenever I place this code anywhere in my program, I get a 3 bytes memory leak when execution is finished.

I am using Visual C++, with the wxWindows framework, under Windows ME.

When executing in Debug mode, wxWindows offers the facility of reporting memory leaks.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1502978244-1005178103=:23354-- From jyl@best.com Wed Nov 7 23:43:50 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA16612; Wed, 7 Nov 2001 23:43:50 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id VAA26746; Wed, 7 Nov 2001 21:36:08 -0800 (PST) Message-ID: <3BEA27D3.9050304@best.com> Date: Wed, 07 Nov 2001 22:36:03 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit , Steve.Ball@zveno.com Subject: Re: [Metakit] XML References: <20011107230052.3585@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: That's very interesting. Of course it's also possible to represent the structure directly as a tree with the e4Graph package, which provides the capability to input XML directly and also to produce XML output. JYL Jean-Claude Wippler wrote: >If anyone is interested, here's a new trial to load XML data into MetaKit: > http://www.equi4.com/previews/xml2.README > http://www.equi4.com/previews/xml2.tar.gz > >The "mk4xml" tool in there is not optimized, and should not be taken as >an indication of how efficient/inefficient things can be made. My first >goal is to get more experience with having such data in MK and finding >ways to effectively deal with it. Things like DOM would be nice (but >take a much larger effort). Transferring data XML <-> MK <-> XML is a >secondary issue for now, and IMO feasible efficiently in several ways. > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From Steve.Ball@zveno.com Thu Nov 8 00:35:55 2001 Received: from waycool.zveno.com (waycool.zveno.com [210.8.44.221]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id AAA17873; Thu, 8 Nov 2001 00:35:50 -0600 From: Steve.Ball@zveno.com Received: from zveno.com (wakool.zveno.office [192.168.1.2]) by waycool.zveno.com (8.9.3/8.9.3) with ESMTP id RAA20814; Thu, 8 Nov 2001 17:46:24 +1100 Message-ID: <3BEA2A3F.E9BBD4AB@zveno.com> Date: Thu, 08 Nov 2001 17:46:23 +1100 Reply-To: Steve.Ball@zveno.com Organization: Zveno Pty Ltd X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-22mdk i686) X-Accept-Language: en MIME-Version: 1.0 To: Jacob Levy CC: Jean-Claude Wippler , metakit Subject: Re: [Metakit] XML References: <20011107230052.3585@mail.planet.nl> <3BEA27D3.9050304@best.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jacob, You wrote: > > That's very interesting. Of course it's also possible to represent the > structure directly as a tree with the e4Graph package, which provides > the capability to input XML directly and also to produce XML output. Oops... I should have mentioned your work to Jean-Claude earlier in our conversation. I had a look at e4graph a while ago. It's good stuff. My only thought was to suggest that a (Tcl)DOM interface be created for it, rather than an API of its own. Jacob: your thoughts? Cheers, Steve Ball > Jean-Claude Wippler wrote: > > >If anyone is interested, here's a new trial to load XML data into MetaKit: > > http://www.equi4.com/previews/xml2.README > > http://www.equi4.com/previews/xml2.tar.gz > > > >The "mk4xml" tool in there is not optimized, and should not be taken as > >an indication of how efficient/inefficient things can be made. My first > >goal is to get more experience with having such data in MK and finding > >ways to effectively deal with it. Things like DOM would be nice (but > >take a much larger effort). Transferring data XML <-> MK <-> XML is a > >secondary issue for now, and IMO feasible efficiently in several ways. > > > >-jcw > > > >_______________________________________________ > >metakit mailing list - metakit@equi4.com > >http://www.equi4.com/mailman/listinfo/metakit > > -- Steve Ball | XSLT Standard Library | Training & Seminars Zveno Pty Ltd | Web Tcl Complete | XML XSL Schemas http://www.zveno.com/ | TclXML TclDOM | Tcl, Web Development Steve.Ball@zveno.com +---------------------------+--------------------- Ph. +61 2 6242 4099 | Mobile (0413) 594 462 | Fax +61 2 6242 4099 From baspey@yahoo.com Thu Nov 8 01:06:32 2001 Received: from web11505.mail.yahoo.com (web11505.mail.yahoo.com [216.136.172.37]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA18629 for ; Thu, 8 Nov 2001 01:06:31 -0600 Message-ID: <20011108070007.79049.qmail@web11505.mail.yahoo.com> Received: from [212.27.48.59] by web11505.mail.yahoo.com via HTTP; Wed, 07 Nov 2001 23:00:07 PST Date: Wed, 7 Nov 2001 23:00:07 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1447379016-1005202807=:79040" Subject: [Metakit] 3 bytes memory leak when creating c4_Storage on the heap Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1447379016-1005202807=:79040 Content-Type: text/plain; charset=us-ascii I apologize for the "popeye sailorman" nickname Let's say my mail account was misconfigured No offense intended ;-) --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1447379016-1005202807=:79040 Content-Type: text/html; charset=us-ascii

I apologize for the "popeye sailorman" nickname

Let's say my mail account was misconfigured

No offense intended   ;-)


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1447379016-1005202807=:79040-- From jcw@equi4.com Thu Nov 8 03:22:24 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA22526 for ; Thu, 8 Nov 2001 03:22:24 -0600 Received: from takkie.equi4.local ([195.121.233.134]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMH5PS00.NR0; Thu, 8 Nov 2001 10:15:28 +0100 Received: from [10.0.1.14] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA89ENI10063; Thu, 8 Nov 2001 10:14:24 +0100 From: Jean-Claude Wippler To: metakit Cc: Subject: Re: [Metakit] XML Date: Thu, 8 Nov 2001 10:08:27 +0100 Message-Id: <20011108090827.26763@mail.planet.nl> In-Reply-To: <3BEA27D3.9050304@best.com> References: <3BEA27D3.9050304@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >That's very interesting. Of course it's also possible to represent the >structure directly as a tree with the e4Graph package, which provides >the capability to input XML directly and also to produce XML output. Yes, I mentioned e4Graph in the README, but let me do better than that: http://sf.net/projects/e4graph My apologies - I should also have mentioned it in the post to this list. -jcw From jcw@equi4.com Thu Nov 8 03:22:53 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA22542; Thu, 8 Nov 2001 03:22:52 -0600 Received: from takkie.equi4.local ([195.121.233.134]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMH5QK00.BR8; Thu, 8 Nov 2001 10:15:56 +0100 Received: from [10.0.1.14] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA89ETI10067; Thu, 8 Nov 2001 10:14:29 +0100 From: Jean-Claude Wippler To: , Subject: Re: [Metakit] 3 bytes memory leak when creating c4_Storage on the heap Date: Thu, 8 Nov 2001 10:16:30 +0100 Message-Id: <20011108091630.13175@mail.planet.nl> In-Reply-To: <20011108000823.25007.qmail@web11505.mail.yahoo.com> References: <20011108000823.25007.qmail@web11505.mail.yahoo.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Wed, Nov 7, 2001, popeye sailorman , also known as Pascal Baspeyras wrote: >c4_Storage * MyStorage = new c4_Storage ("test.dat", true); >delete MyStorage; > >Whenever I place this code anywhere in my program, I get a 3 bytes memory >leak when execution is finished. > >I am using Visual C++, with the wxWindows framework, under Windows ME. > >When executing in Debug mode, wxWindows offers the facility of reporting >memory leaks. Yes, I use MSVC6/MFC/Debug once in a while to do the same. The "leak" you report is, ehm... intentional. It is caused by lines 173 and 178 in src/string.cpp - where a static copy of an empty string is kept for efficiency (sharing it and bumping refcounts instead of allocating a new one). The bad news is that this will not be fixed. The good news, is that the 3-byte leak does not ever increase or repeat: no matter how much you use and allocate things in MK, there will be at most *one* such 3-byte object leak in the whole application. -jcw From j.p.fletcher@aston.ac.uk Thu Nov 8 06:21:57 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA28810 for ; Thu, 8 Nov 2001 06:21:56 -0600 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 161o5v-00067x-00 for metakit@equi4.com; Thu, 08 Nov 2001 12:15:43 +0000 From: "John Fletcher" To: metakit Date: Thu, 8 Nov 2001 12:13:34 -0000 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] XML Priority: normal In-reply-to: <20011108090827.26763@mail.planet.nl> References: <3BEA27D3.9050304@best.com> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler To: metakit Copies to: Subject: Re: [Metakit] XML Date sent: Thu, 8 Nov 2001 10:08:27 +0100 > Jacob Levy wrote: > > >That's very interesting. Of course it's also possible to represent the > >structure directly as a tree with the e4Graph package, which provides > >the capability to input XML directly and also to produce XML output. > > Yes, I mentioned e4Graph in the README, but let me do better than that: > http://sf.net/projects/e4graph > My apologies - I should also have mentioned it in the post to this list. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > I have been experimenting with the E4Graph XML output. The published version includes in the output atributes which enable the XML to be reread by E4Graph to reconstruct the graph even if there are back refences. This means that if the tree was originally built from XML the output will not be the same as the input. I want to be able to store and retrieve MathML so that I need to turn off the extra attributes. I have been using a modified version of the E4Graph XML for this, with an extra parameter to control the output. I was not entirely happy with this because of the possibility of back references. It may well be that the direct Metakit approach will give me a different way of achieving the same results.# John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From baspey@yahoo.com Thu Nov 8 08:42:13 2001 Received: from web11508.mail.yahoo.com (web11508.mail.yahoo.com [216.136.172.40]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA01678 for ; Thu, 8 Nov 2001 08:42:13 -0600 Message-ID: <20011108143547.27962.qmail@web11508.mail.yahoo.com> Received: from [213.228.40.235] by web11508.mail.yahoo.com via HTTP; Thu, 08 Nov 2001 06:35:47 PST Date: Thu, 8 Nov 2001 06:35:47 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1099233020-1005230147=:27065" Subject: [Metakit] 3 bytes memory leak when creating c4_Storage on the heap Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1099233020-1005230147=:27065 Content-Type: text/plain; charset=us-ascii Jean-Claude Wippler wrote: >The bad news is that this will not be fixed. The good news, is that the >3-byte leak does not ever increase or repeat: no matter how much you use >and allocate things in MK, there will be at most *one* such 3-byte object >leak in the whole application. I had indeed noticed that the memory leak was always of 3 bytes, whatever the number of strorage objects allocated on the heap. Let's hope the user won't close and relaunch the application one billion times ;-) You hack too much, Jean-Claude. You hack too much, but your database is great. I definitively adopt it. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1099233020-1005230147=:27065 Content-Type: text/html; charset=us-ascii

Jean-Claude Wippler wrote:

>The bad news is that this will not be fixed.  The good news, is that the
>3-byte leak does not ever increase or repeat: no matter how much you use
>and allocate things in MK, there will be at most *one* such 3-byte object
>leak in the whole application.

I had indeed noticed that the memory leak was always of 3 bytes, whatever the number

of strorage objects allocated on the heap.

Let's hope the user won't close and relaunch the application one billion times  ;-)

You hack too much, Jean-Claude.

You hack too much, but your database is great.

I definitively adopt it.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1099233020-1005230147=:27065-- From jyl@best.com Thu Nov 8 09:26:51 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA03749; Thu, 8 Nov 2001 09:26:49 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id HAA22847; Thu, 8 Nov 2001 07:20:09 -0800 (PST) Message-ID: <3BEAB0B3.1050709@best.com> Date: Thu, 08 Nov 2001 08:20:03 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit , Steve.Ball@zveno.com Subject: Re: [Metakit] XML References: <3BEA27D3.9050304@best.com> <20011108090827.26763@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks JC I'm curious as to the motivation for mkXML and what you can do with it that's hard or impossible with e4Graph. JYL Jean-Claude Wippler wrote: >Jacob Levy wrote: > >>That's very interesting. Of course it's also possible to represent the >>structure directly as a tree with the e4Graph package, which provides >>the capability to input XML directly and also to produce XML output. >> > >Yes, I mentioned e4Graph in the README, but let me do better than that: > http://sf.net/projects/e4graph >My apologies - I should also have mentioned it in the post to this list. > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From jcw@equi4.com Fri Nov 9 12:39:36 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA09250 for ; Fri, 9 Nov 2001 12:39:35 -0600 Received: from takkie.equi4.local ([195.121.234.244]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMJQ6A00.LHP; Fri, 9 Nov 2001 19:32:34 +0100 Received: from [10.0.1.15] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fA9IVJI19752; Fri, 9 Nov 2001 19:31:40 +0100 From: Jean-Claude Wippler Cc: metakit , Subject: Re: [Metakit] XML Date: Fri, 9 Nov 2001 19:31:41 +0100 Message-Id: <20011109183141.5051@mail.planet.nl> In-Reply-To: <3BEAB0B3.1050709@best.com> References: <3BEAB0B3.1050709@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >I'm curious as to the motivation for mkXML and what you can do with it >that's hard or impossible with e4Graph. That was not my focus for doing this. I wanted to explore the issues when the data is "relatively tabular", yet encoded as XML. Things which have structure, although substantial parts are really tables (and possibly subtables). By trying to force XML trees into plain MK views, instead of a generalized node structure, one has the ability to gradually re-structure data further (manually, or perhaps even automatically). It all depends of the context, clearly. There are many ways to use MK, I guess what I'm trying to find out here is to what extent one can reconstruct tabular (and therefore very efficient columnar) formats. Note that this is just exploration - no fancy coding at all, I just dusted off the 3-year old mk4xml utility I had, and threw a few new tricks at that dog... There are *still* more ways to do things, such as the "^" recursive structure definition which was added in 2.3/2.4, i.e. things like "a[b:S,c:I,d:[^]]" if memory serves me well. The big drawback of that, as in hierarchical file systems, is that nodes can only be identified by a variable-length "path" (the path could be an efficient short vector of ints, but it's variable-sized). -jcw From jyl@best.com Fri Nov 9 17:14:27 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA23789; Fri, 9 Nov 2001 17:14:27 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id PAA29378; Fri, 9 Nov 2001 15:06:58 -0800 (PST) Message-ID: <3BEC6F7F.2040300@best.com> Date: Fri, 09 Nov 2001 16:06:23 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit , Steve.Ball@zveno.com Subject: Re: [Metakit] XML References: <3BEAB0B3.1050709@best.com> <20011109183141.5051@mail.planet.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ummm.. Can you explain a bit about the ^ notation? What does it do? --JYL Jean-Claude Wippler wrote: >Jacob Levy wrote: > >>I'm curious as to the motivation for mkXML and what you can do with it >>that's hard or impossible with e4Graph. >> > >That was not my focus for doing this. I wanted to explore the issues >when the data is "relatively tabular", yet encoded as XML. Things which >have structure, although substantial parts are really tables (and >possibly subtables). > >By trying to force XML trees into plain MK views, instead of a >generalized node structure, one has the ability to gradually re-structure >data further (manually, or perhaps even automatically). It all depends >of the context, clearly. > >There are many ways to use MK, I guess what I'm trying to find out here >is to what extent one can reconstruct tabular (and therefore very >efficient columnar) formats. > >Note that this is just exploration - no fancy coding at all, I just >dusted off the 3-year old mk4xml utility I had, and threw a few new >tricks at that dog... > >There are *still* more ways to do things, such as the "^" recursive >structure definition which was added in 2.3/2.4, i.e. things like >"a[b:S,c:I,d:[^]]" if memory serves me well. The big drawback of that, >as in hierarchical file systems, is that nodes can only be identified by >a variable-length "path" (the path could be an efficient short vector of >ints, but it's variable-sized). > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From tito@apple.com Sun Nov 11 19:11:35 2001 Received: from mail-out1.apple.com (mail-out1.apple.com [17.254.0.52]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id TAA20165 for ; Sun, 11 Nov 2001 19:11:35 -0600 Received: from mailgate2.apple.com (A17-129-100-225.apple.com [17.129.100.225]) by mail-out1.apple.com (8.11.3/8.11.3) with ESMTP id fAC14uu08197 for ; Sun, 11 Nov 2001 17:04:56 -0800 (PST) Received: from scv2.apple.com (scv2.apple.com) by mailgate2.apple.com (Content Technologies SMTPRS 4.2.1) with ESMTP id for ; Sun, 11 Nov 2001 17:04:55 -0800 Received: from localhost ([17.254.160.6]) by scv2.apple.com (8.11.3/8.11.3) with ESMTP id fAC14sH17360 for ; Sun, 11 Nov 2001 17:04:54 -0800 (PST) Date: Sun, 11 Nov 2001 17:04:54 -0800 Mime-Version: 1.0 (Apple Message framework v501) Content-Type: multipart/alternative; boundary=Apple-Mail-2--697392676 From: Tito Ciuro To: metakit@equi4.com Message-Id: <48711456-D709-11D5-9F53-00039310A0E8@apple.com> X-Mailer: Apple Mail (2.501) Subject: [Metakit] Performance questions about MetaKit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-2--697392676 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hello, I have a few quick questions regarding MetaKit: 1) Is there a limit of columns per view? 2) Is there a performance hit after any specific number of columns? 3) Is there a performance hit after any specific number of rows? 4) What is the biggest int MetaKit can encode? (long long?) Thanks! -- Tito ____________________________________________ "By the time they had diminished from 50 to 8, the other dwarves began to suspect 'Hungry' ..." -- Gary Larson, "The Far Side" --Apple-Mail-2--697392676 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hello, I have a few quick questions regarding MetaKit: 1) Is there a limit of columns per view? 2) Is there a performance hit after any specific number of columns? 3) Is there a performance hit after any specific number of rows? 4) What is the biggest int MetaKit can encode? (long long?) Thanks! -- Tito ____________________________________________ "By the time they had diminished from 50 to 8, the other dwarves began to suspect 'Hungry' ..." -- Gary Larson, "The Far Side" --Apple-Mail-2--697392676-- From baspey@yahoo.com Sun Nov 11 22:35:15 2001 Received: from web11502.mail.yahoo.com (web11502.mail.yahoo.com [216.136.172.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id WAA28522 for ; Sun, 11 Nov 2001 22:35:15 -0600 Message-ID: <20011112042836.96786.qmail@web11502.mail.yahoo.com> Received: from [212.27.48.193] by web11502.mail.yahoo.com via HTTP; Sun, 11 Nov 2001 20:28:36 PST Date: Sun, 11 Nov 2001 20:28:36 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-641453618-1005539316=:96710" Subject: [Metakit] Strange behaviours using hash views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-641453618-1005539316=:96710 Content-Type: text/plain; charset=us-ascii Hello Jean-Claude, I observe quite strange (and annoying) behaviours when I try to use a hash view. I'm using the latest cvs under Windows. For the test, I parse a text file containing 7000 entries. I build my database quite conventionaly (I hope): c4_Storage MyDatabase = ("MyDatabase.dat", true); c4_View vData = MyStorage .GetAs("Data[Field1:S,Field2:S]"); c4_View vMap = MyStorage .GetAs("Map[_H:I,_R:I]"); c4_View vHash = vData.Hash(vMap, 2); c4_StringProp pField1("Field1"); c4_StringProp pField2("Field2"); c4_Row Row; For each entry read from the text file, I do: pField1(Row) = "found text"; pField2(Row) = "next found text"; vHash.Add(Row); And when finished, I commit. THE FIRST WEIRD BEHAVIOR: I run the program once to construct my database file. OK But then, every time I rerun the program, the file gets bigger (quite bigger), getting fat from the bottom, though the number of rows stays the same (normal), as I can see using KitViewer. Every two times I rerun the program, the map view doubles the number of its rows, which increases of course the size of the database file, but the size of the file also increases when the map view does not double its rows... SECOND WEIRD BEHAVIOUR: I read (just read) my text file, and for each entry, search (just search) for the row position of the corresponding item in the database: int RowIndex = vHash.Find(pField1["blablabla"]); And the big O cost I observe is nothing like 1. Strangely, each search takes more time than the preceding, and it takes several seconds to find the last 500 items... (I have a Celeron 800) THIRD WEIRD BEHAVIOUR: If I compare the time it takes to build the database in the first place, with the time it takes to search each item (see above), well... the building is about 10 times faster than the searching (may be normal due to disk accesses, but is it really ?) Thanks for paying attention to my distress. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-641453618-1005539316=:96710 Content-Type: text/html; charset=us-ascii

Hello Jean-Claude,

I observe quite strange (and annoying) behaviours when I try to use a hash view.

I'm using the latest cvs under Windows.

For the test, I parse a text file containing 7000 entries.

I build my database quite conventionaly (I hope):

c4_Storage MyDatabase = ("MyDatabase.dat", true);

c4_View vData = MyStorage .GetAs("Data[Field1:S,Field2:S]");

c4_View vMap = MyStorage .GetAs("Map[_H:I,_R:I]");

c4_View vHash = vData.Hash(vMap, 2);

c4_StringProp pField1("Field1");

c4_StringProp pField2("Field2");

c4_Row Row;

 

For each entry read from the text file, I do:

pField1(Row) = "found text";

pField2(Row) = "next found text";

vHash.Add(Row);

And when finished, I commit.

 

THE FIRST WEIRD BEHAVIOR:

I run the program once to construct my database file.

OK

But then, every time I rerun the program, the file gets bigger (quite bigger), getting fat from the bottom, though the number of rows stays the same (normal), as I can see using KitViewer.

Every two times I rerun the program, the map view doubles the number of its rows, which increases of course the size of the database file, but the size of the file also increases when the map view does not double its rows...

SECOND WEIRD BEHAVIOUR:

I read (just read) my text file, and for each entry, search (just search) for the row position of the corresponding item in the database:

int RowIndex = vHash.Find(pField1["blablabla"]);

And the big O cost I observe is nothing like 1.

Strangely, each search takes more time than the preceding, and it takes several seconds to find the last 500 items... (I have a Celeron 800)

THIRD WEIRD BEHAVIOUR:

If I compare the time it takes to build the database in the first place, with the time it takes to search each item (see above), well... the building is about 10 times faster than the searching (may be normal due to disk accesses, but is it really ?)

 

Thanks for paying attention to my distress.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-641453618-1005539316=:96710-- From baspey@yahoo.com Sun Nov 11 23:15:13 2001 Received: from web11504.mail.yahoo.com (web11504.mail.yahoo.com [216.136.172.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA31038 for ; Sun, 11 Nov 2001 23:15:13 -0600 Message-ID: <20011112050833.54799.qmail@web11504.mail.yahoo.com> Received: from [212.27.48.102] by web11504.mail.yahoo.com via HTTP; Sun, 11 Nov 2001 21:08:33 PST Date: Sun, 11 Nov 2001 21:08:33 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1496710524-1005541713=:54305" Subject: [Metakit] Strange behaviours using hash views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1496710524-1005541713=:54305 Content-Type: text/plain; charset=us-ascii Oooops... In my precedent posting, please don't pay attention to the third strange behaviour I describe and which simply is a consequence of the increasing search time described in strange behaviour n° 2. And I also apologize for the too long lines. Please have mercy on a poor fellow like me. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1496710524-1005541713=:54305 Content-Type: text/html; charset=us-ascii

Oooops...

In my precedent posting, please don't pay attention to the third

strange behaviour I describe and which simply is a consequence

of the increasing search time described in strange behaviour n° 2.

And I also apologize for the too long lines.

Please have mercy on a poor fellow like me.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1496710524-1005541713=:54305-- From baspey@yahoo.com Mon Nov 12 09:07:48 2001 Received: from web11507.mail.yahoo.com (web11507.mail.yahoo.com [216.136.172.39]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA22164 for ; Mon, 12 Nov 2001 09:07:48 -0600 Message-ID: <20011112150107.87023.qmail@web11507.mail.yahoo.com> Received: from [212.27.48.216] by web11507.mail.yahoo.com via HTTP; Mon, 12 Nov 2001 07:01:07 PST Date: Mon, 12 Nov 2001 07:01:07 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-282460218-1005577267=:86790" Subject: [Metakit] Strange behaviours using hash views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-282460218-1005577267=:86790 Content-Type: text/plain; charset=us-ascii Inspecting Find() with the debugger confirmed that it uses a linear search (from first row to the last one if necessary) in spite of the fact that the view is a hash view. Nonetheless I suppose using Find() is the only possibility, because Search() and Locate() are intended for binary search on sorted views (unless I'm wrong). --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-282460218-1005577267=:86790 Content-Type: text/html; charset=us-ascii

Inspecting Find() with the debugger confirmed that it uses a linear search (from first row to the last one if necessary)

in spite of the fact that the view is a hash view.

Nonetheless I suppose using Find() is the only possibility, because Search() and Locate() are intended for binary

search on sorted views (unless I'm wrong).


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-282460218-1005577267=:86790-- From jcw@equi4.com Mon Nov 12 09:53:12 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA25044 for ; Mon, 12 Nov 2001 09:53:11 -0600 Received: from takkie.equi4.local ([195.121.235.2]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMP2GN01.0VU for ; Mon, 12 Nov 2001 16:45:59 +0100 Received: from [10.0.1.21] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fACFjXI06704 for ; Mon, 12 Nov 2001 16:45:33 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Strange behaviours using hash views Date: Mon, 12 Nov 2001 16:47:26 +0100 Message-Id: <20011112154726.23523@mail.planet.nl> In-Reply-To: <20011112042836.96786.qmail@web11502.mail.yahoo.com> References: <20011112042836.96786.qmail@web11502.mail.yahoo.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Pascal Baspeyras wrote: >c4_View vData = MyStorage .GetAs("Data[Field1:S,Field2:S]"); >c4_View vMap = MyStorage .GetAs("Map[_H:I,_R:I]"); >c4_View vHash = vData.Hash(vMap, 2); [...] >But then, every time I rerun the program, the file gets bigger (quite >bigger), getting fat from the bottom, though the number of rows stays the >same (normal), as I can see using KitViewer. This one I can't quite figure out - still thinking... >int RowIndex = vHash.Find(pField1["blablabla"]); You specified a 2-key hash (the "2" above). MK cannot locate items with hashing when given one field, hence it resorts to linear search. -jcw From baspey@yahoo.com Mon Nov 12 11:44:35 2001 Received: from web11507.mail.yahoo.com (web11507.mail.yahoo.com [216.136.172.39]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id LAA00640 for ; Mon, 12 Nov 2001 11:44:34 -0600 Message-ID: <20011112173753.17702.qmail@web11507.mail.yahoo.com> Received: from [213.228.42.101] by web11507.mail.yahoo.com via HTTP; Mon, 12 Nov 2001 09:37:53 PST Date: Mon, 12 Nov 2001 09:37:53 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1134683454-1005586673=:17251" Subject: [Metakit] Strange behaviours using hash views Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1134683454-1005586673=:17251 Content-Type: text/plain; charset=us-ascii Hello Jean-Claude, Please stop thinking because that's no use... I made a trip into twilight zone: the behaviour I described I can't reproduce anymore. Maybe the mere fact that I shut down my computer for the night. I wanted to join to my mail the 2 incriminated files, but now the second rerun still builds a bigger file, but because each row is present 2 times (I didn't know it was possible with a hash view, thought that if an entry already existed, it replaced the precedent one). As for the find(), you are off course right. Thanks a lot for your help, and who knows, if I can reproduce the pathological behaviour, I'll tell you about. --------------------------------- Do You Yahoo!? Find a job, post your resume on Yahoo! Careers. --0-1134683454-1005586673=:17251 Content-Type: text/html; charset=us-ascii

Hello Jean-Claude,

Please stop thinking because that's no use...

I made a trip into twilight zone: the behaviour I described I can't

reproduce anymore.

Maybe the mere fact that I shut down my computer for the night.

I wanted to join to my mail the 2 incriminated files, but now the second

rerun still builds a bigger file, but because each row is present 2 times

(I didn't know it was possible with a hash view, thought that if an entry

already existed, it replaced the precedent one).

As for the find(), you are off course right.

Thanks a lot for your help, and who knows, if I can reproduce the

pathological behaviour, I'll tell you about.


Do You Yahoo!?
Find a job, post your resume on Yahoo! Careers. --0-1134683454-1005586673=:17251-- From jcw@equi4.com Wed Nov 14 17:16:02 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA20844; Wed, 14 Nov 2001 17:16:01 -0600 Received: from takkie.equi4.local ([195.121.235.10]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMTCAF01.G7U; Thu, 15 Nov 2001 00:08:39 +0100 Received: from [10.0.1.23] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAEN60004372; Thu, 15 Nov 2001 00:06:10 +0100 From: Jean-Claude Wippler To: , Subject: Re: [Metakit] Performance questions about MetaKit Date: Thu, 15 Nov 2001 00:08:43 +0100 Message-Id: <20011114230843.7262@mail.planet.nl> In-Reply-To: <48711456-D709-11D5-9F53-00039310A0E8@apple.com> References: <48711456-D709-11D5-9F53-00039310A0E8@apple.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, >1) Is there a limit of columns per view? Not really. Thousands have been used. Have you tried? >2) Is there a performance hit after any specific number of columns? >3) Is there a performance hit after any specific number of rows? Not really. There may be slow-downs, quadratic slow-downs even. But there's probably no way to answer this in general. It depends. MK can soar and crawl. I'm not trying to give a cheeky answer... it really depends. There are all sorts of surprises, both in what bogs down, and in what one can do to fix it. >4) What is the biggest int MetaKit can encode? (long long?) Yes, see "mk4.h". Note that 64-bit ints are a separate (non-autosizing) type. -jcw From jcw@equi4.com Wed Nov 14 17:35:18 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA21933 for ; Wed, 14 Nov 2001 17:35:18 -0600 Received: from takkie.equi4.local ([195.121.235.194]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMTD6K01.7L1; Thu, 15 Nov 2001 00:27:56 +0100 Received: from [10.0.1.23] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAENPG004742; Thu, 15 Nov 2001 00:25:22 +0100 From: Jean-Claude Wippler Cc: metakit , Subject: Re: [Metakit] XML Date: Thu, 15 Nov 2001 00:27:55 +0100 Message-Id: <20011114232755.19435@mail.planet.nl> In-Reply-To: <3BEC6F7F.2040300@best.com> References: <3BEC6F7F.2040300@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >>There are *still* more ways to do things, such as the "^" recursive >>structure definition which was added in 2.3/2.4, i.e. things like >>"a[b:S,c:I,d:[^]]" if memory serves me well. [...] >Ummm.. Can you explain a bit about the ^ notation? What does it do? It recurses, i.e. re-uses the definition of its parent view: "a[b:S,c:I,d:[^]]" "a[b:S,c:I,d:[b:S,c:I,d:[^]]]" "a[b:S,c:I,d:[b:S,c:I,d:[b:S,c:I,d:[^]]]]" etc... In other words, this defines a tree of same-typed nodes, each with a variable number of children. It's not going to be particularly efficient in things like binary trees, because columns/subcolumns end up being short and numerous. The big surprise (for me, anyway) was that on-the-fly restructuring continued to work without coding anything special for it. See the examples/selfref.py demo. -jcw From steven.leblanc@mayahtt.com Fri Nov 16 15:42:17 2001 Received: from qmail.mayahtt.ca (mayahtt.com [199.84.144.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA21059 for ; Fri, 16 Nov 2001 15:42:17 -0600 Received: from _HOSTNAME_ (indigo23.mayahtt.ca [192.9.201.194]) by qmail.mayahtt.ca (8.11.2/8.11.2) with SMTP id fAGLZ4X08078 for ; Fri, 16 Nov 2001 16:35:04 -0500 Received: by _HOSTNAME_ (sSMTP sendmail emulation); Fri, 16 Nov 2001 16:35:04 -0500 Date: Fri, 16 Nov 2001 16:35:04 -0500 From: Steve Leblanc To: metakit@equi4.com Message-ID: <20011116163504.B267130@mayahtt.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Organization: Maya Heat Transfer Technologies Subject: [Metakit] Problem on SGI and HP Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi I'm really interested in using Metakit for some projects at work, but it's got to work on all our platforms. So, the first thing I did was run the regression test on everything we have. Windows, HP-UX, and Solaris worked but there is a problem on IRIX and AIX. On both platforms TestMapped goes into an infinite loop. This seems to happen during the third call to v3.Add() when it gets down to HashViewer::LookDict. It gets stuck in the for() loop and the value of i just cycles between 4 different values. The loop never meets the termination conditions. I really don't have much time to investigate this, but I'd love to use MK so I'll play around with it a bit. I'll admit that I don't really know how MK works so it's going to be tough to debug. I saw a comment about an earlier infinite loop problem with the hashing stuff, so maybe some- one can give me a hint about where to start. I'd hate to have to trash MK because of this. Thanks -- Steve Leblanc Maya Heat Transfer Technologies Ltd. Phone: (514) 369-5706 Fax: (514) 369-4200 From jcw@equi4.com Fri Nov 16 16:59:16 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA25826; Fri, 16 Nov 2001 16:59:15 -0600 Received: from takkie.equi4.local ([195.121.236.94]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GMX0V300.V03; Fri, 16 Nov 2001 23:52:15 +0100 Received: from [10.0.1.26] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAGMnp019669; Fri, 16 Nov 2001 23:50:02 +0100 From: Jean-Claude Wippler To: , metakit Subject: Re: [Metakit] Problem on SGI and HP Date: Fri, 16 Nov 2001 23:51:36 +0100 Message-Id: <20011116225136.13230@mail.planet.nl> In-Reply-To: <20011116163504.B267130@mayahtt.com> References: <20011116163504.B267130@mayahtt.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Steve Leblanc wrote: >I'm really interested in using Metakit for some projects at work, but >it's got to work on all our platforms. So, the first thing I did was >run the regression test on everything we have. Windows, HP-UX, and >Solaris worked but there is a problem on IRIX and AIX. On both platforms >TestMapped goes into an infinite loop. This seems to happen during the >third call to v3.Add() when it gets down to HashViewer::LookDict. >It gets stuck in the for() loop and the value of i just cycles between >4 different values. The loop never meets the termination conditions. Please make sure you use the latest code in CVS. If you comment out TestMapped, does the rest run properly? if so, we could start narrowing it down to things like word sizes, signed/unsigned, and such... Note that the hash view is relatively young. If all other tests run ok, then you can safely assume that MK is in pretty good shape. I'll walk through the code and re-check things, if you could please tell me a bit about architecture details, especially how they differ from the Win32/ Linux systems (x86), since I know those best. Quick thought: could this be as mundane as "just" an endian-ness issue? Btw, HP-UX is good news. It hasn't always been like that - what system/ os/machine revision would that be? -jcw From baspey@yahoo.com Sun Nov 18 21:22:57 2001 Received: from web11504.mail.yahoo.com (web11504.mail.yahoo.com [216.136.172.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA01747 for ; Sun, 18 Nov 2001 21:22:57 -0600 Message-ID: <20011119031550.51194.qmail@web11504.mail.yahoo.com> Received: from [62.147.36.130] by web11504.mail.yahoo.com via HTTP; Sun, 18 Nov 2001 19:15:50 PST Date: Sun, 18 Nov 2001 19:15:50 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-563452070-1006139750=:49291" Subject: [Metakit] Using hash value as a reference (??) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-563452070-1006139750=:49291 Content-Type: text/plain; charset=us-ascii Hi everybody, In my database, the keys in my first view are movie titles. The second view is a dictionary build from the words contained in these titles. For each word in the dictionary, there is a subview listing all the titles containing this word. Of course, for saving space, the dictionary view only records the index of the movie title. But here is my problem: for efficiency reasons, the main view (containing the movie titles) has to be ordered. Thus the index of a movie is very likely to change after inserting other movies. The solution I have thought of is to construct the view containing the movie titles as a blocked/hash/ordered view. All right. And then, in the dictionary view, the movies would be listed not with their index position, but with their hash value instead. Does this sound crazy or totally stupid to you ? In order to do so, it seems rather easy to find what the hash value of a movie is, but what might be more difficult is to reach a movie by directly giving its hash value. Thank you for giving your opinion. --------------------------------- Do You Yahoo!? Find the one for you at Yahoo! Personals. --0-563452070-1006139750=:49291 Content-Type: text/html; charset=us-ascii

Hi everybody,

In my database, the keys in my first view are movie titles.

The second view is a dictionary build from the words contained in these titles.

For each word in the dictionary, there is a subview listing all the titles containing

this word.

Of course, for saving space, the dictionary view only records the index of the movie title.

But here is my problem: for efficiency reasons, the main view (containing the movie titles)

has to be ordered.

Thus the index of a movie is very likely to change after inserting other movies.

The solution I have thought of is to construct the view containing the movie titles as a

blocked/hash/ordered view. All right. And then, in the dictionary view, the movies would

be listed not with their index position, but with their hash value instead.

Does this sound crazy or totally stupid to you ?

In order to do so, it seems rather easy to find what the hash value of a movie is, but

what might be more difficult is to reach a movie by directly giving its hash value.

Thank you for giving your opinion.


Do You Yahoo!?
Find the one for you at Yahoo! Personals. --0-563452070-1006139750=:49291-- From mitch.chapman@bioreason.com Mon Nov 19 13:10:13 2001 Received: from gateway.bioreason.com (www.bioreason.com [207.108.245.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA14396 for ; Mon, 19 Nov 2001 13:10:13 -0600 Received: by gateway.bioreason.com (Postfix, from userid 200) id B455C2237; Mon, 19 Nov 2001 12:01:28 +0000 (/etc/localtime) Received: by bioreason.com via smwrap Version 1.0 id smwrapVdMpRw; Mon Nov 19 12:01:09 2001 by mailhost.bioreason.com (Postfix) with ESMTP id 5CD9F3577 for ; Mon, 19 Nov 2001 12:01:42 +0000 (/etc/localtime) Message-ID: <3BF958AB.F9AEDFE5@bioreason.com> Date: Mon, 19 Nov 2001 12:08:27 -0700 From: Mitch Chapman Organization: Bioreason, Inc. MIME-Version: 1.0 To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Why does this fail? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm new to Metakit, and am having problems with apparent memory corruption when using Metakit 2.4.1 with either Python 2.1 or Python 2.2b2. The script on which I'm working sometimes segfaults with a gdb stack trace which starts in the Python 2.x garbage collector. I tried to produce a simple problem-demonstration script to include with this message, but succeeded only in getting a different runtime error -- a SystemError. It must be Monday :) Anyway, the demonstration script appears below, together with its output under both Python2.1 and Python2.2b2 (Metakit 2.4.1 having been compiled and installed separately for each). Anybody have any ideas what's going on? Feel free to ask for more information about configurations, versions, etc., than is provided in the script output. Thanks for help. -- Mitch Chapman Mitch.Chapman@bioreason.com ----------------------------------------------------------------------- #!/usr/bin/env python """Demonstrate some odd SystemErrors with Metakit and Python.""" import gc, metakit import sys, os # Show the current platform: osVersion = open("/etc/mandrake-release").read().strip() print "Operating System:", osVersion print " ".join(os.uname()) print print "Python:", sys.version print print "Metakit:", metakit.version print def MKRecord(**kw): return kw db = metakit.storage() version = db.getas("version[ident:S]") # Uncomment this to trigger a SystemError further downstream: del version[:] version.setsize(1) # This causes a 'SystemError: null argument to internal routine' # exception. value = MKRecord(ident="Version 1.0") version[0] = value db.commit() gc.collect() ----------------------------------------------------------------------- Operating System: Linux Mandrake release 7.2 (Odyssey) for i586 Linux calabash.bioreason.com 2.2.17-21mdk #1 Thu Oct 5 13:16:08 CEST 2000 i686 Python: 2.1 (#1, Oct 13 2001, 13:52:22) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Traceback (most recent call last): File "CrashMe.py", line 29, in ? value = MKRecord(ident="Version 1.0") SystemError: null argument to internal routine ----------------------------------------------------------------------- Operating System: Linux Mandrake release 7.2 (Odyssey) for i586 Linux calabash.bioreason.com 2.2.17-21mdk #1 Thu Oct 5 13:16:08 CEST 2000 i686 Python: 2.2b2 (#1, Nov 19 2001, 09:55:10) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Traceback (most recent call last): File "CrashMe.py", line 29, in ? value = MKRecord(ident="Version 1.0") SystemError: null argument to internal routine From jcw@equi4.com Mon Nov 19 13:22:29 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA15367 for ; Mon, 19 Nov 2001 13:22:28 -0600 Received: from takkie.equi4.local ([195.121.237.142]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GN2ASO01.4V2 for ; Mon, 19 Nov 2001 20:14:48 +0100 Received: from [10.0.1.29] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAJJCw006223 for ; Mon, 19 Nov 2001 20:12:58 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Why does this fail? Date: Mon, 19 Nov 2001 20:14:54 +0100 Message-Id: <20011119191454.14624@mail.planet.nl> In-Reply-To: <3BF958AB.F9AEDFE5@bioreason.com> References: <3BF958AB.F9AEDFE5@bioreason.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mitch Chapman wrote: >The script on which I'm working sometimes segfaults with a gdb >stack trace which starts in the Python 2.x garbage collector. The segfault is serious, I hope you can reproduce it - somehow. ># Uncomment this to trigger a SystemError further downstream: >del version[:] This error was solved and reported in the change log: http://www.equi4.com/metakit/CHANGES 2001-10-31 Fixed Mk4py error flag clear on delete When deleting rows, a slice was constructed from a PWOSequence, which generates an otherwise harmless error when its length is checked. The flag was not cleared, causing errors in subsequent Python statements. Changed to a PWOTuple in PyView.cpp (2x). This change is more recent than the 2.4.1 release, see the download page for info on how to obtain the latest source code from CVS: http://www.equi4.com/metakit/download.html -jcw From mitch.chapman@bioreason.com Mon Nov 19 13:38:15 2001 Received: from gateway.bioreason.com (www.bioreason.com [207.108.245.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA16525; Mon, 19 Nov 2001 13:38:14 -0600 Received: by gateway.bioreason.com (Postfix, from userid 200) id 114B82237; Mon, 19 Nov 2001 12:29:30 +0000 (/etc/localtime) Received: by bioreason.com via smwrap Version 1.0 id smwrapJaby99; Mon Nov 19 12:28:41 2001 by mailhost.bioreason.com (Postfix) with ESMTP id DE5403577; Mon, 19 Nov 2001 12:29:14 +0000 (/etc/localtime) Message-ID: <3BF95F20.FBC23F86@bioreason.com> Date: Mon, 19 Nov 2001 12:36:00 -0700 From: Mitch Chapman Organization: Bioreason, Inc. MIME-Version: 1.0 To: Jean-Claude Wippler Cc: metakit Subject: Re: [Metakit] Why does this fail? References: <3BF958AB.F9AEDFE5@bioreason.com> <20011119191454.14624@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > > Mitch Chapman wrote: > ># Uncomment this to trigger a SystemError further downstream: > >del version[:] > > This error was solved and reported in the change log: > http://www.equi4.com/metakit/CHANGES > > 2001-10-31 Fixed Mk4py error flag clear on delete Thanks! > This change is more recent than the 2.4.1 release, see the download page > for info on how to obtain the latest source code from CVS: > http://www.equi4.com/metakit/download.html Is there another way to get at the changes? I seem to be unable to make CVS connections from here... -- Mitch Chapman Mitch.Chapman@bioreason.com From steven.leblanc@mayahtt.com Mon Nov 19 14:15:02 2001 Received: from qmail.mayahtt.ca (mayahtt.com [199.84.144.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA19364 for ; Mon, 19 Nov 2001 14:15:02 -0600 Received: from _HOSTNAME_ (indigo23.mayahtt.ca [192.9.201.194]) by qmail.mayahtt.ca (8.11.2/8.11.2) with SMTP id fAJK7oX22248 for ; Mon, 19 Nov 2001 15:07:52 -0500 Received: by _HOSTNAME_ (sSMTP sendmail emulation); Mon, 19 Nov 2001 15:07:51 -0500 Date: Mon, 19 Nov 2001 15:07:51 -0500 From: Steve Leblanc To: metakit@equi4.com Message-ID: <20011119150751.A281197@mayahtt.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Organization: Maya Heat Transfer Technologies Subject: [Metakit] Problem on SGI and AIX Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi [snip: re infinite loop in TestMapped on SGI and AIX] I tried using the latest CVS code and it gave me the same problem. The good news is that everything works if I comment out the call to TestMapped and I don't need the hash view for my application. Here's what I know about the achitecture of the two platforms on which this test hangs: SGI --- OS: IRIX 6.5 Compiler: MIPS C++ compiler, vers 7.2.1 32 bit big-endian AIX --- OS: AIX 3.4 Compiler: IBM C++ Compiler, vers 3.6.4 32 bit, big-endian (I think) HP-UX worked fine with this configuration: HP-UX B.10.20 using HP ANSI C++ B3910B A.01.21 I don't think it's purely an endianness issue because Solaris is also big-endian and the test runs fine. However, it may well be some low-level thing like that, since the loop in question does a lot of bit-twiddling. P.S. Sorry for breaking the thread. I wasn't subscribed to the mailing list when I sent my first mail, so I can't thread this one to the back of yours. -- Steve Leblanc Maya Heat Transfer Technologies Ltd. Phone: (514) 369-5706 Fax: (514) 369-4200 From mitch.chapman@bioreason.com Mon Nov 19 16:17:22 2001 Received: from gateway.bioreason.com (www.bioreason.com [207.108.245.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA27926 for ; Mon, 19 Nov 2001 16:17:21 -0600 Received: by gateway.bioreason.com (Postfix, from userid 200) id C9D61223B; Mon, 19 Nov 2001 15:08:36 +0000 (/etc/localtime) Received: by bioreason.com via smwrap Version 1.0 id smwrapTYSg3d; Mon Nov 19 15:08:17 2001 by mailhost.bioreason.com (Postfix) with ESMTP id 1CEE23577 for ; Mon, 19 Nov 2001 15:08:50 +0000 (/etc/localtime) Message-ID: <3BF98487.5C434126@bioreason.com> Date: Mon, 19 Nov 2001 15:15:35 -0700 From: Mitch Chapman Organization: Bioreason, Inc. MIME-Version: 1.0 To: Metakit Mailing List Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] Python, Metakit, Linux, SegFaults -- more info Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Below are three Python scripts, with output, which trigger segfaults on Mandrake Linux 7.2. The problems may already have been discovered and fixed,* in which case please pardon the waste of bandwidth. All of these scripts use Metakit incorrectly. They also show usage errors of the sort to which a[t least one] new user is prone, so may be useful for documentation purposes. ("Are you new to Metakit? Well, don't do this.") -- Mitch Chapman Mitch.Chapman@bioreason.com *(I did check the ChangeLog but found no clearly related entries.) ======================================================================== #!/usr/bin/env python """Crash through indexing error -- same cause as SystemError?""" import metakit, sys, os # Show the current platform: osVersion = open("/etc/mandrake-release").read().strip() print "OS:", osVersion print "Python:", sys.version print "Metakit:", metakit.version print db = metakit.storage() schema = db.getas("version[ident:S]") schema[:] = [] # This will crash because it's indexing beyond the end of schema. # Should it produce an IndexError instead? schema[0] = {'ident': 'Kilroy'} ----------------------------------------------------------------------- OS: Linux Mandrake release 7.2 (Odyssey) for i586 Python: 2.2b2 (#1, Nov 19 2001, 09:55:10) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Segmentation fault ======================================================================== #!/usr/bin/env python """Crash by saving an in-memory store?""" import metakit, sys, os # Show the current platform: osVersion = open("/etc/mandrake-release").read().strip() print "OS:", osVersion print "Python:", sys.version print "Metakit:", metakit.version print db = metakit.storage() schema = db.getas("version[ident:S]") schema[:] = [] schema.append({'ident': 'Kilroy'}) # This will crash because storage has no associated file? db.commit() ----------------------------------------------------------------------- OS: Linux Mandrake release 7.2 (Odyssey) for i586 Python: 2.2b2 (#1, Nov 19 2001, 09:55:10) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Segmentation fault ======================================================================== #!/usr/bin/env python """Crash through bad subview manipulation.""" import sys, metakit, os # Show the current platform: osVersion = open("/etc/mandrake-release").read().strip() print "OS:", osVersion print "Python:", sys.version print "Metakit:", metakit.version print db = metakit.storage("crash.dat", 1) records = db.getas("record[ident:I,contents[cid:I]]") numRecords = 500 records[:] = [] records.setsize(numRecords) # Why does this crash instead of complaining about the value of # contents? contents = range(10) for i in range(numRecords): records[i] = {'ident': i, 'contents': contents} print "Saved", i sys.stdout.flush() db.commit() ----------------------------------------------------------------------- OS: Linux Mandrake release 7.2 (Odyssey) for i586 Python: 2.2b2 (#1, Nov 19 2001, 09:55:10) [GCC 2.95.3 19991030 (prerelease)] Metakit: 2.4.1 Saved 0 Saved 1 Saved 2 Saved 3 Saved 4 Saved 5 Saved 6 Saved 7 Segmentation fault From mlh@idi.ntnu.no Mon Nov 19 18:02:53 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id SAA01522 for ; Mon, 19 Nov 2001 18:02:52 -0600 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id AAA03069 for ; Tue, 20 Nov 2001 00:55:37 +0100 (MET) Message-ID: <045301c17155$af6f7280$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: References: <20011119031550.51194.qmail@web11504.mail.yahoo.com> Subject: Re: [Metakit] Using hash value as a reference (??) Date: Tue, 20 Nov 2001 00:55:37 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have another suggestion for you: If the sorting/hashing is only for lookup, and you don't delete titles from your system, what about having a separate view which only contains the titles (not sorted)? Then the dictionary could contain the index of the title in this view, you could look up the title, and use the sorting/hashing to look up that title in the main view. You might want to store a reverse version of this too (name -> index) but that could be done by simply adding an index field to your sorted/hashed view. It's simple, at least. -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.sf.net From baspey@yahoo.com Tue Nov 20 02:18:44 2001 Received: from web11503.mail.yahoo.com (web11503.mail.yahoo.com [216.136.172.35]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id CAA17418 for ; Tue, 20 Nov 2001 02:18:44 -0600 Message-ID: <20011120081127.51629.qmail@web11503.mail.yahoo.com> Received: from [62.147.34.109] by web11503.mail.yahoo.com via HTTP; Tue, 20 Nov 2001 00:11:27 PST Date: Tue, 20 Nov 2001 00:11:27 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1246303962-1006243887=:50488" Subject: [Metakit] Using hash value as a reference (??) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1246303962-1006243887=:50488 Content-Type: text/plain; charset=us-ascii Thanks a lot Magnus, your solution is quite interesting. I wanted my first view to be sorted in order to be able to list all the movies in alphabetical order, the most instantly possible, even with an important number of movies. Of course, having an already ordered view would have been the most efficient. Furthermore, since my last posting, I have implemented a third solution: I have added a field im my first view corresponding to the index of the next movie in the alphabetical order, thus creating same sort of linked list. Everything is quite normal as the rest of the database is concerned: I reference movies in the dictionary thanks to their index, the fact that they are unordered having no incidence. But when I want to do this special thing which is to list all movies orderly, I begin by the first row and then follow down the path given by this additional field. Without entering into all details, adding a movie goes like this: I append the movie at the end of the view; if this movie is alphabetically before the first row, I permute the two rows; otherwise, I randomly pick up 50 movie titles and keep the closest above my new movie title; from this movie I go down the track until I find the place where my new movie should be inserted; I simply make my new movie point toward the movie pointed by the preceding movie , and then make the preceding movie point toward my new movie. The row pointed by the last movie is -1. Although this may seem complicated, it proves itself more efficient to manage than an ordered view. --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1246303962-1006243887=:50488 Content-Type: text/html; charset=us-ascii

Thanks a lot Magnus,

your solution is quite interesting.

I wanted my first view to be sorted in order to be able to list all the movies in alphabetical order,

the most instantly possible, even with an important number of movies.

Of course, having an already ordered view would have been the most efficient.

Furthermore, since my last posting, I have implemented a third solution: I have added a field

im my first view corresponding to the index of the next movie in the alphabetical order, thus creating

same sort of linked list.

Everything is quite normal as the rest of the database is concerned: I reference movies in the dictionary

thanks to their index, the fact that they are unordered having no incidence.

But when I want to do this special thing which is to list all movies orderly, I begin by the first row and

then follow down the path given by this additional field.

Without entering into all details, adding a movie goes like this: I append the movie at the end of the view;

if this movie is alphabetically before the first row, I permute the two rows; otherwise, I randomly pick up 50 movie titles and keep

the closest above my new movie title; from this movie I go down the track until I find the place where my new movie should be

inserted; I simply make my new movie point toward the movie pointed by the preceding movie , and then make the preceding

movie point toward my new movie. The row pointed by the last movie is -1.

Although this may seem complicated, it proves itself more efficient to manage than an ordered view.


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1246303962-1006243887=:50488-- From baspey@yahoo.com Tue Nov 20 18:30:22 2001 Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA17144 for ; Tue, 20 Nov 2001 18:30:21 -0600 Message-ID: <20011121002307.47859.qmail@web11501.mail.yahoo.com> Received: from [62.147.34.236] by web11501.mail.yahoo.com via HTTP; Tue, 20 Nov 2001 16:23:07 PST Date: Tue, 20 Nov 2001 16:23:07 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1608752103-1006302187=:43744" Subject: [Metakit] Using hash value as a reference (??) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1608752103-1006302187=:43744 Content-Type: text/plain; charset=us-ascii Before someone tells me: I had missed the function RemapWith(), and I guess that it will make my solution neater. --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1608752103-1006302187=:43744 Content-Type: text/html; charset=us-ascii

Before someone tells me:

I had missed the function RemapWith(), and I guess that it will make my solution neater.


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1608752103-1006302187=:43744-- From baspey@yahoo.com Wed Nov 21 01:09:43 2001 Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA02064 for ; Wed, 21 Nov 2001 01:09:43 -0600 Message-ID: <20011121070228.5511.qmail@web11501.mail.yahoo.com> Received: from [62.147.39.8] by web11501.mail.yahoo.com via HTTP; Tue, 20 Nov 2001 23:02:28 PST Date: Tue, 20 Nov 2001 23:02:28 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-2061439712-1006326148=:5058" Subject: [Metakit] Using hash value as a reference (??) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-2061439712-1006326148=:5058 Content-Type: text/plain; charset=us-ascii I most certainly soliloquize, but in the dubious case somebody followed my history with interest, here are the latest news (otherwise, sorry for the pollution): RemapWith() will certainly be very helpful when it comes to dictionaries but, in the end, I won't use it to order movie titles, and will stick with my method, because it is faster when inserting new movies (single permutation instead of moving down a bunch of rows). --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-2061439712-1006326148=:5058 Content-Type: text/html; charset=us-ascii

I most certainly soliloquize, but in the dubious case somebody followed my history

with interest, here are the latest news (otherwise, sorry for the pollution):

RemapWith() will certainly be very helpful when it comes to dictionaries but, in the

end, I won't use it to order movie titles, and will stick with my method, because it is faster

when inserting new movies (single permutation instead of moving down a bunch of rows).


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-2061439712-1006326148=:5058-- From baspey@yahoo.com Thu Nov 22 01:04:31 2001 Received: from web11505.mail.yahoo.com (web11505.mail.yahoo.com [216.136.172.37]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA24587 for ; Thu, 22 Nov 2001 01:04:30 -0600 Message-ID: <20011122065709.35661.qmail@web11505.mail.yahoo.com> Received: from [62.147.37.252] by web11505.mail.yahoo.com via HTTP; Wed, 21 Nov 2001 22:57:09 PST Date: Wed, 21 Nov 2001 22:57:09 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1063177582-1006412229=:33046" Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-1063177582-1006412229=:33046 Content-Type: text/plain; charset=us-ascii I am very disappointed by the performance I observe (hope I'm doing something wrong). I use Visual C++ under Windows ME, with the latest CVS depository. I have a blocked/hash view, in which each row consists of 5 string properties, 1 int property and 1 view property. Rows are always added at the end of the view, after having used Find() to check that they're not already present. Finally, I commit every 1000 rows added. And here is what I observe: For the first thousand rows => one Find() takes about 1200 CPU cycles Commit() takes about 170,000 CPU cycles this is satisfying ... except that it degrades steadily ! For the 30th thousand rows => one Find() takes about 1900 CPU cycles Commit() takes about 1,400,000 CPU cycles Since I have close to 300,000 rows to add, that will take ages (days in fact). 30,000 rows only means a database of 1,8 Mo, so I guess that 300,000 rows would mean a 20 Mo database. An unmanageable database of only 20 Mo, this can't be. I have checked that find() was not performing a linear search, but really used the hash value, inducing a theorical O(1) cost.: c4_View::Find() calls c4_CustomSeq::RestrictSearch(), which calls c4_HashViewer::LoopUp(), which calls c4_HashViewer::CalcHash() and c4_HashViewer::LookDict(). Pleeeaase someone tell me what I'm doing wrooong ! --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1063177582-1006412229=:33046 Content-Type: text/html; charset=us-ascii

I am very disappointed by the performance I observe (hope I'm doing something

wrong).

I use Visual C++ under Windows ME, with the latest CVS depository.

I have a blocked/hash view, in which each row consists of 5 string properties,

1 int property and 1 view property.

Rows are always added at the end of the view, after having used Find() to check

that they're not already present.

Finally, I commit every 1000 rows added.

And here is what I observe:

For the first thousand rows => one Find() takes about 1200 CPU cycles

                                             Commit() takes about 170,000 CPU cycles

this is satisfying ... except that it degrades steadily !

For the 30th thousand rows => one Find() takes about 1900 CPU cycles

                                             Commit() takes about 1,400,000 CPU cycles

Since I have close to 300,000 rows to add, that will take ages (days in fact).

30,000 rows only means a database of 1,8 Mo, so I guess that 300,000 rows

would mean a 20 Mo database. An unmanageable database of only 20 Mo,

this can't be.

I have checked that find() was not performing a linear search, but really used

the hash value, inducing a theorical O(1) cost.:

c4_View::Find() calls c4_CustomSeq::RestrictSearch(),

which calls c4_HashViewer::LoopUp(),

which calls c4_HashViewer::CalcHash() and c4_HashViewer::LookDict().

Pleeeaase someone tell me what I'm doing wrooong !




Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-1063177582-1006412229=:33046-- From jcw@equi4.com Thu Nov 22 04:17:24 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA32533 for ; Thu, 22 Nov 2001 04:17:24 -0600 Received: from takkie.equi4.local ([195.121.232.20]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GN750C00.J4Z for ; Thu, 22 Nov 2001 10:57:48 +0100 Received: from [10.0.1.32] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAM9uH023438 for ; Thu, 22 Nov 2001 10:56:18 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Performance can't be that bad Date: Thu, 22 Nov 2001 10:40:07 +0100 Message-Id: <20011122094007.15865@mail.planet.nl> In-Reply-To: <20011122065709.35661.qmail@web11505.mail.yahoo.com> References: <20011122065709.35661.qmail@web11505.mail.yahoo.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Pascal Baspeyras wrote: (Is there a way to alter your email settings? Your text comes out double-spaced all the time - it's quite tedious to read...) >I have a blocked/hash view, in which each row consists of 5 string properties, >1 int property and 1 view property. >Rows are always added at the end of the view, after having used Find() to >check that they're not already present. >Finally, I commit every 1000 rows added. >Since I have close to 300,000 rows to add, that will take ages (days in fact). >30,000 rows only means a database of 1,8 Mo, so I guess that 300,000 rows >would mean a 20 Mo database. An unmanageable database of only 20 Mo, >this can't be. That's 60 bytes of data per row, each 5 strings, an int and a subview. The subview may be killing performance. How's speed when you leave it out of the definition? -jcw From baspey@yahoo.com Thu Nov 22 10:37:42 2001 Received: from web11501.mail.yahoo.com (web11501.mail.yahoo.com [216.136.172.46]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA17485 for ; Thu, 22 Nov 2001 10:37:42 -0600 Message-ID: <20011122163021.73054.qmail@web11501.mail.yahoo.com> Received: from [62.147.35.252] by web11501.mail.yahoo.com via HTTP; Thu, 22 Nov 2001 08:30:21 PST Date: Thu, 22 Nov 2001 08:30:21 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-328133064-1006446621=:68541" Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-328133064-1006446621=:68541 Content-Type: text/plain; charset=us-ascii Thanks a lot Jean-Claude for your reply. >(Is there a way to alter your email settings? Your text comes out >double-spaced all the time - it's quite tedious to read...) I will try, but I don't control much of anything... >That's 60 bytes of data per row, each 5 strings, an int and a subview. Yes indeed. For this test, I only provide 2 strings and 1 int as data.The other fields are left untouched, the subview included. >The subview may be killing performance. How's speed when you leave it >out of the definition? I suppressed the subview from the structure but still got the same result, whichis good news in a sense.I ran the program on another computer, but still had the same result, only slowerbecause it's a slower computer (convert days into weeks ;-) )I used MyView = MyStorage.GetAs() instead of MyView = MyStorage.View() andstill got the same.I tried stopping the program, launch it again and resume the additions, but performancekept on where it left. I run out of ideas... Thanks for your attention. --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-328133064-1006446621=:68541 Content-Type: text/html; charset=us-ascii
Thanks a lot Jean-Claude for your reply.
 
>(Is there a way to alter your email settings?  Your text comes out
>double-spaced all the time - it's quite tedious to read...)
I will try, but I don't control much of anything...
 
>That's 60 bytes of data per row, each 5 strings, an int and a subview.
 
Yes indeed. For this test, I only provide 2 strings and 1 int as data.
The other fields are left untouched, the subview included.
 
>The subview may be killing performance.  How's speed when you leave it
>out of the definition?
 
I suppressed the subview from the structure but still got the same result, which
is good news in a sense.
I ran the program on another computer, but still had the same result, only slower
because it's a slower computer (convert days into weeks ;-)  )
I used MyView = MyStorage.GetAs() instead of MyView = MyStorage.View() and
still got the same.
I tried stopping the program, launch it again and resume the additions, but performance
kept on where it left.
 
I run out of ideas...
 
Thanks for your attention.


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-328133064-1006446621=:68541-- From steve@improvision.com Thu Nov 22 10:56:13 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA18557 for ; Thu, 22 Nov 2001 10:56:12 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id QAA14879 for ; Thu, 22 Nov 2001 16:46:52 GMT Received: from [192.168.1.24] ([192.168.1.24]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Thu, 22 Nov 2001 16:48:51 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Thu, 22 Nov 2001 16:48:49 +0000 Subject: Re: [Metakit] Performance can't be that bad From: Stephen Baxter To: Pascal Baspeyras , Message-ID: In-Reply-To: <20011122163021.73054.qmail@web11501.mail.yahoo.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: How about using a blocked view - might that stop Commit() slowing down as you add more and more rows? -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Pascal Baspeyras > Date: Thu, 22 Nov 2001 08:30:21 -0800 (PST) > To: metakit@equi4.com > Subject: [Metakit] Performance can't be that bad > > Thanks a lot Jean-Claude for your reply. >(Is there a way to alter your email > settings? Your text comes out >> double-spaced all the time - it's quite tedious to read...) > I will try, but I don't control much of anything... >That's 60 bytes of data > per row, each 5 strings, an int and a subview. Yes indeed. For this test, I > only provide 2 strings and 1 int as data.The other fields are left untouched, > the subview included. >The subview may be killing performance. How's speed > when you leave it >> out of the definition? I suppressed the subview from the structure but still >> got the same result, whichis good news in a sense.I ran the program on >> another computer, but still had the same result, only slowerbecause it's a >> slower computer (convert days into weeks ;-) )I used MyView = >> MyStorage.GetAs() instead of MyView = MyStorage.View() andstill got the >> same.I tried stopping the program, launch it again and resume the additions, >> but performancekept on where it left. I run out of ideas... Thanks for your >> attention. > > > --------------------------------- > Do You Yahoo!? > Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. From jcw@equi4.com Thu Nov 22 11:40:13 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA21228 for ; Thu, 22 Nov 2001 11:40:12 -0600 Received: from takkie.equi4.local ([195.121.235.215]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GN7Q1X00.M2G for ; Thu, 22 Nov 2001 18:32:21 +0100 Received: from [10.0.1.32] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAMHUf025628 for ; Thu, 22 Nov 2001 18:30:55 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Performance can't be that bad Date: Thu, 22 Nov 2001 18:32:31 +0100 Message-Id: <20011122173231.18514@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >How about using a blocked view - might that stop Commit() slowing down as >you add more and more rows? Indeed. I believe he mentioned blocked. But I hope it's blocked in the main view, not the hash map (which is going to be just a bunch of fast int vectors). If the hash map persists, that is (this is optional). The other thing that might be happening is that the commit causes a full hash map reconstruction. This could be tested by removing the hash before a commit and timing to see whether it makes a difference until commit returns. -jcw From Murat_Berk@bmc.com Thu Nov 22 12:32:44 2001 Received: from creeper.bmc.com (firebird.bmc.com [198.207.223.228]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA24269 for ; Thu, 22 Nov 2001 12:32:44 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2/8.10.2) with ESMTP id fAMIOWG25557; Thu, 22 Nov 2001 12:24:32 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Thu, 22 Nov 2001 12:24:45 -0600 Message-ID: From: "Berk, Murat" To: "'Stephen Baxter'" , Pascal Baspeyras , metakit@equi4.com Subject: RE: [Metakit] Performance can't be that bad Date: Thu, 22 Nov 2001 12:24:43 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We had similar problems with insertion speed and find/search performace. We find ways around those issues. (We are using 2.01 release) 1) We don't use the Find and Search stuff in the Metakit for string stuff. We wrote custom Find and Search functions which deals with these issues. We figured out that c4_StringRef is very fast and we are using it to access data w/o extracting it out. bool Mk_Utils::Utf8StringIsEqualC4Property(const char* utf8_ptr, size_t len, const c4_StringProp& p, const c4_RowRef& r) { c4_Bytes result; c4_StringRef strRef = p(r); strRef.GetData(result); if ( (result.Size()-1) != (int)len ) return false; const char* ptr1 = result.Size() > 0 ? (const char*) result.Contents() : ""; for ( size_t j = 0; j < len; j++ ) if ( utf8_ptr[j] != ptr1[j] ) return false; return true; } Then we copied Search algorithm from the metakit sources and together they perform much faster overall. I do not remember exact Quantify #'s but most of the time was passing constructing these c4_String objects and in the == operator. 2) Preallocate rows before you insert anything. We allocate 512 rows at a time. But initially it was very slow to add strings into these views. The reason is the internal offset array is reallocated everytime and grows 1 at a time. What we did was set the last row of the preallocated bunch to a dummy value. What this does is allocate all the offset stuff for all 512 rows and also leaves preallocated memory called slack in the data. When you insert new data, it just shifts slack pointer around and works very fast. 3) We really did not have problem w/ the speed of the commits. The only problem we are having is commit uses to much of c4_String which slows it down. 4) We have some memory leaks for the memory allocated in c4_String for nullVector. Everywhere do we use locks around the metakit (1 per open database and we have a lot of databases open at any given time). This works great except the reference counting for the c4_String objects which uses internal nullVector. Since that is shared by all the databases and not protected across threads, we do see reference counting problems and it leaks. I was looking removing all that stuff and using a special value for refcount (like (char)-1) to indicate we do not need to reference count, but it is not much a problem right now. -----Original Message----- From: Stephen Baxter [mailto:steve@improvision.com] Sent: Thursday, November 22, 2001 10:49 AM To: Pascal Baspeyras; metakit@equi4.com Subject: Re: [Metakit] Performance can't be that bad How about using a blocked view - might that stop Commit() slowing down as you add more and more rows? -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Pascal Baspeyras > Date: Thu, 22 Nov 2001 08:30:21 -0800 (PST) > To: metakit@equi4.com > Subject: [Metakit] Performance can't be that bad > > Thanks a lot Jean-Claude for your reply. >(Is there a way to alter your email > settings? Your text comes out >> double-spaced all the time - it's quite tedious to read...) > I will try, but I don't control much of anything... >That's 60 bytes of data > per row, each 5 strings, an int and a subview. Yes indeed. For this test, I > only provide 2 strings and 1 int as data.The other fields are left untouched, > the subview included. >The subview may be killing performance. How's speed > when you leave it >> out of the definition? I suppressed the subview from the structure but still >> got the same result, whichis good news in a sense.I ran the program on >> another computer, but still had the same result, only slowerbecause it's a >> slower computer (convert days into weeks ;-) )I used MyView = >> MyStorage.GetAs() instead of MyView = MyStorage.View() andstill got the >> same.I tried stopping the program, launch it again and resume the additions, >> but performancekept on where it left. I run out of ideas... Thanks for your >> attention. > > > --------------------------------- > Do You Yahoo!? > Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From baspey@yahoo.com Thu Nov 22 12:35:51 2001 Received: from web11504.mail.yahoo.com (web11504.mail.yahoo.com [216.136.172.36]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA24451 for ; Thu, 22 Nov 2001 12:35:50 -0600 Message-ID: <20011122182829.43424.qmail@web11504.mail.yahoo.com> Received: from [62.147.35.96] by web11504.mail.yahoo.com via HTTP; Thu, 22 Nov 2001 10:28:29 PST Date: Thu, 22 Nov 2001 10:28:29 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-2014375866-1006453709=:39863" Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --0-2014375866-1006453709=:39863 Content-Type: text/plain; charset=us-ascii >I believe he mentioned blocked. But I hope it's blocked in the main view, not >the hash map>If the hash map persists, that is. The hash map persists.I first build an empty structure making: c4_View View = Storage.GetAs("Movies[_B[Title:S,Year:S]]"); c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]"); And when it comes to the part of the program where I add rows, I do: c4_View View = Storage.View("Movies"); c4_View ViewBlocked = View.Blocked(); c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]"); c4_View ViewBlockedHash = ViewBlocked.Hash(Map, 2); c4_StringProp pTitle("Title"); c4_StringProp pYear("Year"); And from this point, I make all operations on ViewBlockedHash , i.e: ViewBlockedHash.Add(pTitle["Les bronzés font du ski"] + pYear["1979"]); or ViewBlockedHash.Find(pTitle["Les bronzés font du ski"] + pYear["1979"]); It warms my heart to see you all trying to help. P.S: I missed it completely for the mail format. All my excuses. --------------------------------- Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-2014375866-1006453709=:39863 Content-Type: text/html; charset=us-ascii
>I believe he mentioned blocked.  But I hope it's blocked in the main view, not
>the hash map
>If the hash map persists, that is.
 
The hash map persists.
I first build an empty structure making:
    c4_View View = Storage.GetAs("Movies[_B[Title:S,Year:S]]");
    c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]");
 
And when it comes to the part of the program where I add rows, I do:
    c4_View View = Storage.View("Movies");
    c4_View ViewBlocked = View.Blocked();
    c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]");
    c4_View ViewBlockedHash = ViewBlocked.Hash(Map, 2);
 
    c4_StringProp pTitle("Title");
    c4_StringProp pYear("Year");
 
 
 
And from this point, I make all operations on ViewBlockedHash , i.e:
    ViewBlockedHash.Add(pTitle["Les bronzés font du ski"] + pYear["1979"]);
    or ViewBlockedHash.Find(pTitle["Les bronzés font du ski"] + pYear["1979"]);
 
It warms my heart to see you all trying to help.
 
P.S: I missed it completely for the mail format. All my excuses.


Do You Yahoo!?
Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. --0-2014375866-1006453709=:39863-- From steve@improvision.com Thu Nov 22 12:40:07 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA24688 for ; Thu, 22 Nov 2001 12:40:06 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA23330 for ; Thu, 22 Nov 2001 18:30:47 GMT Received: from [192.168.1.24] ([192.168.1.24]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Thu, 22 Nov 2001 18:32:45 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Thu, 22 Nov 2001 18:32:45 +0000 Subject: Re: [Metakit] Performance can't be that bad From: Stephen Baxter To: "Berk, Murat" , Pascal Baspeyras , Message-ID: In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It would be *really* nice if Metakit did this internally, allocating rows say 512 at a time. This is what std::vector does, it makes a big difference to speed when adding items to an array. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: "Berk, Murat" > Date: Thu, 22 Nov 2001 12:24:43 -0600 > To: "'Stephen Baxter'" , Pascal Baspeyras > , metakit@equi4.com > Subject: RE: [Metakit] Performance can't be that bad > > 2) Preallocate rows before you insert anything. We allocate 512 rows at a > time. But initially it was very slow to add strings into these views. The > reason is the internal offset array is reallocated everytime and grows 1 at > a time. What we did was set the last row of the preallocated bunch to a > dummy value. What this does is allocate all the offset stuff for all 512 > rows and also leaves preallocated memory called slack in the data. When you > insert new data, it just shifts slack pointer around and works very fast. From baspey@yahoo.com Thu Nov 22 13:46:51 2001 Received: from web11502.mail.yahoo.com (web11502.mail.yahoo.com [216.136.172.47]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id NAA27895 for ; Thu, 22 Nov 2001 13:46:51 -0600 Message-ID: <20011122193930.82490.qmail@web11502.mail.yahoo.com> Received: from [62.147.38.85] by web11502.mail.yahoo.com via HTTP; Thu, 22 Nov 2001 11:39:30 PST Date: Thu, 22 Nov 2001 11:39:30 -0800 (PST) From: Pascal Baspeyras To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: My mails are getting worsest. It's beyond my will, I swear. What Berk Murat says about c4_String object seems quite interesting, even though I haven't digested everything yet. But I think he missed one issue: I don't use Find() for brute-force linear search. I use it in conjonction with a hash view. What worries me the most about Find() performance is that it quickly requires 2 more time to perform (and then 3, 4 more times) when it is supposed to perform at O(1) cost. __________________________________________________ Do You Yahoo!? Yahoo! GeoCities - quick and easy web site hosting, just $8.95/month. http://geocities.yahoo.com/ps/info1 From baspey@yahoo.com Thu Nov 22 17:06:33 2001 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id RAA05428 for ; Thu, 22 Nov 2001 17:06:33 -0600 Received: from toulouse-1-a7-32-1.dial.proxad.net (HELO desktop) (62.147.32.1) by smtp.mail.vip.sc5.yahoo.com with SMTP; 22 Nov 2001 22:59:10 -0000 Message-ID: <000f01be166b$8b8036e0$0120933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sun, 22 Nov 1998 23:57:55 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_000A_01BE1673.EC0EDE00" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_000A_01BE1673.EC0EDE00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I repost 2 of my previous messages which where completely unreadable. Hope that will work this time... Message1: >That's 60 bytes of data per row, each 5 strings, an int and a subview. Yes indeed. For this test, I only provide 2 strings and 1 int as data. The other fields are left untouched, the subview included. >The subview may be killing performance. How's speed when you leave it >out of the definition? I suppressed the subview from the structure but still got the same = result, which is good news in a sense. I ran the program on another computer, but still had the same result, = only slower because it's a slower computer (convert days into weeks ;-) ) I used MyView =3D MyStorage.GetAs() instead of MyView =3D = MyStorage.View() and still got the same. I tried stopping the program, launch it again and resume the additions, = but performance kept on where it left. Message2: >I believe he mentioned blocked. But I hope it's blocked in the main = view, not=20 >the hash map >If the hash map persists, that is.=20 The hash map persists.I first build an empty structure making: c4_View View =3D Storage.GetAs("Movies[_B[Title:S,Year:S]]"); c4_View Map =3D Storage.GetAs("MoviesMap[_H:I,_R:I]"); And when it comes to the part of the program where I add rows, I do: c4_View View =3D Storage.View("Movies"); c4_View ViewBlocked =3D View.Blocked(); c4_View Map =3D Storage.GetAs("MoviesMap[_H:I,_R:I]"); c4_View ViewBlockedHash =3D ViewBlocked.Hash(Map, 2); c4_StringProp pTitle("Title"); c4_StringProp pYear("Year"); And from this point, I make all operations on ViewBlockedHash , i.e: ViewBlockedHash.Add(pTitle["Les bronz=E9s font du ski"] + = pYear["1979"]); or ViewBlockedHash.Find(pTitle["Les bronz=E9s font du ski"] + = pYear["1979"]); =20 ------=_NextPart_000_000A_01BE1673.EC0EDE00 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I repost 2 of my previous messages = which where=20 completely unreadable.
Hope that will work this time...
 
Message1:
 
>That's 60 bytes of data per row, each 5 strings, an int and a=20 subview.
 
Yes indeed. For this test, I only = provide 2 strings=20 and 1 int as data.
The other fields are left untouched, = the subview=20 included.
 
>The subview may be killing performance.  How's speed when = you=20 leave it
>out of the definition?
 
I suppressed the subview from the = structure but=20 still got the same result, which
is good news in a sense.
I ran the program on another computer, = but still=20 had the same result, only slower
because it's a slower computer (convert = days into=20 weeks ;-)  )
I used MyView =3D MyStorage.GetAs() = instead of MyView=20 =3D MyStorage.View() and
still got the same.
I tried stopping the program, launch it = again and resume the additions, but performance
kept on where it left.
 
 
Message2:
 
>I believe he mentioned blocked.  But I hope it's blocked = in the=20 main view, not
>the hash map
>If the hash map persists, that is.
 
The hash map persists.I first build an empty structure = making:
    c4_View View =3D=20 Storage.GetAs("Movies[_B[Title:S,Year:S]]");
    c4_View Map =3D=20 Storage.GetAs("MoviesMap[_H:I,_R:I]");
 
 And when it comes to the part of the program where I add = rows, I=20 do:
    c4_View View =3D Storage.View("Movies");
    c4_View ViewBlocked =3D View.Blocked();
    c4_View Map =3D=20 Storage.GetAs("MoviesMap[_H:I,_R:I]");
    c4_View ViewBlockedHash =3D = ViewBlocked.Hash(Map,=20 2);
     c4_StringProp pTitle("Title");
    c4_StringProp pYear("Year");
 
And from this point, I make all operations on ViewBlockedHash , = i.e:
    ViewBlockedHash.Add(pTitle["Les bronz=E9s font = du ski"] +=20 pYear["1979"]);
    or ViewBlockedHash.Find(pTitle["Les bronz=E9s = font du=20 ski"] + pYear["1979"]);

 
------=_NextPart_000_000A_01BE1673.EC0EDE00-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Thu Nov 22 18:06:15 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id SAA08283 for ; Thu, 22 Nov 2001 18:06:14 -0600 Received: from toulouse-2-a7-37-162.dial.proxad.net (HELO desktop) (62.147.37.162) by smtp.mail.vip.sc5.yahoo.com with SMTP; 22 Nov 2001 23:58:51 -0000 Message-ID: <001501be1673$e2179900$a225933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Mon, 23 Nov 1998 00:57:35 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0012_01BE167C.4239E9C0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0012_01BE167C.4239E9C0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable =20 I repost 2 of my previous messages which where completely unreadable. Hope that will work this time... Message1: >That's 60 bytes of data per row, each 5 strings, an int and a subview. Yes indeed. For this test, I only provide 2 strings and 1 int as data. The other fields are left untouched, the subview included. >The subview may be killing performance. How's speed when you leave it >out of the definition? I suppressed the subview from the structure but still got the same = result, which is good news in a sense. I ran the program on another computer, but still had the same result, = only slower because it's a slower computer (convert days into weeks ;-) ) I used MyView =3D MyStorage.GetAs() instead of MyView =3D = MyStorage.View() and still got the same. I tried stopping the program, launch it again and resume the additions, = but performance kept on where it left. Message2: >I believe he mentioned blocked. But I hope it's blocked in the main = view, not=20 >the hash map >If the hash map persists, that is.=20 The hash map persists.I first build an empty structure making: c4_View View =3D Storage.GetAs("Movies[_B[Title:S,Year:S]]"); c4_View Map =3D Storage.GetAs("MoviesMap[_H:I,_R:I]"); And when it comes to the part of the program where I add rows, I do: c4_View View =3D Storage.View("Movies"); c4_View ViewBlocked =3D View.Blocked(); c4_View Map =3D Storage.GetAs("MoviesMap[_H:I,_R:I]"); c4_View ViewBlockedHash =3D ViewBlocked.Hash(Map, 2); c4_StringProp pTitle("Title"); c4_StringProp pYear("Year"); And from this point, I make all operations on ViewBlockedHash , i.e: ViewBlockedHash.Add(pTitle["Les bronz=E9s font du ski"] + = pYear["1979"]); or ViewBlockedHash.Find(pTitle["Les bronz=E9s font du ski"] + = pYear["1979"]); ------=_NextPart_000_0012_01BE167C.4239E9C0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
 
I repost 2 of my previous messages = which where=20 completely unreadable.
Hope that will work this time...
 
Message1:
 
>That's 60 bytes of data per row, each 5 strings, an int and a=20 subview.
 
Yes indeed. For this test, I only = provide 2 strings=20 and 1 int as data.
The other fields are left untouched, = the subview=20 included.
 
>The subview may be killing performance.  How's speed when = you=20 leave it
>out of the definition?
 
I suppressed the subview from the = structure but=20 still got the same result, which
is good news in a sense.
I ran the program on another computer, = but still=20 had the same result, only slower
because it's a slower computer (convert = days into=20 weeks ;-)  )
I used MyView =3D MyStorage.GetAs() = instead of MyView=20 =3D MyStorage.View() and
still got the same.
I tried stopping the program, launch it = again and resume the additions, but performance
kept on where it left.
 
 
Message2:
 
>I believe he mentioned blocked.  But I hope it's blocked = in the=20 main view, not
>the hash map
>If the hash map persists, that is.
 
The hash map persists.I first build an empty structure = making:
    c4_View View =3D=20 Storage.GetAs("Movies[_B[Title:S,Year:S]]");
    c4_View Map =3D=20 Storage.GetAs("MoviesMap[_H:I,_R:I]");
 
 And when it comes to the part of the program where I add = rows, I=20 do:
    c4_View View =3D Storage.View("Movies");
    c4_View ViewBlocked =3D View.Blocked();
    c4_View Map =3D=20 Storage.GetAs("MoviesMap[_H:I,_R:I]");
    c4_View ViewBlockedHash =3D = ViewBlocked.Hash(Map,=20 2);
     c4_StringProp pTitle("Title");
    c4_StringProp pYear("Year");
 
And from this point, I make all operations on ViewBlockedHash , = i.e:
    ViewBlockedHash.Add(pTitle["Les bronz=E9s font = du ski"] +=20 pYear["1979"]);
    or ViewBlockedHash.Find(pTitle["Les bronz=E9s = font du=20 ski"] + pYear["1979"]);
------=_NextPart_000_0012_01BE167C.4239E9C0-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jcw@equi4.com Fri Nov 23 04:50:10 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA28139 for ; Fri, 23 Nov 2001 04:50:09 -0600 Received: from takkie.equi4.local ([195.121.234.170]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GN91NU00.H8O for ; Fri, 23 Nov 2001 11:40:42 +0100 Received: from [10.0.1.33] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fANAdC030062 for ; Fri, 23 Nov 2001 11:39:21 +0100 From: Jean-Claude Wippler To: Metakit Subject: Re: [Metakit] Performance can't be that bad Date: Fri, 23 Nov 2001 11:20:35 +0100 Message-Id: <20011123102035.3201@mail.planet.nl> In-Reply-To: <001501be1673$e2179900$a225933e@desktop> References: <001501be1673$e2179900$a225933e@desktop> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id EAA28139 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Pascal, >I repost 2 of my previous messages which where completely unreadable. And this one was dated 1998, meaning I had to chase it in my mail box :) >The hash map persists.I first build an empty structure making: > c4_View View = Storage.GetAs("Movies[_B[Title:S,Year:S]]"); > c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]"); > > And when it comes to the part of the program where I add rows, I do: > c4_View View = Storage.View("Movies"); > c4_View ViewBlocked = View.Blocked(); > c4_View Map = Storage.GetAs("MoviesMap[_H:I,_R:I]"); > c4_View ViewBlockedHash = ViewBlocked.Hash(Map, 2); > c4_StringProp pTitle("Title"); > c4_StringProp pYear("Year"); Looks ok. You're using the latest code from CVS, right? (Not 2.4.1, it has a bad hashing bug!). Why are you saving Year as string? Int would be much faster. What happens if you do not do commit at all, how's performance as things grow? >And from this point, I make all operations on ViewBlockedHash , i.e: > ViewBlockedHash.Add(pTitle["Les bronzés font du ski"] + pYear["1979"]); > or ViewBlockedHash.Find(pTitle["Les bronzés font du ski"] + >pYear["1979"]); (C'est des livres, ou de la musique, ta librarie?) -jcw From baspey@yahoo.com Fri Nov 23 10:26:18 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA11344 for ; Fri, 23 Nov 2001 10:26:18 -0600 Received: from toulouse-2-a7-38-109.dial.proxad.net (HELO desktop) (62.147.38.109) by smtp.mail.vip.sc5.yahoo.com with SMTP; 23 Nov 2001 16:18:52 -0000 Message-ID: <001501c1743a$581ceb60$6d26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 23 Nov 2001 17:16:18 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0010_01C17442.90546C80" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0010_01C17442.90546C80 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I do hope my nightmarish experience with mails has come to an end ! JCW kindly wrote: >You're using the latest code from CVS, right? Right. >Why are you saving Year as string? Int would be much faster. I'll of course do that later, but for the sake of the test, it should = not matter. >What happens if you do not do commit at all, how's performance as = things grow? As things grow, performance is bad. The program very rapidly slows down = because Find() is called for each new value entered, and Find() takes fastly = increasing time. Commit() is one big issue, and Find() is another one. I could understand Commit() takes increasing time (though I don't like = it), but Find() should not slow down at such a noticeable pace. It's a hash lookup ! (Voyons Jean-Claude... Les bronz=E9s font du ski, c'est pas de la = litt=E9rature) ------=_NextPart_000_0010_01C17442.90546C80 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
I do hope my nightmarish experience = with mails has=20 come to an end !
 
JCW kindly wrote:
>You're using the latest code from = CVS,=20 right?
 
Right.
 
>Why are you saving Year as string?  Int would be much=20 faster.
 
I'll of course do that later, but for the sake of the test, it = should not=20 matter.
 
>What = happens if you=20 do not do commit at all, how's performance as things=20 grow?
As things grow, performance is = bad. The=20 program very rapidly slows down because
Find() is called for each new = value entered,=20 and Find() takes fastly increasing time.
Commit() is one big issue, and = Find() is=20 another one.
I could understand Commit() takes = increasing time=20 (though I don't like it), but Find()
should not slow down at such a = noticeable pace.=20 It's a hash lookup !
 
 
(Voyons Jean-Claude... Les = bronz=E9s font du=20 ski, c'est pas de la litt=E9rature)
------=_NextPart_000_0010_01C17442.90546C80-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Fri Nov 23 12:09:47 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA15787 for ; Fri, 23 Nov 2001 12:09:47 -0600 Received: from toulouse-1-a7-35-123.dial.proxad.net (HELO desktop) (62.147.35.123) by smtp.mail.vip.sc5.yahoo.com with SMTP; 23 Nov 2001 18:02:21 -0000 Message-ID: <003901c17448$ccf59b40$c626933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 23 Nov 2001 18:59:53 +0100 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0026_01C17451.093DC2A0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Sending mails the right way Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0026_01C17451.093DC2A0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello, I send my mails to this group using Outlook Express 5 with my mail account on Yahoo! Mail. As I guess you can see, the result is ugly. If I send a mail to myself, the result is perfect, but if I send to this = group, that's a disaster... Could you please tell me how, yourself, you post to this group. ------=_NextPart_000_0026_01C17451.093DC2A0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hello,
 
I send my mails to this group using = Outlook Express=20 5 with my mail
account on Yahoo! Mail.
As I guess you can see, the result is=20 ugly.
If I send a mail to myself, the result = is perfect,=20 but if I send to this group,
that's a disaster...
 
Could you please tell me how, yourself, =  you=20 post to this group.
------=_NextPart_000_0026_01C17451.093DC2A0-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From tciuro@mac.com Fri Nov 23 12:20:40 2001 Received: from smtpout.mac.com (smtpout.mac.com [204.179.120.89]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA16218 for ; Fri, 23 Nov 2001 12:20:39 -0600 Received: from smtp-relay02.mac.com (server-source-si02 [10.13.10.6]) by smtpout.mac.com (8.12.1/8.10.2/1.0) with ESMTP id fANHSrsD018529 for ; Fri, 23 Nov 2001 09:28:53 -0800 (PST) Received: from asmtp01.mac.com ([10.13.10.65]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GN9MM100.P1Q for ; Fri, 23 Nov 2001 10:13:13 -0800 Received: from localhost ([63.202.175.181]) by asmtp01.mac.com (Netscape Messaging Server 4.15 asmtp01 Jun 21 2001 23:53:48) with ESMTP id GN9MM000.40R; Fri, 23 Nov 2001 10:13:12 -0800 Date: Fri, 23 Nov 2001 10:13:12 -0800 Subject: Re: [Metakit] Performance can't be that bad Content-Type: multipart/alternative; boundary=Apple-Mail-1-314704782 Mime-Version: 1.0 (Apple Message framework v501) Cc: Metakit To: Pascal Baspeyras From: Tito Ciuro In-Reply-To: <001501c1743a$581ceb60$6d26933e@desktop> Message-Id: X-Mailer: Apple Mail (2.501) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-314704782 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Salut Pascal, I'd like to share with you a few things I've noticed regarding MetaKit's=20= performance. 1) If your data file grows very quickly (as in commit-extend mode):=20 compact it upon save. With this mode, data grows quickly because a change in a column requires=20= the entire column to be rewritten. In my data file, I know that a=20 particular record is, at the most, 400 bytes. So, when I save, I=20 calculate roughly the size of the record: (file size/number of records).=20= If the resulting record size is greater than, say, 600 bytes, I trigger=20= the compacting method. This has proven to reduce the file size to its=20 minimum size and improve search speeds quite a bit. 2) GetAs() is expensive: cache the view. It's way better to create one and leave it open, then perform all ops=20 with it. Sampling thereafter shows quite a different picture. 3) A hashed view may be your best friend: using it whenever possible. I use my own call rowWithUID() quite a bit. The first column is a unique=20= ID which I use extensively to obtain its row. Using hashed view + Find()=20= has proven to be an *enormous* gain. 4) Cache the hashed view: powerful combo. Apply (1) and (2) above to speed up things a lot. I hope this helps you a bit, -- Tito On Friday, November 23, 2001, at 08:16 AM, Pascal Baspeyras wrote: > I do hope my nightmarish experience with mails has come to an end ! > =A0 > JCW kindly wrote: > >You're using the latest code from CVS, right? > =A0 > Right. > =A0 > >Why are you saving Year as string?=A0 Int would be much faster. > =A0 > I'll of course do that later, but for the sake of the test, it should=20= > not matter. > =A0 > >What happens if you do not do commit at all, how's performance as=20 > things grow? > As things grow, performance is bad. The program very rapidly slows = down=20 > because > Find() is called for each new value entered, and Find() takes fastly=20= > increasing time. > Commit() is one big issue, and Find() is another one. > I could understand Commit() takes increasing time (though I don't like=20= > it), but Find() > should not slow down at such a noticeable pace. It's a hash lookup ! > =A0 > =A0 > (Voyons Jean-Claude... Les bronz=E9s font du ski, c'est pas de la=20 > litt=E9rature) > ____________________________________________ "Have you ever noticed? Anybody going slower than you is an idiot, and anyone going faster than you is a maniac." George Carlin --Apple-Mail-1-314704782 Content-Transfer-Encoding: quoted-printable Content-Type: text/enriched; charset=ISO-8859-1 Salut Pascal, I'd like to share with you a few things I've noticed regarding MetaKit's performance. 1) If your data file grows very quickly (as in commit-extend mode): compact it upon save. With this mode, data grows quickly because a change in a column requires the entire column to be rewritten. In my data file, I know that a particular record is, at the most, 400 bytes. So, when I save, I calculate roughly the size of the record: (file size/number of records). If the resulting record size is greater than, say, 600 bytes, I trigger the compacting method. This has proven to reduce the file size to its minimum size and improve search speeds quite a bit. 2) GetAs() is expensive: cache the view. It's way better to create one and leave it open, then perform all ops with it. Sampling thereafter shows quite a different picture. 3) A hashed view may be your best friend: using it whenever possible. I use my own call rowWithUID() quite a bit. The first column is a unique ID which I use extensively to obtain its row. Using hashed view + Find() has proven to be an *enormous* gain. 4) Cache the hashed view: powerful combo. Apply (1) and (2) above to speed up things a lot. I hope this helps you a bit, -- Tito On Friday, November 23, 2001, at 08:16 AM, Pascal Baspeyras wrote: ArialI do hope my nightmarish experience with mails has come to an end = ! =A0 ArialJCW kindly = wrote: Arial>You're using the latest code from CVS, right? =A0 ArialRight. =A0 >Why are you saving Year as string?=A0 Int would be much faster. =A0 I'll of course do that later, but for the sake of the test, it should not matter. =A0 Arial>What happens if you do not do commit at all, how's performance as things grow? As things grow, performance is bad. The program very rapidly slows down because ArialFind() is called for each new value entered, and Find() takes fastly increasing = time. ArialCommit() is one big issue, and Find() is another one. ArialI could understand Commit() takes increasing time (though I don't like it), but = Find() Arialshould not slow down at such a noticeable pace. It's a hash lookup ! =A0 =A0 Arial(Voyons Jean-Claude... Les bronz=E9s font du ski, c'est pas de la = litt=E9rature) Lucida Grande = ____________________________________________Lucida Grande 0000,0000,FFFF"Have you ever noticed? Anybody going slower than you is an idiot, and anyone going faster than you is a maniac." = 0000,6666,3333 George Carlin= --Apple-Mail-1-314704782-- From baspey@yahoo.com Fri Nov 23 15:04:42 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id PAA23191 for ; Fri, 23 Nov 2001 15:04:41 -0600 Received: from toulouse-2-a7-36-250.dial.proxad.net (HELO desktop) (62.147.36.250) by smtp.mail.vip.sc5.yahoo.com with SMTP; 23 Nov 2001 20:57:14 -0000 Message-ID: <000e01c17461$3a3e5da0$fa24933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 23 Nov 2001 21:54:26 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bonjour amis francophiles, Many thanks to Tito and Berk (I don't like much Jean-Louis Murat either) for giving out their precious optimization secrets. A special thank to Tito, whose mail looked almost as ugly as mines. I finished a program intended to extensively test Metakit performance: I randomly generate string and add them to a blocked/hash view, and I do nothing else, hence no risk of perturbation. With only 1 field, the Commit() is still as bad, but surprisingly, the Find() is correct. Now, I will try with 2 hashed fields. And after that, I will see if changing the commit mode improves things. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From readonly@getsoft.com Fri Nov 23 15:42:57 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA24946 for ; Fri, 23 Nov 2001 15:42:57 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=OMNIBOOK) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 167Nxd-0003kv-00; Fri, 23 Nov 2001 14:34:13 -0700 Date: Sat, 24 Nov 2001 08:35:50 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <77163008358.20011124083550@getsoft.com> To: "Pascal Baspeyras" CC: "Metakit" Subject: Re: [Metakit] Performance can't be that bad In-Reply-To: <000e01c17461$3a3e5da0$fa24933e@desktop> References: <000e01c17461$3a3e5da0$fa24933e@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It seems that many questions posted on the Metakit list revolve around performance issues, and the various hacks people finish up using to to try and get the performance they need for the specific applications. Jean-Claude states that Metakits performance varies dramatically depending on how you use it, which seems to be related to its column oriented design. From my personal experience I spent a solid 6+ weeks trying to use Metakit in an application. I was able to dramatically improve performance in some areas, but got absolutely nowhere in others. For example sorting a view with a large number of rows took far too long. This was earlier in the year, so things may have improved by now. The bottom line for me was that I wasn't able to get either acceptable performance, nor reliable use out of MK. MK is an elegant and interesting DB system, but in my experience it proved too difficult to use, was too unpredictable in its performance and required trying lots of hacks to get it to perform for each different DB design you through at it. I put this all down to a learning experience and moved on, writing a DB of my own with which exhibits very good performance without me worrying about how big it gets. The DB I'm referring to is the Source Database engine at the heart of the Source Browser/Class View in my programmers editor, ED for Windows (www.getsoft.com). I applaud the work Jean-Claude has done, but at the same time people need to determine whether MK is the appropriate solution for them, and consider how much time they are willing to invest in order to make this decision. Saturday, November 24, 2001, 7:54:26 AM, you wrote: PB> Bonjour amis francophiles, PB> Many thanks to Tito and Berk (I don't like much Jean-Louis Murat either) for PB> giving out their precious optimization secrets. PB> A special thank to Tito, whose mail looked almost as ugly as mines. PB> I finished a program intended to extensively test Metakit performance: PB> I randomly generate string and add them to a blocked/hash view, and I do PB> nothing PB> else, hence no risk of perturbation. PB> With only 1 field, the Commit() is still as bad, but surprisingly, the PB> Find() is correct. PB> Now, I will try with 2 hashed fields. PB> And after that, I will see if changing the commit mode improves things. PB> _________________________________________________________ PB> Do You Yahoo!? PB> Get your free @yahoo.com address at http://mail.yahoo.com PB> _______________________________________________ PB> metakit mailing list - metakit@equi4.com PB> http://www.equi4.com/mailman/listinfo/metakit -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From baspey@yahoo.com Fri Nov 23 16:58:46 2001 Received: from smtp010.mail.yahoo.com (smtp010.mail.yahoo.com [216.136.173.30]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA28742 for ; Fri, 23 Nov 2001 16:58:45 -0600 Received: from toulouse-1-a7-34-199.dial.proxad.net (HELO desktop) (62.147.34.199) by smtp.mail.vip.sc5.yahoo.com with SMTP; 23 Nov 2001 22:51:18 -0000 Message-ID: <000701c17471$2a224520$c722933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 23 Nov 2001 23:48:49 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >From my personal experience I spent a solid 6+ weeks trying to use >Metakit in an application. I stick to Metakit mostly because I also spent a lot of time trying to master it. Indeed, I find documentation awful. Information is often somehere, but the question is: where ? In the end, the source code is often the best source of information (what about a real synthesis, not just "wiki.cgi\4_2a.htm" and the prehistorical "intro.htm"). Now that I have decided for a database design according to my needs, I'm terribly disappointed by the performance. By posting to that group, I was hoping someone tells me what blunt mistake I was making. >I put this all down to a learning experience and moved on, writing a >DB of my own with which exhibits very good performance without me >worrying about how big it gets. The DB I'm referring to is the Source >Database engine at the heart of the Source Browser/Class View in my >programmers editor, ED for Windows (www.getsoft.com). I went to your site, and: 1/ ED isn't free nor open source 2/ ED for Windows does it work on Macintosh and Linux ? 3/ ED database format can it go from one platform to another one transparently ? I may indeed give up Metakit and move on to Berkeley DB, but I will first hand my evaluation program to Jean-Claude for him to give his own interpretation. My guess is that Jean-Claude can't make thorough tests of Metakit because users use it in very various ways, and that there are still imperfections in the recent mapping views. But to tell the truth, I still hope that the malfunction is all my fault. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From readonly@getsoft.com Fri Nov 23 17:23:35 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA30325 for ; Fri, 23 Nov 2001 17:23:35 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 167PYB-0007uT-00; Fri, 23 Nov 2001 16:16:03 -0700 Date: Sat, 24 Nov 2001 10:18:17 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <93286188377.20011124101817@getsoft.com> To: "Pascal Baspeyras" CC: "Metakit" Subject: Re: [Metakit] Performance can't be that bad In-Reply-To: <000701c17471$2a224520$c722933e@desktop> References: <000701c17471$2a224520$c722933e@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: PB> Neville Franks wrote: >>From my personal experience I spent a solid 6+ weeks trying to use >>Metakit in an application. PB> I stick to Metakit mostly because I also spent a lot of time trying to PB> master it. PB> Indeed, I find documentation awful. Information is often somehere, but PB> the question is: where ? In the end, the source code is often the best PB> source PB> of information (what about a real synthesis, not just "wiki.cgi\4_2a.htm" PB> and PB> the prehistorical "intro.htm"). Yes I agree the documentation is awful. In my experience this is a common trait of "Free Open Source Software". I have largely lost all interest in Open Source because of issues like this. There are some very nice and interesting projects around, but the cost to try and use them is far, far, far too high. And again in my experience you invest a lot of time only to find out they are unusable for whatever reason. As a wise person said once, "there is no such thing as a free lunch." PB> Now that I have decided for a database design according to my needs, I'm PB> terribly disappointed by the performance. By posting to that group, I was PB> hoping someone tells me what blunt mistake I was making. >>I put this all down to a learning experience and moved on, writing a >>DB of my own with which exhibits very good performance without me >>worrying about how big it gets. The DB I'm referring to is the Source >>Database engine at the heart of the Source Browser/Class View in my >>programmers editor, ED for Windows (www.getsoft.com). PB> I went to your site, and: PB> 1/ ED isn't free nor open source Correct, I'm afraid I can't put food on my table by giving away my work. This is my livelihood, not something I do as a plaything. PB> 2/ ED for Windows does it work on Macintosh and Linux ? The Mac market is too small, and Linux people expect everything to be available for free, and come with source. PB> 3/ ED database format can it go from one platform to another one PB> transparently ? Like ED itself the Database is not meant to be cross platform. At present it is designed specifically meet EDs requirements - ie. fast, large databases. This includes very fast updates, and lookups and full multithreaded support. As you edit your code the database gets updated in real time, without any impact on your editing. PB> I may indeed give up Metakit and move on to Berkeley DB, but I will first PB> hand PB> my evaluation program to Jean-Claude for him to give his own interpretation. I'd suggest also looking at MySQL. I'm using that on my Web server and have been very impressed with its capabilities. That said it is a totally different product to MK. PB> My guess is that Jean-Claude can't make thorough tests of Metakit because PB> users PB> use it in very various ways, and that there are still imperfections in the PB> recent mapping views. PB> But to tell the truth, I still hope that the malfunction is all my fault. That wasn't my experience, as I stated earlier. PB> _________________________________________________________ PB> Do You Yahoo!? PB> Get your free @yahoo.com address at http://mail.yahoo.com PB> _______________________________________________ PB> metakit mailing list - metakit@equi4.com PB> http://www.equi4.com/mailman/listinfo/metakit -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From baspey@yahoo.com Fri Nov 23 20:08:23 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id UAA11643 for ; Fri, 23 Nov 2001 20:08:21 -0600 Received: from toulouse-2-a7-36-94.dial.proxad.net (HELO desktop) (62.147.36.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 02:00:51 -0000 Message-ID: <000e01c1748b$a43e3160$5e24933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 01:22:17 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >Yes I agree the documentation is awful. What I hate the most is that you read things which might have been stated 2 years ago, and never know if a limitation still applies or not. >In my experience this is a common trait of >"Free Open Source Software". "Free Open Source Software" can be really great. What about Linux itself ? I also use wxWindows which I find marvellous. Speaking of wxWindows, I have conversations directly with the authors, which is always an incredible moment, just like when JCW personally answer my mails. Free software writers are passionated, talented, and they generally do a great job. Quality isn't proportional to the price, far from it. >The Mac market is too small. I have chosen wxWindows and Metakit for my program to be available on a maximum of platforms. It's an important requirement. By the way, I hate the word "Market". >I'd suggest also looking at MySQL. MySQL uses its own database system, but can also integrate the Berkeley Database, which is a sign of quality. Berkeley DB has great portability, small footprint and source availability like another database I know (Metakit). Free software writers deserve the greatest indulgence and the greatest gratitude for giving away their free time like this. They are the "Mother Theresa" of the computer world. In fact, I foretell that they will rule it in the end (bye bye M$...) _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jyl@best.com Fri Nov 23 20:13:12 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA12030 for ; Fri, 23 Nov 2001 20:13:02 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id SAA05285; Fri, 23 Nov 2001 18:04:13 -0800 (PST) Message-ID: <3BFF0E75.7020006@best.com> Date: Fri, 23 Nov 2001 19:05:25 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Neville Franks CC: Pascal Baspeyras , Metakit Subject: Re: [Metakit] Performance can't be that bad References: <000e01c17461$3a3e5da0$fa24933e@desktop> <77163008358.20011124083550@getsoft.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Three comments: * I browsed your web site and couldn't find the DB available as a separate product. Is that intentional? * Since I couldn't examine your DB I cannot comment on the performance and features. I dont know whether the DB in your editor is as rich as MK and in fact whether it needs to be as rich as MK, for the uses you intend. * I've used MK extensively and quickly learned to use it efficiently. My use is the simplest possible. I dont use Find(), hashed views, and so forth. I found all derived views to be too slow to use for my purposes, but learned to do without. You can see how I use MK by examining http://e4graph.sourceforge.net, the source code is all available. All MK related functionality is in the subdirectory mkdriver in the source tree. Regards, --JYL >It seems that many questions posted on the Metakit list revolve around >performance issues, and the various hacks people finish up using to to >try and get the performance they need for the specific applications. > >Jean-Claude states that Metakits performance varies dramatically >depending on how you use it, which seems to be related to its column >oriented design. > >>From my personal experience I spent a solid 6+ weeks trying to use >Metakit in an application. I was able to dramatically improve >performance in some areas, but got absolutely nowhere in others. For >example sorting a view with a large number of rows took far too long. >This was earlier in the year, so things may have improved by now. > >The bottom line for me was that I wasn't able to get either acceptable >performance, nor reliable use out of MK. MK is an elegant and >interesting DB system, but in my experience it proved too difficult to >use, was too unpredictable in its performance and required trying lots >of hacks to get it to perform for each different DB design you >through at it. > >I put this all down to a learning experience and moved on, writing a >DB of my own with which exhibits very good performance without me >worrying about how big it gets. The DB I'm referring to is the Source >Database engine at the heart of the Source Browser/Class View in my >programmers editor, ED for Windows (www.getsoft.com). > >I applaud the work Jean-Claude has done, but at the same time people >need to determine whether MK is the appropriate solution for them, and >consider how much time they are willing to invest in order to make this >decision. > > >Saturday, November 24, 2001, 7:54:26 AM, you wrote: > >PB> Bonjour amis francophiles, > >PB> Many thanks to Tito and Berk (I don't like much Jean-Louis Murat either) for >PB> giving out their precious optimization secrets. >PB> A special thank to Tito, whose mail looked almost as ugly as mines. > >PB> I finished a program intended to extensively test Metakit performance: >PB> I randomly generate string and add them to a blocked/hash view, and I do >PB> nothing >PB> else, hence no risk of perturbation. > >PB> With only 1 field, the Commit() is still as bad, but surprisingly, the >PB> Find() is correct. >PB> Now, I will try with 2 hashed fields. >PB> And after that, I will see if changing the commit mode improves things. > >PB> _________________________________________________________ >PB> Do You Yahoo!? >PB> Get your free @yahoo.com address at http://mail.yahoo.com > >PB> _______________________________________________ >PB> metakit mailing list - metakit@equi4.com >PB> http://www.equi4.com/mailman/listinfo/metakit > > From jcw@equi4.com Fri Nov 23 21:10:32 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA16866 for ; Fri, 23 Nov 2001 21:10:32 -0600 Received: from takkie.equi4.local ([195.121.234.41]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNAB4800.MFY for ; Sat, 24 Nov 2001 04:02:32 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAO317002326 for ; Sat, 24 Nov 2001 04:01:17 +0100 From: Jean-Claude Wippler To: Metakit Subject: Re: [Metakit] Performance can't be that bad Date: Sat, 24 Nov 2001 04:03:54 +0100 Message-Id: <20011124030354.7219@mail.planet.nl> In-Reply-To: <000701c17471$2a224520$c722933e@desktop> References: <000701c17471$2a224520$c722933e@desktop> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tsk, tsk, tsk: "it's a bad worker who blames his tools" ... Let's get this right, folks: MK run circles around Berkeley DB and MySQL, under the right circumstances. Feel free to find out for yourself. Others have. And yes, I keep saying that at times MK's performance can go down a *lot*. With so much finger pointing going on, I simply had to go check things. Below are some results, using the Tcl scripting language on a PIII/650 under SuSE Linux 7.3, with 256 MB RAM. C++ ought to perform better. The results - as I see them, and I apologize if I made mistakes: - the whole test, creating 6 files of 500,000 rows, ran in 11 minutes (it consists of adding unique strings, constructed from /usr/dist/words) - I'm only doing a single commit at the end, and have no subviews in here - times below are in millisecs, f1/f2/f5 are flat, b1/b2/b5 are blocked, f1 is 1 key + 1 val, f2 is 2 keys, f5 is 2 keys + 4 vals (ditto for b*) - in the worst case, it takes under 7s to add 10,000 rows (see "b5" below) (blocked, 2-key hash, 6 string props, at 500,000 rows, file size 52 Mb) - in the unblocked case, performance is linear (.75 sec/10k-rows for f2), in the blocked case it degrades by a factor 2x from start to finish - finds take 54 .. 104 microseconds, commits between 1.5 and 25.7 seconds I do not see a significant slowdown for 1-key vs. 2-key hashes. I do not see O(N) behavior anywhere, other than in the number or properties / file size. I'll ask again, Pascal: are you sure that you have the latest code from CVS? The linear slowdown you have been reporting sounds similar to a recent bug, which was resolved resolved (check the change logs). Now can we please get on with it? I can't really be expected to help everyone debug their software, nor even help them learn it, can I? When problems are reported, and that is what this list is for, then there are two equally effective ways to progress: try to figure out things yourself (and let us all know, please!), or construct a small example that illustrates what is happening in a way others can easily reproduce. I'm happy to say that several people have done this in the past, and it has been a great way to fix/improve MK. And for cases where the "problems" are your own: be glad that many will be answered by subscribers on this list, simply because they can spot some mistakes easily. My software is free, and I'm doing just fine commercially thank you, but my time is limited. The deal is very simple: if we all join forces to improve the software and the docs, then I'll do my best to fix issues where I can, and to integrate improvements others offer. And now that there is a rare opportunity to say this: my warmest thanks to all the people who have helped (and continue to help) find/fix problems. You share the credit for making MK what it is today, and for what it will be tomorrow. -jcw P.S. Here is the complete test output, followed by the complete Tcl script: ===================================== ========================================== % ntclkit slow.tcl w = contributory Sat Nov 24 00:37:31 CET 2001 f1: .................................................. 500000 rows, 22890442 b 0: k1 k1_Chapman0 v v_Chapman0 1: k1 k1_beckoning0 v v_beckoning0 2: k1 k1_belittled0 v v_belittled0 f2: .................................................. 500000 rows, 23390449 b 0: k1 k1_Chapman0 k2 k2_Chapman0 v {} 1: k1 k1_beckoning0 k2 k2_beckoning0 v {} 2: k1 k1_belittled0 k2 k2_belittled0 v {} f5: .................................................. 500000 rows, 52893926 b 0: k1 k1_Chapman0 k2 k2_Chapman0 v1 v1_Chapman0 v2 v2_Chapman0 v3 v3_Chapman0 v v_Chapman0 1: k1 k1_beckoning0 k2 k2_beckoning0 v1 v1_beckoning0 v2 v2_beckoning0 v3 v3_beckoning0 v v_beckoning0 2: k1 k1_belittled0 k2 k2_belittled0 v1 v1_belittled0 v2 v2_belittled0 v3 v3_belittled0 v v_belittled0 b1: .................................................. 500000 rows, 22918775 b 0: k1 k1_Chapman0 v v_Chapman0 1: k1 k1_beckoning0 v v_beckoning0 2: k1 k1_belittled0 v v_belittled0 b2: .................................................. 500000 rows, 23420800 b 0: k1 k1_Chapman0 k2 k2_Chapman0 v {} 1: k1 k1_beckoning0 k2 k2_beckoning0 v {} 2: k1 k1_belittled0 k2 k2_belittled0 v {} b5: .................................................. 500000 rows, 52974159 b 0: k1 k1_Chapman0 k2 k2_Chapman0 v1 v1_Chapman0 v2 v2_Chapman0 v3 v3_Chapman0 v v_Chapman0 1: k1 k1_beckoning0 k2 k2_beckoning0 v1 v1_beckoning0 v2 v2_beckoning0 v3 v3_beckoning0 v v_beckoning0 2: k1 k1_belittled0 k2 k2_belittled0 v1 v1_belittled0 v2 v2_belittled0 v3 v3_belittled0 v v_belittled0 Sat Nov 24 00:48:28 CET 2001 b1 b2 b5 f1 f2 f5 cols( 10000) = 1242.50 1304.00 2680.61 777.08 795.82 1479.10 cols( 20000) = 1335.40 1433.63 2929.18 770.77 842.36 1480.55 cols( 30000) = 1448.52 1549.82 3107.09 837.14 907.41 1414.22 cols( 40000) = 1344.77 1441.45 3042.39 741.56 757.63 1464.62 cols( 50000) = 1683.81 1806.10 3453.16 1013.09 1148.84 1566.98 cols( 60000) = 1358.38 1472.10 3180.01 716.69 889.23 1342.18 cols( 70000) = 1394.52 1527.15 3295.67 724.70 783.99 1406.92 cols( 80000) = 1440.27 1556.47 3344.69 735.12 804.01 1379.41 cols( 90000) = 2264.91 2396.75 4200.34 1190.70 1644.88 1940.82 cols(100000) = 1470.20 1610.09 3472.78 713.38 929.40 1319.38 cols(110000) = 1503.52 1637.00 3518.44 723.18 741.28 1318.49 cols(120000) = 1522.37 1667.66 3631.21 718.91 789.59 1322.77 cols(130000) = 1541.93 1702.36 3726.43 724.64 735.77 1325.02 cols(140000) = 1577.06 1741.61 3728.57 725.03 752.13 1336.44 cols(150000) = 1600.60 1768.46 3836.94 732.58 767.03 1349.27 cols(160000) = 1628.90 1807.21 3864.10 738.50 774.70 1351.62 cols(170000) = 1651.83 1853.33 3944.51 755.37 787.41 1357.57 cols(180000) = 3694.33 3905.01 6139.05 1654.60 1680.87 2274.20 cols(190000) = 1660.46 1857.05 3954.51 715.48 751.46 1326.79 cols(200000) = 1670.83 1878.95 4097.30 708.23 751.60 1329.17 cols(210000) = 1702.75 1906.41 4129.13 716.49 745.58 1354.67 cols(220000) = 1719.62 1926.42 4242.19 719.36 753.08 1334.60 cols(230000) = 1741.48 1968.90 4346.46 720.20 755.74 1313.73 cols(240000) = 1758.04 1992.53 4384.92 714.93 755.37 1322.02 cols(250000) = 1785.96 2018.08 4446.50 724.12 757.57 1320.53 cols(260000) = 1791.03 2016.54 4448.53 719.30 767.97 1344.32 cols(270000) = 1831.66 2072.68 4582.49 724.47 765.00 1354.10 cols(280000) = 1854.82 2093.85 4685.77 734.55 768.16 1348.03 cols(290000) = 1871.86 2131.60 4739.70 725.76 771.90 1373.31 cols(300000) = 1900.85 2159.78 4783.72 740.63 770.87 1357.07 cols(310000) = 1917.00 2179.84 4891.21 736.46 781.40 1364.52 cols(320000) = 1950.14 2204.38 4961.11 740.58 778.29 1379.75 cols(330000) = 1969.40 2233.66 5056.38 750.75 790.07 1371.60 cols(340000) = 1980.81 2270.14 5156.39 748.22 783.81 1381.50 cols(350000) = 7900.49 8180.1911189.29 2625.26 2637.88 3265.47 cols(360000) = 2011.92 2275.95 5314.29 719.10 756.82 1334.90 cols(370000) = 2036.50 2305.38 5391.69 720.18 752.38 1355.60 cols(380000) = 2044.61 2329.94 5495.08 716.65 751.96 1347.62 cols(390000) = 2064.97 2388.83 5586.73 718.49 762.35 1354.92 cols(400000) = 2088.34 2410.12 5722.40 712.94 751.62 1349.71 cols(410000) = 2100.25 2432.17 5781.90 714.60 767.75 1377.17 cols(420000) = 2120.84 2458.69 5895.63 720.20 749.74 1374.17 cols(430000) = 2146.21 2479.88 5981.36 719.72 763.38 1365.93 cols(440000) = 2159.94 2505.02 6129.47 726.59 757.63 1373.81 cols(450000) = 2172.09 2528.55 6223.94 721.32 762.63 1346.21 cols(460000) = 2203.20 2559.08 6342.73 727.50 759.12 1351.56 cols(470000) = 2209.74 2591.13 6494.58 724.56 762.50 1374.38 cols(480000) = 2246.69 2597.51 6622.03 720.50 760.39 1365.53 cols(490000) = 2249.65 2632.48 6716.73 730.03 770.98 1362.94 cols(500000) = 2269.11 2643.57 6849.40 724.77 765.88 1390.11 stats(b1-commit) = 1609.204 stats(b1-find) = 710.0 stats(b2-commit) = 1888.466 stats(b2-find) = 990.0 stats(b5-commit) = 25706.905 stats(b5-find) = 1040.0 stats(f1-commit) = 1575.921 stats(f1-find) = 540.0 stats(f2-commit) = 1851.443 stats(f2-find) = 780.0 stats(f5-commit) = 13972.162 stats(f5-find) = 780.0 Sat Nov 24 00:48:28 CET 2001 % ===================================== ========================================== # Trying to find out how hash performance degrades as size increases if [catch {package require Mk4tcl}] { catch {load ./Mk4tcl.so mk4tcl} catch {load ./Mk4tcl_d.dll mk4tcl} } proc loadwords {step} { global warray set fd [open words] for {set i 0} {$i < $step && [gets $fd line] >= 0} {incr i} { set warray($line) $i } close $fd return $line } proc timedRun {tag count args} { set usec [lindex [time $args $count] 0] lappend ::stats($tag) [expr {$count*$usec/1000.0}] } proc setup {keys type} { file delete _large.mk mk::file open db _large.mk -nocommit set layout "$keys v" if {$type == "b"} { set layout "{_B {$layout}}" } mk::view layout db.words $layout if {$type == "b"} { mk::view open db.words rawdata rename [rawdata view blocked] data } else { mk::view open db.words data } mk::view layout db.words_map {_H:I _R:I} mk::view open db.words_map map rename [data view hash map 1] words } proc teardown {} { catch { rename words "" } catch { rename data "" } catch { rename map "" } catch { rename rawdata "" } mk::file close db } proc fill1 {seq} { global warray foreach {k v} [array get warray] { set x $k$seq words insert end k1 k1_$x v v_$x } } proc fill2 {seq} { global warray foreach {k v} [array get warray] { set x $k$seq words insert end k1 k1_$x k2 k2_$x } } proc fill5 {seq} { global warray foreach {k v} [array get warray] { set x $k$seq words insert end k1 k1_$x k2 k2_$x v1 v1_$x v2 v2_$x v3 v3_$x v v_$x } } proc find1 {w} { words find k1 k1_${w}5 } proc find2 {w} { words find k1 k1_${w}5 k2 k2_${w}5 } proc find5 {w} { words find k1 k1_${w}5 k2 k2_${w}5 } set step 10000 #set step 100 set mult 50 set w [loadwords $step] puts "w = $w" puts [clock format [clock seconds]] foreach type {f b} { foreach keys {{k1} {k1 k2} {k1 k2 v1 v2 v3}} { set nkeys [llength $keys] set mode "$type$nkeys" puts -nonewline stderr "$mode: " setup $keys $type for {set i 0} {$i < $mult} {incr i} { timedRun $mode-fill 1 fill$nkeys $i puts -nonewline stderr . } timedRun $mode-find 10000 find$nkeys $w timedRun $mode-commit 1 mk::file commit db puts stderr " [words size] rows, [file size _large.mk] b" for {set i 0} {$i < 3} {incr i} { puts " $i: [words get $i]" } teardown } } puts [clock format [clock seconds]] puts -nonewline "\n " foreach x [lsort [array names stats *-fill]] { puts -nonewline [format %8s [lindex [split $x -] 0]] set i 0 foreach y $stats($x) { incr i $step append cols([format %6d $i]) [format {%8.2f} $y] } unset stats($x) } puts \n parray cols puts "" parray stats puts "" puts [clock format [clock seconds]] ===================================== ========================================== From baspey@yahoo.com Fri Nov 23 21:31:49 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA17528 for ; Fri, 23 Nov 2001 21:31:49 -0600 Received: from toulouse-2-a7-36-80.dial.proxad.net (HELO desktop) (62.147.36.80) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 03:24:22 -0000 Message-ID: <001901c17497$4f4cc700$5024933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 04:20:37 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >I've used MK extensively and quickly learned to use it efficiently. My >use is the simplest possible. I dont use Find(), hashed views, and so >forth. I found all derived views to be too slow to use for my purposes, >but learned to do without. In my idea, Metakit was simple, rustic. No fancy features like security, SQL, elaborated relational concepts. Furthermore, there are smart ideas like colum-wise data structure and memory mapped files (also on-the-fly restructuring, but this I consider rather marginal). How could this be so ill-programmed that it performs much worse than other databases which perform much complicated tasks ?? I'd like to know what is the secret of these other databases. Because a CPU can perform a certain number of operations per second, and no more. I do hope there isn't a profound flaw in Metakit. Has someone ever achieved ambitious realisation with Metakit ? I mean big database files, or use of advanced functionality like hashing. Not just serialization of initialisation file. I will now repost a message I saw, dated of 14th october 2001, and which sounds like science-fiction (the subject was "VLDB performance ?"). I will try to get in touch with Michael P. Wilson to know how things have turned out. Here is the answer JCW gave him: Michael P. Wilson wrote: >I've been lurking around looking for a relatively low-level library for >maintaining an extremely large (ok, to me anyway) database of fairly >large objects (about 3k of large-ish text columns plus blobs of up to 5 >meg.) > >The semantic needs are very thin. I only ever need append and >read-only access, including fairly basic selection and filtering across >the set. > >Are there any immediate "gotchas" in MetaKit that I should know about >before giving it a shot? I don't need complex data types, advanced >'select' management, record updating, etc. However, all fields are >variable length including the blob field. > >Oh, btw, when I say "extremely large" I mean something in excess of a >1.2Terabytes (and growing at about 100-200 meg a day.) Most things you say are within what I'd consider tough but still feasible. Large objects do not in themselves cause much trouble, because they are stored individually. Access to a row when the large object field is not used, is as fast as if the large object had not been present, due to the column-wise data organization. But the 1.2 Tb kills things. MK is based on memory mapped files and (currently) requires that the entire file be mapped (R/O). On a 32-bit machine, don't even *think* about datasets over 2 Gb. There is only one way to make this fly: use a 64-bit architecture which supports mapping the entire DBMS. I'd be interested to see where bottlenecks turn up, if any, and then try to resolve it. There are numerous tricks to apply, and the view mechanism on top of MK allows for a wide range of "special-purpose cleverness". Read-only data in itself makes the issue feasible IMO. The "append-only" mode you need is trickier, because - again due to the column-wise design - appends, even of a single item, normally have the effect of copying an entire column. One solution is to segment the data, and then recombine it virtually - which the various custom views can IMO be made to do just fine. But let's not forget the reality that 1 Tb is way, way, over what MK has been used for today - and ever considered for AFAIK. It will take deep knowledge of the access/use/modification patterns and of the MK design to have any chance at success with this. Just a thought: why not save the data serially to file(s), and have MK *manage* it, i.e. store offsets and some key navigational data, for example? Transaction safety can surely be obtained, and the amount of data residing in MK could go down dramatically. Again, keep in mind that custom views can hide this physical design choice. -jcw _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Fri Nov 23 21:59:03 2001 Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id VAA18118 for ; Fri, 23 Nov 2001 21:59:03 -0600 Received: from toulouse-2-a7-37-194.dial.proxad.net (HELO desktop) (62.147.37.194) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 03:51:36 -0000 Message-ID: <005101c1749b$1d18c500$5024933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 04:49:11 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >"it's a bad worker who blames his tools" ... I've never been happier to discover I'm a bad worker ! >MK run circles around Berkeley DB and MySQL, >under the right circumstances. I believe you, and that's great news because it would have been terrible to have wasted my time and be compelled to move on to another solution. >I'll ask again, Pascal: are you sure that you have the latest code from >CVS? The linear slowdown you have been reporting sounds similar to a >recent bug, which was resolved resolved (check the change logs). I'm gonna reload and recompile everything. I would be so relieved to discover I've mixed-up somewhere. >Now can we please get on with it? I feel really sorry to have been some source of discordance. But for a while, I was wondering if Metakit wasn't fake... But I'm now convinced it's not. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Fri Nov 23 22:21:46 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id WAA18578 for ; Fri, 23 Nov 2001 22:21:45 -0600 Received: from toulouse-2-a7-38-136.dial.proxad.net (HELO desktop) (62.147.38.136) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 04:14:18 -0000 Message-ID: <000701c1749e$491a4e00$8826933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 05:11:54 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Is JCW rich ? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >My software is free, and I'm doing just fine commercially thank you. Does this mean you get some income from Metakit. That would be good news, but I can't figure out how. And I'm curious... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From readonly@getsoft.com Fri Nov 23 23:24:45 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA19808 for ; Fri, 23 Nov 2001 23:24:45 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 167V9k-0002mw-00; Fri, 23 Nov 2001 22:15:12 -0700 Date: Sat, 24 Nov 2001 16:17:26 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <21307737703.20011124161726@getsoft.com> To: Jacob Levy CC: Pascal Baspeyras , Metakit Subject: Re[2]: [Metakit] Performance can't be that bad In-Reply-To: <3BFF0E75.7020006@best.com> References: <000e01c17461$3a3e5da0$fa24933e@desktop> <77163008358.20011124083550@getsoft.com> <3BFF0E75.7020006@best.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jacob, Saturday, November 24, 2001, 2:05:25 PM, you wrote: JL> Neville JL> Three comments: JL> * I browsed your web site and couldn't find the DB available as a JL> separate product. Is that intentional? The DB is something I've developed purely for use within ED. I have no intention or interest in having it available as a separate product. It is used to track every function, procedure, class, struct etc, within a project, and is updated in real time as you edit the code. It works very effectively with large projects with many thousands of files and 100's of thousands of rows. This is what I was trying to use MK for, but was unable to. JL> * Since I couldn't examine your DB I cannot comment on the performance JL> and features. I dont know whether the DB in your editor is as rich as MK JL> and in fact whether it needs to be as rich as MK, for the uses you intend. It is not in any way intended to be compared or compete with MK. It is specific to my requirements and certainly isn't as feature rich as MK, nor does it need to be. It does have very fast B+Tree access, which is something that was lacking in MK when I tried to use it, and something I very much need. JL> * I've used MK extensively and quickly learned to use it efficiently. My JL> use is the simplest possible. I dont use Find(), hashed views, and so JL> forth. I found all derived views to be too slow to use for my purposes, JL> but learned to do without. I think this is one of the big problems. ie. You need to find some way to get MK to work for you, and this seems to different for each application. If after much effort you still can't find a efficient path what do you do? I had a situation where a sort on 100,000 rows in MK would take 5-10 minutes, which made it impossible for me to use in this specific application. If MK had had indexed access then it would have been a very different picture. JL> You can see how I use MK by examining JL> http://e4graph.sourceforge.net, the source code is all available. All MK JL> related functionality is in the subdirectory mkdriver in the source tree. Thanks. JL> Regards, --JYL >>It seems that many questions posted on the Metakit list revolve around >>performance issues, and the various hacks people finish up using to to >>try and get the performance they need for the specific applications. >> >>Jean-Claude states that Metakits performance varies dramatically >>depending on how you use it, which seems to be related to its column >>oriented design. >> >>>From my personal experience I spent a solid 6+ weeks trying to use >>Metakit in an application. I was able to dramatically improve >>performance in some areas, but got absolutely nowhere in others. For >>example sorting a view with a large number of rows took far too long. >>This was earlier in the year, so things may have improved by now. >> >>The bottom line for me was that I wasn't able to get either acceptable >>performance, nor reliable use out of MK. MK is an elegant and >>interesting DB system, but in my experience it proved too difficult to >>use, was too unpredictable in its performance and required trying lots >>of hacks to get it to perform for each different DB design you >>through at it. >> >>I put this all down to a learning experience and moved on, writing a >>DB of my own with which exhibits very good performance without me >>worrying about how big it gets. The DB I'm referring to is the Source >>Database engine at the heart of the Source Browser/Class View in my >>programmers editor, ED for Windows (www.getsoft.com). >> >>I applaud the work Jean-Claude has done, but at the same time people >>need to determine whether MK is the appropriate solution for them, and >>consider how much time they are willing to invest in order to make this >>decision. >> >> >>Saturday, November 24, 2001, 7:54:26 AM, you wrote: >> >>PB> Bonjour amis francophiles, >> >>PB> Many thanks to Tito and Berk (I don't like much Jean-Louis Murat either) for >>PB> giving out their precious optimization secrets. >>PB> A special thank to Tito, whose mail looked almost as ugly as mines. >> >>PB> I finished a program intended to extensively test Metakit performance: >>PB> I randomly generate string and add them to a blocked/hash view, and I do >>PB> nothing >>PB> else, hence no risk of perturbation. >> >>PB> With only 1 field, the Commit() is still as bad, but surprisingly, the >>PB> Find() is correct. >>PB> Now, I will try with 2 hashed fields. >>PB> And after that, I will see if changing the commit mode improves things. >> >>PB> _________________________________________________________ >>PB> Do You Yahoo!? >>PB> Get your free @yahoo.com address at http://mail.yahoo.com >> >>PB> _______________________________________________ >>PB> metakit mailing list - metakit@equi4.com >>PB> http://www.equi4.com/mailman/listinfo/metakit >> >> -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From jcw@equi4.com Sat Nov 24 05:19:44 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA26504 for ; Sat, 24 Nov 2001 05:19:43 -0600 Received: from takkie.equi4.local ([195.121.238.150]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNAXRE01.LPK for ; Sat, 24 Nov 2001 12:11:38 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAOBAF004339 for ; Sat, 24 Nov 2001 12:10:26 +0100 From: Jean-Claude Wippler Cc: Metakit Subject: [Metakit] Performance can't be that bad Date: Sat, 24 Nov 2001 12:13:08 +0100 Message-Id: <20011124111308.18336@mail.planet.nl> In-Reply-To: <21307737703.20011124161726@getsoft.com> References: <21307737703.20011124161726@getsoft.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >The DB is something I've developed purely for use within ED. Well, it's kind of tricky to compare two things, when one is unavailable. >JL> * I've used MK extensively and quickly learned to use it efficiently. My >JL> use is the simplest possible. I dont use Find(), hashed views, and so >JL> forth. I found all derived views to be too slow to use for my purposes, >JL> but learned to do without. That was pre 2.0 or even earlier. Hashes and blocked views did not exist at the time. So again, it's a bit hard to comment about such a design today. >I think this is one of the big problems. ie. You need to find some way >to get MK to work for you, and this seems to different for each >application. If after much effort you still can't find a efficient >path what do you do? I had a situation where a sort on 100,000 rows >in MK would take 5-10 minutes, which made it impossible for me to use >in this specific application. If MK had had indexed access then it >would have been a very different picture. Gee, 5..10 minutes to sort 100,000 rows, how do you do that? Indexed access is easy. Create a custom sort, have it store the order in a second view, and use RemapWith to make that order persist. This is not 2.4 specific, it could have been done the same way years ago. Neville, could you do me a favor? Can you at least level the playing field by giving me a chance to reproduce and explain this? Perhaps by sending code or a description of code which explains the structure and dataset you used? As for how hard it is to make MK fly in everyone's particular case: if you learn to ride a bike, and then expect to be able to use that experience to drive a car, then boy you're in for a rough ride. Is it a drawback of MK that it is not a bike (i.e. not row-wise, but column-wise) in its design? My advice to everyone is: if you don't like MK, or don't have the ability, time, interest, or whatever to think through the concepts and take advantage of MK, then please... go ride a nice bike, and let the rest of us on this mailing list enjoy and enhance MK. -jcw From jcw@equi4.com Sat Nov 24 06:17:52 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA27644 for ; Sat, 24 Nov 2001 06:17:46 -0600 Received: from takkie.equi4.local ([195.121.235.183]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNB0GC02.13O for ; Sat, 24 Nov 2001 13:09:48 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAOC8a004849 for ; Sat, 24 Nov 2001 13:08:36 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Is JCW rich ? Date: Sat, 24 Nov 2001 12:55:29 +0100 Message-Id: <20011124115529.12245@mail.planet.nl> In-Reply-To: <000701c1749e$491a4e00$8826933e@desktop> References: <000701c1749e$491a4e00$8826933e@desktop> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Dear Pascal / popeye, >Jean-Claude Wippler wrote: >>My software is free, and I'm doing just fine commercially thank you. > >Does this mean you get some income from Metakit. >That would be good news, but I can't figure out how. >And I'm curious... Sure, but does that mean you have to repeatedly waste the time of hundreds of people subscribed to this mailing list with this sort of chit-chat? (FYI, yes I feel rich, in the sense that I make more every year from consulting than I spend. Thanks to MK, which is going a lot more places these days.) Please allow me to adjust the advice I gave to you before. First, I apologize for the time you had to waste on MetaKit. It is a silly design and will most likely never meet your requirements. You will be far better of by using Berkeley DB, MySQL, or Postgres. They are all free, extremely well documented, easy to learn and use, and with lots and lots of people to help. Second, please go play outside and socialize with real people a bit more often. It helps. Now, for the last time... can we get on with it, please? -jcw From jcw@equi4.com Sat Nov 24 06:17:54 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA27645 for ; Sat, 24 Nov 2001 06:17:49 -0600 Received: from takkie.equi4.local ([195.121.235.183]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNB0GD01.XY4 for ; Sat, 24 Nov 2001 13:09:49 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAOC8b004852 for ; Sat, 24 Nov 2001 13:08:37 +0100 From: Jean-Claude Wippler To: Subject: RE: [Metakit] Performance can't be that bad Date: Sat, 24 Nov 2001 13:10:46 +0100 Message-Id: <20011124121046.27655@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Berk, Murat wrote: >We had similar problems with insertion speed and find/search performace. >We find ways around those issues. (We are using 2.01 release) > >1) We don't use the Find and Search stuff in the Metakit for string stuff. You will find that hash/ordered views in 2.3/2.4 now automatically cause Find to work optimally. >We wrote custom Find and Search functions which deals with these issues. We >figured out that c4_StringRef is very fast and we are using it to access >data w/o extracting it out. > >bool Mk_Utils::Utf8StringIsEqualC4Property(const char* utf8_ptr, size_t len, >const c4_StringProp& p, const c4_RowRef& r) Ah, UTF8. Yes, it seems like altering the comparisons to it is long overdue. I look into how much work it would be to make these functions customizable (without introducing globals). >2) Preallocate rows before you insert anything. We allocate 512 rows at a >time. But initially it was very slow to add strings into these views. The >reason is the internal offset array is reallocated everytime and grows 1 at >a time. What we did was set the last row of the preallocated bunch to a >dummy value. What this does is allocate all the offset stuff for all 512 >rows and also leaves preallocated memory called slack in the data. When you >insert new data, it just shifts slack pointer around and works very fast. Thanks. This issue needs to be revisited as well. The grow-by-one behavior has changed, because more data now uses the efficient internal "roving-gap" technique. Let me get this straight: is this about the very simple situation of adding a row with string to a view, and doing so as views grow very large? I'll be happy to time and refine the logic - but can only do so when the exact issue is clear to me. >3) We really did not have problem w/ the speed of the commits. The only >problem we are having is commit uses to much of c4_String which slows it >down. The c4_String class, and that includes build variants which derive from MFC's "CString" and STL'd "string", has been a huge bottleneck in the past, which is why the current MK no longer has them in the public API, and internally greatly reduces their use. Most internal data management now relies on c4_Bytes. Again, show me a case, preferably in code, and I can dive in. >4) We have some memory leaks for the memory allocated in c4_String for >nullVector. Everywhere do we use locks around the metakit (1 per open >database and we have a lot of databases open at any given time). This works >great except the reference counting for the c4_String objects which uses >internal nullVector. Since that is shared by all the databases and not >protected across threads, we do see reference counting problems and it >leaks. >I was looking removing all that stuff and using a special value for refcount >(like (char)-1) to indicate we do not need to reference count, but it is not >much a problem right now. Perhaps simply removing the special case for empty strings would do it? If the optimization to share empty strings is undone, then data sharing would go away. You can't use "(char) -1", it's already used IIRC. What you could do, is alter the decref code to never delete the special empty string, and then simply fully ignore inc/decrefs on that single object. Performance-wise, that will probably not have much impact at all. I'll be happy to look into it - feel free to follow up by email, to give more details (and to refresh my memory on why you haven't yet switched to 2.4.1 :) -jcw From jcw@equi4.com Sat Nov 24 08:56:10 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA30544 for ; Sat, 24 Nov 2001 08:56:09 -0600 Received: from takkie.equi4.local ([195.121.235.232]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNB7S501.O58 for ; Sat, 24 Nov 2001 15:48:05 +0100 Received: from [10.0.1.34] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAOEks006019 for ; Sat, 24 Nov 2001 15:46:54 +0100 From: Jean-Claude Wippler Cc: Metakit Date: Sat, 24 Nov 2001 15:49:37 +0100 Message-Id: <20011124144937.22689@mail.planet.nl> In-Reply-To: <93286188377.20011124101817@getsoft.com> References: <93286188377.20011124101817@getsoft.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Conclusions about speed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: (My apologies for boring everyone with what seems to have become a mud- slinging contest - but when false accusations start flying around, I have to respond) [...] >Yes I agree the documentation is awful. In my experience this is a >common trait of "Free Open Source Software". I have largely lost all >interest in Open Source because of issues like this. It's called a self-fulfilling prophecy. Being someone who sells software (as opposed to services), it would seem to me that you have an interest in seeing free software flounder. It's quite intriguing that you choose to remain on the MK mailing list, even though you do not seem to have an interest in it. From another post: >I think this is one of the big problems. ie. You need to find some way >to get MK to work for you, and this seems to different for each >application. If after much effort you still can't find a efficient >path what do you do? I had a situation where a sort on 100,000 rows >in MK would take 5-10 minutes, which made it impossible for me to use >in this specific application. Yeah, right. The code below sorts 280,000 strings of average length 10 in 28 seconds. Oh, well ... let's just say that Tcl is 20x faster than C++ :) -jcw ===================================== ========================================== % ntclkit sort.tcl Sat Nov 24 15:04:13 CET 2001 ...... 280000 rows, 3090569 b 0: text realest0 1: text Fujitsu0 2: text denotational0 Sat Nov 24 15:06:07 CET 2001 stats(commit) = 359.32 stats(count) = 40000 80000 120000 160000 200000 240000 280000 stats(fill) = 1456.11 884.91 885.72 893.49 884.72 894.40 898.60 stats(sort) = 3229.93 6943.24 10 903.93 14960.48 19230.05 23377.88 27659.37 Sat Nov 24 15:06:07 CET 2001 % ===================================== ========================================== # Timing of the view sort operation package require Mk4tcl proc timedRun {tag count args} { set usec [lindex [time $args $count] 0] append ::stats($tag) [format {%9.2f} [expr {$count*$usec/1000.0}]] } proc fill {seq} { global warray foreach {k v} [array get warray] { mk::row append db.words text $k$seq } } set step 40000 set mult 7 set fd [open /usr/share/dict/words] for {set i 0} {$i < $step && [gets $fd line] >= 0} {incr i} { set warray($line) $i } close $fd puts [clock format [clock seconds]] file delete _large.mk mk::file open db _large.mk -nocommit mk::view layout db.words text for {set i 0} {$i < $mult} {incr i} { append stats(count) [format {%9d} [expr {($i+1)*$step}]] timedRun fill 1 fill $i timedRun sort 1 mk::select db.words -sort text puts -nonewline stderr . } timedRun commit 1 mk::file commit db puts stderr " [mk::view size db.words] rows, [file size _large.mk] b" for {set i 0} {$i < 3} {incr i} { puts " $i: [mk::get db.words!$i]" } mk::file close db puts [clock format [clock seconds]] puts "" parray stats puts "" puts [clock format [clock seconds]] ===================================== ========================================== From jyl@best.com Sat Nov 24 09:29:58 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA31220; Sat, 24 Nov 2001 09:29:57 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id HAA26270; Sat, 24 Nov 2001 07:21:40 -0800 (PST) Message-ID: <3BFFC957.2020706@best.com> Date: Sat, 24 Nov 2001 08:22:47 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Metakit CC: Jean-Claude Wippler Subject: Re: [Metakit] Performance can't be that bad References: <21307737703.20011124161726@getsoft.com> <20011124111308.18336@mail.planet.nl> Content-Type: multipart/alternative; boundary="------------030709000209010804020201" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------030709000209010804020201 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Just want to confirm one thing that JCW said, that I tried to use derived views pre-2.0 and they were not fast enough for me. I am now using the latest MK with much better performance even for my simple use, and I haven't gone back to try the derived views again. I definitely will, to provide more advanced functionality like searching over an entire DB. All in all I'm very happy with what MK provides. Since its free I feel like the value per feature is infinite :) By the way, I had serious problems with deleting values from my DB, using a schema I developed to represent directed graphs inside MK. The end of it was that it was my fault, and JCW helped me patiently throughout the resolution. Thanks JCW :) I'm not yet pushing performance to its limits, but I too would like Commit() to be faster. If I recall, this definitely is something JCW has been working on continuously, and he's promised improvements. So I'm happy -- the issue is out in the open, and the author acknowledges it and promised to do something about it. That's good enough for me. --JYL Jean-Claude Wippler wrote: >Neville Franks wrote: > >>The DB is something I've developed purely for use within ED. >> > >Well, it's kind of tricky to compare two things, when one is unavailable. > >>JL> * I've used MK extensively and quickly learned to use it efficiently. My >>JL> use is the simplest possible. I dont use Find(), hashed views, and so >>JL> forth. I found all derived views to be too slow to use for my purposes, >>JL> but learned to do without. >> > >That was pre 2.0 or even earlier. Hashes and blocked views did not exist >at the time. So again, it's a bit hard to comment about such a design today. > >>I think this is one of the big problems. ie. You need to find some way >>to get MK to work for you, and this seems to different for each >>application. If after much effort you still can't find a efficient >>path what do you do? I had a situation where a sort on 100,000 rows >>in MK would take 5-10 minutes, which made it impossible for me to use >>in this specific application. If MK had had indexed access then it >>would have been a very different picture. >> > >Gee, 5..10 minutes to sort 100,000 rows, how do you do that? > >Indexed access is easy. Create a custom sort, have it store the order in >a second view, and use RemapWith to make that order persist. This is not >2.4 specific, it could have been done the same way years ago. > >Neville, could you do me a favor? Can you at least level the playing >field by giving me a chance to reproduce and explain this? Perhaps by >sending code or a description of code which explains the structure and >dataset you used? > >As for how hard it is to make MK fly in everyone's particular case: if >you learn to ride a bike, and then expect to be able to use that >experience to drive a car, then boy you're in for a rough ride. Is it a >drawback of MK that it is not a bike (i.e. not row-wise, but column-wise) >in its design? > >My advice to everyone is: if you don't like MK, or don't have the >ability, time, interest, or whatever to think through the concepts and >take advantage of MK, then please... go ride a nice bike, and let the >rest of us on this mailing list enjoy and enhance MK. > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > --------------030709000209010804020201 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Just want to confirm one thing that JCW said, that I tried to use derived views pre-2.0 and they were not fast  enough for me. I  am now using the latest MK with much better performance even for my simple use, and I haven't gone back to try the derived views again. I definitely will, to provide more advanced functionality like searching over an entire DB.

All in all I'm very happy with what MK provides. Since its free  I feel like the value per feature is infinite :)

By the way, I had serious problems with deleting values from my DB, using a schema I developed to represent directed graphs inside MK. The end of it was that it was my fault, and JCW helped me patiently throughout the resolution. Thanks JCW :)

I'm not yet pushing performance to its limits, but I too would like Commit() to be faster. If I recall, this definitely is something JCW has been working on continuously, and he's promised improvements. So I'm happy -- the issue is out in the open, and the author acknowledges it and promised to do something about it. That's good enough for me.

--JYL

Jean-Claude Wippler wrote:
Neville Franks <readonly@getsoft.com> wrote:

The DB is something I've developed purely for use within ED.

Well, it's kind of tricky to compare two things, when one is unavailable.

JL> * I've used MK extensively and quickly learned to use it efficiently. My 
JL> use is the simplest possible. I dont use Find(), hashed views, and so 
JL> forth. I found all derived views to be too slow to use for my purposes, 
JL> but learned to do without.

That was pre 2.0 or even earlier.  Hashes and blocked views did not exist
at the time.  So again, it's a bit hard to comment about such a design today.

I think this is one of the big problems. ie. You need to find some way
to get MK to work for you, and this seems to different for each
application. If after much effort you still can't find a efficient
path what do you do?  I had a situation where a sort on 100,000 rows
in MK would take 5-10 minutes, which made it impossible for me to use
in this specific application. If MK had had indexed access then it
would have been a very different picture.

Gee, 5..10 minutes to sort 100,000 rows, how do you do that?

Indexed access is easy.  Create a custom sort, have it store the order in
a second view, and use RemapWith to make that order persist.  This is not
2.4 specific, it could have been done the same way years ago.

Neville, could you do me a favor?  Can you at least level the playing
field by giving me a chance to reproduce and explain this?  Perhaps by
sending code or a description of code which explains the structure and
dataset you used?

As for how hard it is to make MK fly in everyone's particular case: if
you learn to ride a bike, and then expect to be able to use that
experience to drive a car, then boy you're in for a rough ride.  Is it a
drawback of MK that it is not a bike (i.e. not row-wise, but column-wise)
in its design?

My advice to everyone is: if you don't like MK, or don't have the
ability, time, interest, or whatev
er to think through the concepts and
take advantage of MK, then please... go ride a nice bike, and let the
rest of us on this mailing list enjoy and enhance MK.

-jcw

_______________________________________________
metakit mailing list  -  metakit@equi4.com
http://www.equi4.com/mailman/listinfo/metakit


--------------030709000209010804020201-- From jyl@best.com Sat Nov 24 09:33:56 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA31350; Sat, 24 Nov 2001 09:33:55 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id HAA26633; Sat, 24 Nov 2001 07:25:11 -0800 (PST) Message-ID: <3BFFCA2B.4060905@best.com> Date: Sat, 24 Nov 2001 08:26:19 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Subject: Re: [Metakit] Is JCW rich ? References: <000701c1749e$491a4e00$8826933e@desktop> <20011124115529.12245@mail.planet.nl> Content-Type: multipart/alternative; boundary="------------070504090403020806050906" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------070504090403020806050906 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit JC One *important* point that was raised here that shouldn't get lost: the documentation provided for MK could use some enhancing. I agree that the line of discussion about whether you're rich and so forth is obscene and out of place. :( --JYL Jean-Claude Wippler wrote: >Dear Pascal / popeye, > >>Jean-Claude Wippler wrote: >> >>>My software is free, and I'm doing just fine commercially thank you. >>> >>Does this mean you get some income from Metakit. >>That would be good news, but I can't figure out how. >>And I'm curious... >> > >Sure, but does that mean you have to repeatedly waste the time of >hundreds of people subscribed to this mailing list with this sort of >chit-chat? > >(FYI, yes I feel rich, in the sense that I make more every year from >consulting than I spend. Thanks to MK, which is going a lot more places >these days.) > >Please allow me to adjust the advice I gave to you before. > >First, I apologize for the time you had to waste on MetaKit. It is a >silly design and will most likely never meet your requirements. You will >be far better of by using Berkeley DB, MySQL, or Postgres. They are all >free, extremely well documented, easy to learn and use, and with lots and >lots of people to help. > >Second, please go play outside and socialize with real people a bit more >often. It helps. > >Now, for the last time... can we get on with it, please? > >-jcw > >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > --------------070504090403020806050906 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit JC

One *important* point that was raised here that shouldn't get lost: the documentation provided for MK could use some enhancing.

I agree that the line of discussion about whether you're rich and so forth is obscene and out of place. :(

--JYL

Jean-Claude Wippler wrote:
Dear Pascal / popeye,

Jean-Claude Wippler wrote:
My software is free, and I'm doing just fine commercially thank you.
Does this mean you get some income from Metakit.
That would be good news, but I can't figure out how.
And I'm curious...

Sure, but does that mean you have to repeatedly waste the time of
hundreds of people subscribed to this mailing list with this sort of
chit-chat?

(FYI, yes I feel rich, in the sense that I make more every year from
consulting than I spend.  Thanks to MK, which is going a lot more places
these days.)

Please allow me to adjust the advice I gave to you before.  

First, I apologize for the time you had to waste on MetaKit.  It is a
silly design and will most likely never meet your requirements.  You will
be far better of by using Berkeley DB, MySQL, or Postgres.  They are all
free, extremely well documented, easy to learn and use, and with lots and
lots of people to help.

Second, please go play outside and socialize with real people a bit more
often.  It helps.

Now, for the last time... can we get on with it, please?

-jcw

_______________________________________________
me
takit mailing list  -  metakit@equi4.com
http://www.equi4.com/mailman/listinfo/metakit


--------------070504090403020806050906-- From baspey@yahoo.com Sat Nov 24 10:29:15 2001 Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA32399 for ; Sat, 24 Nov 2001 10:29:15 -0600 Received: from toulouse-2-a7-39-28.dial.proxad.net (HELO desktop) (62.147.39.28) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Nov 2001 16:21:46 -0000 Message-ID: <001501c17503$e891b760$1c27933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Sat, 24 Nov 2001 17:19:57 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Performance can't be that bad Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >please go play outside and socialize with real people a bit more >often. It helps. You're a mean person Jean-Claude, you know that ! If you read again all my postings (sorry if there's a lot), you will realize that I'm not a bad person myself. I was simply in the position of someone who thought he had made a blunt mistake but couldn't find which one (the title I chose was "performance *can't be* that bad"). And the only answers I got were that if you don't use clever tricks, performance is indeed that bad. I little went outside because I had launched the subject and thought I had to answer other person's mails, out of politeness and respect. >First, I apologize for the time you had to waste on MetaKit. It is a >silly design and will most likely never meet your requirements. You will >be far better of by using Berkeley DB, MySQL, or Postgres. They are all >free, extremely well documented, easy to learn and use, and with lots and >lots of people to help. I have indeed spent a lot of time on Metakit, and a better documentation would have helped, that's no doubt. Now that I am convinced it is worth it, I will spend the extra time necessary to reach my goal. Sorry, I don't give up Metakit. The only thing I'll give up is this mailing list. I will still listen to your clever conversations but myself, poor peasant, will never post again, by fear that other subscribers may unsubscribe... Jacob Levy wrote (about my subject "is JCW rich ?"): >I agree that the line of discussion about whether you're rich and so >forth is obscene and out of place. :( I suffer, really ! My title was of course only a joke ! But I was really interested in knowing if Jean-Claude got income by providing Metakit's services. Because I never paid a dollar for Metakit and Jean-Claude spends a lot of time on it, I would have been happy to know that Metakit can be a source of income for him. So long. You'll never know if I'd survived all this humiliation or put a bullet beetween my ears. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From konrad@minus1.de Sat Nov 24 13:12:38 2001 Received: from moutvdom01.kundenserver.de (moutvdom01.kundenserver.de [195.20.224.200]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA03190 for ; Sat, 24 Nov 2001 13:12:37 -0600 Received: from [195.20.224.219] (helo=mrvdom03.schlund.de) by moutvdom01.kundenserver.de with esmtp (Exim 2.12 #2) id 167i6v-0003ac-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:05:09 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom03.schlund.de with esmtp (Exim 2.12 #2) id 167i6v-0006KB-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:05:09 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 167i9Q-00019t-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:07:44 +0100 Date: Sat, 24 Nov 2001 20:07:43 +0100 From: Konrad Anton To: metakit@equi4.com Message-ID: <20011124200743.A4397@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.2.5i Subject: [Metakit] accessing row attributes in Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello everybody. I'm using Mk4py 2.4.1 on Python 2.1.0 on Linux. Suppose I've got a metakit view >>> s = metakit.storage("TMPwuff.mk",1) >>> v = s.getas("wuff[bla:S,blub:I]") is there any way to generically get the values of all columns in a row in v, like in: aRow=v[44] for colname in ["bla", "blub"]: print aRow[colname], I can work around it by using print eval("aRow."+colname) instead, but this looks like a dirty hack to me. I've also tried aRow.get(key) and aRow.__getattr__(key), but no success. Ideas, anyone? TIA Konrad PS. shoulder-pat shoulder-pat to JCW for Metakit. It works for me. Some time in the future, I'll announce a little transaction processing system for it, but it is not quite ready for public dégustation. -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: They couldn't have somebody to day. From konrad@minus1.de Sat Nov 24 13:42:33 2001 Received: from moutvdom01.kundenserver.de (moutvdom01.kundenserver.de [195.20.224.200]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA03813 for ; Sat, 24 Nov 2001 13:42:33 -0600 Received: from [195.20.224.208] (helo=mrvdom01.schlund.de) by moutvdom01.kundenserver.de with esmtp (Exim 2.12 #2) id 167iZs-0006cO-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:35:04 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom01.schlund.de with esmtp (Exim 2.12 #2) id 167iZs-0000s1-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:35:04 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 167iGh-0001Ab-00 for metakit@equi4.com; Sat, 24 Nov 2001 20:15:15 +0100 Date: Sat, 24 Nov 2001 20:15:15 +0100 From: Konrad Anton To: metakit@equi4.com Message-ID: <20011124201515.A4488@minus1.de> References: <20011124200743.A4397@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20011124200743.A4397@minus1.de>; from konrad on Sat, Nov 24, 2001 at 08:07:43PM +0100 Subject: [Metakit] Re: accessing row attributes in Python Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Konrad. Long time no see. > instead, but this looks like a dirty hack to me. I've also tried > aRow.get(key) and aRow.__getattr__(key), but no success. use getattr(aRow, key). > TIA > Konrad You're welcome. Konrad. -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: They couldn't have somebody to day. From gmcm@hypernet.com Sat Nov 24 13:42:35 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA03817 for ; Sat, 24 Nov 2001 13:42:35 -0600 From: gmcm@hypernet.com Received: from PARANOIA (204.176.40.83) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Sat, 24 Nov 2001 14:28:58 -0500 To: metakit@equi4.com Date: Sat, 24 Nov 2001 14:35:44 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] accessing row attributes in Python Message-ID: <3BFFB040.22925.BFE04001@localhost> Priority: normal In-reply-to: <20011124200743.A4397@minus1.de> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 24 Nov 2001 at 20:07, Konrad Anton wrote: [snip] > is there any way to generically get the values of all columns in a row > in v, like in: > > aRow=v[44] > for colname in ["bla", "blub"]: > print aRow[colname], > > I can work around it by using > > print eval("aRow."+colname) > > instead, but this looks like a dirty hack to me. I've also tried > aRow.get(key) and aRow.__getattr__(key), but no success. Almost. getattr(row, "colname") (You're dealing with an old-fashioned type object which doesn't have a method "__getattr__", but it does have a getattr slot.) From tciuro@mac.com Sat Nov 24 14:24:43 2001 Received: from smtpout.mac.com (smtpout.mac.com [204.179.120.89]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA04723; Sat, 24 Nov 2001 14:24:43 -0600 Received: from smtp-relay02.mac.com (server-source-si02 [10.13.10.6]) by smtpout.mac.com (8.12.1/8.10.2/1.0) with ESMTP id fAOJTRsD017032; Sat, 24 Nov 2001 11:29:27 -0800 (PST) Received: from asmtp02.mac.com ([10.13.10.66]) by smtp-relay02.mac.com (Netscape Messaging Server 4.15 relay02 Jun 21 2001 23:53:48) with ESMTP id GNBN0Q00.E11; Sat, 24 Nov 2001 12:17:14 -0800 Received: from localhost ([63.202.173.4]) by asmtp02.mac.com (Netscape Messaging Server 4.15 asmtp02 Jun 21 2001 23:53:48) with ESMTP id GNBN0P00.CIS; Sat, 24 Nov 2001 12:17:13 -0800 Date: Sat, 24 Nov 2001 12:17:13 -0800 Subject: Re: [Metakit] Is JCW rich ? Content-Type: multipart/alternative; boundary=Apple-Mail-1-408545894 Mime-Version: 1.0 (Apple Message framework v501) Cc: Jean-Claude Wippler , metakit To: Jacob Levy From: Tito Ciuro In-Reply-To: <3BFFCA2B.4060905@best.com> Message-Id: <3F32B6E0-E118-11D5-BB10-0003934510E2@mac.com> X-Mailer: Apple Mail (2.501) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --Apple-Mail-1-408545894 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Hey, hey, hey! Hold your horses! Let's give Jean-Claude a break, or better yet, two. Without his dedication and hard work, I, for one, wouldn't have had the slightest chance to finish my project. Not only has MK proven to be a workhorse, but The Man himself has gone great lengths making sure that "we get it right." In one of my first posts, sometime in March, I wrote an email complaining the state of the somewhat aged documentation. Yes, it's true. Yes, it's too bad. But guess what? You can read the list's comments. You can read the FAQ. You can read the headers. If in the end you're still bothered by it, why don't you contribute? How come I hear complaints without giving anything back? What the heck is this? C'mon! Let's show some manners and initiative, shall we? Thanks a million Jean-Claude for your immense help and dedication. I owe you a big one; you know what I'm talking about... -- Tito On Saturday, November 24, 2001, at 08:26 AM, Jacob Levy wrote: > JC > > One *important* point that was raised here that shouldn't get lost: the > documentation provided for MK could use some enhancing. > > I agree that the line of discussion about whether you're rich and so > forth is obscene and out of place. :( > > --JYL > ____________________________________________ "I told my psychiatrist that everyone hates me. He said I was being ridiculous - everyone hasn't met me yet." Rodney Dangerfield --Apple-Mail-1-408545894 Content-Transfer-Encoding: 7bit Content-Type: text/enriched; charset=US-ASCII Hey, hey, hey! Hold your horses! Let's give Jean-Claude a break, or better yet, two. Without his dedication and hard work, I, for one, wouldn't have had the slightest chance to finish my project. Not only has MK proven to be a workhorse, but The Man himself has gone great lengths making sure that "we get it right." In one of my first posts, sometime in March, I wrote an email complaining the state of the somewhat aged documentation. Yes, it's true. Yes, it's too bad. But guess what? You can read the list's comments. You can read the FAQ. You can read the headers. If in the end you're still bothered by it, why don't you contribute? How come I hear complaints without giving anything back? What the heck is this? C'mon! Let's show some manners and initiative, shall we? Thanks a million Jean-Claude for your immense help and dedication. I owe you a big one; you know what I'm talking about... -- Tito On Saturday, November 24, 2001, at 08:26 AM, Jacob Levy wrote: JC One *important* point that was raised here that shouldn't get lost: the documentation provided for MK could use some enhancing. I agree that the line of discussion about whether you're rich and so forth is obscene and out of place. :( --JYL Lucida Grande ____________________________________________Lucida Grande 0000,0000,FFFF"I told my psychiatrist that everyone hates me. He said I was being ridiculous - everyone hasn't met me yet." 0000,6666,3333 Rodney Dangerfield --Apple-Mail-1-408545894-- From readonly@getsoft.com Sat Nov 24 17:34:56 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id RAA09407; Sat, 24 Nov 2001 17:34:55 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 167mCi-0002CZ-00; Sat, 24 Nov 2001 16:27:25 -0700 Date: Sun, 25 Nov 2001 10:29:42 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <167373274049.20011125102942@getsoft.com> To: Jean-Claude Wippler CC: Metakit Subject: Re: [Metakit] Performance can't be that bad In-Reply-To: <20011124111308.18336@mail.planet.nl> References: <21307737703.20011124161726@getsoft.com> <20011124111308.18336@mail.planet.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Saturday, November 24, 2001, 10:13:08 PM, you wrote: JCW> Neville Franks wrote: >>The DB is something I've developed purely for use within ED. JCW> Well, it's kind of tricky to compare two things, when one is unavailable. As I stated earlier I was never trying to compare what I've done with MK. I simply wanted to state my experience with MK. Furthermore from the outset I mentioned that I applauded the work you have done with MK. It was unfortunate for me that I wasn't able to get it to meet my needs. >>JL> * I've used MK extensively and quickly learned to use it efficiently. My >>JL> use is the simplest possible. I dont use Find(), hashed views, and so >>JL> forth. I found all derived views to be too slow to use for my purposes, >>JL> but learned to do without. JCW> That was pre 2.0 or even earlier. Hashes and blocked views did not exist JCW> at the time. So again, it's a bit hard to comment about such a design today. >>I think this is one of the big problems. ie. You need to find some way >>to get MK to work for you, and this seems to different for each >>application. If after much effort you still can't find a efficient >>path what do you do? I had a situation where a sort on 100,000 rows >>in MK would take 5-10 minutes, which made it impossible for me to use >>in this specific application. If MK had had indexed access then it >>would have been a very different picture. JCW> Gee, 5..10 minutes to sort 100,000 rows, how do you do that? Using c4_View.Sort() There was approx. 7 columns in the view and the sort was on a string. This was with several hundred thousand rows on a P3 550Mhz PC. JCW> Indexed access is easy. Create a custom sort, have it store the order in JCW> a second view, and use RemapWith to make that order persist. This is not JCW> 2.4 specific, it could have been done the same way years ago. I recall discussing my need to persist the Sort on this list, but can't recall any solution being suggested. Having to wait several minutes or even 30 seconds each time the app starts while the Sort is performed is I'm afraid unacceptable to my users. JCW> Neville, could you do me a favor? Can you at least level the playing JCW> field by giving me a chance to reproduce and explain this? Perhaps by JCW> sending code or a description of code which explains the structure and JCW> dataset you used? Jean-Claude, I think this would be a waste of your valuable time and mine. I was simply using c4_View.Sort()as mentioned above. Sorting was one issue, real time updates with many adds and deletes was another performance problem for me. When you are writing code in an editor any delays to your typing would make the editor unusable. With ED each and every edit will trigger database lookups and potential updates. This needs to have minimal perceived impact to the user. In other words no delays are acceptable. JCW> As for how hard it is to make MK fly in everyone's particular case: if JCW> you learn to ride a bike, and then expect to be able to use that JCW> experience to drive a car, then boy you're in for a rough ride. Is it a JCW> drawback of MK that it is not a bike (i.e. not row-wise, but column-wise) JCW> in its design? JCW> My advice to everyone is: if you don't like MK, or don't have the JCW> ability, time, interest, or whatever to think through the concepts and JCW> take advantage of MK, then please... go ride a nice bike, and let the JCW> rest of us on this mailing list enjoy and enhance MK. I invested considerable time and effort in MK and in the end found that I could not get it to meet my specific needs. This came at a high cost to me, but I do not regret it. Furthermore I'm interested in seeing how MK evolves, which is one of the reasons I've stayed on this list. I have also assisted some MK users in the past, and provided some code to enable custom sort sequences etc. If you would prefer not to have me on this list then please say so and I'll remove myself. I have no interest in seeing free software flounder, as you stated. The lesson I've learnt over the past year is that "free software" has cost me dearly. I estimate I spent approx. $16K of my time on MK. Obviously my expectations were wrong and I know that now. You can't possibly expect to get the level of documentation, support etc.etc. from free software that you can from commercial software. Even many commercial packages fall short, but at least you can rightfully demand a certain level of assistance. Over time commercial products that don't meet their customers needs will fail and disappear from the marketplace. Unfortunately this isn't the case with "free software" which seems to live on forever, potentially costing people dearly. I see this along with a lack of adequate peer review a fundamental problem with "free software". Please note that I am not referring to MK here. It is very unfortunate that this thread has degenerated into a mud slinging match. This was certainly never my intention. I simply wanted to state my observations and personal experience with MK, thinking that they may be of some interest to others. -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From jyl@best.com Sun Nov 25 12:03:34 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA32692; Sun, 25 Nov 2001 12:03:31 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id JAA09438; Sun, 25 Nov 2001 09:55:54 -0800 (PST) Message-ID: <3C013F0B.1020209@best.com> Date: Sun, 25 Nov 2001 10:57:15 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Efficiency question Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude Is there a big difference in efficiency if I create a temporary row object and do my manipulations on that, and afterwards write the row to the view, or if I manipulate individual properties one at a time? For example, the MK driver in e4Graph is full of idioms like this: /* * Initialize the marker. */ pFlags(markers[idx]) = E4_INUSE | E4_DETACHED; pUserData(markers[idx]) = 0; pNameID(markers[idx]) = nameID; Is each expression of the form markers[idx] creating a temporary row object that is immediately discarded, or is it a bit more efficient than that to do the above? Is the above significanly less efficient than: c4_Row row = markers[idx]; pFlags(row) = E4_INUSE | E4_DETACHED; pUserData(row) = 0; pNameID(row) = nameID; markers[idx] = row; Thanks, --JYL From jcw@equi4.com Sun Nov 25 13:19:23 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA01588 for ; Sun, 25 Nov 2001 13:19:22 -0600 Received: from takkie.equi4.local ([195.121.233.70]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNDEMU01.02H for ; Sun, 25 Nov 2001 20:11:18 +0100 Received: from [10.0.1.35] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAPJAH015265 for ; Sun, 25 Nov 2001 20:10:17 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Efficiency question Date: Sun, 25 Nov 2001 20:12:31 +0100 Message-Id: <20011125191231.21914@mail.planet.nl> In-Reply-To: <3C013F0B.1020209@best.com> References: <3C013F0B.1020209@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >Is there a big difference in efficiency if I create a temporary row >object and do my manipulations on that, and afterwards write the row to >the view, or if I manipulate individual properties one at a time? MK stores data column-wise. If you have 5 columns, then fetching them all and copying it into a temp row, then storing it back is 10 data movements. If you need to alter one property, for example, then temp rows would be the worst thing you can do. > pFlags(markers[idx]) = E4_INUSE | E4_DETACHED; > pUserData(markers[idx]) = 0; > pNameID(markers[idx]) = nameID; > >Is each expression of the form markers[idx] creating a temporary row >object that is immediately discarded, or is it a bit more efficient than >that to do the above? > >Is the above significanly less efficient than: > > c4_Row row = markers[idx]; > pFlags(row) = E4_INUSE | E4_DETACHED; > pUserData(row) = 0; > pNameID(row) = nameID; > markers[idx] = row; On the contrary, your first example is *more* efficient. Did you time it? Follow the "Tips and Tricks" link on the API page: http://www.equi4.com/metakit/api/ That leads to: http://www.equi4.com/metakit/api/tips.html#faster The point of column-wise data, is that many loops tend to not deal with all properties. With columns, such unused data never gets loaded. That is why, for example, sorting speed is not related to the number of properties in a view. For a different reason (CPU caches, both instruction- and data), but again due to the columnar structure underneath MK, when you loop over large amounts of data, don't do: for each row: for each column: blah But do, when possible: for each column: for each row: blah I cannot repeat it enough, it seems: the concept of storing data in columns yet presenting a traditional row-wise API, as MK does, is one which has profound implications. The difference between getting it right and getting it wrong, is often several orders of magnitude, especially when aiming for high-end performance. I wish I knew how to make the concept clearer, I really do. -jcw From jcw@equi4.com Sun Nov 25 13:30:13 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA01812; Sun, 25 Nov 2001 13:30:13 -0600 Received: from takkie.equi4.local ([195.121.237.165]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNDF4W02.R33; Sun, 25 Nov 2001 20:22:08 +0100 Received: from [10.0.1.35] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAPJKu015355; Sun, 25 Nov 2001 20:21:06 +0100 From: Jean-Claude Wippler To: Cc: metakit Subject: Re: [Metakit] Efficiency question #2 Date: Sun, 25 Nov 2001 20:23:31 +0100 Message-Id: <20011125192331.26641@mail.planet.nl> In-Reply-To: <3C013F0B.1020209@best.com> References: <3C013F0B.1020209@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > pNameID(markers[idx]) = nameID; > >Is each expression of the form markers[idx] creating a temporary row >object that is immediately discarded, [...] Aha, sorry for missing this on first reading: "markers[idx]" is *not* a temp row. It is totally unrelated to writing things like "pNameID [123]", which indeed is a temp row that gets to hold one property value, briefly. The confusion is most unfortunate, and cause by overloading the same [] operator, simply because C++ offers few alternatives. No, "views[index]" is *the* most efficient way to refer to any (conceptual) row in a view. It is, internall, a c4_RowRef object - which in turn is a very very lightweight object (a "flyweight", in gang-of-four parlance). On the other hand, "property[value]" creates a view, sizes it to hold one row, and allocates one column to hold the value. It's extremely convenient, but needs to be avoided inside tight loops. See the "tips & tricks" page. Thanks for helping clarify this key issue. -jcw From jcw@equi4.com Mon Nov 26 06:10:02 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA09618 for ; Mon, 26 Nov 2001 06:10:01 -0600 Received: from takkie.equi4.local ([195.121.232.195]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNEPF600.T3L for ; Mon, 26 Nov 2001 13:01:54 +0100 Received: from [10.0.1.35] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAQC0w019861 for ; Mon, 26 Nov 2001 13:00:59 +0100 From: Jean-Claude Wippler To: Metakit Mailing List Subject: Re: [Metakit] Python, Metakit, Linux, SegFaults -- more info Date: Mon, 26 Nov 2001 13:02:58 +0100 Message-Id: <20011126120258.9976@mail.planet.nl> In-Reply-To: <3BF98487.5C434126@bioreason.com> References: <3BF98487.5C434126@bioreason.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mitch Chapman wrote: >Below are three Python scripts, with output, which trigger segfaults Ok, I've commited some changes to CVS and updated the snapshot at: http://www.equi4.com/previews/metakit.tar.gz (For this time, this snapshot won't be updated all the time!) >schema[:] = [] ># This will crash because it's indexing beyond the end of schema. ># Should it produce an IndexError instead? >schema[0] = {'ident': 'Kilroy'} Fixed, a check was moved to catch this. It made another problem with exception handling in Mk4py show up, which has been worked around for the time being. >db = metakit.storage() [...] ># This will crash because storage has no associated file? >db.commit() The same exception problem. Commit to such a temp storage generated the proper Python error, but its handling went south. >contents = range(10) >for i in range(numRecords): > records[i] = {'ident': i, 'contents': contents} Fixed. Added code to reject a sequence of scalars in this context. -jcw From nicolasb@maich.gr Mon Nov 26 06:44:36 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA11444 for ; Mon, 26 Nov 2001 06:44:31 -0600 Received: FROM maich.gr BY mail-server ; Mon Nov 26 14:52:14 2001 +0200 Message-ID: <3C023B96.CFF4FEA9@maich.gr> Date: Mon, 26 Nov 2001 14:54:46 +0200 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, Too much un-related noise on this list... FWIW, I feel a need to chime in here, and hopefully put some things in perspective... IMO, Metakit ia a developers tool, with a rich api and the ability to "try it this way if it didn't work as expected the first way..." Developers in general develop. If not let's all stick to .mdb files and a VB front end and resultant 30 MB application files, and install nightmares (maybe that;s another story)... Metakit is free and Open Source...(Used to be 90USD for a full developers version, libraries and source, correct me if I am wrong JC..) Metakit is available on numerous platforms Metakit's file are transportable accross paltforms Metakit has AFAIK, 3 interfaces... Metakit has real support. My problems/bugs are addressed/fixed in a day or less. Usually less than an hour.. Metakit is constantly being improved, and functionality added...(v1.8---v2.4) Core documentation is there. I heve learned to read the sources/lists/faqs for the rest. Yes a bound book would be nice Yes, other options exist, some open, some not, some free some not (Is Faircom still charging ~16K for their cross platform C libs?) Taken as whole, this is a tall order... So where do we stand? Some recommendations? JC, maybe you should re-instate you old Mk pricing along with your open source approach. Then you probably wont get questions on the state of your financial affairs... It would also make it easier for people to show their appreciation for your work and support and such.... Documentation; Someone that must have it; write a book and I'll print it and bind it.... Maybe clearly/simply summarize Mk major developments and current directions and potential limitations (read TeraBytes) on a single html page with big fonts so people can see it if they dont feel like reading changes.txt or the Metakit intros...... Finally a note to Mr. Pascal. Please take a breath, re-look at your project and see if Metakit is appropriate, and let us know. It's how I learn at least... And for JC, as in Tito's last post, thanx a bunch for all you have done for me...It's been too long since we met, I must dream up another project, Is July 2002 open:-? Sincerely Nicolas Boretos Metakit user since V1.8/Mk4tcl 0.5 From baspey@yahoo.com Mon Nov 26 09:44:40 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA22265 for ; Mon, 26 Nov 2001 09:44:39 -0600 Received: from toulouse-2-a7-38-223.dial.proxad.net (HELO desktop) (62.147.38.223) by smtp.mail.vip.sc5.yahoo.com with SMTP; 26 Nov 2001 15:37:03 -0000 Message-ID: <000901c1768f$fb069cc0$df26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Mon, 26 Nov 2001 16:34:32 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, here are my public apologizes to everyone, and most especially to JCW. You may accept them or not... I feel like shit, but claim not having had bad intentions at any time. I have been informed that my style deserved me, and believe it or not, I did not realize at that time that I sounded aggressive. Metakit fits my expectations in all points: embeddable, small footprint, cross-platform, open source, free. After having impregnated myself with Metakit concepts, I have worked out a design for my database according to Metakit's particular architecture (especially subviews). Then came the moment I tried the result and ooops, that's really not good... I did not assault this forum but instead made experiences for 2 days. Convinced I was making some silly mistake but unable to find which one, I posted a mail on this list. And then bad luck fell down on me since I received answers mostly from people telling me that Metakit is pourrave, and on another side, could not find the source of my problem even with the help of the dedicated Jean-Claude. You are people who know that Metakit is great, but not much of you intervened to testify... But those are public apologizes so I'll end up with these words: Thanks a lot Jean-Claude. I apologize to everyone. P.S: please don't call me mister anymore. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From mitch.chapman@bioreason.com Mon Nov 26 10:11:56 2001 Received: from gateway.bioreason.com (www.bioreason.com [207.108.245.3]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA24276; Mon, 26 Nov 2001 10:11:55 -0600 Received: by gateway.bioreason.com (Postfix, from userid 200) id A6B272201; Mon, 26 Nov 2001 09:04:04 +0000 (/etc/localtime) Received: by bioreason.com via smwrap Version 1.0 id smwrappwbGca; Mon Nov 26 09:03:05 2001 by mailhost.bioreason.com (Postfix) with ESMTP id F265630DC; Mon, 26 Nov 2001 09:02:43 +0000 (/etc/localtime) Message-ID: <3C02692F.979A0BDF@bioreason.com> Date: Mon, 26 Nov 2001 09:09:19 -0700 From: Mitch Chapman Organization: Bioreason, Inc. MIME-Version: 1.0 To: Jean-Claude Wippler Cc: Metakit Mailing List Subject: Re: [Metakit] Python, Metakit, Linux, SegFaults -- more info References: <3BF98487.5C434126@bioreason.com> <20011126120258.9976@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: > > On Mitch Chapman wrote: > > >Below are three Python scripts, with output, which trigger segfaults > > Ok, I've commited some changes to CVS and updated the snapshot at: > http://www.equi4.com/previews/metakit.tar.gz > (For this time, this snapshot won't be updated all the time!) Thanks! That was a quick turnaround. Thanks also for making these changes available via HTTP. I think our firewall is to blame for our inability to access remote CVS repositories from work, but in any case you've saved me alot of time (go home, do a CVS update, email the results to work) and hassle. -- Off to get the snapshot, Mitch From steve@improvision.com Mon Nov 26 10:36:02 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA25895 for ; Mon, 26 Nov 2001 10:36:01 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id QAA05222 for ; Mon, 26 Nov 2001 16:26:24 GMT Received: from [192.168.1.25] ([192.168.1.25]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971 for ; Mon, 26 Nov 2001 16:28:21 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 26 Nov 2001 16:28:25 +0000 From: Stephen Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Possible bug (and fix) in remap.cpp Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I have found what might be a small bug in remap.cpp Line 647 reads: bool c4_BlockedViewer::RemoveRows(int pos_, int count_) { d4_assert(count_ > 0); d4_assert(pos_ + count_ < GetSize()); . . . I suspect the assert should read: d4_assert(pos_ + count_ <= GetSize()); As it asserts if pos_ is the last row and count_ is 1 (i.e. delete the last row). Is this correct? Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ From mlh@idi.ntnu.no Mon Nov 26 11:09:54 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA28810 for ; Mon, 26 Nov 2001 11:09:53 -0600 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id SAA16554; Mon, 26 Nov 2001 18:02:13 +0100 (MET) Message-ID: <028901c1769c$18130540$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: "Pascal Baspeyras" Cc: References: <000901c1768f$fb069cc0$df26933e@desktop> Subject: Re: [Metakit] Retro-spection Date: Mon, 26 Nov 2001 18:02:13 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Pascal Baspeyras" > Hello, > > here are my public apologizes to everyone, and most especially > to JCW. > You may accept them or not... > > I feel like shit, but claim not having had bad intentions at > any time. Just for the record, I found the hostile reactions of this list towards you excessive. It seemed to me that intentions were wrongly read into your postings. -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.sf.net From horst@freedict.de Mon Nov 26 11:33:42 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id LAA30647 for ; Mon, 26 Nov 2001 11:33:41 -0600 From: horst@freedict.de Received: (qmail 29958 invoked from network); 26 Nov 2001 17:48:57 -0000 Received: from dialin-145-254-111-247.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.111.247) by mail.netbeat.de with SMTP; 26 Nov 2001 17:48:57 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fAQGeai04683; Mon, 26 Nov 2001 17:40:37 +0100 Message-Id: <200111261640.fAQGeai04683@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Mon, 26 Nov 2001 17:40:33 +0100 (CET) Reply-To: horst@freedict.de To: metakit@equi4.com In-Reply-To: <20011125192331.26641@mail.planet.nl> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] I guess, I did not understand metakits hashed view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I am not sure, I understood correct how the hashed view is meant to work. Attached is the a test file, which demonstrates where I do have difficulties. When I replace vwh.append with view.append, things become slow with a number of entries, but the result is as expected: with each append, the nuber of items increases. Thanks, Horst import metakit import locale import time Auth = "hey" Date = time.time() Quality = 1 Status = 0 dict = "test.mk" db = metakit.storage(dict, 1) db.autocommit() # word: The Heardowrd for the definition # definition: The definition in TEI format # auth: Who last changed the definition # date: when was it last changed # the quality of the entry # 0: no quality # 10: computer generated # 20: novice, non linguist # ... # 100: absolut perfect entry (perhaps can never be achieved) # negative numbers are used when the entry is not conformant # (to the DTD, or ...) - The value still represents who # generted it. - So -10 is invalid computer generated # status # 0: word is new # 1: word is "work in progress" # 2: word is eidted (at least once) view = db.getas("dict[dictionary:S,word:S,collate:B,definition:S,auth:S,date:D,quality:I,status:I]") map = db.getas('map[_H:I,_R:I]') vwh = view.hash(map,1 ) sorted = vwh.sort(vwh.collate) for Word in ("a", "b", "c", "d"): "Add a definition to the database" Def = Word + Word Collate = locale.strxfrm(Word) Word = Word.encode('utf-8') Def = Def.encode('utf-8') vwh.append(word=Word, collate=Collate, definition=Def, auth=Auth, date=Date, quality=Quality, status = Status); for r in vwh: print r.word for r in vwh: print r.word -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From jcw@equi4.com Mon Nov 26 13:09:41 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA06169 for ; Mon, 26 Nov 2001 13:09:40 -0600 Received: from takkie.equi4.local ([195.121.238.184]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNF8UK02.DM1 for ; Mon, 26 Nov 2001 20:01:32 +0100 Received: from [10.0.1.36] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fAQJ0d023765 for ; Mon, 26 Nov 2001 20:00:39 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] I guess, I did not understand metakits hashed view Date: Mon, 26 Nov 2001 20:02:35 +0100 Message-Id: <20011126190235.2360@mail.planet.nl> In-Reply-To: <200111261640.fAQGeai04683@eaglesnest.mceggman> References: <200111261640.fAQGeai04683@eaglesnest.mceggman> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: >view = db.getas("dict[dictionary:S,wo >rd:S,collate:B,definition:S,auth:S,date:D,quality:I,status:I]") >map = db.getas('map[_H:I,_R:I]') >vwh = view.hash(map,1 ) >sorted = vwh.sort(vwh.collate) > >for Word in ("a", "b", "c", "d"): [...] > vwh.append(word=Word, collate=Collate, definition=Def, auth=Auth, >date=Date, quality=Quality, status = Status); You're defining a view with "dictionary" as first property. Then you hash, specifying that its key is the first field. Then you add items which all have the same value for "dictionary" (unset, i.e. the empty string). The result is a view which contains the last entry ("append" is confusing in the context of hashes, which work with unique keys). -jcw From baspey@yahoo.com Mon Nov 26 23:33:55 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA07000 for ; Mon, 26 Nov 2001 23:33:55 -0600 Received: from toulouse-2-a7-36-254.dial.proxad.net (HELO desktop) (62.147.36.254) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 05:26:16 -0000 Message-ID: <007201c17703$d1bb3a00$fe24933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 06:24:06 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Be warned: this is an un-aggressive mail ! Magnus Lie Hetland wrote: >Just for the record, I found the hostile reactions of this list >towards you excessive. It seemed to me that intentions were >wrongly read into your postings. I'm not sure you will like it Magnus, but I LOVE YOU. If we lived in California, I would marry you ! I totally agree with what you say, and I totally agree with what I've said. Nobody can't deny that I made my possible to defend Metakit and to say "I must be doing something wrong". I have already received (a few) supportive mails on my private E-mail address. Thanks a lot guys. And all readers are welcome to send me theirs. I will quote one of them: >Too often I have found in my life people who, because of some misunderstanding >- sometimes cultural, sometimes because of translation problems, sometimes >because of personal problems or issues - end up being 'flamed' in public >and, because of the embarassment, give up on some particular mailing >list or product. >What I have tried to do when I find _myself_ in such situations is this: >1. examine what I have said and determine whether I owe someone an >apology - either public or private >2. examine what benefits I get from the product and decide if a public >apology should be offered - whether I owe it or not. >3. most of all, determine when I am willing to give up having the last word >and just letting things go, hoping that after some decent period of time >I can expect things to 'resume normalcy'. I'll follow this wise advice. I'll follow it right now ! Before someone puts it on the table again, let's talk about my "is JCW rich ?" subject. Title was of course a prank. Richness doesn't always mean money. But on the other hand, subject was serious: I'd really like my application to be available for free. I have never heard of JCW charging for his services, so I would have liked to know what kind of partnership or sponsorship it is possible to contract. This may certainly not be the right forum to discuss such subject, but if someone could tell me about his experience or direct me toward the right forum... I'm French and in France we less easily talk about money than the Americans do. So where could the fellow who called me obcene come from ? End of the digression. I have practiced flagellation in public. But nobody seems interested in knowing what solution I have found to my problem. The answer is simple: at that time, none. You all remember that I've cause much trouble and that a lot of persons hate me and call me Mr Pascal. But what did I complain about ? All right, I'm gonna help you. ;-) I had performance problems with hash lookup and commit. Now I will ask of you an act of solidarity: let's leave JCW apart from this. I have been much of an annoyance to him those days. Let's give him a break, or better, two. Shoulder-pat Jean-Claude. Shoulder-pat. I'm sincere. Hello sir Borretos ! Now that noble defensors of Metakit have arrived on their white horses, we may try to solve this among us, following the ancient code (love this one). My mail is getting real long, so I'll split it in two. To be continued... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Mon Nov 26 23:36:17 2001 Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA07103 for ; Mon, 26 Nov 2001 23:36:17 -0600 Received: from toulouse-2-a7-36-254.dial.proxad.net (HELO desktop) (62.147.36.254) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 05:28:36 -0000 Message-ID: <008501c17704$25690a60$fe24933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 06:27:01 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Part II This mail is the continuation of another. So if you didn't read the first mail, I leave it to your cleverness what you have to do. JCW wrote: >With so much finger pointing going on, I simply had to go check things. >Below are some results, using the Tcl scripting language on a PIII/650 >under SuSE Linux 7.3, with 256 MB RAM. C++ ought to perform better. This was followed by a long series of numbers. But what did those numbers say in fact. They only showed that Add() takes constant time, that's all. They didn't tell us anything interesting about Find() and Commit(), so let's forget them. On the contrary, JCW mailed me other results, which tell a lot more. Those are the results of a test JCW wrote especially for me in C++. I shall never thank him enough for the time he has dedicated to me. I don't include the source for brevity, but here are those results for1 view with 1 column: /* Timing results on Linux Suse 7.3, gcc 2.95.3, PII/650, 256 Mb RAM: NORMAL VIEW ROW ADD FIND COMMIT SIZE 25000 7 uS 81921 uS 30691 uS 1998304 bytes 50000 8 uS 164343 uS 55400 uS 3552352 bytes 75000 7 uS 247122 uS 169282 uS 6408400 bytes 100000 7 uS 327152 uS 162672 uS 6408400 bytes 125000 8 uS 411907 uS 135025 uS 8571424 bytes 150000 8 uS 495606 uS 156729 uS 11469448 bytes 175000 8 uS 580956 uS 181985 uS 11469448 bytes 200000 8 uS 660399 uS 218114 uS 15396472 bytes 225000 7 uS 743788 uS 247174 uS 15396472 bytes 250000 7 uS 824172 uS 259353 uS 20499496 bytes BLOCKED VIEW ROW ADD FIND COMMIT SIZE 25000 11 uS 153098 uS 24847 uS 663167 bytes 50000 13 uS 347281 uS 48650 uS 1275917 bytes 75000 14 uS 579764 uS 72652 uS 1872917 bytes 100000 15 uS 856977 uS 96022 uS 2433167 bytes 125000 17 uS 1160784 uS 119760 uS 2998667 bytes 150000 18 uS 1513262 uS 143609 uS 3609065 bytes 175000 20 uS 1898778 uS 168134 uS 4134665 bytes 200000 22 uS 2339882 uS 195486 uS 4727417 bytes 225000 29 uS 2799415 uS 219007 uS 5329385 bytes 250000 41 uS 3287381 uS 243094 uS 5940569 bytes HASH VIEW ROW ADD FIND COMMIT SIZE 25000 23 uS 69 uS 33690 uS 2037185 bytes 50000 34 uS 73 uS 63885 uS 4713013 bytes 75000 24 uS 83 uS 90765 uS 6285961 bytes 100000 19 uS 83 uS 134531 uS 9706905 bytes 125000 31 uS 88 uS 153336 uS 10755509 bytes 150000 27 uS 89 uS 188538 uS 11804113 bytes 175000 20 uS 83 uS 232263 uS 16506741 bytes 200000 25 uS 84 uS 251398 uS 20701101 bytes 225000 24 uS 86 uS 289060 uS 22798281 bytes 250000 35 uS 96 uS 381825 uS 22798281 bytes BLOCKED HASH VIEW ROW ADD FIND COMMIT SIZE 25000 42 uS 72 uS 28302 uS 1582622 bytes 50000 44 uS 88 uS 60463 uS 3942086 bytes 75000 37 uS 85 uS 83439 uS 4466402 bytes 100000 35 uS 116 uS 120280 uS 7087926 bytes 125000 51 uS 104 uS 146270 uS 9185134 bytes 150000 50 uS 125 uS 164883 uS 9185134 bytes 175000 57 uS 109 uS 236153 uS 12330918 bytes 200000 62 uS 112 uS 240761 uS 14428098 bytes 225000 56 uS 130 uS 267224 uS 14428098 bytes 250000 84 uS 106 uS 281014 uS 20719638 bytes So what do we learn about Commit(). Whatever the kind of view, committing 250,000 rows takes roughly 10 more times that committing 25,000 rows. The relation is in fact linear, as we best see with the hash view (close to 3, 6, 9, 12, 15, 18, etc...). 25,000 x 10 = 250,000 Committing the last 25,000 rows takes as long as committing the 250,000 rows of the full database. We can generalize: whatever the number of new rows you commit, that takes as long as committing the entire database, i.e. if you add 10 rows in a 300,000 rows database, that will take as long as if you commited 300,000 rows. If we enter inside details, that's even worse than that. Let's stick to the hash view example: In 33690 uS, we add 25,000 rows and commit them. In 10 times 33690us, we should be able to add 250,000 rows and commit them. But in fact, in 381825 uS, all we do is adding 25,000 rows and commit them. I hope you see the picture though it's hard to describe. So here is Commit's performance. That seems terribly disabling to me but everyone seems satisfied with it. The conclusion is simple: I'm too demanding. Let's talk about Berkeley DB (excuse me for the swearword, let's call it Bedebeu). I am in no way trying to sell Bedebeu to you. I have never tryed it so I don't know how its Commit performs. But Bedebeu prouds itself to scale up to 256 terabytes. If Bedebeu's Commit performs as Metakit's Commit, programmers certainly think it twice before adding a row. ;-) Sorry Sir Borretos, I've heard you ! "Why don't you use Bedebeu and leave us alone ?" you're saying. Because on paper Metakit satisfies me much more than Bedebeu. Because I've spent quite some time learning Metakit's concepts (by the way, I really don't understand this need to change all commonly used database terms into others). Because I've even designed a structure which suits my needs taking advantage of Metakit's architecture (especially subviews). And because I'm in love with Jean-Claude (I write to him day and night). Let's move on to my second problem: Find() performance. As you can see on JCW's test, there's no problem with Find() at all. It performs smoothly wether there's 25,000 rows or 250,000. And that's where I beg for your assistance, noble assembly. Because I have written an equivalent test program, and with this program I see Find() execution time double, even triple when reaching the 250,000th row. JCW cheched my code and confirmed it was OK. So for a long time, we thought, JCW and I, that it was because my computer did not have 256 Mo of RAM, but 64Mo or 128 Mo according to the computer I used. Que nenni ! My program performed as badly when run by JCW's 256 Mo computer ! Here I am now: I'm pretty sure it's a compilation problem. I'm certain of using the latest cvs sources, so the problem is not there. When I compile \win\msvc60\mksrc.dsp, I get mkvc60.lib, mkvc60_d.lib, mkvc60s.lib, mkvc60s_d.lib, mkvc60s_mfc_d.lib and mkvc60s_std_d.lib. mkvc60s.lib seems to be the right static library to use. But I get with it the bad results I descrive above. And I also get those bad results when using the dll. By the way, mkvc60s.lib is 542 ko. Metakit is supposed to be less than 200 ko... I don't really understand. I would really appreciate if someone confirmed me that I am using Metakit the right way. Or if someone thought of a tricky pitfall I may have fallen into. If you went through all this titanic mail (3h30), you've earned my eternal consideration. ;-) _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From niki@vintech.bg Tue Nov 27 02:38:31 2001 Received: from www.vintech.bg (vintech-ws82.pip.digsys.bg [193.68.201.82] (may be forged)) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA11207 for ; Tue, 27 Nov 2001 02:38:26 -0600 Received: from vintech.bg (sierra [10.0.0.7]) by www.vintech.bg (8.11.2/8.8.7) with ESMTP id fAR8cbs07386 for ; Tue, 27 Nov 2001 10:38:37 +0200 Message-ID: <3C035061.2020608@vintech.bg> Date: Tue, 27 Nov 2001 10:35:45 +0200 From: Niki Spahiev User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:0.9.6) Gecko/20011120 X-Accept-Language: bg, en MIME-Version: 1.0 To: metakit@equi4.com Subject: Re: [Metakit] Retro-spection References: <008501c17704$25690a60$fe24933e@desktop> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Here I am now: I'm pretty sure it's a compilation problem. > I'm certain of using the latest cvs sources, so the problem is not there. > When I compile \win\msvc60\mksrc.dsp, I get mkvc60.lib, mkvc60_d.lib, > mkvc60s.lib, mkvc60s_d.lib, mkvc60s_mfc_d.lib and mkvc60s_std_d.lib. > mkvc60s.lib seems to be the right static library to use. > But I get with it the bad results I descrive above. > And I also get those bad results when using the dll. Well you compare Linux with Windows. We all know which is better. > By the way, mkvc60s.lib is 542 ko. Metakit is supposed to be less than > 200 ko... I don't really understand. Lib files are not 100% code. Even DLL are not 100% code. HTH, Niki Spahiev From steve@improvision.com Tue Nov 27 04:43:01 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA16065 for ; Tue, 27 Nov 2001 04:43:00 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id KAA16821 for ; Tue, 27 Nov 2001 10:33:20 GMT Received: from [192.168.1.25] ([192.168.1.25]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Tue, 27 Nov 2001 10:35:16 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Tue, 27 Nov 2001 10:35:19 +0000 Subject: Re: [Metakit] Retro-spection From: Stephen Baxter To: Pascal Baspeyras , Metakit Message-ID: In-Reply-To: <008501c17704$25690a60$fe24933e@desktop> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is the one that is odd. I thought the whole point of blocked views was that we did *not* have to commit the entire view, just the "block" that has changed. JCW - any thoughts? I did a performance test with a large database last night and commits do rapidly become very slow with a few thousand rows. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: "Pascal Baspeyras" > Date: Tue, 27 Nov 2001 06:27:01 +0100 > To: "Metakit" > Subject: [Metakit] Retro-spection > > BLOCKED VIEW > > ROW ADD FIND COMMIT SIZE > 25000 11 uS 153098 uS 24847 uS 663167 bytes > 50000 13 uS 347281 uS 48650 uS 1275917 bytes > 75000 14 uS 579764 uS 72652 uS 1872917 bytes > 100000 15 uS 856977 uS 96022 uS 2433167 bytes > 125000 17 uS 1160784 uS 119760 uS 2998667 bytes > 150000 18 uS 1513262 uS 143609 uS 3609065 bytes > 175000 20 uS 1898778 uS 168134 uS 4134665 bytes > 200000 22 uS 2339882 uS 195486 uS 4727417 bytes > 225000 29 uS 2799415 uS 219007 uS 5329385 bytes > 250000 41 uS 3287381 uS 243094 uS 5940569 bytes > From lvirden@cas.org Tue Nov 27 06:08:35 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA20639 for ; Tue, 27 Nov 2001 06:08:34 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id fARC0Oj06607 for ; Tue, 27 Nov 2001 07:00:24 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fARC0M020182; Tue, 27 Nov 2001 07:00:22 -0500 (EST) Date: Tue, 27 Nov 2001 07:00:22 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011127070022.AAB19982@cas.org> Subject: Re: [Metakit] Retro-spection In-Reply-To: Your message of Tue, 27 Nov 2001 06:24:06 +0100 To: "Metakit" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: "Pascal Baspeyras" > But on the other hand, subject was serious: I'd really like my application > to be available for free. I have never heard of JCW charging for his > services, > so I would have liked to know what kind of partnership or sponsorship it is > possible to contract. This may certainly not be the right forum to discuss > such subject, but if someone could tell me about his experience or direct me > toward the right forum... Interesting topic - probably not on-topic here. JCW, I'd be happy to redirect elsewhere - do you want us to move over to usenet somewhere? The topic is "how does a programmer make money from open source" and JCW and I have discussed this in private email for years . An interesting experience I recently had was seeing someone who actually had been awarded software patents announcing on comp.lang.tcl some open source software! Here are some ideas - I am not suggesting that any or all of these relate to JCW's situation. Just a variety of thoughts that are available. o custom programming for hire - writing specific applications for a client (which might include some portion of code returnable to the open source community) o support for hire - the priority resolution of particular software in specific applications. While many problems are of a nature that their resolution can come over time, some problems need resolved NOW - and some are willing to pay for that timely service (in this case, it is very important that fixes roll back into the general source tree) o training - the one on one approach to training (for question answering, hand holding, etc.) is sometimes preferred over the reading of man pages (in this case, it would be helpful if improvement to docs, etc. rolled back into the open source community) o extending software/documentation for hire - some people appreciate the open source nature of a product so much they are willing to 'hire' a developer to code or write particular functionality or reference material for the general community. (in this case, the very nature of the work is for improvements to be rolled back into the open source community) > I had performance problems with hash lookup and commit. I don't recall in this painful thread whether the specific code was provided. However, I generally ask people, when reporting problems like "the library is too slow" or "the program doesn't work" to provide small coherent examples of the problem, including the error output, and a detailed description of the environment being used (including version numbers, etc.). I also encourage software writers to develop tools that gather this info to make such problem reports easier to submit... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Tue Nov 27 07:37:13 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA26086 for ; Tue, 27 Nov 2001 07:37:12 -0600 Received: from takkie.equi4.local ([195.121.236.245]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNGO4D00.SZK for ; Tue, 27 Nov 2001 14:29:01 +0100 Received: from [10.0.1.37] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fARDSE028834 for ; Tue, 27 Nov 2001 14:28:14 +0100 From: Jean-Claude Wippler To: Metakit Date: Tue, 27 Nov 2001 14:28:55 +0100 Message-Id: <20011127132855.27220@mail.planet.nl> In-Reply-To: <20011127070022.AAB19982@cas.org> References: <20011127070022.AAB19982@cas.org> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] A plea Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I would like to request a time-out from everyone on this mailing list. There is right and there is wrong. There are facts and there is emotion. There is justice and there is injustice. I am embarrassed to say that I have overlooked something important. Please allow me the time it takes to fully understand and resolve it. When I do, you will all be the first to know. Now that so much energy has been used up, let's try to make the very best use of what still remains. Regards, Jean-Claude From nicolasb@maich.gr Tue Nov 27 07:44:06 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id HAA26530 for ; Tue, 27 Nov 2001 07:44:01 -0600 Received: FROM maich.gr BY mail-server ; Tue Nov 27 15:51:41 2001 +0200 Message-ID: <3C039B04.4A9472B4@maich.gr> Date: Tue, 27 Nov 2001 15:54:12 +0200 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: metakit@equi4.com References: <001801c17708$57df9a00$bc24933e@desktop> Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Subject: [Metakit] Cultural Differences? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Pascal Baspeyras wrote: > > Be warned: this is an un-aggressive mail ! I am sorry Pascal (you asked not to be called mister), to me at this seems like an aggressive mail, or at least mockery to this list.. > > > >Too often I have found in my life people who, because of some > misunderstanding > >- sometimes cultural, sometimes because of translation problems, sometimes > >because of personal problems or issues - end up being 'flamed' in public > >and, because of the embarassment, give up on some particular mailing > >list or product. In the Greek culture, particularly when we do not know someone, out of respect, we refer to someone as Mr. ... Additionally, we address them in the plural but in English there is neither a written nor a spoken distinction between "you guy", and "you guys"; it's just "you". In Greek, "you" singular, trans-literated is "esei", while "you" plural is "eseis" with an "s" appended. I am sorry that you mis-read my intentions. > >What I have tried to do when I find _myself_ in such situations is this: > >1. examine what I have said and determine whether I owe someone an > >apology - either public or private > >2. examine what benefits I get from the product and decide if a public > >apology should be offered - whether I owe it or not. > >3. most of all, determine when I am willing to give up having the last word > >and just letting things go, hoping that after some decent period of time > >I can expect things to 'resume normalcy'. I was really hoping that the last word was your apology yesterday, but alas:-( > > Hello sir Borretos ! Now that noble defensors of Metakit have arrived on > their white horses, we may try to solve this among us, following the ancient > code (love this one). For someone who gets easily irked with monikers, you obviously dont have a problem using them... Please evaluate your sense of humor sir! (no pun intended) and please dont waste my time anymore with your rants poorly disguised as diatribes... > > My mail is getting real long, so I'll split it in two. > > To be continued... Must you??? nicolas boretos (spelled with one r) > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Tue Nov 27 08:20:15 2001 Received: from smtp013.mail.yahoo.com (smtp013.mail.yahoo.com [216.136.173.57]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id IAA28847 for ; Tue, 27 Nov 2001 08:20:15 -0600 Received: from toulouse-2-a7-38-254.dial.proxad.net (HELO desktop) (62.147.38.254) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 14:12:34 -0000 Message-ID: <005b01c1774d$56f3f420$fe26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 15:09:54 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] A plea Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Jean-Claude, You write: >I am embarrassed to say that I have overlooked something important. >Please allow me the time it takes to fully understand and resolve it. >When I do, you will all be the first to know. I suppose you're talking about the Commit() issue which degrades Metakit's performance a lot. It looks like transparent on-the-fly restructuring is a very costly feature which is scarcely used. Wouldn't it be great if we could enable or disable it at will ? Just an idea. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From mlh@idi.ntnu.no Tue Nov 27 09:23:23 2001 Received: from zevs.idi.ntnu.no (IDENT:0@[129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA32559 for ; Tue, 27 Nov 2001 09:23:23 -0600 Received: from bryssel (bryssel.idi.ntnu.no [129.241.111.21]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with SMTP id QAA06288 for ; Tue, 27 Nov 2001 16:15:42 +0100 (MET) Message-ID: <00c601c17756$614af640$156ff181@idi.ntnu.no> From: "Magnus Lie Hetland" To: Date: Tue, 27 Nov 2001 16:15:40 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Recommended article Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Useful guidelines for electronic communication: http://www.rand.org/publications/MR/R3283/ -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.sf.net From baspey@yahoo.com Tue Nov 27 09:50:31 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA01774 for ; Tue, 27 Nov 2001 09:50:30 -0600 Received: from toulouse-2-a7-36-173.dial.proxad.net (HELO desktop) (62.147.36.173) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 15:42:49 -0000 Message-ID: <005c01c17759$f28b61a0$fe26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 16:36:29 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Niki Spahiev wrote: >Lib files are not 100% code. Even DLL are not 100% code. Thanks a lot for this info. I didn't know it could be in such a ratio. Niki Spahiev wrote >Well you compare Linux with Windows. We all know which is better. You're absolutely right. I had thought about that but forgot to mention. It is indeed a possibility that Find() performs right under Linux and wrong under Windows. Here is the C++ program JCW mailed me: #include #include #include #include #include #include long ticks() { struct timeval tv; struct timezone tz; gettimeofday(&tv, &tz); return tv.tv_sec * 1000000 + tv.tv_usec; } int main(int argc, char **argv) { c4_StringProp pKey ("key"); c4_Storage storage ("Database.yumm", true); c4_View data = storage.GetAs("data[key:S]"); if (argc > 1) { if (strchr(argv[1], 'b')) { data = storage.GetAs("data[_B[key:S]]"); data = data.Blocked(); } if (strchr(argv[1], 'h')) { c4_View map = storage.GetAs("map[_H:I,_R:I]"); data = data.Hash(map, 1); } } char buf [25]; c4_Row row; struct stat sb; long t; puts(" ROW ADD FIND COMMIT SIZE"); for (int i = 1; i <= 250000; ++i) { sprintf(buf, "%10d%10d", 100000000 + i, 200000000 + i); pKey (row) = buf; t = ticks(); data.Add(row); long at = ticks() - t; if (i % 1000 == 0) { t = ticks(); int n = data.Find(row); long ft = ticks() - t; if (n < 0) { puts(buf); return 1; } t = ticks(); storage.Commit(); long ct = ticks() - t; if (i % 25000 == 0) { stat("Database.yumm", &sb); printf("%9d %9d uS %9d uS %9d uS %9d bytes \n", i, at, ft, ct, sb.st_size); } } } return 0; } He then launches the program with this command line: % g++ -Dq4_INLINE mkhash.cpp -lmk4 % for i in "" b h bh; do rm -f Database.yumm; a.out $i; done Unfortunatley, Windows API doesn't have gettimeofday() nor stat(). Here are the modifications I propose to run an equivalent program under Windows: int main(int argc, char **argv) { // will be used for performance measuring LARGE_INTEGER Freq, Time1, Time2; QueryPerformanceFrequency(&Freq); double AddTook = 0; double FindTook = 0; double CommitTook = 0; c4_StringProp pKey ("key"); c4_Storage storage ("Database.yumm", true); c4_View data = storage.GetAs("data[key:S]"); /* if (argc > 1) { if (strchr(argv[1], 'b')) { data = storage.GetAs("data[_B[key:S]]"); data = data.Blocked(); } if (strchr(argv[1], 'h')) { c4_View map = storage.GetAs("map[_H:I,_R:I]"); data = data.Hash(map, 1); } } */ // let's do only the blocked/hash test data = storage.GetAs("data[_B[key:S]]"); data = data.Blocked(); c4_View map = storage.GetAs("map[_H:I,_R:I]"); data = data.Hash(map, 1); wxString msg; char buf [25]; c4_Row row; puts(" ROW ADD FIND COMMIT SIZE"); for (int i = 1; i <= 250000; ++i) { sprintf(buf, "%10d%10d", 100000000 + i, 200000000 + i); pKey (row) = buf; QueryPerformanceCounter(&Time1); data.Add(row); QueryPerformanceCounter(&Time2); AddTook = ((double)Time2.QuadPart - (double)Time1.QuadPart) / (double)Freq.QuadPart; if (i % 1000 == 0) { QueryPerformanceCounter(&Time1); int n = data.Find(row); QueryPerformanceCounter(&Time2); FindTook = ((double)Time2.QuadPart - (double)Time1.QuadPart) / (double)Freq.QuadPart; /* if (n < 0) { puts(buf); return; } */ QueryPerformanceCounter(&Time1); storage.Commit(); QueryPerformanceCounter(&Time2); CommitTook = ((double)Time2.QuadPart - (double)Time1.QuadPart) / (double)Freq.QuadPart; if (i % 25000 == 0) { // stat("Database.yumm", &sb); // printf("%9d %9d uS %9d uS %9d uS %9d bytes \n", // i, at, ft, ct, sb.st_size); Printf(_("%d Add() => %f average Find() => %f Commit() => %f"), i, AddTook, FindTook, CommitTook); } } } return 0; } When JCW got almost constant Find times under Linux, here is what I get using the above code under Windows on my Celeron 800 with 64 Mo of RAM: 25,000 rows => 73 us 50,000 rows => 86 us 75,000 rows => 132 us 100,000 rows => 152 us 125,000 rows => 163 us 150,000 rows => 185 us 175,000 rows => 125 us 200,000 rows => 216 us 225,000 rows => 215 us 250,000 rows => 147 us N.B: it's even more worrying if I go beyond 250,000 rows or if I sample more often than every 25,000 rows (code is easy to change). Feel free to try this code on your own computer with your own compiling methods. That may help a bunch. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Tue Nov 27 09:55:50 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA02121 for ; Tue, 27 Nov 2001 09:55:50 -0600 Received: from toulouse-1-a7-33-231.dial.proxad.net (HELO desktop) (62.147.33.231) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 15:48:10 -0000 Message-ID: <006801c1775a$b1678cc0$fe26933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 16:45:03 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Cultural Differences? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas Boretos wrote: >> >> My mail is getting real long, so I'll split it in two. >> >> To be continued... >Must you??? That's the second part which is interesting. The first part was just for relocating things. My apologizes to Magnus Lie Hetland for this first part. ;-) _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Tue Nov 27 10:36:55 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id KAA04470 for ; Tue, 27 Nov 2001 10:36:54 -0600 Received: from toulouse-2-a7-37-171.dial.proxad.net (HELO desktop) (62.147.37.171) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Nov 2001 16:29:14 -0000 Message-ID: <000701c17760$6e23f2e0$ab25933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 27 Nov 2001 17:25:52 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks a lot to Larry W. Virden for rehabilitating my "Is JCW subject". The solutions Larry talk about apply very well to a library like Metakit, and are no real surprise to me. The application I'm working on is rather of that kind: a sympathetical utility which may interest a real lot of users because it's useful and easy to use. Most of this kind of applications are free, though there often is a paying pro version. I think of apps like WinZip, WinAmp, Acrobat Reader, Netscape Navigator, etc... All of them have an important development's cost, but the user don't pays anything. Their fundings are mystery to me. I know this is not the right place, but I'd really like to continue this discussion somewhere. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From horst@freedict.de Tue Nov 27 23:40:37 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA11919 for ; Tue, 27 Nov 2001 23:40:37 -0600 From: horst@freedict.de Received: (qmail 29169 invoked from network); 28 Nov 2001 05:56:01 -0000 Received: from dialin-145-254-112-096.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.112.96) by mail.netbeat.de with SMTP; 28 Nov 2001 05:56:01 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fAS5WBi15086; Wed, 28 Nov 2001 06:32:12 +0100 Message-Id: <200111280532.fAS5WBi15086@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Wed, 28 Nov 2001 06:32:08 +0100 (CET) Reply-To: horst@freedict.de Subject: Re: [Metakit] Retro-spection To: baspey@yahoo.com cc: metakit@equi4.com In-Reply-To: <008501c17704$25690a60$fe24933e@desktop> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, the only remarkable news in this mail I find is: On 27 Nov, Pascal Baspeyras wrote: > Part II > > NORMAL VIEW > > ROW ADD FIND COMMIT SIZE > 250000 7 uS 824172 uS 259353 uS 20499496 bytes > > BLOCKED VIEW > > ROW ADD FIND COMMIT SIZE > 250000 41 uS 3287381 uS 243094 uS 5940569 bytes > > HASH VIEW > > ROW ADD FIND COMMIT SIZE > 250000 35 uS 96 uS 381825 uS 22798281 bytes > > BLOCKED HASH VIEW > > ROW ADD FIND COMMIT SIZE > 250000 84 uS 106 uS 281014 uS 20719638 bytes > and now go back and compare the sizes - I find the blocked view very interesting indeed (not the find performance so) - any hint's? Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From horst@freedict.de Tue Nov 27 23:40:41 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA11923 for ; Tue, 27 Nov 2001 23:40:40 -0600 From: horst@freedict.de Received: (qmail 29193 invoked from network); 28 Nov 2001 05:56:04 -0000 Received: from dialin-145-254-112-096.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.112.96) by mail.netbeat.de with SMTP; 28 Nov 2001 05:56:04 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fAS5Tui15059; Wed, 28 Nov 2001 06:29:57 +0100 Message-Id: <200111280529.fAS5Tui15059@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Wed, 28 Nov 2001 06:29:53 +0100 (CET) Reply-To: horst@freedict.de Subject: Re: [Metakit] Retro-spection To: baspey@yahoo.com cc: metakit@equi4.com In-Reply-To: <008501c17704$25690a60$fe24933e@desktop> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 27 Nov, Pascal Baspeyras wrote: > Part II .... noise .... > > If we enter inside details, that's even worse than that. > Let's stick to the hash view example: > In 33690 uS, we add 25,000 rows and commit them. > In 10 times 33690us, we should be able to add 250,000 rows and > commit them. > But in fact, in 381825 uS, all we do is adding 25,000 rows and > commit them. > I hope you see the picture though it's hard to describe. did you work it out? if you commit 25000 newly added rows at a time, it works out to be 10us per new row, but did you work out how metakit works? it's organised in columns, it's organised in columns, it's organised in columns. The example tells you this: - if things are time critical avoid commits, add as many rows as possible, then commit all new additions - metakit is open source, so as an experienced programmer you are welcome to take the source, have a look at it, and modify it locally to your needs. - When you find them usefull, send patches back to the original author, and he can consider to implement them into the product. - make sure you are fully aware, that you understand that metakit works differently to most other approaches (you mentioned: .. not mentioned ... > > Let's talk about Berkeley DB (excuse me for the swearword, let's call > it Bedebeu). > I am in no way trying to sell Bedebeu to you. > I have never tryed it so I don't know how its Commit performs. > But Bedebeu prouds itself to scale up to 256 terabytes. > If Bedebeu's Commit performs as Metakit's Commit, programmers > certainly think it twice before adding a row. ;-) But adding a row is very fast in MetaKit, it's the commit which appears to bother you. - couldn't you for example just commit on program exit? ... more noise ... -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From baspey@yahoo.com Wed Nov 28 01:30:50 2001 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id BAA15015 for ; Wed, 28 Nov 2001 01:30:49 -0600 Received: from toulouse-2-a7-37-21.dial.proxad.net (HELO desktop) (62.147.37.21) by smtp.mail.vip.sc5.yahoo.com with SMTP; 28 Nov 2001 07:23:05 -0000 Message-ID: <000f01c177dd$4c305c00$1525933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Wed, 28 Nov 2001 08:20:04 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Retro-spection Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tsk, tsk, tsk, Horst... I prefer to forget the aggressive parts of your mails where you say nothing. Let's talk about the argumentative part. If I understand well, Commit() suits your needs perfectly well, and almost everyone must be in your case since nobody complained before me. Maybe Commit's poor performance is only recent, but I couldn't say since I use hash view and thus have to use the latest cvs sources. I am no maniac whose only preoccupation is to measure things. If I took time to measure Find() and Commit() performances, it's because I was looking for a bottleneck I could observe with my bare eyes: the operation of adding 1000 new rows should have (in my idea at least) performed in constant time since I used hash view, but instead it took ever increasing time, very rapidly increasing time. >did you work it out? if you commit 25000 newly added rows at a time, it >works out to be 10us per new row, but did you work out how metakit >works? it's organised in columns, it's organised in columns, it's >organised in columns. Column-wise organisation ? Yeah, heard about something like that. By the way, this test program is only 1 column. So I guess I didn't really work out how Metakit works because I don't understand why column-wise organisation should justify such results against row-wise organisation. I may be wrong but I think that 25,000 newly added rows means 10 us per row only if the database was blank, but it will become 100 us if the database already had 225,000 rows. I come to believe that you really missed my point... >if things are time critical avoid commits, add as many rows as >possible, then commit all new additions. Do you know of many database applications where you have all data from the start and where you just read it after that ? In my case, only the more addicted user will reach 300,000 rows. But as an average user, he won't have any idea of my program's underlying structure, as if I'm using Metakit or not. He will be placed in the case where he only made few additions and wait minutes for program to close (guess he will blame windows as always, unless he uses the Linux or Mac version of my program). 300,000 rows should be the biggest view. But there will 2 such views, plus a dictionary for fast word lookup. On this mailing list, someone once asked JCW if it was better to split a database into many. JCW told him it has no impact and using a single database would be more practical. According to my observation, I must say that this is a false statement. I also disagree with the documentation which states that using a blocked view will allow your database to "scale up indefinitely". I can't see in the test any benefit due to blocked view. On the contrary, "due to the extra level of indirection, blocked views are slightly slower". >But adding a row is very fast in MetaKit, it's the commit which appears >to bother you. - couldn't you for example just commit on program exit? As I said, my program will commit only on closing, but most of the time the user will have only made tens or hundreds of additions, like always in real-life. And I insist: the fact that Add() is fast can be completely ruined by Commit() performance. Maybe we should make a poll to know if people consider Commit's behaviour to be pathological or not ? Let me remind you my case: I really couldn't believe that was normal behaviour ( hence the "Performance can't be that bad" title) and asked help to know what I was doing wrong (did you really read my mails or did you stop to the fact that it's good to hate me ?). JCW agreed with me and tried hard to find out were I was doing wrong. Until the point we are now, were we know that this really is Metakit's "normal" behaviour. >metakit is open source, so as an experienced programmer you are >welcome to take the source, have a look at it, and modify it locally >to your needs. - When you find them usefull, send patches back to the >original author, and he can consider to implement them into the >product. If nothing changes, it's a (very unlikely) possibility that I modify sources for my "special" needs. That don't inspire me much since it will then be tedious to follow Metakit's evolution. Here is my position now: I made a bug report and am waiting for JCW conclusions. I think he understands the situation better than you do, and that's a chance. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jcw@equi4.com Wed Nov 28 08:28:46 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA02023 for ; Wed, 28 Nov 2001 08:28:46 -0600 Received: from takkie.equi4.local ([195.121.234.97]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNIL6602.BNJ for ; Wed, 28 Nov 2001 15:20:30 +0100 Received: from [10.0.1.38] (airsie.equi4.local [195.108.246.51]) by takkie.equi4.local (8.11.2/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id fASEJp006457 for ; Wed, 28 Nov 2001 15:19:52 +0100 From: Jean-Claude Wippler To: metakit Date: Wed, 28 Nov 2001 15:21:48 +0100 Message-Id: <20011128142148.5023@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] commit performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Here are the results of my timing experiments and changes. I'll try to keep it short, but need to throw figures around to explain things. The results below come from examples/mkhash.cpp, now in CVS (and as tar in the previews/ area). All tests were performed on the same Linux (SuSE 7.3) and PIII/650 + 256 Mb RAM notebook as before. First, building up a view of 250,000 rows, committing every 998, then 2 rows (that last commit is timed, i.e. a 2-row change). This is a hash view, with the data stored flat and the map not stored: dhs ROW ADD FIND COMMIT SIZE 25000 21 uS 154 uS 15077 uS 1936098 bytes 50000 36 uS 114 uS 32699 uS 3427104 bytes 75000 21 uS 112 uS 49081 uS 6094110 bytes 100000 22 uS 115 uS 61174 uS 8131092 bytes 125000 28 uS 130 uS 77383 uS 8131092 bytes 150000 29 uS 131 uS 94857 uS 10819116 bytes 175000 16 uS 121 uS 111775 uS 14389140 bytes 200000 26 uS 133 uS 119887 uS 14389140 bytes 225000 21 uS 129 uS 149678 uS 14389140 bytes 250000 39 uS 137 uS 166089 uS 19135164 bytes Adds and finds are O(1). Commit is O(N). File size is roughly O(N). Now the same, using a blocked view for the data: Dhs ROW ADD FIND COMMIT SIZE 25000 25 uS 116 uS 2177 uS 687868 bytes 50000 36 uS 104 uS 3989 uS 1436068 bytes 75000 26 uS 102 uS 5950 uS 2158018 bytes 100000 22 uS 99 uS 7642 uS 2853718 bytes 125000 49 uS 110 uS 9170 uS 3523168 bytes 150000 32 uS 110 uS 11171 uS 4166368 bytes 175000 24 uS 108 uS 12746 uS 4783318 bytes 200000 30 uS 113 uS 14435 uS 5374018 bytes 225000 29 uS 114 uS 16264 uS 5956570 bytes 250000 41 uS 123 uS 17759 uS 6494770 bytes Adds and finds are not affected much. Commit time is lower, because only changes to the modified (subview-) block need to be committed. File size is lower, because many small columns now reuse free space. File space usage is not much different, but the first case leads to a lot of free space in the file (re-used, but present nevertheless). The above figures are what happens after I fixed a major performance bug, if you compare with what has been posted before, there is a huge difference. The bug affects all string/bytes properties, when there are many views/subviews. Strings are stored as 2..3 columns, one with all data, the second with all string sizes, and the third with info about oversize (previously called "memo") entries. The bug was that sizes and oversize info was *always* written, even for columns in which nothing at all had changed in the string data. That means that for blocked views, sizes etc (but not the data itself) was being written to disk on commit for *all* blocks, including untouched ones. The current code no longer does an increasing number of writes in the 10 commits from 25k..250k rows (this has been verified with strace). But there still is an O(N) behavior in here. My current explanation for it is the fact that MK does full free-space re-determination on each commit, and this is proportional to the number of columns that are present in the entire datafile. That calculation could be tuned further (still O(N)), and there are provisions in the file format to cluster subtrees of data so the entire tree space is accounted for in one step. I have not yet explored that, and have to add that I may not get to it for some time to come (this is one reason for offering an Enterprise License: to reorder priorities and make things happen). In short: now that this particular I/O bottleneck has been removed, another CPU-bound - and O(N) - one moves to the surface. I think. I have tweaked a bit of the inlines, which showed up as bottlenecks in some profiling tests, but with mixed results (the German term for it is "verschlimmbesserung", i.e. 2 steps forwards, 2..3 steps back). I suspect that these are CPU cache and cache alignment effects. My priority was to better understand the issues, and subsequently to make sure that the change would continue to produce correct results. There are no leaks AFAIK, all regression tests pass, and so do the Tcl tests (they did bring up one bug in the changes, now resolved). There are still several improvements to make: the blocked view uses a fixed 500..1000 row block size, it could be adaptive based on the total number of rows, to lead to more scalable O(log N) performance. The free-space calculations could be done differently, in ways which will alter the O(x) effects and in simply streamlining it further. So where does this leave MK? Again, I think it illustrates that the performance of column-wise storage continues to be a highly complex tradeoff. Commits have always been the Achilles' heel, which is not so surprising, considering what commit does (even with one change). Blocked views are (now!) a step in the right direction, IMO. The untapped option for now, is commit-aside, which was specifically designed to bring differential commits into the picture. But it is not ready for prime time (in that it is not even differential today). At a higher level, another option is to create a difference custom viewer, i.e. one which caches changes, but makes them appear "real". An where does that leave me? Well, I must say that I was premature in rejecting the possibility that there could be a fault in Metakit. Inside every criticism there is a grain of truth - next time I will spend more effort looking for that. If I have offended anyone on this list with my defensiveness I offer my sincere apologies. But it pleases me that there's a winner in this, after all. MetaKit has evolved further, and thanks to everyone, performs significantly better than before in certain circumstances. Of course, whether that is sufficient for each of your needs remains up to you. Regards, Jean-Claude From lvirden@cas.org Wed Nov 28 08:40:13 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA02779 for ; Wed, 28 Nov 2001 08:40:13 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.9.3/CAS_MAIL_HUB-1.14) with ESMTP id fASEVwj28424 for ; Wed, 28 Nov 2001 09:31:58 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fASEVuk14756; Wed, 28 Nov 2001 09:31:56 -0500 (EST) Date: Wed, 28 Nov 2001 09:31:56 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011128093155.AAB14672@cas.org> Subject: Re: [Metakit] commit performance In-Reply-To: <20011128142148.5023@mail.planet.nl> of Wed, 28 Nov 2001 15:21:48 +0100 To: metakit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler > But it pleases me that there's a winner in this, after all. MetaKit > has evolved further, and thanks to everyone, performs significantly > better than before in certain circumstances. Of course, whether > that is sufficient for each of your needs remains up to you. Despite many criticisms people have of open source code, it is experiences like this that continue to keep me cheering free software on. I have had experiences in the past where I have provided detailed bug reports to vendors about their closed source application behaviors. In one case, I even was able to provide a code fix to a script that they were shipping which had incorrect behavior. The attitude was "well, we don't wish to be bothered fixing that problem - it is too low a priority". In my case, I just continued using my version of the script, and felt bad for all the people also using the product who either were unaware of the problem or not able to create the fix. In other cases, serious data damaging bugs were simply documented as "don't give sort lines longer than 1k as sort will break those lines" and so on. This too seems to me to be a bad approach - to know of a serious bug, and to simply document it as a limitation rather than spending the time to fix things. But, one might argue, the vendor has to allocate their resources carefully, picking the highest priority bugs to fix. And I would not disagree. I would, however, counter with the idea that with open source, one has the option of dealing with one's personal bugs directly, rather than waiting for the problem to rise on someone else's priority list... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From baspey@yahoo.com Wed Nov 28 09:31:44 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA05929 for ; Wed, 28 Nov 2001 09:31:44 -0600 Received: from toulouse-2-a7-39-154.dial.proxad.net (HELO desktop) (62.147.39.154) by smtp.mail.vip.sc5.yahoo.com with SMTP; 28 Nov 2001 15:23:47 -0000 Message-ID: <001001c17820$726e4380$9a27933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Wed, 28 Nov 2001 16:21:07 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] commit performance Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >If I have offended anyone on this list with my defensiveness I >offer my sincere apologies. I take them with pleasure. ;-) Larry W. Virden wrote: >Despite many criticisms people have of open source code, it is experiences >like this that continue to keep me cheering free software on. Me too. I always had had good experiences before. And now, let's go take a look at this cvs ! _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jyl@best.com Wed Nov 28 09:56:32 2001 Received: from sj1-3-4-18.iserver.com (sj1-3-4-18.iserver.com [192.220.127.211]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA07549 for ; Wed, 28 Nov 2001 09:56:32 -0600 Received: (qmail 14087 invoked by uid 19667); 28 Nov 2001 15:48:47 -0000 Received: from unknown (HELO best.com) ([unknown]) (envelope-sender ) by unknown (qmail-ldap-1.03) with SMTP for ; 28 Nov 2001 15:48:47 -0000 Message-ID: <3C050994.77F0298D@best.com> Date: Wed, 28 Nov 2001 07:58:12 -0800 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Subject: Re: [Metakit] commit performance References: <20011128142148.5023@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks, JC! Very good. That explains why even when I changed the data type in the program you wrote and which Pascal posted to I from S, I still saw (much faster! :) O(N) commit behavior. I modified the program to store the index I in the Ith row. I also verified by changing the program to commit only every 25K that that delaying the commit does not make a huge difference, neither in size nor in time, which means that your free space reclamation is a waste of time for each commit -- it seems OK to do it only once in a while. Finally, because I was running on Solaris, I thought to test the effects of NFS and remoteness of the file system on how Commit behaves. I could post the results when I get into work, later. The salient bit of info is that Commit time seems dominated by I/O overhead and that NFS can make Commit take upto 4X more time on a remote mounted FS than on a local one... Sheesh :( (not MK's fault). One question: as I wrote before, I am addressing rows by index and really need to know what row# a row occupied. Does this rule out use of Hashed or Blocked views? I could of course modify my schema to include my own index in each row, but it seems (MUCH) less efficient to find a row by searching with Find than byjust saying view[i]. Any insights appreciated! --JYL Jean-Claude Wippler wrote: > Here are the results of my timing experiments and changes. I'll try > to keep it short, but need to throw figures around to explain things. > The results below come from examples/mkhash.cpp, now in CVS (and as > tar in the previews/ area). All tests were performed on the same > Linux (SuSE 7.3) and PIII/650 + 256 Mb RAM notebook as before. > > First, building up a view of 250,000 rows, committing every 998, then > 2 rows (that last commit is timed, i.e. a 2-row change). This is a > hash view, with the data stored flat and the map not stored: > > dhs > ROW ADD FIND COMMIT SIZE > 25000 21 uS 154 uS 15077 uS 1936098 bytes > 50000 36 uS 114 uS 32699 uS 3427104 bytes > 75000 21 uS 112 uS 49081 uS 6094110 bytes > 100000 22 uS 115 uS 61174 uS 8131092 bytes > 125000 28 uS 130 uS 77383 uS 8131092 bytes > 150000 29 uS 131 uS 94857 uS 10819116 bytes > 175000 16 uS 121 uS 111775 uS 14389140 bytes > 200000 26 uS 133 uS 119887 uS 14389140 bytes > 225000 21 uS 129 uS 149678 uS 14389140 bytes > 250000 39 uS 137 uS 166089 uS 19135164 bytes > > Adds and finds are O(1). Commit is O(N). File size is roughly O(N). > > Now the same, using a blocked view for the data: > > Dhs > ROW ADD FIND COMMIT SIZE > 25000 25 uS 116 uS 2177 uS 687868 bytes > 50000 36 uS 104 uS 3989 uS 1436068 bytes > 75000 26 uS 102 uS 5950 uS 2158018 bytes > 100000 22 uS 99 uS 7642 uS 2853718 bytes > 125000 49 uS 110 uS 9170 uS 3523168 bytes > 150000 32 uS 110 uS 11171 uS 4166368 bytes > 175000 24 uS 108 uS 12746 uS 4783318 bytes > 200000 30 uS 113 uS 14435 uS 5374018 bytes > 225000 29 uS 114 uS 16264 uS 5956570 bytes > 250000 41 uS 123 uS 17759 uS 6494770 bytes > > Adds and finds are not affected much. Commit time is lower, because > only changes to the modified (subview-) block need to be committed. > File size is lower, because many small columns now reuse free space. > File space usage is not much different, but the first case leads to > a lot of free space in the file (re-used, but present nevertheless). > > The above figures are what happens after I fixed a major performance > bug, if you compare with what has been posted before, there is a huge > difference. The bug affects all string/bytes properties, when there > are many views/subviews. Strings are stored as 2..3 columns, one > with all data, the second with all string sizes, and the third with > info about oversize (previously called "memo") entries. The bug was > that sizes and oversize info was *always* written, even for columns > in which nothing at all had changed in the string data. That means > that for blocked views, sizes etc (but not the data itself) was being > written to disk on commit for *all* blocks, including untouched ones. > > The current code no longer does an increasing number of writes in the > 10 commits from 25k..250k rows (this has been verified with strace). > > But there still is an O(N) behavior in here. My current explanation > for it is the fact that MK does full free-space re-determination on > each commit, and this is proportional to the number of columns that > are present in the entire datafile. That calculation could be tuned > further (still O(N)), and there are provisions in the file format to > cluster subtrees of data so the entire tree space is accounted for in > one step. I have not yet explored that, and have to add that I may > not get to it for some time to come (this is one reason for offering > an Enterprise License: to reorder priorities and make things happen). > > In short: now that this particular I/O bottleneck has been removed, > another CPU-bound - and O(N) - one moves to the surface. I think. > > I have tweaked a bit of the inlines, which showed up as bottlenecks > in some profiling tests, but with mixed results (the German term for > it is "verschlimmbesserung", i.e. 2 steps forwards, 2..3 steps back). > I suspect that these are CPU cache and cache alignment effects. > > My priority was to better understand the issues, and subsequently to > make sure that the change would continue to produce correct results. > There are no leaks AFAIK, all regression tests pass, and so do the > Tcl tests (they did bring up one bug in the changes, now resolved). > > There are still several improvements to make: the blocked view uses > a fixed 500..1000 row block size, it could be adaptive based on the > total number of rows, to lead to more scalable O(log N) performance. > The free-space calculations could be done differently, in ways which > will alter the O(x) effects and in simply streamlining it further. > > So where does this leave MK? Again, I think it illustrates that the > performance of column-wise storage continues to be a highly complex > tradeoff. Commits have always been the Achilles' heel, which is not > so surprising, considering what commit does (even with one change). > Blocked views are (now!) a step in the right direction, IMO. > > The untapped option for now, is commit-aside, which was specifically > designed to bring differential commits into the picture. But it is > not ready for prime time (in that it is not even differential today). > At a higher level, another option is to create a difference custom > viewer, i.e. one which caches changes, but makes them appear "real". > > An where does that leave me? Well, I must say that I was premature > in rejecting the possibility that there could be a fault in Metakit. > Inside every criticism there is a grain of truth - next time I will > spend more effort looking for that. If I have offended anyone on > this list with my defensiveness I offer my sincere apologies. > > But it pleases me that there's a winner in this, after all. MetaKit > has evolved further, and thanks to everyone, performs significantly > better than before in certain circumstances. Of course, whether > that is sufficient for each of your needs remains up to you. > > Regards, > Jean-Claude > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jyl@best.com Wed Nov 28 09:56:43 2001 Received: from sj1-3-4-18.iserver.com (sj1-3-4-18.iserver.com [192.220.127.211]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA07575 for ; Wed, 28 Nov 2001 09:56:43 -0600 Received: (qmail 14100 invoked by uid 19667); 28 Nov 2001 15:48:59 -0000 Received: from unknown (HELO best.com) ([unknown]) (envelope-sender ) by unknown (qmail-ldap-1.03) with SMTP for ; 28 Nov 2001 15:48:59 -0000 Message-ID: <3C0509A0.3602CC8D@best.com> Date: Wed, 28 Nov 2001 07:58:24 -0800 From: Jacob Levy Organization: Leage of Retired Slug Programmers X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: metakit Subject: Re: [Metakit] commit performance References: <20011128142148.5023@mail.planet.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks, JC! Very good. That explains why even when I changed the data type in the program you wrote and which Pascal posted to I from S, I still saw (much faster! :) O(N) commit behavior. I modified the program to store the index I in the Ith row. I also verified by changing the program to commit only every 25K that that delaying the commit does not make a huge difference, neither in size nor in time, which means that your free space reclamation is a waste of time for each commit -- it seems OK to do it only once in a while. Finally, because I was running on Solaris, I thought to test the effects of NFS and remoteness of the file system on how Commit behaves. I could post the results when I get into work, later. The salient bit of info is that Commit time seems dominated by I/O overhead and that NFS can make Commit take upto 4X more time on a remote mounted FS than on a local one... Sheesh :( (not MK's fault). One question: as I wrote before, I am addressing rows by index and really need to know what row# a row occupied. Does this rule out use of Hashed or Blocked views? I could of course modify my schema to include my own index in each row, but it seems (MUCH) less efficient to find a row by searching with Find than byjust saying view[i]. Any insights appreciated! --JYL Jean-Claude Wippler wrote: > Here are the results of my timing experiments and changes. I'll try > to keep it short, but need to throw figures around to explain things. > The results below come from examples/mkhash.cpp, now in CVS (and as > tar in the previews/ area). All tests were performed on the same > Linux (SuSE 7.3) and PIII/650 + 256 Mb RAM notebook as before. > > First, building up a view of 250,000 rows, committing every 998, then > 2 rows (that last commit is timed, i.e. a 2-row change). This is a > hash view, with the data stored flat and the map not stored: > > dhs > ROW ADD FIND COMMIT SIZE > 25000 21 uS 154 uS 15077 uS 1936098 bytes > 50000 36 uS 114 uS 32699 uS 3427104 bytes > 75000 21 uS 112 uS 49081 uS 6094110 bytes > 100000 22 uS 115 uS 61174 uS 8131092 bytes > 125000 28 uS 130 uS 77383 uS 8131092 bytes > 150000 29 uS 131 uS 94857 uS 10819116 bytes > 175000 16 uS 121 uS 111775 uS 14389140 bytes > 200000 26 uS 133 uS 119887 uS 14389140 bytes > 225000 21 uS 129 uS 149678 uS 14389140 bytes > 250000 39 uS 137 uS 166089 uS 19135164 bytes > > Adds and finds are O(1). Commit is O(N). File size is roughly O(N). > > Now the same, using a blocked view for the data: > > Dhs > ROW ADD FIND COMMIT SIZE > 25000 25 uS 116 uS 2177 uS 687868 bytes > 50000 36 uS 104 uS 3989 uS 1436068 bytes > 75000 26 uS 102 uS 5950 uS 2158018 bytes > 100000 22 uS 99 uS 7642 uS 2853718 bytes > 125000 49 uS 110 uS 9170 uS 3523168 bytes > 150000 32 uS 110 uS 11171 uS 4166368 bytes > 175000 24 uS 108 uS 12746 uS 4783318 bytes > 200000 30 uS 113 uS 14435 uS 5374018 bytes > 225000 29 uS 114 uS 16264 uS 5956570 bytes > 250000 41 uS 123 uS 17759 uS 6494770 bytes > > Adds and finds are not affected much. Commit time is lower, because > only changes to the modified (subview-) block need to be committed. > File size is lower, because many small columns now reuse free space. > File space usage is not much different, but the first case leads to > a lot of free space in the file (re-used, but present nevertheless). > > The above figures are what happens after I fixed a major performance > bug, if you compare with what has been posted before, there is a huge > difference. The bug affects all string/bytes properties, when there > are many views/subviews. Strings are stored as 2..3 columns, one > with all data, the second with all string sizes, and the third with > info about oversize (previously called "memo") entries. The bug was > that sizes and oversize info was *always* written, even for columns > in which nothing at all had changed in the string data. That means > that for blocked views, sizes etc (but not the data itself) was being > written to disk on commit for *all* blocks, including untouched ones. > > The current code no longer does an increasing number of writes in the > 10 commits from 25k..250k rows (this has been verified with strace). > > But there still is an O(N) behavior in here. My current explanation > for it is the fact that MK does full free-space re-determination on > each commit, and this is proportional to the number of columns that > are present in the entire datafile. That calculation could be tuned > further (still O(N)), and there are provisions in the file format to > cluster subtrees of data so the entire tree space is accounted for in > one step. I have not yet explored that, and have to add that I may > not get to it for some time to come (this is one reason for offering > an Enterprise License: to reorder priorities and make things happen). > > In short: now that this particular I/O bottleneck has been removed, > another CPU-bound - and O(N) - one moves to the surface. I think. > > I have tweaked a bit of the inlines, which showed up as bottlenecks > in some profiling tests, but with mixed results (the German term for > it is "verschlimmbesserung", i.e. 2 steps forwards, 2..3 steps back). > I suspect that these are CPU cache and cache alignment effects. > > My priority was to better understand the issues, and subsequently to > make sure that the change would continue to produce correct results. > There are no leaks AFAIK, all regression tests pass, and so do the > Tcl tests (they did bring up one bug in the changes, now resolved). > > There are still several improvements to make: the blocked view uses > a fixed 500..1000 row block size, it could be adaptive based on the > total number of rows, to lead to more scalable O(log N) performance. > The free-space calculations could be done differently, in ways which > will alter the O(x) effects and in simply streamlining it further. > > So where does this leave MK? Again, I think it illustrates that the > performance of column-wise storage continues to be a highly complex > tradeoff. Commits have always been the Achilles' heel, which is not > so surprising, considering what commit does (even with one change). > Blocked views are (now!) a step in the right direction, IMO. > > The untapped option for now, is commit-aside, which was specifically > designed to bring differential commits into the picture. But it is > not ready for prime time (in that it is not even differential today). > At a higher level, another option is to create a difference custom > viewer, i.e. one which caches changes, but makes them appear "real". > > An where does that leave me? Well, I must say that I was premature > in rejecting the possibility that there could be a fault in Metakit. > Inside every criticism there is a grain of truth - next time I will > spend more effort looking for that. If I have offended anyone on > this list with my defensiveness I offer my sincere apologies. > > But it pleases me that there's a winner in this, after all. MetaKit > has evolved further, and thanks to everyone, performs significantly > better than before in certain circumstances. Of course, whether > that is sufficient for each of your needs remains up to you. > > Regards, > Jean-Claude > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Thu Nov 29 03:36:29 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id DAA03431 for ; Thu, 29 Nov 2001 03:36:28 -0600 Received: from [10.0.1.39] ([195.121.234.104]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNK2AY00.Q2R for ; Thu, 29 Nov 2001 10:28:10 +0100 From: Jean-Claude Wippler Cc: metakit Subject: Re: [Metakit] commit performance Date: Thu, 29 Nov 2001 10:29:34 +0100 Message-Id: <20011129092934.629@mail.planet.nl> In-Reply-To: <3C0509A0.3602CC8D@best.com> References: <3C0509A0.3602CC8D@best.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: > [...] I also verified by changing the program to commit >only every 25K that that delaying the commit does not make a huge >difference, neither in size nor in time, which means that your free space >reclamation is a waste of time for each commit -- it seems OK to do it >only once in a while. That's no so easy to change. I need to carry more info across a commit, right now commit cleans up a lot, and lets the next one re-determine things. Note that free space use is not stored on disk - it is implicit in the tree - so storing anything at all requires knowing the layout of the entire data structure on disk. But it's a good idea, now that we're this far such an optimization might be quite worthwhile. [...] >One question: as I wrote before, I am addressing rows by index and really >need to know what row# a row occupied. Does this rule out use of Hashed >or Blocked views? I could of course modify my schema to include my own >index in each row, but it seems (MUCH) less efficient to find a row by >searching with Find than byjust saying view[i]. Any insights appreciated! Blocked views will behave like any other view, you can in principle replace anything by blocked views. The trade-off is that access to row #i takes a bit more time (there's now a tiny scan loop, whereas flat indexing is immediate). Find can search fast, but only under certain circumstances: - if the underlying view is a hashed one, and if the find key includes all of the ones on which the has works (usually simply the first prop in the view), then it takes advantage of hashing, as the examples/mkhash.cpp code shows - if the underlying view is ordered, and as above, the key is appropriate, then again Find switches to binary search internally (as doing Search yourself) - in other cases, and for remaining properties, Find resorts to linear scan You say: >I am addressing rows by index and really >need to know what row# a row occupied. I don't understand. By index in which view? Row# in which view? Aren't those two the same thing? -jcw From baspey@yahoo.com Thu Nov 29 05:21:11 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id FAA07662 for ; Thu, 29 Nov 2001 05:21:11 -0600 Received: from toulouse-1-a7-32-121.dial.proxad.net (HELO desktop) (62.147.32.121) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 11:13:21 -0000 Message-ID: <003701c178c6$9f85fa60$7920933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 12:11:38 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello everybody, I guess you will say that I'm the kind of guy who's never satisfied, but I still have problems with Metakit. The fact is that JCW performed all his tests under Linux, while my application performed under Windows. And when I ran my application, it was really obvious, even before measuring, that something was wrong with Commit() and Find(). At last, I have come to understand why so many persons talked about "my special needs": the figures given by JWC were good, and are even better since the latest cvs changes. Indeed, even before the recent bug fix, Commit() never took more than 1/3 second, which would have been quite satisfying for my needs. Alas, I was far from having those good results under Windows... And if JCW found a bug, that's a pure coincidence if I spotted it. If I did not have had my Windows' problems, he would not have sent me his Commit() performance at all. Let me remind you the results JCW gets using examples\mkhash.cpp on his PIII/650 + 256 Mb RAM notebook: dhs ROW ADD FIND COMMIT SIZE 25000 21 uS 154 uS 15077 uS 1936098 bytes 50000 36 uS 114 uS 32699 uS 3427104 bytes 75000 21 uS 112 uS 49081 uS 6094110 bytes 100000 22 uS 115 uS 61174 uS 8131092 bytes 125000 28 uS 130 uS 77383 uS 8131092 bytes 150000 29 uS 131 uS 94857 uS 10819116 bytes 175000 16 uS 121 uS 111775 uS 14389140 bytes 200000 26 uS 133 uS 119887 uS 14389140 bytes 225000 21 uS 129 uS 149678 uS 14389140 bytes 250000 39 uS 137 uS 166089 uS 19135164 bytes Dhs ROW ADD FIND COMMIT SIZE 25000 25 uS 116 uS 2177 uS 687868 bytes 50000 36 uS 104 uS 3989 uS 1436068 bytes 75000 26 uS 102 uS 5950 uS 2158018 bytes 100000 22 uS 99 uS 7642 uS 2853718 bytes 125000 49 uS 110 uS 9170 uS 3523168 bytes 150000 32 uS 110 uS 11171 uS 4166368 bytes 175000 24 uS 108 uS 12746 uS 4783318 bytes 200000 30 uS 113 uS 14435 uS 5374018 bytes 225000 29 uS 114 uS 16264 uS 5956570 bytes 250000 41 uS 123 uS 17759 uS 6494770 bytes And here is what I get under Windows with my Celeron 800 + 64 Mb RAM notebook: dhs ROW ADD FIND COMMIT 25000 18 uS 368 uS 274497 uS 50000 36 uS 416 uS 518412 uS 75000 20 uS 450 uS 740287 uS 100000 27 uS 452 uS 970711 uS 125000 25 uS 217 uS 1096427 uS 150000 33 uS 505 uS 1432881 uS 175000 17 uS 603 uS 4129216 uS 200000 33 uS 593 uS 6728838 uS 225000 22 uS 299 uS 8930381 uS 250000 41 uS 336 uS 7629655 uS Dhs ROW ADD FIND COMMIT 25000 20 uS 223 uS 69844 uS 50000 29 uS 247 uS 52325 uS 75000 22 uS 246 uS 56790 uS 100000 20 uS 269 uS 73695 uS 125000 72 uS 244 uS 69494 uS 150000 28 uS 256 uS 52919 uS 175000 21 uS 218 uS 69990 uS 200000 29 uS 423 uS 44764 uS 225000 26 uS 554 uS 83888 uS 250000 38 uS 510 uS 61994 uS (I don't think this big difference may be due to the fact that my computer has "only" 64 Mb RAM. Indeed, I had already sent to JCW a test program which performed poorly with his 256 Mb computer) I have to precise that Dhs performs good thanks to the recent bug fix, but performed bad before. But since I am under Windows, I couldn't run examples\mkhash.cpp as it, because Windows' API does not have such things as gettimeofday() or stat(). I had to modify things, and there's always a possibility that I made wrong somewhere. For your information, here is how I made to execute examples\mkhash.cpp under Windows: I used \win\msvc60\mhtest.dsp as a fundation. I replaced the main procedure (found in file \test\regress.cpp) with this one: #include #include #include #include #include #include int main() { char buf [25]; c4_Row row; //long t; LARGE_INTEGER Freq, t1, t2; QueryPerformanceFrequency(&Freq); double at, ct, ft, ht; int nkeys = 1; setvbuf(stdout, NULL, _IOLBF, BUFSIZ); c4_Storage storage ("test.dat", true); c4_StringProp pKey ("key"), pKey2 ("key2"); c4_View data = pKey; c4_IntProp pH ("_H"), pR ("_R"); c4_View map = (pH, pR); //const char *s = argc > 1 ? argv[1] : "-"; char *s = ""; gets(s); if (strchr(s, '2')) { pKey2 (row) = "abcdefghijklmnopqrstuvwxyz"; nkeys = 2; if (strchr(s, 'd')) data = storage.GetAs("data[key:S,key2:S]"); if (strchr(s, 'D')) { data = storage.GetAs("data[_B[key:S,key2:S]]"); data = data.Blocked(); } } else { if (strchr(s, 'd')) data = storage.GetAs("data[key:S]"); if (strchr(s, 'D')) { data = storage.GetAs("data[_B[key:S]]"); data = data.Blocked(); } } if (strchr(s, 'm')) map = storage.GetAs("map[_H:I,_R:I]"); if (strchr(s, 'M')) { map = storage.GetAs("map[_B[_H:I,_R:I]]"); map = map.Blocked(); } if (strchr(s, 'h')) data = data.Hash(map, nkeys); if (strchr(s, 'o')) data = data.Ordered(); int limit = 250000; /* if (argc > 2) limit = atoi(argv[2]); puts(s); */ puts(" ROW ADD FIND COMMIT"); for (int i = 1; i <= limit; ++i) { sprintf(buf, "%10d%10d", 100000000 + i, 200000000 + i); pKey (row) = buf; // t = ticks(); QueryPerformanceCounter(&t1); data.Add(row); // long at = ticks() - t; QueryPerformanceCounter(&t2); at = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; // if ((i+2) % 1000 == 0 && strchr(s, 's')) if ((i+2) % 1000 == 0) storage.Commit(); if (i % 1000 == 0) { // t = ticks(); QueryPerformanceCounter(&t1); storage.Commit(); // long ct = ticks() - t; QueryPerformanceCounter(&t2); ct = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; if (i % (limit/10) == 0) { // t = ticks(); QueryPerformanceCounter(&t1); int n = data.Find(row); // long ft = ticks() - t; QueryPerformanceCounter(&t2); ft = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; if (n < 0) { puts(buf); return 1; } /* struct stat sb; stat("test.dat", &sb); printf("%9d %9d uS %9d uS %9d uS %9d bytes \n", i, at, ft, ct, sb.st_size); */ printf("%9d %f uS %f uS %f uS\n", i, at*1000000, ft*1000000, ct*1000000); } } } if (strchr(s, 'H')) { // t = ticks(); QueryPerformanceCounter(&t1); data = data.Hash(map, nkeys); // long ht = ticks() - t; QueryPerformanceCounter(&t2); ht = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; // t = ticks(); QueryPerformanceCounter(&t1); storage.Commit(); // long ct = ticks() - t; QueryPerformanceCounter(&t2); ct = ((double)t2.QuadPart - (double)t1.QuadPart) / (double)Freq.QuadPart; printf("construct hash: %d uS, then commit: %d uS\n", ht, ct); } return 0; } Then, I selected the menu item "Build" - "Set Active Configuration" and chose "mktest - Win32Release". And to finish, I selected the menu item "Project" - "Settings", then the "Link" tab, and in the "Object/library modules" textbox, I wrote "..\..\builds\mk4vc60s.lib". In my idea, Metakit has a bug under Windows, which is the source of all my troubles from the beginning... Does someone object ? _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Thu Nov 29 06:00:18 2001 Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA09194 for ; Thu, 29 Nov 2001 06:00:17 -0600 Received: from toulouse-1-a7-32-121.dial.proxad.net (HELO desktop) (62.147.32.121) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 11:52:26 -0000 Message-ID: <001e01c178cc$15376d20$7920933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 12:50:43 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'd just like to add that I now understand perfectly the attitude of JCW (and others) toward me. This attitude was really fully justified, and that's even incredible JCW granted me so much attention ! JCW, YOU'RE REALLY A GREAT MAN. That's really a shame we did not localize the source of the problem sooner... At that time, I was talking about my mails formatting, when I said to JCW: >when something can go wrong, it always goes wrong with me. >I'm that kind of person. >Just have a look at my mails... This time, it went wrong from beginning till end. You get the picture of what my life can be... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From lvirden@cas.org Thu Nov 29 06:16:45 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA09924 for ; Thu, 29 Nov 2001 06:16:45 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id fATC8Qv07563 for ; Thu, 29 Nov 2001 07:08:27 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fATC8PH25321; Thu, 29 Nov 2001 07:08:25 -0500 (EST) Date: Thu, 29 Nov 2001 07:08:25 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011129070825.AAB25282@cas.org> Subject: Re: [Metakit] Metakit and Windows In-Reply-To: Your message of Thu, 29 Nov 2001 12:11:38 +0100 To: "Metakit" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Can I try to get a summary of what you see as the problem? Is it that on Windows (which windows was it again?) Metakit is taking as much as 18 times longer to perform the comit than it does on Linux - even when using a comperable program? Could someone work with Pascal and his version of the modified benchmark to make certain that it is being compiled comperably (I'm not a C++ programmer, so I'm not aware of what optimization flags , etc. one needs to include/avoid on various platforms)? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Thu Nov 29 06:21:31 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA10159 for ; Thu, 29 Nov 2001 06:21:30 -0600 Received: from [10.0.1.39] ([195.121.232.210]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNK9Y101.PA9 for ; Thu, 29 Nov 2001 13:13:13 +0100 From: Jean-Claude Wippler To: Metakit Subject: Re: [Metakit] Metakit and Windows Date: Thu, 29 Nov 2001 13:14:42 +0100 Message-Id: <20011129121442.27964@mail.planet.nl> In-Reply-To: <20011129070825.AAB25282@cas.org> References: <20011129070825.AAB25282@cas.org> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >Can I try to get a summary of what you see as the problem? >Is it that on Windows (which windows was it again?) Metakit is taking >as much as 18 times longer to perform the comit than it does on Linux - >even when using a comperable program? There more... it's also O(1), which is most surprising. >Could someone work with Pascal and his version of the modified benchmark >to make certain that it is being compiled comperably (I'm not a C++ >programmer, so I'm not aware of what optimization flags , etc. one needs >to include/avoid on various platforms)? I'm on the case, sir :) -jcw From baspey@yahoo.com Thu Nov 29 06:34:34 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id GAA10828 for ; Thu, 29 Nov 2001 06:34:33 -0600 Received: from toulouse-1-a7-32-121.dial.proxad.net (HELO desktop) (62.147.32.121) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 12:26:46 -0000 Message-ID: <000801c178d0$e0dd1340$7920933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 13:24:46 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >Can I try to get a summary of what you see as the problem? >Is it that on Windows (which windows was it again?) Metakit is taking >as much as 18 times longer to perform the comit than it does on Linux - >even when using a comperable program? I'm using Windows Millenium Edition, and Visual C++ SP5. When I see the results of MkHash.cpp, only Commit() looks worrying. But when I was running my application, I also had blatant performance problem with Find(). I have no explanation. Thanks a lot for your attention. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Thu Nov 29 07:46:28 2001 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id HAA14594 for ; Thu, 29 Nov 2001 07:46:28 -0600 Received: from toulouse-2-a7-36-195.dial.proxad.net (HELO desktop) (62.147.36.195) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 13:38:40 -0000 Message-ID: <001b01c178da$eba8dac0$c324933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 14:36:57 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, There's at least one thing I could specify about the blatant Find() performance problem in my app: before adding any row, I use Find() to check that the entry doesn't already exist. And then I use Add(), which induces another Find(), a useless one I don't know how to get rid of, but that's another problem ;-) In one word: my app uses lots of Find(), which made the problem blatant. P.S: someone has reminded me of the poor performance of Windows OS compared to Linux OS. Hope that's not the only explanation. And there's still the possibility I did wrong somewhere. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From lvirden@cas.org Thu Nov 29 08:03:46 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA15541 for ; Thu, 29 Nov 2001 08:03:45 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id fATDtHv02428 for ; Thu, 29 Nov 2001 08:55:17 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fATDtGQ27499; Thu, 29 Nov 2001 08:55:16 -0500 (EST) Date: Thu, 29 Nov 2001 08:55:16 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011129085516.AAB27405@cas.org> Subject: Re: [Metakit] Metakit and Windows In-Reply-To: Your message of Thu, 29 Nov 2001 14:36:57 +0100 To: "Metakit" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Certainly I would not be surprised to see an overall difference in performance between Linux and Windows/ME. I would however be somewhat surprised if one particular aspect of Metakit were greatly slower than all others. HOWEVER, I would not be willing to rule out the possibility that some limitation in any particular OS could have negative impact on some specific function of the library. I would hope, however, that in that case, some OS specific work around might be possible. Pascal, I know that in general this is a frustrating time. I would make one suggestion - while your application has its own characteristic behaviors which you are attempting to resolve, it would be MOST productive if you could come up with a relatively small, coherent example which demonstrates the problems you are seeing with Find, independant of all the various processing that is going on. I would be the first to recognize that this is not always possible. However, it is the preferable place to begin tracking down a problem such as you are encountering with Find. If this is not possible, then I would recommend begining the effort to create some sort of minimal bundle containing all the pieces necessary to recreate the problem - this would include application, data, etc. Finally, and I know this may sound silly, but have you tried using the latest metakit and your app, but with a database newly created with the new metakit? Perhaps something inside the database got mangled and that is the source of your problem? I've seen cases with other apps where use of an underlying dll or library resulted in a slightly incorrect database, resulting in various misbehaviors that were unexplainable. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jyl@best.com Thu Nov 29 09:06:45 2001 Received: from proxy2.ba.best.com (root@proxy2.ba.best.com [206.184.139.14]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA19587 for ; Thu, 29 Nov 2001 09:06:45 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id GAA11686; Thu, 29 Nov 2001 06:58:26 -0800 (PST) Message-ID: <3C065B53.8030905@best.com> Date: Thu, 29 Nov 2001 07:59:15 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: "Larry W. Virden" CC: Metakit , Jacob At Sun Subject: Re: [Metakit] Metakit and Windows References: <20011129070825.AAB25282@cas.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A large portion of the overhead of Commit is due to file system issues and I/O costs. I'll post some numbers later today (against the unfixed Metakit unfortunately) which demonstrate up to four times slower commit on an NFS mounted file system than on a local UFS mounted file system. Comparing file system performance accross OSes is very dicey, one cannot draw any conclusions from performance of MK Commit on one system to predict performance on another, because the way the OS does I/O has a large effect. JYL Larry W. Virden wrote: >Can I try to get a summary of what you see as the problem? >Is it that on Windows (which windows was it again?) Metakit is taking >as much as 18 times longer to perform the comit than it does on Linux - >even when using a comperable program? > >Could someone work with Pascal and his version of the modified benchmark >to make certain that it is being compiled comperably (I'm not a C++ >programmer, so I'm not aware of what optimization flags , etc. one needs >to include/avoid on various platforms)? > From baspey@yahoo.com Thu Nov 29 09:52:04 2001 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id JAA22873 for ; Thu, 29 Nov 2001 09:52:04 -0600 Received: from toulouse-2-a7-37-160.dial.proxad.net (HELO desktop) (62.147.37.160) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 15:44:16 -0000 Message-ID: <002801c178ec$772ae780$a025933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 16:41:39 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Larry W. Virden wrote: >HOWEVER, I would not be willing to rule out the possibility that some limitation >in any particular OS could have negative impact on some specific function of >the library. I would hope, however, that in that case, some OS specific >work around might be possible. That's incredible! Either you read in my mind, either that's an obvious point ! I had thought of writing the same phrase as your last one but did not. I also used the word "work around" and wanted to check it inside the dictionary! Larry also wrote: >it would be MOST productive if you could come up with a relatively small, >coherent example which demonstrates the problems you are seeing with Find, >independant of all the various processing that is going on. As I said elsewhere, I have already sent an executable + source to JCW. The timing was expressed in CPU cycles instead of uS, but Jean-Claude saw it clearly that it performed poorly. This example was built with the pre-bug sources and was in fact a stripped version of my full app and randomly generated a string, used find(), then Add(), and finally committed every 1000 additions. It measured the average Find(), that is it took into account the value of all 1000 Find(). This sample have most certainly been trashed, but I may mail it again on demand. Regards. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From readonly@getsoft.com Thu Nov 29 11:27:09 2001 Received: from freedom.vosn.net (IDENT:root@[157.238.46.18]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA30391 for ; Thu, 29 Nov 2001 11:27:08 -0600 Received: from c1702.eburwd2.vic.optusnet.com.au ([203.164.254.182] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.33 #1) id 169UqJ-00017F-00; Thu, 29 Nov 2001 10:19:23 -0700 Date: Fri, 30 Nov 2001 04:21:25 +1100 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <152783213601.20011130042125@getsoft.com> To: "Pascal Baspeyras" CC: "Metakit" Subject: Re: [Metakit] Metakit and Windows In-Reply-To: <002801c178ec$772ae780$a025933e@desktop> References: <002801c178ec$772ae780$a025933e@desktop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Might I humbly suggest that Pascal try increasing the amount of RAM in his computer. 64M is not really adequate in todays world. Windows performance can improve considerably with more RAM. When doing your tests you also need to take into account what other applications are running on the PC. I'd suggest closing everything possible, including VC++. You might also want to Defragment your drive and perform any other functions which may help to improve your PCs performance, such as increasing the size of your Windows swap file, or make sure Windows has the optimum settings for it. You should also be using FAT32 and not FAT. From memory you have a Celeron which may also have some overall impact. I have heard about various problems with Win ME and would not recommend anyone using it. W2K is by far a better development platform. Of course if the potential users of your product have machines with ME and 64M of RAM then you must consider that. SiSoft Sandra is a very good program for doing benchmarks on a PC and seeing where performance may be able to be improved. VC5 is obviously quite old now, so updating to VC6 is another consideration. And there are issues such as Release Build vs. Debug Build and compiler optimization settings. Debug Builds will be inherently slower than Release Builds. I hope this has been of some help. Friday, November 30, 2001, 2:41:39 AM, you wrote: PB> Larry W. Virden wrote: >>HOWEVER, I would not be willing to rule out the possibility that some PB> limitation >>in any particular OS could have negative impact on some specific function PB> of >>the library. I would hope, however, that in that case, some OS specific >>work around might be possible. PB> That's incredible! Either you read in my mind, either that's an obvious PB> point ! PB> I had thought of writing the same phrase as your last one but did not. PB> I also used the word "work around" and wanted to check it inside the PB> dictionary! PB> Larry also wrote: >>it would be MOST productive if you could come up with a relatively small, >>coherent example which demonstrates the problems you are seeing with Find, >>independant of all the various processing that is going on. PB> As I said elsewhere, I have already sent an executable + source to JCW. PB> The timing was expressed in CPU cycles instead of uS, but Jean-Claude saw PB> it clearly that it performed poorly. PB> This example was built with the pre-bug sources and was in fact a stripped PB> version of my full app and randomly generated a string, used find(), then PB> Add(), and finally committed every 1000 additions. PB> It measured the average Find(), that is it took into account the value of PB> all PB> 1000 Find(). PB> This sample have most certainly been trashed, but I may mail it again on PB> demand. PB> Regards. -- Best Regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 just released, with a C++ Class View that actually works. mailto:readonly@getsoft.com From baspey@yahoo.com Thu Nov 29 12:09:15 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA01407 for ; Thu, 29 Nov 2001 12:09:14 -0600 Received: from toulouse-2-a7-39-199.dial.proxad.net (HELO desktop) (62.147.39.199) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 18:01:18 -0000 Message-ID: <001701c178ff$9c2a5c60$c727933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 18:58:01 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >Might I humbly suggest that Pascal try increasing the amount of RAM in >his computer. 64M is not really adequate in todays world. Windows >performance can improve considerably with more RAM. I may be wrong, but I really think we can discard the memory problem. I have same problems with my desktop 128 Mb computer, and JCW had problems when running my test program with his 256 Mb notebook. I may once again be wrong, but I think we may discard the compilation problem too. To run MkHash.cpp, I simply changed MkTest.dsp, that is a project configured by JCW himself. As always, all suggestions are welcome and appreciated. Neville also wrote: >VC5 is obviously quite old now, so updating to VC6 is another >consideration. I made a typo: I meant Visual Studio 6 Service Pack 5. Thanks Neville for giving me the opportunity to precise those important points. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From steve@improvision.com Thu Nov 29 12:10:37 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA01559 for ; Thu, 29 Nov 2001 12:10:36 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA14524 for ; Thu, 29 Nov 2001 18:00:47 GMT Received: from [192.168.1.25] ([192.168.1.25]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Thu, 29 Nov 2001 18:02:48 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Thu, 29 Nov 2001 18:02:48 +0000 Subject: Re: [Metakit] Metakit and Windows From: Stephen Baxter To: Neville Franks , Pascal Baspeyras CC: Metakit Message-ID: In-Reply-To: <152783213601.20011130042125@getsoft.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Just to second this - debug builds under Win can be 100s of times slower than release because they use the debugging versions of MFC, memory management etc. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Neville Franks > Organization: Soft As It Gets Pty Ltd > Reply-To: Neville Franks > Date: Fri, 30 Nov 2001 04:21:25 +1100 > To: "Pascal Baspeyras" > Cc: "Metakit" > Subject: Re: [Metakit] Metakit and Windows > > Might I humbly suggest that Pascal try increasing the amount of RAM in > his computer. 64M is not really adequate in todays world. Windows > performance can improve considerably with more RAM. > > When doing your tests you also need to take into account what other > applications are running on the PC. I'd suggest closing everything > possible, including VC++. > > You might also want to Defragment your drive and perform any other > functions which may help to improve your PCs performance, such as > increasing the size of your Windows swap file, or make sure Windows > has the optimum settings for it. You should also be using FAT32 and > not FAT. From memory you have a Celeron which may also have some > overall impact. > > I have heard about various problems with Win ME and would not > recommend anyone using it. W2K is by far a better development > platform. Of course if the potential users of your product have > machines with ME and 64M of RAM then you must consider that. > > SiSoft Sandra is a very good program for doing benchmarks on a PC and > seeing where performance may be able to be improved. > > VC5 is obviously quite old now, so updating to VC6 is another > consideration. And there are issues such as Release Build vs. Debug > Build and compiler optimization settings. Debug Builds will be > inherently slower than Release Builds. > > I hope this has been of some help. > > > Friday, November 30, 2001, 2:41:39 AM, you wrote: > > PB> Larry W. Virden wrote: > >>> HOWEVER, I would not be willing to rule out the possibility that some > PB> limitation >>> in any particular OS could have negative impact on some specific function > PB> of >>> the library. I would hope, however, that in that case, some OS specific >>> work around might be possible. > > PB> That's incredible! Either you read in my mind, either that's an obvious > PB> point ! > PB> I had thought of writing the same phrase as your last one but did not. > PB> I also used the word "work around" and wanted to check it inside the > PB> dictionary! > > PB> Larry also wrote: >>> it would be MOST productive if you could come up with a relatively small, >>> coherent example which demonstrates the problems you are seeing with Find, >>> independant of all the various processing that is going on. > > PB> As I said elsewhere, I have already sent an executable + source to JCW. > PB> The timing was expressed in CPU cycles instead of uS, but Jean-Claude saw > PB> it clearly that it performed poorly. > PB> This example was built with the pre-bug sources and was in fact a stripped > PB> version of my full app and randomly generated a string, used find(), then > PB> Add(), and finally committed every 1000 additions. > PB> It measured the average Find(), that is it took into account the value of > PB> all > PB> 1000 Find(). > > PB> This sample have most certainly been trashed, but I may mail it again on > PB> demand. > > PB> Regards. > > > -- > Best Regards, > Neville Franks, Author of ED for Windows - the programmers editor, an > indispensable tool for great software development. > http://www.getsoft.com Version 4.01 just released, with a C++ Class View that > actually works. > mailto:readonly@getsoft.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From nicolasb@maich.gr Thu Nov 29 12:11:58 2001 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA01687 for ; Thu, 29 Nov 2001 12:11:57 -0600 Received: FROM maich.gr BY mail-server ; Thu Nov 29 20:19:35 2001 +0200 Message-ID: <3C067CCF.B76B9E87@maich.gr> Date: Thu, 29 Nov 2001 20:22:07 +0200 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: Neville Franks CC: Pascal Baspeyras , Metakit Subject: Re: [Metakit] Metakit and Windows References: <002801c178ec$772ae780$a025933e@desktop> <152783213601.20011130042125@getsoft.com> Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: > > Might I humbly suggest that Pascal try increasing the amount of RAM in > his computer. 64M is not really adequate in todays world. Windows > performance can improve considerably with more RAM. Hi, While I tend to think that win and memory is not (at least not) the main problem, a seat of the pants experience that I have had is the following; Notebook, 750Mhz, 256Mb, Win2k on one partition and Redhat6.5 on the other. Following sw on both. Oracle 8.1.7 ACS3.4 AOL3.2 Response times from a seperate browser pc. Linux averaged ~1-2 seconds for most page requests Win2K averaged at least ~40 seconds and many times longer for many others; essentially rendering the system useless. Defragged win, similar swaps etc... You figure... nicolas boretos P.S. This is why I love tclhttpd/metakit for most things I do..... From baspey@yahoo.com Thu Nov 29 13:23:33 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id NAA07749 for ; Thu, 29 Nov 2001 13:23:32 -0600 Received: from toulouse-2-a7-37-237.dial.proxad.net (HELO desktop) (62.147.37.237) by smtp.mail.vip.sc5.yahoo.com with SMTP; 29 Nov 2001 19:15:02 -0000 Message-ID: <001901c17909$e8fdd3a0$ed25933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Thu, 29 Nov 2001 20:12:21 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Metakit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen Baxter wrote: >Just to second this - debug builds under Win can be 100s of times slower >than release because they use the debugging versions of MFC, memory >management etc. I agree that debug version is much slower than release version. But let this be clear, we're talking about RELEASE VERSION here. I know that Stephen understood right, but I wouldn't like people to get confused... _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From horst@freedict.de Fri Nov 30 13:44:58 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id NAA07928 for ; Fri, 30 Nov 2001 13:44:58 -0600 From: horst@freedict.de Received: (qmail 23962 invoked from network); 30 Nov 2001 20:00:33 -0000 Received: from dialin-145-254-112-143.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.112.143) by mail.netbeat.de with SMTP; 30 Nov 2001 20:00:33 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fATLEjl02046; Thu, 29 Nov 2001 22:14:46 +0100 Message-Id: <200111292114.fATLEjl02046@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Thu, 29 Nov 2001 22:14:42 +0100 (CET) Reply-To: horst@freedict.de Subject: Re: [Metakit] Metakit and Windows To: baspey@yahoo.com cc: metakit@equi4.com In-Reply-To: <001901c17909$e8fdd3a0$ed25933e@desktop> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, do you know the cygwin tools? - With them you can compile (GNU) unix programs and run them under windows. This might help to establish where problems are. - I guess people would be able to help you more with options and other things, when you use these tools. What is the other's opinion? Horst On 29 Nov, Pascal Baspeyras wrote: > Stephen Baxter wrote: >>Just to second this - debug builds under Win can be 100s of times slower >>than release because they use the debugging versions of MFC, memory >>management etc. > > I agree that debug version is much slower than release version. > > But let this be clear, we're talking about RELEASE VERSION here. > > I know that Stephen understood right, but I wouldn't like people to get > confused... > > -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From baspey@yahoo.com Fri Nov 30 14:54:30 2001 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id OAA11768 for ; Fri, 30 Nov 2001 14:54:30 -0600 Received: from toulouse-2-a7-37-57.dial.proxad.net (HELO desktop) (62.147.37.57) by smtp.mail.vip.sc5.yahoo.com with SMTP; 30 Nov 2001 20:46:32 -0000 Message-ID: <000b01c179df$db8cf300$3925933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Fri, 30 Nov 2001 21:44:33 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Meatkit and Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Horst wrote: >do you know the cygwin tools? - With them you can compile (GNU) unix >programs and run them under windows. >This might help to establish where problems are. - I guess people would >be able to help you more with options and other things, when you use >these tools. JCW last wrote, 32 hours ago: >>Could someone work with Pascal and his version of the modified benchmark >>to make certain that it is being compiled comperably (I'm not a C++ >>programmer, so I'm not aware of what optimization flags , etc. one needs >>to include/avoid on various platforms)? >I'm on the case, sir :) I suppose that JCW has observed the poor Windows' performance I was talking about. From this point, I guess he'll have to determine if that's a bug or an OS limitation, if it can or not be improved. It's most certainly a tricky problem which takes time to solve, and JCW certainly also has other important things to do. He may manifest himself if I'm wrong. Otherwise, let's give him all the time he needs. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jcw@equi4.com Fri Nov 30 15:21:18 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA13056 for ; Fri, 30 Nov 2001 15:21:17 -0600 Received: from [10.0.1.55] ([195.121.233.47]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNMTLH00.AKE for ; Fri, 30 Nov 2001 22:12:53 +0100 From: Jean-Claude Wippler To: metakit Date: Fri, 30 Nov 2001 22:14:33 +0100 Message-Id: <20011130211433.20709@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Commit timing Unix/Windows Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: My recent changes introduced a small bug. Now fixed, with thanks to Steve Baxter, who narrowed it down with a small test case. The (further reduced) code is now in the regression test suite (m02). The timing program examples/mkhash.cpp has been upgraded with more options, and now also works under Windows. For further notes on changes made, please review the change log. Everything has been checked into CVS. As Jacob Levy pointed out, cross-platform timing is hairy stuff. I've been able to finally build more or less equivalent versions of the mkhash timing program, which now shows more or less identical behavior (one key issue is that inlining must be turned on, in the library and in the app using it: -Dq4_INLINE). The results for Linux: % a.out Dhs ROW ADD FIND COMMIT SIZE 25000 26 uS 179 uS 2402 uS 687868 bytes 50000 37 uS 104 uS 4137 uS 1436068 bytes 75000 27 uS 100 uS 5575 uS 2158018 bytes 100000 22 uS 109 uS 7432 uS 2853718 bytes 125000 40 uS 109 uS 8733 uS 3523168 bytes 150000 31 uS 109 uS 10256 uS 4166368 bytes 175000 23 uS 187 uS 12276 uS 4783318 bytes 200000 30 uS 109 uS 13499 uS 5374018 bytes 225000 29 uS 112 uS 15244 uS 5956570 bytes 250000 41 uS 121 uS 16814 uS 6494770 bytes For Win NT4: C:\> mkhash Dhs ROW ADD FIND COMMIT SIZE 25000 73 uS 270 uS 8864 uS 687868 bytes 50000 50 uS 462 uS 13426 uS 1436068 bytes 75000 79 uS 399 uS 15424 uS 2158018 bytes 100000 35 uS 409 uS 19489 uS 2853718 bytes 125000 84 uS 435 uS 18298 uS 3523168 bytes 150000 65 uS 446 uS 21482 uS 4166368 bytes 175000 51 uS 307 uS 26864 uS 4783318 bytes 200000 66 uS 389 uS 45472 uS 5374018 bytes 225000 65 uS 327 uS 36261 uS 5956570 bytes 250000 93 uS 404 uS 35041 uS 6494770 bytes The Linux build uses gcc 2.95.3, Windows build is MSVC6 (which I have seen to optimize and malloc substantially better than VC5 at times). There's considerably more disk activity under Windows, indicating that either the "fflush" is causing actual sync-to-disk, or that its general sync setting are very different from Linux (I use ReiserFS, a modern journalling FS). One conclusion seems to be: your mileage *WILL* vary! -jcw From steve@improvision.com Sat Dec 1 12:24:36 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA11480; Sat, 1 Dec 2001 12:24:35 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id SAA18333; Sat, 1 Dec 2001 18:14:36 GMT Received: from [10.0.1.4] ([213.105.144.17]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Sat, 01 Dec 2001 18:16:37 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Sat, 01 Dec 2001 18:16:35 +0000 Subject: Re: [Metakit] Commit timing Unix/Windows From: Steve Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011130211433.20709@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Excellent - thanks for the quick work JC! I'll try it first thing Monday. I'll also try and get some timing info for my G4 Mac to add into this mix. Cheers, Steve. > From: Jean-Claude Wippler > Date: Fri, 30 Nov 2001 22:14:33 +0100 > To: metakit > Subject: [Metakit] Commit timing Unix/Windows > > My recent changes introduced a small bug. Now fixed, with thanks to > Steve Baxter, who narrowed it down with a small test case. The (further > reduced) code is now in the regression test suite (m02). > > The timing program examples/mkhash.cpp has been upgraded with more > options, and now also works under Windows. > > For further notes on changes made, please review the change log. > Everything has been checked into CVS. > > As Jacob Levy pointed out, cross-platform timing is hairy stuff. I've > been able to finally build more or less equivalent versions of the mkhash > timing program, which now shows more or less identical behavior (one key > issue is that inlining must be turned on, in the library and in the app > using it: -Dq4_INLINE). > > The results for Linux: > % a.out > Dhs > ROW ADD FIND COMMIT SIZE > 25000 26 uS 179 uS 2402 uS 687868 bytes > 50000 37 uS 104 uS 4137 uS 1436068 bytes > 75000 27 uS 100 uS 5575 uS 2158018 bytes > 100000 22 uS 109 uS 7432 uS 2853718 bytes > 125000 40 uS 109 uS 8733 uS 3523168 bytes > 150000 31 uS 109 uS 10256 uS 4166368 bytes > 175000 23 uS 187 uS 12276 uS 4783318 bytes > 200000 30 uS 109 uS 13499 uS 5374018 bytes > 225000 29 uS 112 uS 15244 uS 5956570 bytes > 250000 41 uS 121 uS 16814 uS 6494770 bytes > > For Win NT4: > C:\> mkhash > Dhs > ROW ADD FIND COMMIT SIZE > 25000 73 uS 270 uS 8864 uS 687868 bytes > 50000 50 uS 462 uS 13426 uS 1436068 bytes > 75000 79 uS 399 uS 15424 uS 2158018 bytes > 100000 35 uS 409 uS 19489 uS 2853718 bytes > 125000 84 uS 435 uS 18298 uS 3523168 bytes > 150000 65 uS 446 uS 21482 uS 4166368 bytes > 175000 51 uS 307 uS 26864 uS 4783318 bytes > 200000 66 uS 389 uS 45472 uS 5374018 bytes > 225000 65 uS 327 uS 36261 uS 5956570 bytes > 250000 93 uS 404 uS 35041 uS 6494770 bytes > > The Linux build uses gcc 2.95.3, Windows build is MSVC6 (which I have > seen to optimize and malloc substantially better than VC5 at times). > > There's considerably more disk activity under Windows, indicating that > either the "fflush" is causing actual sync-to-disk, or that its general > sync setting are very different from Linux (I use ReiserFS, a modern > journalling FS). > > One conclusion seems to be: your mileage *WILL* vary! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From konrad@minus1.de Sun Dec 2 15:17:47 2001 Received: from moutvdom00.kundenserver.de (moutvdom00.kundenserver.de [195.20.224.149]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA12164 for ; Sun, 2 Dec 2001 15:17:46 -0600 Received: from [195.20.224.219] (helo=mrvdom03.schlund.de) by moutvdom00.kundenserver.de with esmtp (Exim 2.12 #2) id 16Adrq-0004C3-00 for metakit@equi4.com; Sun, 2 Dec 2001 22:09:42 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom03.schlund.de with esmtp (Exim 2.12 #2) id 16Adrq-00077h-00 for metakit@equi4.com; Sun, 2 Dec 2001 22:09:42 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16AduN-0000Xw-00 for metakit@equi4.com; Sun, 02 Dec 2001 22:12:19 +0100 Date: Sun, 2 Dec 2001 22:12:19 +0100 From: Konrad Anton To: Metakit list Message-ID: <20011202221219.A2000@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] Behaviour in disk-full condition Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all... While playing with Mk4py, trying to invent nasty tests for my application, I get the following behaviour: The following Python programme is supposed to append rows to a view until disk space runs out. Running it under Linux (Debian potato, kernel 2.2.20, Metakit 2.4.1 for Python 2.1.0), it keeps running until the filesystem is full, but then it segfaults, which is a little impolite for a Python script. #--------------------------------------------------------------- import metakit s=metakit.storage("/mnt/misc/bla.mk", 1) v=s.getas("wuff[a:S,b:S]") for i in range(10000000): v.append(a=str(i)+"bla di bla",b=str(i)+"bla blub") if (i+1)%1000 == 0: print "row",i s.commit() #--------------------------------------------------------------- As you see, it keeps creating unique rows consisting of string attributes and committing every 1000 rows. I tested it on a 300k loop mounted filesystem (both minix and ext2) and a 1M loop mounted ext2 filesystem. Additionally, top claims that "python" is 157 MB big in the "SIZE" and "RSS" categories. In all of these cases, the test programme finally segfaults. Is this a bug, or am I just supposed to not let disk space run short (which is a bit tricky in multitasking environments)? yes-I-know,-disk-space-is-cheap-ly y'rs, Konrad -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: Alice, had no notion poor man. From steve@improvision.com Mon Dec 3 05:10:39 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA27774; Mon, 3 Dec 2001 05:10:38 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id LAA23753; Mon, 3 Dec 2001 11:00:32 GMT Received: from [192.168.1.11] ([192.168.1.11]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 03 Dec 2001 11:02:31 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 03 Dec 2001 11:02:33 +0000 Subject: Re: [Metakit] Commit timing Unix/Windows From: Stephen Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011130211433.20709@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Some more results from a dual G4/500 Mac on both MacOS X and MacOS 9.2.1: MacOS X ======= Dhs ROW ADD FIND COMMIT SIZE 25000 24 uS 807 uS 3431 uS 687868 bytes 50000 35 uS 975 uS 5438 uS 1436068 bytes 75000 31 uS 768 uS 6263 uS 2158018 bytes 100000 21 uS 660 uS 8174 uS 2853718 bytes 125000 36 uS 621 uS 8938 uS 3523168 bytes 150000 30 uS 490 uS 11184 uS 4166368 bytes 175000 23 uS 633 uS 13882 uS 4783318 bytes 200000 30 uS 578 uS 14659 uS 5374018 bytes 225000 30 uS 505 uS 17378 uS 5956570 bytes 250000 43 uS 518 uS 18537 uS 6494770 bytes MacOS 9.2.1 =========== Dhs ROW ADD FIND COMMIT SIZE 25000 23 uS 569 uS 2837 uS 687868 bytes 50000 38 uS 520 uS 4481 uS 1436068 bytes 75000 27 uS 503 uS 5720 uS 2158018 bytes 100000 23 uS 496 uS 7472 uS 2853718 bytes 125000 42 uS 471 uS 9300 uS 3523168 bytes 150000 34 uS 475 uS 11427 uS 4166368 bytes 175000 30 uS 464 uS 13151 uS 4783318 bytes 200000 38 uS 466 uS 15227 uS 5374018 bytes 225000 37 uS 430 uS 17286 uS 5956570 bytes 250000 52 uS 427 uS 19304 uS 6494770 bytes These are with the latest changes from CVS. Quite interesting that the Mac seems to be faster than Windows - is this on an NTFS partition? Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Fri, 30 Nov 2001 22:14:33 +0100 > To: metakit > Subject: [Metakit] Commit timing Unix/Windows > > My recent changes introduced a small bug. Now fixed, with thanks to > Steve Baxter, who narrowed it down with a small test case. The (further > reduced) code is now in the regression test suite (m02). > > The timing program examples/mkhash.cpp has been upgraded with more > options, and now also works under Windows. > > For further notes on changes made, please review the change log. > Everything has been checked into CVS. > > As Jacob Levy pointed out, cross-platform timing is hairy stuff. I've > been able to finally build more or less equivalent versions of the mkhash > timing program, which now shows more or less identical behavior (one key > issue is that inlining must be turned on, in the library and in the app > using it: -Dq4_INLINE). > > The results for Linux: > % a.out > Dhs > ROW ADD FIND COMMIT SIZE > 25000 26 uS 179 uS 2402 uS 687868 bytes > 50000 37 uS 104 uS 4137 uS 1436068 bytes > 75000 27 uS 100 uS 5575 uS 2158018 bytes > 100000 22 uS 109 uS 7432 uS 2853718 bytes > 125000 40 uS 109 uS 8733 uS 3523168 bytes > 150000 31 uS 109 uS 10256 uS 4166368 bytes > 175000 23 uS 187 uS 12276 uS 4783318 bytes > 200000 30 uS 109 uS 13499 uS 5374018 bytes > 225000 29 uS 112 uS 15244 uS 5956570 bytes > 250000 41 uS 121 uS 16814 uS 6494770 bytes > > For Win NT4: > C:\> mkhash > Dhs > ROW ADD FIND COMMIT SIZE > 25000 73 uS 270 uS 8864 uS 687868 bytes > 50000 50 uS 462 uS 13426 uS 1436068 bytes > 75000 79 uS 399 uS 15424 uS 2158018 bytes > 100000 35 uS 409 uS 19489 uS 2853718 bytes > 125000 84 uS 435 uS 18298 uS 3523168 bytes > 150000 65 uS 446 uS 21482 uS 4166368 bytes > 175000 51 uS 307 uS 26864 uS 4783318 bytes > 200000 66 uS 389 uS 45472 uS 5374018 bytes > 225000 65 uS 327 uS 36261 uS 5956570 bytes > 250000 93 uS 404 uS 35041 uS 6494770 bytes > > The Linux build uses gcc 2.95.3, Windows build is MSVC6 (which I have > seen to optimize and malloc substantially better than VC5 at times). > > There's considerably more disk activity under Windows, indicating that > either the "fflush" is causing actual sync-to-disk, or that its general > sync setting are very different from Linux (I use ReiserFS, a modern > journalling FS). > > One conclusion seems to be: your mileage *WILL* vary! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From karl@magicnotes.com Mon Dec 3 05:40:56 2001 Received: from mta05ps.bigpond.com (mta05ps.bigpond.com [144.135.25.137]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA29220 for ; Mon, 3 Dec 2001 05:40:55 -0600 Received: from SuperBoy ([144.135.25.84]) by mta05ps.bigpond.com (Netscape Messaging Server 4.15) with SMTP id GNRN1X00.6U3 for ; Mon, 3 Dec 2001 21:39:33 +1000 Received: from CPE-203-45-10-217.vic.bigpond.net.au ([203.45.10.217]) by psmam06.mailsvc.email.bigpond.com(MailRouter V2.9k 8419/28092434); 03 Dec 2001 21:32:47 Date: Mon, 3 Dec 2001 22:32:33 +1100 From: Karl Edwall X-Mailer: The Bat! (v1.53t) Personal Reply-To: Karl Edwall Organization: Eskil Software X-Priority: 3 (Normal) Message-ID: <1107103003.20011203223233@magicnotes.com> To: metakit Subject: Re[2]: [Metakit] Commit timing Unix/Windows In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Sorry if I'm stating the obvious but has anybody run these using some profiling tools like Numega TrueTime or or similar? If not I'd be happy to do so. I've always found that gives you a better idea of obvious bottlenecks. Regards, Karl Edwall Stephen Baxter wrote; >> Some more results from a dual G4/500 Mac on both MacOS X and MacOS 9.2.1: >> >> MacOS X >> ======= >> >>Dhs >> ROW ADD FIND COMMIT SIZE >> 25000 24 uS 807 uS 3431 uS 687868 bytes From jcw@equi4.com Mon Dec 3 05:55:58 2001 Received: from smtp04.wxs.nl (smtp04.wxs.nl [195.121.6.59]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA30018 for ; Mon, 3 Dec 2001 05:55:58 -0600 Received: from [10.0.1.55] ([195.121.233.114]) by smtp04.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNRNEZ02.40R for ; Mon, 3 Dec 2001 12:47:23 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Commit timing Unix/Windows Date: Mon, 3 Dec 2001 12:48:17 +0100 Message-Id: <20011203114817.13241@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Stephen, >Some more results from a dual G4/500 Mac on both MacOS X and MacOS 9.2.1: [...] >These are with the latest changes from CVS. Quite interesting that the Mac >seems to be faster than Windows - is this on an NTFS partition? Yes. Eh, no. Eh, hm. I've been running off a Samba network drive and off the C: drive (NTFS), not sure which ones my last results were from. I think C:\. One explanation for the performance diff of PowerPC G4 w.r.t. Pentium PIII could be that PowerPC is more "super-scalar/pipelined" (and RISC vs. CISC). It's been plaguing Apple for ages now that a "measly 300 MHz" machine can be as fast as a Wintel machine at much higher clock rates. Imagine buying a machine, same price, same features, but one 300 and the other 650 MHz - which do you pick? I'm not so surprised that tight loops and well-optimized code end up making a RISC machine go in overdrive. That's exactly what their design is about. Hence my last remark: >> One conclusion seems to be: your mileage *WILL* vary! With G4's "AltiVec" vector instructions, one might push MK yet further, but that would require custom coding (and a deep understanding of that side of things). FWIW, one goal for the next G5's appears to be to go way up in MHz, just to get over this perception (see http://www.mosr.com/ for rumors about such things). I'd like to add that this wasn't about comparing the various CPU's or OS'es (not for me, anyway). Though it's always interesting to compare figures. (and pushing me another nudge to using Mac OS X + G4 for development) -jcw From jcw@equi4.com Mon Dec 3 06:06:25 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA30607 for ; Mon, 3 Dec 2001 06:06:25 -0600 Received: from [10.0.1.55] ([195.121.233.253]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNRNWF01.BHS for ; Mon, 3 Dec 2001 12:57:51 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: Re[2]: [Metakit] Commit timing Unix/Windows Date: Mon, 3 Dec 2001 12:59:01 +0100 Message-Id: <20011203115901.31868@mail.planet.nl> In-Reply-To: <1107103003.20011203223233@magicnotes.com> References: <1107103003.20011203223233@magicnotes.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Karl Edwall wrote: >Sorry if I'm stating the obvious but has anybody run these using some >profiling tools like Numega TrueTime or or similar? If not I'd be happy to do >so. I've always found that gives you a better idea of obvious >bottlenecks. Yes, I did (a while back using MSVC's profiling), last week or so using gcc/gprof on Linux. Helped me tweak a little, but it was only a cursory check. There are still several things happening at commit time which could be done differently - but for now this ought to be sufficient. -jcw From jcw@equi4.com Mon Dec 3 06:21:12 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA31411 for ; Mon, 3 Dec 2001 06:21:12 -0600 Received: from [10.0.1.55] ([195.121.235.154]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNROL100.2HB for ; Mon, 3 Dec 2001 13:12:37 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Behaviour in disk-full condition Date: Mon, 3 Dec 2001 13:13:08 +0100 Message-Id: <20011203121308.3990@mail.planet.nl> In-Reply-To: <20011202221219.A2000@minus1.de> References: <20011202221219.A2000@minus1.de> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Konrad Anton wrote: [...] >import metakit > >s=metakit.storage("/mnt/misc/bla.mk", 1) >v=s.getas("wuff[a:S,b:S]") > >for i in range(10000000): > v.append(a=str(i)+"bla di bla",b=str(i)+"bla blub") > if (i+1)%1000 == 0: > print "row",i > s.commit() >#--------------------------------------------------------------- > >As you see, it keeps creating unique rows consisting of string >attributes and committing every 1000 rows. Good you mention said it, email indentation messed it up. >I tested it on a 300k loop mounted filesystem (both minix and ext2) >and a 1M loop mounted ext2 filesystem. Additionally, top claims that >"python" is 157 MB big in the "SIZE" and "RSS" categories. Yes, you are likely to get huge memory use without "blocking views". It's them "column-wise storage" again. Replace this: v=s.getas("wuff[a:S,b:S]") By this: v=s.getas("_B[wuff[a:S,b:S]]").blocked() Should scale far better, in mem-use and in commit speed. >In all of these cases, the test programme finally segfaults. This could be one of the three problems reported by Mitch Chapman (20- 11), which were fixed on 26-11 (in CVS). The commit fails, but passing back the error to Python was hosed. >Is this a bug, or am I just supposed to not let disk space run short >(which is a bit tricky in multitasking environments)? It is (was, I hope) a bug. MK should detect failure to commit properly, regardless what the reason was, and report this. With so many changes and fixes in the recent past, I'm planning to have a new 2.4.2 release - within at most a week or two. Those who have CVS access, and an interest in getting everything top notch, please *do* continue to update and give the code a very heavy workout (last CVS check-in was a few minutes ago, a silly oversight in my last changes). -jcw From steve@improvision.com Mon Dec 3 06:24:54 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA31647; Mon, 3 Dec 2001 06:24:53 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id MAA06913; Mon, 3 Dec 2001 12:14:49 GMT Received: from [192.168.1.11] ([192.168.1.11]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 03 Dec 2001 12:16:45 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 03 Dec 2001 12:16:48 +0000 Subject: Re: Re[2]: [Metakit] Commit timing Unix/Windows From: Stephen Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011203115901.31868@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I will do this on the Mac now everything is working again. Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jean-Claude Wippler > Date: Mon, 3 Dec 2001 12:59:01 +0100 > To: metakit > Subject: Re: Re[2]: [Metakit] Commit timing Unix/Windows > > Karl Edwall wrote: > >> Sorry if I'm stating the obvious but has anybody run these using some >> profiling tools like Numega TrueTime or or similar? If not I'd be happy to do >> so. I've always found that gives you a better idea of obvious >> bottlenecks. > > Yes, I did (a while back using MSVC's profiling), last week or so using > gcc/gprof on Linux. Helped me tweak a little, but it was only a cursory > check. There are still several things happening at commit time which > could be done differently - but for now this ought to be sufficient. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From steve@improvision.com Mon Dec 3 06:33:42 2001 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA32164; Mon, 3 Dec 2001 06:33:41 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id MAA11891; Mon, 3 Dec 2001 12:23:36 GMT Received: from [192.168.1.11] ([192.168.1.11]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Mon, 03 Dec 2001 12:25:35 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 03 Dec 2001 12:25:38 +0000 Subject: Re: [Metakit] Commit timing Unix/Windows From: Stephen Baxter To: Jean-Claude Wippler , metakit Message-ID: In-Reply-To: <20011203114817.13241@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Stephen, > >> Some more results from a dual G4/500 Mac on both MacOS X and MacOS 9.2.1: > [...] >> These are with the latest changes from CVS. Quite interesting that the Mac >> seems to be faster than Windows - is this on an NTFS partition? > > Yes. Eh, no. Eh, hm. I've been running off a Samba network drive and > off the C: drive (NTFS), not sure which ones my last results were from. > I think C:\. > > One explanation for the performance diff of PowerPC G4 w.r.t. Pentium > PIII could be that PowerPC is more "super-scalar/pipelined" (and RISC vs. > CISC). It's been plaguing Apple for ages now that a "measly 300 MHz" > machine can be as fast as a Wintel machine at much higher clock rates. > Imagine buying a machine, same price, same features, but one 300 and the > other 650 MHz - which do you pick? > > I'm not so surprised that tight loops and well-optimized code end up > making a RISC machine go in overdrive. That's exactly what their design > is about. > > Hence my last remark: >>> One conclusion seems to be: your mileage *WILL* vary! Yes indeed - it's useful to know what to expect on different platforms though - it at least tells me everything is being compiled and inlined in the right way. Lots of other things come into play as well. We have found that non-inlined calls can be very slow on the Mac (much worse than Intel platforms), maybe because of all the registers that need to be saved in each stack frame. Inlining a tight-loop method can increase performance by as much as 10x quite instantly. Also the Mac filesystem does not seem to be as fast as Windows in general. > With G4's "AltiVec" vector instructions, one might push MK yet further, > but that would require custom coding (and a deep understanding of that > side of things). Altivec is great for crunching through truly vast amounts of data (we use it for accelerated 3D volume rendering). Writing good Altivec code is *hard* though (every time I try the Altivec code ends up slower than scalar code - fortunately I have one guy in my team who is gifted at this though). > FWIW, one goal for the next G5's appears to be to go way up in MHz, just > to get over this perception (see http://www.mosr.com/ for rumors about > such things). If you believe MacOS Rumors and The Register, it looks like the 1GHz G5 will outperform a 2GHz P4 quite easily. God knows how fast the 2.8GHz G5 will be (the fastest G5s are testing at this speed though it is going to be quite a while before we see them anywhere). I'll let you know as soon as they come out and I persuade my boss to buy me one :). > I'd like to add that this wasn't about comparing the various CPU's or > OS'es (not for me, anyway). Though it's always interesting to compare > figures. > > (and pushing me another nudge to using Mac OS X + G4 for development) Personally I feel that MacOS X is not yet ready for primetime, at least when compared to 9. If you're coming from Linux or Windows your view might be different! It *is* making progress and the rough edges are gradually being rounded off though... Cheers, Steve. From konrad@minus1.de Mon Dec 3 11:02:00 2001 Received: from mout02.kundenserver.de (mout02.kundenserver.de [195.20.224.133]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA16536 for ; Mon, 3 Dec 2001 11:01:59 -0600 Received: from [195.20.224.220] (helo=mrvdom04.kundenserver.de) by mout02.kundenserver.de with esmtp (Exim 2.12 #2) id 16AwLq-0007j1-00 for metakit@equi4.com; Mon, 3 Dec 2001 17:53:54 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom04.kundenserver.de with esmtp (Exim 2.12 #2) id 16AwLp-0000Id-00 for metakit@equi4.com; Mon, 3 Dec 2001 17:53:53 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16AwNa-0000Dj-00 for metakit@equi4.com; Mon, 03 Dec 2001 17:55:42 +0100 Date: Mon, 3 Dec 2001 17:55:42 +0100 From: Konrad Anton To: Metakit list Message-ID: <20011203175542.A790@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] Segfault when committing a read-only storage Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all. The following Python program segfaults (Python 2.1.0, Metakit release 2.4.1, Debian GNU/Linux potato). What it does is * make sure there is a nonempty storage * open it with opening mode 0 (readonly) * write and commit to it Here is the source. #------------------------------------------------------------------ import metakit print "creating storage bla.mk" s = metakit.storage("bla.mk", 1) v = s.getas("wuff[a:S,b:S]") v.append(a="Eins",b="Zwei") s.commit() del s print "open readonly, then write" s = metakit.storage("bla.mk", 0) v = s.view("wuff") print "append..." v.append(a="drei",b="vier") print "commit..." s.commit() print "success" #------------------------------------------------------------------- The segfault occurs after "commit..." is printed. While I understand that I have no right to commit changes to a read-only storage, I wish it just told me so with a catchable exception. I haven't verified if this problem is among those fixed in the latest CVS version (is there a public CVS somewhere? I haven't seen any announced anywhere), but I report it anyway, since the bug report yesterday was so much fun ;) Have a nice whatever-time-of-day, Konrad. -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The end dance? From jcw@equi4.com Mon Dec 3 11:11:52 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA17249 for ; Mon, 3 Dec 2001 11:11:51 -0600 Received: from [10.0.1.55] ([195.121.236.174]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNS21H02.BGC for ; Mon, 3 Dec 2001 18:03:17 +0100 From: Jean-Claude Wippler To: Metakit list Subject: Re: [Metakit] Segfault when committing a read-only storage Date: Mon, 3 Dec 2001 18:05:05 +0100 Message-Id: <20011203170505.15774@mail.planet.nl> In-Reply-To: <20011203175542.A790@minus1.de> References: <20011203175542.A790@minus1.de> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Konrad Anton wrote: >s = metakit.storage("bla.mk", 0) >v = s.view("wuff") >print "append..." >v.append(a="drei",b="vier") >print "commit..." >s.commit() [...] >The segfault occurs after "commit..." is printed. While I understand >that I have no right to commit changes to a read-only storage, I wish >it just told me so with a catchable exception. > >I haven't verified if this problem is among those fixed in the latest >CVS version (is there a public CVS somewhere? I haven't seen any >announced anywhere), but I report it anyway, since the bug report >yesterday was so much fun ;) Well, I don't want to spoil the fun for you - but yes, this is exactly what Mitch Chapman reported on this list... The MK CVS repository is on www.equi4.com, its access is documented on the MK home page: http://www.equi4.com/metakit/ - please let me know if that isn't clear or accurate. There's a snapshot of MK in the previews/ area, which very likely solves the above, but I'm not guaranteeing that it gets updated each time. If you want bleeding edge, then you need to set up CVS. And as always, the change log is at: http://www.equi4.com/metakit/CHANGES/ I try to keep it up to date with the latest in CVS - please make sure you check the top entries before submitting a problem which may well have been resolved. -jcw From jcw@equi4.com Mon Dec 3 11:15:36 2001 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA17494 for ; Mon, 3 Dec 2001 11:15:35 -0600 Received: from [10.0.1.55] ([195.121.236.174]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNS27K02.7FG for ; Mon, 3 Dec 2001 18:06:56 +0100 From: Jean-Claude Wippler To: Metakit list Date: Mon, 3 Dec 2001 18:08:47 +0100 Message-Id: <20011203170847.22260@mail.planet.nl> In-Reply-To: <20011203170505.15774@mail.planet.nl> References: <20011203170505.15774@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Correction Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >And as always, the change log is at: > http://www.equi4.com/metakit/CHANGES/ Whoops, it is and always has been at: http://www.equi4.com/metakit/CHANGES -jcw From konrad@minus1.de Mon Dec 3 11:56:06 2001 Received: from mout02.kundenserver.de (mout02.kundenserver.de [195.20.224.133]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA19572 for ; Mon, 3 Dec 2001 11:56:05 -0600 Received: from [195.20.224.208] (helo=mrvdom01.schlund.de) by mout02.kundenserver.de with esmtp (Exim 2.12 #2) id 16AxCE-0000aL-00 for metakit@equi4.com; Mon, 3 Dec 2001 18:48:02 +0100 Received: from gw.sw-stusie.uni-freiburg.de ([132.230.131.220] helo=fohnhab.minus1.de) by mrvdom01.schlund.de with esmtp (Exim 2.12 #2) id 16AwnF-0001DO-00 for metakit@equi4.com; Mon, 3 Dec 2001 18:22:13 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16Awpl-0001KF-00 for metakit@equi4.com; Mon, 03 Dec 2001 18:24:49 +0100 Date: Mon, 3 Dec 2001 18:24:49 +0100 From: Konrad Anton To: Metakit list Message-ID: <20011203182449.A976@minus1.de> References: <20011203175542.A790@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20011203175542.A790@minus1.de>; from konrad@minus1.de on Mon, Dec 03, 2001 at 05:55:42PM +0100 Subject: [Metakit] problem solved in CVS (was: Segfault when committing a read-only storage) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, Dec 03, 2001 at 05:55:42PM +0100, Konrad Anton wrote: > The following Python program segfaults (Python 2.1.0, Metakit release > 2.4.1, Debian GNU/Linux potato). > (...) > I haven't verified if this problem is among those fixed in the latest > CVS version (is there a public CVS somewhere? I haven't seen any > announced anywhere), but I report it anyway, since the bug report > yesterday was so much fun ;) Haha. Well, the CVS wasn't that *hard* to find once I took off the sunglasses -- sorry. Results with the CVS version of Metakit of just a couple of minutes ago show that both former segfault conditions now raise proper IOError exceptions. So the time-to-bugfix-in-CVS-version is actually negative. That's just great! Sorry for the wasted time, have a nice remaining-of-day, Konrad. -- Konrad Anton gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The end dance? From Christophe.Muller@research.gemplus.com Wed Dec 5 11:43:34 2001 Received: from difool.gemplus.com (difool.gemplus.com [195.25.133.180]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA02816 for ; Wed, 5 Dec 2001 11:43:33 -0600 Received: from research.gemplus.com (unknown [172.28.50.32]) by difool.gemplus.com (Postfix) with ESMTP id 8DF0DC1FF; Wed, 5 Dec 2001 18:33:43 +0100 (MET) Received: from aphrodite (aphrodite [172.28.50.32]) by research.gemplus.com (8.8.8+Sun/8.8.8) with SMTP id SAA22303; Wed, 5 Dec 2001 18:30:17 +0100 (MET) Message-ID: <3C0E59A7.7324@research.gemplus.com> Date: Wed, 05 Dec 2001 18:30:15 +0100 From: Christophe Muller Organization: GEMPLUS X-Mailer: Mozilla 3.0Gold (X11; I; SunOS 5.5.1 sun4u) MIME-Version: 1.0 To: metakit@equi4.com Cc: lvirden@cas.org Content-Type: multipart/mixed; boundary="------------42852FDF6E5D" Subject: [Metakit] WiKit DB corruption and a "wikit-dump" script Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --------------42852FDF6E5D Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hello, [Note: this is a message about WiKit, more than about MetaKit, although I'm not sure the corruption I got comes from the former or the latter..?? If there is a more appropriate place to talk about Wikit, please let me know and sorry about the disturbance.] [Note to JC: I was the 'CM' on you wiki.. :-)] Larry W. Virden wrote (in September): > I have a wikit that I would like to learn to manipulate. > 1. How can I determine the titles of each of the pages in the > wikit? > 2. How can I extract the latest versions of all the pages in the > wikit? > 3. How can I take these extracted pages and insert them into a > new, empty, wikit (say one that has been built with updated code, > etc.) > 4. How can I access the history of changes to any specific page > on the wikit? Like Larry, I'm trying to set-up a wikit internally. I have just grabbed a copy of WiKit at: http://www.equi4.com/pub/tk/examples/wikit.bin I first started to compile everything by myself (with the latest sources taken from /pub/tars/) and ran it under Apache but I got a very badly corrupted database yesterday evening and spent hours to get back data from an hexa dump sdx could gave me... I couldn't figure out how it happened. First the dates got corrupted (i.e., dozens of fancy entries in [Changes], from 1930 to 2067, you get the idea), then after a while all the page titles went wrong and I could see a mixed of text in each page.. I don't really know how to reproduce that. It seems that it happened because I was using at the same time the Tk GUI (./wikit) and the CGI way... I seem to recall to have read somewhere that they cannot mix yet.. If there is a more stable version of Wikit/Tclkit than the one I'm using, I'd be glad to know.. (I'd rather stick to the stable version when editing content). Anyway, today (is a new day) and I got a fresh compiled copy of tclkit from: http://www.equi4.com/pub/tk/tclkit-8.4-26-linux/ have put everything back in place, have used the local mode *only at the beginning* and then I have used only the CGI mode. Also, like Larry, I was so concerned about saving the content I typed today that I have looked inside the tcl scripts (by unpacking them with sdx) and started developing a small script of my own that basically (I think) addresses points 1 and 2 of his request. However, I still have no clue on how to use that to re-create a wikit from scratch in reverse. Any help would be appreciated. Also any doc on wikit design, how to modify and repack custom versions, how to easily extract and reload the database data, etc. would be welcomed. It would be nice if we could see a file in the .vfs tree that contains the data (e.g., pages.db), and if we could pretty-print and parse it back from ASCII form.. I have the idea of saving RCS/CVS modifs at each "Save".. This would definitely reassure me :-). I have attached my script draft. At least with it, my content is not lost in case of corruption. Any help on how to "build" a wikit database programmatically would be appreciated. Also it seems to me that when two persons are editing the same page, the last "Save" will erase the first one: Am I right? Isn't it possible to use MetaKit commit features? Thanks for your help. If I improve the script, I'll post the results. Cheers, Christophe. = Most of the VAX instructions are in microcode, = = but HALT and NO-OP are in hardware for efficiency. = -- ------------------------------------------------------------- Christophe.Muller@research.gemplus.com - Gemplus Research Lab Phone: +33 4-42-36-57-83 | Disclaimer: I don't speak for Gemplus Gemplus doesn't speak for me... it is better that way! ------------------------------------------------------------- --------------42852FDF6E5D Content-Type: text/plain; charset=us-ascii; name="wikit-dump" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="wikit-dump" #!/usr/local/bin/tclkit # Dumps a wikit that has been filled with pages in "source" form # (i.e., with the wikit page markup) # # 2001-12-05, cm set header {FILE AUTOMATICALLY SAVED BY wikit-dump. PLEASE DO NOT EDIT} if {[llength $argv] != 2} { puts stderr "Usage: $argv0 infile outfile" exit 1 } mk::file open db [lindex $argv 0] -readonly set all [mk::select db.pages] puts "The following pages will be saved: $all" puts -nonewline "Saving..." set out [open [lindex $argv 1] w] fconfigure $out -translation binary -encoding binary puts $out $header foreach i $all { puts -nonewline "." puts $out "__________________________________________________" puts $out "Page: $i in wikit source form." puts $out "Name: [mk::get db.pages!$i name]" puts $out "Date: [mk::get db.pages!$i date]" puts $out "" puts $out [mk::get db.pages!$i page] } close $out puts "Done." exit --------------42852FDF6E5D-- From lvirden@cas.org Wed Dec 5 12:24:21 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA04386 for ; Wed, 5 Dec 2001 12:24:21 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id fB5IFa214809 for ; Wed, 5 Dec 2001 13:15:36 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fB5IFY625463; Wed, 5 Dec 2001 13:15:34 -0500 (EST) Date: Wed, 5 Dec 2001 13:15:34 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011205131534.AAB25421@cas.org> Subject: Re: [Metakit] WiKit DB corruption and a "wikit-dump" script In-Reply-To: Your message of Wed, 05 Dec 2001 18:30:15 +0100 To: metakit@equi4.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: In the past, this has been 'the place' to discuss wikits. I don't know what JCW's thoughts are on this topic. I do know, however, that JCW's been hard at work at the wikit at , fixing problems that sound amazinging like what you describe, during the past few weeks. Hopefully he will have more to add when he gets to his mail. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Thu Dec 6 06:57:43 2001 Received: from smtp02.wxs.nl (smtp02.wxs.nl [195.121.6.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id GAA04188 for ; Thu, 6 Dec 2001 06:57:42 -0600 Received: from [10.0.1.55] ([195.121.232.191]) by smtp02.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GNXA9T01.39L for ; Thu, 6 Dec 2001 13:49:05 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] WiKit DB corruption and a "wikit-dump" script Date: Thu, 6 Dec 2001 13:50:18 +0100 Message-Id: <20011206125018.32530@mail.planet.nl> In-Reply-To: <3C0E59A7.7324@research.gemplus.com> References: <3C0E59A7.7324@research.gemplus.com> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Christophe Muller wrote: >[Note: this is a message about WiKit, more than about MetaKit, >although I'm not sure the corruption I got comes from the former >or the latter..?? If there is a more appropriate place to talk >about Wikit, please let me know and sorry about the disturbance.] No, there is no better place right now - though this topic is likely to interest only a few people on this list. I'll answer here, perhaps we could continue the discussion on a wiki page on the Tcl'ers Wiki? >Like Larry, I'm trying to set-up a wikit internally. > >I have just grabbed a copy of WiKit at: >http://www.equi4.com/pub/tk/examples/wikit.bin > >I first started to compile everything by myself (with the latest >sources taken from /pub/tars/) and ran it under Apache but I got >a very badly corrupted database yesterday evening and spent hours >to get back data from an hexa dump sdx could gave me... I couldn't >figure out how it happened. First the dates got corrupted (i.e., >dozens of fancy entries in [Changes], from 1930 to 2067, you get >the idea), then after a while all the page titles went wrong and >I could see a mixed of text in each page.. > >I don't really know how to reproduce that. It seems that it >happened because I was using at the same time the Tk GUI (./wikit) >and the CGI way... I seem to recall to have read somewhere that >they cannot mix yet.. Your problem appears to be caused by two processes accessing the MK datafile at the same time (and modifying it). Column-wise storage means that damage, however small, may end up looking very ugly and bad all over the datafile. So although the effect is indeed "corruption", this really is caused by multi-user access to a design which does not deal with it. The wikit does do its own manual locking, so I'm a bit puzzled how this could have happened. When you do "tclkit wikit.tkd dump", do you see a "lock.tcl" file listed? if so, then locking should really have been active at every launch (you weren't doing this as root, I assume?). >Anyway, today (is a new day) and I got a fresh compiled copy of >tclkit from: http://www.equi4.com/pub/tk/tclkit-8.4-26-linux/ >have put everything back in place, have used the local mode >*only at the beginning* and then I have used only the CGI mode. That sounds like a good idea. >Also, like Larry, I was so concerned about saving the content >I typed today that I have looked inside the tcl scripts (by >unpacking them with sdx) and started developing a small script [...] The wikit scripts are in an intermediate state. Though they use VFS and you can indeed use SDX to get/set them, keep in mind that the wiki pages are still stored in "raw" MetaKit format, i.e. as a (non-VFS) view. >I have attached my script draft. Did you look at the (quite recent) "wikitool" mentioned over here? http://www.equi4.com/wikit/ >Also it seems to me that when two persons are editing the >same page, the last "Save" will erase the first one: Am I >right? Isn't it possible to use MetaKit commit features? Yes. That is not so easy to change: the main use is over the web, and HTTP is stateless... it is not obvious whether someone is "currently editing" a page, especially not because all requests are handled by separate CGI launches. Your script looks ok to me, btw. You're doing the essentials for looping and fetching data. It ought to be possible to do the reverse as well, but there are issues such as cerating new pages (and page id's) which are not trivial. Then again, if it's merely for restoring existing content, then it should be feasible. FWIW, I'm working on and off on a very much simpler wiki, and taking advantage of the latest VFS to get rid of all the issues you are running into. But it's hard to say when it will be ready / or at least useful enough for others to work with. -jcw From jcw@equi4.com Wed Dec 12 09:32:03 2001 Received: from smtp07.wxs.nl (smtp07.wxs.nl [195.121.6.39]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA07231; Wed, 12 Dec 2001 09:32:02 -0600 Received: from [10.0.1.55] ([195.121.236.29]) by smtp07.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GO8LE401.QUJ; Wed, 12 Dec 2001 16:22:52 +0100 From: Jean-Claude Wippler To: announce , metakit Date: Wed, 12 Dec 2001 16:19:47 +0100 Message-Id: <20011212151947.15191@mail.planet.nl> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Exactly two years (to this very day) after MetaKit 2.0 was released as open source software, I am proud to announce a new MetaKit 2.4.2 release. This release passes all my tests, and contains several changes since 2.4.1: - the hashing view has been improved and now works well with blocking - commit speed improved with many subviews containing strings - the tcl/kit/ directory has been removed (TclKit is being split off) - optional builds of Mk4py and Mk4tcl have been streamlined - more sample/timing code added to "examples/" in the src distribution - various bug fixes, warning msgs cleaned up, and minor tweaks The detailed change log is in the source distribution and on the web: http://www.equi4.com/metakit/CHANGES Downloads for the source, as well as builds for Windows, Linux, MacOS 9+X, FreeBSD, and more (Solaris coming soon) are all at: http://www.equi4.com/pub/mk/ MetaKit's home page is, as before, at http://www.equi4.com/metakit/ I've started to take some notes w.r.t. documentation on the MetaKit wiki: http://www.equi4.com/metakit/wiki.cgi Feel free to add/amend, or even add your own pages and examples. If you create a build for another platform and it sails cleanly through "make test", then I'd be most happy (eager!) to include it in the download area. If the build is not successful, please post info to metakit@equi4.com so readers can try to help. Thanks to everyone for reporting bugs, providing help on the mailing list, and more generally for helping clarify trouble spots and broaden the use of MetaKit. May it serve you well, also in 2002! Jean-Claude From mmesser@masari.com Wed Dec 12 13:20:24 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA19330 for ; Wed, 12 Dec 2001 13:20:14 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000017; 12 Dec 01 09:57:52 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 12 Dec 01 09:57:15 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Wed, 12 Dec 2001 09:56:56 -0800 MIME-Version: 1.0 Message-ID: <3C1729E8.25321.75FCE80@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] Mk4py: problem with view.hash() ?? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: # This code crashes the interpreter. I tried 2.4.1 ad 2.4.2. # Am I doing something wrong here? import metakit db = metakit.storage("datafile.mk",1) vw = db.getas("people[first:S,last:S,shoesize:I]") vw.append(first='John',last='Lennon',shoesize=44) vw.append(first='Flash',last='Gordon',shoesize=42) db.commit() for r in vw: print r.first, r.last, r.shoesize tbl = {} newvw = vw.hash(tbl, numkeys=1) #for item in tbl.items(): print item #for r in newvw: print r.first, r.last, r.shoesize From jcw@equi4.com Wed Dec 12 13:59:16 2001 Received: from smtp01.wxs.nl (smtp01.wxs.nl [195.121.6.61]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id NAA21382 for ; Wed, 12 Dec 2001 13:59:15 -0600 Received: from [10.0.1.55] ([195.121.234.63]) by smtp01.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GO8XRH00.YJP for ; Wed, 12 Dec 2001 20:50:05 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Mk4py: problem with view.hash() ?? Date: Wed, 12 Dec 2001 20:40:58 +0100 Message-Id: <20011212194058.5245@mail.planet.nl> In-Reply-To: <3C1729E8.25321.75FCE80@localhost> References: <3C1729E8.25321.75FCE80@localhost> X-Mailer: CTM PowerMail 3.0.9v2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Messer wrote: >This code crashes the interpreter. Am I doing something wrong? [...] >tbl = {} >newvw = vw.hash(tbl, numkeys=1) Well... tbl needs to me a MK view, not a Python dict. For example: tbl = metakit.view() Or look at the *.py files in the examples/ dir (in the MK source dist). Hm, MK shouldn't crash... that is definitely a weakness in MK. -jcw From mmesser@masari.com Wed Dec 12 23:16:40 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA10145 for ; Wed, 12 Dec 2001 23:16:40 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000020; 12 Dec 01 19:54:04 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 12 Dec 01 19:53:49 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Wed, 12 Dec 2001 19:53:34 -0800 MIME-Version: 1.0 Message-ID: <3C17B5BE.11018.9820919@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] mk4py: outer joins Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This code gives the following error: Traceback (most recent call last): File "test.py", line 20, in ? for r in vw.join(vw2, vw.shoesize, outer=1): print r.first, r.last, r.shoesize, r.desc SystemError: error return without exception set It works fine when I remove outer=1 from the join. It still gives and error when outer=0 is specified. __________________________________________________________________ import metakit import sys db = metakit.storage() #Create a view (this is the MetaKit term for "table"): vw = db.getas("people[first:S,last:S,shoesize:I]") vw2 = db.getas("sizes[shoesize:I,desc:S]") #Add two rows (this is the MetaKit term for "record"): vw.append(first='John',last='Lennon',shoesize=44) vw.append(first='Flash',last='Gordon',shoesize=45) vw.append(first='John',last='Gordon',shoesize=46) vw.append(first='Flash',last='Lennon',shoesize=47) vw2.append(shoesize=44, desc='Small') vw2.append(shoesize=45, desc='Medium') for r in vw.join(vw2, vw.shoesize, outer=1): print r.first, r.last, r.shoesize, r.desc print From mmesser@masari.com Wed Dec 12 23:21:28 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA10272 for ; Wed, 12 Dec 2001 23:21:28 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000022; 12 Dec 01 19:58:52 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 12 Dec 01 19:58:46 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Wed, 12 Dec 2001 19:58:25 -0800 MIME-Version: 1.0 Message-ID: <3C17B6E1.10816.9867AAB@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] mk4py: infinite loop when passing dictionary criteria to view.find() Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The invocation of the "bat" function results in an infinite loop. Isn't it legal to pass criteria in dictionary form to view.find() ? _______________________________________________________________-- import metakit db = metakit.storage() #Create a view (this is the MetaKit term for "table"): vw = db.getas("people[first:S,last:S,shoesize:I]") vw.append(first='John',last='Lennon',shoesize=44) vw.append(first='Flash',last='Gordon',shoesize=42) def foo(v, criteria): for r in v.select(criteria): print r.first, r.last, r.shoesize def bar(v): i = v.find(last='Lennon', start=0) while(i >= 0): print v[i].first, v[i].last, v[i].shoesize i = v.find(last='Lennon', start=i+1) def bat(v, criteria): i = v.find(criteria, start=0) while(i >= 0): print v[i].first, v[i].last, v[i].shoesize i = v.find(criteria, start=i+1) criteria = {'last':'Lennon'} foo(vw, criteria) print bar(vw) print bat(vw, criteria) print From j.p.fletcher@aston.ac.uk Thu Dec 13 07:25:19 2001 Received: from email.aston.ac.uk (email.aston.ac.uk [134.151.79.13]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA25357 for ; Thu, 13 Dec 2001 07:25:18 -0600 Received: from [134.151.70.134] (helo=ceac034) by email.aston.ac.uk with smtp (Exim 3.30 #1) id 16EVjQ-0005Lk-00 for metakit@equi4.com; Thu, 13 Dec 2001 13:17:00 +0000 From: "John Fletcher" To: metakit Date: Thu, 13 Dec 2001 13:14:31 -0000 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: [Metakit] MetaKit 2.4.2 Priority: normal In-reply-to: <20011212151947.15191@mail.planet.nl> X-mailer: Pegasus Mail for Win32 (v3.01d) Message-Id: Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Jean-Claude Wippler To: announce , metakit Subject: [Metakit] MetaKit 2.4.2 Date sent: Wed, 12 Dec 2001 16:19:47 +0100 > Exactly two years (to this very day) after MetaKit 2.0 was released as > open source software, I am proud to announce a new MetaKit 2.4.2 release. > Thanks for this and for all the work. Are there any implications for the public interface? In particular, can I substitute this release in builds for E4Graph? Thanks John Fletcher ------------------------------------------------------------------- Dr John P. Fletcher Tel: (44) 121 359 3611 ext 4625 Chemical Engineering and Applied Chemistry (CEAC), School of Engineering and Applied Science (SEAS), Aston University, Fax: (44) 121 359 4094 Aston Triangle, Email: J.P.Fletcher@aston.ac.uk BIRMINGHAM B4 7ET U.K. CEAC Web site http://www.ceac.aston.ac.uk/ From gmcm@hypernet.com Thu Dec 13 07:39:34 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA25998 for ; Thu, 13 Dec 2001 07:39:33 -0600 Received: from PARANOIA (204.176.40.64) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 13 Dec 2001 08:24:31 -0500 From: "Gordon McMillan" To: "Mark Messer" , metakit@equi4.com Date: Thu, 13 Dec 2001 08:30:36 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] mk4py: outer joins Reply-to: gmcm@hypernet.com Message-ID: <3C18672C.8079.5E37B26@localhost> Priority: normal In-reply-to: <3C17B5BE.11018.9820919@localhost> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 12 Dec 2001 at 19:53, Mark Messer wrote: > > This code gives the following error: > > Traceback (most recent call last): > File "test.py", line 20, in ? > for r in vw.join(vw2, vw.shoesize, outer=1): print r.first, > r.last, r.shoesize, r.desc > SystemError: error return without exception set join is not a keyword-accepting method. vw.jon(vw2, vw.shoesize, 1) will give you an outer join, while vw.join(vw2, vw.shoesize, 0) is equivalent to vw.join(vw2, vw.shoesize) -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Thu Dec 13 07:39:35 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA26001 for ; Thu, 13 Dec 2001 07:39:34 -0600 Received: from PARANOIA (204.176.40.64) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 13 Dec 2001 08:24:33 -0500 From: "Gordon McMillan" To: "Mark Messer" , metakit@equi4.com Date: Thu, 13 Dec 2001 08:30:36 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] mk4py: infinite loop when passing dictionary criteria to view.find() Reply-to: gmcm@hypernet.com Message-ID: <3C18672C.25988.5E37B80@localhost> Priority: normal In-reply-to: <3C17B6E1.10816.9867AAB@localhost> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 12 Dec 2001 at 19:58, Mark Messer wrote: > The invocation of the "bat" function results in an infinite loop. > > Isn't it legal to pass criteria in dictionary form to view.find() ? Yes, but it's not legal to use a criteria dictionary AND a keyword arg at the same time. Instead of v.find(criteria, start=i+1) you could: criteria['start'] = i+1 v.find(criteria) > _______________________________________________________________-- > > > import metakit > > db = metakit.storage() > > #Create a view (this is the MetaKit term for "table"): > vw = db.getas("people[first:S,last:S,shoesize:I]") > vw.append(first='John',last='Lennon',shoesize=44) > vw.append(first='Flash',last='Gordon',shoesize=42) > > def foo(v, criteria): > for r in v.select(criteria): print r.first, r.last, r.shoesize > > def bar(v): > i = v.find(last='Lennon', start=0) > while(i >= 0): > print v[i].first, v[i].last, v[i].shoesize > i = v.find(last='Lennon', start=i+1) > > def bat(v, criteria): > i = v.find(criteria, start=0) > while(i >= 0): > print v[i].first, v[i].last, v[i].shoesize > i = v.find(criteria, start=i+1) > > criteria = {'last':'Lennon'} > foo(vw, criteria) > print > bar(vw) > print > bat(vw, criteria) > print > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Thu Dec 13 07:52:58 2001 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA26644 for ; Thu, 13 Dec 2001 07:52:58 -0600 Received: from PARANOIA (204.176.40.64) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 13 Dec 2001 08:38:24 -0500 From: "Gordon McMillan" To: "Mark Messer" , metakit@equi4.com Date: Thu, 13 Dec 2001 08:44:29 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] mk4py: infinite loop when passing dictionary criteria to view.find() Reply-to: gmcm@hypernet.com Message-ID: <3C186A6D.4510.5F03116@localhost> Priority: normal In-reply-to: <3C18672C.25988.5E37B80@localhost> References: <3C17B6E1.10816.9867AAB@localhost> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 13 Dec 2001 at 8:30, Gordon McMillan wrote: > Instead of > v.find(criteria, start=i+1) > you could: > criteria['start'] = i+1 > v.find(criteria) Oops. If you want to use start, you can't use a dictionary for criteria. That can probably be improved. BTW, if you expect a number of hits on criteria, you're often better off using select. -- Gordon http://www.mcmillan-inc.com/ From mmesser@masari.com Thu Dec 13 15:28:47 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA19526 for ; Thu, 13 Dec 2001 15:28:46 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000033; 13 Dec 01 12:06:05 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 13 Dec 01 12:05:51 -0800 Received: from abusimbel (63.94.91.2) by masari.com (Mercury/32 v3.21c) with ESMTP ID MG000032; 13 Dec 01 12:05:46 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Thu, 13 Dec 2001 13:19:45 -0800 MIME-Version: 1.0 Message-ID: <3C18AAF1.19696.F9246C1@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] mk2py: more weird stuff with vs.hash() and vs.order() Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: import metakit db = metakit.storage() #Create a view (this is the MetaKit term for "table"): vw = db.getas("people[last:S,first:S,shoesize:I]") vw.append(first='John',last='Lennon',shoesize=44) vw.append(first='Flash',last='Gordon',shoesize=42) mapvw = db.getas("map[_H:I,_R:I]") # THIS CAUSES A CRASH!! # hashvw1a = vw.hash(mapvw, numkeys=1) hashvw1b = vw.hash(mapvw, 1) ordvw1a = vw.ordered(numkeys=1) ordvw1b = vw.ordered(1) ordvw2a = vw.ordered(numkeys=2) ordvw2b = vw.ordered(2) def foo(v, criteria): for r in v.select(criteria): print 'foo:', r.first, r.last, r.shoesize print 'foo: done.\n' def bar(v): i = v.find(last='Gordon', start=0) while(i >= 0): print 'bar:', v[i].first, v[i].last, v[i].shoesize i = v.find(last='Gordon', start=i+1) print 'bar: done.\n' criteria = {'last':'Gordon'} foo(vw, criteria) foo(hashvw1b, criteria) foo(ordvw1a, criteria) foo(ordvw1b, criteria) foo(ordvw2a, criteria) foo(ordvw2b, criteria) bar(vw) bar(hashvw1b) bar(ordvw1a) # no results - isn't "ordered" an identity view???? bar(ordvw1b) # no results bar(ordvw2a) # no results bar(ordvw2b) From mmesser@masari.com Thu Dec 13 15:38:07 2001 Received: from masari.com (adsl-63-192-208-25.dsl.snfc21.pacbell.net [63.192.208.25]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA20045 for ; Thu, 13 Dec 2001 15:38:07 -0600 Received: from Spooler by masari.com (Mercury/32 v3.21c) ID MO000035; 13 Dec 01 12:15:25 -0800 Received: from spooler by masari.com (Mercury/32 v3.21c); 13 Dec 01 12:15:23 -0800 Received: from abusimbel (63.94.91.2) by masari.com (Mercury/32 v3.21c) with ESMTP ID MG000034; 13 Dec 01 12:15:14 -0800 From: "Mark Messer" To: metakit@equi4.com Date: Thu, 13 Dec 2001 13:29:13 -0800 MIME-Version: 1.0 Message-ID: <3C18AD29.19923.F9AEFD0@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] mk2py: more weird stuff with vs.hash() and vs.order() (FOLLOW UP) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: By the way, if you change the "getas" call to this: vw = db.getas("people[test:I,last:S,first:S,shoesize:I]") it works for all cases. From horst@freedict.de Sun Dec 16 03:15:29 2001 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id DAA21603 for ; Sun, 16 Dec 2001 03:15:13 -0600 From: horst@freedict.de Received: (qmail 6932 invoked from network); 16 Dec 2001 09:31:56 -0000 Received: from dialin-145-254-117-023.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.117.23) by mail.netbeat.de with SMTP; 16 Dec 2001 09:31:56 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id fBG8nrM23586; Sun, 16 Dec 2001 09:49:54 +0100 Message-Id: <200112160849.fBG8nrM23586@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Sun, 16 Dec 2001 09:49:50 +0100 (CET) Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] Metakit for Python and File locking Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I did not find information on metakit for Python and file locking on the net. Is there a prefered way of locking metakit databases, and / or are there things to keep in mind, when implementing file locking. Note: I am implementing file locking, for a email based system, primarily running on linux, but platform independence certainly is desirable. Thanks for hints, Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From jcw@equi4.com Sun Dec 16 04:06:38 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id EAA22079 for ; Sun, 16 Dec 2001 04:06:38 -0600 Received: from [10.0.1.55] ([195.121.234.16]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GOFKZD02.HOK for ; Sun, 16 Dec 2001 10:57:13 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Metakit for Python and File locking Date: Sun, 16 Dec 2001 10:53:58 +0100 Message-Id: <20011216095358.2032@mail.planet.nl> In-Reply-To: <200112160849.fBG8nrM23586@eaglesnest.mceggman> References: <200112160849.fBG8nrM23586@eaglesnest.mceggman> X-Mailer: CTM PowerMail 3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: >I did not find information on metakit for Python and file locking on the >net. Is there a prefered way of locking metakit databases, and / or are >there things to keep in mind, when implementing file locking. File locking was at one point part of Mk4py and Mk4tcl, but I took it out again, after struggling hard to make it work reliably on all platforms. I was looking for a solution which did not require an extra lock file, and which did not need cleaning up if an application was aborted prematurely. It's tricky stuff, across Unix variants and Windows to get this right. And a sort-of-usually-working solution which gives a false sense of working properly but doesn't is worse than none... hence the decision to take it out again. Nowadays, I tend to use lock files. Write the pid in th elock file, and when another process wants to acquire the lock: check that the locking process is still alive - if not, break the lock by removing the lock file. On Unix, there is the /proc// to check things, else do a "ps". Even then however, truly reliable locking on NFS file systems, in all their (sometimes broken) variations, seems to be next to impossible. Even with lockfiles, apparently. There's a discussion (Tcl-centric) about this on the Tcl'ers wiki: Python might be in a somewhat better position, since it implements lockf (in the Posix layer, AFAIK). Another source of information (perhaps dated by now) for me, was "procmail" which seems to goo out of its way to try to solve this. No URL, you'll have to chase it on Google to find sources and look there. I'm still very interested in solving this. Even a partial, but very clearly specified, solution would be useful to have in MK. -jcw From jcw@equi4.com Mon Dec 17 14:10:52 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA28170 for ; Mon, 17 Dec 2001 14:10:52 -0600 Received: from [10.0.1.55] ([195.121.233.224]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GOI7N401.956 for ; Mon, 17 Dec 2001 21:01:52 +0100 From: Jean-Claude Wippler To: Subject: Re: [Metakit] mk2py: more weird stuff with vs.hash() and vs.order() Date: Mon, 17 Dec 2001 20:55:19 +0100 Message-Id: <20011217195519.16006@mail.planet.nl> In-Reply-To: <3C18AAF1.19696.F9246C1@localhost> References: <3C18AAF1.19696.F9246C1@localhost> X-Mailer: CTM PowerMail 3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Messer wrote: (Sorry for the late response, I was tied up with some other work) >import metakit > >db = metakit.storage() > >#Create a view (this is the MetaKit term for "table"): >vw = db.getas("people[last:S,first:S,shoesize:I]") >vw.append(first='John',last='Lennon',shoesize=44) >vw.append(first='Flash',last='Gordon',shoesize=42) > >mapvw = db.getas("map[_H:I,_R:I]") > ># THIS CAUSES A CRASH!! ># hashvw1a = vw.hash(mapvw, numkeys=1) Resolved in Gordon's recent email: drop the "numkeys=". >hashvw1b = vw.hash(mapvw, 1) >ordvw1a = vw.ordered(numkeys=1) [drop the above] >ordvw1b = vw.ordered(1) >ordvw2a = vw.ordered(numkeys=2) [drop the above] >ordvw2b = vw.ordered(2) > >def foo(v, criteria): > for r in v.select(criteria): print 'foo:', r.first, r.last, r.shoesize > print 'foo: done.\n' > >def bar(v): > i = v.find(last='Gordon', start=0) > while(i >= 0): > print 'bar:', v[i].first, v[i].last, v[i].shoesize > i = v.find(last='Gordon', start=i+1) > print 'bar: done.\n' > >criteria = {'last':'Gordon'} > >foo(vw, criteria) >foo(hashvw1b, criteria) >foo(ordvw1a, criteria) >foo(ordvw1b, criteria) >foo(ordvw2a, criteria) >foo(ordvw2b, criteria) > >bar(vw) >bar(hashvw1b) >bar(ordvw1a) # no results - isn't "ordered" an identity view???? [ignore, see above] >bar(ordvw1b) # no results >bar(ordvw2a) # no results [ignore, see above] >bar(ordvw2b) The *one* key issue in this email is: bar(ordvw1b) # no results This can be explained by the fact that "ordered" is a mapping view which maintains the inderlying view in sort order. What needs to be documented better, though, is that it also requires the view as given to it at definition to already be in the proper sort order. In other words: bar(ordvw1b) # no results <-- you got bitten While: bar(ordvw2b) <-- you got lucky The last case just "happened" to find the key you were looking for. Thanks for reporting this. There is a clear distinction of which I was not aware between the hashed and the ordered view: - The hashed view uses a secondary view to maintain a hashing order, and builds it on-the-fly when defined on a view which already contains data. Whereas: - The ordered view does not use a secondary view. It expects the view to be sorted properly, and then maintains the sort order when insertions and deletions are applied. Things get more complicated, though. There is also an "indexed" mapping view, which is similar to "hash" in that it maintains a secondary sort index (a set of row numbers, basically). It too will create the index if it is found to be empty at definition time. And to make a long story even longer: there are restrictions in what these views do when keys are modified, i.e. properies which participate in the hash/sort order. The only proper solution for now is to *never* alter keys. If you have to change a key property, use a row deletion + insertion instead. Conclusion: there is still a lot of work left to do in MK to get all these special cases resolved properly, and fully automatically... -jcw From hans.yandle@cingular.com Thu Dec 20 10:37:18 2001 Received: from bscc-gateway.bscc.bls.com (smtp.cingular.com [170.35.240.204]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA24430 for ; Thu, 20 Dec 2001 10:37:18 -0600 From: hans.yandle@cingular.com Received: from laho-gateway.bscc.bls.com (mailhub-laho [30.146.28.2]) by bscc-gateway.bscc.bls.com (8.9.3/8.9.3) with ESMTP id LAA06831 for ; Thu, 20 Dec 2001 11:27:39 -0500 (EST) Received: from zeus.houstoncellular.com (apollo.houstoncell.com [172.1.9.3]) by laho-gateway.bscc.bls.com (8.9.3/8.9.3) with ESMTP id LAA12712 for ; Thu, 20 Dec 2001 11:27:38 -0500 (EST) Received: by scorpexc4.houstoncell.com with Internet Mail Service (5.5.2653.19) id ; Thu, 20 Dec 2001 10:27:37 -0600 Message-ID: <5B572F4E690AD411ADC200508BCF39920324D26B@scorpexc2.houstoncell.com> To: metakit@equi4.com Date: Thu, 20 Dec 2001 10:27:24 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C18973.3512BF70" Subject: [Metakit] SPARC/Solaris 8 Installation and Compile Instructions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C18973.3512BF70 Content-Type: text/plain; charset="iso-8859-1" I am interested in using your product with Python 2.1.1 on a Sun Solaris 8 Sparc server. At present, all my attempts to compile and install have failed. Can anyone help on this effort? Hints, comments, suggestions, etc are all welcome. ------_=_NextPart_001_01C18973.3512BF70 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable SPARC/Solaris 8 Installation and Compile Instructions

I am interested in using your product = with Python 2.1.1 on a Sun Solaris 8 Sparc server. At present, all my = attempts to compile and install have failed. Can anyone help on this = effort? Hints, comments, suggestions, etc are all welcome.

------_=_NextPart_001_01C18973.3512BF70-- From mlh@idi.ntnu.no Thu Dec 20 15:44:41 2001 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA05497 for ; Thu, 20 Dec 2001 15:44:41 -0600 Received: from vier.idi.ntnu.no (IDENT:22955@vier.idi.ntnu.no [129.241.103.4]) by zevs.idi.ntnu.no (8.9.3/8.9.3) with ESMTP id WAA16563; Thu, 20 Dec 2001 22:35:30 +0100 (MET) Received: (from mlh@localhost) by vier.idi.ntnu.no (8.9.3/8.9.3) id WAA04535; Thu, 20 Dec 2001 22:35:30 +0100 (MET) From: Magnus Lie Hetland Date: Thu, 20 Dec 2001 22:35:30 +0100 To: hans.yandle@cingular.com Cc: metakit@equi4.com Subject: Re: [Metakit] SPARC/Solaris 8 Installation and Compile Instructions Message-ID: <20011220223530.F3753@idi.ntnu.no> References: <5B572F4E690AD411ADC200508BCF39920324D26B@scorpexc2.houstoncell.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <5B572F4E690AD411ADC200508BCF39920324D26B@scorpexc2.houstoncell.com>; from hans.yandle@cingular.com on Thu, Dec 20, 2001 at 10:27:24AM -0600 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Thu, Dec 20, 2001 at 10:27:24AM -0600, hans.yandle@cingular.com wrote: > I am interested in using your product with Python 2.1.1 on a Sun Solaris 8 > Sparc server. At present, all my attempts to compile and install have > failed. Can anyone help on this effort? Hints, comments, suggestions, etc > are all welcome. I'm sure this won't be very helpful, but I've used MetaKit with Python 2.1 (and 2.2) on Solaris 8 without any problems at all. Since I had no problems, I also have no real hints, but perhaps the fact that it can be done could serve as an encouragement? :) -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From lvirden@cas.org Thu Dec 20 21:56:42 2001 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id VAA16098 for ; Thu, 20 Dec 2001 21:56:42 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id fBL3l2223656 for ; Thu, 20 Dec 2001 22:47:02 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id fBL3l0V17077; Thu, 20 Dec 2001 22:47:00 -0500 (EST) Date: Thu, 20 Dec 2001 22:47:00 -0500 (EST) From: "Larry W. Virden" Message-Id: <20011220224659.AAB17061@cas.org> Subject: Re: [Metakit] SPARC/Solaris 8 Installation and Compile Instructions In-Reply-To: Your message of Thu, 20 Dec 2001 10:27:24 -0600 To: metakit@equi4.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I suggest that you describe what compiler you are using, what version of python, metakit, etc. you are using, what configure options you are using, etc. as well as showing how it fails. -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jyl@best.com Thu Dec 20 23:38:45 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id XAA17618 for ; Thu, 20 Dec 2001 23:38:45 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id VAA17777; Thu, 20 Dec 2001 21:29:07 -0800 (PST) Message-ID: <3C22D765.4000202@best.com> Date: Thu, 20 Dec 2001 22:32:05 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Magnus Lie Hetland CC: hans.yandle@cingular.com, metakit@equi4.com Subject: Re: [Metakit] SPARC/Solaris 8 Installation and Compile Instructions References: <5B572F4E690AD411ADC200508BCF39920324D26B@scorpexc2.houstoncell.com> <20011220223530.F3753@idi.ntnu.no> Content-Type: multipart/alternative; boundary="------------010908050002090903010707" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --------------010908050002090903010707 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit I've just finished porting my e4Graph software (http://e4graph.sourceforge.net/) to Metakit 2.4.2 on both Solaris 7 and on Windows 98, with absolutely no problems. All tests of both Metakit and e4Graph pass flawlessly, and I think Metakit 2.4.2 is the best release so far. Upgrade if you can, is my advice. JYL Magnus Lie Hetland wrote: >On Thu, Dec 20, 2001 at 10:27:24AM -0600, hans.yandle@cingular.com wrote: > >>I am interested in using your product with Python 2.1.1 on a Sun Solaris 8 >>Sparc server. At present, all my attempts to compile and install have >>failed. Can anyone help on this effort? Hints, comments, suggestions, etc >>are all welcome. >> > >I'm sure this won't be very helpful, but I've used MetaKit with >Python 2.1 (and 2.2) on Solaris 8 without any problems at all. >Since I had no problems, I also have no real hints, but perhaps >the fact that it can be done could serve as an encouragement? > >:) > --------------010908050002090903010707 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit I've just finished porting my e4Graph software (http://e4graph.sourceforge.net/) to Metakit 2.4.2 on both Solaris 7 and on Windows 98, with absolutely no problems. All tests of both Metakit and e4Graph pass flawlessly, and I think Metakit 2.4.2 is the best release so far. Upgrade if you can, is my advice.

JYL

Magnus Lie Hetland wrote:
On Thu, Dec 20, 2001 at 10:27:24AM -0600, hans.yandle@cingular.com wrote:
I am interested in using your product with Python 2.1.1 on a Sun Solaris 8
Sparc server. At present, all my attempts to compile and install have
failed. Can anyone help on this effort? Hints, comments, suggestions, etc
are all welcome.

I'm sure this won't be very helpful, but I've used MetaKit with
Python 2.1 (and 2.2) on Solaris 8 without any problems at all.
Since I had no problems, I also have no real hints, but perhaps
the fact that it can be done could serve as an encouragement?

:)


--------------010908050002090903010707-- From jcw@equi4.com Fri Dec 21 07:26:04 2001 Received: from smtp03.wxs.nl (smtp03.wxs.nl [195.121.6.37]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id HAA24964 for ; Fri, 21 Dec 2001 07:26:04 -0600 Received: from [10.0.1.55] ([195.121.236.58]) by smtp03.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GOP3J801.9Z8 for ; Fri, 21 Dec 2001 14:16:20 +0100 From: Jean-Claude Wippler To: metakit Date: Fri, 21 Dec 2001 14:13:08 +0100 Message-Id: <20011221131308.15747@mail.planet.nl> In-Reply-To: <3C22D765.4000202@best.com> References: <3C22D765.4000202@best.com> X-Mailer: CTM PowerMail 3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Breaking with the past Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jacob Levy wrote: >I've just finished porting my e4Graph software >(http://e4graph.sourceforge.net/) to Metakit 2.4.2 on both Solaris 7 and >on Windows 98, with absolutely no problems. All tests of both Metakit >and e4Graph pass flawlessly, and I think Metakit 2.4.2 is the best >release so far. Upgrade if you can, is my advice. Thanks. The state of 2.4.2, or rather HEAD-of-CVS (fixes continue to be made), is such that by now I urge everyone who hasn't migrated before to reconsider and start planning for it. The stability of the original 2.01 release is almost becoming a *disadvantage* now, in terms of version management. Let me try to summarize the main issues which affect a move towards 2.4.2: - File format: moving from 2.01 to 2.4 introduces a file format change, which is final after the first commit. This process is automatic, and there are no problems I know of (other than a very old 1.8.6 conversion issue marked in the source code), but it is important to note that after commit, 2.01 code can no longer use the file. - File open times have improved dramatically for complex datasets (many views) in that theuy are now essentially constant-time, but some of the time gained does come back as being slower at commit time. The reason is that free space determination still takes a full traversal of the "column-tree", even though data is not being loaded. - The M(emo) datatype is gone and has been merged with B(ytes). Memo- type storage is now automatic and adaptive, and works for B(ytes) as well as S(tring). This means that the choice between storing data as B(ytes) or S(tring) can now be made in terms of whether you want null-bytes, and the convenience of passing char* pointers in and out of MK calls. Comparisons on S(tring) continue to be vase-insensitive ASCII (with UTF-8 planned), while B(ytes) sort order is based on memcmp. - The c4_Storage::Store() call is gone. Restructuring must be done with GetAs. - Serialization no longer generates datafiles which *only* can be serialized back in. The new format allows normal memory-mapped and random-access use and is more suitable for streaming because the header includes the bytecount of what follows. Serialization is now the proper way to efficiently generate a new optimally-packed copy of a MK datafile. - Storage objects are now a special type of view (and derived from c4_View). One benefit is that you can init a storage as empty, then assign a new storage, so the need to allocate storage objects on the heap is gone, even when used as members in other objects. - There are plenty new features to help tempt you to make the switch, most notably the blocked and hash mapping views, which can deal far more efficiently with large datasets. Commit-extend and commit-aside need mentioning too, though these have not yet been optimized or as thoroughly tested as the rest. I would like to urge everyone who is still using a pre-2.4.2 release to make plans for a transition. We're in a chicken-and-egg type of situation: unless MK gets beaten up massively, I can never hope to resolve any remaining issues, and if I don't you may think that switching is still too risky. For those who use a recent release: please do report anything you find that does not work as expected, but also feel free to mention successful uses, so that everyone can get a feel for the amount of real use and correct operation. If confidentiality is an issue, feel free to email me and I'll post things without mentioning the source of the information. Let me finish off with all remaining issues in MK 2.4.2 that I'm aware of: - a crash on exit on Windows which *might* be a crash-on-commit in MK (long-time bug, can't debug because a debug build works) - just reported: Description() fails when the storage has no views at all (will resolve this asap) - hangs on LinuxPPC with m01 and m03, not yet investigated (gcc 2.95.3) And here's the list of 2.4.2 (or CVS) builds which works 100%, AFAIK: Windows/Linux/FreeBSD, Solaris (Sparc+x86), Mac (68k+PPC), MacOS X Also, as a to the past: a recent 2.4.x build on Win 3.1x (16-bit). Regards, Jean-Claude From jyl@best.com Fri Dec 21 09:02:57 2001 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA28348 for ; Fri, 21 Dec 2001 09:02:55 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id GAA14895 for ; Fri, 21 Dec 2001 06:52:50 -0800 (PST) Message-ID: <3C235B7F.5080304@best.com> Date: Fri, 21 Dec 2001 07:55:43 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Stupid questions: how to get rid of a column in a view,how to discard a view Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: System: Win98 Metakit 2.4.2. Due to format changes, I want to convert an old storage to the new format. In this conversion I'll lose several columns. Is it the case that when you GetAs in the old format, do other transformations, then GetAs in the new format, then commit, the old unneeded columns are discarded? That was my plan, however I'm not sure it's correct. Concrete example (showing stylized changes for only one view): Old format: nodes[firstMarker:I,firstVertex:I,lastVertex:I,vertexCount:I,...] New format: nodes[firstVerex:I,lastVertex:I,vertexCount:I,...] In other words the firstMarker:I should be discarded going from the old format to the new format. Second question: one of the views in the storage's old format should not be present when the storage is saved in the new format. Does setting its length to zero before committing make it not be saved during commit? Thanks! JYL From jcw@equi4.com Fri Dec 21 11:23:31 2001 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA00582 for ; Fri, 21 Dec 2001 11:23:30 -0600 Received: from [10.0.1.55] ([195.121.236.72]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GOPEJR00.Q02 for ; Fri, 21 Dec 2001 18:14:15 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Breaking with the past Date: Fri, 21 Dec 2001 18:10:43 +0100 Message-Id: <20011221171043.28161@mail.planet.nl> In-Reply-To: <20011221131308.15747@mail.planet.nl> References: <20011221131308.15747@mail.planet.nl> X-Mailer: CTM PowerMail 3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: To follow up on my own post: > - just reported: Description() fails when the storage has no views at all > (will resolve this asap) It looks like this was ok, but the catch is that this case returns NULL. I've just checked an optimization for GetAs into CVS, which makes it an order of magnitude faster than before when no restructuring is needed and when the storage contains many (or very complex) views. -jcw From baspey@yahoo.com Mon Dec 24 12:23:56 2001 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id MAA11371 for ; Mon, 24 Dec 2001 12:23:55 -0600 Received: from toulouse-1-a7-35-155.dial.proxad.net (HELO desktop) (62.147.35.155) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Dec 2001 18:14:30 -0000 Message-ID: <000e01c18ca6$6f860b40$9b23933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Mon, 24 Dec 2001 19:11:16 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Memory leak when scope of c4_View is global Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello and merry Christmas to everybody, I use C++ and Metakit 2.4.2 (hope lots of people will follow JCW's advice and break with the past like me). I have a view which is used by many different objects. One solution would be to declare this view as a globlal variable. I know that using globals is not elegant programming, but that would simplify my code alot. Another solution would be to use a pointer pointing toward this view. But I don't know the equivalent to MyView[RowIndex] when MyView is a pointer instead of a real c4_View. So I choose the global solution, but I get important memory leaks. Adding "c4_View MyView;" somewhere globally, generates a 48-bytes memory leak. And if I use this view and forget to do "MyView = c4_View();" before the program stops, there are then lots lots of memory leaks. Any comment appreciated. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From baspey@yahoo.com Tue Jan 1 05:58:39 2002 Received: from smtp013.mail.yahoo.com (smtp013.mail.yahoo.com [216.136.173.57]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id FAA07771 for ; Tue, 1 Jan 2002 05:58:39 -0600 Received: from toulouse-1-a7-34-34.dial.proxad.net (HELO desktop) (62.147.34.34) by smtp.mail.vip.sc5.yahoo.com with SMTP; 1 Jan 2002 11:48:44 -0000 Message-ID: <000b01c192b9$d0ed4420$2222933e@desktop> From: "Pascal Baspeyras" To: "Metakit" Date: Tue, 1 Jan 2002 12:43:30 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Memory leak when scope of c4_View is global Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello and Happy New Year to everyone, the memory leak I reported before has nothing to do with Metakit. It's all my fault as I could see when I tried to declare as global another variable, and also got memory leaks. Globals are tricky, and I now understand better why they should be avoided as much as possible. I won't go any further on the subject since it's only a C++ related question. _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From Mark.Oakden@ubsw.com Thu Jan 10 05:18:32 2002 Received: from gate2.ldn.ubswarburg.com (gate2.ldn.ubswarburg.com [139.149.1.38]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id FAA29271 for ; Thu, 10 Jan 2002 05:18:31 -0600 From: Mark.Oakden@ubsw.com Received: (from smap@localhost) by gate2.ldn.ubswarburg.com (8.8.8/8.8.8) id LAA29808 for ; Thu, 10 Jan 2002 11:08:01 GMT Received: from (nine.ubswarburg.com [192.168.0.4]) by gate2 via smap (V2.0) id xma029719; Thu, 10 Jan 2002 11:07:57 GMT Received: from ln4p1013pos.ldn.swissbank.com (virscan2 [192.168.0.4]) by virscan2.swissbank.com (8.8.8/8.8.8) with ESMTP id LAA11561 for ; Thu, 10 Jan 2002 11:10:01 GMT Received: from ln4p2350.ldn.swissbank.com (ln4p2350.ldn.swissbank.com [172.16.232.73]) by ln4p1013pos.ldn.swissbank.com (8.8.8/8.8.8) with ESMTP id LAA27128 for ; Thu, 10 Jan 2002 11:07:54 GMT Received: from localhost (root@localhost) by ln4p2350.ldn.swissbank.com (8.8.6 (PHNE_14041)/8.8.6/WDR gamma evision: 1.4 $) with ESMTP id LAA07978 for ; Thu, 10 Jan 2002 11:07:54 GMT X-OpenMail-Hops: 1 Date: Thu, 10 Jan 2002 11:07:53 +0000 Message-Id: MIME-Version: 1.0 TO: metakit@equi4.com Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline ;Creation-Date="Thu, 10 Jan 2002 11:07:52 +0000" Content-Transfer-Encoding: 7bit X-WDR-Disclaimer: Version $Revision: 1.15 $ Subject: [Metakit] Wikit corruption Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Sorry if this is the wrong forum to ask this question. I suspect that the cause of my problem is Wikit specific, but that the recovery (if it is possible) may require Metakit knowledge. [I have also posted this question to comp.lang.tcl] I have been setting up an internal Wiki using Wikit. I got it working but yesterday everything got corrupted somehow. I suspect that it happened when two people were editing via CGI at the same time, but can't prove that. The main page has now got a block of binary looking junk on the top and all the links on it to pages we had created have reverted to the "[New Link]" appearance (with linked "[" brackets to edit the "new" page). Has anybody seen this before? I assume it can be prevented with appropriate script/tclkit versions. Can I retrieve my pages from the corrupt Wikit file? Cheers, Mark -- Mark Oakden O:+44 20 75686190 M:+44 7703 531701 Platform Architecture and Design www.perotsystems.com perotsystems @ UBS Private Banking Europe www.ubs.com ---> any legalese/disclaimer below this line was added in transit <--- Visit our website at http://www.ubswarburg.com This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. From stephen@myseraph.org Thu Jan 10 05:19:37 2002 Received: from c007.snv.cp.net (c007-h011.c007.snv.cp.net [209.228.33.217]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id FAA29318 for ; Thu, 10 Jan 2002 05:19:35 -0600 Received: (cpmta 15160 invoked from network); 10 Jan 2002 03:09:00 -0800 Received: from 65.184.159.237 (HELO JEREMY) by smtp.directvinternet.com (209.228.33.217) with SMTP; 10 Jan 2002 03:09:00 -0800 X-Sent: 10 Jan 2002 11:09:00 GMT Message-ID: <001101c199c7$3a0dd050$0100a8c0@JEREMY> From: "Stephen Hansen" To: Date: Thu, 10 Jan 2002 03:09:09 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Is Metakit an answer to this problem? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, This has a few different sub-questions all lumped together, i'm sorry :) In a project i'm working on, my database needs will conform to a structure of approximately this shape: Objects[ ID:I, Owner:I, Attributes[ Name:S, Value:S], Body:S] That is, a database of items, each with a unique integer id and integer owner, and then a series of arbitrary Name:Value pairs and the actual data of the object. Its really not a complicated database, but it may start getting large in theory-- although certainly not into the GB range of things. In theory, thousands of items, each with anywhere between ten to twenty name:value pairs. Now, speed is absolutely critical. What I'll be doing with the above structure is a periodic search to answer such questions as, "for owner#, find all items who have attribute 'Flagged' set to 'Yes'." These searches may be frequent, and are not necessarily cacheable, and response time is very important. It is potentially possible that I can store a list of the most common attributes on the db record itself, making the description essentially: Objects[ ID:I, Owner:I, Flagged:S, Shown:S, etc:S, Attributes[ Name:S, Value:S], Body:S] There are approximately 10-15 attributes which are most common, and which I can thus hardwire into the db there-- and if you think it would bring a significant performance boost, I will. There will still be the need for an arbitrary name-value section though, even though it would then be significantly less used. Due to Metakit's column-wise setup, it seems to me that it is perhaps the right tool for the job. What do you think? For instance, I imagine something of the effect of: usersItems = items.select(id=UserId) flaggedItems = userItems.select(...) The above is in Python, which is what will be used for everything until it becomes necessary to recode the bottlenecks in C/C++. I'm not precisely sure how the second line should work, due to my current inexperience with Metakit's syntax; i'm almost tempted to just use a 'for' loop and go over the usersItems, and doing a find() on the Attributes subview of each item then compare it. There may or may not be a better way :) Anyways. I'm somewhat groggy, sorry for any horrendous grammar or logic errors in the above. :) Thanks in advance. --Stephen From gmcm@hypernet.com Thu Jan 10 08:19:36 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id IAA03963 for ; Thu, 10 Jan 2002 08:19:35 -0600 Received: from PARANOIA (204.176.40.121) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Thu, 10 Jan 2002 09:09:12 -0500 From: "Gordon McMillan" To: Date: Thu, 10 Jan 2002 09:03:46 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Is Metakit an answer to this problem? Reply-to: gmcm@hypernet.com Message-ID: <3C3D58F2.4853.3E43A294@localhost> Priority: normal In-reply-to: <001101c199c7$3a0dd050$0100a8c0@JEREMY> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 10 Jan 2002 at 3:09, Stephen Hansen wrote: > In a project i'm working on, my database needs will conform > to a structure of approximately this shape: > > Objects[ > ID:I, > Owner:I, > Attributes[ > Name:S, > Value:S], > Body:S] > > .... In theory, thousands of items, each with anywhere > between ten to twenty name:value pairs. Sounds very reasonable. > Now, speed is absolutely critical. What I'll be doing > with the above structure is a periodic search to answer such > questions as, "for owner#, find all items who have > attribute 'Flagged' set to 'Yes'." These searches may be > frequent, and are not necessarily cacheable, and response > time is very important. It is potentially possible that I > can store a list of the most common attributes on > the db record itself [snip] > which I can thus hardwire into the db there-- and if you > think it would bring a significant performance boost, I > will. I certainly wouldn't start out that way! I kind of doubt you'll need it. > There will still be the need for an arbitrary > name-value section though, even though it would then be > significantly less used. [snip] > For instance, I imagine something of the effect of: > > usersItems = items.select(id=UserId) > flaggedItems = userItems.select(...) > userrecs = items.select(id=Userid) useritems = userrecs.flatten(userrecs.Attributes) Now you have a flat view, and your search becomes flagged = useritems.select(useritems.Name="Flagged", useritems.Value="Yes") If you just interested in existence, then a find would be faster in the last step. -- Gordon http://www.mcmillan-inc.com/ From erlavigne@wanadoo.fr Fri Jan 11 14:10:29 2002 Received: from mel-rto1.wanadoo.fr (smtp-out-1.wanadoo.fr [193.252.19.188]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA09796 for ; Fri, 11 Jan 2002 14:10:23 -0600 Received: from mel-rta5.wanadoo.fr (193.252.19.122) by mel-rto1.wanadoo.fr; 11 Jan 2002 20:59:16 +0100 Received: from [192.168.0.2] (193.252.31.240) by mel-rta5.wanadoo.fr; 11 Jan 2002 20:58:56 +0100 User-Agent: Microsoft-Entourage/9.0.2509 Date: Fri, 11 Jan 2002 20:58:55 +0100 From: Eric To: Message-ID: In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Metakit memory usage issue Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add data into a nested view. Addind data into a nested view seeems to use a lot of memory (and on macOS memory is a limited resource). Adding about 200 entries (with just one entry in the nested view for each) requires more than 1 Mb. This memory is not lost because when the storage is closed this memory is released. But importing a few hundred entries requires a few megabytes of available memory, and that's not very convenient. May be I'm doing something wrong in my code? You will find below a sample source code (for MacOS) reproducing this problem. Any help (or workaround) will be appreciated. Thanks in advance. -- Eric //======================================================================== #include "mk4.h" static long sFreeMem, sStartFreeMem; static void DumpFreeMem() { long theFreeMem = ::FreeMem(); printf("\tMemory change: %ld Kb %ld b\n", ((theFreeMem - sFreeMem) / 1024L), ((theFreeMem - sFreeMem) % 1024L)); sFreeMem = theFreeMem; } #define kTest_Desc "value[text:S,names[name:S,nickname:S]]" static void TestMetaKit() { sFreeMem = sStartFreeMem = ::FreeMem(); c4_Storage theStorage("test", true); c4_StringProp pText("text"); c4_ViewProp pNames("names"); c4_StringProp pNickName("nickname"); c4_StringProp pName("name"); c4_View theView = theStorage.GetAs(kTest_Desc); c4_Row theRow; theView.SetSize(0); for (size_t i = 0 ; i < 10 ; i++) { pText(theRow) = "test"; int n = theView.Add(theRow); for (size_t j = 0 ; j < 1 ; j++) { c4_View theNamesView = pNames(theView[n]); theNamesView.Add(pName["John"] + pNickName["johnny"]); } DumpFreeMem(); } long theFreeMem = ::FreeMem(); printf("\tTotal change: %ld Kb %ld b\n", ((theFreeMem - sStartFreeMem) / 1024L), ((theFreeMem - sStartFreeMem) % 1024L)); } From Murat_Berk@bmc.com Fri Jan 11 14:25:14 2002 Received: from creeper.bmc.com (camaro.bmc.com [198.207.223.231]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id OAA10274 for ; Fri, 11 Jan 2002 14:25:14 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2/8.10.2) with ESMTP id g0BKDW710422; Fri, 11 Jan 2002 14:13:32 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Fri, 11 Jan 2002 14:13:50 -0600 Message-ID: From: "Berk, Murat" To: "'Eric'" , metakit@equi4.com Subject: RE: [Metakit] Metakit memory usage issue Date: Fri, 11 Jan 2002 14:13:48 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We had exact some problem w/ 2.0.1 This happens if you create nested views. Each nested view's column allocates 4KB memory. So even if you have a nested view w/ only 2-3 columns, (name, value), you use around 16K min (since string fields require 2 columns, one of them index or someting like that) -----Original Message----- From: Eric [mailto:erlavigne@wanadoo.fr] Sent: Friday, January 11, 2002 1:59 PM To: metakit@equi4.com Subject: [Metakit] Metakit memory usage issue Hello, I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add data into a nested view. Addind data into a nested view seeems to use a lot of memory (and on macOS memory is a limited resource). Adding about 200 entries (with just one entry in the nested view for each) requires more than 1 Mb. This memory is not lost because when the storage is closed this memory is released. But importing a few hundred entries requires a few megabytes of available memory, and that's not very convenient. May be I'm doing something wrong in my code? You will find below a sample source code (for MacOS) reproducing this problem. Any help (or workaround) will be appreciated. Thanks in advance. -- Eric //======================================================================== #include "mk4.h" static long sFreeMem, sStartFreeMem; static void DumpFreeMem() { long theFreeMem = ::FreeMem(); printf("\tMemory change: %ld Kb %ld b\n", ((theFreeMem - sFreeMem) / 1024L), ((theFreeMem - sFreeMem) % 1024L)); sFreeMem = theFreeMem; } #define kTest_Desc "value[text:S,names[name:S,nickname:S]]" static void TestMetaKit() { sFreeMem = sStartFreeMem = ::FreeMem(); c4_Storage theStorage("test", true); c4_StringProp pText("text"); c4_ViewProp pNames("names"); c4_StringProp pNickName("nickname"); c4_StringProp pName("name"); c4_View theView = theStorage.GetAs(kTest_Desc); c4_Row theRow; theView.SetSize(0); for (size_t i = 0 ; i < 10 ; i++) { pText(theRow) = "test"; int n = theView.Add(theRow); for (size_t j = 0 ; j < 1 ; j++) { c4_View theNamesView = pNames(theView[n]); theNamesView.Add(pName["John"] + pNickName["johnny"]); } DumpFreeMem(); } long theFreeMem = ::FreeMem(); printf("\tTotal change: %ld Kb %ld b\n", ((theFreeMem - sStartFreeMem) / 1024L), ((theFreeMem - sStartFreeMem) % 1024L)); } _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Fri Jan 11 15:40:34 2002 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA12382 for ; Fri, 11 Jan 2002 15:40:33 -0600 Received: from [10.0.1.55] ([195.121.233.99]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GPSMDX02.9KF for ; Fri, 11 Jan 2002 22:29:57 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Metakit memory usage issue Date: Fri, 11 Jan 2002 22:27:11 +0100 Message-Id: <20020111212711.17344@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Eric, >I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add >data into a nested view. >Addind data into a nested view seeems to use a lot of memory (and on macOS >memory is a limited resource). > >Adding about 200 entries (with just one entry in the nested view for each) >requires more than 1 Mb. This memory is not lost because when the storage is >closed this memory is released. But importing a few hundred entries requires >a few megabytes of available memory, and that's not very convenient. Yes, at some point, I changed MK to allocate columns in 4 Kb increments, and with subviews, you'll end up with at least one of those each as soon as there is some data. The trouble is that on the Mac (OS 8/9), things are particularly tricky because of the lack of memory-mapped files. It'll take some work to improve this. Basically, what needs to be done is to track the "over-allocated" columns, and then truncate the ones which haven't been modified for a while - so as to limit the number of columns which waste so much space. I'm quite hesitant to do this, since it is Mac-specific (more accurately: an issue only of platforms without mmap support). Dropping all the slack is not a real option, because that could severely reduce modify-performance. If anyone reading this has an idea for a simpler fix, please share your wisdom! -jcw From jcw@equi4.com Fri Jan 11 15:41:03 2002 Received: from smtp06.wxs.nl (smtp06.wxs.nl [195.121.6.58]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA12409 for ; Fri, 11 Jan 2002 15:41:02 -0600 Received: from [10.0.1.55] ([195.121.233.99]) by smtp06.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GPSMDW00.IGZ for ; Fri, 11 Jan 2002 22:29:56 +0100 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Wikit corruption Date: Fri, 11 Jan 2002 22:16:12 +0100 Message-Id: <20020111211612.32007@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark.Oakden@ubsw.com wrote: >I have been setting up an internal Wiki using Wikit. I got it working >but yesterday everything got corrupted somehow. > >I suspect that it happened when two people were editing via CGI at the >same time, but can't prove that. Yes, this seems plausible. I've responded in more detail to your post on the Tcl'ers Wiki mailing list at . -jcw From Murat_Berk@bmc.com Fri Jan 11 15:53:51 2002 Received: from babbler.bmc.com (firebird.bmc.com [198.207.223.228]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA12815; Fri, 11 Jan 2002 15:53:51 -0600 Received: from ec02-hou.bmc.com (localhost [127.0.0.1]) by babbler.bmc.com (8.10.2/8.10.2) with ESMTP id g0BLhN114442; Fri, 11 Jan 2002 15:43:23 -0600 (CST) Received: by EC02-HOU.bmc.com with Internet Mail Service (5.5.2653.19) id ; Fri, 11 Jan 2002 15:40:38 -0600 Message-ID: From: "Berk, Murat" To: "'Jean-Claude Wippler'" , metakit Subject: RE: [Metakit] Metakit memory usage issue Date: Fri, 11 Jan 2002 15:40:38 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: When we save them, it is saved in a compact form. Because size of database is not huge. Does this means that when it is loaded, we do not use memory mapped area, but copy to the memory? Murat -----Original Message----- From: Jean-Claude Wippler [mailto:jcw@equi4.com] Sent: Friday, January 11, 2002 3:27 PM To: metakit Subject: Re: [Metakit] Metakit memory usage issue Eric, >I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add >data into a nested view. >Addind data into a nested view seeems to use a lot of memory (and on macOS >memory is a limited resource). > >Adding about 200 entries (with just one entry in the nested view for each) >requires more than 1 Mb. This memory is not lost because when the storage is >closed this memory is released. But importing a few hundred entries requires >a few megabytes of available memory, and that's not very convenient. Yes, at some point, I changed MK to allocate columns in 4 Kb increments, and with subviews, you'll end up with at least one of those each as soon as there is some data. The trouble is that on the Mac (OS 8/9), things are particularly tricky because of the lack of memory-mapped files. It'll take some work to improve this. Basically, what needs to be done is to track the "over-allocated" columns, and then truncate the ones which haven't been modified for a while - so as to limit the number of columns which waste so much space. I'm quite hesitant to do this, since it is Mac-specific (more accurately: an issue only of platforms without mmap support). Dropping all the slack is not a real option, because that could severely reduce modify-performance. If anyone reading this has an idea for a simpler fix, please share your wisdom! -jcw _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Fri Jan 11 16:08:27 2002 Received: from smtp05.wxs.nl (smtp05.wxs.nl [195.121.6.57]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA13285 for ; Fri, 11 Jan 2002 16:08:27 -0600 Received: from [10.0.1.55] ([195.121.233.181]) by smtp05.wxs.nl (Netscape Messaging Server 4.15) with ESMTP id GPSNO601.3NB for ; Fri, 11 Jan 2002 22:57:42 +0100 From: Jean-Claude Wippler To: metakit Subject: RE: [Metakit] Metakit memory usage issue Date: Fri, 11 Jan 2002 22:55:04 +0100 Message-Id: <20020111215504.19817@mail.planet.nl> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Berk, Murat wrote: >When we save them, it is saved in a compact form. Because size of database >is not huge. Does this means that when it is loaded, we do not use memory >mapped area, but copy to the memory? If I understand you right: no, the allocation happens when buffers get allocated for columns. When reading data, that never happens. It's upon modifying it. So... yes, perhaps in this case a workaround would be to commit somewhat more frequently on the Mac (this will slow it down, i.e. trade speed for memory use). -jcw From erlavigne@wanadoo.fr Fri Jan 11 16:15:15 2002 Received: from mel-rto7.wanadoo.fr (smtp-out-7.wanadoo.fr [193.252.19.26]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id QAA13496 for ; Fri, 11 Jan 2002 16:15:14 -0600 Received: from mel-rta2.wanadoo.fr (193.252.19.152) by mel-rto7.wanadoo.fr; 11 Jan 2002 23:04:10 +0100 Received: from [192.168.0.2] (193.252.31.240) by mel-rta2.wanadoo.fr; 11 Jan 2002 23:03:53 +0100 User-Agent: Microsoft-Entourage/9.0.2509 Date: Fri, 11 Jan 2002 23:03:52 +0100 Subject: Re: [Metakit] Metakit memory usage issue From: Eric To: metakit Message-ID: In-Reply-To: <20020111212711.17344@mail.planet.nl> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude, Why not adding a 'Compact' method to c4_View class that will do this work. It will be the developer responsibility to call this method when memory will get low (or on a regular basis whenn adding a lot of entries). This should not impact performance in all other cases and will offer a clean way to deal with this special case. I don't know if it's possible (and how complex it is) to implement this 'Compact' algorithm but personnally I will be very happy with it. -- Eric > Eric, > >> I'm using Metakit 2.4.2 on MacOS. Everything works fine except when I add >> data into a nested view. >> Addind data into a nested view seeems to use a lot of memory (and on macOS >> memory is a limited resource). >> >> Adding about 200 entries (with just one entry in the nested view for each) >> requires more than 1 Mb. This memory is not lost because when the storage is >> closed this memory is released. But importing a few hundred entries requires >> a few megabytes of available memory, and that's not very convenient. > > Yes, at some point, I changed MK to allocate columns in 4 Kb increments, > and with subviews, you'll end up with at least one of those each as soon > as there is some data. > > The trouble is that on the Mac (OS 8/9), things are particularly tricky > because of the lack of memory-mapped files. > > It'll take some work to improve this. Basically, what needs to be done > is to track the "over-allocated" columns, and then truncate the ones > which haven't been modified for a while - so as to limit the number of > columns which waste so much space. > > I'm quite hesitant to do this, since it is Mac-specific (more accurately: > an issue only of platforms without mmap support). Dropping all the slack > is not a real option, because that could severely reduce modify-performance. > > If anyone reading this has an idea for a simpler fix, please share your > wisdom! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jvasco@bellatlantic.net Sun Jan 13 20:51:18 2002 Received: from out001pub.verizon.net (out001pub.verizon.net [206.46.170.101]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id UAA04386 for ; Sun, 13 Jan 2002 20:51:17 -0600 Received: from localhost.localdomain (pool-138-89-75-20.mad.east.verizon.net [138.89.75.20]) by out001pub.verizon.net with ESMTP for ; id g0E2e4OB006892 Sun, 13 Jan 2002 20:40:04 -0600 (CST) From: Jose Vasconcellos To: metakit@equi4.com Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/1.0 (Preview Release) Date: 13 Jan 2002 21:39:15 -0500 Message-Id: <1010975956.11969.0.camel@localhost.localdomain> Mime-Version: 1.0 Subject: [Metakit] Building Mk4py with python2.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For the adventurous that want to try using python2.2 be warned! The release has a declaration error such that Mk4py won't compile. The fix is to change the declaration in the python include file abstract.h. There is a define PyMapping_DelItemString that should be set to PyDict_DelItemString. It is erroneously set to PyObject_DelItemString which doesn't exist. Regards, Jose Vasconcellos From stephen@myseraph.org Sun Jan 13 23:16:09 2002 Received: from c007.snv.cp.net (c007-h014.c007.snv.cp.net [209.228.33.221]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA05163 for ; Sun, 13 Jan 2002 23:16:06 -0600 Received: (cpmta 9519 invoked from network); 13 Jan 2002 21:04:52 -0800 Received: from 65.184.159.237 (HELO JEREMY) by smtp.directvinternet.com (209.228.33.221) with SMTP; 13 Jan 2002 21:04:52 -0800 X-Sent: 14 Jan 2002 05:04:52 GMT Message-ID: <000501c19cb9$118eab50$0100a8c0@JEREMY> From: "Stephen Hansen" To: References: <1010975956.11969.0.camel@localhost.localdomain> Subject: Re: [Metakit] Building Mk4py with python2.2 Date: Sun, 13 Jan 2002 21:05:21 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Actually, the "real" fix is to use CVS to grab the release22-maint branch, where this bug is fixed. It is actually suspsoed to be PyObject_DelItemString, which was simply ommitted from abstract.c before the release. :) Go to, http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Object s/abstract.c, and you'll see on the top of the list is the fix we're discussing here. --S > For the adventurous that want to try using python2.2 be > warned! The release has a declaration error such that > Mk4py won't compile. The fix is to change the declaration > in the python include file abstract.h. There is a define > PyMapping_DelItemString that should be set to PyDict_DelItemString. > It is erroneously set to PyObject_DelItemString which doesn't > exist. > > Regards, > > Jose Vasconcellos > > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From ia@innovatech.dk Mon Jan 14 09:32:29 2002 Received: from mail.web-solutions.dk (ns2.web-solutions.dk [212.130.48.54]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA19362 for ; Mon, 14 Jan 2002 09:32:29 -0600 Message-Id: <200201141532.JAA19362@trixie.triqs.com> Received: from there ([80.62.45.76]) by mail.web-solutions.dk (Merak 4.10.050) with SMTP id JGB36881 for ; Mon, 14 Jan 2002 16:21:10 +0100 Content-Type: text/plain; charset="iso-8859-1" From: Israel Alvarez Reply-To: ia@innovatech.dk Organization: InnovaTech To: metakit@equi4.com Date: Mon, 14 Jan 2002 16:19:38 -0500 X-Mailer: KMail [version 1.3.1] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Metakit] Installation problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all!, I'm trying to install Metakit 2.4.2 in my Red Hat Linux System 7.2, with gcc 2.96. The problem I have is that when I run the ./configure i get: .... > checking for ANSI C header files... no > checking for working const... no > checking for inline... no > checking size of long... 0 > ./configure: line 5886: syntax error near unexpected token `AC_CHECK_TYPES(long' > ./configure: line 5886: `AC_CHECK_TYPES(long long)' and it fails. If I try to run ./autoconf I get: > autoconf: Undefined macros: > configure.in:68:AC_CHECK_TYPES(long long) and it fails. It seems like I don't have autoconf properly running, but I've never worked with it so I don't know what to do. In the other way, Is is possible to use your libmk4.so and libmk4.a libraries downloaded from your site and linking them to my C++ application? If I could do this, I think I would not need to install the sources. Is it OK? Thanks in advance -- Israel Alvarez ____________________________ InnovaTech Dortheavej 10 DK-2400 København NV ____________________________ Phone +45 70 27 28 11 Cell. Phone +45 20 93 00 53 ____________________________ http://www.InnovaTech.dk From gmcm@hypernet.com Mon Jan 14 11:40:03 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA24018 for ; Mon, 14 Jan 2002 11:40:02 -0600 Received: from PARANOIA (204.176.40.69) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Mon, 14 Jan 2002 12:29:31 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Mon, 14 Jan 2002 12:28:53 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Installation problems Reply-to: gmcm@hypernet.com Message-ID: <3C42CF05.27878.1082B02@localhost> Priority: normal In-reply-to: <200201141532.JAA19362@trixie.triqs.com> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=ISO-8859-1 Content-description: Mail message body Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from Quoted-printable to 8bit by trixie.triqs.com id LAA24018 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Israel, Don't know why that happens, but start over by running configure yourself, and it goes away. -- Gordon http://www.mcmillan-inc.com/ On 14 Jan 2002 at 16:19, Israel Alvarez wrote: > Hi all!, > I'm trying to install Metakit 2.4.2 in my Red Hat Linux System > 7.2, with gcc 2.96. The problem I have is that when I run the > ./configure i get: > > .... > > checking for ANSI C header files... no > > checking for working const... no > > checking for inline... no > > checking size of long... 0 > > ./configure: line 5886: syntax error near unexpected token > `AC_CHECK_TYPES(long' > > ./configure: line 5886: `AC_CHECK_TYPES(long long)' > > and it fails. > > If I try to run ./autoconf I get: > > > autoconf: Undefined macros: > > configure.in:68:AC_CHECK_TYPES(long long) > > and it fails. > > It seems like I don't have autoconf properly running, but I've > never worked with it so I don't know what to do. > > In the other way, Is is possible to use your libmk4.so and > libmk4.a libraries downloaded from your site and linking them to > my C++ application? If I could do this, I think I would not need > to install the sources. Is it OK? > > Thanks in advance > > -- > > Israel Alvarez > ____________________________ > > InnovaTech > Dortheavej 10 > DK-2400 København NV > ____________________________ > > Phone +45 70 27 28 11 > Cell. Phone +45 20 93 00 53 > ____________________________ > > http://www.InnovaTech.dk > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From mlh@idi.ntnu.no Tue Jan 15 15:41:55 2002 Received: from zevs.idi.ntnu.no (IDENT:0@zevs.idi.ntnu.no [129.241.110.12]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id PAA15736 for ; Tue, 15 Jan 2002 15:41:54 -0600 Received: from vier.idi.ntnu.no (IDENT:22955@vier.idi.ntnu.no [129.241.103.4]) by zevs.idi.ntnu.no (8.10.2+Sun/8.9.3) with ESMTP id g0FLV3p06255 for ; Tue, 15 Jan 2002 22:31:03 +0100 (MET) Received: (from mlh@localhost) by vier.idi.ntnu.no (8.9.3/8.9.3) id WAA07881 for metakit@equi4.com; Tue, 15 Jan 2002 22:31:03 +0100 (MET) Date: Tue, 15 Jan 2002 22:31:03 +0100 From: Magnus Lie Hetland To: Metakit Mailing List Message-ID: <20020115223103.A7437@idi.ntnu.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] Swig for PHP... Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: By chance I looked at the Swig pages and it seems there is an experimental PHP module now... Perhaps this means that we can get PHP bindings for Metakit? - Magnus (who doesn't feel kompetent to do the binding himself...) -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From horst@freedict.de Tue Jan 15 23:05:23 2002 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id XAA25218 for ; Tue, 15 Jan 2002 23:05:22 -0600 From: horst@freedict.de Received: (qmail 12933 invoked from network); 16 Jan 2002 05:24:40 -0000 Received: from dialin-145-254-115-239.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.115.239) by mail.netbeat.de with SMTP; 16 Jan 2002 05:24:40 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id g0FLJOk14855; Tue, 15 Jan 2002 22:19:25 +0100 Message-Id: <200201152119.g0FLJOk14855@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Tue, 15 Jan 2002 22:19:20 +0100 (CET) Reply-To: horst@freedict.de To: metakit MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] how to use hash with two keys? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I try to use a hashed view with to key fields. But the performance I get is poor. To see what I do, apply the patch to find.py in the examples dir 80c80 < r = vw.find(p1=k) --- > r = vw.find(p1=k, p2=k+k) 96c96 < vwh = vw.hash(map) --- > vwh = vw.hash(map,2) 102c102 < vwh = vw.hash(map) --- > vwh = vw.hash(map,2) 114,115c114,115 < hash_key(n/2, 10000) < hash_key(n+1, 10000) --- > hash_key(n/2, 1000000/n) > hash_key(n+1, 1000000/n) What should I have done differently? Thanks, Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From peter@tbots.com Wed Jan 16 11:41:08 2002 Received: from tbots.com (pop.cloud-nine.co.uk [62.172.32.16]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id LAA16228 for ; Wed, 16 Jan 2002 11:41:07 -0600 Date: Wed, 16 Jan 2002 17:29:28 +0000 Message-Id: <200201161729.AA263717176@tbots.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii From: "peter" Reply-To: To: X-Mailer: Subject: [Metakit] metakit questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: hi Jean Claude There have been a few new releases of metakit since we last emailed each other. I hope you are well!! I have a couple of questions regarding metakit which I hope you can answer. The questions are primarily related to larger database files where I would require very fast operations. I wonder if metakit is the right choice given all the performance enhancements that have been made over the last 2 years. 1) when calling 'commit' - is the entire datafile rewritten, or just the part that has changed? 2) since metakit does not index its tables/columns , and I need best possible performance, is there any way to pick between binary search or some form of hashing? how does metakit compare to say a 'b-tree on disk' type of storage form? I know I could do an exhaustive performance test on this :-), but I'm a few weeks from a major product release, and I just wondered about the theoretical aspects of this. I havent found any notes on this in the documentation. 3) Does metakit have the equivalent of the SQL ...WHERE IN (1,2,3) , and if so, how does metakit go about the looking up the records? (I guess this relates to question 2 4) Does metakit now have equivalent of DELETE WHERE .... I remember having to reverse iterate through views do delete rows. If these questions are already answered in available literature somewhere, I would be grateful for a pointer. Best Regards, Peter Ritter ******************************************************** Peter Ritter Eurasia Software 0049 (0)172 695 2846 0049 (0)69 68600700 ******************************************************** From guenther.fischer@hrz.tu-chemnitz.de Thu Jan 17 01:24:48 2002 Received: from obelix.hrz.tu-chemnitz.de (obelix.hrz.tu-chemnitz.de [134.109.132.55]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id BAA10364 for ; Thu, 17 Jan 2002 01:24:48 -0600 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=fischer) by obelix.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #1) id 16R6kC-0005vp-00 for metakit@equi4.com; Thu, 17 Jan 2002 08:13:52 +0100 Date: Thu, 17 Jan 2002 08:13:51 +0100 (MET) From: Guenther Fischer To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from QUOTED-PRINTABLE to 8bit by trixie.triqs.com id BAA10364 Subject: [Metakit] 2 questions about tclkit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, 1. I use tclkit for linux/windows for a wine-database for german users (all texts in german language). With the latest version of tclkit I got the button names not shown correct (only those with german umlaut like ÄÖÜ and so on). Where there some changes in encoding? 2. I use tkhtml with tclkit to show help and other views. Now I also want to use images inside the html pages. gif will work but not jpeg. So I insert Img1.2 with libimg1.2.so and package require Img and get no a undefined Tk_PhotoExpand. Is there any help? thanks -- ~Guenther Fischer From jtate@mi-corporation.com Thu Jan 17 12:27:03 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA00567 for ; Thu, 17 Jan 2002 12:27:03 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0HIG0f12985 for ; Thu, 17 Jan 2002 13:16:00 -0500 From: "Joseph Tate" To: "MetaKit" Date: Thu, 17 Jan 2002 13:16:55 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Porting issues Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm trying to port Metakit to Windows CE so that our application (which runs under wince and hopefully windows 9x/NT) can have a common database backend. I've run into a few problems in the port and have a few questions. First of all, am I reinventing the wheel? Has someone else done this port? Second, is this a good idea given the limited memory and space on the PocketPC platform? If not, is there another similar solution that would be better suited? Third, the biggest issue in moving from Windows to Windows CE is the move from single byte to double byte characters. I've #def'd char->short, and replaced all the str* functions with their wide char equivalents, fixed memcopy issues in the string classes, etc, but now have issues with Views. Is there somewhere else in the Metakit code that relies on char being a single byte? (Not having a console or access to stdout/stderr is another issue, but I think I've got that handled.). Thanks in advance for your help. Joseph From guenther.fischer@hrz.tu-chemnitz.de Fri Jan 18 02:15:23 2002 Received: from asterix.hrz.tu-chemnitz.de (asterix.hrz.tu-chemnitz.de [134.109.132.84]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id CAA20545 for ; Fri, 18 Jan 2002 02:15:22 -0600 Received: from pat.hrz.tu-chemnitz.de ([134.109.132.143] ident=mail) by asterix.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #1) id 16RU0c-0006wf-00 for metakit@equi4.com; Fri, 18 Jan 2002 09:04:22 +0100 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=root) by pat.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #2) id 16RU0b-0007kC-00 for metakit@equi4.com; Fri, 18 Jan 2002 09:04:21 +0100 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.9.3/8.9.3) with ESMTP id JAA24185 for ; Fri, 18 Jan 2002 09:04:21 +0100 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Fri, 18 Jan 2002 09:04:21 +0100 (MET) From: Guenther Fischer To: metakit@equi4.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] tclkit and iso8859-1 ? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi again, I try to verify the differences in encoding betwen the 2 tclkit's I use. (the latest and some releases back). I try to change the system encoding (I know I should do that in real ...): tclkit % encoding system identity % encoding system iso8859-1 % encoding system iso8859-1 and with the latest: ~/tclkit % encoding system identity % encoding system iso8859-1 unknown encoding "iso8859-1" and with an actual tclsh: /usr/local/ActiveTcl/bin/tclsh % encoding system iso8859-1 (the last is the best :-) What can I do? -- ~Guenther Fischer From genec@goldsmithsinc.com Wed Jan 23 09:06:43 2002 Received: from mail.goldsmithsinc.com ([64.66.89.67]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA13995 for ; Wed, 23 Jan 2002 09:06:43 -0600 Received: from genec (genec [10.1.1.100]) by mail.goldsmithsinc.com (Switch-2.0.0/Switch-2.0.0) with SMTP id g0NEjeV16602 for ; Wed, 23 Jan 2002 08:45:40 -0600 Message-ID: <003501c1a41d$7f8fc560$6401010a@genec> From: "Gene Christian" To: Date: Wed, 23 Jan 2002 08:51:54 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-Mimeole: Produced By Microsoft MimeOLE V5.50.4807.1700 Subject: [Metakit] Python questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been working with the 2.4.2 version of metakit for a few days, and I have run into some problems with the python interface. First, I can't get the wrap function to work properly. The wrap.py file that comes in the distribution shows correct results only for the tuple (position based) wrapping. The class wrapping shows three rows, but the string data is blank, and the integer data is all zeros. The dictionary wrapping gets the integers right, but the strings show junk. Here is the output I get from the wrap.py under Windows 2000: class objects: a b c - - - 0 0 0 - - - Total: 3 rows dictionary elements: a b c --- --- - x?y x?y 1 x?y x?y 2 x?y x?y 3 --- --- - Total: 3 rows tuples (by position): a b c ----- ----- - one un 1 two deux 2 three trois 3 ----- ----- - Total: 3 rows I have tested this on a Windows 2000 PC using ActiveState's 2.1.1 distribution and the new 2.1.2 distribution from python.org. I was using the Mk4py.dll binary. I also tested the wrap.py function on a Debian 3.0 (Woody) system using the debian python 2.1.1 and the metakit binary. In all three cases, the result was the same. Also, I have been exploring the derived views and sub-views and I don't think that the results I am getting are correct. I first tried to use the view.select() function to create a derived view, and I expected to be able to append a record to the derived view and have the main view updated as well. I couldn't get this to work. I have included below a copy of the commands I used to test this: >>> import metakit >>> mk = metakit >>> v = mk.view() >>> p1 = mk.property('S','group') >>> p2 = mk.property('S','data') >>> v.addproperty(p1) 0 >>> v.addproperty(p2) 1 >>> v.structure() [Property('S', 'group'), Property('S', 'data')] >>> v.append(group="a",data="one") 0 >>> v.append(group="a",data="two") 1 >>> v.append(group="b",data="X") 2 >>> v.append(group="b",data="Y") 3 >>> mk.dump(v) group data ----- ---- a one a two b X b Y ----- ---- Total: 4 rows >>> v1 = v.select(group='a') >>> mk.dump(v1) group data ----- ---- a one a two ----- ---- Total: 2 rows >>> v1.append(group='a',data='three') 2 >>> mk.dump(v1) group data ----- ---- a one a two ----- ---- Total: 2 rows >>> mk.dump(v) group data ----- ---- a one a two b X b Y ----- ---- Total: 4 rows >>> v.append(group='a',data='three') 4 >>> mk.dump(v) group data ----- ----- a one a two b X b Y a three ----- ----- Total: 5 rows >>> mk.dump(v1) group data ----- ----- a one a two a three ----- ----- Total: 3 rows >>> I have had similar results with the groupby function. Am I using the wrong version of python (is it compiled for 2.0)? I could compile metakit on Linux, but my target platform is Windows and I don't have a windows compiler. Is this a bug or am I using it wrong? Thanks. Gene Christian From gmcm@hypernet.com Wed Jan 23 10:20:27 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id KAA16506 for ; Wed, 23 Jan 2002 10:20:27 -0600 Received: from PARANOIA (204.176.40.51) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Wed, 23 Jan 2002 11:09:15 -0500 From: "Gordon McMillan" To: Date: Wed, 23 Jan 2002 11:09:00 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Python questions Reply-to: gmcm@hypernet.com Message-ID: <3C4E99CC.19083.2F184439@localhost> Priority: normal In-reply-to: <003501c1a41d$7f8fc560$6401010a@genec> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 23 Jan 2002 at 8:51, Gene Christian wrote: > I have been working with the 2.4.2 version of metakit for a few > days, and I have run into some problems with the python > interface. > > First, I can't get the wrap function to work properly. > The wrap.py file that comes in the distribution shows > correct results only for the tuple (position based) > wrapping. The class wrapping shows three rows, but the > string data is blank, and the integer data is all zeros. > The dictionary wrapping gets the integers right, but the > strings show junk. You're right. I'll look into it. I've never used wrap with anything but tuples, so I don't know how long that's been broken. > Also, I have been exploring the derived views and > sub-views and I don't think that the results I am getting > are correct. I first tried to use the view.select() > function to create a derived view, and I expected to be > able to append a record to the derived view and have the > main view updated as well. Nope. You can alter a row in the selected view, and the changes will be reflected in the base view, but you can't change the (derived) container. If you append to the base view, a selected or sorted view will see the changes. You've stumbled into a confusing area. If you pull Mk4py from equi4.com's cvs, I've straightened it out so that it's much clearer what you can & cannot do (no more silent errors). There are now Views, Viewers, ROViewers. Also RORowRefs as well as RowRefs. Base views, view copies and the mapping views are all Views (for a mapping view, don't touch the base view). You can do anything. Viewers let you modify rows, but not the container (sort, select return Viewers). Basically, anything else gives you a ROViewer. Of course, sorting / selecting a ROViewer leaves it a ROViewer. > I have had similar results with the groupby function. Which is RO. > Am I using the wrong version of python (is it compiled for > 2.0)? I could compile metakit on Linux, but my target > platform is Windows and I don't have a windows compiler. > Is this a bug or am I using it wrong? I could send you a Mk4py built from CVS for 2.1 if you like. It's still "experimental", though I'm using it fulltime. -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Thu Jan 24 09:06:37 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id JAA19332 for ; Thu, 24 Jan 2002 09:06:35 -0600 Received: from PARANOIA (204.176.40.101) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 24 Jan 2002 09:55:18 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Thu, 24 Jan 2002 09:55:01 -0500 MIME-Version: 1.0 Reply-to: gmcm@hypernet.com CC: David Ascher Message-ID: <3C4FD9F5.16568.33FAE306@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Subject: [Metakit] Windows DLLs Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For the benefit of Windows users without compilers, I have placed builds of the CVS Mk4py for Python21 and Python22 at http://www.mcmillan-inc.com/dnld/Mk4py2.1-cvs.zip http://www.mcmillan-inc.com/dnld/Mk4py2.2-cvs.zip In addition to the fancy type stuff discussed yesterday, these have the fixes for wrap (which used to work only for tuples). Warning: this used to work: wrap([0,1], [property('I','x')]) although it shouldn't have. That now needs to be: wrap([(0,), (1,)], [property('I', 'x')], 1) -- Gordon http://www.mcmillan-inc.com/ From mh@pixar.com Thu Jan 24 12:50:44 2002 Received: from pixar.pixar.com (pixar.pixar.com [138.72.10.20]) by trixie.triqs.com (8.9.3/8.9.3) with ESMTP id MAA29045 for ; Thu, 24 Jan 2002 12:50:44 -0600 Received: from postal.pixar.com (postal.pixar.com [138.72.126.101]) by pixar.pixar.com (8.9.3/8.9.3) with ESMTP id KAA26612 for ; Thu, 24 Jan 2002 10:38:48 -0800 (PST) Received: from pixar.com (dreadnok [138.72.16.110]) by postal.pixar.com (8.11.1/8.11.1) with ESMTP id g0OIceJ14495 for ; Thu, 24 Jan 2002 10:38:40 -0800 (PST) Message-ID: <3C5054B7.5050107@pixar.com> Date: Thu, 24 Jan 2002 10:38:47 -0800 From: Mark Harrison User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.7) Gecko/20011221 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] cannot run more than one instance of wikit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is on linux, apache. If two instances of wikit are executed, one of them returns an error 500 with this in the log file: [error] (26)Text file busy: exec of /mh/htdocs/t1/wikit.cgi failed [error] [client 138.72.16.110] Premature end of script headers: /mh/htdocs/t1/wikit.cgi For now I'm working around it with setting the apache parameters to > MinSpareServers 1 > MaxSpareServers 1 > StartServers 1 > MaxClients 1 > MaxRequestsPerChild 100 Any ideas? Thanks, Mark --- Mark Harrison -- mh@pixar.com Pixar Animation Studios From nathan.thompson-amato@dnamerican.com Thu Jan 24 16:23:34 2002 Received: from montecarlo.DNAMERICAN.COM ([66.109.177.66]) by trixie.triqs.com (8.9.3/8.9.3) with SMTP id QAA05660 for ; Thu, 24 Jan 2002 16:23:24 -0600 Received: from [66.109.177.116] by montecarlo.DNAMERICAN.COM (NTMail 3.02.13) with ESMTP id ma067976 for ; Thu, 24 Jan 2002 17:26:03 -0500 Message-ID: <3C50867C.9020906@dnamerican.com> Date: Thu, 24 Jan 2002 17:11:08 -0500 From: Nathan Thompson-Amato Organization: D.N. American User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.6) Gecko/20011120 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Multiple databases in one file: possible? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, I've been using Metakit for a single database (one basic view with a hash table on top of it) for a while now, and it works beautifully. However, I'd like to add an entirely independent database without using a separate storage object. I've tried creating a new view with something like c4_View *otherView = new c4_View(storage->GetAs("otherData[name:I]")); but attempts to access this database cause crashes. Should I expect this to work? If not, is there some other technique I should be using? Thanks, Nathan From jcw@equi4.com Mon Jan 28 10:38:19 2002 Received: from [10.0.1.55] (ipc379e8df.dial.wxs.nl [195.121.232.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGcI830203 for ; Mon, 28 Jan 2002 10:38:18 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] metakit questions Date: Mon, 28 Jan 2002 17:34:30 +0100 Message-Id: <20020128163430.14182@triqs.com> In-Reply-To: <200201161729.AA263717176@tbots.com> References: <200201161729.AA263717176@tbots.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: peter wrote: >I have a couple of questions regarding metakit which I hope you >can answer. The questions are primarily related to larger database >files where I would require very fast operations. I wonder if >metakit is the right choice given all the performance enhancements >that have been made over the last 2 years. > >1) when calling 'commit' - is the entire datafile rewritten, or >just the part that has changed? Just the part that has changed, but this has to be qualified further: "part" being columns. With blocked views, columns are broken down in smaller chunks, which is why blocked views greatly improve commit speed in large views. >2) since metakit does not index its tables/columns , and I need >best possible performance, is there any way to pick between binary >search or some form of hashing? how does metakit compare to say >a 'b-tree on disk' type of storage form? I know I could do an >exhaustive performance test on this :-), but I'm a few weeks from >a major product release, and I just wondered about the theoretical >aspects of this. I havent found any notes on this in the >documentation. The hashing performance of MK has been, eh, hashed over quite a bit lately. I think that you will find that hashed views provide excellent access-by key performance. By layering an ordered view over a blocked view, you get something which is very similar to a B-tree (but fixed at two levels). There are still some optimizations left to do to take full advantage of locality-of- reference for non-leaf blocks (speaking in B-tree terms, not MK), but it's a start. And it has the same semantics: an ordered view, which can be efficiently traversed. Note also that c4_View::Locate() is an improved API for (binary) Search, which lets you find out how many rows match. The logic of this all is automatic: if you do a Find on the first property in a hashed view, MK will take advantage of hashing. On other Finds, it reverts to the linear approach. Same for locate, ordered, etc. >3) Does metakit have the equivalent of the SQL ...WHERE IN (1,2,3) >, and if so, how does metakit go about the looking up the records? >(I guess this relates to question 2 No. Do a sequential scan. Keep in mind that sequential scans continue to be blazingly fast (even if O(N)). >4) Does metakit now have equivalent of DELETE WHERE .... >I remember having to reverse iterate through views do delete rows. This too has not changed. Issue 3) and 4) are basically limited by the lack of an API in C++ to specify these things (how to pass a "where expression" in C++ for example, without having to implement a complete parser). I admit that I haven't given it much thought recently. -jcw From jcw@equi4.com Mon Jan 28 10:40:49 2002 Received: from [10.0.1.55] (ipc379e8df.dial.wxs.nl [195.121.232.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGem830343 for ; Mon, 28 Jan 2002 10:40:48 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] tclkit and iso8859-1 ? Date: Mon, 28 Jan 2002 17:35:05 +0100 Message-Id: <20020128163505.25706@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: >Hi again, >I try to verify the differences in encoding betwen the 2 tclkit's I use. >(the latest and some releases back). [1] >% encoding system >iso8859-1 [2] >% encoding system iso8859-1 >unknown encoding "iso8859-1" The Unicode encoding logic has been causing me a headache for ages. The good news is that in the new TclKit builds, things are starting to look better. The version which is most likely to work for you is at http://www.equi4.com/previews/tclkit.* It behaves as [1], as far as I can tell. You are right in saying that the default encoding should have been set up right away, but there seems to be a nasty chicken-and-egg issue: TclKit needs to open the MK database before it has access to its runtime files, such as encodings. For now, the above will let you set the encoding, AFAIK. On Windows, things are better - and properly set up. I am looking into Linux/Unix, to see why the standard Tcl startup logic is not getting things right. -jcw From jcw@equi4.com Mon Jan 28 10:40:51 2002 Received: from [10.0.1.55] (ipc379e8df.dial.wxs.nl [195.121.232.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGeo830347 for ; Mon, 28 Jan 2002 10:40:50 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] cannot run more than one instance of wikit? Date: Mon, 28 Jan 2002 17:36:17 +0100 Message-Id: <20020128163617.29498@triqs.com> In-Reply-To: <3C5054B7.5050107@pixar.com> References: <3C5054B7.5050107@pixar.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Harrison wrote: >This is on linux, apache. > >If two instances of wikit are executed, one of them returns >an error 500 with this in the log file: > >[error] (26)Text file busy: exec of /mh/htdocs/t1/wikit.cgi failed >[error] [client 138.72.16.110] Premature end of script headers: > /mh/htdocs/t1/wikit.cgi > >For now I'm working around it with setting the apache parameters to [...] This looks like a Unix issue. TclKit is an executable with data et the end. That data is in fact a MetaKit database, which gets opened read- only as memory-mapped file. It seems that on some Unix systems, that extra open or mmap prevents an exec on it from running a second instance. Try running things through a shell script, my CGI's usually look like: #!/bin/sh cd /myhome/data exec /myhome/bin/tclkit wikit.tkd -jcw From jcw@equi4.com Mon Jan 28 10:40:52 2002 Received: from [10.0.1.55] (ipc379e8df.dial.wxs.nl [195.121.232.223]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGep830351 for ; Mon, 28 Jan 2002 10:40:52 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] how to use hash with two keys? Date: Mon, 28 Jan 2002 17:36:18 +0100 Message-Id: <20020128163618.22596@triqs.com> In-Reply-To: <20020128154353.30922@triqs.com> References: <20020128154353.30922@triqs.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: horst@freedict.de wrote: >I try to use a hashed view with to key fields. >But the performance I get is poor. > >To see what I do, apply the patch to find.py in the examples dir > >80c80 >< r = vw.find(p1=k) >--- >> r = vw.find(p1=k, p2=k+k) >96c96 >< vwh = vw.hash(map) >--- >> vwh = vw.hash(map,2) >102c102 >< vwh = vw.hash(map) >--- >> vwh = vw.hash(map,2) >114,115c114,115 >< hash_key(n/2, 10000) >< hash_key(n+1, 10000) >--- >> hash_key(n/2, 1000000/n) >> hash_key(n+1, 1000000/n) > >What should I have done differently? The line "r = vw.find(p1=k, p2=k+k)" passes p1 and p2 as Python keyword arguments to Mk4py (which internally is a dict). That, unfortunately means that the order is not preserved. My hunch is that Mk4py then builds a MK row from it in the opposite order (p2,p1). The problem with multi-key optimized finds in MK, is that the key given must be in the exact same order as declared in the view. In this case, the Python hashing exposes this weakness. I think you should try using a list (or a tuple) instead: r = vw.find([k, k+k]) The order is now implicit from the view, and hence always right. -jcw From jtate@mi-corporation.com Mon Jan 28 10:47:38 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SGlc830686 for ; Mon, 28 Jan 2002 10:47:38 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0SGlWf07528 for ; Mon, 28 Jan 2002 11:47:32 -0500 From: "Joseph Tate" To: "MetaKit" Date: Mon, 28 Jan 2002 11:48:44 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Metakit porting issue. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I've ported Metakit to Windows CE replacing all char-centric code with two-byte characters. I've also ported most of the test suite. I am now to the hash mapping functions and have found an infinite loop. The loop happens in c4_HashViewer::LookDict when //tmapped.cpp line 18 v3.Add(p1 [_T("b93655249726e5ef4c68e45033c2e0850570e1e07")]); is called, but arises from the creation of the view in lines above. The problem is that the empty placeholder row is created with the value -1, which is fine, but when IsUnused is finally called within LookDict the value has been morphed to 255 (if changed in the debug editor, it works fine). It's taking the "empty" value -1, stored in 8 bits, and reading it as if it was stored in 16 bits. The value -1 is copied multiple times between the creation of the view and the addition of the first element, but I can't seem to find where the loss of bitlength occurs. Can anyone point me in the right direction? Thanks, Joseph From jtate@mi-corporation.com Mon Jan 28 12:34:49 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SIYn802511 for ; Mon, 28 Jan 2002 12:34:49 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0SIYgf08032 for ; Mon, 28 Jan 2002 13:34:42 -0500 From: "Joseph Tate" To: "MetaKit" Subject: RE: [Metakit] Metakit porting issue. Date: Mon, 28 Jan 2002 13:35:55 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Figured out that issue. Problem was I was using an unsigned entity in Get_8i, and unsigned -1 is 255 when you're only using one byte. Joseph -----Original Message----- From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf Of Joseph Tate Sent: Monday, January 28, 2002 11:49 AM To: MetaKit Subject: [Metakit] Metakit porting issue. I've ported Metakit to Windows CE replacing all char-centric code with two-byte characters. I've also ported most of the test suite. I am now to the hash mapping functions and have found an infinite loop. The loop happens in c4_HashViewer::LookDict when //tmapped.cpp line 18 v3.Add(p1 [_T("b93655249726e5ef4c68e45033c2e0850570e1e07")]); is called, but arises from the creation of the view in lines above. The problem is that the empty placeholder row is created with the value -1, which is fine, but when IsUnused is finally called within LookDict the value has been morphed to 255 (if changed in the debug editor, it works fine). It's taking the "empty" value -1, stored in 8 bits, and reading it as if it was stored in 16 bits. The value -1 is copied multiple times between the creation of the view and the addition of the first element, but I can't seem to find where the loss of bitlength occurs. Can anyone point me in the right direction? Thanks, Joseph _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From guenther.fischer@hrz.tu-chemnitz.de Mon Jan 28 13:26:37 2002 Received: from asterix.hrz.tu-chemnitz.de (asterix.hrz.tu-chemnitz.de [134.109.132.84]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SJQa804762 for ; Mon, 28 Jan 2002 13:26:37 -0600 Received: from pat.hrz.tu-chemnitz.de ([134.109.132.143] ident=mail) by asterix.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #1) id 16VHQJ-0002gh-00 for metakit@equi4.com; Mon, 28 Jan 2002 20:26:35 +0100 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=root) by pat.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #2) id 16VHQJ-0003JS-00 for metakit@equi4.com; Mon, 28 Jan 2002 20:26:35 +0100 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.9.3/8.9.3) with ESMTP id UAA15699 for ; Mon, 28 Jan 2002 20:26:35 +0100 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Mon, 28 Jan 2002 20:26:35 +0100 (MET) From: Guenther Fischer To: metakit Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Jpeg images with tclkit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I tried to use the Img to add jpeg to view in tclkit. I found no way to add this package always some errors. I also build a tclkit for linux from the source an the img library with the same source. I didn't get it run - is there anybody who has done this. Could you add the Img to the new tclkit or to kitten Jean Cloude? -- ~Guenther Fischer From tciuro@yahoo.com Mon Jan 28 13:49:22 2002 Received: from web20603.mail.yahoo.com (web20603.mail.yahoo.com [216.136.226.161]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g0SJnM805680 for ; Mon, 28 Jan 2002 13:49:22 -0600 Message-ID: <20020128194914.71226.qmail@web20603.mail.yahoo.com> Received: from [80.32.136.3] by web20603.mail.yahoo.com via HTTP; Mon, 28 Jan 2002 11:49:14 PST Date: Mon, 28 Jan 2002 11:49:14 -0800 (PST) From: Tito Ciuro To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] MetaKit: Little vs big endian Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all, What is the status of MetaKit regarding little vs big endian? Is this something that I can count on in the current release? (2.4.2) Has anyone tried to access the same MetaKit data file from two machines with different endieness at the same time? Issues anyone? TIA, -- Tito __________________________________________________ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com From jtate@mi-corporation.com Mon Jan 28 14:08:45 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0SK8i806393; Mon, 28 Jan 2002 14:08:44 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0SK8bf08635; Mon, 28 Jan 2002 15:08:37 -0500 From: "Joseph Tate" To: "MetaKit" , "Jean-Claude Wippler" Date: Mon, 28 Jan 2002 15:09:51 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Metakit Ported to Windows CE Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have ported Metakit verison 2.4.2 to Windows CE. Please let me know if you would like sources/projects/binary builds. I'm cleaning it up right now: making sure that with all my #ifdef statements that it still compiles under windows, etc. Joseph From jyl@best.com Mon Jan 28 23:57:05 2002 Received: from proxy4.ba.best.com (root@proxy4.ba.best.com [206.184.139.15]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0T5v5819660 for ; Mon, 28 Jan 2002 23:57:05 -0600 Received: from best.com (adsl-209-233-24-218.dsl.snfc21.pacbell.net [209.233.24.218]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id VAA04615; Mon, 28 Jan 2002 21:56:28 -0800 (PST) Message-ID: <3C564862.4080907@best.com> Date: Mon, 28 Jan 2002 22:59:46 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Tito Ciuro CC: metakit@equi4.com Subject: Re: [Metakit] MetaKit: Little vs big endian References: <20020128194914.71226.qmail@web20603.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Tito I regularly transport Metakit storages between Sparc and Intel, no issues so far. I'd be surprised if there were any, Metakit storages are supposed to be portable and architecture neutral. JYL Tito Ciuro wrote: >Hello all, > >What is the status of MetaKit regarding little vs big >endian? Is this something that I can count on in the >current release? (2.4.2) > >Has anyone tried to access the same MetaKit data file >from two machines with different endieness at the same >time? Issues anyone? > >TIA, > >-- Tito > >__________________________________________________ >Do You Yahoo!? >Great stuff seeking new owners in Yahoo! Auctions! >http://auctions.yahoo.com >_______________________________________________ >metakit mailing list - metakit@equi4.com >http://www.equi4.com/mailman/listinfo/metakit > From steve@improvision.com Tue Jan 29 04:19:38 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TAJc824227 for ; Tue, 29 Jan 2002 04:19:38 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id KAA21880 for ; Tue, 29 Jan 2002 10:17:10 GMT Received: from [192.168.1.91] ([192.168.1.91]) by mail.improvision.com (Merak 4.00.100) with ESMTP id LVC36971; Tue, 29 Jan 2002 10:19:34 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Tue, 29 Jan 2002 10:19:37 +0000 Subject: Re: [Metakit] MetaKit: Little vs big endian From: Stephen Baxter To: Jacob Levy , Tito Ciuro CC: Message-ID: In-Reply-To: <3C564862.4080907@best.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Works for me - we're using it between Mac and PC with no problems. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Jacob Levy > Date: Mon, 28 Jan 2002 22:59:46 -0800 > To: Tito Ciuro > Cc: metakit@equi4.com > Subject: Re: [Metakit] MetaKit: Little vs big endian > > Hi Tito > > I regularly transport Metakit storages between Sparc and Intel, no > issues so far. I'd be surprised if there were any, Metakit storages are > supposed to be portable and architecture neutral. > > JYL > > Tito Ciuro wrote: > >> Hello all, >> >> What is the status of MetaKit regarding little vs big >> endian? Is this something that I can count on in the >> current release? (2.4.2) >> >> Has anyone tried to access the same MetaKit data file >> from two machines with different endieness at the same >> time? Issues anyone? >> >> TIA, >> >> -- Tito >> >> __________________________________________________ >> Do You Yahoo!? >> Great stuff seeking new owners in Yahoo! Auctions! >> http://auctions.yahoo.com >> _______________________________________________ >> metakit mailing list - metakit@equi4.com >> http://www.equi4.com/mailman/listinfo/metakit >> > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jcw@equi4.com Tue Jan 29 04:33:20 2002 Received: from [10.0.1.55] (ipc379e9a0.dial.wxs.nl [195.121.233.160]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TAXJ824821 for ; Tue, 29 Jan 2002 04:33:19 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] MetaKit: Little vs big endian Date: Tue, 29 Jan 2002 11:23:39 +0100 Message-Id: <20020129102339.19092@triqs.com> In-Reply-To: <20020128194914.71226.qmail@web20603.mail.yahoo.com> References: <20020128194914.71226.qmail@web20603.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito Ciuro wrote: >What is the status of MetaKit regarding little vs big >endian? Is this something that I can count on in the >current release? (2.4.2) Yes, it works fine. With one exception, which I am chasing right now: there is an issue with load/save (i.e. seraalizing the datafile out), when the datafile was created on a machine with the opposite endianness. But for normal opens/commit, it should not cause problems anywhere. -jcw From jcw@equi4.com Tue Jan 29 04:33:22 2002 Received: from [10.0.1.55] (ipc379e9a0.dial.wxs.nl [195.121.233.160]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TAXL824825 for ; Tue, 29 Jan 2002 04:33:21 -0600 From: Jean-Claude Wippler To: MetaKit Subject: Re: [Metakit] Metakit Ported to Windows CE Date: Tue, 29 Jan 2002 11:30:20 +0100 Message-Id: <20020129103020.19246@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joseph Tate wrote: >I have ported Metakit verison 2.4.2 to Windows CE. Please let me know if >you would like sources/projects/binary builds. I'm cleaning it up right >now: making sure that with all my #ifdef statements that it still compiles >under windows, etc. Joseph, thank you very much for doing this - I apologize for not responding sooner (I had started to several times, but each time thought of some detail for this port which needed for thought). I'm impressed that you got this far and would very much like to provide your build on the website for others to download, if you permit. Also, I'd like to check a few things out w.r.t. the changes you needed to make o get things working. Ultimately, it would be great if we can merge the source code changes so WinCE support becomes part of the standard distribution. SInce I have a WinCE 2.11 PDA lying around (Cassiopeia E11 - i.e. MIPS), I could check things and try them out even. But the big Q is how to build for it: last time I looked into this, there was a free D/L from Microsoft for the MSVC (5? 6? I have both) environment. Is that the way to go - or is there a gcc option nowadays. -jcw From jcw@equi4.com Tue Jan 29 07:03:32 2002 Received: from [10.0.1.55] (ipc379e836.dial.wxs.nl [195.121.232.54]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TD3V831813 for ; Tue, 29 Jan 2002 07:03:32 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Jpeg images with tclkit Date: Tue, 29 Jan 2002 13:57:20 +0100 Message-Id: <20020129125720.21189@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther Fischer wrote: >I tried to use the Img to add jpeg to view in tclkit. I found no way >to add this package always some errors. (For non-Tcl'ers: img is an extension which lets Tk understand more graphic file formats) Here's how I got it to work: - took the img1.2.4/ copy from browsex (since I had it lying around) - did "./configure" - PATCHED the Makefile to also link in X11 (see below) (maybe it even works as is, *if* you "package require Tk" first?) (hm, no, in Linux it won't work - X11 is linked in statically) - did "make all" ("all" is needed to make jpg, png, etc) - then launched tclkit: % pwd /home/jcw/img1.2.4 % load ./libimg1.2.so % ls libjpeg/*.jpg libjpeg/: testimg.jpg testimgp.jpg testorig.jpg testprog.jpg % image create photo i -file libjpeg/testimg.jpg i % pack [button .b -image i] % And bingo... I'm seeing a red rose. The above tells me that the build and stubs logic are ok. But you may run into problems trying to embed this stuff into a scripted document. The reason is that "libimg1.2.so" depends on a few other .so's (jpeg, png, etc). The trick is a nasty but common idiom in Tcl: before the "load blah/libimg1.2.so" you must insert lines such as: catch {load blah/libjpeg.so} That will fail, since there is no tcl init setup in there - but it will nevertheless (!) have loaded the shared library (be sure to place those shared libs next to libimg1.2.so). A kludge, but effective... -jcw P.S. The necessary Makefile change is to add X libs to the last line of the snippet shown here: # Directories in which the X11 includes and libraries can be found TK_XINCLUDES = TK_XLIBSW = -L/usr/X11R6/lib -lX11 # Libraries to be included with libimg$(VERSION).so TCL_SHARED_LIBS = -L/usr/lib -ltkstub8.3 -L/usr/lib \ -ltclstub8.3${TCL_DBGX} ${TK_XLIBSW} P.P.S. Yes, adding "img" to Kitten would be a good idea - I'll keep it in mind in the next update round for it. From guenther.fischer@hrz.tu-chemnitz.de Tue Jan 29 08:11:22 2002 Received: from asterix.hrz.tu-chemnitz.de (asterix.hrz.tu-chemnitz.de [134.109.132.84]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TEBL801016 for ; Tue, 29 Jan 2002 08:11:21 -0600 Received: from pat.hrz.tu-chemnitz.de ([134.109.132.143] ident=mail) by asterix.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #1) id 16VYym-0000Rf-00 for metakit@equi4.com; Tue, 29 Jan 2002 15:11:20 +0100 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=root) by pat.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #2) id 16VYym-0008Cn-00 for metakit@equi4.com; Tue, 29 Jan 2002 15:11:20 +0100 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.9.3/8.9.3) with ESMTP id PAA31346 for ; Tue, 29 Jan 2002 15:11:20 +0100 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Tue, 29 Jan 2002 15:11:19 +0100 (MET) From: Guenther Fischer To: metakit@equi4.com Subject: Re: [Metakit] Jpeg images with tclkit In-Reply-To: <20020129125720.21189@triqs.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude, thank you very much for your answer. I hope for the future to see Img in kitten because I need Img for linux and windows. Thank you too for the answer around the iso-8859-1. The tclkit.bin and the tclkit.exe does the needed help. BTW - I had compiled tclkit, metakit etc. from your sources and the characterset problem was fixed for linux. But for now I will take the two tclkit's from the previews area. I tried your linux solution with tclkit and img1.2.4 as described and got: couldn't load file "libimg1.2.so": /afs/tu-chemnitz.de/home/urz/f/fischer/img1.2.4/libimg1.2.so: undefined symbol: log If you see the problem let me know, if not I will wait because the linux solution is only the half I need ... It is a good idea to collect the extensions for tclkit in the kitten library. Perhaps somebody can help to fill the kitten library ... ??? On Tue, 29 Jan 2002, Jean-Claude Wippler wrote: > Guenther Fischer wrote: > > >I tried to use the Img to add jpeg to view in tclkit. I found no way > >to add this package always some errors. > > (For non-Tcl'ers: img is an extension which lets Tk understand more > graphic file formats) > > Here's how I got it to work: > - took the img1.2.4/ copy from browsex (since I had it lying around) > - did "./configure" > - PATCHED the Makefile to also link in X11 (see below) > (maybe it even works as is, *if* you "package require Tk" first?) > (hm, no, in Linux it won't work - X11 is linked in statically) > - did "make all" ("all" is needed to make jpg, png, etc) > - then launched tclkit: > % pwd > /home/jcw/img1.2.4 > % load ./libimg1.2.so > % ls libjpeg/*.jpg > libjpeg/: > testimg.jpg testimgp.jpg testorig.jpg testprog.jpg > % image create photo i -file libjpeg/testimg.jpg > i > % pack [button .b -image i] > % > > And bingo... I'm seeing a red rose. > > The above tells me that the build and stubs logic are ok. But you may > run into problems trying to embed this stuff into a scripted document. > The reason is that "libimg1.2.so" depends on a few other .so's (jpeg, > png, etc). The trick is a nasty but common idiom in Tcl: before the > "load blah/libimg1.2.so" you must insert lines such as: > catch {load blah/libjpeg.so} > That will fail, since there is no tcl init setup in there - but it will > nevertheless (!) have loaded the shared library (be sure to place those > shared libs next to libimg1.2.so). A kludge, but effective... > > -jcw > > P.S. The necessary Makefile change is to add X libs to the last line of > the snippet shown here: > > # Directories in which the X11 includes and libraries can be found > TK_XINCLUDES = > TK_XLIBSW = -L/usr/X11R6/lib -lX11 > > # Libraries to be included with libimg$(VERSION).so > TCL_SHARED_LIBS = -L/usr/lib -ltkstub8.3 -L/usr/lib \ > -ltclstub8.3${TCL_DBGX} ${TK_XLIBSW} > > P.P.S. Yes, adding "img" to Kitten would be a good idea - I'll keep it > in mind in the next update round for it. > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > -- ~Guenther Fischer From tciuro@yahoo.com Tue Jan 29 09:41:32 2002 Received: from web20609.mail.yahoo.com (web20609.mail.yahoo.com [216.136.226.167]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g0TFfW803816 for ; Tue, 29 Jan 2002 09:41:32 -0600 Message-ID: <20020129154127.76964.qmail@web20609.mail.yahoo.com> Received: from [80.26.55.39] by web20609.mail.yahoo.com via HTTP; Tue, 29 Jan 2002 07:41:27 PST Date: Tue, 29 Jan 2002 07:41:27 -0800 (PST) From: Tito Ciuro Subject: Re: [Metakit] MetaKit: Little vs big endian To: Jean-Claude Wippler , metakit@equi4.com In-Reply-To: <20020129102339.19092@triqs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Jean-Claude, This is great news. Serializing is something that I use in order to compact MK data files (since I'm opening the file in commit-extend mode.) Just being curious... is this open issue relatively easy to solve? Thank you, -- Tito --- Jean-Claude Wippler wrote: > Tito Ciuro wrote: > > >What is the status of MetaKit regarding little vs > big > >endian? Is this something that I can count on in > the > >current release? (2.4.2) > > Yes, it works fine. > > With one exception, which I am chasing right now: > there is an issue with > load/save (i.e. seraalizing the datafile out), when > the datafile was > created on a machine with the opposite endianness. > > But for normal opens/commit, it should not cause > problems anywhere. > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com From tciuro@yahoo.com Tue Jan 29 09:43:38 2002 Received: from web20606.mail.yahoo.com (web20606.mail.yahoo.com [216.136.226.164]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g0TFhc803951 for ; Tue, 29 Jan 2002 09:43:38 -0600 Message-ID: <20020129154337.93182.qmail@web20606.mail.yahoo.com> Received: from [80.26.55.39] by web20606.mail.yahoo.com via HTTP; Tue, 29 Jan 2002 07:43:37 PST Date: Tue, 29 Jan 2002 07:43:37 -0800 (PST) From: Tito Ciuro Subject: Re: [Metakit] MetaKit: Little vs big endian To: Jacob Levy Cc: metakit@equi4.com In-Reply-To: <3C564862.4080907@best.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jacob, I'm aware that MK is multi-platform, but sometimes there are some issues that may complicate things a bit... :) It's great to hear that you haven't had major issues. Thanks for your reply, -- Tito --- Jacob Levy wrote: > Hi Tito > > I regularly transport Metakit storages between Sparc > and Intel, no > issues so far. I'd be surprised if there were any, > Metakit storages are > supposed to be portable and architecture neutral. > > JYL > > Tito Ciuro wrote: > > >Hello all, > > > >What is the status of MetaKit regarding little vs > big > >endian? Is this something that I can count on in > the > >current release? (2.4.2) > > > >Has anyone tried to access the same MetaKit data > file > >from two machines with different endieness at the > same > >time? Issues anyone? > > > >TIA, > > > >-- Tito > > > >__________________________________________________ > >Do You Yahoo!? > >Great stuff seeking new owners in Yahoo! Auctions! > >http://auctions.yahoo.com > >_______________________________________________ > >metakit mailing list - metakit@equi4.com > >http://www.equi4.com/mailman/listinfo/metakit > > > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com From jtate@mi-corporation.com Tue Jan 29 10:49:47 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0TGnl807316; Tue, 29 Jan 2002 10:49:47 -0600 Received: from jtatework ([216.21.174.97]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g0TGnff13805; Tue, 29 Jan 2002 11:49:41 -0500 From: "Joseph Tate" To: "Jean-Claude Wippler" , "MetaKit" Subject: RE: [Metakit] Metakit Ported to Windows CE Date: Tue, 29 Jan 2002 11:50:55 -0500 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0010_01C1A8BB.3585B3B0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <20020129103020.19246@triqs.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_0010_01C1A8BB.3585B3B0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit > Joseph, thank you very much for doing this - I apologize for not > responding sooner (I had started to several times, but each time thought > of some detail for this port which needed for thought). Not a problem. I feel like I have a pretty good feel now for the inner workings of Metakit. > I'm impressed that you got this far and would very much like to provide > your build on the website for others to download, if you permit. I'll package it up and send it to you. > Also, I'd like to check a few things out w.r.t. the changes you needed to > make o get things working. Ultimately, it would be great if we can merge > the source code changes so WinCE support becomes part of the standard > distribution. Well, I've ported it with that in mind. My metakit tree still compiles and passes the test suite under win2k. It should patch ok. I decided to use Metakit so that I could use the same underlying embedded database structure in an application that I'm porting from Pocket PC to Windows. The Pocket PC architecture provides a limited functionality database system, but it's ram based and subject to corruption if the device crashes or whatever. Plus the database is not available on Windows, so we either needed to have two different database backends, or port Metakit to CE. We opted for the added functionality provided by porting Metakit to CE. The biggest thing with portability (and this may require a change to file format if the datafiles are to be moved accross platforms) is the treatment of strings as two-byte character arrays instead of single byte chars. What I did was created a wince.h header file that defined char as a unsigned short. Then elsewhere in the code where an array of single bytes was needed I used t4_byte and a new type, t4_sbyte, for unsigned and signed single bytes (respectively). So in the code anywhere that had a variable definition of char * mmm that wasn't referring to strings was changed to either t4_byte* or t4_sbyte*. I also #def'd a lot of the str maninip functions to their wide char counterparts. wince.h is attached so that you can see what I've done with it. I'll also include header.h so you can see what changes I made there. Eventually I'd like to see all of the string classes changed so that they're based on CHAR and not char. That way a char remains as defined in the C and C++ specs as a single signed byte. I'm sure it'd be pretty simple to change. First priority was getting it to work, then making it pretty. The other thing I did was surround almost every "string" with the _T("string") construct which casts it to a wide char array. The only place where this was not done was with memo fields. Now, about the string problem in the file format. I'm not familiar with the file format at all. I do know that I had to expressly change the file length when I got to the textend.cpp test: #include "regress.h" #ifndef q4_WCE const int kSize1 = 41; const int kSize2 = 85; #else const int kSize1 = 48; const int kSize2 = 99; #endif An extra 7 bytes per entry. I've not delved into why that is, but I'm pretty sure it has to do with the the multi-byte nature of strings on CE. How can I make this play well? I'm worried that a file created on a non-CE system will be read in with two bytes per character, munging the data. And vice versa, a file created on CE will be read in with null characters between each and every character on non-CE systems. Is there some flag we can set in the data file header? Some kind of bytes per character indicator? > SInce I have a WinCE 2.11 PDA lying around (Cassiopeia E11 - i.e. MIPS), > I could check things and try them out even. But the big Q is how to > build for it: last time I looked into this, there was a free D/L from > Microsoft for the MSVC (5? 6? I have both) environment. Is that the way > to go - or is there a gcc option nowadays. There is a free download from Microsoft called embedded Visual C++. It's available for download from http://www.microsoft.com/mobile/downloads/emvt30.asp . Take note of the CDKey on the page. It uses a unique project workspace format, so you can't open it with Vis Studio. I've not tested the port on anything except the x86 emulator, so your help in testing/debugging would be much appreciated. Well, that's enough for one e-mail. I'll package up my metakit tree and send it in another e-mail. Joseph ------=_NextPart_000_0010_01C1A8BB.3585B3B0 Content-Type: application/octet-stream; name="header.h" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="header.h" // header.h -- // $Id: header.h,v 1.6 2001/12/11 01:59:51 wcvs Exp $ // This is part of MetaKit, the homepage is = http://www.equi4.com/metakit/ /** @file * The internal header included in all source files */ #ifndef __HEADER_H__ #define __HEADER_H__ /////////////////////////////////////////////////////////////////////////= //// #include "config.h" /////////////////////////////////////////////////////////////////////////= //// // A number of preprocessor options are used in the source code // // q4_DOS MS-DOS real-mode OS // q4_MAC Apple Macintosh OS // q4_UNIX Unix, any flavor // q4_VMS DEC OpenVMS OS // q4_WIN Microsoft Windows OS, any flavor // q4_WIN32 Microsoft Windows OS, 32-bit // // q4_MFC Microsoft MFC framework // q4_STD Standard STL version // q4_UNIV Universal version // // q4_BOOL compiler supports bool datatype // q4_CHECK enable assertion checks // q4_FIX manual header fix (see above) // q4_INLINE enable inline expansion // q4_KITDLL compile as DLL (shared library) // q4_MULTI compile for multi-threading // q4_NOLIB do not add automatic lib linkage (MSVC5) // q4_NO_NS don't use namespaces for STL // q4_OK assume all software is perfect // q4_STRICT do not disable any compiler warnings // q4_TINY small version, no floating point // /////////////////////////////////////////////////////////////////////////= //// #define __K4CONF_H__ // skip section in "mk4.h", since we use = "header.h" // if neither MFC nor STD are specified, default to Universal = version #if !q4_MFC && !q4_STD && !defined (q4_UNIV) #define q4_UNIV 1 #endif #if defined (_WIN32_WCE) // Microsoft Windows CE #include "wince.h" #else //#define _T(x) x //Conversion to wchar_t* disabled. #endif /////////////////////////////////////////////////////////////////////////= //// // You can either use '#define q4_xxx 1' to flag the choice of an OS, or // use a '#define d4_OS_H "file.h"' to force inclusion of a header = later. #if defined (__MINGW32__) #define d4_OS_H "win.h" #elif defined (MSDOS) && defined (__GNUC__) #define q4_DOS 1 #elif defined(unix) || defined(__unix__) || defined(__GNUC__) || = defined(_AIX) #define q4_UNIX 1 #elif defined (__VMS) #define q4_VMS 1 #elif defined (macintosh) #define q4_MAC 1 #elif !defined (d4_OS_H) #define d4_OS_H "win.h" #endif /////////////////////////////////////////////////////////////////////////= //// // Use '#define q4_xxx 1' to flag the choice of a CPU. #if defined (_M_I86) || defined (_M_IX86) || defined (i386) #define q4_I86 1 #if defined (_M_I86SM) #define q4_TINY 1 #endif #elif defined (__powerc) #define q4_PPC 1 #elif defined (__alpha) #define q4_AXP 1 #define q4_LONG64 1 #elif defined (__VMS) #define q4_VAX 1 #else #define q4_M68K 1 #endif /////////////////////////////////////////////////////////////////////////= //// // Use '#define q4_xxx 1' to flag the choice of an IDE, and optionally = also // add '#include "file.h"' to force inclusion of a header file right = here. #if defined (__BORLANDC__) // Borland C++ #include "borc.h" #elif defined (__DECCXX) // DEC C++ #define q4_DECC 1 #elif defined (__GNUC__) // GNU C++ #include "gnuc.h" #elif defined (__MWERKS__) // Metrowerks CodeWarrior = C++ #include "mwcw.h" #elif defined (_MSC_VER) // Microsoft Visual C++ #include "msvc.h" #elif defined (__SC__) // Symantec C++ #define q4_SYMC 1 #elif defined (__WATCOMC__) // Watcom C++ #define q4_WATC 1 #endif /////////////////////////////////////////////////////////////////////////= //// // Some of the options take precedence over others #if !q4_BOOL && !q4_STD // define a bool datatype #define false 0 #define true 1 #define bool int #endif #if !q4_CHECK // disable assertions #undef d4_assert #define d4_dbgdef(x) #define d4_assert(x) #endif #if q4_NO_NS // don't use namespaces #define d4_std #else #define d4_std std #endif #if HAVE_MEMMOVE #define d4_memmove(d,s,n) memmove(d,s,n) #elif HAVE_BCOPY #define d4_memmove(d,s,n) bcopy(s,d,n) #else #define d4_memmove f4_memmove extern void f4_memmove(void* d, const void* s, int n); #endif #ifndef q4_WCE typedef unsigned char t4_byte; // create typedefs for t4_byte, etc. typedef char t4_sbyte; // Need both unsigned and signed bytes #endif //defined in wince.h #if SIZEOF_LONG =3D=3D 8 typedef int t4_i32; // longs are 64b, so int must be 32b #else typedef long t4_i32; // longs aren't 64b, so they are 32b #endif /////////////////////////////////////////////////////////////////////////= //// // Include header files which contain additional os/cpu/ide/fw specifics #ifdef d4_OS_H // operating system dependencies #include d4_OS_H #endif /////////////////////////////////////////////////////////////////////////= //// // Several defines should always be set #ifndef d4_assert // assertion macro #include #define d4_assert assert #endif #ifndef d4_dbgdef // conditionally compiled #ifdef NDEBUG #define d4_dbgdef(x) #else #define d4_dbgdef(x) x #endif #endif #ifndef d4_new // heap allocator #define d4_new new #endif #ifndef d4_reentrant // thread-local storage #define d4_reentrant #endif /////////////////////////////////////////////////////////////////////////= //// // Public definitions, plus a few more framework-specific ones #include "mk4.h" #if q4_MFC #include "mfc.h" #elif q4_STD #include "std.h" #elif q4_UNIV #include "univ.h" #endif #ifdef _MSC_VER #pragma warning(disable: 4100 4127 4135 4244 4511 4512 4514) #endif #include /////////////////////////////////////////////////////////////////////////= //// // Report unexpected combinations of settings #if !q4_FIX #if (q4_DOS+q4_MAC+q4_UNIX+q4_VMS+q4_WIN) !=3D 1 #error Exactly one operating system should have been defined #endif #if (q4_MFC+q4_STD+q4_UNIV) !=3D 1 #error Exactly one container library should have been defined #endif #endif =20 /////////////////////////////////////////////////////////////////////////= //// #endif ------=_NextPart_000_0010_01C1A8BB.3585B3B0 Content-Type: application/octet-stream; name="wince.h" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="wince.h" #ifndef __WINCE_H__INCLUDED__ #define __WINCE_H__INCLUDED__ //Moved from header.h typedef unsigned char t4_byte; // create typedefs for t4_byte, etc. typedef char t4_sbyte; // Need both unsigned and signed bytes //Definitions needed elsewhere #include "tchar.h" #include #define q4_CHECK 1 #define d4_assert(x) ASSERT(x) #define d4_dbgdef(x) x //Set up the char handling #defines #define _CHAR_DEFINED 1 #define char unsigned short #define sprintf swprintf #define fprintf fwprintf #define printf wprintf #define strcspn wcscspn #define strchr wcschr #define strlen wcslen #define strcmp wcscmp #define stricmp _wcsicmp #define strrchr wcsrchr #define strpbrk wcspbrk #define strstr wcsstr #define strspn wcsspn #define _strdup _wcsdup #define strcpy wcscpy #define strtol wcstol //file handling routines #define fopen _wfopen #define puts _putws #define fputs fputws #endif // __WINCE_H__INCLUDED__ ------=_NextPart_000_0010_01C1A8BB.3585B3B0-- From jrichards@starband.net Thu Jan 31 08:22:19 2002 Received: from C9Mailgw07.amadis.com ([216.163.188.203]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0VEMJ829661 for ; Thu, 31 Jan 2002 08:22:19 -0600 Received: from c9service11.amadis.com (10.9.0.1) by C9Mailgw07.amadis.com (NPlex 5.5.029) id 3C508FA10005940A for metakit@equi4.com; Thu, 31 Jan 2002 06:18:55 -0800 Received: from localhost (148.63.243.140) by c9service11.amadis.com (NPlex 5.5.042) id 3C4292B200079AFD for metakit@equi4.com; Thu, 31 Jan 2002 06:18:55 -0800 Date: Thu, 31 Jan 2002 09:12:46 -0500 Mime-Version: 1.0 (Apple Message framework v472) Content-Type: text/plain; charset=US-ASCII; format=flowed From: Jim Richards To: metakit@equi4.com Content-Transfer-Encoding: 7bit Message-Id: <9983A778-1654-11D6-A822-000393085C6C@starband.net> X-Mailer: Apple Mail (2.472) Subject: [Metakit] Install MetaKit on OS X Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi All; I am looking for some guidance on installing MetaKit on OS X 10.1. I have downloaded the binary files libmk4.a.gz and libmk4.dylib.gz as well as metakit.py. I decompressed both lib files, moved them to $PYTHONHOME/lib and renamed them Mk4py.a and Mk4py.dylib. I can import metakit but if I try print metakit.version all I get is an error as follows: AttributeError: 'module' object has no attribute 'version' I am using Python 2.2 Thanks Jim Richards From lvirden@cas.org Thu Jan 31 08:50:29 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g0VEoT830835 for ; Thu, 31 Jan 2002 08:50:29 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g0VEoM224359 for ; Thu, 31 Jan 2002 09:50:22 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g0VEoJk21037; Thu, 31 Jan 2002 09:50:19 -0500 (EST) Date: Thu, 31 Jan 2002 09:50:19 -0500 (EST) From: "Larry W. Virden" Message-Id: <20020131095019.AAB21013@cas.org> Subject: Re: [Metakit] Jpeg images with tclkit In-Reply-To: Your message of Tue, 29 Jan 2002 15:11:19 +0100 (MET) To: metakit@equi4.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: From: Guenther Fischer > I tried your linux solution with tclkit and img1.2.4 as described and > got: > couldn't load file "libimg1.2.so": > /afs/tu-chemnitz.de/home/urz/f/fischer/img1.2.4/libimg1.2.so: undefined > symbol: log > > If you see the problem let me know, if not I will wait because the linux > solution is only the half I need ... Add a -lm as the last of the series of libraries used when creating libimg... -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Fri Feb 1 10:50:21 2002 Received: from [10.0.1.55] (ipc379eb91.dial.wxs.nl [195.121.235.145]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11GoK809824; Fri, 1 Feb 2002 10:50:20 -0600 From: Jean-Claude Wippler To: metakit , announce Date: Fri, 1 Feb 2002 17:47:51 +0100 Message-Id: <20020201164751.4817@texas.triqs.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There is a new release of MetaKit, "the database that fits in the palm of your hand". Some details about this 2.4.3 release: - sources plus builds for Windows, Linux, and Mac are on the web - various bug fixes, see http://www.equi4.com/metakit/CHANGES - a cross-platform byte-order serialization bug has been fixed - the Python builds are for Python 2.2 (several improvements by GM) - the Tcl builds should run on any Tcl installation >= 8.1 - www.equi4.com now runs on a new server, it appears to be stable The following problems remain, as far as I know: - conversion from 2.0 to 2.4 has been reported to fail, workaround is to serialize the old file to a new one, and then use that - one report of excessive file growth (after doing lots of commits?) A note on where MK stands and where it's going: - I will only build for the above 3 platforms from now on (but of course as many platforms as possible remain supported) - release 2.4.3 is solid, future 2.4.x releases are for bug fixes - one day soon, I'll go to 2.5.x for adding new features to MK See the home page for all further details and downloads: http://www.equi4.com/metakit/ Happy programming :) Regards, Jean-Claude From jgodfrey@optinest.com Fri Feb 1 15:32:53 2002 Received: from linux2.optinest.com (node-20.65.210.128.in-addr.arpa [65.210.128.20] (may be forged)) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11LWq001493 for ; Fri, 1 Feb 2002 15:32:52 -0600 Received: from nt40ld (nt4-0ld [65.210.128.93]) by linux2.optinest.com (8.11.3/8.11.3) with SMTP id g11KZVi08878 for ; Fri, 1 Feb 2002 14:35:49 -0600 From: "Jeff Godfrey" To: Date: Fri, 1 Feb 2002 14:46:48 -0600 Message-ID: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_005F_01C1AB2F.46E4B1D0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] metakit and mk4tcl newbie Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_005F_01C1AB2F.46E4B1D0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Hi, I have just started using "metakit" through "mk4tcl". My question is about the "mk::file commit" command and when it should be used. I am writing a simple client server app where the clients will request database activities (retrieve, update, store data) and the server will perform the tasks by accessing a metakit database. I assume that in the event of a problem (hardware failure, server crash, bug, etc) any data that has not been stored using "mk::file commit" will be lost? If this is true, is there any reason I would not want to commit every change immediately after making it (bad form, performance, ??)? Basically I am thinking about doing the following in the server code: - Start server - Open metakit database file - make change - commit change - make change - commit change - make change - commit change - Close metakit database file upon shutting down the server Does this seem logical, or is there a problem with this simple design? Thanks for any input. Jeff Godfrey Optimation USA (800) 523-4744 jgodfrey@optinest.com ------=_NextPart_000_005F_01C1AB2F.46E4B1D0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi,

I have just started using "metakit"=20 through "mk4tcl".  My question is about the "mk::file = commit"=20 command and when it should be used.  I am writing a simple client = server=20 app where the clients will request database activities (retrieve, = update, store=20 data) and the server will perform the tasks by accessing a metakit=20 database.  I assume that in the event of a problem (hardware = failure,=20 server crash, bug, etc) any data that has not been stored using = "mk::file=20 commit" will be lost?  If this is true, is there any reason I=20 would not want to commit every change immediately after making it = (bad=20 form, performance, ??)?
 
Basically I am=20 thinking about doing the following in the server=20 code:
 
- Start=20 server
- Open metakit=20 database file
 
- make=20 change
- commit=20 change
 
- make=20 change
- commit=20 change
 
- make=20 change
- commit=20 change
 
- Close=20 metakit database file upon shutting down the = server
 
Does this seem=20 logical, or is there a problem with this simple design? =20
 
Thanks for any=20 input.
 
Jeff Godfrey
Optimation USA
(800)=20 523-4744
jgodfrey@optinest.com =

 

 
------=_NextPart_000_005F_01C1AB2F.46E4B1D0-- From jcw@equi4.com Fri Feb 1 15:55:28 2002 Received: from [10.0.1.55] (ipc379ed7a.dial.wxs.nl [195.121.237.122]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11LtR002433 for ; Fri, 1 Feb 2002 15:55:27 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] metakit and mk4tcl newbie Date: Fri, 1 Feb 2002 22:53:08 +0100 Message-Id: <20020201215308.26827@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeff Godfrey wrote: >I assume that in the event of a problem >(hardware failure, server crash, bug, etc) any data that has not been stored >using "mk::file commit" will be lost? Correct. >If this is true, is there any reason >I would not want to commit every change immediately after making it (bad >form, performance, ??)? Performance. But don't worry about it until you need to. You might find that the performance you need is there no matter how often you commit. >Basically I am thinking about doing the following in the server code: > >- Start server >- Open metakit database file > >- make change >- commit change > >- make change >- commit change > >- make change >- commit change > >- Close metakit database file upon shutting down the server > >Does this seem logical, or is there a problem with this simple design? Looks fine. There can only be one process opening the file. -jcw From genec@goldsmithsinc.com Fri Feb 1 16:01:47 2002 Received: from mail.goldsmithsinc.com ([64.66.89.67]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11M1j002725 for ; Fri, 1 Feb 2002 16:01:46 -0600 Received: from genec (genec [10.1.1.100]) by mail.goldsmithsinc.com (Switch-2.0.0/Switch-2.0.0) with SMTP id g11Lphq14386 for ; Fri, 1 Feb 2002 15:51:43 -0600 Message-ID: <00df01c1ab6b$82bbca40$6401010a@genec> From: "Gene Christian" To: "metakit" Date: Fri, 1 Feb 2002 15:57:58 -0600 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00DC_01C1AB39.38152540" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Subject: [Metakit] 2.4.3 and python 2.1? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. ------=_NextPart_000_00DC_01C1AB39.38152540 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Is it still possible to build version 2.4.3 for python 2.1 or are some = of the changes specific to the 2.2 version? If it is possible, has = anybody done it? On a related note, would the MS Visual C++ 6.0 standard edition be the = right choice for compiling python extensions? I feel bad asking about = Windows binaries, and would rather be able to produce them myself. The = standard edition isn't that expensive (under $100.00), but if it takes = the professional edition then I'll have to skip it. Thanks! Gene Christian ------=_NextPart_000_00DC_01C1AB39.38152540 Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Is it still possible to build version 2.4.3 for = python 2.1 or=20 are some of the changes specific to the 2.2 version?  If it is possible, has anybody done it?
 
On a related note, would the MS Visual C++ 6.0 = standard=20 edition be the right choice for compiling python extensions?  I = feel bad=20 asking about Windows binaries, and would rather be able to produce them=20 myself.  The standard edition isn't that expensive (under $100.00), = but if=20 it takes the professional edition then I'll have to skip = it.
 
Thanks!
 
Gene Christian
------=_NextPart_000_00DC_01C1AB39.38152540-- From jcw@equi4.com Fri Feb 1 16:10:53 2002 Received: from [10.0.1.55] (ipc379e99a.dial.wxs.nl [195.121.233.154]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g11MAq003110 for ; Fri, 1 Feb 2002 16:10:53 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] 2.4.3 and python 2.1? Date: Fri, 1 Feb 2002 23:08:30 +0100 Message-Id: <20020201220830.30136@triqs.com> In-Reply-To: <00df01c1ab6b$82bbca40$6401010a@genec> References: <00df01c1ab6b$82bbca40$6401010a@genec> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Gene Christian wrote: >Is it still possible to build version 2.4.3 for python 2.1 or are some of >the changes specific to the 2.2 version? If it is possible, has anybody >done it? No problem at all. I don't have a build for you, though :( The easy way out is to stick to 2.4.2 ... >On a related note, would the MS Visual C++ 6.0 standard edition be the >right choice for compiling python extensions? I feel bad asking about >Windows binaries, and would rather be able to produce them myself. The >standard edition isn't that expensive (under $100.00), but if it takes >the professional edition then I'll have to skip it. I'm not sure how "crippled" the std ed is. I build with 6 all the time. having said that: with 2.4.2, I also built with MinGW. Even cross platform, i.e. from Linux generating Win binaries. One suggestion could be to give MinGW a shot, it's good stuff. -jcw From juneaftn@hananet.net Sat Feb 2 01:52:40 2002 Received: from mo02.hananet.net ([211.202.13.165]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g127qd011353 for ; Sat, 2 Feb 2002 01:52:39 -0600 Received: from [211.201.213.181] by (Terrace internet messaging server 3.0 (for Hananet)) with ESMTP id 2002020216:52:32:301091.303.253 for ; Sat, 02 Feb 2002 16:52:32 +0900 (KST) Message-ID: <008001c1abbe$74bd8d40$b5d5c9d3@hananet.net> From: "Changjune Kim" To: Date: Sat, 2 Feb 2002 16:51:42 +0900 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] MetaKit 2.4.3 Storage Locking Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, First of all, thank you for this wonderful software. but, I have some problem here. As I read in the documentation, it says MetaKit has a simple locking model for "Storage." When opening(or creating) a file you can choose read-only, or read-write mode. It doesn't seem to work on win32 (esp. Win 98SE). Isn't it suppposed to raise an error or block when you try to open the same file with read-write mode again in a different process? Thanks in advance. June From dgporter@erols.com Sat Feb 2 19:09:39 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1319c017614 for ; Sat, 2 Feb 2002 19:09:38 -0600 Received: from 66-44-7-242.s2020.apx1.lnh.md.dialup.rcn.com ([66.44.7.242] helo=there) by smtp01.mrf.mail.rcn.net with smtp (Exim 3.33 #10) id 16XB9w-0000n6-00 for metakit@equi4.com; Sat, 02 Feb 2002 20:09:33 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Don Porter Reply-To: dgporter@erols.com To: metakit@equi4.com Date: Sat, 2 Feb 2002 20:07:58 -0500 X-Mailer: KMail [version 1.3.1] References: <20020201164751.4817@texas.triqs.com> In-Reply-To: <20020201164751.4817@texas.triqs.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: Subject: [Metakit] Mk4Tcl concurrency Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, The 2.4.3 release seems like a good time to give Metakit another try. First let me contribute something. If others are using the very latest HEAD version of Tcl from CVS like I am, they should add the flag -DUSE_NON_CONST to the CXX_SWITCHES_TCL definition in the Metakit Makefile in order to get Mk4tcl to compile. I apologize in advance if I'm asking newbie questions, but it appears to me that most of the documentation and FAQs I find are for version 1.8.5 of Metakit, so I don't trust them to have any relevance to the current release. I'll be using Metakit through its Tcl binding. The documentation suggests that commands like [mk::select] offer only a synchronous, blocking interface. I can imagine that this command might take a perceptible length of time to complete, while blocking screen redraws of Tk, or service of network connections. Am I correct that the only solution available is to use multiple threads? Assuming so, it seems it would be a straightforward matter to write a Tcl package that wraps around Mk4Tcl, and makes use of the Tcl Thread package to create asynchronous callback forms of the Metakit commands. I think that would be preferred to pushing thread management issues onto all users of Mk4Tcl. Has this already been done? If not, are others interested in this idea, or does Mk4tcl offer other solutions for this problem that I don't know about. Also, which Mk4Tcl commands are the ones where blocking time is a problem? -- | Don Porter dgporter@erols.com | | "Some days you just can't get rid of a bomb!" | | -- Adam West as BATMAN | |______________________________________________________________________| From gmcm@hypernet.com Sat Feb 2 20:13:45 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g132Dj018060 for ; Sat, 2 Feb 2002 20:13:45 -0600 Received: from PARANOIA (204.176.40.97) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Sat, 2 Feb 2002 21:13:46 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Sat, 2 Feb 2002 21:13:55 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] Mk4Tcl concurrency Reply-to: gmcm@hypernet.com Message-ID: <3C5C5693.32590.64C1AB2A@localhost> Priority: normal References: <20020201164751.4817@texas.triqs.com> In-reply-to: X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 2 Feb 2002 at 20:07, Don Porter wrote: > I'll be using Metakit through its Tcl binding. The > documentation suggests that commands like > [mk::select] offer only a synchronous, blocking > interface. I can imagine that this command might > take a perceptible length of time to complete, while > blocking screen redraws of Tk, or service of network > connections. Am I correct that the only solution > available is to use multiple threads? Can't speak to Mk4tcl, but as an Mk4py user, I would say that for a typical GUI app: - you don't have to think about if your db is in the small Ks of rows - it require some thought in the <100K range - after that, you might have to consider your algorithms seriously. YMMV, but I've never had to dedicate a thread to Mk yet. -- Gordon http://www.mcmillan-inc.com/ From mark@markroseman.com Sun Feb 3 16:03:25 2002 Received: from admin.cgocable.net (admin.cgocable.net [24.226.1.21]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g13M3Pa01410 for ; Sun, 3 Feb 2002 16:03:25 -0600 Received: from eyak-can4.markroseman.com (d141-164-68.home.cgocable.net [24.141.164.68]) by admin.cgocable.net (8.10.0/8.9.3) with ESMTP id g13IKmn17166 for ; Sun, 3 Feb 2002 13:22:15 -0500 (EST) Message-Id: <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> X-Sender: X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Sun, 03 Feb 2002 13:15:25 -0500 To: metakit@equi4.com From: Mark Roseman Subject: Re: [Metakit] Mk4Tcl concurrency In-Reply-To: References: <20020201164751.4817@texas.triqs.com> <20020201164751.4817@texas.triqs.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: As Gordon suggested, for most small-ish datasets, the time to complete most any operation in Metakit is very quick. For the odd time when you are doing operations that will cause issues, yes, running in a thread or another process may be useful. However, I'd strongly suggest not dealing with it until the point you know its an issue (whether through practice, or experimentation ahead of time). As an example, our web-based app runs under AOLserver, and we keep a lot of central data held in Metakit databases. The multiple AOLserver threads make socket connections to a separate single-threaded process running the database, so in effect requests have to wait their turn to be processed. Despite some fairly substantial queries and reasonable numbers of simultaneous users, we've never had issues with hugely perceptible delays on the client side. Mark Roseman http://www.markroseman.com From jyl@best.com Sun Feb 3 16:57:47 2002 Received: from sj1-3-4-18.securesites.net (sj1-3-4-18.securesites.net [192.220.127.211]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g13Mvla02086 for ; Sun, 3 Feb 2002 16:57:47 -0600 Received: (qmail 19362 invoked by uid 19667); 3 Feb 2002 17:31:02 -0000 Received: from unknown (HELO best.com) ([209.233.24.218]) (envelope-sender ) by 192.220.76.211 (qmail-ldap-1.03) with SMTP for ; 3 Feb 2002 17:31:02 -0000 Message-ID: <3C5D8360.3020506@best.com> Date: Sun, 03 Feb 2002 10:37:20 -0800 From: Jacob Levy User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] [ANNOUNCE] e4Graph 1.0a5 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: WHERE TO GET: ============= File releases: http://sourceforge.net/projects/e4graph/ Documentation: http://e4graph.sourceforge.net/ Change log: http://e4graph.sourceforge.net/changes.txt WHAT IS IT: =========== e4Graph is a C++ and Tcl library that allows programs to store graph-like data persistently and to access and manipulate that data efficiently. With e4Graph, you can arrange your data in the most natural form that reflects the relationships between its parts, rather than having to force it into a table-like format. The e4Graph library also allows you to concentrate on the relationships you want to represent, and not on how to store them in a database. You can modify data items, and add and remove connections and relationships between pieces of data on the fly. e4Graph allows you to represent an unlimited number of different connections between pieces of data, and your program can selectively manipulate the data according to the relationships it cares about, not having to know about other connections represented in the data set. CHANGES: ======== For all changes, see: http://e4graph.sourceforge.net/changes.txt Major changes: Markers are no longer supported as a storage entity; instead, each storage has a distinguished root node. Nodes and vertices are no longer explicitly deleted from a storage; instead, they are detached from their containing entity or entities, and when the user program no longer has a reference to the detached node or vertex, the resources associated with this entity are automatically reclaimed. DEPENDENCIES: ============= This release requires Metakit 2.4.3, Tcl 8.3 and optionally Expat 1.95.2. From dgporter@erols.com Sun Feb 3 17:45:28 2002 Received: from smtp03.mrf.mail.rcn.net (smtp03.mrf.mail.rcn.net [207.172.4.62]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g13NjRa02675 for ; Sun, 3 Feb 2002 17:45:27 -0600 Received: from 66-44-18-88.s2120.apx2.lnh.md.dialup.rcn.com ([66.44.18.88] helo=there) by smtp03.mrf.mail.rcn.net with smtp (Exim 3.33 #10) id 16XS4o-0000LE-00 for metakit@equi4.com; Sun, 03 Feb 2002 14:13:23 -0500 Content-Type: text/plain; charset="iso-8859-1" From: Don Porter Reply-To: dgporter@erols.com To: metakit@equi4.com Date: Sun, 3 Feb 2002 14:11:51 -0500 X-Mailer: KMail [version 1.3.1] References: <20020201164751.4817@texas.triqs.com> <3C5C5693.32590.64C1AB2A@localhost> In-Reply-To: <3C5C5693.32590.64C1AB2A@localhost> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: Subject: [Metakit] Patches? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is this the right place to offer patches for MetaKit? I have a patch that corrects Mk4tcl's failure to export its public commands. Where do I send it? -- | Don Porter dgporter@erols.com | | "Some days you just can't get rid of a bomb!" | | -- Adam West as BATMAN | |______________________________________________________________________| From nicolasb@maich.gr Mon Feb 4 03:01:04 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g14913a06957 for ; Mon, 4 Feb 2002 03:01:03 -0600 Received: FROM maich.gr BY mail-server ; Mon Feb 04 11:19:21 2002 +0200 Message-ID: <3C5E5268.C21291E3@maich.gr> Date: Mon, 04 Feb 2002 11:20:40 +0200 From: Nicolas Boretos Organization: MAICh X-Mailer: Mozilla 4.75 [en] (Windows NT 5.0; U) X-Accept-Language: en,el MIME-Version: 1.0 To: Mark Roseman CC: metakit@equi4.com Subject: Re: [Metakit] Mk4Tcl concurrency References: <20020201164751.4817@texas.triqs.com> <20020201164751.4817@texas.triqs.com> <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> Content-Type: text/plain; charset=iso-8859-7 Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, Is your metakit db driver for aolserver (and api?) something that could be made available to the public. We currently use tclhttpd, but would like to migrate some stuff to aolserver... tia, nicolas boretos Mark Roseman wrote: > > As Gordon suggested, for most small-ish datasets, the time to > complete most any operation in Metakit is very quick. For the > odd time when you are doing operations that will cause issues, > yes, running in a thread or another process may be useful. > However, I'd strongly suggest not dealing with it until the > point you know its an issue (whether through practice, or > experimentation ahead of time). > > As an example, our web-based app runs under AOLserver, and we > keep a lot of central data held in Metakit databases. The > multiple AOLserver threads make socket connections to a separate > single-threaded process running the database, so in effect > requests have to wait their turn to be processed. Despite some > fairly substantial queries and reasonable numbers of simultaneous > users, we've never had issues with hugely perceptible delays on > the client side. > > Mark Roseman > http://www.markroseman.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From mark@markroseman.com Mon Feb 4 12:54:58 2002 Received: from admin.cgocable.net (admin.cgocable.net [24.226.1.21]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g14Iswa29157 for ; Mon, 4 Feb 2002 12:54:58 -0600 Received: from eyak-can4.markroseman.com (d141-164-68.home.cgocable.net [24.141.164.68]) by admin.cgocable.net (8.10.0/8.9.3) with ESMTP id g14IvBn28443 for ; Mon, 4 Feb 2002 13:57:14 -0500 (EST) Message-Id: <5.1.0.14.0.20020204135036.00ac7768@roseman.myservers.org> X-Sender: X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Mon, 04 Feb 2002 13:51:42 -0500 To: metakit@equi4.com From: Mark Roseman Subject: Re: [Metakit] Mk4Tcl concurrency In-Reply-To: <3C5E5268.C21291E3@maich.gr> References: <20020201164751.4817@texas.triqs.com> <20020201164751.4817@texas.triqs.com> <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We didn't do a generic database driver for Metakit, but instead did something thats fairly application specific. So no, its not available. Is your metakit db driver for aolserver (and api?) something that could >be made available to the public. >We currently use tclhttpd, but would like to migrate some stuff to >aolserver... Mark Roseman http://www.markroseman.com From genec@goldsmithsinc.com Mon Feb 4 13:01:41 2002 Received: from mail.goldsmithsinc.com ([64.66.89.67]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g14J1ea29504 for ; Mon, 4 Feb 2002 13:01:40 -0600 Received: from genec (genec [10.1.1.100]) by mail.goldsmithsinc.com (Switch-2.0.0/Switch-2.0.0) with SMTP id g14IpVq29320 for ; Mon, 4 Feb 2002 12:51:31 -0600 Message-ID: <015201c1adad$d6814cb0$6401010a@genec> From: "Gene Christian" To: "metakit" References: <00df01c1ab6b$82bbca40$6401010a@genec> <20020201220830.30136@triqs.com> Subject: Re: [Metakit] 2.4.3 and python 2.1? Date: Mon, 4 Feb 2002 12:57:48 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > I'm not sure how "crippled" the std ed is. I build with 6 all the time. > having said that: with 2.4.2, I also built with MinGW. Even cross > platform, i.e. from Linux generating Win binaries. > > One suggestion could be to give MinGW a shot, it's good stuff. Thanks for the tip. It took a little while to get set up, but after a little experimentation MinGW worked just fine. Gene. From wlb@multisysid.com Tue Feb 5 17:26:01 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g15NQ1a24348 for ; Tue, 5 Feb 2002 17:26:01 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id TAA19501 for ; Tue, 5 Feb 2002 19:24:45 -0500 Message-ID: <000d01c1ae9d$a9e41480$6101a8c0@WB02> From: "wlb" To: Date: Tue, 5 Feb 2002 19:34:32 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Porting to Windows CE Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I spent the day porting to Windows CE before reading the archives to see that someone else has made some good progress. daaahhhh. But it looks like it was not a waste of time at all. The previous port to WCE looks to be H/PC compatible. My port focuses on PocketPC, which is mostly a subset of H/PC. I had to deal with the problem of no _osfhandle() to convert from a FILE * to a HANDLE, so I had to replace the FILE *'s with HANDLE's, fopen()'s with CreateFile()'s, etc. If you have a solution to this problem, let me know. These changes are very compatible not only with H/PC, but all versions of Windows. I created a new config variable, q4_WCE. I did not convert text to Unicode. I'm still not 100% sure that that is what I want to do, but then again, it does indeed sound like a good idea. If you can merge the for CE port soon, I would love to add my diffs to that version / release. So far, here are my regression results. It still has a few problems, but close indeed. Extend fails for me also. That's high on my list to understand better. wlb b00 - Should fail *** Failed: A(false) *** <<< done. b01 - Should succeed <<< done. b02 - Int property <<< done. b03 - Float property <<< done. b04 - String property <<< done. b05 - View property <<< done. b06 - View construction <<< done. b07 - Row manipulation <<< done. b08 - Row expressions <<< done. b09 - View manipulation <<< done. b10 - View sorting <<< done. b11 - View selection <<< done. b12 - Add after remove <<< done. b13 - Clear view entry <<< done. b14 - Empty view outlives temp storage <<< done. b15 - View outlives temp storage <<< done. b16 - View outlives cleared temp storage <<< done. b17 - Double property <<< done. b18 - SetAtGrow usage <<< done. b19 - Bytes property <<< done. b20 - Search sorted view <<< done. b21 - Memo property <<< done. b22 - Stored view references <<< done. b23 - Sort comparison fix <<< done. b24 - Custom view comparisons <<< done. b25 - Copy row from derived <<< done. b26 - Partial memo field access <<< done. b27 - Copy value to another row <<< done. n01 - Add to selection <<< done. n02 - Remove from selection <<< done. n03 - Modify into selection <<< done. n04 - Modify out of selection <<< done. n05 - Add to sorted <<< done. n06 - Remove from sorted <<< done. n07 - New property through sort <<< done. n08 - Nested project and select <<< done. n09 - Multiple dependencies <<< done. n10 - Modify sorted duplicates <<< done. n11 - Resize compound derived view <<< done. n12 - Alter multiply derived view <<< done. n13 - Project without <<< done. c01 - Slice forward <<< done. c02 - Slice backward <<< done. c03 - Slice reverse <<< done. c04 - Cartesian product <<< done. c05 - Remapping <<< done. c06 - Pairwise combination <<< done. c07 - Concatenate views <<< done. c08 - Rename property <<< done. c09 - GroupBy operation *** Failed: A(false) *** <<< done. c10 - Counts operation *** Failed: A(false) *** <<< done. c11 - Unique operation <<< done. c12 - Union operation <<< done. c13 - Intersect operation <<< done. c14 - Different operation <<< done. c15 - Minus operation <<< done. c16 - View comparisons <<< done. c17 - Join operation *** Failed: A(false) *** <<< done. c18 - Groupby sort fix <<< done. c19 - JoinProp operation <<< done. c20 - Wide cartesian product <<< done. c21 - Join on compound key <<< done. r00 - Simple insert <<< done. r01 - Simple removes <<< done. r02 - Large inserts and removes <<< done. r03 - Binary property insertions <<< done. r04 - Scripted string property tests <<< done. s00 - Simple storage <<< done. s01 - Integer storage <<< done. s02 - Float storage <<< done. s03 - String storage <<< done. s04 - View storage <<< done. s05 - Store and reload <<< done. s06 - Commit twice <<< done. s07 - Commit modified <<< done. s08 - View after storage <<< done. s09 - Copy storage <<< done. s10 - Stream storage *** Failed: A(v1.GetSize() == 3) *** *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(v3.GetSize() == 3) *** *** Failed: A(v4.GetSize() == 2) *** *** Failed: A(v1.GetSize() == 3) *** *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(v3.GetSize() == 3) *** *** Failed: A(v4.GetSize() == 2) *** *** Failed: A(v1.GetSize() == 3) *** *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(v3.GetSize() == 3) *** *** Failed: A(v4.GetSize() == 2) *** *** Failed: A(v1.GetSize() == 4) *** *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(v3.GetSize() == 3) *** *** Failed: A(v4.GetSize() == 2) *** <<< done. s11 - Commit and rollback <<< done. s12 - Remove subview <<< done. s13 - Remove middle subview <<< done. s14 - Replace attached subview <<< done. s15 - Add after removed subviews <<< done. s16 - Add after removed ints <<< done. s17 - Add after removed strings <<< done. s18 - Empty storage <<< done. s19 - Empty view outlives storage <<< done. s20 - View outlives storage <<< done. s21 - Test demo scenario <<< done. s22 - Double storage <<< done. s23 - Find absent record <<< done. s24 - Bitwise storage <<< done. s25 - Bytes storage <<< done. s26 - Bitwise autosizing <<< done. s27 - Bytes restructuring <<< done. s28 - Doubles added later <<< done. s29 - Delete bytes property <<< done. d01 - Commit aside <<< done. e01 - Extend new file <<< done. e02 - Extend committing twice <<< done. e03 - Read during extend *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(p1 (v2[0]) == 123) *** *** Failed: A(v3.GetSize() == 2) *** *** Failed: A(p1 (v3[0]) == 123) *** *** Failed: A(p1 (v3[1]) == 456) *** <<< done. e04 - Extend during read *** Failed: A(false) *** <<< done. e05 - Test memory mapping <<< done. e06 - Rollback during extend *** Failed: A(v2.GetSize() == 1) *** *** Failed: A(p1 (v2[0]) == 123) *** *** Failed: A(v2a.GetSize() == 1) *** *** Failed: A(p1 (v2a[0]) == 123) *** <<< done. f01 - Add view to format <<< done. f02 - Remove view from format <<< done. f03 - Rollback format change <<< done. f04 - Rearrange format <<< done. f05 - Nested reformat <<< done. f06 - Flip foreign data <<< done. f07 - Automatic structure info (obsolete) <<< done. f08 - Automatic storage format <<< done. f09 - Partial restructuring <<< done. f10 - Committed restructuring <<< done. f11 - Delete missing view <<< done. m01 - Hash mapping <<< done. m02 - Blocked view bug <<< done. m03 - Hash adds <<< done. l00 - Lots of properties *** Failed: A(false) *** <<< done. l01 - Over 32 Kb of integers <<< done. l02 - Over 64 Kb of strings <<< done. l03 - Force sections in storage *** Failed: A(false) *** <<< done. l04 - Modify sections in storage <<< done. l05 - Delete from 32 Kb of strings <<< done. l06 - Bit field manipulations <<< done. l07 - Huge description *** Failed: A(false) *** <<< done. From wlb@multisysid.com Wed Feb 6 08:04:19 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16E4Ia12753 for ; Wed, 6 Feb 2002 08:04:18 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id KAA21981 for ; Wed, 6 Feb 2002 10:03:04 -0500 Message-ID: <000501c1af18$5e3c71a0$6101a8c0@WB02> From: "wlb" To: Date: Wed, 6 Feb 2002 10:12:52 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] WCE port update Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: My WCE port now passes all regression tests....well, I didn't actually run TestStore4 since I still need to #if the fopen()'s and replace with CreateFile()'s for q4_WCE. Previous tests failed because I was being too stingy with the FILE_SHARE_READ and FILE_SHARE_WRITE flags. Together with a couple of errors in GENERIC_READ and GENERIC_WRITE flags in the regression tests. When files cannot be opened with the requested permissions catastrophic failures result in the most unexpected places. I'll look at the code some more and see if I can't detect those problems before something explodes. Mine port is still ASCII, with no testing using MFC or Unicode. Question to the the previous WCE porter: What syntax did you use for Unicode constants? _T("text"), _("text"), L"text", MYMACRO("text") ? I would like to be consistant with you. wlb From jtate@mi-corporation.com Wed Feb 6 09:28:22 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16FSKa15933 for ; Wed, 6 Feb 2002 09:28:22 -0600 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g16FSAf24227; Wed, 6 Feb 2002 10:28:10 -0500 From: "Joseph Tate" To: "wlb" , Subject: RE: [Metakit] WCE port update Date: Wed, 6 Feb 2002 10:29:37 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <000501c1af18$5e3c71a0$6101a8c0@WB02> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I used _T(""). I'll send you my project files so that you can take a look at them. I couldn't post them to the list because they were too big. Joseph > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of wlb > Sent: Wednesday, February 06, 2002 9:13 AM > To: metakit@equi4.com > Subject: [Metakit] WCE port update > > > > My WCE port now passes all regression tests....well, I > didn't actually run TestStore4 since I still need to #if > the fopen()'s and replace with CreateFile()'s for q4_WCE. > > Previous tests failed because I was being too stingy with the > FILE_SHARE_READ and FILE_SHARE_WRITE flags. > Together with a couple of errors in GENERIC_READ and > GENERIC_WRITE flags in the regression tests. > > When files cannot be opened with the requested permissions > catastrophic failures result in the most unexpected places. > I'll look at the code some more and see if I can't detect those > problems before something explodes. > > Mine port is still ASCII, with no testing using MFC or Unicode. > > Question to the the previous WCE porter: What syntax > did you use for Unicode constants? _T("text"), _("text"), L"text", > MYMACRO("text") ? > > I would like to be consistant with you. > > wlb > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From jtate@mi-corporation.com Wed Feb 6 09:57:04 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16Fv4a17362 for ; Wed, 6 Feb 2002 09:57:04 -0600 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g16Fuvf24415; Wed, 6 Feb 2002 10:56:57 -0500 From: "Joseph Tate" To: "wlb" , Subject: RE: [Metakit] Porting to Windows CE Date: Wed, 6 Feb 2002 10:58:24 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <000d01c1ae9d$a9e41480$6101a8c0@WB02> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: If you take a look at the header files that I posted to the list on Jan 29 you can see how I handled the file creation/handle problem. I didn't have to replace all FILE* items with HANDLEs. See the following snippet from fileio.cpp: #ifndef q4_WCE FlushFileBuffers((HANDLE) _get_osfhandle(_fileno(_file))); HANDLE h = ::CreateFileMapping((HANDLE) _get_osfhandle(_fileno(_file)), 0, PAGE_READONLY, 0, len, 0); #else FlushFileBuffers((HANDLE) _fileno(_file)); HANDLE h = ::CreateFileMapping((HANDLE) _fileno(_file), 0, PAGE_READONLY, 0, len, 0); #endif //q4_WCE It turns out that _fileno returns a handle. You just have to cast it. I couldn't find an acceptable replacement for freopen, but that was only used for creating the output files during testing. It passes all the regression tests, even teststore4. The issue that my port has is that everything is w_char rather than char, which is a serious issue because if a file is created on CE it may not open under !CE. It definitely needs more work, but as of right now, my resources have been redirected elsewhere. I'll be getting back to work on this in a week or two. Good luck. Joseph > -----Original Message----- > From: metakit-admin@equi4.com [mailto:metakit-admin@equi4.com]On Behalf > Of wlb > Sent: Tuesday, February 05, 2002 6:35 PM > To: metakit@equi4.com > Subject: [Metakit] Porting to Windows CE > > > I spent the day porting to Windows CE before reading the > archives to see that someone else has made some good > progress. > > daaahhhh. > > But it looks like it was not a waste of time at all. The previous > port to WCE looks to be H/PC compatible. My port focuses > on PocketPC, which is mostly a subset of H/PC. I had to > deal with the problem of no _osfhandle() to convert from a > FILE * to a HANDLE, so I had to replace the FILE *'s > with HANDLE's, fopen()'s with CreateFile()'s, etc. If you > have a solution to this problem, let me know. > > These changes are very compatible not only with H/PC, but > all versions of Windows. > > I created a new config variable, q4_WCE. > > I did not convert text to Unicode. I'm still not 100% sure that > that is what I want to do, but then again, it does indeed sound > like a good idea. > > If you can merge the for CE port soon, I would love to add my > diffs to that version / release. > > So far, here are my regression results. It still has a few problems, > but close indeed. > > Extend fails for me also. That's high on my list to understand better. > > wlb > > b00 - Should fail *** Failed: A(false) *** > <<< done. > b01 - Should succeed <<< done. > b02 - Int property <<< done. > b03 - Float property <<< done. > b04 - String property <<< done. > b05 - View property <<< done. > b06 - View construction <<< done. > b07 - Row manipulation <<< done. > b08 - Row expressions <<< done. > b09 - View manipulation <<< done. > b10 - View sorting <<< done. > b11 - View selection <<< done. > b12 - Add after remove <<< done. > b13 - Clear view entry <<< done. > b14 - Empty view outlives temp storage <<< done. > b15 - View outlives temp storage <<< done. > b16 - View outlives cleared temp storage <<< done. > b17 - Double property <<< done. > b18 - SetAtGrow usage <<< done. > b19 - Bytes property <<< done. > b20 - Search sorted view <<< done. > b21 - Memo property <<< done. > b22 - Stored view references <<< done. > b23 - Sort comparison fix <<< done. > b24 - Custom view comparisons <<< done. > b25 - Copy row from derived <<< done. > b26 - Partial memo field access <<< done. > b27 - Copy value to another row <<< done. > n01 - Add to selection <<< done. > n02 - Remove from selection <<< done. > n03 - Modify into selection <<< done. > n04 - Modify out of selection <<< done. > n05 - Add to sorted <<< done. > n06 - Remove from sorted <<< done. > n07 - New property through sort <<< done. > n08 - Nested project and select <<< done. > n09 - Multiple dependencies <<< done. > n10 - Modify sorted duplicates <<< done. > n11 - Resize compound derived view <<< done. > n12 - Alter multiply derived view <<< done. > n13 - Project without <<< done. > c01 - Slice forward <<< done. > c02 - Slice backward <<< done. > c03 - Slice reverse <<< done. > c04 - Cartesian product <<< done. > c05 - Remapping <<< done. > c06 - Pairwise combination <<< done. > c07 - Concatenate views <<< done. > c08 - Rename property <<< done. > c09 - GroupBy operation *** Failed: A(false) *** > <<< done. > c10 - Counts operation *** Failed: A(false) *** > <<< done. > c11 - Unique operation <<< done. > c12 - Union operation <<< done. > c13 - Intersect operation <<< done. > c14 - Different operation <<< done. > c15 - Minus operation <<< done. > c16 - View comparisons <<< done. > c17 - Join operation *** Failed: A(false) *** > <<< done. > c18 - Groupby sort fix <<< done. > c19 - JoinProp operation <<< done. > c20 - Wide cartesian product <<< done. > c21 - Join on compound key <<< done. > r00 - Simple insert <<< done. > r01 - Simple removes <<< done. > r02 - Large inserts and removes <<< done. > r03 - Binary property insertions <<< done. > r04 - Scripted string property tests <<< done. > s00 - Simple storage <<< done. > s01 - Integer storage <<< done. > s02 - Float storage <<< done. > s03 - String storage <<< done. > s04 - View storage <<< done. > s05 - Store and reload <<< done. > s06 - Commit twice <<< done. > s07 - Commit modified <<< done. > s08 - View after storage <<< done. > s09 - Copy storage <<< done. > s10 - Stream storage *** Failed: A(v1.GetSize() == 3) *** > *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(v3.GetSize() == 3) *** > *** Failed: A(v4.GetSize() == 2) *** > *** Failed: A(v1.GetSize() == 3) *** > *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(v3.GetSize() == 3) *** > *** Failed: A(v4.GetSize() == 2) *** > *** Failed: A(v1.GetSize() == 3) *** > *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(v3.GetSize() == 3) *** > *** Failed: A(v4.GetSize() == 2) *** > *** Failed: A(v1.GetSize() == 4) *** > *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(v3.GetSize() == 3) *** > *** Failed: A(v4.GetSize() == 2) *** > <<< done. > s11 - Commit and rollback <<< done. > s12 - Remove subview <<< done. > s13 - Remove middle subview <<< done. > s14 - Replace attached subview <<< done. > s15 - Add after removed subviews <<< done. > s16 - Add after removed ints <<< done. > s17 - Add after removed strings <<< done. > s18 - Empty storage <<< done. > s19 - Empty view outlives storage <<< done. > s20 - View outlives storage <<< done. > s21 - Test demo scenario <<< done. > s22 - Double storage <<< done. > s23 - Find absent record <<< done. > s24 - Bitwise storage <<< done. > s25 - Bytes storage <<< done. > s26 - Bitwise autosizing <<< done. > s27 - Bytes restructuring <<< done. > s28 - Doubles added later <<< done. > s29 - Delete bytes property <<< done. > d01 - Commit aside <<< done. > e01 - Extend new file <<< done. > e02 - Extend committing twice <<< done. > e03 - Read during extend *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(p1 (v2[0]) == 123) *** > *** Failed: A(v3.GetSize() == 2) *** > *** Failed: A(p1 (v3[0]) == 123) *** > *** Failed: A(p1 (v3[1]) == 456) *** > <<< done. > e04 - Extend during read *** Failed: A(false) *** > <<< done. > e05 - Test memory mapping <<< done. > e06 - Rollback during extend *** Failed: A(v2.GetSize() == 1) *** > *** Failed: A(p1 (v2[0]) == 123) *** > *** Failed: A(v2a.GetSize() == 1) *** > *** Failed: A(p1 (v2a[0]) == 123) *** > <<< done. > f01 - Add view to format <<< done. > f02 - Remove view from format <<< done. > f03 - Rollback format change <<< done. > f04 - Rearrange format <<< done. > f05 - Nested reformat <<< done. > f06 - Flip foreign data <<< done. > f07 - Automatic structure info (obsolete) <<< done. > f08 - Automatic storage format <<< done. > f09 - Partial restructuring <<< done. > f10 - Committed restructuring <<< done. > f11 - Delete missing view <<< done. > m01 - Hash mapping <<< done. > m02 - Blocked view bug <<< done. > m03 - Hash adds <<< done. > l00 - Lots of properties *** Failed: A(false) *** > <<< done. > l01 - Over 32 Kb of integers <<< done. > l02 - Over 64 Kb of strings <<< done. > l03 - Force sections in storage *** Failed: A(false) *** > <<< done. > l04 - Modify sections in storage <<< done. > l05 - Delete from 32 Kb of strings <<< done. > l06 - Bit field manipulations <<< done. > l07 - Huge description *** Failed: A(false) *** > <<< done. > > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From wlb@multisysid.com Wed Feb 6 10:37:10 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16Gb9a19952 for ; Wed, 6 Feb 2002 10:37:09 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id MAA23341 for ; Wed, 6 Feb 2002 12:35:55 -0500 Message-ID: <001b01c1af2d$b92df330$6101a8c0@WB02> From: "wlb" To: Date: Wed, 6 Feb 2002 12:45:45 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Porting to Windows CE Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > It turns out that _fileno returns a handle. You just have to cast it. Well I'll be. Hats off to you. It also appears you have spent more time on this than I. At least for now, I do have time and energy to dedicate to this port. I understand the problem of portability of data files to non-Unicode platforms. I would think that portability would require a library which simultaneously supported unicode and non-unicode methods, as well as an indicator in the data file header. I guess that last statement is not 100% correct: you could have two DLL's, one with Unicode and the other with ASCII. I prefer one DLL. One DLL which handles both cases would preclude the use of (the MicroSoft standard) _tcscmp and their other _t??? functions. (Thinking aloud.) Other than the c4_String class, there are few uses of char *'s. Most of the occurrances are in naming properties, descriptions, and the storage format string used in GetAs( ). c4_String. Huh. I have noticed that CE's MFC-CString can be assigned and ASCII value directly without first converting to Unicode. I haven't checked to see if he stores it ASCII or silently converts to Unicode. If I'm not mistaken, there is something in the data file header which indicates big-endian / little-endian. Maybe there are a few more flags or values in that area that could be useful. I'll dig some more unless jcw shines some light on it. I'm very interested in knowning what jcw thinks would be the correct method to integrate Unicode. I'm also interested in knowing what you (Mr. Tate) think would be the most interesting / needed areas of work. I'm looking forward to reviewing the code. wlb From jtate@mi-corporation.com Wed Feb 6 13:14:32 2002 Received: from edsel.mi-messages.net ([209.116.71.28]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16JEVa28441 for ; Wed, 6 Feb 2002 13:14:31 -0600 Received: from jtatework ([204.85.2.183]) by edsel.mi-messages.net (Switch-2.0.0/Switch-2.0.0) with SMTP id g16JENf25328; Wed, 6 Feb 2002 14:14:23 -0500 From: "Joseph Tate" To: "wlb" , Subject: RE: [Metakit] Porting to Windows CE Date: Wed, 6 Feb 2002 14:15:52 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <001b01c1af2d$b92df330$6101a8c0@WB02> X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Well I'll be. Hats off to you. It also appears you have spent more > time on this than I. Thanks. I spent about a week and a half on it. > Other than the c4_String class, there are few uses of char *'s. Most > of the occurrances are in naming properties, descriptions, and the > storage format string used in GetAs( ). c4_String. Huh. I have > noticed that CE's MFC-CString can be assigned and ASCII value > directly without first converting to Unicode. I haven't checked to > see if he stores it ASCII or silently converts to Unicode. The CE CString class is strictly ASCII. You can convert CString's to unicode, but only through the MultiByteToWideChar function and then through WideCharToMultiByte to get unicode back to ASCII. This doesn't help us at all. CStrings are evil anyway. If you want to know why, ask me off list. As you'll see (see wince.h), what I did to make my life easier was to #define char to short, and then enclose everything that needed to be a short* in _T(""). > If I'm not mistaken, there is something in the data file header which > indicates big-endian / little-endian. Maybe there are a few more flags > or values in that area that could be useful. I'll dig some more unless > jcw shines some light on it. I would support this addition to the header. > I'm very interested in knowning what jcw thinks would be the correct > method to integrate Unicode. I think it's something that definitely needs to be looked at as metakit is used in more and more world wide applications. We could do it in a way that would make it totally transparent to the user, especially if we used something like UTF-8 unicode, where a file that would normally be pure ascii remains pure ascii. If done correctly, no-one needs to know of the change, and we can even use the same file format. People who want unicode support just use different parameters or function calls, or compile options. > I'm also interested in knowing what you (Mr. Tate) think would be > the most interesting / needed areas of work. The next step I believe would be to replace all remaining instances of char in the code with CHAR, i.e. something that can be typedef'd according to the platform that it was compiled on (t4_char perhaps?). Either that or duplicate every function that takes a char* parameter so that it's overloaded to take a short* as well (we may wish to do this anyway so that we don't have to have _T("") everywhere in the metakit code). There are still places in the code (mostly in the integer handling) that need a signed single byte entity, and here it would be nice to still have char available, though having t4_byte and t4_sbyte (s for signed) functions well. Some more thought needs to go into this though. Perhaps the best option is to keep the t4_string class with a unicode backend (short*), but provide operators and conversion functions to make it easy to insert and extract ascii if needs be. I don't know. Another thing to look at is textend.cpp in the test suite, figuring out why there's an extra seven bytes written per entry. This would be a good place to start to get us closer to files that work across all platforms. > I'm looking forward to reviewing the code. Let me know if you need any help with clarification or understanding. It's nice to not have to think about this on my own anymore. Joseph From wlb@multisysid.com Wed Feb 6 15:15:48 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g16LFla02682 for ; Wed, 6 Feb 2002 15:15:47 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id RAA25809 for ; Wed, 6 Feb 2002 17:14:32 -0500 Message-ID: <001301c1af54$a609fa70$6101a8c0@WB02> From: "wlb" To: References: Subject: Re: [Metakit] Porting to Windows CE Date: Wed, 6 Feb 2002 17:24:21 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: >> Other than the c4_String class, there are few uses of char *'s. Most >> of the occurrances are in naming properties, descriptions, and the >> storage format string used in GetAs( ). c4_String. Huh. I have >> noticed that CE's MFC-CString can be assigned and ASCII value >> directly without first converting to Unicode. I haven't checked to >> see if he stores it ASCII or silently converts to Unicode. > > The CE CString class is strictly ASCII. You can convert CString's to > unicode, but only through the MultiByteToWideChar function and then through > WideCharToMultiByte to get unicode back to ASCII. This doesn't help us at > all. CStrings are evil anyway. If you want to know why, ask me off list. > As you'll see (see wince.h), what I did to make my life easier was to > #define char to short, and then enclose everything that needed to be a > short* in _T(""). There are a couple of shortcut functions that don't take nearly as many parameters: mbstowcs() and wcstombs(). I have always used CString in CE as Unicode... CString csText = _T("my text"); LPCTSTR szText = csText, and it has always given me the expected results...so I thought it was Unicode. Even more evil are people who write functions like: CString FindText( CString key ) { ... } without understanding what is a copy constructor. I have stories (and code) to make you laugh. Ask me off list. >> If I'm not mistaken, there is something in the data file header which >> indicates big-endian / little-endian. Maybe there are a few more flags >> or values in that area that could be useful. I'll dig some more unless >> jcw shines some light on it. > >I would support this addition to the header. I looked through the code. There is no real big/little endian checking. There was an earlier header format which was little ending and the code has a comment to that effect. Officially, strings are stored UTF-8. The "Overview of Metadata Format" document on the web site states: { This string, as well as all strings stored as properties of rows, is stored in UTF-8 format, which allow storing Unicode strings while maintaining compatibility of datafiles across all platforms. } I'll dig through the file format and textend.cpp. Worst case scenario, I can create a file ASCII and one UTF-16 (your port) and hexdump them to see the differences. > I think it's something that definitely needs to be looked at as metakit is > used in more and more world wide applications. We could do it in a way that > would make it totally transparent to the user, especially if we used > something like UTF-8 unicode, where a file that would normally be pure ascii > remains pure ascii. If done correctly, no-one needs to know of the change, > and we can even use the same file format. People who want unicode support > just use different parameters or function calls, or compile options. UTF-8 is nice. For CE, we would still have to watch the UTF-8 to UTF-16 conversions. Vice versa, for metakit we need to watch the UTF-16 to UTF-8 conversions. The only languages which I am handling (English/Spanish/Portugese/ French) are happy with mbstowcs() and wcwtombs() functions, then treat as ASCII or UTF-16. Not theoretically correct, but it works for me :) I've been programming CE for a long time, but metakit for only two days. It will take a few more days for me to get up to speed on the internals of metakit, read all the docs, and read all the code. wlb From mbloore@yahoo.com Wed Feb 6 21:30:56 2002 Received: from web9102.mail.yahoo.com (web9102.mail.yahoo.com [216.136.128.239]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g173Uua17360 for ; Wed, 6 Feb 2002 21:30:56 -0600 Message-ID: <20020207033055.72906.qmail@web9102.mail.yahoo.com> Received: from [65.93.22.92] by web9102.mail.yahoo.com via HTTP; Wed, 06 Feb 2002 19:30:55 PST Date: Wed, 6 Feb 2002 19:30:55 -0800 (PST) From: mARK bLOORE To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] empty data file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: running the following program, under RH linux 7.1, compiled with gcc 2.96: #include int main() { cout << endl << endl << "Hello World, this is MetaKit!" << endl; c4_StringProp pName ("name"); c4_StringProp pCountry ("country"); c4_Storage vAddress ("myfile.dat", 1); c4_Row row; cout << "rows: " << vAddress.GetSize() << endl; pName (row) = "John Williams"; pCountry (row) = "UK"; vAddress.Add(row); vAddress.Add(pName ["Julien Coco"] + pCountry ["Netherlands"]); cout << "name[0]: " << pName (vAddress[0]) << endl; cout << "name[1]: " << pName (vAddress[1]) << endl; bool committed = vAddress.Commit(); cout << "committed: " << (committed ? "yes" : "no") << endl; c4_Strategy& strat = vAddress.Strategy(); bool valid = strat.IsValid(); cout << "strategy valid: " << (valid ? "yes" : "no") << endl; cout << "rows: " << vAddress.GetSize() << endl; cout << "Goodbye World" << endl; return 0; } prints the following, on the first and subsequent runs: Hello World, this is MetaKit! rows: 1 name[0]: name[1]: John Williams committed: yes strategy valid: yes rows: 3 Goodbye World myfile.dat is created on the first run, but is empty, and stays empty on later runs. debugging, i see that during Commit() 'limit' in c4_SaveContext::SaveIt() is set to 11, which causes the routine to exit rather than write an empty file. i am not clear on how its value was decided, except that it came from a c4_Allocator. what am i missing? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com From jcw@equi4.com Thu Feb 7 04:53:54 2002 Received: from [10.0.1.55] (ipc379e97b.dial.wxs.nl [195.121.233.123]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g17Arra22776 for ; Thu, 7 Feb 2002 04:53:53 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] empty data file Date: Thu, 7 Feb 2002 11:51:36 +0100 Message-Id: <20020207105137.23502@triqs.com> In-Reply-To: <20020207033055.72906.qmail@web9102.mail.yahoo.com> References: <20020207033055.72906.qmail@web9102.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: mARK bLOORE wrote: > c4_Storage vAddress ("myfile.dat", 1); [...] > vAddress.Add(row); [...] >myfile.dat is created on the first run, but is empty, and stays empty >on later runs. Ouch - this use should not have been possible. It's a side effect of the generalizations made not so long ago. The way to do it is: c4_Storage storage ("myfile.dat", 1); c4_View vAddress = storage.GetAs("address[name:S,country:S]"); [etc] A storage object holds one or more named views. It may act like a view itself, but please just consider that a design flaw for now. See sample code in examples/, demos/, and tests/ in the source distribution. -jcw From mbloore@yahoo.com Mon Feb 11 17:49:08 2002 Received: from web14301.mail.yahoo.com (web14301.mail.yahoo.com [216.136.173.77]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1BNn8a01219 for ; Mon, 11 Feb 2002 17:49:08 -0600 Message-ID: <20020211234908.86930.qmail@web14301.mail.yahoo.com> Received: from [65.93.23.229] by web14301.mail.yahoo.com via HTTP; Mon, 11 Feb 2002 15:49:08 PST Date: Mon, 11 Feb 2002 15:49:08 -0800 (PST) From: mARK bLOORE To: metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Metakit] storing large numbers of strings Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i am working on an application that will store data about large numbers of files, perhaps 100,000 in a directory. i am using essentilly the CatFish structure, and finding that saving the file names is very slow. (storing my extra data will only make it worse.) i have tried putting the names into memos, but that does not help (not really a surprise). i tried setting the growth granularity to 100100 without effect (but i don't suppose that could help, since strings are not of fixed size). storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, and 100,000 takes almost 700 sec. without the names (just date and size) i get 100,000 in about 15 sec. can anyone suggest anything i might try to improve this? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com From Murat_Berk@bmc.com Mon Feb 11 22:43:31 2002 Received: from babbler.bmc.com (camaro.bmc.com [198.207.223.231]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1C4hVa07976 for ; Mon, 11 Feb 2002 22:43:31 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by babbler.bmc.com (8.10.2/8.10.2) with ESMTP id g1C4gcm25327; Mon, 11 Feb 2002 22:42:38 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Mon, 11 Feb 2002 22:39:34 -0600 Message-ID: From: "Berk, Murat" To: "'mARK bLOORE'" , metakit Subject: RE: [Metakit] storing large numbers of strings Date: Mon, 11 Feb 2002 22:39:33 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The growth granularity is ignored. Set the size of view to 100,000. Set the lost row for name column to a dummy value (which forces all internal structures to be created to track string offsets. Then set it using for ( i = 1...100000 ) { c4_RowRef r = view[i]; pName(r) = string value } -----Original Message----- From: mARK bLOORE [mailto:mbloore@yahoo.com] Sent: Monday, February 11, 2002 5:49 PM To: metakit Subject: [Metakit] storing large numbers of strings i am working on an application that will store data about large numbers of files, perhaps 100,000 in a directory. i am using essentilly the CatFish structure, and finding that saving the file names is very slow. (storing my extra data will only make it worse.) i have tried putting the names into memos, but that does not help (not really a surprise). i tried setting the growth granularity to 100100 without effect (but i don't suppose that could help, since strings are not of fixed size). storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, and 100,000 takes almost 700 sec. without the names (just date and size) i get 100,000 in about 15 sec. can anyone suggest anything i might try to improve this? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From wlb@multisysid.com Tue Feb 12 09:40:15 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1CFeFa26931 for ; Tue, 12 Feb 2002 09:40:15 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id LAA22334 for ; Tue, 12 Feb 2002 11:39:16 -0500 Message-ID: <002401c1b3dc$cd1ab0a0$6101a8c0@WB02> From: "wlb" To: Date: Tue, 12 Feb 2002 11:49:05 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Unicode recommendation and WCE Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Metakit is capable of UTF-8. However, it would be more appropriate to say that Metakit handles single-byte null terminated strings as a fundamental data type. Metakit neither validates, enforces, nor converts UTF-8 to/from ASCII. A lot of the data which we save as strings is not UTF-8 encoded. ASCII and UTF-8 only overlap on 7-bit characters. Accented characters which we commenly know and love in most non-English languages are usually 8-bit ASCII, but require a two-byte representation in UTF-8. Indeed. Even in English, we are accustomed to borrowing words from other languages with these accents. Cafe' is the only word which immediately comes to mind. Personal names with accents should also conserve those accents in English. So, if you want to save your data UTF-8, it will work fine (null terminated 8-bit strings), or ASCII. But if you save your data UTF-8, expect to read it UTF-8, and vice versa. Metakit is agnostic. Only the application knows the true encoding. In Unicode speak, there are two commonly used double-byte (or wide character) representations. UCS-2 and UTF-16. For our purposes, I will consider them interchangable. The rules of manipulation are simple and fixed such as our standard null terminated strings, except that each 'code' is two bytes. I suggest creating a database column type. Call it '2', call it 'U', call it anything you want. It would deal with wchar_t *'s as the standard 'S'tring type deals with char *'s. In addition, I would recommend overloading the 12 or so functions which take variable names or structure information to provide wchar_t *'s as well as char *'s. Internal structures should be either one or the other, but on-the-fly conversions here should not suffer any measurable performance penalty. Structure and property name lookups really should not be inside a loop. Some operators would also need to be overloaded. This approach would have the following advantages: - a database can be copied from one machine to another - no penalty on the native machine to convert to/from ASCII or UTF8 if machine natively handles UCS-2 or UTF-16. - Any build of metakit can handle simultaneously wchar_t *'s or char *'s (where the host compiler handles wchar_t's, which is ANSI.) You might also want to consider the following: - There is a standard method documented in the Unicode literature for attaching a two-byte prefix to a string to determine byte order. The Unicode documentation is online at www.unicode.org. It is longish and committee like but very thorough. Please excuse the long-ish explanation. Let me know what you think. wlb From mbloore@yahoo.com Tue Feb 12 19:53:58 2002 Received: from web14308.mail.yahoo.com (web14308.mail.yahoo.com [216.136.173.156]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1D1rwa25792 for ; Tue, 12 Feb 2002 19:53:58 -0600 Message-ID: <20020213015349.56824.qmail@web14308.mail.yahoo.com> Received: from [65.93.23.229] by web14308.mail.yahoo.com via HTTP; Tue, 12 Feb 2002 17:53:49 PST Date: Tue, 12 Feb 2002 17:53:49 -0800 (PST) From: mARK bLOORE Subject: RE: [Metakit] storing large numbers of strings To: metakit In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thank you for your reply. Unfortunatly, preallocating the view causes things to get much slower! I am at a loss to explain this. Preallocating 10,000 rows and listing 10,000 files is pretty quick, though still slower than not preallocating. Preallocating 100,000 rows and listing the same 10,000 files is about twenty times slower (60 sec vs. 3). The slowdown happens in the for ( i = 1...10000 ) { c4_RowRef r = view[i]; pName(r) = string value } step. The step which copies that view into the directory's row takes an additional 3 seconds either way. --- "Berk, Murat" wrote: > The growth granularity is ignored. > > Set the size of view to 100,000. > Set the lost row for name column to a dummy value (which forces all > internal > structures to be created to track string offsets. > Then set it using > for ( i = 1...100000 ) > { > c4_RowRef r = view[i]; > pName(r) = string value > } > > -----Original Message----- > From: mARK bLOORE [mailto:mbloore@yahoo.com] > Sent: Monday, February 11, 2002 5:49 PM > To: metakit > Subject: [Metakit] storing large numbers of strings > > > i am working on an application that will store data about large > numbers > of files, perhaps 100,000 in a directory. i am using essentilly the > CatFish structure, and finding that saving the file names is very > slow. > (storing my extra data will only make it worse.) i have tried > putting > the names into memos, but that does not help (not really a surprise). > i tried setting the growth granularity to 100100 without effect (but > i > don't suppose that could help, since strings are not of fixed size). > > storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, > and > 100,000 takes almost 700 sec. without the names (just date and size) > i > get 100,000 in about 15 sec. > > can anyone suggest anything i might try to improve this? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com From Murat_Berk@bmc.com Tue Feb 12 21:05:21 2002 Received: from babbler.bmc.com (firebird.bmc.com [198.207.223.228]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1D35La27411 for ; Tue, 12 Feb 2002 21:05:21 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by babbler.bmc.com (8.10.2/8.10.2) with ESMTP id g1D38BG14337; Tue, 12 Feb 2002 21:08:11 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Tue, 12 Feb 2002 21:05:07 -0600 Message-ID: From: "Berk, Murat" To: "'mARK bLOORE'" , metakit Subject: RE: [Metakit] storing large numbers of strings Date: Tue, 12 Feb 2002 21:05:03 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Did you do pName(_view[10000-1]) = dummy string before entering the row. That is the key not preallocating. Otherwise you are right it gets slower. Murat -----Original Message----- From: mARK bLOORE [mailto:mbloore@yahoo.com] Sent: Tuesday, February 12, 2002 7:54 PM To: metakit Subject: RE: [Metakit] storing large numbers of strings Thank you for your reply. Unfortunatly, preallocating the view causes things to get much slower! I am at a loss to explain this. Preallocating 10,000 rows and listing 10,000 files is pretty quick, though still slower than not preallocating. Preallocating 100,000 rows and listing the same 10,000 files is about twenty times slower (60 sec vs. 3). The slowdown happens in the for ( i = 1...10000 ) { c4_RowRef r = view[i]; pName(r) = string value } step. The step which copies that view into the directory's row takes an additional 3 seconds either way. --- "Berk, Murat" wrote: > The growth granularity is ignored. > > Set the size of view to 100,000. > Set the lost row for name column to a dummy value (which forces all > internal > structures to be created to track string offsets. > Then set it using > for ( i = 1...100000 ) > { > c4_RowRef r = view[i]; > pName(r) = string value > } > > -----Original Message----- > From: mARK bLOORE [mailto:mbloore@yahoo.com] > Sent: Monday, February 11, 2002 5:49 PM > To: metakit > Subject: [Metakit] storing large numbers of strings > > > i am working on an application that will store data about large > numbers > of files, perhaps 100,000 in a directory. i am using essentilly the > CatFish structure, and finding that saving the file names is very > slow. > (storing my extra data will only make it worse.) i have tried > putting > the names into memos, but that does not help (not really a surprise). > i tried setting the growth granularity to 100100 without effect (but > i > don't suppose that could help, since strings are not of fixed size). > > storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, > and > 100,000 takes almost 700 sec. without the names (just date and size) > i > get 100,000 in about 15 sec. > > can anyone suggest anything i might try to improve this? __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From mbloore@yahoo.com Tue Feb 12 21:33:30 2002 Received: from web14303.mail.yahoo.com (web14303.mail.yahoo.com [216.136.173.79]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1D3XUa28124 for ; Tue, 12 Feb 2002 21:33:30 -0600 Message-ID: <20020213033325.65183.qmail@web14303.mail.yahoo.com> Received: from [65.93.23.229] by web14303.mail.yahoo.com via HTTP; Tue, 12 Feb 2002 19:33:25 PST Date: Tue, 12 Feb 2002 19:33:25 -0800 (PST) From: mARK bLOORE Subject: RE: [Metakit] storing large numbers of strings To: metakit In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I did. I later tried removing that, but it made no difference. I also tried allocating 100,000 but only setting view[9999]. That was just as slow as setting view[99999], when only creating 10,000 rows. The important thing, I think, is that preallocating 10x too much makes for a huge slowdown. I find that if i don't store the file names (just two longs, for date and size), then things are fast. Also, storing the names speeds up towards the end. I think that putting names into the string column must be doing memory moves that are slowing it down, even though nothing has been stored in the rows that are moved. --- "Berk, Murat" wrote: > Did you do > pName(_view[10000-1]) = dummy string > before entering the row. That is the key not preallocating. > Otherwise you are right it gets slower. > > Murat > > -----Original Message----- > From: mARK bLOORE [mailto:mbloore@yahoo.com] > Sent: Tuesday, February 12, 2002 7:54 PM > To: metakit > Subject: RE: [Metakit] storing large numbers of strings > > > Thank you for your reply. > > Unfortunatly, preallocating the view causes things to get much > slower! > I am at a loss to explain this. > > Preallocating 10,000 rows and listing 10,000 files is pretty quick, > though still slower than not preallocating. Preallocating 100,000 > rows > and listing the same 10,000 files is about twenty times slower (60 > sec > vs. 3). > The slowdown happens in the > for ( i = 1...10000 ) > { > c4_RowRef r = view[i]; > pName(r) = string value > } > step. The step which copies that view into the directory's row takes > an additional 3 seconds either way. > > > --- "Berk, Murat" wrote: > > The growth granularity is ignored. > > > > Set the size of view to 100,000. > > Set the lost row for name column to a dummy value (which forces all > > internal > > structures to be created to track string offsets. > > Then set it using > > for ( i = 1...100000 ) > > { > > c4_RowRef r = view[i]; > > pName(r) = string value > > } > > > > -----Original Message----- > > From: mARK bLOORE [mailto:mbloore@yahoo.com] > > Sent: Monday, February 11, 2002 5:49 PM > > To: metakit > > Subject: [Metakit] storing large numbers of strings > > > > > > i am working on an application that will store data about large > > numbers > > of files, perhaps 100,000 in a directory. i am using essentilly > the > > CatFish structure, and finding that saving the file names is very > > slow. > > (storing my extra data will only make it worse.) i have tried > > putting > > the names into memos, but that does not help (not really a > surprise). > > i tried setting the growth granularity to 100100 without effect > (but > > i > > don't suppose that could help, since strings are not of fixed > size). > > > > storing 1,000 names takes about 0.3 seconds. 10,000 takes 10 sec, > > and > > 100,000 takes almost 700 sec. without the names (just date and > size) > > i > > get 100,000 in about 15 sec. > > > > can anyone suggest anything i might try to improve this? > > > __________________________________________________ > Do You Yahoo!? > Send FREE Valentine eCards with Yahoo! Greetings! > http://greetings.yahoo.com > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Send FREE Valentine eCards with Yahoo! Greetings! http://greetings.yahoo.com From gerard@xsystemsinc.com Wed Feb 13 12:34:01 2002 Received: from dom01.xsystemsinc.com (hidden-user@dom01.xsystemsinc.com [12.5.16.43]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1DIY1a25133 for ; Wed, 13 Feb 2002 12:34:01 -0600 Received: from GERARD ([64.204.192.3]) by dom01.xsystemsinc.com (Lotus Domino Release 5.0.8) with SMTP id 2002021313335799:3043 ; Wed, 13 Feb 2002 13:33:57 -0500 Message-ID: <007d01c1b4bd$0d374870$0c00a8c0@GERARD> From: "Gerard Duquette" To: Date: Wed, 13 Feb 2002 13:34:19 -0500 MIME-Version: 1.0 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-MIMETrack: Itemize by SMTP Server on dom01/xsystems(Release 5.0.8 |June 18, 2001) at 02/13/2002 01:33:58 PM, Serialize by Router on dom01/xsystems(Release 5.0.8 |June 18, 2001) at 02/13/2002 01:34:01 PM, Serialize complete at 02/13/2002 01:34:01 PM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] AIX metakit shared object Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I am in the process of installing metakit for one of our customers. So far thats not a problem, the problem is that they are on IBM AIX servers. I have tried to find the mk4tcl.so shared object for AIX with no success. The best I can do is get the source code and compile it. Now it gets complicated, because they are not really experienced when it comes to compiling. I have tried to setup a cross compiler on one of our servers and quickly found out this would take way to much time. This is where the world comes in, specifically the world of metakit users. Has anyone succesfully compiled metakit for IBM AIX RS600, and would you be willing to share the fruits of your success, if not would you help in getting it compiled? I admit I am new to this list and this is a bit much to ask the first time, but I dont have much options left. This would be greatly appreciated. Thank you Gerard Duquette X.Systems, Inc (703) 330-1645 ext111 gerard@xsystemsinc.com From cloudshadow@21cn.com Thu Feb 14 07:26:23 2002 Received: from 21cn.com ([61.140.60.248]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1EDQMa29450 for ; Thu, 14 Feb 2002 07:26:22 -0600 Message-Id: <200202141326.g1EDQMa29450@trixie.triqs.com> Received: from shadow([61.142.221.12]) by 21cn.com(AIMC 2.9.5.2) with SMTP id jm1c3c6c2e7f; Thr, 14 Feb 2002 21:27:30 +0800 Date: Thu, 14 Feb 2002 21:26:11 +0800 From: cloudshadow To: "metakit@equi4.com" X-mailer: FoxMail 4.0 Alpha 1 [cn] Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=====002_Dragon680087657523_=====" Subject: [Metakit] how can i change the DBF files to the metakit files type? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is a multi-part message in MIME format. --=====002_Dragon680087657523_===== Content-Type: text/plain; charset="GB2312" Content-Transfer-Encoding: base64 --=====002_Dragon680087657523_===== Content-Type: text/html; charset="GB2312" Content-Transfer-Encoding: base64 PEhUTUw+DQo8SEVBRD4NCjxtZXRhIGh0dHAtZXF1aXZlPSJDb250ZW50LVR5cGUiIGNvbnRlbnQ9 InRleHQvaHRtbDsgY2hhcnNldD1HQjIzMTIiPg0KPG1ldGEgaHR0cC1lcXVpdmU9IkNvbnRlbnQt VHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PUdCMjMxMiI+DQo8TUVUQSBOQU1FPSJH RU5FUkFUT1IiIENvbnRlbnQ9Ik1pY3Jvc29mdCBESFRNTCBFZGl0aW5nIENvbnRyb2wiPg0KPFRJ VExFPjwvVElUTEU+DQo8L0hFQUQ+DQo8Qk9EWT4NCjxESVY+Jm5ic3A7DQo8L0RJVj4NCjwvQk9E WT4NCjwvSFRNTD4NCg== --=====002_Dragon680087657523_=====-- From horst@freedict.de Thu Feb 14 12:15:11 2002 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1EIFAa09189 for ; Thu, 14 Feb 2002 12:15:11 -0600 Received: (qmail 3098 invoked from network); 14 Feb 2002 18:49:26 -0000 Received: from dialin-145-254-115-124.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.115.124) by mail.netbeat.de with SMTP; 14 Feb 2002 18:49:26 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id g1EI7Bm00968; Thu, 14 Feb 2002 19:07:12 +0100 Message-Id: <200202141807.g1EI7Bm00968@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Thu, 14 Feb 2002 19:07:08 +0100 (CET) From: horst@freedict.de Reply-To: horst@freedict.de To: metakit@equi4.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Subject: [Metakit] python metakit / blocked / hashed Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I failed to get a blocked (and hashed) view working with python metakit. - Are there examples available? By the way, some python examples include metakit, others Mk4py. Which one really should be used, or are there no differences? And yet another remark: the python documentation does not mention how to install subviews yet. It would be nice to include that at some point in the future. (I managed to install subviews (looking at tcl), but it would have helped). Thanks, Horst -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From gmcm@hypernet.com Fri Feb 15 09:02:46 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1FF2ha07210 for ; Fri, 15 Feb 2002 09:02:44 -0600 Received: from PARANOIA (204.176.40.107) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0) for ; Fri, 15 Feb 2002 10:02:40 -0500 From: "Gordon McMillan" To: metakit@equi4.com Date: Fri, 15 Feb 2002 10:02:06 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] python metakit / blocked / hashed Reply-to: gmcm@hypernet.com Message-ID: <3C6CDC9E.2807.42FAE64@localhost> Priority: normal In-reply-to: <200202141807.g1EI7Bm00968@eaglesnest.mceggman> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 14 Feb 2002 at 19:07, horst@freedict.de wrote: > I failed to get a blocked (and hashed) view working > with python metakit. - Are there examples available? Probably not. Meaningful examples are harder to write than you might think. Make sure your base view is in the right order (or empty). Create the block / hash from the base view. Now do everything through the block / hash view. > By the way, some python examples include metakit, > others Mk4py. Which one really should be used, or > are there no differences? The only thing metakit adds is dump(). > And yet another remark: the python documentation > does not mention how to install subviews yet. It > would be nice to include that at some point in the > future. (I managed to install subviews (looking at > tcl), but it would have helped). Contributions gratefully accepted. -- Gordon http://www.mcmillan-inc.com/ From mh@pixar.com Fri Feb 15 09:59:51 2002 Received: from pixar.pixar.com (pixar.pixar.com [138.72.10.20]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1FFxoa09241 for ; Fri, 15 Feb 2002 09:59:50 -0600 Received: from postal.pixar.com (postal.pixar.com [138.72.126.101]) by pixar.pixar.com (8.9.3/8.9.3) with ESMTP id HAA13479 for ; Fri, 15 Feb 2002 07:59:44 -0800 (PST) Received: from pixar.com (dreadnok [138.72.16.110]) by postal.pixar.com (8.11.1/8.11.1) with ESMTP id g1FFxaF11735 for ; Fri, 15 Feb 2002 07:59:37 -0800 (PST) Message-ID: <3C6D306F.7090708@pixar.com> Date: Fri, 15 Feb 2002 07:59:43 -0800 From: Mark Harrison User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.7) Gecko/20011221 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com References: <3C6CDC9E.2807.42FAE64@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] newest wikit? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is this the right place for wikit questions? Here are a couple... 1. How can I tell what version of the wiki software I'm running? 2. I think I have the latest, but it does not seem to support the inline display of images... I can make it work on the tclers wiki but not on my local copy. 3. I downloaded the full tclwiki, but cannot get it to run. error messages below. 4. Is the wiki src available? I want to add a "bug" tag that will refer to our internal bug database page, and an "isbn" tag for book references. Many thanks, Mark ~/bin/linux/tclkit bigwiki.tkd don't know how to start 'main.tcl' while executing "error "don't know how to start '$argMain.tcl'"" invoked from within "if {$failed && $errMsg != ""} { error $errMsg $savedInfo puts $errMsg exit 2 }" ("eval" body line 527) invoked from within "eval $script" (file "bigwiki.tkd" line 8) From horst@freedict.de Fri Feb 15 14:59:11 2002 Received: from mail.netbeat.de (mail.netbeat.de [62.208.140.19]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1FKxAa22627 for ; Fri, 15 Feb 2002 14:59:10 -0600 Received: (qmail 19294 invoked from network); 15 Feb 2002 21:33:33 -0000 Received: from dialin-145-254-119-253.arcor-ip.net (HELO eaglesnest.mceggman) (root@145.254.119.253) by mail.netbeat.de with SMTP; 15 Feb 2002 21:33:33 -0000 Received: from freedict.de (localhost [127.0.0.1]) by eaglesnest.mceggman (8.10.2/8.10.2/SuSE Linux 8.10.0-0.3) with ESMTP id g1FHjkQ17142; Fri, 15 Feb 2002 18:45:47 +0100 Message-Id: <200202151745.g1FHjkQ17142@eaglesnest.mceggman> X-Authentication-Warning: eaglesnest.mceggman: Host localhost [127.0.0.1] claimed to be freedict.de Date: Fri, 15 Feb 2002 18:45:44 +0100 (CET) From: horst@freedict.de Reply-To: horst@freedict.de Subject: Re: [Metakit] python metakit / blocked / hashed To: gmcm@hypernet.com cc: metakit@equi4.com In-Reply-To: <3C6CDC9E.2807.42FAE64@localhost> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 15 Feb, Gordon McMillan wrote: > On 14 Feb 2002 at 19:07, horst@freedict.de wrote: > >> I failed to get a blocked (and hashed) view working >> with python metakit. - Are there examples available? > > Probably not. Meaningful examples are harder > to write than you might think. to bad - So I prepared some example code, which failed for my setup. Any hits? #################################################### import sys; sys.path.append('../builds'); import Mk4py; mk = Mk4py print sys.argv[0], '-', 'Mk4py', mk.version, '-', sys.platform from time import time db = mk.storage() vw = db.getas('_B[vw[p1:I,p2:I]]').blocked() map = db.getas('_B[map[_H:I,_R:I]]').blocked() def filldb(n=1000): del vw[:] t0 = time(); for i in xrange(n): vw.append(p1=i, p2=i+i) print 'filldb %d rows: %g sec' % (n, time() - t0) # this find will do a linear scan def find_raw(k,n=1000): t0 = time(); for i in xrange(n): r = vw.find(p1=k) print ' find_raw %d times, key %d -> %d: %g sec' % (n, k, r, time() - t0) for n in (100, 1000, 10000, 100000): filldb(n) find_raw(n/2, 1000000/n) find_raw(n+1, 1000000/n) find_ord(n/2, 1000000/n) #################################################### The output actually is: find3.py - Mk4py 2.4.3 - linux2 filldb 100 rows: 0.00252903 sec Traceback (most recent call last): File "find3.py", line 28, in ? find_raw(n/2, 1000000/n) File "find3.py", line 22, in find_raw r = vw.find(p1=k) ValueError: Object has no usable attributes > > Make sure your base view is in the right order (or > empty). Create the block / hash from the base view. > Now do everything through the block / hash view. > >> By the way, some python examples include metakit, >> others Mk4py. Which one really should be used, or >> are there no differences? > > The only thing metakit adds is dump(). I guess, what I really asked was: which one (Mk4py or metakit) should be used for beeing save in > >> And yet another remark: the python documentation >> does not mention how to install subviews yet. It >> would be nice to include that at some point in the >> future. (I managed to install subviews (looking at >> tcl), but it would have helped). > > Contributions gratefully accepted. Structure definitions consist of a list of properties. Subviews are specified as a sublist of two entries: the name and the list of properties in that subview. Note that subviews add two levels of nesting (see phones in the phonebook example below). w = db.getas("name:S,Address:S,phones:[category:S,phone:S]]") or simply: subviews can be created as in the given example. (see above) > > -- Gordon > http://www.mcmillan-inc.com/ > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit -- Horst@freedict.de Horst Eyermann Germany You need a dictionary? - visit http://www.freedict.de for free (GPL) dictionaries (unix; windows work in progress) For windows, visit http://www.freedict.de/wbuch A article (in German) about dictionary efforts on the net http://www.heise.de/tp/deutsch/inhalt/on/5927/1.html From vaclav.slavik@matfyz.cz Fri Feb 15 13:48:03 2002 Received: from smtp3.vol.cz (smtp3.vol.cz [195.250.128.83]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1FJm1a19459 for ; Fri, 15 Feb 2002 13:48:02 -0600 Received: from there (prahad-3-78.dialup.vol.cz [212.27.197.246]) by smtp3.vol.cz (8.11.6/8.11.3) with SMTP id g1FJltO21509 for ; Fri, 15 Feb 2002 20:47:56 +0100 (CET) (envelope-from vaclav.slavik@matfyz.cz) Message-Id: <200202151947.g1FJltO21509@smtp3.vol.cz> Content-Type: text/plain; charset="iso-8859-1" From: Vaclav Slavik Date: Fri, 15 Feb 2002 20:47:43 +0100 X-Mailer: KMail [version 1.3] MIME-Version: 1.0 To: metakit@equi4.com Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g1FJm1a19459 Subject: [Metakit] hashed views never stored in file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I'm trying to use hashes in MetaKit 2.4.3-33 (which I found a week ago and which is a perfect library for my needs, thanks a lot!) and there's a problem I don't know how to solve: it appears that data stored via hash view are never stored in the storage object! It can be shown by this piece of code: c4_IntProp p1("p1"); c4_StringProp p2("p2"); c4_Storage storage("myfile.dat", true); c4_View data = storage.GetAs("data[key:S,val1:S,val2:I]"); c4_View data_h = storage.GetAs("data_H[_H:I,_R:I]"); c4_View hashed = data.Hash(data_h,1); hashed.Add(p1[1] + p2["one"]); hashed.Add(p1[2] + p2["two"]); storage.Commit(); // myfile.dat has 0 bytes size! It can also be seen in examples/mkhash.cpp when run as "./mkhash h". Same thing will happen, empty file... Am I doing something wrong? Thanks! Vaclav From jcw@equi4.com Sun Feb 17 17:23:29 2002 Received: from [10.0.1.55] (ipc379e83a.dial.wxs.nl [195.121.232.58]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1HNNSa22909 for ; Sun, 17 Feb 2002 17:23:29 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] hashed views never stored in file Date: Mon, 18 Feb 2002 00:13:19 +0100 Message-Id: <20020217231319.29933@triqs.com> In-Reply-To: <200202151947.g1FJltO21509@smtp3.vol.cz> References: <200202151947.g1FJltO21509@smtp3.vol.cz> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Vaclav Slavik wrote: > c4_IntProp p1("p1"); > c4_StringProp p2("p2"); You've just defined properties "p1" and "p2". > c4_Storage storage("myfile.dat", true); > c4_View data = storage.GetAs("data[key:S,val1:S,val2:I]"); So you better use those names in the stored view as well. Replace the above line with something compatible, e.g.: c4_View data = storage.GetAs("data[p1:I,p2:S]"); > c4_View data_h = storage.GetAs("data_H[_H:I,_R:I]"); > c4_View hashed = data.Hash(data_h,1); > hashed.Add(p1[1] + p2["one"]); > hashed.Add(p1[2] + p2["two"]); > storage.Commit(); // myfile.dat has 0 bytes size! -jcw From jcw@equi4.com Sun Feb 17 17:23:31 2002 Received: from [10.0.1.55] (ipc379e83a.dial.wxs.nl [195.121.232.58]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1HNNUa22912 for ; Sun, 17 Feb 2002 17:23:30 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] newest wikit? Date: Mon, 18 Feb 2002 00:21:51 +0100 Message-Id: <20020217232151.32392@triqs.com> In-Reply-To: <3C6D306F.7090708@pixar.com> References: <3C6D306F.7090708@pixar.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Harrison wrote: >Is this the right place for wikit questions? Yes, though probably only few readers will be interested/involved. Might also try the TclersWiki mailing list on yahoo. >Here are a couple... > >1. How can I tell what version of the wiki software I'm running? There's no versioning, other than checking script file dates. >2. I think I have the latest, but it does not seem to support the > inline display of images... I can make it work on the tclers wiki > but not on my local copy. The Tcl'ers Wiki was customized a few times. I haven't gotten around to consolidating changes so far. >3. I downloaded the full tclwiki, but cannot get it to run. error > messages below. Same answer as 2, but even nastier: recent changes to the Tcl'ers Wiki broke the ability to use it in a general context, because it now stores data differently. This is in anticipation of more changes, and a planned move to a new "kiwi" based re-implementation of the wikit code (www.equi4.com/kiwi/). >4. Is the wiki src available? I want to add a "bug" tag that will > refer to our internal bug database page, and an "isbn" tag > for book references. Wikit is coded in Tcl, all sources are stored inside as a "scripted document". There's some documentation on that concept, and on how to use such beasts. The one complicating factor is that wikit precedes the current VFS-based design of scripted documents, so scripts are being stored in ways which differ from one revision to the next. For example: >~/bin/linux/tclkit bigwiki.tkd >don't know how to start 'main.tcl' > while executing >"error "don't know how to start '$argMain.tcl'"" This is a remnant of the old system. Name your file "wikit.something" and try again. The startup script looks at the way it was started - and that has to contain the prefix "wikit" to work. -jcw From jcw@equi4.com Sun Feb 17 17:26:42 2002 Received: from [10.0.1.55] (ipc379e83a.dial.wxs.nl [195.121.232.58]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1HNQfa22980 for ; Sun, 17 Feb 2002 17:26:41 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] how can i change the DBF files to the metakit files type? Date: Mon, 18 Feb 2002 00:25:04 +0100 Message-Id: <20020217232504.20279@triqs.com> In-Reply-To: <200202141326.g1EDQMa29450@trixie.triqs.com> References: <200202141326.g1EDQMa29450@trixie.triqs.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: cloudshadow wrote: >how can i change the DBF files to the metakit files type? There's an old example of a little C++ program which can read DBase files and present the content as a MK view. It's at: http://www.equi4.com/pub/mk/metakit-1.99/win-examples.zip You could use that to copy the records to an MK datafile. Or, if you use Python or Tcl, you could look for packages which read DBF files, and write a little script which copies the records. -jcw From mark@markroseman.com Sun Feb 17 17:39:49 2002 Received: from admin.cgocable.net (admin.cgocable.net [24.226.1.21]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1HNdna23213 for ; Sun, 17 Feb 2002 17:39:49 -0600 Received: from eyak-can4.markroseman.com (d141-160-232.home.cgocable.net [24.141.160.232]) by admin.cgocable.net (8.10.0/8.9.3) with ESMTP id g1HNg8n05135 for ; Sun, 17 Feb 2002 18:42:08 -0500 (EST) Message-Id: <5.1.0.14.0.20020217183342.03a96ff0@demo.teamwave.com> X-Sender: X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Sun, 17 Feb 2002 18:38:02 -0500 To: metakit@equi4.com From: Mark Roseman Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: [Metakit] metakit for tcl tutorial documentation Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: i've put together some new, more tutorial-style documentation intended to introduce metakit to tcl developers. i hope this will serve as a good complement to jean-claude's existing documentation. you can find a first draft at: http://www.markroseman.com/tcl/ i would greatly appreciate any comments or suggestions you might have on this draft. thanks mark From wlb@multisysid.com Mon Feb 18 12:32:58 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1IIWva13553 for ; Mon, 18 Feb 2002 12:32:57 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id OAA24064 for ; Mon, 18 Feb 2002 14:32:14 -0500 Message-ID: <000501c1b8ab$fbe323d0$6101a8c0@WB02> From: "wlb" To: Date: Mon, 18 Feb 2002 14:42:12 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Indexes as of 2.4.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I see support for indexes in the header files, and apparently in the bindings for tcl and python also. Alas, no regression tests demonstrate their usage. Do they work? Any docs/tests/samples? I will even read the code if someone could confirm that they are supposed to work. wlb From Murat_Berk@bmc.com Mon Feb 18 16:51:33 2002 Received: from creeper.bmc.com (camaro.bmc.com [198.207.223.231]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1IMpXa29551; Mon, 18 Feb 2002 16:51:33 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by creeper.bmc.com (8.10.2/8.10.2) with ESMTP id g1IMp9c07428; Mon, 18 Feb 2002 16:51:09 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Mon, 18 Feb 2002 16:51:26 -0600 Message-ID: From: "Berk, Murat" To: "'Jean-Claude Wippler'" , metakit Date: Mon, 18 Feb 2002 16:51:23 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Commit problems Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude, We are experiencing commit problems w/ this simple test case using latest CVS version. When we commit database, the view is not committed. i.e If I move the addition of item before Commit() call, everything works. But if we commit after GetAs() w/o inserting any row, none of the rows get inserted. (It behaves as if it is detached.) Murat #include "mk4.h" #include "mk4str.h" #include int main() { c4_StringProp pName ("name"); c4_StringProp pCountry ("country"); c4_Storage storage ("myfile.dat", true); c4_View vAddress = storage.GetAs("address[name:S,country:S]"); storage.Commit(); c4_Row row; pName (row) = "John Williams"; pCountry (row) = "UK"; vAddress.Add(row); storage.Commit(); // Should have 1 row, have 0 rows return 0; } From wlb@multisysid.com Mon Feb 18 17:07:41 2002 Received: from nutmeg.multisysid.com ([63.105.180.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1IN7ea30466 for ; Mon, 18 Feb 2002 17:07:40 -0600 Received: from WB02 ([192.168.1.97]) by nutmeg.multisysid.com (8.9.3/8.8.7) with SMTP id TAA25824 for ; Mon, 18 Feb 2002 19:06:57 -0500 Message-ID: <000801c1b8d2$5d3e6e20$6101a8c0@WB02> From: "wlb" To: Date: Mon, 18 Feb 2002 19:16:56 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Metakit] Indexes as of 2.4.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: ...thinking aloud. In remap.cpp line around line 930, it says: bool c4_IndexedViewer::SetItem(int row_, int col_, const c4_Bytes& buf_) { ... if (keyMod) { // TODO adjust index } Which means, indexes may work, but don't update a key value, I believe. I really don't see when the map is stored. It is stored, isn't it? Or is this just a large index in memory for now? wlb From slok00@yahoo.com Tue Feb 19 09:32:48 2002 Received: from smtp012.mail.yahoo.com (smtp012.mail.yahoo.com [216.136.173.32]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1JFWma21432 for ; Tue, 19 Feb 2002 09:32:48 -0600 Received: from mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 19 Feb 2002 15:32:44 -0000 Message-Id: <5.1.0.14.0.20020219231806.02d76e78@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Tue, 19 Feb 2002 23:30:23 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-7F3E5D62; boundary="=======72EC1BC4=======" Subject: [Metakit] Mk4tcl.so installation question.. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======72EC1BC4======= Content-Type: text/plain; x-avg-checked=avg-ok-7F3E5D62; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit I downloaded Mk4tcl.so.gz and uncompress it to a directory "/home/user/test" I created a simple script mytest.tcl and put it in the same directory as Mk4tcl.so in "/home/user/test" When I run "wish mytest.tcl", I got the following messages ==== Error in startup script: can't find Mk4tcl while executing "package require Mk4tcl" {file "mytest.tcl" line 1} ===== How or where should I put Mk4tcl.so? Thanks. YekSoon --=======72EC1BC4======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-7F3E5D62 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.323 / Virus Database: 180 - Release Date: 2/8/2002 --=======72EC1BC4=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From jcw@equi4.com Tue Feb 19 10:44:02 2002 Received: from [10.0.1.55] (ipc379eb0a.dial.wxs.nl [195.121.235.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1JGi1a24465 for ; Tue, 19 Feb 2002 10:44:01 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Indexes as of 2.4.3 Date: Tue, 19 Feb 2002 17:20:09 +0100 Message-Id: <20020219162009.8468@triqs.com> In-Reply-To: <000801c1b8d2$5d3e6e20$6101a8c0@WB02> References: <000801c1b8d2$5d3e6e20$6101a8c0@WB02> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: wlb wrote: >...thinking aloud. In remap.cpp line around line 930, it says: > >bool c4_IndexedViewer::SetItem(int row_, int col_, const c4_Bytes& buf_) >{ > >... > > if (keyMod) > { > // TODO adjust index > } > > >Which means, indexes may work, but don't update a key value, I believe. Exactly. This is the reason I did not take this further - the indexed mapping view is able to deal with the case of adding/deleting rows, and is ok as long as the key is not changed. Not sure just how far I got with inserting a second row with duplicate keys, but I think that was ok. But given that it is not yet solving the problem fully, >I really don't see when the map is stored. It is stored, isn't it? Or is >this just a large index in memory for now? Whether it persist or not depends on whether the index view is setup to persist prior to being used in the indexed mapping viewer. So yes, you can make it persist - by creating an appropriate view beforehand - or you can choose to use it only in a transient way. -jcw From jcw@equi4.com Tue Feb 19 10:44:04 2002 Received: from [10.0.1.55] (ipc379eb0a.dial.wxs.nl [195.121.235.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1JGi3a24471 for ; Tue, 19 Feb 2002 10:44:03 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Mk4tcl.so installation question.. Date: Tue, 19 Feb 2002 17:23:18 +0100 Message-Id: <20020219162318.13425@triqs.com> In-Reply-To: <5.1.0.14.0.20020219231806.02d76e78@pop.mail.yahoo.com> References: <5.1.0.14.0.20020219231806.02d76e78@pop.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Lok Yek Soon wrote: >I downloaded Mk4tcl.so.gz and uncompress it to a directory "/home/user/test" > >I created a simple script mytest.tcl and put it in the same directory as >Mk4tcl.so in "/home/user/test" >When I run "wish mytest.tcl", I got the following messages > >==== >Error in startup script: can't find Mk4tcl > while executing >"package require Mk4tcl" > {file "mytest.tcl" line 1} >===== > > >How or where should I put Mk4tcl.so? As with any shared library for Tcl, you could put it in the current dir and use "load ./Mk4tcl.so", or you could use "mk_pkgIndex" to create the required "pkgIndex.tcl" for using MK as a Tcl package. -jcw From teamwork@andinet.com Tue Feb 19 11:26:16 2002 Received: from kepler.andinet.com (kepler.andinet.com [208.221.129.18]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1JHQFa27120 for ; Tue, 19 Feb 2002 11:26:16 -0600 Received: by kepler.andinet.com (5.1.056) id 3C7289EF000000B6 for metakit@equi4.com; Tue, 19 Feb 2002 12:23:38 -0500 Message-ID: <3C727D8D00000035@kepler.andinet.com> Date: Tue, 19 Feb 2002 12:23:38 -0500 From: teamwork@andinet.com To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g1JHQFa27120 Subject: [Metakit] database sizes Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi: metakit last version one question? append records this way db = storage('demo',2) f = db.getas('frequents[drinker,bar,perweek:I]') for i in range(1000): f.append(drinker='lola', bar='lolas', perweek=6) db.commit() database size (demo) is 13kb but!!!!!!!!!!!!!!! append records this way db = storage('demo',2) f = db.getas('frequents[drinker,bar,perweek:I]') for i in range(1000): f.append(drinker='lola', bar='lolas', perweek=6) db.commit() database size (demo) is 6207kb help me please!!! libraries and programs: PythonWin 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32. MK 2.4.3 metakit.py Juan Carlos Rodriguez O. Team Work Ltda. Bogota Colombia. quirogaco@yahoo.es teamwork@andinet.com From jcw@equi4.com Tue Feb 19 11:33:28 2002 Received: from [10.0.1.55] (ipc379e8de.dial.wxs.nl [195.121.232.222]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1JHXRa27522 for ; Tue, 19 Feb 2002 11:33:27 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] database sizes Date: Tue, 19 Feb 2002 18:31:23 +0100 Message-Id: <20020219173123.9969@triqs.com> In-Reply-To: <3C727D8D00000035@kepler.andinet.com> References: <3C727D8D00000035@kepler.andinet.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: teamwork@andinet.com wrote: > for i in range(1000): > f.append(drinker='lola', bar='lolas', perweek=6) > db.commit() > >database size (demo) is 13kb You're opening in mode 2 (commit extend), which *only* writes changes at the end of the datafile. > for i in range(1000): > f.append(drinker='lola', bar='lolas', perweek=6) > db.commit() A thousand copies of the entire database. Opening in mode 1 will lead to a (much) smaller file. -jcw From quirogaco@yahoo.es Wed Feb 20 13:54:06 2002 Received: from web20107.mail.yahoo.com (web20107.mail.yahoo.com [216.136.226.44]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1KJs6a20497 for ; Wed, 20 Feb 2002 13:54:06 -0600 Message-ID: <20020220195405.73745.qmail@web20107.mail.yahoo.com> Received: from [200.13.218.176] by web20107.mail.yahoo.com via HTTP; Wed, 20 Feb 2002 20:54:05 CET Date: Wed, 20 Feb 2002 20:54:05 +0100 (CET) From: =?iso-8859-1?q?Juan=20carlos=20Rodriguez?= To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] dynamic - find Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi: JCW thank's, quick response before question. other question: situation one: n = 'FERRARI PAOLO' vw.find(name = n) work fine. how make dynamic find?, example: a = 'name' n = 'FERRARI PAOLO' vw.find(a = n) don't work exist other way? diferent of : eval("vw.find(name = 'FERRARI PAOLO')") thank's Juan Carlos Rodriguez O. Bogota, Colombia quirogaco@yahoo.es teamwork@andinet.com _______________________________________________________________ Do You Yahoo!? Yahoo! Messenger Comunicación instantánea gratis con tu gente. http://messenger.yahoo.es From jerry@usd1.com Wed Feb 20 15:37:45 2002 Received: from bfish.hurrah.com (bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1KLbja25873 for ; Wed, 20 Feb 2002 15:37:45 -0600 Received: from 12-225-195-8.client.attbi.com (12-225-195-8.client.attbi.com [12.225.195.8]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id g1KLbgx25102 for ; Wed, 20 Feb 2002 13:37:42 -0800 Date: Wed, 20 Feb 2002 13:36:01 -0800 From: Jerry McRae X-Mailer: The Bat! (v1.53d) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <921316543.20020220133601@usd1.com> To: Metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] .so files Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I tried to install Mk4py.so on the Linus (v 2.2.18) box for my website. Just copied in the latest. Date of 2/1/02, Linux version. I try import Mk4py (from Python 2.1) and get the message: ImportError: libstdc++-libc6.2-2.so.3: cannot open shared object file: No such file or directory I don't know Linus or C++, so I have no clue. I can import other librarys. This just happens to be used at the core of my product, so I cannot go any farther. I thought all I would have to do is copy the current .so file instead of my .dll on my system. Mk4py.so permissions: -rw-r--r Ideas? -- Thanks, Jerry McRae MetaKit 2.4.3, Python 2.1, wxPython 2.2.5, Win98, 384M RAM From lvirden@cas.org Fri Feb 22 05:57:02 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1MBv2a29179 for ; Fri, 22 Feb 2002 05:57:02 -0600 Received: from lwv26awu.cas.org (lwv26awu [134.243.55.72]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g1MBusb07910 for ; Fri, 22 Feb 2002 06:56:54 -0500 (EST) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g1MBuqn22429; Fri, 22 Feb 2002 06:56:52 -0500 (EST) Date: Fri, 22 Feb 2002 06:56:52 -0500 (EST) From: "Larry W. Virden" Message-Id: <20020222065652.AAB22420@cas.org> To: Metakit Subject: [Metakit] Call For Papers: 9th Annual Tcl/Tk Conference, Sept 16-20, 2002 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Call For Papers 9th Annual Tcl/Tk Conference September 16-20, 2002 Vancouver, BC, Canada We are pleased to announce the 9th Annual Tcl/Tk conference, to be held September 16-20, 2002, in Vancouver, BC, Canada. This conference is a forum to: * bring together Tcl/Tk researchers and practitioners * publish and present current work involving Tcl/Tk * learn about the latest developments in Tcl/Tk * plan for future Tcl/Tk related developments The conference program will include paper presentations, tutorials, Birds of a Feather (BOF) sessions and invited key-note talks. This call invites you to submit extended abstracts to be considered for the conference program. The conference schedule will consist of 2 days of tutorials (Monday - Tuesday) and 3 days for the main conference (Wednesday - Friday). Submission of Extended Abstracts The conference provides an opportunity to report on original Tcl/Tk research. The audience is practitioners and researchers who are intermediate or experienced users of Tcl/Tk. For this reason, reports on experiences and applications must draw out lessons for other Tcl/Tk developers. Topics include, but are not limited to: * System extensions * Novel Tcl/Tk-based applications * Experience reports on building applications in Tcl/Tk * Comparative evaluations of Tcl/Tk and other languages or toolkits for building applications * Use of different programming paradigms in Tcl/Tk and proposals for new directions. * New areas of exploration for the Tcl/Tk language Extended Abstracts should be written in English and 600 to 1000 words long (1-2 pages). Omit extraneous or redundant information. Length is not a direct factor in judging the quality of the submission. Send submissions as plain text to no later than April 28, 2002. Authors of accepted abstracts will have until August 20, 2002 to submit their final paper for the inclusion in the conference proceedings. The proceedings will be made available on CD-ROM, so extra materials like code samples are welcome. The authors will have 20-25 minutes to present the paper at the conference. The program committee will review and evaluate papers according to the following criteria: * Quantity and quality of novel content * Relevance and interest to the Tcl/Tk community * Suitability of content for presentation at the conference Proposals may report on commercial or non-commercial systems, but those with only blatant marketing content will not be accepted. Application and experience papers need to strike a balance between background on the application domain and the relevance of Tcl/Tk to the application. Application and experience papers should clearly explain how the application or experience illustrates a novel use of Tcl/Tk, and what lessons the Tcl/Tk community can derive from the application or experience to apply to their own development efforts. Papers accompanied by non-disclosure agreement forms will be returned to the author(s) unread. All submissions are held in the highest confidentiality prior to publication in the Proceedings, both as a matter of policy and in accord with the U. S. Copyright Act of 1976. Other Forms of Participation The program committee also welcomes proposals for panel discussions of up to 90 minutes. Proposals should include a list of confirmed panelists, a title and format, and a panel description with position statements from each panelist. Panels should have no more than four speakers, including the panel moderator, and should allow time for substantial interaction with attendees. Panels are not presentations of related research papers. Slots for Works-in-Progress (WIP) presentations and Birds-of-a-Feather sessions (BOFs) are available on a first-come, first-served basis starting in August 2002. Specific instructions for reserving WIP and BOF time slots will be provided in the registration information available in June 2002. Some WIP and BOF time slots will be held open for on-site reservation, so we encourage all attendees with interesting work in progress to consider presenting that work at the conference. Registration Information More information on the conference will be available in Spring 2002 at the conference web site (http://www.tcl.tk/community/tcl2002/) and published on various Tcl/Tk-related information channels. To keep in touch with conference announcements and Tcl events in general, subscribe to the tcl-announce list at: http://listserv.activestate.com/mailman/mysubs?show=announce by entering your email and selecting Tcl-announce. Conference Committee Jeff Hobbs ActiveState Corp General Chair Gerald Lester Aspen Technology Program Co-chair Kevin Kenny GE Global Research Center Program Co-chair Andreas Kupries ActiveState Corp Andrej Vckovski NetCetera Clif Flynt Noumena Corp Ken Jones Avia Training Larry Virden Chemical Abstracts Service (CAS) Mac Cody Raytheon Company Mark Roseman Sonexis, Inc. Steve Landers Digital Smarties Contact Information tcl2002@-SPAM-.tcl.tk http://www.tcl.tk/community/tcl2002/ -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Fri Feb 22 07:25:47 2002 Received: from [10.0.1.55] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g1MDPka31920 for ; Fri, 22 Feb 2002 07:25:46 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] .so files Date: Fri, 22 Feb 2002 05:15:09 -0800 Message-Id: <20020222131509.13028@triqs.com> In-Reply-To: <921316543.20020220133601@usd1.com> References: <921316543.20020220133601@usd1.com> X-Mailer: CTM PowerMail 3.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jerry McRae wrote: >I tried to install Mk4py.so on the Linus (v 2.2.18) box for my website. >Just copied in the latest. Date of 2/1/02, Linux version. I try import >Mk4py (from Python 2.1) and get the message: > >ImportError: libstdc++-libc6.2-2.so.3: cannot open shared object file: No >such file or directory > >I don't know Linus or C++, so I have no clue. > >I can import other librarys. This just happens to be used at the core of my >product, so I cannot go any farther. I thought all I would have to do >is copy the current .so file instead of my .dll on my system. On Linux, there are several versions of the C/C++ runtime library (glibc 2.0/2.1/2.2) - it looks like you are loading a build of MK which does not match your environment. You can use "ldd Mk4py.so" to find out. You will have to either rebuild MK on your machine (it's not too hard, something like "cd builds; ../unix/configure --enable-python; make" is likely to work). Or you can upgrade your Linux system. Or you can try to find someone who has a similar setup and has a build ready or is willing to do it for you. Last option is to look for an slightly older build of MK on the www.equi4.com website, which does work with your runtime - you'll need to download and try. Regards, Jean-Claude From andreas.muegge@gmx.de Mon Feb 25 06:54:22 2002 Received: from mx0.gmx.net (mx0.gmx.net [213.165.64.100]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1PCsLa09395 for ; Mon, 25 Feb 2002 06:54:21 -0600 Received: (qmail 12626 invoked by uid 0); 25 Feb 2002 12:54:10 -0000 Date: Mon, 25 Feb 2002 13:54:10 +0100 (MET) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: Metakit MIME-Version: 1.0 X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [217.194.32.8] Message-ID: <3022.1014641650@www53.gmx.net> X-Mailer: WWW-Mail 1.5 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: [Metakit] Quick way to do a substring search on a property Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, in my application the metakit database looks like this: recipe[index:I,name:S,cat1:S,cat2:S,cat3:S,cat4:S,cat5:S,count:S,descr:S] The user should be able to find a "name" by entering parts of it - a typical substring search where searching for "potatoe" will find all recipes with "potatoe" somewhere in the name. My first approach was like this: c4_StringProp pName("name"); for (int i=0; i < nRecipesCount; i++) { const char* szFound = pName ( myView[i]); // ... some tolower() and strstr() stuff } Unfortunately this turns out to be rather slow, on my P3 I can only parse 2000 recipes per second - 25 sec just for retrieving 50'000 names from the database is not acceptable. Is there a better way to do it? Ciao, Andreas From slok00@yahoo.com Tue Feb 26 04:18:25 2002 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1QAIPa32445 for ; Tue, 26 Feb 2002 04:18:25 -0600 Received: from slok00 (AUTH LOGIN) at mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 26 Feb 2002 10:18:20 -0000 Message-Id: <5.1.0.14.0.20020226180255.026bd560@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Tue, 26 Feb 2002 18:03:54 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-68004E7; boundary="=======40956BAE=======" Subject: [Metakit] Metakit support for images? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======40956BAE======= Content-Type: text/plain; x-avg-checked=avg-ok-68004E7; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit Hi, Does MetaKit supports images? If so, how can I do that? Thanks YekSoon --=======40956BAE======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-68004E7 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 --=======40956BAE=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From slok00@yahoo.com Wed Feb 27 08:39:12 2002 Received: from smtp015.mail.yahoo.com (smtp015.mail.yahoo.com [216.136.173.59]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1REdCa13800 for ; Wed, 27 Feb 2002 08:39:12 -0600 Received: from slok00 (AUTH LOGIN) at mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 27 Feb 2002 14:37:30 -0000 Message-Id: <5.1.0.14.0.20020227223043.02db0320@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Wed, 27 Feb 2002 22:35:00 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-3851DF4; boundary="=======B3A1155=======" Subject: [Metakit] tclKit and tclHttpd problem Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======B3A1155======= Content-Type: text/plain; x-avg-checked=avg-ok-3851DF4; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit My apologies if this is not the right mailing list to post with regards to TclKit. But, I am having trouble finding it. I am trying to turn tclHttpd into a single scripted document. I downloaded tclhttpd33.zip from http://tcl.activestate.com:8002/resource/software/tools/www/ and the latest tclkit.exe and follows the instructions at http://equi4.com/pub/tk/examples/tclhttpd.README to turn tclHttpd into a scripted document. However, after all the steps, when I tried running the scripted document (tclhttpd.bat) that I create , I got the following dialog error: == Fatal Error Don't know how to run tclhttpd for tclhttpd === The only thing that I didn't do is download tcllib as I believe it is already included in tclhttpd33.zip btw, I''m trying this on win2k without any wish or Tcl installed. Any ideas what could be wrong? Thanks YekSoon --=======B3A1155======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-3851DF4 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 --=======B3A1155=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From nicolasb@mail.maich.gr Wed Feb 27 09:33:28 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g1RFXRa15978 for ; Wed, 27 Feb 2002 09:33:27 -0600 Received: FROM mail.maich.gr BY mail-server ; Wed Feb 27 17:53:17 2002 +0200 Message-ID: <3C7D00FA.6000407@mail.maich.gr> Date: Wed, 27 Feb 2002 17:53:30 +0200 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Lok Yek Soon CC: metakit@equi4.com Subject: Re: [Metakit] tclKit and tclHttpd problem References: <5.1.0.14.0.20020227223043.02db0320@pop.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I just tried it and had no problem.. Here's what I did 1. Extract tclhttpd3.3 to c:\ 2. rename to tclhttpd3.3.vfs 3. Add a file in c:\tclhttpd.vfs\bin called main.tcl containing set env(TCL_HTTPD_LIBRARY) [file join $scripdoc::self lib] source [file join $scripdoc::self bin httpd.tcl] 4. Create a batch file, sdxcreate.bat containing @tclkit sdx %1 %2 %3 %4 %5 %6 %7 %8 %9 5. Copy sdx.bin into c:\ and renameto sdx 6. Create a file called tclhttpd.bat in c:\ with the following @tclkit tclhttpd %1 %2 %3 %4 %5 %6 %7 %8 %9 7. in c:\ type in sdxcreate fs2sd tclhttpd after the Tk window comes up, you know it's done 8. Run tclhttpd.bat Note, tclkit.exe should be in c:\, or at least where system can findit or edit the batch file with fullpath. Note, the current tclhttpd3.3.1, looks for tcllib in the same level as tclhttpd. So either adjust as necessary, or use tcllib0.8 Hope this helps, Nicolas Lok Yek Soon wrote: > My apologies if this is not the right mailing list to post with > regards to TclKit. > But, I am having trouble finding it. > > I am trying to turn tclHttpd into a single scripted document. > I downloaded tclhttpd33.zip from > http://tcl.activestate.com:8002/resource/software/tools/www/ > and the latest tclkit.exe and follows the instructions at > http://equi4.com/pub/tk/examples/tclhttpd.README > to turn tclHttpd into a scripted document. > > However, after all the steps, when I tried running the scripted > document (tclhttpd.bat) that I create , > I got the following dialog error: > > == > Fatal Error > Don't know how to run tclhttpd for tclhttpd > === > > The only thing that I didn't do is download tcllib as I believe it is > already included in tclhttpd33.zip > > btw, I''m trying this on win2k without any wish or Tcl installed. > > Any ideas what could be wrong? > > Thanks > YekSoon > > >------------------------------------------------------------------------ > > >--- >Outgoing mail is certified Virus Free. >Checked by AVG anti-virus system (http://www.grisoft.com). >Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 > From slok00@yahoo.com Fri Mar 1 02:38:27 2002 Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g218cRa05837 for ; Fri, 1 Mar 2002 02:38:27 -0600 Received: from slok00 (AUTH LOGIN) at mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 1 Mar 2002 08:31:57 -0000 Message-Id: <5.1.0.14.0.20020301161228.03372db0@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Fri, 01 Mar 2002 16:29:23 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-2435493D; boundary="=======7876096=======" Subject: [Metakit] tclhttpd's cgi-bin not working in scripted document. Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======7876096======= Content-Type: text/plain; x-avg-checked=avg-ok-2435493D; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit The cgi-bin in the scripted doc "tclhttpd" that I created does not work. It return the following message when I request for the env.cgi that comes with tclhttpd. Erro Message on the browser as follows: = Got the error Bad Request while trying to obtain /cgi-bin/env.cgi/extra/path couldn't execute "/home/test/tclhttpd3.3-dist/tclhttpd/htdocs/cgi-bin/env.cgi": not a directory ===== These are the steps I do to create the scripted document tclhttpd 1. Extract tclhttpd3.3-dist.tar.gz to /home/test 2. cd tclhttpd3.3-dist 3. rename tclhttpd3.3 to tclhttpd.vfs 4. mv tcllib0.8 to tclhttpd.vfs/lib 5. add a file in c:\tclhttpd.vfs\bin called main.tcl containing set env(TCL_HTTPD_LIBRARY) [file join $scripdoc::self lib] source [file join $scripdoc::self bin httpd.tcl] 6.. copy tclkit and sdx.bin into tclhttpd3.3-dist 7. type "tclkit sdx.bin fs2sd tclhttpd" to create scripted doc. 8. start scripted document's webserver by typing ./tclhttpd 9. access the localsite http://localhost:8015/cgi-bin/env.cgi The cgi works when run from the webserver which is not in scripted doc. Is there any steps that needs to be done to make this works? Thanks YekSoon after the Tk window comes up, you know it's done 8. Run tclhttpd.bat --=======7876096======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-2435493D Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 --=======7876096=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From DavidA@ActiveState.com Sat Mar 2 00:05:01 2002 Received: from smtp1.ActiveState.com (gw.activestate.com [209.17.183.249]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g22650a04159 for ; Sat, 2 Mar 2002 00:05:01 -0600 Received: from smtp3.ActiveState.com (smtp3.ActiveState.com [192.168.3.19]) by smtp1.ActiveState.com (8.11.6/8.11.6) with ESMTP id g2264oV29799 for ; Fri, 1 Mar 2002 22:04:51 -0800 Received: from ActiveState.com (vpn6.ActiveState.com [192.168.3.236]) by smtp3.ActiveState.com (8.11.6/8.11.6) with ESMTP id g2264nL13910 for ; Fri, 1 Mar 2002 22:04:49 -0800 Message-ID: <3C806C94.FBF30FF6@ActiveState.com> Date: Fri, 01 Mar 2002 22:09:24 -0800 From: David Ascher X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: metakit Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Filtered-By: PerlMx makes it fast and easy. See http://www.ActiveState.com/Products/PerlMx/Header Subject: [Metakit] concurrent access layer for metakit/python? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm looking at using metakit in a web site (currently under Apache/Windows/"simple CGI", but I'd like the solution to work in a variety of web env'ts, such as mod_python). The site isn't high-traffic, so it's ok to serialize database accesses even relatively 'primitively'. My primary concern is to avoid data corruption due to the unpredictable timings of web hits, not performance. Are there any tricks that people use to make coding that kind of access as easy as possible? --david From jcw@equi4.com Sun Mar 3 03:27:10 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g239R9a25797 for ; Sun, 3 Mar 2002 03:27:09 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] concurrent access layer for metakit/python? Date: Sun, 3 Mar 2002 10:26:57 +0100 Message-Id: <20020303092657.21014@triqs.com> In-Reply-To: <3C806C94.FBF30FF6@ActiveState.com> References: <3C806C94.FBF30FF6@ActiveState.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: David Ascher wrote: >I'm looking at using metakit in a web site (currently under >Apache/Windows/"simple CGI", but I'd like the solution to work in a >variety of web env'ts, such as mod_python). > >The site isn't high-traffic, so it's ok to serialize database accesses >even relatively 'primitively'. My primary concern is to avoid data >corruption due to the unpredictable timings of web hits, not >performance. > >Are there any tricks that people use to make coding that kind of access >as easy as possible? Hi David, I can't think of a simple way to do it in the general case. For simple CGI (which is what I use in the Tcl'ers Wiki, plus page caching), I just lock/open-db/use/close/unlock. With a bit of logic to get rid of dangling locks (the database is safe): store PID and check /proc/PID to break locks. For more demanding uses, I'd use TclHttpd/AolServer (or a Python equivalent in your case) as server and run MK in it - that way all page accesses are properly serialized. There is a considerable amount of caching in MK, meaning that the lock really has to around the entire open/use/close group. Lastly, there is a "commit-extend mode" (open mode 2), which a single process can use while all others open read-only. That grows a datafile *very* quickly, but is a way as well and fast. It is badly documented, but solid nevertheless. The idea is that each process can do: open ro handle req if just reading if writing is needed: close lock, waiting open extend alter close release The only thing you need to do is occasionally compact the file: open ro serialize contents to a new file close replace original with newly created compact file The solution I have in mind for high-performance concurrent access is a combination of commit-extend and commit-aside (which saves diffs efficiently), but the implementation of that is not finished. This design will allow multiple-reader single writer with little overhead and no locking at all. For now, if I were you, I'd start off with the CGI model and full locks in Python. Changes to the locking model later on need IMO not affect much of the application coding. -jcw From jcw@equi4.com Sun Mar 3 10:17:22 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g23GHLa06207 for ; Sun, 3 Mar 2002 10:17:21 -0600 From: Jean-Claude Wippler To: metakit Date: Sun, 3 Mar 2002 17:17:15 +0100 Message-Id: <20020303161715.16068@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] TclKit status Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: For those interested in TclKit (the single-file Tcl/Tk/MK runtime), here is a brief status report: - From now on, TclKit will have IncrTcl included (but not incrTk). - The latest source has been checked into the "tclkit" project (i.e. as a CVS project on equi4.com). It is no longer part of the MetaKit distribution - this seems more logical, long term. - TclKit now builds using the latest *unmodified* Tcl, Tk, IncrTcl, TclVFS, and MetaKit distributions. I intend to track these. - The 8.4-26 release that has been around for months in /pub/tk/ uses a pure-tcl VFS scheme. The new system uses the latest core Virtual File System as now present in the official Tcl 8.4a4. - One effect is that all file calls will be fully compatible with plain Tcl systems. Another effect is that Vince Darley's TclVFS is used for scripted documents and that his http:// logic works. See for some examples. Lastly, the performance is up (way up for large dirs, due to more caching). - There are new binaries in http://www.equi4.com/previews/tclkit.*, as of today. These builds for Linux and Windows really are very close to becoming the "official" TclKit release. Still to do: * tweak how shared libs are placed into temp and cleaned up * some more testing to verify good backward compatibility * a recently discovered readlink bug must be fixed (in VFS) - The current builds no longer have threading enabled, but this is temporary. TclKit fully supports Tcl's threading (including MK). I just need to figure out config/build issues again and test it. - Daniel Steffen recently built a TclKit for Mac, using Carbon for use with OS 8/9 and with X, but also "fat", i.e. PPC *and* 68K. I'm tracking MacTcl issues on , and intend to create a MacOS X version of TclKit in the near future. Both MetaKit and IncrTcl work fine, I just need to get to it. - In an attempt to keep (regain?) my sanity, I will only provide builds for Windows, Linux, and Macintosh from now on. Binaries for other platforms will be gratefully accepted and included in the equi4.com download area - as will all source code tweaks. My goal is still to have TclKit (and MetaKit) running on as many platforms as possible - but I have to depend on others for that. If you use TclKit under Windows or Linux, please give the latest builds a good workout. I will fix issues as soon as possible, and hope to announce solid revisions as soon as the latest Tcl/Tk revs are frozen (I *think* it's going to be called 8.4b1). -jcw From rjent@rjent.pair.com Sun Mar 3 20:39:23 2002 Received: from portal2.isz ([204.184.227.216]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g242dLa18355 for ; Sun, 3 Mar 2002 20:39:22 -0600 Received: (from nobody@localhost) by portal2.isz (8.11.6/8.11.6) id g242hOa06298; Sun, 3 Mar 2002 20:43:24 -0600 (CST) (envelope-from rjent@rjent.pair.com) Date: Sun, 3 Mar 2002 20:43:24 -0600 (CST) Message-Id: <200203040243.g242hOa06298@portal2.isz> X-Authentication-Warning: portal2.isz: nobody set sender to rjent@rjent.pair.com using -f From: "RJ Ent." To: metakit@equi4.com X-Mailer: NeoMail 1.25 with (Patch-2a by Nelson) X-IPAddress: 192.168.2.3 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Subject: [Metakit] Multiuser version? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Greetings, I am curious about the potential for a multiuser version of metakit. Many thanks! From slok00@yahoo.com Tue Mar 5 01:22:08 2002 Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g257M8a21639 for ; Tue, 5 Mar 2002 01:22:08 -0600 Received: from slok00 (AUTH login) at mcns94.docsis216.scvmaxonline.com.sg (HELO reciproc-z9e5n5.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 5 Mar 2002 07:22:07 -0000 Message-Id: <5.1.0.14.0.20020305151625.02d527b0@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Tue, 05 Mar 2002 15:17:46 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-29D74A98; boundary="=======71D1585B=======" Subject: [Metakit] tclkit.exe and tclkitsh.exe Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======71D1585B======= Content-Type: text/plain; x-avg-checked=avg-ok-29D74A98; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit On windows, what's the difference between tclkit.exe and tclkitsh.exe? Under what circumstances do you use each? Thanks YekSoon --=======71D1585B======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-29D74A98 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.325 / Virus Database: 182 - Release Date: 2/19/2002 --=======71D1585B=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From nicole@bwh.harvard.edu Tue Mar 5 17:12:21 2002 Received: from clifford.bwh.harvard.edu (clifford.bwh.harvard.edu [134.174.9.41]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g25NCLa02579 for ; Tue, 5 Mar 2002 17:12:21 -0600 Received: from tango.bwh.harvard.edu (tango [134.174.9.35]) by clifford.bwh.harvard.edu (8.10.2+Sun/8.11.0) with ESMTP id g25NBOu12039 for ; Tue, 5 Mar 2002 18:11:25 -0500 (EST) Received: from localhost (nicole@localhost) by tango.bwh.harvard.edu (8.10.2+Sun/8.11.0) with ESMTP id g25NBOS10252 for ; Tue, 5 Mar 2002 18:11:24 -0500 (EST) Date: Tue, 5 Mar 2002 18:11:24 -0500 (EST) From: Nicole Aucoin To: metakit@equi4.com Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] MetaKit and cgi scripting Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, I'm fairly new to tcl and I'm almost positive that this is a question with a simple answer, but I've been unable to dig it up (web searches and a grep of the archives of this list) or figure it out on my own, so please bear with me. I have a fairly low access volume web script that I'd like to have use Mk4tcl. I can run the cgi script that calls the tcl script with no problems from my Solaris account, but when it's called from the web page with a user id of nobody, it can't load Mk4tcl.so. I catch the following error: couldn't load file "./Mk4tcl.so": ld.so.1: /local/os/bin/tclsh: fatal: libstdc++.so.2.10.0: open failed: No such file or directory I've tried setting env(LD_LIBRARY_PATH) and env(TCLLIBPATH) to the directory that this file lives in, but no success. I downloaded the latest Solaris Mk4tcl.so from the web site, will I need to compile the source and link it statically to get this to work? thanks, Nicole From reza@psych.utoronto.ca Wed Mar 6 11:13:25 2002 Received: from brodmann (CPE000102e3f968.cpe.net.cable.rogers.com [24.157.195.240]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g26HDLa13415 for ; Wed, 6 Mar 2002 11:13:25 -0600 Received: from [127.0.0.1] by brodmann (ArGoSoft Mail Server Freeware, Version 1.804a (1.8.0.4)); Wed, 6 Mar 2002 12:13:20 -0500 Date: Wed, 6 Mar 2002 12:13:18 -0500 From: Reza Habib X-Mailer: The Bat! (v1.53d) Personal Reply-To: Reza Habib Organization: Rotman Research Institute X-Priority: 3 (Normal) Message-ID: <17513781146.20020306121318@psych.utoronto.ca> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] possible bug? [MK 2.4.3, win2000, mingw 2.95.3-6] Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi. I'm new to metakit. I think I have found a bug or maybe I'm doing something wrong (the subject shows the version and platform I'm using). Any help would be appreciated. I have a data structure that looks like this: #define FORMAT "Subject[Name:S,Number:I,Age:I,PresRate:I,ISI:I,Trials:I,List[Words_Section:I,Replace_Section:I],Random:I,Stimuli[Section:I,ListEntered:I,ListDeparted:I,Type:S,Word:S]]" I then get a view to this structure like this: c4_View vSubject = sDataFile.GetAs(FORMAT); and then a view to the subview 'Stimuli' like this: c4_ViewProp pStimuli = (c4_ViewProp&)vSubject.NthProperty(8); c4_View vWords = pStimuli(vSubject[1]); Then I add some rows to this view and use select to get two different views to this view like this: c4_View vSection = vWords.Select(pSection[i+1] + pType["Study"]); c4_View vReplace = vWords.Select(pSection[0] + pType["Replacement"]); I then want to change some of the values in these views. I do so like this: int r = rand() % vSection.GetSize(); //To get a random row c4_Row TempRow; //Get a row and set some properties pSection(TempRow) = i+1; pListEntered(TempRow) = CurrentTrial; pListDeparted(TempRow) = 0; pType(TempRow) = "Study"; pWord(TempRow) = pWord(vReplace[0]); vSection[r] = TempRow; //Assign row to view sDataFile.Commit(); //Save changes c4_Row TempRow2; //Same as above pSection(TempRow2) = 0; pListEntered(TempRow2) = 1;//pListEntered(vSection[r]); pListDeparted(TempRow2) = CurrentTrial; pType(TempRow2) = "Replaced"; pWord(TempRow2) = "Test";//pWord(vSection[r]); vReplace[0] = TempRow2; sDataFile.Commit(); Now, the first commit works as expected. That is the value in the correct view is changed, however, the second commit that is supposed to change the subview pointed to by vReplace ends up changing the subview pointed to by vSection. Here is a bit of the output from dump: This is what the file looks like before anything happens. 0: subview 'Stimuli' VIEW 196 rows = Section:I ListEntered:I ListDeparted:I Type:S Word:S 0: 1 1 0 'Study' 'purely' 1: 1 1 0 'Study' 'vacuum' 2: 1 1 0 'Study' 'warmth' 3: 1 1 0 'Study' 'affair' 4: 2 1 0 'Study' 'settle' 5: 2 1 0 'Study' 'worthy' .... 12: 0 0 0 'Replacement' 'phrase' 13: 0 0 0 'Replacement' 'uneasy' 14: 0 0 0 'Replacement' 'utopia' 15: 0 0 0 'Replacement' 'resist' Then, after the first commit, the first part (rows 1 - 5) should be changed but not the second part. This output shows that this happens: 0: 1 2 0 'Study' 'phrase' <------- this line changed 1: 1 1 0 'Study' 'vacuum' 2: 1 1 0 'Study' 'warmth' 3: 1 1 0 'Study' 'affair' 4: 2 1 0 'Study' 'settle' 5: 2 1 0 'Study' 'worthy' .... 12: 0 0 0 'Replacement' 'phrase' 13: 0 0 0 'Replacement' 'uneasy' 14: 0 0 0 'Replacement' 'utopia' 15: 0 0 0 'Replacement' 'resist' Now, after the second commit, the first row in the second part (row 12) should be changed. As you can see, this is not what happens, but rather the first row of the first part is changed: 0: 0 2 2 'Replaced' 'phrase' <----This was changed but shouldn't have been 1: 1 1 0 'Study' 'vacuum' 2: 1 1 0 'Study' 'warmth' 3: 1 1 0 'Study' 'affair' 4: 2 1 0 'Study' 'settle' 5: 2 1 0 'Study' 'worthy' .... 12: 0 0 0 'Replacement' 'phrase' <---- This should have been changed 13: 0 0 0 'Replacement' 'uneasy' 14: 0 0 0 'Replacement' 'utopia' 15: 0 0 0 'Replacement' 'resist' I did various tests to check to see whether I was doing something wrong. If I use the GetIndexOf() function to find the row number pointed to by vReplace[0], I get the correct answer either when I check within vReplace (i.e. vReplace.GetIndexOf(vReplace[0] = 0) or within the vWords view (i.e. vWords.GetIndexOf(vReplace[0]) = 12). Now, have I done something wrong in assigning to the row of this view or is this a bug? And if the latter, is there a workaround? Thanks (and sorry for the long email). Reza From squeezeboxguy@yahoo.co.uk Wed Mar 6 17:14:55 2002 Received: from smtp018.mail.yahoo.com (smtp018.mail.yahoo.com [216.136.174.115]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g26NEsa02765 for ; Wed, 6 Mar 2002 17:14:54 -0600 Received: from squeezeboxguy (AUTH login) at pc-80-195-147-244-du.blueyonder.co.uk (HELO ?192.168.57.2?) (squeezeboxguy@80.195.147.244) by smtp.mail.vip.sc5.yahoo.com with SMTP; 6 Mar 2002 22:08:39 -0000 From: Jeremy Cope To: metakit@equi4.com Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/1.0.1 Date: 06 Mar 2002 22:08:00 +0000 Message-Id: <1015452522.13955.9.camel@lancre> Mime-Version: 1.0 Subject: [Metakit] Mk4py.dll and Python 2.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Is there available somewhere a build of Mk4py.dll for Python 2.2 on windows? I had some trouble with "Cannot find a required DLL" type errors, and was stumped until I had a look at the dll itself and discovered that it expects python 2.1 (ie python21.dll). I tried a quick hack with a hex editor, but this doesn't seem to have worked, so I need another solution. Please could someone help? Thanks in advance, Jez :) PS Thanks to everyone involved in development of metakit, it works just great on my Linux box and the interface is so easy to use! -- Fabricate diem, punc. Jeremy Cope, UK _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From genec@goldsmithsinc.com Thu Mar 7 07:44:49 2002 Received: from mail.goldsmithsinc.com ([64.66.89.67]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g27Dima30814 for ; Thu, 7 Mar 2002 07:44:48 -0600 Received: from genec (genec [10.1.1.100]) by mail.goldsmithsinc.com (Switch-2.0.0/Switch-2.0.0) with SMTP id g27DXQq29006; Thu, 7 Mar 2002 07:33:26 -0600 Message-ID: <003101c1c5dd$8dc2c240$6401010a@genec> From: "Gene Christian" To: "Jeremy Cope" , References: <1015452522.13955.9.camel@lancre> Subject: Re: [Metakit] Mk4py.dll and Python 2.2 Date: Thu, 7 Mar 2002 07:39:48 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I ran into the same thing, and somebody on this list recommended compiling it using mingw (which was something of an experience :-). I can email the dll to you if you would like (it runs 453K, so let me know if you me to send it). I have compiled a couple of python extensions using mingw, and it has worked very well so far. If you try it, you will want the msys (?) system that emulates a unix shell environment (so you can run the configure scripts that come with many packages). Gene ----- Original Message ----- From: "Jeremy Cope" To: Sent: Wednesday, March 06, 2002 4:08 PM Subject: [Metakit] Mk4py.dll and Python 2.2 > Is there available somewhere a build of Mk4py.dll for Python 2.2 on > windows? I had some trouble with "Cannot find a required DLL" type > errors, and was stumped until I had a look at the dll itself and > discovered that it expects python 2.1 (ie python21.dll). I tried a quick > hack with a hex editor, but this doesn't seem to have worked, so I need > another solution. Please could someone help? > > Thanks in advance, Jez :) > > PS Thanks to everyone involved in development of metakit, it works just > great on my Linux box and the interface is so easy to use! > -- > Fabricate diem, punc. > > Jeremy Cope, UK > > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From squeezeboxguy@yahoo.co.uk Thu Mar 7 08:00:04 2002 Received: from web14704.mail.yahoo.com (web14704.mail.yahoo.com [216.136.224.121]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g27E03a31602 for ; Thu, 7 Mar 2002 08:00:03 -0600 Message-ID: <20020307140002.63250.qmail@web14704.mail.yahoo.com> Received: from [212.219.131.2] by web14704.mail.yahoo.com via HTTP; Thu, 07 Mar 2002 06:00:02 PST Date: Thu, 7 Mar 2002 06:00:02 -0800 (PST) From: Jeremy Cope Subject: Re: [Metakit] Mk4py.dll and Python 2.2 To: Gene Christian , metakit@equi4.com In-Reply-To: <003101c1c5dd$8dc2c240$6401010a@genec> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --- Gene Christian wrote: > I ran into the same thing, and somebody on this list recommended > compiling > it using mingw (which was something of an experience :-). I can > email the > dll to you if you would like (it runs 453K, so let me know if you me > to send > it). Yes, please could you email it to me, this address will be fine. I would have given mingw a go myself, but I don't have enough free rein on the college computers, and I run Linux at home. One of these days I'll get round to setting up a cross compiler. But I'll shut up now before I go seriously offtopic :) Thanks very much, Jez ===== "Fabricate diem, punc" - Motto, Ankh-Morpork City Watch Jeremy Cope ICQ: 33490230; MSN: jezcope@yahoo.com; Jabber: jezcope@jabber.com __________________________________________________ Do You Yahoo!? Try FREE Yahoo! Mail - the world's greatest free email! http://mail.yahoo.com/ From jcw@equi4.com Sun Mar 10 17:23:41 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANNea03568 for ; Sun, 10 Mar 2002 17:23:41 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Mk4py.dll and Python 2.2 Date: Mon, 11 Mar 2002 00:23:18 +0100 Message-Id: <20020310232318.17175@triqs.com> In-Reply-To: <20020307140002.63250.qmail@web14704.mail.yahoo.com> References: <20020307140002.63250.qmail@web14704.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jeremy Cope wrote: Gene Christian wrote: [we want Mk4py for Py2.2] The build I had was still erroneously linking against 2.1 - I've now removed 2.1 so my MSVC project only finds 2.2. New builds have been placed in Note that this is based on the latest CVS code, not 2.4.3 - but I don't want to move to a new release number just yet. -jcw P.S. Anyone using MK on Win32 with NTFS should check the CHANGE log - there has been a bug fix which addresses this particular combination. From jcw@equi4.com Sun Mar 10 17:46:34 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkXa03970 for ; Sun, 10 Mar 2002 17:46:33 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] MetaKit and cgi scripting Date: Mon, 11 Mar 2002 00:37:52 +0100 Message-Id: <20020310233753.1574@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicole Aucoin wrote: >I have a fairly low access volume web script that I'd like to have use >Mk4tcl. I can run the cgi script that calls the tcl script with no >problems from my Solaris account, but when it's called from the web page >with a user id of nobody, it can't load Mk4tcl.so. I catch the following >error: > >couldn't load file "./Mk4tcl.so": ld.so.1: /local/os/bin/tclsh: fatal: >libstdc++.so.2.10.0: open failed: No such file or directory > >I've tried setting env(LD_LIBRARY_PATH) and env(TCLLIBPATH) to the >directory that this file lives in, but no success. I downloaded the latest >Solaris Mk4tcl.so from the web site, will I need to compile the source and >link it statically to get this to work? I'm assuming it works one way and not the other, all on the same machine. Your webserver may be restricting access in many different ways. If it does a "chroot()", then that means you are running in a new area - you will need to check that it contains all the shared libraries. One way to find out, is to code a CGI script which returns the output of "ldd Mk4tcl.so", i.e. a list of shared library linkages and how they are being resolved. Or there is a UID permission issue. Neither of these areally related to MK, I'm afraid. It may well be that no other C++ shared libs have been run from the webserver, in which case this is a C++ / Webserver issue. An alternative is to have a look at TclKit, which combines Tcl, Tk, and MK in a single standalone executable. That build has no dynamic linkage (othe rthan std X11). There are builds for Solaris, see www.equi4.com/ pub/tk/. -jcw From jcw@equi4.com Sun Mar 10 17:46:36 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkZa03974 for ; Sun, 10 Mar 2002 17:46:35 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Metakit support for images? Date: Mon, 11 Mar 2002 00:38:08 +0100 Message-Id: <20020310233808.27229@triqs.com> In-Reply-To: <5.1.0.14.0.20020226180255.026bd560@pop.mail.yahoo.com> References: <5.1.0.14.0.20020226180255.026bd560@pop.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Lok Yek Soon wrote: >Does MetaKit supports images? If so, how can I do that? Images are binary data. Store them in fields of type "B", e.g. in Tcl: mk::view layout db.images {name data:B} -jcw From jcw@equi4.com Sun Mar 10 17:46:37 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkaa03977 for ; Sun, 10 Mar 2002 17:46:37 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Multiuser version? Date: Mon, 11 Mar 2002 00:38:17 +0100 Message-Id: <20020310233817.23131@triqs.com> In-Reply-To: <200203040243.g242hOa06298@portal2.isz> References: <200203040243.g242hOa06298@portal2.isz> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: RJ Ent. wrote: >I am curious about the potential for a multiuser version of metakit. The only multi-user scenario's supported by metakit are client/server (i.e. all access of MK datafiles in a single server process/thread), or the use of commit-extend (with suitable extra locking). See the MK wiki for more info: -jcw From jcw@equi4.com Sun Mar 10 17:46:39 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkca03980 for ; Sun, 10 Mar 2002 17:46:38 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] tclkit.exe and tclkitsh.exe Date: Mon, 11 Mar 2002 00:38:25 +0100 Message-Id: <20020310233825.19574@triqs.com> In-Reply-To: <5.1.0.14.0.20020305151625.02d527b0@pop.mail.yahoo.com> References: <5.1.0.14.0.20020305151625.02d527b0@pop.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Lok Yek Soon wrote: >On windows, what's the difference between tclkit.exe and tclkitsh.exe? >Under what circumstances do you use each? - tclkit includes Tk, and runs as GUI application - tclkitsh does not include Tk, and runs in console mode -jcw From jcw@equi4.com Sun Mar 10 17:46:41 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ANkea03984 for ; Sun, 10 Mar 2002 17:46:40 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] possible bug? [MK 2.4.3, win2000, mingw 2.95.3-6] Date: Mon, 11 Mar 2002 00:46:26 +0100 Message-Id: <20020310234626.6218@triqs.com> In-Reply-To: <17513781146.20020306121318@psych.utoronto.ca> References: <17513781146.20020306121318@psych.utoronto.ca> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Reza Habib wrote: >Hi. I'm new to metakit. I think I have found a bug or maybe I'm >doing something wrong (the subject shows the version and platform I'm >using). Any help would be appreciated. > >I have a data structure that looks like this: (Please keep your examples short - I can't deal with too much code...) >#define FORMAT "Subject[Name:S,Number >:I,Age:I,PresRate:I,ISI:I,Trials:I,Li >st[Words_Section:I,Replace_Section:I] >,Random:I,Stimuli[Section:I,ListEntered:I,ListDeparted:I,Type:S,Word:S]]" > >I then get a view to this structure like this: > > c4_View vSubject = sDataFile.GetAs(FORMAT); [...] > vSection[r] = TempRow; //Assign row to view > sDataFile.Commit(); //Save changes [...] > vReplace[0] = TempRow2; > sDataFile.Commit(); > >Now, the first commit works as expected. That is the value in the >correct view is changed, however, the second commit that is supposed >to change the subview pointed to by vReplace ends up changing the >subview pointed to by vSection. This may be the dreaded problem of MK losing track of "attached" vs. "unattached" views. I have two questions: - what happens if you comment out the first commit? - what happens if you do a commit after the GetAs? actually, to be really sure, replace it by the following: sDataFile.GetAs(FORMAT); sDataFile.Commit(); c4_View vSubject = sDataFile.View("Subject"); does that solve the commit weirdness? -jcw From jcw@equi4.com Sun Mar 10 18:39:21 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2B0dJa04805 for ; Sun, 10 Mar 2002 18:39:20 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Mk4Tcl concurrency Date: Mon, 11 Mar 2002 01:39:14 +0100 Message-Id: <20020311003914.30196@triqs.com> In-Reply-To: <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> References: <5.1.0.14.0.20020203130753.03878cc0@roseman.myservers.org> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Mark Roseman wrote: >As an example, our web-based app runs under AOLserver, and we >keep a lot of central data held in Metakit databases. The >multiple AOLserver threads make socket connections to a separate >single-threaded process running the database, so in effect >requests have to wait their turn to be processed. Despite some >fairly substantial queries and reasonable numbers of simultaneous >users, we've never had issues with hugely perceptible delays on >the client side. Note that 2.4.x builds of Mk4tcl (can't remember when it was introduced) support safe access to it from a threaded Tcl environment. The contention is resolved through one big mutex preventing entry to all but one thread inside Mk4tcl. Simple, but effective (single process still, just for MT). This only has value with "mk::file open db file.dat -shared", which will expose "db" to all interpreters. In fact, this could be used as a way to transfer data between interps (note that file.dat can be omitted, thus turning this into an in-mem shared data structure). -jcw From jerry@usd1.com Mon Mar 11 13:41:54 2002 Received: from bfish.hurrah.com (bfish.hurrah.com [208.151.247.36]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2BJfsa07949 for ; Mon, 11 Mar 2002 13:41:54 -0600 Received: from 12-225-195-8.client.attbi.com (12-225-195-8.client.attbi.com [12.225.195.8]) by bfish.hurrah.com (8.11.3/8.11.3) with ESMTP id g2BJfoG15859 for ; Mon, 11 Mar 2002 11:41:50 -0800 Date: Mon, 11 Mar 2002 11:26:00 -0800 From: Jerry McRae X-Mailer: The Bat! (v1.53d) Personal Reply-To: Jerry McRae Organization: usd1 X-Priority: 3 (Normal) Message-ID: <17100167637.20020311112600@usd1.com> To: Metakit MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] subclassing MK types in py2.2 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I was experimenting with Python 2.2 and attempting to define a class inheriting RowRef and View types, and was not successful. I got this: >>> class vwtest(mk.RowRefType): ... pass ... Traceback (most recent call last): File "", line 1, in ? TypeError: type 'PyRowRef' is not an acceptable base type Is this possible? I would like to add a smart dictionary, but if it is not there, to use the original methods. -- Thanks, Jerry McRae MetaKit 2.4.3, Python 2.2, wxPython 2.2.5, Win98, 384M RAM From gmcm@hypernet.com Mon Mar 11 14:45:32 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2BKjWa12044 for ; Mon, 11 Mar 2002 14:45:32 -0600 Received: from PARANOIA (204.176.40.68) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Mon, 11 Mar 2002 15:45:39 -0500 From: "Gordon McMillan" To: Jerry McRae Date: Mon, 11 Mar 2002 15:45:51 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] subclassing MK types in py2.2 Reply-to: gmcm@hypernet.com CC: metakit@equi4.com Message-ID: <3C8CD12F.5277.810307A4@localhost> Priority: normal In-reply-to: <17100167637.20020311112600@usd1.com> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 11 Mar 2002 at 11:26, Jerry McRae wrote: > Hi, > > I was experimenting with Python 2.2 and attempting > to define a class inheriting RowRef and View types, > and was not successful. I got this: [snip] No. These are still old-fashioned types. Mk4py already plays some type object games. I don't know what it will take to make them new-style type objects. -- Gordon http://www.mcmillan-inc.com/ From marc.vollmer@handshake.de Mon Mar 11 15:21:49 2002 Received: from hs-gate.handshake.de (root@hs-gate.handshake.de [193.141.176.10]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2BLLna14024 for ; Mon, 11 Mar 2002 15:21:49 -0600 Received: from linux (hs3-200.handshake.de [193.141.176.200]) by hs-gate.handshake.de (8.9.3/8.9.3) with SMTP id WAA03392 for ; Mon, 11 Mar 2002 22:21:44 +0100 From: Marc Vollmer Reply-To: marc.vollmer@handshake.de To: metakit@equi4.com Date: Mon, 11 Mar 2002 22:04:24 +0100 X-Mailer: KMail [version 1.0.29.2] Content-Type: text/plain MIME-Version: 1.0 Message-Id: <02031122252200.04170@linux> Content-Transfer-Encoding: 8bit Subject: [Metakit] Newbie: How can I find a property? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I don't understand, how I can find a value of a proberty: My database is: Food.db: 1 properties element[bez:S,kal:F,fet:F,koh:F,eiw:F] VIEW 1 rows = element:V 0: subview 'element' VIEW 2 rows = bez:S kal:F fet:F koh:F eiw:F 0: 'Maultaschen schwäbisch' 180.1 11.6 8.9 10 1: 'Butter' 741.9 83.2 0.6 0.7 I search the Property "Butter", so I open the database and search: const char * tmp1; c4_Storage fooddb ("Food.db", false); c4_View v1 = fooddb.View("element"); c4_RowRef row = v1.ElementAt (0); c4_Property prop = v1.NthProperty(0); for (int i=0;i; Mon, 11 Mar 2002 15:40:00 -0600 Received: from squeezeboxguy (AUTH login) at pc-80-195-147-244-du.blueyonder.co.uk (HELO ?192.168.57.2?) (squeezeboxguy@80.195.147.244) by smtp.mail.vip.sc5.yahoo.com with SMTP; 11 Mar 2002 21:39:59 -0000 Subject: Re: [Metakit] Newbie: How can I find a property? From: Jeremy Cope To: marc.vollmer@handshake.de, metakit@equi4.com In-Reply-To: <02031122252200.04170@linux> References: <02031122252200.04170@linux> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Evolution/1.0.2 Date: 11 Mar 2002 21:39:28 +0000 Message-Id: <1015882810.8524.14.camel@lancre> Mime-Version: 1.0 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Phew! I'm impressed, I wouldn't have managed that! To start off with, you need to use the specifically type property classes, in this case c4_StringProp, rather than the base c4_Property. I don't think it's necessary to use NthProperty() at all unless you're writing something generic like the dump utility in the source distribution. I would go about it like this: c4_Storage fooddb("Food.db", false); c4_View v1 = fooddb.View("element"); c4_StringProp prop("bez"); c4_View v2 = v1.Select(prop["Butter"]); Then, you'll have a view v2 which contains all the rows where the "bez" property is "Butter", to do with what you will. Now I just hope the code above actually compiles and does what I think it does, in order to save myself from total embarressment. Try looking at the examples on the website (http://www.equi4.com/metakit/intro.html#codingExamples). The syntax isn't obvious at first, but it's got a rundown of how to do what. Hope that helps, Jez :) -- Fabricate diem, punc. Jeremy Cope, UK _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From marc.vollmer@handshake.de Tue Mar 12 15:13:08 2002 Received: from hs-lan.handshake.de (hs-lan.handshake.de [193.141.176.12]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2CLD7a15455 for ; Tue, 12 Mar 2002 15:13:08 -0600 Received: from linux (hs3-247.handshake.de [193.141.176.247]) by hs-lan.handshake.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with SMTP id WAA05594 for ; Tue, 12 Mar 2002 22:12:10 +0100 From: marc.vollmer@handshake.de To: metakit@equi4.com Subject: Re: [Metakit] Newbie: How can I find a property? Date: Tue, 12 Mar 2002 21:44:42 +0100 X-Mailer: KMail [version 1.0.29.2] Content-Type: text/plain References: <1015882810.8524.14.camel@lancre> In-Reply-To: <1015882810.8524.14.camel@lancre> MIME-Version: 1.0 Message-Id: <02031221524200.00796@linux> Content-Transfer-Encoding: 8bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks. Now, I understand a litte more. Am Mon, 11 Mär 2002 schrieb Jeremy Cope: > c4_Storage fooddb("Food.db", false); > c4_View v1 = fooddb.View("element"); > c4_StringProp prop("bez"); > c4_View v2 = v1.Select(prop["Butter"]); It works fine, but I search the substring "Butter" or "butter". I added mydatabase: Food.db: 1 properties element[bez:S,kal:F,fet:F,koh:F,eiw:F] VIEW 1 rows = element:V 0: subview 'element' VIEW 3 rows = bez:S kal:F fet:F koh:F eiw:F 0: 'Maultaschen schwäbisch' 180.1 11.6 8.9 10 1: 'Butter' 741.9 83.2 0.6 0.7 2: 'Butterkäse' 299 23.5 0 21.7 By Select I find only item 1. And by Find: int count; count = v1.Find(prop1["Butter"]); count = v1.Find(prop1["Butter"]); The result is item 1 and item 1, not item 1 and item 2. :-( By Search the result is 0 and 0 :-(( How can I find the item 1 and 2? 1: 'Butter' 741.9 83.2 0.6 0.7 2: 'Butterkäse' 299 23.5 0 21.7 > Try looking at the examples on the website > (http://www.equi4.com/metakit/intro.html#codingExamples). The syntax > isn't obvious at first, but it's got a rundown of how to do what. This was a good tip, I have looked only in the HTML-Docs. From jcw@equi4.com Tue Mar 12 17:17:07 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2CNH6a24853 for ; Tue, 12 Mar 2002 17:17:06 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Newbie: How can I find a property? Date: Wed, 13 Mar 2002 00:16:54 +0100 Message-Id: <20020312231654.25608@triqs.com> In-Reply-To: <02031221524200.00796@linux> References: <02031221524200.00796@linux> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g2CNH6a24853 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: marc.vollmer@handshake.de wrote: [...] > VIEW 3 rows = bez:S kal:F fet:F koh:F eiw:F > 0: 'Maultaschen schwäbisch' 180.1 11.6 8.9 10 > 1: 'Butter' 741.9 83.2 0.6 0.7 > 2: 'Butterkäse' 299 23.5 0 21.7 > >By Select I find only item 1. And by Find: > > int count; > count = v1.Find(prop1["Butter"]); > count = v1.Find(prop1["Butter"]); > >The result is item 1 and item 1, not item 1 and item 2. :-( > >By Search the result is 0 and 0 :-(( > >How can I find the item 1 and 2? > > 1: 'Butter' 741.9 83.2 0.6 0.7 > 2: 'Butterkäse' 299 23.5 0 21.7 c4_StringProp pBez("bez"); for (int i = 0; i < v1.GetSize(); ++i) if (stristr(pBez (v1[i]), "butter") != NULL) puts("bingo"); (Assuming "stristr" is a case-insensitive version of strstr - I don't think it exists, but you can code it up, surely). This looks like brute force. It is. But as you will see, the time to perform such tight loops is usually surprisingly good (though O(N)). -jcw From dave@primco.org Thu Mar 14 03:15:14 2002 Received: from minerva.host4u.net (minerva.host4u.net [216.71.64.49]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2E9FDa32761 for ; Thu, 14 Mar 2002 03:15:13 -0600 Received: from redcloud (adsl-63-193-248-16.dsl.snfc21.pacbell.net [63.193.248.16]) by minerva.host4u.net (8.11.6/8.11.6) with ESMTP id g2E9EIC01775 for ; Thu, 14 Mar 2002 03:14:18 -0600 From: "David Primmer" To: Date: Thu, 14 Mar 2002 01:11:29 -0800 Message-ID: <000001c1cb38$3d93fb80$6401a8c0@redcloud> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] how to do substring selections Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I'm using python and I have a column called 'utc_datetime' with data like this: 2002-02-22T03:40:26-08:00 And I'd like to select rows based on a substring of utc_datetime but I don't know how to do this in metakit language. I've tried select(utc_datetime[0:10]=2002-02-22) but that didn't work of course. I'm generally very confused as to how to use the criteria. And what type of objects are they? I know that a view is can be used like a list but what about properties? I was disappointed to find out that I couldn't do python code inside the structure definition and it looks like I can't do any python inside the criteria. All I've seen is the overly simple name='John' examples. C'mon! I'm frequently doing selects on substrings in SQL. Have I just missed the examples in the docs or on this list? Should I switch to MetaSQL for this function? Starting to get really frustrated with metakit. Just doesn't seem like anyone's using it with python. Seems like there'd be better docs if there were. davep From slok00@yahoo.com Thu Mar 14 05:52:43 2002 Received: from smtp014.mail.yahoo.com (smtp014.mail.yahoo.com [216.136.173.58]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g2EBqga08293 for ; Thu, 14 Mar 2002 05:52:42 -0600 Received: from slok00 (AUTH login) at mcns94.docsis216.scvmaxonline.com.sg (HELO hermes.yahoo.com) (slok00@202.156.216.94) by smtp.mail.vip.sc5.yahoo.com with SMTP; 14 Mar 2002 11:52:41 -0000 Message-Id: <5.1.0.14.0.20020314194846.00b0cb78@pop.mail.yahoo.com> X-Sender: slok00@pop.mail.yahoo.com X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Thu, 14 Mar 2002 19:51:23 +0800 To: metakit@equi4.com From: Lok Yek Soon Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-6FA557B3; boundary="=======20103AF5=======" Subject: [Metakit] tclkit - scripted doc partial working Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: --=======20103AF5======= Content-Type: text/plain; x-avg-checked=avg-ok-6FA557B3; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit on win2k, I create a scripted doc version of tclhttpd using tclkit.exe the scripted tclhttpd can serve static html files but requesting a standard cgi that comes with it gives me the following error: eg. If I request for env.cgi, which should dumps the environment variables gives me this error dialog: == Error in startup script Couldn't read file "c:/tclhttpd/htdocs/cgi-bin/env.cgi": no such file or directory" === What could be the problem? The main web server script "httpd.tcl" works fine, but a simple "env.cgi" doesn't. Both have "exec tclsh8.3" at the top of the script. YekSoon --=======20103AF5======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-6FA557B3 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.336 / Virus Database: 188 - Release Date: 3/11/2002 --=======20103AF5=======-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com From gmcm@hypernet.com Thu Mar 14 07:01:07 2002 Received: from hypernet.com (media1.hypernet.com [204.176.40.2]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ED17a12017 for ; Thu, 14 Mar 2002 07:01:07 -0600 Received: from PARANOIA (204.176.40.75) by hypernet.com with ESMTP (Eudora Internet Mail Server 3.0); Thu, 14 Mar 2002 08:01:20 -0500 From: "Gordon McMillan" To: "David Primmer" , Date: Thu, 14 Mar 2002 08:01:31 -0500 MIME-Version: 1.0 Subject: Re: [Metakit] how to do substring selections Reply-to: gmcm@hypernet.com Message-ID: <3C9058DB.9530.8ECCFF01@localhost> Priority: normal In-reply-to: <000001c1cb38$3d93fb80$6401a8c0@redcloud> X-mailer: Pegasus Mail for Windows (v4.01) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On 14 Mar 2002 at 1:11, David Primmer wrote: > I'm using python and I have a column called > 'utc_datetime' with data like this: > > 2002-02-22T03:40:26-08:00 > > And I'd like to select rows based on a substring of > utc_datetime but I don't know how to do this in > metakit language. > > I've tried select(utc_datetime[0:10]=2002-02-22) but > that didn't work of course. Use the select range form: select({'utc_datetime':'2002-02-22'}, {'utc_datetime':'2002-02-22\377'}) > I'm generally very confused as to how to use the > criteria. And what type of objects are they? Dictionaries of the form {propname=value}, or keyword args propname=value. In sort (and others where value is not applicable), you pass in property objects. You can use view.propname, or construct a property object (2 properties are equal if they have the same name and type). > I know that a view is can be used like a list but > what about properties? What about them? > I was disappointed to find > out that I couldn't do python code inside the > structure definition and it looks like I can't do > any python inside the criteria. Metakit is a C++ library. Use view.filter if you want complex selection logic. You can then use remapwith to get a subset. > All I've seen is the > overly simple name='John' examples. C'mon! I'm > frequently doing selects on substrings in SQL. Have > I just missed the examples in the docs or on this > list? > > Should I switch to MetaSQL for this function? MkSQL doesn't do LIKE, either. > Starting to get really frustrated with metakit. Just > doesn't seem like anyone's using it with python. Not true. > Seems > like there'd be better docs if there were. As with most open source projects, complaints outnumber contributions. -- Gordon http://www.mcmillan-inc.com/ From dave@primco.org Thu Mar 14 07:07:22 2002 Received: from minerva.host4u.net (minerva.host4u.net [216.71.64.49]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ED7Ma12374 for ; Thu, 14 Mar 2002 07:07:22 -0600 Received: from redcloud (adsl-63-193-248-16.dsl.snfc21.pacbell.net [63.193.248.16]) by minerva.host4u.net (8.11.6/8.11.6) with ESMTP id g2ED6QX01817 for ; Thu, 14 Mar 2002 07:06:26 -0600 From: "David Primmer" To: Date: Thu, 14 Mar 2002 05:03:28 -0800 Message-ID: <000301c1cb58$a58aeb70$6401a8c0@redcloud> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-reply-to: Subject: [Metakit] RE: how to do substring selections - now sortrev Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Well I figured this one out for r in myview: if r.utc_datetime[0:10] == '2002-02-22': print r.utc_datetime now my question is, how to do a reverse sort? There's no detail on the arguments of sortrev and google can find no example of where it's used. In my case, I don't want some of the rows in natural order and some reversed, I want them all reversed. BTW, Why isn't there a simple view.sort(property,order) where order is descending or ascending? The docs for the C++ implementation state: This sorts descending on A: sorted = view.SortOnReverse(A, A) but using this argument syntax for python gives me sortedview = myview.sortrev(myview.utc_datetime,myview.utc_datetime) TypeError: Not a sequence Thanks for any help. > -----Original Message----- > From: David Primmer [mailto:dave@primco.org] > Sent: Thursday, March 14, 2002 1:11 AM > To: 'metakit@equi4.com' > Subject: how to do substring selections > > I'm using python and I have a column called 'utc_datetime' with data like > this: > > 2002-02-22T03:40:26-08:00 > > And I'd like to select rows based on a substring of utc_datetime but I > don't know how to do this in metakit language. > > I've tried select(utc_datetime[0:10]=2002-02-22) but that didn't work of > course. > > I'm generally very confused as to how to use the criteria. And what type > of objects are they? I know that a view is can be used like a list but > what about properties? I was disappointed to find out that I couldn't do > python code inside the structure definition and it looks like I can't do > any python inside the criteria. All I've seen is the overly simple > name='John' examples. C'mon! I'm frequently doing selects on substrings in > SQL. Have I just missed the examples in the docs or on this list? > > Should I switch to MetaSQL for this function? > > Starting to get really frustrated with metakit. Just doesn't seem like > anyone's using it with python. Seems like there'd be better docs if there > were. > > davep From squeezeboxguy@yahoo.co.uk Thu Mar 14 07:58:19 2002 Received: from web14708.mail.yahoo.com (web14708.mail.yahoo.com [216.136.224.125]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g2EDwJa15071 for ; Thu, 14 Mar 2002 07:58:19 -0600 Message-ID: <20020314135815.28470.qmail@web14708.mail.yahoo.com> Received: from [212.219.131.2] by web14708.mail.yahoo.com via HTTP; Thu, 14 Mar 2002 05:58:15 PST Date: Thu, 14 Mar 2002 05:58:15 -0800 (PST) From: Jeremy Cope Subject: Fwd: Re: [Metakit] RE: how to do substring selections - now sortrev To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > The docs for the C++ implementation state: > > This sorts descending on A: > sorted = view.SortOnReverse(A, A) > > but using this argument syntax for python gives me > > sortedview = > myview.sortrev(myview.utc_datetime,myview.utc_datetime) > TypeError: Not a sequence sortrev takes two tuples as arguments, the first taking, effectively, arguments for sort (in the usual order), the second taking arguments to sort in reverse order. So for the first argument in this case, you need an empty tuple for the first argument - an empty set of brackets (). For the second argument, you need to pass a tuple with just a single item, which by a quirk of Python syntax means enclosing the item in brackets followed by a comma. So this ought to work: sortedview = myview.sortrev( (), (myview.utc_datetime,) ) The apparently redundant trailing comma in the second argument is necessary. Hope this helps Jez :) ===== "Fabricate diem, punc" - Motto, Ankh-Morpork City Watch Jeremy Cope ICQ: 33490230; MSN: jezcope@yahoo.com; Jabber: jezcope@jabber.com __________________________________________________ Do You Yahoo!? Yahoo! Sports - live college hoops coverage http://sports.yahoo.com/ From dave@primco.org Thu Mar 14 10:32:31 2002 Received: from minerva.host4u.net (minerva.host4u.net [216.71.64.49]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2EGWVa24633 for ; Thu, 14 Mar 2002 10:32:31 -0600 Received: from redcloud (adsl-63-193-248-16.dsl.snfc21.pacbell.net [63.193.248.16]) by minerva.host4u.net (8.11.6/8.11.6) with ESMTP id g2EGVYf04978; Thu, 14 Mar 2002 10:31:34 -0600 From: "David Primmer" To: "'Jeremy Cope'" , Subject: RE: [Metakit] RE: how to do substring selections - now sortrev Date: Thu, 14 Mar 2002 08:28:33 -0800 Message-ID: <000001c1cb75$4d4145a0$6401a8c0@redcloud> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal In-Reply-To: <20020314135642.28007.qmail@web14708.mail.yahoo.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Thanks Jeremy That was almost it. Not sure why but your solution didn't quite work. I had to modify it to sortedview = myview.sortrev((myview.utc_datetime,),(myview.utc_datetime,)) without specifying the first argument, my results would sort on the first property field in my view (which happens to be called "id"). Not sure if it would have "subsorted" by date after id but it works fine with the modification. Davep > > sortrev takes two tuples as arguments, the first taking, effectively, > arguments for sort (in the usual order), the second taking arguments to > sort in reverse order. So for the first argument in this case, you need > an empty tuple for the first argument - an empty set of brackets (). > For the second argument, you need to pass a tuple with just a single > item, which by a quirk of Python syntax means enclosing the item in > brackets followed by a comma. So this ought to work: > > sortedview = myview.sortrev( (), (myview.utc_datetime,) ) > > The apparently redundant trailing comma in the second argument is > necessary. From konrad@minus1.de Sun Mar 17 14:37:57 2002 Received: from moutvdomng1.kundenserver.de (moutvdomng1.kundenserver.de [212.227.126.181]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2HKbua01494 for ; Sun, 17 Mar 2002 14:37:56 -0600 Received: from [212.227.126.161] (helo=mrvdomng0.kundenserver.de) by moutvdomng1.kundenserver.de with esmtp (Exim 3.22 #2) id 16mhPf-0002jU-00 for metakit@equi4.com; Sun, 17 Mar 2002 21:37:55 +0100 Received: from [132.230.131.220] (helo=fohnhab.minus1.de) by mrvdomng0.kundenserver.de with esmtp (Exim 3.22 #2) id 16mhPf-00078T-00 for metakit@equi4.com; Sun, 17 Mar 2002 21:37:55 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16mh4m-00028M-00 for metakit@equi4.com; Sun, 17 Mar 2002 21:16:20 +0100 Date: Sun, 17 Mar 2002 21:16:20 +0100 From: Konrad Anton To: Metakit list Message-ID: <20020317211620.A8073@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Metakit] Commitaside-Readonly problems with Mk4py Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all... I'm experimenting with the highly useful commit-aside modes and have run into a problem when the main file is opened read-only. My sample program creates a storage with a view "v" in it, then reopens the storage read-only and an empty commit-aside file read-write and puts them together. Then it appends a row, commits, appends another row and rolls back. ################################################### #prepare a storage s=metakit.storage("TMPtest.mk",1) v=s.getas("v[x:I]") v.append(x=0) s.commit() #open commit-aside with readonly main storage s=metakit.storage("TMPtest.mk",0) sa=metakit.storage("TMPtest.mka",1) s.aside(sa) v=s.view("v") v.append(x=1) print "appending 1", [r.x for r in v] sa.commit() # s.commit() would raise IOError print "committing", [r.x for r in v] v.append(x=2) print "appending 2", [r.x for r in v] sa.rollback() v=s.view("v") print "rollback aside storage", [r.x for r in v] s.rollback(0) #s.rollback() or s.rollback(1) give same result! v=s.view("v") print "rollback main storage", [r.x for r in v] ################################################### The output is: ################################################### appending 1 [0, 1] committing [0, 1] appending 2 [0, 1, 2] rollback aside storage [0, 1, 2] rollback main storage [0] ################################################### Is there a way I can rollback to the state after committing (second line in output)? Calling rollback() on the aside storage does nothing, whereas rollback(0) on the main storage does a full rollback, seemingly discarding the entire aside storage. I'm using Python 2.1.1 and Metakit 2.4.3-33 (but 2.4.2 gives same result). I can try the CVS version, too, but first I'd like to know if my expectations are right at all. TIA Konrad -- Konrad Anton ; Web: www.minus1.de ; gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: Who are you will you, have just the mushroom cat; remarked, till she remarked. From petitsoldat@rediffmail.com Mon Mar 18 06:28:47 2002 Received: from mailweb22.rediffmail.com ([203.199.83.146]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g2ICSja18373 for ; Mon, 18 Mar 2002 06:28:46 -0600 Received: (qmail 17729 invoked by uid 510); 18 Mar 2002 12:26:01 -0000 Date: 18 Mar 2002 12:26:01 -0000 Message-ID: <20020318122601.17728.qmail@mailweb22.rediffmail.com> Received: from unknown (203.196.128.8) by rediffmail.com via HTTP; 18 Mar 2002 12:26:01 -0000 MIME-Version: 1.0 From: "petitsoldat" Reply-To: "petitsoldat" To: metakit@equi4.com Content-type: text/plain; format=flowed Content-Disposition: inline Subject: [Metakit] Some help with using Metakit for large file Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I need some help with using the Metakit C++ library in the best possible way. Here is a description of the problem. It only gives the context but you can skip to the next paragraph without reading it. I am trying to use Metakit for storing and reading a table from disk. The total number of records in the table will be around 1,000,000 while Metakit seems to work well for 100,000 rows per file. It implies that the table must be saved in multiple files (e.g. 10 files of 10,000 records each). Record consists of a string key and object pair with each key having multiple objects (10,000 object on average for each key, may be more) and the object size being 100 KB in some cases. Memory usage is a limiting factor on my computer so, ideally, I would like to limit the memory used. I am thinking of using multiple files and using a block_size parameter to limit the number of records in each file. I looked at the last few months archives and came across the term "blocked column" that if used can make commit() faster. What is a blocked column and how does it make commit faster? Another question I want to ask is on how Metakit does on-demand loading. What portion of the file is in memory at a given point of time, things like how many rows surrounding a row being read, entire rows or just the columns being used, page size, etc? Can someone throw light on these questions. Thanks Tarun From jcw@equi4.com Mon Mar 18 07:04:20 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ID4Ja20036 for ; Mon, 18 Mar 2002 07:04:19 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Some help with using Metakit for large file Date: Mon, 18 Mar 2002 14:00:45 +0100 Message-Id: <20020318130045.20280@triqs.com> In-Reply-To: <20020318122601.17728.qmail@mailweb22.rediffmail.com> References: <20020318122601.17728.qmail@mailweb22.rediffmail.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: petitsoldat wrote: >Here is a description of the problem. It only gives the context >but you can skip to the next paragraph without reading it. I am >trying to use Metakit for storing and reading a table from disk. >The total number of records in the table will be around 1,000,000 >while Metakit seems to work well for 100,000 rows per file. It >implies that the table must be saved in multiple files (e.g. 10 >files of 10,000 records each). Record consists of a string key and >object pair with each key having multiple objects (10,000 object >on average for each key, may be more) and the object size being >100 KB in some cases. Memory usage is a limiting factor on my >computer so, ideally, I would like to limit the memory used. I am >thinking of using multiple files and using a block_size parameter >to limit the number of records in each file. What is the average/expected total size of your data? 1,000,000 x 10,000 x 100 Kb would be terabytes... In general, you are better off storing things in a single file (but 2 Gb is a hard limit, if you don't hit other roadblocks first). >I looked at the last few months archives and came across the term >"blocked column" that if used can make commit() faster. What is a >blocked column and how does it make commit faster? It stores data as a view of subviews, but makes them appear as a single view. Commits are column-wise, so this segmented/blocked format reduces the size of columns. The story is rather more complex nowadays, though - large strings/bindata is stored as "memos", i.e. not columnwise at all. >Another question I want to ask is on how Metakit does on-demand >loading. What portion of the file is in memory at a given point of >time, things like how many rows surrounding a row being read, >entire rows or just the columns being used, page size, etc? MK works with memory mapped files. Data is not "read", it is "paged in" as it gets accessed (and paged out again as the OS juggles with VM). Pages tend to be 4 Kb on most hardware, so that's what gets transferred to/from disk. When you make changes, MK tends to read more of columns in. To keep memory use low, you need to find a balance for the frquency of commits (which flushes buffers and reduces memory use), and performance (which is higher when not committing too often). In read-only mode, memory use is essentially absent. Adding lots of data and only comitting at the end will require lots of (temporary) memory. -jcw From jcw@equi4.com Mon Mar 18 07:04:21 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2ID4Ka20042 for ; Mon, 18 Mar 2002 07:04:21 -0600 From: Jean-Claude Wippler To: Metakit list Subject: Re: [Metakit] Commitaside-Readonly problems with Mk4py Date: Mon, 18 Mar 2002 14:03:17 +0100 Message-Id: <20020318130317.10621@triqs.com> In-Reply-To: <20020317211620.A8073@minus1.de> References: <20020317211620.A8073@minus1.de> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Konrad Anton wrote: (Thanks for making this example so simple to read) >#prepare a storage >s=metakit.storage("TMPtest.mk",1) >v=s.getas("v[x:I]") >v.append(x=0) >s.commit() > >#open commit-aside with readonly main storage >s=metakit.storage("TMPtest.mk",0) >sa=metakit.storage("TMPtest.mka",1) >s.aside(sa) >v=s.view("v") >v.append(x=1) >print "appending 1", [r.x for r in v] >sa.commit() ># s.commit() would raise IOError >print "committing", [r.x for r in v] >v.append(x=2) >print "appending 2", [r.x for r in v] >sa.rollback() >v=s.view("v") >print "rollback aside storage", [r.x for r in v] >s.rollback(0) >#s.rollback() or s.rollback(1) give same result! >v=s.view("v") >print "rollback main storage", [r.x for r in v] > >The output is: > >appending 1 [0, 1] >committing [0, 1] Correct, s now has extra changes, stored "aside" in sa. >appending 2 [0, 1, 2] >rollback aside storage [0, 1, 2] Appended, but not committed... hm, you're right: rollback should take you back to the last committed state. Try: reopen both and redo "s.aside(sa)". Should be [0, 1] - it it? >rollback main storage [0] > >Is there a way I can rollback to the state after committing (second >line in output)? Calling rollback() on the aside storage does nothing, >whereas rollback(0) on the main storage does a full rollback, seemingly >discarding the entire aside storage. That last part is correct. >I'm using Python 2.1.1 and Metakit 2.4.3-33 (but 2.4.2 gives same >result). I can try the CVS version, too, but first I'd like to know if >my expectations are right at all. Yes, I think they are. No need to go for CVS just yet, I have not made changes w.r.t. commit aside. Note also that commit-aside is lacking an important optimization (which is in the file format, but not yet coded): commit aside currently stores entire columns - it is intended to store byte-wise differences, making it *far* more space efficient than it is now. I will look into this - it looks like a commit-aside bug. Thanks. -jcw From konrad@minus1.de Mon Mar 18 14:08:00 2002 Received: from moutvdom00.kundenserver.de (moutvdom00.kundenserver.de [195.20.224.149]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2IK80a15993 for ; Mon, 18 Mar 2002 14:08:00 -0600 Received: from [172.19.20.63] (helo=mrvdomng2.kundenserver.de) by moutvdom00.kundenserver.de with esmtp (Exim 2.12 #2) id 16n3QD-00020B-00 for metakit@equi4.com; Mon, 18 Mar 2002 21:07:57 +0100 Received: from [132.230.131.220] (helo=fohnhab.minus1.de) by mrvdomng2.kundenserver.de with esmtp (Exim 3.22 #2) id 16n3QD-00078s-00 for metakit@equi4.com; Mon, 18 Mar 2002 21:07:57 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16n3Dn-0000lU-00 for metakit@equi4.com; Mon, 18 Mar 2002 20:55:07 +0100 Date: Mon, 18 Mar 2002 20:55:07 +0100 From: Konrad Anton To: Metakit list Subject: Re: [Metakit] Commitaside-Readonly problems with Mk4py Message-ID: <20020318205507.A2916@minus1.de> References: <20020317211620.A8073@minus1.de> <20020318130317.10621@triqs.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020318130317.10621@triqs.com>; from jcw@equi4.com on Mon, Mar 18, 2002 at 02:03:17PM +0100 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, Mar 18, 2002 at 02:03:17PM +0100, Jean-Claude Wippler wrote: > (Thanks for making this example so simple to read) How ironical was that? > Appended, but not committed... hm, you're right: rollback should take you > back to the last committed state. > > Try: reopen both and redo "s.aside(sa)". Should be [0, 1] - it it? New version of the script: ################################################################ import os try: os.unlink("TMPtest.mk") except: pass try: os.unlink("TMPtest.mka") except: pass import metakit #prepare a storage s=metakit.storage("TMPtest.mk",1) v=s.getas("v[x:I]") v.append(x=0) s.commit() #open commit-aside with readonly main storage s = metakit.storage("TMPtest.mk",0) sa= metakit.storage("TMPtest.mka",1) s.aside(sa) v=s.view("v") v.append(x=1) print "appending 1", [r.x for r in v] sa.commit() print "committing", [r.x for r in v] v.append(x=2) print "appending 2", [r.x for r in v] #trying JCW's idea s=metakit.storage("TMPtest.mk",0) sa=metakit.storage("TMPtest.mka",1) s.aside(sa) v=s.view("v") print "reopen", [r.x for r in v] ################################################################### With result: ################################################################### appending 1 [0, 1] committing [0, 1] appending 2 [0, 1, 2] reopen [0] ################################################################### Which is still not optimal. BTW: A commit-aside setup with read-write main storage does the same. I-always-thought-a-computer-could-produce-both-zeros-and-ones-ly y'rs Konrad -- Konrad Anton ; Web: www.minus1.de ; gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The blame on their own ears and mine the mushroom Cat and low and I suppose? From konrad@minus1.de Tue Mar 19 06:38:03 2002 Received: from moutvdomng1.kundenserver.de (moutvdomng1.kundenserver.de [212.227.126.181]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2JCc2a08392 for ; Tue, 19 Mar 2002 06:38:02 -0600 Received: from [212.227.126.161] (helo=mrvdomng0.kundenserver.de) by moutvdomng1.kundenserver.de with esmtp (Exim 3.22 #2) id 16nIsJ-00031q-00 for metakit@equi4.com; Tue, 19 Mar 2002 13:37:59 +0100 Received: from [132.230.131.220] (helo=fohnhab.minus1.de) by mrvdomng0.kundenserver.de with esmtp (Exim 3.22 #2) id 16nIsI-0007Ix-00 for metakit@equi4.com; Tue, 19 Mar 2002 13:37:58 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16nIde-0000NN-00 for metakit@equi4.com; Tue, 19 Mar 2002 13:22:50 +0100 Date: Tue, 19 Mar 2002 13:22:50 +0100 From: Konrad Anton To: Metakit list Subject: solved (was: Re: [Metakit] Commitaside-Readonly problems with Mk4py) Message-ID: <20020319132250.A1408@minus1.de> References: <20020317211620.A8073@minus1.de> <20020318130317.10621@triqs.com> <20020318205507.A2916@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020318205507.A2916@minus1.de>; from konrad@minus1.de on Mon, Mar 18, 2002 at 08:55:07PM +0100 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, Mar 18, 2002 at 08:55:07PM +0100, Konrad Anton wrote: > BTW: A commit-aside setup with read-write > main storage does the same. I've found the mistake. I was calling commit(0) on the aside storage for partial commits. If I call commit(0) on the main storage, it all works. The idea that I get IOErrors from committing the (read-only) main storage must have come from an experiment with a full commit. Great! Have a nice remaining day. Konrad -- Konrad Anton ; Web: www.minus1.de ; gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The blame on their own ears and see how Is But the three. From konrad@minus1.de Tue Mar 19 07:09:48 2002 Received: from moutng0.schlund.de (moutng0.kundenserver.de [212.227.126.170]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2JD9la11278 for ; Tue, 19 Mar 2002 07:09:47 -0600 Received: from [195.20.224.204] (helo=mrvdom00.kundenserver.de) by moutng0.schlund.de with esmtp (Exim 3.22 #2) id 16nJN4-0001Tc-00 for metakit@equi4.com; Tue, 19 Mar 2002 14:09:46 +0100 Received: from [132.230.131.220] (helo=fohnhab.minus1.de) by mrvdom00.kundenserver.de with esmtp (Exim 2.12 #2) id 16nJLL-0001pm-00 for metakit@equi4.com; Tue, 19 Mar 2002 14:07:59 +0100 Received: from konrad by fohnhab.minus1.de with local (Exim 3.12 #1) id 16nJDi-0000OK-00 for metakit@equi4.com; Tue, 19 Mar 2002 14:00:06 +0100 Date: Tue, 19 Mar 2002 14:00:06 +0100 From: Konrad Anton To: Metakit list Subject: Re: solved (was: Re: [Metakit] Commitaside-Readonly problems with Mk4py) Message-ID: <20020319140006.A1487@minus1.de> References: <20020317211620.A8073@minus1.de> <20020318130317.10621@triqs.com> <20020318205507.A2916@minus1.de> <20020319132250.A1408@minus1.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020319132250.A1408@minus1.de>; from konrad@minus1.de on Tue, Mar 19, 2002 at 01:22:50PM +0100 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Tue, Mar 19, 2002 at 01:22:50PM +0100, Konrad Anton wrote: > On Mon, Mar 18, 2002 at 08:55:07PM +0100, Konrad Anton wrote: > > BTW: A commit-aside setup with read-write > > main storage does the same. > > I've found the mistake. I was calling commit(0) on the aside storage > for partial commits. If I call commit(0) on the main storage, it all > works. The idea that I get IOErrors from committing the (read-only) > main storage must have come from an experiment with a full commit. No, it came from a fragment like sto=storage("somefile",0) # where somefile already contains views asideSto=storage("asidefile",1) sto.rollback(1) #(missing:) sto.aside(asideSto) va = sto.view("viewa") va.append(x=20) sto.commit(0) # raises IOError The IOError is raised iff I do not reassign the aside storage. That's something I can live with, but maybe it could be announced in the documentation? like (in python.html)... ,------------------ storage.rollback(full=0) Revert data and structure as was last committed to disk. In commit-aside mode, a "full" rollback reverts to the state of the original file and forgets about the aside file. (new:) After a rollback, your view objects are invalid (use the view or getas methods on your storage object to get them back). Furthermore, after a full rollback, the aside storage is detached from the main storage. Use the aside method on your main storage object to reattach it. If you do not reattach it, further commits will (try to) write to the main storage. `------------------ it's-getting-clearer-every-time-ly y'rs Konrad -- Konrad Anton ; Web: www.minus1.de ; gpg: 0x22954D8A Tel. +49-761-881-2122, Fax +49-721-151318943. Alice: The blame on their own ears and see how Is But the three. From gerard@xsystemsinc.com Tue Mar 19 09:10:40 2002 Received: from dom01.xsystemsinc.com (hidden-user@dom01.xsystemsinc.com [12.5.16.43]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2JFAea22271 for ; Tue, 19 Mar 2002 09:10:40 -0600 Received: from GERARD ([64.204.192.3]) by dom01.xsystemsinc.com (Lotus Domino Release 5.0.8) with SMTP id 2002031910103883:1929 ; Tue, 19 Mar 2002 10:10:38 -0500 Message-ID: <009f01c1cf58$43b14fd0$0c00a8c0@GERARD> From: "Gerard Duquette" To: Date: Tue, 19 Mar 2002 10:10:53 -0500 MIME-Version: 1.0 X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-MIMETrack: Itemize by SMTP Server on dom01/xsystems(Release 5.0.8 |June 18, 2001) at 03/19/2002 10:10:39 AM, Serialize by Router on dom01/xsystems(Release 5.0.8 |June 18, 2001) at 03/19/2002 10:10:40 AM, Serialize complete at 03/19/2002 10:10:40 AM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="iso-8859-1" Subject: [Metakit] Compiling Metakit for Unix Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I have been trying to compile metakit for a Unix box for some time now with no success. Originaly I wanted to compile it for AIX but that proved to be a big problem so now I am trying for HPUX10.2. Basically all I need is the shared object i.e. mk4tcl.so. I will be using metakit with tcl. I can not use the compiled HPUX object from the website because it was compiled for HPUX11 which is a 64bit OS and HPUX10.2 is a 32bit one. My question is simply has anybody tried to compile metakit, just the shared object, for either of these OS and would you be willing to share your experience. I dont need the shared object, just some pointers on how to get through this task without too much pain. I simply need to know what to look for, how to go about it, possibly a quick and dirty way to do it, anything would help. Thanks Gerard Duquette X.Systems, Inc (703) 330-1645 ext111 From Murat_Berk@bmc.com Tue Mar 19 09:25:23 2002 Received: from babbler.bmc.com (camaro.bmc.com [198.207.223.231]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2JFPNa23188 for ; Tue, 19 Mar 2002 09:25:23 -0600 Received: from ec03-hou.bmc.com (localhost [127.0.0.1]) by babbler.bmc.com (8.10.2/8.10.2) with ESMTP id g2JFSfV07025; Tue, 19 Mar 2002 09:28:41 -0600 (CST) Received: by ec03-hou.bmc.com with Internet Mail Service (5.5.2653.19) id ; Tue, 19 Mar 2002 09:25:15 -0600 Message-ID: From: "Berk, Murat" To: "'Gerard Duquette'" , metakit@equi4.com Subject: RE: [Metakit] Compiling Metakit for Unix Date: Tue, 19 Mar 2002 09:25:12 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: We do compile it regularly for those and some other platforms both 32 and 64 bit mode. We do not use the configure script since it is integrated to our make system. On AIX, it will not compile out of box. I send fixes for AIX XLC 3.64 and above to Jean-Claude for 2.01 long time ago and same fixes applies to 2.4.3 Add these before c4_Cursor #if defined(os_aix) && defined(compiler_ibmcxx) && (compiler_ibmcxx > 500) bool operator== (const c4_RowRef& a_, const c4_RowRef& b_); bool operator!= (const c4_RowRef& a_, const c4_RowRef& b_); bool operator<= (const c4_RowRef& a_, const c4_RowRef& b_); bool operator>= (const c4_RowRef& a_, const c4_RowRef& b_); bool operator> (const c4_RowRef& a_, const c4_RowRef& b_); bool operator< (const c4_RowRef& a_, const c4_RowRef& b_); #endif Unless these are defined before the class definition, it does not compile. Other than that, it should compile pretty much on all platforms. And we do compile as shared object using our own make system. Murat -----Original Message----- From: Gerard Duquette [mailto:gerard@xsystemsinc.com] Sent: Tuesday, March 19, 2002 9:11 AM To: metakit@equi4.com Subject: [Metakit] Compiling Metakit for Unix I have been trying to compile metakit for a Unix box for some time now with no success. Originaly I wanted to compile it for AIX but that proved to be a big problem so now I am trying for HPUX10.2. Basically all I need is the shared object i.e. mk4tcl.so. I will be using metakit with tcl. I can not use the compiled HPUX object from the website because it was compiled for HPUX11 which is a 64bit OS and HPUX10.2 is a 32bit one. My question is simply has anybody tried to compile metakit, just the shared object, for either of these OS and would you be willing to share your experience. I dont need the shared object, just some pointers on how to get through this task without too much pain. I simply need to know what to look for, how to go about it, possibly a quick and dirty way to do it, anything would help. Thanks Gerard Duquette X.Systems, Inc (703) 330-1645 ext111 _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit From petitsoldat@rediffmail.com Tue Mar 19 09:40:17 2002 Received: from mailweb9.rediffmail.com ([203.199.83.21]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g2JFeGa25451 for ; Tue, 19 Mar 2002 09:40:17 -0600 Received: (qmail 606 invoked by uid 510); 19 Mar 2002 15:39:26 -0000 Date: 19 Mar 2002 15:39:26 -0000 Message-ID: <20020319153926.605.qmail@mailweb9.rediffmail.com> Received: from unknown (203.196.128.8) by rediffmail.com via HTTP; 19 Mar 2002 15:39:26 -0000 MIME-Version: 1.0 From: "petitsoldat" Reply-To: "petitsoldat" To: metakit@equi4.com Subject: Re: Re: [Metakit] Some help with using Metakit for large file Content-type: text/plain; format=flowed Content-Disposition: inline Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: On Mon, 18 Mar 2002 Jean-Claude Wippler wrote : >petitsoldat wrote: > > >What is the average/expected total size of your data? > 1,000,000 x 10,000 x 100 Kb would be terabytes... 100KB is the maximum size of an object (the average size is perhaps much smaller but is hard to predict). May be I should have said that there will be 2^7 keys at max with each key having O(10^4) objects on an average. I think this will keep the number of records in the table to around 1 million and total data size will be: 1,000,000 x 100 KB = 100 GB in the worst-case. Average case should bring this down by at least one order of magnitude. > >In general, you are better off storing things in a >single file (but 2 Gb is a hard limit, if you don't hit >other roadblocks first). > Thanks. Is 10,000 still the hard limit for views in the latest version of Metakit? > >I looked at the last few months archives and came > >across the term "blocked column" that if used can > >make commit() faster. What is a blocked column and > >how does it make commit faster? > >It stores data as a view of subviews, but makes them >appear as a single view. Commits are column-wise, so >this segmented/blocked format reduces the size of >columns. The story is rather more complex nowadays, >though - large strings/bindata is stored as "memos", >i.e. not columnwise at all. > It is interesting since this is just what I am trying to do by storing a table in several files. Let me know if there is any further explanation on blocked columns, somewhere on the wiki may be. And thanks for the quick reply. How are "memos" stored then? I apologise if it is mere confusion on my part, but isn't it true that memo fields are the same as c4_BytesProp in the newer releases. From jcw@equi4.com Fri Mar 22 17:08:16 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2MN8Fa22254 for ; Fri, 22 Mar 2002 17:08:15 -0600 From: Jean-Claude Wippler To: metakit Date: Sat, 23 Mar 2002 00:08:13 +0100 Message-Id: <20020322230813.19205@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] TclKit news Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is to let you know that there are new TclKit builds on the website. Many bug fixes, many new platforms, much streamlining, it's all at: http://www.equi4.com/tclkit/ Also of interest to Tcl'ers on this list, is a very young but growing collection of scripted documents, both applications and packages - an initiative by Steve Landers: http://mini.net/sdarchive/ Oh, wait, perhaps I should add what this is all about... "TclKit: is a single-file runtime for projects built with Tcl/Tk, IncrTcl, and MetaKit. It's all of these, but instantly deployable. See http://wiki.tcl.tk/TclKit "Scripted Documents" are a packaging scheme which can be used in combination with TclKit to create single-file multi-platform apps. See http://wiki.tcl.tk/Scripted It can all be summarized with the following three words: deployment, deployment, deployment... -jcw From steve@improvision.com Tue Mar 26 10:58:03 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2QGw2a30201 for ; Tue, 26 Mar 2002 10:58:02 -0600 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id QAA13119 for ; Tue, 26 Mar 2002 16:55:11 GMT Received: from [192.168.1.76] ([192.168.1.76]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971 for ; Tue, 26 Mar 2002 16:58:00 -0000 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Tue, 26 Mar 2002 16:57:59 +0000 From: Stephen Baxter To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Metakit] Corrupted byte properties Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi all, We seem to be getting problems with corrupted byte properties being written to the database again. I can't find a pattern yet, but the corrupted items all seem to be small (<50 bytes). I'm using the latest sources out of CVS - has anyone seen any similar problems? Cheers, Steve. -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ From kord@grub.org Wed Mar 27 10:12:15 2002 Received: from ant.grub.org (ant.grub.org [206.30.142.107]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2RGCFa24554 for ; Wed, 27 Mar 2002 10:12:15 -0600 Received: from localhost (kord@localhost) by ant.grub.org (8.11.2/8.11.2) with ESMTP id g2RGCC927174; Wed, 27 Mar 2002 10:12:12 -0600 X-Authentication-Warning: ant.grub.org: kord owned process doing -bs Date: Wed, 27 Mar 2002 10:12:12 -0600 (CST) From: Kord Campbell To: cc: In-Reply-To: <20020327072537.GC3819@asgard.pte.at> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] Re: [Grub-client] successful compile and bugs (fwd) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Alfie, I'm not sure about compatability between Metakit 2.01 and 2.4.3. We never had any other version besides 2.4.3 installed, so we are (obviously) in the dark about the ramification of making people upgrade their Metakit library to run the client and whether or not this will break existing dependencys. It is my experience that it is more likely that it will break them than not! ;) I would imagine that a work around could be done by staticly linking the client with the metakit libary. That way, you could install the newer metakit in another directory, compile with it, and then still have the old shared linked library in use by flightgear. Question is, do Debian packages use source code, binaries, or both? BTW, I emailed Chris - I think you had the correct address in the cc! Kord On Wed, 27 Mar 2002 grub-client-admin@lists.sourceforge.net wrote: > * [2002-03-26 11:28]: > > ---------- Forwarded message ---------- > > Date: Mon, 25 Mar 2002 12:43:34 -0500 > > From: Christopher Thomas Nicodemus > > To: support@grub.org > > Subject: successful compile and bugs > > > > I successfully compiled grubclient 0.30 on Debian 3.0 > > uname -a > > Linux NN2kLinux 2.4.18 #13 Mon Mar 25 08:52:36 EST 2002 i686 unknown > > Kord, could you hand me Christopher's mail address? I can't compile > grub on Debian without an updated metakit package (silly metakit doesn't > install itself with a distinctive api-version so you can install > different versions of metakit side-by-side) so I'd like to know if he > used an updated metakit or what else trick he used to get it working? > > At least I have a pre-version at > for Debian that was working > already before this mail came in ,) It currently is just the question > if flightgear works with the newly build metakit library. If that > problem is solved (or if I found a way to install metakit-2.4.3 > side-by-side with metakit-2.01) I'll upload the package immediately. > > If you have any ideas about metakit that doesn't break flightgear it > would be nice to share them :) > > Kord, feel free to forward this message to Christopher, thanks. > Alfie > -- -------------------------------------------------------------- Kord Campbell Grub.Org Inc. President 6051 N. Brookline #118 Oklahoma City, OK 73112 kord@grub.org Voice: (405) 843-6336 http://www.grub.org Fax: (405) 848-5477 -------------------------------------------------------------- From jcw@equi4.com Wed Mar 27 10:39:42 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2RGdfa25958; Wed, 27 Mar 2002 10:39:42 -0600 From: Jean-Claude Wippler To: Cc: Subject: Re: [Metakit] Re: [Grub-client] successful compile and bugs (fwd) Date: Wed, 27 Mar 2002 17:38:29 +0100 Message-Id: <20020327163829.14101@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Kord Campbell wrote: [...] >I'm not sure about compatability between Metakit 2.01 and 2.4.3. >We never had any other version besides 2.4.3 installed, so we are >(obviously) in the dark about the ramification of making people >upgrade their Metakit library to run the client and whether or >not this will break existing dependencys. It is my experience >that it is more likely that it will break them than not! ;) > >I would imagine that a work around could be done by staticly linking >the client with the metakit libary. That way, you could install the >newer metakit in another directory, compile with it, and then still >have the old shared linked library in use by flightgear. MetaKit's file format changed from 2.0 to 2.3/2.4 - the current code is such that conversion from 2.01 to 2.4.3 should be automatic and effortless. This conversion does imply that older code will no longer recognize the dataformat. One of the design goals of MK, is that it always offers an upward migration path for datafiles. In the case of 2.01 -> 2.4.3, this conversion takes place on open (even with r/o files) and gets applied on commit. That sounds more complex than it really is, the file format has not changed all that much. My advice would be to go for a full update to 2.4.3 if possible (FWIW, I'll bring out a new release soon, because there are some valuable changes in CVS - so you might want to plan this but hold off for just a tad longer). There has been one reported case whereby the conversion caused problems, but I was not able to delve into that further - apart from that, I consider 2.4.3 to be just as stable as 2.01 (and preferable due to new features and higher performance). Please email me if you have concerns about upgrading, or run into any issues with the deployment side of it. -jcw From jcw@equi4.com Wed Mar 27 10:41:39 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2RGfca26134 for ; Wed, 27 Mar 2002 10:41:38 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: Re: [Metakit] Some help with using Metakit for large file Date: Wed, 27 Mar 2002 17:40:26 +0100 Message-Id: <20020327164026.9331@triqs.com> In-Reply-To: <20020319153926.605.qmail@mailweb9.rediffmail.com> References: <20020319153926.605.qmail@mailweb9.rediffmail.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: petitsoldat wrote: [...] > total data size will be: > >1,000,000 x 100 KB = 100 GB > >in the worst-case. Average case should bring this down by at least >one order of magnitude. This is well beyond what MK can deal with. -jcw From jcw@equi4.com Wed Mar 27 11:01:06 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2RH15a27397 for ; Wed, 27 Mar 2002 11:01:05 -0600 From: Jean-Claude Wippler To: Subject: RE: [Metakit] Compiling Metakit for Unix Date: Wed, 27 Mar 2002 18:01:01 +0100 Message-Id: <20020327170101.19707@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Berk, Murat wrote: [Gerard Duquette asks for info about building for AIX & HPUX 10.2] >We do compile it regularly for those and some other platforms both 32 and 64 >bit mode. We do not use the configure script since it is integrated to our >make system. > >On AIX, it will not compile out of box. I send fixes for AIX XLC 3.64 and >above to Jean-Claude for 2.01 long time ago [...] I've just committed your patch to CVS. [...] >Other than that, it should compile pretty much on all platforms. FWIW, I've started testing builds, and am trying to simplify the autoconf + libtool use of MK. One success, is that the ".0.0.0" suffix is gone, MK now generates and installs "normal" .so files. Successful builds within the past week or so include several Win/Mac setups, and Linux x86/alpha/ppc/sparc, FreeBSD, BSD/OS, Solaris, Tru64. The portability to things like AIX and HPUX is limited mostly by my lack of access, and/or working closely with people to get things right. I'm still struggling a bit with libtool in unix/Makefile.in - if anyone has tips on how to improve things: your comments will be very much appreciated. -jcw From dndposey@jps.net Wed Mar 27 18:58:31 2002 Received: from epic.mail.pas.earthlink.net (epic.mail.pas.earthlink.net [207.217.120.181]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2S0wSa20519 for ; Wed, 27 Mar 2002 18:58:29 -0600 Received: from 216-224-155-215.thegrid.net ([216.224.155.215] helo=merlot) by epic.mail.pas.earthlink.net with esmtp (Exim 3.33 #2) id 16qOF9-0000ow-00 for metakit@equi4.com; Wed, 27 Mar 2002 16:58:20 -0800 From: dndposey@jps.net To: metakit@equi4.com Date: Wed, 27 Mar 2002 17:02:32 -0800 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Message-ID: <3CA1FB28.596.2225F93@localhost> Priority: normal X-mailer: Pegasus Mail for Win32 (v3.12c) Subject: [Metakit] Latest TclKit binaries Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The latest Win32 binaries for on my Win98 system seem do to well *until* it exits. Then it crashes with an invalid page fault. I run Win98 on a Sony VAIO laptop w/128meg. Dale Posey ex of IBM (thank God!) From bobhicks@adelphia.net Sat Mar 30 12:18:19 2002 Received: from smtprelay9.dc2.adelphia.net (smtprelay9.dc2.adelphia.net [64.8.50.53]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2UIIIa10840 for ; Sat, 30 Mar 2002 12:18:18 -0600 Received: from homepcxuhigci7 ([24.49.32.46]) by smtprelay9.dc2.adelphia.net (Netscape Messaging Server 4.15) with SMTP id GTSTJT01.H3R for ; Sat, 30 Mar 2002 13:19:05 -0500 Message-ID: <000901c1d817$41dacbb0$6401a8c0@homepcxuhigci7> From: "Bob Hicks" To: Date: Sat, 30 Mar 2002 13:18:13 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Subject: [Metakit] Ruby bindings Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Would anyone be in the process of creating Ruby bindings for Metakit? Is anyone interested in such a beast? Bob From frank@canyon-medical.com Sat Mar 30 12:59:31 2002 Received: from mail3.centurytel.net (mail3.centurytel.net [209.142.136.99]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2UIxVa11188 for ; Sat, 30 Mar 2002 12:59:31 -0600 Received: from aSqueakSystem (pppoe0034.sm.centurytel.net [209.142.160.163]) by mail3.centurytel.net (8.12.2/8.12.2) with SMTP id g2UIxUu5016368 for ; Sat, 30 Mar 2002 12:59:31 -0600 (CST) Message-Id: <200203301859.g2UIxUu5016368@mail3.centurytel.net> X-Mailer: Celeste 2.0.4599 Date: Sat, 30 Mar 2002 12:58:02 -0600 Subject: Re: [Metakit] Ruby bindings To: From: Frank Sergeant Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: "Bob Hicks" wrote: > Would anyone be in the process of creating Ruby bindings for Metakit? Is > anyone interested in such a beast? I have an interest in creating Smalltalk (Dolphin or Squeak) bindings, but haven't gotten close to beginning. I seem to be getting further behind rather than catching up. -- Frank From jcw@equi4.com Sat Mar 30 14:46:49 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2UKkma12059 for ; Sat, 30 Mar 2002 14:46:48 -0600 From: Jean-Claude Wippler To: Subject: Re: [Metakit] Latest TclKit binaries Date: Sat, 30 Mar 2002 21:46:25 +0100 Message-Id: <20020330204625.489@triqs.com> In-Reply-To: <3CA1FB28.596.2225F93@localhost> References: <3CA1FB28.596.2225F93@localhost> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: dndposey@jps.net wrote: >The latest Win32 binaries for on my Win98 system seem do to well >*until* it exits. >Then it crashes with an invalid page fault. >I run Win98 on a Sony VAIO laptop w/128meg. Yes, I've been hitting crash-on-exit in Windows for some time. The latest builds seem to be very consistent on Win98 - :( - but not unfortunately on NT4, where I have all the tools needed to chase this issue. Am currently working with Tcl'ers to try to get this resolved. There have been MetaKit-related crash-on-exit bugs in the past, but right now it looks more like something that is related to Tk 8.4a5 in some way. If you can research this some more, that would be a tremendous help: does it always crash, even just launch+exit? Is tclkitsh.exe ok? Please feel free to follow up by email - I really need to find out what the issue is and solve it. -jcw From jtaylor@shift-right.com Sat Mar 30 16:55:18 2002 Received: from mactufus.net2atlanta.com (ds2.net2atlanta.com [207.51.14.3]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2UMtIa12951 for ; Sat, 30 Mar 2002 16:55:18 -0600 Received: from lefty (208.147.135.177 [208.147.135.177]) by mactufus.net2atlanta.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id H91CMDG2; Sat, 30 Mar 2002 17:54:57 -0500 From: "John Taylor" To: Date: Sat, 30 Mar 2002 17:54:50 -0500 Message-ID: <000001c1d83d$e8ebd9a0$6401a8c0@lefty> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Multithreaded in C++ Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: FYI, If anyone one is interested. In the course of using Metakit in our products at work, we have created a Multi-threaded C++ wrapper for Metakit. It is a client/server model using ITC (inter-thread-communications) to isolate all database actions for a given a database to a single thread. The wrapper is part of my company's open source C++ library. The library can be downloaded at http://sourceforge.net/projects/openrepo. After downloading, the example project for the Metakit wrapper is: projects//metakit/examples/mt John T. Taylor Shift-Right Technologies, LLC. Tel: 678-344-3115 Fax: 770-736-9276 email: jtaylor@shift-right.com url: www.shift-right.com From jcw@equi4.com Sun Mar 31 14:43:41 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g2VKhea26105; Sun, 31 Mar 2002 14:43:40 -0600 From: Jean-Claude Wippler To: announce , metakit Date: Sun, 31 Mar 2002 22:42:26 +0200 Message-Id: <20020331204226.29553@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] MetaKit 2.4.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is to announce a new stable release of MetaKit, the database that fits in the palm of your hand. There are several bug fixes and a few enhancements, see: http://www.equi4.com/metakit/CHANGES Most notable improvements occur with blocked view operations. Storing 100,000,000 ints, with one commit every 5,000,000: bigblock.tcl - Mk4tcl 2.4.4 - Linux filled commit #blocks filesize memuse total 216 s 575.2 ms 5001 19980575 b 23372 K 5000000 rows 362 s 674.9 ms 10001 40094082 b 25568 K 10000000 rows 749 s 543.6 ms 15001 60204629 b 26028 K 15000000 rows 285 s 576.9 ms 20001 80316241 b 27268 K 20000000 rows 289 s 609.1 ms 25001 100428917 b 28500 K 25000000 rows 296 s 649.2 ms 30001 120542657 b 29688 K 30000000 rows 291 s 696.6 ms 35001 140653465 b 31084 K 35000000 rows 281 s 719.9 ms 40001 160765337 b 32388 K 40000000 rows 287 s 757.3 ms 45001 180878273 b 33608 K 45000000 rows 281 s 790.1 ms 50001 200992273 b 34888 K 50000000 rows 292 s 834.0 ms 55001 221099345 b 36128 K 55000000 rows 282 s 1127.7 ms 60001 241211477 b 37284 K 60000000 rows 287 s 894.1 ms 65001 261324673 b 38508 K 65000000 rows 285 s 932.3 ms 70001 281437963 b 39736 K 70000000 rows 284 s 967.2 ms 75001 301557105 b 40960 K 75000000 rows 282 s 1002.8 ms 80001 321678315 b 41908 K 80000000 rows 285 s 1038.5 ms 85001 341801593 b 43132 K 85000000 rows 280 s 1071.2 ms 90001 361922943 b 44348 K 90000000 rows 281 s 1110.3 ms 95001 382046361 b 45580 K 95000000 rows 289 s 1404.0 ms 100001 402167851 b 46852 K 100000000 rows Notes: - memory use grows, but very very slowly, with 20 Mb being the hard baseline, since that represents 5 million 32-bit ints - filling performance remains constant (one unexplained peak), mostly because these are all appends, i.e. creating new blocks - this took over an hour, a lot of overhead is no doubt caused by using Tcl in such a tight loop, and by a known blocked view index access bottleneck (could be optimized for linear scans) - these performance figures are from a PIII/650, Linux SuSE 7.3 Warning: the build process has been changed to no longer create *.so.0.0.0 libraries, nor special Mk4py.so and Mk4tcl.so targets. Instead, please use libmk4py.so and libmk4tcl.so, or rename them to those old names if you prefer. As before, the MetaKit homepage and download area are at: http://www.equi4.com/metakit/ http://www.equi4.com/pub/mk/ Regards, Jean-Claude From joshm@taconic.net Sun Mar 31 18:45:45 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g310jja28067 for ; Sun, 31 Mar 2002 18:45:45 -0600 Received: from gatekeeper (ch-asc4-p5.taconic.net [205.231.28.149]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g310jfK06247 for ; Sun, 31 Mar 2002 19:45:42 -0500 (EST) Message-ID: <011801c1d916$65de7430$3396e7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" Date: Sun, 31 Mar 2002 19:44:35 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] Newbie questions Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I love metakit (so far), but there are some things that I just don't seem able to grok: 1. I have a simple view into a storage object. I can add or replace fields in this view by simply setting them and calling commit on the storage object. But, if I create a new view based on the original view (for example, calling select to extract a revelent subset), and I make changes to this new view, the storage doesn't update, even if I call commit on it. What I've found that works is to use GetIndexOf to find the index into the original view, make the change in the original view, and commit. Why does modifying the original view change the underlying storage, but modifying the derived view does not? 2. I have a nested view as part of my main table. I can access this table by creating a view from the c4_ViewProp field in the main table. But then how do I add or remove rows from this nested view, and have the storage actually update? 3. One final question. If I want to search in the main table for rows where something is found *in the nested table*, is there any way to do this? For an example, suppose my main table is made up of "book" records, and each book contains a set of "chapter" records as a subview. Is there a logical way to search for all the books that have certain types of chapters? Is there useful documentation which describes how all of this works? The tutorial talks a little about nested views, but doesn't go into any real details about replacing data, deleting data, and so on. -- j From jcw@equi4.com Mon Apr 1 09:49:28 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31FnQa04674; Mon, 1 Apr 2002 09:49:26 -0600 From: Jean-Claude Wippler To: announce , metakit Date: Mon, 1 Apr 2002 17:48:08 +0200 Message-Id: <20020401154808.28447@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Tclkit 8.4-34 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: FYI, there is a new release of TclKit on the web, based on Metakit 2.4.4, Tcl/Tk 8.4a5, IncrTcl 3.2, and the latest TclVFS interface, see: http://www.equi4.com/tclkit/ Includes single-file builds for about a dozen platforms, which should work out of the box (i.e. after gunzip, "chmod +x", and a move/rename on Unix). Regards, Jean-Claude From jalupo@attbi.com Mon Apr 1 11:24:26 2002 Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31HOQa08913 for ; Mon, 1 Apr 2002 11:24:26 -0600 Received: from egor.linux.home ([12.253.55.237]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020401172420.VBDB1147.rwcrmhc52.attbi.com@egor.linux.home> for ; Mon, 1 Apr 2002 17:24:20 +0000 Received: from egor.linux.home (localhost [127.0.0.1]) by egor.linux.home (8.12.2/8.12.2/Debian -7) with ESMTP id g31HOFps019420; Mon, 1 Apr 2002 10:24:15 -0700 Received: (from jalupo@localhost) by egor.linux.home (8.12.2/8.12.2/Debian -7) id g31HO4fF019415; Mon, 1 Apr 2002 10:24:04 -0700 Date: Mon, 1 Apr 2002 10:24:04 -0700 From: "James A. Lupo" Message-Id: <200204011724.g31HO4fF019415@egor.linux.home> To: metakit@equi4.com Subject: [Metakit] Error Compiling metakit-2.4.4 on Linux Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I downloaded and attempted to compile the latest Metakit with these steps: cd builds ../unix/configure --with-tcl=/usr/include/tcl8.3 make The error I get is: /bin/sh ./libtool --tag=CXX g++ -c -O2 -I../unix/../include \ -I/usr/include/tcl8.3/generic -I/usr/include/tcl8.3 \ ../unix/../tcl/mk4tcl.cpp g++ -c -O2 -I../unix/../include -I/usr/include/tcl8.3/generic \ -I/usr/include/tcl8.3 ../unix/../tcl/mk4tcl.cpp -fPIC -DPIC \ -o .libs/mk4tcl.o ../unix/../tcl/mk4tcl.cpp: In method \ `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = "option")': ../unix/../tcl/mk4tcl.cpp:1483: argument passing to `char **' from `const char **' discards qualifiers make: *** [mk4tcl.o] Error 1 mk4tcl.cpp:1483 is: _error = Tcl_GetIndexFromObj(interp, obj_, table_, msg_, 0, &index); Tcl_GetIndexFromOBJ is defined in tclDecls.h as: /* 36 */ EXTERN int Tcl_GetIndexFromObj _ANSI_ARGS_((Tcl_Interp * interp, Tcl_Obj * objPtr, char ** tablePtr, char * msg, int flags, int * indexPtr)); I am tracking the Debian Unstable distribution. Kernel: 2.4.19 gcc: 2.95.4 g++: 2.95.4 tcl: 8.3.3-8 -- Jim ------------------- Optional Methods ----------------------- Dr James A. Lupo (303) 423-2652 9667 Independence Dr. jalupo@attbi.com Westminster CO 80021-6845 jalupo@regis.edu From jcw@equi4.com Mon Apr 1 11:33:25 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31HXNa09376; Mon, 1 Apr 2002 11:33:24 -0600 From: Jean-Claude Wippler To: "James A. Lupo" , Subject: Re: [Metakit] Error Compiling metakit-2.4.4 on Linux Date: Mon, 1 Apr 2002 19:31:49 +0200 Message-Id: <20020401173149.32091@triqs.com> In-Reply-To: <200204011724.g31HO4fF019415@egor.linux.home> References: <200204011724.g31HO4fF019415@egor.linux.home> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: James A. Lupo wrote: >I downloaded and attempted to compile the latest Metakit with these >steps: > > cd builds > ../unix/configure --with-tcl=/usr/include/tcl8.3 > make [...] > `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = >"option")': > ../unix/../tcl/mk4tcl.cpp:1483: argument passing to `char **' from > `const char **' discards qualifiers [...] Yes, this is the one instance where I have not been able to work around an incompatibility in the Tcl headers. Tcl 8.4a4 has recently been "constified", which in itself is a good thing. Unfortunately, no matter what cast I use in my code, the compile will break either in older tcl configs (as it does for you), or in the latest one. Solution: either change "table" to "(char**) table" on line 1483 of mk4tcl.cpp, or upgrade your tcl source tree to a recent 8.4-ish one. -jcw From dgporter@erols.com Mon Apr 1 12:58:19 2002 Received: from smtp01.mrf.mail.rcn.net (smtp01.mrf.mail.rcn.net [207.172.4.60]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31IwIa13180; Mon, 1 Apr 2002 12:58:19 -0600 X-Info: This message was accepted for relay by smtp01.mrf.mail.rcn.net as the sender used SMTP authentication X-Trace: UmFuZG9tSVZzdqsEEpQLHVgM2SLuSTo6nKqjN48oy75wAPZU/9pirnvb25h0fu1H3NntaUJvYck= Received: from hamming.cam.nist.gov ([129.6.88.137] helo=erols.com) by smtp01.mrf.mail.rcn.net with asmtp (Exim 3.33 #10) id 16s70T-0005BQ-00; Mon, 01 Apr 2002 13:58:17 -0500 Message-ID: <3CA8ADC5.14C2D816@erols.com> Date: Mon, 01 Apr 2002 13:58:13 -0500 From: Donald G Porter Reply-To: dgporter@erols.com X-Mailer: Mozilla 4.79 [en] (X11; U; SunOS 5.8 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Claude Wippler CC: "James A. Lupo" , metakit@equi4.com Subject: Re: [Metakit] Error Compiling metakit-2.4.4 on Linux References: <200204011724.g31HO4fF019415@egor.linux.home> <20020401173149.32091@triqs.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: James A. Lupo wrote: > > `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = > >"option")': > > ../unix/../tcl/mk4tcl.cpp:1483: argument passing to `char **' from > > `const char **' discards qualifiers Jean-Claude Wippler wrote: > Yes, this is the one instance where I have not been able to work around > an incompatibility in the Tcl headers. Tcl 8.4a4 has recently been > "constified", which in itself is a good thing. > > Unfortunately, no matter what cast I use in my code, the compile will > break either in older tcl configs (as it does for you), or in the latest one. Have you tried using the -DUSE_NON_CONST option in the Makefile? That should keep you 8.3 header-compatible until you're ready to drop 8.3 support. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@nist.gov Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From g.lancaster@mbf.com.au Mon Apr 1 16:15:31 2002 Received: from fs1-smtp.mbf.com.au ([61.88.251.20]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g31MFUa23902 for ; Mon, 1 Apr 2002 16:15:31 -0600 Received: from MBF-INT1-Message_Server by fs1-smtp.mbf.com.au with Novell_GroupWise; Tue, 02 Apr 2002 08:02:13 +1000 Message-Id: X-Mailer: Novell GroupWise 5.5.2 Date: Tue, 02 Apr 2002 08:13:09 +1000 From: "Garth Lancaster" To: , Subject: Re: [Metakit] Multithreaded in C++ Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline X-Guinevere: 1.0.13 ; Medical Benefits Fun Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g31MFUa23902 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hiyah John - this is one thing that has been a long time coming, a C++ MT 'wrapper' for Metakit - good stuff I followed your link to sourceforge, but couldnt see any files released by the project - and at the moment my ISP seems to have serious DNS problems, so I cant get to www.shift-right.com to have a look there... I'll keep trying .. congrats on putting something very useful within reach :-) regards, Garth Lancaster APATHY ERROR: Don't bother striking any key. Application has reported a "Not My Fault" in module KRNL.EXE in line 0200:103F Garth Lancaster MBF InfoTech (Infrastructure) Integration Engineer g.lancaster@mbf.com.au Ph : +61 2 9323 9534 Fax : +61 2 9267 7359 >>> "John Taylor" 03/31/02 08:54am >>> FYI, If anyone one is interested. In the course of using Metakit in our products at work, we have created a Multi-threaded C++ wrapper for Metakit. It is a client/server model using ITC (inter-thread-communications) to isolate all database actions for a given a database to a single thread. The wrapper is part of my company's open source C++ library. The library can be downloaded at http://sourceforge.net/projects/openrepo. After downloading, the example project for the Metakit wrapper is: projects//metakit/examples/mt John T. Taylor Shift-Right Technologies, LLC. Tel: 678-344-3115 Fax: 770-736-9276 email: jtaylor@shift-right.com url: www.shift-right.com _______________________________________________ metakit mailing list - metakit@equi4.com http://www.equi4.com/mailman/listinfo/metakit This email contains confidential information intended only for the addressee. If you received it in error, please inform us by reply email or by calling us on 131 137. Please also delete this email and destroy any hard copy. You must not disclose or use the information in this email. From jcw@equi4.com Mon Apr 1 16:45:42 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g31Mjfa25457 for ; Mon, 1 Apr 2002 16:45:41 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Error Compiling metakit-2.4.4 on Linux Date: Tue, 2 Apr 2002 00:43:07 +0200 Message-Id: <20020401224307.17757@triqs.com> In-Reply-To: <3CA8ADC5.14C2D816@erols.com> References: <3CA8ADC5.14C2D816@erols.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Donald G Porter wrote: >James A. Lupo wrote: >> > `int Tcl::tcl_GetIndexFromObj(Tcl_Obj *, const char **, char * = >> >"option")': >> > ../unix/../tcl/mk4tcl.cpp:1483: argument passing to `char **' from >> > `const char **' discards qualifiers [...] >Have you tried using the -DUSE_NON_CONST option in the Makefile? That >should keep you 8.3 header-compatible until you're ready to drop 8.3 >support. Ah, that and "CONST84" solved it. I've applied changes to mk4tcl.* so the code should now compile properly with previous Tcl headers - now in CVS. Thanks. -jcw P.S. Also reverted to an earlier libtool to undo the "--tag" trouble that was introduced in unix/Makefile.in - why oh why is is that whenever a release is made, then the *next* minute a useful change needs to be committed to CVS? From jcw@equi4.com Tue Apr 2 10:36:11 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g32Ga9a22986 for ; Tue, 2 Apr 2002 10:36:10 -0600 From: Jean-Claude Wippler To: metakit Date: Tue, 2 Apr 2002 18:34:53 +0200 Message-Id: <20020402163453.11193@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Tweak for 2.4.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There is a bad check in remap.cpp, line 531 - when compiled with assertion checking on. Just comment out the d4_assert call and things will be fine. Another more complete fix has been comitted to CVS. Thanks to several people for pointing this out. -jcw From luislavena@hotmail.com Wed Apr 3 09:25:30 2002 Received: from hotmail.com (oe41.law8.hotmail.com [216.33.240.98]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g33FPUa01333 for ; Wed, 3 Apr 2002 09:25:30 -0600 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 3 Apr 2002 07:25:24 -0800 X-Originating-IP: [200.43.142.2] From: "Luis Lavena" To: Date: Wed, 3 Apr 2002 12:25:15 -0300 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Message-ID: X-OriginalArrivalTime: 03 Apr 2002 15:25:24.0717 (UTC) FILETIME=[C6B4F1D0:01C1DB23] Subject: [Metakit] No 2.4.4 Mk4py for Python 2.1? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I check at equi4.com about the new release of 2.4.4 of Metakit. But after downloaded it (from http://www.equi4.com/pub/mk/metakit-2.4.4-34-win/) only get the Mk4py.dll wich aren't compatible with 2.1 (currently using 2.1.2). is the new only available to 2.2 pythonist? I can't do a build from cvs due lack of VC or any C/C++ compiler on Windows. Anyway, thanks. Regards. Luis From jcw@equi4.com Wed Apr 3 17:33:55 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g33NXsa23534 for ; Wed, 3 Apr 2002 17:33:54 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] No 2.4.4 Mk4py for Python 2.1? Date: Thu, 4 Apr 2002 01:33:14 +0200 Message-Id: <20020403233314.17009@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Luis Lavena wrote: >Hello, I check at equi4.com about the new release of 2.4.4 of Metakit. But >after downloaded it (from http://www.equi4.com/pub/mk/metakit-2.4.4-34-win/) >only get the Mk4py.dll wich aren't compatible with 2.1 (currently using >2.1.2). I'll be happy to put a copy on the server, if someone sends me one - but I don't have Python 2.1 installed anymore myself, unfortunately. You can use the 2.4.3 build, though, it's at: -jcw From joshm@taconic.net Thu Apr 4 13:36:45 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g34Jaia29091 for ; Thu, 4 Apr 2002 13:36:45 -0600 Received: from gatekeeper (ch-asc6-p21.taconic.net [205.231.148.21]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g34JahK12113 for ; Thu, 4 Apr 2002 14:36:43 -0500 (EST) Message-ID: <022b01c1dc0f$d9e26e80$9e1de7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: <011801c1d916$65de7430$3396e7cd@gatekeeper> Subject: Re: [Metakit] Newbie questions Date: Thu, 4 Apr 2002 14:35:16 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I wasn't sure if this made it across the mailing list or not, since I didn't get any replies. :-( Basically, I'm looking for documentation on how to add to/modify/replace/delete information stored in a subview (in C++). This is an extension of a problem I could not figure out, which is as follows: I create a view based on a storage object. In this view, I can easily add and modify fields in a table. This works well. However, when I search this view (create another view via select(), for example), while I can examine this view, if I make changes to it, the underlying storage isn't updated. When I manually follow the fields back to the original view (via GetIndexOf()), I'm able to update the original view and the underlying storage. Question 1: Why doesn't my modification of the view created via select() update the underlying storage? Question 2: If there isn't a way for it to update the underlying storage, how does one modify subviews, since they are only accessible indirectly? If my questions aren't clear, I can try to provide some source to help explain them, but at the moment, I don't have a simple test case. Any any all help will be greatly appreciated. -- j ----- Original Message ----- From: "Joshua Muskovitz" To: "MetaKit Mailing List" Sent: Sunday, March 31, 2002 7:44 PM Subject: [Metakit] Newbie questions > Hi, > > I love metakit (so far), but there are some things that I just don't seem > able to grok: > > 1. I have a simple view into a storage object. I can add or replace fields > in this view by simply setting them and calling commit on the storage > object. But, if I create a new view based on the original view (for > example, calling select to extract a revelent subset), and I make changes to > this new view, the storage doesn't update, even if I call commit on it. > What I've found that works is to use GetIndexOf to find the index into the > original view, make the change in the original view, and commit. Why does > modifying the original view change the underlying storage, but modifying the > derived view does not? > > 2. I have a nested view as part of my main table. I can access this table > by creating a view from the c4_ViewProp field in the main table. But then > how do I add or remove rows from this nested view, and have the storage > actually update? > > 3. One final question. If I want to search in the main table for rows > where something is found *in the nested table*, is there any way to do this? > For an example, suppose my main table is made up of "book" records, and each > book contains a set of "chapter" records as a subview. Is there a logical > way to search for all the books that have certain types of chapters? > > Is there useful documentation which describes how all of this works? The > tutorial talks a little about nested views, but doesn't go into any real > details about replacing data, deleting data, and so on. > > -- j > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit > > From jcw@equi4.com Fri Apr 5 01:56:29 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g357uSa22853; Fri, 5 Apr 2002 01:56:28 -0600 From: Jean-Claude Wippler To: Joshua Muskovitz , metakit Subject: Re: [Metakit] Newbie questions Date: Fri, 5 Apr 2002 09:49:08 +0200 Message-Id: <20020405074908.9726@triqs.com> In-Reply-To: <022b01c1dc0f$d9e26e80$9e1de7cd@gatekeeper> References: <022b01c1dc0f$d9e26e80$9e1de7cd@gatekeeper> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joshua Muskovitz wrote: >Hi, I wasn't sure if this made it across the mailing list or not, since I >didn't get any replies. :-( Whoops... I totally missed this (it did get posted). >Question 1: Why doesn't my modification of the view created via select() >update the underlying storage? It's a limitation of the implementation - derived views are not as modifiable as I had hoped they would become (my only excuse is: it's tricky to get this right when derived views are stacked, i.e. a join of a select of a sort). The GetIndexOf call is the way to do it. Even then, you have to be careful because not all derived view combinations will see the changes (and some things become terribly slow, while propagating each change). A safer approach is to modify the originals, and tear-down/re-build the derived views before/after doing so, well at least in complex cases or when making massive changes. >Question 2: If there isn't a way for it to update the underlying storage, >how does one modify subviews, since they are only accessible indirectly? The logic is reversed: you create a row, then make a view *from* it: c4_Row row = ... int n = myview.Add(row); c4_View subn = pSubProp (myview [n]); ... now make changes to subn, which *is* in storage already ... The reason is that the reverse, creating a view full of data and then assigning it to some row in storage as being a subview, takes twice as much memory (first, it's a standalone view, then it gets copied into storage). It's still possible to say: pSubProp (myview [n]) = a_view_full_of_data; ... but it's causes a lot more overhead. >> Is there useful documentation which describes how all of this works? The >> tutorial talks a little about nested views, but doesn't go into any real >> details about replacing data, deleting data, and so on. Some sample code in examples/ (in the MetaKit source .tar.gz), lots of little snippets of C++ code, all independent in the regression test suite - tests/. Contributions are most welcome. You can edit a page on the MetaKit wiki (or ask me to set it up for you if the process is not clear), or send me text to add - I'm sure we can find a good spot for it, in the distribution, on the web, whatever. I'm feeling the (increasing) pain of not having good docs for each of the currently supported C++, Python, and Tcl languages (actually, Tcl is covered best now, due to Mark Roseman's recent contribution). Make a difference! Become part of the MK story! Share in the fame! See your name in virtual gold on the web! Share in the pride of doing the right thing! (Just having fun... but I *do* mean it, nevertheless!) -jcw From andreas.olsson@ctakt.com Fri Apr 5 09:14:22 2002 Received: from linus.ctakt.com (linus.ctakt.com [62.119.45.234]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g35FELa06544 for ; Fri, 5 Apr 2002 09:14:22 -0600 Received: from flint ([192.168.2.115]) by linus.ctakt.com (8.11.6/8.11.2) with SMTP id g35FEKp03034 for ; Fri, 5 Apr 2002 17:14:20 +0200 From: "Andreas Olsson" To: Date: Fri, 5 Apr 2002 16:14:19 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] loss of data at commit?! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I just downloaded and tried MetaKit 2.4.4-34. It works fine until I commit my changes to a view. When I print the content of my view before commit it contains three columns (key, name, value). After storage.Commit() I only get column key when I print. Any one having a solution? /Andreas From jcw@equi4.com Fri Apr 5 09:25:21 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g35FPKa07233 for ; Fri, 5 Apr 2002 09:25:20 -0600 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] loss of data at commit?! Date: Fri, 5 Apr 2002 17:25:13 +0200 Message-Id: <20020405152513.26497@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas, >I just downloaded and tried MetaKit 2.4.4-34. It works fine until I commit >my changes to a view. When I print the content of my view before commit it >contains three columns (key, name, value). After storage.Commit() I only get >column key when I print. Did you use storage.GetAs("...") to define the view with all three columns? If not, the commit will store the columns it knows about, and make the in-memory data match the definition, i.e. drop the other two columns. -jcw From andreas.olsson@ctakt.com Sat Apr 6 09:47:19 2002 Received: from linus.ctakt.com (linus.ctakt.com [62.119.45.234]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g36FlIa26424 for ; Sat, 6 Apr 2002 09:47:19 -0600 Received: from flint ([192.168.2.115]) by linus.ctakt.com (8.11.6/8.11.2) with SMTP id g36FlDp05062 for ; Sat, 6 Apr 2002 17:47:13 +0200 From: "Andreas Olsson" To: Date: Sat, 6 Apr 2002 17:47:12 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] Problems with RemoveAt in 2.4.4-34 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! As I wrote in an earlier posting I'm a C++ newbie and also new to MetaKit. Therefore I apology for my simple questions. The problem is that I'm having a tight deadline for this project and I have no time to look up the solutions myself. Question: why doesn't view.RemoveAt(i) work? when I debug my application I end up in... void c4_FilterSeq::RemoveAt(int, int) { d4_assert(0); } where it of course fails. My code looks like this: void CTLocalDb::deleteRow(char *key) { c4_View deleteview; deleteview = view.Select(pKey [key]); int rowc = deleteview.GetSize(); deleteview.RemoveAt(0, rowc); // deleteview.SetSize(0); } Thanks / Andreas From jcw@equi4.com Tue Apr 9 10:37:20 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g39FbIa07345; Tue, 9 Apr 2002 10:37:18 -0500 From: Jean-Claude Wippler To: metakit Cc: will Date: Tue, 9 Apr 2002 17:36:47 +0200 Message-Id: <20020409153647.12221@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] tclkit crash-on-exit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There IS a crash-on-exit bug in the current TclKit windows builds. Reports vary as far as windows releases go - for some people it works on 2k but not 98, for others it works fine on 98. Several reports where launching and immediately stopping will crash on 98. I have one sequence of steps (involving alphatk) I run which crashes on all win platforms I tried, but *NOT* in the debug build. For some people things work fine all the time, even when trying to force the problem. In my most recent debugging rounds, crashes were never in Mk4tcl (though they have been in the past, perhaps there were multiple bugs). There may be an issue with Tk menu cleanup, but I'm not 100% sure, and even if accurate - knowing this does not solve things either, it merely would explain why the problem surfaces in different ways (even handling order, OS differences). Just uploaded which is a debug build made with MSVC6 (and quite large unpacked). So far, debug builds have never crashed on exit - which is one of the main reasons why I cannot resolve the issue: I can't make it fail in debug mode! If you run into crash-on-exit, I'd appreciate it is you could give the above version a try and tell me whether it works for you. If you get a crash and have MSVC6, please please go into debug mode and try to obtain a stack trace. Feel free to distribute the debug build, if it solves your problem for now. It is complete, but it's compiled with optimizations turned off. It's a very extreme measure, but it's all I can come up with... -jcw From smalllinux@netscape.net Tue Apr 9 15:40:42 2002 Received: from imo-r03.mx.aol.com (imo-r03.mx.aol.com [152.163.225.99]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g39Kega24535 for ; Tue, 9 Apr 2002 15:40:42 -0500 Received: from smalllinux@netscape.net by imo-r03.mx.aol.com (mail_out_v32.5.) id d.ed.383e25f (16238) for ; Tue, 9 Apr 2002 16:40:34 -0400 (EDT) Received: from netscape.com (mow-m11.webmail.aol.com [64.12.184.139]) by air-in03.mx.aol.com (v84.14) with ESMTP id MAILININ32-0409164034; Tue, 09 Apr 2002 16:40:34 -0500 Date: Tue, 09 Apr 2002 16:40:34 -0400 From: smalllinux@netscape.net To: metakit@equi4.com Message-ID: <5D30BF3B.5BBBD809.1080C5D1@netscape.net> X-Mailer: Atlas Mailer 2.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Subject: [Metakit] tclkit crash on exit - win32 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Ok, now I'm confused. Linudent crashes on exit on win98 using tclkit Jan 2002 build. On that same system, most tclkit applications exit fine, ml editor, wikit, etc. Linudent sources many different tcl files, uses kitten.bin, itcl and itk and iwidgets. Click on the window icon or exiting through the menu, both call a exit proc opening an iwidget. The last line of the dialog box would call the tcl exit builtin command. Using tclkit_d.exe does NOT crash on exit using the shutdown button choice. But it does crash on exit clicking the window icon. So I created a test script to show it. I don't get it. normal tclkit exit through shutdown = crash exit through windows = crash debug tclkit exit through shutdown = NO crash exit through windows = crash Steven G. http://linudent.sourceforge.net/ #!/usr/local/bin/tclkit package require scripdoc mk4vfs::mount kitten.bin ../libs/kitten.bin -readonly scripdoc::extendPath kitten.bin # main.tcl - application startup script package require Tk package require Itcl package require Itk package require Iwidgets # main.tcl - application startup script set MainGraphic drtux.gif set office_name testoffice set this_version "3.10" set version_date "April 9 2002" set beta_version "normal" set msglocale off proc dialog_position {dlg} { set width [winfo screenwidth .] set height [winfo screenwidth .] set x [expr {($width/2) - 260}] set y [expr {($height/2) - 300}] wm geometry $dlg +$x+$y } # copyright 2001 LinuDent # GPL version 2 #wm protocol . WM_DELETE_WINDOW shutdown #source exit_dialog.itk ;# provides proc shutdown # exit_application # we ask close before closing this is the parent of all modules proc exit_main_book {} { shutdown } wm title . Start_Book iwidgets::tabnotebook .notebook -tabpos n -equaltabs 0 -height 4i -width 6i\ -padx 3 -angle 30 -background cyan -backdrop grey65 set linudent [.notebook add -label [::msgcat::mc "linudent"] -command\ {destroy .notebook.canvas.notebook.button}] frame $linudent.frame -relief ridge -bd 3 -bg peachpuff label $linudent.frame.label -text [::msgcat::mc "$office_name"] pack $linudent.frame.label -side top -pady 2 pack $linudent.frame -fill both -expand 1 set status [.notebook add -label [::msgcat::mc "status"] -command\ {destroy .notebook.canvas.notebook.button}] label $status.label -text [::msgcat::mc "[::msgcat::mc "Status Screen\ - LinuDent"]"] pack $status.label -side top -pady 2 button $status.button -text [::msgcat::mc "[::msgcat::mc "click to reload Linudent config"]"] -command { load_config show_status puts "load config" } pack $status.button -side top -pady 2 proc show_status {} { destroy $::status.label1 label $::status.label1 -text [::msgcat::mc "[::msgcat::mc "You are running\ LinuDent version"] - $::this_version"] pack $::status.label1 destroy $::status.label2 label $::status.label2 -text [::msgcat::mc "[::msgcat::mc "This is the current\ version as of"] $::version_date "] pack $::status.label2 destroy $::status.label3 label $::status.label3 -text [::msgcat::mc "[::msgcat::mc "You are configured to\ use the"] $::beta_version version "] pack $::status.label3 destroy $::status.labelmsgcat label $::status.labelmsgcat -text [::msgcat::mc "[::msgcat::mc "Your locale is\ set to: "] [::msgcat::mclocale] "] pack $::status.labelmsgcat #frame $accounting.frame -bg green -container 1 -width 1400 -height 1200 #pack $accounting.frame -pady 37 } ;# end of proc show status show_status set modules [.notebook add -label [::msgcat::mc "modules"] -command\ {destroy .notebook.canvas.notebook.button}] label $modules.label -text [::msgcat::mc "modules Screen - LinuDent"] pack $modules.label -side top -pady 2 frame $modules.frame -height 500 -width 700 pack $modules.frame frame $modules.frame.buttonframe -height 500 -width 250 pack $modules.frame.buttonframe -side left -expand true -fill y frame $modules.frame.buttonframe2 -bg ivory -height 500 -width 700 pack $modules.frame.buttonframe2 -expand true -fill y set administration [.notebook add -label [::msgcat::mc "shutdown"]\ -command {destroy .notebook.canvas.notebook.button}] label $administration.label -text [::msgcat::mc "Shutdown Screen - LinuDent"] pack $administration.label -side top -pady 2 frame $administration.frame -height 500 -width 700 pack $administration.frame frame $administration.frame.buttonframe -height 100 -width 150 pack $administration.frame.buttonframe -side left -expand true -fill y frame $administration.frame.buttonframe2 -bg ivory -height 100 -width 700 pack $administration.frame.buttonframe2 -expand true -fill y button $administration.frame.buttonframe2.shutdown -text [::msgcat::mc\ "F10 Shut Down"] -bg red -fg yellow -command {shutdown} button $administration.frame.buttonframe.datashutdown -text [::msgcat::mc\ "shutdown database server"] -bg red -fg yellow -command { send databaseserver.tcl "exit" } button $administration.frame.buttonframe.demoshutdown -text [::msgcat::mc\ "close demo book"] -bg red -fg yellow -command { puts "hide patient book" } button $administration.frame.buttonframe2.schedshutdown -text [::msgcat::mc\ "close schedule book"] -bg red -fg yellow -command { puts "destroy scheduler" } button $administration.frame.buttonframe.reportshutdown -text [::msgcat::mc\ "close report book"] -bg red -fg yellow -command { puts "destroy reportbook" } button $administration.frame.buttonframe2.accountshutdown -text [::msgcat::mc\ "close account book"] -bg red -fg yellow -command { puts "destroy accountingbook" } pack $administration.frame.buttonframe.demoshutdown -expand true -fill x pack $administration.frame.buttonframe2.schedshutdown -expand true -fill x pack $administration.frame.buttonframe.datashutdown -expand true -fill x pack $administration.frame.buttonframe.reportshutdown -expand true -fill x pack $administration.frame.buttonframe2.accountshutdown -expand true -fill x pack $administration.frame.buttonframe2.shutdown -expand true -fill x frame .exit_frame -bd 0 button .exit_frame.exit -text [::msgcat::mc "Exit"] -command exit pack .exit_frame.exit pack .notebook -expand true -fill both .notebook select 1 update destroy .top update idletasks focus -force .notebook # ---------------------------------------------------------------------- # exit_dialog in [incr Widgets] # ---------------------------------------------------------------------- #\ # # Build a generic dialog # proc shutdown {} { if [winfo exist .exit_d] { .exit_d activate } else { iwidgets::dialog .exit_d .exit_d buttonconfigure OK -command { puts "pushed: OK" .exit_d deactivate 1 exit } .exit_d hide Apply .exit_d buttonconfigure Cancel -command { puts "pushed: Cancel" wm withdraw .exit_d .exit_d deactivate 0 } .exit_d hide Help # # Add something to the top of the dialog... # set win [.exit_d childsite] label $win.ex -text "Shuting down LinuDent\n(are you sure)" \ -background red \ -width 25 -height 5 pack $win.ex -expand yes -fill both -padx 4 -pady 4 .exit_d activate } focus -force .exit_d } __________________________________________________________________ Your favorite stores, helpful shopping tools and great gift ideas. Experience the convenience of buying online with Shop@Netscape! http://shopnow.netscape.com/ Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From guenther.fischer@hrz.tu-chemnitz.de Thu Apr 11 11:19:14 2002 Received: from meg.hrz.tu-chemnitz.de (mail@meg.hrz.tu-chemnitz.de [134.109.132.57]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3BGJDa08314 for ; Thu, 11 Apr 2002 11:19:13 -0500 Received: from pat.hrz.tu-chemnitz.de ([134.109.132.143] ident=mail) by meg.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #3) id 16vhI0-0005kg-00 for metakit@equi4.com; Thu, 11 Apr 2002 18:19:12 +0200 Received: from bruford.hrz.tu-chemnitz.de ([134.109.132.46] ident=root) by pat.hrz.tu-chemnitz.de with esmtp (Exim 3.32 #2) id 16vhI0-00014f-00 for metakit@equi4.com; Thu, 11 Apr 2002 18:19:12 +0200 Received: from localhost (fischer@localhost) by bruford.hrz.tu-chemnitz.de (8.9.3/8.9.3) with ESMTP id SAA16247 for ; Thu, 11 Apr 2002 18:19:11 +0200 X-Authentication-Warning: bruford.hrz.tu-chemnitz.de: fischer owned process doing -bs Date: Thu, 11 Apr 2002 18:19:11 +0200 (MEST) From: Guenther Fischer To: metakit@equi4.com In-Reply-To: <20020409153647.12221@triqs.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Metakit] tclkit error on new redhad skipjack Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I do a redhat system update and tclkit doesn't work anymore ... tclkit-x86-linux.bin: dynamic-link.h:62: elf_get_dynamic_info: Assertion `! "bad dynamic tag"' failed. I got this error with all the last versions any idea? -- ~Guenther Fischer From jcw@equi4.com Thu Apr 11 11:31:02 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3BGV1a09243 for ; Thu, 11 Apr 2002 11:31:01 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] tclkit error on new redhad skipjack Date: Thu, 11 Apr 2002 18:29:06 +0200 Message-Id: <20020411162906.3668@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Guenther, >I do a redhat system update and tclkit doesn't work anymore ... > >tclkit-x86-linux.bin: dynamic-link.h:62: elf_get_dynamic_info: Assertion >`! "bad dynamic tag"' failed. > >I got this error with all the last versions What is last versions, one of the following? What does the command "ldd tclkit-x86-linux.bin" return? And what does "file tclkit-x86-linux.bin" say? -jcw From jcw@equi4.com Fri Apr 12 08:29:59 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3CDTta25148; Fri, 12 Apr 2002 08:29:55 -0500 From: Jean-Claude Wippler To: Cc: Jeff Hobbs , Vince Darley , will Subject: Re: [Metakit] tclkit crash on exit - win32 Date: Fri, 12 Apr 2002 15:29:40 +0200 Message-Id: <20020412132940.31407@triqs.com> In-Reply-To: <5D30BF3B.5BBBD809.1080C5D1@netscape.net> References: <5D30BF3B.5BBBD809.1080C5D1@netscape.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The saga continues... I can now force a crash and get a stack trace with function names in it. You can help solve this issue, by downloading the new build of tclkit for Windows, which is a release build, but with symbols fully enabled: If you can make it crash *and* have MSVC6 installed, then you should be able to obtain a stack trace, by pressing "cancel" when the crash dialog appears. The stack trace is in the "call stack" windows, and can be copied and pasted to another application. It would be a great help if you could send those stack traces. I've got one on NT4, but before drawing conclusions it would be really helpful to see the issues on Windows 98 and 2k. The more there is to compare, the better. Please be sure to include info on windows version, and what you did to make the crash happen - in some cases a plain start + exit has been all it takes to trigger this problem. But even if you cannot reproduce or describe the exact steps taken before the crash - a stack trace would be most welcome. -jcw From jcw@equi4.com Sat Apr 13 06:13:04 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3DBD0a09390; Sat, 13 Apr 2002 06:13:00 -0500 From: Jean-Claude Wippler To: Cc: will , Jeff Hobbs Subject: Re: [Metakit] tclkit crash on exit - win32 Date: Sat, 13 Apr 2002 13:12:22 +0200 Message-Id: <20020413111222.22858@triqs.com> In-Reply-To: <20020412132940.31407@triqs.com> References: <20020412132940.31407@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: There are new win32* builds for tclkit at: http://www.equi4.com/pub/tk/newer/ This fixes (to be accurate: works around) a, what I'd call "fully identified", problem in Tk menu cleanup. Please use these latest builds. Many thanks to Jeff Hobbs for helping figure out this tricky bug. Remaining problems (oh, there'll always be some) are not necessarily the same issue, so please do continue to let me know if bad things happen. -jcw From joshm@taconic.net Sun Apr 14 23:40:29 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3F4eTa07339 for ; Sun, 14 Apr 2002 23:40:29 -0500 Received: from gatekeeper (ch-asc5-p40.taconic.net [205.231.28.232]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g3F4eRB14068 for ; Mon, 15 Apr 2002 00:40:27 -0400 (EDT) Message-ID: <0e6f01c1e437$576e1190$9e1de7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" Date: Mon, 15 Apr 2002 00:38:06 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Subject: [Metakit] How do I search for a substring? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Another novice question: I've got a column of names (say last names), and I want to do a lookup for some sort of typeahead completion in a UI. So I want to find all the names where the string in the column starts with some string. More specifically, the shortest one that matches (case INsensitive). Please tell me there is a better way then sorting the view and doing a binary search for the item. -- j Out The Door In 2K4! From nicolasb@maich.gr Mon Apr 15 08:09:03 2002 Received: from mail-server (mail.maich.gr [193.218.36.74]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3FD92a21105 for ; Mon, 15 Apr 2002 08:09:02 -0500 Received: FROM maich.gr BY mail-server ; Mon Apr 15 16:30:22 2002 +0300 Message-ID: <3CBAD11B.5060406@maich.gr> Date: Mon, 15 Apr 2002 16:09:47 +0300 From: Nicolas Boretos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.4) Gecko/20011128 Netscape6/6.2.1 X-Accept-Language: en-us MIME-Version: 1.0 To: metakit@equi4.com References: <20020412132940.31407@triqs.com> <20020413111222.22858@triqs.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: [Metakit] Serializing Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi, I serialize a metakit file as below, (in tclkit) mk::file open db data.dat set fd [open out.dat w] mk::file save db $fd close $fd The file shrinks from ~5Mb to 1.5Mb When I re-open the new out.dat,I notice that some "large string records" are truncated... Is this how to correctly serialize the file..??? Sincerely, Nicolas Boretos From antonio@mac.ps100.net Mon Apr 15 09:10:20 2002 Received: from mac.ps100.net (h00045a2a5bb8.ne.client2.attbi.com [65.96.154.86]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FEAHa23442 for ; Mon, 15 Apr 2002 09:10:20 -0500 Received: from tintin ([127.0.0.1] helo=mac.ps100.net) by mac.ps100.net with smtp (Exim 3.12 #1 (Debian)) id 16x76h-0006lu-00 for ; Mon, 15 Apr 2002 10:05:25 -0400 Message-ID: <2814131.1018879481642.JavaMail.upgradewebapp@mac.ps100.net> From: Antonio Rodriguez To: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit x-memora-stamp: 1018879480124 Date: Mon, 15 Apr 2002 10:05:25 -0400 Subject: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello all, Trying to build Metakit on a portable Mac (I'm temporarily away from my linux machines) and I am running into a problem during the compile phase. I'm not used to building anything on OSX, so I'm not quite sure of how to go about fixing this: cd metakit/builds ../unix/configure --enable-python << lots of autoconf stuff here >> make << lots of stuff, then error: >> c++ -shared .libs/column.o .libs/custom.o .libs/derived.o .libs/fileio.o .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o .libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o .libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4.so /usr/bin/ld: Undefined symbols: _main ___builtin_delete ___builtin_vec_delete ___builtin_vec_new ___throw _terminate__Fv ___builtin_new ___pure_virtual ___rtti_si ___rtti_user make: *** [libmk4.la] Error 1 Thanks Antonio PS Please cc me on replies as I am not a regular list member yet. From jcw@equi4.com Mon Apr 15 09:46:00 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FEjxa24928 for ; Mon, 15 Apr 2002 09:45:59 -0500 From: Jean-Claude Wippler To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? Date: Mon, 15 Apr 2002 16:38:11 +0200 Message-Id: <20020415143811.9400@triqs.com> In-Reply-To: <0e6f01c1e437$576e1190$9e1de7cd@gatekeeper> References: <0e6f01c1e437$576e1190$9e1de7cd@gatekeeper> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joshua Muskovitz wrote: >I've got a column of names (say last names), and I want to do a lookup for >some sort of typeahead completion in a UI. So I want to find all the names >where the string in the column starts with some string. More specifically, >the shortest one that matches (case INsensitive). The "shortest" confuses me, if I search for "abx" in the following: aa abxdd abxe abxff ac Which one(s) do you want? >Please tell me there is a better way then sorting the view and doing a >binary search for the item. Can you elaborate? What search mechanism would you consider optimal? -jcw From jcw@equi4.com Mon Apr 15 09:46:04 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FEk3a24946; Mon, 15 Apr 2002 09:46:03 -0500 From: Jean-Claude Wippler To: Cc: Antonio Rodriguez Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 Date: Mon, 15 Apr 2002 16:44:48 +0200 Message-Id: <20020415144448.20573@triqs.com> In-Reply-To: <2814131.1018879481642.JavaMail.upgradewebapp@mac.ps100.net> References: <2814131.1018879481642.JavaMail.upgradewebapp@mac.ps100.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Antonio Rodriguez wrote: >Trying to build Metakit on a portable Mac (I'm temporarily away from my >linux machines) and I am running into a problem during the compile phase. > >I'm not used to building anything on OSX, so I'm not quite sure of how to >go about fixing this: > >cd metakit/builds >../unix/configure --enable-python ><< lots of autoconf stuff here >> >make ><< lots of stuff, then error: >> > >c++ -shared .libs/column.o .libs/custom.o .libs/derived.o .libs/ >fileio.o .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o >.libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4.so >/usr/bin/ld: Undefined symbols: >_main >___builtin_delete >___builtin_vec_delete >___builtin_vec_new >___throw >_terminate__Fv >___builtin_new >___pure_virtual >___rtti_si >___rtti_user >make: *** [libmk4.la] Error 1 Ah yes, I've been struggling to understand this myself. What I had to do was edit builds/libtool and replace the "-shared" with "-dynamiclib" to make it work - it seems that otherwise c++ is trying to build a self- contained "main" executable, not a shared lib for use by others. That is of course a kludge. If there is a Mac OSX expert on this list who can help with this, please do follow up if possible. Libtool remains as clear as mud to me, so I'm essentially just feeling my way around in considerable darkness... >PS Please cc me on replies as I am not a regular list member yet. Done! -jcw From jcw@equi4.com Mon Apr 15 09:46:02 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FEk1a24933 for ; Mon, 15 Apr 2002 09:46:01 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Serializing Date: Mon, 15 Apr 2002 16:39:55 +0200 Message-Id: <20020415143955.4301@triqs.com> In-Reply-To: <3CBAD11B.5060406@maich.gr> References: <3CBAD11B.5060406@maich.gr> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Nicolas Boretos wrote: >I serialize a metakit file as below, (in tclkit) > >mk::file open db data.dat >set fd [open out.dat w] >mk::file save db $fd >close $fd > >The file shrinks from ~5Mb to 1.5Mb > >When I re-open the new out.dat,I notice that some "large string records" >are truncated... > >Is this how to correctly serialize the file..??? Yes. Note that there have been recent updates w.r.t serialization, if you are using the latest builds of TclKit, then you may have found a bug. -jcw From joshm@taconic.net Mon Apr 15 11:32:18 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FGWIa29811 for ; Mon, 15 Apr 2002 11:32:18 -0500 Received: from gatekeeper (ch-as5396-2-p86.taconic.net [205.231.30.133]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g3FGW8f11360 for ; Mon, 15 Apr 2002 12:32:09 -0400 (EDT) Message-ID: <0ec901c1e49a$c1df54e0$9e1de7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: <0e6f01c1e437$576e1190$9e1de7cd@gatekeeper> <20020415143811.9400@triqs.com> Subject: Re: [Metakit] How do I search for a substring? Date: Mon, 15 Apr 2002 12:29:45 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > The "shortest" confuses me, if I search for "abx" in the following: > aa > abxdd > abxe > abxff > ac > Which one(s) do you want? Silly me. After all these years, you'd think I could write a more coherent question. Given a column with: aa aabx abxdd abxe abxff ac I have two search questions: 1. LefthandFind("abx") should find "abxe". This is the string which matches the input string from the left (ala CString::StartsWith()) *and* is the shortest (CString::GetLength()) of the possible results. If two strings match this criteria, then I'm flexible on which one to find. 2. FindAllContaining("abx") should find *all* rows containing "abx" as a substring. (CString::Find() != -1). This would find 4 rows. They could be returned as a view, or there could be some sort of iterator. -- j From steve@improvision.com Mon Apr 15 11:45:17 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FGjHa30557 for ; Mon, 15 Apr 2002 11:45:17 -0500 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id RAA03849; Mon, 15 Apr 2002 17:42:16 +0100 (BST) Received: from [192.168.1.82] ([192.168.1.82]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971; Mon, 15 Apr 2002 17:45:10 +0100 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Mon, 15 Apr 2002 17:45:10 +0100 Subject: Re: [Metakit] How do I search for a substring? From: Stephen Baxter To: Joshua Muskovitz , MetaKit Mailing List Message-ID: In-Reply-To: <0ec901c1e49a$c1df54e0$9e1de7cd@gatekeeper> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A brute force search is actually blazingly fast in Metakit (the latency for getting a string from the database is very low) - how many rows are you talking about? -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: "Joshua Muskovitz" > Date: Mon, 15 Apr 2002 12:29:45 -0400 > To: "MetaKit Mailing List" > Subject: Re: [Metakit] How do I search for a substring? > >> The "shortest" confuses me, if I search for "abx" in the following: >> aa >> abxdd >> abxe >> abxff >> ac >> Which one(s) do you want? > > Silly me. After all these years, you'd think I could write a more coherent > question. > > Given a column with: > > aa > aabx > abxdd > abxe > abxff > ac > > I have two search questions: > > 1. LefthandFind("abx") should find "abxe". This is the string which > matches the input string from the left (ala CString::StartsWith()) *and* is > the shortest (CString::GetLength()) of the possible results. If two strings > match this criteria, then I'm flexible on which one to find. > > 2. FindAllContaining("abx") should find *all* rows containing "abx" as a > substring. (CString::Find() != -1). This would find 4 rows. They could be > returned as a view, or there could be some sort of iterator. > > -- j > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From joshm@taconic.net Mon Apr 15 11:48:37 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3FGmaa30768 for ; Mon, 15 Apr 2002 11:48:36 -0500 Received: from gatekeeper (ch-as5396-2-p86.taconic.net [205.231.30.133]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g3FGmYf16418 for ; Mon, 15 Apr 2002 12:48:35 -0400 (EDT) Message-ID: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: Subject: Re: [Metakit] How do I search for a substring? Date: Mon, 15 Apr 2002 12:46:08 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > A brute force search is actually blazingly fast in Metakit (the latency for > getting a string from the database is very low) - how many rows are you > talking about? I would say in the 10k - 100k row range. -- j From lvirden@cas.org Tue Apr 16 06:03:35 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3GB3Za13899 for ; Tue, 16 Apr 2002 06:03:35 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g3GB3SS11181 for ; Tue, 16 Apr 2002 07:03:28 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g3GB3RY16631; Tue, 16 Apr 2002 07:03:27 -0400 (EDT) Date: Tue, 16 Apr 2002 07:03:27 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020416070327.AAB16622@cas.org> To: "MetaKit Mailing List" Subject: [Metakit] Submit your papers! Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: It is time to submit your abstract for a paper or tutorial to be presented at the Tcl Conference! This is your chance to let others know what you are doing. Don't think "Oh, I just don't have anything to contribute to the conference" - letting people know what you are doing helps others as well as helps yourself! See the details at the conference web page! Join us in Sept. for Tcl'2002: http://www.tcl.tk/community/tcl2002/ -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From tciuro@yahoo.com Tue Apr 16 10:18:34 2002 Received: from web20606.mail.yahoo.com (web20606.mail.yahoo.com [216.136.226.164]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3GFIYa26326 for ; Tue, 16 Apr 2002 10:18:34 -0500 Message-ID: <20020416151829.89572.qmail@web20606.mail.yahoo.com> Received: from [63.202.173.197] by web20606.mail.yahoo.com via HTTP; Tue, 16 Apr 2002 08:18:29 PDT Date: Tue, 16 Apr 2002 08:18:29 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 To: Jean-Claude Wippler , metakit@equi4.com Cc: Antonio Rodriguez In-Reply-To: <20020415144448.20573@triqs.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, On Mac OS X, I decided from day #1 to use a different route: use ProjectBuilder (it's included with Mac OS X) and package everything in a framework. Forget about those Libtool blues... It's dead easy and it works fine. -- Tito --- Jean-Claude Wippler wrote: > Antonio Rodriguez wrote: > > >Trying to build Metakit on a portable Mac (I'm > temporarily away from my > >linux machines) and I am running into a problem > during the compile phase. > > > >I'm not used to building anything on OSX, so I'm > not quite sure of how to > >go about fixing this: > > > >cd metakit/builds > >../unix/configure --enable-python > ><< lots of autoconf stuff here >> > >make > ><< lots of stuff, then error: >> > > > >c++ -shared .libs/column.o .libs/custom.o > .libs/derived.o .libs/ > >fileio.o .libs/field.o .libs/format.o > .libs/handler.o .libs/persist.o > >.libs/remap.o .libs/std.o .libs/store.o > .libs/string.o .libs/table.o > >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o > .libs/libmk4.so > >/usr/bin/ld: Undefined symbols: > >_main > >___builtin_delete > >___builtin_vec_delete > >___builtin_vec_new > >___throw > >_terminate__Fv > >___builtin_new > >___pure_virtual > >___rtti_si > >___rtti_user > >make: *** [libmk4.la] Error 1 > > Ah yes, I've been struggling to understand this > myself. What I had to do > was edit builds/libtool and replace the "-shared" > with "-dynamiclib" to > make it work - it seems that otherwise c++ is trying > to build a self- > contained "main" executable, not a shared lib for > use by others. That is > of course a kludge. > > If there is a Mac OSX expert on this list who can > help with this, please > do follow up if possible. Libtool remains as clear > as mud to me, so I'm > essentially just feeling my way around in > considerable darkness... > > >PS Please cc me on replies as I am not a regular > list member yet. > > Done! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From antonio@mac.ps100.net Tue Apr 16 10:46:46 2002 Received: from mac.ps100.net (h00045a2a5bb8.ne.client2.attbi.com [65.96.154.86]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3GFkka28272; Tue, 16 Apr 2002 10:46:46 -0500 Received: from tintin ([127.0.0.1] helo=mac.ps100.net) by mac.ps100.net with smtp (Exim 3.12 #1 (Debian)) id 16xV5C-0000Cm-00; Tue, 16 Apr 2002 11:41:26 -0400 Message-ID: <6667222.1018971656287.JavaMail.upgradewebapp@mac.ps100.net> From: Antonio Rodriguez To: jcw@equi4.com, metakit@equi4.com, tciuro@yahoo.com Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 Cc: metakit@equi4.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit x-memora-stamp: 1018971654816 Date: Tue, 16 Apr 2002 11:41:26 -0400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, Do you have your project builder project file? I'm not familiar with PB, but I'm assuming that it saves project options (compilation flags, target dirs, etc) to some sort of file(s). If so, maybe the could be included in an ext/ dir in the metakit dir. Since everyone with OSX has PB by default, this might be pretty handy. Thanks Antonio -|Tito Ciuro wrote on Tue Apr 16 11:14:12 EDT 2002: Hello, On Mac OS X, I decided from day #1 to use a different route: use ProjectBuilder (it's included with Mac OS X) and package everything in a framework. Forget about those Libtool blues... It's dead easy and it works fine. -- Tito --- Jean-Claude Wippler wrote: > Antonio Rodriguez wrote: > > >Trying to build Metakit on a portable Mac (I'm > temporarily away from my > >linux machines) and I am running into a problem > during the compile phase. > > > >I'm not used to building anything on OSX, so I'm > not quite sure of how to > >go about fixing this: > > > >cd metakit/builds > >../unix/configure --enable-python > ><< lots of autoconf stuff here >> > >make > ><< lots of stuff, then error: >> > > > >c++ -shared .libs/column.o .libs/custom.o > .libs/derived.o .libs/ > >fileio.o .libs/field.o .libs/format.o > .libs/handler.o .libs/persist.o > >.libs/remap.o .libs/std.o .libs/store.o > .libs/string.o .libs/table.o > >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o > .libs/libmk4.so > >/usr/bin/ld: Undefined symbols: > >_main > >___builtin_delete > >___builtin_vec_delete > >___builtin_vec_new > >___throw > >_terminate__Fv > >___builtin_new > >___pure_virtual > >___rtti_si > >___rtti_user > >make: *** [libmk4.la] Error 1 > > Ah yes, I've been struggling to understand this > myself. What I had to do > was edit builds/libtool and replace the "-shared" > with "-dynamiclib" to > make it work - it seems that otherwise c++ is trying > to build a self- > contained "main" executable, not a shared lib for > use by others. That is > of course a kludge. > > If there is a Mac OSX expert on this list who can > help with this, please > do follow up if possible. Libtool remains as clear > as mud to me, so I'm > essentially just feeling my way around in > considerable darkness... > > >PS Please cc me on replies as I am not a regular > list member yet. > > Done! > > -jcw > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From tciuro@yahoo.com Tue Apr 16 19:41:55 2002 Received: from web20606.mail.yahoo.com (web20606.mail.yahoo.com [216.136.226.164]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3H0fsa03263 for ; Tue, 16 Apr 2002 19:41:54 -0500 Message-ID: <20020417004152.21010.qmail@web20606.mail.yahoo.com> Received: from [17.202.13.44] by web20606.mail.yahoo.com via HTTP; Tue, 16 Apr 2002 17:41:52 PDT Date: Tue, 16 Apr 2002 17:41:52 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 To: Antonio Rodriguez , jcw@equi4.com, metakit@equi4.com Cc: metakit@equi4.com In-Reply-To: <6667222.1018971656287.JavaMail.upgradewebapp@mac.ps100.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: No problem... Jean-Claude? Do you have any comments on this? Can I send you the pbproj file containing the settings? BTW... The pbproj has been created with the latest PB, which will be delivered in WWDC 2002 (May 6th). You should be able to open it with the the version delivered with 10.1.X though... Otherwise, let me know and I'll see what I can do... -- Tito --- Antonio Rodriguez wrote: > Tito, > > Do you have your project builder project file? I'm > not familiar with PB, but I'm assuming that it saves > project options (compilation flags, target dirs, > etc) to some sort of file(s). > > If so, maybe the could be included in an ext/ dir in > the metakit dir. Since everyone with OSX has PB by > default, this might be pretty handy. > > Thanks > > Antonio > > > -|Tito Ciuro wrote on Tue Apr 16 > 11:14:12 EDT 2002: > > Hello, > > On Mac OS X, I decided from day #1 to use a > different > route: use ProjectBuilder (it's included with Mac OS > X) and package everything in a framework. Forget > about > those Libtool blues... > > It's dead easy and it works fine. > > -- Tito > > --- Jean-Claude Wippler wrote: > > Antonio Rodriguez wrote: > > > > >Trying to build Metakit on a portable Mac (I'm > > temporarily away from my > > >linux machines) and I am running into a problem > > during the compile phase. > > > > > >I'm not used to building anything on OSX, so I'm > > not quite sure of how to > > >go about fixing this: > > > > > >cd metakit/builds > > >../unix/configure --enable-python > > ><< lots of autoconf stuff here >> > > >make > > ><< lots of stuff, then error: >> > > > > > >c++ -shared .libs/column.o .libs/custom.o > > .libs/derived.o .libs/ > > >fileio.o .libs/field.o .libs/format.o > > .libs/handler.o .libs/persist.o > > >.libs/remap.o .libs/std.o .libs/store.o > > .libs/string.o .libs/table.o > > >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o > > .libs/libmk4.so > > >/usr/bin/ld: Undefined symbols: > > >_main > > >___builtin_delete > > >___builtin_vec_delete > > >___builtin_vec_new > > >___throw > > >_terminate__Fv > > >___builtin_new > > >___pure_virtual > > >___rtti_si > > >___rtti_user > > >make: *** [libmk4.la] Error 1 > > > > Ah yes, I've been struggling to understand this > > myself. What I had to do > > was edit builds/libtool and replace the "-shared" > > with "-dynamiclib" to > > make it work - it seems that otherwise c++ is > trying > > to build a self- > > contained "main" executable, not a shared lib for > > use by others. That is > > of course a kludge. > > > > If there is a Mac OSX expert on this list who can > > help with this, please > > do follow up if possible. Libtool remains as > clear > > as mud to me, so I'm > > essentially just feeling my way around in > > considerable darkness... > > > > >PS Please cc me on replies as I am not a regular > > list member yet. > > > > Done! > > > > -jcw > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > __________________________________________________ > Do You Yahoo!? > Yahoo! Tax Center - online filing with TurboTax > http://taxes.yahoo.com/ > > __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From tciuro@yahoo.com Tue Apr 16 19:41:55 2002 Received: from web20606.mail.yahoo.com (web20606.mail.yahoo.com [216.136.226.164]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3H0fta03265 for ; Tue, 16 Apr 2002 19:41:55 -0500 Message-ID: <20020417004152.21010.qmail@web20606.mail.yahoo.com> Received: from [17.202.13.44] by web20606.mail.yahoo.com via HTTP; Tue, 16 Apr 2002 17:41:52 PDT Date: Tue, 16 Apr 2002 17:41:52 -0700 (PDT) From: Tito Ciuro Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 To: Antonio Rodriguez , jcw@equi4.com, metakit@equi4.com Cc: metakit@equi4.com In-Reply-To: <6667222.1018971656287.JavaMail.upgradewebapp@mac.ps100.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: No problem... Jean-Claude? Do you have any comments on this? Can I send you the pbproj file containing the settings? BTW... The pbproj has been created with the latest PB, which will be delivered in WWDC 2002 (May 6th). You should be able to open it with the the version delivered with 10.1.X though... Otherwise, let me know and I'll see what I can do... -- Tito --- Antonio Rodriguez wrote: > Tito, > > Do you have your project builder project file? I'm > not familiar with PB, but I'm assuming that it saves > project options (compilation flags, target dirs, > etc) to some sort of file(s). > > If so, maybe the could be included in an ext/ dir in > the metakit dir. Since everyone with OSX has PB by > default, this might be pretty handy. > > Thanks > > Antonio > > > -|Tito Ciuro wrote on Tue Apr 16 > 11:14:12 EDT 2002: > > Hello, > > On Mac OS X, I decided from day #1 to use a > different > route: use ProjectBuilder (it's included with Mac OS > X) and package everything in a framework. Forget > about > those Libtool blues... > > It's dead easy and it works fine. > > -- Tito > > --- Jean-Claude Wippler wrote: > > Antonio Rodriguez wrote: > > > > >Trying to build Metakit on a portable Mac (I'm > > temporarily away from my > > >linux machines) and I am running into a problem > > during the compile phase. > > > > > >I'm not used to building anything on OSX, so I'm > > not quite sure of how to > > >go about fixing this: > > > > > >cd metakit/builds > > >../unix/configure --enable-python > > ><< lots of autoconf stuff here >> > > >make > > ><< lots of stuff, then error: >> > > > > > >c++ -shared .libs/column.o .libs/custom.o > > .libs/derived.o .libs/ > > >fileio.o .libs/field.o .libs/format.o > > .libs/handler.o .libs/persist.o > > >.libs/remap.o .libs/std.o .libs/store.o > > .libs/string.o .libs/table.o > > >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o > > .libs/libmk4.so > > >/usr/bin/ld: Undefined symbols: > > >_main > > >___builtin_delete > > >___builtin_vec_delete > > >___builtin_vec_new > > >___throw > > >_terminate__Fv > > >___builtin_new > > >___pure_virtual > > >___rtti_si > > >___rtti_user > > >make: *** [libmk4.la] Error 1 > > > > Ah yes, I've been struggling to understand this > > myself. What I had to do > > was edit builds/libtool and replace the "-shared" > > with "-dynamiclib" to > > make it work - it seems that otherwise c++ is > trying > > to build a self- > > contained "main" executable, not a shared lib for > > use by others. That is > > of course a kludge. > > > > If there is a Mac OSX expert on this list who can > > help with this, please > > do follow up if possible. Libtool remains as > clear > > as mud to me, so I'm > > essentially just feeling my way around in > > considerable darkness... > > > > >PS Please cc me on replies as I am not a regular > > list member yet. > > > > Done! > > > > -jcw > > > > _______________________________________________ > > metakit mailing list - metakit@equi4.com > > http://www.equi4.com/mailman/listinfo/metakit > > > __________________________________________________ > Do You Yahoo!? > Yahoo! Tax Center - online filing with TurboTax > http://taxes.yahoo.com/ > > __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From jcw@equi4.com Wed Apr 17 01:46:48 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3H6kla18924; Wed, 17 Apr 2002 01:46:47 -0500 From: Jean-Claude Wippler To: Antonio Rodriguez , Date: Wed, 17 Apr 2002 08:46:30 +0200 Message-Id: <20020417064630.1458@triqs.com> In-Reply-To: <20020417004152.21010.qmail@web20606.mail.yahoo.com> References: <20020417004152.21010.qmail@web20606.mail.yahoo.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Taking if further (Was: Building metakit 2.3.34 on Mac OSX 10.1.3) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Tito, [macosx ProjectBuilder] >Jean-Claude? Do you have any comments on this? Can I >send you the pbproj file containing the settings? I'd be delighted to add a PB setup to the MK distribution. The deliverables that I think will matter most are - in decreasing order of importance: - MK static library - MK shared library - regression test (non-gui) application - debug build of the above, to help catch more issues As you know, i have PB set up (dec 2001 developer CD). I'll be more than happy to help tweak, test, extend it as needed. PB is still new to me, but I got surprising mileage out of it so far, it really is a great setup harnessing the gcc and gdb tools - so *YES* please! >BTW... The pbproj has been created with the latest PB, >which will be delivered in WWDC 2002 (May 6th). You >should be able to open it with the the version >delivered with 10.1.X though... Good point. That is indeed a concern - if this thing also works for people who do not have access to the latest and greatest PB, then so much the better. Let me re-iterate that I welcome all contributions to MetaKit: large and small, code and docs. MK might look like a one-man show to some, but it isn't by far anymore - and the more you help improve it in ways *you* find important, the better. MK is a foundation for almost everything I do, and I know a couple companies and people who pound & rely *very* heavily on it. You can take it forward too - trust me, MK's future looks extremely bright, for all of C++, Python, and Tcl. For reasons of confidentiality, I cannot always list all people involved, but I do always try to acknowledge all contributions as much as I can on the MK, Python, and Tcl web pages. Go for it, Tito & Antonio :) -jcw PS. No need to CC me, as I get all email from the list anyway. From jcw@equi4.com Wed Apr 17 02:08:54 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3H78ra20021 for ; Wed, 17 Apr 2002 02:08:53 -0500 From: Jean-Claude Wippler To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? Date: Wed, 17 Apr 2002 09:08:31 +0200 Message-Id: <20020417070832.22300@triqs.com> In-Reply-To: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Joshua Muskovitz wrote: [Stephen Baxter] >> A brute force search is actually blazingly fast in Metakit >> [...] - how many rows are you talking about? > >I would say in the 10k - 100k row range. Ok, if you want instant GUI responsiveness per key hit, and given that you're doing this with strings, not numbers (which are one step faster still), you *might* need to add tricks. There are plenty to think of IMO. > 1. LefthandFind("abx") should find "abxe". Use c4_View's Search or Locate. They will not find a match, but they will find the position of the first entry >= your key (lexicographically, and ignoring case). This uses binary search. If the results are not as you want them, it'd be very easy to write a binary search yourself, and it'd be just as fast as these routines. > 2. FindAllContaining("abx") That is - without extra indexing preparations - not doable through any other means than brute force linear search. Use a loop from 0 to GetSize()-1 and do a strstr-like match yourself. The key to performance is to not create a copy of the string you get from MK. In 99% of the cases, you will be accessing a string which is in memory-mapped file, i.e. *straight* off the paged-in virtual memory mapped over the search column. To give an indication of speed: it would not surprise me if *even* brute force for both requirements leads to tenths-of-a-second access for 100k rows on today's desktop hardware. My suggestion would be to just try the brute force approach, as Stephen suggested. Avoid string copies, definitely avoid constructing a string object on each iteration, and you might just find out that "simplest" works fine. If you needs more speed in case #1, switch to the binary search. Of you need to optimize case #2, consider creating an extra int column, perhaps a bitmap for all the letters in the item - and then do a loop using "if ((bitmap & searchbits) == searchbits)". I've used similar tricks for full-text keyword searches at times, with very good results. -jcw From andreas.muegge@gmx.de Wed Apr 17 03:25:43 2002 Received: from mx0.gmx.net (mx0.gmx.de [213.165.64.100]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3H8Pha23624 for ; Wed, 17 Apr 2002 03:25:43 -0500 Received: (qmail 3783 invoked by uid 0); 17 Apr 2002 08:25:36 -0000 Date: Wed, 17 Apr 2002 10:25:35 +0200 (MEST) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: MetaKit Mailing List MIME-Version: 1.0 References: <20020417070832.22300@triqs.com> Subject: Re: [Metakit] How do I search for a substring? X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [217.194.32.16] Message-ID: <22644.1019031935@www47.gmx.net> X-Mailer: WWW-Mail 1.5 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi Jean-Claude, [fulltext search] > If you needs more speed in case #1, switch to the binary search. Of you > need to optimize case #2, consider creating an extra int column, perhaps > a bitmap for all the letters in the item - and then do a loop using "if > ((bitmap & searchbits) == searchbits)". I've used similar tricks for > full-text keyword searches at times, with very good results. Can you show me a simple example for case 2? I much interested in speeding up my substring search. It's astonishing fast already but decreasing the response time for the user even further would be great. Andreas Muegge From ken@clres.com Wed Apr 17 10:03:12 2002 Received: from mail.his.com (root@mail.his.com [216.194.225.77]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HF3Ca14022; Wed, 17 Apr 2002 10:03:12 -0500 Received: from clres.com (max1k-10.his.com [216.194.203.10]) by mail.his.com (8.9.3/8.9.3) with ESMTP id LAA06723; Wed, 17 Apr 2002 11:03:06 -0400 (EDT) Message-ID: <3CBD8EA9.8080709@clres.com> Date: Wed, 17 Apr 2002 11:03:05 -0400 From: Ken Litkowski Organization: CL Research User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Jean-Claude Wippler CC: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Jean-Claude Wippler wrote: >>1. LefthandFind("abx") should find "abxe". > > Use c4_View's Search or Locate. They will not find a match, but they > will find the position of the first entry >= your key (lexicographically, > and ignoring case). This uses binary search. If the results are not as > you want them, it'd be very easy to write a binary search yourself, and > it'd be just as fast as these routines. > Are you saying that this is case-insensitive, so that using "Search" or "search" will return the same position when using Search or Locate? I have always gone through elaborate procedures to deal with case and it would certainly help to know that Search and Locate are case-insensitive. Ken -- Ken Litkowski TEL.: 301-482-0237 CL Research EMAIL: ken@clres.com 9208 Gue Road Damascus, MD 20872-1025 USA Home Page: http://www.clres.com From steve@improvision.com Wed Apr 17 10:16:32 2002 Received: from mail1lo.highwayone.net (mail1lo.highwayone.net [195.70.64.22]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HFGVa15075; Wed, 17 Apr 2002 10:16:31 -0500 Received: from mail.improvision.com ([62.232.123.113]) by mail1lo.highwayone.net (8.9.3+Sun/8.9.3) with ESMTP id QAA00847; Wed, 17 Apr 2002 16:13:32 +0100 (BST) Received: from [192.168.1.82] ([192.168.1.82]) by mail.improvision.com (Merak 4.2.2) with ESMTP id LVC36971; Wed, 17 Apr 2002 16:16:27 +0100 User-Agent: Microsoft-Outlook-Express-Macintosh-Edition/5.02.2022 Date: Wed, 17 Apr 2002 16:16:28 +0100 Subject: Re: [Metakit] How do I search for a substring? From: Stephen Baxter To: Ken Litkowski , Jean-Claude Wippler CC: MetaKit Mailing List Message-ID: In-Reply-To: <3CBD8EA9.8080709@clres.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I found Search() and Find() could be slow for large datasets because of the number of virtual function calls involved. I ended up writing my own (simple) routine for doing case-insensitive string searches: /*************************************************************************** ************ FindString Find the string s in property prop in view. Returns the row if found or -1 otherwise. This basically replaces c4_View::Find() with a more efficient implementation. *************************************************************************** ************/ SInt32 XMetakitDB::FindString( const XString& s1, const c4_StringProp& prop, const c4_View& view, SInt32 start ) { // Do a direct comparison for speed register char c1; register char c2; char delta = 'A'-'a'; SInt32 count = view.GetSize(); SInt32 index = start; while( index < count ) { // Get the property const char *p1 = prop.Get(view[index]); // Do the compare const char *p2 = s1; while( 1 ) { c1 = *p1; c2 = *p2; p1++; p2++; // OK, we have two valid chars - make them both upper case if( (c1 >= 'a') && (c1 <= 'z') ) { c1 += delta; } if( (c2 >= 'a') && (c2 <= 'z') ) { c2 += delta; } if( c1 != c2 ) { // Strings are different break; } else if( !c1 ) { // We have reached the end of both of the strings // and they are identical return index; } } index++; } return -1; } -- Stephen Baxter Development Manager Improvision steve@improvision.com Tel:+44-2476-692229 Fax:+44-2476-690091 ------------------------------------------------------ Volocity - New Dimensions in High Performance Imaging http://www.improvision.com/products/Volocity ------------------------------------------------------ > From: Ken Litkowski > Organization: CL Research > Date: Wed, 17 Apr 2002 11:03:05 -0400 > To: Jean-Claude Wippler > Cc: MetaKit Mailing List > Subject: Re: [Metakit] How do I search for a substring? > > Jean-Claude Wippler wrote: > >>> 1. LefthandFind("abx") should find "abxe". >> >> Use c4_View's Search or Locate. They will not find a match, but they >> will find the position of the first entry >= your key (lexicographically, >> and ignoring case). This uses binary search. If the results are not as >> you want them, it'd be very easy to write a binary search yourself, and >> it'd be just as fast as these routines. >> > > > Are you saying that this is case-insensitive, so that using "Search" or > "search" will return the same position when using Search or Locate? I > have always gone through elaborate procedures to deal with case and it > would certainly help to know that Search and Locate are case-insensitive. > > Ken > -- > Ken Litkowski TEL.: 301-482-0237 > CL Research EMAIL: ken@clres.com > 9208 Gue Road > Damascus, MD 20872-1025 USA Home Page: http://www.clres.com > > _______________________________________________ > metakit mailing list - metakit@equi4.com > http://www.equi4.com/mailman/listinfo/metakit From joshm@taconic.net Wed Apr 17 14:57:50 2002 Received: from mail.taconic.net (mail.taconic.net [205.231.144.5]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HJvoa03243 for ; Wed, 17 Apr 2002 14:57:50 -0500 Received: from gatekeeper (ch-asc7-p6.taconic.net [205.231.148.54]) by mail.taconic.net (8.11.6+Sun/8.11.4) with SMTP id g3HJvmf13732 for ; Wed, 17 Apr 2002 15:57:48 -0400 (EDT) Message-ID: <00ab01c1e649$cb0a5e60$0f94e7cd@gatekeeper> From: "Joshua Muskovitz" To: "MetaKit Mailing List" References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> <3CBD8EA9.8080709@clres.com> Subject: Re: [Metakit] How do I search for a substring? Date: Wed, 17 Apr 2002 15:55:14 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: > Are you saying that this is case-insensitive, so that using "Search" or > "search" will return the same position when using Search or Locate? I > have always gone through elaborate procedures to deal with case and it > would certainly help to know that Search and Locate are case-insensitive. We solved this problem by keeping an additional column of forced lower-case representations (not that much waste, given our particular design), and forcing the search key to lower case before doing the search. J-C, The idea of keeping a bitfield of used characters is ingenious, and it fits into an int! I'll have to contemplate whether this is useful in our particular application, but I'm sure I'll use it at some point in my life. Thanks for another cool algorithm to add to my heap. -- j From DavidA@ActiveState.com Wed Apr 17 15:09:15 2002 Received: from smtp1.ActiveState.com (gw.activestate.com [209.17.183.249]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HK9Fa04020 for ; Wed, 17 Apr 2002 15:09:15 -0500 Received: from smtp3.ActiveState.com (smtp3.ActiveState.com [192.168.3.19]) by smtp1.ActiveState.com (8.11.6/8.11.6) with ESMTP id g3HK8A228785 for ; Wed, 17 Apr 2002 13:08:10 -0700 Received: from ActiveState.com (loom.activestate.com [192.168.3.137]) by smtp3.ActiveState.com (8.11.6/8.11.6) with ESMTP id g3HK88e03614 for ; Wed, 17 Apr 2002 13:08:08 -0700 Message-ID: <3CBDD628.6090908@ActiveState.com> Date: Wed, 17 Apr 2002 13:08:08 -0700 From: David Ascher User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.9) Gecko/20020311 X-Accept-Language: en-us, en MIME-Version: 1.0 To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Filtered-By: PerlMx makes it fast and easy. See http://www.ActiveState.com/Products/PerlMx/Header Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: The discussion re: how to do text searches in metakit makes me wonder if someone (jcw?) wouldn't want to use metakit to come up with a reusable full text indexer that was more portable and easier to integrate than any of the other open source ones I've seen. I've spent more hours than I care to admit researching the topic, and finding basically nothing which is both high-performing, under a non-restrictive license, portable, and easy to setup... There's a lot of blind alleys to go in when it comes to full text searching, but I think that JCW could come up with a framework that would solve the problem for 90% of the cases where it's likely to be used (inside apps for on-line help, inside simple web sites, etc.). Just a suggestion. --da From readonly@getsoft.com Wed Apr 17 15:54:34 2002 Received: from freedom.vosn.net ([157.238.46.77]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HKsYa06981 for ; Wed, 17 Apr 2002 15:54:34 -0500 Received: from c18520.eburwd2.vic.optusnet.com.au ([210.49.191.115] helo=192.168.100.153) by freedom.vosn.net with esmtp (Exim 3.35 #1) id 16xwRU-0008B0-00; Wed, 17 Apr 2002 14:54:16 -0600 Date: Thu, 18 Apr 2002 06:54:16 +1000 From: Neville Franks X-Mailer: The Bat! (v1.53d) Reply-To: Neville Franks Organization: Soft As It Gets Pty Ltd X-Priority: 3 (Normal) Message-ID: <16346665221.20020418065416@getsoft.com> To: David Ascher CC: MetaKit Mailing List Subject: Re[2]: [Metakit] How do I search for a substring? In-Reply-To: <3CBDD628.6090908@ActiveState.com> References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> <3CBDD628.6090908@ActiveState.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - freedom.vosn.net X-AntiAbuse: Original Domain - equi4.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - getsoft.com Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi David, I'm not exactly sure what you are looking for when you say "full text indexer" but you may find Mifluz http://www.gnu.org/software/mifluz/ of interest. From memory there are others like this as well. Thursday, April 18, 2002, 6:08:08 AM, you wrote: DA> The discussion re: how to do text searches in metakit makes me wonder if DA> someone (jcw?) wouldn't want to use metakit to come up with a reusable DA> full text indexer that was more portable and easier to integrate than DA> any of the other open source ones I've seen. I've spent more hours than DA> I care to admit researching the topic, and finding basically nothing DA> which is both high-performing, under a non-restrictive license, DA> portable, and easy to setup... DA> There's a lot of blind alleys to go in when it comes to full text DA> searching, but I think that JCW could come up with a framework that DA> would solve the problem for 90% of the cases where it's likely to be DA> used (inside apps for on-line help, inside simple web sites, etc.). DA> Just a suggestion. -- Best regards, Neville Franks, Author of ED for Windows - the programmers editor, an indispensable tool for great software development. http://www.getsoft.com Version 4.01 now available. New: Projects, Source Database, Difference Analysis.... Soft As It Gets Pty Ltd, 12 Fairview Grove, Glen Iris, VIC, 3146 Australia Ph +61 3 9885 4445, Fax +61 3 9885 4444, mailto:readonly@getsoft.com From DavidA@ActiveState.com Wed Apr 17 16:28:26 2002 Received: from smtp1.ActiveState.com (gw.activestate.com [209.17.183.249]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3HLSPa09457 for ; Wed, 17 Apr 2002 16:28:25 -0500 Received: from smtp3.ActiveState.com (smtp3.ActiveState.com [192.168.3.19]) by smtp1.ActiveState.com (8.11.6/8.11.6) with ESMTP id g3HLS0213145; Wed, 17 Apr 2002 14:28:00 -0700 Received: from ActiveState.com (loom.activestate.com [192.168.3.137]) by smtp3.ActiveState.com (8.11.6/8.11.6) with ESMTP id g3HLRxe12030; Wed, 17 Apr 2002 14:27:59 -0700 Message-ID: <3CBDE8DF.4090801@ActiveState.com> Date: Wed, 17 Apr 2002 14:27:59 -0700 From: David Ascher User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.9) Gecko/20020311 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Neville Franks CC: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? References: <0efa01c1e49d$0da1f200$9e1de7cd@gatekeeper> <20020417070832.22300@triqs.com> <3CBDD628.6090908@ActiveState.com> <16346665221.20020418065416@getsoft.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Filtered-By: PerlMx makes it fast and easy. See http://www.ActiveState.com/Products/PerlMx/Header Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Neville Franks wrote: >Hi David, >I'm not exactly sure what you are looking for when you say "full text >indexer" but you may find Mifluz http://www.gnu.org/software/mifluz/ >of interest. From memory there are others like this as well. > There are lots. Mifluz fails my particular test because it appears to have no users on Windows, and the GPL license is one I can't use in some projects. But thanks for the pointer, I hadn't heard of that particular one.-- there should be a list of all of the open source searching solutions out there somewhere... --david From antonio@mac.ps100.net Wed Apr 17 22:16:10 2002 Received: from mac.ps100.net (h00045a2a5bb8.ne.client2.attbi.com [65.96.154.86]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3I3G9a27196 for ; Wed, 17 Apr 2002 22:16:09 -0500 Received: from tintin ([127.0.0.1] helo=mac.ps100.net) by mac.ps100.net with smtp (Exim 3.12 #1 (Debian)) id 16y2K4-0003ak-00 for ; Wed, 17 Apr 2002 23:11:00 -0400 Message-ID: <2490106.1019099434764.JavaMail.upgradewebapp@mac.ps100.net> From: Antonio Rodriguez To: metakit@equi4.com Subject: Re: [Metakit] Building metakit 2.3.34 on Mac OSX 10.1.3 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit x-memora-stamp: 1019099434698 Date: Wed, 17 Apr 2002 23:11:00 -0400 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hey all, So I made the changes you suggested to the libtool wrapper script J-C and I'm now getting what may be a more basic build error. I think that long-term Tito's approach is better (using native OSX dev. tools) but I am now being stubborn about getting this to work in the PB-less way. Below is my new error-- note that python headers for me is in /sw/include because I use fink: /bin/sh ./libtool --tag=CXX c++ -o libmk4py.la -O2 -I../unix/../include -I../unix/../src -I. -module -avoid-version \ -rpath /sw/lib PyProperty.lo PyRowRef.lo PyStorage.lo PyView.lo PWOImp.lo column.lo custom.lo derived.lo fileio.lo field.lo format.lo handler.lo persist.lo remap.lo std.lo store.lo string.lo table.lo univ.lo view.lo viewx.lo c++ -dynamiclib .libs/PyProperty.o .libs/PyRowRef.o .libs/PyStorage.o .libs/PyView.o .libs/PWOImp.o .libs/column.o .libs/custom.o .libs/derived.o .libs/fileio.o .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o .libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o .libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4py.so ld: Undefined symbols: _PyExc_IndexError _PyExc_TypeError _PyInt_FromLong _PyNumber_Check _PySequence_Check _PySequence_GetItem _PyString_AsString _PyString_FromString _PyString_FromStringAndSize _PyString_Type _PyType_IsSubtype _PyType_Type _Py_FindMethod _PyErr_Clear _PyErr_SetString _PyExc_AttributeError _PyFloat_FromDouble _PyFloat_Type _PyInt_Type _PyList_New _PyList_SetItem _PyLong_AsLong _PyLong_FromLong _PyLong_Type _PyNumber_Float _PyNumber_Int _PySequence_Size __Py_NoneStruct _PyDict_GetItemString _PyDict_SetItemString _PyDict_Type _PyExc_IOError _PyExc_ValueError _PyFile_AsFile _PyFile_Type _PyInstance_Type _PyObject_CallMethod _PyObject_GetAttrString _PyObject_SetAttrString _PyString_Size _Py_InitModule4 _PyCallable_Check _PyDict_DelItemString _PyDict_New _PyExc_KeyError _PyExc_RuntimeError _PyInt_AsLong _PyList_GetItem _PyList_Type _PyMapping_Check _PyMapping_GetItemString _PyMapping_HasKeyString _PyObject_IsTrue _PyObject_Size _PySequence_GetSlice _PyTuple_New _PyTuple_SetItem _PyTuple_Type __PyString_Resize _PyDict_SetItem _PyEval_CallObjectWithKeywords _PyList_AsTuple /usr/bin/libtool: internal link edit command failed make: *** [libmk4py.la] Error 1 Antonio -|Jean-Claude Wippler wrote on Mon Apr 15 10:42:06 EDT 2002: Antonio Rodriguez wrote: >Trying to build Metakit on a portable Mac (I'm temporarily away from my >linux machines) and I am running into a problem during the compile phase. > >I'm not used to building anything on OSX, so I'm not quite sure of how to >go about fixing this: > >cd metakit/builds >../unix/configure --enable-python ><< lots of autoconf stuff here >> >make ><< lots of stuff, then error: >> > >c++ -shared .libs/column.o .libs/custom.o .libs/derived.o .libs/ >fileio.o .libs/field.o .libs/format.o .libs/handler.o .libs/persist.o >.libs/remap.o .libs/std.o .libs/store.o .libs/string.o .libs/table.o >.libs/univ.o .libs/view.o .libs/viewx.o -lc -o .libs/libmk4.so >/usr/bin/ld: Undefined symbols: >_main >___builtin_delete >___builtin_vec_delete >___builtin_vec_new >___throw >_terminate__Fv >___builtin_new >___pure_virtual >___rtti_si >___rtti_user >make: *** [libmk4.la] Error 1 Ah yes, I've been struggling to understand this myself. What I had to do was edit builds/libtool and replace the "-shared" with "-dynamiclib" to make it work - it seems that otherwise c++ is trying to build a self- contained "main" executable, not a shared lib for use by others. That is of course a kludge. If there is a Mac OSX expert on this list who can help with this, please do follow up if possible. Libtool remains as clear as mud to me, so I'm essentially just feeling my way around in considerable darkness... >PS Please cc me on replies as I am not a regular list member yet. Done! -jcw From jcw@equi4.com Fri Apr 19 04:33:34 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3J9XWa24809 for ; Fri, 19 Apr 2002 04:33:33 -0500 From: Jean-Claude Wippler To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? Date: Fri, 19 Apr 2002 11:32:52 +0200 Message-Id: <20020419093252.28655@triqs.com> In-Reply-To: <22644.1019031935@www47.gmx.net> References: <22644.1019031935@www47.gmx.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g3J9XWa24809 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas Mügge wrote: >[fulltext search] >> If you needs more speed in case #1, switch to the binary search. Of you >> need to optimize case #2, consider creating an extra int column, perhaps >> a bitmap for all the letters in the item - and then do a loop using "if >> ((bitmap & searchbits) == searchbits)". I've used similar tricks for >> full-text keyword searches at times, with very good results. > >Can you show me a simple example for case 2? I much interested in speeding >up my substring search. It's astonishing fast already but decreasing the >response time for the user even further would be great. Let me describe the logic: - add an int property to your view, say "pMap" - for each row, set the value as follows: - if key contains 'a' or 'A', set bit 0 - if key contains 'b' or 'B', set bit 1 - etc... - that leaves a few bits to spare, you could eve refine things by using bits for "has digits", "has punctuation", "is all uppercase", "1st upper + rest lower", and so on - when you want to do a "string contains" lookup, first calculate its mask, using the same rules as above Now the key is that you have a way to very quickly decide whether a row cannot possible match, using the expression shown above. On a miss, quickly move on, on a match, you'll need to check further using a normal string match. This idea of using a quick "filtering pre-check" is not new. In the case of MK, it makes brute force more effective. Int access is more efficient that string access which needs to deal with variable-size offest calculations. Here's another example of the same, which helps with full-text searching, assuming that one only wants to find matches on word-beginnings: - add 3 int properties, say: p1, p2, p3 - bit 0 in p1 means: there is at least one word in the text which starts with 'a' or 'A' (bit 1 for [bB], etc) - bit 0 in p2 means: there is at least one word in the text which has 'a' or 'A' in the second position - bit 0 in p2 means: there is at least one word in the text which has 'a' or 'A' in the third position - and so on, for a total overhead of 12 bytes per row Now, searching for "abe" means creating three searchbit ints, with respectively b0, b1, and b4 set. To find all matching rows, each of those three ints must match with p1/p2/p3, respectively. Of they do, then you will still find false positives, but a simple normal string search will weed those out. The result, pretty good full-text retrieval performance for moderate amounts of text. The idea can be refined to work with 1..N char searches (while still using 3 ints, these are probably often sufficient). Note that you can mix and match blocked viewers with normal views. Even if the text is stored in blocked form, you can choose to place these ints in normal flat (and huge) int vector form. That makes them far more efficient for brute force scanning, and the combination can be made through either the c4_View::Pair or the c4_View::RemapWith operators. There is a lot of hidden speed in such designs. It would be *grand* to add one or more new custom viewers to MK which support such search modes. I've been wanting to for ages, but haven't found the opportunity and time to work on it so far. If you'd like to have a go at this, and get at least the basics of such a mechanism in place, I'll gladly help tweak it and integrate it all into a future MetaKit release. As with so many things, simple tricks sometimes can go a long way. -jcw From andreas.muegge@gmx.de Fri Apr 19 07:19:12 2002 Received: from mx0.gmx.net (mx0.gmx.net [213.165.64.100]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3JCJBa01693 for ; Fri, 19 Apr 2002 07:19:11 -0500 Received: (qmail 1099 invoked by uid 0); 19 Apr 2002 12:19:00 -0000 Date: Fri, 19 Apr 2002 14:19:00 +0200 (MEST) From: Andreas =?ISO-8859-1?Q?M=FCgge?= To: MetaKit Mailing List MIME-Version: 1.0 References: <20020419093252.28655@triqs.com> Subject: Re: [Metakit] How do I search for a substring? X-Priority: 3 (Normal) X-Authenticated-Sender: #0001987504@gmx.net X-Authenticated-IP: [217.194.32.11] Message-ID: <15233.1019218740@www17.gmx.net> X-Mailer: WWW-Mail 1.5 (Global Message Exchange) X-Flags: 0001 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello Jean-Claude, > Let me describe the logic: > - add an int property to your view, say "pMap" > - for each row, set the value as follows: > - if key contains 'a' or 'A', set bit 0 > - if key contains 'b' or 'B', set bit 1 > - etc... > ... Thank you! Now everything is clear, indeed a clever approach which I will try out in the next future. The only drawback is that it won't work when I must store longer strings (1k or more) but in that case I need some kind of fulltext indexing anyway. Btw, how could I store such an index in Metakit? Let's take for example something like this: , , , ..., For every word which occurs I create a new entry and then I must add a variable number of integers to this row (for each document/record where the word was found). How can I do this? With Subviews (and if yes how - this is a topic I haven't touched before)? Ciao, Andreas Muegge From brunobl@ifrance.com Thu Apr 25 19:08:17 2002 Received: from pop.univ-lyon1.fr (postfix@pop.univ-lyon1.fr [134.214.100.7]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3Q08Ha03826 for ; Thu, 25 Apr 2002 19:08:17 -0500 Received: by pop.univ-lyon1.fr (Postfix, from userid 3081) id F33C823EAC; Fri, 26 Apr 2002 02:08:15 +0200 (CEST) Received: from [134.214.164.67] (I435.resI.insa-lyon.fr [134.214.164.67]) by pop.univ-lyon1.fr (Postfix) with ESMTP id 288F9BF07 for ; Fri, 26 Apr 2002 02:08:15 +0200 (CEST) User-Agent: Microsoft-Entourage/10.0.0.1309 Date: Fri, 26 Apr 2002 02:08:14 +0200 From: Bruno Blondeau To: Message-ID: In-Reply-To: <200204191701.g3JH12a16229@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Subject: [Metakit] Unable to update from 2.4.1 to 2.4.4 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello, I recently updated my application, and also included a newer version of Metakit (2.4.4 instead of 2.4.1). I thought the data file would be converted automatically or the file format remain the same. However, it seems not to be the case. My application is buggy now. After spending several hours, I eventually reverted to 2.4.1, and everything works fine again... Do I have to do something to ask Metakit to convert my data file from the old format to the new one? Thanks, Bruno Blondeau From jcw@equi4.com Sun Apr 28 06:33:09 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SBX8a06555 for ; Sun, 28 Apr 2002 06:33:08 -0500 From: Jean-Claude Wippler To: metakit Date: Sun, 28 Apr 2002 13:32:19 +0200 Message-Id: <20020428113219.13489@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] Bug fix, small ints Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Two bug fixes have just been checked into CVS - these were serious enough to point out, so here are the changelog entries: 2002-04-28 Fix small-int re-use view bug A nasty bug was reported by VPI which caused upper bytes to be truncated from int values. The problem appears when storing ints of 1..4 bits in a view, then clearing the view (so a gap is placed past the end of the column), then adding a row with an int of 2 or more bytes. This uncovered a bug in forgetting to truncate columns with sub-byte int storage. Now fixed, added regression test s43. This bug could also have affected string and byte storage, since these use int columns to store item sizes. Under very specific conditions, it may have lead to truncated or even mixed-up values. 2002-04-27 Fix nested mk4tcl loop bug Loops would exit prematurely when nested - due to objc/objv being overwritten in the inner loop. Affects mk4tcl.cpp and mk4too.cpp. FYI, there are now web interfaces to the MetaKit and the TclKit CVS repositories, based on ViewCVS - see the respective home pages. I'm still looking into a "clobbered string" issue, which appears to happen only when there are strings over 100b *and* there are at least several thousand rows. Unfortunately, I've not been able to reproduce the issue myself despite numerous ways of banging on MetaKit, so if you have run into such a problem, please let me know. -jcw From jcw@equi4.com Sun Apr 28 06:55:30 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SBtTa07068 for ; Sun, 28 Apr 2002 06:55:29 -0500 From: Jean-Claude Wippler To: metakit Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 Date: Sun, 28 Apr 2002 13:54:37 +0200 Message-Id: <20020428115437.31702@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Bruno Blondeau wrote: >I recently updated my application, and also included a newer version of >Metakit (2.4.4 instead of 2.4.1). > >I thought the data file would be converted automatically or the file format >remain the same. > >However, it seems not to be the case. > >My application is buggy now. After spending several hours, I eventually >reverted to 2.4.1, and everything works fine again... > >Do I have to do something to ask Metakit to convert my data file from the >old format to the new one? No, never. Either it has the same format (as it does from 2.3 to now), or it does conversion fully automatically (as it does for 2.01 and older). Maybe something else is the problem. I just checked in a bug fix, so I would definitely advise to either try the CVS latest source, or wait until there is a 2.4.5 release - which is forthcoming. Now that the 2.4.4 release is being used far far more intensively and by more people (well, better late than never), so it looks like we are finally banging hard enough on MK to make all remaining bugs show up. I'd appreciate it if you could give me some more information on what sort of problems you were seeing - especially with the latest source code. -jcw From jcw@equi4.com Sun Apr 28 07:10:06 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SCA5a07449 for ; Sun, 28 Apr 2002 07:10:05 -0500 From: Jean-Claude Wippler To: MetaKit Mailing List Subject: Re: [Metakit] How do I search for a substring? Date: Sun, 28 Apr 2002 14:09:10 +0200 Message-Id: <20020428120910.32592@triqs.com> In-Reply-To: <15233.1019218740@www17.gmx.net> References: <15233.1019218740@www17.gmx.net> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by trixie.triqs.com id g3SCA5a07449 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas Mügge wrote: [...] >store longer strings (1k or more) but in that case I need some kind of >fulltext indexing anyway. > >Btw, how could I store such an index in Metakit? Let's take for example >something like this: > >, , , ..., > >For every word which occurs I create a new entry and then I must add a >variable number of integers to this row (for each document/record where >the word was found). How can I do this? With Subviews (and if yes how - >this is a topic I haven't touched before)? Subviews are definitely an option, but you'll find that with 10k rows and more, each having a small subview, things do tend to slow down. You'll need to experiment and compare performance for your cases - it's hard to predict what works best for you. From what you describe, I can see two candidates worth trying: - create a hash, first prop is word, 2nd prop is subview of ints (and consider blocking, i.e. a hash on top of a blocked view) - store "flat", i.e. , ... - this would be wasteful since "word" is stored n times, but with an extra view containing only words, you can replace it with an index: words[text:S],index[wpos:I,docnum:I] (In this case, "words" would be a candidate for hashing) It'll depend a lot on expected sizes, but I wouldn't immediately worry if that last approach had a million rows in the "index" view (you can switch to a blocking structure later without affecting much of the rest of your code). It really depends a lot on access patterns, amount/type of change, how much performance you need to achieve, how much effort you are willing to spend on timing various approaches, and how much logic you are willing to add. For me, the ground rule is still: go for brute force until it becomes a bottleneck. -jcw From brunobl@ifrance.com Sun Apr 28 16:17:34 2002 Received: from pop.univ-lyon1.fr (postfix@pop.univ-lyon1.fr [134.214.100.7]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SLHXa19879 for ; Sun, 28 Apr 2002 16:17:33 -0500 Received: by pop.univ-lyon1.fr (Postfix, from userid 3081) id 0536D24264; Sun, 28 Apr 2002 23:17:31 +0200 (CEST) Received: from [134.214.164.67] (I435.resI.insa-lyon.fr [134.214.164.67]) by pop.univ-lyon1.fr (Postfix) with ESMTP id 30A57C1F8 for ; Sun, 28 Apr 2002 23:17:31 +0200 (CEST) User-Agent: Microsoft-Entourage/10.0.0.1309 Date: Sun, 28 Apr 2002 23:17:29 +0200 Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 From: Bruno Blondeau To: Message-ID: In-Reply-To: <200204281701.g3SH15a13834@trixie.triqs.com> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: on 28/04/02 19:01, metakit-request@equi4.com - metakit-request@equi4.com wrote: > Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 > Date: Sun, 28 Apr 2002 13:54:37 +0200 > > Bruno Blondeau wrote: > >> I recently updated my application, and also included a newer version of >> Metakit (2.4.4 instead of 2.4.1). >> >> I thought the data file would be converted automatically or the file format >> remain the same. >> >> However, it seems not to be the case. >> >> My application is buggy now. After spending several hours, I eventually >> reverted to 2.4.1, and everything works fine again... >> >> Do I have to do something to ask Metakit to convert my data file from the >> old format to the new one? > > No, never. Either it has the same format (as it does from 2.3 to now), > or it does conversion fully automatically (as it does for 2.01 and older). > > Maybe something else is the problem. I just checked in a bug fix, so I > would definitely advise to either try the CVS latest source, or wait > until there is a 2.4.5 release - which is forthcoming. > > Now that the 2.4.4 release is being used far far more intensively and by > more people (well, better late than never), so it looks like we are > finally banging hard enough on MK to make all remaining bugs show up. > > I'd appreciate it if you could give me some more information on what sort > of problems you were seeing - especially with the latest source code. The previous version of my application was using Metakit 2.4.1 The new version is using Metakit 2.4.4, and also adds some columns to the database. Maybe the problem is here. It also seems there is no problem using the new version using the new version if the database is created with this version (only people who updated to the new version have problem). The new version with the database change but still Metakit 2.4.1 doesn't have any problem reading the old files (but has problems to read database that were converted with Metakit 2.4.4, so I have a user base that is stuck with a corrupted database... ) Also, there is only a problem with one column. A binary one. Which already existed in the previous version. I'm using objective-C++, but it shouldn't matter. The problem occures when I'm doing this. c4_Bytes data = NOTE_CONTENT(NOTE_VIEW[index]); The data content seems to be wrong when I'm reading it (but the length is right) Reading the first bytes, I get something like: 616c0a5c 63663020 4c617572 656e7420 50696e65 When it should look like (well, the first half are always the same given the kind of data structure that is used): 72746664 00000000 03000000 02000000 01000000 Also, I'm working using MacOS X, so maybe there is an endianness issue? I'm not used to download files using a CVS, but I will try to do my best to see if it works. -- Bruno Blondeau http://www.brunoblondeau.com From brunobl@ifrance.com Sun Apr 28 17:10:41 2002 Received: from pop.univ-lyon1.fr (postfix@pop.univ-lyon1.fr [134.214.100.7]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3SMAfa21122 for ; Sun, 28 Apr 2002 17:10:41 -0500 Received: by pop.univ-lyon1.fr (Postfix, from userid 3081) id 5B6F224267; Mon, 29 Apr 2002 00:10:40 +0200 (CEST) Received: from [134.214.164.67] (I435.resI.insa-lyon.fr [134.214.164.67]) by pop.univ-lyon1.fr (Postfix) with ESMTP id 98FAAC059 for ; Mon, 29 Apr 2002 00:10:39 +0200 (CEST) User-Agent: Microsoft-Entourage/10.0.0.1309 Date: Mon, 29 Apr 2002 00:10:37 +0200 Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 From: Bruno Blondeau To: Message-ID: In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: I tried the latest version that is in the CVS, but results aren't better. I randomly get complete garbage from the database. For example, below, there wasn't any problem for rows 150 to 156, but garbage for the other ones. I'm really beginning to think there either was a problem in the previous version, or a new problem in the current version... (if you have any idea, please cc me, as I only receive a digest of this list, and I'm getting flooded by users with problems...) Thanks in advance, Bruno Blondeau 2002-04-28 23:49:13.746 iOrganizeX[2435] 150 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 0f2b0000 01000000 2002-04-28 23:49:14.832 iOrganizeX[2435] 151 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 0f2b0000 01000000 2002-04-28 23:49:20.489 iOrganizeX[2435] 152 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 dc030000 01000000 2002-04-28 23:49:21.011 iOrganizeX[2435] 153 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 15010000 01000000 2002-04-28 23:49:21.334 iOrganizeX[2435] 154 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 4b010000 01000000 2002-04-28 23:49:21.656 iOrganizeX[2435] 155 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 1d010000 01000000 2002-04-28 23:49:21.977 iOrganizeX[2435] 156 : desc = <72746664 00000000 03000000 02000000 01000000 2e070000 00545854 2e727466 2b000000 30010000 01000000 2002-04-28 23:49:22.318 iOrganizeX[2435] 157 : desc = <2f202020 28667261 6e5c2738 64616973 295c0a66 74703a2f 2f667470 2e726561 6c736f66 74776172 652e636f 2002-04-28 23:49:22.650 iOrganizeX[2435] 158 : desc = <35355c67 7265656e 3235355c 626c7565 3235353b 7d0a5c70 6172645c 74783536 305c7478 31313230 5c747831 2002-04-28 23:49:23.005 iOrganizeX[2435] 159 : desc = <6c0a5c63 66302049 20737563 63656564 65642077 69746820 73657474 696e6720 74686520 44726976 6572206e 2002-04-28 23:49:23.312 iOrganizeX[2435] 160 : desc = <305c7478 32323430 5c747832 3830305c 74783333 36305c74 78333932 305c7478 34343830 5c747835 3034305c 2002-04-28 23:49:23.614 iOrganizeX[2435] 161 : desc = <73752e69 6e666f2e 6170706c 652e636f 6d2f6865 6c706e65 78742f68 656c706e 6578742f 68656c70 68656c70 2002-04-28 23:49:23.917 iOrganizeX[2435] 162 : desc = <2e636974 65676c6f 62652e63 6f6d5c0a 68747470 3a2f2f77 77772e73 6b696470 65726665 63742e63 6f6d2f65 on 28/04/02 23:17, Bruno Blondeau - brunobl@ifrance.com wrote: > on 28/04/02 19:01, metakit-request@equi4.com - metakit-request@equi4.com > wrote: > >> Subject: Re: [Metakit] Unable to update from 2.4.1 to 2.4.4 >> Date: Sun, 28 Apr 2002 13:54:37 +0200 >> >> Bruno Blondeau wrote: >> >>> I recently updated my application, and also included a newer version of >>> Metakit (2.4.4 instead of 2.4.1). >>> >>> I thought the data file would be converted automatically or the file format >>> remain the same. >>> >>> However, it seems not to be the case. >>> >>> My application is buggy now. After spending several hours, I eventually >>> reverted to 2.4.1, and everything works fine again... >>> >>> Do I have to do something to ask Metakit to convert my data file from the >>> old format to the new one? >> >> No, never. Either it has the same format (as it does from 2.3 to now), >> or it does conversion fully automatically (as it does for 2.01 and older). >> >> Maybe something else is the problem. I just checked in a bug fix, so I >> would definitely advise to either try the CVS latest source, or wait >> until there is a 2.4.5 release - which is forthcoming. >> >> Now that the 2.4.4 release is being used far far more intensively and by >> more people (well, better late than never), so it looks like we are >> finally banging hard enough on MK to make all remaining bugs show up. >> >> I'd appreciate it if you could give me some more information on what sort >> of problems you were seeing - especially with the latest source code. > > The previous version of my application was using Metakit 2.4.1 > > The new version is using Metakit 2.4.4, and also adds some columns to the > database. Maybe the problem is here. It also seems there is no problem using > the new version using the new version if the database is created with this > version (only people who updated to the new version have problem). > > The new version with the database change but still Metakit 2.4.1 doesn't have > any problem reading the old files (but has problems to read database that were > converted with Metakit 2.4.4, so I have a user base that is stuck with a > corrupted database... ) > > Also, there is only a problem with one column. A binary one. Which already > existed in the previous version. > > > I'm using objective-C++, but it shouldn't matter. The problem occures when I'm > doing this. > > c4_Bytes data = NOTE_CONTENT(NOTE_VIEW[index]); > > The data content seems to be wrong when I'm reading it (but the length is > right) > > > Reading the first bytes, I get something like: > > 616c0a5c 63663020 4c617572 656e7420 50696e65 > > When it should look like (well, the first half are always the same given the > kind of data structure that is used): > 72746664 00000000 03000000 02000000 01000000 > > > Also, I'm working using MacOS X, so maybe there is an endianness issue? > > > I'm not used to download files using a CVS, but I will try to do my best to > see if it works. -- Bruno Blondeau http://www.brunoblondeau.com From keks@instock.ru Mon Apr 29 06:13:55 2002 Received: from mx.instock.ru ([194.226.223.175]) by trixie.triqs.com (8.11.6/8.11.6) with SMTP id g3TBDsa08523 for ; Mon, 29 Apr 2002 06:13:54 -0500 Received: (qmail 4956 invoked by uid 404); 29 Apr 2002 11:14:57 -0000 Received: from keks@instock.ru by ns.instock.ru by uid 0 with qmail-scanner-1.10 (drweb. Clear:0. Processed in 0.061129 secs); 29 Apr 2002 11:14:57 -0000 Received: from c001-pautina-swamp.jacksattelecom.butovo.biz (HELO necropol-pc) (keks@instock.ru@62.117.74.178) by 0 with SMTP; 29 Apr 2002 11:14:57 -0000 Date: Mon, 29 Apr 2002 15:15:13 +0400 From: "Yuriy S. Polyakov" X-Mailer: The Bat! (v1.52f) Personal Reply-To: "Yuriy S. Polyakov" X-Priority: 3 (Normal) Message-ID: <140213179035.20020429151513@instock.ru> To: metakit@equi4.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: [Metakit] _indexed search Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hello metakit, Whether allows MetaKit to make indexed search in value of a field? -- Yuris S. Polyakov From andreas.olsson@ctakt.com Mon Apr 29 06:35:48 2002 Received: from linus.ctakt.com (linus.ctakt.com [62.119.45.234]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3TBZla09469 for ; Mon, 29 Apr 2002 06:35:47 -0500 Received: from flint ([192.168.2.115]) by linus.ctakt.com (8.11.6/8.11.2) with SMTP id g3TBZkp30410 for ; Mon, 29 Apr 2002 13:35:46 +0200 From: "Andreas Olsson" To: Date: Mon, 29 Apr 2002 13:35:45 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Subject: [Metakit] A bug in Locate? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Hi! I get different results when using Find and Locate. Find works fine but Locate returns 2 when there only is one matching row in the view. My call to Locate looks like this: int count = table.Locate( colName [pKey] , &pos ); I'd like to use Locate because I need both the count and pos value. /Andreas Olsson From jcw@equi4.com Mon Apr 29 06:54:53 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3TBspa10373 for ; Mon, 29 Apr 2002 06:54:52 -0500 From: Jean-Claude Wippler To: Subject: Re: [Metakit] A bug in Locate? Date: Mon, 29 Apr 2002 13:54:13 +0200 Message-Id: <20020429115413.20066@triqs.com> In-Reply-To: References: X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: Andreas Olsson wrote: >I get different results when using Find and Locate. Find works fine but >Locate returns 2 when there only is one matching row in the view. Please verify that the view you are using locate on is indeed sorted. A quick test from 0..N comparing values will tell you if things are indeed sorted. -jcw From lvirden@cas.org Tue Apr 30 10:04:34 2002 Received: from srv01.cas.org (srv01s4.cas.org [134.243.50.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3UF4Ya06110 for ; Tue, 30 Apr 2002 10:04:34 -0500 Received: from lwv26awu.cas.org (lwv26awu [134.243.40.138]) by srv01.cas.org (8.10.2+Sun/m4_8.10.2/CAS_MAIL_HUB-1.15) with ESMTP id g3UF4RS16885 for ; Tue, 30 Apr 2002 11:04:27 -0400 (EDT) Received: (from lwv26@localhost) by lwv26awu.cas.org (8.10.2+Sun/m4_8.10.2/CAS_CLIENT-1.18) id g3UF4Oq22774; Tue, 30 Apr 2002 11:04:24 -0400 (EDT) Date: Tue, 30 Apr 2002 11:04:24 -0400 (EDT) From: "Larry W. Virden" Message-Id: <20020430110424.AAB22765@cas.org> To: Subject: [Metakit] Did anyone ever write Metakit bindings for perl? Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: A friend is looking for a simple database with which he could manage a personal comic book inventory database. His experience is with Oracle - but apparently there isn't a free version of Oracle server for Windows XP Home. I thought of Metakit, but then discovered to my horror that I could not find a Metakit binding for Perl, the language he codes in. Did anyone ever get around to writing a Tie module or something for Perl and Metakit? -- Never apply a Star Trek solution to a Babylon 5 problem. Larry W. Virden Even if explicitly stated to the contrary, nothing in this posting should be construed as representing my employer's opinions. -><- From jcw@equi4.com Tue Apr 30 11:30:35 2002 Received: from [10.0.1.192] (madder.xs4all.nl [80.126.24.9]) by trixie.triqs.com (8.11.6/8.11.6) with ESMTP id g3UGUYa10324; Tue, 30 Apr 2002 11:30:34 -0500 From: Jean-Claude Wippler To: announce , metakit Date: Tue, 30 Apr 2002 18:28:03 +0200 Message-Id: <20020430162803.2864@triqs.com> X-Mailer: CTM PowerMail 3.1.2 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Metakit] TclKit 8.4-35 Sender: metakit-admin@equi4.com Errors-To: metakit-admin@equi4.com X-BeenThere: metakit@equi4.com X-Mailman-Version: 2.0.3 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Discussions about the MetaKit database library. List-Unsubscribe: , List-Archive: This is to announce a new release of TclKit, the self-contained runtime with Tcl/Tk 8.4a4+, MetaKit 2.4.5, and incrTcl 3.2+. This release combines the latest revisions, "puts $tcl_platform(vfs)" will print the string