At last, some more information here and here (see also Wikipedia) about why column-wise database storage makes sense. Brought to my attention via Slashdot.

Although I don't actually see "relational" and "column-wise" as opposites at all...

Having read several interesting articles lately, I've decided to get a few ideas out of my system. Seeds. Food for thought. Things I like. I'm calling them Croissants and have placed them next to the Tech Notes on the Vlerq site.

Update: fixed the above link...

There is a new area on the Vlerq website where I've added a first "Tech Note" - TN01 is about Nulls in Vlerq. To comment, please post to the Vlerq mailing list.

Tech Notes are more technical / factual than my musings and more concrete / focused than my earlier ramblings. Oh, and they are exclusively about Vlerq & Ratcl.

At last - more column-wise databases are starting to emerge: C-Store (BSD) and Vertica (proprietary). It's about time!!!

From a paper linked on Slashdot.

There's an interesting article about TM and STM in ACM's Queue. Somehow, I get the impression that Vlerq's mutable views are very similar.

Brian Theado pointed me to Dabble DB (blog). Fascinating (more discussion on this here).

A very good article by El Reg. Reading between the lines, I can see Vlerq going there on several levels.

A column-oriented database. Looks very interesting!

Here is an impressive quick introduction with a few small videos describing how to create a "Core Data" application from scratch - this is Apple's way of building data-aware apps (using Objective C, Cocoa, Interface Builder, etc).

The interesting pieces are the very slick GUI tools to create a data model, to visually design the GUI, and the way to tie actions and code together. This seems to work with native data formats, SQL, and XML as storage engines.

A superb intro, though I'm not convinced that this is the way to go, ultimately.

Every once in a while (especially with scripting languages), I come across the intriguing phrase "no database required". Often as a feature (example).

It sounds as silly as "no file system required, prefer raw disk blocks!" to me... What's next - no data structures required? no programming language required?

I s'pose the cause of this is the deployment hell involved in using any of the mainstream databases. Sensible objection, but wrong conclusion.

The more I look, the more I see people everywhere trying to solve the problem of marrying databases and SQL (and XML and XPath) without using either. There's LINQ, and there's Ruby for example.

A solid relational foundation inside a programming language (whichever one is used) still seems to be a long way off. I sure wish I had more answers ready.

A fascinating post which proposes to implement Chris Date's "Third Manifesto" database concepts (and I guess its "Tutorial D" language) on top of a substantially modified implementation of SQLite. I'd love to see this fly (but I doubt it will, for several reasons).

Views and sub-views are not really new or unique. Everybody struggles with nested structures. Master / detail, XML, group by, pivot tables ... all have some aspects related to nesting and grouping.

Here's one I had not seen before: Data Shaping and Advanced Data Shaping.

I'm still not convinced that either SQL extensions or XPath-/XQuery-like solutions are the "right" way to deal with these concepts.

An independent study (heh!) confirms the value of columns-oriented storage (with thanks to Nikos B for passing on this link).

STM has been mentioned a few times on LtU. The latest entry points to a recent SCOOL workshop which goes into this in great detail. It turns out to be amazingly appropriate and timely, given my last post on the Vlerq forum. Lots of reading to do ... (yummie + sigh)

IBM calls it Viper. And yes, they do have eh... slightly more pushing power. But I've got Vlerq and am going all out to design a system which works well with both traditional and very sparse relational data. Let's not underestimate the power of columns just yet.

Interesting article about "type managers", by Ben Meyer. He explains how apps such as iTunes stop making you think of the disk as a file system with lots of files. It's not about technology, it's about content. And addressing one problem domain.

Interesting article about where MySQL is headed. Explaining why others can go after Oracle:

If you're working in a zoo you don't want to be the one who has to brush the teeth of the lion.

Chuckle. And while the world feasts on SQL, I can focus on other stuff.

There's this big issue of "normalizing" a database design - it comes up all the time (if you're into modeling information that lives in the real world, that is). It has implications for how data is stored, consistency, redundancy, and performance. In short: it's impoitant ...

I'd like to be able to take existing (filled!) data tables and play around with alternate representations, while at the same time exploring the implications for performance and the impact on normalization. Drastic changes, sort of like a "refactoring browser for data" - preferably in a highly visual manner.

Have not found such a utility so far.

Maybe one day, the views and relational algebra of Vlerq will make it possible to support such a data model evolution tool. The dynamics of views, and derived views (also blocked views, as first explored in Metakit) ought to be up to the task.

Days are too short!

The Register has an interesting news item about an innovative database called the ANTs Data Server (ADS). The way they deal with contention while avoiding locking is particularly intriguing.

The MySQL 5.0 release candidate is out. Interesting changes listed here include: updatable views and compact archival storage.

From a post on the SQLite mailing list:

The more I learn about NULLs in SQL the less sense they make...

So true, Richard, so true.

There's a new book out by Chris Date, titled "Database in Depth, Relational Theory for Practitioners" (ISBN 0-596-10012-4). It's a 200-page paperback full of very concise wisdom. All of Date's key arguments in a nutshell. A must read if you're into databases (both usage and design), because it highlights and justifies the main points of relational theory.

With a title like Beyond Relational Databases, I was really looking forward to reading Margo Seltzer's ACM article (SleepyCat's founder). Alas, the word "algebra" is not mentioned. Omission or oversight, who knows.

Every time I come across things like these, I can't help but think that two trends which have wandered too far off course (SQL and OO) are being shoe-horned back into something which to me seems... odd.

Given that I consider data on-disk and in-memory to be two sides of the same coin, I can only conclude that SQL and OO are going out of their way to be as different from each other as can be.

Why marry two opposite mindsets? It makes no sense.