2006 ISSUE 4 Contents

THE INTERBASE
& FIREBIRD
Contents
DEVELOPER Editor notes
MAGAZINE by Alexey Kovyazin
Money for nothing ................................................................. 4
Oldest Active
by Helen Borrie
RTFM — regarding those free manuals ........................................... 5
Credits Community
by Carlos Cantu
Alexey Kovyazin,
About Dolphins and Birds ......................................................... 6
Chief Editor
Cover Story
Dmitri Kuzmenko,
by Paul Ruizendaal
Editor
The story of Fyracle ................................................................ 8
Helen Borrie,
Development area
Editor
by Paul Ruizendaal
Lev Tashchilin, Common Table Expression in Fyracle ............................................ 12
Designer
Products Overview
Editorial Office IBSurgeon Products .............................................................. 17
IBase IBDeveloper, office 5, Development area
1-st Novokuznetsky lane, 10 by Paul Ruizendaal
zip: 115184 Morfik's WebOS: Innovating beyond LAMP ..................................... 18
Moscow, Russia by Vlad Horsun
Phone: +7495 6869763 Global Temporary Tables in Fyracle .............................................. 23
Fax: +7495 9531334
TestBed
Email: by Alexey Kovyazin
[email protected] TPC-C based tests results ........................................................ 26
www.ibdeveloper.com Bonus
by Alexey Kovyazin
© Copyright 2005-2006 by
IB Developer. All rights reserved.
Comprehensive Repairing Guide. Part 1 ........................................ 35

No part of this publication may be

reproduced or transmitted in any
form of or by any means, electronic
or mechanical, including photo-
copy or any information storage
and retrieval system, without per-
mission.
For promotional reprints, contact re-
print coordinator Alexey Kovyazin,
[email protected].
IBDeveloper reserves the right to
revise, republish and authorize its
readers to use the articles submit-
ted for publication. All brand and
product names used in on these
pages are trade names, service
marks or trademarks of their re-
spective owners.

 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 EDITOR'S NOTE 2006 ISSUE 4

Money for nothing

I guess some of our readers have be-
come a bit tired of waiting and won-
dering when the new issue would be re-
a «Borland Developer Studio» confer-
ence and a lot of things got clearer.
By Alexey Kovyazin,
[email protected]

In essence, Borland is splitting into two

leased. I am really sorry about the long companies: one is solution-targeted
delay, but there have been some serious (ALM) and the other is oriented on de-
financial and time issues holding up veloper tools. As I understand, the De-
production. veloper Company (alias “DevCo”) guys
I want to begin with sincere thanks were the initiators of the spin-off be-
to people who made donations to our cause they considered the IDE product (rather interesting) point of view. Jim’s
magazine. Orlin Chankov and Steve line too different from the ALM busi- departure from active development on
Kramer, thank you, guys, for your help. ness. I can even guess that Delphi and Firebird provided an excellent opportu-
Your donations were much appreciated. the other IDEs were the financial props nity to promote Firebird. Now everyone
for ALM! Jason did not say so, of course, knows where successful companies are
We’ve come to realise that producing
but it seems quite obvious to me. looking to headhunt developers and
the magazine with only advertising and
sponsors to support it is impossible. According to their statements, there’s get good ideas :)
We’ve decided to change our business no problem with Delphi sales—Delphi Fyracle
model slightly. We are going to run 2006 is a bestseller—and no problem
This issue is mostly devoted to Fyracle.
special proprietary editions of each is- with its development. Jason demon-
I’d like to express great thanks to Paul
sue with bonus articles. These special strated an impressive roadmap for Del-
Ruizendaal, Vlad Horsun and Eugeniy
editions will be available in electronic phi up to 2009, with many exciting fea-
Putilin for their help with creating this
(printable PDF) and paper versions. tures, and things of interest for JBuilder
issue. Fyracle is on the cutting edge of
From this issue forward, the latest issue and C++ Builder too.
new Firebird technologies. Many things
will be available only for purchase for According to the team, InterBase sales that are already implemented there are
a period—say one or two weeks from are good and its roadmap is interesting, planned for inclusion in Firebird 3.0, so
publication—and will be fully released too. There is no indication that DevCo you can download the future right now.
afterwards for everyone, minus the bo- intends to drop all these profitable Read on about Fyracle’s exciting fea-
nus articles. products and, personally, I feel much tures and give it a try!
It’s regrettable to have to move in this better about it all now. We can only
Unfortunately, we couldn’t include the
direction but it seems the only way to await the completion of the spin-off.
article about External Procedures in
make the magazine viable. Orlin and Just one other thing—why isn’t there Java and .NET in this issue. We’ll try to
Steve, for their generosity, will have any name for “DevCo” yet? One guess is ensure that it makes it in time for the
free, unlimited subscriptions to the that the “Borland” brand is one of the next issue.
electronic version of IBDeveloper mag- most precious assets of the company
azine. TPC-C based test
and both the IDE and ALM sides want to
Borland have it. Anyway, Delphi and InterBase And yes, we’ve finished the TPC-C series
are here to stay, regardless of whether testing. There are some very interesting
On February 8, Borland published plans
the branding is “Borland” or “DevCo”! results, so don’t miss the article about
to spin off the developer product line,
it in the TestBed section of our maga-
including Delphi and InterBase. It Jim zine!
looked so peculiar at the time that
Jim Starkey was one of the main news-
many developers said “Borland is dead”, Bonus article
makers since the previous issue. Jim is
and probably several less printable ex-
a living legend and becomes more and Our bonus material for this issue is the
pressions. It was really strange and,
more popular (I think he should start first part of “The Comprehensive Re-
for months, all we had were rumors and
to sell autographs via eBay!). MySQL pairing Guide for InterBase and Fire-
fuzzy press-releases.
bought Jim’s company Netfrastructure bird”. This part is “Corruption reasons,
Recently, however, the chief Borland and hired Jim to make a proprietary Part 1”. If you are keen to know where
executives of the new “DevCo” (David database engine for them. In this is- the pitfalls and traps for your database
I, Jason Vokes and others) went on a sue Carlos H. Cantu comments on all are, order the bonus version of “The
worldwide tour to clarify events and these events in his article “From Birds InterBase and Firebird Developer Maga-
plans for the developer community. I to Dolphins”. As editor I should em- zine”!
met with Jason in Moscow on May 23 at phasize that this article is his personal

www.ibdeveloper.com 
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 OLDEST ACTIVE

RTFM - Regarding Those

Free Manuals By Helen Borrie
[email protected]

A quaint side effect of the Firebird

“wanna-be” culture is the oft-sung
lament, entitled “Where Are My Free
discover that full, free-beer documen-
tation is available and can be at your
fingertips with minimal effort.
Manuals?” I’m undoubtedly a bit over- Besides that, and top of the agenda
sensitive to the assertion that users the original chapters need more work
currently, Firebird-Docs also has in
of a totally free database engine have than others, while some others need
hand a large volume of user documen-
some indisputable right to expect to to be laundered to eliminate content
tation that is work-in-progress, around
pay nothing, ever, for up-to-date user that seems too Borlandish for peace of
20 weighty chapters that began life in
documentation. As the author of a Very mind. Paul has a couple of chapters
2002, as a Firebird 1.0 ebook manual
Fat Book about Firebird, I doubt I will of “Using Firebird” almost ready to go
that was distributed to subscribers of
ever get happy about those list mes- now. As the various chapters become
the IBPhoenix CD service. The Volume
sages and private emails that criticise ready for use with Firebird 2.0 and 1.5,
I manual, “Using Firebird”, was given to
Greedy Me for getting it published com- so they will be built and published.
the Firebird Project in 2004 as Frame-
mercially instead of just giving it to you That’s where your free manuals are: in
maker 6 source code under open docu-
as a gift. It took a year and I’m not even the hands of the willing few. The Docs
ment licensing. Converting the arcane
close to making any profit out of it. I project (see http://www.firebirdsql.
Adobe sources to the Firebird Docs proj-
never expected I would. But–ouch!–it org/index.php?op=devel&sub=doc) ur-
ect’s custom Docbook XML format took
hurts. gently needs some reasonably accom-
a lot of work for a couple of people.
Contrary to popular belief, Borland plished writers of English, with Fire-
Since then, updating those chapters
did not release the IB 6.0 documenta- bird skills, to pick up the XML sources
has fallen largely to Paul Vinkenoog,
tion set sources in 2000, nor any other of these chapters and upgrade them
the very same guy who picked up the
form of documentation. One of the few to Firebird 2.0, taking in Firebird 1.5
unfinished, unusable Docbook system
things Borland was prepared to be clear along the way. Waiting in the wings are
begun by David Jencks and refactored
about was that it would act against any- Spanish, Japanese, Russian and Brazil
it into the sophisticated, multi-lan-
one in Firebird’s community who tried Portuguese translators. There are tasks
guage system that today can build our
to re-publish or update any documen- aplenty here for those with some time
release notes and all of the project’s
tation to which it claimed copyright. to commit and the will to participate in
currently released user docs in a vari-
Fortunately for those who demand free documenting Firebird.
ety of formats and languages. Some of
documentation beyond the basics, un-
authorised downloads of these seven
PDF books have remained available
around the Web to this day.
What that means to us as a community
is that every bit of Firebird published
documentation has to be written on
blank paper. It takes time, care, effort,
skills and bodies. That’s why, almost
six years on from the opening of the In-
terBase code, the Firebird Project still
doesn’t have its own complete, inte-
grated, free documentation set.
It does have Quick Start Guides for its
two released versions and a few papers
on topics of interest to newbies, experi-
enced users, or both. It does have three
complete sets of detailed release notes
that document every bug-fix, change
and enhancement and tell users how
to install and configure that particu-
lar Firebird version. Those prepared to
reach past the end of their noses can

 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 COMMUNITY 2006 ISSUE 4

About Dolphins and Birds...

By Carlos H. Cantu
It seems to me that we are living in ises a lot of new features, including
[email protected]
unprecedented times. As never be- support for stored procedures, triggers,
fore, Open Source software has tra- referential integrity, etc., introducing
ditional companies like Microsoft, more advanced capabilities that could
Oracle and IBM running scared. If make MySQL suitable for than more just
you think about Linux vs. Windows, it simple data repositories.
has been the case for a while already. So what does all of this have to do with work of the Firebird development team,
Now, it seems to be carrying over to Firebird? Firebird is Open Source soft- a group of people from many places
database server systems. ware, making the Firebird community around the world. Jim jumped into the

T hose who follow some of the tech-

nology news sites are probably aware
of recent announcements from the “big
direct or indirect players in this game...
but there is more than that.
scene almost three years ago, when SAS
contracted IBPhoenix to fork Firebird
1.5 and develop a systemically multi-
guys”: Oracle and IBM both released From Birds to Dolphins threaded version for its own purposes.
free versions of their respective data- Jim Starkey, creator of the original In- IBPhoenix hired Jim to do the work.
base server products. Microsoft already terBase and main developer of the Vul- From that point, he became an active
had MSDE, along similar lines. can fork of Firebird, announced on Feb- participant in the Firebird discussion
ruary 18 that MySQL AB had bought his lists and involved himself with the
Those free versions do come with a
company (Netfrastructure) and he was team. When Jim’s contract with SAS
variety of limitations, such as restrict-
moving on with it (see announcement ended, SAS gave Vulcan to the Firebird
ing RAM/CPU or database size, and,
box). Ann Harrison, who is a partner in Project.
of course, although free, they are NOT
the Firebird/InterBase support compa-
Open Source. They notably lack one of I’m sure Jim would not get directly in-
ny IBPhoenix and also Jim’s wife, is to
the base benefits of open source, the volved with Firebird if he wasn’t being
work part-time for MySQL AB. Ann has
ability to get your hands on the source paid to do his work. The Vulcan proj-
stated publicly that she has no inten-
code and make your own, custom builds. ect was based on a forked version of
tion of leaving the Firebird Project.
And the question of whether those free Firebird 1.5 code and remains a totally
versions will be maintained or updated Before talking about how this can af- separate project from Firebird 2.0. The
by the vendors remains unknown. fect the future of Firebird development, next major Firebird release, Firebird 3.0,
let’s check why MySQL did this. is planned to be a merger of Vulcan and
The important thing to note is that
things are changing! Two or three years Jim has a great reputation as a soft- Firebird 2.0 into a single code base.
ago, it would have been hard to imagine ware architect and for his first-hand The merger process is being conducted
Oracle or IBM releasing free versions knowledge about RDBMS development by the Firebird team. Jim has stated that
of their flagship software. In the in- and architecture. Transactions, stored he would continue to lurk in the Firebird
tervening years, open source database procedures, concurrency and all the lists from time to time, so I don’t doubt
engines have been growing in features, other powerful relational database fea- that he will be around if his advice is
power and market space conquest. The tures are no mystery to him. Acquir- needed during the merger process.
threat of being overtaken is now forc- ing Netfrastructure and bringing Jim However, the Firebird team comprises a
ing the “big guys” to change the way aboard along with it could be viewed highly competent group of developers
they do business. as MySQL’s answer to Oracle’s attacks. with a thorough understanding of the
They want Jim’s brain so they can start code and the requirements. The Inter-
MySQL Under Attack developing their own fully featured Base 6.0 code was released with no ac-
Recently Oracle bought Innobase, the database engine for MySQL. It won’t cess to internal documentation and, in
Finland-based manufacturer of InnoDB, surprise me to find a lot of similarities almost six years, the Firebird developers
the most fully featured engine used by between Firebird and the next genera- have been digging and figuring out for
MySQL database. Next, Oracle acquired tions of MySQL. themselves how to build, change and
Sleepycat, the company behind Berke- clean that buggy code. These guys are
leyDB, another database engine used by What About Firebird? really good and I’m sure they will do a
MySQL. It is clear to me that Oracle’s ac- If you are worried about Firebird’s fu- great job in the merge process!
tions are attempts directed at affecting ture, calm down! The project loses noth-
market confidence in MySQL. Remember ing from the recent events. Actually, it What does Firebird win from
that MySQL has an impressive number can even win, as I will explain. this?
of installations all over the world and Jim Starkey had no direct involvement Jim’s announcement attracted a lot of
claims to be the most-used database for in the development of Firebird 1.0, 1.5 attention to Firebird. FireBirdNews, one
Internet applications. MySQL 5 prom- and 2.0. All three versions were the of the first sites to spread the notice
www.ibdeveloper.com 
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 COMMUNITY
about the Starkey move, got a lot of Announcement box: nologies would be unlikely to meet the
links from many other technology sites goals of either project.
My company, Netfrastructure, Inc.,
and blogs all over the internet. That MySQL and Firebird have never seen
has been acquired by MySQL, AB. As part
provided opportunities for potential each other as competitors and I doubt
of the agreement, I will be working full
new users, who previously had no con- this will change in the future. The proj-
time for MySQL. I expect to lurk on the ar-
tact with Firebird, to discover how good ects have different open source philoso-
chitecture list from time to time and may
and special the product is. A special phies, different technologies, different
contribute the occasional wolf-o-gram,
paper was created to introduce Firebird customer bases, and different sweet
but I will not be taking an active part in
to those people: “Get to know Firebird spots. The ideas behind the two projects
Firebird development. Although Ann will
in 2 minutes” is now available at least are, happily, public and available to all.
work for MySQL, part time, translating
eight languages, with the intention of If MySQL and Firebird compete, it is only
from wolf to English, she will continue to
presenting Firebird to outsiders in a competition in offering the best possible
be active in the Firebird project.
simple, concise way. support to their respective customers.
My decision to join MySQL has al-
I’m sure Firebird is getting a lot of new I am pleased to have had the oppor-
most nothing to do with Firebird and
users who can be persuaded to become tunity to finish the Vulcan project. The
everything to do with Netfrastructure.
Firebird Foundation members and so combination of Vulcan SMP and archi-
The Netfrastructure platform represents
start to contribute to the project fund- tecture combined the rich feature set of
what I feel about contemporary comput-
ing. Some of them might join the devel- Firebird 2 will make a solid release and a
ing hardware and future application re-
opment team and help with bug fixing superb platform for future development.
quirements, and has been the center of
and implementing new features. Others
my technical heart and soul for six year. I wish the Firebird project all the
might contribute to documentation
Some aspects of Netfrastructure technol- best in years to come. And if you need
and testing.
ogy have already been contributed to the an opinion, please feel free to call.
In other words, Firebird is out there in Vulcan project, but Firebird and Netfra-
the media and the community is get- structure are architecturally incompat- Jim Starkey
ting the kind of exposure it needs to ible. An attempt to integrate the tech-
get bigger! Consider my line of think-
ing: why should you wait years until
MySQL can finish a new engine, if Fire- Customer satisfaction is our top priority! Integrating the Enterprise!
bird offers all the features of a real RD-
BMS right now?
What if Oracle wants to buy
Firebird?
It would be a very difficult (should I
say “impossible”?) thing to make hap-
pen. Firebird has no owner and there is
no individual or company that owns or better office is a German company specialized
controls its inherent rights and assets. in Borland and Microsoft development environments
Oracle (or anybody else) can take the With offices in Oldenburg (head office),
code for free and roll their own Firebird,
Frankfurt, Berlin, Pennsylvania (USA)
as long as they comply with the open
source licensing, but nobody can come and a Team of appr. 25 associates.
and buy Firebird. We don’t need to worry better office provide development tools,
about this. Firebird belongs to the com- custom made software development,
munity! software consultancy,
Conclusion Project management, training and support.
I hope this article provided some clari- They have extensive experience in developing client-server
fication of recent events and was able and web based database systems.
to show how Firebird can gain from the
Benefits:
extra exposure. Firebird is a great prod- Head Office: Oldenburg
Borland Delphi WIN32 and .Net
uct, and it is getting better and better Stau 19 – 6.OG -
Microsoft .Net Framework
at each new release. 26122 Oldenburg
(C# Visual Studio)
You can find some more information JAVA JBuilder and Eclipse Tel.: +49 (0)441 926740
about this by checking the posts at Borland InterBase Fax: +49 (0)441 2488675
www.FirebirdNews.org. Microsoft SQL-Server
IBM DB/2/40 (iSeries) E-Mail: [email protected]

 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 THE STORY of FYRACLE 2006 ISSUE 4

The story of Fyracle

The birth of
this could perhaps be modeled on the By Paul Ruizendaal
Oracle-mode Firebird existing code for external flat file ta-
[email protected]

S ome of the best conversation takes

places over dinner. Picture yourself
in Brussels, Februrary 2003. It was the
bles. This, I think, was a good idea and
remains on my list of future projects for
Firebird. Today, I would look at general-
weekend of the annual FOSDEM confer- izing the external table mechanism to a
ence and I had just attended Ann Har- variety of sources, the most important
rison’s talk about Firebird. For several being another Firebird instance. The After tossing the idea around for
years I had been looking for a database toughest hill to climb would be to fig- about an hour, everybody agreed that
to accompany Phoenix Object Basic, a ure out how to deal with transactions. it might work. In theory. Covering ev-
cross-platform work-alike of Visual Ba- ery nook and cranny of Oracle’s some-
The more elaborate part of the dinner
sic. It needed something equivalent to times weird behaviors would be hard.
conversation was about Compiere and
Microsoft’s Jet database engine, only it To make a general solution would be a
how Firebird could support it. Sequenc-
had to run on both Windows and Linux major amount of work. It would always
es were not an issue, generators fit the
and it had to be less fragile than Jet. It be out of date with the latest release
bill perfectly. Embedded transactions
looked like Firebird could fit the bill. of Oracle. All very sensible comments.
was a bigger topic. At the time I did not
But I only want to run one specific pro-
About half a year earlier I had been understand all that much and it took a
gram, I countered. The problem set is
searching for an accounting package while before we could figure out that
defined and not unbounded. Skeptical
and had come across something called what was needed were user level save-
faces remained. With one exception,
“Compiere”. Compiere is an open source points. Around that time, these had just
Paul Reeves. With a deep, thoughtful
ERP package, written in Java, running on been added to the Firebird 1.5 develop-
look he agreed: it actually might work.
top of a Jboss application server. With ment branch, so that hurdle seemed
several hundred thousand downloads, taken as well. The challenge was on the table. It
Compiere was getting noticed. It had looked like there was a plan that could
Remained the issue of PL/SQL and syn-
one big drawback: it only ran on Oracle. give Firebird an oracle-compatibility
tax differences between Firebird’s SQL
The author of Compiere, Jorg Janke, had mode. But who was going to do it? The
and Oracle’s SQL. Various approaches
tried to port his software to Postgres, IBPhoenix crew looked rather exhaust-
were discussed. One such option was
but had given up. I asked him what ed from the 2000-2002 experiences
converting PL/SQL to the equivalent
caused him to abandon the project, and and had enough struggles of their own
PSQL text and working from there.
he said that Postgres lacked three es- to deal with. They were not going to
There are some products on the mar-
sential features: sequences, embedded shoot of on a tangent just because of
ket that attempt to do this for other
transactions (he meant savepoints) a pleasant dinner. So, during the din-
conversions (e.g. Oracle to DB2, etc.).
and PL/SQL. This reply intrigued me ner I resolved to look into how hard it
None of these appear to work very well,
and I wondered whether Firebird would would really be and Ann kindly agreed
because if they did, database system
be up to the job. to answer questions I may have. Paul
migration would not be seen as such a
Beach promised to dig up a copy of an
With these two issues in mind I had major task.
old Interbase 3.3 manual for BLR for
e-mailed Ann and asked if I could in-
A second approach was suggested by me, which he did.
vite her to dinner to talk about Firebird
Paul Beach. He reminded everyone at
during the FOSDEM conference. Ann, So, Fyracle was born in a pub-eatery in
the table of the old BLR interface to
probably cautious not to have dinner Brussels. If the project ever gets to be
Firebird that still existed. It was once
alone with a complete stranger asked famous, I will have to go back to the
the main connection API into the en-
if Paul Beach could be there and as it place and put a plaque on the wall. Not
gine, in a time when client programs
happened most of the IBPhoenix crew sure I can still find it, though.
used pre-processing instead of dynamic
joined the dinner. So there we were, en-
joying hearty Flemish food and discuss-
SQL to communicate with the database Building the architecture
engine. BLR is an acronym for Binary
ing where Firebird could go. In March of that year work began on
Language Representation. It is a low
exploring the work needed to make
Using Firebird as a better Jet for Phoe- level language that high level relational
Compiere run on Firebird, with mini-
nix Object Basic was not much of a top- languages like SQL, GDML and QUEL can
mal, or even better no change to Com-
ic. It was quickly concluded that it had be compiled to. Paul suggested that it
piere. Work started with getting the
nearly all features on my wish list, with might be possible to write a compiler
table definitions across and moving
one exception. I would have liked for that mapped PL/SQL to BLR. The issue
the data. As it was obvious that a full-
the possibility to link to external ODBC of accepting Oracle’s flavour of SQL
blown SQL compiler would be needed
database tables. It was suggested that could be handled in a similar way.
further down the road, the first item to
www.ibdeveloper.com 
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 THE STORY of FYRACLE
be written was a lexer and a simple recursive descent parser that could handle the
essential oracle-style DDL statements. Rather than writing out BLR, the back-end
of the compiler wrote out a text file with Firebird-style SQL. This was completed
about a month later.
The compiler was then extended with the ability to parse a subset of Oracle’s DML
language, covering the most common constructs. The hardest part in this early
phase was figuring out how to handle Oracle’s join syntax. As most of you probably
know, Oracle has a non-standard syntax for joining tables, using the (+) operator.
SELECT tbl1.a, tbl2.x FROM tbl1, tbl2 WHERE tbl1.a=tbl2.x(+);
What the compiler does is analyzing the WHERE clause and find the sub-expres-
sions that use the (+) operator. These expressions are then ordered and potential
cycles are detected. Cycles are illegal and an error results. The WHERE clause is
then rewritten to contain only the other parts. The sub-expressions using the (+)
operator are moved to the JOIN ON clauses.
SELECT tbl1.a, tbl2.x FROM tbl1 LEFT JOIN tbl2 ON tbl1.a=tbl2.b;
With this work done, some of the basic At this point we had some basic con-
elements were in place, but Fyracle was cepts in place. We could take an Oracle
still a text-to-text tool. For Fyracle to DDL file and use it to define the data-
support Compiere it would need to work base structure in Firebird. We could
together with the Jaybird JDBC driver. connect a Java client and use basic
Ann kindly introduced me to Roman oracle-style SQL. This SQL was translat-
Rokytskyy, who was kind enough to ed on-the-fly to firebird-style SQL and
help and has been providing invaluable passed to the engine. The result set, if
input ever since. After some discussion, any, was passed back to the Java client
it was decided that Fyracle should ex- in the usual way.
pose a standard Firebird client API, so
that the JDBC driver could connect to Working with PL/SQL
it as a Type II driver. Roman added code In parallel work had taken place on
to load “fyracle.dll” when an oracle- compiling PL/SQL into something that
mode connection was specified and I Firebird could digest. With the help of
wrapped the text-to-text tool in a dy- Paul’s manual and Ann’s input, I was be-
namic library exposing the standard ginning to understand the concepts be-
API. So, the connection stack would hind BLR. It wasn’t long before I had to
look like the below graphic: accept that there were some real chal-
lenges here: BLR is not a natural match
with PL/SQL. It was designed to be a
low level representation of relational
languages with some extensions added
to help it handle procedural elements
like you would find in for instance the
“qli” command line tool.
PL/SQL in contrast has a very different
design philosophy. It started life as a
client side scripting language in Oracle
extensions like the “Forms” package. In
its concepts it is - according to Oracle
- modeled on the ADA programming
language, which is rooted in the Algol
family of programming languages. The
most well known member of this fam-
ily is Pascal. Being in the Algol family
means that PL/SQL supports something
called lexical scoping, which in turn
means that procedures can be defined
inside of other procedures. The vari-
ables of the current instance of the
figure 1 outer procedure are available in every
 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 THE STORY of FYRACLE 2006 ISSUE 4
instance of the inner procedure. and transaction.
At first I tried to figure out how the re- New code had to be created to build a superset of the UDF mechanism. The engine
quirements of PL/SQL could be mapped was modified to accept new syntax that called an external function and provided
to regular BLR. At a few points I was that external function with the means to call back into the engine. The code was
considering to add significant new modeled on both the UDF code and the EXECUTE STATEMENT code that had just
features to it, even revamping it com- been added to the Firebird 1.5 development branch. Much debate took place on
pletely. The execution engine for BLR the Architecture mailing list about how external procedures should call back into
evolved from the co-routine struc- the engine. The discussion revolved on the best design of the API and the proper
ture that Jim had once developed for architectural layering of code. Fyracle needed to keep moving, so it took a short-
Datatrieve and it had evolved so much cut: the API was kludged together and the layering was broken. The latter didn’t
during 20 years that a major redesign mean much as Borland had already broken the layering and proper refactoring
would make sense. In the end I decided would not occur before Firebird 3.
that such a task would go beyond my
will and skill and chose another route.
The solution was to create a simple byte
code interpreter for an Algol-like pro-
cedural language, which would be sep-
arate, but have tight integration with
the relational engine. The design for
this byte code engine is similar to the
byte code engines that used to support
microcomputer Pascal back in the early
eighties. We have probably all written
such an engine back in university, as an
assignment in compiler construction
class. For lack of a more creative name,
I call it the “btc engine”, for byte code
engine. Often I refer to the btc engine figure 2
as the “procedural engine”, to distin- By now time had progressed to about November 2003 and all the basic elements
guish it from the blr engine, which I seemed in place. Fyracle could connect to a Java client and accept oracle-style
refer to as the “relational engine”. SQL statements. There was a compiler to take the source of a PL/SQL procedure,
As in PL/SQL, this engine started life as function or trigger and compile it to byte code and the Fyracle build of Firebird
a client side test implementation. Us- could execute that byte code (see below picture).
ing the lexer and SQL parser that was You can imagine our joy when the Compiere application came up whilst connected
already taking shape, code was added to Fyracle for the first time.
to parse the various expressions and
control structures. A semantic mod- Step-wise refinement
ule was added that verified data types The joy soon turned to a long grind of bug fixing. Almost every function in Com-
and various other checks. A back-end piere failed because a built-in oracle function had not been implemented yet, be-
was added that took the parse tree and cause the PL/SQL compiler had made an error or because the SQL translator did not
generated code for the btc engine. cover a wide enough range of the full syntax.
From simple beginnings, the parser, the The harder cases were those where Firebird did not have a certain bit of functional-
semantic checks and the btc engine ity that was required by Compiere. On such a case was Oracle’s CONNECT BY syntax,
gradually grew in capabilities. which allows hierarchical queries. Another round of debate on the Architecture
At this point, the btc engine was still list ensued. In those discussions it became clear that hierarchical queries are best
a client side only entity, much like PL/ conceptualized as a special form of union and that the standard’s WITH syntax was
SQL itself had been in its earliest incar- a clearer way to express that (see separate article about WITH in this issue on page
nations. It needed an implementation …). So, the Fyracle build of Firebird was extended with support for recursive WITH
inside the engine and a way for the rela- and the oracle connector used this syntax to express CONNECT BY statements.
tional engine to call into the procedur- By March 2004, almost exactly one year after that dinner in Brussels, Fyracle was
al engine and vice versa. The first thing able to run large parts of the Compiere application. As it happened, I had to be in
to look at was the existing UDF code, the USA that month and took the opportunity to fly out to Massachusetts and meet
which enables the relational engine to with Ann and Jim, proudly showing the result. On the whiteboard the component
call into external code. It seemed like architecture was drawn out (similar to figure 2) and it was agreed that it was a
this would work, but it offered no way good way to do things. The client-side SQL-connector, of course, is not such a nice
for the external code to call back into part and it would be better if it moved to the server-side.
the engine, using the same connection

www.ibdeveloper.com 10
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 THE STORY of FYRACLE

In discussions it came up that it would a reasonable cross-platform GUI admin and a 128-bit data type come to mind.
be so much nicer if the architectural tool and agreed to create a special Fyr- • On the infrastructure side, Fyracle
layering of Firebird would be restored. acle plug-in for it. Tim O’Reilly was nice needs to implement a few of the more
This would mean that Firebird’s SQL enough to allow me to include some of commonly used predefined packages, in-
compiler (module “dsql” in the source his manuals into the developer kit. Fyr- cluding integration with a web server.
tree) would move from the Y-valve com- acle 0.8.0 was born in November 2004
ponent to the core engine component. and has gone through 10 dot-releases • Last but not least, it would be nice to
This would enable the use of dynamic since. resurrect the clustering code that was
SQL in the core engine, whereas it cur- once written for VAX clusters and use it
Two of those release contained major
rently relies on preprocessing. I argued to allow using Firebird on a Linux clus-
functionality provided by other team
that if the way this was done provided ter.
members. Vlad Horsun had worked out
a clean API for pluggable SQL compil- Fyracle has been fun for the team that
how to add global temporary tables to
ers, I could swap in my oracle-mode created it. I hope that it has been fun
Firebird, but his code was too late to be
SQL compiler for clients connecting in for you, the reader and user as well.
included in the FB2 release. Vlad kindly
oracle-mode. agreed to back-port his code from FB2
Special thanks
Jim did implement such a structure to FB15 and from there it was included
in the Vulcan branch of Firebird. I had in Fyracle in May 2005. I would like to use this opportunity to
promised to look into cleaning up the thank everybody who helped making
Eugeney Putilin had been working on Fyracle a reality, by giving advice, con-
code in a specific core engine module creating stored procedures written in tributing code or simply moral support.
which I never found the time to do. Java for a long time, probably starting Special thanks go to, in alphabetical
As a result of this mal performance, as early as 2003. By 2005 he had fig- order:
Jim promised to string me up in the ured out the issues and gotten to work-
tree next to his house. Luckily, he in- Tiberiu Adorei
ing developer builds. Roman Rokytskyy
formed me last November that the wind had modified the Jaybird driver to also Paul Beach
had blown over the tree he had had in support usage inside the server, rather
mind. Arno Brinkman
than only from a client. Eugeney, Vlad
The “coming out party” for Fyracle was and Roman worked hard late sum- Carlos Guzman
the Firebird conference in Fulda in May mer 2005 and delivered working code
Ann Harrison
2004. Most of the summer little hap- early in the autumn. It was included
pened in further development of func- in Fyracle in October 2005. Soon after, Arek Heldt
tionality, as the initial goal had been Carlos Guzman added a plugin for dot- Vlad Horsun
reached. Towards the end of the sum- NET based languages and the CLR. This
mer, Marius Popa managed to get Fyra- plugin and the dotNet driver will be in- Holger Klemt
cle discussed on Slashdot and interest cluded in the next release of Fyracle. Pascal Legrand
slowly but surely started to rise. For one,
the “oracle-mode” thing was picked up Outlook Marek Mosiewicz
by Computer Associates (“CA”) as a What once started as a one-off project Eugeney Putilin
nice marketing gimmick for the open to see if it could be done has evolved Roman Rokytskyy
source release of Ingres. CA announced into a long-term project. Much has
the “million dollar challenge”, which been achieved, but much more remains Nikolay Samofatov
amongst other projects promised $600K to be done. Jim Starkey
for a working migration tool for oracle- • The PL/SQL supported by Fyracle is
based applications. After some thought Claudio Valderrama
roughly Oracle 8i with some 9i exten-
and a look at Ingres, I decided to stick sions. Some things are glaringly miss- Dmitry Yemanov
with Firebird and not take part in the ing and will be added soon, such as
competition. Also in that Summer, En- table functions and collections. Other
terpriseDB got started on oracle-mode features to be added are the object-
Postgres, although they did not come oriented features of PL/SQL that were
out of “stealth mode” until May of mostly added in 9i. The language en-
2005. hancements in version 10g are not all
So, motivated by competition, work that significant.
started to make Fyracle a real prod- • On the relational side, Firebird needs
uct. In order to create an easy to use to get some upgrades for future releas-
package, the Fyracle configuration was es of Fyracle. Things like statement and
cleaned up and a graphical installer for database level triggers, deferred trig-
both Windows and Linux was added. A gers, materialized views, 10-base floats
guy named Arek Heldt showed up with
11 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 DEVELOPER AREA 2006 ISSUE 4

Common Table Expressions in Fyracle

By Paul Ruizendaal
[email protected]
Introduction

W hilst building the oracle-to-firebird translation module for Fyracle, it became

clear that Firebird does not have support for hierarchical queries. To Oracle
users, such queries are known as CONNECT BY queries, after the syntax that Oracle
uses to express them. The SQL standard does not support oracle’s CONNECT BY
syntax, but does support a different syntax known as recursive common table ex-
pressions. This syntax is implemented by DB2 and by SQLServer in its latest release
(2005). Expert's note
In order to be able to support CONNECT BY statements in Fyracle’s oracle-mode,
Firebird had to be enhanced to support hierarchical queries. In order to stay close
to the standard, it was implemented using recursive common table expressions. Fyracle … Yes!
In this article, I will explain the basic syntax of common table expressions, of re- Fyracle … Yes, it has lot of Oracle
cursive common table expressions and the relation to Oracle’s CONNECT BY syntax. specific features.
Really, InterBase and Firebird some-
Syntax definition times used as “small Oracle’s broth-
Common table expressions (CTE’s) are define using a new syntax element around ers”. Main office uses Oracle, sub-
the WITH keyword. Essentially, the normal SELECT syntax is extended with a pre- units uses Firebird.
amble that defines common expressions. The WITH syntax specifies a temporary
Want to move from Firebird or Inter-
named result set. One can think of the named result set as a named derived table.
base to work with >250 gigabytes
It is derived from a simple query and defined within the execution scope of a
database? Go to Oracle.
single SELECT statement. A common table expression can include references to
itself. This is referred to as a recursive common table expression and in this usage But, now Fyracle can be used not
the keyword RECURSIVE is obligatory. only as Oracle-like RDBMS.
Syntax It’s on the edge of new technologies,
like Java Stored Procedures, Global
<with statement> ::= Temporary Tables, and all other cool
WITH [RECURSIVE] <common table expression> [ , <common things.
table expression>... ] <select_statement>
Try Fyracle, if you want to dig in to
<common table expression> ::= the future.
expression name [ ( column name [ ,...n ] ) ]
AS
( CTE query definition )

Expression name
This is a valid identifier for the common table expression. An expression name
must be different from the name of any other common table expression defined
in the same WITH <common table expression> clause, but expression name can be
the same as the name of a base table or view. Any reference to expression name in
the query uses the common table expression and not the base object.
Our
Column name expert
Specifies a column name in the common table expression. Duplicate names within
a single CTE definition are not allowed. The number of column names specified
must match the number of columns in the result set of the CTE query definition.
The list of column names is optional only if distinct names for all resulting col-
umns are supplied in the query definition.
Dmitri Kuzmenko is the chief expert
CTE query definition of “The InterBase and Firebird De-
Specifies a SELECT statement whose result set populates the common table expres- veloper Magazine” with 20 years of
sion. The SELECT statement for CTE query definition must meet the same require- InterBase and Firebird experience
ments as for creating a view, except a CTE cannot define another CTE
e-mail: [email protected]
Specifying more than one WITH clause in a CTE is not allowed. For example, if a
www.ibdeveloper.com 12
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 DEVELOPER AREA
CTE query definition contains a subquery, that subquery cannot contain a nested
WITH clause that defines another CTE. The recursive form of WITH must contain
exactly two query definitions, an anchor member and a recursive member, joined
together by a UNION ALL operator.
Setting up an example
To start, we will define and fill an employee table. The table has an employee ID,
which is the primary key, a manager ID, which links the employee to his manager,
and name and salary columns:
CREATE TABLE emp(empid INTEGER NOT NULL PRIMARY KEY,
name VARCHAR(10),
salary DECIMAL(9, 2),
mgrid INTEGER);

INSERT INTO emp VALUES ( 1, ‘Winkel’, 30000, 10);

INSERT INTO emp VALUES ( 2, ‘Henkel’, 35000, 10);
INSERT INTO emp VALUES ( 3, ‘Bassinger’, 40000, 10);
INSERT INTO emp VALUES ( 4, ‘Lessig’, 38000, 10);
INSERT INTO emp VALUES ( 5, ‘Korning’, 42000, 11);
INSERT INTO emp VALUES ( 6, ‘Beagle’, 41000, 11);
INSERT INTO emp VALUES ( 7, ‘Reilly’, 36000, 12);
INSERT INTO emp VALUES ( 8, ‘Smith’, 34000, 12);
INSERT INTO emp VALUES ( 9, ‘Boot’, 33000, 12);
INSERT INTO emp VALUES (10, ‘Monroe’, 50000, 15);
INSERT INTO emp VALUES (11, ‘Schindler’, 52000, 16);
INSERT INTO emp VALUES (12, ‘King’, 51000, 16);
INSERT INTO emp VALUES (13, ‘Jones’, 54000, 15);
INSERT INTO emp VALUES (14, ‘Scott’, 53000, 16);
INSERT INTO emp VALUES (15, ‘Mills’, 70000, 17);
INSERT INTO emp VALUES (16, ‘Gaastra’, 80000, 17);
INSERT INTO emp VALUES (17, ‘Gates’, 95000, NULL);
Looking at the inserted rows, it is apparent that employee ‘Gates’ is the top man-
ager for whom ‘Gaastra’ and ‘Mills’ work. ‘Scott’ in turn works for ‘Gaastra’ and has
no employees of his own. ‘Monroe,’ on the other hand, manages ‘Henkel,’ ‘Bassing-
er,’ and ‘Lessig’ and is an employee of ‘Mills.’
Using common table expressions
Common table expressions can make complex SELECT statements more legible.
For example, the following example shows the number of employees reporting
directly to each manager:

WITH DirReps(mgrid, dirreps) AS

(
SELECT mgrid, COUNT(*) as dirreps
FROM emp
WHERE mgrid IS NOT NULL
GROUP BY mgrid
)
SELECT mgrid, dirreps
FROM DirReps
ORDER BY mgrid;

The resulting output is:

MGRID DIRREPS
============ ============

10 4
11 2
12 3
15 2
16 3
17 2

13 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 DEVELOPER AREA 2006 ISSUE 4
And the following example shows the maximum number of employees reporting
to a single manager:
WITH DirReps (mgrid, dirreps) AS
(
SELECT mgrid, COUNT(*) AS dirreps
FROM emp
GROUP BY mgrid
)
SELECT MAX(dirreps) AS “Max. Number of Direct Reports”
FROM DirReps
WHERE dirreps>= 2 ;

The resulting output is:

Max. Number of Direct Reports
=============================
4

Although common table expressions can be used as a hint to the optimiser about
how a subquery is used, and to make it explicit that a single subquery is used
at more than one place in a query, such optimisations are currently not imple-
mented.
A simple recursive query
A reasonable question to ask is: “Who works directly or indirectly for ‘Gaastra’?”
To answer the question of who works for ‘Gaastra,’ an Oracle developer might write
the following query:
SELECT name
FROM emp
START WITH name = ‘Gaastra’
CONNECT BY PRIOR empid = mgrid

START WITH denotes the seed of the recursion while CONNECT BY describes the re-
cursive step. That is how to get from step n to step (n + 1). Since it is important to
distinguigh between the nth and the (n + 1)th step during name resolution, PRIOR
is used to show that empid belongs to the nth step while mgrid belongs to step (n
+ 1)th. So with empid being 16 for step 1, mgrid must be 16 as well and hence step
2 produces ‘Scott,’ ‘King,’ and ‘Schindler.’ Their empids will now serve as PRIOR to
step 3, and so on and so forth.
The Oracle syntax is very concise. The SQL standard WITH syntax uses regular SQL
to describe the exact same relationships. As you will see, it is more verbose, but
equally straightforward:
WITH RECURSIVE n(empid, name) AS
(SELECT empid, name
FROM emp
WHERE name = ‘Gaastra’
UNION ALL
SELECT nplus1.empid, nplus1.name
FROM emp as nplus1, n
WHERE n.empid = nplus1.mgrid)
SELECT name FROM n;
What makes this CTE special is that it is referred to within its very own defintion.
This is what distinguishes a regular CTE from a recursive CTE. Here I named the
CTE n to correlate to the recursive steps. A recursive CTE consists of two parts
combined with a UNION ALL:
• The seed or step 1 of the recursion. This is what is described in Oracle using
START WITH. In a recursive CTE, it is simply any query providing a set of rows. In
this case we query the emp table and filter for ‘Gaastra.’ We select the name of
course and also the empid, because we need it for the recursive step.
• The recursive step going from n to (n + 1). Here we refer to step n (the CTE n)

www.ibdeveloper.com 14
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 DEVELOPER AREA
and join in step (n + 1) using the same predicate used in CONNECT BY. Instead of
PRIOR, regular correlation names are used to distinguish n from (n + 1).
It is noteworthy to look at the output of this query and realize that the recursive
process is a depth first recursion:
NAME
==========

Gaastra
Schindler
Korning
Beagle
King
Reilly
Smith
Boot
Scott

Before moving on to pseudo columns and more complex examples, I want to briefly
explain where the WHERE predicate of an Oracle recursion needs to be placed. We
will modify the example to return all employees, including their salaries, working
for ‘Gaastra’ who earn more than 40,000.

SELECT name, salary

FROM emp
WHERE salary > 40000
START WITH name = ‘Gaastra’
CONNECT BY PRIOR empid = mgrid

It is interesting to note how the WHERE clause preceded the recursive specifica-
tion. For a non-oracle developer, it might seem that the predicate belongs to the
set of rows being considered to begin with. This, however, is not correct. Instead,
the WHERE filters the final result and belongs at the end of the matching CTE
based query:

WITH RECURSIVE n(empid, name, salary) AS

(SELECT empid, name, salary
FROM emp
WHERE name = ‘Gaastra’
UNION ALL
SELECT nplus1.empid, nplus1.name, nplus1.salary
FROM emp as nplus1, n
WHERE n.empid = nplus1.mgrid)
SELECT name, salary FROM n WHERE salary > 40000;

This query results in the following output:

NAME SALARY
========== ============

Gaastra 80000.00
Schindler 52000.00
Korning 42000.00
Beagle 41000.00
King 51000.00
Scott 53000.00

The LEVEL pseudo column

The most well-known of the pseudo columns is LEVEL. The purpose of this column
is to show the number of the recursive step n that produced the row. In our ex-
ample, it indicates the levels of management between ‘Gaastra’ and the employee
plus 1 (because LEVEL starts with 1). Here is the original Oracle example enhanced
with LEVEL:
15 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 DEVELOPER AREA 2006 ISSUE 4
SELECT LEVEL, name
FROM emp
START WITH name = ‘Gaastra’
CONNECT BY PRIOR empid = mgrid

The SQL standard saw no need to add syntax for this feature because it can be
expressed using regular SQL:
WITH RECURSIVE n(lvl, empid, name) AS
(SELECT 1 lvl, empid, name
FROM emp
WHERE name = ‘Gaastra’
UNION ALL
SELECT n.lvl + 1, nplus1.empid, nplus1.name
FROM emp as nplus1, n
WHERE n.empid = nplus1.mgrid)
SELECT lvl, name FROM n;

The output for this query is:

LVL NAME
===================== ==========
1 Gaastra
2 Schindler
3 Korning
3 Beagle
2 King
3 Reilly
3 Smith
3 Boot
2 Scott

All I have done here is to introduce a level column, which starts with 1 and incre-
ments by 1. Of course, any semantics is possible, but this one happens to provide
the same semantics as LEVEL.
Conclusion
In this article I provided generic mappings from the Oracle style CONNECT BY
recursive query syntax to Firebird’s standard compliant recursive common table
expressions using UNION ALL. While the Oracle syntax is less verbose because it
provides keywords for various common semantics, this also means that it is less
expressive since little new semantics can be added without changes to the DBMS.

www.ibdeveloper.com 16
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4

IBSurgeon
Products
IBFirstAID
IBFirstAID is a tool that can be
used for automatically diagnosing
and repairing corrupted Firebird or
InterBase databases. It can fix up
to 80% of often corruptions.
Supports InterBase 5.x-7.x and
Firebird 1.x-2.x databases. IBSurgeon Pack
IBSurgeon Pack is a comprehensive set of
tools to repair InterBase and Firebird data-
bases and backups, undelete occasionally
deleted records and foresee database prob-
lems.
Buy IBSurgeon Pack
for EUR 399 and save 197 EUR!

IBUndelete
IBUndelete is a tool which can undelete occasionally deleted records
in InterBase or Firebird databases. It uses unique IBSurgeon core en-
gine for direct work with data inside database.
Supports InterBase 5.x-7.x and Firebird 1.x-2.x

IBBackupSurgeon
IBBackupSurgeon is a tool to read and save data
from corrupted Firebird or InterBase backup
files. With this tool you can browse a backup
file, select tables you need and then extract
them to a new or existing database.
Supports InterBase 5.x-7.x and Firebird 1.x-2.x
backups.

IBAnalyst
IBAnalyst is a tool that assists a user to analyze in detail Firebird or InterBase
database statistics and identify possible problems with database performance,
maintenance and how an application interacts with the database.
Supports InterBase 5.x-7.x and Firebird 1.x-2.x databases.

17 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 DEVELOPER AREA 2006 ISSUE 4

MORFIK’s WebOS:
Innovating beyond LAMP
Last month saw the first public beta of a revolutionary new tool to develop web ap- By Paul Ruizendaal
plications. Several developers have commented that using WebOS for the first time [email protected]
gives that same sensation of excitement that they felt when they first worked with
Delphi 1.0 back in the early nineties. WebOS comes with a bundled Firebird database
server.

Introduction Google Maps and Flickr have shown

that the user experience in a browser
I n less than two decades since its
humble beginnings, the World Wide
Web has not only permeated a fair por-
can rival that of desktop applications,
with the browser powered by nothing
A new role for traditional
operating systems
more than JavaScript, HTML/XML and The browser is already the undisputed
tion of our lives but has also become
XMLHttpRequest. platform of choice and indeed the driv-
the subject of much discussion and
ing force behind a new class of appli-
speculation as a viable alternative to However, for AJAX to succeed in the
cations which are centered around the
traditional platforms for business ap- long term, it must be supported by pro-
user and their social interactions. Ex-
plications. For many years the web fessional development tools that are
amples include e-mail, blogs, CRM, col-
community has tried to overcome the specifically designed for creating web
laboration software, discussion forums,
limitations of web browsers and their applications and also incorporate the
e-commerce and community portals.
related internet protocols by extending design methodologies and features of-
the capabilities of browsers and servers fered by products such as Visual Studio® With the advent of AJAX, browsers have
in a variety of ways. Some have tried to and Delphi®. Ideally, developers should demonstrated an ability to provide a
extend the functionality of browsers be able to leverage their existing lan- user experience that rivals the desktop
using plug-ins and applets, while others guage skills and use a familiar Inte- enabling the browser to claim new terri-
have tried to push the entire computing grated Development Environment (IDE) tory within the corporate and business
task to the server. to develop AJAX applications without applications sphere. Today, many ac-
the need for learning JavaScript or for counting and financial applications –
If taken to either of these extremes,
hand-coding HTML. inventory control, sales and marketing,
the status of the browser is essentially
production planning, human resources
reduced to that of a “dumb terminal”. This is precisely what Morfik’s WebOS
management and payroll to name a
With applets and plug-ins, the browser AppsBuilder offers. It allows program-
few – can not only be implemented as
is merely a host for another application mers to implement the business logic
browser applications but can also ben-
which runs inside a box. With server-side of their application in a high-level ob-
efit from the portability and wider con-
computing, the browser simply displays ject oriented language of their choice
nectivity that the browser provides.
what it receives like a slide-show. While and develop the presentation layer of
centralized systems, such as corporate their application using a visual design As the browser continues its march in
server farms or applications hosted by environment. WebOS AppsBuilder com- the direction of both traditional desk-
ASPs (application service providers), piles the project’s code, resources and top territory and new and innovative
are believed to need little more from objects into an AJAX application that landscapes, traditional operating sys-
the browser than the functionality of a can run both on-line and off-line. tems will gradually take a back seat
dumb-terminal, in real life the browser and revert to their original mission of
has steadily grown in functionality to
the extent that it is now a viable alter-
native platform for both the web and
desktop applications – irrespective of
their on-line or off-line state.
Recently, developments such as Asyn-
chronous JavaScript And XML (AJAX),
have captured the imagination of de-
velopers worldwide and have allowed us
to re-examine the capabilities that have
existed in web browsers for some time.
Applications such as Google Gmail,
www.ibdeveloper.com 18
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 DEVELOPER AREA
providing a hardware abstraction layer. Traditional web applications are com- of HTML and JavaScript, which rivals
This will allow traditional applications prised of a number of web pages that desktop applications in user experience
to be gradually superseded by a new are separate from one another both and computational integrity.
class of applications that fully utilize spatially and computationally. Un- • Limitations of HTML page layout
this new operating system and are de- like the user interface of conventional
signed from the outset to be user-cen- desktop applications, the browser con- HTML pages are designed to display
tric. tent changes in a disjointed fashion their content in a fluid layout much like
– sometimes with unacceptable time- a word processor. Fluid layouts take the
A new approach delays. This spatial separation of web shape of their container. Changing the
“Our objective is to bring web applica- pages has a detrimental effect on the size of the browser’s window will change
tions to the desktop and take desktop user experience. Recent developments, the alignment of page components.
applications to the web” such as AJAX, have proven successful Web designers have managed to find
Can we use the browser as a platform in enhancing the user experience, and work-arounds to address many undesir-
for real-life business applications? Will applications such as Google Gmail and able side effects of this fluid model.
such applications provide a user expe- Google Maps are examples of this. With the advent of Cascading Style
rience similar to conventional desktop Those who develop for the web are Sheets, browsers gained the ability to
applications? Will these applications forced to scatter the business logic display the elements of the page in
work both on-line and off-line? across the application space and find a predetermined fixed position – ir-
The answer to all these questions is a workarounds for managing the appli- respective of the shape or size of the
resounding yes! The capabilities re- cation state in an otherwise stateless containing window. This fixed layout
quired for this have been natively avail- environment. This computational sepa- is similar to conventional desktop ap-
able in browsers for some time, yet we ration of web pages has a detrimental plications. But since neither the size of
have been hampered by the difficulty of effect on the reliability and scalability the browser window nor the resolution
writing JavaScript code and distracted of the application. In contrast, conven- of the client display is controlled by
by inadequate attempts to extend the tional software engineering empha- the developer, the CSS model has some
browser’s functionality. sizes a unified computational model to undesirable side-effects also. The ideal
ensure reliability and scalability. model is a plastic layout that combines
By combining the following elements in the strengths of fluid and fixed models.
a single stand alone package, web ap- One option is to push all of the comput- Such a plastic layout is particularly use-
plications can be brought to the desk- ing to the server-side in order to unify ful when database reports and tabulat-
top and desktop applications taken to the computational space and make the ed data are displayed or page elements
the web: application more reliable. But this has change their size and position due to
three undesirable side-effects: first, it
1. HTML - for the user interface user interaction or business logic.
further limits scalability; second, it is
2. JavaScript - for the application logic dependant on both the availability of WebOS AppsBuilder’s plastic layout of-
band-width and the reliability of the fers the strengths of both fluid and
3. XMLHttpRequest - for asynchronous
connection; and third, the application fixed layouts. In Morfik applications,
browser requests
is no longer available when it is un- the content of the browser is a hierar-
4. Web Server - for handling the browser plugged. chy of heterogeneous nodes. Each node
requests is aware of the existence, state and be-
Morfik has the following solution to
5. Database - for managing data haviour of other nodes and can respond
limitations of a page-centric archi- to layout changes at run-time according
This concept is not new and appears tecture: WebOS AppsBuilder applica- to the rules set out by the programmer
deceptively simple, yet its implemen- tions are not page-centric. The browser at design-time. In other words, at de-
tation presents enormous technical content “morphs” according to the sign-time, the programmer can define
challenges. Unless these challenges are requirements of the application. This both the fixed position of each element
addressed and overcome, the concept simplifies state management and works as well as its run-time plasticity. This
shall remain on the drawing board. hand-in-hand with an application-spe- uniquely incorporates the best of both
cific AJAX engine. WebOS AppsBuilder
Morfik has identified and addressed worlds in layout design.
creates this AJAX engine from the busi-
these technical challenges and in the • Limitations of JavaScript
ness logic written in a high level lan-
process has created a unique Inte-
guage of choice using its unique and The difficulties of writing extensive
grated Development Environment for
patented JavaScript Synthesis Technol- and yet coherent JavaScript code is
developing business applications on
ogy (‘JST’). This approach unifies the the Achilles’ heel of AJAX and if not ad-
the browser platform. The following is
computational space of the application dressed could eventually slow down its
a technical account of these challenges
across the server and the client. widespread uptake. Few acknowledge or
and their solutions.
WebOS AppsBuilder applications do not recognize that JavaScript’s capabilities
• Limitations of a page-centric archi- extend beyond that of a mere scripting
use applets, plug-ins or cookies. The re-
tecture language for light programming tasks.
sult is an application comprised purely

19 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 DEVELOPER AREA 2006 ISSUE 4
However, market acceptance of JavaS- The process is a true compilation and tool for building large and complex ap-
cript for implementing large-scale ap- avoids boilerplates or code snippet li- plications.
plications faces the following practical braries. The source code written in the • Compiled code is, by definition, more
challenges: object-oriented, strongly typed lan- reliable and scalable than interpreted
guage of choice is first passed through
a) Syntax - JavaScript borrows its code. Although the output of the Mor-
a parser that includes a tokenizer and
syntax from C. While C/C++/C# and fik compiler is interpreted JavaScript,
syntax analyzer. The output of the pars-
Java programmers feel at home with the compilation process nevertheless
er is passed through a semantic map
case-sensitive short-hand syntax, the ensures rigorous type checking and
builder which builds a detailed seman-
large number of developers who have enforcement of referential integrity in
tic map of the entire application. This
mastered other languages such as Vi- producing reliable and scalable appli-
map conveys the full “meaning” of the
sual Basic and Delphi face a frustrating cations.
application logic. This semantic map is
transition.
then compiled into JavaScript code that • The AJAX engine created through
b) Semantic Design - JavaScript is a pro- is semantically identical to the original the Morfik process is an exact seman-
totype-based (instead of class-based) code written by the programmer and tic equivalent of the high level source
object-oriented language. This also conveys the exact same “meaning”. code which reflects the programmer’s
presents developers with the challenge intentions without imposing limita-
This process offers several advantages:
of adopting a whole new mindset. tions. Code-snippet libraries and pre-
• Mastering a programming language is fabricated component frameworks, by
c) Lack of rigor - JavaScript does not
a long and arduous process. Switching their very nature, limit the program-
provide the developer with a rigorous
to a new language requires much more mer’s ability to choose the appropriate
programming model. For example, it
than learning a new syntax. It requires level of abstraction.
does not support type declarations, nor
an idiomatic shift in the programmer’s
can functions receive their parameters Since the output of this process is nei-
thought processes. This skill takes time
by reference, to name a few. This limi- ther an executable in machine-code,
to acquire. By allowing developers to
tation is a disadvantage in large scale nor a one-to-one translation of source
program in the language they have al-
programming. code, nor a collection of predefined
ready mastered, Morfik not only encour- code snippets, WebOS AppsBuilder‘s
d) Interpreted - Interpreted languages ages the uptake of AJAX and improves process is referred to as JavaScript Syn-
do not offer the benefit of compilers in the quality of the output, but also re- thesis Technology (‘JST’):
rigorous enforcement of application in- duces the development cost of AJAX
tegrity at compile time. Small changes applications. Client-side handling of brows-
to an otherwise perfectly working pro- er requests
• The clean separation of the parser,
gram can cause unpredictable run-time
semantic analyzer and compiler al- Browsers are designed to work with
crashes. For example the most common
lows Morfik to compile from a variety servers using a stateless request-re-
error messages in existing web-page
of source languages into a variety of sponse model. Normally, the browser
scripts are “object expected” or “object
target languages. The only condition is and the server do not co-exist on the
does not support this property”.
that the source language is either ob- same hardware, and browsers need to
Morfik solves these problems with its ject-oriented (e.g. C++/C#/Java) or has stay connected to the web in order to
revolutionary JavaScript Synthesis been specifically modified to support maintain a dialog with the server. Of
Technology (‘JST’). Using WebOS Ap- object-oriented constructs. (See Ap- course, this is natural and the way of
psBuilder, programmers implement the pendix A for further details on) “multi- things on the web. However, to bring
business logic of their application in a language support in Morfik”. the web applications to the desktop
high-level object oriented language of and enable them to work after they are
• Strongly typed compiled languages
their choice (e.g. C++, C#, Java, Delphi). unplugged from the web, one must be
follow a rigorous programming model
WebOS AppsBuilder then compiles this able to handle the browser requests lo-
that has made compilers the preferred
code into a JavaScript AJAX engine. cally.

www.ibdeveloper.com 20
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 DEVELOPER AREA
One obvious solution is to install an WebOS AppsBuilder has an open and
HTTP server on the local host. This highly flexible architecture and can
will also fulfill the second part of the integrate local database engines or
objective, namely taking desktop ap- provide remote database connectivity
plications to the web. Many mature in a variety of ways. To provide a ma-
and stable examples of HTTP servers are ture, stable and well tested option that
freely available. Some are stand-alone can allow its applications run under
applications whilst others are small em- Windows, Linux and OS X, WebOS Ap-
bedded systems. Alternatively, the rel- psBuilder tightly integrates a Firebird
evant subset of HTTP server protocols server into its applications by default.
can be implemented within the appli- This is not an exclusive option and will
cation so it can natively communicate not limit the developer’s choice of oth-
with the browser and process browser er database engines.
requests.
Productivity
More importantly, the browser vendors
AJAX applications have created a lot
have recognized the benefits and pos-
of excitement but until professional
sibilities that local handling of browser
Integrated Development Environments
requests can offer, and they are plan-
become available, developers will find
ning to add this ability to the browser
it difficult to justify the effort – and
itself – effectively merging the HTTP
therefore the cost – associated with
server and the browser into a unified
AJAX programming.
system.
It is Morfik’s belief that designing an
WebOS AppsBuilder has an open and
IDE around code libraries and compo-
highly flexible architecture which can
nent frameworks is a repetition of past
support the local handling of browser
mistakes. Such tools make it decep-
requests in a variety of ways. To provide
tively easy for developers to get simple
a mature, stable and well tested option
applications up and running quickly
that can allow its applications to run
only to have them hit the wall as their
under Windows, Linux and OS X, WebOS
projects grow in size and scope.
AppsBuilder tightly integrates an em-
bedded open-source Apache server into To ensure reliability, scalability and
Morfik applications by default. This is maximum productivity, JavaScript Syn-
not an exclusive option and will not thesis Technology leverages the power
limit the developer’s choice of other of high level languages and the exist-
methods ing skills of programmers. WebOS Ap-
psBuilder is a world class system which
Database needs incorporates the features that are re-
Business applications are predomi- quired for a truly professional AJAX-
nantly data-driven and depend heav- based Integrated Development Envi-
ily on a relational database system. In ronment.
the corporate world, data is centralized
and database servers do not normally Multi language support in
co-exist with the client applications on WebOS
the same hardware. Consequently, the At the core of the WebOS AppsBuilder
client needs to stay connected to the Integrated Development Environment
network in order to maintain a dialog for AJAX lies the time-tested and prov-
with the server. en technique of using a strongly typed
To bring data-driven web applications object-oriented language and a com-
to the desktop and enable them to work piler, which strictly enforces referential
after they are unplugged from the net- integrity, high levels of modularity, re-
work requires a local database engine. factoring and polymorphism within the
This will not only fulfill the second part source code, to develop applications
of the objective, namely taking desktop that are reliable, scalable and easy to
applications to the web, but also will maintain.
facilitate the implementation of large The output of a compiler is invariably
scale distributed databases and distrib- in a language different from that of the
uted computing. source code – usually machine code but

21 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 DEVELOPER AREA 2006 ISSUE 4
sometimes an intermediate language. A and Morfik Object Pascal (with more to highly modular architecture. There is a
good example of this is the Microsoft come) for the AJAX platform. Program- clean separation of the syntax parser,
compiler that takes the source code in mers choose their favorite syntax (or semantic analyzers and the target code
C#, VB.NET or J# and produces code in even mix and match a number of them) synthesizer. This enables WebOS Ap-
Microsoft Intermediate Language (MIL). to develop their application. This al- psBuilder to compile from a variety of
When examining a snippet of sample lows them to benefit from the rigorous source languages into a variety of target
code in Microsoft Developers Network software engineering techniques and languages. The only condition is that
(MSDN), one is usually given the option system design methodologies inherent the source language must be strongly
of seeing it in one of these languages. in the object-oriented programming typed and either object-oriented (e.g.
Yet if these code snippets are carefully paradigm. WebOS AppsBuilder then syn- C++/C#/Java) or has been specifically
compared, one realizes that semanti- thesizes the source code into semanti- modified to support object-oriented
cally all three languages are identical! cally equivalent JavaScript code and constructs.
Therefore, the MIL code generated is packages it into an AJAX application.
also the same. Free download
In this model, the synthesized JavaScript
The beta releases of Morfik's WebOS Ap-
To achieve this, in the case of Visual in WebOS AppsBuilder is the equivalent
psBuilder is a free download.
Basic, Microsoft had to take it to the of MIL in .NET. In addition, WebOS Apps-
next level, and developed the strongly Builder does not prevent extending the Please check http://www.morfik.com
typed object oriented VB.NET. Although JavaScript output by linking-in external to download your own evaluation copy
unpopular with some VB programmers hand written JavaScript code libraries. and get a feel for the future of web ap-
this move was nevertheless a technical However, such external code would not plication development.
necessity. benefit from the same rigorous checks Move over LAMP, WebOS is here!
as native WebOS AppsBuilder code.
Morfik has used this strategy in develop-
ing Morfik Basic, Morfik C#, Morfik Java WebOS AppsBuilder has a flexible and

www.ibdeveloper.com 22
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 DEVELOPER AREA

Global Temporary
Tables in Fyracle By Vlad Horsun, [email protected]

P rogrammers sometimes need to store

data that is temporary in nature: for
example, to create a data set, work with
the ordinary programmer! :). The solution comes as various kinds of temporary
tables. In this article we’ll consider global temporary tables, first as they were
implemented in IB 7.5; then, the Fyracle implementation, that is to be included in
it and remove it. If it is an infrequent Firebird soon, after version 2.0.
or one-off necessity, just creating and
dropping a table is feasible. A Definition of GTTs
Global temporary tables (GTTs) are tables with permanent metadata, stored in the
But what if the need for temporary ta-
system catalogue, but with the temporary data.
bles arises regularly? In practical terms,
constantly creating and dropping tables GTT’s may be associated with data having two kinds of persistence: within the life-
is not good from the security point of time (scope) of the connection in which the given GTT was referenced and within
view—end-users don’t usually have the the lifetime (scope) of just the referencing transaction. The data in GTTs from
appropriate rights—and performing different connections or transactions, respectively, are isolated from one other,
DDL on production databases is against but the metadata of the global temporary table is shared by all connections and
all recommendations, anyway. Thirdly, transactions.
such an approach is far from perfect for
productivity.
Syntax and semantics
The statement syntax for creating the metadata of a temporary table in the system
All that apart, if the same sets of tem-
catalogue is as follows:
porary data are needed for several users
simultaneously, each would be com- CREATE GLOBAL TEMPORARY TABLE <table_name> <table_elements>
pelled to assign them unique names, [ON COMMIT {PRESERVE | DELETE} ROWS]
making such tables unworkable with
stored procedures. The ON COMMIT clause sets the scope of the data persistence for the temporary
table:
The “Service Field” Approach
ON COMMIT PRESERVE ROWS causes data left in the table after a transaction
One widespread approach to eliminate
ends to remain in database until the end of the connection
these problems is to create the table as
a permanent table with an additional ON COMMIT DELETE ROWS causes the data to be be deleted from database im-
service field populated during the mediately after a transaction ends
life of the working data set with CUR- If the optional ON COMMIT clause is not specified, then ON COMMIT DELTE ROWS
RENT_USER or with a private value from is used by default.
a generator.
CREATE GLOBAL TEMPORARY TABLE is an ordinary DDL statement that is pro-
Although this approach is free from the cessed by the engine the same way as a CREATE TABLE operation, so creating and
aforesaid deficiencies, it is not without dropping a GTT within a stored procedure or trigger is similarly prohibited. The
deficiencies of its own, not least being SQL standard has other kinds of temporary tables that can be allowed such treat-
the accumulation of many back versions ment, the most suitable for this purpose being the DECLARED LOCAL TEMPO-
as the subsequently unwanted rows are RARY TABLE—but that’s another story.
deleted. Since the additional field is
usually indexed, the index remains in- A GTT differs from a permanent table by way of two new flags in RDB$RELATIONS.
teresting to all queries on these tables. RDB$FLAGS. Because it involves no change to the on-disk structure (ODS), it was
The removed records are never visited possible to include GTTs in Fyracle, which is based on Firebird 1.5.x. A GTT may
again, to make the back versions eli- have indexes, triggers, field¬¬ level and table level constraints, just like ordinary
gible for garbage collection, and this tables.
garbage will sit in the database until However, constraints between temporary and persistent tables follow some spe-
the a sweep is run. cific rules:
Fortunately, the SQL standard defines a) references between permanent and temporary tables are forbidden
a solution for such problems. True—it
doesn’t always promise a quiet life for b) a GTT with ON COMMIT PRESERVE ROWS cannot refer to a GTT with ON COM-
MIT DELETE ROWS
23 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 DEVELOPER AREA 2006 ISSUE 4
c) a domain constraints cannot refer to a GTT.
The following tabulation simplifies how the rules apply:
master\detail persistent preserve rows delete rows
persistent allow
preserve rows allow
delete rows allow allow

Implementation details
A GTT instance—a set of data rows created by and visible within the given con-
nection or transaction—is created when it is first referenced, usually at statement
prepare time. Each instance has its own private set of pages on which data and
indexes are stored. Data rows and indexes have the same physical storage lay-
out as permanent tables. When persistence ends (the connection detaches or the
transaction ends, depending on the scope) all pages of a GTT instance are released
immediately. Although this is similar to when you do DROP TABLE, of course the
metadata remains in the database.
The clearing of a GTT is much quicker than the ordinary row-by-row delete and its
associated garbage collection. Each data (index) page contains rows (keys) from
the same GTT instance, meaning the connection or transaction should read\write
fewer pages than they would if GTT instances were isolated by the service field.
With the “service field” design, in contrast, data pages would continue to contain
rows from old transactions and dead connections until it was garbage collected.
It can happen that the GTT design requires more space than the “service field” de-
sign. For example, 10 instances of a GTT, each with one row, would occupy 10 data
pages, while, under the “service field” design, only one page would be used, as long
as it could accommodate 10 rows. This difference doesn’t really become evident
except with a large number of small tables, so we can accept that as a reasonable
price for the benefits of speed and almost instant cleanup.
In Fyracle, the data pages of all of the GTT instances are placed in the database file.
In Firebird 2+, with better performance the objective, these pages will be placed in
separate files to allow temporary data to be written to another hard disk. This also
enables temporary files always to be opened with forced writes off, regardless of
the database setting. And—yes—there are plans to consider this mechanism for
implementing tablespaces. (No, I didn’t write this :)
There is no limit to the number of GTT instances active in the database. If you have
N transactions active simultaneously and each transaction has referenced some
GTT then you’ll have N instances of that GTT.
The InterBase implementation of GTT has one “featurebug”. If you have two trans-
actions in one connection and both transactions insert some rows into a GTT ON
COMMIT DELETE table and then one transaction commits, the data in the second
transaction’s GTT instance will disappear. Firebird’s implementation of GTT is free
of this problem.
Examples
— create usual (permanent) table :
CREATE TABLE PERSISTENT (
ID INT NOT NULL
);
— create temporary table with scope of transaction
CREATE GLOBAL TEMPORARY TABLE GTT_DELETE (
ID INT
) ON COMMIT DELETE ROWS;

— create temporary table with scope of connection

— try to reference on permanent table (this is forbidden!)

www.ibdeveloper.com 24
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 DEVELOPER AREA
CREATE GLOBAL TEMPORARY TABLE GTT_PRESERVE (
ID INT
CHECK (EXISTS(SELECT * FROM PERSISTENT))
) ON COMMIT PRESERVE ROWS;
This operation is not defined for system tables.
unsuccessful metadata update.
global temporary table “GTT_PRESERVE” of type on commit
preserve rows cannot depend on persistent table “PERSIS-
TENT”.

— create temporary table with scope of connection

CREATE GLOBAL TEMPORARY TABLE GTT_PRESERVE (
ID INT
) ON COMMIT PRESERVE ROWS;
— Lets look how rows in different transactions and connec-
tion are handled
INSERT INTO GTT_DELETE VALUES (CURRENT_TRANSACTION);
INSERT INTO GTT_PRESERVE VALUES (CURRENT_TRANSACTION);
SELECT * FROM GTT_DELETE;

ID
============
5

SELECT * FROM GTT_PRESERVE;

ID
============
5

COMMIT;
SELECT * FROM GTT_DELETE;
— no rows

SELECT * FROM GTT_PRESERVE;

ID
============
5
— our one row still here
start another isql session:
SELECT * FROM GTT_PRESERVE;
— no rows, as expected

25 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 TESTBED 2006 ISSUE 4

TPC-C based tests results

W ell, it’s time to say who the winner
of our comparison test is. Usu-
ally tests are similar to beauty contests
The test toolkit contains 3 parts – 1)
SQL scripts to create the database, 2)
the load_ib.exe module to populate the
By Alexey Kovyazin,
[email protected]

– someone should be the winner. But I database with sample data, and 3) the
have to disappoint you – it wasn’t the tpcc.exe module to run user processes
case for our tests and there is no abso- simultaneously in order to simulate the
lute winner. OLTP workload.
While the test results indicated certain All source code for these tools is open
trends that were fairly dependent on and can be downloaded from http://ibdeveloper.com/tests/tpc-c/ , so you can
settings and hardware configurations, compile and check its workings.
it did not bring a single overlord into How the test is conducted
the light. For the impatient amongst
the readers I can say that InterBase 1) Database creation. The test database is always created from scratch using SQL
7.5.1 showed the best results in most scripts. We used page size 4096 for all databases. 4096 was chosen because our
but not all of the high-end configura- previous tests1 did not show significant difference between a 4 Kb and an 8 Kb
tions, with some stability issues. page size, and because 4096 is the default page size for the modern server versions
of InterBase and Firebird.
Let’s proceed to the test results.
2) Populating the database. The module load_ib.exe populates the test database
Test team with sample values. Look below in the “How to configure tests” section to see how
First of all, I need to express my sincere to set the database size.
gratitude to Alexey Karyakin and Vlad 3) Index creation. All indices are created after data are loaded into the database,
Horsun who created and adapted the not just to speed up data loading but also to avoid spoiling the selectivity of indi-
TPC-C based test toolkit for public us- ces and thus forestall optimizer mistakes.
age.
4) Testing. Testing is performed by tpcc.exe module which runs N simultaneous
Then I’d like to thank people who threads with user queries to simulate real-world OLTP- activity. Please notice that
helped to perform tests on different all actions are done by stored procedures, so the client (tpcc.exe) consumes very
hardware and software configurations: little CPU and RAM resources. Tpcc.exe is run on the same host as the database
Vadim Dokukin (www.niklaus.ru), An- server. It can be run remotely, but it requires very wide bandwidth: during the
ton Glazunov, Sergey Chernyak, Eugeny course of testing our attempt with a regular 100Mbps network produced a lot net-
Putilin and Sergey Mereutsa. work errors (10054), so I suppose a gigabit network is called for.
Special thanks to Thomas Pfister (www. 5) Database checking. After the test is complete, the “gfix –v –full” command line
nevrona.com) and Daniel Magin (www. tool is run to check database state after intensive work and ensure it is correct.
better-office.com) for great help in
6) Database statistics. Then we run “gstat –r” to gather database statistics with
carrying out tests on special configura-
record versions.
tions.
Test duration (testing only, without loading, checking and statistics time) is usu-
Test technique ally three hours – 0.5 hours for startup and a 2.5-hour period of measurement.
I anticipate a lot of questions related Where we increased times we note it specifically.
to the execution of the tests, so please
All test results are written to appropriate logs into “Log” folder.
read this part carefully to avoid a lot of
them. Raw results
We used a TPC-C based toolkit to carry Test logs can be downloaded from http://ibdeveloper.com/tests/tpc-c/
out our tests. In essence this test simu-
lates a database warehousing system How to run tests
with W warehouses and T terminals (i.e. It might seem a difficult thing to get these tests set up to run but that’s not the
users) working simultaneously with case at all. Thanks to a useful toolkit, all you need to do to run the test is change
these warehouses, mostly inserting and the location of the appropriate server version in the setup_VERNN files, set the
updating records. desired database size and user count in prepare.cmd and then launch RUN.CMD.
All tests were carried out on Windows How to configure the test
platform. I am first to agree that it is
Let’s consider a simple example of configuring the test. Assume that you have In-
a major omission but we’ve had insuffi-
terBase 7.5 installed in folder “C:\InterBase751”, so you need to change the first
cient time and resources so far to adapt
line in configuration file “setup_ib751.cmd”
the tests for Linux.
1
Probably we will publish their results in the next issue
www.ibdeveloper.com 26
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 TESTBED
SET IB=C:\interbase751\bin
SET NAME=IB751 Expert's note
SET DATABASE=%~dp0%NAME%_tpcc.fdb
SET ISC_USER=SYSDBA
SET ISC_PASSWORD=masterkey
SET PATH_SAVE=%PATH%
Yaffil
SET PATH=%IB%;%PATH% Yaffil is a Russian clone of Firebird
SET BUFFERS = 99999 1.0. It was made in the middle of
%IB%\instreg install %IB%\.. instance gds_db 2001 to check where optimizations
%IB%\instsvc install %IB%\.. instance gds_db can be done, and how it will affect
net start IBS_gds_db server performance. Since Yaffil was
Please note that database buffers are set in setup_VERNN too. made only for Windows, there were
a lot of windows-related optimiza-
Then you need to select the desired database size and user count. It involves
tions. The whole list of optimizations
changing two lines in the prepare.cmd file.
is very long – new memory manager,
Database size is set by –W variable (W means warehouses) at the database creation faster inserts, disk i/o, additional
step. Each warehouse adds ~30Mb to database, so –W50 means to create 4.5Gb configuration parameters, optimizer
database: fixes, query scheduler, garbage col-
bin\load_ib -W50 -D\\.\%DATABASE% > log\%NAME%_load.log lection, compatibility with previous
User count and number of actually used terminals are set up at the testing step: InterBase versions (5.x and lower),
and more. Also some little parts of
bin\tpcc -W50 -T50 -Dlocalhost:%DATABASE% -r30 –R180 -i30 > Firebird 1.0 code were rewritten in
log\%name%_tpcc.log pure assembler.
Please note that we always set W= T. It’s done to avoid annoying lock conflict
errors in the logs. In fact, lock conflicts are perfectly normal in real-world OLTP Thus, some users that moved from
systems, but we’re trying to avoid exceptions for the testing because they pollute InterBase 4.x and 5.x to Yaffil, got
the test logs and make them difficult to analyse. not only better compatibility with
legacy databases than InterBase
Once all the locations in the setup_VERNN.cmd files are set up, the test – RUN.CMD 6.0 and Firebird had, but from 2 to
– is ready to launch. 6 times speedup of common server
Test participants performance. Optimization of the
internal server scheduler still allows
Which InterBase and Firebird versions were tested? We tested the following server Yaffil SuperServer to have less pro-
versions: cessor load (from 5 to 40 percent)
Server version Notes than Firebird SuperServer on some
applications (with short and long-
InterBase 7.5.1 All configurations running queries being executed si-
Firebird 1.5 SuperServer Omitted in some configurations multaneously).
due to the lack of time
Yaffil was the pioneer in reconstruct-
Firebird 1.5 Classic All configurations ing Classic for Windows functional-
Firebird 2 Release Omitted in some configurations ity, because Borland nearly dumped
Candidate 1 SuperServer due to the lack of time Classic architecture before version
Firebird 2 Release All configurations 6.0, and the published source code
Candidate 1 Classic for InterBase Classic for Windows was
broken. Yaffil successfully revived
Yaffil Classic Omitted in some configurations Classic architecture for Windows
due to the lack of time (with services api support), having
Yaffil SuperServer Omitted in some configurations high stability on ~40 gigabytes da-
due to the lack of time tabases nearly year before Firebird
Classic for Windows was released.
Because a lot of people are interested in the Vulcan results, we tried to run the
public pre-alpha Vulcan of Vulcan. Unfortunately it was too unstable to run and
crashed every time, so we had to omit it. We still hope there will be an alpha ver- Our expert
sion before too long that will be stable enough to sustain our test. Dmitri Kuzmenko
Test results Dmitri Kuzmenko is the chief expert
We ran the tests many times in different software and hardware configurations and of “The InterBase and Firebird De-
received a lot of raw results that are not yet fully processed. Space in this issue veloper Magazine” with 20 years of
of magazine is limited too, so we decided to publish a special issue of “The Inter- InterBase and Firebird experience
Base and Firebird Developer Magazine” devoted solely to test results analysis and e-mail: [email protected]
27 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 TESTBED 2006 ISSUE 4
performance optimization.
Here we will highlight the main trends and commonalities in the results to illus-
trate the general picture.
Configuration 1
Hardware configuration:
Hardware:
2xCPU 3.0Ghz, 2GB RAM,
2x36Gb Ultra SCSI 320 in RAID1+0 with 128 RAM,
Windows 2003.
50 users, Forced Writes off

Server Performance, Tpmc Buffers

Firebird 2 RC1 Classic 1218.29 2048
Firebird 1.5.3 Classic 905.14 2048
Firebird 1.5.3 704.02 9900
SuperServer
InterBase 7.5.1 1746.16 99999
Yaffil Classic Server 1.0 1075.08 2048
Yaffil SuperServer 1.0 743.86 9900

This was the most balanced hardware configuration – a very good disk system is
combined with good CPUs and a lot of RAM. The database size for 50 users is 4.5Gb
would not fit into RAM, so disk performance was high in importance.
We set Forced writes off to get maximum performance. It’s the default mode in
InterBase 7.5.1. For Firebird, in addition to setting Forced Writes off, we set the
parameters MaxUnflushedWrites and MaxUnflushedWriteTime were to -1 in fire-
bird.conf to disable flushing.
So, InterBase 7.5.1 is winner here, with Firebird 2 Classic Release Candidate 1
showing its enhanced capability, compared to 1.5.3, to work with disks.
Firebird 1.5.3 and Yaffil 1.0 SuperServer versions showed lower performance be-
cause, in this test, dual CPUs played an important role – fast disks and a lot of RAM
being able to keep the CPUs busy.
100 users, Forced Writes On
After that, simulated a heavily loaded system. We increased the workload and ran
the test with 100 users and 100 terminals, so the database became 9.5Gb. We also
set Forced Writes on and increased test time to 6 hours.

www.ibdeveloper.com 28
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 TESTBED
We launched only 3 servers to compare the maximum workload of InterBase SMP
and the Firebird 1.5.3 and Firebird 2 Release Candidate 1 Classic architectures.
Firebird 1.5.3 SuperServer, Firebird 2 Release Candidate 1 SuperServer and also
Tips
Yaffil were omitted because of time constraints. Below are results:
Do you know how
Server tpmC Buffers many transactions
FFirebird 2 Release 479.95 2048 your application starts
Candidate 1 Classic
every day?
Firebird 1.5.3. Classic 88.43 2048
You can check it by running gstat
InterBase 7.5.1 754.76 1000000
–h db.gdb every day, but IBAna-
Below is the results graph. As you can see, Firebird 1.5.3 showed the lowest perfor- lyst will show daily average transac-
mance, and InterBase 7.5.1 is the winner here. tions count on the fly.
System with 50-70 users usually
starts about 50-500 thousands
transactions per day.
Is it much, or not?
It depends on how you have written
your application, i.e. how you man-
age transactions within application.
There may be not many transactions
per day, but if these transactions are
long, they can cause lot of record
versions in database, and perfor-
mance may degrade (as usual).
On the other side, even 1-5 million
short transactions per day may not
load server much.
But there are some circumstances which are rather important to mention. The
activity of InterBase 7.5.1 was rather unstable and performance tended to degrade
– please see graph below.

First, InterBase 7.5.1 had a lot of deadlocks. As I commented earlier, it’s rather normal, but InterBase didn’t exhibit them in
the test with the lower user count. More importantly, Firebird 2 Release Candidate 1 and Firebird 1.5.3 exhibited no deadlocks
at all, even with 100 users.
The second point is InterBase’s tendency to decrease in performance, with no obvious explanation. This test cycle (all three
servers) required more than 24 hours to complete, which is why we repeated it only twice. We got the same results (+- 1%)
on the repeat runs. It would be interesting to run InterBase alone for 2-3 days and see what happens.
29 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 TESTBED 2006 ISSUE 4
The third instability we saw with InterBase was that InterBase, three times, went
to zero performance for 30-60 seconds – just stopped to answer to client requests.
The yellow vertical lines on the graph were where these trouble spots occurred.
I’m guessing that, as a result of these stops, database checking (gfix -v -full) indi-
cated 1 record error:
Summary of validation errors
Number of record level errors : 1

Configuration 2
by Daniel Magin, [email protected]
Hardware: Pentium 3.0, HT ON, 2Gb/1Gb, SATA 300Gb
Many of our customers run InterBase on standard workgroup servers so I was in-
terested to analyse the results of testing the different server versions on a host
machine that was specifically not a high-speed system with multiple CPUs. My
base test platform was a simple machine with a Pentium 4 3.0 GHz processor with
Hyperthreading and 2 Gb RAM on an Asus P4P800-E mainboard. The hard drive
was a 300 Gb Maxtor SATA2 V300FO with a 16 Mb Cache but it was running on the
mainboard’s SATA1 interface.
On the only existing partition C (90GB) I installed a new Windows 2003 Server
Enterprise Edition. All tests ran on this single partition. Though there’s no ques-
tion that results could be sped up by spending more money on SCSI-RAID and more
RAM, the focus of my interest for this test was how this fairly ordinary configura-
tion would respond.
I used the latest official version of each of the different database engines. I start-
ed out by playing around quite intensively with some settings to find the best
configuration for each one for speed. I settled on the following buffer settings:

Server version Buffers

Firebird 1.5.3 Classic 2048
Firebird 1.5.3 SuperServer 9000
Firebird 2 Release Candidate 1 Classic 2048
Firebird 2 Release Candidate 1 SuperServer 100000
InterBase 7.5.1 100000
Yaffil Classic 2048e
Yaffil SuperServer 9000
I ran Alexey’s complete TPC-C Test for many hours. The test system was a ware-
housing system with 30 terminals, with a starting database size of around 2.8 Gb.
The 30 terminals would represent, for example, a web application with between
500 and 1000 users on-line.

Server tpmC Loadingtime RunningTime Transactions Deadlocks

Firebird 1.5.3 238,83 872,59 10800 82422 0
Classic
Firebird 1.5.3 522,55 815,67 10800 180251 3
SuperServer
Firebird2 Classic 252,85 951,7 10800 87234 0
Firebird 2 488,77 944,891 10800 168615 1
SuperServer
InterBase 7.5.1 832,64 1010,57 10800 287260 1
Yaffil Classic 697,77 768,531 10800 234516 0
Yaffil 659,77 765,53 10800 227630 0
SuperServer

www.ibdeveloper.com 30
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 TESTBED

News line
Devrace
has acquired
Metadataforge
Devrace has recently acquired Meta-
dataforge company (SQLHammer
project) and will immediately start
selling the product.
The CEO of Devrace Serg Vostrikov,
says: “We are very happy to have
signed the contract with Dmitry
Kovalenko, the author of SQLHam-
mer. SQL Hammer is a simple, fast
and elegant IDE for database devel-
opment and invaluable tool for real
InterBase, current Version 7.5.1, was the winner in my TPC-C based test, except administrators. When developing
it came last in loading the datascript. I don’t know why. Most significant is the SQL Hammer Dmitry was concentrat-
extremely good speed InterBase delivers if it can find 2 Gb of RAM. ed on improving the performance of
With only 1Gb of RAM, InterBase 7.5.1 and Firebird 2 Release Candidate 1 Classic access to databases, and so far judg-
delivered similar speeds, and Firebird 1.5.3 SuperServer became the leader: ing by recent tests these attempts
are very successful. I am sure soon
Server tpmC Loadingtime Transactions Set Buffers we will prove InterBase/Firebird da-
Firebird 1.5.3 159.18 1475.36 54900 2048 tabase developers that SQL Hammer
Classic is worth buying!”
Firebird 1.5.3 443.11 825.65 152889 9900 Sophisticated internal SQLHammer
SuperServer architecture is based on plug-in
Firebird 2 202.88 951.53 69977 2048 modules which extend IDE func-
Classic tionality. For example centralized
Firebird 2 400.46 940.68 138213 6400 SQL Editor is used in most database
SuperServer objects and editors, whereas it is
InterBase 341.47 1151.56 117820 6400 physically present only in a single
7.5.1 module. Such a module architecture
Yaffil Classic 377.3 776.18 130212 2048 provides developers with additional
advantages:
Yaffil 359.09 783.26 121807 9900
SuperServer • saving computer resources,
• an ability to use only necessary
functions and deactivate the other
tools,
• full support of all Borland Inter-
Base/Firebird versions, as we use
different access means for each
server,
• better support of client develop-
ment tools, in particular, SQLHam-
mer has SQLMonitor for FIBPlus and
SQLMonitor for IBX,
• more convenient updates: devel-
So if you use InterBase, having the full 2 Gb of RAM available will be rewarding. opers get an update of a certain
Yaffil—which is based on Firebird 1.0 but has a better memory manager—also module or of the product core in-
showed itself very good at handling the bigger memory. If you use open source stead of full package at once.
versions and you have at least 2 Gb RAM, Yaffil would be worth a try if you need
more speed. www.devrace.com
31 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 TESTBED 2006 ISSUE 4
Hyperthreading
The next point for analysis was hyperthreading. Does HT really speed up the sys-
tem? Which versions actually work with HT?
I decided to increase the work for the server by changing the terminal/warehouse
setting to 50.
Bigger DataBase + More Thread Connections:=more work for the server :-)
The following results are just for comparing the differences between having HT
enabled and having it disabled. With a high-speed hard disk system such as SCSI,
you could vastly expand the Delta because the CPUs would not be kept waiting for
read/write operations.

Server HT ON HT OFF Delta

Firebird 1.5 311,62 301,23 103,45%
Firebird 2 323,08 353,25 91,46%
InterBase 7.5 442,82 414,52 106,83%
The Firebird 2 results are interesting: enabling HT actually causes the TPC-C values
to decrease, apparently with a negative impact that is higher than the gains shown
by using HT with InterBase, Firebird 1.5 and Yaffil.
Daniel Magin has more than 18 years of experience in a variety of hosting, multi-tier,
and client/server projects. He is a frequent tutorial speaker in Germany and is a certi-
fied trainer for Borland Delphi, InterBase and Microsoft.net. He is also employed by
Better-Office. Daniel has spoken at multiple international conferences in the USA, Eu-
rope and the Arab Emirates on object-oriented programming, Java, Delphi, InterBase,
workflow management, application servers, and database and application design.

Customer satisfaction is our top priority! Integrating the Enterprise!

better office is a German company specialized

in Borland and Microsoft development environments
With offices in Oldenburg (head office),
Frankfurt, Berlin, Pennsylvania (USA)
and a Team of appr. 25 associates.
better office provide development tools,
custom made software development,
software consultancy,
Project management, training and support.
They have extensive experience in developing client-server
and web based database systems.
Benefits:
Borland Delphi WIN32 and .Net Head Office: Oldenburg
Microsoft .Net Framework Stau 19 – 6.OG -
(C# Visual Studio) 26122 Oldenburg
JAVA JBuilder and Eclipse
Tel.: +49 (0)441 926740
Borland InterBase
Fax: +49 (0)441 2488675
Microsoft SQL-Server
IBM DB/2/40 (iSeries) E-Mail: [email protected]

www.ibdeveloper.com 32
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4 TESTBED
Configuration 3
Hardware: Fujitsu-Siemens 200TX S2, 1xXeon 3.2Ghz 64EM-T, HT OFF, 2Gb RAM, Expert's note
6x36Gb UltraSCSI320, RAID 1+0, Windows 2003 64 bit Edition
30 users, Forced Writes off 64-bit
This test was interesting because of the abnormal speed of Yaffil Classic Server 64-bit processors now are nearly
– it was more than twice as fast as InterBase 7.5. We believe the reason for it everywhere, from desktop to server.
is Windows-optimized code in Yaffil (as you probably know, Yaffil is for Windows What benefit could be gained from
platform only) and its spinlock-based lock manager. Critical low-level operations these processors?
are implemented in Yaffil in Assembler. Yaffil’s memory manager also differs from The initial win is for gamers, espe-
both the InterBase and Firebird memory managers. cially those who already have AMD 64
64-bit based CPUs and OS are coming soon, so it may be useful for InterBase and processors in their systems. The next
Firebird engineers to take a look at the Yaffil implementation. comes when 64-bit operating sys-
tems come into currency. Windows
Server Performance, Tpmc Buffers XP Professional and Windows 2003
Firebird 1.5.3 Classic 252.170 2048 Server 64-bit editions are available
Firebird 1.5.3 497.010 9000 already but the big problem for both
SuperServer is drivers and software.
Firebird 2 Release 448.33 9000 Before installing these operating
Candidate 1 SuperServer systems you need to be sure that sta-
ble drivers exist for your hardware. If
Firebird 2 Release 440.400 2048
you will use only 32-bit programs,
Candidate 1 Classic
performance will be the same as for
InterBase 7.5.1 863.760 100000 the 32-bit operating system (except
Yaffil Server 1.0 Classic 2,038.160 2048 strange Yaffil results in tpc-c test),
and could be worse (especially in
games, up to -40%).
For now, it doesn’t make a lot of
sense to install 64-bit operating sys-
tem unless you have a specific need
to do so and the software and driver
support to make it work. Neverthess,
be prepared to install it in the not-
too-far-distant future.

Yaffil 1.0 SuperServer was omitted due to a service startup problem that we didn’t
notice until it was too late to fix it. The hardware was no longer available to repeat
the test. Our
Servers results overview expert
InterBase 7.5.1
Currently it’s the most powerful server for the most powerful SMP hardware con-
figurations. It has some problems with stability and is sensitive to the amount of
installed RAM. And we should not forget about TPC-R results which showed that
InterBase 7.5.1 required a lot of query optimizations in real-world applications.
But, if SQL developers are optimizing queries effectively, the two tests indicate Dmitri Kuzmenko is the chief expert
there is a performance advantage for InterBase from investing in multi-CPU com- of “The InterBase and Firebird De-
puters with fast disk systems. veloper Magazine” with 20 years of
InterBase and Firebird experience
Firebird 1.5.3
e-mail: [email protected]
It’s an old horse, but it’s not too bad! For single CPU computers SuperServer does
33 www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 TESTBED 2006 ISSUE 4
quite well. On a 64-bit computer with Yaffil 1.0
Windows 2003 it does even better than And about our dark horse. I suppose many of our readers first got to know of the
Firebird 2 Release Candidate 1. Firebird Yaffil project in this article. Firebird 2 incorporated quite a lot of Yaffil’s function-
1.5.3 looks good for small to medium ality (mostly System Defined Functions and some improvements), but there are
size databases and user bases. And we still many interesting things, especially in speed issues and heavy-load user man-
should not overlook its very high sta- agement. Yaffil 1.0 is rather common in Russia – it is used for quick replacement
bility. for old InterBase 4.x and InterBase 5.x databases due its special backward capa-
Firebird 2 Release Candidate 1 bilities and high speed. With single backup and restore and without any changes
in application code users get faster and more stable databases. Of course, Yaffil
It’s not good to test beta versions, be-
isn’t in line for new development, but there are a lot of ideas inside it to explore.
cause they change so quickly. For ex-
ample, Firebird 2 Release Candidate 2 Summary
is available already and we’ll certainly
I think many readers (especially fans of Firebird or InterBase) will be dissatisfied
test it and publish the results in the
with this article…. It’s not possible to determine an absolute leader and too many
next issue. But we have to admit that, things depend on hardware, operatng system and configuration parameters that
for a beta, Release Candidate 1 is doing we haven’t specified here.
very well. Its performance is only 30-
40% lower than InterBase 7.5.1 run- It was a real discovery for us that performance of modern InterBase and Firebird
ning on powerful SMP configurations versions can vary so much. As we discovered, performance is not a simple thing.
with a lot of RAM installed (2Gb+). On It’s hard to describe all the testing we did in a single article. We’ll be having a
more modest hardware configurations special issue of our magazine devoted exclusively to the results and their inter-
it is even better, and also demonstrated pretation. It will cover configuration aspects like playing with firebird.conf and
very good stability. The forthcoming IBCONFIG parameters, the impact of new settings like GROUP COMMIT on perfor-
release of Firebird 2 looks likely to be mance, comparing tables of results for different settings, detailed explanation of
more than a match for InterBase. We the interdependencies between elements of hardware configuration.
intend to pay a lot of attention to its We hope that some readers will be able to repeat the tests using our toolkit and
evolution. send us the results for publication.

www.ibdeveloper.com 34
© Copyright 2005-2006, All right reserved www.ibdeveloper.com
 2006 ISSUE 4

Buy electronic version now

and get instant access
to full PDF!

www.ibdeveloper.com
© Copyright 2005-2006, All right reserved www.ibdeveloper.com