Synopse

To content | To menu | To search

mORMot Framework

Entries feed

2012, Tuesday February 14

ORM cache

Here is the definition of "cache", as stated by Wikipedia:

In computer engineering, a cache is a component that transparently stores data so that future requests for that data can be served faster. The data that is stored within a cache might be values that have been computed earlier or duplicates of original values that are stored elsewhere. If requested data is contained in the cache (cache hit), this request can be served by simply reading the cache, which is comparatively faster. Otherwise (cache miss), the data has to be recomputed or fetched from its original storage location, which is comparatively slower. Hence, the greater the number of requests that can be served from the cache, the faster the overall system performance becomes.

To be cost efficient and to enable an efficient use of data, caches are relatively small. Nevertheless, caches have proven themselves in many areas of computing because access patterns in typical computer applications have locality of reference. References exhibit temporal locality if data is requested again that has been recently requested already. References exhibit spatial locality if data is requested that is physically stored close to data that has been requested already.

In our ORM framework, since performance was one goal since the beginning, cache has been implemented at four levels:

  • Statement cache for reuse of SQL prepared statements, and bound parameters on the fly - note that this cache is available not only for the SQlite3 database engine, but also for any external engine; 
  • Global JSON result cache at the database level, which is flushed globally on any INSERT / UPDATE
  • Tuned record cache at the CRUD/RESTful level for specified tables or records on the server side; 
  • Tuned record cache at the CRUD/RESTful level for specified tables or records on the client side.

Continue reading...

2012, Monday February 6

Modification of TSQLRestServerCallBack method prototype (bis)

In order to implement some RESTful Services, a callback has to be defined on the server side.

The prototype of these methods has been modified one more time, to supply an unique parameter:
This is a CODE BREAK change and you shall refresh ALL your server-side code to match the new signature.

This unique parameter will let the signature remain untouched in your code implementation, even if the framework evolves (like adding a new parameter).

Continue reading...

2012, Sunday January 29

Microsoft states: OleDB out - enjoy ODBC!

For our native connection to any DB, we developed a set of classes and several units.

We implemented at first OleDB, then native Oracle direct access and SQlite3 static engine.

Now, Microsoft is officially deprecating OleDB, and urge all developers to switch to the open and cross-platform ODBC API for native connection.

Continue reading...

2011, Friday December 30

Hash collision attack

A variety of programming languages suffer from a denial-of-service (DoS) condition against storage functions of key/value pairs in hash data structures, the condition can be leveraged by exploiting predictable collisions in the underlying hashing algorithms.

The issue finds particular exposure in web server applications and/or frameworks. In particular, the lack of sufficient limits for the number of parameters in POST requests in conjunction with the predictable collision properties in the hashing functions of the underlying languages can render web applications vulnerable to the DoS condition. The attacker, using specially crafted HTTP requests, can lead to a 100% of CPU usage which can last up to several hours depending on the targeted application and server performance, the amplification effect is considerable and requires little bandwidth and time on the attacker side.

Source: #2011-003 multiple implementations denial-of-service via hash algorithm collision

Continue reading...

2011, Tuesday December 6

Automatic JOIN query

In mORMot, all the methods available to handle many-to-many relationship (ManySelect, DestGetJoined...) are used to retrieve the relations between tables from the pivot table point of view. This saves bandwidth, and can be used in most simple cases, but it is not the only way to perform requests on many-to-many relationships. And you may have several TSQLRecordMany instances in the same main record - in this case, those methods won't help you.

It is very common, in the SQL world, to create a JOINed request at the main "Source" table level, and combine records from two or more tables in a database. It creates a set that can be saved as a table or used as is. A JOIN is a means for combining fields from two or more tables by using values common to each. Writing such JOINed statements is not so easy by hand, especially because you'll have to work with several tables, and have to specify the exact fields to be retrieved; if you have several pivot tables, it may start to be a nightmare.

Let's see how our ORM will handle it.

Continue reading...

2011, Wednesday November 30

AJAX authentication

A nice framework user, named esmondb, did write and publish some JavaScript code to handle our RESTful authentication mechanism.

It seems to work well, and implements all secure hashing and challenging.
Our authentication mechanism is much more advanced than the one used by DataSnap - which is a basic HTTP authentication with the password transmitted in clear (this is the reason why it shall better be used over HTTPS, whereas mORMot can be used over plain HTTP).
Resulting JavaScript code seems not difficult to follow, even for a no JS expert like me.

Continue reading...

2011, Sunday November 27

Modification of TSQLRestServerCallBack method prototype

In order to implement some RESTful Services, a callback has to be defined on the server side.

The prototype of these method has been modified, to supply an additional aSession: cardinal parameter: this is a CODE BREAK change and you shall refresh ALL your server-side code to match the new signature.

Continue reading...

2011, Sunday September 25

Synopse SQLite3/mORMot framework 1.15

Our Client-Server ORM framework is now available in revision 1.15.

This is a major upgrade of the framework:

  • It is now called mORMot - so please update your T-Shirts or coffee cups ;)
  • It is able to use any Database engine back-end - in fact, it is SQLite3 powered, not SQLite3 limited;
  • In particular, direct OleDB and native Oracle have been implemented;
  • It makes use of the genuine SQlite3 Virtual Table mechanism everywhere to allow mixed access to any database engine;
  • New TModTime / TCreateTime kind of fields;
  • Enhanced stability, speed and multi-thread implementation;
  • Methods and functions have been enhanced, according to user feedback (thanks you all for your interest and forum posts!);
  • Extended documentation (more than 700 pdf pages), with new diagrams and a lot of new content;
  • New associated tools, like LogViewer or SynDBExplorer;
  • The SQLite3 core can now be used without our ORM - it has been updated to the latest 3.7.8 version;
  • Open Source (under GPL/LGPL/MPL license), running from Delphi 6 up to XE2.

Continue reading...

2011, Wednesday September 14

L10n and i18n in our framework

In computing, internationalization and localization (also spelled internationalisation and localisation) are means of adapting computer software to different languages, regional differences and technical requirements of a target market:

  • Internationalization (i18n) is the process of designing a software application so that it can be adapted to various languages; 
  • Localization (L10n) is the process of adapting internationalized software for a specific region or language by adding locale-specific components and translating text, e.g. for dates display.

Our framework handle both features, via the SQLite3i18n.pas unit. For instance, resourcestring defined in the source code are retrieved from the executable and can be translated on the fly. The unit extends this to visual forms, and even captions generated from RTTI.

In short, making your software open to any language is handled by the framework, from the bottom-up.

Continue reading...

2011, Wednesday August 10

Framework documentation updated for revision 1.15

The framework documentation was just updated.

The general organization of the SAD document (which is the one to be read in all cases) has been refreshed, and is now separated in smaller chapters.

The new official name has been changed into "Synopse SQLite3/mORMot framework"...

Continue reading...

2011, Monday August 8

Our mORMot won't hibernate this winter, thanks to FireMonkey

Everybody is buzzing about FireMonkey...

Our little mORMot will like FireMonkey!
Here is why...

Continue reading...

2011, Sunday August 7

SQLite3-powered, not SQLite3-limited

Our downloadable documentation has been enhanced, and contains now a description about the main feature of the near-to-come 1.15 version, i.e. "database agnosticism".

The core database of our mORMot / SQLite3 framework uses the SQLite3 library, which is a Free, Secure, Zero-Configuration, Server-less, Single Stable Cross-Platform Database File database engine.

As stated below, you can use any other database access layer, if you wish. A fast in-memory engine is included, and can be used instead or together with the SQLite3 engine. Since revision 1.15 of the framework you may be able to access any remote database, and use one or more OleDB (or Oracle) connections to store your precious ORM objects. The SQlite3 will be used as the main SQL engine, able to JOIN all those tables, thanks to its Virtual Table unique feature.

Continue reading...

2011, Monday July 25

Close future of the framework: database agnosticism

Our ORM RESTful Framework is about to access any available database engine.

It will probably change its name (since it won't use only SQlite3 as database), to become mORMot - could be an acronym for "Manage Object Relational Mapping Over Tables", or whatever you may think of...

We'll still rely on SQLite3 on the server, but a dedicated mechanism will allow to access via OleDB any remote database, and mix those tables content with the native ORM tables of the framework. A flexible Virtual Tables and column mapping will allow any possible architecture: either a new project in pure ORM, either a project relying on an existing database with its own table layout.

Continue reading...

2011, Saturday July 9

SynDBOracle: Open Source native Oracle access

For our upcoming mORMot framework, and in completion to our SynOleDB unit, we just added a new Open Source unit, named SynDBOracle. It allows direct access to any remote Oracle server, using the Oracle Call Interface.

Oracle Call Interface (OCI) is the most comprehensive, high performance, native unmanaged interface to the Oracle Database that exposes the full power of the Oracle Database. We wrote a direct call of the oci.dll library, using our DB abstraction classes introduced for SynOleDB.

We tried to implement all best-practice patterns detailed in the official Building High Performance Drivers for Oracle document

Resulting speed is quite impressive: for all requests, SynDBOracle is 3 to 5 times faster than a SynOleDB connection using the native OleDB Provider supplied by Oracle. We noted also that our implementation is 10 times faster than the one provided with ZEOS/ZDBC, which is far from optimized.

You can use the latest version of the Oracle Instant Client provided by Oracle - see this link - which allows you to run your applications without installing the standard (huge) Oracle client or having an ORACLE_HOME. Just deliver the dll files in the same directory than your application, and it will work.

Continue reading...

2011, Saturday July 2

Is Object-Relational Mapping the Paradise of Computer Science?

There is a well known syndrome around, against ORM.

Do you remember The Vietnam of Computer Science article?

It is worth reading... and commenting.
Sounds a bit outdated by now. Tempus fugit!

Continue reading...

Faster variant late binding

For both our SynOleDB and SynBigTable units, we allow late-binding of data row values, using a variant and direct named access of properties. Thanks to this unique feature (as far as I know in the Delphi database world),

This allows clear and valid code as such:

var Customer: Variant;
begin
  with Props.Execute(
    'select * from Sales.Customer where AccountNumber like ?',
    ['AW000001%'],@Customer) do
    while Step do
      assert(Copy(Customer.AccountNumber,1,8)='AW000001');
end;

In practice, this code is slower than using a standard property based access, like this:

    while Step do
      assert(Copy(Column['AccountNumber'],1,8)='AW000001');

But the first version, using late-binding of column name, just sounds more natural.

Of course, since it's late-binding, we are not able to let the compiler check at compile time for the column name. If the column name in the source code is wrong, an error will be triggered at runtime only. But it would not be an issue, since it would be the same for the SQL code inserted: it's only executed at runtime (this is one of the benefits of using an ORM, by the way: the ORM will generate correct SQL code for you...).

The default VCL implementation of this late-binding was a bit slow for our purpose.
Since it has to deal with Ole Automation, and because it's fun, we hacked the VCL to provide a lighter and faster version for our custom variant types.

Continue reading...

2011, Friday July 1

SynOleDB: OpenSource Unit for direct access to any database via OleDB

That's it, our SynOleDB unit seems alive and running well.

OLE DB (Object Linking and Embedding, Database, sometimes written as OLEDB or OLE-DB) is an API designed by Microsoft for accessing data from a variety of sources in a uniform manner. It was designed as a higher-level replacement for, and successor to, ODBC, extending its feature set to support a wider variety of non-relational databases, such as object databases and spreadsheets that do not necessarily implement SQL.

SynOleDB unit implementation has been made with several points in mind:

  • Tested with SQL Server 2008 R2 and Oracle 11g providers from Microsoft and Oracle; 
  • Ability to be truly Unicode, even with pre-Unicode version of Delphi (like Delphi 7 or 2007); 
  • Could access any local or remote Database, from any version of Delphi, since it doesn't use the DB.pas unit or any related part of the VCL (even the Delphi 7 personal or the Turbo Explorer editions), just for free; 
  • Handle NULL or BLOB content for parameters and results; 
  • Avoid most memory copy or unnecessary allocation: we tried to access the data directly from the retrieved data buffer, just as given from OleDB; 
  • Was therefore designed to achieve the best performance possible: most time is spent in OleDB: the code layer added to the OleDB customer is very thin; 
  • True OOP architecture, to be used with any OleDB provider (allowing custom parameters or such), and even without OleDB (in the future, direct access to any DB client could be used); 
  • Could be safely used in a multi-threaded application/server (with one TOleDBConnection per thread); 
  • Allow parameter bindings of requests, with fast access to any parameter or column name (thanks to TDynArrayHashed);
  • Late binding of column values in Delphi code;
  • Direct JSON content creation, with no temporary data copy nor allocation; 
  • Designed to be used with our mORMot ORM, but could be used stand-alone (a full Delphi 7 client executable is just about 200 KB), or even in any existing Delphi application, thanks to a TQuery-like wrapper.

Continue reading...