Synopse Open Source

After having tested and enhanced the external database speed (including BATCH mode), we are now able to benchmark all database engines available in mORMot.

In fact, the ORM part of our framework has several potential database backends, in addition to the default SQLite3 file-based engine.
Each engine may have its own purpose, according to the application expectation.

The following tables try to sum up all available possibilities, and give some benchmark (average rows/seconds for writing or read). 

In these tables:

• 'internal' means use of the internal SQLite3 engine;
• 'external' stands for an external access via SynDB;
• 'TObjectList' indicates a TSQLRestServerStaticInMemory instance either static (with no SQL support) or virtual (i.e. SQL featured via SQLite3 virtual table mechanism) which may persist the data on disk as JSON or compressed binary;
• 'trans' stands for Transaction, i.e. when the write process is nested within BeginTransaction / Commit calls;
• 'batch' mode will be described in this article;
• 'read one' states that one object is read per call (ORM generates a SELECT * FROM table WHERE ID=?);
• 'read all' is when all 5000 objects are read in a single call (i.e. running SELECT * FROM table);
• ACID is an acronym for "Atomicity Consistency Isolation Durability" properties, which guarantee that database transactions are processed reliably: for instance, in case of a power loss or hardware failure, the data will be saved on disk in a consistent way, with no potential loss of data.

In short: depending on the database you can persist up to 150,000 objects per second, or retrieve  240,000 objects per second.
With a high-performance database like Oracle and our direct access classes, you write 53,000 and read 72,000 objects per second.
Difficult to find a faster ORM, I suspect. :)

A common issue with Client-Server databases is the latency introduced for each query.

For example, suppose you have a requirement to first collect some information from your application’s users and then insert that information into a table in Oracle Database.

The first obvious option is to insert these multiple rows into the table through a loop in your program. This loop iterates over the data to be inserted and does what is known as a single-row insert , because the application sends one single row of data to the database at a time. Due to the network latency (typically around 1 ms over a corporate network), it would achieve not more than 500-600 requests per second to let the work done, since for each INSERT, a so-called round-trip occurs: a message is sent to Oracle, then a response is sent back to the client.

You have another option for inserting multiple rows of data into the table— that reduces the number of round-trips and improves application performance, database performance, and network resource use. Rather than having the application send a single row of data to the database at a time, it can use array binding to send the data in batches of rows. Therefore, you reduce a lot the number of round-trips to be processed, and enhance performance by a factor of about 100.

Our SynDB unit has been enhanced to introduce new TSQLDBStatement.BindArray() methods, introducing array binding for faster database batch modifications (only implemented in SynDBOracle by now - but MS SQL has a similar feature called OleDB bulk insert).
It is available from the ORM side or mORMot, when working with external tables, in BATCH mode.

Thanks to this enhancement, inserting records within Oracle comes from 400-500 rows per second to more than 50000 rows per second!
It was also a good opportunity to speed up the BATCH process globally, and to benchmark our Oracle back-end against existing external databases, i.e. SQLite3 (as file or in-memory), and Jet / MS Access / .mdb engine.

Note that this article scope is only about virtual tables linked to external databases (i.e. TSQLRecordExternal). Plain TSQLRecord classes will access directly to the SQLite3 engine or in-memory TList, so speed will be even higher than the below values.

Featuring benchmark source code and nice performance charts.

By default, both GDI+ anti-aliased drawing and PDF file generation using our libraries did lack of font-fallback implementation.

If some of the characters in a string are not supported in a requested font, they were drawn as square symbols on the GDI+ canvas or the PDF content.

The font-fallback mechanism implemented will use a third-party font (Arial Unicode MS by default, which is installed with Microsoft Office), for any glyph that are missing when rendering the content. Of course, for PDF generation, you are able to embed the font within the generated file (this is by the way mandatory for PDF/A-1 format).

Both of these new features are handled by the SQlite3Pages unit, i.e. for report anti-aliased drawing preview and PDF generation.

By the way, the TGDIPages class itself (the one used for report generation from code) now handles by default Unicode text, even with Delphi versions prior to 2009 (via the SynUnicode string type, which maps WideString before Delphi 2009, then UnicodeString starting with Delphi 2009). Does make sense with such a font-fallback mechanism.