Synopse Open Source

Mustache is a well-known logic-less template engine.
There is plenty of Open Source implementations around (including in JavaScript, which can be very convenient for AJAX applications on client side, for instance).
For mORMot, we created the first pure Delphi implementation of it, with a perfect integration with other bricks of the framework.

In this second part of this series of blog articles, we will introduce the Mustache syntax.
You can download this documentation as one single pdf file.

We have just released a set of slides introducing  ORM, SOA, REST, JSON, MVC, MVVM, SOLID, Mocks/Stubs, Domain-Driven Design concepts with Delphi,  and showing some sample code using our Open Source mORMot framework. You can follow the public link on Google Drive! This is a great opportunity to  […]

For several units of our framework, we allow late-binding of data values, using a variant and direct named access to properties:
- In SynCommons, we defined our TDocVariant custom variant type, able to store any JSON/BSON document-based content;
- In SynBigTable, we use the TSynTableVariantType custom variant type, as defined in SynCommons;
- In SynDB, we defined a TSQLDBRowVariantType, ready to access any column of a RDBMS data result set row;
- In mORMot, we allow access to TSQLTableRowVariantType column values.

It's a very convenient way of accessing result rows values. Code is still very readable, and safe at the same time.

For instance, we can write:

var V: variant;
 ...
  TDocVariant.New(V); // or slightly slower V := TDocVariant.New;
  V.name := 'John';
  V.year := 1972;
  // now V contains {"name":"john","year":1982}

This is just another implementation of KISS design in our framework.

Since Delphi XE2, some modifications were introduced to the official DispInvoke() RTL implementation:

1. A new varUStrArg kind of parameter has been defined, which will allow to transmit UnicodeString property values;
2. All text property values would be transmitted as BSTR / WideString / varOleStr variants to the invoked variant type;
3. All textual property names were normalized to be in UPPERCASE.

Those modifications are worth considering...
And we may have discovered two regressions: one about speed, and the other about an unexpected logic bug...

We just tested, benchmarked and validated Oracle MySQL, IBM DB2 and PostgreSQL support for our SynDB database classes and the mORMot's ORM core.
This article will also show all updated results, including our newly introduced multi-value INSERT statement generations, which speed up a lot BATCH insertion.

Stay tuned!

Purpose here is not to say that one library or database is better or faster than another, but publish a snapshot of mORMot persistence layer abilities, depending on each access library.

In this timing, we do not benchmark only the "pure" SQL/DB layer access (SynDB units), but the whole Client-Server ORM of our framework.

Process below includes all aspects of our ORM:

• Access via high level CRUD methods (Add/Update/Delete/Retrieve, either per-object or in BATCH mode);
• Read and write access of TSQLRecord instances, via optimized RTTI;
• JSON marshaling of all values (ready to be transmitted over a network);
• REST routing, with security, logging and statistic;
• Virtual cross-database layer using its SQLite3 kernel;
• SQL on-the-fly generation and translation (in virtual mode);
• Access to the database engines via several libraries or providers.

In those tests, we just bypassed the communication layer, since TSQLRestClient and TSQLRestServer are run in-process, in the same thread - as a TSQLRestServerDB instance. So you have here some raw performance testimony of our framework's ORM and RESTful core, and may expect good scaling abilities when running on high-end hardware, over a network.

On a recent notebook computer (Core i7 and SSD drive), depending on the back-end database interfaced, mORMot excels in speed, as will show the following benchmark:

• You can persist up to 570,000 objects per second, or retrieve 870,000 objects per second (for our pure Delphi in-memory engine);
• When data is retrieved from server or client 38, you can read more than 900,000 objects per second, whatever the database back-end is;
• With a high-performance database like Oracle, and our direct access classes, you can write 70,000 (via array binding) and read 160,000 objects per second, over a 100 MB network;
• When using alternate database access libraries (e.g. Zeos, or DB.pas based classes), speed is lower (even if comparable for DB2, MS SQL, PostgreSQL, MySQL) but still enough for most work, due to some optimizations in the mORMot code (e.g. caching of prepared statements, SQL multi-values insertion, direct export to/from JSON, SQlite3 virtual mode design, avoid most temporary memory allocation...).

Difficult to find a faster ORM, I suspect.

One of the main features you may miss when discovering NoSQL ("Not-Only SQL"?) databases, coming from a RDBMS background, is ACID.

ACID (Atomicity, Consistency, Isolation, Durability) is a set of properties that guarantee that database transactions are processed reliably. In the context of databases, a single logical operation on the data is called a transaction. For example, a transfer of funds from one bank account to another, even involving multiple changes such as debiting one account and crediting another, is a single transaction. (Wikipedia)

But are there any ACID NoSQL database?

Please ensure you read the Martin Fowler introduction about NoSQL databases.
And the corresponding video.

First of all, we can distinguish two types of NoSQL databases:

1. Aggregate-oriented databases;
2. Graph-oriented databases (e.g. Neo4J).

By design, most Graph-oriented databases are ACID!
This is a first good point.

Then, what about the other type?
In Aggregate-oriented databases, we can identify three sub-types:

• Document-based NoSQL databases (e.g. MongoDB, CouchDB);
• Key/Value NoSQL databases (e.g. Redis);
• Column family NoSQL databases (e.g. Cassandra).

With revision 1.18 of the framework, we just introduced two new custom types of variants:

• TDocVariant kind of variant;
• TBSONVariant kind of variant.

The second custom type (which handles MongoDB-specific extensions - like ObjectID or other specific types like dates or binary) will be presented later, when dealing with MongoDB support in mORMot, together with the BSON kind of content. BSON / MongoDB support is implemented in the SynMongoDB.pas unit.

We will now focus on TDocVariant itself, which is a generic container of JSON-like objects or arrays.
This custom variant type is implemented in SynCommons.pas unit, so is ready to be used everywhere in your code, even without any link to the mORMot ORM kernel, or MongoDB.

TDocVariant documents

TDocVariant implements a custom variant type which can be used to store any JSON/BSON document-based content, i.e. either:

• Name/value pairs, for object-oriented documents;
• An array of values (including nested documents), for array-oriented documents;
• Any combination of the two, by nesting TDocVariant instances.

Here are the main features of this custom variant type:

• DOM approach of any object or array documents;
• Perfect storage for dynamic value-objects content, with a schema-less approach (as you may be used to in scripting languages like Python or JavaScript);
• Allow nested documents, with no depth limitation but the available memory;
• Assignment can be either per-value (default, safest but slower when containing a lot of nested data), or per-reference (immediate reference-counted assignment);
• Very fast JSON serialization / un-serialization with support of MongoDB-like extended syntax;
• Access to properties in code, via late-binding (including almost no speed penalty due to our VCL hack as already detailed);
• Direct access to the internal variant names and values arrays from code, by trans-typing into a TDocVariantData record;
• Instance life-time is managed by the compiler (like any other variant type), without the need to use interfaces or explicit try..finally blocks;
• Optimized to use as little memory and CPU resource as possible (in contrast to most other libraries, it does not allocate one class instance per node, but rely on pre-allocated arrays);
• Opened to extension of any content storage - for instance, it will perfectly integrate with BSON serialization and custom MongoDB types (ObjectID, RegEx...), to be used in conjunction with MongoDB servers;
• Perfectly integrated with our Dynamic array wrapper and its JSON serialization as with the record serialization;
• Designed to work with our mORMot ORM: any TSQLRecord instance containing such variant custom types as published properties will be recognized by the ORM core, and work as expected with any database back-end (storing the content as JSON in a TEXT column);
• Designed to work with our mORMot SOA: any interface-based service is able to consume or publish such kind of content, as variant kind of parameters;
• Fully integrated with the Delphi IDE: any variant instance will be displayed as JSON in the IDE debugger, making it very convenient to work with.

To create instances of such variant, you can use some easy-to-remember functions:

• _Obj() _ObjFast() global functions to create a variant object document;
• _Arr() _ArrFast() global functions to create a variant array document;
• _Json() _JsonFast() _JsonFmt() _JsonFastFmt() global functions to create any variant object or array document from JSON, supplied either with standard or MongoDB-extended syntax.

In addition to regular HTTPS flow encryption, which is not easy to setup due to the needed certificates, mORMot proposes a proprietary encryption scheme. It is based on SHA-256 and AES-256/CTR algorithms, so is known to be secure.

You do not need to setup anything on the server or the client configuration, just run the TSQLHttpClient and TSQLHttpServer classes with the corresponding parameters.

One year ago, we already made a quick presentation of Domain-Driven Design, in the context of our mORMot framework.
After one year of real-world application of those patterns, it is now time to give more light to DDD.

Let's continue with part 3, which will define Domain-Driven Design patterns and principles - this will be the main article of the whole serie!

One year ago, we already made a quick presentation of Domain-Driven Design, in the context of our mORMot framework.
After one year of real-world application of those patterns, and a training made by a great French software designer named Jérémie Grodziski, it is now time to give more light to DDD.

Let's start with part 1, which will be a general introduction to Domain-Driven Design, trying to state how it may be interesting (or not) for your projects.