Synopse Open Source

You will find in the SQLite3\Sample\16 - Execute SQL via services folder of mORMot source code a Client-Server sample able to access any external database via JSON and HTTP.
It is a good demonstration of how to use an interface-based service between a client and a server.
It will also show how our SynDB classes have a quite abstract design, and are easy to work with, whatever database provider you need to use.

The corresponding service contract has been defined:

  TRemoteSQLEngine = (rseOleDB, rseODBC, rseOracle, rseSQlite3, rseJet, rseMSSQL);

  IRemoteSQL = interface(IInvokable)
    ['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}']
    procedure Connect(aEngine: TRemoteSQLEngine; const aServerName, aDatabaseName,
      aUserID, aPassWord: RawUTF8);
    function GetTableNames: TRawUTF8DynArray;
    function Execute(const aSQL: RawUTF8; aExpectResults, aExpanded: Boolean): RawJSON;
  end;

Purpose of this service is:
- To Connect() to external databases, given the parameters of a standard TSQLDBConnectionProperties. Create() constructor;
- Retrieve all table names of this external database as a list;
- Execute any SQL statement, returning the content as JSON array, ready to be consumed by AJAX applications (if aExpanded is true), or a Delphi client (e.g. via a TSQLTableJSON and the mORMotUI unit).

Of course, this service will be define as sicClientDriven mode, that is, the framework will be able to manage a client-driven TSQLDBProperties instance life time.

Benefit of this service is that no database connection is required on the client side: a regular HTTP connection is enough.
No need to install nor configure any database provider, and full SQL access to the remote databases.

Due to our optimized JSON serialization, it will probably be faster to work with such plain HTTP / JSON services, instead of a database connection through a VPN. In fact, database connections are made to work on a local network, and do not like high-latency connections, which are typical on the Internet.

Some major speed improvements have been made to our SynDB* units, and how they are used within the mORMot persistence layer.
It results in an amazing speed increase, in some cases.

Here are some of the optimizations how took place in the source code trunk:

• SQL statement client-side cache in ODBC and OleDB;
• SQL statement server-side cache for Oracle;
• When inserting individual data rows in an external table, the last inserted IDs are maintained in memory instead of executing "select max(id)" -  we added a new property EngineAddUseSelectMaxID to unset this optimization - we noted that this modification circumvented a known limitation of Firebird very efficiently.

Overall, I observed from x2 to x10 performance boost with simple Add() operations, using ODBC, OleDB and direct Oracle access, when compare to previous benchmarks (which were already impressive).
BATCH mode performance is less impacted, since it by-passed some of those limitations, but even in this operation mode, there is some benefits (especially with ODBC and OleDB).

Here are some results, directly generated by the supplied "15 - External DB performance" sample.

Due to the current implementation pattern of the TCollection type in Delphi, it was not possible to implement directly this kind of parameter.

In fact, the TCollection constructor is defined as such:

 constructor Create(ItemClass: TCollectionItemClass);

And, on the server side, we do not know which kind of TCollectionItemClass is to be passed. Therefore, the TServiceFactoryServer is unable to properly instantiate the object instances, supplying the expected item class.

The framework propose two potential solutions:

• You can let your collection class inherit from the new TInterfaceCollection type;
• You can call the TJSONSerializer.RegisterCollectionForJSON() method to register the collection type and its associated item class.

First solution has already been detailed in this blog.
We will now describe the second (and new) way.

Little mORMots have a very efficient transmission protocol, in their mountains...

Most mORMots are highly social and use loud whistles to communicate with one another, especially when alarmed.
As stated by Wikipedia

In a comment of the already quoted blog article about DataSnap performance issues, Tom asked this interesting question about mORMot, after I presented how we like to use http.sys kernel-mode server to achieve best performance possible over HTTP:

What about TCP communication, http is not the only Internet protocol. Protobuf, Thrift, MsgPack, BSON etc, is binary protocol and http i big overhead in some systems.
Did http.sys enable use of tcp/ip socket protocol?

With a refreshed hardware, and the latest code modifications of the framework code, I run again the benchmark sample.

The new PC has an Intel Core i7-2600 CPU, running on Windows Seven 64-bit, with anti-virus (Norton) fully enabled.
Oracle 11g database is remotely accessed over a corporate network, so latency and bandwidth is not optimal.
Still no SSD, but a standard 7200 rpm hard drive of 500 GB.

The results are impressive, when compared to the previous run using a Core 2 Duo CPU - mORMot's optimized code achieves amazing speed on this platform.
And I guess the 8MB of L3 cache of the Core i7 does wonders with our code.

In our documentation, and in all our code source, we avoid using the VCL DB.pas related units, and all the associated RAD components.

This is by design, since our experiment encouraged us to "think ORM, forget anything about RAD (and even SQL in most cases)" in mORMot.
And it introduced some nice border-side effect to Delphi users, e.g. that even a "Delphi Starter Edition" is able to use mORMot, have access to SQLite3, MS SQL or Oracle or any other DB, add interface-based RESTful JSON services over it, just for free...

But in the real world, you may need to upgrade some existing application, get rid of the BDE, or add a SOA layer over an existing business intelligence.
And mORMot is able to serve you well in those scenarios.
That's why we just added a first attempt to expose SynDB results and mORMOt TSQLTableJSON content into a TDataSet.

With a previous article, we introduced the concept of "Domain-Driven design" into our framework presentation.

It's now time to detail a bit more this very nice software architecture design, and how mORMot is able to achieve such an implementation pattern.

In a discussion with Henrick Hellström, in Embarcadero forums, I wrote some high-level information about mORMot.

It was clear to me that our little mORMot is now far away from a simple Client-Server solution.

The Henrick point was that with Real Thin Client (RTC), you are able to write any Client-Server solution, even a RESTful / JSON based one.

He is of course right, but it made clear to me all the work done in mORMot since its beginning.
From a Client-Server ORM, it is now a complete SOA framework, ready to serve Domain-Driven-Design solutions.

By default, our mORMot framework is able to serialize any class into a true JSON object.

All published properties will be written as JSON object members.

In some cases, it may be handy to have a custom serialization, for instance if you want to manage some third-party classes, or to adapt the serialization scheme to a particular purpose, at runtime.

As stated by this previous article, the default answer format is a valid JSON object.

In some cases, it may be useful to have a service operation (i.e. an interface method) returning any content, e.g. some plain TEXT, HTML or binary data (like a picture).