Synopse Open Source

Do you know QDAC3 ?
This is an open source project, from China (with Chinese comments and exception errors messages, but the methods and variables are in English).
It is cross-platform, and told to be very fast about JSON process.

You can download this Open Source project code from http://sourceforge.net/projects/qdac3
And their blog - in Chinese - is at http://blog.qdac.cc/

So I included QDAC3 in our "25 - JSON performance" sample.
Numbers are talking, now.

How would be 2015 like for our little rodents?
Due to popular request of several users of mORMot, we identified and designed some feature requests dedicated to BigData process.

In fact, your data is the new value, especially if you propose SaaS (Software As A Service) hosting to your customers, with a farm of mORMot servers.
Recent Linux support for mORMot servers, together with the high performance and installation ease of our executable, open the gate to cheap cloud-based hosting.
As a consequence, a lot of information would certainly be gathered by your mORMot servers, and a single monolithic database is not an option any more.

For mORMot solutions hosted in cloud, a lot of data may be generated. The default SQLite3 storage engine may be less convenient, once it reaches some GB of file content. Backup becomes to be slow and inefficient, and hosting this oldest data in the main DB, probably stored on an expensive SSD, may be a lost of resource. Vertical scaling is limited by hardware and price factors.

This is were data sharding comes into scene.
Note that sharding is not replication/backup, nor clustering, nor just spreading. We are speaking about application-level data splitting, to ease maintenance and horizontal scalability of mORMot servers.

Data sharding could already be implemented with mORMot servers, thanks to TSQLRestStorage:

• Using TSQLRestStorageExternal: any table may have its own external SQL database engine, may be in its separated DB server;
• Using TSQLRestStorageMongoDB: any table may be stored on a MongoDB cluster, with its own sharding abilities;
• Using TSQLRestStorageRemote: each table may have its own remote ORM/REST server.

But when data stored in a single table tends to grow without limit, this feature is not enough.
Let's see how the close future of mORMot looks like.

We just finished our Be-Delphi 2014 sessions and drank our last beers, so here we are.

I published some slides for this great event.

Working with objects is pretty powerful, but requires to handle manually the created instances life time, via try .. finally blocks. Most of the time, the TSQLRecord life time would be very short: we allocate one instance on a local variable, then release it when it goes out of scope.

If we take again the TSQLBaby sample, we may write:

function NewMaleBaby(Client: TSQLRest; const Name,Address: RawUTF8): TID;
var Baby: TSQLBaby;   // store a record
begin
  Baby := TSQLBaby.Create;
  try
    Baby.Name := Name;
    Baby.Address := Address;
    Baby.BirthDate := Date;
    Baby.Sex := sMale;
    result := Client.Add(Baby);
  finally
    Baby.Free;
  end;
end;

To ease this pretty usual pattern, the framework offers some kind of automatic memory management at TSQLRecord level:

function NewMaleBaby(Client: TSQLRest; const Name,Address: RawUTF8): TID;
var Baby: TSQLBaby;   // store a record
begin
  TSQLBaby.AutoFree(Baby);  // no try..finally needed!
  Baby.Name := Name;
  Baby.Address := Address;
  Baby.BirthDate := Date;
  Baby.Sex := sMale;
  result := Client.Add(Baby);
end; // local Baby instance will be released here

I just wanted to share a great article by Martin Fowler, about Micro Services. IMHO such "Micro Services" are the proper way of defining a SOA project, following SOLID principles. If we follow the "Single Responsibility" principle, we will define small uncoupled services, which  […]

It is pretty much possible that you would have to maintain and evolve a legacy project, based on an existing database, with a lot of already written SQL statements - see Legacy code and existing projects.

For instance, you would like to use mORMot for new features, and/or add mobile or HTML clients - see Cross-Platform clients.
In this case, the ORM advanced features - like ORM Cache or BATCH process, see BATCH sequences for adding/updating/deleting records - may conflict with the legacy code, for the tables which may have to be shared.
Here are some guidelines when working on such a project.

To be exhaustive about your question, we need to consider each ORM CRUD operation.
We may have to divide them in three kinds: read queries, insertions, and modifications of existing data.

You certainly noticed that WebSocket is the current trendy flavor for any modern web framework.
But does it scale? Would it replace HTTP/REST?
There is a feature request ticket about them for mORMot, so here are some thoughts - matter of debate, of course!
I started all this by answering a StackOverflow question, in which the actual answers were not accurate enough, to my opinion.

From my point of view, Websocket - as a protocol - is some kind of monster.

You start a HTTP stateless connection, then switch to WebSocket mode which releases the TCP/IP dual-direction layer, then you may switch later on back to HTTP...
It reminds me some kind of monstrosity, just like encapsulating everything over HTTP, using XML messages... Just to bypass the security barriers... Just breaking the OSI layered model...
It reminds me the fact that our mobile phone data providers do not use broadcasting for streaming audio and video, but regular Internet HTTP servers, so the mobile phone data bandwidth is just wasted when a sport event occurs: every single smart phone has its own connection to the server, and the same video is transmitted in parallel, saturating the single communication channel... Smart phones are not so smart, aren't they?

WebSocket sounds like a clever way to circumvent a limitation...
But why not use a dedicated layer?
I hope HTTP 2.0 would allow pushing information from the server, as part of the standard... and in one decade, we probably will see WebSocket as a deprecated technology.
You have been warned. Do not invest too much in WebSockets..

OK. Back to our existential questions...
First of all, does the WebSocket protocol scale?
Today, any modern single server is able to server millions of clients at once.
Its HTTP server software has just to be is Event-Driven (IOCP) oriented (we are not in the old Apache's one connection = one thread/process equation any more).
Even the HTTP server built in Windows (http.sys - which is used in mORMot) is IOCP oriented and very efficient (running in kernel mode).
From this point of view, there won't be a lot of difference at scaling between WebSocket and a regular HTTP connection. One TCP/IP connection uses a little resource (much less than a thread), and modern OS are optimized for handling a lot of concurrent connections: WebSocket and HTTP are just OSI 7 application layer protocols, inheriting from this TCP/IP specifications.

But, from experiment, I've seen two main problems with WebSocket:

1. It does not support CDN;
2. It has potential security issues.

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

• Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
• FreePascal Compiler 2.7.1;
• Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

• Units and platforms;
• Generating the client code wrappers;
• Delphi / FreePascal clients;
• Smart Mobile Studio clients.

Any feedback is welcome in our forum, as usual!

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

• Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
• FreePascal Compiler 2.7.1;
• Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

• Units and platforms;
• Generating the client code wrappers;
• Delphi / FreePascal clients;
• Smart Mobile Studio clients.

Any feedback is welcome in our forum, as usual!