We just committed a deep refactoring of the SynSQlite3Static.pas unit - and all units using static linking for FPC. It also includes a new encryption format for SQlite3, using AES, so much more secure than the previous one. This is a breaking change, so worth a blog article! Now all static .o .a […]
In the last weeks/months, we worked a lot with FPC.
Delphi is still our main IDE, due to its better debugging experience under Windows, but we target to have premium support of FPC, on all platforms, especially Linux.
The new Delphi Linux compiler is out of scope, since it is heavily priced,
its performance is not so good, and ARC broke memory management so would need a
deep review/rewrite of our source code, which we can't afford - since we have
FPC which is, from our
opinion, a much better compiler for Linux.
Of course, you can create clients for Delphi Linux and FMX, as usual, using the cross-platform client parts of mORMot. But for server side, this compiler is not supported, and will probably never be.
Everyone knows about the pascal random()
It returns some numbers, using a linear congruential generator, with a multiplier of 134775813, in its Delphi implementation.
It is fast, but not really secure. Output is very predictable, especially if you forgot to execute the RandSeed() procedure.
In real world scenarios, safety always requires random numbers, e.g. for
The less predictable, the better.
We just included a Cryptographically Secure Pseudo-Random Number Generator (CSPRNG) into our SynCrypto.pas unit.
TAESPRNG class would use real system entropy to generate
a sequence of pseudorandom bytes, using AES-256, so returning highly
If you compare with existing client/server SOA solutions (in Delphi, Java,
C# or even in Go or other frameworks), mORMot's
callback mechanism sounds pretty unique and easy to work with.
Most Events Oriented solutions do use a set of dedicated
messages to propagate the events, with a centralized Message
Bus (like MSMQ or
JMS), or a
P2P/decentralized approach (see e.g. ZeroMQ or NanoMsg). In practice, you are expected to
class per message, the
class fields being
the message values. You would define e.g. one
class to notify a
successful process, and another
class to notify an error. SOA
services would eventually tend to be defined by a huge number of individual
classes, with the temptation of re-using existing classes in several
interface-based approach allows to gather all events:
- In a single
interfacetype per notification, i.e. probably per service operation;
- With one method per event;
- Using method parameters defining the event values.
Since asynchronous notifications are needed most of the time, method
parameters would be one-way, i.e. defined only
const - in such case, an evolved algorithm would
transparently gather those outgoing messages, to enhance scalability when
processing such asynchronous events. Blocking request may also be defined
var/out, as we will see below, inWorkflow
Behind the scene, the framework would still transmit raw messages over IP
sockets (currently over a
WebSockets connection), like other systems, but events notification would
benefit from using interfaces, on both server and client sides.
We will now see how...
We just noticed a nice feedback from a mORMot user. Vojko Cendak commented the well-known DataSnap analysis based on Speed & Stability tests blog article written by Roberto some months years (!) ago. It is not meant to be the final word, perhaps there was some tuning possible for RTC (which is […]
Back in 2013, I found out an implementation weakness in the implementation
of ARC weak references in the RTL.
A giant lock was freezing all threads and cores, so would decrease a lot the performance abilities of any ARC application, especially in multi thread.
I just investigated that things are now better.
Among all its means of transmission, our mORMot framework
WebSockets, allowing bidirectional communications, and interface-based
callbacks for real time notification of SOA events.
After several months of use in production, we identified some needed changes for this just emerged feature.
today a breaking change of the data layout used for our proprietary
WebSockets binary protocol.
From our tests, it would increase the performance and decrease the resource consumption, especially in case of high number of messages.
A lot of our code, and probably yours, is highly relying on text process. In our mORMot framework, most of its features use JSON text, encoded as UTF-8. Profiling shows that a lot of time is spent computing the end of a text buffer, or comparing text content. You may know that In its SSE4.2 feature […]
2015-06-21. Pascal Programming
Almost every time I'm debugging some core part of our framework, I like to see the generated asm, and trying to optimize the pascal code for better speed - when it is worth it, of course! I just made a nice observation, when comparing the assembler generated by Delphi to FPC's output. Imagine you […]
We introduced DDD concepts some time ago, in a series of articles in this blog. At that time, we proposed a simple way of using mORMot types to implement DDD in your applications. But all Domain Entitities being tied to the framework TSQLRecord class did appear as a limitation, breaking the […]
2015-05-08. Pascal Programming
There was some discussion about the new
TStringHelper feature introduced in latest versions of
I was told to be some kind of archaic guy, not able to see the benefit of this.
Reducing opinions to a conservative/progressive approach - another famous 10 kinds of coders - is very reductive.
Of course, this was IMHO unfair and my point was that I have the feeling
that some decisions about the Delphi language and RTL are inadequate.
Some changes are welcome. I enjoy the introduction of generics - even if it is was painful, and even buggy (do not use TList<T> with managed record types in XE8!).
But some upcoming changes about the
string policy - breaking
everything just because we want to align with mainstream C# or Java habits -
are just non sense to me.
I really think that Embarcadero deciders like to shoot their own foot.
Or - certainly worse - our own feet!
I will post here some part of the discussion...
So that we may be able to share our ideas.
We have just included some optimized x64 assembler to our Open
Source SynCrypto.pas unit
so that SHA-256 hashing will perform at best speed.
It is an adaptation from tuned Intel's assembly macros, which makes use of the SSE4 instruction set, if available.
2015-02-16. Open Source
And in fact, it is fast, and sounds pretty great!
Here are some numbers, compared with SuperObject, standard DBXJson, dwsJSON, QDAC and mORMot.
Please refer to previous benchmark articles about those libraries. We will now focus on JsonDataObjects.
Do you know QDAC3 ?
This is an open source project, from China (with Chinese comments and exception errors messages, but the methods and variables are in English).
It is cross-platform, and told to be very fast about JSON process.
So I included QDAC3 in
our "25 - JSON performance" sample.
Numbers are talking, now.
2015-01-15. Open Source
Today, we committed a new patch to enable AES-NI hardware acceleration to our SynCrypto.pas unit. Intel® AES-NI is a new encryption instruction set that improves on the Advanced Encryption Standard (AES) algorithm and accelerates the encryption of data on newer processors. Of course, all this is […]
You can use the FreePascal Compiler (FPC) to compile the mORMot framework source code, targetting Windows and Linux.
Linux is a premium target for cheap and efficient server Hosting. Since mORMot has no dependency, installing a new mORMot server is as easy as copying its executable on a blank Linux host, then run it. No need to install any framework nor runtime. You could even use diverse operating systems (several Linux or Windows Server versions) in your mORMot servers farm, with minimal system requirements, and updates.
We will now see how to write your software with Linux-compiling in mind, and also give some notes about how to install a Linux Virtual Machine with Lazarus on your Windows computer, compiling both FPC and Lazarus from their SVN latest sources!
How would be 2015 like for our little rodents?
Due to popular request of several users of mORMot, we identified and designed some feature requests dedicated to BigData process.
In fact, your data is the new value, especially if you propose
SaaS (Software As A Service)
hosting to your customers, with a farm of
Recent Linux support for mORMot servers, together with the high performance and installation ease of our executable, open the gate to cheap cloud-based hosting.
As a consequence, a lot of information would certainly be gathered by your mORMot servers, and a single monolithic database is not an option any more.
For mORMot solutions hosted in cloud, a lot of data may be generated. The default SQLite3 storage engine may be less convenient, once it reaches some GB of file content. Backup becomes to be slow and inefficient, and hosting this oldest data in the main DB, probably stored on an expensive SSD, may be a lost of resource. Vertical scaling is limited by hardware and price factors.
This is were data sharding comes into scene.
Note that sharding is not replication/backup, nor clustering, nor just spreading. We are speaking about application-level data splitting, to ease maintenance and horizontal scalability of mORMot servers.
Data sharding could already be implemented with mORMot servers,
TSQLRestStorageExternal: any table may have its own external SQL database engine, may be in its separated DB server;
TSQLRestStorageMongoDB: any table may be stored on a MongoDB cluster, with its own sharding abilities;
TSQLRestStorageRemote: each table may have its own remote ORM/REST server.
But when data stored in a single table tends to grow without limit, this
feature is not enough.
Let's see how the close future of mORMot looks like.
For mORMot, we developed a fully feature direct access layer to any RDBMS, implemented in the SynDB.pas unit.
You can use those
SynDB classes to execute any SQL statement,
without any link to the framework ORM.
At reading, the resulting performance is much higher than using the standard
TDataSet component, which is in fact a true performance
It has genuine features, like column access via late-binding, an innovative
ISQLDBRows interface, and ability to directly access
the low-level binary buffers of the database clients.
We just added a nice feature to those classes: the ability to access
remotely, via plain HTTP, to any
SynDB supported database!
We almost finished implementing a long-standing feature request,
introducing MVC / MVVM for Web Applications (like RubyOnRails) in mORMot.
This is a huge step forward, opening new perspectives not only to our framework, but for the Delphi community.
In respect to the existing MVC frameworks for Delphi, our solution is closer to Delphi On Rails (by the convention-over-configuration pattern) than the Delphi MVC Framework or XMM.
The mORMot point of view is unique, and quite powerful, since it is integrated with other parts of our framework, as its ORM/ODM or interface-based services.
Of course, this is a work in progress, so you are welcome to put your feedback, patches or new features!
We will now explain how to build a MVC/MVVM web application using
mORMot, starting from the "30
- MVC Server" sample.
First of all, check the source in our GitHub repository: two
.pas files, and
a set of minimalist Mustache
This little web application publishes a simple BLOG, not fully finished yet
(this is a Sample, remember!).
But you can still execute it in your desktop browser, or any mobile device (thanks to a simple Bootstrap-based responsive design), and see the articles list, view one article and its comments, view the author information, log in and out.
This sample is implemented as such:
||Mustache template engine in the Views sub-folder|
||Defined as one
For the sake of the simplicity, the sample will create some fake data in its own local SQlite3 database, the first time it is executed.
For pre-Unicode versions of Delphi, the unique way of having UTF-16 native
type is to use the
This type, under Windows, matched the BSTR managed type, as used by OLE and COM components.
WideString implementation calls directly the
corresponding Windows API, and do not use the main Delphi heap manager.
Even if since Vista, this API did have a huge speed-up, it is still in practice much slower than the regular
string type. Problems is not about
UTF-16 encoding, but about the memory allocation, which is shared among
processes, using the Windows global heap, and is much slower than our beloved
Newer versions of Delphi (since Delphi 2009) feature a refactored
UnicodeString type, which relies on
FastMM4 and not the Windows API, and is much faster than
Within our mORMot framework,
we by-passed this limitation by using our
RawUTF8 type, which is
UTF-8 encoded, so as Unicode ready as the new
and pretty fast.
In a recent internal project, we had to use a lot of
WideString instances, to support UTF-16 encoding in Delphi
7/2007, involving a lot of text.
It sounded to be very slow, so we had to do something!
This is where our new SynFastWideString unit comes in.
Purpose of this unit is to patch the
system.pas unit for older
versions of Delphi, so that
WideString memory allocation would use
FastMM4 instead of the slow BSTR Windows API.
It will speed up the
WideString process a lot, especially when a
lot of content is allocated, since FastMM4 is much more aggressive
than Windows' global heap and the BSTR slow API. It could be more than 50 times
faster, especially when releasing the used memory.
WideString implementation pattern does NOT feature Copy-On-Write, so is still
slower than the
string UnicodeString type as implemented since
Delphi 2009. This is the reason why this unit won't do anything on
Unicode versions of the compiler, since the new string type is to be