To content | To menu | To search

Tag - mORMot

Entries feed

2015, Monday September 28

mORMot show case: Illustrated Spare Parts Catalog

Illustrated Spare Parts Catalog is, as its name suggests, a software for creating and publishing spare parts catalogs.
It uses mORMot for client-server communication and ORM, and SynPdf for the reporting.

Sounds like a powerful solution.
It is also a testimony that you could use big databases (20 GB of blobs) with a SQlite3 engine, and access them via REST using mORMot, without the hassle of setting up a regular RDBMS.

If you (or Google Translate) know a little of Russian, it is worth reading this previous blog article, about how the software author interacted with our Open Source project.
In fact, Chaa did provide a lot of feedback, patches and new features (like direct authentication via Active Directory).
Open Source could be great!

Thanks Chaa for the feedback, and interest!

2015, Friday September 25

ORM TNullable* fields for NULL storage

In Delphi code, NULLable types do not exist as such. There is no native int? type, as in C#.
But at SQL and JSON levels, the NULL value does exist and should be converted as expected by the ORM.

In SQLite3 itself, NULL is handled as stated in (see e.g. IS and IS NOT operators).
It is worth noting that NULL handling is not consistent among all existing database engines, e.g. when you are comparing NULL with non NULL values... so we recommend using it with care in any database statements, or only with proper (unit) testing, when you switch from one database engine to another.

By default, in the mORMot ORM/SQL code, NULL will appear only in case of a BLOB storage with a size of 0 bytes.
Otherwise, you should not see it as a value, in most kinds of ORM properties.

Null-oriented value types have been implemented in our framework, since the object pascal language does not allow defining a nullable type (yet)

We choose to store those values as variant, with a set of TNullable dedicated types, as defined in mORMot.pas:

  TNullableInteger = type variant;
  TNullableBoolean = type variant;
  TNullableFloat = type variant;
  TNullableCurrency = type variant;
  TNullableDateTime = type variant;
  TNullableTimeLog = type variant;
  TNullableUTF8Text = type variant;

Continue reading...

2015, Wednesday September 16

Feedback from the Wild

We just noticed a nice feedback from a mORMot user.

Vojko Cendak commented the well-known DataSnap analysis based on Speed & Stability tests blog article written by Roberto some months years (!) ago.
It is not meant to be the final word, perhaps there was some tuning possible for RTC (which is told to be very reliable), but it is worth a look:

We used 3 products: RO, RTC and Mormot.. I won’t speak about RO ( slow and heavy ). We tried RTC but was too very slow and CPU consuming in getting lots of 1000 .. 5000 dynamically fetching OPC tags (let’s say list of small objects) – at least once per second (one client). I mean Mormot is FAST and we’re glad to be so. We use Mormot in actual productions 24/7 on several sites: servers don’t even blink on client requests and run smoothly and reliably.

Thanks for the kind words!
We have a lot of feedback, around the world, from users of our little Open Source project, very happy with its abilities.
We try to make it always better! Open Source, and Delphi as a platform, do rock!

2015, Monday August 31

Delphi 10 = DX Seattle is out, mORMot supports it

We expected Delphi XE9, and now we have Rad Studio 10 Seattle, with Delphi renamed as Delphi 10 Seattle, or simply DX.

No big news for the Delphi compiler itself (we are still waiting for Linux server support), but a lot of FireMonkey updates, Windows 10 compatibility enhancements, enhancements to JSON (better performance using a SAX approach), and NoSQL/MongoDB support in FireDAC.
The documentation is rather sparse for the new features, but it goes into the right direction (we support MongoDB since a long time, in our ORM/ODM).
See what's new in details.

Of course, our Open Source mORMot framework supports this version.
Feedback is welcome, as usual!
Enjoy the new DX IDE!

2015, Sunday August 23

"SQL and NoSQL", not "SQL vs NoSQL"

You know certainly that our mORMot Open Source framework is an ORM, i.e. mapping objects to a relational / SQL database (Object Relational Mapping).
You may have followed also that it is able to connect to a NoSQL database, like MongoDB, and that the objects are then mapped via an ODM (Object Document Mapping) - the original SQL SELECT are even translated on the fly to MongoDB queries.

But thanks to mORMot, it is not "SQL vs NoSQL" - but "SQL and NoSQL".
You are not required to make an exclusive choice.
You can share best of both worlds, depending on your application needs.

In fact, the framework is able to add NoSQL features to a regular relational / SQL database, by storing JSON documents in TEXT columns.

In your end-user code, you just define a variant field in the ORM, and store a TDocVariant document within.
We also added some dedicated functions at SQL level, so that SQLite3 could be used as embedded fast engine, and provide advanced WHERE clauses on this JSON content.

Continue reading...

2015, Saturday August 15

Breaking Change in mORMot WebSockets binary protocol

Among all its means of transmission, our mORMot framework features WebSockets, allowing bidirectional communications, and interface-based callbacks for real time notification of SOA events.
After several months of use in production, we identified some needed changes for this just emerged feature.

We committed today a breaking change of the data layout used for our proprietary WebSockets binary protocol.
From our tests, it would increase the performance and decrease the resource consumption, especially in case of high number of messages.

Continue reading...

2015, Tuesday July 14

New blog about mORMot

An enthusiastic mORMot user, named willo in the forum, just started a blog about his experiments with our framework.

The information there is clear, simple, and right to the point.
If you are a little lost in our huge documentation, it is a good place to start!

Continue reading...

2015, Tuesday June 30

Faster String process using SSE 4.2 Text Processing Instructions STTNI

A lot of our code, and probably yours, is highly relying on text process.
In our mORMot framework, most of its features use JSON text, encoded as UTF-8.
Profiling shows that a lot of time is spent computing the end of a text buffer, or comparing text content.

You may know that In its SSE4.2 feature set, Intel added STTNI (String and Text New Instructions) opcodes.
They are several new instructions that perform character searches and comparison on two operands of 16 bytes at a time.

I've just committed optimized version of StrComp() and StrLen(), also used for our TDynArrayHashed wrapper.
The patch works from Delphi 5 up to XE8, and with FPC - unknown SSE4.2 opcodes have been entered as hexadecimal bytes, for compatibility with the last century compilers!
The resulting speed up may be worth it!

Next logical step would be to use those instruction in the JSON process itself.
It may speed up the parsing speed of our core functions (which is already very optimized, but written in a classical one-char-at-a-time reading).
Main benefit would be to read the incoming UTF-8 text buffer by blocks of 16 bytes, and performing several characters comparison in a few CPU cycles, with no branching.
Also JSON writing would benefit for it, since escaping could be speed up thanks to STTNI instructions.

Any feedback is welcome, as usual!

2015, Tuesday June 16

Handling Cross-Platform Time Zones

One common problem when handling dates and times, is that time is shown and entered as local, whereas the computer should better use non-geographic information - especially on a Client-Server architecture, where both ends may not be on the same physical region.

A time zone is a region that observes a uniform standard time for legal, commercial, and social purposes.
Time zones tend to follow the boundaries of countries and their subdivisions because it is convenient for areas in close commercial or other communication to keep the same time.
Most of the time zones on land are offset from Coordinated Universal Time (UTC) by a whole number of hours, or minutes.
Even worse, some countries use daylight saving time for part of the year, typically by changing clocks by an hour, twice every year.

The main rule is that any date and time stored should be stored in UTC, or with an explicit Zone identifier (i.e. an explicit offset to the UTC value).
Our framework expects this behavior: every date/time value stored and handled by the ORM, SOA, or any other part of it, is expected to be UTC-encoded.
At presentation layer (e.g. the User Interface), conversion to/from local times should take place, so that the end-user is provided with friendly clock-wall compatible timing.

As you may guess, handling time zones is a complex task, which should be managed by the Operating System itself.
Since this cultural material is constantly involving, it is updated as part of the OS.

In practice, current local time could be converted from UTC from the current system-wide time zone. One of the only parameters you have to set when installing an Operating System is to pickup the keyboard layout... and the current time zone to be used. But in a client-server environment, you may have to manage several time zones on the server side: so you can't rely on this global setting.

One sad - but predictable - news is that there is no common way of encoding time zone information.
Under Windows, the registry contains a list of time zones, and the associated time bias data. Most POSIX systems (including Linux and Mac OSX) do rely on the IANA database, also called tzdata - you may have noticed that this particular package is often updated with your system.
Both zone identifiers do not map, so our framework needed something to be shared on all systems.

Continue reading...

2015, Saturday June 6

GetIt "Spirit" Concerns

I'm confused by the GetIt Submitting official blog page.
Reminds me the darker ages of Delphi licensing change of XE3.

GetIt is the new XE8 package manager for RAD Studio. Information about how to submit your libraries to GetIt has just been made available by Embarcadero. The idea behind GetIt is really to make is easier and faster to discover, install, and keep updated some of the best open source libraries for Delphi and C++Builder.

When you look at the approval conditions, it sounds like if mORMot would not find its way in this package manager:

Replacing key capabilities that are part of the core platforms definitions such as Client/Server FireDAC pack or the DataSnap/EMS Enterprise middleware, would be less likely to be accepted.
The different SKUs are meant for different types of developers, and multi-tier capabilities with strong client/server RDBMS integration require an Enterprise edition license.
We will bias acceptance toward GetIt libraries that respect the spirit of our licensing and editions, not just use the letter of the license and the technical boundaries. If you are unsure about your submission please check with us first. 

What is this "spirit of our licensing and editions"?
Why is it not part of the official license terms?
Where does this assumption comes from?
Would the licensing conditions change in the close future, as with the XE3 "episode"?
Would Marco's interpretation become the new rule?

It clearly reminds me the XE3 time where there was an attempt from Embarcadero to modify their licence terms, so that third party vendors or Open Source libraries would not be allowed to create multi-tier frameworks with Delphi!
Is it true that "strong client/server RDBMS integration require an Enterprise edition" ?
Last time I checked the licence terms, it was not stated.
Why on earth would we have to pay for the Entreprise edition, if the Professionnal edition is all that you need?

Still the same "closed" spirit.
It is like if they know their own n-Tier solution weaknesses, so they try to avoid any other possibility, but to use their own.
They clearly do not understand the benefit and dynamic of Open Source.

I guess our little mORMot falls directly into this "unwelcomed" category.
I did not make the submission yet. But should I?
Perhaps sub-part of the framework may find its way in: SynPdf, SynCrypto, SynGdiPlus, SynCommons...
But the main ORM/SOA/REST/MVC/StubMock features would certainly be rejected.

Our Open Source project is sometimes preferred to DataSnap/EMS (or even FireDAC), not only for licence cost, but also about features, documentation, stability, compatibility with older versions of Delphi, performance, and Linux compatibility.
I have encountered several companies which are still using Delphi because of mORMot: if they did not have found it, they would have moved to C# or Java, just to be able to use a full SOA/MVC stack, which was not available, even in the "Enterprise" version of Delphi.

Story repeats itself.
I just wanted to ensure that the licensing terms would not change in that direction.
I - as many Delphi users - would not let this GetIt "spirit" become the new rule.
We have to react, as we did for XE3, otherwise we may all suffer!
IMHO Embacardero should better focus on the compiler and IDE, not cutting the branch on which they are seated...

What do you think? Comments and feebacks are welcome!

2015, Monday June 1

Updated Slides about ORM SOA MVC SOLID DDD

One year ago, we published a set of slides about the main concepts implemented by our framework.
Mainly about ORM (and ODM), NoSQL, JSON, SOA, MVC (and MVVM), SOLID, DDD, CQRS and some patterns like Stubs, Mocks, Factory, Repository, Unit-Of-Work.
Worth a look, if you want to find out the benefits of the latest software development techniques.
They try to open the landscape of any Delphi developer (probably with a mostly RAD and OOP background) to some new areas.

I just updated the slides from our public GoogleDrive folder.
They now reflect the latest state of the framework (e.g. ORM real-time synchronization, asynchronous callbacks, DDD CQRS services...).
They have also been polished after several public presentations, since I used them as base for trainings I made for some European companies.

If you want to go further, or have some more enlightenment, ensure you took a look at our framework Documentation, which would detail all those patterns, and how mORMot may help implementing them for your projects!

Feedback is welcome on our forum, as usual!

2015, Monday May 18

CQRS Persistence Service of any DDD object with mORMot

We introduced DDD concepts some time ago, in a series of articles in this blog.
At that time, we proposed a simple way of using mORMot types to implement DDD in your applications.
But all Domain Entitities being tied to the framework TSQLRecord class did appear as a limitation, breaking the Persistence Ignorance principle, since it couples the DDD objects to the framework implementation details.

We introduced a new mORMotDDD.pas unit, which is able to easily create CQRS Persistence services for any plain Delphi class (the famous PODOs - Plain Old Delphi Objects).
No need to inherit from TSQLRecord, or pollute your class definition with attributes!

For instance, a TUser class may be persisted via such a service:

  IDomUserCommand = interface(IDomUserQuery)
    function Add(const aAggregate: TUser): TCQRSResult;
    function Update(const aUpdatedAggregate: TUser): TCQRSResult;
    function Delete: TCQRSResult;
    function Commit: TCQRSResult;

Here, the write operations are defined in a IDomUserCommand service, which is separated (but inherits) from IDomUserQuery, which is used for read operations.
Separating reads and writes is a powerful pattern also known as CQRS, i.e. Command Query Responsibility Segregation, which we followed when defining our persistence services.
The framework make it pretty easy to create such services for storing any kind of class type in any SQL or NoSQL engine, with almost no code to write.
Last but not least, using such interface-based services for data persistence will allow to stub or mock the data access layer, making unit testing straightforward: you would not fear to write TDD code any more!

Please refer to our updated documentation for this unique and powerful feature.
You may take a look at the corresponding dddDomUserTypes.pas, dddDomUserCQRS.pas, and dddInfraRepoUser.pas units, detailed as sample reference.
Feedback is welcome in our forum, as usual!

2015, Thursday May 14

Using TSynLog with a lot of threads? PYou should better upgrade your source

We identified and fixed today several issues which may affect applications creating a lot of threads (i.e. not using a thread pool).
The symptom was an unexpected access violation, when you reach a multiple of 256 threads count.

You should better upgrade to at least revision 1.18.1351 if your application creates a lot of custom threads.
Note that a regular mORMot server, using http.sys and its thread pool won't be affected by this issue.

We also refactored the whole threading process in TSynLog, so that:

  • The thread numbers in the .log file would be re-used;
  • The memory resources associated with each thread would be released and re-used;
  • It would handle an infinite number of thread creation (previous implementation did have a hard limit of 32,768 created threads during a process lifetime);
  • Exception would never be intercepted during logging process;
  • Made code stronger, and re-entrant in case of concurrent TSynLog classes in the same process;
  • Of course, this works also under Linux, via FPC or (Cross)Kylix.

See our Source Code Repository TimeLine.

Feedback is welcome on our forum, as usual!

2015, Sunday April 12

Why Transmitting Exceptions in SOA services is not a good idea

Usually, in Delphi application (like in most high-level languages), errors are handled via exceptions. By default, any Exception raised on the server side, within an interface-based service method, will be intercepted, and transmitted as an error to the client side, then a safe but somewhat obfuscated EInterfaceFactoryException will be raised on the client side, containing additional information serialized as JSON.

You may wonder why exceptions are not transmitted and raised directly on the client side, with our mORMot framework interface-based services, as if they were executed locally.

We will now detail some arguments, and patterns to be followed.

Continue reading...

2015, Tuesday March 31

ORM Master/Slave Replication

As stated during TSQLRecord fields definition, the ORM is able to maintain a revision number for any TSQLRecord table, so that it the table may be easily synchronized remotely by another TSQLRestServer instance.
If you define a TRecordVersion published property, the ORM core will fill this field just before any write with a monotonically increasing revision number, and will take care of any deletion, so that those modifications may be replayed later on any other database.

This synchronization will work as a strict master/slave replication scheme, as a one-way on demand refresh of a replicated table.
Each write operation on the master database on a given table may be easily reflected on one or several slave databases, with almost no speed nor storage size penalty.

Continue reading...

2015, Tuesday March 17

Framework Documentation Enhanced By Links

The mORMot framework documentation, in its HTML online form, has been enhanced to include links to almost of the code symbols.

In fact, the latest version of our SynProject tool will search for code symbols (types, methods, constants, functions): 

Some minor cosmetic changes did also occur, especially in the API Reference.
We hope it would help you discover and work with out little mORMot!

2015, Sunday March 1

ShowCase: mORMot with FPC on Android

I just received a mail from Alfred (aka Alf in the source code), which did a lot of work to let our little mORMot compiles and run with FPC, especially under Linux, and also with an ARM processor.

Hello Arnaud,
A nice surprise ...
Sample 2 native on Android !!!!
See picture.
 Works 100% !!!
 Greetings, Alfred.

This was compiled with FPC, and LCL for Android... 

This is not just the cross-platform client library, but an Embedded SQLite3 ORM stand-alone application running on the Android device!

You can download the .apk directly from
Note how the application compiled with laz4android, using native Android components, if much smaller than a FMX's.
The apk has two huge JPEG pictures, but the binary itself is only 800 KB...

Great, isn't it?
Comments are welcome on our forum, as usual!

2015, Thursday January 15

AES-NI enabled for SynCrypto

Today, we committed a new patch to enable AES-NI hardware acceleration to our SynCrypto.pas unit.

Intel® AES-NI is a new encryption instruction set that improves on the Advanced Encryption Standard (AES) algorithm and accelerates the encryption of data on newer processors.

Of course, all this is available in the Delphi unit, from Delphi 6 to XE7: no external dll nor OS update is needed.
And it will work also on Linux, so could help encrypting the mORMot transmission with no power loss.

You have nothing to do: just upgrade your mORMot source code, then AES-NI instructions will be used, if the CPU offers it.
We have seen performance boost of more than 5x, depending on the size of the data to be encrypted.


2015, Saturday January 10

mORMot under Linux thanks to FPC

You can use the FreePascal Compiler (FPC) to compile the mORMot framework source code, targetting Windows and Linux.

Linux is a premium target for cheap and efficient server Hosting. Since mORMot has no dependency, installing a new mORMot server is as easy as copying its executable on a blank Linux host, then run it. No need to install any framework nor runtime. You could even use diverse operating systems (several Linux or Windows Server versions) in your mORMot servers farm, with minimal system requirements, and updates.

We will now see how to write your software with Linux-compiling in mind, and also give some notes about how to install a Linux Virtual Machine with Lazarus on your Windows computer, compiling both FPC and Lazarus from their SVN latest sources!

Continue reading...

2014, Wednesday December 31

2015: the future of mORMot is BigData

How would be 2015 like for our little rodents?
Due to popular request of several users of mORMot, we identified and designed some feature requests dedicated to BigData process.

In fact, your data is the new value, especially if you propose SaaS (Software As A Service) hosting to your customers, with a farm of mORMot servers.
Recent Linux support for mORMot servers, together with the high performance and installation ease of our executable, open the gate to cheap cloud-based hosting.
As a consequence, a lot of information would certainly be gathered by your mORMot servers, and a single monolithic database is not an option any more.

For mORMot solutions hosted in cloud, a lot of data may be generated. The default SQLite3 storage engine may be less convenient, once it reaches some GB of file content. Backup becomes to be slow and inefficient, and hosting this oldest data in the main DB, probably stored on an expensive SSD, may be a lost of resource. Vertical scaling is limited by hardware and price factors.

This is were data sharding comes into scene.
Note that sharding is not replication/backup, nor clustering, nor just spreading. We are speaking about application-level data splitting, to ease maintenance and horizontal scalability of mORMot servers.

Data sharding could already be implemented with mORMot servers, thanks to TSQLRestStorage:

  • Using TSQLRestStorageExternal: any table may have its own external SQL database engine, may be in its separated DB server;
  • Using TSQLRestStorageMongoDB: any table may be stored on a MongoDB cluster, with its own sharding abilities;
  • Using TSQLRestStorageRemote: each table may have its own remote ORM/REST server.

But when data stored in a single table tends to grow without limit, this feature is not enough.
Let's see how the close future of mORMot looks like.

Continue reading...

- page 1 of 9