Synopse

To content | To menu | To search

Tag - ORM

Entries feed

2016, Thursday November 10

EKON20 mORMot Conferences

EKON20 is now over, and there was a lot of people, great speakers, beautiful T-Shirt, and fresh beer!

I've published the slides of my mORMot conferences on SlideShare...

The "classic" Synopse/mORMot slides have also been uploaded to their latest revision, so don't hesitate to check what's new!

The PDF are also available for direct download from our server.

Enjoy!

2016, Tuesday September 6

Ride the mORMot at EKON 20 in Dusseldorf!

There are still a few days for "very early birds" offer for EKON 20 conference, and meet us for 3 sessions (including a half-day training/introduction to mORMot)!
Join us the 7-9th of November in Düsseldorf!

Our sessions are not restricted to mORMot, but will use mORMot to illustrate some concepts and design ideas:

You can contact me if you want to join, so that I may give you an additional discount password!

Hope we can meet for a chat and (a few) beers!

2015, Friday December 11

Audit Trail for Services

We have seen previously how the ORM part of the framework is able to provide an Audit Trail for change tracking.
It is a very convenient way of storing the change of state of the data.

On the other side, in any modern SOA solution, data is not at the center any more, but services.
Sometimes, the data is not stored within your server, but in a third-party Service-Oriented Architecture (SOA).
Being able to monitor the service execution of the whole system becomes sooner or later mandatory.

Our framework allows to create an Audit Trail of any incoming or outgoing service operation, in a secure, efficient and automated way.

Continue reading...

2015, Saturday November 21

Try to avoid RTTI (ab)use

There is a very trendy move, since a few years, to value so called "meta-programming".
In short, it is about the ability to treat programs as their data.
It is a very powerful paradigm in functional languages, and it was also introduced to OOP languages, even in SmallTalk a long time before this concept was trendy in Ruby, C# or Java.

In OOP compiled languages, reflection is used to achieve a similar behavior at run-time, mainly via RTTI (Run-Time Type Information).
Delphi supports RTTI since its version 1, as it was heavily used e.g. for all UI streaming.
In our framework, we rely on RTTI for its main features: ORMSOA and MVC - and even in some other parts, like Desktop UI generation.

But RTTI could easily be abused.
Here are some thoughts, started as a comment in a good old Mason's blog article about how RTTI performance may be a bottleneck.
My comment was to get rid of RTTI, and follow a SOLID implementation with explicit OOP code, like use of interface.

Continue reading...

2015, Friday September 25

ORM TNullable* fields for NULL storage

In Delphi code, NULLable types do not exist as such. There is no native int? type, as in C#.
But at SQL and JSON levels, the NULL value does exist and should be converted as expected by the ORM.

In SQLite3 itself, NULL is handled as stated in http://www.sqlite.org/lang_expr.html (see e.g. IS and IS NOT operators).
It is worth noting that NULL handling is not consistent among all existing database engines, e.g. when you are comparing NULL with non NULL values... so we recommend using it with care in any database statements, or only with proper (unit) testing, when you switch from one database engine to another.

By default, in the mORMot ORM/SQL code, NULL will appear only in case of a BLOB storage with a size of 0 bytes.
Otherwise, you should not see it as a value, in most kinds of ORM properties.

Null-oriented value types have been implemented in our framework, since the object pascal language does not allow defining a nullable type (yet)

We choose to store those values as variant, with a set of TNullable dedicated types, as defined in mORMot.pas:

type
  TNullableInteger = type variant;
  TNullableBoolean = type variant;
  TNullableFloat = type variant;
  TNullableCurrency = type variant;
  TNullableDateTime = type variant;
  TNullableTimeLog = type variant;
  TNullableUTF8Text = type variant;

Continue reading...

2015, Monday August 31

Delphi 10 = DX Seattle is out, mORMot supports it

We expected Delphi XE9, and now we have Rad Studio 10 Seattle, with Delphi renamed as Delphi 10 Seattle, or simply DX.

No big news for the Delphi compiler itself (we are still waiting for Linux server support), but a lot of FireMonkey updates, Windows 10 compatibility enhancements, enhancements to JSON (better performance using a SAX approach), and NoSQL/MongoDB support in FireDAC.
The documentation is rather sparse for the new features, but it goes into the right direction (we support MongoDB since a long time, in our ORM/ODM).
See what's new in details.

Of course, our Open Source mORMot framework supports this version.
Feedback is welcome, as usual!
Enjoy the new DX IDE!

2015, Sunday August 23

"SQL and NoSQL", not "SQL vs NoSQL"

You know certainly that our mORMot Open Source framework is an ORM, i.e. mapping objects to a relational / SQL database (Object Relational Mapping).
You may have followed also that it is able to connect to a NoSQL database, like MongoDB, and that the objects are then mapped via an ODM (Object Document Mapping) - the original SQL SELECT are even translated on the fly to MongoDB queries.

But thanks to mORMot, it is not "SQL vs NoSQL" - but "SQL and NoSQL".
You are not required to make an exclusive choice.
You can share best of both worlds, depending on your application needs.

In fact, the framework is able to add NoSQL features to a regular relational / SQL database, by storing JSON documents in TEXT columns.

In your end-user code, you just define a variant field in the ORM, and store a TDocVariant document within.
We also added some dedicated functions at SQL level, so that SQLite3 could be used as embedded fast engine, and provide advanced WHERE clauses on this JSON content.

Continue reading...

2015, Tuesday July 14

New blog about mORMot

An enthusiastic mORMot user, named willo in the forum, just started a blog about his experiments with our framework.

The information there is clear, simple, and right to the point.
If you are a little lost in our huge documentation, it is a good place to start!

Continue reading...

2015, Saturday June 6

GetIt "Spirit" Concerns

I'm confused by the GetIt Submitting official blog page.
Reminds me the darker ages of Delphi licensing change of XE3.

GetIt is the new XE8 package manager for RAD Studio. Information about how to submit your libraries to GetIt has just been made available by Embarcadero. The idea behind GetIt is really to make is easier and faster to discover, install, and keep updated some of the best open source libraries for Delphi and C++Builder.

When you look at the approval conditions, it sounds like if mORMot would not find its way in this package manager:

Replacing key capabilities that are part of the core platforms definitions such as Client/Server FireDAC pack or the DataSnap/EMS Enterprise middleware, would be less likely to be accepted.
The different SKUs are meant for different types of developers, and multi-tier capabilities with strong client/server RDBMS integration require an Enterprise edition license.
We will bias acceptance toward GetIt libraries that respect the spirit of our licensing and editions, not just use the letter of the license and the technical boundaries. If you are unsure about your submission please check with us first. 

What is this "spirit of our licensing and editions"?
Why is it not part of the official license terms?
Where does this assumption comes from?
Would the licensing conditions change in the close future, as with the XE3 "episode"?
Would Marco's interpretation become the new rule?

It clearly reminds me the XE3 time where there was an attempt from Embarcadero to modify their licence terms, so that third party vendors or Open Source libraries would not be allowed to create multi-tier frameworks with Delphi!
Is it true that "strong client/server RDBMS integration require an Enterprise edition" ?
Last time I checked the licence terms, it was not stated.
Why on earth would we have to pay for the Entreprise edition, if the Professionnal edition is all that you need?

Still the same "closed" spirit.
It is like if they know their own n-Tier solution weaknesses, so they try to avoid any other possibility, but to use their own.
They clearly do not understand the benefit and dynamic of Open Source.

I guess our little mORMot falls directly into this "unwelcomed" category.
I did not make the submission yet. But should I?
Perhaps sub-part of the framework may find its way in: SynPdf, SynCrypto, SynGdiPlus, SynCommons...
But the main ORM/SOA/REST/MVC/StubMock features would certainly be rejected.

Our Open Source project is sometimes preferred to DataSnap/EMS (or even FireDAC), not only for licence cost, but also about features, documentation, stability, compatibility with older versions of Delphi, performance, and Linux compatibility.
I have encountered several companies which are still using Delphi because of mORMot: if they did not have found it, they would have moved to C# or Java, just to be able to use a full SOA/MVC stack, which was not available, even in the "Enterprise" version of Delphi.

Story repeats itself.
I just wanted to ensure that the licensing terms would not change in that direction.
I - as many Delphi users - would not let this GetIt "spirit" become the new rule.
We have to react, as we did for XE3, otherwise we may all suffer!
IMHO Embacardero should better focus on the compiler and IDE, not cutting the branch on which they are seated...

What do you think? Comments and feebacks are welcome!

2015, Monday June 1

Updated Slides about ORM SOA MVC SOLID DDD

One year ago, we published a set of slides about the main concepts implemented by our framework.
Mainly about ORM (and ODM), NoSQL, JSON, SOA, MVC (and MVVM), SOLID, DDD, CQRS and some patterns like Stubs, Mocks, Factory, Repository, Unit-Of-Work.
Worth a look, if you want to find out the benefits of the latest software development techniques.
They try to open the landscape of any Delphi developer (probably with a mostly RAD and OOP background) to some new areas.

I just updated the slides from our public GoogleDrive folder.
They now reflect the latest state of the framework (e.g. ORM real-time synchronization, asynchronous callbacks, DDD CQRS services...).
They have also been polished after several public presentations, since I used them as base for trainings I made for some European companies.

If you want to go further, or have some more enlightenment, ensure you took a look at our framework Documentation, which would detail all those patterns, and how mORMot may help implementing them for your projects!

Feedback is welcome on our forum, as usual!

2015, Monday May 18

CQRS Persistence Service of any DDD object with mORMot

We introduced DDD concepts some time ago, in a series of articles in this blog.
At that time, we proposed a simple way of using mORMot types to implement DDD in your applications.
But all Domain Entitities being tied to the framework TSQLRecord class did appear as a limitation, breaking the Persistence Ignorance principle, since it couples the DDD objects to the framework implementation details.

We introduced a new mORMotDDD.pas unit, which is able to easily create CQRS Persistence services for any plain Delphi class (the famous PODOs - Plain Old Delphi Objects).
No need to inherit from TSQLRecord, or pollute your class definition with attributes!

For instance, a TUser class may be persisted via such a service:

type
  IDomUserCommand = interface(IDomUserQuery)
    ['{D345854F-7337-4006-B324-5D635FBED312}']
    function Add(const aAggregate: TUser): TCQRSResult;
    function Update(const aUpdatedAggregate: TUser): TCQRSResult;
    function Delete: TCQRSResult;
    function Commit: TCQRSResult;
  end;

Here, the write operations are defined in a IDomUserCommand service, which is separated (but inherits) from IDomUserQuery, which is used for read operations.
Separating reads and writes is a powerful pattern also known as CQRS, i.e. Command Query Responsibility Segregation, which we followed when defining our persistence services.
The framework make it pretty easy to create such services for storing any kind of class type in any SQL or NoSQL engine, with almost no code to write.
Last but not least, using such interface-based services for data persistence will allow to stub or mock the data access layer, making unit testing straightforward: you would not fear to write TDD code any more!

Please refer to our updated documentation for this unique and powerful feature.
You may take a look at the corresponding dddDomUserTypes.pas, dddDomUserCQRS.pas, and dddInfraRepoUser.pas units, detailed as sample reference.
Feedback is welcome in our forum, as usual!

2015, Monday April 6

Asynchronous Service - WebSockets, Callbacks and Publish-Subscribe

When publishing SOA services, most of them are defined as stateless, in a typical query/answer pattern - see Service-Oriented Architecture (SOA).
This fits exactly with the RESTful approach of Client-Server services via interfaces, as proposed by the framework.

But it may happen that a client application (or service) needs to know the state of a given service. In a pure stateless implementation, it will have to query the server for any state change, i.e. for any pending notification - this is called polling.

Polling may take place for instance:

  • When a time consuming work is to be processed on the server side. In this case, the client could not wait for it to be finished, without raising a timeout on the HTTP connection: as a workaround, the client may start the work, then ask for its progress status regularly using a timer and a dedicated method call;
  • When an unpredictable event is to be notified from the server side. In this case, the client should ask regularly (using a timer, e.g. every second), for any pending event, then react on purpose.

It may therefore sounds preferred, and in some case necessary, to have the ability to let the server notify one or several clients without any prior query, nor having the requirement of a client-side timer:

  • Polling may be pretty resource consuming on both client and server sides, and add some unwanted latency;
  • If immediate notification is needed, some kind of "long polling" algorithm may take place, i.e. the server will wait for a long time before returning the notification state if no event did happen: in this case, a dedicated connection is required, in addition to the REST one;
  • In an event-driven systems, a lot of messages are sent to the clients: a proper publish/subscribe mechanism is preferred, otherwise the complexity of polling methods may increase and become inefficient and unmaintainable;
  • Explicit push notifications may be necessary, e.g. when a lot of potential events, associated with a complex set of parameters, are likely to be sent by the client.

Our mORMot framework is therefore able to easily implement asynchronous callbacks over WebSockets, defining the callbacks as interface parameters in service method definitions - see Available types for methods parameters.

Continue reading...

Real-Time ORM Master/Slave Replication via WebSockets

In a previous article, we presented how Master/Slave replication may be easily implemented in mORMot's RESTful ORM.
Do not forget to visit the corresponding paragraphs of our online documentation, which has been updated, and is more accurate!

Sometimes, the on-demand synchronization is not enough.
So we have just introduced real-time replication via WebSockets.
For instance, you may need to:

  • Synchronize a short list of always evolving items which should be reflected as soon as possible;
  • Involve some kind of ACID-like behavior (e.g. handle money!) in your replicated data;
  • Replicate not from a GUI application, but from a service, so use of a TTimer is not an option;
  • Combine REST requests (for ORM or services) and master/slave ORM replication on the same wire, e.g. in a multi-threaded application.

In this case, the framework is able to use WebSockets and asynchronous callbacks to let the master/slave replication - see Asynchronous callbacks - take place without the need to ask explicitly for pending data.
You would need to use TSQLRestServer.RecordVersionSynchronizeMasterStart, TSQLRestServer.RecordVersionSynchronizeSlaveStart and TSQLRestServer.RecordVersionSynchronizeSlaveStop methods over the proper kind of bidirectional connection.

Continue reading...

2015, Monday February 16

Benchmarking JsonDataObjects JSON parser

There is a new player in town.
Since it has been written by Andreas Hausladen, the maintainer of the great Delphi IDE fix packs, this new JSON library is very promising.

And in fact, it is fast, and sounds pretty great!
Here are some numbers, compared with SuperObject, standard DBXJson, dwsJSON, QDAC and mORMot.
Please refer to previous benchmark articles about those libraries. We will now focus on JsonDataObjects.

Continue reading...

2015, Sunday February 1

Benchmarking QDAC3 JSON parser

Do you know QDAC3 ?
This is an open source project, from China (with Chinese comments and exception errors messages, but the methods and variables are in English).
It is cross-platform, and told to be very fast about JSON process.

You can download this Open Source project code from http://sourceforge.net/projects/qdac3
And their blog - in Chinese - is at http://blog.qdac.cc/

So I included QDAC3 in our "25 - JSON performance" sample.
Numbers are talking, now.

Continue reading...

2015, Saturday January 10

mORMot under Linux thanks to FPC

You can use the FreePascal Compiler (FPC) to compile the mORMot framework source code, targetting Windows and Linux.

Linux is a premium target for cheap and efficient server Hosting. Since mORMot has no dependency, installing a new mORMot server is as easy as copying its executable on a blank Linux host, then run it. No need to install any framework nor runtime. You could even use diverse operating systems (several Linux or Windows Server versions) in your mORMot servers farm, with minimal system requirements, and updates.

We will now see how to write your software with Linux-compiling in mind, and also give some notes about how to install a Linux Virtual Machine with Lazarus on your Windows computer, compiling both FPC and Lazarus from their SVN latest sources!

Continue reading...

2014, Wednesday December 31

2015: the future of mORMot is BigData

How would be 2015 like for our little rodents?
Due to popular request of several users of mORMot, we identified and designed some feature requests dedicated to BigData process.

In fact, your data is the new value, especially if you propose SaaS (Software As A Service) hosting to your customers, with a farm of mORMot servers.
Recent Linux support for mORMot servers, together with the high performance and installation ease of our executable, open the gate to cheap cloud-based hosting.
As a consequence, a lot of information would certainly be gathered by your mORMot servers, and a single monolithic database is not an option any more.

For mORMot solutions hosted in cloud, a lot of data may be generated. The default SQLite3 storage engine may be less convenient, once it reaches some GB of file content. Backup becomes to be slow and inefficient, and hosting this oldest data in the main DB, probably stored on an expensive SSD, may be a lost of resource. Vertical scaling is limited by hardware and price factors.

This is were data sharding comes into scene.
Note that sharding is not replication/backup, nor clustering, nor just spreading. We are speaking about application-level data splitting, to ease maintenance and horizontal scalability of mORMot servers.

Data sharding could already be implemented with mORMot servers, thanks to TSQLRestStorage:

  • Using TSQLRestStorageExternal: any table may have its own external SQL database engine, may be in its separated DB server;
  • Using TSQLRestStorageMongoDB: any table may be stored on a MongoDB cluster, with its own sharding abilities;
  • Using TSQLRestStorageRemote: each table may have its own remote ORM/REST server.

But when data stored in a single table tends to grow without limit, this feature is not enough.
Let's see how the close future of mORMot looks like.

Continue reading...

2014, Friday November 28

ODM magic: complex queries over NoSQL / MongoDB

You know that our mORMot is able to access directly any MongoDB database engine, allowing its ORM to become an ODM, and using NoSQL instead of SQL for the query languages.

But at mORMot level, you could share the same code between your RDBMS and NoSQL databases.
The ORM/ODM is able to do all the conversions by itself!
Since we have just improved this feature, it is time to enlighten its current status.

Continue reading...

2014, Tuesday November 18

HTTP remote access for SynDB SQL execution

For mORMot, we developed a fully feature direct access layer to any RDBMS, implemented in the SynDB.pas unit.

You can use those SynDB classes to execute any SQL statement, without any link to the framework ORM.
At reading, the resulting performance is much higher than using the standard TDataSet component, which is in fact a true performance bottleneck.
It has genuine features, like column access via late-binding, an innovative ISQLDBRows interface, and ability to directly access the low-level binary buffers of the database clients.

We just added a nice feature to those classes: the ability to access remotely, via plain HTTP, to any SynDB supported database!

Continue reading...

2014, Friday November 14

BREAKING CHANGE - TSQLRecord.ID primary key changed to TID: Int64

Up to now, the TSQLRecord.ID property was defined in mORMot.pas as a plain PtrInt/NativeInt (i.e. Integer under Win32), since it was type-cast as pointer for TSQLRecord published properties.
We introduced a new TID type, so that the ORM primary key would now be defined as Int64.

All the framework ORM process relies on the TSQLRecord class.
This abstract TSQLRecord class features a lot of built-in methods, convenient to do most of the ORM process in a generic way, at record level.

It first defines a primary key field, defined as ID: TID, i.e. as Int64 in mORMot.pas:

type
  TID = type Int64;
  ...
  TSQLRecord = class(TObject)
  ...
    property ID: TID read GetID write fID;
  ...

In fact, our ORM relies now on a Int64 primary key, matching the SQLite3 ID/RowID primary key.
This primary key will be used as RESTful resource identifier, for all CRUD operations.

Continue reading...

- page 1 of 6