Synopse

To content | To menu | To search

Tag - mORMot

Entries feed

2014, Friday June 27

RoadMap refreshed

We did some cleaning in the mORMot official RoadMap.
Now feature requests tickets will detail all to-do items we would like to implement.

Current framework RoadMap and implementation is in fact going into a pragmatic direction.
No need to make all framework's unit compatible at once: so we introduced some client-dedicated units, without any dependency on SynCommons.pas.

We would like to implement (in this order):

The CrossPlatform folder already contains units which compile under all Delphi compilers (VCL and FMX), and FPC.

But perhaps we would move the server to Linux, either via FPC, or using Delphi itself!

Continue reading...

2014, Sunday June 22

Audit-trail for ORM change tracking

Since most CRUD operations are centered within the scope of our mORMot server, we implemented in the ORM an integrated mean of tracking changes (aka Audit Trail) of any TSQLRecord.
In short, our ORM is transformed into a time-machine, just like the good old DeLorean!

Keeping a track of the history of business objects is one very common need for software modeling, and a must-have for any accurate data modeling, like Domain-Driven Design.
By default, as expected by the OOP model, any change to an object will forget any previous state of this object. But thanks to mORMot's exclusive change-tracking feature, you can persist the history of your objects.

Enabling audit-trail

By default, change-tracking feature will be disabled, saving performance and disk use.
But you can enable change tracking for any class, by calling the following method, on server side:

 aServer.TrackChanges([TSQLInvoice]);

This single line will let aServer: TSQLRestServer monitor all CRUD operations, and store all changes of the TSQLInvoice table within a TSQLRecordHistory table.

Continue reading...

2014, Monday June 9

Performance comparison from Delphi 6, 7, 2007, XE4 and XE6

Since there was recently some articles about performance comparison between several versions of the Delphi compiler, we had to react, and gives our personal point of view.

IMHO there won't be any definitive statement about this.
I'm always doubtful about any conclusion which may be achieved with such kind of benchmarks.
Asking "which compiler is better?" is IMHO a wrong question.
As if there was some "compiler magic": the new compiler will be just like a new laundry detergent - it will be cleaner and whiter...

Performance is not about marketing.
Performance is an iterative process, always a matter of circumstances, and implementation.

Circumstances of the benchmark itself.
Each benchmark will report only information about the process it measured.
What you compare is a limited set of features, running most of the time an idealized and simplified pattern, which shares nothing with real-world process.

Implementation is what gives performance.
Changing a compiler will only gives you some percents of time change.
Identifying the true bottlenecks of an application via a profiler, then changing the implementation of the identified bottlenecks may give order of magnitudes of speed improvement.
For instance, multi-threading abilities can be achieved by following some simple rules.

With our huge set of regression tests, we have at hand more than 16,500,000 individual checks, covering low-level features (like numerical and text marshaling), or high-level process (like concurrent client/server and database multi-threaded process).

You will find here some benchmarks run with Delphi 6, 7, 2007, XE4 and XE6 under Win32, and XE4 and XE6 under Win64.
In short, all compilers performs more or less at the same speed.
Win64 is a little slower than Win32, and the fastest appears to be Delphi 7, using our enhanced and optimized RTL.

Continue reading...

2014, Friday May 30

Software Design, Brook, mORMot, RAD, SOLID and OOP

We got a very instructive discussion in our forums, with Silvio, the maintainer of the Brook Framework.
Brook is a nice framework for writing web applications using Free Pascal.

It comes to my mind what mORMot can offer.
We did not want to compare the features or say that one framework is better than the other, but it appeared to me that a lot of object pascal programmers are tied to 20th century programming model.

In fact, to embrace the potentials of mORMot, you need to switch your mind, and enhanced your RAD and OOP background, into 21th century SOLID model.

Continue reading...

2014, Sunday May 25

New crc32c() function using optimized asm and SSE 4.2 instruction

Cyclic Redundancy Check (CRC) codes are widely used for integrity checking of data in fields such as storage and networking.
There is an ever-increasing need for very high-speed CRC computations on processors for end-to-end integrity checks.

We just introduced to mORMot's core unit (SynCommons.pas) a fast and efficient crc32c() function.

It will use either:

  • Optimized x86 asm code, with unrolled loops;
  • SSE 4.2 hardware crc32 instruction, if available.

Resulting speed is very good.
This is for sure the fastest CRC function available in Delphi.
Note that there is a version dedicated to each Win32 and Win64 platform - both performs at the same speed!

In fact, most popular file formats and protocols (Ethernet, MPEG-2, ZIP, RAR, 7-Zip, GZip, and PNG) use the polynomial $04C11DB7, while Intel's hardware implementation is based on another polynomial, $1EDC6F41 (used in iSCSI and Btrfs).
So you would not use this new crc32c() function to replace the zlib's crc32() function, but as a convenient very fast hashing function at application level.
For instance, our TDynArray wrapper will use it for fast items hashing.

Continue reading...

2014, Sunday May 18

Automatic JSON serialization of record or dynamic arrays via Enhanced RTTI

Since Delphi 2010, the compiler generates additional RTTI at compilation, so that all record fields are described, and available at runtime.
By the way, this enhanced RTTI is one of the reasons why executables did grow so much in newer versions of the compiler.

Our SynCommons.pas unit is now able to use this enhanced information, and let any record be serialized via RecordLoad() and RecordSave() functions, and all internal JSON marshalling process.

In short, you have nothing to do.
Just use your record as parameters, and, with Delphi 2010 and up, they will be serialized as valid JSON objects.

Of course, text-based definition or callback-based registration are still at hand, and will be used with older versions of Delphi.
But you could be used to by-pass or extend the enhanced-RTTI serialization, even on newer versions of the compiler.

Continue reading...

New sample for JSON performance: mORMot vs SuperObject/XSuperObject/dwsJSON/DBXJSON

We have just added a new "25 - JSON performance" sample to benchmark JSON process, using well most known Delphi libraries...

A new fight
featuring
mORMot vs SuperObject/XSuperObject/dwsJSON/DBXJSON

On mORMot side, it covers TDocVariant, late binding, TSQLTable, ORM, record access, BSON...

We tried to face several scenarios:

  • parse/access/write iteration over a small JSON document,
  • read of deeply nested 680 KB JSON (here mORMot is slower than SO/dwsJSON),
  • read of one 180 MB JSON file (with on-the-fly adaptation to fit a record layout),
  • named access to all rows and columns of a 1 MB JSON table, extracted from a SQL request (with comparison with our ORM performance).

On average and in details, mORMot is the fastest in almost all scenarios (with an amazing performance for table/ORM processing), dwsJSON performs very well (better than SuperObject), and DBXJSON is the slowest (by far, but XE6 version is faster than XE4).

Continue reading...

"Native" means CPU-native... even Microsoft admits it!

There were a lot of debate about what "native" was..

Especially for some great companies, you want to sell their compiler technologies...
For them, "native" is not CPU-native, but framework-native...

Even Microsoft claimed since the beginning of C# that the managed .Net model was faster, due to "optimized JIT or NGEN compilation"...

But even Microsoft is clearly changing its mind!

They switch to "Native" for their framework, especially when targeting mobile platforms!

Continue reading...

2014, Friday May 9

BREAKING CHANGE: TSQLRestServerStatic* classes are now renamed as TSQLRestStorage*

From the beginning, server-side storage tables which were not store in a SQLite3 database were implemented via some classes inheriting from TSQLRestServerStatic.
This TSQLRestServerStatic was inheriting from TSQLRestServer
, which did not make much sense (but was made for laziness years ago, if I remember well).

Now, a new TSQLRestStorage class, directly inheriting from TSQLRest, is used for per-table storage.
This huge code refactoring results in a much cleaner design, and will enhance code maintainability.
Documentation has been updated to reflect the changes.

Note that this won't change anything when using the framework (but the new class names): it is an implementation detail, which had to be fixed.

Continue reading...

2014, Wednesday May 7

Benchmarking Mustache libraries: native SynMustache vs mustache.js/SpiderMonkey

I just wrote a small sample program, for benchmarking Mustache libraries: native SynMustache vs mustache.js running on SpiderMonkey 24...

And the winner is ...SynMustache, which is 10 times faster, uses almost no memory during process, and handles inlined {{>partials}} natively (whereas we have to handle them manually with mustache.js)!

Who says that Garbage Collection and immutable strings in modern JITted runtimes are faster than "native" Delphi applications?
Are you still preferring the "NextGen" roadmap?

Continue reading...

MongoDB + mORMot benchmark

Here are some benchmark charts about MongoDB integration in mORMot's ORM.

MongoDB appears as a serious competitor to SQL databases, with the potential benefit of horizontal scaling and installation/administration ease - performance is very high, and its document-based storage fits perfectly with mORMot's advanced ORM features like Shared nothing architecture (or sharding).

Continue reading...

MongoDB + mORMot ORM = ODM

MongoDB (from "humongous") is a cross-platform document-oriented database system, and certainly the best known NoSQL database.
According to http://db-engines.com in April 2014, MongoDB is in 5th place of the most popular types of database management systems, and first place for NoSQL database management systems.
Our mORMot gives premium access to this database, featuring full NoSQL and Object-Document Mapping (ODM) abilities to the framework.

Integration is made at two levels:

  • Direct low-level access to the MongoDB server, in the SynMongoDB.pas unit;
  • Close integration with our ORM (which becomes defacto an ODM), in the mORMotMongoDB.pas unit.

MongoDB eschews the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (MongoDB calls the format BSON), which matches perfectly mORMot's RESTful approach.

This second article will focus on integration of MongoDB with our ORM.

Continue reading...

Direct MongoDB database access

MongoDB (from "humongous") is a cross-platform document-oriented database system, and certainly the best known NoSQL database.
According to http://db-engines.com in April 2014, MongoDB is in 5th place of the most popular types of database management systems, and first place for NoSQL database management systems.
Our mORMot framework gives premium access to this database, featuring full NoSQL and Object-Document Mapping (ODM) abilities to the framework.

Integration is made at two levels:

  • Direct low-level access to the MongoDB server, in the SynMongoDB.pas unit;
  • Close integration with our ORM (which becomes defacto an ODM), in the mORMotMongoDB.pas unit.

MongoDB eschews the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (MongoDB calls the format BSON), which matches perfectly mORMot's RESTful approach.

In this first article, we will detail direct low-level access to the MongoDB server, via the SynMongoDB.pas unit.

Continue reading...

2014, Friday May 2

mORMot on GitHub

There was a long-standing request from customers, about putting all our source code repository to GitHub.

We like a lot our self-hosted Fossil repository, and will continue to use it as our main system, including issue tracking and wiki, for our official web site.

But we created a repository on GitHub, on https://github.com/synopse/mORMot

Continue reading...

2014, Monday April 28

Mustache Logic-less templates for Delphi - part 3

Mustache is a well-known logic-less template engine.
There is plenty of Open Source implementations around (including in JavaScript, which can be very convenient for AJAX applications on client side, for instance).
For mORMot, we created the first pure Delphi implementation of it, with a perfect integration with other bricks of the framework.

In last part of this series of blog articles, we will introduce the Mustache library included within mORMot source code tree.
You can download this documentation as one single pdf file.

Continue reading...

Mustache Logic-less templates for Delphi - part 2

Mustache is a well-known logic-less template engine.
There is plenty of Open Source implementations around (including in JavaScript, which can be very convenient for AJAX applications on client side, for instance).
For mORMot, we created the first pure Delphi implementation of it, with a perfect integration with other bricks of the framework.

In this second part of this series of blog articles, we will introduce the Mustache syntax.
You can download this documentation as one single pdf file.

Continue reading...

Mustache Logic-less templates for Delphi - part 1

Mustache is a well-known logic-less template engine.
There is plenty of Open Source implementations around (including in JavaScript, which can be very convenient for AJAX applications on client side, for instance).
For mORMot, we created the first pure Delphi implementation of it, with a perfect integration with other bricks of the framework.

In this first part of this series of blog articles, we will introduce the Mustache design.
You can download this documentation as one single pdf file.

Continue reading...

2014, Friday April 18

Introducing mORMot's architecture and design principles

We have just released a set of slides introducing 

  • ORM, SOA, REST, JSON, MVC, MVVM, SOLID, Mocks/Stubs, Domain-Driven Design concepts with Delphi, 
  • and showing some sample code using our Open Source mORMot framework.

You can follow the public link on Google Drive!

This is a great opportunity to discovers some patterns you may not be familiar with, and find out how mORMot try to implement them.
This set of slides may be less intimidating than our huge documentation - do not be terrified by our 1400 pages Software Architecture Design pdf!

Feedback is welcome on our forum, as usual.

2014, Monday April 7

JavaScript support in mORMot via SpiderMonkey

As we already stated, we finished the first step of integration of the SpiderMonkey engine to our mORMot framework.
Version 1.8.5 of the library is already integrated, and latest official revision will be soon merged, thanks to mpv's great contribution.
It can be seen as stable, since it is already used on production site to serve more than 1,000,000 requests per day.

You can now easily uses JavaScript on both client and server side.
On server side, mORMot's implementation offers an unique concept, i.e. true multi-threading, which is IMHO a huge enhancement when compared to the regular node.js mono-threaded implementation, and its callback hell.
In fact, node.js official marketing states its non-blocking scheme is a plus. It allows to define a HTTP server in a few lines, but huge server applications need JavaScript experts not to sink into a state a disgrace.

Continue reading...

2014, Saturday March 29

Enhanced and fixed late-binding of variants for Delphi XE2 and up

For several units of our framework, we allow late-binding of data values, using a variant and direct named access to properties:
- In SynCommons, we defined our TDocVariant custom variant type, able to store any JSON/BSON document-based content;
- In SynBigTable, we use the TSynTableVariantType custom variant type, as defined in SynCommons;
- In SynDB, we defined a TSQLDBRowVariantType, ready to access any column of a RDBMS data result set row;
- In mORMot, we allow access to TSQLTableRowVariantType column values.

It's a very convenient way of accessing result rows values. Code is still very readable, and safe at the same time.

For instance, we can write:

var V: variant;
 ...
  TDocVariant.New(V); // or slightly slower V := TDocVariant.New;
  V.name := 'John';
  V.year := 1972;
  // now V contains {"name":"john","year":1982}

This is just another implementation of KISS design in our framework.

Since Delphi XE2, some modifications were introduced to the official DispInvoke() RTL implementation:

  1. A new varUStrArg kind of parameter has been defined, which will allow to transmit UnicodeString property values;
  2. All text property values would be transmitted as BSTR / WideString / varOleStr variants to the invoked variant type;
  3. All textual property names were normalized to be in UPPERCASE.

Those modifications are worth considering...
And we may have discovered two regressions: one about speed, and the other about an unexpected logic bug...

Continue reading...

- page 1 of 7