Synopse

To content | To menu | To search

Tag - Documentation

Entries feed

2014, Friday September 12

Legacy code, mORMot, and database sharing

It is pretty much possible that you would have to maintain and evolve a legacy project, based on an existing database, with a lot of already written SQL statements - see Legacy code and existing projects.

For instance, you would like to use mORMot for new features, and/or add mobile or HTML clients - see Cross-Platform clients.
In this case, the ORM advanced features - like ORM Cache or BATCH process, see BATCH sequences for adding/updating/deleting records - may conflict with the legacy code, for the tables which may have to be shared.
Here are some guidelines when working on such a project.

To be exhaustive about your question, we need to consider each ORM CRUD operation.
We may have to divide them in three kinds: read queries, insertions, and modifications of existing data.

Continue reading...

2014, Monday August 11

Cross-Platform mORMot Clients - Smart Mobile Studio

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

  • Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
  • FreePascal Compiler 2.7.1;
  • Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

Any feedback is welcome in our forum, as usual!

Continue reading...

Cross-Platform mORMot Clients - Delphi / FreePascal

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

  • Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
  • FreePascal Compiler 2.7.1;
  • Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

Any feedback is welcome in our forum, as usual!

Continue reading...

Cross-Platform mORMot Clients - Generating Code Wrappers

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

  • Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
  • FreePascal Compiler 2.7.1;
  • Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

Any feedback is welcome in our forum, as usual!

Continue reading...

Cross-Platform mORMot Clients - Units and Platforms

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

  • Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
  • FreePascal Compiler 2.7.1;
  • Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

Any feedback is welcome in our forum, as usual!

Continue reading...

2014, Tuesday August 5

Returning content as XML

By default, interface-based services of a mORMot server will always return a JSON array (or a JSON object, if TServiceFactoryServer.ResultAsJSONObject is true).
With some kind of clients (e.g. if they are made by a third party), it could be useful to return XML content instead.

Your mORMot server is able to let its interface-based services return XML context instead, or in addition to the default JSON format.

Continue reading...

2014, Sunday June 22

Audit-trail for ORM change tracking

Since most CRUD operations are centered within the scope of our mORMot server, we implemented in the ORM an integrated mean of tracking changes (aka Audit Trail) of any TSQLRecord.
In short, our ORM is transformed into a time-machine, just like the good old DeLorean!

Keeping a track of the history of business objects is one very common need for software modeling, and a must-have for any accurate data modeling, like Domain-Driven Design.
By default, as expected by the OOP model, any change to an object will forget any previous state of this object. But thanks to mORMot's exclusive change-tracking feature, you can persist the history of your objects.

Enabling audit-trail

By default, change-tracking feature will be disabled, saving performance and disk use.
But you can enable change tracking for any class, by calling the following method, on server side:

 aServer.TrackChanges([TSQLInvoice]);

This single line will let aServer: TSQLRestServer monitor all CRUD operations, and store all changes of the TSQLInvoice table within a TSQLRecordHistory table.

Continue reading...

2014, Monday June 9

Performance comparison from Delphi 6, 7, 2007, XE4 and XE6

Since there was recently some articles about performance comparison between several versions of the Delphi compiler, we had to react, and gives our personal point of view.

IMHO there won't be any definitive statement about this.
I'm always doubtful about any conclusion which may be achieved with such kind of benchmarks.
Asking "which compiler is better?" is IMHO a wrong question.
As if there was some "compiler magic": the new compiler will be just like a new laundry detergent - it will be cleaner and whiter...

Performance is not about marketing.
Performance is an iterative process, always a matter of circumstances, and implementation.

Circumstances of the benchmark itself.
Each benchmark will report only information about the process it measured.
What you compare is a limited set of features, running most of the time an idealized and simplified pattern, which shares nothing with real-world process.

Implementation is what gives performance.
Changing a compiler will only gives you some percents of time change.
Identifying the true bottlenecks of an application via a profiler, then changing the implementation of the identified bottlenecks may give order of magnitudes of speed improvement.
For instance, multi-threading abilities can be achieved by following some simple rules.

With our huge set of regression tests, we have at hand more than 16,500,000 individual checks, covering low-level features (like numerical and text marshaling), or high-level process (like concurrent client/server and database multi-threaded process).

You will find here some benchmarks run with Delphi 6, 7, 2007, XE4 and XE6 under Win32, and XE4 and XE6 under Win64.
In short, all compilers performs more or less at the same speed.
Win64 is a little slower than Win32, and the fastest appears to be Delphi 7, using our enhanced and optimized RTL.

Continue reading...

2014, Friday May 9

BREAKING CHANGE: TSQLRestServerStatic* classes are now renamed as TSQLRestStorage*

From the beginning, server-side storage tables which were not store in a SQLite3 database were implemented via some classes inheriting from TSQLRestServerStatic.
This TSQLRestServerStatic was inheriting from TSQLRestServer
, which did not make much sense (but was made for laziness years ago, if I remember well).

Now, a new TSQLRestStorage class, directly inheriting from TSQLRest, is used for per-table storage.
This huge code refactoring results in a much cleaner design, and will enhance code maintainability.
Documentation has been updated to reflect the changes.

Note that this won't change anything when using the framework (but the new class names): it is an implementation detail, which had to be fixed.

Continue reading...

2014, Wednesday May 7

MongoDB + mORMot benchmark

Here are some benchmark charts about MongoDB integration in mORMot's ORM.

MongoDB appears as a serious competitor to SQL databases, with the potential benefit of horizontal scaling and installation/administration ease - performance is very high, and its document-based storage fits perfectly with mORMot's advanced ORM features like Shared nothing architecture (or sharding).

Continue reading...

MongoDB + mORMot ORM = ODM

MongoDB (from "humongous") is a cross-platform document-oriented database system, and certainly the best known NoSQL database.
According to http://db-engines.com in April 2014, MongoDB is in 5th place of the most popular types of database management systems, and first place for NoSQL database management systems.
Our mORMot gives premium access to this database, featuring full NoSQL and Object-Document Mapping (ODM) abilities to the framework.

Integration is made at two levels:

  • Direct low-level access to the MongoDB server, in the SynMongoDB.pas unit;
  • Close integration with our ORM (which becomes defacto an ODM), in the mORMotMongoDB.pas unit.

MongoDB eschews the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (MongoDB calls the format BSON), which matches perfectly mORMot's RESTful approach.

This second article will focus on integration of MongoDB with our ORM.

Continue reading...

Direct MongoDB database access

MongoDB (from "humongous") is a cross-platform document-oriented database system, and certainly the best known NoSQL database.
According to http://db-engines.com in April 2014, MongoDB is in 5th place of the most popular types of database management systems, and first place for NoSQL database management systems.
Our mORMot framework gives premium access to this database, featuring full NoSQL and Object-Document Mapping (ODM) abilities to the framework.

Integration is made at two levels:

  • Direct low-level access to the MongoDB server, in the SynMongoDB.pas unit;
  • Close integration with our ORM (which becomes defacto an ODM), in the mORMotMongoDB.pas unit.

MongoDB eschews the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (MongoDB calls the format BSON), which matches perfectly mORMot's RESTful approach.

In this first article, we will detail direct low-level access to the MongoDB server, via the SynMongoDB.pas unit.

Continue reading...

2014, Monday April 28

Mustache Logic-less templates for Delphi - part 3

Mustache is a well-known logic-less template engine.
There is plenty of Open Source implementations around (including in JavaScript, which can be very convenient for AJAX applications on client side, for instance).
For mORMot, we created the first pure Delphi implementation of it, with a perfect integration with other bricks of the framework.

In last part of this series of blog articles, we will introduce the Mustache library included within mORMot source code tree.
You can download this documentation as one single pdf file.

Continue reading...

Mustache Logic-less templates for Delphi - part 2

Mustache is a well-known logic-less template engine.
There is plenty of Open Source implementations around (including in JavaScript, which can be very convenient for AJAX applications on client side, for instance).
For mORMot, we created the first pure Delphi implementation of it, with a perfect integration with other bricks of the framework.

In this second part of this series of blog articles, we will introduce the Mustache syntax.
You can download this documentation as one single pdf file.

Continue reading...

Mustache Logic-less templates for Delphi - part 1

Mustache is a well-known logic-less template engine.
There is plenty of Open Source implementations around (including in JavaScript, which can be very convenient for AJAX applications on client side, for instance).
For mORMot, we created the first pure Delphi implementation of it, with a perfect integration with other bricks of the framework.

In this first part of this series of blog articles, we will introduce the Mustache design.
You can download this documentation as one single pdf file.

Continue reading...

2014, Friday April 18

Introducing mORMot's architecture and design principles

We have just released a set of slides introducing 

  • ORM, SOA, REST, JSON, MVC, MVVM, SOLID, Mocks/Stubs, Domain-Driven Design concepts with Delphi, 
  • and showing some sample code using our Open Source mORMot framework.

You can follow the public link on Google Drive!

This is a great opportunity to discovers some patterns you may not be familiar with, and find out how mORMot try to implement them.
This set of slides may be less intimidating than our huge documentation - do not be terrified by our 1400 pages Software Architecture Design pdf!

Feedback is welcome on our forum, as usual.

2014, Monday April 7

JavaScript support in mORMot via SpiderMonkey

As we already stated, we finished the first step of integration of the SpiderMonkey engine to our mORMot framework.
Version 1.8.5 of the library is already integrated, and latest official revision will be soon merged, thanks to mpv's great contribution.
It can be seen as stable, since it is already used on production site to serve more than 1,000,000 requests per day.

You can now easily uses JavaScript on both client and server side.
On server side, mORMot's implementation offers an unique concept, i.e. true multi-threading, which is IMHO a huge enhancement when compared to the regular node.js mono-threaded implementation, and its callback hell.
In fact, node.js official marketing states its non-blocking scheme is a plus. It allows to define a HTTP server in a few lines, but huge server applications need JavaScript experts not to sink into a state a disgrace.

Continue reading...

2014, Friday March 7

Support of MySQL, DB2 and PostgreSQL

We just tested, benchmarked and validated Oracle MySQL, IBM DB2 and PostgreSQL support for our SynDB database classes and the mORMot's ORM core.
This article will also show all updated results, including our newly introduced multi-value INSERT statement generations, which speed up a lot BATCH insertion.

Stay tuned!

Purpose here is not to say that one library or database is better or faster than another, but publish a snapshot of mORMot persistence layer abilities, depending on each access library.

In this timing, we do not benchmark only the "pure" SQL/DB layer access (SynDB units), but the whole Client-Server ORM of our framework.

Process below includes all aspects of our ORM:

  • Access via high level CRUD methods (Add/Update/Delete/Retrieve, either per-object or in BATCH mode);
  • Read and write access of TSQLRecord instances, via optimized RTTI;
  • JSON marshaling of all values (ready to be transmitted over a network);
  • REST routing, with security, logging and statistic;
  • Virtual cross-database layer using its SQLite3 kernel;
  • SQL on-the-fly generation and translation (in virtual mode);
  • Access to the database engines via several libraries or providers.

In those tests, we just bypassed the communication layer, since TSQLRestClient and TSQLRestServer are run in-process, in the same thread - as a TSQLRestServerDB instance. So you have here some raw performance testimony of our framework's ORM and RESTful core, and may expect good scaling abilities when running on high-end hardware, over a network.

On a recent notebook computer (Core i7 and SSD drive), depending on the back-end database interfaced, mORMot excels in speed, as will show the following benchmark:

  • You can persist up to 570,000 objects per second, or retrieve 870,000 objects per second (for our pure Delphi in-memory engine);
  • When data is retrieved from server or client 38, you can read more than 900,000 objects per second, whatever the database back-end is;
  • With a high-performance database like Oracle, and our direct access classes, you can write 70,000 (via array binding) and read 160,000 objects per second, over a 100 MB network;
  • When using alternate database access libraries (e.g. Zeos, or DB.pas based classes), speed is lower (even if comparable for DB2, MS SQL, PostgreSQL, MySQL) but still enough for most work, due to some optimizations in the mORMot code (e.g. caching of prepared statements, SQL multi-values insertion, direct export to/from JSON, SQlite3 virtual mode design, avoid most temporary memory allocation...).

Difficult to find a faster ORM, I suspect.

Continue reading...

2014, Tuesday February 25

TDocVariant custom variant type

With revision 1.18 of the framework, we just introduced two new custom types of variants:

  • TDocVariant kind of variant;
  • TBSONVariant kind of variant.

The second custom type (which handles MongoDB-specific extensions - like ObjectID or other specific types like dates or binary) will be presented later, when dealing with MongoDB support in mORMot, together with the BSON kind of content. BSON / MongoDB support is implemented in the SynMongoDB.pas unit.

We will now focus on TDocVariant itself, which is a generic container of JSON-like objects or arrays.
This custom variant type is implemented in SynCommons.pas unit, so is ready to be used everywhere in your code, even without any link to the mORMot ORM kernel, or MongoDB.

TDocVariant documents

TDocVariant implements a custom variant type which can be used to store any JSON/BSON document-based content, i.e. either:

  • Name/value pairs, for object-oriented documents;
  • An array of values (including nested documents), for array-oriented documents;
  • Any combination of the two, by nesting TDocVariant instances.

Here are the main features of this custom variant type:

  • DOM approach of any object or array documents;
  • Perfect storage for dynamic value-objects content, with a schema-less approach (as you may be used to in scripting languages like Python or JavaScript);
  • Allow nested documents, with no depth limitation but the available memory;
  • Assignment can be either per-value (default, safest but slower when containing a lot of nested data), or per-reference (immediate reference-counted assignment);
  • Very fast JSON serialization / un-serialization with support of MongoDB-like extended syntax;
  • Access to properties in code, via late-binding (including almost no speed penalty due to our VCL hack as already detailed);
  • Direct access to the internal variant names and values arrays from code, by trans-typing into a TDocVariantData record;
  • Instance life-time is managed by the compiler (like any other variant type), without the need to use interfaces or explicit try..finally blocks;
  • Optimized to use as little memory and CPU resource as possible (in contrast to most other libraries, it does not allocate one class instance per node, but rely on pre-allocated arrays);
  • Opened to extension of any content storage - for instance, it will perfectly integrate with BSON serialization and custom MongoDB types (ObjectID, RegEx...), to be used in conjunction with MongoDB servers;
  • Perfectly integrated with our Dynamic array wrapper and its JSON serialization as with the record serialization;
  • Designed to work with our mORMot ORM: any TSQLRecord instance containing such variant custom types as published properties will be recognized by the ORM core, and work as expected with any database back-end (storing the content as JSON in a TEXT column);
  • Designed to work with our mORMot SOA: any interface-based service is able to consume or publish such kind of content, as variant kind of parameters;
  • Fully integrated with the Delphi IDE: any variant instance will be displayed as JSON in the IDE debugger, making it very convenient to work with.

To create instances of such variant, you can use some easy-to-remember functions:

  • _Obj() _ObjFast() global functions to create a variant object document;
  • _Arr() _ArrFast() global functions to create a variant array document;
  • _Json() _JsonFast() _JsonFmt() _JsonFastFmt() global functions to create any variant object or array document from JSON, supplied either with standard or MongoDB-extended syntax.

Continue reading...

2013, Monday November 4

Updated mORMot database benchmark - including MS SQL and PostgreSQL

On an recent notebook computer (Core i7 and SSD drive), depending on the back-end database interfaced, mORMot excels in speed:

  • You can persist up to 570,000 objects per second, or retrieve more than 900,000 objects per second (for our pure Delphi in-memory engine);
  • When data is retrieved from server or client cache, you can read more than 900,000 objects per second, whatever the database back-end is;
  • With a high-performance database like Oracle and our direct access classes, you can write 65,000 (via array binding) and read 160,000 objects per second, over a 100 MB network;
  • When using alternate database access libraries (e.g. Zeos, or DB.pas based classes), speed is lower, but still enough for most work.

Difficult to find a faster ORM, I suspect.

The following tables try to sum up all available possibilities, and give some benchmark (average objects/second for writing or read).

In these tables:

  • 'SQLite3 (file full/off/exc)' indicates use of the internal SQLite3 engine, with or without Synchronous := smOff and/or DB.LockingMode := lmExclusive;
  • 'SQLite3 (mem)' stands for the internal SQLite3 engine running in memory;
  • 'SQLite3 (ext ...)' is about access to a SQLite3 engine as external database - either as file or memory;
  • 'TObjectList' indicates a TSQLRestServerStaticInMemory instance, either static (with no SQL support) or virtual (i.e. SQL featured via SQLite3 virtual table mechanism) which may persist the data on disk as JSON or compressed binary;
  • 'Oracle' shows the results of our direct OCI access layer (SynDBOracle.pas);
  • 'NexusDB' is the free embedded edition, available from official site;
  • 'Zeos *' indicates that the database was accessed directly via the ZDBC layer;
  • 'FireDAC *' stands for FireDAC library;
  • 'UniDAC *' stands for UniDAC library;
  • 'BDE *' when using a BDE connection;
  • 'ODBC *' for a direct access to ODBC;
  • 'Jet' stands for a MSAccess database engine, accessed via OleDB;
  • 'MSSQL local' for a local connection to a MS SQL Express 2008 R2 running instance (this was the version installed with Visual Studio 2010), accessed via OleDB.

This list of database providers is to be extended in the future. Any feedback is welcome!

Numbers are expressed in rows/second (or objects/second). This benchmark was compiled with Delphi 7, so newer compilers may give even better results, with in-lining and advanced optimizations.

Note that these tests are not about the relative speed of each database engine, but reflect the current status of the integration of several DB libraries within the mORMot database access.

Purpose here is not to say that one library or database is better or faster than another, but publish a snapshot of current mORMot persistence layer abilities.

In this timing, we do not benchmark only the "pure" SQL/DB layer access (SynDB units), but the whole Client-Server ORM of our framework: process below includes read and write RTTI access of a TSQLRecord, JSON marshaling, CRUD/REST routing, virtual cross-database layer, SQL on-the-fly translation. We just bypass the communication layer, since TSQLRestClient and TSQLRestServer are run in-process, in the same thread - as a TSQLRestServerDB instance. So you have here some raw performance testimony of our framework's ORM and RESTful core.

You can compile the "15 - External DB performance" supplied sample code, and run the very same benchmark on your own configuration.

Continue reading...

- page 1 of 3