Synopse

To content | To menu | To search

Tag - JSON

Entries feed

2015, Tuesday June 30

Faster String process using SSE 4.2 Text Processing Instructions STTNI

A lot of our code, and probably yours, is highly relying on text process.
In our mORMot framework, most of its features use JSON text, encoded as UTF-8.
Profiling shows that a lot of time is spent computing the end of a text buffer, or comparing text content.

You may know that In its SSE4.2 feature set, Intel added STTNI (String and Text New Instructions) opcodes.
They are several new instructions that perform character searches and comparison on two operands of 16 bytes at a time.

I've just committed optimized version of StrComp() and StrLen(), also used for our TDynArrayHashed wrapper.
The patch works from Delphi 5 up to XE8, and with FPC - unknown SSE4.2 opcodes have been entered as hexadecimal bytes, for compatibility with the last century compilers!
The resulting speed up may be worth it!

Next logical step would be to use those instruction in the JSON process itself.
It may speed up the parsing speed of our core functions (which is already very optimized, but written in a classical one-char-at-a-time reading).
Main benefit would be to read the incoming UTF-8 text buffer by blocks of 16 bytes, and performing several characters comparison in a few CPU cycles, with no branching.
Also JSON writing would benefit for it, since escaping could be speed up thanks to STTNI instructions.

Any feedback is welcome, as usual!

2015, Sunday April 12

Why Transmitting Exceptions in SOA services is not a good idea

Usually, in Delphi application (like in most high-level languages), errors are handled via exceptions. By default, any Exception raised on the server side, within an interface-based service method, will be intercepted, and transmitted as an error to the client side, then a safe but somewhat obfuscated EInterfaceFactoryException will be raised on the client side, containing additional information serialized as JSON.

You may wonder why exceptions are not transmitted and raised directly on the client side, with our mORMot framework interface-based services, as if they were executed locally.

We will now detail some arguments, and patterns to be followed.

Continue reading...

2015, Monday February 16

Benchmarking JsonDataObjects JSON parser

There is a new player in town.
Since it has been written by Andreas Hausladen, the maintainer of the great Delphi IDE fix packs, this new JSON library is very promising.

And in fact, it is fast, and sounds pretty great!
Here are some numbers, compared with SuperObject, standard DBXJson, dwsJSON, QDAC and mORMot.
Please refer to previous benchmark articles about those libraries. We will now focus on JsonDataObjects.

Continue reading...

2015, Sunday February 1

Benchmarking QDAC3 JSON parser

Do you know QDAC3 ?
This is an open source project, from China (with Chinese comments and exception errors messages, but the methods and variables are in English).
It is cross-platform, and told to be very fast about JSON process.

You can download this Open Source project code from http://sourceforge.net/projects/qdac3
And their blog - in Chinese - is at http://blog.qdac.cc/

So I included QDAC3 in our "25 - JSON performance" sample.
Numbers are talking, now.

Continue reading...

2014, Friday November 28

ODM magic: complex queries over NoSQL / MongoDB

You know that our mORMot is able to access directly any MongoDB database engine, allowing its ORM to become an ODM, and using NoSQL instead of SQL for the query languages.

But at mORMot level, you could share the same code between your RDBMS and NoSQL databases.
The ORM/ODM is able to do all the conversions by itself!
Since we have just improved this feature, it is time to enlighten its current status.

Continue reading...

2014, Friday October 24

MVC/MVVM Web Applications

We almost finished implementing a long-standing feature request, introducing MVC / MVVM for Web Applications (like RubyOnRails) in mORMot.
This is a huge step forward, opening new perspectives not only to our framework, but for the Delphi community.
In respect to the existing MVC frameworks for Delphi, our  solution is closer to Delphi On Rails (by the convention-over-configuration pattern) than the Delphi MVC Framework or XMM.
The mORMot point of view is unique, and quite powerful, since it is integrated with other parts of our framework, as its ORM/ODM or interface-based services.
Of course, this is a work in progress, so you are welcome to put your feedback, patches or new features!

We will now explain how to build a MVC/MVVM web application using mORMot, starting from the "30 - MVC Server" sample.
First of all, check the source in our GitHub repository: two .pas files, and a set of minimalist Mustache views.

This little web application publishes a simple BLOG, not fully finished yet (this is a Sample, remember!).
But you can still execute it in your desktop browser, or any mobile device (thanks to a simple Bootstrap-based responsive design), and see the articles list, view one article and its comments, view the author information, log in and out.

This sample is implemented as such:

MVVM Source mORMot
Model MVCModel.pas TSQLRestServerDB ORM over a SQlite3 database
View *.html Mustache template engine in the Views sub-folder
ViewModel MVCViewModel.pas Defined as one IBlogApplication interface

For the sake of the simplicity, the sample will create some fake data in its own local SQlite3 database, the first time it is executed.

Continue reading...

2014, Saturday August 16

Will WebSocket replace HTTP? Does it scale?

You certainly noticed that WebSocket is the current trendy flavor for any modern web framework.
But does it scale? Would it replace HTTP/REST?
There is a feature request ticket about them for mORMot, so here are some thoughts - matter of debate, of course!
I started all this by answering a StackOverflow question, in which the actual answers were not accurate enough, to my opinion.

From my point of view, Websocket - as a protocol - is some kind of monster.

You start a HTTP stateless connection, then switch to WebSocket mode which releases the TCP/IP dual-direction layer, then you may switch later on back to HTTP...
It reminds me some kind of monstrosity, just like encapsulating everything over HTTP, using XML messages... Just to bypass the security barriers... Just breaking the OSI layered model...
It reminds me the fact that our mobile phone data providers do not use broadcasting for streaming audio and video, but regular Internet HTTP servers, so the mobile phone data bandwidth is just wasted when a sport event occurs: every single smart phone has its own connection to the server, and the same video is transmitted in parallel, saturating the single communication channel... Smart phones are not so smart, aren't they?

WebSocket sounds like a clever way to circumvent a limitation...
But why not use a dedicated layer?
I hope HTTP 2.0 would allow pushing information from the server, as part of the standard... and in one decade, we probably will see WebSocket as a deprecated technology.
You have been warned. Do not invest too much in WebSockets..

OK. Back to our existential questions...
First of all, does the WebSocket protocol scale?
Today, any modern single server is able to server millions of clients at once.
Its HTTP server software has just to be is Event-Driven (IOCP) oriented (we are not in the old Apache's one connection = one thread/process equation any more).
Even the HTTP server built in Windows (http.sys - which is used in mORMot) is IOCP oriented and very efficient (running in kernel mode).
From this point of view, there won't be a lot of difference at scaling between WebSocket and a regular HTTP connection. One TCP/IP connection uses a little resource (much less than a thread), and modern OS are optimized for handling a lot of concurrent connections: WebSocket and HTTP are just OSI 7 application layer protocols, inheriting from this TCP/IP specifications.

But, from experiment, I've seen two main problems with WebSocket:

  1. It does not support CDN;
  2. It has potential security issues.

Continue reading...

2014, Monday August 11

Cross-Platform mORMot Clients - Smart Mobile Studio

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

  • Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
  • FreePascal Compiler 2.7.1;
  • Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

Any feedback is welcome in our forum, as usual!

Continue reading...

Cross-Platform mORMot Clients - Delphi / FreePascal

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

  • Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
  • FreePascal Compiler 2.7.1;
  • Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

Any feedback is welcome in our forum, as usual!

Continue reading...

Cross-Platform mORMot Clients - Generating Code Wrappers

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

  • Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
  • FreePascal Compiler 2.7.1;
  • Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

Any feedback is welcome in our forum, as usual!

Continue reading...

Cross-Platform mORMot Clients - Units and Platforms

Current version of the main framework units target only Win32 and Win64 systems.

It allows to make easy self-hosting of mORMot servers for local business applications in any corporation, or pay cheap hosting in the Cloud, since mORMot CPU and RAM expectations are much lower than a regular IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need to create clients for platforms outside the Windows world, especially mobile devices.

A set of cross-platform client units is therefore available in the CrossPlatform sub-folder of the source code repository. It allows writing any client in modern object pascal language, for:

  • Any version of Delphi, on any platform (Mac OSX, or any mobile supported devices);
  • FreePascal Compiler 2.7.1;
  • Smart Mobile Studio 2.1, to create AJAX or mobile applications (via PhoneGap, if needed).

This series of articles will introduce you to mORMot's Cross-Platform abilities:

Any feedback is welcome in our forum, as usual!

Continue reading...

2014, Tuesday August 5

Returning content as XML

By default, interface-based services of a mORMot server will always return a JSON array (or a JSON object, if TServiceFactoryServer.ResultAsJSONObject is true).
With some kind of clients (e.g. if they are made by a third party), it could be useful to return XML content instead.

Your mORMot server is able to let its interface-based services return XML context instead, or in addition to the default JSON format.

Continue reading...

2014, Sunday June 22

Audit-trail for ORM change tracking

Since most CRUD operations are centered within the scope of our mORMot server, we implemented in the ORM an integrated mean of tracking changes (aka Audit Trail) of any TSQLRecord.
In short, our ORM is transformed into a time-machine, just like the good old DeLorean!

Keeping a track of the history of business objects is one very common need for software modeling, and a must-have for any accurate data modeling, like Domain-Driven Design.
By default, as expected by the OOP model, any change to an object will forget any previous state of this object. But thanks to mORMot's exclusive change-tracking feature, you can persist the history of your objects.

Enabling audit-trail

By default, change-tracking feature will be disabled, saving performance and disk use.
But you can enable change tracking for any class, by calling the following method, on server side:

 aServer.TrackChanges([TSQLInvoice]);

This single line will let aServer: TSQLRestServer monitor all CRUD operations, and store all changes of the TSQLInvoice table within a TSQLRecordHistory table.

Continue reading...

2014, Sunday May 18

Automatic JSON serialization of record or dynamic arrays via Enhanced RTTI

Since Delphi 2010, the compiler generates additional RTTI at compilation, so that all record fields are described, and available at runtime.
By the way, this enhanced RTTI is one of the reasons why executables did grow so much in newer versions of the compiler.

Our SynCommons.pas unit is now able to use this enhanced information, and let any record be serialized via RecordLoad() and RecordSave() functions, and all internal JSON marshalling process.

In short, you have nothing to do.
Just use your record as parameters, and, with Delphi 2010 and up, they will be serialized as valid JSON objects.

Of course, text-based definition or callback-based registration are still at hand, and will be used with older versions of Delphi.
But you could be used to by-pass or extend the enhanced-RTTI serialization, even on newer versions of the compiler.

Continue reading...

New sample for JSON performance: mORMot vs SuperObject/XSuperObject/dwsJSON/DBXJSON

We have just added a new "25 - JSON performance" sample to benchmark JSON process, using well most known Delphi libraries...

A new fight
featuring
mORMot vs SuperObject/XSuperObject/dwsJSON/DBXJSON

On mORMot side, it covers TDocVariant, late binding, TSQLTable, ORM, record access, BSON...

We tried to face several scenarios:

  • parse/access/write iteration over a small JSON document,
  • read of deeply nested 680 KB JSON (here mORMot is slower than SO/dwsJSON),
  • read of one 180 MB JSON file (with on-the-fly adaptation to fit a record layout),
  • named access to all rows and columns of a 1 MB JSON table, extracted from a SQL request (with comparison with our ORM performance).

On average and in details, mORMot is the fastest in almost all scenarios (with an amazing performance for table/ORM processing), dwsJSON performs very well (better than SuperObject), and DBXJSON is the slowest (by far, but XE6 version is faster than XE4).

Continue reading...

2014, Wednesday May 7

Benchmarking Mustache libraries: native SynMustache vs mustache.js/SpiderMonkey

I just wrote a small sample program, for benchmarking Mustache libraries: native SynMustache vs mustache.js running on SpiderMonkey 24...

And the winner is ...SynMustache, which is 10 times faster, uses almost no memory during process, and handles inlined {{>partials}} natively (whereas we have to handle them manually with mustache.js)!

Who says that Garbage Collection and immutable strings in modern JITted runtimes are faster than "native" Delphi applications?
Are you still preferring the "NextGen" roadmap?

Continue reading...

MongoDB + mORMot benchmark

Here are some benchmark charts about MongoDB integration in mORMot's ORM.

MongoDB appears as a serious competitor to SQL databases, with the potential benefit of horizontal scaling and installation/administration ease - performance is very high, and its document-based storage fits perfectly with mORMot's advanced ORM features like Shared nothing architecture (or sharding).

Continue reading...

MongoDB + mORMot ORM = ODM

MongoDB (from "humongous") is a cross-platform document-oriented database system, and certainly the best known NoSQL database.
According to http://db-engines.com in April 2014, MongoDB is in 5th place of the most popular types of database management systems, and first place for NoSQL database management systems.
Our mORMot gives premium access to this database, featuring full NoSQL and Object-Document Mapping (ODM) abilities to the framework.

Integration is made at two levels:

  • Direct low-level access to the MongoDB server, in the SynMongoDB.pas unit;
  • Close integration with our ORM (which becomes defacto an ODM), in the mORMotMongoDB.pas unit.

MongoDB eschews the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (MongoDB calls the format BSON), which matches perfectly mORMot's RESTful approach.

This second article will focus on integration of MongoDB with our ORM.

Continue reading...

Direct MongoDB database access

MongoDB (from "humongous") is a cross-platform document-oriented database system, and certainly the best known NoSQL database.
According to http://db-engines.com in April 2014, MongoDB is in 5th place of the most popular types of database management systems, and first place for NoSQL database management systems.
Our mORMot framework gives premium access to this database, featuring full NoSQL and Object-Document Mapping (ODM) abilities to the framework.

Integration is made at two levels:

  • Direct low-level access to the MongoDB server, in the SynMongoDB.pas unit;
  • Close integration with our ORM (which becomes defacto an ODM), in the mORMotMongoDB.pas unit.

MongoDB eschews the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (MongoDB calls the format BSON), which matches perfectly mORMot's RESTful approach.

In this first article, we will detail direct low-level access to the MongoDB server, via the SynMongoDB.pas unit.

Continue reading...

2014, Friday April 18

Introducing mORMot's architecture and design principles

We have just released a set of slides introducing 

  • ORM, SOA, REST, JSON, MVC, MVVM, SOLID, Mocks/Stubs, Domain-Driven Design concepts with Delphi, 
  • and showing some sample code using our Open Source mORMot framework.

You can follow the public link on Google Drive!

This is a great opportunity to discovers some patterns you may not be familiar with, and find out how mORMot try to implement them.
This set of slides may be less intimidating than our huge documentation - do not be terrified by our Online Documentation!
The first set of pages (presenting architecture and design principles) is worth reading.

Feedback is welcome on our forum, as usual.

- page 1 of 4