Cross-Platform mORMot Clients - Units and Platforms

Involved units

The units are the following:

Unit Name	Description
`SynCrossPlatformREST.pas`	Main unit, implementing secured ORM and SOA RESTful client
`SynCrossPlatformCrypto.pas`	SHA-256 and crc32 algorithms, used for authentication
`SynCrossPlatformJSON.pas`	Optimized JSON process (not used by Smart)
`SynCrossPlatformSpecific.pas`	System-specific functions, e.g. HTTP clients

This set of units will provide a solid and shared ground for the any kind of clients:

Connection to a mORMot server via HTTP, with full REST support;
Support of weak or default authentication to secure the transfer - see Authentication;
Definition of the TSQLRecord class, using RTTI when available on Delphi or FreePascal, and generated code for Smart Mobile Studio;
Remote CRUD operations, via JSON and REST, with a TSQLRestClientURI class, with the same methods as with the mORMot.pas framework unit;
Optimized TSQLTableJSON class to handle a JSON result table, as returned by mORMot's REST server ORM - see JSON (not) expanded layouts;
Batch process - see BATCH sequences for adding/updating/deleting records - for transactional and high-speed writes;
Client-Server services via methods with parameters marshaling;
Client-Server services via interfaces with parameters marshaling and instance-life time;
Mapping of most supported field types, including e.g. ISO 8601 date/time encoding, BLOBs and TModTime/TCreateTime - see TSQLRecord fields definition in the SAD 1.18 pdf;
Complex record types are also exported and consumed via JSON, on all platforms (for both ORM and SOA methods);
Some cross-platform low-level functions and types definitions, to help share as much code as possible between your projects.

In the future, C# or Java clients may be written.
The CrossPlatform sub-folder code could be used as reference, to write minimal and efficient clients on any platform. Our REST model is pretty straightforward and standard, and use of JSON tends to leverage a lot of potential marshaling issues which may occur with XML or binary formats.

In practice, a code generator embedded in the mORMot server can be used to create the client wrappers, using the Mustache template engine included on the server side. With a click, you can generate and download a client source file for any supported platform. A set of .mustache templates is available, and can be customized or extended to support any new platform: any help is welcome, especially for targeting Java or C# clients.

Available client platforms

Delphi FMX / FreePascal FCL cross-platform support

Latest versions of Delphi include the FireMonkey (FMX) framework, able to deliver multi-device, true native applications for Windows, Mac OSX, Android and iOS (iPhone/iPad).
Our SynCrossPlatform* units are able to easily create clients for those platforms.

Similarly, these units can be compiled with FreePascal, so that any mORMot server could be consumed from the numerous supported platforms of this compiler.

In order to use those units, ensure in your IDE that the CrossPlatform sub-folder of the mORMot source code repository is defined in your Library Search Path.

Cross-platform JSON

We developed our own cross-platform JSON process unit in SynCrossPlatformJSON.pas, shared with Delphi and FreePascal.
In fact, it appears to be easier to use (since it is variant-based and with late-binding abilities) and run much faster than the official DBXJSON.pas unit shipped with latest versions of Delphi, as stated by the "25 - JSON performance" sample:

 2.2. Table content:
- Synopse crossplatform: 41,135 assertions passed  20.56ms  400,048/s  1.9 MB
- DBXJSON: 41,136 assertions passed  240.84ms  34,159/s  9.9 MB

Our TSQLTableJSON class is more than 10 times faster than standard DBXJSON unit, when processing a list of results as returned by a mORMot server.
The latest value on each line above is the memory consumption. It should be of high interest on mobile platforms, where memory allocation tends to be much slower and sensitive than on Windows (where FastMM4 memory manager does wonders). Our unit consumes 5 times less memory than the RTL's version.

We did not include XSuperObject here, which is cross-platform, but performs even worse than DBXJSON in terms of speed. Other libraries - as SuperObject or dwsJSON - are not cross-platform.
See http://blog.synopse.info/post/json-benchmark-delphi-mormot-superobject-dwsjson-dbxjson for details about this comparison.

A special mention is due to dwsJSON, which performs very well, but only on Windows, and is slower than mORMot's implementation:

- Synopse ORM loop: 41,135 assertions passed  6.18ms  1,330,153/s  1.1 MB
- Synopse ORM list: 41,135 assertions passed  6.47ms  1,270,775/s  952 KB
- Synopse crossplatform: 41,135 assertions passed  20.56ms  400,048/s  1.9 MB
- Super object properties: 41,136 assertions passed  2.20s  3,739/s  6.3 MB
- dwsJSON: 41,136 assertions passed  32.05ms  256,628/s  4.7 MB
- DBXJSON: 41,136 assertions passed  240.84ms  34,159/s  9.9 MB

The "Synopse ORM" lines stand for the TSQLTableJSON class as implemented in mORMot.pas. It uses our optimized UTF-8 functions and classes, in-place escaping together with our RawUTF8 custom string type, so that it is 3 times faster than our cross-platform units, and 40 times than DBXJSON, using much less memory. Some tricks used by Synopse ORM rely on pointers and are not compatible with the NextGen compiler or the official Delphi road-map, so the Synopse crossplatform uses diverse algorithm, but offers still pretty good performance.

This unit features a TJSONVariantData custom variant type, similar to TDocVariant custom variant type, available in the main mORMot framework.
It allows writing such nice and readable code, with late-binding:

var doc: variant;
    json,json2: string;
...
  doc := JSONVariant('{"test":1234,"name":"Joh\"n\r","zero":0.0}');
  assert(doc.test=1234);
  assert(doc.name='Joh"n'#13);
  assert(doc.name2=null);
  assert(doc.zero=0);
  json := doc; // conversion to JSON text when assigned to a string variable
  assert(json='{"test":1234,"name":"Joh\"n\r","zero":0}');
  doc.name2 := 3.1415926;
  doc.name := 'John';
  json := doc;
  assert(json='{"test":1234,"name":"John","zero":0,"name2":3.1415926}');

The unit is also able to serialize any TPersistent class, i.e. all published properties could be written or read from a JSON object representation. It also handles nested objects, stored as TCollection.
See for instance in the SynCrossPlatformTests unit:

type
  TMainNested = class(TCollectionItem)
  private
    fNumber: double;
    fIdent: RawUTF8;
  published
    property Ident: RawUTF8 read fIdent write fIdent;
    property Number: double read fNumber write fNumber;
  end;

  TMain = class(TPersistent)
  private
    fName: RawUTF8;
    fNested: TCollection;
    fList: TStringList;
  public
    constructor Create;
    destructor Destroy; override;
  published
    property Name: RawUTF8 read fName write fName;
    property Nested: TCollection read fNested;
    property List: TStringList read fList;
  end;

  obj1 := TMain.Create;
  obj2 := TMain.Create;
...
  obj1.Name := IntToStr(i);
  item := obj1.Nested.Add as TMainNested;
  item.Ident := obj1.Name;
  item.Number := i/2;
  obj1.list.Add(obj1.Name);
  json := ObjectToJSON(obj1);
  if i=1 then
    assert(json='{"Name":"1","Nested":[{"Ident":"1","Number":0.5}],"List":["1"]}');
  JSONToObject(obj2,json);
  assert(obj2.Nested.Count=i);
  json2 := ObjectToJSON(obj2);
  assert(json2=json);
...

Of course, this serialization feature is used for the TSQLRecord ORM class.

Due to lack of RTTI, record serialization is supported via some functions generated by the server with the code wrappers.

Delphi OSX and NextGen

In order to be compliant with the NextGen revision, our SynCrossPlatform* units follow the expectations of this new family of cross-compilers, which targets Android and iOS.
In particular, we rely only on the string type for text process and storage, even at JSON level, and we tried to make object allocation ARC-compatible. Some types have been defined, e.g. THttpBody, TUTF8Buffer or AnsiChar, to ensure that our units would compile on all supported platforms.

On Delphi, the Indy library is used for HTTP requests. It is cross-platform by nature, so should work on any supported system. For SSL support with iOS and Android clients, please follow instructions at http://blog.marcocantu.com/blog/using_ssl_delphi_ios.html you may also download the needed libcrypto.a and libssl.a files from http://indy.fulgan.com/SSL/OpenSSLStaticLibs.7z

Feedback is needed for the mobile targets, via FMX.
In fact, we rely for our own projects on Smart Mobile Studio for our mobile applications, so the Synopse team did not test Delphi NextGen platforms (i.e. iOS and Android) as deep as other systems. Your input would be very valuable and welcome, here!

FreePascal clients

SynCrossPlatform* units support the FreePascal Compiler, in its 2.7.1 revision.
Most of the code is shared with Delphi, including RTTI support and all supported types.

Some restrictions apply, though.

Due to a bug in FreePascal implementation of variant late binding, the following code won't work as expected:

  doc.name2 := 3.1415926;
  doc.name := 'John';

Under FreePascal, you have to write:

  TJSONVariantData(doc)['name2'] := 3.1415926;
  TJSONVariantData(doc)['name'] := 'John';

In fact, the way late-binding properties are implemented in the FreePascal RTL forbid to modify the content of the associated variant. A private copy of the variant is made, which is not only slower, but disallows modification of its stored value.
Any feedback and help from the FreePascal maintainers may be welcome!

As a result, direct access to TJSONVariantData instances, and not a variant variable, would be faster and less error-prone when using this compiler, until the issue is fixed.

Another issue with the 2.7.1 revision is how the new string type is implemented.
In fact, if you use a string variable containing an UTF-8 encoded text, then the following line would reset the result code page to the system code page:

function StringToJSON(const Text: string): string;
  ...
  result := '"'+copy(Text,1,j-1); // here FPC 2.7.1 erases UTF-8 encoding
  ...

It sounds like if '"' will force the code page of result to be not an UTF-8 content.
With Delphi, this kind of statements work as expected, even for AnsiString values, and '"' constant is handled as RawByteString. We were not able to find an easy and safe workaround for FPC yet. Input is welcome in this area, from any expert!

You have to take care of this limitation, if you target the Windows operating system with FPC (and Lazarus). Under other systems, the default code page is likely to be UTF-8, so in this case our SynCrossPlatform* units will work as expected.

We found out the FreePascal compiler to work very well, and result in small and fast executables. For most common work, timing is comparable with Delphi. The memory manager is less optimized than FastMM4 for rough simple threaded tests, but is cross-platform and much more efficient in multi-thread mode: in fact, it has no giant lock, as FastMM4 suffers.

Smart Mobile Studio support

Smart Mobile Studio - see http://www.smartmobilestudio.com - is a complete RAD environment for writing cutting edge HTML5 mobile applications. It ships with a fully fledged compiler capable of compiling Object Pascal (in a modern dialect call SmartPascal) into highly optimized and raw JavaScript.

There are several solutions able to compile to JavaScript.
In fact, we can find several families of compilers:

JavaScript super-sets, adding optional strong typing, and classes, close to the ECMAScript Sixth Edition: the current main language in this category is certainly TypeScript, designed by Anders Hejlsberg (father of both the Delphi language and C#), and published by Microsoft;
New languages, dedicated to make writing JavaScript programs easier, with an alternative syntax and new concepts (like classes, lambdas, scoping, splats, comprehensions...): most relevant languages of this family are CoffeeScript and Dart;
High-level languages, like Google Web Toolkit (compiling Java code), JSIL (from C# via Mono), or Smart Mobile Studio (from object pascal);
Low-level languages, like Emscripten (compiling C/C++ from LLVM byte-code, using asm.js).

Of course, from our point of view, use of modern object pascal is of great interest, since it will leverage our own coding skills, and make us able to share code between client and server sides.

Beyond JavaScript

The so-called Smart Pascal language brings strong typing, true OOP to JavaScript, including classes, partial classes, interfaces, inheritance, polymorphism, virtual and abstract classes and methods, helpers, closures, lambdas, enumerations and sets, getter/setter expressions, operator overloading, contract programming. But you can still unleash the power of JavaScript (some may say "the good parts"), if needed: the variant type is used to allow dynamic typing, and you can write some JavaScript code as an asm .. end block.
See http://en.wikipedia.org/wiki/The_Smart_Pascal_programming_language

The resulting HTML5 project is self-sufficient with no external JavaScript library, and is compiled as a single index.html file (including its css, if needed). The JavaScript code generated by the compiler (written in Delphi by Eric Grange), is of very high quality, optimized for best execution performance (either in JIT or V8), has low memory consumption, and can be compressed and/or obfuscated.

The SmartCL runtime library encapsulate HTML5 APIs in a set of pure pascal classes and functions, and an IDE with an integrated form designer is available. You can debug your application directly within the IDE (since revision 2.1 - even if it is not yet always stable) or within your browser (IE, Chrome or FireBug have great debuggers), with step-by-step execution of the object pascal code (if you define "Add source map (for debugging)" in Project Options / Linker).

Using a third-party tool like PhoneGap - see http://phonegap.com - you would be able to supply your customers with true native iOS or Android applications, running without any network, and accessing the full power of any modern Smart Phone. Resulting applications will be much smaller in size than the one generated by Delphi FMX (a simple Smart RESTful client with a login form and ORM + SOA tests is zipped as 40 KB), and will work seamlessly on all HTML5 platforms, including most mobile (like Windows Phone, Blackberry, Firefox OS, or webOS) or desktop (Windows, Linux, BSD, MacOS) architectures.

Smart Mobile Studio is therefore a great platform for implementing rich client-side AJAX or Mobile applications, to work with our client-server mORMot framework.

Using Smart Mobile Studio with mORMot

There is no package to be installed within the Smart Mobile Studio IDE. The client units will be generated directly from the mORMot server.
Any edition of Smart - see http://smartmobilestudio.com/feature-matrix - is enough: you do not need to pay for the Enterprise edition to consume mORMot services. But of course, the Professionnal edition is recommended, since the Basic edition does not allow to create forms from the IDE, which is the main point for an AJAX application.

In contrast to the wrappers available in the Entreprise edition of Smart, for accessing RemObjects or DataSnap servers, our mORMot clients are 100% written in the SmartPascal dialect. There is no need to link an external .js library to your executable, and you will benefit of the obfuscation and smart linking features of the Smart compiler.

The only requirement is to copy the mORMot cross-platform units to your Smart Mobile Studio installation. This can be done in three copy instructions:

xcopy SynCrossPlatformSpecific.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y
xcopy SynCrossPlatformCrypto.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y
xcopy SynCrossPlatformREST.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y

You can find a corresponding BATCH file in the CrossPlatform folder, and in SQLite3- SmartMobileStudio ClientCopySynCrossPlatformUnits.bat.

In fact, the SynCrossPlatformJSON.pas unit is not used under Smart Mobile Studio: we use the built-in JSON serialization features of JavaScript, using variant dynamic type, and the standard JSON.Stringify() and JSON.Parse() functions.

Synopse Open Source