Synopse Open Source - Tag - OpenSourcemORMot MVC / SOA / ORM and friends2024-02-02T17:08:25+00:00urn:md5:cc547126eb580a9adbec2349d7c65274DotclearNative X.509, RSA and HSM Supporturn:md5:6404121d596d3c5cee045e29cf93e02b2023-12-09T11:01:00+00:002023-12-10T18:13:06+00:00Arnaud BouchezmORMot FrameworkAsymmetricCrossPlatformCSPRNGDelphiECCed25519forensicFPCFreePascalGoodPracticeinterfacemORMotmORMot2OpenSourceOpenSSLperformancePKCS11RSAsecurityX509<p>Today, almost all computer security relies on asymmetric cryptography and X.509 certificates as file or hardware modules.<br />
And the RSA algorithm is still used to sign the vast majority of those certificates. Even if there are better options (like ECC-256), RSA-2048 seems the actual standard, at least still allowed for a few years.</p>
<p><img src="https://blog.synopse.info?post/public/blog/mormotSecurity.jpg" alt="" /></p>
<p>So we added pure pascal RSA cryptography and X.509 certificates support in <em>mORMot</em>.<br />
Last but not least, we also added Hardware Security Modules support via the PKCS#11 standard.<br />
Until now, we were mostly relying on OpenSSL, but a native embedded solution would be smaller in code size, better for reducing dependencies, and easier to work with (especially for HSM). The main idea is to offer only safe algorithms and methods, so that you can write reliable software, even if you are no cryptographic expert. <img src="https://blog.synopse.info?pf=smile.svg" alt=":)" class="smiley" /></p> <h4>Rivest-Shamir-Adleman (RSA) Public-Key Cryptography</h4>
<p>The RSA public-key algorithm was designed back in 1977, and is still the most widely used. In order to fully implement it, we need to generate new key pairs (public and private keys), then sign and verify (or encrypt or decrypt) data with the key. For instance, a private key is kept secret, and used for an Authority to sign a certificate, and the public key is published, and able to verify a certificate. It is based on large prime numbers, so we needed to develop a Big Integer library, which is not part of Delphi or FPC RTL.</p>
<p><img src="https://blog.synopse.info?post/public/blog/RSAalgo.png" alt="" /></p>
<p>Here as some notes about our implementation in <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.rsa.pas">mormot.crypt.rsa.pas</a>:</p>
<ul>
<li>new pure pascal OOP design of BigInt computation optimized for RSA process;</li>
<li>dedicated x86_64/i386 asm for core computation routines (noticeable speedup);</li>
<li>use half-registers (HalfUInt) for efficient computation on all CPUs (arm and aarch64 numbers are good);</li>
<li>slower than OpenSSL, but likely to be the fastest FPC or Delphi native RSA library, thanks to our optimized asm: for instance, you can generate a new RSA-2048 keypair in less than a second;</li>
<li>internal garbage collection of BigInt instances, to minimize heap pressure during computation, and ensure all values are wiped once used during the process - as proven anti-forensic measure;</li>
<li>includes FIPS-level RSA keypair validation and generation, using a safe random source, with efficient prime number detection, and minimal code size;</li>
<li>features both RSASSA-PKCS1-v1_5 and RSASSA-PSS signature schemes;</li>
<li>started as a fcl-hash fork, but full rewrite inspired by Mbed TLS source because this initial code is slow and incomplete;</li>
<li>references: we followed <a href="https://github.com/Mbed-TLS/mbedtls">the Mbded TLS</a> implementation (which is much easier to follow than OpenSSL), and the well known <a href="https://cacr.uwaterloo.ca/hac/about/chap4.pdf">Handbook of Applied Cryptography (HAC)</a> recommendations;</li>
<li>includes full coverage of unit tests to avoid any regression, validated against the OpenSSL library as audited reference;</li>
<li>this unit will register as <code>Asym</code> 'RS256','RS384','RS512' algorithms (if not overridden by the faster <code>mormot.crypt.openssl</code>), keeping 'RS256-int' and 'PS256-int' available to use our unit;</li>
<li>as used by <code>mormot.crypt.x509</code> (see below) to handle RSA signatures of its X.509 Certificates.</li>
</ul>
<p>For instance, if you want to access a <code>TCryptAsym</code> digital signature instance with RSA-2048 and SHA-256 hashing, you can just use the <code>CryptAsym</code> global variable with <code>caaRS256</code> algorithm as factory.<br />
If you need just public/private key support, you can use <code>CryptPublicKey</code> or <code>CryptPrivateKey</code> factories with <code>ckaRsa</code> algorithm.</p>
<p>About RSA security:</p>
<ul>
<li>RSA-512 or RSA-1024 are considered unsafe and should not be used.</li>
<li>RSA-2048 confers 112-bit of security, and is the usual choice today when this algorithm is to be used.</li>
<li>RSA-3072 could confer 128-bit of security, at the expense of being slower and 50% bigger - so switching to ECC-256 may be a better option, for the same level of security.</li>
<li>RSA-4096 is not worth it in respect to RSA-3072, and RSA-7680 is very big and slow, and only gives 192-bit of security, so should be avoided.</li>
</ul>
<p>Anyway, our library is able to support all those key sizes, up to RSA-7680 is you really need it.<br />
See <a href="https://stackoverflow.com/a/589850/458259">this SO response</a> as reference about RSA keysizes.</p>
<h4>X.509 Certificates</h4>
<p>As we wrote in introduction, X.509 certificates are the base of most computer security.<br />
The whole TLS/HTTPS stack makes use of it, and the whole Internet would collapse without it.</p>
<p>We developed our <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.x509.pas">mormot.crypt.x509.pas</a> unit from scratch, featuring:</p>
<ul>
<li>X.509 Certificates Fields Logic (e.g. X.501 Type Names);</li>
<li>X.509 Certificates and Certificate Signing Request (CSR);</li>
<li>X509 Certificate Revocation List (CRL);</li>
<li>X509 Private Key Infrastructure (PKI);</li>
<li>Registration of our X.509 Engine to the <code>TCryptCert</code>/<code>TCryptStore</code> Factories.</li>
</ul>
<p><img src="https://blog.synopse.info?post/public/blog/X509certificate.png" alt="" /></p>
<p>The raw binary encoding is using the (weird) ASN.1 syntax, which is now implemented as part of the <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.secure.pas">mormot.crypt.secure.pas</a> unit.<br />
We followed the RFC 5280 specifications, and mapped latest X.509 Certificates / CSR / CRL extensions, with some low-level but very readable pascal code using classes, records and enumerates. It features perfect compatibility with our <code>ICryptCert</code> high-level interface wrappers, ready to be used in a very convenient way. We support all basic functions, but also advanced features like open/sealing or text peer information in a human readable format.<br />
When using our unit, your end-user code should not be lost within the complex details and notions of the X.509 format (like OIDs, versions or extensions), but use high-level pascal code, with no possibility to use a weak or invalid configuration.</p>
<p>Of course, it can support not only our new RSA keys, but also ECC-256 as implemented by our native <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.ecc.pas">mormot.crypt.ecc.pas</a>, or any other algorithm, e.g. available from OpenSSL.</p>
<h4>X.509 Private Key Infrastructure (PKI)</h4>
<p>Our unit features a full Private Key Infrastructure (PKI) implementation.<br />
In fact, X.509 certificates are as weak as the PKI they are used on. You can have strong certificates, but a weak verification pattern. In end-user applications, it is typical to see all the security being lost by a poor (e.g. naive) implementation of the keys interaction.</p>
<p><img src="https://blog.synopse.info?post/public/blog/PKI.png" alt="" /></p>
<p>This is why our unit publishes a 'x509-pki' <code>ICryptStore </code> as a full featured PKI:</p>
<ul>
<li>using our <code>TX509</code> and <code>TX509Crl</code> classes for actual certificates process;</li>
<li>clean verification of the chain of trust, with customized depth and proper root Certificate Authority (CA) support, following the RFC 5280 section 6 requirements of a clean "Certification Path Validation";</li>
<li>maintaining a cache of <code>ICryptCert</code> instances, which makes a huge performance benefit in the context of a PKI (e.g. you don't need to parse the X.509 binary, or verify the chain of trust each time).</li>
</ul>
<p>We tried to make performance and usability in the highest possible standards, to let you focus on your business logic, and keep the hard cryptography work done in the <em>mORMot</em> library code.</p>
<h4>Hardware Security Modules (HSM) via PKCS#11</h4>
<p>The PKCS#11 standard is a way to define some software access to Hardware Security Modules, via a set of defined API calls.<br />
We just published the <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.pkcs11.pas">mormot.crypt.pkcs11.pas</a> unit to interface those devices with the other <em>mORMot</em> PKI.</p>
<p><img src="https://blog.synopse.info?post/public/blog/HSM.png" alt="" /></p>
<p>Once you have loaded the library of your actual hardware (typically a <code>.dll</code> or <code>.so</code>) using a <code>TCryptCertAlgoPkcs11</code> instance, you can see all stored certificates and keys, as high-level regular <code>ICryptCert</code> instances, and sign or verify any kind of data (some binary or some other certificates), using the private key safely stored on in the hardware device.<br />
This is usually slower than a pure software verification, but it is much safer, because the private key is sealed within the hardware token, and never leave it. So it can't be intercepted and stolen.</p>
<h4>You are Welcome!</h4>
<p>With those <em>mORMot</em> cryptography units, you now have anything at hand to use standard and proven public-key cryptography in your applications, on both Delphi or FPC, with no external dll deployment issue, and minimal code size increase.<br />
We can thank a lot <a href="https://www.tranquil.it/en">my employer</a> for needing those nice features, therefore letting me work on them.<br />
Open Source rocks! :)</p>Native TLS Support for mORMot 2 REST or WebSockets Serversurn:md5:0d13d921c925b27dacea2bb1bda2c2852022-07-09T11:08:00+01:002022-07-10T06:39:19+01:00Arnaud BouchezmORMot FrameworkAsymmetricFreePascalHTTPhttp.sysHTTPSLinuxmORMotmORMot2OpenSourceOpenSSLperformancePublicKeyRESTSChannelTLSWebSockets<p>Since the beginning, we delegated the TLS encryption support to a reverse proxy server, mainly <a href="https://nginx.org">Nginx</a>. Under Windows, you could setup the http.sys HTTPS layer as usual, as a native - even a bit complicated - solution.<br />
Nginx has several advantages, the first being a proven and efficient technology, with plenty of documentation and configuration tips. It interfaces nicely with Let's Encrypt, and is very good for any regular website, using static content and PHP. This very blog and the <a href="https://synopse.info">Synopse</a> web site is hosted via Ngnix on a small Linux server.</p>
<p><img src="https://blog.synopse.info?post/public/blog/TlsServer.png" alt="" /></p>
<p>But in mORMot 2, we introduced a new set of <a href="https://blog.synopse.info/?post/2022/05/21/New-Async-HTTP/WebSocket-Server-on-mORMot-2">asynchronous web server classes</a>. So stability and performance are not a problem any more. Some benchmarks even consider this server to <a href="https://synopse.info/forum/viewtopic.php?pid=36546#p36546">be faster than nginx</a> (the stability issue mentioned in this post has been fixed in-between).<br />
We just introduced TLS support of our socket-based servers, both the blocking and asynchronous classes. It would use OpenSSL if available, or the SChannel API layer of Windows. Serving HTTPS or WSS with a self-signed certificate is just a matter of a single parameter now, and performance seems pretty good, especially with OpenSSL.</p> <h4>From HTTP to HTTPS</h4>
<p>Here is how you publish a <code>TRestServer</code> instance over HTTP, on port 8888, and with 16 threads for the thread pool:</p>
<pre>
Server := TRestHttpServer.Create([RestServer], '8888', 16);
</pre>
<p>Note the new constructor, easier to use than before, if you just want the default asynchronous server.</p>
<p>And to publish over HTTPS, on the very same port, with a self-signed certificate:</p>
<pre>
Server := TRestHttpServer.Create([RestServer], '8888', 16, secTLSSelfSigned);
</pre>
<p>For a <em>mORMot</em> client, you should also specify that you expect TLS support, and ignore the fact that this self-signed certificate is unknown by the system:</p>
<pre>
Client := TRestHttpClientSocket.Create('127.0.0.1', '8888', Model, {https=}true);
Client.IgnoreTlsCertificateErrors := true;
</pre>
<p>Then, at least with OpenSSL, you could serve TLS 1.3 content from now on, with a safe cipher negotiation by default (which could be tuned if needed).<br />
Nice and easy!</p>
<h4>Give Me the Keys</h4>
<p>And if you generated your own public/private keys pair, you could specify it:</p>
<pre>
Server := TRestHttpServer.Create([RestServer], '8888', 16, secTLS,
HTTPSERVER_DEFAULT_OPTIONS, 'mycert.pem', 'mypriv.pem', 'privpassw');
</pre>
<p>And don't forget to keep your private key... private. :)</p>
<h4>Keys Rule</h4>
<p>Where do those certificates come from? Do I need to read endless and complex OpenSSL command line samples, and mess with files and passwords?<br />
Our framework make it easy. You can now use the new <em>mORMot 2</em> high-level cryptography interfaces to generate the keys you want in simple pascal code.</p>
<p>Here is how the framework generates the self-signed server certificate on OpenSSL, or use a pre-computed one for SChannel:</p>
<pre>
procedure InitNetTlsContextSelfSignedServer(var TLS: TNetTlsContext;
Algo: TCryptAsymAlgo);
var
cert: ICryptCert;
certfile, keyfile: TFileName;
keypass: RawUtf8;
begin
certfile := TemporaryFileName;
if CryptCertAlgoOpenSsl[Algo] = nil then
begin
FileFromString(PrivKeyCertPfx, certfile); // use pre-computed key
keypass := 'pass';
end
else
begin
keyfile := TemporaryFileName;
keypass := CardinalToHexLower(Random32);
cert := CryptCertAlgoOpenSsl[Algo].
Generate(CU_TLS_SERVER, '127.0.0.1', {authority=}nil, 3650);
cert.SaveToFile(certfile, cccCertOnly, '', ccfPem);
cert.SaveToFile(keyfile, cccPrivateKeyOnly, keypass, ccfPem);
//writeln(BinToSource('PRIVKEY_PFX', '',
// cert.Save(cccCertWithPrivateKey, 'pass', ccfBinary)));
end;
InitNetTlsContext(TLS, {server=}true, certfile, keyfile, keypass);
end;
</pre>
<p>As you can see, the <code>ICryptCert</code> interface is very simple to use, and hide all the complexity of X509 and OpenSSL. We provided <code>nil</code> as authority, but you could specify a <code>ICryptCert</code> instance to sign your certificate, if needed.<br />
Under comments in the above source, you can see how to export the keys pair as PKCS#12 certificate, ready to be used for SChannel.</p>
<h4>TLS Everywhere</h4>
<p>Offering TLS as part of your software solution could be a game-changer for serious business, even over a corporate network, with self-signed certificates. It would help your IT and management people trust your <em>mORMot</em> / pascal solution in an heterogeneous and complex mesh of services. Modern object pascal is still on track for the next decades! :)</p>
<p>Feedback is <a href="https://synopse.info/forum/viewtopic.php?id=6291">welcome on our forum</a>, as usual. :)</p>Support of Delphi 10.1 Berlinurn:md5:e51dd652892edaaf7cb9d90bd85f55a52016-04-22T11:29:00+02:002016-04-22T11:29:00+02:00AB4327-GANDIOpen SourceblogDelphiDocumentationEmbarcaderomORMotOpenSourceSynopse <p>You should have noticed that <a href="http://docwiki.embarcadero.com/RADStudio/Berlin/en/What's_New">Delphi 10.1
Berlin</a> has been released.</p>
<p><img src="http://www.blong.com/images/Eggs/Delphi1a.gif" alt="" /></p>
<p>Our <a href="http://synopse.info/fossil/wiki/Synopse+OpenSource">Open Source
projects</a>, including <em>mORMot</em> and <em>SynPDF</em> and their
<a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html">
associated documentation</a> have been updated to support this new
revision.<br />
Any <a href="http://synopse.info/forum/viewtopic.php?id=3280">additional
feedback is welcome</a>, as usual!</p>"SQL and NoSQL", not "SQL vs NoSQL"urn:md5:c22e98fb0648c165bc9ee01f23a34cdd2015-08-23T13:34:00+02:002015-08-23T13:03:13+02:00AB4327-GANDImORMot FrameworkblogDatabaseDelphiJSONMongoDBmORMotNoSQLODMOpenSourceORMSQLSQLite3TDocVariant<p>You know certainly that our <em>mORMot</em> Open Source framework is an ORM,
i.e. mapping objects to a relational / SQL database (<a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_13">Object
Relational Mapping</a>).<br />
You may have followed also that it is able to connect to a <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_83">
NoSQL database</a>, like <a href="https://www.mongodb.org/">MongoDB</a>, and
that the objects are then mapped via an ODM (<a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_82">Object
Document Mapping</a>) - the original SQL SELECT are even <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITLE_207">
translated on the fly to MongoDB queries</a>.</p>
<p>But thanks to <em>mORMot</em>, it is not "<em>SQL vs NoSQL</em>" - but
"<em>SQL and NoSQL</em>".<br />
You are not required to make an exclusive choice.<br />
You can share best of both worlds, depending on your application needs.</p>
<p><img src="http://bodescu.me/wp-content/uploads/2015/02/sql-vs-nosql-450x217.png" alt="" /></p>
<p>In fact, the framework is able to <em>add NoSQL features to a regular
relational / SQL database</em>, by storing JSON documents in TEXT columns.</p>
<p>In your end-user code, you just define a <code>variant</code> field in
the ORM, and store a <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_80">
TDocVariant document</a> within.<br />
We also added some dedicated functions at SQL level, so that
<em>SQLite3</em> could be used as embedded fast engine, and provide
advanced WHERE clauses on this JSON content.</p> <h3>Schemaless storage via a variant</h3>
<p>As we just wrote, a first-class candidate for <em>data sharding</em> in a
<code>TSQLRecord</code> is our <em>TDocVariant custom variant type</em>.</p>
<p>You may define:</p>
<pre>
TSQLRecordData = <strong>class</strong>(TSQLRecord)
<strong>private</strong>
fName: RawUTF8;
fData: <strong>variant</strong>;
<strong>public</strong>
<strong>published</strong>
<strong>property</strong> Name: RawUTF8 <strong>read</strong> fTest <strong>write</strong> fTest <strong>stored</strong> AS_UNIQUE;
<span style="background-color:yellow;"><strong>property</strong> Data: <strong>variant read</strong> fData <strong>write</strong> fData;</span>
<strong>end</strong>;
</pre>
<p>Here, we defined two indexed keys, ready to access any data record:</p>
<ul>
<li>Via the <code>ID: TID</code> property defined at <code>TSQLRecord</code>
level, which will map the <em>SQLite3</em> <code>RowID</code> primary key;</li>
<li>Via the <code>Name: RawUTF8</code> property, which will was marked to be
indexed by setting the "<code>stored AS_UNIQUE</code>" attribute.</li>
</ul>
<p>Then, any kind of data may be stored in the <code>Data: variant</code>
published property. In the database, it will be stored as JSON UTF-8 text,
ready to be retrieved from any client, including AJAX / HTML5
applications.<br />
<em>Delphi</em> clients or servers will access those data via
<em>late-binding</em>, from its <code>TDocVariant</code> instance.</p>
<p>You just reproduced the <em>schema-less</em> approach of the NoSQL database
engines, in a few lines of code! Thanks to the <em>mORMot</em>'s <em><a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_6">
JSON RESTful Client-Server</a></em> design, your applications are able to store
any kind of document, and easily access to them via HTTP.</p>
<p>The documents stored in such a database can have varying sets of fields,
with different types for each field. One could have the following objects in a
single collection of our <code>Data: variant</code> rows:</p>
<pre>
<em>{ name : "Joe", x : 3.3, y : [1,2,3] }</em>
<em>{ name : "Kate", x : "abc" }</em>
<em>{ q : 456 }</em>
</pre>
<p>Of course, when using the database for real problems, the data does have a
fairly consistent structure. Something like the following would be more common,
e.g. for a table persisting <em>student</em> objects:</p>
<pre>
<em>{ name : "Joe", age : 30, interests : "football" }</em>
<em>{ name : "Kate", age : 25 }</em>
</pre>
<p>Generally, there is a direct analogy between this <em>schema-less</em> style
and dynamically typed languages. Constructs such as those above are easy to
represent in <em>PHP</em>, <em>Python</em> and <em>Ruby</em>. And, thanks to
our <code>TDocVariant</code> <em>late-binding</em> magic, even our good
<em>Delphi</em> is able to handle those structures in our code. What we are
trying to do here is make this mapping to the database natural, like:</p>
<pre>
<strong>var</strong> aRec: TSQLRecordData;
aID: TID;
<strong>begin</strong>
<em>// initialization of one record</em>
aRec := TSQLRecordData.Create;
aRec.Name := 'Joe'; <em>// one unique key</em>
aRec.data := _JSONFast('{name:"Joe",age:30}'); <em>// create a TDocVariant</em>
<em>// or we can use this overloaded constructor for simple fields</em>
aRec := TSQLRecordData.Create(['Joe',_ObjFast(['name','Joe','age',30])]);
<em>// now we can play with the data, e.g. via late-binding:</em>
writeln(aRec.Name); <em>// will write 'Joe'</em>
writeln(aRec.Data); <em>// will write '{"name":"Joe","age":30}' (auto-converted to JSON string)</em>
aRec.Data.age := aRec.Data.age+1; <em>// one year older</em>
aRec.Data.interests := 'football'; <em>// add a property to the schema</em>
aID := aClient.Add(aRec,true); <em>// will store {"name":"Joe","age":31,"interests":"footbal"}</em>
aRec.Free;
<em>// now we can retrieve the data either via the aID created integer, or via Name='Joe'</em>
<strong>end</strong>;
</pre>
<p>One of the great benefits of these dynamic objects is that schema migrations
become very easy.<br />
With a traditional RDBMS, releases of code might contain data migration
scripts. Further, each release should have a reverse migration script in case a
rollback is necessary.<br />
<code>ALTER TABLE</code> operations can be very slow and result in scheduled
downtime.</p>
<p>With a <em>schema-less</em> organization of the data, 90% of the time
adjustments to the database become transparent and automatic.<br />
For example, if we wish to add GPA to the <em>student</em> objects, we add the
attribute, re-save, and all is well - if we look up an existing student and
reference GPA, we just get back null.<br />
Furthermore, if we roll back our code, the new GPA fields in the existing
objects are unlikely to cause problems if our code was well written.</p>
<p>In fact, <em>SQlite3</em> is so efficient about its indexes B-TREE storage,
that such a structure may be used as a credible alternative to much heavier
<em>NoSQL</em> engines, like <em>MongoDB</em> or <em>CouchDB</em>.<br />
With the possibility to add some "regular" fields, e.g. plain numbers (like
ahead-computed aggregation values), or text (like a summary or description
field), you can still use any needed fast SQL query, without the complexity of
<em>map/reduce</em> algorithm used by the <em>NoSQL</em> paradigm. You could
even use the <em>Full Text Search</em> - see <em><a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_8">
FTS3/FTS4</a></em> - or RTREE extension advanced features of <em>SQLite3</em>
to perform your queries.<br />
Then, thanks to <em>mORMot</em>'s ability to access any external database
engine, you are able to perform a JOINed query of your <em>schema-less</em>
data with some data stored e.g. in an Oracle, PostgreSQL or MS SQL enterprise
database. Or switch later to a true <em>MongoDB</em> storage, in just one line
of code - see <em><a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_84">
MongoDB + ORM = ODM</a></em>.</p>
<h3>JSON operations from SQL code</h3>
<p>As we stated, any <code>variant</code> field would be serialized as JSON,
then stored as plain TEXT in the database.<br />
In order to make a complex query on the stored JSON, you could retrieve it in
your end-user code, then use the corresponding <code>TDocVariant</code>
instance to perform the search on its content.<br />
Of course, all this has a noticeable performance cost, especially when the data
tend to grow.</p>
<p>The natural way of solving those performance issue is to add some "regular"
RDBMS fields, with a proper index, then perform the requests on those
fields.<br />
But sometimes, you may need to do some addition query, perhaps in conjunction
with "regular" field lookup, on the JSON data stored itself.<br />
In order to avoid the slowest conversion to the ORM client side, we defined
some SQL functions, dedicated to JSON process.</p>
<p>The first is <code>JsonGet()</code>, and is able to extract any value from
the TEXT field, mapping a <code>variant</code>:</p>
<table>
<tbody>
<tr>
<td><code>JsonGet(ArrColumn,0)</code></td>
<td>returns a property value by index, from a JSON array</td>
</tr>
<tr>
<td><code>JsonGet(ObjColumn,'PropName')</code></td>
<td>returns a property value by name, from a JSON object</td>
</tr>
<tr>
<td><code>JsonGet(ObjColumn,'Obj1.Obj2.Prop')</code></td>
<td>returns a property value by path, including nested JSON objects</td>
</tr>
<tr>
<td><code>JsonGet(ObjColumn,'Prop1,Prop2')</code></td>
<td>extract properties by name, from a JSON object</td>
</tr>
<tr>
<td><code>JsonGet(ObjColumn,'Prop1,Obj1.Prop')</code></td>
<td>extract properties by name (including nested JSON objects), from a JSON
object</td>
</tr>
<tr>
<td><code>JsonGet(ObjColumn,'Prop*')</code></td>
<td>extract properties by wildchar name, from a JSON object</td>
</tr>
<tr>
<td><code>JsonGet(ObjColumn,'Prop*,Obj1.P*')</code></td>
<td>extract properties by wildchar name (including nested JSON objects), from a
JSON object</td>
</tr>
</tbody>
</table>
<p>If no value does match, this function would return the SQL
<code>NULL</code>. If the matching value is a simple JSON text or number, it
will be returned as a TEXT, INTEGER or DOUBLE value, ready to be passed as a
result column or any WHERE clause. If the returned value is a nested JSON
object or array, it will be returned as TEXT, serialized as JSON; as a
consequence, you may use it as the source of another <code>JsonGet()</code>
function, or even able to gather the results via the <code>CONCAT()</code>
aggregate function.</p>
<p>The comma-separated syntax allowed in the property name parameter (e.g.
<code>'Prop1,Prop2,Prop3'</code>), would search for several properties at once
in a single object, returning a JSON object of all matching values - e.g.
<code>'{"Prop2":"Value2","Prop3":123}'</code> if the <code>Prop1</code>
property did not appear in the stored JSON object.</p>
<p>If you end the property name with a <code>*</code> character, it would
return a JSON object, with all matching properties.<br />
Any nested object would have its property names be flattened as
{<code>"Obj1.Prop":...}</code>, within the returned JSON object.<br />
Note that the comma-separated syntax also allows such wildchar search, so that
e.g.</p>
<pre>
JsonGet(ObjColumn,'owner') = {"login":"smith","id":123456} as TEXT
JsonGet(ObjColumn,'owner.login') = "smith" as TEXT
JsonGet(ObjColumn,'owner.id') = 123456 as INTEGER
JsonGet(ObjColumn,'owner.name') = NULL
JsonGet(ObjColumn,'owner.login,owner.id') = {"owner.login":"smith","owner.id":123456} as TEXT
JsonGet(ObjColumn,'owner.I*') = {"owner.id:123456} as TEXT
JsonGet(ObjColumn,'owner.*') = {"owner.login":"smith","owner.id":123456} as TEXT
JsonGet(ObjColumn,'unknown.*') = NULL
</pre>
<p>Another function, named <code>JsonHas()</code> is similar to
<code>JsonGet()</code>, but will return TRUE or FALSE depending if the supplied
property (specified by name or index) do exist. It may be faster to use
<code>JsonHas()</code> than <code>JsonGet()</code> e.g. in a WHERE clause, when
you do not want to process this property value, but only return data rows
containing needed information.</p>
<pre>
JsonHas(ObjColumn,'owner') = true
JsonHas(ObjColumn,'owner.login') = true
JsonHas(ObjColumn,'owner.name') = false
JsonHas(ObjColumn,'owner.i*') = true
JsonHas(ObjColumn,'owner.n*') = false
</pre>
<p>Since the process would take place within the <em>SQLite3</em> engine
itself, and since they use a SAX-like fast approach (without any temporary
memory allocation during its search), those JSON functions could be pretty
efficient, and proudly compare to some dedicated NoSQL engines.</p>
<h3>The upcoming official SQlite3 extension</h3>
<p>As you may have noticed <a href="http://www.sqlite.org/src/info/178f9a352c6c9e15">in the SQlite3 timeline</a>,
some extension functions for processing JSON are currently being
implemented.</p>
<p>The <em>mORMot</em> native implementation is very proven, since it uses the
<a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_2">
JSON core of the framework</a>, and is able to handle the <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITLE_39">
"extended" JSON syntax of MongoDB</a>: field names can be serialized
<em>without</em> double quotes, which reduces a lot the storage size, and the
transmission size, when the JSON is stored within a TEXT column, which would
escape all double quote characters.</p>
<p>I guess the performance of our implementation is probably faster than
current <em>SQlite3</em>'s extension.<br />
We use a SAX-like fast approach, without any temporary memory allocation during
its search, whereas AFAIK <em>SQlite3</em>'s extension is doing a lot of
memory allocations, creating all nodes of the JSON documents, like a DOM.</p>
<p>But here again, we are not exclusive guys. If you like this upcoming JSON
extension, you are free to use it! No offense taken!</p>
<p>Feedback <a href="http://synopse.info/forum/viewtopic.php?pid=17448#p17448">is welcome on our
forum</a>, as usual.</p>GetIt "Spirit" Concernsurn:md5:c8befaf992f7ae8f2053d260241925262015-06-06T10:52:00+02:002015-06-06T12:23:00+02:00AB4327-GANDIPascal ProgrammingblogDatabaseDelphiGetItmORMotOpenSourceORMSOA <p>I'm confused by the <a href="http://community.embarcadero.com/index.php/article/news/16027-submitting-libraries-to-getit">
GetIt Submitting official blog page</a>.<br />
Reminds me the darker ages of <a href="http://www.deltics.co.nz/blog/posts/1097">Delphi licensing change of
XE3</a>.</p>
<p><a href="http://blog.marcocantu.com/blog/2015-june-more-info-getit.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+marcocantublog+%28marcocantu.blog%29">
GetIt</a> is the new XE8 package manager for RAD Studio. Information about how
to submit your libraries to GetIt has just been made available by Embarcadero.
The idea behind GetIt is really to make is easier and faster to discover,
install, and keep updated some of the best open source libraries for Delphi and
C++Builder.</p>
<p>When you <a href="http://community.embarcadero.com/index.php/article/news/16027-submitting-libraries-to-getit">
look at the approval conditions</a>, it sounds like if mORMot would not find
its way in this package manager:</p>
<blockquote>
<p>Replacing key capabilities that are part of the core platforms definitions
such as Client/Server FireDAC pack or the DataSnap/EMS Enterprise middleware,
would be less likely to be accepted.<br />
The different SKUs are meant for different types of developers, and multi-tier
capabilities with strong client/server RDBMS integration
<strong>require</strong> an Enterprise edition license.<br />
We will bias acceptance toward GetIt libraries that respect the <strong>spirit
of our licensing and editions</strong>, not just use the letter of the license
and the technical boundaries. If you are unsure about your submission please
check with us first. </p>
</blockquote>
<p>What is this "<em>spirit of our licensing and editions</em>"?<br />
Why is it not part of the official license terms?<br />
Where does this assumption comes from?<br />
Would the licensing conditions change in the close future, as with the XE3
"episode"?<br />
Would Marco's interpretation become the new rule?</p>
<p>It clearly <a href="https://blog.synopse.info?post/post/2012/09/03/Client-Server-allowed-back-to-XE3-pro">reminds me the XE3
time where there was an attempt from Embarcadero to modify their licence
terms</a>, so that third party vendors or Open Source libraries would not be
allowed to create multi-tier frameworks with Delphi!<br />
Is it true that "<em>strong client/server RDBMS integration
<strong>require</strong> an Enterprise edition</em>" ?<br />
Last time I checked the licence terms, it was not stated.<br />
Why on earth would we have to pay for the <em>Entreprise</em> edition, if the
<em>Professionnal</em> edition is all that you need?</p>
<p><img src="http://static.tvtropes.org/pmwiki/pub/images/suicidalstupidity_6656.jpg" alt="" /></p>
<p>Still the same "closed" spirit.<br />
It is like if they know their own n-Tier solution weaknesses, so they try to
avoid any other possibility, but to use their own.<br />
They clearly do not understand the benefit and dynamic of Open Source.</p>
<p>I guess our little <em>mORMot</em> falls directly into this "unwelcomed"
category.<br />
I did not make the submission yet. But should I?<br />
Perhaps sub-part of the framework may find its way in: <a href="http://synopse.info/fossil/wiki?name=PDF+Engine">SynPdf</a>, <a href="http://synopse.info/files/html/api-1.18/SynCrypto.html#">SynCrypto</a>,
<a href="http://synopse.info/files/html/api-1.18/SynGdiPlus.html#">SynGdiPlus</a>,
<a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_45">
SynCommons</a>...<br />
But the main ORM/SOA/REST/MVC/StubMock features would certainly be
rejected.</p>
<p>Our Open Source project is sometimes preferred to DataSnap/EMS (or even
FireDAC), not only for licence cost, but also about features, documentation,
stability, compatibility with older versions of Delphi, performance, and Linux
compatibility.<br />
I have encountered several companies which are still using Delphi <em>because
of <em>mORMot</em></em>: if they did not have found it, they would have moved
to C# or Java, just to be able to use a full SOA/MVC stack, which was not
available, even in the "Enterprise" version of Delphi.</p>
<p>Story repeats itself.<br />
I just wanted to ensure that the licensing terms would not change in that
direction.<br />
I - as many Delphi users - would not let this GetIt "spirit" become the new
rule.<br />
We have to react, as we did for XE3, otherwise we may all suffer!<br />
IMHO Embacardero should better focus on the compiler and IDE, not cutting the
branch on which they are seated...</p>
<p>What do you think? <a href="http://synopse.info/forum/viewtopic.php?id=2623">Comments and feebacks are
welcome</a>!</p>mORMot under Linux thanks to FPCurn:md5:a93fc22b55453318d6aef9fbb4a126f12015-01-10T13:48:00+01:002015-01-10T14:23:06+01:00AB4327-GANDImORMot FrameworkblogCrossPlatformDelphiDocumentationFreePascalinterfaceLinuxmockmORMotOpenSourceORMperformanceSOASourceSQLite3<p>You can use the <em><a href="http://freepascal.org/">FreePascal
Compiler</a></em> (FPC) to compile the <em>mORMot</em> framework source code,
targetting <em>Windows</em> and <em>Linux</em>.</p>
<p><img src="http://www.cloudreviews.com/blog/wp-content/uploads/2012/10/linux.jpg" /></p>
<p><em>Linux</em> is a premium target for cheap and efficient server
<em>Hosting</em>. Since <em>mORMot</em> has no dependency, installing a new
<em>mORMot</em> server is as easy as copying its executable on a blank
<em>Linux</em> host, then run it. No need to install any framework nor runtime.
You could even use diverse operating systems (several <em>Linux</em> or
<em>Windows Server</em> versions) in your <em>mORMot</em> servers farm, with
minimal system requirements, and updates.</p>
<p>We will now see how to write your software with Linux-compiling in mind, and
also give some notes about how to install a Linux Virtual Machine with
<em>Lazarus</em> on your <em>Windows</em> computer, compiling both FPC and
Lazarus from their SVN latest sources!</p> <h3>Compiler expectations</h3>
<p>You should better use the latest SVN trunk version of the FPC 2.7.1 / 3.1.1
compiler, and the corresponding <em>Lazarus</em> IDE.</p>
<p>If you want to use <em>TDocVariant custom variant type</em>, ensure that
your revision includes the fix for <a href="http://mantis.freepascal.org/view.php?id=26773">http://mantis.freepascal.org/view.php?id=26773</a>
bug, i.e. newer than revision 28995 from 2014-11-05T22:17:54. This bug has not
been fixed in 2.6.4 branch.</p>
<p>We recommend using the fpcup tool, as published at <a href="http://wiki.freepascal.org/fpcup">http://wiki.freepascal.org/fpcup</a><br />
To compile the latest svn version of the trunk, just write:</p>
<pre>
fpcup.exe --fpcURL="trunk" --lazURL="trunk"
</pre>
<p>Then ensure you set the static <em>SQlite3</em> .o files for Windows or
Linux in the right folder, as stated about the <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_113">
Installation</a>.</p>
<h3>Creating the missing RTTI for interfaces</h3>
<p>Sadly, we have to face some unresolved FPC compiler-level issue, which does
not supply the needed <code>interface</code> RTTI - see <a href="http://bugs.freepascal.org/view.php?id=26774">http://bugs.freepascal.org/view.php?id=26774</a></p>
<p>As a result, SOA, mock/stub and MVC features would not working
directly.<br />
We propose a workaround to compile such applications with FPC. You could use
Delphi to generate one unit containing the needed information.</p>
<p>The <code>mORMotWrappers.pas</code> unit proposes a
<code>ComputeFPCInterfacesUnit()</code> function, which could be used on Delphi
to generate the RTTI unit for FPC, as such:</p>
<ul>
<li>Ensure that the application will use all its needed <code>interface</code>:
for instance, run all your regression tests, and/or use all its SOA/MVC
features if you are not confident about your test coverage;</li>
<li>Just before the application exists, add a call to
<code>ComputeFPCInterfacesUnit()</code> with the proper folders, e.g. at the
very end of your <code>.dpr</code> code.</li>
</ul>
<p>For instance, here is how <code>TestSQL3.dpr</code> has been modified:</p>
<pre>
<strong>program</strong> TestSQL3;
...
<strong>uses</strong>
...
<span style="background-color:yellow;">mORMotWrappers.pas,</span>
...
<strong>begin</strong>
SQLite3ConsoleTests;
<em>{$ifdef COMPUTEFPCINTERFACES}</em>
ChDir(ExtractFilePath(ParamStr(0)));
<span style="background-color:yellow;">ComputeFPCInterfacesUnit(</span>
<span style="background-color:yellow;">['..\CrossPlatform\templates','..\..\CrossPlatform\templates'],</span>
<span style="background-color:yellow;">'\..\..\SQlite3\TestSQL3FPCInterfaces.pas');</span>
<em>{$endif}</em>
<strong>end</strong>.
</pre>
<p>If you define the <code>COMPUTEFPCINTERFACES</code> conditional, the
<code>TestSQL3FPCInterfaces.pas</code> unit will be generated.<br />
You can take a look at <a href="https://github.com/synopse/mORMot/blob/master/SQLite3/TestSQL3FPCInterfaces.pas">
the committed version of this generated file</a>.</p>
<p>Of course, for your own application, you may use absolute path names: here
we used relative naming, via <code>..\</code>, so that it would work on any
development folder configuration.</p>
<p>Then, add it to any of your uses clause, as such:</p>
<pre>
<strong>uses</strong>
...
<span style="background-color:yellow;">TestSQL3FPCInterfaces, <em>// will register RTTI for interfaces under FPC</em></span>
...
</pre>
<p>This unit will do nothing when compiled under <em>Delphi</em>: it will
register the RTTI only when compiled with <em>FPC</em>.<br />
The rest of your code would be untouched, and could be shared between Delphi
and FPC.</p>
<p>If you do not modify the <code>interface</code> methods definition, this
generation step could be safely bypassed.</p>
<p>We hope that in a close future, the FPC team would fix the <a href="http://bugs.freepascal.org/view.php?id=26774">http://bugs.freepascal.org/view.php?id=26774</a>
issue, but the ticket seems pretty inactive since its creation.</p>
<h3>Writing your project for FPC</h3>
<p>If you want your application to compile with FPC, some little patterns
should be followed.</p>
<p>In all your source code file, the easiest is to including the following
<code>mORMot</code> file, which will define all compiler options and
conditionals as expected:</p>
<pre>
<em>{$I Synopse.inc} // define HASINLINE USETYPEINFO CPU32 CPU64 OWNNORMTOUPPER</em>
</pre>
<p>Then in your <code>.dpr</code> file, you should write:</p>
<pre>
<strong>uses</strong>
<em>{$ifdef Linux}</em>
<em>// if you use threads</em>
cthreads,
<em>// widestring manager for Linux if needed !!</em>
<em>// could also be put in another unit ... but doc states: as early as possible</em>
cwstring, <em>// optional</em>
<em>{$endif}</em>
</pre>
<p>For instance a minimal FPC project to run the regression tests may be:</p>
<pre>
<strong>program</strong> LinuxSynTestFPCLinuxi386;
<br /><em>{$I Synopse.inc}
{$APPTYPE CONSOLE}</em>
<br /><strong>uses</strong> <em>{$ifdef Linux}</em> cthreads, cwstring, <em>{$endif}</em> mORMotSelfTests;
<br /><strong>begin</strong>
SQLite3ConsoleTests;
<strong>end</strong>.
</pre>
<p>In your user code, ensure you do not directly link to the
<code>Windows</code> unit, but rely on the cross-platform classes and functions
as defined in <code>SysUtils.pas</code>, <code>Classes.pas</code> and
<code>SynCommons.pas</code>.<br />
You could find in <code>SynFPCTypInfo.pas</code> and
<code>SynFPCLinux.pas</code> some low-level functions dedicated to FPC and
Linux compilation, to be used with legacy units - your new code should better
rely on higher level functions and classes.</p>
<p>If you rely on <em>mORMot</em> classes and types, e.g. use
<code>RawUTF8</code> for all your <code>string</code> process in the business
logic, and do not use Delphi-specific features (like generics, or new syntax
sugar), it would be very easy to let your application compile with FPC.</p>
<p>In practice, we use Delphi as our main IDE, then switch to Lazarus under a
small integrated <em>Linux VirtualBox</em>, running a low resource
<em>XFCE</em> desktop. We defined the <em>Windows</em> folder containing the
source as a <em>VirtualBox</em> shared folder, so that we are able to compile,
debug and test the <em>Linux</em> version of any executable in native
<em>Linux</em>, on the same computer, from the very same sources. We found out
that <em>Lazarus</em> debugging was pretty smooth on <em>Linux</em> - GDB is
smoother on <em>Linux</em> than <em>Windows</em>, by the way. Then switching
from <em>Delphi/Windows</em> to <em>Lazarus/Linux</em> is direct and natural,
especially when the <em>VirtualBox</em> "Integrated Desktop" feature is
enabled.</p>
<h3>Linux installation tips</h3>
<p>Here are a few informal notes about getting running a FPC/Lazarus virtual
machine running <em>XUbuntu</em>, on a <em>Windows</em> host.<br />
They are published as a general guideline, and we would not provide any
reference procedure, nor support it.</p>
<ul>
<li>Install the latest <em>VirtualBox</em> version from <a href="http://www.virtualbox.org/">http://www.virtualbox.org/</a> to Windows;</li>
<li>Download the latest <code>.iso</code> version published at <a href="http://xubuntu.org/">http://xubuntu.org/</a> or any other place - we use XFCE
since it is a very lightweight desktop, perfect to run <em>Lazarus</em>, and we
selected an Ubuntu LTS revision (14.04 at the time of this writing), which
would be the same used on Internet servers;</li>
<li>Create a new virtual machine (VM) in <em>VirtualBox</em>, with 1 or 2 CPUs,
more than 512 MB of RAM (we use 777 MB), and an automatic-growing disk storage,
with a maximal size of 15 GB; ensure that the disk storage is marked as SSD if
your real host storage is a SSD;</li>
<li>Let the CDROM storage point to the <code>.iso</code> you downloaded;</li>
<li>Start the VM and install <em>Linux</em> locally, as usual - you may select
to download the updated packages during the installation, for safety;</li>
<li>When the system restarts, if it asks for software updates, accept and wait
for the update installation to finish - it is a good idea to have the latest
version of the kernel and libraries before installing the <em>VirtualBox</em>
drivers;</li>
<li>Restart your VM when asked to;</li>
<li>Under a Ubuntu/Debian terminal, write the following commands:</li>
</ul>
<pre>
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install dkms
</pre>
<ul>
<li>Restart the VM, then select "Insert Guest Additions CD image" from the VM
"Devices" menu: a virtual CD would be mounted on your system and appear on your
desktop;</li>
<li>Run the following command, according to your current user name and
<em>VirtualBox</em> version:</li>
</ul>
<pre>
sudo sh /media/...user.../VBOXADDITIONS_..../VBoxLinuxAdditions.run
</pre>
<ul>
<li>Restart the VM, then add a permanent shared folder in the VM configuration,
named <code>Lib</code>, and pointing to your local <em>mORMot</em> installation
(e.g. <code>d:\dev\lib</code>;</li>
<li>Create a void folder, e.g. in your home:</li>
</ul>
<pre>
mkdir lib
</pre>
<ul>
<li>Create a launcher for the following command, to mount the shared folder as
expected:</li>
</ul>
<pre>
sudo mount -t vboxsf lib /home/...user.../lib
</pre>
<ul>
<li>Download the version of <code>fpcup</code> for your system, e.g. <a href="https://bitbucket.org/reiniero/fpcup/downloads/fpcup_linux_x86">https://bitbucket.org/reiniero/fpcup/downloads/fpcup_linux_x86</a></li>
<li>Execute the following commands:</li>
</ul>
<pre>
sudo apt-get install build-essential mingw32-binutils subversion libgtk2.0-dev
sudo ln -s /usr/bin/i586-mingw32msvc-windres /usr/bin/windres
./fpcup_linux_x86 --fpcURL="trunk" --lazURL="trunk"
</pre>
<ul>
<li>Now wait for all source to be retrieved by SVN, then the whole build to
take places;</li>
<li>If you have issues during SVN retrieval, go the
<code>development/fpc</code> folder, then run the following before trying again
the <code>fpcup_linux_x86</code> command:</li>
</ul>
<pre>
svn cleanup
svn update
</pre>
<ul>
<li>On success, you can create a launcher pointing to
<code>development/lazarus/startlazarus</code>.</li>
</ul>
<p>If you followed the above steps, you should now have at least a Lazarus IDE
v1.3 and the corresponding FPC 3.1.1 compiler. It is amazing seeing the whole
compiler + IDE being compiled from the official sources, for free, and in a few
minutes.</p>
<p>In fact, this article has been extracted from our official documentation,
which <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_125">
should be checked for its latest version</a>!</p>
<p>Feedback is <a href="http://synopse.info/forum/viewtopic.php?id=2279">welcome on our forum</a>, as
usual!</p>HTTP remote access for SynDB SQL executionurn:md5:8ac51ed0a6e45f22746c8ba9e6d3e63c2014-11-18T01:08:00+01:002014-11-18T01:11:52+01:00AB4327-GANDIOpen Source librariesarray bindingblogcompressionDatabaseDelphiFirebirdFireDACHTTPhttp.sysHTTPSmORMotMSSQLMySQLOpenSourceOracleORMperformancesecuritySQLSynDBSynDBExplorerTDataSetwebWinHTTPWinINetZEOS<p>For <em>mORMot</em>, we developed a fully feature direct access layer to any
RDBMS, implemented <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_126">
in the SynDB.pas unit</a>.</p>
<p>You can use those <code>SynDB</code> classes to execute any SQL statement,
without any link to the framework ORM.<br />
At reading, the resulting performance is much higher than using the standard
<code>TDataSet</code> component, which is in fact a true performance
bottleneck.<br />
It has genuine features, like <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_136">
column access via late-binding</a>, <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_137">
an innovative <code>ISQLDBRows</code> interface, and ability to directly access
the low-level binary buffers of the database clients.</a></p>
<p><img src="http://vpn-services.bestreviews.net/files/vpn-remote-access.jpg" alt="" /></p>
<p>We just added a nice feature to those classes: the ability to access
remotely, via plain HTTP, to any <code>SynDB</code> supported database!</p> <p>To define a HTTP server, you may write:</p>
<pre>
<strong>uses</strong> SynDB, <em>// RDBMS core</em>
SynDBSQLite3, SynSQLite3Static, <em>// static SQLite3 engine</em>
SynDBRemote; <em>// for HTTP server</em>
...
<strong>var</strong> Props: TSQLDBConnectionProperties;
HttpServer: TSQLDBServerAbstract;
...
Props := TSQLDBSQLite3ConnectionProperties.Create('data.db3','','','');
<span style="background-color:yellow;">HttpServer := TSQLDBServerHttpApi.Create(Props,'remote','8092','user','pass');</span>
</pre>
<p>The above code will initialize a connection to a local <code>data.db3</code>
<em>SQlite3</em> database (in the <code>Props</code> variable), and then
publish it using the <code>http.sys</code> kernel mode HTTP server to the
<code>http://1.2.3.4:8092/remote</code> URI - if the server's IP is
<code>1.2.3.4</code>.</p>
<p>On the client side, you can then write:</p>
<pre>
<strong>uses</strong> SynDB, <em>// RDBMS core</em>
SynDBRemote; <em>// for HTTP server</em>
...
<strong>var</strong> Props: TSQLDBConnectionProperties;
...
Props := TSQLDBWinHTTPConnectionProperties.Create('1.2.3.4:8092','root','user','pass');
</pre>
<p>As you can see, there is no link to <code>SynDBSQLite3.pas</code> nor
<code>SynSQLite3Static.pas</code> on the client side.<br />
Just the HTTP link is needed.<br />
No need to deploy the RDBMS client libraries with your application, nor setup
the local network firewall.</p>
<p>We defined here a single user, with 'user' / 'pass' credentials, but you may
manage more users on the server side, using the <code>Authenticate</code>
property of <code>TSQLDBServerAbstract</code>.<br />
Note that in our remote access, user management does not match the RDBMS user
rights: you should better have your own set of users at application level, for
higher security, and a better integration with your business logic. If creating
a new user on a RDBMS could be painful, it is pretty easy on our
<code>SynDBRemote.pas</code> side.</p>
<p>Then, you execute your favorite SQL using the connection just as usual:</p>
<pre>
<strong>procedure</strong> Test(Props: TSQLDBConnectionProperties);
<strong>var</strong> Stmt: ISQLDBRows;
<strong>begin</strong>
Stmt := Props.Execute('select * from People where YearOfDeath=?',[1519]);
<strong>while</strong> Stmt.Step <strong>do begin</strong>
assert(Stmt.ColumnInt('ID')>0);
assert(Stmt.ColumnInt('YearOfDeath')=1519);
<strong>end</strong>;
<strong>end</strong>;
</pre>
<p>You may even use this remote connection e.g. using a stand-alone shared
<em>SQLite3</em> database as high performance but low maintenance client-server
database engine.</p>
<p>The transmission protocol uses an optimized binary format, which is
compressed and digitally signed on both ends, and the remote user
authentication will be performed via a challenge validation scheme.<br />
You could publish your server over HTTPS, if needed.</p>
<p>Even if you may be tempted to use such remote access to implement a
<em>n-Tier architecture</em>, you should rather use <em>mORMot</em>'s
Client-Server ORM instead. Our little <em>mORMot</em> is not an ORM on
which we added a data transmission layer: it is a full RESTful system, with a
true SOA design.<br />
But for integrating some legacy SQL code into a new architecture,
<code>SynDBRemote.pas</code> may have its benefits, used in conjunction with
<em>mORMot</em>'s higher level features.</p>
<p>Feel free to <a href="http://synopse.info/forum/viewtopic.php?id=2178">post
your feedback on our forum</a>, as usual, or <a href="http://synopse.info/files/html/Synopse%20mORMot%20Framework%20SAD%201.18.html#TITL_131">
browse the updated documentation</a>!</p>MVC/MVVM Web Applicationsurn:md5:0f6dae7b4dfdc4fb48483baae67ae6512014-10-24T19:41:00+02:002020-07-03T09:29:59+02:00AB4327-GANDImORMot FrameworkblogDatabaseDelphiDocumentationHTMLinterfaceJSONmORMotmultithreadMustacheMVCMVVMOpenSourceORMperformanceSOASource<p>We almost finished implementing a <a href="http://synopse.info/fossil/info/bd94c11ab1">long-standing feature request</a>,
introducing MVC / MVVM for Web Applications (like <a href="http://rubyonrails.org/">RubyOnRails</a>) in <em>mORMot</em>.<br />
This is a huge step forward, opening new perspectives not only to our
framework, but for the Delphi community.<br />
In respect to the <a href="http://stackoverflow.com/q/9853766/458259">existing
MVC frameworks for Delphi</a>, our solution is closer to <a href="https://code.google.com/p/delphionrails/">Delphi On Rails</a> (by the
<a href="http://en.wikipedia.org/wiki/Convention_over_configuration">convention-over-configuration</a>
pattern) than the <a href="https://code.google.com/p/delphimvcframework/">Delphi MVC
Framework</a> or <a href="http://xxm.sourceforge.net/">XMM</a>.<br />
The <em>mORMot</em> point of view is unique, and quite powerful,
since it is integrated with other parts of our framework, as its <a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-mORMot-ORM-ODM">ORM/ODM</a> or <a href="https://blog.synopse.info?post/post/2012/03/07/Interface-based-services">interface-based services</a>.<br />
Of course, this is a work in progress, so you are welcome to put your feedback,
patches or new features!</p>
<p><img src="http://d2o0t5hpnwv4c1.cloudfront.net/1122_zend2/images/ash-mvc-architecture.gif" alt="" /></p>
<p>We will now explain how to build a <em>MVC/MVVM web application</em> using
<em>mORMot</em>, starting from the "<em><a href="https://github.com/synopse/mORMot/tree/master/SQLite3/Samples/30%20-%20MVC%20Server">30
- MVC Server</a></em>" sample.<br />
First of all, <a href="https://github.com/synopse/mORMot/tree/master/SQLite3/Samples/30%20-%20MVC%20Server">
check the source in our GitHub repository</a>: two <code>.pas</code> files, and
a set of minimalist <a href="https://blog.synopse.info?post/post/2014/04/28/Mustache-Logic-less-templates-for-Delphi-part-3">Mustache
views</a>.</p>
<p>This little web application publishes a simple BLOG, not fully finished yet
(this is a <em>Sample</em>, remember!).<br />
But you can still execute it in your desktop browser, or any mobile device
(thanks to a simple <em>Bootstrap</em>-based responsive design), and see the
articles list, view one article and its comments, view the author information,
log in and out.</p>
<p>This sample is implemented as such:</p>
<table>
<tbody>
<tr>
<td><strong>MVVM</strong></td>
<td><strong>Source</strong></td>
<td><strong>mORMot</strong></td>
</tr>
<tr>
<td><em>Model</em></td>
<td><code>MVCModel.pas</code></td>
<td><code>TSQLRestServerDB</code> ORM over a <em>SQlite3</em> database</td>
</tr>
<tr>
<td><em>View</em></td>
<td><code>*.html</code></td>
<td><em>Mustache template engine</em> in the <em>Views</em> sub-folder</td>
</tr>
<tr>
<td><em>ViewModel</em></td>
<td><code>MVCViewModel.pas</code></td>
<td>Defined as one <code>IBlogApplication</code> interface</td>
</tr>
</tbody>
</table>
<p>For the sake of the simplicity, the sample will create some fake data in its
own local <em>SQlite3</em> database, the first time it is executed.</p> <p>To discover more, the easiest is to read the <a href="https://blog.synopse.info?post/public/mORMot/mORMotMVC.pdf"><em>mORMot</em> MVC Draft
Documentation</a>!<br />
(which is a chapter extracted from today's version of our <a href="http://synopse.info/files/pdf/Synopse%20mORMot%20Framework%20SAD%201.18.pdf">Software
Architecture Document</a> pdf)</p>
<p>As usual, <a href="http://synopse.info/forum/viewtopic.php?id=2119">feedback
is welcome in our forum</a>!</p>Cross-Platform mORMot Clients - Smart Mobile Studiourn:md5:c91c39eba98fded8fbc28d7f62d2954b2014-08-11T20:44:00+02:002020-07-03T09:29:59+02:00AB4327-GANDImORMot FrameworkAJAXblogCrossPlatformDelphiDocumentationDTOFireMonkeyFreePascalHTMLHTTPHTTPSiOSJavaScriptJSONModelmORMotMustachenativeNextGenOpenSourceORMperformancerecordRestSOASynopseUserInterfaceweb<p>Current version of the main framework units target only <em>Win32</em> and
<em>Win64</em> systems.</p>
<p>It allows to make easy self-hosting of <em>mORMot</em> servers for local
business applications in any corporation, or pay cheap hosting in the Cloud,
since <em>mORMot</em> CPU and RAM expectations are much lower than a regular
<code>IIS-WCF-MSSQL-.Net</code> stack.<br />
But in a <em>Service-Oriented Architecture (SOA)</em>, you would probably need
to create clients for platforms outside the <em>Windows</em> world, especially
mobile devices.<br />
<img src="https://blog.synopse.info?post/public/mORMot/CrossPlatform.jpg" alt="" title="CrossPlatform, Aug 2014" /></p>
<p>A set of cross-platform client units is therefore available in the
<code>CrossPlatform</code> sub-folder of the source code repository. It allows
writing any client in modern <em>object pascal</em> language, for:</p>
<ul>
<li>Any version of <em>Delphi</em>, on any platform (<em>Mac OSX</em>, or any
mobile supported devices);</li>
<li><em>FreePascal</em> Compiler 2.7.1;</li>
<li><em>Smart Mobile Studio</em> 2.1, to create AJAX or mobile applications
(via <em>PhoneGap</em>, if needed).</li>
</ul>
<p>This series of articles will introduce you to <em>mORMot</em>'s
Cross-Platform abilities:</p>
<ul>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Units-Platforms">Units and
platforms</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Generate-Code">Generating
the client code wrappers</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Delphi-FreePascal"><em>Delphi</em> /
<em>FreePascal</em> clients</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/SmartMobileStudio"><em>Smart Mobile
Studio</em> clients</a>.</li>
</ul>
<p>Any feedback is <a href="http://synopse.info/forum/viewtopic.php?id=1939">welcome in our forum</a>, as
usual!</p> <h3>Smart Mobile Studio client samples</h3>
<p>In addition to <em>Delphi</em> and <em>FreePascal</em> clients, our
framework is able to access any <em>mORMot</em> server from HTML5 / AJAX rich
client, thanks to <em>Smart Mobile Studio</em>.</p>
<h3>Adding two numbers in AJAX</h3>
<p>You can find in <code>SQLite3- CrossPlatform ClientsSmartMobileStudio</code>
a simple client for the <code>TServiceCalculator.Add()</code> interface-based
service.<br />
If your <code>Project14ServerHttpWrapper</code> server is running, you can just
point to the supplied <code>wwwhtml</code> file in the sub-folder.<br />
You would then see a web page with a "<code>Server Connect</code>" button, and
if you click on it, you would be able to add two numbers. This a full HTML5 web
application, connecting securely to your <em>mORMot</em> server, which will
work from any desktop browser (on <em>Windows</em>, <em>Mac OS X</em>, or
<em>Linux</em>), or from any mobile device (either <em>iPhone</em> /
<em>iPad</em> / <em>Android</em> / <em>Windows 8 Mobile</em>).</p>
<p>In order to create the application, we just clicked on "<ins>download as
file</ins>" in the <strong>SmartMobileStudio</strong> link in the web page, and
copied the generated file in the source folder of a new <em>Smart Mobile</em>
project.<br />
Of course, we did copy the needed <code>SynCrossPlatform*.pas</code> units from
the <em>mORMot</em> source code tree into the Smart library folder, as stated
above. Just ensure you run <code>CopySynCrossPlatformUnits.bat</code> from the
<code>CrossPlatform</code> folder at least once from the latest revision of the
framework source code.</p>
<p>Then, on the form visual editor, we added a <code>BtnConnect</code> button,
then a <code>PanelCompute</code> panel with two edit fields named
<code>EditA</code> and <code>EditB</code>, and two other buttons, named
<code>BtnComputeAsynch</code> and <code>BtnComputeSynch</code>. A
<code>LabelResult</code> label will be used to display the computation result.
The <code>BtnConnect</code> is a toggle which will show or display the
<code>PanelCompute</code> panel, which is hidden by default, depending on the
connection status.</p>
<p><img src="https://blog.synopse.info?post/public/mORMot/SmartCalculator.png" alt="" title="Smart Mobile Studio Calculator Service Client Sample, Aug 2014" /></p>
<p>In the <code>Form1.pas</code> unit source code side, we added a reference to
our both <code>SynCrossPlatformREST</code> and <code>mORMotClient</code> units,
and some events to the buttons:</p>
<pre>
<strong>unit</strong> Form1;
<strong>interface
uses</strong>
SmartCL.System, SmartCL.Graphics, SmartCL.Components, SmartCL.Forms,
SmartCL.Fonts, SmartCL.Borders, SmartCL.Application, SmartCL.Controls.Panel,
SmartCL.Controls.<strong>Label</strong>, SmartCL.Controls.EditBox, SmartCL.Controls.Button,
<span style="background-color:yellow;">SynCrossPlatformREST, mORMotClient;</span>
<br /><strong>type</strong>
TForm1 = <strong>class</strong>(TW3Form)
<strong>procedure</strong> BtnComputeSynchClick(Sender: TObject);
<strong>procedure</strong> BtnComputeAsynchClick(Sender: TObject);
<strong>procedure</strong> BtnConnectClick(Sender: TObject);
<strong>private</strong>
<em>{$I 'Form1:intf'}</em>
<strong>protected</strong>
<span style="background-color:yellow;">Client: TSQLRestClientURI;</span>
<strong>procedure</strong> InitializeForm; <strong>override</strong>;
<strong>procedure</strong> InitializeObject; <strong>override</strong>;
<strong>procedure</strong> Resize; <strong>override</strong>;
<strong>end</strong>;
</pre>
<p>The <code>BtnConnect</code> event will connect asynchronously to the server,
using <code>'User'</code> as log-on name, and <code>'synopse'</code> as
password (those as the framework defaults).<br />
We just use the <code>GetClient()</code> function, as published in our
generated <code>mORMotClient.pas</code> unit:</p>
<pre>
<em>/// create a TSQLRestClientHTTP instance and connect to the server
// - it will use by default port 888
// - secure connection will be established via TSQLRestServerAuthenticationDefault
// with the supplied credentials
// - request will be asynchronous, and trigger onSuccess or onError event</em>
<strong>procedure</strong> GetClient(<strong>const</strong> aServerAddress, aUserName,aPassword: <strong>string</strong>;
onSuccess, onError: TSQLRestEvent; aServerPort: integer=SERVER_PORT);
</pre>
<p>It uses two callbacks, the first in case of success, and the second
triggered on failure. On success, we will set the global <code>Client</code>
variable with the <code>TSQLRestClientURI</code> instance just created, then
display the two fields and compute buttons:</p>
<pre>
<strong>procedure</strong> TForm1.BtnConnectClick(Sender: TObject);
<strong>begin</strong>
<strong>if</strong> Client=<strong>nil then</strong>
<span style="background-color:yellow;">GetClient('127.0.0.1','User','synopse',</span>
<span style="background-color:yellow;">lambda (aClient: TSQLRestClientURI)</span>
PanelCompute.Visible := true;
W3Label1.Visible := true;
W3Label2.Visible := true;
LabelConnect.Caption := '';
BtnConnect.Caption := 'Disconnect';
LabelResult.Caption := '';
<span style="background-color:yellow;">Client := aClient;</span>
<strong>end</strong>,
lambda
ShowMessage('Impossible to connect to the server!');
<strong>end</strong>)
<strong>else begin</strong>
PanelCompute.Visible := false;
BtnConnect.Caption := 'Server Connect';
Client.Free;
Client := <strong>nil</strong>;
<strong>end</strong>;
<strong>end</strong>;
</pre>
<p>The <code>GetClient()</code> function expects two callbacks, respectively
<code>onSuccess</code> and <code>onError</code>, which are implemented here
with two <em>SmartPascal</em> <code>lambda</code> blocks.</p>
<p>Now that we are connected to the server, let's do some useful
computation!<br />
As you can see in the <code>mORMotClient.pas</code> generated unit, our
interface-based service can be accessed via a <em>SmartPascal</em>
<code>TServiceCalculator</code> class (and not an <code>interface</code>), with
two variations of each methods: one <em>asynchronous</em> method - e.g.
<code>TServiceCalculator.Add()</code> - expecting success/error callbacks, and
one <em>synchronous</em> (blocking) method - e.g.
<code>TServiceCalculator._Add()</code>:</p>
<pre>
<strong>type</strong>
<em>/// service accessible via http://localhost:888/root/Calculator</em>
<em>// - this service will run in sicShared mode</em>
<em>// - synchronous and asynchronous methods are available, depending on use case</em>
<em>// - synchronous _*() methods will block the browser execution, so won't be</em>
<em>// appropriate for long process - on error, they may raise EServiceException</em>
TServiceCalculator = <strong>class</strong>(TServiceClientAbstract)
<strong>public</strong>
<em>/// will initialize an access to the remote service</em>
<strong>constructor</strong> Create(aClient: TSQLRestClientURI); <strong>override</strong>;
<strong>procedure</strong> Add(n1: integer; n2: integer;
onSuccess: <strong>procedure</strong>(Result: integer); onError: TSQLRestEvent);
<strong>function</strong> _Add(<strong>const</strong> n1: integer; <strong>const</strong> n2: integer): integer;
<strong>end</strong>;
</pre>
<p>We can therefore execute asynchronously the <code>Add()</code> service as
such:</p>
<pre>
<strong>procedure</strong> TForm1.BtnComputeAsynchClick(Sender: TObject);
<strong>begin</strong>
TServiceCalculator.Create(Client).Add(
StrToInt(EditA.Text),StrToInt(EditB.Text),
lambda (res: integer)
LabelResult.Caption := format('Result = %d',[res]);
<strong>end</strong>,
lambda
ShowMessage('Error calling the method!');
<strong>end</strong>);
<strong>end</strong>;
</pre>
<p>Or execute synchronously the <code>_Add()</code> service:</p>
<pre>
<strong>procedure</strong> TForm1.BtnComputeSynchClick(Sender: TObject);
<strong>begin</strong>
LabelResult.Caption := format('Result = %d',
[TServiceCalculator.Create(Client)._Add(
StrToInt(EditA.Text),StrToInt(EditB.Text))]);
<strong>end</strong>;
</pre>
<p>Of course, the synchronous code is much easier to follow and maintain. To be
fair, the <em>SmartPascal</em> <code>lambda</code> syntax is not difficult to
read nor write. In the browser debugger, you can easily set a break point
within any <code>lambda</code> block, and debug your code.</p>
<p>Note that if the server is slow to answer, your whole web application will
be unresponsive, and the browser may even complain about the page, proposing
the kill its process!<br />
As a consequence, simple services may be written in a synchronous manner, but
your serious business code should rather use asynchronous callbacks, just as
with any modern AJAX application.</p>
<p>Thanks to the <em>Smart Linking</em> feature of its compiler, only the used
version of the unit will be converted to <em>JavaScript</em> and included in
the final <code>index.html</code> HTML5 file. So having both synchronous and
asynchronous versions of each method at hand is not an issue.</p>
<h3>CRUD/ORM remote access</h3>
<p>If the server did have some ORM model, its <code>TSQLRecord</code> classes
will also be part of the <code>mORMotClient.pas</code> generated unit. All
types, even complex record structures, will be marshaled as expected.</p>
<p>For instance, if you run the <code>RegressionTestsServer.dpr</code> server
(available in the same folder), a much more complete unit could be generated
from <code>http://localhost:888/root/wrapper</code>:</p>
<pre>
<strong>type</strong> <em>// define some enumeration types, used below</em>
TPeopleSexe = (sFemale, sMale);
TRecordEnum = (reOne, reTwo, reLast);
<br /><strong>type</strong> <em>// define some record types, used as properties below</em>
TTestCustomJSONArraySimpleArray = <strong>record</strong>
F: <strong>string</strong>;
G: <strong>array of string</strong>;
H: <strong>record</strong>
H1: integer;
H2: <strong>string</strong>;
H3: <strong>record</strong>
H3a: boolean;
H3b: TSQLRawBlob;
<strong>end</strong>;
<strong>end</strong>;
I: TDateTime;
J: <strong>array of record</strong>
J1: byte;
J2: TGUID;
J3: TRecordEnum;
<strong>end</strong>;
<strong>end</strong>;
<strong>type</strong>
<em>/// service accessible via http://localhost:888/root/Calculator</em>
<em>// - this service will run in sicShared mode</em>
<em>// - synchronous and asynchronous methods are available, depending on use case</em>
<em>// - synchronous _*() methods will block the browser execution, so won't be</em>
<em>// appropriate for long process - on error, they may raise EServiceException</em>
TServiceCalculator = <strong>class</strong>(TServiceClientAbstract)
<strong>public</strong>
<em>/// will initialize an access to the remote service</em>
<strong>constructor</strong> Create(aClient: TSQLRestClientURI); <strong>override</strong>;
<strong>procedure</strong> Add(n1: integer; n2: integer;
onSuccess: <strong>procedure</strong>(Result: integer); onError: TSQLRestEvent);
<strong>function</strong> _Add(<strong>const</strong> n1: integer; <strong>const</strong> n2: integer): integer;
<strong>procedure</strong> ToText(Value: currency; Curr: <strong>string</strong>; Sexe: TPeopleSexe; Name: <strong>string</strong>;
onSuccess: <strong>procedure</strong>(Sexe: TPeopleSexe; Name: <strong>string</strong>); onError: TSQLRestEvent);
<strong>procedure</strong> _ToText(<strong>const</strong> Value: currency; <strong>const</strong> Curr: RawUTF8; <strong>var</strong> Sexe: TPeopleSexe; <strong>var</strong> Name: RawUTF8);
<strong>procedure</strong> RecordToText(Rec: TTestCustomJSONArraySimpleArray;
onSuccess: <strong>procedure</strong>(Rec: TTestCustomJSONArraySimpleArray; Result: <strong>string</strong>); onError: TSQLRestEvent);
<strong>function</strong> _RecordToText(<strong>var</strong> Rec: TTestCustomJSONArraySimpleArray): <strong>string</strong>;
<strong>end</strong>;
<br /> <em>/// map "People" table</em>
TSQLRecordPeople = <strong>class</strong>(TSQLRecord)
<strong>protected</strong>
fFirstName: <strong>string</strong>;
fLastName: <strong>string</strong>;
fData: TSQLRawBlob;
fYearOfBirth: integer;
fYearOfDeath: word;
fSexe: TPeopleSexe;
fSimple: TTestCustomJSONArraySimpleArray;
<em>// those overriden methods will emulate the needed RTTI</em>
<strong>class function</strong> ComputeRTTI: TRTTIPropInfos; <strong>override</strong>;
<strong>procedure</strong> SetProperty(FieldIndex: integer; <strong>const</strong> Value: <strong>variant</strong>); <strong>override</strong>;
<strong>function</strong> GetProperty(FieldIndex: integer): <strong>variant</strong>; <strong>override</strong>;
<strong>public</strong>
<strong>property</strong> FirstName: <strong>string read</strong> fFirstName <strong>write</strong> fFirstName;
<strong>property</strong> LastName: <strong>string read</strong> fLastName <strong>write</strong> fLastName;
<strong>property</strong> Data: TSQLRawBlob <strong>read</strong> fData <strong>write</strong> fData;
<strong>property</strong> YearOfBirth: integer <strong>read</strong> fYearOfBirth <strong>write</strong> fYearOfBirth;
<strong>property</strong> YearOfDeath: word <strong>read</strong> fYearOfDeath <strong>write</strong> fYearOfDeath;
<strong>property</strong> Sexe: TPeopleSexe <strong>read</strong> fSexe <strong>write</strong> fSexe;
<strong>property</strong> Simple: TTestCustomJSONArraySimpleArray <strong>read</strong> fSimple <strong>write</strong> fSimple;
<strong>end</strong>;
</pre>
<p>In the above code, you can see several methods to the
<code>ICalculator</code> service, some involving the complex
<code>TTestCustomJSONArraySimpleArray</code> record type. The
<code>implementation</code> section of the unit will in fact allow
serialization of such records to/from JSON, even with obfuscated
<em>JavaScript</em> field names, via <code>ComputeRTTI() GetProperty()</code>
and <code>SetProperty()</code>.</p>
<p>Some <em>enumerations</em> types are also defined, so will help your
business code be very expressive, thanks to the <em>SmartPascal</em> strong
typing. This is a huge improvement when compared to <em>JavaScript</em> native
weak and dynamic typing.</p>
<p>There is a <code>TSQLRecordPeople</code> class generated, which will map the
following <em>Delphi</em> class type, as defined in the
<code>PeopleServer.pas</code> unit:</p>
<pre>
TSQLRecordPeople = <strong>class</strong>(TSQLRecord)
<strong>protected</strong>
fData: TSQLRawBlob;
fFirstName: RawUTF8;
fLastName: RawUTF8;
fYearOfBirth: integer;
fYearOfDeath: word;
fSexe: TPeopleSexe;
fSimple: TTestCustomJSONArraySimpleArray;
<strong>public</strong>
<strong>class procedure</strong> InternalRegisterCustomProperties(Props: TSQLRecordProperties); <strong>override</strong>;
<strong>published</strong>
<strong>property</strong> FirstName: RawUTF8 <strong>read</strong> fFirstName <strong>write</strong> fFirstName;
<strong>property</strong> LastName: RawUTF8 <strong>read</strong> fLastName <strong>write</strong> fLastName;
<strong>property</strong> Data: TSQLRawBlob <strong>read</strong> fData <strong>write</strong> fData;
<strong>property</strong> YearOfBirth: integer <strong>read</strong> fYearOfBirth <strong>write</strong> fYearOfBirth;
<strong>property</strong> YearOfDeath: word <strong>read</strong> fYearOfDeath <strong>write</strong> fYearOfDeath;
<strong>property</strong> Sexe: TPeopleSexe <strong>read</strong> fSexe <strong>write</strong> fSexe;
<strong>public</strong>
<strong>property</strong> Simple: TTestCustomJSONArraySimpleArray <strong>read</strong> fSimple;
<strong>end</strong>;
</pre>
<p>Here, a complex <code>TTestCustomJSONArraySimpleArray</code> record field
has been published, thanks to a manual
<code>InternalRegisterCustomProperties()</code> registration, as we already
stated above.<br />
You can see that types like <code>RawUTF8</code> were mapped to the standard
<em>SmartPascal</em> <code>string</code> type, as expected, when converted to
the <code>mORMotClient.pas</code> generated unit.</p>
<p>Your AJAX client can then access to this <code>TSQLRecordPeople</code>
content easily, via standard CRUD operations.<br />
See the <code>SQLite3- SmartMobileStudio Client</code> sample, for instance the
following line:</p>
<pre>
<span style="background-color:yellow;">people := new TSQLRecordPeople;</span>
<strong>for</strong> i := 1 <strong>to</strong> 200 <strong>do begin</strong>
<span style="background-color:yellow;">assert(client.Retrieve(i,people));</span>
assert(people.ID=i);
assert(people.FirstName='First'+IntToStr(i));
assert(people.LastName='Last'+IntToStr(i));
assert(people.YearOfBirth=id+1800);
assert(people.YearOfDeath=id+1825);
<strong>end</strong>;
</pre>
<p>Here, the <code>client</code> variable is a <code>TSQLRestClientURI</code>
instance, as returned by the <code>GetClient() onSuccess</code> callback
generated in <code>mORMotClient.pas</code>.<br />
You have <code>Add() Delete() Update() FillPrepare()
CreateAndFillPrepare()</code> and <code>Batch*()</code> methods available,
ready to safely access your data from your AJAX client.</p>
<p>If you update your data model on the server, just re-generate your
<code>mORMotClient.pas</code> unit from
<code>http://localhost:888/root/wrapper</code>, then rebuild your <em>Smart
Mobile Studio</em> project to reflect all changes made to your ORM data model,
or your SOA available services.</p>
<p>Thanks to the <em>SmartPascal</em> strong typing, any breaking change of the
server expectations would immediately be reported at compilation, and not at
runtime, as it would with regular <em>JavaScript</em> clients.</p>Cross-Platform mORMot Clients - Delphi / FreePascalurn:md5:1a7657aad2fbf5869145db279b552ec32014-08-11T20:42:00+02:002020-07-03T09:29:59+02:00AB4327-GANDImORMot FrameworkAJAXblogCrossPlatformDelphiDocumentationDTOFireMonkeyFreePascalHTMLHTTPHTTPSiOSJavaScriptJSONModelmORMotMustachenativeNextGenOpenSourceORMperformancerecordRestSOASynopseUserInterfaceweb<p>Current version of the main framework units target only <em>Win32</em> and
<em>Win64</em> systems.</p>
<p>It allows to make easy self-hosting of <em>mORMot</em> servers for local
business applications in any corporation, or pay cheap hosting in the Cloud,
since <em>mORMot</em> CPU and RAM expectations are much lower than a regular
<code>IIS-WCF-MSSQL-.Net</code> stack.<br />
But in a <em>Service-Oriented Architecture (SOA)</em>, you would probably need
to create clients for platforms outside the <em>Windows</em> world, especially
mobile devices.<br />
<img src="https://blog.synopse.info?post/public/mORMot/CrossPlatform.jpg" alt="" title="CrossPlatform, Aug 2014" /></p>
<p>A set of cross-platform client units is therefore available in the
<code>CrossPlatform</code> sub-folder of the source code repository. It allows
writing any client in modern <em>object pascal</em> language, for:</p>
<ul>
<li>Any version of <em>Delphi</em>, on any platform (<em>Mac OSX</em>, or any
mobile supported devices);</li>
<li><em>FreePascal</em> Compiler 2.7.1;</li>
<li><em>Smart Mobile Studio</em> 2.1, to create AJAX or mobile applications
(via <em>PhoneGap</em>, if needed).</li>
</ul>
<p>This series of articles will introduce you to <em>mORMot</em>'s
Cross-Platform abilities:</p>
<ul>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Units-Platforms">Units and
platforms</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Generate-Code">Generating
the client code wrappers</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Delphi-FreePascal"><em>Delphi</em> /
<em>FreePascal</em> clients</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/SmartMobileStudio"><em>Smart Mobile
Studio</em> clients</a>.</li>
</ul>
<p>Any feedback is <a href="http://synopse.info/forum/viewtopic.php?id=1939">welcome in our forum</a>, as
usual!</p> <h3>Delphi / FreePascal client samples</h3>
<p>The "<code>27 - CrossPlatform ClientsRegressionTests.dpr</code>" sample
creates a <em>mORMot</em> server with its own ORM data model, containing a
<code>TSQLRecordPeople</code> class, and a set of interface-based SOA services,
some including complex types like a record.</p>
<p>Then this sample uses a generated <code>mORMotClient.pas</code>, retrieved
from the "<ins>download as file</ins>" link of the
<strong>CrossPlatform</strong> template above.<br />
Its set of regression tests (written using a small cross-platform
<code>TSynTest</code> unit test class) will then perform remote ORM and SOA
calls to the <code>PeopleServer</code> embedded instance, over all supported
authentication schemes - see <em><a href="https://blog.synopse.info?post/post/2011/05/24/How-to-implement-RESTful-authentication">Authentication</a></em>:</p>
<pre>
Cross Platform Units for mORMot
---------------------------------
<br />1. Running "Iso8601DateTime"
30003 tests passed in 00:00:018
2. Running "Base64Encoding"
304 tests passed in 00:00:000
3. Running "JSON"
18628 tests passed in 00:00:056
4. Running "Model"
1013 tests passed in 00:00:003
5. Running "Cryptography"
4 tests passed in 00:00:000
<br />Tests failed: 0 / 49952
Time elapsed: 00:00:080
<br />Cross Platform Client for mORMot without authentication
---------------------------------------------------------
<br />1. Running "Connection"
2 tests passed in 00:00:010
2. Running "ORM"
4549 tests passed in 00:00:160
3. Running "ORMBatch"
4564 tests passed in 00:00:097
4. Running "Services"
26253 tests passed in 00:00:302
5. Running "CleanUp"
1 tests passed in 00:00:000
<br />Tests failed: 0 / 35369
Time elapsed: 00:00:574
<br />Cross Platform Client for mORMot using TSQLRestServerAuthenticationNone
-------------------------------------------------------------------------
...
<br />Cross Platform Client for mORMot using TSQLRestServerAuthenticationDefault
----------------------------------------------------------------------------
...
</pre>
<p>The generated <code>mORMotClient.pas</code> unit is used for all
"<code>Cross Platform Client</code>" tests above, covering both ORM and SOA
features of the framework.</p>
<h3>Connection to the server</h3>
<p>You could manually connect to a <em>mORMot</em> server as such:</p>
<pre>
<strong>var</strong> Model: TSQLModel;
Client: TSQLRestClientHTTP;
...
Model := TSQLModel.Create([TSQLAuthUser,TSQLAuthGroup,TSQLRecordPeople]);
Client := TSQLRestClientHTTP.Create('localhost',SERVER_PORT,Model);
<strong>if not</strong> Client.Connect <strong>then</strong>
<strong>raise</strong> Exception.Create('Impossible to connect to the server');
<strong>if</strong> Client.ServerTimeStamp=0 <strong>then</strong>
<strong>raise</strong> Exception.Create('Incorrect server');
<strong>if not</strong> Client.SetUser(TSQLRestAuthenticationDefault,'User','synopse') <strong>then</strong>
<strong>raise</strong> Exception.Create('Impossible to authenticate to the server');
...
</pre>
<p>Or you may use the <code>GetClient()</code> function generated in
<code>mORMotClient.pas</code>:</p>
<pre>
<em>/// create a TSQLRestClientHTTP instance and connect to the server
// - it will use by default port 888
// - secure connection will be established via TSQLRestServerAuthenticationDefault
// with the supplied credentials - on connection or authentication error,
// this function will raise a corresponding exception</em>
<strong>function</strong> GetClient(<strong>const</strong> aServerAddress, aUserName,aPassword: <strong>string</strong>;
aServerPort: integer=SERVER_PORT): TSQLRestClientHTTP;
</pre>
<p>Which could be used as such:</p>
<pre>
<strong>var</strong> Client: TSQLRestClientHTTP;
...
Client := GetClient('localhost','User','synopse')
</pre>
<p>The data model and the expected authentication scheme were included in the
<code>GetClient()</code> function, which will raise the expected
<code>ERestException</code> in case of any connection or authentication
issue.</p>
<h3>CRUD/ORM remote access</h3>
<p>Thanks to <code>SynCrossPlatform*</code> units, you could easily perform any
remote ORM operation on your <em>mORMot</em> server, with the usual
<code>TSQLRest</code> CRUD methods.<br />
For instance, the <code>RegressionTests.dpr</code> sample performs the
following operations</p>
<pre>
<span style="background-color:yellow;">fClient.CallBackGet('DropTable',[],Call,TSQLRecordPeople); <em>// call of method-based service</em></span>
check(Call.OutStatus=HTML_SUCCESS);
people := TSQLRecordPeople.Create; <em>// create a record ORM</em>
<strong>try</strong>
<strong>for</strong> i := 1 <strong>to</strong> 200 <strong>do begin</strong>
people.FirstName := 'First'+IntToStr(i);
people.LastName := 'Last'+IntToStr(i);
people.YearOfBirth := i+1800;
people.YearOfDeath := i+1825;
people.Sexe := TPeopleSexe(i <strong>and</strong> 1);
<span style="background-color:yellow;">check(Client.Add(people,true)=i); <em>// add one record</em></span>
<strong>end</strong>;
<strong>finally</strong>
people.Free;
<strong>end</strong>;
...
<span style="background-color:yellow;">people := TSQLRecordPeople.CreateAndFillPrepare(fClient,'',</span>
<span style="background-color:yellow;">'yearofbirth=?',[1900]); <em>// parameterized query returning one or several rows</em></span>
<strong>try</strong>
n := 0;
<span style="background-color:yellow;"><strong>while</strong> people.FillOne <strong>do begin</strong></span>
inc(n);
check(people.ID=100);
check(people.FirstName='First100');
check(people.LastName='Last100');
check(people.YearOfBirth=1900);
check(people.YearOfDeath=1925);
<strong>end</strong>;
check(n=1); <em>// we expected only one record here</em>
<strong>finally</strong>
people.Free;
<strong>end</strong>;
<strong>for</strong> i := 1 <strong>to</strong> 200 <strong>do</strong>
<strong>if</strong> i <strong>and</strong> 15=0 <strong>then</strong>
<span style="background-color:yellow;">fClient.Delete(TSQLRecordPeople,i) <strong>else</strong> <em>// record deletion</em></span>
<strong>if</strong> i <strong>mod</strong> 82=0 <strong>then begin</strong>
people := TSQLRecordPeople.Create;
<strong>try</strong>
id := i+1;
people.ID := i;
people.YearOfBirth := id+1800;
people.YearOfDeath := id+1825;
<span style="background-color:yellow;">check(fClient.Update(people,'YEarOFBIRTH,YEarOfDeath')); <em>// record modification</em></span>
<strong>finally</strong>
people.Free;
<strong>end</strong>;
<strong>end</strong>;
<strong>for</strong> i := 1 <strong>to</strong> 200 <strong>do begin</strong>
<span style="background-color:yellow;">people := TSQLRecordPeople.Create(fClient,i); <em>// retrieve one instance from ID</em></span>
<strong>try</strong>
<strong>if</strong> i <strong>and</strong> 15=0 <strong>then</strong> <em>// was deleted</em>
Check(people.ID=0) <strong>else begin</strong>
<strong>if</strong> i <strong>mod</strong> 82=0 <strong>then</strong>
id := i+1 <strong>else</strong> <em>// was modified</em>
id := i;
Check(people.ID=i);
Check(people.FirstName='First'+IntToStr(i));
Check(people.LastName='Last'+IntToStr(i));
Check(people.YearOfBirth=id+1800);
Check(people.YearOfDeath=id+1825);
Check(ord(people.Sexe)=i <strong>and</strong> 1);
<strong>end</strong>;
<strong>finally</strong>
people.Free;
<strong>end</strong>;
<strong>end</strong>;
</pre>
<p>As we already stated, BATCH mode is also supported, with the classic
<em>mORMot</em> syntax:</p>
<pre>
...
res: TIntegerDynArray;
...
<span style="background-color:yellow;">fClient.BatchStart(TSQLRecordPeople);</span>
people := TSQLRecordPeople.Create;
<strong>try</strong>
<strong>for</strong> i := 1 <strong>to</strong> 200 <strong>do begin</strong>
people.FirstName := 'First'+IntToStr(i);
people.LastName := 'Last'+IntToStr(i);
people.YearOfBirth := i+1800;
people.YearOfDeath := i+1825;
<span style="background-color:yellow;">fClient.BatchAdd(people,true);</span>
<strong>end</strong>;
<strong>finally</strong>
people.Free;
<strong>end</strong>;
<span style="background-color:yellow;">fClient.fBatchSend(res)=HTML_SUCCESS);</span>
check(length(res)=200);
<strong>for</strong> i := 1 <strong>to</strong> 200 <strong>do</strong>
check(res[i-1]=i); <em>// server returned the IDs of the newly created records</em>
</pre>
<p>Those <code>BatchAdd</code> / <code>BatchDelete</code> /
<code>BatchUpdate</code> methods of <code>TSQLRest</code> have the benefit to
introduce at client level:</p>
<ul>
<li>Much higher performance, especially on multi-insertion or multi-update of
data;</li>
<li>Transactional support: <code>TSQLRest.BatchStart()</code> has an optional
<code>AutomaticTransactionPerRow</code> parameter, set to <code>10000</code> by
default, which will create a server-side transaction during the write process,
and an ACID rollback in case of any failure.</li>
</ul>
<p>You can note that all above code has exactly the same structure and methods
than standard <em>mORMot</em> clients.</p>
<p>The generated <code>mORMotClient.pas</code> unit contains all needed
<code>TSQLRecord</code> types, and its used properties, including enumerations
or complex records. The only dependency of this unit are
<code>SynCrossPlatform*</code> units, so would be perfectly cross-platform
(whereas our main <code>SynCommons.pas</code> and <code>mORMot.pas</code> units
do target only <em>Win32</em> and <em>Win64</em>).</p>
<p>As a result, you are able to <em>share</em> server and client code between a
Windows project and any supported platform, even AJAX (see "<em>Smart Mobile
Studio client samples</em>" below). A shared unique code base would eventually
reduce both implementation and debugging time, which is essential to unleash
your business code potential and maximize your ROI.</p>
<h3>Service consumption</h3>
<p>The ultimate goal of the <em>mORMot</em> framework is to publish your
business via a <em>Service-Oriented Architecture (SOA)</em>.<br />
As a consequence, those services should be made available from any kind of
device or platform, even outside the <em>Windows</em> world. The server is able
to generate client wrappers code, which could be used to consume any
<em>Client-Server services via interfaces</em> using any supported
authentication scheme - see <em><a href="https://blog.synopse.info?post/post/2013/06/07/Authentication-and-Authorization">Authentication</a></em>.</p>
<p>Here is an extract of the <code>mORMotClient.pas</code> unit as generated
for the <code>RegressionTests.dpr</code> sample:</p>
<pre>
<strong>type</strong>
<em>/// service implemented by TServiceCalculator</em>
<em>// - you can access this service as such:</em>
<em>// !var aCalculator: ICalculator;</em>
<em>// !begin</em>
<em>// ! aCalculator := TCalculator.Create(aClient);</em>
<em>// ! // now you can use aCalculator methods</em>
<em>// !...</em>
ICalculator = <strong>interface</strong>(IServiceAbstract)
['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}']
<strong>function</strong> Add(<strong>const</strong> n1: integer; <strong>const</strong> n2: integer): integer;
<strong>procedure</strong> ToText(<strong>const</strong> Value: currency; <strong>const</strong> Curr: <strong>string</strong>; <strong>var</strong> Sexe: TPeopleSexe; <strong>var</strong> Name: <strong>string</strong>);
<strong>function</strong> RecordToText(<strong>var</strong> Rec: TTestCustomJSONArraySimpleArray): <strong>string</strong>;
<strong>end</strong>;
<br /> <em>/// implements ICalculator from http://localhost:888/root/Calculator</em>
<em>// - this service will run in sicShared mode</em>
TServiceCalculator = <strong>class</strong>(TServiceClientAbstract,ICalculator)
<strong>public</strong>
<strong>constructor</strong> Create(aClient: TSQLRestClientURI); <strong>override</strong>;
<strong>function</strong> Add(<strong>const</strong> n1: integer; <strong>const</strong> n2: integer): integer;
<strong>procedure</strong> ToText(<strong>const</strong> Value: currency; <strong>const</strong> Curr: <strong>string</strong>; <strong>var</strong> Sexe: TPeopleSexe; <strong>var</strong> Name: <strong>string</strong>);
<strong>function</strong> RecordToText(<strong>var</strong> Rec: TTestCustomJSONArraySimpleArray): <strong>string</strong>;
<strong>end</strong>;
</pre>
<p>As you can see, a dedicated class has been generated to consume the
server-side <code>ICalculator</code> interface-based service, in its own
<code>ICalculator</code> client-side type.<br />
It is able to handle complex types, like enumerations (e.g.
<code>TPeopleSexe</code>) and records (e.g.
<code>TTestCustomJSONArraySimpleArray</code>), which are also defined in the
very same <code>mORMotClient.pas</code> unit.<br />
You can note that the <code>RawUTF8</code> type has been changed into the
standard <em>Delphi</em> / <em>FreePascal</em> <code>string</code> type, since
it is the native type used by our <code>SynCrossPlatformJSON.pas</code> unit
for all its JSON marshaling. Of course, under latest version of <em>Delphi</em>
and <em>FreePascal</em>, this kind of content may be Unicode encoded (either as
UTF-16 for the <code>string</code> = <code>UnicodeString</code> <em>Delphi</em>
type, or as UTF-8 for the <em>FreePascal</em> / <em>Lazarus</em>
<code>string</code> type).</p>
<p>The supplied regression tests show how to use remotely those services:</p>
<pre>
<span style="background-color:yellow;"><strong>var</strong> calc: ICalculator;</span>
i,j: integer;
sex: TPeopleSexe;
name: <strong>string</strong>;
...
<span style="background-color:yellow;">calc := TServiceCalculator.Create(fClient);</span>
check(calc.InstanceImplementation=sicShared);
check(calc.ServiceName='Calculator');
<strong>for</strong> i := 1 <strong>to</strong> 200 <strong>do</strong>
<span style="background-color:yellow;">check(calc.Add(i,i+1)=i*2+1);</span>
<strong>for</strong> i := 1 <strong>to</strong> 200 <strong>do begin</strong>
sex := TPeopleSexe(i <strong>and</strong> 1);
name := 'Smith';
<span style="background-color:yellow;">calc.ToText(i,'$',sex,name);</span>
check(sex=sFemale);
check(name=format('$ %d for %s Smith',[i,SEX_TEXT[i <strong>and</strong> 1]]));
<strong>end</strong>;
...
</pre>
<p>As with regular <em>mORMot</em> client code, a
<code>TServiceCalculator</code> instance is created and is assigned to a
<code>ICalculator</code> local variable. As such, no <code>try ... finally
Calc.Free end</code> block is mandatory here, to avoid any memory leak: the
compiler will create such an hidden block for the <code>Calc:
ICalculator</code> variable scope.</p>
<p>The service-side contract of the <code>ICalculator</code> signature is
retrieved and checked within <code>TServiceCalculator.Create</code>, and would
raise an <code>ERestException</code> if it does not match the contract
identified in <code>mORMotClient.pas</code>.</p>
<p>The cross-platform clients are able to manage the service instance
life-time, especially the <code>sicPerClient</code> mode. In this case, an
implementation class instance will be created on the server for each client,
until the corresponding <code>interface</code> instance will released (i.e. out
of scope or assigned to <code>nil</code>), which will release the server-side
instance - just like with a regular <em>mORMot</em> client code.</p>
<p>Note that all process here is executed <em>synchronously</em>, i.e. in
blocking mode. It is up to you to ensure that your application is able to still
be responsive, even if the server does a lot of process, so may be late to
answer. A dedicated thread may help in this case.</p>Cross-Platform mORMot Clients - Generating Code Wrappersurn:md5:86cc040b427c36a5c9fec81b3bc9a9b32014-08-11T20:41:00+02:002014-08-12T08:54:36+02:00AB4327-GANDImORMot FrameworkAJAXblogCrossPlatformDelphiDocumentationDTOFireMonkeyFreePascalHTMLHTTPHTTPSiOSJavaScriptJSONModelmORMotMustachenativeNextGenOpenSourceORMperformancerecordRestSOASynopseUserInterfaceweb<p>Current version of the main framework units target only <em>Win32</em> and
<em>Win64</em> systems.</p>
<p>It allows to make easy self-hosting of <em>mORMot</em> servers for local
business applications in any corporation, or pay cheap hosting in the Cloud,
since <em>mORMot</em> CPU and RAM expectations are much lower than a regular
<code>IIS-WCF-MSSQL-.Net</code> stack.<br />
But in a <em>Service-Oriented Architecture (SOA)</em>, you would probably need
to create clients for platforms outside the <em>Windows</em> world, especially
mobile devices.<br />
<img src="https://blog.synopse.info?post/public/mORMot/CrossPlatform.jpg" alt="" title="CrossPlatform, Aug 2014" /></p>
<p>A set of cross-platform client units is therefore available in the
<code>CrossPlatform</code> sub-folder of the source code repository. It allows
writing any client in modern <em>object pascal</em> language, for:</p>
<ul>
<li>Any version of <em>Delphi</em>, on any platform (<em>Mac OSX</em>, or any
mobile supported devices);</li>
<li><em>FreePascal</em> Compiler 2.7.1;</li>
<li><em>Smart Mobile Studio</em> 2.1, to create AJAX or mobile applications
(via <em>PhoneGap</em>, if needed).</li>
</ul>
<p>This series of articles will introduce you to <em>mORMot</em>'s
Cross-Platform abilities:</p>
<ul>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Units-Platforms">Units and
platforms</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Generate-Code">Generating
the client code wrappers</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Delphi-FreePascal"><em>Delphi</em> /
<em>FreePascal</em> clients</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/SmartMobileStudio"><em>Smart Mobile
Studio</em> clients</a>.</li>
</ul>
<p>Any feedback is <a href="http://synopse.info/forum/viewtopic.php?id=1939">welcome in our forum</a>, as
usual!</p> <h3>Generating client wrappers</h3>
<p>Even if it feasible to write the client code by hand, your <em>mORMot</em>
server is able to create the source code needed for client access, via a
dedicated method-based service, and set of <em>Mustache</em>-based templates -
see <em><a href="https://blog.synopse.info?post/post/2014/04/28/Mustache-Logic-less-templates-for-Delphi-part-1">Mustache
template engine</a></em>.</p>
<p>The following templates are available in the
<code>CrossPlatformtemplates</code> folder:</p>
<table>
<tbody>
<tr>
<td><strong>Unit Name</strong></td>
<td><strong>Compiler Target</strong></td>
</tr>
<tr>
<td><code>CrossPlatform.pas.mustache</code></td>
<td><em>Delphi</em> / FPC SynCrossPlatform* units</td>
</tr>
<tr>
<td><code>Delphi.pas.mustache</code></td>
<td><em>Delphi</em> Win32/Win64 <em>mORMot</em> units</td>
</tr>
<tr>
<td><code>SmartMobileStudio.pas.mustache</code></td>
<td>Smart Mobile Studio 2.1</td>
</tr>
</tbody>
</table>
<p>In the future, other wrappers may be added. And you can write your own,
which could be included within the framework source! Your input is warmly
welcome, especially if you want to write a template for <em>Java</em> or
<code>C#</code> client. The generated data context already contains the data
types corresponding to those compilers: e.g. a <em>mORMot</em>'s
<code>RawUTF8</code> field or parameter could be identified as
<code>"typeCS":"string"</code> or <code>"typeJava":"String"</code> in addition
to <code>"typeDelphi":"RawUTF8"</code> and
<code>"typePascal":"string"</code>.</p>
<h3>Publishing the code generator</h3>
<p>By default, and for security reasons, the code generation is not embedded to
your <em>mORMot</em> RESTful server. In fact, the
<code>mORMotWrapper.pas</code> unit will link both <code>mORMot.pas</code> and
<code>SynMustache.pas</code> units, and use <em>Mustache</em> templates to
generate code for a given <code>TSQLRestServer</code> instance.</p>
<p>We will start from the interface-based service <em>Sample code</em> as
defined in the<br />
"<code>SQLite3- Interface based services</code>" folder.<br />
After some minor modifications, we copied the server source code into<br />
"<code>SQLite3- CrossPlatform
ClientsProject14ServerHttpWrapper.dpr</code>":</p>
<pre>
<strong>program</strong> Project14ServerHttpWrapper;
<br /><em>{$APPTYPE CONSOLE}</em>
<br /><strong>uses</strong>
SysUtils,
Classes,
SynCommons,
mORMot,
mORMotHttpServer,
<span style="background-color:yellow;">mORMotWrappers,</span>
Project14Interface <strong>in</strong> '..\14 - Interface based services\Project14Interface.pas';
<br /><strong>type</strong>
TServiceCalculator = <strong>class</strong>(TInterfacedObject, ICalculator)
<strong>public</strong>
<strong>function</strong> Add(n1,n2: integer): integer;
<strong>end</strong>;
<br /><strong>function</strong> TServiceCalculator.Add(n1, n2: integer): integer;
<strong>begin</strong>
result := n1+n2;
<strong>end</strong>;
<br /><strong>var</strong>
aModel: TSQLModel;
aServer: TSQLRestServer;
aHTTPServer: TSQLHttpServer;
<strong>begin</strong>
<em>// create a Data Model</em>
aModel := TSQLModel.Create([],ROOT_NAME);
<strong>try</strong>
<em>// initialize a TObjectList-based database engine</em>
aServer := TSQLRestServerFullMemory.Create(aModel,'test.json',false,true);
<strong>try</strong>
<span style="background-color:yellow;"><em>// add the http://localhost:888/root/wrapper code generation web page</em></span>
<span style="background-color:yellow;">AddToServerWrapperMethod(aServer,</span>
<span style="background-color:yellow;">['..\..\..\CrossPlatform\templates','..\..\..\..\CrossPlatform\templates']);</span>
<em>// register our ICalculator service on the server side</em>
aServer.ServiceRegister(TServiceCalculator,[TypeInfo(ICalculator)],sicShared);
<em>// launch the HTTP server</em>
aHTTPServer := TSQLHttpServer.Create(PORT_NAME,[aServer],'+',useHttpApiRegisteringURI);
<strong>try</strong>
aHTTPServer.AccessControlAllowOrigin := '*'; <em>// for AJAX requests to work</em>
writeln(#10'Background server is running.');
writeln('You can test http://localhost:',PORT_NAME,'/wrapper');
writeln(#10'Press [Enter] to close the server.'#10);
readln;
<strong>finally</strong>
aHTTPServer.Free;
<strong>end</strong>;
<strong>finally</strong>
aServer.Free;
<strong>end</strong>;
<strong>finally</strong>
aModel.Free;
<strong>end</strong>;
<strong>end</strong>.
</pre>
<p>As you can see, we just added a reference to the <code>mORMotWrappers</code>
unit, and a call to <code>AddToServerWrapperMethod()</code> in order to publish
the available code generators.</p>
<p>Now, if you run the <code>Project14ServerHttpWrapper</code> server, and
point your favorite browser to <code>http://localhost:888/root/wrapper</code>
you will see the following page:</p>
<blockquote>
<p><strong>Client Wrappers</strong></p>
<p><strong>Available Templates:</strong></p>
<p>* <strong>CrossPlatform<br /></strong><em>mORMotClient.pas</em> -
<ins>download as file</ins> - <ins>see as text</ins> - <ins>see
template</ins></p>
<p>* <strong>Delphi<br /></strong><em>mORMotClient.pas</em> - <ins>download as
file</ins> - <ins>see as text</ins> - <ins>see template</ins></p>
<p>* <strong>SmartMobileStudio<br /></strong><em>mORMotClient.pas</em> -
<ins>download as file</ins> - <ins>see as text</ins> - <ins>see
template</ins></p>
<p>You can also retrieve the corresponding <ins>template context</ins>.</p>
</blockquote>
<p>Each of the <code>*.mustache</code> template available in the specified
folder is listed here. Links above will allow downloading a client source code
unit, or displaying it as text in the browser. The template can also be
displayed un-rendered, for reference. As true <em>Mustache</em> templates, the
source code files are generated from a <em>data context</em>, which can be
displayed, as JSON, from the "<ins>template context</ins>" link. It may help
you when debugging your own templates. Note that if you modify and save a
<code>.mustache</code> template file, just re-load the "<ins>see as text</ins>"
browser page and your modification is taken in account immediately (you do not
need to restart the server).</p>
<p>Generated source code will follow the template name, and here will always be
downloaded as <code>mORMotClient.pas</code>. Of course, you can change the unit
name for your end-user application. It could be even mandatory if the same
client would access to several <em>mORMot</em> servers at once, which could be
the case in a <em>Service-Oriented Architecture (SOA)</em> project.</p>
<p>Just ensure that you will never change the <code>mORMotClient.pas</code>
generated content by hand. If necessary, you can create and customize your own
<em>Mustache</em> template, to be used for your exact purpose. By design, such
automated code generation will require to re-create the client unit each time
the server ORM or SOA structure is modified. In fact, as stated in the
<code>mORMotClient.pas</code> comment, any manual modification of this file may
be lost after regeneration. You have been warned!</p>
<p>If you feel that the current templates have some issues or need some
enhancements, you are very welcome to send us your change requests on our
forums. Once you are used at it, <em>Mustache</em> templates are fairly easy to
work with. Similarly, if you find out that some information is missing in the
generated <em>data context</em>, e.g. for a new platform or language, we would
be pleased to enhance the official <code>mORMotWrapper.pas</code> process.</p>
<h2>Visit our source code repository</h2>
<p>There is nothing better than some real files.</p>
<p>You can take a look at the following files in our source code
repository:</p>
<ul>
<li>The <a href="https://github.com/synopse/mORMot/tree/master/CrossPlatform/templates"><em>.mustache</em>
template files</a>;</li>
<li>The <em><a href="https://github.com/synopse/mORMot/tree/master/CrossPlatform">SynCrossPlatform*.pas</a></em>
units;</li>
<li>A generated <em><a href="https://github.com/synopse/mORMot/blob/master/SQLite3/Samples/27%20-%20CrossPlatform%20Clients/mORMotClient.pas">
mORMotClient.pas</a></em> unit for <em>Delphi</em> / <em>FPC</em> -
generated from <em><a href="https://github.com/synopse/mORMot/tree/master/SQLite3/Samples/27%20-%20CrossPlatform%20Clients">
RegressionTestsServer.dpr</a></em>;</li>
<li>The same <em><a href="https://github.com/synopse/mORMot/blob/master/SQLite3/Samples/29%20-%20SmartMobileStudio%20Client/mORMotClient.pas">
mORMotClient.pas</a></em> unit for <em>Smart Mobile Studio</em> - also
generated from <em><a href="https://github.com/synopse/mORMot/tree/master/SQLite3/Samples/27%20-%20CrossPlatform%20Clients">RegressionTestsServer.dpr</a></em>;</li>
<li>Another simplier <a href="https://github.com/synopse/mORMot/blob/master/SQLite3/Samples/27%20-%20CrossPlatform%20Clients/SmartMobileStudio/mORMotClient.pas">
<em>mORMotClient.pas</em></a> unit for <em>Smart Mobile Studio</em> - generated
from <em><a href="https://github.com/synopse/mORMot/blob/master/SQLite3/Samples/27%20-%20CrossPlatform%20Clients/Project14ServerHttpWrapper.dpr">
Project14ServerHttpWrapper.dpr</a></em>.</li>
</ul>
<div>Enjoy!</div>Cross-Platform mORMot Clients - Units and Platformsurn:md5:150a3105836e67fd082f8e9ff8032f532014-08-11T20:40:00+02:002014-08-12T08:50:34+02:00AB4327-GANDImORMot FrameworkAJAXblogCrossPlatformDelphiDocumentationDTOFireMonkeyFreePascalHTMLHTTPHTTPSiOSJavaScriptJSONModelmORMotMustachenativeNextGenOpenSourceORMperformancerecordRestSOASynopseUserInterfaceweb<p>Current version of the main framework units target only <em>Win32</em> and
<em>Win64</em> systems.</p>
<p>It allows to make easy self-hosting of <em>mORMot</em> servers for local
business applications in any corporation, or pay cheap hosting in the Cloud,
since <em>mORMot</em> CPU and RAM expectations are much lower than a regular
<code>IIS-WCF-MSSQL-.Net</code> stack.<br />
But in a <em>Service-Oriented Architecture (SOA)</em>, you would probably need
to create clients for platforms outside the <em>Windows</em> world, especially
mobile devices.<br />
<img src="https://blog.synopse.info?post/public/mORMot/CrossPlatform.jpg" alt="" title="CrossPlatform, Aug 2014" /></p>
<p>A set of cross-platform client units is therefore available in the
<code>CrossPlatform</code> sub-folder of the source code repository. It allows
writing any client in modern <em>object pascal</em> language, for:</p>
<ul>
<li>Any version of <em>Delphi</em>, on any platform (<em>Mac OSX</em>, or any
mobile supported devices);</li>
<li><em>FreePascal</em> Compiler 2.7.1;</li>
<li><em>Smart Mobile Studio</em> 2.1, to create AJAX or mobile applications
(via <em>PhoneGap</em>, if needed).</li>
</ul>
<p>This series of articles will introduce you to <em>mORMot</em>'s
Cross-Platform abilities:</p>
<ul>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Units-Platforms">Units and
platforms</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Generate-Code">Generating
the client code wrappers</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/Delphi-FreePascal"><em>Delphi</em> /
<em>FreePascal</em> clients</a>;</li>
<li><a href="https://blog.synopse.info?post/post/2014/08/11/CrossPlatform-Clients/SmartMobileStudio"><em>Smart Mobile
Studio</em> clients</a>.</li>
</ul>
<p>Any feedback is <a href="http://synopse.info/forum/viewtopic.php?id=1939">welcome in our forum</a>, as
usual!</p> <h2>Involved units</h2>
The units are the following:
<table>
<tbody>
<tr>
<td><strong>Unit Name</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td><code>SynCrossPlatformREST.pas</code></td>
<td>Main unit, implementing secured ORM and SOA RESTful client</td>
</tr>
<tr>
<td><code>SynCrossPlatformCrypto.pas</code></td>
<td>SHA-256 and crc32 algorithms, used for authentication</td>
</tr>
<tr>
<td><code>SynCrossPlatformJSON.pas</code></td>
<td>Optimized JSON process (not used by <em>Smart</em>)</td>
</tr>
<tr>
<td><code>SynCrossPlatformSpecific.pas</code></td>
<td>System-specific functions, e.g. HTTP clients</td>
</tr>
</tbody>
</table>
<p>This set of units will provide a solid and shared ground for the any kind of
clients:</p>
<ul>
<li>Connection to a <em>mORMot</em> server via HTTP, with full REST
support;</li>
<li>Support of weak or default authentication to secure the transfer - see
<em><a href="https://blog.synopse.info?post/post/2013/06/07/Authentication-and-Authorization">Authentication</a></em>;</li>
<li>Definition of the <code>TSQLRecord</code> class, using RTTI when available
on <em>Delphi</em> or <em>FreePascal</em>, and generated code for <em>Smart
Mobile Studio</em>;</li>
<li>Remote CRUD operations, via JSON and REST, with a
<code>TSQLRestClientURI</code> class, with the same methods as with the
<code>mORMot.pas</code> framework unit;</li>
<li>Optimized <code>TSQLTableJSON</code> class to handle a JSON result table,
as returned by <em>mORMot</em>'s REST server ORM - see <em><a href="https://blog.synopse.info?post/post/2010/07/02/JSON-format-of-a-RESTful-application">JSON (not) expanded
layouts</a></em>;</li>
<li>Batch process - see <em>BATCH sequences for adding/updating/deleting
records</em> - for transactional and high-speed writes;</li>
<li><em>Client-Server services via methods</em> with parameters
marshaling;</li>
<li><em>Client-Server services via interfaces</em> with parameters marshaling
and instance-life time;</li>
<li>Mapping of most supported field types, including e.g. ISO 8601 date/time
encoding, BLOBs and <code>TModTime</code>/<code>TCreateTime</code> - see
<em>TSQLRecord fields definition </em>in the SAD 1.18 pdf;</li>
<li>Complex <code>record</code> types are also exported and consumed via JSON,
on all platforms (for both ORM and SOA methods);</li>
<li>Some cross-platform low-level functions and types definitions, to help
share as much code as possible between your projects.</li>
</ul>
<p>In the future, C# or Java clients may be written.<br />
The <code>CrossPlatform</code> sub-folder code could be used as reference, to
write minimal and efficient clients on any platform. Our REST model is pretty
straightforward and standard, and use of JSON tends to leverage a lot of
potential marshaling issues which may occur with XML or binary formats.</p>
<p>In practice, a code generator embedded in the <em>mORMot</em> server can be
used to create the client wrappers, using the <em>Mustache template engine</em>
included on the server side. With a click, you can generate and download a
client source file for any supported platform. A set of <code>.mustache</code>
templates is available, and can be customized or extended to support any new
platform: any help is welcome, especially for targeting Java or C# clients.</p>
<h2>Available client platforms</h2>
<h3>Delphi FMX / FreePascal FCL cross-platform support</h3>
<p>Latest versions of <em>Delphi</em> include the <em>FireMonkey</em> (FMX)
framework, able to deliver multi-device, true native applications for
<em>Windows</em>, <em>Mac OSX</em>, <em>Android</em> and <em>iOS</em>
(<em>iPhone</em>/<em>iPad</em>).<br />
Our <code>SynCrossPlatform*</code> units are able to easily create clients for
those platforms.</p>
<p>Similarly, these units can be compiled with FreePascal, so that any
<em>mORMot</em> server could be consumed from the numerous supported platforms
of this compiler.</p>
<p>In order to use those units, ensure in your IDE that the
<code>CrossPlatform</code> sub-folder of the <em>mORMot</em> source code
repository is defined in your <em>Library Search Path</em>.</p>
<h3>Cross-platform JSON</h3>
<p>We developed our own cross-platform JSON process unit in
<code>SynCrossPlatformJSON.pas</code>, shared with <em>Delphi</em> and
<em>FreePascal</em>.<br />
In fact, it appears to be easier to use (since it is <code>variant</code>-based
and with <em>late-binding</em> abilities) and run much faster than the official
<code>DBXJSON.pas</code> unit shipped with latest versions of <em>Delphi</em>,
as stated by the "<code>25 - JSON performance</code>" sample:</p>
<pre>
2.2. Table content:
- Synopse crossplatform: 41,135 assertions passed 20.56ms 400,048/s 1.9 MB
- DBXJSON: 41,136 assertions passed 240.84ms 34,159/s 9.9 MB
</pre>
<p>Our <code>TSQLTableJSON</code> class is more than 10 times faster than
standard <code>DBXJSON</code> unit, when processing a list of results as
returned by a <em>mORMot</em> server.<br />
The latest value on each line above is the memory consumption. It should be of
high interest on mobile platforms, where memory allocation tends to be much
slower and sensitive than on Windows (where <em>FastMM4</em> memory manager
does wonders). Our unit consumes 5 times less memory than the RTL's
version.</p>
<p>We did not include <em>XSuperObject</em> here, which is cross-platform, but
performs even worse than <code>DBXJSON</code> in terms of speed. Other
libraries - as <em>SuperObject</em> or <em>dwsJSON</em> - are not
cross-platform.<br />
See <a href="http://blog.synopse.info/post/json-benchmark-delphi-mormot-superobject-dwsjson-dbxjson">
http://blog.synopse.info/post/json-benchmark-delphi-mormot-superobject-dwsjson-dbxjson</a>
for details about this comparison.</p>
<p>A special mention is due to <em>dwsJSON</em>, which performs very well, but
only on Windows, and is slower than <em>mORMot</em>'s implementation:</p>
<pre>
- Synopse ORM loop: 41,135 assertions passed 6.18ms 1,330,153/s 1.1 MB
- Synopse ORM list: 41,135 assertions passed 6.47ms 1,270,775/s 952 KB
- Synopse crossplatform: 41,135 assertions passed 20.56ms 400,048/s 1.9 MB
- Super object properties: 41,136 assertions passed 2.20s 3,739/s 6.3 MB
- dwsJSON: 41,136 assertions passed 32.05ms 256,628/s 4.7 MB
- DBXJSON: 41,136 assertions passed 240.84ms 34,159/s 9.9 MB
</pre>
<p>The "<code>Synopse ORM</code>" lines stand for the
<code>TSQLTableJSON</code> class as implemented in <code>mORMot.pas</code>. It
uses our optimized UTF-8 functions and classes, in-place escaping together with
our <code>RawUTF8</code> custom string type, so that it is 3 times faster than
our cross-platform units, and 40 times than <code>DBXJSON</code>, using much
less memory. Some tricks used by <code>Synopse ORM</code> rely on pointers and
are not compatible with the <em>NextGen</em> compiler or the official
<em>Delphi</em> road-map, so the <code>Synopse crossplatform</code> uses
diverse algorithm, but offers still pretty good performance.</p>
<p>This unit features a <code>TJSONVariantData</code> custom variant type,
similar to <em>TDocVariant custom variant type</em>, available in the main
<em>mORMot</em> framework.<br />
It allows writing such nice and readable code, with late-binding:</p>
<pre>
<strong>var</strong> doc: <strong>variant</strong>;
json,json2: <strong>string</strong>;
...
doc := JSONVariant('{"test":1234,"name":"Joh\"n\r","zero":0.0}');
assert(doc.test=1234);
assert(doc.name='Joh"n'#13);
assert(doc.name2=null);
assert(doc.zero=0);
json := doc; <em>// conversion to JSON text when assigned to a string variable</em>
assert(json='{"test":1234,"name":"Joh\"n\r","zero":0}');
doc.name2 := 3.1415926;
doc.name := 'John';
json := doc;
assert(json='{"test":1234,"name":"John","zero":0,"name2":3.1415926}');
</pre>
<p>The unit is also able to serialize any <code>TPersistent</code> class, i.e.
all published properties could be written or read from a JSON object
representation. It also handles nested objects, stored as
<code>TCollection</code>.<br />
See for instance in the <code>SynCrossPlatformTests</code> unit:</p>
<pre>
<strong>type</strong>
TMainNested = <strong>class</strong>(TCollectionItem)
<strong>private</strong>
fNumber: double;
fIdent: RawUTF8;
<strong>published</strong>
<strong>property</strong> Ident: RawUTF8 <strong>read</strong> fIdent <strong>write</strong> fIdent;
<strong>property</strong> Number: double <strong>read</strong> fNumber <strong>write</strong> fNumber;
<strong>end</strong>;
<br /> TMain = <strong>class</strong>(TPersistent)
<strong>private</strong>
fName: RawUTF8;
fNested: TCollection;
fList: TStringList;
<strong>public</strong>
<strong>constructor</strong> Create;
<strong>destructor</strong> Destroy; <strong>override</strong>;
<strong>published</strong>
<strong>property</strong> Name: RawUTF8 <strong>read</strong> fName <strong>write</strong> fName;
<strong>property</strong> Nested: TCollection <strong>read</strong> fNested;
<strong>property</strong> List: TStringList <strong>read</strong> fList;
<strong>end</strong>;
<br /> obj1 := TMain.Create;
obj2 := TMain.Create;
...
obj1.Name := IntToStr(i);
item := obj1.Nested.Add <strong>as</strong> TMainNested;
item.Ident := obj1.Name;
item.Number := i/2;
obj1.list.Add(obj1.Name);
json := ObjectToJSON(obj1);
<strong>if</strong> i=1 <strong>then</strong>
assert(json='{"Name":"1","Nested":[{"Ident":"1","Number":0.5}],"List":["1"]}');
JSONToObject(obj2,json);
assert(obj2.Nested.Count=i);
json2 := ObjectToJSON(obj2);
assert(json2=json);
...
</pre>
<p>Of course, this serialization feature is used for the
<code>TSQLRecord</code> ORM class.</p>
<p>Due to lack of RTTI, <code>record</code> serialization is supported via some
functions generated by the server with the code wrappers.</p>
<h3>Delphi OSX and NextGen</h3>
<p>In order to be compliant with the <a href="https://blog.synopse.info?post/post/2013/05/11/Delphi-XE4-NextGen-compiler-is-disapointing"><em>NextGen</em></a> revision,
our <code>SynCrossPlatform*</code> units follow the expectations of this new
family of cross-compilers, which targets <em>Android</em> and
<em>iOS</em>.<br />
In particular, we rely only on the <code>string</code> type for text process
and storage, even at JSON level, and we tried to make object allocation
ARC-compatible. Some types have been defined, e.g. <code>THttpBody</code>,
<code>TUTF8Buffer</code> or <code>AnsiChar</code>, to ensure that our units
would compile on all supported platforms.</p>
<p>On <em>Delphi</em>, the <em>Indy</em> library is used for HTTP requests. It
is cross-platform by nature, so should work on any supported system. For SSL
support with <em>iOS</em> and <em>Android</em> clients, please follow
instructions at <a href="http://blog.marcocantu.com/blog/using_ssl_delphi_ios.html">http://blog.marcocantu.com/blog/using_ssl_delphi_ios.html</a>
you may also download the needed <code>libcrypto.a</code> and
<code>libssl.a</code> files from <a href="http://indy.fulgan.com/SSL/OpenSSLStaticLibs.7z">http://indy.fulgan.com/SSL/OpenSSLStaticLibs.7z</a></p>
<p>Feedback is needed for the mobile targets, via FMX.<br />
In fact, we rely for our own projects on <em>Smart Mobile Studio</em> for our
mobile applications, so the <em>Synopse</em> team did not test <em>Delphi
NextGen</em> platforms (i.e. <em>iOS</em> and <em>Android</em>) as deep as
other systems. Your input would be very valuable and welcome, here!</p>
<h3>FreePascal clients</h3>
<p><code>SynCrossPlatform*</code> units support the <em>FreePascal</em>
Compiler, in its 2.7.1 revision.<br />
Most of the code is shared with <em>Delphi</em>, including RTTI support and all
supported types.</p>
<p>Some restrictions apply, though.</p>
<p>Due to a bug in <em>FreePascal</em> implementation of <code>variant</code>
late binding, the following code won't work as expected:</p>
<pre>
doc.name2 := 3.1415926;
doc.name := 'John';
</pre>
<p>Under <em>FreePascal</em>, you have to write:</p>
<pre>
TJSONVariantData(doc)['name2'] := 3.1415926;
TJSONVariantData(doc)['name'] := 'John';
</pre>
<p>In fact, the way late-binding properties are implemented in the
<em>FreePascal</em> RTL forbid to modify the content of the associated
<code>variant</code>. A private copy of the <code>variant</code> is made, which
is not only slower, but disallows modification of its stored value.<br />
Any feedback and help from the <em>FreePascal</em> maintainers may be
welcome!</p>
<p>As a result, direct access to <code>TJSONVariantData</code> instances, and
not a <code>variant</code> variable, would be faster and less error-prone when
using this compiler, until the issue is fixed.</p>
<p>Another issue with the 2.7.1 revision is how the new <code>string</code>
type is implemented.<br />
In fact, if you use a string variable containing an UTF-8 encoded text, then
the following line would reset the result code page to the system code
page:</p>
<pre>
<strong>function</strong> StringToJSON(<strong>const</strong> Text: <strong>string</strong>): <strong>string</strong>;
...
result := '"'+copy(Text,1,j-1); <em>// here FPC 2.7.1 erases UTF-8 encoding</em>
...
</pre>
<p>It sounds like if <code>'"'</code> will force the code page of
<code>result</code> to be not an UTF-8 content.<br />
With <em>Delphi</em>, this kind of statements work as expected, even for
<code>AnsiString</code> values, and <code>'"'</code> constant is handled as
<code>RawByteString</code>. We were not able to find an easy and safe
workaround for FPC yet. Input is welcome in this area, from any expert!</p>
<p>You have to take care of this limitation, if you target the <em>Windows</em>
operating system with FPC (and <em>Lazarus</em>). Under other systems, the
default code page is likely to be UTF-8, so in this case our
<code>SynCrossPlatform*</code> units will work as expected.</p>
<p>We found out the <em>FreePascal</em> compiler to work very well, and result
in small and fast executables. For most common work, timing is comparable with
<em>Delphi</em>. The memory manager is less optimized than <em>FastMM4</em> for
rough simple threaded tests, but is cross-platform and much more efficient in
multi-thread mode: in fact, it has no giant lock, as <em>FastMM4</em>
suffers.</p>
<h2>Smart Mobile Studio support</h2>
<p><em>Smart Mobile Studio</em> - see <a href="http://www.smartmobilestudio.com">http://www.smartmobilestudio.com</a> - is a
complete RAD environment for writing cutting edge HTML5 mobile applications. It
ships with a fully fledged compiler capable of compiling <em>Object Pascal</em>
(in a modern dialect call <em>SmartPascal</em>) into highly optimized and raw
<em>JavaScript</em>.</p>
<p>There are several solutions able to compile to <em>JavaScript</em>.<br />
In fact, we can find several families of compilers:</p>
<ul>
<li><em>JavaScript</em> super-sets, adding optional <em>strong typing</em>, and
classes, close to the <em>ECMAScript Sixth Edition</em>: the current main
language in this category is certainly <em><a href="http://www.typescriptlang.org/">TypeScript</a></em>, designed by Anders
Hejlsberg (father of both the <em>Delphi</em> language and <em>C#</em>), and
published by <em>Microsoft</em>;</li>
<li>New languages, dedicated to make writing <em>JavaScript</em> programs
easier, with an alternative syntax and new concepts (like classes, lambdas,
scoping, splats, comprehensions...): most relevant languages of this family are
<em><a href="http://coffeescript.org/">CoffeeScript</a></em> and <em><a href="https://www.dartlang.org/">Dart</a></em>;</li>
<li>High-level languages, like <em><a href="http://www.gwtproject.org/">Google
Web Toolkit</a></em> (compiling <em>Java</em> code), <em><a href="http://jsil.org/">JSIL</a></em> (from <em>C#</em> via <em>Mono</em>), or
<em><a href="http://smartmobilestudio.com/">Smart Mobile Studio</a></em> (from
<em>object pascal</em>);</li>
<li>Low-level languages, like <em><a href="https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Emscripten">Emscripten</a></em>
(compiling C/C++ from LLVM byte-code, using <em>asm.js</em>).</li>
</ul>
<p>Of course, from our point of view, use of modern <em>object pascal</em> is
of great interest, since it will leverage our own coding skills, and make us
able to share code between client and server sides.</p>
<h3>Beyond JavaScript</h3>
<p>The so-called <em><a href="http://en.wikipedia.org/wiki/The_Smart_Pascal_programming_language">Smart
Pascal</a></em> language brings strong typing, true OOP to <em>JavaScript</em>,
including classes, partial classes, interfaces, inheritance, polymorphism,
virtual and abstract classes and methods, helpers, closures, lambdas,
enumerations and sets, getter/setter expressions, operator overloading,
contract programming. But you can still unleash the power of
<em>JavaScript</em> (some may say "the good parts"), if needed: the
<code>variant</code> type is used to allow dynamic typing, and you can write
some <em>JavaScript</em> code as an <code>asm .. end</code> block.<br />
See <a href="http://en.wikipedia.org/wiki/The_Smart_Pascal_programming_language">http://en.wikipedia.org/wiki/The_Smart_Pascal_programming_language</a></p>
<p>The resulting HTML5 project is self-sufficient with no external
<em>JavaScript</em> library, and is compiled as a single
<code>index.html</code> file (including its <code>css</code>, if needed). The
<em>JavaScript</em> code generated by the compiler (written in <em>Delphi</em>
by Eric Grange), is of very high quality, optimized for best execution
performance (either in JIT or V8), has low memory consumption, and can be
compressed and/or obfuscated.</p>
<p>The <code>SmartCL</code> runtime library encapsulate HTML5 APIs in a set of
pure pascal classes and functions, and an IDE with an integrated form designer
is available. You can debug your application directly within the IDE (since
revision 2.1 - even if it is not yet always stable) or within your browser (IE,
Chrome or FireBug have great debuggers), with step-by-step execution of the
object pascal code (if you define "<em>Add source map (for debugging)</em>" in
<code>Project Options</code> / <code>Linker</code>).</p>
<p>Using a third-party tool like <em>PhoneGap</em> - see <a href="http://phonegap.com">http://phonegap.com</a> - you would be able to supply
your customers with true native <em>iOS</em> or <em>Android</em> applications,
running without any network, and accessing the full power of any modern
<em>Smart Phone</em>. Resulting applications will be much smaller in size than
the one generated by <em>Delphi</em> FMX (a simple <em>Smart</em> RESTful
client with a login form and ORM + SOA tests is zipped as 40 KB), and will work
seamlessly on all HTML5 platforms, including most mobile (like Windows Phone,
Blackberry, Firefox OS, or webOS) or desktop (Windows, Linux, BSD, MacOS)
architectures.</p>
<p><em>Smart Mobile Studio</em> is therefore a great platform for implementing
rich client-side AJAX or <em>Mobile</em> applications, to work with our
client-server <em>mORMot</em> framework.</p>
<h3>Using Smart Mobile Studio with mORMot</h3>
<p>There is no package to be installed within the <em>Smart Mobile Studio</em>
IDE. The client units will be generated directly from the <em>mORMot</em>
server.<br />
Any edition of <em>Smart</em> - see <a href="http://smartmobilestudio.com/feature-matrix">http://smartmobilestudio.com/feature-matrix</a>
- is enough: you do not need to pay for the <em>Enterprise</em> edition to
consume <em>mORMot</em> services. But of course, the <em>Professionnal</em>
edition is recommended, since the <em>Basic</em> edition does not allow to
create forms from the IDE, which is the main point for an AJAX application.</p>
<p>In contrast to the wrappers available in the
<em>Entreprise </em>edition of Smart, for accessing <em>RemObjects</em> or
<em>DataSnap</em> servers, our <em>mORMot</em> clients are 100% written in the
<em>SmartPascal</em> dialect. There is no need to link an external
<code>.js</code> library to your executable, and you will benefit of the
obfuscation and smart linking features of the Smart compiler.</p>
<p>The only requirement is to copy the <em>mORMot</em> cross-platform units to
your <em>Smart Mobile Studio</em> installation. This can be done in three
<code>copy</code> instructions:</p>
<pre>
xcopy SynCrossPlatformSpecific.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y
xcopy SynCrossPlatformCrypto.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y
xcopy SynCrossPlatformREST.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y
</pre>
<p>You can find a corresponding BATCH file in the <code>CrossPlatform</code>
folder, and in <code>SQLite3- SmartMobileStudio
ClientCopySynCrossPlatformUnits.bat</code>.</p>
<p>In fact, the <code>SynCrossPlatformJSON.pas</code> unit is not used under
<em>Smart Mobile Studio</em>: we use the built-in JSON serialization features
of <em>JavaScript</em>, using <code>variant</code> dynamic type, and the
standard <code>JSON.Stringify()</code> and <code>JSON.Parse()</code>
functions.</p>New crc32c() function using optimized asm and SSE 4.2 instructionurn:md5:313be743e42b9b38e9fc71776e85142c2014-05-25T14:53:00+02:002020-07-03T09:29:59+02:00AB4327-GANDImORMot Framework64bitasmblogcrc32cDelphiGoodPracticemORMotOpenSourceperformanceSourcesse42<p>Cyclic Redundancy Check (CRC) codes are widely used for integrity checking
of data in fields such as storage and networking.<br />
There is an ever-increasing need for very high-speed CRC computations on
processors for end-to-end integrity checks.</p>
<p>We just introduced to <em>mORMot</em>'s core unit
(<code>SynCommons.pas</code>) a <a href="http://synopse.info/fossil/info/8fe7cc53b7">fast and efficient
<code>crc32c()</code> function</a>.</p>
<p><img src="http://www.xbitlabs.com/images/cpu/nehalem-microarchitecture/sse.png" alt="" /></p>
<p>It will use either:</p>
<ul>
<li>Optimized <em>x86 asm</em> code, with unrolled loops;</li>
<li>SSE 4.2 hardware crc32 instruction, if available.</li>
</ul>
<p>Resulting speed is very good.<br />
This is for sure the fastest CRC function available in Delphi.<br />
Note that there is a version dedicated to each Win32 and Win64 platform - both
performs at the same speed!</p>
<p>In fact, most popular file formats and protocols (Ethernet, MPEG-2, ZIP,
RAR, 7-Zip, GZip, and PNG) use the polynomial <code>$04C11DB7</code>, while
Intel's hardware implementation is based on another polynomial,
<code>$1EDC6F41</code> (used in iSCSI and Btrfs).<br />
So you would not use this new <code>crc32c()</code> function to
<em>replace</em> the zlib's <code>crc32()</code> function, but as a
convenient very fast hashing function at application level.<br />
For instance, our <code>TDynArray</code> wrapper will use it for fast items
hashing.</p> <p>Here are some speed result, run on a Core i7 notebook.<br />
We did hash 10000 random strings, from 1 to 1250 chars long.</p>
<ul>
<li>Our optimized unrolled x86 version - aka <code>crc32cfast()</code> -
performs the test at a very good pace of 1.7 GB/s;</li>
<li>SSE 4.2 version - aka <code>crc32csse42()</code> - gives an amazing 3.7
GB/s speed (on both Win32 and Win64 platforms);</li>
<li>simple rolled version of the algorithm (similar to the one in Delphi
<code>zlib</code> unit) runs at 330 MB/s.</li>
</ul>
<p>For comparison, on the same random content:</p>
<ul>
<li>Our optimized unrolled <code>kr32()</code> function (i.e. the standard
<em>Kernighan & Ritchie</em> hash taken from "The C programming Language",
3rd edition) hashes at 898.8 MB/s;</li>
<li>Our simple proprietary <code>Hash32()</code> function runs
at 2.5 GB/s, but with much more collisions.</li>
</ul>
<div>Feedback and numbers is <a href="http://synopse.info/forum/viewtopic.php?id=1784">welcome in our forum</a>, as
usual!</div>MongoDB + mORMot benchmarkurn:md5:ffe569e390738cbf6244be3f0a64bf512014-05-07T09:31:00+02:002014-05-07T08:57:07+02:00AB4327-GANDImORMot FrameworkACIDAJAXBatchblogDatabaseDelphiDocumentationDomainDrivenGoodPracticeJSONmappingModelMongoDBmORMotNoSQLOpenSourceORMperformanceRestshardingSourceSQLtransactionUnicode<p>Here are some benchmark charts about <a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-mORMot-ORM-ODM"><em>MongoDB</em> integration in
<em>mORMot</em>'s ORM</a>.</p>
<p><img src="http://www.ceillac.com/IMAGES%20SITE/animaux/marmot-s2.jpg" alt="" /></p>
<p><em>MongoDB</em> appears as a serious competitor to SQL databases, with the
potential benefit of horizontal scaling and installation/administration ease -
performance is very high, and its document-based storage fits perfectly with
<em>mORMot</em>'s advanced ORM features like <em><a href="https://blog.synopse.info?post/post/2011/07/03/Sharde-nothing-architecture">Shared nothing architecture (or
sharding)</a></em>.</p> <p>Following tests were using Synopse <em>mORMot</em> framework 1.18, compiled
with Delphi XE4, against SQLite 3.8.4.3.</p>
<p>We won't show all database engine, but the most representative ones.<br />
Please refer to <a href="https://blog.synopse.info?post/post/2014/03/07/Support-of-MySQL%2C-DB2-and-PostgreSQL">this other benchmark
article for some more complete information</a>.</p>
<h1>Insertion speed</h1>
<p>'MongoDB ack/no ack' for direct <em>MongoDB</em> access
(<code>SynMongoDB.pas</code>) with or without write acknowledge.</p>
<p>For the testings, we used a local <em>MongoDB</em> 2.6 instance in 64 bit
mode.</p>
<table>
<tbody>
<tr align="center">
<td> </td>
<td><strong>Direct</strong></td>
<td><strong>Batch</strong></td>
<td><strong>Trans</strong></td>
<td><strong>Batch Trans</strong></td>
</tr>
<tr align="center">
<td><strong>SQLite3 (file full)</strong></td>
<td>466</td>
<td>437</td>
<td>81754</td>
<td>108752</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (file off)</strong></td>
<td>2012</td>
<td>2044</td>
<td>84550</td>
<td>111731</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (file off exc)</strong></td>
<td>25079</td>
<td>28192</td>
<td>83943</td>
<td>115159</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (mem)</strong></td>
<td>69961</td>
<td>94871</td>
<td>87279</td>
<td>118657</td>
</tr>
<tr align="center">
<td><strong>TObjectList (static)</strong></td>
<td>232385</td>
<td>400608</td>
<td>252678</td>
<td>402803</td>
</tr>
<tr align="center">
<td><strong>TObjectList (virtual)</strong></td>
<td>242812</td>
<td>409131</td>
<td>240003</td>
<td>405712</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (ext full)</strong></td>
<td>490</td>
<td>11918</td>
<td>87556</td>
<td>151144</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (ext off)</strong></td>
<td>2141</td>
<td>47266</td>
<td>89249</td>
<td>160616</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (ext off exc)</strong></td>
<td>33199</td>
<td>145471</td>
<td>90025</td>
<td>158815</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (ext mem)</strong></td>
<td>76411</td>
<td>184706</td>
<td>89834</td>
<td>192618</td>
</tr>
<tr align="center">
<td><strong>MongoDB (ack)</strong></td>
<td>10081</td>
<td>84585</td>
<td>9800</td>
<td>85232</td>
</tr>
<tr align="center">
<td><strong>MongoDB (no ack)</strong></td>
<td>33223</td>
<td>189186</td>
<td>27974</td>
<td>206355</td>
</tr>
<tr align="center">
<td><strong>ZEOS SQlite3</strong></td>
<td>474</td>
<td>11917</td>
<td>36740</td>
<td>55767</td>
</tr>
<tr align="center">
<td><strong>FireDAC SQlite3</strong></td>
<td>20735</td>
<td>40083</td>
<td>40408</td>
<td>121359</td>
</tr>
<tr align="center">
<td><strong>ZEOS Firebird</strong></td>
<td>10056</td>
<td>10155</td>
<td>18769</td>
<td>20335</td>
</tr>
<tr align="center">
<td><strong>FireDAC Firebird</strong></td>
<td>19742</td>
<td>48684</td>
<td>19904</td>
<td>47803</td>
</tr>
<tr align="center">
<td><strong>MSSQL2012 local</strong></td>
<td>3470</td>
<td>35510</td>
<td>10750</td>
<td>47653</td>
</tr>
<tr align="center">
<td><strong>ODBC MSSQL2012</strong></td>
<td>3659</td>
<td>6252</td>
<td>5537</td>
<td>6290</td>
</tr>
<tr align="center">
<td><strong>FireDAC MSSQL2012</strong></td>
<td>3276</td>
<td>5838</td>
<td>9540</td>
<td>40040</td>
</tr>
<tr align="center">
<td><strong>ZEOS PostgreSQL</strong></td>
<td>2953</td>
<td>23740</td>
<td>6913</td>
<td>29780</td>
</tr>
<tr align="center">
<td><strong>ODBC PostgreSQL</strong></td>
<td>2902</td>
<td>25040</td>
<td>3576</td>
<td>28714</td>
</tr>
<tr align="center">
<td><strong>FireDAC PostgreSQL</strong></td>
<td>3054</td>
<td>23329</td>
<td>7149</td>
<td>24844</td>
</tr>
</tbody>
</table>
<p><img src="http://chart.apis.google.com/chart?chtt=Insertion+speed+%28rows%2Fsecond%29&chxl=1:|Batch+Trans|Trans|Batch|Direct&chxt=x,y&chbh=a&chs=600x500&cht=bhg&chco=3D7930,3D8930,309F30,40C355&chxr=0,0,409131&chds=0,409131,0,409131,0,409131,0,409131,0,409131&chd=t:466,437,81754,108752|2012,2044,84550,111731|25079,28192,83943,115159|69961,94871,87279,118657|232385,400608,252678,402803|242812,409131,240003,405712|490,11918,87556,151144|2141,47266,89249,160616|33199,145471,90025,158815|76411,184706,89834,192618|10081,84585,9800,85232|33223,189186,27974,206355|474,11917,36740,55767|20735,40083,40408,121359|10056,10155,18769,20335|19742,48684,19904,47803|3470,35510,10750,47653|3659,6252,5537,6290|3276,5838,9540,40040|2953,23740,6913,29780|2902,25040,3576,28714|3054,23329,7149,24844&chdl=SQLite3+%28file+full%29|SQLite3+%28file+off%29|SQLite3+%28file+off+exc%29|SQLite3+%28mem%29|TObjectList+%28static%29|TObjectList+%28virtual%29|SQLite3+%28ext+full%29|SQLite3+%28ext+off%29|SQLite3+%28ext+off+exc%29|SQLite3+%28ext+mem%29|MongoDB+%28ack%29|MongoDB+%28no+ack%29|ZEOS+SQlite3|FireDAC+SQlite3|ZEOS+Firebird|FireDAC+Firebird|MSSQL2012+local|ODBC+MSSQL2012|FireDAC+MSSQL2012|ZEOS+PostgreSQL|ODBC+PostgreSQL|FireDAC+PostgreSQL" /></p>
<p><img src="http://chart.apis.google.com/chart?chtt=Insertion+speed+%28rows%2Fsecond%29&chxl=1:|FireDAC+PostgreSQL|ODBC+PostgreSQL|ZEOS+PostgreSQL|FireDAC+MSSQL2012|ODBC+MSSQL2012|MSSQL2012+local|FireDAC+Firebird|ZEOS+Firebird|FireDAC+SQlite3|ZEOS+SQlite3|MongoDB+%28no+ack%29|MongoDB+%28ack%29|SQLite3+%28ext+mem%29|SQLite3+%28ext+off+exc%29|SQLite3+%28ext+off%29|SQLite3+%28ext+full%29|TObjectList+%28virtual%29|TObjectList+%28static%29|SQLite3+%28mem%29|SQLite3+%28file+off+exc%29|SQLite3+%28file+off%29|SQLite3+%28file+full%29&chxt=x,y&chbh=a&chs=600x500&cht=bhg&chco=3D7930,3D8930,309F30,40C355&chxr=0,0,409131&chds=0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131,0,409131&chd=t:466,2012,25079,69961,232385,242812,490,2141,33199,76411,10081,33223,474,20735,10056,19742,3470,3659,3276,2953,2902,3054|437,2044,28192,94871,400608,409131,11918,47266,145471,184706,84585,189186,11917,40083,10155,48684,35510,6252,5838,23740,25040,23329|81754,84550,83943,87279,252678,240003,87556,89249,90025,89834,9800,27974,36740,40408,18769,19904,10750,5537,9540,6913,3576,7149|108752,111731,115159,118657,402803,405712,151144,160616,158815,192618,85232,206355,55767,121359,20335,47803,47653,6290,40040,29780,28714,24844&chdl=Direct|Batch|Trans|Batch+Trans" /></p>
<p><em>MongoDB</em> bulk insertion has been implemented, which shows an amazing
speed increase in Batch mode. Depending on the <em>MongoDB write concern</em>
mode, insertion speed can be very high: by default, every write process will be
acknowledge by the server, but you can by-pass this request if you set the
<code>wcUnacknowledged</code> mode - note that in this case, any error (e.g. an
unique field duplicated value) will never be notified, so it should not be used
in production, unless you need this feature to quickly populate a database, or
consolidate some data as fast as possible.</p>
<h1>Read speed</h1>
<p>The same records are now read, either one by one, either as a list:</p>
<table>
<tbody>
<tr align="center">
<td> </td>
<td><strong>By one</strong></td>
<td><strong>All Virtual</strong></td>
<td><strong>All Direct</strong></td>
</tr>
<tr align="center">
<td><strong>SQLite3 (file full)</strong></td>
<td>21607</td>
<td>455083</td>
<td>458757</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (file off)</strong></td>
<td>22177</td>
<td>456454</td>
<td>458001</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (file off exc)</strong></td>
<td>98014</td>
<td>454215</td>
<td>457540</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (mem)</strong></td>
<td>99190</td>
<td>461808</td>
<td>464252</td>
</tr>
<tr align="center">
<td><strong>TObjectList (static)</strong></td>
<td>235504</td>
<td>756773</td>
<td>750300</td>
</tr>
<tr align="center">
<td><strong>TObjectList (virtual)</strong></td>
<td>233666</td>
<td>332402</td>
<td>733460</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (ext full)</strong></td>
<td>103917</td>
<td>210863</td>
<td>458379</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (ext off)</strong></td>
<td>101498</td>
<td>209634</td>
<td>441033</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (ext off exc)</strong></td>
<td>101839</td>
<td>218292</td>
<td>439947</td>
</tr>
<tr align="center">
<td><strong>SQLite3 (ext mem)</strong></td>
<td>102414</td>
<td>185494</td>
<td>438904</td>
</tr>
<tr align="center">
<td><strong>MongoDB (ack)</strong></td>
<td>8002</td>
<td>242353</td>
<td>251268</td>
</tr>
<tr align="center">
<td><strong>MongoDB (no ack)</strong></td>
<td>8234</td>
<td>252079</td>
<td>254582</td>
</tr>
<tr align="center">
<td><strong>ZEOS SQlite3</strong></td>
<td>31135</td>
<td>173593</td>
<td>263060</td>
</tr>
<tr align="center">
<td><strong>FireDAC SQlite3</strong></td>
<td>6318</td>
<td>67169</td>
<td>92291</td>
</tr>
<tr align="center">
<td><strong>ZEOS Firebird</strong></td>
<td>12076</td>
<td>67853</td>
<td>85828</td>
</tr>
<tr align="center">
<td><strong>FireDAC Firebird</strong></td>
<td>1918</td>
<td>37113</td>
<td>44894</td>
</tr>
<tr align="center">
<td><strong>MSSQL2012 local</strong></td>
<td>7904</td>
<td>182401</td>
<td>349797</td>
</tr>
<tr align="center">
<td><strong>ODBC MSSQL2012</strong></td>
<td>8693</td>
<td>113973</td>
<td>178526</td>
</tr>
<tr align="center">
<td><strong>FireDAC MSSQL2012</strong></td>
<td>3054</td>
<td>63730</td>
<td>86051</td>
</tr>
<tr align="center">
<td><strong>ZEOS PostgreSQL</strong></td>
<td>7031</td>
<td>122327</td>
<td>176298</td>
</tr>
<tr align="center">
<td><strong>ODBC PostgreSQL</strong></td>
<td>7281</td>
<td>66843</td>
<td>91489</td>
</tr>
<tr align="center">
<td><strong>FireDAC PostgreSQL</strong></td>
<td>1644</td>
<td>45184</td>
<td>61252</td>
</tr>
</tbody>
</table>
<p><img src="http://chart.apis.google.com/chart?chtt=Read+speed+%28rows%2Fsecond%29&chxl=1:|All+Direct|All+Virtual|By+one&chxt=x,y&chbh=a&chs=600x500&cht=bhg&chco=3D7930,3D8930,309F30,40C355&chxr=0,0,756773&chds=0,756773,0,756773,0,756773&chd=t:21607,455083,458757|22177,456454,458001|98014,454215,457540|99190,461808,464252|235504,756773,750300|233666,332402,733460|103917,210863,458379|101498,209634,441033|101839,218292,439947|102414,185494,438904|8002,242353,251268|8234,252079,254582|31135,173593,263060|6318,67169,92291|12076,67853,85828|1918,37113,44894|7904,182401,349797|8693,113973,178526|3054,63730,86051|7031,122327,176298|7281,66843,91489|1644,45184,61252&chdl=SQLite3+%28file+full%29|SQLite3+%28file+off%29|SQLite3+%28file+off+exc%29|SQLite3+%28mem%29|TObjectList+%28static%29|TObjectList+%28virtual%29|SQLite3+%28ext+full%29|SQLite3+%28ext+off%29|SQLite3+%28ext+off+exc%29|SQLite3+%28ext+mem%29|MongoDB+%28ack%29|MongoDB+%28no+ack%29|ZEOS+SQlite3|FireDAC+SQlite3|ZEOS+Firebird|FireDAC+Firebird|MSSQL2012+local|ODBC+MSSQL2012|FireDAC+MSSQL2012|ZEOS+PostgreSQL|ODBC+PostgreSQL|FireDAC+PostgreSQL" /></p>
<p><img src="http://chart.apis.google.com/chart?chtt=Read+speed+%28rows%2Fsecond%29&chxl=1:|FireDAC+PostgreSQL|ODBC+PostgreSQL|ZEOS+PostgreSQL|FireDAC+MSSQL2012|ODBC+MSSQL2012|MSSQL2012+local|FireDAC+Firebird|ZEOS+Firebird|FireDAC+SQlite3|ZEOS+SQlite3|MongoDB+%28no+ack%29|MongoDB+%28ack%29|SQLite3+%28ext+mem%29|SQLite3+%28ext+off+exc%29|SQLite3+%28ext+off%29|SQLite3+%28ext+full%29|TObjectList+%28virtual%29|TObjectList+%28static%29|SQLite3+%28mem%29|SQLite3+%28file+off+exc%29|SQLite3+%28file+off%29|SQLite3+%28file+full%29&chxt=x,y&chbh=a&chs=600x500&cht=bhg&chco=3D7930,3D8930,309F30,40C355&chxr=0,0,756773&chds=0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773,0,756773&chd=t:21607,22177,98014,99190,235504,233666,103917,101498,101839,102414,8002,8234,31135,6318,12076,1918,7904,8693,3054,7031,7281,1644|455083,456454,454215,461808,756773,332402,210863,209634,218292,185494,242353,252079,173593,67169,67853,37113,182401,113973,63730,122327,66843,45184|458757,458001,457540,464252,750300,733460,458379,441033,439947,438904,251268,254582,263060,92291,85828,44894,349797,178526,86051,176298,91489,61252&chdl=By+one|All+Virtual|All+Direct" /></p>
<p><em>MongoDB</em> appears as a serious competitor to SQL databases, with the
potential benefit of horizontal scaling and installation/administration ease -
performance is very high, and its document-based storage fits perfectly with
<em>mORMot</em>'s advanced ORM features like <em><a href="https://blog.synopse.info?post/post/2011/07/03/Sharde-nothing-architecture">Shared nothing architecture (or
sharding)</a></em>.</p>
<p>You can get more information about <a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-database-access">low-level integration of
<em>MongoDB</em></a> in <em>mORMot</em>, or our integrated <a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-mORMot-ORM-ODM">ORM/ODM support in the
framework</a>.<br />
Feedback is <a href="http://synopse.info/forum/viewtopic.php?id=1742">welcome
on our forum</a>, as usual!</p>MongoDB + mORMot ORM = ODMurn:md5:232877d989279d738aab47cacf5358f82014-05-07T09:14:00+02:002014-05-07T13:52:17+02:00AB4327-GANDImORMot FrameworkACIDAJAXBatchblogDatabaseDelphiDocumentationDomainDrivenGoodPracticeJSONmappingModelMongoDBmORMotNoSQLOpenSourceORMperformanceRestshardingSourceSQLtransactionUnicode<p><em><a href="http://www.mongodb.org/">MongoDB</a></em> (from "humongous") is
a cross-platform document-oriented database system, and certainly the best
known NoSQL database.<br />
According to <a href="http://db-engines.com">http://db-engines.com</a> in April
2014, <em>MongoDB</em> is in 5th place of the most popular types of database
management systems, and first place for NoSQL database management
systems.<br />
Our <em>mORMot</em> gives premium access to this database, featuring full
<em>NoSQL and Object-Document Mapping (ODM)</em> abilities to the
framework.</p>
<p><img src="http://upload.wikimedia.org/wikipedia/en/thumb/e/eb/MongoDB_Logo.png/320px-MongoDB_Logo.png" alt="" /></p>
<p>Integration is made at two levels:</p>
<ul>
<li>Direct low-level access to the <em>MongoDB</em> server, in the
<code>SynMongoDB.pas</code> unit;</li>
<li>Close integration with our ORM (which becomes <em>defacto</em> an ODM), in
the <code>mORMotMongoDB.pas</code> unit.</li>
</ul>
<p><em>MongoDB</em> eschews the traditional table-based relational database
structure in favor of JSON-like documents with dynamic schemas
(<em>MongoDB</em> calls the format BSON), which matches perfectly
<em>mORMot</em>'s RESTful approach.</p>
<p>This second article will focus on integration of <em>MongoDB</em> with our
ORM.</p> <h3>MongoDB + ORM = ODM</h3>
<p>The <code>mORMotMongoDB.pas</code> unit is able to let any
<code>TSQLRecord</code> class be persisted on a remote <em>MongoDB</em>
server.</p>
<p>As a result, our ORM is able to be used as a <em>NoSQL and Object-Document
Mapping (ODM)</em> framework, with almost no code change.<br />
Any <em>MongoDB</em> database can be accessed via RESTful commands, using JSON
over HTTP.</p>
<p>This integration benefits from the other parts of the framework (e.g. our
UTF-8 dedicated process, which is also the native encoding for BSON), so you
can easily mix SQL and NoSQL databases with the exact same code, and are still
able to tune any SQL or <em>MongoDB</em> request in your code, if
necessary.</p>
<h3>Register the TSQLRecord class</h3>
<p>In the database model, we define a <code>TSQLRecord</code> class, as
usual:</p>
<pre>
TSQLORM = <strong>class</strong>(TSQLRecord)
<strong>private</strong>
fAge: integer;
fName: RawUTF8;
fDate: TDateTime;
fValue: <strong>variant</strong>;
fInts: TIntegerDynArray;
fCreateTime: TCreateTime;
fData: TSQLRawBlob;
<strong>published</strong>
<strong>property</strong> Name: RawUTF8 <strong>read</strong> fName <strong>write</strong> fName <strong>stored</strong> AS_UNIQUE;
<strong>property</strong> Age: integer <strong>read</strong> fAge <strong>write</strong> fAge;
<strong>property</strong> Date: TDateTime <strong>read</strong> fDate <strong>write</strong> fDate;
<strong>property</strong> Value: <strong>variant read</strong> fValue <strong>write</strong> fValue;
<strong>property</strong> Ints: TIntegerDynArray <strong>index</strong> 1 <strong>read</strong> fInts <strong>write</strong> fInts;
<strong>property</strong> Data: TSQLRawBlob <strong>read</strong> fData <strong>write</strong> fData;
<strong>property</strong> CreateTime: TCreateTime <strong>read</strong> fCreateTime <strong>write</strong> fCreateTime;
<strong>end</strong>;
</pre>
<p>Note that we did not define any <strong><code>index</code></strong>
<code>...</code> values for the <code>RawUTF8</code> property, as we need for
external SQL databases, since <em>MongoDB</em> does not expect any restriction
about text fields length.</p>
<p>The property values will be stored in the native <em>MongoDB</em> layout,
i.e. with a better coverage than the SQL types:</p>
<table>
<tbody>
<tr>
<td><strong>Delphi</strong></td>
<td><strong><em>MongoDB</em></strong></td>
<td><strong>Remarks</strong></td>
</tr>
<tr>
<td><code>byte</code></td>
<td>int32</td>
</tr>
<tr>
<td><code>word</code></td>
<td>int32</td>
</tr>
<tr>
<td><code>integer</code></td>
<td>int32</td>
</tr>
<tr>
<td><code>cardinal</code></td>
<td>N/A</td>
<td>You should use <code>Int64</code> instead</td>
</tr>
<tr>
<td><code>Int64</code></td>
<td>int64</td>
</tr>
<tr>
<td><code>boolean</code></td>
<td>boolean</td>
<td>0 is <code>false</code>, anything else is <code>true</code></td>
</tr>
<tr>
<td>enumeration</td>
<td>int32</td>
<td>store the ordinal value of the enumerated item(i.e. starting at 0 for the
first element)</td>
</tr>
<tr>
<td>set</td>
<td>int32</td>
<td>each bit corresponding to an enumerated item (therefore a set of up to 64
elements can be stored in such a field)</td>
</tr>
<tr>
<td><code>single</code></td>
<td>double</td>
</tr>
<tr>
<td><code>double</code></td>
<td>double</td>
</tr>
<tr>
<td><code>extended</code></td>
<td>double</td>
<td>stored as <code>double</code> (precision lost)</td>
</tr>
<tr>
<td><code>currency</code></td>
<td>double</td>
<td>safely converted to/from <code>currency</code> type with fixed decimals,
without rounding error</td>
</tr>
<tr>
<td><code>RawUTF8</code></td>
<td>UTF-8</td>
<td>this is the <strong>preferred</strong> field type for storing some textual
content in the ORM</td>
</tr>
<tr>
<td><code>WinAnsiString</code></td>
<td>UTF-8</td>
<td><em>WinAnsi</em> char-set (code page 1252) in Delphi</td>
</tr>
<tr>
<td><code>RawUnicode</code></td>
<td>UTF-8</td>
<td><em>UCS2</em> char-set in Delphi, as <code>AnsiString</code></td>
</tr>
<tr>
<td><code>WideString</code></td>
<td>UTF-8</td>
<td><em>UCS2</em> char-set, as COM BSTR type (Unicode in all version of
Delphi)</td>
</tr>
<tr>
<td><code>SynUnicode</code></td>
<td>UTF-8</td>
<td>Will be either <code>WideString</code> before Delphi 2009, or
<code>UnicodeString</code> later</td>
</tr>
<tr>
<td><code>string</code></td>
<td>UTF-8</td>
<td>Not to be used before Delphi 2009 (unless you may loose some data during
conversion) - <code>RawUTF8</code> is preferred in all cases</td>
</tr>
<tr>
<td><code>TDateTime</code></td>
<td>datetime</td>
<td>ISO 8601 encoded date time</td>
</tr>
<tr>
<td><code>TTimeLog</code></td>
<td>int64</td>
<td>as proprietary fast <code>Int64</code> date time</td>
</tr>
<tr>
<td><code>TModTime</code></td>
<td>int64</td>
<td>the server date time will be stored when a record is modified (as
proprietary fast <code>Int64</code>)</td>
</tr>
<tr>
<td><code>TCreateTime</code></td>
<td>int64</td>
<td>the server date time will be stored when a record is created (as
proprietary fast <code>Int64</code>)</td>
</tr>
<tr>
<td><code>TSQLRecord</code></td>
<td>int32</td>
<td><code>RowID</code> pointing to another record (warning: the field value
contains <code>pointer(RowID)</code>, not a valid object instance - the record
content must be retrieved with late-binding via its <code>ID</code> using a
<code>PtrInt(Field)</code> typecast or the <code>Field.ID</code> method), or by
using e.g. <code>CreateJoined()</code></td>
</tr>
<tr>
<td><code>TSQLRecordMany</code></td>
<td>nothing</td>
<td>data is stored in a separate <em>pivot</em> table; for MongoDB, you should
better use <em>data sharding</em>, and an embedded sub-document</td>
</tr>
<tr>
<td><code>TRecordReference</code></td>
<td>int32</td>
<td>store both <code>ID</code> and <code>TSQLRecord</code> type in a
<code>RecordRef</code>-like value (use e.g. <code>TSQLRest.
Retrieve(Reference)</code> to get a record content)</td>
</tr>
<tr>
<td><code>TPersistent</code></td>
<td>object</td>
<td>BSON object (from <code>ObjectToJSON</code>)</td>
</tr>
<tr>
<td><code>TCollection</code></td>
<td>array</td>
<td>BSON array of objects (from <code>ObjectToJSON</code>)</td>
</tr>
<tr>
<td><code>TObjectList</code></td>
<td>array</td>
<td>BSON array of objects (from <code>ObjectToJSON</code>) - see
<code>TJSONSerializer. RegisterClassForJSON</code> <em>TObjectList
serialization</em></td>
</tr>
<tr>
<td><code>TStrings</code></td>
<td>array</td>
<td>BSON array of strings (from <code>ObjectToJSON</code>)</td>
</tr>
<tr>
<td><code>TRawUTF8List</code></td>
<td>array</td>
<td>BSON array of string (from <code>ObjectToJSON</code>)</td>
</tr>
<tr>
<td>any <code>TObject</code></td>
<td>object</td>
<td>See <code>TJSONSerializer. RegisterCustomSerializer</code> <em>TObject
serialization</em></td>
</tr>
<tr>
<td><code>TSQLRawBlob</code></td>
<td>binary</td>
<td>BSON binary for blob storage - note that you need to retrieve explicitly
such fields, which are not part of the default <code>Retrieve()</code> method;
this type is an alias to <code>RawByteString</code></td>
</tr>
<tr>
<td><em>dynamic arrays</em></td>
<td>array<br />
binary</td>
<td>if the dynamic array can be saved as true JSON, will be stored as BSON
array - otherwise, will be stored in the <code>TDynArray.SaveTo</code> binary
format - note that <code>TByteDynArray</code> will always be stored as BSON
binary, so that <code>TSQLRawBlob</code> may be reserved to store content on
<em>GridFS</em> in the future</td>
</tr>
<tr>
<td><code>variant</code></td>
<td>array<br />
object</td>
<td>BSON number, text, object or array, depending on <em>TDocVariant custom
variant type</em> or <code>TBSONVariant</code> stored value</td>
</tr>
<tr>
<td><code>record</code></td>
<td>binary<br />
object</td>
<td>BSON as defined in code by overriding
<code>TSQLRecord.InternalRegisterCustomProperties</code></td>
</tr>
</tbody>
</table>
<p>On the server side (there won't be any difference for the client), you
define a <code>TMongoDBClient</code>, and assign it to a given
<code>TSQLRecord</code> class:</p>
<pre>
MongoClient := TMongoClient.Create('localhost',27017);
DB := MongoClient.Database['dbname'];
Model := TSQLModel.Create([TSQLORM]);
Client := TSQLRestClientDB.Create(Model,<strong>nil</strong>,':memory:',TSQLRestServerDB);
<span style="background-color:yellow;"><strong>if</strong> StaticMongoDBRegister(TSQLORM,fClient.Server,fDB,'collectionname')=<strong>nil then</strong></span>
<strong>raise</strong> Exception.Create('Error');
</pre>
<p>And... that's all!</p>
<p>You can then use any ORM command, as usual:</p>
<pre>
writeln(Client.TableRowCount(TSQLORM)=0);
</pre>
<p>As with external databases, you can specify the field names mapping between
the objects and the <em>MongoDB</em> collection.<br />
By default, the <code>TSQLRecord.ID</code> property is mapped to the
<em>MongoDB</em>'s <code>_id</code> field, and the ORM will populate this
<code>_id</code> field with a sequence of integer values, just like any
<code>TSQLRecord</code> table.<br />
You can specify your own mapping, using for instance:</p>
<pre>
aModel.Props[aClass].ExternalDB.MapField(..)
</pre>
<p>Since the field names are stored within the document itself, it may be a
good idea to use shorter naming for the <em>MongoDB</em> collection. It may
save some storage space, when working with a huge number of documents.</p>
<p>Once the <code>TSQLRecord</code> is mapped to a <em>MongoDB</em> collection,
you can always have direct access to the <code>TMongoCollection</code> instance
later on, by calling:</p>
<pre>
(aServer.StaticDataServer[aClass] <strong>as</strong> TSQLRestServerStaticMongoDB).Collection
</pre>
<p>This may allow any specific task, including any tuned query or process.</p>
<h3>ORM/ODM CRUD methods</h3>
<p>You can add documents with the standard CRUD methods of the ORM, as
usual:</p>
<pre>
R := TSQLORM.Create;
<strong>try</strong>
<strong>for</strong> i := 1 <strong>to</strong> COLL_COUNT <strong>do begin</strong>
R.Name := 'Name '+Int32ToUTF8(i);
R.Age := i;
R.Date := 1.0*(30000+i);
R.Value := _ObjFast(['num',i]);
R.Ints := <strong>nil</strong>;
R.DynArray(1).Add(i);
<span style="background-color:yellow;">assert(Client.Add(R,True)=i);</span>
<strong>end</strong>;
<strong>finally</strong>
R.Free;
<strong>end</strong>;
</pre>
<p>As we already saw, the framework is able to handle any kind of properties,
including complex types like <em>dynamic arrays</em> or
<code>variant</code>.<br />
In the above code, a <code>TDocVariant</code> document has been stored in
<code>R.Value</code>, and a dynamic array of <code>integer</code> values is
accessed via its <code>index 1</code> shortcut and the
<code>TSQLRecord.DynArray()</code> method.</p>
<p>The usual <code>Retrieve</code> / <code>Delete</code> / <code>Update</code>
methods are available:</p>
<pre>
R := TSQLORM.Create;
<strong>try</strong>
<strong>for</strong> i := 1 <strong>to</strong> COLL_COUNT <strong>do begin</strong>
<span style="background-color:yellow;">Check(Client.Retrieve(i,R));</span>
<em>// here R instance contains all values of one document, excluding BLOBs</em>
<strong>end</strong>;
<strong>finally</strong>
R.Free;
<strong>end</strong>;
</pre>
<p>You can define a WHERE clause, as if the back-end where a regular SQL
database:</p>
<pre>
R := TSQLORM.CreateAndFillPrepare(Client,'ID=?',[i]);
<strong>try</strong>
...
</pre>
<p>Current implementation understand one condition over one single field, with
<code>= > >= < <= IN</code> clauses. More advanced queries are
possible, but won't be handled as SQL, but via direct access to the
<code>TMongoDBCollection</code>.</p>
<p>To perform a query and retrieve the content of several documents, you can
use regular <code>CreateAndFillPrepare</code> or <code>FillPrepare</code>
methods:</p>
<pre>
<span style="background-color:yellow;">R := TSQLORM.CreateAndFillPrepare(Client,'');</span>
<strong>try</strong>
n := 0;
<span style="background-color:yellow;"><strong>while</strong> R.FillOne <strong>do begin</strong></span>
<em>// here R instance contains all values of one document, excluding BLOBs</em>
inc(n);
<strong>end</strong>;
assert(n=COLL_COUNT);
<strong>finally</strong>
R.Free;
<strong>end</strong>;
</pre>
<p>A WHERE clause can also be defined for <code>CreateAndFillPrepare</code> or
<code>FillPrepare</code> methods.</p>
<h3>BATCH mode</h3>
<p>In addition to individual CRUD operations, our <em>MongoDB</em> is able to
use BATCH mode for adding or deleting documents.</p>
<p>You can write the exact same code as with any SQL back-end:</p>
<pre>
Client.BatchStart(TSQLORM);
R := TSQLORM.Create;
<strong>try</strong>
<strong>for</strong> i := 1 <strong>to</strong> COLL_COUNT <strong>do begin</strong>
R.Name := 'Name '+Int32ToUTF8(i);
R.Age := i;
R.Date := 1.0*(30000+i);
R.Value := _ObjFast(['num',i]);
R.Ints := <strong>nil</strong>;
R.DynArray(1).Add(i);
assert(Client.BatchAdd(R,True)>=0);
<strong>end</strong>;
<strong>finally</strong>
R.Free;
<strong>end</strong>;
assert(Client.BatchSend(IDs)=HTML_SUCCESS);
</pre>
<p>Or for deletion:</p>
<pre>
Client.BatchStart(TSQLORM);
<strong>for</strong> i := 5 <strong>to</strong> COLL_COUNT <strong>do</strong>
<strong>if</strong> i <strong>mod</strong> 5=0 <strong>then</strong>
assert(fClient.BatchDelete(i)>=0);
assert(Client.BatchSend(IDs)=HTML_SUCCESS);
</pre>
<p>Speed benefit may be huge in regard to individual Add/Delete operations,
even on a local <em>MongoDB</em> server. We will see some benchmark numbers
now.</p>
<h3>ORM/ODM performance</h3>
<p>You can take a look at <em><a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-mORMot-benchmark">Data access benchmark</a></em> to
compare <em>MongoDB</em> as back-end for our ORM classes.</p>
<p>In respect to external SQL engines, it features very high speed, low CPU
use, and almost no difference in use. We interfaced the <code>BatchAdd()</code>
and <code>BatchDelete()</code> methods to benefit of <em>MongoDB</em> BULK
process, and avoided most memory allocation during the process.</p>
<p>Here are some numbers, extracted from the <code>MongoDBTests.dpr</code>
sample, which reflects the performance of our ORM/ODM, depending on the
<em>Write Concern</em> mode used:</p>
<pre>
2. ORM
<br />2.1. ORM with acknowledge:
- Connect to local server: 6 assertions passed 18.65ms
- Insert: 5,002 assertions passed 521.25ms
<span style="background-color:yellow;"> 5000 rows inserted in 520.65ms i.e. 9603/s, aver. 104us, 2.9 MB/s</span>
- Insert in batch mode: 5,004 assertions passed 65.37ms
<span style="background-color:yellow;"> 5000 rows inserted in 65.07ms i.e. 76836/s, aver. 13us, 8.4 MB/s</span>
- Retrieve: 45,001 assertions passed 640.95ms
5000 rows retrieved in 640.75ms i.e. 7803/s, aver. 128us, 2.1 MB/s
- Retrieve all: 40,001 assertions passed 20.79ms
<span style="background-color:yellow;"> 5000 rows retrieved in 20.33ms i.e. 245941/s, aver. 4us, 27.1 MB/s</span>
- Retrieve one with where clause: 45,410 assertions passed 673.01ms
5000 rows retrieved in 667.17ms i.e. 7494/s, aver. 133us, 2.0 MB/s
- Update: 40,002 assertions passed 681.31ms
5000 rows updated in 660.85ms i.e. 7565/s, aver. 132us, 2.4 MB/s
- Blobs: 125,003 assertions passed 2.16s
5000 rows updated in 525.97ms i.e. 9506/s, aver. 105us, 2.4 MB/s
- Delete: 38,003 assertions passed 175.86ms
1000 rows deleted in 91.37ms i.e. 10944/s, aver. 91us, 2.3 MB/s
- Delete in batch mode: 33,003 assertions passed 34.71ms
<span style="background-color:yellow;"> 1000 rows deleted in 14.90ms i.e. 67078/s, aver. 14us, 597 KB/s</span>
Total failed: 0 / 376,435 - ORM with acknowledge PASSED 5.00s
<br />2.2. ORM without acknowledge:
- Connect to local server: 6 assertions passed 16.83ms
- Insert: 5,002 assertions passed 179.79ms
<span style="background-color:yellow;"> 5000 rows inserted in 179.15ms i.e. 27908/s, aver. 35us, 3.9 MB/s</span>
- Insert in batch mode: 5,004 assertions passed 66.30ms
<span style="background-color:yellow;"> 5000 rows inserted in 31.46ms i.e. 158891/s, aver. 6us, 17.5 MB/s</span>
- Retrieve: 45,001 assertions passed 642.05ms
5000 rows retrieved in 641.85ms i.e. 7789/s, aver. 128us, 2.1 MB/s
- Retrieve all: 40,001 assertions passed 20.68ms
<span style="background-color:yellow;"> 5000 rows retrieved in 20.26ms i.e. 246718/s, aver. 4us, 27.2 MB/s</span>
- Retrieve one with where clause: 45,410 assertions passed 680.99ms
5000 rows retrieved in 675.24ms i.e. 7404/s, aver. 135us, 2.0 MB/s
- Update: 40,002 assertions passed 231.75ms
5000 rows updated in 193.74ms i.e. 25807/s, aver. 38us, 3.6 MB/s
- Blobs: 125,003 assertions passed 1.44s
5000 rows updated in 150.58ms i.e. 33202/s, aver. 30us, 2.6 MB/s
- Delete: 38,003 assertions passed 103.57ms
1000 rows deleted in 19.73ms i.e. 50668/s, aver. 19us, 2.4 MB/s
- Delete in batch mode: 33,003 assertions passed 47.50ms
<span style="background-color:yellow;"> 1000 rows deleted in 364us i.e. 2747252/s, aver. 0us, 23.4 MB/s</span>
Total failed: 0 / 376,435 - ORM without acknowledge PASSED 3.44s
</pre>
<p>As for direct <em>MongoDB</em> access, the <code>wcUnacknowledged</code> is
not to be used on production, but may be very useful in some particular
scenarios.<br />
As expected, the reading process is not impacted by the <em>Write Concern</em>
mode set.</p>
<p>You can take a look at the previous blog article, about <a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-database-access">low-level <em>MongoDB</em> direct
access</a>.<br />
Or check out <a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-mORMot-benchmark">more complete
benchmark results</a>, to compare <em>MongoDB</em> with other SQL back-end
handled by our ORM.<br />
Feedback is <a href="http://synopse.info/forum/viewtopic.php?id=1742">welcome
on our forum</a>, as usual!</p>Direct MongoDB database accessurn:md5:783573bb416f11adc20f4cffa50477cd2014-05-07T09:10:00+02:002020-07-03T09:29:59+02:00AB4327-GANDImORMot FrameworkACIDAJAXBatchblogDatabaseDelphiDocumentationDomainDrivenGoodPracticeJSONmappingModelMongoDBmORMotNoSQLOpenSourceORMperformanceRestshardingSourceSQLtransactionUnicode<p><em><a href="http://www.mongodb.org/">MongoDB</a></em> (from "humongous") is
a cross-platform document-oriented database system, and certainly the best
known NoSQL database.<br />
According to <a href="http://db-engines.com">http://db-engines.com</a> in April
2014, <em>MongoDB</em> is in 5th place of the most popular types of database
management systems, and first place for NoSQL database management
systems.<br />
Our <em>mORMot</em> framework gives premium access to this database,
featuring full <em>NoSQL and Object-Document Mapping (ODM)</em> abilities to
the framework.</p>
<p><img src="http://upload.wikimedia.org/wikipedia/en/thumb/e/eb/MongoDB_Logo.png/320px-MongoDB_Logo.png" alt="" /></p>
<p>Integration is made at two levels:</p>
<ul>
<li>Direct low-level access to the <em>MongoDB</em> server, in the
<code>SynMongoDB.pas</code> unit;</li>
<li>Close integration with our ORM (which becomes <em>defacto</em> an ODM), in
the <code>mORMotMongoDB.pas</code> unit.</li>
</ul>
<p><em>MongoDB</em> eschews the traditional table-based relational database
structure in favor of JSON-like documents with dynamic schemas
(<em>MongoDB</em> calls the format BSON), which matches perfectly
<em>mORMot</em>'s RESTful approach.</p>
<p>In this first article, we will detail direct low-level access to the
<em>MongoDB</em> server, via the <code>SynMongoDB.pas</code> unit.</p> <h3>MongoDB client</h3>
<p>The <code>SynMongoDB.pas</code> unit features direct optimized access to a
<em>MongoDB</em> server.</p>
<p>It gives access to any BSON data, including documents, arrays, and
<em>MongoDB</em>'s custom types (like ObjectID, dates, binary, regex or
Javascript):</p>
<ul>
<li>For instance, a <code>TBSONObjectID</code> can be used to create some
genuine document identifiers on the client side (<em>MongoDB</em> does not
generate the IDs for you: a common way is to generate unique IDs on the client
side);</li>
<li>Generation of BSON content from any Delphi types (via
<code>TBSONWriter</code>);</li>
<li>Fast in-place parsing of the BSON stream, without any memory allocation
(via <code>TBSONElement</code>);</li>
<li>A <code>TBSONVariant</code> custom variant type, to store
<em>MongoDB</em>'s custom type values;</li>
<li>Interaction with the <code>SynCommons</code>' <em>TDocVariant custom
variant type</em> as document storage and late-binding access;</li>
<li>Marshalling BSON to and from JSON, with the <em>MongoDB</em> extended
syntax for handling its custom types.</li>
</ul>
<p>This unit defines some objects able to connect and manage databases and
collections of documents on any <em>MongoDB</em> servers farm:</p>
<ul>
<li>Connection to one or several servers, including secondary hosts, via the
<code>TMongoClient</code> class;</li>
<li>Access to any database instance, via the <code>TMongoDatabase</code>
class;</li>
<li>Access to any collection, via the <code>TMongoCollection</code> class;</li>
<li>It features some nice abilities about speed, like BULK insert or delete
mode, and explicit <em>Write Concern</em> settings.</li>
</ul>
<p>At collection level, you can have direct access to the data, with high level
structures like <code>TDocVariant</code>/<code>TBSONVariant</code>, with
easy-to-read JSON, or low level BSON content.<br />
You can also tune most aspects of the client process, e.g. about error handling
or <em>write concerns</em> (i.e. how remote data modifications are
acknowledged).</p>
<h3>Connecting to a server</h3>
<p>Here is some sample code, which is able to connect to a <em>MongoDB</em>
server, and returns the server time:</p>
<pre>
<strong>var</strong> Client: TMongoClient;
DB: TMongoDatabase;
serverTime: TDateTime;
res: <strong>variant</strong>; <em>// we will return the command result as TDocVariant</em>
errmsg: RawUTF8;
<strong>begin</strong>
Client := TMongoClient.Create('localhost',27017);
<strong>try</strong>
DB := Client.Database['mydb'];
writeln('Connecting to ',DB.Name); <em>// will write 'mydb'</em>
errmsg := DB.RunCommand('hostInfo',res); <em>// run a command</em>
<strong>if</strong> errmsg<>'' <strong>then</strong>
exit; <em>// quit on any error</em>
serverTime := res.system.currentTime; <em>// direct conversion to TDateTime</em>
writeln('Server time is ',DateTimeToStr(serverTime));
<strong>finally</strong>
Client.Free; <em>// will release the DB instance</em>
<strong>end</strong>;
<strong>end</strong>;
</pre>
<p>Note that for this low-level command, we used a <code>TDocVariant</code>,
and its late-binding abilities.</p>
<p>In fact, if you put your mouse over the <code>res</code> variable during
debugging, you will see the following JSON content:</p>
<pre>
{"system":{"currentTime":"2014-05-06T15:24:25","hostname":"Acer","cpuAddrSize":64,"memSizeMB":3934,"numCores":4,"cpuArch":"x86_64","numaEnabled":false},"os":{"type":"Windows","name":"Microsoft Windows 7","version":"6.1 SP1 (build 7601)"},"extra":{"pageSize":4096},"ok":1}
</pre>
<p>And we simply access to the server time by writing
<code>res.system.currentTime</code>.</p>
<h3>Adding some documents to the collection</h3>
<p>We will now explain how to add documents to a given collection.</p>
<p>We assume that we have a <code>DB: TMongoDatabase</code> instance available.
Then we will create the documents with a <code>TDocVariant</code> instance,
which will be filled via late-binding, and via a <code>doc.Clear</code>
pseudo-method used to flush any previous property value:</p>
<pre>
<strong>var</strong> Coll: TMongoCollection;
doc: <strong>variant</strong>;
i: integer;
<strong>begin</strong>
Coll := DB.CollectionOrCreate[COLL_NAME];
TDocVariant.New(doc);
<strong>for</strong> i := 1 <strong>to</strong> 10 <strong>do</strong>
<strong>begin</strong>
doc.Clear;
doc.Name := 'Name '+IntToStr(i+1);
doc.Number := i;
<span style="background-color:yellow;">Coll.Save(doc);</span>
writeln('Inserted with _id=',doc._id);
<strong>end</strong>;
<strong>end</strong>;
</pre>
<p>Thanks to <code>TDocVariant</code> late-binding abilities, code is pretty
easy to understand and maintain.</p>
<p>This code will display the following on the console:</p>
<pre>
Inserted with _id=5369029E4F901EE8114799D9
Inserted with _id=5369029E4F901EE8114799DA
Inserted with _id=5369029E4F901EE8114799DB
Inserted with _id=5369029E4F901EE8114799DC
Inserted with _id=5369029E4F901EE8114799DD
Inserted with _id=5369029E4F901EE8114799DE
Inserted with _id=5369029E4F901EE8114799DF
Inserted with _id=5369029E4F901EE8114799E0
Inserted with _id=5369029E4F901EE8114799E1
Inserted with _id=5369029E4F901EE8114799E2
</pre>
<p>It means that the <code>Coll.Save()</code> method was clever enough to
understand that the supplied document does not have any <code>_id</code> field,
so will compute one on the client side before sending the document data to the
<em>MongoDB</em> server.</p>
<p>We may have written:</p>
<pre>
<strong>for</strong> i := 1 <strong>to</strong> 10 <strong>do</strong>
<strong>begin</strong>
doc.Clear;
<span style="background-color:yellow;">doc._id := ObjectID;</span>
doc.Name := 'Name '+IntToStr(i+1);
doc.Number := i;
Coll.Save(doc);
writeln('Inserted with _id=',doc._id);
<strong>end</strong>;
<strong>end</strong>;
</pre>
<p>Which will compute the document identifier explicitly before calling
<code>Coll.Save()</code>.<br />
In this case, we may have called directly <code>Coll.Insert()</code>, which is
somewhat faster.</p>
<p>Note that you are not obliged to use a <em>MongoDB</em> ObjectID as
identifier. You can use any value, if you are sure that it will be genuine. For
instance, you can use an integer:</p>
<pre>
<strong>for</strong> i := 1 <strong>to</strong> 10 <strong>do</strong>
<strong>begin</strong>
doc.Clear;
<span style="background-color:yellow;">doc._id := i;</span>
doc.Name := 'Name '+IntToStr(i+1);
doc.Number := i;
<span style="background-color:yellow;">Coll.Insert(doc);</span>
writeln('Inserted with _id=',doc._id);
<strong>end</strong>;
<strong>end</strong>;
</pre>
<p>The console will display now:</p>
<pre>
Inserted with _id=1
Inserted with _id=2
Inserted with _id=3
Inserted with _id=4
Inserted with _id=5
Inserted with _id=6
Inserted with _id=7
Inserted with _id=8
Inserted with _id=9
Inserted with _id=10
</pre>
<p>Note that the <em>mORMot</em> ORM will compute a genuine series of integers
in a similar way, which will be used as expected by the
<code>TSQLRecord.ID</code> primary key property.</p>
<p>The <code>TMongoCollection</code> class can also write a list of documents,
and send them at once to the <em>MongoDB</em> server: this BULK insert mode -
close to the <em>Array Binding</em> feature of some SQL providers, and
implemented in our <em>SynDB</em> classes - see <em>BATCH sequences for
adding/updating/deleting records</em> - can increase the insertion by a factor
of 10 times, even when connected to a local instance: imagine how much time it
may save over a physical network!</p>
<p>For instance, you may write:</p>
<pre>
<strong>var</strong> docs: TVariantDynArray;
...
SetLength(docs,COLL_COUNT);
<strong>for</strong> i := 0 <strong>to</strong> COLL_COUNT-1 <strong>do begin</strong>
TDocVariant.New(docs[i]);
docs[i]._id := ObjectID; <em>// compute new ObjectID on the client side</em>
docs[i].Name := 'Name '+IntToStr(i+1);
docs[i].FirstName := 'FirstName '+IntToStr(i+COLL_COUNT);
docs[i].Number := i;
<strong>end</strong>;
Coll.Insert(docs); <em>// insert all values at once</em>
...
</pre>
<p>You will find out later for some numbers about the speed increase due to
such BULK insert.</p>
<h3>Retrieving the documents</h3>
<p>You can retrieve the document as a <code>TDocVariant</code> instance:</p>
<pre>
<strong>var</strong> doc: <strong>variant</strong>;
...
doc := Coll.FindOne(5);
writeln('Name: ',doc.Name);
writeln('Number: ',doc.Number);
</pre>
<p>Which will write on the console:</p>
<pre>
Name: Name 6
Number: 5
</pre>
<p>You have access to the whole <code>Query</code> parameter, if needed:</p>
<pre>
doc := Coll.FindDoc('{_id:?}',[5]);
doc := Coll.FindOne(5); <em>// same as previous</em>
</pre>
<p>This <code>Query</code> filter is similar to a WHERE clause in SQL. You can
write complex search patterns, if needed - see <a href="http://docs.mongodb.org/manual/reference/method/db.collection.find">http://docs.mongodb.org/manual/reference/method/db.collection.find</a>
for reference.</p>
<p>You can retrieve a list of documents, as a dynamic array of
<code>TDocVariant</code>:</p>
<pre>
<strong>var</strong> docs: TVariantDynArray;
...
Coll.FindDocs(docs);
<strong>for</strong> i := 0 <strong>to</strong> high(docs) <strong>do</strong>
writeln('Name: ',docs[i].Name,' Number: ',docs[i].Number);
</pre>
<p>Which will output:</p>
<pre>
Name: Name 2 Number: 1
Name: Name 3 Number: 2
Name: Name 4 Number: 3
Name: Name 5 Number: 4
Name: Name 6 Number: 5
Name: Name 7 Number: 6
Name: Name 8 Number: 7
Name: Name 9 Number: 8
Name: Name 10 Number: 9
Name: Name 11 Number: 10
</pre>
<p>If you want to retrieve the documents directly as JSON, we can write:</p>
<pre>
<strong>var</strong> json: RawUTF8;
...
json := Coll.FindJSON(null,null);
writeln(json);
...
</pre>
<p>This will append the following to the console:</p>
<pre>
[{"_id":1,"Name":"Name 2","Number":1},{"_id":2,"Name":"Name 3","Number":2},{"_id
":3,"Name":"Name 4","Number":3},{"_id":4,"Name":"Name 5","Number":4},{"_id":5,"N
ame":"Name 6","Number":5},{"_id":6,"Name":"Name 7","Number":6},{"_id":7,"Name":"
Name 8","Number":7},{"_id":8,"Name":"Name 9","Number":8},{"_id":9,"Name":"Name 1
0","Number":9},{"_id":10,"Name":"Name 11","Number":10}]
</pre>
<p>You can note that <code>FindJSON()</code> has two properties, which are the
<code>Query</code> filter, and a <code>Projection</code> mapping (similar to
the column names in a <code>SELECT col1,col2</code>).<br />
So we may have written:</p>
<pre>
json := Coll.FindJSON('{_id:?}',[5]);
writeln(json);
</pre>
<p>Which would output:</p>
<pre>
[{"_id":5,"Name":"Name 6","Number":5}]
</pre>
<p>Note here than we used an overloaded <code>FindJSON()</code> method, which
accept the <em>MongoDB</em> extended syntax (here, the field name is unquoted),
and parameters as variables.</p>
<p>We can specify a projection:</p>
<pre>
json := Coll.FindJSON('{_id:?}',[5],'{Name:1}');
writeln(json);
</pre>
<p>Which will only return the "Name" and "_id" fields (since <code>_id</code>
is, by <em>MongoDB</em> convention, always returned:</p>
<pre>
[{"_id":5,"Name":"Name 6"}]
</pre>
<p>To return only the "Name" field, you can specify <code>'_id:0,Name:1'</code>
as extended JSON for the <em>projection</em> parameter.</p>
<pre>
[{"Name":"Name 6"}]
</pre>
<p>There are other methods able to retrieve data, also directly as BSON binary
data. They will be used for best speed e.g. in conjunction with our ORM, but
for most end-user code, using <code>TDocVariant</code> is safer and easier to
maintain.</p>
<h3>Updating or deleting documents</h3>
<p>The <code>TMongoCollection</code> class has some methods dedicated to alter
existing documents.</p>
<p>At first, the <code>Save()</code> method can be used to update a document
which has been first retrieved:</p>
<pre>
doc := Coll.FindOne(5);
doc.Name := 'New!';
Coll.Save(doc);
writeln('Name: ',Coll.FindOne(5).Name);
</pre>
<p>Which will write:</p>
<pre>
Name: New!
</pre>
<p>Note that we used here an integer value (5) as key, but we may use an
<em>ObjectID</em> instead, if needed.</p>
<p>The <code>Coll.Save()</code> method could be changed into
<code>Coll.Update()</code>, which expects an explicit <code>Query</code>
operator, in addition to the updated document content:</p>
<pre>
doc := Coll.FindOne(5);
doc.Name := 'New!';
Coll.Update(BSONVariant(['_id',5]),doc);
writeln('Name: ',Coll.FindOne(5).Name);
</pre>
<p>Note that by <em>MongoDB</em>'s design, any call to <code>Update()</code>
will <em>replace</em> the whole document.</p>
<p>For instance, if you write:</p>
<pre>
writeln('Before: ',Coll.FindOne(3));
Coll.Update('{_id:?}',[3],'{Name:?}',['New Name!']);
writeln('After: ',Coll.FindOne(3));
</pre>
<p>Then the <code>Number</code> field will disappear!</p>
<pre>
Before: {"_id":3,"Name":"Name 4","Number":3}
After: {"_id":3,"Name":"New Name!"}
</pre>
<p>If you need to update only some fields, you will have to use the
<code>$set</code> modifier:</p>
<pre>
writeln('Before: ',Coll.FindOne(4));
Coll.Update('{_id:?}',[4],'{$set:{Name:?}}',['New Name!']);
writeln('After: ',Coll.FindOne(4));
</pre>
<p>Which will write on the console the value as expected:</p>
<pre>
Before: {"_id":4,"Name":"Name 5","Number":4}
After: {"_id":4,"Name":"New Name!","Number":4}
</pre>
<p>Now the <code>Number</code> field remains untouched.</p>
<p>You can also use the <code>Coll.UpdateOne()</code> method, which will update
the supplied fields, and leave the non specified fields untouched:</p>
<pre>
writeln('Before: ',Coll.FindOne(2));
Coll.UpdateOne(2,_Obj(['Name','NEW']));
writeln('After: ',Coll.FindOne(2));
</pre>
<p>Which will output as expected:</p>
<pre>
Before: {"_id":2,"Name":"Name 3","Number":2}
After: {"_id":2,"Name":"NEW","Number":2}
</pre>
<p>You can refer to the documentation of the <code>SynMongoDB.pas</code> unit,
to find out all functions, classes and methods available to work with
<em>MongoDB</em>.</p>
<h3>Write Concern and Performance</h3>
<p>You can take a look at the <code>MongoDBTests.dpr</code> sample - located in
the <code>SQLite3- MongoDB</code> sub-folder of the source code repository, and
the <code>TTestDirect</code> classes, to find out some performance
information.</p>
<p>In fact, this <code>TTestDirect</code> is inherited twice, to run the same
tests with diverse write concern.</p>
<p>The difference between the two classes will take place at client
initialization:</p>
<pre>
<strong>procedure</strong> TTestDirect.ConnectToLocalServer;
...
fClient := TMongoClient.Create('localhost',27017);
<strong>if</strong> ClassType=TTestDirectWithAcknowledge <strong>then</strong>
<span style="background-color:yellow;">fClient.WriteConcern := wcAcknowledged <strong>else</strong></span>
<strong>if</strong> ClassType=TTestDirectWithoutAcknowledge <strong>then</strong>
<span style="background-color:yellow;">fClient.WriteConcern := wcUnacknowledged;</span>
...
</pre>
<p><code>wcAcknowledged</code> is the default safe mode: the <em>MongoDB</em>
server confirms the receipt of the write operation. Acknowledged write concern
allows clients to catch network, duplicate key, and other errors. But it adds
an additional round-trip from the client to the server, and wait for the
command to be finished before returning the error status: so it will slow down
the write process.</p>
<p>With <code>wcUnacknowledged</code>, <em>MongoDB</em> does not acknowledge
the receipt of write operation. Unacknowledged is similar to errors ignored;
however, drivers attempt to receive and handle network errors when possible.
The driver's ability to detect network errors depends on the system's
networking configuration.</p>
<p>The speed difference between the two is worth mentioning, as stated by the
regression tests status, running on a local <em>MongoDB</em> instance:</p>
<pre>
1. Direct access
<br />1.1. Direct with acknowledge:
- Connect to local server: 6 assertions passed 4.72ms
- Drop and prepare collection: 8 assertions passed 9.38ms
<span style="background-color:yellow;"> - Fill collection: 15,003 assertions passed 558.79ms</span>
<span style="background-color:yellow;"> 5000 rows inserted in 548.83ms i.e. 9110/s, aver. 109us, 3.1 MB/s</span>
- Drop collection: no assertion 856us
<span style="background-color:yellow;"> - Fill collection bulk: 2 assertions passed 74.59ms</span>
<span style="background-color:yellow;"> 5000 rows inserted in 64.76ms i.e. 77204/s, aver. 12us, 7.2 MB/s</span>
- Read collection: 30,003 assertions passed 2.75s
5000 rows read at once in 9.66ms i.e. 517330/s, aver. 1us, 39.8 MB/s
<span style="background-color:yellow;"> - Update collection: 7,503 assertions passed 784.26ms</span>
<span style="background-color:yellow;"> 5000 rows updated in 435.30ms i.e. 11486/s, aver. 87us, 3.7 MB/s</span>
<span style="background-color:yellow;"> - Delete some items: 4,002 assertions passed 370.57ms</span>
<span style="background-color:yellow;"> 1000 rows deleted in 96.76ms i.e. 10334/s, aver. 96us, 2.2 MB/s</span>
Total failed: 0 / 56,527 - Direct with acknowledge PASSED 4.56s
<br />1.2. Direct without acknowledge:
- Connect to local server: 6 assertions passed 1.30ms
- Drop and prepare collection: 8 assertions passed 8.59ms
<span style="background-color:yellow;"> - Fill collection: 15,003 assertions passed 192.59ms</span>
<span style="background-color:yellow;"> 5000 rows inserted in 168.50ms i.e. 29673/s, aver. 33us, 4.4 MB/s</span>
- Drop collection: no assertion 845us
<span style="background-color:yellow;"> - Fill collection bulk: 2 assertions passed 68.54ms</span>
<span style="background-color:yellow;"> 5000 rows inserted in 58.67ms i.e. 85215/s, aver. 11us, 7.9 MB/s</span>
- Read collection: 30,003 assertions passed 2.75s
5000 rows read at once in 9.99ms i.e. 500150/s, aver. 1us, 38.5 MB/s
<span style="background-color:yellow;"> - Update collection: 7,503 assertions passed 446.48ms</span>
<span style="background-color:yellow;"> 5000 rows updated in 96.27ms i.e. 51933/s, aver. 19us, 7.7 MB/s</span>
<span style="background-color:yellow;"> - Delete some items: 4,002 assertions passed 297.26ms</span>
<span style="background-color:yellow;"> 1000 rows deleted in 19.16ms i.e. 52186/s, aver. 19us, 2.8 MB/s</span>
Total failed: 0 / 56,527 - Direct without acknowledge PASSED 3.77s
</pre>
<p>As you can see, the reading speed is not affected by the <em>Write
Concern</em> settings.<br />
But data writing can be multiple times faster, when each write command is not
acknowledged.</p>
<p>Since there is no error handling, <code>wcUnacknowledged</code> is not to be
used on production. You may use it for replication, or for data consolidation,
e.g. feeding a database with a lot of existing data as fast as possible.</p>
<p>Stay tuned for the next article, which <a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-mORMot-ORM-ODM">will detail the <em>MongoDB</em>
integration 's ORM</a>... and the last one of the series, with <a href="https://blog.synopse.info?post/post/2014/05/07/MongoDB-mORMot-benchmark">benchmarks of <em>MongoDB</em>
against SQL engines, via our ORM</a>...<br />
Feedback is <a href="http://synopse.info/forum/viewtopic.php?id=1742">welcome
on our forum</a>, as usual!</p>mORMot on GitHuburn:md5:49968ae9786ffddbe238f5ad2f34d8cd2014-05-02T14:31:00+02:002020-07-03T09:29:59+02:00AB4327-GANDImORMot FrameworkblogDelphiFossilgitgithubmORMotOpenSourceSource<p>There was a long-standing request from customers, about putting all our
source code repository to <a href="http://github.com">GitHub</a>.</p>
<p>We like a lot our self-hosted <a href="https://www.fossil-scm.org">Fossil
repository</a>, and will continue to use it as our main system, including issue
tracking and wiki, for <a href="http://synopse.info/fossil">our official web
site</a>.</p>
<p><img src="http://wiki.scribus.net/wiki/images/4/40/Octocat,_a_Mascot_of_Github.jpg" alt="" /></p>
<p>But we created a repository on GitHub, on <a href="https://github.com/synopse/mORMot">https://github.com/synopse/mORMot</a></p> <p><a href="http://git-scm.com">Git</a>, as a source control manager system,
sounds pretty good for handling the source code tree.<br />
Pretty similar to fossil (it is a distributed SCM), but much heavier and
difficult to install/configure.</p>
<p>But I would not say the same for the "GitHub for Windows" tool.<br />
For a somewhat huge project like <em>mORMot</em>, it is slow, non responsive,
and, after waiting a lot, uses more than 400 MB just to display the
repository.<br />
Sometimes, it just crashes. The UI looks nice, but is very difficult to work
with.<br />
Another awfully fat and non working WPF / .Net application!</p>
<p>Tickets and wiki will remain on our own site.<br />
But source code repository will be committed on both system, i.e. our
self-hosted Fossil repository, and GitHub.</p>
<p>We wrote a <a href="https://github.com/synopse/mORMot/tree/master/SQLite3/Documentation/SourceCodeRep">
simple tool to make this easy</a>:</p>
<p><img src="https://blog.synopse.info?post/public/mORMot/SourceCodeRep.png" alt="" title="SourceCodeRep Main Window, mai 2014" /></p>
<p>"GitHub for Windows" was not an option for us!<br />
What a bloatware!</p>
<p>Feedback is <a href="http://synopse.info/forum/viewtopic.php?pid=10413#p10413">welcome on our
forum</a>, as usual.</p>Mustache Logic-less templates for Delphi - part 3urn:md5:177e56c3d5bed092677caa99fb1f52322014-04-28T13:39:00+02:002014-04-28T15:55:54+02:00AB4327-GANDIOpen Source librariesblogBusinessRulesCrossPlatformDelphiDocumentationGoodPracticeHTMLinterfaceJavaScriptmORMotmultithreadMustacheMVCOpenSourceperformanceSOASourcesyntaxtemplatesUserInterfaceweb<p><em><a href="https://blog.synopse.info?post/post/2012/03/07/Interface-based-services">Mustache</a></em> is
a well-known <em>logic-less</em> template engine.<br />
There is plenty of Open Source implementations around (including in JavaScript,
which can be very convenient for AJAX applications on client side, for
instance).<br />
For <em>mORMot</em>, we created the first pure Delphi implementation of it,
with a perfect integration with other bricks of the framework.</p>
<p><img src="http://santateresaschool.org/v2/wp-content/uploads/2014/03/mustache-much-8.jpg" alt="" width="250" height="292/" /></p>
<p>In last part of this series of blog articles, we will introduce the
<em>Mustache</em> library included within <em>mORMot</em> source code
tree.<br />
You can <a href="https://blog.synopse.info?post/public/Documents/SynMustache.pdf">download this documentation
as one single pdf file</a>.</p> <p>Part of our <em>mORMot</em> framework, we implemented an optimized
<em>Mustache</em> template engine in the <code>SynMustache</code> unit:</p>
<ul>
<li>It is the first Delphi implementation of <em>Mustache</em>;</li>
<li>It has a separate parser and renderer (so you can compile your templates
ahead of time);</li>
<li>The parser features a shared cache of compiled templates;</li>
<li>It passes all official <a href="http://github.com/mustache/spec"><em>Mustache</em> specification tests</a> -
including all weird whitespace process;</li>
<li>External partials can be supplied as <code>TSynMustachePartials</code>
dictionaries;</li>
<li><code>{{.}}</code>, <code>{{-index}}</code> and <code>{{"some text}}</code>
pseudo-variables were added to the standard <em>Mustache</em> syntax;</li>
<li><code>{{#-first}}</code>, <code>{{#-last}}</code> and
<code>{{#-odd}}</code> pseudo-sections were added to the standard
<em>Mustache</em> syntax;</li>
<li>Internal partials can be defined via <code>{{<partial}}</code> - also a
nice addition to the standard <em>Mustache</em> syntax;</li>
<li>It allows the data context to be supplied as JSON or our <a href="https://blog.synopse.info?post/post/2014/02/25/TDocVariant-custom-variant-type"><em>TDocVariant</em> custom
type</a>;</li>
<li>Almost no memory allocation is performed during the rendering;</li>
<li>It is natively UTF-8, from the ground up, with optimized conversion of any
string data;</li>
<li>Performance has been tuned and grounded in <code>SynCommons</code>'s
optimized code;</li>
<li>Each parsed template is thread-safe and re-entrant;</li>
<li>It follows the <em><a href="https://blog.synopse.info?post/post/2011/11/27/SOLID-design-principles">Open/Close principle</a></em> so
that any aspect of the process can be customized and extended (e.g. for any
kind of data context);</li>
<li>It is perfectly integrated with the other bricks of our <em>mORMot</em>
framework, ready to implement dynamic web sites with true 10 design, and full
separation of concerns in the views written in <em>Mustache</em>, the
controllers being e.g. interface-based services;</li>
<li>API is flexible and easy to use.</li>
</ul>
<h3>Variables</h3>
<p>Now, let's see some code.</p>
<p>First, we define our needed variables:</p>
<pre>
<strong>var</strong> mustache: TSynMustache;
doc: <strong>variant</strong>;
</pre>
<p>In order to parse a template, you just need to call:</p>
<pre>
mustache := TSynMustache.Parse(
'Hello {{name}}'#13#10'You have just won {{value}} dollars!');
</pre>
<p>It will return a compiled instance of the template.<br />
The <code>Parse()</code> class method will use the shared cache, so you won't
need to release the <code>mustache</code> instance once you are done with it:
no need to write a <code>try ... finally mustache.Free; end</code> block.</p>
<p>You can use a <code>TDocVariant</code> to supply the context data (with
late-binding):</p>
<pre>
TDocVariant.New(doc);
doc.name := 'Chris';
doc.value := 10000;
</pre>
<p>As an alternative, you may have defined the context data as such:</p>
<pre>
doc := _ObjFast(['name','Chris','value',1000]);
</pre>
<p>Now you can render the template with this context:</p>
<pre>
html := mustache.Render(doc);
<em>// now html='Hello Chris'#13#10'You have just won 10000 dollars!'</em>
</pre>
<p>If you want to supply the context data as JSON, then render it, you may
write:</p>
<pre>
mustache := TSynMustache.Parse(
'Hello {{value.name}}'#13#10'You have just won {{value.value}} dollars!');
html := mustache.RenderJSON('{value:{name:"Chris",value:10000}}');
<em>// now html='Hello Chris'#13#10'You have just won 10000 dollars!'</em>
</pre>
<p>Note that here, the JSON is supplied with an extended syntax (i.e. field
names are unquoted), and that <code>TSynMustache</code> is able to identify a
dotted-named variable within the execution context.</p>
<p>As an alternative, you could use the following syntax to create the data
context as JSON, with a set of parameters, therefore easier to work with in
real code storing data in variables (for instance, any <code>string</code>
variable is quoted as expected by JSON, and converted into UTF-8):</p>
<pre>
mustache := TSynMustache.Parse(
'Hello {{name}}'#13#10'You have just won {{value}} dollars!');
html := mustache.RenderJSON('{name:?,value:?}',[],['Chris',10000]);
html='Hello Chris'#13#10'You have just won 10000 dollars!'
</pre>
<p>You can find in the <code>mORMot.pas</code> unit the
<code>ObjectToJSON()</code> function which is able to transform any
<code>TPersistent</code> instance into valid JSON content, ready to be supplied
to a <code>TSynMustache</code> compiled instance.<br />
If the object's published properties have some getter functions, they will be
called on the fly to process the data (e.g. returning 'FirstName Name' as
FullName by concatenating both sub-fields).</p>
<h3>Sections</h3>
<p>Sections are handled as expected:</p>
<pre>
mustache := TSynMustache.Parse('Shown.{{#person}}As {{name}}!{{/person}}end{{name}}');
html := mustache.RenderJSON('{person:{age:?,name:?}}',[10,'toto']);
<em>// now html='Shown.As toto!end'</em>
</pre>
<p>Note that the sections change the data context, so that within the
<code>#person</code> section, you can directly access to the data context
<code>person</code> member, i.e. writing directly <code>name</code></p>
<p>It supports also inverted sections:</p>
<pre>
mustache := TSynMustache.Parse('Shown.{{^person}}Never shown!{{/person}}end');
html := mustache.RenderJSON('{person:true}');
<em>// now html='Shown.end'</em>
</pre>
<p>To render a list of items, you can write for instance (using the
<code>.</code> pseudo-variable):</p>
<pre>
mustache := TSynMustache.Parse('{{#things}}{{.}}{{/things}}');
html := mustache.RenderJSON('{things:["one", "two", "three"]}');
<em>// now html='onetwothree'</em>
</pre>
<p>The <code>-index</code> pseudo-variable allows to numerate the list items,
when rendering:</p>
<pre>
mustache := TSynMustache.Parse(
'My favorite things:'#$A'{{#things}}{{-index}}. {{.}}'#$A'{{/things}}');
html := mustache.RenderJSON('{things:["Peanut butter", "Pen spinning", "Handstands"]}');
<em>// now html='My favorite things:'#$A'1. Peanut butter'#$A'2. Pen spinning'#$A+</em>
<em>// '3. Handstands'#$A,'-index pseudo variable'</em>
</pre>
<h3>Partials</h3>
<p>External partials (i.e. standard <em>Mustache</em> partials) can be defined
using <code>TSynMustachePartials</code>. You can define and maintain a list of
<code>TSynMustachePartials</code> instances, or you can use a one-time partial,
for a given rendering process, as such:</p>
<pre>
mustache := TSynMustache.Parse('{{>partial}}'#$A'3');
html := mustache.RenderJSON('{}',TSynMustachePartials.CreateOwned(['partial','1'#$A'2']));
<em>// now html='1'#$A'23','external partials'</em>
</pre>
<p>Here <code>TSynMustachePartials.CreateOwned()</code> expects the partials to
be supplied as name/value pairs.</p>
<p>Internal partials (one of the <code>SynMustache</code> extensions), can be
defined directly in the main template:</p>
<pre>
mustache := TSynMustache.Parse('{{<partial}}1'#$A'2{{name}}{{/partial}}{{>partial}}4');
html := mustache.RenderJSON('{name:3}');
<em>// now html='1'#$A'234','internal partials'</em>
</pre>
<h3>Internationalization</h3>
<p>You can define <code>{{"some text}}</code> pseudo-variables in your
templates, which text will be supplied to a callback, ready to be transformed
on the fly: it may be convenient for <em>i18n</em> of web applications.</p>
<p>By default, the text will be written directly to the output buffer, but you
can define a callback which may be used e.g. for text translation:</p>
<pre>
<strong>procedure</strong> TTestLowLevelTypes.MustacheTranslate(<strong>var</strong> English: <strong>string</strong>);
<strong>begin</strong>
<strong>if</strong> English='Hello' <strong>then</strong>
English := 'Bonjour' <strong>else</strong>
<strong>if</strong> English='You have just won' <strong>then</strong>
English := 'Vous venez de gagner';
<strong>end</strong>;
</pre>
<p>Of course, in a real application, you may assign one
<code>TLanguageFile.Translate(var English: string)</code> method, as defined in
the <code>mORMoti18n.pas</code> unit.</p>
<p>Then, you will be able to define your template as such:</p>
<pre>
mustache := TSynMustache.Parse(
'{{"Hello}} {{name}}'#13#10'{{"You have just won}} {{value}} {{"dollars}}!');
html := mustache.RenderJSON('{name:?,value:?}',[],['Chris',10000],<strong>nil</strong>,MustacheTranslate);
<em>// now html='Bonjour Chris'#$D#$A'Vous venez de gagner 10000 dollars!'</em>
</pre>
<p>All text has indeed been translated as expected.</p>
<p>Feedback is <a href="http://synopse.info/forum/viewtopic.php?id=1720">welcome on our forum, as
usual</a>!</p>Mustache Logic-less templates for Delphi - part 2urn:md5:933cfcf13ca723e4ff09e08550ceccfb2014-04-28T13:38:00+02:002014-04-28T15:55:39+02:00AB4327-GANDIOpen Source librariesblogBusinessRulesCrossPlatformDelphiDocumentationGoodPracticeHTMLinterfaceJavaScriptmORMotmultithreadMustacheMVCOpenSourceperformanceSOASourcesyntaxtemplatesUserInterfaceweb<p><em><a href="http://mustache.github.io/">Mustache</a></em> is a well-known
<em>logic-less</em> template engine.<br />
There is plenty of Open Source implementations around (including in JavaScript,
which can be very convenient for AJAX applications on client side, for
instance).<br />
For <em>mORMot</em>, we created the first pure Delphi implementation of it,
with a perfect integration with other bricks of the framework.</p>
<p><img src="http://santateresaschool.org/v2/wp-content/uploads/2014/03/mustache-much-8.jpg" alt="" width="250" height="292/" /></p>
<p>In this second part of this series of blog articles, we will introduce the
<em>Mustache</em> syntax.<br />
You can <a href="https://blog.synopse.info?post/public/Documents/SynMustache.pdf">download this documentation
as one single pdf file</a>.</p> <p>The <em>Mustache</em> template logic-less language has five types of
tags:</p>
<ol>
<li>Variables;</li>
<li>Sections;</li>
<li>Inverted Sections;</li>
<li>Comments;</li>
<li>Partials.</li>
</ol>
<p>All those tags will be identified with mustaches, i.e.
<code>{{...}}</code>.<br />
Anything found in a template of this form is interpreted as a template
marker.<br />
All other text is considered formatting text and is output verbatim at template
expansion time.</p>
<table>
<tbody>
<tr>
<td><strong>Marker</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td><code>{{variable}}</code></td>
<td>The <code>variable</code> name will be searched recursively within the
current context (possibly with dotted names), and, if found, will be written as
escaped HTML.<br />
If there is no such key, nothing will be rendered.</td>
</tr>
<tr>
<td><code>{{{variable}}}<br />
{{& variable}}</code></td>
<td>The <code>variable</code> name will be searched recursively within the
current context, and, if found, will be written directly, <em>without any HTML
escape</em>.<br />
If there is no such key, nothing will be rendered.</td>
</tr>
<tr>
<td><code>{{#section}}<br /></code>...<br />
<code>{{/section}}</code></td>
<td>Defines a block of text, aka <em>section</em>, which will be rendered
depending of the <code>section</code> variable value, as searched in the
current context:<br />
- If <code>section</code> equals <code>false</code> or is an <em>empty
list</em> <code>[]</code>, the whole block won't be rendered;<br />
- If <code>section</code> is non-<code>false</code> but not a list, it will be
used as the context for a single rendering of the block;<br />
- If <code>section</code> is a non-empty list, the text in the block will be
rendered once for each item in the list - the context of the block will be set
to the current item for each iteration.</td>
</tr>
<tr>
<td><code>{{^section}}<br /></code>...<br />
<code>{{/section}}</code></td>
<td>Defines a block of text, aka <em>inverted section</em>, which will be
rendered depending of the <code>section</code> variable <em>inverted</em>
value, as searched in the current context:<br />
- If <code>section</code> equals <code>false</code> or is an <em>empty
list</em>, the whole block <em>will</em> be rendered;<br />
- If <code>section</code> is non-<code>false</code> or a non-empty list, it
won't be rendered.</td>
</tr>
<tr>
<td><code>{{! comment}}</code></td>
<td>The comment text will just be ignored.</td>
</tr>
<tr>
<td><code>{{>partial}}</code></td>
<td>The <code>partial</code> name will be searched within the registered
<em>partials list</em>, then will be executed at run-time (so recursive
partials are possible), with the current execution context.</td>
</tr>
<tr>
<td><code>{{=...=}}</code></td>
<td>The delimiters (i.e. by default <code>{{ }}</code>) will be replaced by the
specified characters (may be convenient when double-braces may appear in the
text).</td>
</tr>
</tbody>
</table>
<p>In addition to those standard markers, the <em>mORMot</em> implementation of
<em>Mustache</em> features:</p>
<table>
<tbody>
<tr>
<td><strong>Marker</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td><code>{{.}}</code></td>
<td>This pseudo-variable refers to the context object itself instead of one of
its members. This is particularly useful when iterating over lists.</td>
</tr>
<tr>
<td><code>{{-index}}</code></td>
<td>This pseudo-variable returns the current item number when iterating over
lists, starting counting at 1</td>
</tr>
<tr>
<td><code>{{#-first}}<br /></code>...<br />
<code>{{/-first}}</code></td>
<td>Defines a block of text (pseudo-section), which will be rendered - or
<em>not</em> rendered for inverted <code>{{^-first}}</code> - for the
<em>first</em> item when iterating over lists</td>
</tr>
<tr>
<td><code>{{#-last}}<br /></code>...<br />
<code>{{/-last}}</code></td>
<td>Defines a block of text (pseudo-section), which will be rendered - or
<em>not</em> rendered for inverted <code>{{^-last}}</code> - for the
<em>last</em> item when iterating over lists</td>
</tr>
<tr>
<td><code>{{#-odd}}<br /></code>...<br />
<code>{{/-odd}}</code></td>
<td>Defines a block of text (pseudo-section), which will be rendered - or
<em>not</em> rendered for inverted <code>{{^-odd}}</code> - for the
<em>odd</em> item number when iterating over lists: it can be very usefull e.g.
to display a list with alternating row colors</td>
</tr>
<tr>
<td><code>{{<partial}}<br /></code>...<br />
<code>{{/partial}}</code></td>
<td>Defines an in-lined <em>partial</em> - to be called later via
<code>{{>partial}}</code> - within the scope of the current template</td>
</tr>
<tr>
<td><code>{{"some text}}</code></td>
<td>This pseudo-variable will supply the given text to a callback, which will
for instance transform its content (e.g. translate it), before writing it to
the output</td>
</tr>
</tbody>
</table>
<p>This set of markers will allow to easily write any kind of content, without
any explicit logic nor nested code.<br />
As a major benefit, the template content could be edited and verified without
the need of any <em>Mustache</em> compiler, since all those
<code>{{...}}</code> markers will identify very clearly the resulting
layout.</p>
<h3>Variables</h3>
<p>A typical Mustache template:</p>
<pre>
Hello {{name}}
You have just won {{value}} dollars!
Well, {{taxed_value}} dollars, after taxes.
</pre>
<p>Given the following hash:</p>
<pre>
{
"name": "Chris",
"value": 10000,
"taxed_value": 6000
}
</pre>
<p>Will produce the following:</p>
<pre>
Hello Chris
You have just won 10000 dollars!
Well, 6000 dollars, after taxes.
</pre>
<p>You can note that <code>variable</code> tags are escaped for HTML by
default. This is a mandatory security feature. In fact, all web applications
which create HTML documents can be vulnerable to Cross-Site-Scripting (XSS)
attacks unless data inserted into a template is appropriately sanitized and/or
escaped. With Mustache, this is done by default. Of course, you can override it
and force to <em>not-escape</em> the value, using <code>variable or &
variable</code>.</p>
<p>For instance:</p>
<table>
<tbody>
<tr>
<td><strong>Template</strong></td>
<td><strong>Context</strong></td>
<td><strong>Output</strong></td>
</tr>
<tr>
<td><code>* {{name}}<br />
* {{age}}<br />
* {{company}}<br />
* {{{company}}}</code></td>
<td><code>{<br />
"name": "Chris",<br />
"company": "<b>GitHub</b>"<br />
}</code></td>
<td><code>* Chris<br />
*<br />
* &lt;b&gt;GitHub&lt;/b&gt;<br />
* <b>GitHub</b></code></td>
</tr>
</tbody>
</table>
<p>Variables resolve names within the current context with an optional dotted
syntax, for instance:</p>
<table>
<tbody>
<tr>
<td><strong>Template</strong></td>
<td><strong>Context</strong></td>
<td><strong>Output</strong></td>
</tr>
<tr>
<td><code>* {{people.name}}<br />
* {{people.age}}<br />
* {{people.company}}<br />
* {{{people.company}}}</code></td>
<td><code>{<br />
"name": "Chris",<br />
"company": "<b>GitHub</b>"<br />
}</code></td>
<td><code>* Chris<br />
*<br />
* &lt;b&gt;GitHub&lt;/b&gt;<br />
* <b>GitHub</b></code></td>
</tr>
</tbody>
</table>
<h3>Sections</h3>
<p><em>Sections</em> render blocks of text one or more times, depending on the
value of the key in the current context.</p>
<p>In our "wining template" above, what happen if we do want to hide the tax
details?<br />
In most script languages, we may write an <code>if ...</code> block within the
template. This is what <em>Mustache</em> avoids. So we define a section, which
will be rendered on need.</p>
<p>The template becomes:</p>
<pre>
Hello {{name}}
You have just won {{value}} dollars!
{{#in_ca}}
Well, {{taxed_value}} dollars, after taxes.
{{/in_ca}}
</pre>
<p>Here, we created a new section, named <code>in_ca</code>.</p>
<p>Given the hash value of <code>in_ca</code> (and its presence), the section
will be rendered, or not:</p>
<table>
<tbody>
<tr>
<td><strong>Context</strong></td>
<td><strong>Output</strong></td>
</tr>
<tr>
<td><code>{<br />
"name": "Chris",<br />
"value": 10000,<br />
"taxed_value": 6000,<br />
"in_ca": true<br />
}</code></td>
<td><code>Hello Chris<br />
You have just won 10000 dollars!<br />
Well, 6000 dollars, after taxes.</code></td>
</tr>
<tr>
<td><code>{<br />
"name": "Chris",<br />
"value": 10000,<br />
"taxed_value": 6000,<br />
"in_ca": false<br />
}</code></td>
<td><code>Hello Chris<br />
You have just won 10000 dollars!</code></td>
</tr>
<tr>
<td><code>{<br />
"name": "Chris",<br />
"value": 10000,<br />
"taxed_value": 6000<br />
}</code></td>
<td><code>Hello Chris<br />
You have just won 10000 dollars!</code></td>
</tr>
</tbody>
</table>
<p>Sections also change the context of its inner block. It means that the
section variable content becomes the top-most context which will be used to
identify any supplied variable key.</p>
<p>Therefore, the following context will be perfectly valid: we can define
<code>taxed_value</code> as a member of <code>in_ca</code>, and it will be
rendered directly, since it is part of the new context.</p>
<table>
<tbody>
<tr>
<td><strong>Context</strong></td>
<td><strong>Output</strong></td>
</tr>
<tr>
<td><code>{<br />
"name": "Chris",<br />
"value": 10000,<br />
"in_ca": {<br />
"taxed_value": 6000<br />
}<br />
}<br />
<br /></code></td>
<td><code>Hello Chris<br />
You have just won 10000 dollars!<br />
Well, 6000 dollars, after taxes.</code></td>
</tr>
<tr>
<td><code>{<br />
"name": "Chris",<br />
"value": 10000,<br />
"taxed_value": 6000<br />
}</code></td>
<td><code>Hello Chris<br />
You have just won 10000 dollars!</code></td>
</tr>
<tr>
<td><code>{<br />
"name": "Chris",<br />
"value": 10000,<br />
"taxed_value": 3000,<br />
"in_ca": {<br />
"taxed_value": 6000<br />
}<br />
}<br /></code></td>
<td><code>Hello Chris<br />
You have just won 10000 dollars!<br />
Well, 6000 dollars, after taxes.</code></td>
</tr>
</tbody>
</table>
<p>In the latest context above, there are two <code>taxed_value</code>
variables.<br />
The engine will use the one defined by the context in the <code>in_ca</code>
section, i.e. <code>in_ca.taxed_value</code>; the one defined at the root
context level (which equals 3000) is just ignored.</p>
<p>If the variable pointed by the section name is a list, the text in the block
will be rendered once for each item in the list.<br />
The context of the block will be set to the current item for each
iteration.</p>
<p>In this way we can loop over collections.<br />
<em>Mustache</em> allows any depth of nested loops (e.g. any level of
master/details information).</p>
<table>
<tbody>
<tr>
<td><strong>Template</strong></td>
<td><strong>Context</strong></td>
<td><strong>Output</strong></td>
</tr>
<tr>
<td><code>{{#repo}}<br />
<b>name</b><br />
{{/repo}}</code></td>
<td><code>{<br />
"repo": [<br />
{"name": "resque"},<br />
{"name": "hub"},<br />
{"name": "rip"} <br />
]<br />
}</code></td>
<td><code><b>resque</b><br />
<b>hub</b><br />
<b>rip</b></code></td>
</tr>
<tr>
<td><code>{{#repo}}<br />
<b>.</b><br />
{{/repo}}</code></td>
<td><code>{<br />
"repo":<br />
["resque", "hub", "rip"]<br />
}</code></td>
<td><code><b>resque</b><br />
<b>hub</b><br />
<b>rip</b></code></td>
</tr>
</tbody>
</table>
<p>The latest template makes use of the <code>.</code> pseudo-variable, which
allows to render the current item of the list.</p>
<h3>Inverted Sections</h3>
<p>An inverted section begins with a caret (<code>^</code>) and ends as a
standard (non-inverted) section.<br />
They may render text once, based on the <em>inverse</em> value of the key. That
is, the text block will be rendered if the key doesn't exist, is false, or is
an empty list.</p>
<p>Inverted sections are usually defined after a standard section, to render
some message in case no information will be written in the non-inverted
section:</p>
<table>
<tbody>
<tr>
<td><strong>Template</strong></td>
<td><strong>Context</strong></td>
<td><strong>Output</strong></td>
</tr>
<tr>
<td><code>{{#repo}}<br />
<b>.</b><br />
{{/repo}}<br />
{{^repo}}<br />
No repos <img src="https://blog.synopse.info?pf=sad.svg" alt=":(" class="smiley" /><br />
{{/repo}}</code></td>
<td><code>{<br />
"repo":<br />
[]<br />
}</code></td>
<td><code>No repos <img src="https://blog.synopse.info?pf=sad.svg" alt=":(" class="smiley" /></code></td>
</tr>
</tbody>
</table>
<h3>Partials</h3>
<p>Partials are some kind of external sub-templates which can be included
within a main template, for instance to follow the same rendering at several
places.<br />
Just like functions in code, they do ease template maintainability and spare
development time.</p>
<p>Partials are rendered at runtime (as opposed to compile time), so recursive
partials are possible. Just avoid infinite loops.<br />
They also inherit the calling context, so can easily be re-used within a list
section, or together with plain variables.</p>
<p>In practice, partials shall be supplied together with the data context -
they could be seen as "template context".</p>
<p>For example, this "main" template uses a <code>> user</code> partial:</p>
<pre>
<h2>Names</h2>
{{#names}}
{{> user}}
{{/names}}
</pre>
<p>With the following template registered as "user":</p>
<pre>
<strong>{{name}}</strong>
</pre>
<p>Can be thought of as a single, expanded template:</p>
<pre>
<h2>Names</h2>
{{#names}}
<strong>{{name}}</strong>
{{/names}}
</pre>
<p>In <em>mORMot</em>'s implementations, you can also create some
<em>internal</em> partials, defined as <code><partial ... /partial</code>
pseudo-sections.<br />
It may decrease the need of maintaining multiple template files, and refine the
rendering layout.</p>
<p>For instance, the previous template may be defined at once:</p>
<pre>
<h2>Names</h2>
{{#names}}
{{>user}}
{{/names}}
{{<user}}
<strong>{{name}}</strong>
{{/user}}
</pre>
<p>The same file will define both the partial and the main template. </p>
<p>Note that we defined the internal partial after the main template, but we
may have defined it anywhere in the main template logic: internal partials
definitions are ignored when rendering the main template, just like
comments.</p>
<p><a href="https://blog.synopse.info?post/post/2014/04/28/Mustache-Logic-less-templates-for-Delphi-part-3">Next article
will detail the <em>Mustache</em> engine as implemented in <em>mORMot</em>'s
source code tree</a>.<br />
Now, a bit of practice!</p>Mustache Logic-less templates for Delphi - part 1urn:md5:5e26b8841fcba48a85895f3f787cd0172014-04-28T13:37:00+02:002014-04-28T15:55:15+02:00AB4327-GANDIOpen Source librariesblogBusinessRulesCrossPlatformDelphiDocumentationGoodPracticeHTMLinterfaceJavaScriptmORMotmultithreadMustacheMVCOpenSourceperformanceSOASourcesyntaxtemplatesUserInterfaceweb<p><em><a href="http://mustache.github.io/">Mustache</a></em> is a well-known
<em>logic-less</em> template engine.<br />
There is plenty of Open Source implementations around (including in JavaScript,
which can be very convenient for AJAX applications on client side, for
instance).<br />
For <em>mORMot</em>, we created the first pure Delphi implementation of it,
with a perfect integration with other bricks of the framework.</p>
<p><img src="http://santateresaschool.org/v2/wp-content/uploads/2014/03/mustache-much-8.jpg" alt="" width="250" height="292/" /></p>
<p>In this first part of this series of blog articles, we will introduce the
<em>Mustache</em> design.<br />
You can <a href="https://blog.synopse.info?post/public/Documents/SynMustache.pdf">download this documentation
as one single pdf file</a>.</p> <p>Generally speaking, a Template system can be used to separate output
formatting specifications, which govern the appearance and location of output
text and data elements, from the executable logic which prepares the data and
makes decisions about what appears in the output.</p>
<p>Most template systems (e.g. PHP, smarty, Razor...) feature in fact a full
scripting engine within the template content.<br />
It allows powerful constructs like variable assignment or conditional
statements in the middle of the HTML content. It makes it easy to modify the
look of an application within the template system exclusively, without having
to modify any of the underlying "application logic". They do so, however, at
the cost of separation, turning the templates themselves into part of the
application logic.</p>
<p><em>Mustache</em> inherits from Google's <em>ctemplate</em> library, and is
used in many famous applications, including the "main" <a href="http://code.google.com/p/ctemplate">Google web search</a>, or the Twitter web
site.<br />
The <em>Mustache</em> template system leans strongly towards preserving the
separation of logic and presentation, therefore ensures a perfect <a href="https://blog.synopse.info?post/post/2014/04/18/Introducing-mORMot-s-architecture-and-design-principles">MVC
design</a>, and ready to consume SOA services.</p>
<p><em>Mustache</em> is intentionally constrained in the features it supports
and, as a result, applications tend to require quite a bit of code to
instantiate a template: all the application logic will be defined within the
<em>Controller</em> code, not in the <em>View</em> source.<br />
This may not be to everybody's tastes. However, while this design limits the
power of the template language, it does not limit the power or flexibility of
the template system. This system supports arbitrarily complex text
formatting.</p>
<p>Finally, <em>Mustache</em> is designed with an eye towards efficiency.
Template instantiation is very quick, with an eye towards minimizing both
memory use and memory fragmentation. As a result, it sounds like a perfect
template system for our <em>mORMot</em> framework.</p>
<h3>Mustache principles</h3>
<p>There are two main parts to the <em>Mustache</em> template system:</p>
<ol>
<li>Templates (which are plain text files);</li>
<li>Data dictionaries (aka <em>Context</em>).</li>
</ol>
<p>For instance, given the following template:</p>
<pre>
<h1>{{header}}</h1>
<br />{{#items}}
{{#first}}
<li><strong>{{name}}</strong></li>
{{/first}}
{{#link}}
<li><a href="{{url}}">{{name}}</a></li>
{{/link}}
{{/items}}
<br />{{#empty}}
<p>The list is empty.</p>
{{/empty}}
</pre>
<p>and the following data context:</p>
<pre>
{
"header": "Colors",
"items": [
{"name": "red", "first": <strong>true</strong>, "url": "#Red"},
{"name": "green", "link": <strong>true</strong>, "url": "#Green"},
{"name": "blue", "link": <strong>true</strong>, "url": "#Blue"}
],
"empty": <strong>true</strong>
}
</pre>
<p>The <em>Mustache</em> engine will render this data as such:</p>
<pre>
<h1>Colors</h1>
<li><strong>red</strong></li>
<li><a href="#Green">green</a></li>
<li><a href="#Blue">blue</a></li>
<p>The list is empty.</p>
</pre>
<p>In fact, you did not see any "<code>if</code>" nor "<em>for</em>" loop in
the template, but <em>Mustache</em> conventions make it easy to render the
supplied data as the expected HTML output. It is up to the MVC
<em>Controller</em> to render the data as expected by the template, e.g. for
formatting dates or currency values.</p>
<p><a href="https://blog.synopse.info?post/post/2014/04/28/Mustache-Logic-less-templates-for-Delphi-part-2">Next article
will detail the Mustache syntax itself</a>.<br />
Stay tuned!</p>