Synopse Open Source - Tag - OpenSSLmORMot MVC / SOA / ORM and friends2024-02-02T17:08:25+00:00urn:md5:cc547126eb580a9adbec2349d7c65274DotclearNative X.509, RSA and HSM Supporturn:md5:6404121d596d3c5cee045e29cf93e02b2023-12-09T11:01:00+00:002023-12-10T18:13:06+00:00Arnaud BouchezmORMot FrameworkAsymmetricCrossPlatformCSPRNGDelphiECCed25519forensicFPCFreePascalGoodPracticeinterfacemORMotmORMot2OpenSourceOpenSSLperformancePKCS11RSAsecurityX509<p>Today, almost all computer security relies on asymmetric cryptography and X.509 certificates as file or hardware modules.<br />
And the RSA algorithm is still used to sign the vast majority of those certificates. Even if there are better options (like ECC-256), RSA-2048 seems the actual standard, at least still allowed for a few years.</p>
<p><img src="https://blog.synopse.info?post/public/blog/mormotSecurity.jpg" alt="" /></p>
<p>So we added pure pascal RSA cryptography and X.509 certificates support in <em>mORMot</em>.<br />
Last but not least, we also added Hardware Security Modules support via the PKCS#11 standard.<br />
Until now, we were mostly relying on OpenSSL, but a native embedded solution would be smaller in code size, better for reducing dependencies, and easier to work with (especially for HSM). The main idea is to offer only safe algorithms and methods, so that you can write reliable software, even if you are no cryptographic expert. <img src="https://blog.synopse.info?pf=smile.svg" alt=":)" class="smiley" /></p> <h4>Rivest-Shamir-Adleman (RSA) Public-Key Cryptography</h4>
<p>The RSA public-key algorithm was designed back in 1977, and is still the most widely used. In order to fully implement it, we need to generate new key pairs (public and private keys), then sign and verify (or encrypt or decrypt) data with the key. For instance, a private key is kept secret, and used for an Authority to sign a certificate, and the public key is published, and able to verify a certificate. It is based on large prime numbers, so we needed to develop a Big Integer library, which is not part of Delphi or FPC RTL.</p>
<p><img src="https://blog.synopse.info?post/public/blog/RSAalgo.png" alt="" /></p>
<p>Here as some notes about our implementation in <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.rsa.pas">mormot.crypt.rsa.pas</a>:</p>
<ul>
<li>new pure pascal OOP design of BigInt computation optimized for RSA process;</li>
<li>dedicated x86_64/i386 asm for core computation routines (noticeable speedup);</li>
<li>use half-registers (HalfUInt) for efficient computation on all CPUs (arm and aarch64 numbers are good);</li>
<li>slower than OpenSSL, but likely to be the fastest FPC or Delphi native RSA library, thanks to our optimized asm: for instance, you can generate a new RSA-2048 keypair in less than a second;</li>
<li>internal garbage collection of BigInt instances, to minimize heap pressure during computation, and ensure all values are wiped once used during the process - as proven anti-forensic measure;</li>
<li>includes FIPS-level RSA keypair validation and generation, using a safe random source, with efficient prime number detection, and minimal code size;</li>
<li>features both RSASSA-PKCS1-v1_5 and RSASSA-PSS signature schemes;</li>
<li>started as a fcl-hash fork, but full rewrite inspired by Mbed TLS source because this initial code is slow and incomplete;</li>
<li>references: we followed <a href="https://github.com/Mbed-TLS/mbedtls">the Mbded TLS</a> implementation (which is much easier to follow than OpenSSL), and the well known <a href="https://cacr.uwaterloo.ca/hac/about/chap4.pdf">Handbook of Applied Cryptography (HAC)</a> recommendations;</li>
<li>includes full coverage of unit tests to avoid any regression, validated against the OpenSSL library as audited reference;</li>
<li>this unit will register as <code>Asym</code> 'RS256','RS384','RS512' algorithms (if not overridden by the faster <code>mormot.crypt.openssl</code>), keeping 'RS256-int' and 'PS256-int' available to use our unit;</li>
<li>as used by <code>mormot.crypt.x509</code> (see below) to handle RSA signatures of its X.509 Certificates.</li>
</ul>
<p>For instance, if you want to access a <code>TCryptAsym</code> digital signature instance with RSA-2048 and SHA-256 hashing, you can just use the <code>CryptAsym</code> global variable with <code>caaRS256</code> algorithm as factory.<br />
If you need just public/private key support, you can use <code>CryptPublicKey</code> or <code>CryptPrivateKey</code> factories with <code>ckaRsa</code> algorithm.</p>
<p>About RSA security:</p>
<ul>
<li>RSA-512 or RSA-1024 are considered unsafe and should not be used.</li>
<li>RSA-2048 confers 112-bit of security, and is the usual choice today when this algorithm is to be used.</li>
<li>RSA-3072 could confer 128-bit of security, at the expense of being slower and 50% bigger - so switching to ECC-256 may be a better option, for the same level of security.</li>
<li>RSA-4096 is not worth it in respect to RSA-3072, and RSA-7680 is very big and slow, and only gives 192-bit of security, so should be avoided.</li>
</ul>
<p>Anyway, our library is able to support all those key sizes, up to RSA-7680 is you really need it.<br />
See <a href="https://stackoverflow.com/a/589850/458259">this SO response</a> as reference about RSA keysizes.</p>
<h4>X.509 Certificates</h4>
<p>As we wrote in introduction, X.509 certificates are the base of most computer security.<br />
The whole TLS/HTTPS stack makes use of it, and the whole Internet would collapse without it.</p>
<p>We developed our <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.x509.pas">mormot.crypt.x509.pas</a> unit from scratch, featuring:</p>
<ul>
<li>X.509 Certificates Fields Logic (e.g. X.501 Type Names);</li>
<li>X.509 Certificates and Certificate Signing Request (CSR);</li>
<li>X509 Certificate Revocation List (CRL);</li>
<li>X509 Private Key Infrastructure (PKI);</li>
<li>Registration of our X.509 Engine to the <code>TCryptCert</code>/<code>TCryptStore</code> Factories.</li>
</ul>
<p><img src="https://blog.synopse.info?post/public/blog/X509certificate.png" alt="" /></p>
<p>The raw binary encoding is using the (weird) ASN.1 syntax, which is now implemented as part of the <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.secure.pas">mormot.crypt.secure.pas</a> unit.<br />
We followed the RFC 5280 specifications, and mapped latest X.509 Certificates / CSR / CRL extensions, with some low-level but very readable pascal code using classes, records and enumerates. It features perfect compatibility with our <code>ICryptCert</code> high-level interface wrappers, ready to be used in a very convenient way. We support all basic functions, but also advanced features like open/sealing or text peer information in a human readable format.<br />
When using our unit, your end-user code should not be lost within the complex details and notions of the X.509 format (like OIDs, versions or extensions), but use high-level pascal code, with no possibility to use a weak or invalid configuration.</p>
<p>Of course, it can support not only our new RSA keys, but also ECC-256 as implemented by our native <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.ecc.pas">mormot.crypt.ecc.pas</a>, or any other algorithm, e.g. available from OpenSSL.</p>
<h4>X.509 Private Key Infrastructure (PKI)</h4>
<p>Our unit features a full Private Key Infrastructure (PKI) implementation.<br />
In fact, X.509 certificates are as weak as the PKI they are used on. You can have strong certificates, but a weak verification pattern. In end-user applications, it is typical to see all the security being lost by a poor (e.g. naive) implementation of the keys interaction.</p>
<p><img src="https://blog.synopse.info?post/public/blog/PKI.png" alt="" /></p>
<p>This is why our unit publishes a 'x509-pki' <code>ICryptStore </code> as a full featured PKI:</p>
<ul>
<li>using our <code>TX509</code> and <code>TX509Crl</code> classes for actual certificates process;</li>
<li>clean verification of the chain of trust, with customized depth and proper root Certificate Authority (CA) support, following the RFC 5280 section 6 requirements of a clean "Certification Path Validation";</li>
<li>maintaining a cache of <code>ICryptCert</code> instances, which makes a huge performance benefit in the context of a PKI (e.g. you don't need to parse the X.509 binary, or verify the chain of trust each time).</li>
</ul>
<p>We tried to make performance and usability in the highest possible standards, to let you focus on your business logic, and keep the hard cryptography work done in the <em>mORMot</em> library code.</p>
<h4>Hardware Security Modules (HSM) via PKCS#11</h4>
<p>The PKCS#11 standard is a way to define some software access to Hardware Security Modules, via a set of defined API calls.<br />
We just published the <a href="https://github.com/synopse/mORMot2/blob/master/src/crypt/mormot.crypt.pkcs11.pas">mormot.crypt.pkcs11.pas</a> unit to interface those devices with the other <em>mORMot</em> PKI.</p>
<p><img src="https://blog.synopse.info?post/public/blog/HSM.png" alt="" /></p>
<p>Once you have loaded the library of your actual hardware (typically a <code>.dll</code> or <code>.so</code>) using a <code>TCryptCertAlgoPkcs11</code> instance, you can see all stored certificates and keys, as high-level regular <code>ICryptCert</code> instances, and sign or verify any kind of data (some binary or some other certificates), using the private key safely stored on in the hardware device.<br />
This is usually slower than a pure software verification, but it is much safer, because the private key is sealed within the hardware token, and never leave it. So it can't be intercepted and stolen.</p>
<h4>You are Welcome!</h4>
<p>With those <em>mORMot</em> cryptography units, you now have anything at hand to use standard and proven public-key cryptography in your applications, on both Delphi or FPC, with no external dll deployment issue, and minimal code size increase.<br />
We can thank a lot <a href="https://www.tranquil.it/en">my employer</a> for needing those nice features, therefore letting me work on them.<br />
Open Source rocks! :)</p>End Of Live OpenSSL 1.1 vs Slow OpenSSL 3.0urn:md5:f20e1a3a1c96e8f65f1fc8ef5a04498c2023-09-08T11:59:00+01:002023-09-08T14:13:49+01:00Arnaud BouchezOpen SourceAESCertificatesDelphiFreePascalGoodPracticeLateBindingLazarusMaxOSXmORMotmORMot2OpenSSLperformancesecuritySource<p>You may have noticed that the OpenSSL 1.1.1 series will reach End of Life (EOL) next Monday...<br />
Most sensible options are to switch to 3.0 or 3.1 as soon as possible.</p>
<p><img src="https://blog.synopse.info?post/public/blog/mormotSecurity.jpg" alt="mormotSecurity.jpg, Sep 2023" /></p>
<p>Of course, our <a href="https://github.com/synopse/mORMot2/blob/master/src/lib/mormot.lib.openssl11.pas"><em>mORMot 2</em> OpenSSL unit</a> runs on 1.1 and 3.x branches, and self-adapt at runtime to the various API incompatibilities existing between each branch.<br />
But we also discovered that switching to OpenSSL 3.0 could led into big performance regressions... so which version do you need to use?</p> <h4>OpenSSL 1.1 End Of Live</h4>
<p><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/6/6a/OpenSSL_logo.svg/320px-OpenSSL_logo.svg.png" alt="OpenSSL logo" /></p>
<p>The well known and well established OpenSSL 1.1.1 series will reach End of Life (EOL) on 11th September 2023. So next Monday! <img src="https://blog.synopse.info?pf=sad.svg" alt=":(" class="smiley" /> <br />
Users of OpenSSL 1.1.1 should consider their options and plan any actions they might need to take.</p>
<p>Note that Indy users are <a href="https://github.com/IndySockets/Indy/issues/183">still stuck to the OpenSSL 1.0 branch</a>, even 1.1 is not yet officially supported. Some <a href="https://github.com/IndySockets/Indy/pull/299">alternate IO handlers</a> are able to use newest releases - to some extend.<br />
Indy users should rather move to a better supported library, like our little <em>mORMot</em>.</p>
<p>Also note that there are some API incompatibilities between 1.1 and 3.x. Functions have been renamed, or even removed; new context constructors appeared; some parameters types even changed!<br />
Our unit tries to address all those problems at runtime, and is tested against several version of the OpenSSL library, to ensure you do not have to worry about those low-level issues.</p>
<h4>OpenSSL 3.x Benefits</h4>
<p>With OpenSSL 3.0, the developpers did a huge refactoring of the library internals.<br />
To be fair, the 1.x source code of OpenSSL was kind of a mess, and difficult to maintain. The biggest IT companies did even made their own forks or switched to other libraries. The best known is <a href="https://boringssl.googlesource.com/boringssl/">BoringSSL</a>, maintained by Google, and used e.g. in Chrome and Android.<br />
So it was time for a refactoring, especially for a library as critical as OpenSSL for so many projects.</p>
<p>With the new 3.x branch, a lot of low-level API functions have been deprecated.<br />
In practice, you don't have direct access any more to the internal structures of the library, and should now always use the high-level API to access a context property, or execute the processing methods. For instance, the low-level <code>AES_encrypt</code> function is not available any more: from now on, you need to use the high-level <code>EVP_Encrypt*</code> API.<br />
The official <a href="https://www.openssl.org/docs/man3.0/man7/migration_guide.html">Migration Guide page</a> is clearly huge, and worth reading if you want to prepare yourself to the upcoming years with OpenSSL.</p>
<h4>OpenSSL 3.0 Performance Regression</h4>
<p>The 3.0 branch new code may seem more beautiful and more maintainable, but it had its drawbacks. Newer is not always better.<br />
Most users of this new release <a href="https://github.com/openssl/openssl/issues/17064">observed a huge performance regression</a> when switching from 1.x to 3.0. It affected a lot of projects, from various languages, even script languages which were not already shining about performance. Time regression from 3x up to 10x were reported. On our side, X509 certificates manipulation was really slower than before - the worse being about X509 stores.</p>
<p>Some slowdown were expected and documented (like RSA key generation, which now uses 64 rounds). But the regression was much deeper.<br />
The culprit seems not to be the core cryptographic code, like AES buffer encoding (which asm claims to have been optimized even further on 3.x branch), but the OpenSSL context structures themselves. They were rewritten for future maintainability, but not focusing on their actual performance.</p>
<h4>OpenSSL 3.1 Numbers</h4>
<p>The 3.1 branch claims to have addressed most of these problems.</p>
<p><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/The_Tortoise_and_the_Hare_-_Project_Gutenberg_etext_19994.jpg/334px-The_Tortoise_and_the_Hare_-_Project_Gutenberg_etext_19994.jpg" alt="The Tortoise and the Hare" /></p>
<p>To be sure, we run the <em>mORMot</em> cryptographic regression tests with several versions of OpenSSL. And in fact, OpenSSL 3.1 was much faster than OpenSSL 3.0, but still behind OpenSSL 1.1.<br />
Here are the numbers we observed for the whole <code>TTestCoreCrypto</code> method execution, executed on Win32:</p>
<ul>
<li>OpenSSL 1.1 = 15 sec</li>
<li>OpenSSL 3.0 = 33 sec</li>
<li>OpenSSL 3.1 = 18 sec</li>
</ul>
<p>There are several aspects to emphasize:</p>
<ul>
<li>Those tests runs also <em>mORMot</em> engine cryptography, so you don't only test OpenSSL: the "pure mORMot" tests take around 4.5 seconds in the above numbers;</li>
<li>Any serious project should consider compiling on Win64, and running a server on a x86_64 Linux - on this platform, the regression does exist, but only slightly better;</li>
<li>The slowdown was less affecting <code>TTestCoreCrypto.Benchmark</code> (i.e. raw buffer encryption) than <code>TTestCoreCrypto.Catalog</code> (i.e. certificates process);</li>
<li>Our tests were mono-threaded, and worse slow down were reported on heavily threaded process (up to x10).</li>
</ul>
<p>Within the <em>mORMot</em> OpenSSL wrapper, we try to cache as many context as possible. For instance, we don't lookup the OpenSSL algorithm by name for each call, but we cache it at runtime to avoid any slowdown.<br />
But it seems not enough with OpenSSL 3.0, which may affect your application performance.</p>
<h4>To Support or Not Support</h4>
<p>So OpenSSL 3.1 seems to be the way to go.</p>
<p>On Linux (or other POSIX systems), you are likely to use the library shipped with the system.<br />
So you would not worry about which version to use. And, sadly, it is very likely that your distribution provides OpenSSL 3.0 and not OpenSSL 3.1.</p>
<p>On Windows (or Mac), you could (should?) use your "own" dll/so files, so you have to take into account the support level of the library.<br /></p>
<ul>
<li>OpenSSL 3.0 is a Long Term Support (LTS) version, which will be maintained until 7th September 2026.<br /></li>
<li>OpenSSL 3.1 will be supported only until 14th March 2025.</li>
</ul>
<p>These support end dates could appear counter-intuitive, but this is an usual way in Open Source projects, the best known being perhaps <a href="https://ubuntu.com/blog/what-is-an-ubuntu-lts-release">Ubuntu LTS versions</a>.<br />
For more information about OpenSSL support lifetime, look at the <a href="https://www.openssl.org/source/">official OpenSSL Downloads page</a>.</p>
<p>So, for most projects, especially on Windows where you are likely to publish OpenSSL dll with your own executable, switching to OpenSSL 3.1 is likely to be the way to go.<br />
If you need to gather some security certification for your product, you may consider using OpenSSL 3.0 LTS version, which may help your certification remain active for a longer period.</p>
<p>Any feedback is <a href="https://synopse.info/forum/viewtopic.php?id=6697">welcome on our forum</a>, as usual!</p>mORMot 2.1 Releasedurn:md5:4b0b47efd0e18fe5bdbea7a1e836a7662023-08-24T00:39:00+01:002023-08-29T08:53:25+01:00Arnaud BouchezmORMot Framework7zipAES-NiangelizeauthenticationCommandLineLDAPLUTImORMotmORMot2OpenSSLperformancereleaseshaSQLite3TFB<p>We are pleased to announce the release of <strong>mORMot 2.1</strong>.<br />
The <a href="https://github.com/synopse/mORMot2/releases/tag/2.1.stable">download link is available on github</a>.</p>
<p><img src="https://blog.synopse.info?post/public/blog/mORMot21.jpg" alt="" /></p>
<p>The <em>mORMot</em> family is growing up. <img src="https://blog.synopse.info?pf=smile.svg" alt=":)" class="smiley" /></p> <h4>The Release Cycle</h4>
<p>Like any living form, even silicon-based, the <em>mORMot</em> has its own cycle of life, following the coding seasons and commits history.</p>
<p>This was time to publish a new stable version of our Open Source framework for Delphi and FPC.<br />
We want indeed to make some official releases in a regular way, to ease integration into end user projects.</p>
<p>Here is an extract of the release notes:</p>
<h4>Added</h4>
<ul>
<li>(C)LDAP, DNS, (S)NTP clients</li>
<li>Command Line Parser</li>
<li>Native digest/basic HTTP servers authentication</li>
<li>Angelize services/daemons manager</li>
<li>TTunnelLocal TCP port forwarding</li>
<li>SHA-1/SHA-256 HW opcodes asm</li>
<li>7Zip dll wrapper</li>
<li>OpenSSL CSR support</li>
<li>PostgreSQL async DB with HTTP async backend (for TFB)</li>
<li>LUTI continous integration cross-platform farm</li>
<li></li>
</ul>
<h4>Changed</h4>
<ul>
<li>Upgraded SQLite3 to 3.42.0</li>
<li>Stabilized Mac x86_64/aarch64 platforms</li>
<li>Lots of bug fixes and enhancements</li>
</ul>
<h4>Feedback Welcome</h4>
<p>Some of the new features were already discussed in this blog, but <a href="https://synopse.info/forum/viewtopic.php?id=6681">you are free to ask for more on our forum, as usual</a>.</p>
<p>There was also a lot of small but meaningful changes and enhancements in the source code tree.<br />
Especially in the HTTP/REST part, <a href="https://synopse.info/forum/viewtopic.php?id=6443">to reach the top of the TechEmpower Framework Benchmark ranks</a>.</p>
<p><img src="https://blog.synopse.info?post/public/blog/mORMot21Release.jpeg" alt="" /></p>Native TLS Support for mORMot 2 REST or WebSockets Serversurn:md5:0d13d921c925b27dacea2bb1bda2c2852022-07-09T11:08:00+01:002022-07-10T06:39:19+01:00Arnaud BouchezmORMot FrameworkAsymmetricFreePascalHTTPhttp.sysHTTPSLinuxmORMotmORMot2OpenSourceOpenSSLperformancePublicKeyRESTSChannelTLSWebSockets<p>Since the beginning, we delegated the TLS encryption support to a reverse proxy server, mainly <a href="https://nginx.org">Nginx</a>. Under Windows, you could setup the http.sys HTTPS layer as usual, as a native - even a bit complicated - solution.<br />
Nginx has several advantages, the first being a proven and efficient technology, with plenty of documentation and configuration tips. It interfaces nicely with Let's Encrypt, and is very good for any regular website, using static content and PHP. This very blog and the <a href="https://synopse.info">Synopse</a> web site is hosted via Ngnix on a small Linux server.</p>
<p><img src="https://blog.synopse.info?post/public/blog/TlsServer.png" alt="" /></p>
<p>But in mORMot 2, we introduced a new set of <a href="https://blog.synopse.info/?post/2022/05/21/New-Async-HTTP/WebSocket-Server-on-mORMot-2">asynchronous web server classes</a>. So stability and performance are not a problem any more. Some benchmarks even consider this server to <a href="https://synopse.info/forum/viewtopic.php?pid=36546#p36546">be faster than nginx</a> (the stability issue mentioned in this post has been fixed in-between).<br />
We just introduced TLS support of our socket-based servers, both the blocking and asynchronous classes. It would use OpenSSL if available, or the SChannel API layer of Windows. Serving HTTPS or WSS with a self-signed certificate is just a matter of a single parameter now, and performance seems pretty good, especially with OpenSSL.</p> <h4>From HTTP to HTTPS</h4>
<p>Here is how you publish a <code>TRestServer</code> instance over HTTP, on port 8888, and with 16 threads for the thread pool:</p>
<pre>
Server := TRestHttpServer.Create([RestServer], '8888', 16);
</pre>
<p>Note the new constructor, easier to use than before, if you just want the default asynchronous server.</p>
<p>And to publish over HTTPS, on the very same port, with a self-signed certificate:</p>
<pre>
Server := TRestHttpServer.Create([RestServer], '8888', 16, secTLSSelfSigned);
</pre>
<p>For a <em>mORMot</em> client, you should also specify that you expect TLS support, and ignore the fact that this self-signed certificate is unknown by the system:</p>
<pre>
Client := TRestHttpClientSocket.Create('127.0.0.1', '8888', Model, {https=}true);
Client.IgnoreTlsCertificateErrors := true;
</pre>
<p>Then, at least with OpenSSL, you could serve TLS 1.3 content from now on, with a safe cipher negotiation by default (which could be tuned if needed).<br />
Nice and easy!</p>
<h4>Give Me the Keys</h4>
<p>And if you generated your own public/private keys pair, you could specify it:</p>
<pre>
Server := TRestHttpServer.Create([RestServer], '8888', 16, secTLS,
HTTPSERVER_DEFAULT_OPTIONS, 'mycert.pem', 'mypriv.pem', 'privpassw');
</pre>
<p>And don't forget to keep your private key... private. :)</p>
<h4>Keys Rule</h4>
<p>Where do those certificates come from? Do I need to read endless and complex OpenSSL command line samples, and mess with files and passwords?<br />
Our framework make it easy. You can now use the new <em>mORMot 2</em> high-level cryptography interfaces to generate the keys you want in simple pascal code.</p>
<p>Here is how the framework generates the self-signed server certificate on OpenSSL, or use a pre-computed one for SChannel:</p>
<pre>
procedure InitNetTlsContextSelfSignedServer(var TLS: TNetTlsContext;
Algo: TCryptAsymAlgo);
var
cert: ICryptCert;
certfile, keyfile: TFileName;
keypass: RawUtf8;
begin
certfile := TemporaryFileName;
if CryptCertAlgoOpenSsl[Algo] = nil then
begin
FileFromString(PrivKeyCertPfx, certfile); // use pre-computed key
keypass := 'pass';
end
else
begin
keyfile := TemporaryFileName;
keypass := CardinalToHexLower(Random32);
cert := CryptCertAlgoOpenSsl[Algo].
Generate(CU_TLS_SERVER, '127.0.0.1', {authority=}nil, 3650);
cert.SaveToFile(certfile, cccCertOnly, '', ccfPem);
cert.SaveToFile(keyfile, cccPrivateKeyOnly, keypass, ccfPem);
//writeln(BinToSource('PRIVKEY_PFX', '',
// cert.Save(cccCertWithPrivateKey, 'pass', ccfBinary)));
end;
InitNetTlsContext(TLS, {server=}true, certfile, keyfile, keypass);
end;
</pre>
<p>As you can see, the <code>ICryptCert</code> interface is very simple to use, and hide all the complexity of X509 and OpenSSL. We provided <code>nil</code> as authority, but you could specify a <code>ICryptCert</code> instance to sign your certificate, if needed.<br />
Under comments in the above source, you can see how to export the keys pair as PKCS#12 certificate, ready to be used for SChannel.</p>
<h4>TLS Everywhere</h4>
<p>Offering TLS as part of your software solution could be a game-changer for serious business, even over a corporate network, with self-signed certificates. It would help your IT and management people trust your <em>mORMot</em> / pascal solution in an heterogeneous and complex mesh of services. Modern object pascal is still on track for the next decades! :)</p>
<p>Feedback is <a href="https://synopse.info/forum/viewtopic.php?id=6291">welcome on our forum</a>, as usual. :)</p>EKON 25 Slidesurn:md5:aed86aeed11190901cf050e9842ec0bc2021-11-16T12:34:00+00:002021-11-16T12:34:00+00:00Arnaud BouchezmORMot Framework64bitAESAES-CTRAES-GCMAES-NiauthenticationCertificatesCrossPlatformDDDDelphiECCECDHECIESECSDAed25519EKONFreePascalinterfacelibdeflatemORMotmORMot2multithreadOpenSSLperformancerandomSOASourceWebSockets<p><a href="https://entwickler-konferenz.de/">EKON 25 at Düsseldorf</a> was a great conference (konference?).</p>
<p>At last, a <strong>physical</strong> gathering of Delphi developers, mostly from Germany, but also from Europe - and even some from USA! No more virtual meetings, which may trigger the well known 'Abstract Error' on modern pascal coders.<br />
There were some happy FPC users too - as I am now. <img src="https://blog.synopse.info?pf=smile.svg" alt=":)" class="smiley" /></p>
<p><img src="https://blog.synopse.info?post/public/blog/Ekon25.png" alt="" /></p>
<p>I have published the slides of my conferences, mostly about mORMot 2.<br />
By the way, I wish we would be able to release officially mORMot 2 in December, before Christmas. I think it starts to be stabilized and already known to be used on production. We expect no more breaking change in the next weeks.</p> <p>Here are the slides of my two 1-hour sessions.</p>
<h5>mORMot Cryptography</h5>
<p>The OpenSource mORMot framework has a strong set of cryptography features. It offers symmetric cryptography with hashing and encryption, together with asymmetric cryptography via private/public key pairs. Its optimized pascal and assembly engines can be embedded into your executable, but you could also call an external OpenSSL library if needed. This session will present mormot.crypt.* units, and apply them to some use cases, from low-level algorithms to high-level JWT or file encryption and signing.</p>
<p><a href="https://www.slideshare.net/ArnaudBouchez1/ekon25-mormot-2-cryptography">mORMot 2 Cryptography on SlideShare</a></p>
<p>I just had an interesting discussion with Michael on <a href="https://gitlab.com/freepascal.org/fpc/source/-/commit/3229cb712e33374b85258aed43726058be633bed#note_734398698">FPC new gitlab platform</a>: the FPC RTL is gaining some official cryptography functions, and I proposed to use mORMot code base as reference, and to introduce some RTL wrapper functions which can redirect to a plain pascal FPC RTL version, or use another engines, like OpenSSL or mORMot, if available.</p>
<h5>Server-Side REST Notifications with mORMot</h5>
<p>The most powerful way of writing REST services is to define them via interfaces, then let the SOA/REST framework do all the routing, data marshalling and communication behind the scenes. One distinctive feature of mORMot is to define a method parameter as a notification interface, and let the server call back the client when needed, as with regular Delphi code. This session will present the benefit of defining REST services using interfaces, and how WebSockets can offer real-time notifications into your rich Delphi client applications.</p>
<p><a href="https://www.slideshare.net/ArnaudBouchez1/ekon25-mormot-2-serverside-notifications">mORMot 2 Server-Side Notifications on SlideShare</a></p>
<p>Feedback is <a href="https://synopse.info/forum/viewtopic.php?id=6051">welcome on our forum, as usual.</a></p>Enhanced HTTP/HTTPS Support in mORMot 2urn:md5:fa0f372038d4e743aa1f81b28cdea9c32021-05-14T12:29:00+01:002021-05-14T12:29:00+01:00Arnaud BouchezmORMot FrameworkDelphiFreePascalhashHTTPHTTPSmORMotmORMot2OpenSSLpasswordperformanceRESTsecuritySSPIAuthWGet<p>HTTP(S) is the main protocol of the Internet.<br />
We enhanced the mORMot 2 socket client to push its implementation into more use cases. The main new feature is perhaps WGET-like processing, with hashing, resuming, console feedback, and direct file download.</p>
<p><img src="https://blog.synopse.info?post/public/blog/wget.png" alt="" /></p> <h3>Several Clients to Rule Them All</h3>
<p>Our framework supports several HTTP clients implementations/classes:</p>
<ul>
<li>Socket-based <code>THttpClientSocket</code>;</li>
<li>Windows WinHTTP API via <code>TWinHttp</code>;</li>
<li>Windows WinINET API via <code>TWinINet</code>;</li>
<li>LibCurl API via <code>TCurlHttp</code>.</li>
</ul>
<p>Depending on the system, you may want to use either of the classes.</p>
<p>In practice, they are not equivalent. For instance, our <code>THttpClientSocket</code> is the fastest over all - especially on UNIX, when using Unix Sockets instead of TCP to communicate (even it is not true HTTP any more, but it is supported by mORMot classes and also by reverse proxies like <code>nginx</code>).</p>
<p>This blog article will detail the latest modifications in mORMot 2 <code>THttpClientSocket</code> class.</p>
<h3>OpenSSL TLS Fixes</h3>
<p>During an intensive test phase, some fixes have been applied to the TLS/HTTPS layer. On Windows, it is now easier to load the OpenSSL dll, and the direct SSPI layer did have some fixes applied.</p>
<h3>Authentication and Proxy Support</h3>
<p><code>THttpClientSocket</code> has now callbacks for HTTP authentication, using any kind of custom scheme, with built-in support of regular Basic algorithm, or a cross-platform Kerberos algorithm, using SSPI on Windows or the GSSAPI on UNIX.</p>
<p>This class can now connect via a HTTP Proxy tunnel, with proper password authentication if needed.</p>
<h3>MultiPart Support</h3>
<p>One of the most complex use of HTTP is the ability to cut a stream as multiple parts. Up to now, there was limited support of the MultiPart encoding.</p>
<p>As consequences:</p>
<ul>
<li>The <code>THttpClientSocket</code> class now accepts not only memory buffers (encoded as <code>RawByteString</code>) but also <code>TStream</code>, so that data of any size could be downloaded (GET) or uploaded (POST);</li>
<li>We added the <code>THttpMultiPartStream</code> class for multipart/formdata HTTP POST.</li>
</ul>
<h3>WGet Support</h3>
<p><a href="https://www.gnu.org/software/wget/">GNU Wget</a> is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS, the most widely used Internet protocols.</p>
<p>We have implemented its basic features set - with extensions - in <code>THttpClientSocket.WGet()</code> method, as a convenient high-level way of downloading files.</p>
<ul>
<li>Direct file download with low resource consumption;</li>
<li>Can resume any previously aborted file, without restarting from the start - it would be very welcome for huge files;</li>
<li>Optional redirection from the original URI following the 30x status codes from the server;</li>
<li>Optional on-the-fly hashing of the content - supporting all <code>mormot.crypt.core</code> algorithms;</li>
<li>Optional progress event, via a callback method or built-in console display;</li>
<li>Optional folder to maintain a list of locally cached files;</li>
<li>Optional limitation of the bandwidth consumed during download, to reduce the network consumption e.g. on slow connections;</li>
<li>Optional timeout check for the whole download process (not only at connection level).</li>
</ul>
<p>We hope these new features will help you work on HTTP, and perhaps switch from other HTTP client libraries if you want to focus on our framework. We are open to any missing feature you may need to implement.</p>OpenSSL 1.1.1 Support for mORMot 2urn:md5:4cf412e6fbfe63673ee7f189fd8a3f9c2021-02-22T09:06:00+00:002021-02-22T10:40:23+00:00Arnaud BouchezmORMot FrameworkAESAES-CTRAES-GCMAES-NiasmblogCACertificatesDelphiECCECDHECIESECSDAed25519FreePascalJWTLinuxMaxOSXmORMot2OpenSSLperformancePublicKeyRSATLS <h3>Why OpenSSL?</h3>
<p><img src="https://blog.synopse.info?post/public/blog/OpenSSL_logo.png" alt="" /></p>
<p><a href="https://www.openssl.org/">OpenSSL</a> is the reference library for cryptography and secure TLS/HTTPS communication. It is part of most Linux/BSD systems, and covers a lot of use cases and algorithms. Even if it had some vulnerabilities in the past, it has been audited and validated for business use. Some algorithms have been deeply optimized, with very tuned assembly code.</p>
<p><em>mORMot 2</em> can now access all OpenSSL 1.1.1 features, on all supported targets.<br />
OpenSSL 1.1.1 is the latest stable branch, with LTS support, and nice new additions like TLS 1.3 and ed25519.</p>
<p>On Windows, you can use our internal assembly code for encryption, hashing and randomness (e.g. AES or SHA2/3), and we provide <a href="https://docs.microsoft.com/en-us/windows-server/security/tls/tls-ssl-schannel-ssp-overview">SChannel</a> support since <em>mORMot 1</em>, which allows to create a TLS/HTTPS layer without shipping any external library - using OpenSSL <code>.dll</code> files can be a real PITA on Windows, since it is very likely that several versions do exist on your systems, even in your path! We just <a href="https://synopse.info/forum/viewtopic.php?pid=34389#p34389">discovered and fixed some problems with SChannel</a>, which seems now stable and working fine even on latest Windows revisions.</p>
<p>Of course, if you really need to use OpenSSL on Windows, you can, and our little mORMot will try to do its best to find the right <code>.dll</code> for you. Windows' SChannel is behind OpenSSL in terms of ciphers support, and also about performance - especially if your Windows version is old or not updated. But it is part of the system, so you can rely on it if you expect basic TLS support.</p>
<p>On Linux / BSD / POSIX, OpenSSL 1.1.1 is the reference cryptographic library. It is already installed on any recent system.<br />
On MacOS and Android, you can use static linking to your application.</p>
<h3>OpenSSL 1.1.1 in mORMot 2</h3>
<p>Our OpenSSL integration in <em>mORMot 2</em> has several key features:</p>
<ul>
<li>Split into two units: <code>mormot.lib.openssl11.pas</code> for the raw API, and <code>mormot.core.crypto.openssl.pas</code> for integration with the other <em>mORMot 2</em> features;</li>
<li><code>mormot.lib.openssl11.pas</code> leverages OpenSSL 1.1.1, which is the latest stable version, and the current TLS branch - when OpenSSL 3 will be released, the API should remain compatible;</li>
<li><code>mormot.lib.openssl11.pas</code> by default loads dynamically OpenSSL 1.1.1 libraries, so even if your system won't have OpenSSL installed, your application will still start, with explicit error messages when you would try to use OpenSSL features;</li>
<li><code>mormot.lib.openssl11.pas</code> has a "full-API" conditional, which is used as reference when adding new features, and could be defined in your projects if you need specific OpenSSL features;</li>
<li><code>mormot.lib.openssl11.pas</code> publishes a TLS layer, recognized by our <code>mormot.net.sock.pas</code> unit, just like SChannel on Windows, to enabled TLS/HTTPS client connections;</li>
<li><code>mormot.core.crypto.openssl.pas</code> exposes the OpenSSL PRNG, which may be used for regulatory purposes, if our own PRNG doesn't meet your requirements;</li>
<li><code>mormot.core.crypto.openssl.pas</code> exposes the main (and safest) AES ciphers, and Hashing/Signing offered by OpenSSL, in a <code>mormot.core.crypto.pas</code> compatible way;</li>
<li><code>mormot.core.crypto.openssl.pas</code> defines some JWT classes, with full coverage of <strong>all</strong> the algorithms defined by the official <a href="https://jwt.io">JWT website</a> (it is the only Delphi library defining all of them);</li>
<li><code>mormot.core.crypto.openssl.pas</code> replaces the <code>mormot.core.crypto.pas</code> and <code>mormot.core.ecc256r1.pas</code> functions with the OpenSSL version, if they are faster;</li>
<li>We (re)validated our <code>mormot.core.crypto.pas</code> and <code>mormot.core.ecc256r1.pas</code> implementation against OpenSSL as reference, and also enhanced the test coverage (with more reference vectors for instance) for <em>mORMot 2</em>.</li>
</ul>
<p>In a nutshell, OpenSSL is slower than our <code>mormot.core.crypto.pas</code> assembly on x86_64 for most AES process (ciphers and PRNG) - but AES-GCM. But OpenSSL is much faster than <code>mormot.core.ecc256r1.pas</code> for ECC public key cryptography, so is automatically selected, which is a good idea e.g. on a Linux server environment.</p>
<h3>AES-PRNG, AES-CTR and AES-GCM</h3>
<p>See <a href="https://blog.synopse.info/?post/2021/02/13/Fastest-AES-PRNG%2C-AES-CTR-and-AES-GCM-Delphi-implementation">this article</a> for actual performance numbers of the AES encryption, and how our internal <code>mormot.core.crypto.pas</code> assembly is faster than OpenSSL in most cases.<br />
Our x86_64 tuned assembly, with full AES-NI and PCLMUL support does wonders.</p>
<h3>Public Key Cryptography and JWT</h3>
<p>The <code>mormot.core.crypto.openssl.pas</code> unit covers all possible JWT algorithms, with very interesting results, as shown by our regression tests:</p>
<pre>
mormot.core.crypto.pas algorithms:
1000 HS256 in 2.16ms i.e. 461,041/s, aver. 2us
1000 HS384 in 2.19ms i.e. 456,204/s, aver. 2us
1000 HS512 in 2.18ms i.e. 457,456/s, aver. 2us
1000 S3224 in 1.91ms i.e. 521,376/s, aver. 1us
1000 S3256 in 1.89ms i.e. 526,870/s, aver. 1us
1000 S3384 in 1.92ms i.e. 518,941/s, aver. 1us
1000 S3512 in 1.94ms i.e. 513,874/s, aver. 1us
1000 S3S128 in 1.92ms i.e. 520,833/s, aver. 1us
1000 S3S256 in 1.94ms i.e. 513,610/s, aver. 1us
100 ES256 in 14.65ms i.e. 6,823/s, aver. 146us
mormot.core.crypto.openssl.pas algorithms:
100 RS256 in 5.35ms i.e. 18,674/s, aver. 53us
100 RS384 in 5.16ms i.e. 19,368/s, aver. 51us
100 RS512 in 5.16ms i.e. 19,357/s, aver. 51us
100 PS256 in 5.56ms i.e. 17,985/s, aver. 55us
100 PS384 in 5.52ms i.e. 18,102/s, aver. 55us
100 PS512 in 5.65ms i.e. 17,677/s, aver. 56us
100 ES256 in 14.38ms i.e. 6,951/s, aver. 143us
100 ES384 in 123.79ms i.e. 807/s, aver. 1.23ms
100 ES512 in 94.50ms i.e. 1,058/s, aver. 945us
100 ES256K in 63.09ms i.e. 1,584/s, aver. 630us
100 EdDSA in 18.83ms i.e. 5,310/s, aver. 188us
</pre>
<p>The timing above are to verify a JWT signature.<br />
We can see that the ES256 has very tuned assembly code, and that the other ECC algorithms (ES384/ES512/ES256K) use standard coding, which are much slower. EdDSA is the ed25519 public cryptography algorithm, which is efficient on non-Intel/AMD platforms too, e.g. on ARM.</p>
<p>Raw asymmetric / public key cryptography using OpenSSL is therefore consistent with the expectations:</p>
<pre>
3 RSA 2048 Generation in 184.43ms i.e. 16/s, aver. 61.47ms
30 RSA 2048 Sign in 48.66ms i.e. 616/s, aver. 1.62ms
30 RSA 2048 Verify in 1.53ms i.e. 19,505/s, aver. 51us
3 RSA-PSS 2048 Generation in 205.60ms i.e. 14/s, aver. 68.53ms
30 RSA-PSS 2048 Sign in 47.32ms i.e. 633/s, aver. 1.57ms
30 RSA-PSS 2048 Verify in 1.57ms i.e. 19,047/s, aver. 52us
100 prime256v1 Generation in 8.26ms i.e. 12,096/s, aver. 82us
100 prime256v1 Sign in 7.02ms i.e. 14,226/s, aver. 70us
100 prime256v1 Verify in 13.45ms i.e. 7,433/s, aver. 134us
100 ed25519 Generation in 7.33ms i.e. 13,642/s, aver. 73us
100 ed25519 Sign in 13.67ms i.e. 7,310/s, aver. 136us
100 ed25519 Verify in 18.44ms i.e. 5,422/s, aver. 184us
</pre>
<p>RSA generation is really slow in comparison to other options (but verification is fast). In practice, prime256v1 (ES256) is to be privileged - especially since we offer our own pure standalone version if OpenSSL is not available.<br />
Note that any other algorithm exposed by OpenSSL are available - you are not restricted to the list above.</p>
<p>We introduced an easy way to generate public/private key pairs in a single function call, which is mandatory for any automated server work. No need to try the obfuscated <code>openssl ecparam -name prime256v1 -genkey -noout -out key.pem && openssl ec -in key.pem -pubout -out public.pem</code> command line switches: just call <code>OpenSslGenerateKeys()</code> or <code>TJwtAbstractOsl.GenerateKeys</code> and you are ready to go!</p>
<h3>TLS Layer</h3>
<p>We integrated the OpenSSL TLS layer to our <code>mormot.net.sock.pas</code> unit, with a powerful and easy way of using it:</p>
<pre>
with THttpClientSocket.Create do
try
TLS.WithPeerInfo := true;
TLS.IgnoreCertificateErrors := true;
TLS.CipherList := 'ECDHE-RSA-AES256-GCM-SHA384';
OpenBind('synopse.info', '443', {bind=}false, {tls=}true);
writeln(TLS.PeerInfo);
writeln(TLS.CipherName);
writeln(Get('/forum/', 1000), ' len=', ContentLength);
writeln(Get('/fossil/wiki/Synopse+OpenSource', 1000));
finally
Free;
end;
</pre>
<p>The <code>TCrtSocket.TLS</code> new field can be used to set the private and public keys or root certificates, if needed. Its <code>PeerInfo</code> return the whole certificate of the server side, and <code>CipherName</code> returns the current cipher used, e.g. <code>'ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD'</code>.</p>
<p>The very same code can be used with the SChannel version of <code>mormot.net.sock.windows.inc</code> - but with less features by now.</p>
<h3>OpenSource ForEver</h3>
<p>The source code of the units is available for review:</p>
<ul>
<li><a href="https://github.com/synopse/mORMot2/blob/master/src/lib/mormot.lib.openssl11.pas">/src/lib/mormot.lib.openssl11.pas</a>;</li>
<li><a href="https://github.com/synopse/mORMot2/blob/master/src/core/mormot.core.crypto.openssl.pas">/src/core/mormot.core.crypto.openssl.pas</a>.</li>
</ul>
<p>It works for FPC and Delphi, in the usual targets supported by <em>mORMot 2</em>.</p>
<p>Interfacing OpenSSL is really tricky sometimes, due to the opaque API names, lack of documentation, and weak/deprecated samples on the net.<br />
We tried to leverage our integration, so that using OpenSSL would be seamless for the <em>mORMot 2</em> users: you just use the mORMot classes, and don't have to care about which engine (mORMot's or OpenSSL's) is actually running them.</p>
<p>And feedback is <a href="https://synopse.info/forum/viewtopic.php?pid=34388#p34388">welcome on our forum</a>, as usual!</p>