Synopse Open Source

A recent poll on the Lazarus/FPC forum highlighted a fact: pascal coders are older than most coders. Usually, at our age, we should be managers, not developers. But we like coding in pascal. It is still fun after decades!
But does it mean that you should not use pascal for any new project? Are the language/compilers/libraries outdated?
In the company I currently work for, we have young coders, just out-of-school or still-in-school, which joined the team and write great code!

And a recent thread in this very same forum was about comparing languages to implement a REST server, in C#, Go, Scala, TypeScript, Elixir and Rust.
Several pascal versions are about to be contributed, one in which mORMot shines.

Since the beginning, we delegated the TLS encryption support to a reverse proxy server, mainly Nginx. Under Windows, you could setup the http.sys HTTPS layer as usual, as a native - even a bit complicated - solution.
Nginx has several advantages, the first being a proven and efficient technology, with plenty of documentation and configuration tips. It interfaces nicely with Let's Encrypt, and is very good for any regular website, using static content and PHP. This very blog and the Synopse web site is hosted via Ngnix on a small Linux server.

But in mORMot 2, we introduced a new set of asynchronous web server classes. So stability and performance are not a problem any more. Some benchmarks even consider this server to be faster than nginx (the stability issue mentioned in this post has been fixed in-between).
We just introduced TLS support of our socket-based servers, both the blocking and asynchronous classes. It would use OpenSSL if available, or the SChannel API layer of Windows. Serving HTTPS or WSS with a self-signed certificate is just a matter of a single parameter now, and performance seems pretty good, especially with OpenSSL.

The official release of mORMot 2 is around the edge. It may be the occasion to show some data persistence performance numbers, in respect to mORMot 1.

For the version 2 of our framework, its ORM feature has been enhanced and tuned in several aspects: REST routing optimization, ORM/JSON serialization, and in-memory and SQL engines tuning. Numbers are talking. You could compare with any other solution, and compile and run the tests by yourself for both framework, and see how it goes on your own computer or server.
In a nutshell, we almost reach 1 million inserts per second on SQLite3, and are above the million inserts in our in-memory engine. Reading speed is 1.2 million and 1.7 million respectively. From the object to the storage, and back. And forcing AES-CTR encryption on disk almost don't change anything. Now we are talking.

Generics are a clever way of writing some code once, then reuse it for several types.
They are like templates, or compiler-time shortcuts for type definitions.

In the last weeks, we added a new mormot.core.collections.pas unit, which features:

• JSON-aware IList<> List Storage;
• JSON-aware IKeyValue<> Dictionary Storage.

In respect to Delphi or FPC RTL generics.collections, this unit uses interfaces as variable holders, and leverage them to reduce the generated code as much as possible, as the Spring4D 2.0 framework does, but for both Delphi and FPC. It publishes TDynArray and TSynDictionary high-level features like indexing, sorting, JSON/binary serialization or thread safety as Generics strong typing.

Resulting performance is great, especially for its enumerators, and your resulting executable size won't blow up as with the regular RTL unit.

The latest revision of Delphi, named Delphi 11 Alexandria, is out.
A lot of new features, some enhanced platforms. Nice!
But it is also for us the opportunity to come back to some breaking changes, which appeared in Delphi 10.4 earlier this year, and are now "officially" part of Delphi 11.

The main breaking change of Delphi 10.4 and later, as reported by mORMot users, is the new lifetime of local variables.
TL&LR: a local variable which is not explicitly declared, but returned by a function may be released as soon as it is not used any more, whereas in the original implementation, it was allocated as a regular local variable, and we could expect its lifetime to remain active up to the end of the function. With Delphi 10.4, it is not the case any more: the compiler could release/clear the local variable sooner, to reduce the allocation pressure.

Idea behind this change is that it may have better register allocation within the function, so it "may" theoretically result in faster code. Not convinced about it, anyway - we will discuss that.
The main thing is that it could break existing code, because it breaks the Delphi compiler expectation since decades.
Some perfectly fine working code would end to work as expected. We have identified several use cases with mORMot which are affected by this change. Since it seems there will be no coming back from Delphi point of view, it is worth a blog article.

HTTP(S) is the main protocol of the Internet.
We enhanced the mORMot 2 socket client to push its implementation into more use cases. The main new feature is perhaps WGET-like processing, with hashing, resuming, console feedback, and direct file download.

Why OpenSSL? OpenSSL is the reference library for cryptography and secure TLS/HTTPS communication. It is part of most Linux/BSD systems, and covers a lot of use cases and algorithms. Even if it had some vulnerabilities in the past, it has been audited and validated for business use. Some algorithms  […]

I have just committed some new AesNiHash32 AesNiHash64 AesNiHash128 Hashers for mORMot 2. They are using AES-NI and SSE4.1 opcodes on x86_64 and i386. This implementation is faster than the fastest SSE4.1 crc32c and with a much higher usability (less collisions). Logic was extracted from the Go  […]

About latest SQlite3 3.32.xxx there is a big problem with codecs.

Critical changes to the public SQLite code were introduced on Feb 7, 2020: “Simplify the code by removing the unsupported and undocumented SQLITE_HAS_CODEC compile-time option”. With the release of SQLite version 3.32.0 on May 22, 2020 these changes finally took officially effect, although they weren't officially announced.

As a sad and unexpected consequence, we are NOT ANY MORE able to compile the new SQlite3 amalgamation with our encryption patch.

On server side, a lot of CPU is done processing conversions to or from text. Mainly JSON these days.

In mORMot, we take care a lot about performance, so we have rewritten most conversion functions to have something faster than the Delphi or FPC RTL can offer.
Only float to text conversion was not available. And RTL str/floattexttext performance, at least under Delphi, is not consistent among platforms.
So we just added a new Double-To-Text set of functions.