2022-05-21

New Async HTTP/WebSocket Server on mORMot 2

The HTTP server is one main part of any SOA/REST service, by design.
It is the main entry point of all incoming requests. So it should better be stable and efficient. And should be able to scale in the future, if needed.

There have always been several HTTP servers in mORMot. You can use the HTTP server class you need.
In mORMot 2, we added two new server classes, one for publishing over HTTP, another able to upgrade to WebSockets. The main difference is that they are fully event-driven, so their thread pool is able to scale with thousands of concurrent connections, with a fixed number of threads. They are a response to the limitations of our previous socket server.

HTTP Is Not REST

In mORMot, the HTTP server does not match the REST server. They are two concepts, with diverse units, classes and even... source code folders.
The HTTP server can publish its own process, using a callback, or can leverage one or several REST servers. Or the REST server could be with no communication at all, e.g. be run in the service process within the current thread, executing ORM or SOA requests without any HTTP or WebSockets involved. No difference in the user code: just some interface methods to call, and they will return their answer, whatever it is over HTTP, locally or remotely, on the server side or the client side, in a thread pool or in-process... pascal code just runs its magic for you.

For instance, take a look at httpServerRaw as a low-level HTTP server sample, using a callback for every incoming request.
Here you don't have any automatic routing: you just parse the input URI, then return the proper HTTP response, with its status code, headers and body.
Those low-level HTTP servers are implemented in the server units of folder src/net - as we will detail below.

Then, the REST process is abstracted from HTTP. Even if both HTTP and REST historically share the same father/author/initiator, you could have a REST approach without HTTP. For instance, you could use WebSockets, or direct in-process call.
So in mORMot, the REST process is implemented in the server units of folder src/rest, abstracted from any communication protocol. It does not know nor assume anything about TCP, WebSockets or TLS. Just about URI, headers and body texts, and follow the routing as defined.

Two folders, two uncoupled feature sets. Perhaps a bit confusing when you discover it first. But for the best maintainability and code design.

Several Servers To Rule Them All

mORMot 2 adopted all HTTP server classes from mORMot 1 source code. Then include some new "asynchronous" servers.
They all inherit from a THttpServerGeneric parent class, so you can follow the Liskow Substitution Principle, and change the class at runtime or compilation, as needed, without altering your actual logic.

HTTP servers are implemented in several units:

  • mORMot.net.server.pas offers the THttpServerSocket/THttpServer HTTP/1.1 server, the THttpApiServer HTTP/1.1 server over Windows http.sys module, and THttpApiWebSocketServer over Windows http.sys module;
  • mORMot.net.ws.server.pas offers the TWebSocketServerRest server, which uses WebSockets as mean of transmission, but enable a REST-like blocking request/answer protocol on top of it, with optional bi-directional notifications, using a one-thread-per-connection server;
  • mORMot.net.async.pas offers the new THttpAsyncServer event-driven HTTP server;
  • mORMot.net.ws.async.pas offers the new TWebSocketAsyncServerRest server, which uses WebSockets as mean of transmission, but enable a REST-like blocking request/answer protocol on top of it, with optional bi-directional notifications, using an event-driven server.

On Windows, the http.sys module gives you very good stability, and uses the same Windows-centric way of publishing servers as used by IIS and DotNet. You could even share the same port between several services, if needed.

Our socket-based servers are cross-plaform, and compile and run on both Windows and POSIX (Linux, BSD, MacOS). They use a thread pool for HTTP/1.0 short living requests, and one thread per connection on HTTP/1.1. So they are meant to be used behind a reverse proxy like nginx, which could transmit over HTTP/1.0 with mORMot, but keep efficient HTTP/1.1 or HTTP/2.0 to communicate with the clients.

Both mORMot.net.async.pas and mORMot.net.ws.async.pas are new to mORMot 2. They use an event-driven model, i.e. the opened connections are tracked using a fast API (like epoll on Linux), and the thread pool is used only when there is actually new data pending.

Events Forever

Asynchronous socket access, and event loops are the key for best server scalability. In respect to our regular THttpServerSocket class which uses one thread per HTTP/1.1 or WebSockets connection, our asynchronous classes (e.g. THttpAsyncServer) can have thousands of concurrent clients, with minimal CPU and RAM resource consumption.

Here is a typical event-driven socket access:

In mORMot 2 network core, i.e. in unit mORMot.net.sock.pas, we define an abstract event-driven class:

  /// implements efficient polling of multiple sockets
  // - will maintain a pool of TPollSocketAbstract instances, to monitor
  // incoming data or outgoing availability for a set of active connections
  // - call Subscribe/Unsubscribe to setup the monitored sockets
  // - call GetOne from a main thread, optionally GetOnePending from sub-threads
  TPollSockets = class(TPollAbstract)
  ...
    /// initialize the sockets polling
    constructor Create(aPollClass: TPollSocketClass = nil);
    /// finalize the sockets polling, and release all used memory
    destructor Destroy; override;
    /// track modifications on one specified TSocket and tag
    function Subscribe(socket: TNetSocket; events: TPollSocketEvents;
      tag: TPollSocketTag): boolean; override;
    /// stop status modifications tracking on one specified TSocket and tag
    procedure Unsubscribe(socket: TNetSocket; tag: TPollSocketTag); virtual;
    /// retrieve the next pending notification, or let the poll wait for new
    function GetOne(timeoutMS: integer; const call: RawUtf8;
      out notif: TPollSocketResult): boolean; virtual;
    /// retrieve the next pending notification
    function GetOnePending(out notif: TPollSocketResult; const call: RawUtf8): boolean;
    /// let the poll check for pending events and apend them to fPending results
    function PollForPendingEvents(timeoutMS: integer): integer; virtual;
    /// manually append one event to the pending nodifications
    procedure AddOnePending(aTag: TPollSocketTag; aEvents: TPollSocketEvents;
      aNoSearch: boolean);
    /// notify any GetOne waiting method to stop its polling loop
    procedure Terminate; override;
    ...

Depending on the operating system, it will mimic the epoll api with the underlying low-level system calls.

The events are very abstract, and are in fact just the basic R/W operations on each connection, associated with a "tag", which is likely to be a class pointer associated with a socket/connection:

  TPollSocketEvent = (
    pseRead,
    pseWrite,
    pseError,
    pseClosed);
  TPollSocketEvents = set of TPollSocketEvent;

  TPollSocketResult = record
    tag: TPollSocketTag;
    events: TPollSocketEvents;
  end;
  TPollSocketResults = record
    Events: array of TPollSocketResult;
    Count: PtrInt;
  end;

Then whole new HTTP/1.0, HTTP/1.1 and WebSockets stacks have been written on top of those basic socket-driven events. Instead of blocking threads, they use internal state machines, which are much lighter than a thread, and even lighter than a coroutine/goroutine. Each connection is just a class instance, which maintains the state of each client/server communication, and accesses its own socket.
The mORMot asynchronous TCP server has by default one thread to accept the connection, one thread to poll for pending events (calling the GetOne method), then a dedicated number of threads to consume the Read/Write/Close events (via the GetOnePending method). We used as many non-blocking structures as possible, we minimized memory allocation by reusing the same buffers e.g. for the headers or small responses, we pickup the best locks possible in each case, so that this server could scales nice and smoothly. And simple to use, because handling a new protocol is as easy as inheriting and writing a new connection class.

Our asynchronous servers classes seem now stable, and fast (reported to be twice faster than nginx and six time faster than nodejs!).
But of course, as any new complex code, they may be caveats. And some of them have already be identified and fixed - as reported in our forum. Therefore, feedback is welcome, and a nginx, haproxy or caddy reverse proxy frontend is always a good idea on production.

Writing those servers took more time than previewed, and was sometimes painful. Because debugging multi-thread process is not easy, and especially on several operating systems. There are some subtle differences between the OS, which could lead to unexpected blocking or degraded performance. But we are proud of the result, which compares to the best-in-class servers. Still in modern pascal code, and Open Source software.

Don't hesitate to take a look at the source, and try some samples.
Feedback is welcome in our forum, as usual.

2022-02-15

mORMot 2 ORM Performance

The official release of mORMot 2 is around the edge. It may be the occasion to show some data persistence performance numbers, in respect to mORMot 1.

For the version 2 of our framework, its ORM feature has been enhanced and tuned in several aspects: REST routing optimization, ORM/JSON serialization, and in-memory and SQL engines tuning. Numbers are talking. You could compare with any other solution, and compile and run the tests by yourself for both framework, and see how it goes on your own computer or server.
In a nutshell, we almost reach 1 million inserts per second on SQLite3, and are above the million inserts in our in-memory engine. Reading speed is 1.2 million and 1.7 million respectively. From the object to the storage, and back. And forcing AES-CTR encryption on disk almost don't change anything. Now we are talking. ;)

Continue reading

2022-01-22

Three Locks To Rule Them All

To ensure thread-safety, especially on server side, we usually protect code with critical sections, or locks. In recent Delphi revisions, we have the TMonitor feature, but I would rather trust the OS for locks, which are implemented using Windows Critical Sections, or POSIX futex/mutex.

But all locks are not born equal. Most of the time, the overhead of a Critical Section WinAPI or the pthread library is not needed.
So, in mORMot 2, we introduced several native locks in addition to those OS locks, with multi-read/single-write abilities, or re-entrancy.

Continue reading

2021-12-19

mORMot 2 Generics and Collections

Generics are a clever way of writing some code once, then reuse it for several types.
They are like templates, or compiler-time shortcuts for type definitions.

In the last weeks, we added a new mormot.core.collections.pas unit, which features:

  • JSON-aware IList<> List Storage;
  • JSON-aware IKeyValue<> Dictionary Storage.

In respect to Delphi or FPC RTL generics.collections, this unit uses interfaces as variable holders, and leverage them to reduce the generated code as much as possible, as the Spring4D 2.0 framework does, but for both Delphi and FPC. It publishes TDynArray and TSynDictionary high-level features like indexing, sorting, JSON/binary serialization or thread safety as Generics strong typing.

Resulting performance is great, especially for its enumerators, and your resulting executable size won't blow up as with the regular RTL unit.

Continue reading

2021-11-16

EKON 25 Slides

EKON 25 at Düsseldorf was a great conference (konference?).

At last, a physical gathering of Delphi developers, mostly from Germany, but also from Europe - and even some from USA! No more virtual meetings, which may trigger the well known 'Abstract Error' on modern pascal coders.
There were some happy FPC users too - as I am now. :)

I have published the slides of my conferences, mostly about mORMot 2.
By the way, I wish we would be able to release officially mORMot 2 in December, before Christmas. I think it starts to be stabilized and already known to be used on production. We expect no more breaking change in the next weeks.

Continue reading

2021-09-21

Delphi 10.4 / Delphi 11 Alexandria Breaking Changes

The latest revision of Delphi, named Delphi 11 Alexandria, is out.
A lot of new features, some enhanced platforms. Nice!
But it is also for us the opportunity to come back to some breaking changes, which appeared in Delphi 10.4 earlier this year, and are now "officially" part of Delphi 11.

The main breaking change of Delphi 10.4 and later, as reported by mORMot users, is the new lifetime of local variables.
TL&LR: a local variable which is not explicitly declared, but returned by a function may be released as soon as it is not used any more, whereas in the original implementation, it was allocated as a regular local variable, and we could expect its lifetime to remain active up to the end of the function. With Delphi 10.4, it is not the case any more: the compiler could release/clear the local variable sooner, to reduce the allocation pressure.

Idea behind this change is that it may have better register allocation within the function, so it "may" theoretically result in faster code. Not convinced about it, anyway - we will discuss that.
The main thing is that it could break existing code, because it breaks the Delphi compiler expectation since decades.
Some perfectly fine working code would end to work as expected. We have identified several use cases with mORMot which are affected by this change. Since it seems there will be no coming back from Delphi point of view, it is worth a blog article. ;)

Continue reading

2021-08-17

mORMot 2 on Ampere AARM64 CPU

Last weeks, we have enhanced mORMot support to one of the more powerful AARM64 CPU available: the Ampere Altra CPU, as made available on the Oracle Cloud Infrastructure.

Long story short, this is an amazing hardware to run on server side, with performance close to what Intel/AMD offers, but with almost linear multi-core scalability. The FPC compiler is able to run good code on it, and our mORMot 2 library is able to use the hardware accelerated opcodes for AES, SHA2, and crc32/crc32c.

Continue reading

2021-07-08

Job Offer: FPC mORMot 2 and WAPT

Good news!
The French company I work for, Tranquil IT, is hiring FPC / Lazarus / mORMot developers. Remote work possible.

I share below the Job Offer from my boss Vincent.
We look forward working with you on this great mORMot-powered project!

https://www.tranquil.it/en/who-are-we/join-us

Continue reading

- page 1 of 48