Synopse Open Source

I'm currently fighting against out of memory errors on an heavy-loaded Java server.

If only it had been implemented in Delphi and mORMot!
But at this time, the mORMot was still in its burrow.
Copy-On-Write and a good heap manager can do wonders of stability.

Here are some thoughts about Garbage Collector, and how to temper their limitations.
They may apply to both the JVM and the .Net runtime, by the way.

In Delphi, you have several ways of handling data life time, therefore several ways of handling memory:

• For simple value objects (e.g.  byte integer double shortstring and fixed size arrays or record containing only such types), the value is copied in fixed-size buffers;
• For more complex value objets (e.g. string and dynamic arrays or record containing such types), there is a reference counter handled by each instance, with copy-on-write feature and compiler-generated reference counting at code scope level (with hidden try..finally blocks);
• For most class instances (e.g. deriving from TObject), you have to Create then Free each instance, and manage its life time by hand - with explicit try..finally blocks;
• For class deriving from TInterfacedObject, you have a RefCount property, with _AddRef _Release methods (this is the reference-counted COM model), and you can use Delphi interface to work with such instances - see this blog article.

Writing working multi-threaded code is not easy - it's even hard, as as a Delphi expert just wrote in his blog.

In fact, the first step into multi-thread application development could be:

"protect your shared variables with locks (aka critical sections), because you are not sure that the data you read/write is the same for all threads".

The CPU per-core cache is just one of the possible issues, which will lead into reading wrong values. Another issue which may lead into race condition is two threads writing to a resource at the same time: it's impossible to know which value will be stored afterward.

How to make your software run fast, especially in a multi-threaded architecture?

We tried to remove the Memory Manager scaling problems in our SynScaleMM. It worked as expected in a multi-threaded server environment. Scaling is much better than FastMM4, for some critical tests. But it's not ready for production yet...

To be honest, the Memory Manager is perhaps not the bigger bottleneck in Multi-Threaded applications.

Here are some (not dogmatic, just from experiment and knowledge of low-level Delphi RTL) advice if you want to write FAST multi-threaded application in Delphi.

It was told on this Blog that the Delphi memory manager (FastMM4 since Borland 2006 - but it was even worse with the previous "Borland's" MM), doesn't scale well on multi-code CPU. That is, if you have a multi-threaded application with a lot of memory handling (e.g. aString := aString+someString), the Delphi MM won't scale with multi cores. When I mean "don't scale", I mean that the optimistic though of "my CPU has 4 cores, therefore the same work run in 4 threads will be 4 times faster than with 1 thread" is false. It performs even worse with multiple threads than with 1 thread...

So we went into forking a nice project, named ScaleMM, and created our scalable optimized MM, named SynScaleMM. Our forked modifications were even included in the main ScaleMM branch.

During my profiling of our SynScaleMM, I discovered some very nice results with Delphi compiler inlining features.

As I wrote in a previous post, Delphi string, dynamic array and memory manager don't like multi-core CPU.

My proposal is to add a threadlocalvar keyword, to be used instead of var in your code, to mark some variables to be used in only the current thread. Then the compiler and RTL won't have to use the LOCK instruction, and the application will be MUCH faster in multi-thread environment.