Synopse Open Source

2012-02-14

ORM cache

By AB4327-GANDI, 2012-02-14. Open Source › mORMot Framework

Here is the definition of "cache", as stated by Wikipedia:

In computer engineering, a cache is a component that transparently stores data so that future requests for that data can be served faster. The data that is stored within a cache might be values that have been computed earlier or duplicates of original values that are stored elsewhere. If requested data is contained in the cache (cache hit), this request can be served by simply reading the cache, which is comparatively faster. Otherwise (cache miss), the data has to be recomputed or fetched from its original storage location, which is comparatively slower. Hence, the greater the number of requests that can be served from the cache, the faster the overall system performance becomes.

To be cost efficient and to enable an efficient use of data, caches are relatively small. Nevertheless, caches have proven themselves in many areas of computing because access patterns in typical computer applications have locality of reference. References exhibit temporal locality if data is requested again that has been recently requested already. References exhibit spatial locality if data is requested that is physically stored close to data that has been requested already.

In our ORM framework, since performance was one goal since the beginning, cache has been implemented at four levels:

Statement cache for reuse of SQL prepared statements, and bound parameters on the fly - note that this cache is available not only for the SQlite3 database engine, but also for any external engine;
Global JSON result cache at the database level, which is flushed globally on any INSERT / UPDATE;
Tuned record cache at the CRUD/RESTful level for specified tables or records on the server side;
Tuned record cache at the CRUD/RESTful level for specified tables or records on the client side.

2012-02-09

Using SetLength or SetString

By AB4327-GANDI, 2012-02-09. Pascal Programming

Some Delphi users even do not know the existence of the SetString function.

As stated by the official documentation:

procedure SetString(var S: String; Buffer: PChar; Length: Integer);
For a long string variable, SetString sets S to reference a newly allocated string of the given length. If the Buffer parameter is not nil, SetString then copies Len characters from Buffer into the string; otherwise, the content of the new string is left uninitialized. If there is not enough memory available to create the string, an EOutOfMemory exception is raised. Following a call to SetString, S is guaranteed to reference a unique string (a string with a reference count of one).

Some have noticed that in our libraries, I sometimes use SetString instead of SetLength.
When the string is already allocated, it could be faster to use SetString, if you are sure that you will overwrite the string content.

2012-02-06

Modification of TSQLRestServerCallBack method prototype (bis)

By AB4327-GANDI, 2012-02-06. Open Source › mORMot Framework

In order to implement some RESTful Services, a callback has to be defined on the server side.

The prototype of these methods has been modified one more time, to supply an unique parameter:
This is a CODE BREAK change and you shall refresh ALL your server-side code to match the new signature.

This unique parameter will let the signature remain untouched in your code implementation, even if the framework evolves (like adding a new parameter).

2012-01-17

SynDBExplorer fast direct export

By AB4327-GANDI, 2012-01-17. Open Source › Open Source libraries

The Open Source SynDBExplorer tool has been enhanced these days.

Main new features are:

Execution of the current selected text (if any) instead of the whole memo content;
"Exec & Export" new button, for direct export to file.

I really like the selection execution feature - this speed up SQL process a lot, and allow to switch from one statement to another.
And the new exporting features are opening new possibilities.

2011-12-30

Hash collision attack

By AB4327-GANDI, 2011-12-30. Open Source › mORMot Framework

A variety of programming languages suffer from a denial-of-service (DoS) condition against storage functions of key/value pairs in hash data structures, the condition can be leveraged by exploiting predictable collisions in the underlying hashing algorithms.

The issue finds particular exposure in web server applications and/or frameworks. In particular, the lack of sufficient limits for the number of parameters in POST requests in conjunction with the predictable collision properties in the hashing functions of the underlying languages can render web applications vulnerable to the DoS condition. The attacker, using specially crafted HTTP requests, can lead to a 100% of CPU usage which can last up to several hours depending on the targeted application and server performance, the amplification effect is considerable and requires little bandwidth and time on the attacker side.

Source: #2011-003 multiple implementations denial-of-service via hash algorithm collision

2011-12-11

Strong-typing just rocks

By AB4327-GANDI, 2011-12-11. Pascal Programming

To my understanding, the so-called "strong-typing" feature is one big benefit of the Delphi object pascal language.

As stated by wikipedia:

Most generally, "strong typing" implies that the programming language places severe restrictions on the intermixing that is permitted to occur, preventing the compiling or running of source code which uses data in what is considered to be an invalid way. For instance, an addition operation may not be used with an integer and string values; a procedure which operates upon linked lists may not be used upon numbers. However, the nature and strength of these restrictions is highly variable.

Some Delphi users may find this is a limitation of the language, in comparison with other "fashionable" script idioms (like Python, Javascript of Ruby). For me, runtime strong typing (alla Python or Ruby) is not true strong typing. Simon Stuart just proposed a smartstring kind of string, which is in fact a weakstring type. As far as I understood his point, he wanted to get rid of all the warnings emitted by Unicode-version of the Delphi compiler, about explicit string conversion.

In fact, I use to go in the opposite direction. For wide projects, strong-typing is one of the big benefit of using Delphi (like other main "serious" languages like Java, C, C++, Ada or C#).

2011-12-08

Avoiding Garbage Collector: Delphi and Apple side by side

By AB4327-GANDI, 2011-12-08. Pascal Programming

Among all trolling subject in forums, you'll find out the great Garbage Collection theme.

Fashion languages rely on it. At the core of the .Net and Java framework, and all scripting languages (like JavaScript, Perl, Python or Ruby), you'll find a Garbage Collector. New developers, just released from schools, do learn about handling memory only in theory, and just can't understand how is memory allocated - we all have seen such rookies involved in Delphi code maintenance, leaking memory as much as they type. In fact, most of them did not understood how a computer works. I warned you this will be a trolling subject.

And, in Delphi, there is no such collector. We handle memory in several ways:

Creating static variables - e.g. on the stack, inside a class or globally;
Creating objects with class instances allocated on heap - in at least three ways: with a try..finally Free block, with a TComponent ownership model in the VCL, or by using an interface (which creates an hidden try..finally Free block);
Creating reference-counted variables, i.e. string, array of, interface or variant kind of variables.

It is a bit complex, but it is also deadly powerful. You have several memory allocation models at hand, which can be very handy if you want to tune your performance and let program scale. Just like manual recycling at home will save the planet. Some programmers will tell you that it's a waste of cell brain, typing and time. Linux kernel gurus would not say so, I'm afraid.

Then came the big Apple company, which presented its new ARC model (introduced in Mac OS X 10.7 Lion) as a huge benefit for Objective-C in comparison with the Garbage Collection model. And let's face it: this ARC just sounds like the Delphi memory model.

2011-12-06

Automatic JOIN query

By AB4327-GANDI, 2011-12-06. Open Source › mORMot Framework

In mORMot, all the methods available to handle many-to-many relationship (ManySelect, DestGetJoined...) are used to retrieve the relations between tables from the pivot table point of view. This saves bandwidth, and can be used in most simple cases, but it is not the only way to perform requests on many-to-many relationships. And you may have several TSQLRecordMany instances in the same main record - in this case, those methods won't help you.

It is very common, in the SQL world, to create a JOINed request at the main "Source" table level, and combine records from two or more tables in a database. It creates a set that can be saved as a table or used as is. A JOIN is a means for combining fields from two or more tables by using values common to each. Writing such JOINed statements is not so easy by hand, especially because you'll have to work with several tables, and have to specify the exact fields to be retrieved; if you have several pivot tables, it may start to be a nightmare.

Let's see how our ORM will handle it.