Synopse Open Source

2011-06-07

True per-class variable

By AB4327-GANDI, 2011-06-07. Pascal Programming

For our ORM, we needed a class variable to be available for each TSQLRecord class type.

This variable is used to store the properties of this class type, i.e. the database Table properties (e.g. table and column names and types) associated with a particular TSQLRecord class, from which all our ORM objects inherit.

The class var statement was not enough for us:
- It's not available on earlier Delphi versions, and we try to have our framework work with Delphi 6-7 up to XE;
- This class var instance will be shared by all classes inheriting from the class where it is defined - and we need ONE instance PER class type, not ONE instance for ALL

We needed to find another way to implement this class variable.

An unused VMT slot in the class type description was identified, then each class definition was patched in the process memory to contain our class variable.

2011-06-06

Synopse PDF Engine 1.13

By AB4327-GANDI, 2011-06-06. Open Source › Synopse PDF engine

Our SynPdf unit has been refreshed to the 1.13 version.

code modifications to compile with Delphi 5 compiler;
added horizontal scaling for GDI enumeration in case of text kerning (could occur for small fonts);
fixed "Save when closing with Acrobat Reader X" - thanks to Ondrej for the fix;
fixed clipping problems and vertical font positioning issue in GDI enumeration - thanks to Ondrej for those corrections!

Our pdf engine sounds almost stable by now.
Thanks to the feedback provided by some user of this Open Source library!

2011-06-05

Synopse SQLite3 Framework 1.13

By AB4327-GANDI, 2011-06-05. Open Source › mORMot Framework

This is a major step for the framework.

Among a lot of new features and bug fixes:

Dynamic arrays and Record handling, including low-level access and high-level handling for the ORM part of the framework (using TDynArray wrapper);
Business rules now handled via filtering and validating classes for both our ORM and the Synopse Big Table;
New full-featured logging classes, with exception handling, stack trace and JSON serialization;
Custom SQL functions and TSQLRecordRTree to implement R-Tree virtual tables;
Batch sequences for adding/updating/deleting multiple records;
Secure RESTful authentication via per-user light sessions, and per-query signature;
Define custom Virtual Tables (i.e. access any kind of data from SQL) in Delphi code;
New HTTP server using fast http.sys kernel-mode server and zip / SynLZ compression;
SynTaskDialog unit native under Vista,Seven with emulation code under XP/2000 ;
User Interface automated Generation using standard VCL or proprietary TMS components;
Documentation completed - more than 600 pages by now, including general User Guide and exhaustive Technical Reference;
Updated SQLite3 database engine to version 3.7.6.3;
Numerous bug fixes, speed and feature enhancements (including in the PDF and other UI-related units).

Open Source project, for Delphi 6 up to XE, licensed under a MPL/LGPL/GPL tri-license.

2011-06-02

Custom SQL functions

By AB4327-GANDI, 2011-06-02. Open Source › mORMot Framework

The SQLite3 engine defines some standard SQL functions, like abs() min() max() or upper().
A complete list is available at http://www.sqlite.org/lang_corefunc.html

One of the greatest SQLite3 feature is the ability to define custom SQL functions in high-level language. In fact, its C API allows to implement new functions which may be called within a SQL query. In other database engine, such functions are usually named UDF (for User Defined Functions).

Our framework allows you to add easily such custom functions, directly from Delphi classes.

no trackbacks

Fast JSON parsing

By AB4327-GANDI, 2011-06-02. Open Source › mORMot Framework

When it deals with parsing some (textual) content, two directions are usually envisaged. In the XML world, you have usually to make a choice between:
- A DOM parser, which creates an in-memory tree structure of objects mapping the XML nodes;
- A SAX parser, which reads the XML content, then call pre-defined events for each XML content element.

In fact, DOM parsers use internally a SAX parser to read the XML content. Therefore, with the overhead of object creation and their property initialization, DOM parsers are typically three to five times slower than SAX. But, DOM parsers are much more powerful for handling the data: as soon as it's mapped in native objects, code can access with no time to any given node, whereas a SAX-based access will have to read again the whole XML content.

Most JSON parser available in Delphi use a DOM-like approach. For instance, the DBXJSON unit included since Delphi 2010 or the SuperObject or DWS libraries create a class instance mapping each JSON node.

In a JSON-based Client-Server ORM like ours, profiling shows that a lot of time is spent in JSON parsing, on both Client and Server side. Therefore, we tried to optimize this part of the library.

no trackbacks

BATCH sequences for adding/updating/deleting records

By AB4327-GANDI, 2011-06-02. Open Source › mORMot Framework

When use the so-called BATCH sequences?

In a standard Client-Server architecture, especially with the common understanding (and most implementations) of a RESTful service, any Add / Update / Delete method call requires a back and forth flow to then from the remote server.

In case of a remote connection via the Internet (or a slow network), you could have some 100 ms of latency: it's just the "ping" timing, i.e. the time spent for your IP packet to go to the server, then back to you.

If you are making a number of such calls (e.g. add 1000 records), you'll have 100*1000 ms = 100 s = 1:40 min just because of this network latency!

The BATCH sequence allows you to regroup those statements into just ONE remote call. Internally, it builds a JSON stream, then post this stream at once to the server. Then the server answers at once, after having performed all the modifications.

no trackbacks

TSQLRecordRTree to implement R-Tree virtual tables

By AB4327-GANDI, 2011-06-02. Open Source › mORMot Framework

An R-Tree is a special index that is designed for doing range queries.

R-Trees are most commonly used in geospatial systems where each entry is a rectangle with minimum and maximum X and Y coordinates. Given a query rectangle, an R-Tree is able to quickly find all entries that are contained within the query rectangle or which overlap the query rectangle.

This idea is easily extended to three dimensions for use in CAD systems. R-Trees also find use in time-domain range look-ups. For example, suppose a database records the starting and ending times for a large number of events. A R-Tree is able to quickly find all events, for example, that were active at any time during a given time interval, or all events that started during a particular time interval, or all events that both started and ended within a given time interval. And so forth. See http://www.sqlite.org/rtree.html

Since the 2010-06-25 source code repository update, the RTREE extension is compiled by default within all supplied .obj files of the framework.

A dedicated ORM class, named TSQLRecordRTree, is available to create such tables. It inherits from TSQLRecordVirtual, like the other virtual tables types (e.g. TSQLRecordFTS3 or our custom virtual tables).

no trackbacks

2011-06-01

Business rules: validate and filter data

By AB4327-GANDI, 2011-06-01. Open Source › mORMot Framework

According to the n-Tier architecture, data filtering and validation should be implemented in the business logic, not in the User Interface.

If you were used to develop RAD database application using Delphi, you may have to change a bit your habits here. Data filtering and validation should be implemented not in the User Interface, but in pure Delphi code.

Data filtering is a process run on the User entry: for instance, it will trim left or right spaces, make the text uppercase...
Data validating is performed before saving the data to the database: for instance, an email address is checked for consistency, a field value to be unique...

Some try to implement this using an external scripting engine, in a procedure/event mode. Back to the 80th...
In our ORM framework, filtering and validation can be performed by creating some Delphi classes, which may be used on both Server and Client side.

no trackbacks