Synopse Open Source

In case you were redirected from the previous "Synopse SQLite3 framework" category link, here is the new thread to be used instead: http://blog.synopse.info/category/Open-Source-Projects/mORMot-Framework Since revision 1.15 of the framework, it is able to connect to any database engine  […]

Everybody is buzzing about FireMonkey...

Our little mORMot will like FireMonkey!
Here is why...

For our ORM framework, we implemented an efficient SQLite3 wrapper, joining statically (i.e. without any external dll) the SQLite3 engine to the executable. SQLite3 is in fact used as the DB kernel of the framework. For instance, thanks to its unique virtual table mechanism, even tables in other databases (like Oracle or MSSQL) are available as if they were SQLite3 tables.

We just made this wrapper independent from our ORM, in a new dedicated unit, named SynSQLite3.pas.

It was an easy task to let this unit be called from our SynDB database abstract classes.

Our ORM framework has been released as version 1.14.

It's mainly a bug-fix release:

• Integrated SQLite3 engine updated to latest version 3.7.7.1;
• Fix several issues about JSON generation layout;
• Enhanced automated User Interface generation for object on-screen edition;
• SynPdf unit now handles Bézier curves from TCanvas, and some CMYK functions; also enhanced PDF/A-1compatibility;
• Some speed enhancements, and new functions for the SynOleDB unit.

This is a major step for the framework.

Among a lot of new features and bug fixes:

• Dynamic arrays and Record handling, including low-level access and high-level handling for the ORM part of the framework (using TDynArray wrapper);
• Business rules now handled via filtering and validating classes for both our ORM and the Synopse Big Table;
• New full-featured logging classes, with exception handling, stack trace and JSON serialization;
• Custom SQL functions and TSQLRecordRTree to implement R-Tree virtual tables;
• Batch sequences for adding/updating/deleting multiple records;
• Secure RESTful authentication via per-user light sessions, and per-query signature;
• Define custom Virtual Tables (i.e. access any kind of data from SQL) in Delphi code;
• New HTTP server using fast http.sys kernel-mode server and zip / SynLZ compression;
• SynTaskDialog unit native under Vista,Seven with emulation code under XP/2000 ;
• User Interface automated Generation using standard VCL or proprietary TMS components;
• Documentation completed - more than 600 pages by now, including general User Guide and exhaustive Technical Reference;
• Updated SQLite3 database engine to version 3.7.6.3;
• Numerous bug fixes, speed and feature enhancements (including in the PDF and other UI-related units).

Open Source project, for Delphi 6 up to XE, licensed under a MPL/LGPL/GPL tri-license.

When use the so-called BATCH sequences?

In a standard Client-Server architecture, especially with the common understanding (and most implementations) of a RESTful service, any Add / Update / Delete method call requires a back and forth flow to then from the remote server.

In case of a remote connection via the Internet (or a slow network), you could have some 100 ms of latency: it's just the "ping" timing, i.e. the time spent for your IP packet to go to the server, then back to you.

If you are making a number of such calls (e.g. add 1000 records), you'll have 100*1000 ms = 100 s = 1:40 min just because of this network latency!

The BATCH sequence allows you to regroup those statements into just ONE remote call. Internally, it builds a JSON stream, then post this stream at once to the server. Then the server answers at once, after having performed all the modifications.

According to the n-Tier architecture, data filtering and validation should be implemented in the business logic, not in the User Interface.

If you were used to develop RAD database application using Delphi, you may have to change a bit your habits here. Data filtering and validation should be implemented not in the User Interface, but in pure Delphi code.

Data filtering is a process run on the User entry: for instance, it will trim left or right spaces, make the text uppercase...
Data validating is performed before saving the data to the database: for instance, an email address is checked for consistency, a field value to be unique...

Some try to implement this using an external scripting engine, in a procedure/event mode. Back to the 80th...
In our ORM framework, filtering and validation can be performed by creating some Delphi classes, which may be used on both Server and Client side.

The SQlite3 engine has ability to create Virtual Tables from code. From the perspective of an SQL statement, the virtual table object looks like any other table or view. But behind the scenes, queries from and updates to a virtual table invoke callback methods on the virtual table object instead of reading and writing to the database file.

The virtual table mechanism allows an application to publish interfaces that are accessible from SQL statements as if they were tables. SQL statements can in general do anything to a virtual table that they can do to a real table, with the following exceptions:
- One cannot create a trigger on a virtual table.
- One cannot create additional indices on a virtual table. (Virtual tables can have indices but that must be built into the virtual table implementation. Indices cannot be added separately using CREATE INDEX statements.)
- One cannot run ALTER TABLE ... ADD COLUMN commands against a virtual table.
- Particular virtual table implementations might impose additional constraints. For example, some virtual implementations might provide read-only tables. Or some virtual table implementations might allow INSERT or DELETE but not UPDATE. Or some virtual table implementations might limit the kinds of UPDATEs that can be made.

Example of virtual tables, already included in the SQLite3 engine, are FTS or RTREE tables. A custom virtual table might represent in-memory data structures (like TSQLVirtualTableJSON, TSQLVirtualTableBinary). Or it might represent a view of data on disk that is not in the SQLite3 format (e.g. TSQLVirtualTableLog). Or the application might compute the content of the virtual table on demand.

Thanks to the generic implementation of Virtual Table in SQLite3, you can use such tables in your SQL statement, and even safely execute a SELECT statement with JOIN or custom functions, mixing normal SQLite3 tables and any other Virtual Table.

A dedicated mechanism has been added to the framework, beginning with revision 1.13, in order to easily add such virtual tables with pure Delphi code, just by inheriting some classes.

Since Windows XP SP2 and Windows Server 2003, the Operating System provides a kernel stack to handle HTTP requests. This http.sys driver is in fact a full featured HTTP server, running in kernel mode. It is part of the networking subsystem of the Windows operating system, as a core component.

The SynCrtSock unit can now implement a HTTP server based on this component. Of course, our SQLite3 framework will use it. If it's not available, it will launch our pure Delphi optimized HTTP server, using I/O completion ports and a Thread Pool.