Synopse

In case you were redirected from the previous "Synopse SQLite3 framework" category link, here is the new thread to be used instead:
http://blog.synopse.info/category/Open-Source-Projects/mORMot-Framework

Since revision 1.15 of the framework, it is able to connect to any database engine (therefore is not limited to SQLite3), and is now called mORMot.

Here is what wikipedia states at http://en.wikipedia.org/wiki/Shared_nothing_architecture:

A shared nothing architecture (SN) is a distributed computing architecture in which each node is independent and self-sufficient, and there is no single point of contention across the system. People typically contrast SN with systems that keep a large amount of centrally-stored state information, whether in a database, an application server, or any other similar single point of contention.

This is just one approach of "sharding". Sharding is indeed related to a shared nothing architecture - once sharded, each shard can live in a totally separate logical schema instance.
"I sharded, therefore it scales"...

You can do this in Delphi... and opens a new world of scaling opportunities... Just as Google, Facebook, or eBay do... 

This is a major step for the framework.

Among a lot of new features and bug fixes:

• Dynamic arrays and Record handling, including low-level access and high-level handling for the ORM part of the framework (using TDynArray wrapper);
• Business rules now handled via filtering and validating classes for both our ORM and the Synopse Big Table;
• New full-featured logging classes, with exception handling, stack trace and JSON serialization;
• Custom SQL functions and TSQLRecordRTree to implement R-Tree virtual tables;
• Batch sequences for adding/updating/deleting multiple records;
• Secure RESTful authentication via per-user light sessions, and per-query signature;
• Define custom Virtual Tables (i.e. access any kind of data from SQL) in Delphi code;
• New HTTP server using fast http.sys kernel-mode server and zip / SynLZ compression;
• SynTaskDialog unit native under Vista,Seven with emulation code under XP/2000 ;
• User Interface automated Generation using standard VCL or proprietary TMS components;
• Documentation completed - more than 600 pages by now, including general User Guide and exhaustive Technical Reference;
• Updated SQLite3 database engine to version 3.7.6.3;
• Numerous bug fixes, speed and feature enhancements (including in the PDF and other UI-related units).

Open Source project, for Delphi 6 up to XE, licensed under a MPL/LGPL/GPL tri-license.

When it deals with parsing some (textual) content, two directions are usually envisaged. In the XML world, you have usually to make a choice between:
- A DOM parser, which creates an in-memory tree structure of objects mapping the XML nodes;
- A SAX parser, which reads the XML content, then call pre-defined events for each XML content element.

In fact, DOM parsers use internally a SAX parser to read the XML content. Therefore, with the overhead of object creation and their property initialization, DOM parsers are typically three to five times slower than SAX. But, DOM parsers are much more powerful for handling the data: as soon as it's mapped in native objects, code can access with no time to any given node, whereas a SAX-based access will have to read again the whole XML content.

Most JSON parser available in Delphi use a DOM-like approach. For instance, the DBXJSON unit included since Delphi 2010 or the SuperObject or DWS libraries create a class instance mapping each JSON node.

In a JSON-based Client-Server ORM like ours, profiling shows that a lot of time is spent in JSON parsing, on both Client and Server side. Therefore, we tried to optimize this part of the library.

An R-Tree is a special index that is designed for doing range queries.

R-Trees are most commonly used in geospatial systems where each entry is a rectangle with minimum and maximum X and Y coordinates. Given a query rectangle, an R-Tree is able to quickly find all entries that are contained within the query rectangle or which overlap the query rectangle.

 This idea is easily extended to three dimensions for use in CAD systems. R-Trees also find use in time-domain range look-ups. For example, suppose a database records the starting and ending times for a large number of events. A R-Tree is able to quickly find all events, for example, that were active at any time during a given time interval, or all events that started during a particular time interval, or all events that both started and ended within a given time interval. And so forth. See http://www.sqlite.org/rtree.html

Since the 2010-06-25 source code repository update, the RTREE extension is compiled by default within all supplied .obj files of the framework.

A dedicated ORM class, named TSQLRecordRTree, is available to create such tables. It inherits from TSQLRecordVirtual, like the other virtual tables types (e.g. TSQLRecordFTS3 or our custom virtual tables).

Commonly, it can be achieved, in the SOA over HTTP world via:
- HTTP basic auth over HTTPS;
- Cookies and session management;
- Query Authentication with additional signature parameters.

We'll have to adapt, or even better mix those techniques, to match our framework architecture at best.

Each authentication scheme has its own PROs and CONs, depending on the purpose of your security policy and software architecture.

Since Windows XP SP2 and Windows Server 2003, the Operating System provides a kernel stack to handle HTTP requests. This http.sys driver is in fact a full featured HTTP server, running in kernel mode. It is part of the networking subsystem of the Windows operating system, as a core component.

The SynCrtSock unit can now implement a HTTP server based on this component. Of course, our SQLite3 framework will use it. If it's not available, it will launch our pure Delphi optimized HTTP server, using I/O completion ports and a Thread Pool.

The Synopse SQLite3 Database Framework was just released under version 1.12:

• internal SQLite3 database engine was updated to version 3.7.5, and SQL functions have been enhanced;
• now handle prepared SQL statements with a neutral automated syntax;
• with Delphi 2009/2010/XE, you can define directly Unicode string properties in TSQLRecord (will be stored as UTF-8);
• include some overloaded methods using Variants for direct property access (e.g. in any grid);
• SynPdf has now direct bookmarks and links methods, can reuse any existing bitmap, and generate PDF/A-1 files;
• internal reporting now supports bookmarks and links (with on screen navigation), and better integrates with SynPdf;
• a lot of bug fixes, new methods and enhancements.

The Synopse SQLite3 Database Framework was just released under version 1.11:

- internal SQLite3 database engine is updated to version 3.7.4;
- new TSQLRecordMany to handle "has many" and "has many through" relationships;
- new TSQLRecordFTS4 class, to handle the new FTS4 extension module;
- new RANK() function available in SQL statements for ranking FTS3/FTS4;
- introduces new TSQLite3DB, TSQLite3Statement, TSQLite3Blob, TSQLite3Value and TSQLite3FunctionContext types to clarify SQLite3 internal handle usage;
- new sqlite3_busy_timeout and sqlite3_busy_handler low-level functions, with new TSQLDataBase.BusyTimeout property;
- now handles User Defined Functions, via sqlite3_create_function_v2 and corresponding sqlite3_result_* functions;
- new MOD() and SOUNDEX() functions available in SQL statements;
- a lot of code refactoring, speed improvements, numerous fixes, new methods.

The Synopse SQLite3 Database Framework was just released under version 1.10:
- internal SQLite3 database engine is updated to version 3.7.3;
- code modifications to compile with Delphi 6 compiler;
- enhancements in TSQLRestServerStatic, for easier stand-alone work of this in-memory database engine;
- new SQLite3Edit unit, for automated creation of a UI window, ready to edit any TSQLRecord, with no RAD necessary (all components are created from RTTI): think this is an ORM for User Interface.

This version compiles from Delphi 6 up to Delphi XE.