2011-05-14

Virtual Tables in the SQLite3 framework

The SQlite3 engine has ability to create Virtual Tables from code. From the perspective of an SQL statement, the virtual table object looks like any other table or view. But behind the scenes, queries from and updates to a virtual table invoke callback methods on the virtual table object instead of reading and writing to the database file.

The virtual table mechanism allows an application to publish interfaces that are accessible from SQL statements as if they were tables. SQL statements can in general do anything to a virtual table that they can do to a real table, with the following exceptions:
- One cannot create a trigger on a virtual table.
- One cannot create additional indices on a virtual table. (Virtual tables can have indices but that must be built into the virtual table implementation. Indices cannot be added separately using CREATE INDEX statements.)
- One cannot run ALTER TABLE ... ADD COLUMN commands against a virtual table.
- Particular virtual table implementations might impose additional constraints. For example, some virtual implementations might provide read-only tables. Or some virtual table implementations might allow INSERT or DELETE but not UPDATE. Or some virtual table implementations might limit the kinds of UPDATEs that can be made.

Example of virtual tables, already included in the SQLite3 engine, are FTS or RTREE tables. A custom virtual table might represent in-memory data structures (like TSQLVirtualTableJSON, TSQLVirtualTableBinary). Or it might represent a view of data on disk that is not in the SQLite3 format (e.g. TSQLVirtualTableLog). Or the application might compute the content of the virtual table on demand.

Thanks to the generic implementation of Virtual Table in SQLite3, you can use such tables in your SQL statement, and even safely execute a SELECT statement with JOIN or custom functions, mixing normal SQLite3 tables and any other Virtual Table.

A dedicated mechanism has been added to the framework, beginning with revision 1.13, in order to easily add such virtual tables with pure Delphi code, just by inheriting some classes.

Continue reading

2011-04-14

Enhanced logging in SynCommons

Logging is everything... unless you never wrote a bug in your program! :)

Let us introduce a new logging class:

  • logging with a set of levels;
  • fast, low execution overhead;
  • can load .map file symbols to be used in logging;
  • compression of .map into binary .mab (900 KB -> 70 KB);
  • inclusion of the .map/.mab into the .exe;
  • reading of an external .map to add unit names and line numbers to a log file without .map available information at execution;
  • exception logging (Delphi or low-level exceptions) with unit names and line numbers;
  • optional stack trace with units and line numbers;
  • methods or procedure recursive tracing, with Enter and auto-Leave;
  • high resolution time stamps, for customer-side profiling of the application execution;
  • set / enumerates / TList / TPersistent / TObjectList / TContainer / dynamic array JSON serialization;
  • per-thread or global logging;
  • multiple log files on the same process;
  • integrated log archival (in zip or any other format);
  • Open Source, works from Delphi 6 up to XE.

Continue reading

2011-03-12

TDynArray and Record compare/load/save using fast RTTI

The SynCommons unit has been enhanced:
- new BinToBase64 and Base64ToBin conversion functions;
- new low-level RTTI functions for handling record types: RecordEquals, RecordSave, RecordSaveLength, RecordLoad;
- new TDynArray object, which is a wrapper around any dynamic array.

With TDynArray, you can access any dynamic array (like TIntegerDynArray = array of integer) using TList-like properties and methods, e.g. Count, Add, Insert, Delete, Clear, IndexOf, Find, Sort and some new methods like LoadFromStream, SaveToStream, LoadFrom and SaveTo which allow fast binary serialization of any dynamic array, even containing strings or records - a CreateOrderedIndex method is also available to create individual index according to the dynamic array content. You can also serialize the array content into JSON, if you wish.

What I like with dynamic arrays is that they are reference-counted, don't need any Create/try..finally...Free code, and are well handled by the Delphi compiler.
They are no replacement to a TCollection nor a TList (which are the standard and efficient way of storing class instances), but they are very handy way of having a list of content or a dictionary at hand, with no class nor properties definition.
You can look at them like Python's list, tuples (via records handling) and dictionaries (via Find method), in pure Delphi. Our new methods (about searching and serialization) allow most usage of those script-level structures in your Delphi code.

Continue reading

2011-03-11

HTTP server using fast http.sys kernel-mode server

Since Windows XP SP2 and Windows Server 2003, the Operating System provides a kernel stack to handle HTTP requests. This http.sys driver is in fact a full featured HTTP server, running in kernel mode. It is part of the networking subsystem of the Windows operating system, as a core component.

The SynCrtSock unit can now implement a HTTP server based on this component. Of course, our SQLite3 framework will use it. If it's not available, it will launch our pure Delphi optimized HTTP server, using I/O completion ports and a Thread Pool.

Continue reading

2011-03-05

Open Source SynTaskDialog unit for XP,Vista,Seven

A task dialog is a dialog box that can be used to display information and receive simple input from the user. Like a message box, it is formatted by the operating system according to parameters you set. However, a task dialog has many more features than a message box.

Windows provides a generic task dialog available since Vista/Seven. But there is none available with previous versions of Windows, i.e. Windows XP or 2K.

This unit (licensed under a MPL/GPL/LGPL tri-license) will use the new TaskDialog API under Vista/Seven, and emulate it with pure Delphi code and standard themed VCL components under XP or 2K. It will compile from Delphi 6 up to XE, and is Unicode ready.

Continue reading

2011-02-23

Fast JSON (un)serialization

The core of our SQLite3 framework is using JSON for all its Client/Server communication, and also for the internal cache of previous SQL requests.

There is indeed a damn fast JSON generator and parser in the framework. 

I've written some functions able to serialize and unserialize not only TSQLRecord (which is the main parent class of our ORM), but any TPersistent class instance.

Continue reading

2011-02-13

SynProject 1.12 released

Synopse SynProject is an open source (GPL) application for code source versioning and automated documentation of Delphi projects.

It has been updated to version 1.12.

Main enhancement of this update (among various fixes or new commands) is the generation of vectorial diagrams for all documents, for better rendering than PNG or JPEG.

Continue reading

2011-02-08

Synopse PDF Engine 1.12

Our SynPdf unit has been refreshed to the 1.12 version.

The main enhancements are:

  • Can now generate PDF/A-1 files;
  • Direct methods for creating bookmarks, links and document outline;
  • Document information can now be Unicode encoded;
  • The bitmaps can be cached and reused  if drawn multiple times;
  • Some corrected issues.

Continue reading

- page 42 of 52 -