Synopse Open Source

Here are the main modifications, grouped by unit:

SynCommons

- unit now compiles and works with Delphi 5 compiler
- new low-level RTTI functions for handling record types: RecordEquals, RecordSave, RecordSaveLength, RecordLoad, RecordClear and RecordCopy
- new TDynArray object, which is a wrapper around any dynamic array: you can now access to the dynamic array using TList-like properties and methods, e.g. Count, Add, Insert, Delete, Clear, IndexOf, Find, Sort and some new methods like LoadFromStream, SaveToStream, LoadFrom and SaveTo which allow fast binary serialization of any dynamic array, even containing strings or records; a CreateOrderedIndex method is also available to create individual index according to the dynamic array content; and any dynamic array can be serialized as UTF-8 JSON via TTextWriter.AddDynArrayJSON and TDynArray.LoadFromJSON methods
- introducing direct content filtering and validation using TSynFilterOrValidate dedicated classes, for both TSQLRecord and SynBigTable
- filtering is handled via some TSynFilter classes - TSynFilterUpperCase, TSynFilterUpperCaseU, TSynFilterLowerCase, TSynFilterLowerCaseU and TSynFilterTrim e.g.
- validation is handled via some TSynValidate classes - TSynValidateRest, TSynValidateIPAddress, TSynValidateEmail, TSynValidatePattern, TSynValidatePatternI, TSynValidateText, TSynValidatePassWord e.g.
- dedicated TSynTableFieldProperties.Validate method for validation (e.g. a TSynValidateTableUniqueField instance is created if tfoUnique is in Options)
- dedicated TSynTableFieldProperties.Filter method for filtering (using common TSynFilter classes, working at UTF-8 Text content)
- faster implementation of Move() for Delphi versions with no FastCode inside
- new ConvertCaseUTF8(), UpperCaseU(), LowerCaseU(), Int64ToUInt32(), GetCardinalDef(), IsValidEmail, IsValidIP4Address(), PatchCodePtrUInt(), GetCaptionFromClass(), GetDisplayNameFromClass(), DateTimeToIso8601Text() StrUInt32(), StringBufferToUtf8(), IsZero(), AddPrefixToCSV(), IntToString(), RawUTF8DynArrayEquals(), FromVarString(), GetBitCSV(), SetBitCSV() procedures or functions (with associated tests)
- new grep-like IsMatch() function for basic pattern matching
- new BinToBase64, Base64ToBin and IsBase64 *fast* conversion functions (with optimized assembler version, using CPU pipelining and lookup table)
- introducing the GarbageCollector TObjectList for handling a global garbage collector for instances which must live during the whole executable process (used e.g. to avoid a memory leak for "class var" or such variables)
- new TSynLog class to handle enhanced logging to any application, with exception handling (+stack trace) and customer-side performance profiling
- new TSynMapFile class to retrieve debugging information from .map file (and able to save and read smaller .mab files) - used by TSynLog if available
- new TSynTestsLogged test suit class, with automated test case logging
- Windows version detection enhanced, now retrieving TWindowsVersion enumerate
- great performance improvement in TSynTableFieldProperties for update process
- added TMemoryMap and TSynMemoryStreamMapped to handle memory-mapped files
- added TMemoryMapText class to fast handle big UTF-8 text files (like logs)
- now TTextWriter can have a custom internal buffer size (default 1024 bytes)
- now TFileBufferWriter and TFileVersion are regular classes, not an object/record any more (this was incoherent since Delphi 2010)
- new TFileBufferReader.OpenFrom(Stream) and ReadRawUTF8 methods
- now TSynCache will use faster TDynArrayHashed for its content hashing
- new Escape: TTextWriterKind optional parameter for TTextWriter.Add()
- new SynLZ related compression functions: FileSynLZ/FileUnSynLZ and StreamSynLZ/StreamUnSynLZ
- source can now be parsed per all Delphi IDE pre-parser (dual declaration as record or object because of Delphi 2010 and up was not understood)
- fixed issue in TSynTable.Data() method: ID was not set as expected
- fixed issue in TSynTableFieldProperties: wrong constraint evaluation and index refresh at records update
- fixed issue in ToVarUInt32Length/ToVarUInt32LengthWithData

SynCrtSock

- code modifications to compile with Delphi 5 compiler
- new THttpApiServer class, using fast http.sys kernel-mode server for better performance and less resource usage
- DOS / TCP SYN Flood detection time enhanced to 5 seconds
- fixed HTTP client stream layout (to be more RFC compliant)
- new generic compression handling mechanism: can handle gzip, deflate or custom synlz / synlzo algorithms via THttpSocketCompress functions
- new THttpServerGeneric.Request virtual abstract method prototype
- new TWinINet class, using WinINet API (very slow, do not use)
- new TWinHTTP class, using WinHTTP API (faster than THttpClientSocket): this is the class to be used

SynCrypto

- code modifications to compile with Delphi 5 compiler

SynGdiPlus

- code modifications to compile with Delphi 5 compiler
- handle TCanvas.DrawCurve() method in TMetaFile enumeration
- suppress GDI+ library back thread which may hang up application when using this unit in a DLL - manual hook and unhook is done at statup/shutdown see http://mikevdm.com/BlogEntry/Key/GdiplusShutdown-Hangs-Mysteriously

SynLZ

- code modifications to compile with Delphi 5 compiler
- comment refactoring (mostly for inclusion in SynProject documentation)
- new CompressSynLZ function, for THttpSocket.RegisterCompress - this function will return 'synlzo' as "ACCEPT-ENCODING:" HTTP header parameter

SynLZO

- code modifications to compile with Delphi 5 compiler
- comment refactoring (mostly for inclusion in SynProject documentation)
- new CompressSynLZO function, for THttpSocket.RegisterCompress - those functions will return 'synlzo' as ACCEPT-ENCODING: HTTP header parameter

SynPdf

- code modifications to compile with Delphi 5 compiler
- added horizontal scaling for GDI enumeration in case of text kerning (could occur for small fonts)
- fixed "Save when closing with Acrobat Reader X" - thanks to Ondrej
- fixed clipping problems and vertical font positioning issue in GDI enumeration - thanks to Ondrej for those corrections!

SynZip

- code modifications to compile with Delphi 5 compiler
- new CompressGZip and CompressDeflate functions, for THttpSocket.RegisterCompress
- now handle Unicode file names UTF-8 encoded inside .Zip archive
- new TZipWrite.CreateFrom constructor, to add some new content to an existing .Zip archive
- EventArchiveZip function can be used as a TSynLogArchiveEvent handler to compress old .log files into a .zip standard archive

SynZipFiles

- code modifications to compile with Delphi 5 compiler
- handle Unicode versions of Delphi (Delphi 2009/2010/XE)
- now officialy handle UTF-8 encoded file names inside .Zip archive

SQLite3Commons

- the ORM will now include all published properties of the parents, up to TSQLRecord, to the database fields (it was only using the published properties at the topmost class level)
- dynamic arrays can now be specified for TSQLRecord published properties: a new sftBlobDynArray field kind has been added - will be stored as BLOB in the database (following the TDynArray.SaveTo binary stream layout), and will be transmitted as Base64 encoded in the JSON stream - we implemented a sftBlobRecord field kind, but Delphi RTTI is not generated for published properties of records: so our code is disabled (see PUBLISHRECORD conditional) - but you can use dynamic arrays of records
- TPersistent can be now be specified for TSQLRecord published properties: a new sftObject field kind has been added - will be stored as TEXT in the database (following the ObjectToJSON serialization format) - TStrings or TRawUTF8List will be stored as a JSON array of string, and TCollection as a JSON array of objects, other TPersistent classes will have their published properties serialized as a JSON object
- introducing direct content filtering and validation using TSynFilterOrValidate dedicated classes
- filtering is handled directly in the new TSQLRecord.Filter virtual method, or via some TSynFilter classes - TSynFilterUpperCase, TSynFilterUpperCaseU, TSynFilterLowerCase, TSynFilterLowerCaseU and TSynFilterTrim e.g.
- validation is handled in the new TSQLRecord.Validate virtual method, or via some TSynValidate classes - TSynValidateRest, TSynValidateIPAddress, TSynValidateEmail, TSynValidatePattern, TSynValidatePatternI, TSynValidateText, TSynValidatePassWord e.g.
- introducing TSQLRecordRTree to implement R-Tree virtual tables - and new TSQLRecordVirtual parent table for all virtual tables like TSQLRecordFTS*
- new TSQLRestClientURI methods to implement BATCH sequences to speed up database modifications: after a call to BatchStart, database modification statements are added to the sequence via BatchAdd/BatchUpdate/BatchDelete, then all statments are sent as once to the remote server via BatchSend - this is MUCH faster than individual calls to Add/Update/Delete in case of a slow remote connection (typically HTTP over Internet)
- introducing TSQLVirtualTableModule / TSQLVirtualTable / TSQLVirtualTableCursor classes for a generic Virtual table mechanism (used e.g. by TSQLVirtualTableModuleDB in the SQLite3 unit)
- new TSQLRecordVirtualTableAutoID and TSQLRecordVirtualTableForcedID classes, used to access any TSQLVirtualTable in our ORM
- security and per-user access rights is now implemented in the framework core using per-User authentication via in-memory sessions (stored as TAuthSession), with group-defined associated security parameters (via TSQLAuthUser and TSQLAuthGroup tables), and RESTful Query Authentication via URI signature; should avoid most MITM and replay attacks
- new TJSONSerializer class and ObjectToJSON/JSONToObject method (handles also dynamic arrays following the TTextWriter.AddDynArrayJSON format, i.e. plain JSON array for common types aka '[1,2,3]', but Base64 encoded stream aka '["\uFFF0base64encodedbinary"]' for other arrays) and corresponding UrlDecodeObject() function (to be called by RESTful Services implementation on Server side)
- wider usage of TSQLRecordProperties, for faster RTTI access, via the new class function TSQLRecord.RecordProps: TSQLRecordProperties: only virtual class function or procedure are now defined in TSQLRecord
- enhanced TPropInfo.GetLongStrValue/SetLongStrValue methods, now converting RawUnicode, WinAnsiString, TSQLRawBlob and AnsiString properties
- now ensure that no published property named ID or RowID was defined (this unique primary key field must be handled directly by TSQLRecord)
- MAX_SQLFIELDS default is still 64, but can now be set to any value (64, 128, 192 and 256 have optimized fast code) so that you can have any number of fields in a Table
- MAX_SQLTABLES default is now 256, i.e. you can have up to 256 tables in a TSQLModel instance (you can set any other value, on need)
- TSQLModel implementation speed up, in case of a huge number of registered TSQLRecord in the database Model (since MAX_SQLTABLES=256 by default)
- enhanced TSQLRecordMany.DestGetJoinedTable method to handle custom fields
- TSQLRecordMany.DestGetJoined* methods now accept generic TSQLRest class
- new aCustomFieldsCSV parameter for FillPrepare / CreateAndFillPrepare methods of TSQLRecord, to retrieve only neeeded fields: be aware that not specified fields will be left untouched, so a later Update() call may corrupt the row data - this optional parameter is about to save bandwidth when retrieving records field in a loop
- TSQLRestServerStaticInMemory can now store its content into UTF-8 JSON or an optimized (SynLZ) compressed binary format - associated TPropInfo GetBinary/SetBinary and TSQLRecord GetBinaryValues/SetBinaryValues methods
- the generic TVarData type is now used as a standard way of fast values communication: only handled VType are varNull, varInt64, varDouble, varString (mapping a constant PUTF8Char), and varAny (BLOB with size = VLongs[0]) - used e.g. by SQLite3 unit (VarDataToContext/VarDataFromValue)
- new TSQLRest.Retrieve(aPublishedRecord, aValue: TSQLRecord) and TSQLRecord.Create(aClient: TSQLRest; aPublishedRecord: TSQLRecord..) methods
- ExecuteList defined in TSQLRest, so now available also in TSQLRestServer
- added a magic pattern check to ignore broadcasted WM_COPYDATA message
- fixed issue in serializing sftCurrency type in TSQLRecord.GetJSONValue

SQLite3

- update SQLite3 engine to version 3.7.6.3
- added sqlite3InternalFreeObject(), sqlite3_malloc/realloc/free(), sqlite3_memory_used/highwater(), sqlite3_set_authorizer() and sqlite3_update/commit/rollback_hook() functions
- introducing TSQLVirtualTableModuleSQLite3 / TSQLVirtualTableModuleServerDB classes, and associated low-level sqlite3 functions and memory structures, in order to implement Virtual Table modules in pure Delphi code via common TSQLVirtualTable classes as defined in SQLite3Commons
- new IntegerDynArrayContains(), RawUTF8DynArrayContainsCase/NoCase() and Byte/Word/Cardinal/Int64/CurrencyDynArrayContains(BlobField,I64) SQL functions, able to fast search data within T*DynArray and TRawUTF8DynArray published properties BLOB (Currency mapped as PInt64)
- new TSQLDataBaseSQLFunction and TSQLDataBase.RegisterSQLFunction method, to implement custom SQL functions with any kind of BLOB data
- regression test now exclude unfixed order for select (may vary, just like happened for 3.7.6 when e.g. indexes started to be used)
- added regression tests for sftBlobDynArray and sftObject kind of records
- TSQLRestServerDB now uses faster TDynArrayHashed for its internal prepared statement cache
- fixed issue in TSQLRestClientDB.URI: wrong InternalState returned
- fastest internal cdecl qsort() function (optimized for PAnsiChar)

SQLite3HttpClient

- now can compress its content using deflate or faster SynLZ algorithm: by default, the SynLZ algorithm will be used between a Delphi Client and Server over HTTP/1.1 - there will be no speed penalty on the server side, whereas deflate would use much more CPU
- can make TCP/IP stream not HTTP compliant (against antivirus slowdown)
- new TSQLite3HttpClientWinINet class, using WinINet API (very slow)
- new TSQLite3HttpClientWinHTTP class, using WinHTTP API (fast and stable): this class should be considered to be used instead of TSQLite3HttpClient for any HTTP/1.1 client connection over a network - it is therefore the default TSQLite3HttpClient class since this 1.13 revision

SQLite3HttpServer

- can now use fast http.sys kernel-mode server (THttpApiServer) if available, and our pure Delphi THttpServer on default
- now can compress its content using deflate or faster SynLZ algorithm
- by default, will handle SynLZ compression for TSQLite3HttpClient
- can make TCP/IP stream not HTTP compliant (for security visions)
- TSQLite3HttpServer.Request now uses the new THttpServerGeneric.Request virtual abstract method prototype to handle THttpApiServer

SQLite3i18n

- fix URW699 issue when compiling with Delphi 6

SQLite3ToolBar

- now uses TSQLRecord.RecordProps instead of lowest level RTTI calls
- by default, will use only VCL components to create the Ribbon; can use proprietary TMS component pack if USETMSPACK global conditional is defined

SQLite3UI

- now compiles with Delphi 6
- new TSQLTableToGrid.SelectedRecordCreate method
- added generic TSynLabeledEdit to handle Integer, Int64, Currency and double kind of values

SQLite3UIEdit

- new OnComponentValidate property to allow custom field content validation
- now handle TSQLRecord automated filtering (using TSynFilter classes) and validation (using TSynValidate classes)
- unique field validation is now in TSQLRecord.Validate (better multi-tier architecture)
- hanle sftTimeLog and sftDateTime with a TDateTimePicker
- handle sftInteger (including Int64 fields), sftCurrency and sftFloat with a TSynLabeledEdit field
- now use TMS component pack only if USETMSPACK global conditional is defined: by default, will use only VCL components (i.e. TSynButton) for the form

SQLite3UILogin

- now use TMS component pack only if USETMSPACK global conditional is defined
- introducing new TSynButton, mapping either to the default TBitButton, either to TAdvGlowButton (if USETMSPACK conditional is defined)
- introducing a new TTaskDialog record/object, to access the new Vista/Seven TaskDialog, with a fallback dialog written in Delphi under XP
- use a best available font (Calibri or Tahoma), and useful bitmaps

SQLite3UIOptions

- Delphi 2009/2010/XE compatibility fixes

Full source code available to download, or from our Source Code repository.

Feedback and comments are welcome on our forum.