There were a lot of changes in this release!
Stay tuned... the list is quite huge...
You can go down directly to the download and forum links, if you want to.


Synopse mORMot 1.16 fixes and enhancements

This is a per-unit list of changes for the 1.16 release of mORMot:

Changes in details:

Unit SynBigTable.pas

  • New overriden TSynBigTableTable.Clear method which will clear fields;
  • New TSynBigTable.FileFormatCheck class method to recognize file format;
  • Replaced sbtGetMagic kind of header with InternalMagic class function;
  • Fixed issue in TSynTableFieldProperties.SaveTo about saving wrong indexes;
  • New sbtBeforeWrite step available (e.g. to safely update indexes).

Unit SynCommons.pas

  • Introducing new TSynAnsiConvert and TSynAnsiFixedWidth classes, able to process Unicode to/from Ansi conversion in all possible code pages, with generic access methods and optimized handling of fixed width encodings;
  • Added dedicated Exception classes (ESynException, ETableDataException);
  • TSynLog allows read sharing of the .log created file;
  • TSynLog now stores the executable build time, and library name (if any) - this is a small change of the .log format as expected by the LogViewer tool (so you should upgrade your LogViewer.exe to its latest version);
  • TSynLog and TSynMapFile now handle libraries (.dll/.ocx/.bpl) .map/.mab debugging information (only .exe was previously handled);
  • TSynCache now handles an optional Tag: PtrInt value parameter (used e.g. to store the row counts of a SQL result cache in mORMot);
  • TSynCache now uses the generic TSynNameValue object from its internal hashed list implementation (avoid duplicated code);
  • TMemoryMapText class (and therefore TSynLogFile) is now able to map/open an existing file: it will allow e.g. the SynLogViewer to browse a .log file which is actually still opened and working by the main application;
  • Faster RawUnicodeToUtf8() and UTF8ToWideChar() functions, thanks to very clever speed-up proposals by Sha (also included in TSynAnsi* classes);
  • JSONDecode() overloaded functions now accept parameter names without case sensibility (and a new HandleValuesAsObjectOrArray parameter);
  • New JSONDecode() overloaded function, to properly handle unserialization of a JSON object within a buffer (used e.g. for TDynArrayJSONCustomReader);
  • JSON functions now handle '0' as number according to http://json.org specs;
  • New TTextWriter.AddJSONEscape() overloaded function, to be used to directly serialize some name/value pairs as a JSON object content (used e.g. for TDynArrayJSONCustomWriter callbacks);
  • New FileSize(), RoundTo2Digits() and RawByteStringArrayConcat() functions;
  • New TPrecisionTimer Pause and Resume methods;
  • New TSynTestCase.CheckFailed method (most of the time, Check is sufficient);
  • New TSynLogFamily.IncludeComputerNameInFileName property;
  • New TTextWriter.WrRecord method for direct adding of a Base-64 record content;
  • New TTextWriter.AddNoJSONEscapeString method;
  • New TRawUTF8ListHashed class, with extend TRawUTF8List by using an internal hash table to optimized IndexOf() method call (including case sensitivity);
  • New ToVarInt64() and FromVarInt64() functions to encode and decode variable-length signed Int64 values (with the corresponding new tftVarInt64 kind of variable-length column in TSynTableFieldType enumeration);
  • New GotoNextJSONObjectOrArray() and RawUTF8ArrayToQuotedCSV() functions;
  • New ReadStringFromStream() and WriteStringToStream() functions;
  • Fixed some compilation warnings with Delphi XE and XE2;
  • Fixed issue in TDynArrayHashed if you do not use the external Count;
  • Fixed potential GPF in TDynArrayHashed.ReHash after TDynArray.Clear call;
  • Fixed issue in TSynTableFieldProperties.SaveTo about saving wrong indexes;
  • Fixed issue TSynTableStatement when only one column was retrieved;
  • Fixed rounding issue in ExtendedToString() and all corresponding wrappers like DoubleToStr*, Add(Double...);
  • Fixed issue in Hash32() implementation (potential GPF when reading ahead by DWORD - get rid of unnecessary asm optimization);
  • Fixed issues in function IsJSONString() which returned TRUE for '-' or '+', or false positives in some border-line cases (due to wrong uppercase guess): now this function is split into IsString() and IsStringJSON() functions to explicitely handle null/false/true constant recognition;
  • Fixed potential false positives of null/false/true in function GetJSONField;
  • Get rid of wrong "Decimal" parameter in float to text conversion;
  • TFileBufferWriter.Create now accepts up to 4 MB internal buffer size;
  • Increased TDynArrayHashed number of void entries (for speed);
  • Modified TDynArray.SaveToStream/LoadFromStream to read or save the data from the current stream position;
  • Fixed GPF in TDynArray.SaveTo in case of invalid internal record layout;
  • Modified StreamUnSynLZ() so that Source stream will point after all read data;
  • TDynArray.SaveToStream() method can now save to any TStream class;
  • Added TTextWriter.RegisterCustomJSONSerializer() method to allow JSON serialization of any dynamic array content (used by TDynArray.LoadFromJSON and TTextWriter.AddDynArrayJSON) and record content (used by RecordLoadJSON and TTextWriter.AddRecordJSON);
  • Added USEPACKAGES conditional to help compiling the unit within packages;
  • Added optional DOPATCHTRTL to patch the RTL (RecordCopy low-level function) only if needed (not patched by default, for compatibility reasons) - you may want to use our Enhanced RTL patchs instead for a whole better response;
  • New function BinToBase64URI();
  • Circumvent some bugs of Delphi XE2 background compiler (main compiler is OK);
  • Add premilinary Windows 8 operating system detection (as wEight/wEightServer).

Unit SynCrtSock.pas

  • Fixed issue in case of wrong void parameter e.g. in THttpApiServer.AddUrl;
  • Circumvent some bugs of Delphi XE2 background compiler (main compiler is OK);
  • Added 'RemoteIP: 127.0.0.1' to the retrieved HTTP headers;
  • Major speed up of THttpApiServer for Windows Vista and up, by processing huge content in chunks: upload of 100Mb file take 25 sec before and 6 sec after changes, according to feedback by MPV - ticket 711247b998;
  • New THttpServerGeneric.OnHttpThreadTerminate event, available to clean-up any process in the thread context, when it is terminated (to call e.g. TSQLDBConnectionPropertiesThreadSafe.EndCurrentThread in order to call CoUnInitialize from thread in which CoInitialize was initialy made) - see http://synopse.info/fossil/tktview?name=213544b2f5.

Unit SynCrypto.pas

  • Added TAESECB, TAESCBC, TAESCFB, TAESOFB and TAESCTR classes to handle AES encryption of memory buffers in ECB, CBC, CFB, OFB and CTR mode (including PKCS7 padding);
  • Added pure pascal version (for XE2 64 compilation) of all algorithms.

Unit SynDB.pas

  • Both TSQLDBStatement.FetchAllToJSON and FetchAllAsJSON methods now return the number of rows data fetched (excluding field names);
  • New class method TSQLDBConnectionProperties.GetFieldDefinition();
  • New method TSQLDBStatement.FetchAllToCSVValues() for fast to-file CSV export;
  • New TSQLDBStatement.ColumnsToSQLInsert() and BindFromRows() methods to allow fast data conversion/export between databases;
  • New TSQLDBConnectionProperties.SQLSelectAll method to retrieve a SELECT statement according to a DB column expected layout;
  • New TSQLDBConnectionProperties.ClearConnectionPool method (could be used to recreate all connections in case of DB or network failure/timeout);
  • Fixed issue in TSQLDBConnection.GetServerTimeStamp method.

Unit SynDBODBC.pas

  • First public release, corresponding to mORMot Framework 1.16.

Unit SynDBOracle.pas

  • LONG columns will be handled as ftUTF8 fields, truncated to 32 KB of text (fix error ORA-00932 at OCI client level);
  • We found out that a computed column is returned as SQLT_NUM with Scale=0 even if it is numeric with decimals (OCI 11.2 bug) -> so SQLT_INT trick is disabled;
  • Fixed issue when using transactions methods with Oracle.

Unit SynDBSQLite3.pas

  • Implemented the SQLite3 private encryption using a password (beta feature - better not to be used on production);
  • Added RowsToSQLite3() functions for direct export of any DB statement rows into a SQLite3 database;
  • New TSQLDBSQLite3ConnectionProperties.UseMormotCollations property to allow SQL table creation statement with or without the mORMot collations.

Unit SynGdiPlus.pas

  • Made the TMetaFile rendering engine stronger to malformed EMF content (e.g. when a EMR_SELECTOBJECT item refers to an out-of-range object);
  • Fixed TLockModeOption definition by adding a TLockModeOptions set - see ticket b5a31dd269.

Unit SynLZ.pas

  • Fixed potential GPF issue in Hash32() function.

Unit SynOleDB.pas

  • Add some reference to http://synopse.info/fossil/tktview?name=213544b2f5 in case of wrong implementation of multi-thread connection (within the THttpServerGeneric mORMot server, for instance).

Unit SynPdf.pas

  • Includes new TSynAnsiConvert classes for handling Ansi charsets;
  • Do not stop TMetaFile enumeration in case of invalid EMF content (e.g. if the EMR_SELECTOBJECT refers to an out-of-range object): this is the default behavior of GDI and GDI+ renders (and our SynGdiPlus), so we'll stay to it - may fix issue with some badly formatted objects - also made the TMetaFile rendering stronger to badly formated EMF input;
  • Fixed issue in TPdfDocument.CreateOrGetImage about guessing if a bitmap is to be reused as a pdf object;
  • Added TPdfDocument.ForceNoBitmapReuse property;
  • Added a "Decimals: cardinal=6" parameter to TPdfCanvas.ConcatToCTM;
  • TPdfCanvas.SetDash parameter is now an array of integer;
  • Set PDF_MAX_FONTSIZE limit to 2000 - should be big enough in practice;
  • Fixed an issue when handling bitmap palette;
  • Fixed an issue when the first time a font was used is as Unicode;
  • Fixed a potential GPF issue in function HashOf() in PUREPASCAL mode (used to reuse any existing bitmap content within the PDF document).

Unit SynSelfTests.pas

  • Added interface-based remote service implementation tests;
  • Added test about per-database encryption in TTestExternalDatabase.CryptedDatabase;
  • Added TAESECB, TAESCBC, TAESCFB, TAESOFB and TAESCTR classes tests (+ PKCS7);
  • Enhanced SynLZ tests (comparing asm and pas versions of the implementation).

Unit SynSQLite3.pas

  • Updated SQLite3 engine to version 3.7.12.1;
  • Unit now includes FTS3/FTS4 by default (i.e. INCLUDE_FTS3 conditional is set in both SQLite3.pas and SynSQLite3.pas units);
  • Added sqlite3_changes() and sqlite3_total_changes() function prototypes;
  • New TSQLDataBase.LastChangeCount method (wrapper around sqlite3_changes);
  • New IsSQLite3FileEncrypted() function;
  • New TSQLRequest.FieldBlobToStream and Bind(TCustomMemoryStream) methods;
  • New parameter in TSQLDataBase.ExecuteJSON, LockJSON, UnLockJSON methods, for an optional integer pointer, to return the count of row data;
  • Added an optional behavior parameter to TSQLDataBase.TransactionBegin method;
  • Reintroduce TSQLDataBaseSQLFunction.Create() constructor, and added some TSQLDataBase.RegisterSQLFunction() overloaded methods;
  • Fixed issue in TSQLRequest.Reset() which was triggered an error about the latest statement execution;
  • Fixed potential issue after TSQLStatementCached.ReleaseAllDBStatements;
  • Fixed rounding issue when exporting DOUBLE columns into JSON;
  • Fixed issue of unraised exception in TSQLRequest.PrepareNext;
  • TSQLRequest.Execute(JSON: TStream) now allows field names at least, even with no data (as expected by TSQLRestClientURI.UpdateFromServer);
  • Renamed ESQLException into ESQLite3Exception;
  • Engine is now compiled including tracing within the FTS3 extension - added sqlite3_trace() function prototype to register your own tracing callback.

Unit SynTaskDialog.pas

  • Fixed issue when changing the current application with Alt+Tab - see http://synopse.info/fossil/tktview?name=01395e5932;
  • Fixed compiler error when using the unit with runtime packages enabled (known compiler issue about string resources, referenced as E2201);
  • Default modal dialog parent changed into any current active form;
  • Added tdfQueryFieldFocused optional flag to focus the input field component;
  • Some aesthetical rendering changes and code clean-up (e.g. no temporary form necessary), thanks to uligerhardt proposals.

Unit SynZip.pas

  • Unit now compiles with Delphi XE2 (64 Bit);
  • TZipWrite.AddDeflated(const aFileName) method will use streaming instead of in-memory compression (will handle huge files much efficiently, e.g. log files as for EventArchiveZip).

Unit SQLite3.pas

  • Updated SQLite3 engine to version 3.7.12.1;
  • Unit now includes FTS3/FTS4 by default (i.e. INCLUDE_FTS3 conditional is set in both SQLite3.pas and SynSQLite3.pas units);
  • Fixed TSQLRestServerDB.UpdateField(ByID=true) implementation;
  • Fixed VACUUM failure if there are one or more active SQL statements;
  • New overloaded TSQLRestServerDB.UpdateField method;
  • TSQLRestServerDB.EngineList() method now handles an optional integer pointer, to return the count of row data;
  • Updated TSQLRestServerTest published methods to use the new parameter layout using "var aParams: TSQLRestServerCallBackParams".

Unit SQLite3Commons.pas

  • mORMot framework now implements Client-Server service implementation using regular Delphi interfaces (over a RESTful or JSON-RPC protocol), using TServiceContainer / TServiceFactory classes, accessible via TSQLRest.Services property, on both client and server side, with auto-marshaling, JSON serialization, and built-in security;
  • Added dedicated Exception classes (EORMException, EParsingException, ESecurityException, ECommunicationException, EBusinessLayerException, EServiceException) all inheriting from SynCommons.ESynException;
  • Added a generic JSON error message mechanism within the framework (including error code as integer and text, with custom error messages in RecordCanBeUpdated method and also in TSQLRestServerCallBackParams);
  • Added TSQLRestServerFullMemory class to implement a basic REST server using only in-memory tables (will only handle CRUD commands, but is enough to handle authentication with optional persistence in JSON file);
  • Added TSQLRestServerRemoteDB class to implement a REST server using a remote TSQLRestClient connection for all its ORM process: can be used e.g. to publish services with a dedicated process in a DMZ;
  • Deep refactoring of TSQLRestClient / TSQLRestClientURI methods, for better compliance with the Liskov substitution principle (LSP);
  • TSQLRestServer published method names are now hash-stored for speed;
  • The TSQLRestServerCallBack method prototype has been modified to supply "var aParams: TSQLRestServerCallBackParams: cardinal" as unique parameter: this is a CODE BREAK change and you shall refresh ALL your server-side code to match the new signature (using a record passed by value as parameter will ensure faster code and seamless evolution of this structure);
  • New TSQLRestServer.SessionGetUser method to be used e.g. by any TSQLRestServerCallBack method implementation to retrieve the connected user;
  • Now the TSQLAuthUser instance retrieved during the session opening will retrieve the Data BLOB field (ready to be consummed on the server side);
  • Introduced TSQLRestServerSessionContext structure to include current User and Group ID to the execution context (in addition to the Session ID);
  • TSQLRestServerStaticInMemory binary format changed: now will store its content per field (to ensure better compression) - magic identifier changed;
  • Now TSQLRestClientURI BATCH sequences methods will allow to mix several TSQLRecord in its sequence of BatchAdd/BatchUpdate/BatchDelete calls - if initiated as BatchStart(nil);
  • Code clean-up of TSQLRestClientURI.SetUser + added aHashedPassword optional parameter (to use already hashed password);
  • Added TSQLOccasion to handle the special case of field type (like TCreateTime) in case of Upate/Insert/Select;
  • TCreateTime published fields now are not modified at update;
  • Fixed unexpected exception raised in TSQLRecord.FillOne if FillPrepare was successfull, but did not return any row;
  • Introducing TSQLRest.Cache and TSQLRestCache class to handle Client or Server side fast in-memory cache (with tuned configuration and timeout);
  • Associated TSQLRestServer.CacheFlush service for flushing the Server cache, and remote TSQLRestClientURI.ServerCacheFlush() method for the client;
  • Fixed issue in TSQLRecord.FillPrepare when the table has less columns that the filling TSQLTable (can occur e.g. when using aCustomFieldsCSV parameter in FillPrepare method);
  • EngineList methods (including TSQLRestServerStaticInMemory class) now handles an optional integer pointer, to return the count of row data;
  • Uses new generic TSynAnsiConvert classes for code page process: that is, SQLite3i18n S2U() and U2S() match the SynCommons StringToUTF8() and UTF8ToString() functions - therefore, the TUTF8ToStringEvent parameter is not useful any more;
  • More than MAX_SQLFIELDS-1 columns (by default, 63) will raise an exception;
  • Added TJSONSerializer.RegisterCustomSerializer() method to allow JSON serialization of any class (thanks Pavel "aka mpv" for the idea and patch);
  • Added TSQLRestServer.ServiceMethodByPassAuthentication method in order to allow by-pass of the RESTful authentication scheme for some methods (can be used e.g. to server some HTML content for a non SOA client);
  • Fix issue about missing last item in JSONToObject() function;
  • Fix issue when handling null JSON objects in GetJSONObjectAsSQL() function;
  • JSON functions now handle '0' as number according to http://json.org specs;
  • Fix issue about record locking in TSQLRestClientURI.Retrieve method;
  • Fix execution issue in TSQLRestServer.AfterDeleteForceCoherency();
  • Fix issue about abusive session timeout: TSQLRestServer.SessionGet is now renamed SessionAccess and refreshes the session access timestamp each time a session is retrieved (+internal implementation fix);
  • Fix issue in SetInt64Prop() procedure which failed the update of a property with no explicit setter;
  • Fix issue in TSQLRecord.FillFrom() which forgot to update InternalState;
  • Fix issue in TPropInfo.SetValue + TSQLRecord.ClearProperties with Value=nil;
  • Fix potential formating issue in TSQLTable.GetJSONValues/GetCSVValues methods which may create some wrong formating if TEXT is null/false/true (were formerly recognized as JSON keywords, whereas it should have already been transformed into nil, '0' or '1');
  • Fix issue of unhandled buffer in TSQLTableJSON.UpdateFrom();
  • Fix issue about transactions not working with TSQLRestServerStaticInMemory;
  • Fix issue in TSQLRestServerStaticInMemory on SELECT with only one column;
  • Fix TSQLTable.GetCSVValues() format (adding UTF-8 BOM);
  • TSQLRestServer.URI now returns "Location:" header without the digital signature (e.g. 'Location: People/11012') for a POST (=CRUD create/add);
  • TSQLRestClient.List and ListFmt methods now use TSQLRecordClass open array instead of TClass (for consistency);
  • New global RecordClassesToClasses() wrapper function to convert an array of TSQLRecordClass into the expected array of TClass;
  • TPropInfo.CopyValue method now specifically handle copy of TCollection published properties items (used e.g. in TSQLRecord.FillFrom);
  • New GetEnumNameTrimed() wrapper function;
  • New TRecordType definition, and TTypeinfo.RecordType associated method;
  • Now JSONToObject/ObjectToJSON functions and WriteObject method will handle standard TPersistent class serialization into/from JSON object;
  • Now ObjectToJSON/JSONToObject will unserialize sets and enumerations as an array of string, if HumanReadable is set to TRUE;
  • Now TSQLRestServer.Auth service returns true JSON response as specified by its content type (for better AJAX compatibility);
  • Re-declared TSQLAccessRights record as an object, and added some dedicated methods: FromString, ToString, Edit;
  • Faster and more generic TSQLRecord.FillPrepare/FillRow implementation, including enhanced TSQLRecordFill class;
  • Faster TSQLRestServerStaticInMemory.LoadFromJSON and LoadFromBinary methods;
  • ReUrlEncodedSQL remote access right allows execution of SQL statement from a GET with the content encoded on the URI (as from XMLHTTPRequest);
  • New TSQLRest.EngineUpdateField protected method for a field content update (with PUT ModelRoot/TableName?setname=..&set=..&wherename=..&where=..);
  • New TSQLRecord.CreateAndFillPrepareMany and FillPrepareMany methods, able to create a JOINed SELECT statement including all nested TSQLRecordMany properties (including custom WHERE clause if necessary);
  • Now nested TCollection and TStringList/TRawUTF8List objects are transmitted as true JSON arrays or objects for adding (POST) and update (PUT) - this will save bandwidth and increase compatibility with AJAX clients (they were formerly transmitted as JSON strings) - note that retrieval (GET) is not yet implemented, since it is faster to transmit directly the TEXT value as stored within the database;
  • New TSQLRest.MainFieldIDs() method;
  • New ForceID parameter for TSQLRest.Add() and TSQLRestClientURI.BatchAdd() to allow adding a record with a given ID;
  • Added TSQLRestClientURI.OnSetUser notification event (called from SetUser);
  • Now TSQLRestClientURI.BatchUpdate() will set only ID, TModTime and mapped fields when called over a TSQLRecord on which a FillPrepare() was made (and no FillClose was performed);
  • Now TSQLRestServerStats is a plain TPersistent class, and will be sent as a JSON object to the client;
  • Added function IsNotAjaxJSON() function - formerly internal IsExpanded();
  • Added RecordManySourceProp / RecordManyDestProp / RecordManySourceClass / RecordManyDestClass to the TSQLRecordProperties;
  • TSQLRestClientURI.CallBackPut() will now return any HTTP response content (even if it is not HTTP/1.1 compliant, and not work over some networks);
  • Circumvent some bugs of Delphi XE2 background compiler (main compiler is OK).

Unit SQLite3DB.pas

  • TSQLRestServerStaticExternal.EngineList method now handles an optional integer pointer, to return the count of row data (excluding field names).

Unit SQLite3HttpClient.pas

  • Fixed GPF issue at closing;
  • Fixed unnecessary dual URL signing (when authentication actived).

Unit SQLite3HttpServer.pas

  • Added optional aRestAccessRights parameter in TSQLite3HttpServer.AddServer to override the default HTTP_DEFAULT_ACCESS_RIGHTS settings;
  • Added ServerThreadPoolCount parameter to TSQLite3HttpServer.Create() constructor, set by default to 32 - will speed up process of slow requests (e.g. a POST with some huge data transmitted at slow rate);
  • Fixed error in case of URI similar to 'root?session_signature=...';
  • Fixed incorect thread count in TSQLite3HttpServer.Create;
  • Regression tests are now extracted from this unit, in order to allow construction of a TSQLite3HTTPServer instance without the need of linking the SQLite3 engine to the executable.

Unit SQLite3i18n.pas

  • Fixed some compilation warnings with Delphi XE and XE2;
  • Uses new generic TSynAnsiConvert classes for code page process: since TLanguageFile.Create() will set CurrentAnsiConvert global instance, applications should use CurrentAnsiConvert instead of previous TLanguageFile methods;
  • Now Iso2S() - i.e. overriden i18nDateText global function pointer - will handle a date-only or time-only supplied value as expected.

Unit SQLite3Pages.pas

  • Includes new TSynAnsiConvert classes for handling Ansi charsets;
  • Some minor fixes (e.g. preview landscape or keys for popup menu);
  • Fix issue in TGDIPages.AppendRichEdit() when called on a blank page;
  • Enhanced the print preview screen with a left-sided button bar;
  • New TGdiPages.RenderGraphic method (accepting both TBitmap and TMetaFile).

Unit SQLite3SelfTests.pas

  • First public release, corresponding to SQLite3 Framework 1.16;
  • All mORMot tests are now implemented in this separated unit: this is requested by bugs in the Delphi XE2 background compilers: main compiler was OK with our code (i.e. it compiles into .exe and run as expected), but background IDE compilers (used e.g. for syntax checking) was not able to compile the tests within the main .dpr source code.

Unit SQLite3Service.pas

  • Code refactoring after Leander007 proposals for better compatibility - see http://synopse.info/forum/viewtopic.php?id=584.

Unit SQLite3UI.pas

  • New FillStringGrid() function, ready to fill a regular TStringGrid;
  • Includes new TSynAnsiConvert classes for handling Ansi charsets.

Unit SQLite3UIEdit.pas

  • Moved OnComponentCreated event call after all component initialization (allow adding paired components on purpose).

Unit SQLite3UILogin.pas

  • InputBox function will now focus the input field component by default.


Synopse mORMot download and forum

To get it, go to this download page, or use the source...

Do not forget to get and read the full reference documentation available there (mainly the "SAD" - Software Architecture Design - document).

Feedback and questions are welcome in our forum, just as usual.