Go down to the download and forum links.

Synopse mORMot 1.17 fixes and enhancements

This is a per-unit list of changes for the 1.17 release of mORMot:

Changes in details:

Unit SynCommons.pas

  • Check of QueryPerformanceFrequency failure, and rollback to low-resolution timer;
  • Handle properly old .synlz layout (reading compatibility was broken);
  • Added TObjectListHashed class, which behaves like TList/TObjectList, but will use hashing for (much) faster IndexOf() method, and associated TObjectListPropertyHashed class, which allows hashing of a sub-property of an object (including some changes made to TDynArray/TDynArrayHashed);
  • New TTextWriter.AddDateTime() overloaded method able to quote the output;
  • New TTextWriter.AddFloatStr() method handling partial floating-point text;
  • Both TTextWriter.AddDateTime() overloaded methods will store '' when value is 0, or a pure ISO-8601 date or time if the value is defined as such, just as expected by http://www.sqlite.org/lang_datefunc.html - it will also reduce average generated JSON/text content size;
  • Fixed issue about BLOB unproperly serialized into JSON (e.g. now uses null);
  • Fixed ticket [e5ad3684b2] about some .map files parsing in TSynMapFile;
  • TSynLog stack tracing uses low-level RtlCaptureStackBackTrace() API on CPU64;
  • Changed the non expanded JSON format to use lowercase first column names: {"fieldCount":1,"values":["col1"... instead of {"FieldCount":1,"Values":[..;
  • New SetInt64() procedure for direct assignment of the result;
  • TSynTableStatement class now accepts '_' in table and column identifiers;
  • Fixed implementation issue in function FindNextUTF8WordBegin();
  • Fixed false negative issue in TSynSoundEx.UTF8 and TSynSoundEx.Ansi;
  • Fixed wrong UTF-8 encoding of U+FFF0 used for JSON_BASE64_MAGIC;
  • Added an optional parameter to StrToCurr64() function, able to return a true Int64 value if no decimal is supplied within the input text buffer;
  • Enhanced TSynAnsiFixedWidth.UnicodeBufferToAnsi average process speed;
  • TSynCache.Reset now returns a boolean stating if something was flushed;
  • New SynUnicodeToUtf8(), StringToSynUnicode(), SynUnicodeToString() functions;
  • New StrToCurrency() wrapper function;
  • New UTF8UpperCopy() and UTF8UpperCopy255() optimized functions;
  • New GotoNextNotSpace() and GotoEndOfQuotedString() functions;
  • New TMemoryMap.Map() method expecting a file name as parameter;
  • New TRawUTF8List.LoadFromFile() method;
  • New DateToSQL(), DateTimeToSQL() and Iso8601ToSQL() functions, returning a string with a JSON_SQLDATE_MAGIC prefix and proper UTF-8/ISO-8601 encoding to be inlined as ? bound parameter in any SQL query (allow binding of date/time parameters as request by some external database engine which does not accept ISO-8601 text in this case);
  • Added TDynArray.Equals() method to compare two arrays efficiently;
  • Added TDynArray and TDynArrayHashed InitSpecific() method able to specify how (hashing and) comparison should be processed for a given record (includes also some TDynArray/TDynArrayHashed refactoring and optimization);
  • New TObjectHash abstract class to use hashing to find an object;
  • TTextWriter.AddJSONEscape() method speed up;
  • Moved logging threadvars and associated structures into hidden internal declaration, for better work with packages (avoid W1032 warning);
  • Now JSON parser will handle #1..' ' chars as whitespace (not only ' ');
  • Now JSON parser will allow whitespace inserted between any pair of tokens, even after true/false/null, as expected by the specification;
  • Fixed potential Integer Overflow error in Iso8601ToDateTimePUTF8Char*();
  • Added PatchCode() and RedirectCodeRestore() procedures, and some code refactoring about process in-memory code patching;
  • Internal FillChar() will now use faster SSE2 instructions on supported CPUs.

Unit SynCrtSock.pas

  • Replaced TSockData string type to the generic RawByteString type (and the default AnsiString for non-Unicode version of Delphi);
  • Added optional aProxyName, aProxyByPass parameters to TWinHttpAPI / TWinInet and TWinHTTP constructors;
  • Added THttpServerGeneric.OnHttpThreadStart property, and associated TNotifyThreadEvent event prototype;
  • Handle 'Range: bytes=***-***' request in THttpApiServer.

Unit SynDB.pas

  • Code refactoring to allow direct ODBC connection implementation;
  • Fixed random issue in TSQLDBConnection.GetServerTimeStamp method (using wrongly TTimeLog direct arithmetic, therefore raising EncodeTime() errors);
  • Fixed issue about creating unexisting NCLOB instead of CLOB/NCLOB;
  • Fixed TQuery implementation to match the expected original behavior (e.g. SQL.Clear) - also undefined buggy Last method (use ORDER DESC instead);
  • Fixed issue in TQuery when executing requests with parameters;
  • Fixed issues in TQuery when translated SQL from named parameters to positioned (?) parameters, and escaping strings;
  • Enhanced MySQL DBMS back-end compatibility;
  • TQuery will now accept reused parameters in the SQL statement (just like the original class);
  • Added TQueryValue.AsLargeInt property alias for better compatibility;
  • Enhanced TSQLDBStatement.BindVariant() to handle varBoolean value as integer, and to avoid most temporary conversions to string;
  • Enhanced TSQLDBStatement.Bind(Params: TVarDataDynArray) to handle varDate, and modified TQueryValue in consequence;
  • Enhanced TSQLDBStatement.Bind(const Params: array of const) to accept BLOB content, when transmitted after BinToBase64WithMagic() conversion, and TDateTime parameters via Date[Time]ToSQL() encoding;
  • Declared TSQLDBConnectionProperties.GetMainConnection() method as virtual, then override it for thread-safe connections - see ticket [65e24b2de4];
  • Now TSQLDBStatement.ColumnToVarData method will store '' when TDateTime value is 0, or a pure date or a pure time if the value is defined as such, just as expected by http://www.sqlite.org/lang_datefunc.html - i.e. SQLite3DB;
  • Added FieldSize optional parameter to TSQLDBStatement.ColumnType() method (used e.g. by SynDBVCL to provide the expected field size on TDataSet);
  • Added TSQLDBStatement.ColumnBlobBytes() methods to retrieve TBytes BLOBs;
  • Added TSQLDBConnection.InTransaction property;
  • Added TSQLDBConnectionProperties.EngineName property;
  • Added TSQLDBConnectionProperties.DBMS property, and huge code refactoring among all SynDB* units for generic handling of DBMS-specific properties;
  • Added TSQLDBConnectionProperties.AdaptSQLLimitForEngineList for handling the LIMIT # statement in a database-agnostic form;
  • Added TSQLDBConnectionProperties.BatchSendingAbilities property to define the CRUD modes available in batch sending (see e.g. Oracle's array bind, or MS SQL bulk insert feature);
  • Added direct access to the columns description via new property TSQLDBStatementWithParamsAndColumns.Columns;
  • Added TSQLDBColumnProperty.ColumnUnique property (mainly for TSQLDBConnectionProperties.SQLFieldCreate to create proper SQL);
  • New TSQLDBStatement.BindArray*() methods, introducing array binding for faster database batch modifications (only implemented in SynDBOracle by now).

Unit SynDBODBC.pas

  • Initial working code, tested with ODBC Oracle provider.

Unit SynDBOracle.pas

  • Now NLS_LANG environnement variable or current thread local will be used by TSQLDBOracleConnectionProperties.Create if supplied code page parameter is 0;
  • Now TSQLDBOracleConnectionProperties.Create constructor can be used directly;
  • Previous TSQLDBOracleConnectionProperties reintroduced constructor renamed CreateWithCodePage, to specify the code page to be used;
  • Fixed issue with SQL requests containing '' and parameters;
  • Fixed issue with SQLT_BIN/RAW kind of columns;
  • Fixed issue with SQLT_IBFLOAT/SQLT_IBDOUBLE kind of columns;
  • Fixed issue with CLOB/NCLOB/NVARCHAR kind of columns (including charset);
  • Added TSQLDBOracleStatement.ColumnToVarData() overriden method, for faster process from mORMot external tables (i.e. SQLite3DB);
  • Fixed issue with TDateTime parameter (SQLT_DAT replaced by SQLT_TIMESTAMP);
  • Fixed issue with NULL parameter (avoid ORA-01024 error);
  • Fixed random issue when converting a SQLT_DATE result column value;
  • Force ftCurrency content to be returned with '.' as decimal separator (as expected by JSON), even with any other NLS_LANG encoding;
  • Enhance OUT and IN/OUT bound parameters handling (for stored procedures), with dedicated exceptions for any unhandled kind of parameter;
  • Generated logging level reduced to the necessary (but with timing);
  • Added TSQLDBOracleStatement.ColumnBlobBytes method to retrieve TBytes BLOBs;
  • Speed up of TSQLDBOracleStatement.ColumnInt() and Prepare() methods;
  • Fixed issue in TSQLDBOracleStatement.ColumnToVarData() when retrieving date;
  • Fixed issue in TSQLDBOracleStatement.Prepare() for some kinds of columns;
  • Fixed issue in TSQLDBOracleStatement.ExecutePrepared at SQLT_LVC binding;
  • Fixed BLOB memory leak in TSQLDBOracleStatement.FreeHandles time fields from Oracle as external virtual tables;
  • Fixed issue when NLS_DATE_FORMAT is not ISO-8601: since mORMot's ORM will send date/time as ISO-8601 text, we now force proper NLS_DATE_FORMAT;
  • TOracleDate will now handle Cent=Year=Month=Day=Hour=Main=Sec=0 as TDateTime(0);
  • New TOracleDate.From/ToIso8601 methods for direct SQLT_DAT handling;
  • Code refactoring, especially about error handling and ODBC integration.

Unit SynDBSQLite3.pas

  • Now allow compilation with Delphi 5;
  • Now TSQLDBSQLite3Statement.BindDateTime() will store '' when value is 0, or a pure date or a pure time if the value is defined as such - by the way, it will match SQlite3 expectations of internal date/time functions, as defined at http://www.sqlite.org/lang_datefunc.html;
  • Fixed TSQLDBSQLite3Statement.Step to update CurrentRow and TotalRowsRetrieved properties as expected;
  • Added TSQLDBSQLite3Connection.Synchronous property;
  • Code refactoring, especially about error handling and ODBC integration.

Unit SynDBVCL.pas

  • First public release, corresponding to Synopse mORMOT Framework 1.17.

Unit SynGdiPlus.pas

  • New TGDIPlusFull.ForceUseDrawString property for properly handling font fall-back if needed when drawing text (disabled by default), and corresponding parameter in DrawEmfGdip() function;
  • Added BitmapSetResolution optional parameter for SaveAs() functions to specify the destination image DPI.

Unit SynLZ.pas

  • Use RawByteString type for CompressSynLZ() function prototype.

Unit SynOleDB.pas

  • Fix issue (depending on OleDB Provider) about ordinal binding error;
  • Fix issue about void string parameter binding;
  • Fix issue in TOleDBStatement.BindCurrency() method;
  • Fix retrieval of table and field metadata, for tables without schema (e.g. Jet/MSAccess database);
  • Added TOleDBJetConnectionProperties kind of connection to direct access of Microsoft Jet databases (.mdb files);
  • Added corresponding IsJetFile() function;
  • Added FieldSize optional parameter to TOleDBStatement.ColumnType() method (used e.g. by SynDBVCL to provide the expected field size on TDataSet);
  • Added TOleDBConnectionProperties.CreateDatabase able to create a database from the supplied connection string (used e.g. to initialize .mdb files);
  • Code refactoring, especially about error handling and ODBC integration;
  • CoInit and CoUninit made public for user convenience, e.g. when using custom COM objects in a mORMot multi-thread servers.

Unit SynPdf.pas

  • New TPdfDocument.UseFontFallBack property (enabled by default) and associated FontFallBackName property (set to 'Arial Unicode MS' by default), used to define if the PDF document will handle "font fallback" for characters not existing in the current font: it will avoid rendering block/square symbols instead of the correct characters (e.g. for Chinese text);
  • Now handle device or bitmap fonts as the most close true-type font available;
  • Speed-up of internal true-type fonts list (using binary search);
  • SynPdf unit can now link to standard ZLib.pas unit if you want to use SynPdf stand-alone and do not need SynZip.pas + deflate.obj + trees.obj (but SQLite3Commons.pas main unit of mORMot will need SynZip, so it is enabled by default for use within the framework).

Unit SynSelfTests.pas

  • Added test for TInterfaceCollection kind of parameter;
  • Added multi-thread testing of ExecuteInMainThread() method;
  • Removed TSQLRecordExternal class type, to allow any TSQLRecord (e.g. TSQLRecordMany) to be used with VirtualTableExternalRegister();
  • Added DBMS full test coverage in TTestExternalDatabase.AutoAdaptSQL }.

Unit SynSQLite3.pas

  • Updated SQLite3 engine to version 3.7.14;
  • Allow compilation with Delphi 5;
  • Added TSQLDataBase.CacheFlush method (needed by SQLite3DB);
  • Added TSQLDataBase.Synchronous and TSQLDataBase.WALMode properties;
  • Added TSQLDataBase.ExecuteNoException() overloaded methods;
  • Fixed ticket [8dc4d49ea9] in TSQLDataBase.GetFieldNames()about result array truncated to 64.

Unit SQLite3.pas

  • Updated SQLite3 engine to version 3.7.14;
  • Added overriden TSQLRestServerDB.FlushInternalDBCache method;
  • Added TSQLRestServerDB.BackupGZ method for live database backup into a compressed .gz archive file.

Unit SQLite3Commons.pas

  • TSQLRecord.Create/FillPrepare/CreateAndFillPrepare and TSQLRest.OneFieldValue/MultiFieldValues methods signature BREAKING CHANGE: array of const used to be ParamsSQLWhere and expecting '%' in the FormatSQLWhere statement, is now called BoundsSQLWhere, and expects bound parameters specified as '?' in the FormatSQLWhere statement - this is less confusing for new users, and more close to the usual way of preparing database queries; but your existing user code SHALL BE CHECKED and fixed;
  • Fixed issue in TSQLTable.GetJSONValues about JSON number encoding;
  • Added optional "rowCount": in TSQLRestServerStaticInMemory.GetJSONValues, TSQLTable.GetJSONValues and in TSQLTableJSON.ParseAndConvert for about 5% faster process of huge content (mpv proposal);
  • Fixed issue about BLOB unproperly serialized into JSON (e.g. now uses null);
  • Fixed issue about harcoded 'ID' column, not compatible with virtual tables;
  • Fixed issue about pessimistic TSQLRestServerStaticInMemory.fIDSorted value;
  • Fixed random issue in TSQLRest.GetServerTimeStamp method (using wrongly TTimeLog direct arithmetic, therefore raising EncodeTime() errors);
  • Fixed ticket [fdf7158601] - about incorrect null value parsing in JSONToObject when isObj = oCustom (fix by mpv - thanks!);
  • Fixed ticket [a1d9e9148e] - about incorrect reading empty JSON object by JSONToObject (fix by mpv - thanks!);
  • Fixed ticket [4f5df7f18f] - about potential overflow of TSQLRestServerStats values (changed to QWord kind of property);
  • Implemented feature request [7f6828999d] - about the possibility to use standard read/write in conjunction with custom read/write in JSONToObject and ObjectToJSON (mpv proposal - thanks!);
  • JSONToObject is now able to un-serialize a TObjectList class, when a class for its items is supplied as TObjectListItemClass optional parameter;
  • ExtractInlineParameters() and SQLParamContent() decode ':("\uFFF12012-05-04"):' inlined parameters (i.e. text starting with JSON_SQLDATE_MAGIC after UTF-8 encoding) as sftDateTime kind of parameter;
  • Internal cache added in TSQLRest.GetServerTimeStamp method for better speed;
  • Added TSQLRest.Retrieve() overloaded method for easy parameter binding;
  • Added TSQLRest.Delete() overloaded method with a WHERE clause parameter;
  • Implemented transaction process for (external database) virtual tables;
  • Added ReplaceRowIDWithID optional parameter to GetJSONObjectAsSQL(), in order to allow working with external DB not allowing RowID (e.g. Oracle);
  • TSQLRestCache.Notify*() methods made public for low-level potential use;
  • Made URI check case-insensitive (as for official RFC);
  • New TPropInfo.GetHash and TPropInfo.SameValue methods, with optional case sentivity handling;
  • Changed TSQLRecordProperties.BlobFieldsBits property into BlobFields, as an array of PPropInfo (for faster process);
  • Added TSQLRecordProperties.HasTypeFields containing set of field types appearing in the record - replaces HasModTimeFields and HasCreateTimeField;
  • New TListFieldHash class for efficient O(1) search using hashing handling;
  • Now unique fields are hashed in TSQLRestServerStaticInMemory implementation: "stored: false" properties are now checked for unicity before adding or update, and search will use the hash table for very fast O(1) process;
  • Speed optimization: all TSQLRestServerStaticInMemory search methods will now call a generic FindWhereEqual() for better code speed and maintenance;
  • Added ObjectFromInterface() function working also with TInterfacedObjectFake;
  • Introducing SetWeak() function to handle Weak interface assignment;
  • Added SetWeakZero() function and TObject class helper to handle ZEROed Weak interface assignment (with small performance penalty and memory use), corresponding to the ARC's Zeroing Weak pointers model;
  • CopyObject() procedure now handle TCollection kind of object not only as sub properties;
  • Introducing TInterfacedCollection dedicated class, properly handling collection item creation on the Server side, with interface-based services: all contract operations shall use it instead of TCollection;
  • Changed the non expanded JSON format to use lowercase first column names: {"fieldCount":1,"values":["col1"... instead of {"FieldCount":1,"Values":[..;
  • Added TSQLTable.FieldLengthMax() and ExpandAsSynUnicode() methods;
  • Added BlobToBytes() function and TSQLTable.GetBytes/GetStream methods;
  • Added virtual TSQLRestServer.FlushInternalDBCache method and dedicated TSQLRestServerStaticInMemoryExternal class, to properly handle external DB modification for virtual tables (i.e. flush SQL/JSON cache as expected);
  • Added virtual TSQLRestServer.BeginCurrentThread method;
  • Added virtual TSQLRestServer.EndCurrentThread method which will be called e.g. by TSQLite3HttpServer or TSQLRestServerNamedPipeResponse for each terminating threads, to release any thread-specific resources (like external DB connections defined in SQlite3DB);
  • Added new TServiceMethod.ExecutionOptions member, and the new TServiceMethodExecutionOption[s] types - used by ExecuteInMainThread();
  • Added TServiceFactoryServer.ExecuteInMainThread() method, to force a method to be executed with RunningThread.Synchronize() call on multi-thread server instances (e.g. TSQLite3HttpServer or TSQLRestServerNamedPipeResponse);
  • Refactoring of TServiceMethod.InternalExecute low-level asm code, and changed the Instance parameter to be specified as an open array for fast execution over multiple instances of implementation classes;
  • 'SELECT *' statements on virtual/external tables will by-pass the SQLite3 virtual table module: TSQLRecord.FillPrepare can be up to 30% faster - added TSQLRestServerStatic.AdaptSQLForEngineList virtual method to handle most generic SELECT cases (overriden e.g. in SQLite3DB unit);
  • TSQLRestServerStaticInMemory.GetJSONValues will now generate expanded JSON content, if specified (only non-expanded format was implemented), via the new TSQLRestServerStaticInMemory.ExpandedJSON property;
  • Added TSQLRestServerStatic.InternalBatchStart / InternalBatchStop methods to handle fast grouped sending to remote database engine (e.g. Oracle bound arrays or MS SQL bulk insert via SynDB);
  • Fixed issue in TSQLRestClientURI.EngineAdd() when server returned -1;
  • Changed TSQLRestServerCallBackParams content to be used as a generic parameters wrapper for both method callbacks and interface-based services: now aParams.Context.ID is to be used insted of aParams.ID;
  • Added TJSONObjectDecoder record/object helper for JSON object decoding (used e.g. by GetJSONObjectAsSQL() function, and for SQlite3DB process);
  • Removed TSQLRecordExternal class type, to allow any TSQLRecord (e.g. TSQLRecordMany) to be used with VirtualTableExternalRegister() - there was indeed no implementation requirement to force a specific class type;
  • Added aUseBatchMode optional parameter to TSQLRecordMany.ManyDelete() method;
  • Now JSON parser will handle #1..' ' chars as whitespace (not only ' ');
  • Now huge service JSON response is truncated (to default 20 KB) in logs.

Unit SQLite3DB.pas

  • Changed column named 'RowID' into 'ID' since it is reserved e.g. in Oracle;
  • External direct insert, update or delete actions (i.e. when the TSQLRestServerStaticExternal instance is called directly) will now flush the low-level SQLite3 DB cache, as expected by the virtual tables;
  • Added TSQLRestServerStaticExternal.AdaptSQLForEngineList overriden method to handle most generic SELECT to by-pass the SQLite3 virtual module for speed;
  • Added TSQLRestServerStaticExternal.EndCurrentThread overriden method which will be called e.g. by TSQLite3HttpServer or TSQLRestServerNamedPipeResponse for each terminating threads, to release external connection resource (calling TSQLDBConnectionPropertiesThreadSafe.EndCurrentThread method);
  • Any direct or virtual-table based insertion to the external database will now use a binding matching the exact time of each column: it will e.g. allow to support DBMS which does not accept date/time to be supplied as ISO-8601 text, and make more efficient data conversion (like avoid conversion to floating-point from a currency value) - code shared with BATCH mode and newly added TSQLRestServerStaticExternal.ExecuteFromJSON() protected method;
  • Inlined parameters in any SQL query will bind explicitely TDateTime values if the parameter is transmitted as DateToSQL() or DateTimeToSQL() TEXT;
  • Removed TSQLRecordExternal class type, to allow any TSQLRecord (e.g. TSQLRecordMany) to be used with VirtualTableExternalRegister() - there was indeed no implementation requirement to force a specific class type;
  • Now create properly UNIQUE fields (i.e. "stored false") in external tables;
  • Handle NULL values for BLOBs as expected.

Unit SQLite3HttpClient.pas

  • Added optional aProxyName, aProxyByPass parameters to TSQLite3HttpClientWinGeneric / TSQLite3HttpClientWinINet and TSQLite3HttpClientWinHTTP constructors.

Unit SQLite3HttpServer.pas

  • Made URI check case-insensitive (as for official RFC);
  • TSQLite3HttpServer will now call virtual TSQLRestServer.EndCurrentThread method in each of its terminating threads, to release any thread-specific resources (for instance, external connections in SQlite3DB).

Unit SQLite3i18n.pas

  • Some refactoring about process in-memory code patching.

Unit SQLite3Pages.pas

  • Now whole text process is UNICODE-ready, even on pre-Delphi-2009 versions;
  • Now implements font fall-back in internal Anti-Aliaised drawing, if the new ForceInternalAntiAliasedFontFallBack property is set to TRUE.

Unit SQLite3SelfTests.pas

  • Fixed LVCL and Delphi 5 compilation issues.

Unit SQLite3UI.pas

  • Global functions AddApplicationToFirewall() and AddPortToFirewall() are now compatible with Windows XP, Vista and Seven - renamed on purpose.

Unit SQLite3VCL.pas

  • First public release, corresponding to Synopse mORMot Framework 1.17.


Synopse mORMot download and forum

To get it, go to this download page, or use the source...
Do not forget to get and read the full reference documentation available there (mainly the "SAD" - Software Architecture Design - document).

Feedback and questions are welcome in our forum, just as usual.