To content | To menu | To search

Tag - TDynArray

Entries feed

2014, Friday April 18

Introducing mORMot's architecture and design principles

We have just released a set of slides introducing 

  • ORM, SOA, REST, JSON, MVC, MVVM, SOLID, Mocks/Stubs, Domain-Driven Design concepts with Delphi, 
  • and showing some sample code using our Open Source mORMot framework.

You can follow the public link on Google Drive!

This is a great opportunity to discovers some patterns you may not be familiar with, and find out how mORMot try to implement them.
This set of slides may be less intimidating than our huge documentation - do not be terrified by our Online Documentation!
The first set of pages (presenting architecture and design principles) is worth reading.

Feedback is welcome on our forum, as usual.

2014, Tuesday February 25

TDocVariant custom variant type

With revision 1.18 of the framework, we just introduced two new custom types of variants:

  • TDocVariant kind of variant;
  • TBSONVariant kind of variant.

The second custom type (which handles MongoDB-specific extensions - like ObjectID or other specific types like dates or binary) will be presented later, when dealing with MongoDB support in mORMot, together with the BSON kind of content. BSON / MongoDB support is implemented in the SynMongoDB.pas unit.

We will now focus on TDocVariant itself, which is a generic container of JSON-like objects or arrays.
This custom variant type is implemented in SynCommons.pas unit, so is ready to be used everywhere in your code, even without any link to the mORMot ORM kernel, or MongoDB.

TDocVariant documents

TDocVariant implements a custom variant type which can be used to store any JSON/BSON document-based content, i.e. either:

  • Name/value pairs, for object-oriented documents;
  • An array of values (including nested documents), for array-oriented documents;
  • Any combination of the two, by nesting TDocVariant instances.

Here are the main features of this custom variant type:

  • DOM approach of any object or array documents;
  • Perfect storage for dynamic value-objects content, with a schema-less approach (as you may be used to in scripting languages like Python or JavaScript);
  • Allow nested documents, with no depth limitation but the available memory;
  • Assignment can be either per-value (default, safest but slower when containing a lot of nested data), or per-reference (immediate reference-counted assignment);
  • Very fast JSON serialization / un-serialization with support of MongoDB-like extended syntax;
  • Access to properties in code, via late-binding (including almost no speed penalty due to our VCL hack as already detailed);
  • Direct access to the internal variant names and values arrays from code, by trans-typing into a TDocVariantData record;
  • Instance life-time is managed by the compiler (like any other variant type), without the need to use interfaces or explicit try..finally blocks;
  • Optimized to use as little memory and CPU resource as possible (in contrast to most other libraries, it does not allocate one class instance per node, but rely on pre-allocated arrays);
  • Opened to extension of any content storage - for instance, it will perfectly integrate with BSON serialization and custom MongoDB types (ObjectID, RegEx...), to be used in conjunction with MongoDB servers;
  • Perfectly integrated with our Dynamic array wrapper and its JSON serialization as with the record serialization;
  • Designed to work with our mORMot ORM: any TSQLRecord instance containing such variant custom types as published properties will be recognized by the ORM core, and work as expected with any database back-end (storing the content as JSON in a TEXT column);
  • Designed to work with our mORMot SOA: any interface-based service is able to consume or publish such kind of content, as variant kind of parameters;
  • Fully integrated with the Delphi IDE: any variant instance will be displayed as JSON in the IDE debugger, making it very convenient to work with.

To create instances of such variant, you can use some easy-to-remember functions:

  • _Obj() _ObjFast() global functions to create a variant object document;
  • _Arr() _ArrFast() global functions to create a variant array document;
  • _Json() _JsonFast() _JsonFmt() _JsonFastFmt() global functions to create any variant object or array document from JSON, supplied either with standard or MongoDB-extended syntax.

Continue reading...

2012, Thursday May 3

Custom JSON serialization of records

By default, during interface-based service call, any record parameter or function result will be serialized with our proprietary binary (and optimized layout) - i.e. RecordLoad and RecordSave functions - then encoded in Base-64, to be stored as plain text within the JSON stream.

But custom record JSON serialization can be defined, as with any class - see this article - or dynamic array - see this article.

Continue reading...

2012, Thursday April 12

Custom JSON serialization of any dynamic array content

One nice feature of the TDynArray wrapper and its associated methods, as defined and used in our framework, is the ability to serialize any dynamic array as JSON content.

By default, only "standard" dynamic arrays (like TIntegerDynArray) are serialized as true JSON array: other not known kind of arrays are serialized as binary content, within a Base64 encoding.

This is a very efficient solution for a pure Delphi application, since it will be fast and always works, but won't be easy to deal with from an AJAX client.

Applications can now supply a custom JSON serialization for any other dynamic array, via the TTextWriter.RegisterCustomJSONSerializer() class method.
Two callbacks are to be supplied for a dynamic array type information, in order to handle proper serialization and un-serialization of the JSON array.

Continue reading...

2011, Thursday December 8

Avoiding Garbage Collector: Delphi and Apple side by side

Among all trolling subject in forums, you'll find out the great Garbage Collection theme.

Fashion languages rely on it. At the core of the .Net and Java framework, and all scripting languages (like JavaScript, Perl, Python or Ruby), you'll find a Garbage Collector. New developers, just released from schools, do learn about handling memory only in theory, and just can't understand how is memory allocated - we all have seen such rookies involved in Delphi code maintenance, leaking memory as much as they type. In fact, most of them did not understood how a computer works. I warned you this will be a trolling subject.

And, in Delphi, there is no such collector. We handle memory in several ways:

  • Creating static variables - e.g. on the stack, inside a class or globally;
  • Creating objects with class instances allocated on heap - in at least three ways: with a try..finally Free block, with a TComponent ownership model in the VCL, or by using an interface (which creates an hidden try..finally Free block);
  • Creating reference-counted variables, i.e. string, array of, interface or variant kind of variables.

It is a bit complex, but it is also deadly powerful. You have several memory allocation models at hand, which can be very handy if you want to tune your performance and let program scale. Just like manual recycling at home will save the planet. Some programmers will tell you that it's a waste of cell brain, typing and time. Linux kernel gurus would not say so, I'm afraid.

Then came the big Apple company, which presented its new ARC model (introduced in Mac OS X 10.7 Lion) as a huge benefit for Objective-C in comparison with the Garbage Collection model. And let's face it: this ARC just sounds like the Delphi memory model.

Continue reading...

2011, Saturday July 2

Is Object-Relational Mapping the Paradise of Computer Science?

There is a well known syndrome around, against ORM.

Do you remember The Vietnam of Computer Science article?

It is worth reading... and commenting.
Sounds a bit outdated by now. Tempus fugit!

Continue reading...

2011, Friday July 1

SynOleDB: OpenSource Unit for direct access to any database via OleDB

That's it, our SynOleDB unit seems alive and running well.

OLE DB (Object Linking and Embedding, Database, sometimes written as OLEDB or OLE-DB) is an API designed by Microsoft for accessing data from a variety of sources in a uniform manner. It was designed as a higher-level replacement for, and successor to, ODBC, extending its feature set to support a wider variety of non-relational databases, such as object databases and spreadsheets that do not necessarily implement SQL.

SynOleDB unit implementation has been made with several points in mind:

  • Tested with SQL Server 2008 R2 and Oracle 11g providers from Microsoft and Oracle; 
  • Ability to be truly Unicode, even with pre-Unicode version of Delphi (like Delphi 7 or 2007); 
  • Could access any local or remote Database, from any version of Delphi, since it doesn't use the DB.pas unit or any related part of the VCL (even the Delphi 7 personal or the Turbo Explorer editions), just for free; 
  • Handle NULL or BLOB content for parameters and results; 
  • Avoid most memory copy or unnecessary allocation: we tried to access the data directly from the retrieved data buffer, just as given from OleDB; 
  • Was therefore designed to achieve the best performance possible: most time is spent in OleDB: the code layer added to the OleDB customer is very thin; 
  • True OOP architecture, to be used with any OleDB provider (allowing custom parameters or such), and even without OleDB (in the future, direct access to any DB client could be used); 
  • Could be safely used in a multi-threaded application/server (with one TOleDBConnection per thread); 
  • Allow parameter bindings of requests, with fast access to any parameter or column name (thanks to TDynArrayHashed);
  • Late binding of column values in Delphi code;
  • Direct JSON content creation, with no temporary data copy nor allocation; 
  • Designed to be used with our mORMot ORM, but could be used stand-alone (a full Delphi 7 client executable is just about 200 KB), or even in any existing Delphi application, thanks to a TQuery-like wrapper.

Continue reading...

2011, Saturday March 12

TDynArray and Record compare/load/save using fast RTTI

The SynCommons unit has been enhanced:
- new BinToBase64 and Base64ToBin conversion functions;
- new low-level RTTI functions for handling record types: RecordEquals, RecordSave, RecordSaveLength, RecordLoad;
- new TDynArray object, which is a wrapper around any dynamic array.

With TDynArray, you can access any dynamic array (like TIntegerDynArray = array of integer) using TList-like properties and methods, e.g. Count, Add, Insert, Delete, Clear, IndexOf, Find, Sort and some new methods like LoadFromStream, SaveToStream, LoadFrom and SaveTo which allow fast binary serialization of any dynamic array, even containing strings or records - a CreateOrderedIndex method is also available to create individual index according to the dynamic array content. You can also serialize the array content into JSON, if you wish.

What I like with dynamic arrays is that they are reference-counted, don't need any Create/try..finally...Free code, and are well handled by the Delphi compiler.
They are no replacement to a TCollection nor a TList (which are the standard and efficient way of storing class instances), but they are very handy way of having a list of content or a dictionary at hand, with no class nor properties definition.
You can look at them like Python's list, tuples (via records handling) and dictionaries (via Find method), in pure Delphi. Our new methods (about searching and serialization) allow most usage of those script-level structures in your Delphi code.

Continue reading...