Abstract
The main purpose of this modification was made after a question posted
in How
to store dynamic arrays in a TList?
I also would like to handle dynamic arrays in our ORM, but we didn't have
enough RTTI-based structure for this yet.
I've created a wrapper around dynamic array RTTI, in order to
provide TList
-like properties, and even more features.
Since I wanted to handle dynamic array of records, I also created some
low-level fast access to the record content, using the common RTTI, which a
much faster than the "new" enhanced RTTI available since Delphi 2010.
1. TList-like properties
Here is how you can have method-driven access to the dynamic array:
type
TGroup: array of integer;
var
Group: TGroup;
GroupA: TDynArray;
i, v: integer;
begin
GroupA.Init(TypeInfo(TGroup),Group); // associate GroupA with Group
for i := 0 to 1000 do begin
v := i+1000; // need argument passed as a const variable
GroupA.Add(v);
end;
v := 1500;
if GroupA.IndexOf(v)<0 then // search by content
ShowMessage('Error: 1500 not found!');
for i := GroupA.Count-1 downto 0 do
if i and 3=0 then
GroupA.Delete(i); // delete integer at index i
end;
This TDynArray
wrapper will work also with array of string or
array of records...
Records need only to be packed and have only not reference counted fields
(byte, integer, double
...) or string
reference-counted fields (no Variant
nor Interface
within).
Yes, you read well: it will handle a dynamic array of records, in which you can put some strings or whatever data you need.
The IndexOf()
method will search by content. That is e.g. for
an array of record
, all record fields content (including strings)
must match.
Note that TDynArray
is just a wrapper around an existing
dynamic array. In the code above, Add
and Delete
methods are modifying the content of the Group
variable.
You can therefore initialize a TDynArray
wrapper on need, to
access more efficiently any native Delphi dynamic array.
TDynArray
doesn't contain any data: the elements are stored in the
dynamic array.
2. Enhanced features
I've added some methods to the TDynArray
record/object, which
are not available in a plain TList
- with those methods, we come
closer to some native generics implementation:
- now you can save and load a dynamic array content to or from a stream or a
string (using LoadFromStream/SaveToStream
or
LoadFrom/SaveTo
methods) - it will use a proprietary but very fast
binary stream layout;
- and you can sort the dynamic array content by two means: either in-place
(i.e. the array elements content is exchanged - use the Sort
method in this case) or via an external integer index look-up array (using the
CreateOrderedIndex
method - in this case, you can have several
orders to the same data);
- you can specify any custom comparison function, and there is a new
Find
method will can use fast binary search if available.
Here is how those new methods work:
var
Test: RawByteString;
...
Test := GroupA.SaveTo;
GroupA.Clear;
GroupA.LoadFrom(Test);
GroupA.Compare := SortDynArrayInteger;
GroupA.Sort;
for i := 1 to GroupA.Count-1 do
if Group[i]<Group[i-1] then
ShowMessage('Error: unsorted!');
v := 1500;
if GroupA.Find(v)<0 then // fast binary search
ShowMessage('Error: 1500 not found!');
Some unique methods like Slice, Reverse
or
AddArray
are also available, and mimic well-known Python
methods.
Still closer to the generic paradigm, working for Delphi 6 up to XE, without the need of the slow enhanced RTTI...
3. Capacity handling via an external Count
One common speed issue with the TDynArray
is that the internal
memory buffer is reallocated when you change its length.
That is, whenever you call Add
or Delete
methods,
an internal call to SetLength(DynArray)
is performed. This could
be slow, because it always executes some extra code, including a call to
ReallocMem
.
In order not to suffer for this, you can define an external
Count
value, as an Integer
variable.
In this case, the Length(DynArray)
will be the memory capacity of
the dynamic array, and the exact number of stored item will be available from
this Count
variable. A Count
property is exposed by
TDynArray
, and will always reflect the number of items stored in
the dynamic array. It will point either to the external Count
variable, if defined; or it will reflect the Length(DynArray)
,
just as usual.
As a result, adding or deleting items could be much faster.
4. JSON serialization
Take a look at the JSON serialization features:
TTextWriter.AddDynArrayJSON
and
TDynArray.LoadFromJSON
methods are available for UTF-8 JSON
serialization of dynamic arrays.
Most common kind of dynamic arrays (array of byte, word, integer,
cardinal, Int64, double, currency, RawUTF8, WinAnsiString, string
) will
be serialized as a list of JSON elements.
Other not-known dynamic arrays (like any array of packed record
)
will be serialized as binary, then Base64 encoded.
If you have any ideas of standard dynamic arrays which should be handled, feel
free to post your proposal in the forum!
These methods were the purpose of adding in SynCommons both
BinToBase64
and Base64ToBin
functions, very optimized
for speed. In fact, we will use the Base64 encoding to load or save
any dynamic array of records from a TSQLRecord
published
property... using SQLite3 blob fields for storage, but Base64
for JSON transmission (much more efficient than hexa).
This JSON serialization will indeed be used in our main ORM to support dynamic
arrays as enhanced properties (stored as blob).
5. One step further
The TTestLowLevelCommon._TDynArray
method implements the
automated unitary tests associated with this wrapper.
You'll find out there samples of array of records and more advanced features, with various kind of data.
Feedback and comments, full interface description of the TDynArray
type are available in
our forum.