TDocVariant Pros and Cons
Since years, our TDocVariant
can store any JSON/BSON document-based content, i.e. either:
- Name/value pairs, for object-oriented documents - internally identified as
dvObject
sub-type; - An array of values (including nested documents), for array-oriented documents - internally identified as
dvArray
sub-type; - Any combination of the two, by nesting
TDocVariant
instances.
Every TDocVariant
instance is also a custom variant
type:
- So you can just store or convert it to or from
variant
variables; - You can use late binding to access its object properties, which is some kind of magic in the rigid world of modern pascal;
- The Delphi IDE (and Lazarus 3.x) debuggers have native support of it, so can display its
variant
content as JSON; - If you define
variant
types in any class or record, our framework will recognizeTDocVariant
content and (un)serialize it as JSON, e.g. in its ORM, SOA or Mustache/MVC parts.
Several drawbacks come also from this power:
- Switching between
variant
and itsTDocVariantData
record may be tricky, and it sometimes requires some confusing pointer references; - Each
TDocVariant
instance could be used as a weak reference to other data, or maintain its own content - in some corner cases, incorrect use may leak memory or get some GPF issues; - A
TDocVariant
could be either an object/dictionary or an array/list, so finding the right methods may be difficult, or raise exceptions at runtime; - It evolved from a simple store to a full in-memory engine, so the advanced features are usually underestimated;
- The
TDocVariantData
record is far away from the class system most Delphi/FPC are used to; - By default,
double
values are not parsed - onlycurrency
- which makes sense if you don't want to loose any precision, but has been found confusing.
Enough complains.
Just make it better.
Entering IDocList and IDocDict Interfaces
We introduced two high-level wrapper interface
types:
IDocList
(or its aliasIDocArray
) to store a list of elements;IDocDict
(or its aliasIDocObject
) to store a dictionary of key:value pairs.
The interface
methods and naming follows the usual Python List and Dictionaries, and wrap their own TDocVariant
storage inside safe and class dedicated IDocList
and IDocDict
types.
You may be able to write on modern Delphi:
var list: IDocList; dict: IDocDict; v: variant; i: integer; begin // creating a new list/array from items list := DocList([1, 2, 3, 'four', 1.0594631]); // double are allowed by default // iterating over the list for v in list do Listbox1.Items.Add(v); // convert from variant to string // or a sub-range of the list (with Python-like negative indexes) for i in list.Range(0, -3) do Listbox2.Items.Add(IntToStr(i)); // [1, 2] as integer // search for the existence of some elements assert(list.Exists(2)); assert(list.Exists('four')); // a list of objects, from JSON, with an intruder list := DocList('[{"a":0,"b":20},{"a":1,"b":21},"to be ignored",{"a":2,"b":22}]'); // enumerate all objects/dictionaries, ignoring non-objects elements for dict in list.Objects do begin if dict.Exists('b') then ListBox2.Items.Add(dict['b']); if dict.Get('a', i) then ListBox3.Items.Add(IntToStr(i)); end; // delete one element list.Del(1); assert(list.Json = '[{"a":0,"b":20},"to be ignored",{"a":2,"b":22}]'); // extract one element if list.PopItem(v, 1) then assert(v = 'to be ignored'); // convert to a JSON string Label1.Caption := list.ToString; // display '[{"a":0,"b":20},{"a":2,"b":22}]' end;
and even more advanced features, like sorting, searching, and expression filtering:
var v: variant; f: TDocDictFields; list, list2: IDocList; dict: IDocDict; begin list := DocList('[{"a":10,"b":20},{"a":1,"b":21},{"a":11,"b":20}]'); // sort a list/array by the nested objects field(s) list.SortByKeyValue(['b', 'a']); assert(list.Json = '[{"a":10,"b":20},{"a":11,"b":20},{"a":1,"b":21}]'); // enumerate a list/array with a conditional expression for dict in list.Objects('b<21') do assert(dict.I['b'] < 21); // another enumeration with a variable as conditional expression for dict in list.Objects('a=', 10) do assert(dict.I['a'] = 10); // create a new IDocList from a conditional expression list2 := list.Filter('b =', 20); assert(list2.Json = '[{"a":10,"b":20},{"a":11,"b":20}]'); // direct access to the internal TDocVariantData storage assert(list.Value^.Count = 3); assert(list.Value^.Kind = dvArray); assert(dict.Value^.Kind = dvObject); // TDocVariantData from a variant intermediary v := list.AsVariant; assert(_Safe(v)^.Count = 3); v := dict.AsVariant; assert(_Safe(v)^.Count = 2); // high-level Python-like methods if list.Len > 0 then while list.PopItem(v) do begin assert(list.Count(v) = 0); // count the number of appearances assert(not list.Exists(v)); Listbox1.Items.Add(v.a); // late binding dict := DocDictFrom(v); // transtyping from variant to IDocDict assert(dict.Exists('a') and dict.Exists('b')); // enumerate the key:value elements of this dictionary for f in dict do begin Listbox2.Items.Add(f.Key); Listbox3.Items.Add(f.Value); end; end; // create from any complex "compact" JSON // (note the key names are not "quoted") list := DocList('[{ab:1,cd:{ef:"two"}}]'); // we still have the late binding magic working assert(list[0].ab = 1); assert(list[0].cd.ef = 'two'); // create a dictionary from key:value pairs supplied from code dict := DocDict(['one', 1, 'two', 2, 'three', _Arr([5, 6, 7, 'huit'])]); assert(dict.Len = 3); // one dictionary with 3 elements assert(dict.Json = '{"one":1,"two":2,"three":[5,6,7,"huit"]}'); // convert to JSON with nice formatting (line feeds and spaces) Memo1.Caption := dic.ToString(jsonHumanReadable); // sort by key names dict.Sort; assert(dict.Json = '{"one":1,"three":[5,6,7,"huit"],"two":2}'); // note that it will ensure faster O(log(n)) key lookup after Sort: // (beneficial for performance on objects with a high number of keys) assert(dict['two'] = 2); // default lookup as variant value assert(dict.I['two'] = 2); // explicit conversion to integer end;
Since the high-level instances are interface
and the internal content is variant
, their life time are both safe and usual - and you don't need to write any try..finaly list.Free
code.
And performance is still high, because e.g. a huge JSON array would have a single IDocList
allocated, and all the nested nodes will be hold as efficient dynamic arrays of variants.
Two last one-liners may show how our mORMot library is quite unique in the forest/jungle of JSON libraries for Delphi and FPC:
assert(DocList('[{ab:1,cd:{ef:"two"}}]')[0].cd.ef = 'two');
assert(DocList('[{ab:1,cd:{ef:"two"}}]').First('ab<>0').cd.ef = 'two');
If you compare e.g. to how the standard Delphi JSON library works, with all its per-node classes, you may find quite a difference!
Note that those both lines compile and run with the antique Delphi 7 compiler - who said the pascal language was not expressive, even back in the day?
We hope we succeeded in forging a new way to work with JSON documents, so that you may consider it for your projects on Delphi or FPC.
Any feedback is welcome in our forum, as usual!
BTW, do you know why I picked up this 1.0594631 number in the code?
Hint: this is something I used when I was a kid programming music on a Z80 CPU... and I still remember this constant. :D