You have the number of iterations per second, and the peak memory used during each process.
JSON benchmarking
-------------------
1. Small content
1.1. Synopse record:
- Read: 25,000 assertions passed 70.67ms 353,721/s
- Access: 50,000 assertions passed 493us 50,709,939/s
- Write: 25,000 assertions passed 49.25ms 507,562/s
Total failed: 0 / 100,000 - Synopse record PASSED 121.03ms
1.2. Synopse variant:
- Read: 25,000 assertions passed 120.29ms 207,827/s
- Access direct: 50,000 assertions passed 29.04ms 860,614/s
- Access late binding: 50,000 assertions passed 98.13ms 254,764/s
- Write: 25,000 assertions passed 57.84ms 432,204/s
Total failed: 0 / 150,000 - Synopse variant PASSED 306.04ms
1.3. Super object record:
- Read: 25,000 assertions passed 2.00s 12,470/s
- Access: 50,000 assertions passed 408us 61,274,509/s
- Write: 25,000 assertions passed 1.71s 14,539/s
Total failed: 0 / 100,000 - Super object record PASSED 3.72s
1.4. Super object properties:
- Read: 25,000 assertions passed 2.14s 11,631/s
- Access: 50,000 assertions passed 1.92s 12,971/s
- Write: 25,000 assertions passed 186.63ms 133,952/s
Total failed: 0 / 100,000 - Super object properties PASSED 4.26s
1.5. dws JSON:
- Read: 25,000 assertions passed 136.42ms 183,250/s
- Access: 50,000 assertions passed 37.07ms 674,236/s
- Write: 25,000 assertions passed 97.86ms 255,464/s
Total failed: 0 / 100,000 - dws JSON PASSED 273.66ms
1.6. DBXJSON:
- Read: 25,000 assertions passed 2.35s 10,622/s
- Access: 50,000 assertions passed 23.38ms 1,069,244/s
- Write: 25,000 assertions passed 309.64ms 80,737/s
Total failed: 0 / 100,000 - DBXJSON PASSED 2.68s
2. Big content
2.1. Depth content:
- Download files if necessary: no assertion 384us
- Synopse read variant: 1 assertion passed 87.99ms 284,100/s 337 KB
- Synopse read to BSON: 2 assertions passed 2.55ms 9,784,735/s 155 KB
- Super object read: 2 assertions passed 9.20ms 2,716,210/s 529 KB
- dws JSON read: 1 assertion passed 5.55ms 4,503,693/s 439 KB
- DBXJSON read: 1 assertion passed 92.20ms 271,126/s 679 KB
Total failed: 0 / 7 - Depth content PASSED 202.86ms
2.2. Table content:
- Download files if necessary: no assertion 356us 23,112,359/s
- Synopse parse: 1 assertion passed 2.69ms 3,052,690/s 1.2 MB
- Synopse ORM loop: 41,135 assertions passed 6.14ms 1,339,465/s 1.2 MB
- Synopse ORM list: 41,135 assertions passed 6.52ms 1,260,070/s 951 KB
- Synopse table direct: 41,135 assertions passed 20.40ms 403,126/s 1.2 MB
- Synopse table variant: 41,135 assertions passed 20.29ms 405,330/s 1.2 MB
- Synopse doc variant: 41,137 assertions passed 39.80ms 206,661/s 4.6 MB
- Synopse late binding: 41,137 assertions passed 34.45ms 238,768/s 4.6 MB
- Synopse to BSON: 2 assertions passed 8.92ms 922,206/s 1.1 MB
- Super object properties: 41,136 assertions passed 2.14s 3,840/s 6.3 MB
- Super object record: 41,136 assertions passed 148.57ms 55,373/s 6.3 MB
- dws JSON: 41,136 assertions passed 28.87ms 284,888/s 4.7 MB
- DBXJSON: 1 assertion passed 236.75ms 34,749/s 9.9 MB
Total failed: 0 / 370,226 - Table content PASSED 2.70s
2.3. Huge content:
- Download files if necessary: no assertion 428us
- Synopse read record: 4 assertions passed 1.52s 135,810/s 122.6 MB
- Synopse read variant: 2 assertions passed 2.45s 84,134/s 512.9 MB
- Synopse read to BSON: 3 assertions passed 2.01s 102,333/s 168.1 MB
- Super object read: 2 assertions passed 9.07s 22,769/s 1.1 GB
- dws JSON read: 2 assertions passed 3.26s 63,323/s 672.7 MB
- DBXJSON read: no assertion 703us 35,561,877/s
DBXJSON will raise EOutOfMemory for 185 MB JSON in Win32 -> skip
Total failed: 0 / 13 - Huge content PASSED 18.92s
Generated with: Delphi XE6 compiler
Time elapsed for all tests: 33.22s
Tests performed at 17/05/2014 08:47:02
Total assertions failed for all test suits: 0 / 1,020,246
! All tests passed successfully.
SuperObject has some issues for property names lookup...
I've written a TTestTableContent.SuperObjectRecord dedicated
method: accessing the values via a record (and RTTI) is much faster than using
S[...] I[...] and such methods.
Current version did not support XE6 compiler (I had to write some
$ifdef by hand), and when compiled for Win64, the sample program
just exploded... SuperObject needs some tuning!
It is worth saying that dwsJSON performs very well, for its
purpose.
What is written in this blog
article is perfectly true, in comparison to SuperObject or
DBXJSON.
Even on Win64 platform.
Great work, Eric!
DBXJSON is pretty slow, and is even giving an
EOutOfMEmory error in Win32 for the huge content (more than 2GB is
used!) - under Win64, it passes, with 3GB used for the 180 MB JSON file.
In the meanwhile, mORMot uses 150 MB of memory with records. :)
Here are some number concerning XSuperObject:
1.3. X super object record:
- Read: 25,000 assertions passed 12.68s 1,971/s
- Access: 50,000 assertions passed 517us 48,355,899/s
- Write: 25,000 assertions passed 2.32s 10,737/s
Total failed: 0 / 100,000 - X super object record PASSED 15.01s
1.4. X super object properties:
- Read: 25,000 assertions passed 10.14s 2,463/s
- Access: 50,000 assertions passed 307.49ms 81,302/s
- Write: 25,000 assertions passed 435.44ms 57,412/s
Total failed: 0 / 100,000 - X super object properties PASSED 10.89s
I was not able to run SuperObject and XSuperObject in the
same application at once... so you have to use compiler defines in the sample
source code, to let one of the two libraries be compiled.
But XSuperObject is not optimized for speed, it is in fact very slow,
even slower than SuperObject - and not in the race when compared to
dwsJSON or mORMot.
The mORMot code has some advantages, especially for ORM / table
process.
The ability to use record and dynamic arrays to store the
content make it very convenient, and also powerful (see how we used enhanced
RTTI for serialization, but a custom sub-record type for a "polygon /
multi-polygon coordinates" structure which wouldn't be able to be accessed via
regular records from JSON. Our record-based RTTI gives also impressive
results, in both terms of speed and memory consumption.
And late-binding for properties access gives very readable code.
To conclude, which syntax do you prefer?
// Synopse direct record access
Check(gloss.glossary.GlossDiv.GlossList.GlossEntry.GlossDef.GlossSeeAlso[0]='GML');
// Synopse TDocVariant with properties
Check(DocVariantData(doc.GetValueByPath([
'glossary','GlossDiv','GlossList','GlossEntry','GlossDef','GlossSeeAlso'])).Value[0]='GML');
// Synopse TDocVariant with late binding
Check(doc.glossary.GlossDiv.GlossList.GlossEntry.GlossDef.GlossSeeAlso._(0)='GML');
// SuperObject properties
check(obj['glossary.GlossDiv.GlossList.GlossEntry.GlossDef.GlossSeeAlso[0]'].AsString='GML');
// SuperObject direct record access
Check(gloss.glossary.GlossDiv.GlossList.GlossEntry.GlossDef.GlossSeeAlso[0]='GML');
// XSuperObject
check(obj['glossary.GlossDiv.GlossList.GlossEntry.GlossDef.GlossSeeAlso[0]'].AsString='GML');
// dwsJSON
check(obj['glossary']['GlossDiv']['GlossList']['GlossEntry']['GlossDef']['GlossSeeAlso'][0].AsString='GML');
// DBXJSON
check(((((((obj.GetValue('glossary') as TJSONObject).
GetValue('GlossDiv') as TJSONObject).
GetValue('GlossList') as TJSONObject).
GetValue('GlossEntry') as TJSONObject).
GetValue('GlossDef') as TJSONObject).
GetValue('GlossSeeAlso') as TJSONArray).Get(0).Value='GML');
Any feedback is welcome, including your own benchmark results, in our forum, as usual!