JSON benchmarking ------------------- 1. Small content 1.1. Synopse record: - Read: 50,000 assertions passed 153.07ms 326,641/s - Access: 100,000 assertions passed 958us 52,192,066/s - Write: 50,000 assertions passed 105.84ms 472,411/s Total failed: 0 / 200,000 - Synopse record PASSED 260.33ms 1.2. Synopse variant: - Read: 50,000 assertions passed 381.95ms 130,904/s - Access direct: 100,000 assertions passed 67.59ms 739,677/s - Access late binding: 100,000 assertions passed 699.14ms 71,516/s - Write: 50,000 assertions passed 215.65ms 231,857/s Total failed: 0 / 300,000 - Synopse variant PASSED 1.36s 1.3. Synopse cross platform variant: - Read: 50,000 assertions passed 645.38ms 77,473/s - Access direct: 100,000 assertions passed 41.87ms 1,194,086/s - Access late binding: 100,000 assertions passed 612.42ms 81,643/s - Write: 50,000 assertions passed 625.52ms 79,932/s Total failed: 0 / 300,000 - Synopse cross platform variant PASSED 1.92s 1.4. Super object record: - Read: 50,000 assertions passed 2.37s 21,014/s - Access: 100,000 assertions passed 1.01ms 49,504,950/s - Write: 50,000 assertions passed 1.80s 27,651/s Total failed: 0 / 200,000 - Super object record PASSED 4.19s 1.5. Super object properties: - Read: 50,000 assertions passed 895.68ms 55,823/s - Access: 100,000 assertions passed 369.84ms 135,192/s - Write: 50,000 assertions passed 370.30ms 135,025/s Total failed: 0 / 200,000 - Super object properties PASSED 1.63s 1.6. dws JSON: - Read: 50,000 assertions passed 273.69ms 182,683/s - Access: 100,000 assertions passed 87.51ms 571,356/s - Write: 50,000 assertions passed 204.84ms 244,089/s Total failed: 0 / 200,000 - dws JSON PASSED 569.01ms 1.7. DBXJSON: - Read: 50,000 assertions passed 4.83s 10,340/s - Access: 100,000 assertions passed 49.40ms 1,012,043/s - Write: 50,000 assertions passed 616.26ms 81,133/s Total failed: 0 / 200,000 - DBXJSON PASSED 5.50s 1.8. QDAC: - Read: 50,000 assertions passed 639.13ms 78,230/s - Access: 100,000 assertions passed 95.95ms 521,061/s - Write: 50,000 assertions passed 386.40ms 129,398/s Total failed: 0 / 200,000 - QDAC PASSED 1.12s 1.9. Json data objects: - Read: 50,000 assertions passed 248.87ms 200,900/s - Access: 100,000 assertions passed 7.54ms 6,629,541/s - Write: 50,000 assertions passed 109.55ms 456,379/s Total failed: 0 / 200,000 - Json data objects PASSED 368.36ms 2. Big content 2.1. Depth content: - Download files if necessary: no assertion 408us - Synopse read variant: 1 assertion passed 167.89ms 297,799/s 227 KB - Synopse read to BSON: 2 assertions passed 2.87ms 17,379,214/s 159 KB - Synopse cross platform: 1 assertion passed 3.94ms 12,690,355/s 300 KB ! - Super object read: 1 / 1 FAILED 17.67ms 2,828,374/s - dws JSON read: 1 assertion passed 4.83ms 10,349,824/s 1.5 MB - DBXJSON read: 1 assertion passed 92.25ms 541,958/s 1.9 MB - Json data objects read: 1 assertion passed 2.73ms 18,254,837/s 1.5 MB Total failed: 1 / 8 - Depth content FAILED 304.11ms 2.2. Table content: - Download files if necessary: no assertion 469us 17,543,710/s - Synopse parse: 1 assertion passed 2.70ms 3,043,655/s 1.2 MB - Synopse ORM loop: 41,135 assertions passed 6.67ms 1,233,433/s 1.2 MB - Synopse ORM list: 41,135 assertions passed 6.94ms 1,184,422/s 1016 KB - Synopse table direct: 41,135 assertions passed 19.38ms 424,509/s 1.2 MB - Synopse table variant: 41,135 assertions passed 97.61ms 84,283/s 1.2 MB - Synopse doc variant: 41,137 assertions passed 31.50ms 261,141/s 3.0 MB - Synopse late binding: 41,137 assertions passed 106.78ms 77,045/s 3.0 MB - Synopse cross ORM: 41,135 assertions passed 18.99ms 433,228/s 1.9 MB - Synopse cross direct: 41,135 assertions passed 20.19ms 407,418/s 1.9 MB - Synopse cross variant: 41,135 assertions passed 100.13ms 82,162/s 1.9 MB - Synopse to BSON: 2 assertions passed 10.57ms 777,893/s 1.0 MB - Super object properties: 41,136 assertions passed 180.95ms 45,463/s 6.3 M B - Super object record: 41,136 assertions passed 150.22ms 54,763/s 6.3 MB - dws JSON: 41,136 assertions passed 29.29ms 280,871/s 4.7 MB - DBXJSON: 41,136 assertions passed 265.78ms 30,953/s 9.10 MB - JsonDataObjects: 41,136 assertions passed 15.41ms 533,770/s 2.6 MB - QDAC: 41,136 assertions passed 41.53ms 198,073/s 5.9 MB Total failed: 0 / 617,038 - Table content PASSED 1.13s 2.3. Huge content: - Download files if necessary: no assertion 447us - Synopse beautifier: 1 assertion passed 1.58s 31,558/s 104 B - Synopse read record: 4 assertions passed 1.44s 143,250/s 113.5 MB - Synopse read variant: 2 assertions passed 3.38s 61,027/s 377.4 MB - Synopse cross platform: 2 assertions passed 5.46s 37,824/s 424.8 MB - Synopse read to BSON: 3 assertions passed 1.96s 105,013/s 168.1 MB - Super object read: 2 assertions passed 8.99s 22,951/s 1.1 GB - dws JSON read: 2 assertions passed 3.20s 64,466/s 672.7 MB - DBXJSON read: no assertion 151us 331,125,827/s DBXJSON will raise EOutOfMemory for 185 MB JSON in Win32 -> skip - Json data objects read: 2 assertions passed 1.70s 121,237/s 329.6 MB - Json data objects beautifier: no assertion 7.59s 6,585/s 510.6 MB - QDAC read: 2 assertions passed 8.41s 24,558/s 1.1 GB
Total failed: 0 / 20 - Huge content PASSED 36.90s Generated with: Delphi XE7 compiler Time elapsed for all tests: 55.30s Tests performed at 16/02/2015 09:41:34 Total assertions failed for all test suits: 1 / 2,617,066 ! Some tests FAILED: please correct the code.
As you can see, the latest revision of SuperObject, retrieved
from SVN, failed to pass a test.
There is a weird depth restriction in this library.
Since we don't use this library, which is slow and just blowing under Win64, it
is not a problem for us.
We did not include XSuperObject here, since we already benchmarked it: it is even slower than SuperObject.
As you can see, JsonDataObjects is a great library, fastest
that any other implementations, when using the general purpose
DOM/node-oriented use case.
As always, our ORM-optimized version is fastest for table reading
(see Synopse ORM
loop and Synopse ORM
list), but it is not 100% fair since it has been optimized for a single
use case - which is the main bottleneck of any JSON-based ORM like
mORMot.
Take a look at our Synopse
parse: it is in fact our SAX-like JSON parser, which is able to
read, parse, unescape and fill a list of pointers with each data in more than
3,000,000 rows per seconds, with almost no memory use. A DOM/node approach can
only reach 500,000 rows per second, for JsonDataObjects.
All those numbers are always high: in practice, keeping away from the RTL
DBXJson.pas, or SuperObject units is enough.
In a multi-threaded environment, there may be some bottlenecks of
JsonDataObjects writing, which uses temporary strings for building the
JSON text from number values, so relies heavily on the memory manager (more
than dwsJSON or mORMot implementations).
You can see this behavior when comparing the Json data
objects beautifier time and memory consumption with our
SAX-based Synopse
beautifier (which, BTW preserve the original floating point
precision).
IMHO the only missing feature is a "path query", i.e. allow to retrieve
values by a path.
It could be nice to be able to write (as with other libraries):
Check(doc.A['glossary.GlossDiv.GlossList.GlossEntry.GlossDef.GlossSeeAlso'].S[0]='GML');
Check(doc.O['glossary'].O['GlossDiv'].O['GlossList'].
O['GlossEntry'].O['GlossDef'].A['GlossSeeAlso'].S[0]='GML');
Congrats Andreas for your great work!
Source code of the test is available in
our source code repository.
And feedback is welcome on our forum, as
usual.
