XML may no longer be fashionable, but it is still everywhere.
Scientific datasets, configuration files, industry standards, legacy systems, government data, document formats and countless integration interfaces continue to produce XML. And when XML documents become large, the choice of parser can make a surprisingly large difference.
For mORMot 2, we have recently written a new XML parser, TXmlParser, with a slightly different goal from the traditional XML libraries found in the Delphi and Free Pascal ecosystems. It started from an AI-driven pull request from zen010101, then it was fully rewritten by hand for performance and enhanced to support several API modes.
Instead of choosing between a low-level SAX parser and a convenient DOM parser, the same parser provides several levels of access:
- a zero-allocation SAX/pull API;
- a streaming/navigation API for selecting only the parts of the document that matter;
- a hybrid SAX/DOM API, where selected subtrees can be materialized as TDocVariantData;
- and a full XML-to-TDocVariant DOM representation, when that is what the application actually needs.
The interesting part is that the performance remains very high at all these levels.
Using the 24 MB nasa.xml reference document from the University of Washington XML repository, the raw parser reaches about 2.1 GB/s, while the full DOM conversion reaches about 424 MB/s. Selective streaming reaches about 1.6 GB/s, and once the DOM exists, traversing it through the new TDocVariantData.Product() enumerator reaches about 7 GB/s.
There is also an interesting memory optimization: enabling dvoInternNames reduces the complete DOM footprint from approximately 60 MB to 40 MB, with essentially no performance penalty.
The complete benchmark is shown below.
