From an AI pull request to production code
The initial contribution was an AI-assisted pull request adding ZIP encryption support.
AI coding tools are becoming increasingly useful for generating substantial amounts of code, especially when the requested functionality follows an existing specification. But generated code, even when it compiles and passes basic tests, is not necessarily ready for production.
Cryptography makes this distinction particularly important.
An encryption implementation needs to be correct not only for successful round trips, but also for malformed archives, invalid passwords, truncated ciphertext, modified authentication tags, and unusual file sizes. We also need to consider memory consumption, performance, portability, and backward compatibility with existing mORMot applications.
So we took the initial contribution as a starting point, not as a finished implementation.
We reviewed the design against the ZIP specifications and the existing mORMot cryptographic primitives, simplified the class hierarchy, added independent reference vectors, and exercised the code with both Delphi and Free Pascal in mind.
During this work, we also extended the original scope to the new AE-3 format.
Several subtle bugs and missing cases were uncovered by this review, especially in the interaction between AES-GCM authentication and streaming ZIP extraction.
This is probably a good example of how we see AI-generated code fitting into an Open Source project: AI can accelerate the initial development, but the library maintainer remains responsible for the architecture, testing, and actual security properties.
Thanks again to the initial contributor of the initial Pull Request! It is amazing how AI tools do like the mORMot 2 code base, and are aware to propose some nice code, saving countless hours of coding, especially to write extensive regression tests.
Three encryption formats
The new unit implements three families of ZIP encryption.
- ZipCrypto is the original PKWARE encryption algorithm. It remains useful for compatibility with old tools, including Windows Explorer, but its cryptographic security is very weak. It should not be used to protect sensitive information.
- AE-1 / AE-2 are the established WinZip AES formats, supporting AES-128, AES-192 and AES-256 in CTR mode, with HMAC-SHA1 authentication. These formats are widely implemented by ZIP utilities, but their password derivation uses only 1,000 PBKDF2-HMAC-SHA1 iterations.
- AE-3 is the new WinZip format, using AES-256-GCM authenticated encryption and PBKDF2-HMAC-SHA256 with 600,000 iterations by default. It offers a much more modern construction, at the cost of significantly more expensive password derivation.
For most applications, AE-1 or AE-2 remain an interesting compatibility choice. AE-3 is preferable when stronger password protection is required and interoperability with older ZIP software is not a concern. Deprecated ZipCrypto may still be needed if you expect compatibility with Windows or MacOS file explorers, which only support this legacy unsafe encoding - consider it as obfuscation not as encryption.
The new enumeration makes the selection explicit:
TZipEncryption = ( zeAes256, zeZipCrypto, zeAes256Gcm);
The existing TZipWrite and TZipRead APIs remain unchanged apart from the new encryption properties.
For instance:
w.Password := 'correct horse battery staple';
w.Encryption := zeAes256Gcm;
w.AddDeflated('hello.txt', pointer(Plain), length(Plain));
Reading an encrypted archive is equally straightforward: set TZipRead.Password before extracting its contents.
Of course, the actual security still depends heavily on the quality of the password.
Keeping the ZIP core lightweight
One important design decision was to keep the cryptographic implementation outside mormot.core.zip.
We did not want every application using ZIP compression to depend on the cryptographic units, nor to increase executable size unnecessarily.
Instead, mormot.core.zip defines a small abstract TZipCrypt interface and an array of implementation classes.
The new mormot.crypt.zip.pas unit implements all the actual algorithms and registers them during initialization:
initialization ZipCryptClass[zeAes256] := TZipCryptAes; ZipCryptClass[zeZipCrypto] := TZipCryptPkware; ZipCryptClass[zeAes256Gcm] := TZipCryptGcm;
This is a simple form of late binding, following a pattern already used elsewhere in mORMot.
If an application only needs compression, it can keep using mormot.core.zip alone.
If it also needs encryption, it just adds mormot.crypt.zip to its uses clause. The classes are automatically registered, and TZipRead / TZipWrite can instantiate the appropriate implementation.
There is no explicit registration call, no dynamic OpenSSL dependency, and no change to the normal ZIP API.
We like this kind of integration: a small optional unit can add significant functionality without making the core more complicated.
Reusing and optimizing our cryptographic engine
All three ZIP encryption formats reuse the low-level cryptographic code already available in mORMot.
In particular, AES-GCM is implemented by TAesGcmEngine from mormot.crypt.core, which already provides optimized AES-NI and PCLMULQDQ execution paths.
We also took the opportunity to improve the general AES implementation.
TAes.DoBlocks() now includes dedicated four-way interleaved AES-NI assembly routines, for both encryption and decryption, on x86 and x86-64.
Instead of processing each AES block independently, we can process four blocks together, interleaving their AES rounds across several XMM registers. This allows the processor to execute independent AES instructions while previous rounds are still in flight.
The optimized path is selected automatically when AES-NI is available and at least four blocks can be processed. The remaining blocks use the existing single-block path.
These routines are written in assembly for both 32-bit and 64-bit targets, preserve the required calling conventions, and leave the AES context untouched.
The AES-GCM engine has its own eight-way interleaved x86-64 acceleration, combining AES-NI with carry-less multiplication for authentication.
As usual, portable Pascal implementations remain available for platforms without these hardware instructions.
AE-3: the most interesting technical challenge
The newest WinZip AE-3 format deserves a closer look.
To our knowledge, mORMot 2 is the first OpenSource library in any language to support this new format. We used infozip-av source as reference, but it is a tool, not a library.
Compared with AE-2, AE-3 replaces the separate AES-CTR and HMAC-SHA1 combination with authenticated AES-256-GCM encryption. Its password derivation also becomes considerably more expensive: 600,000 PBKDF2-HMAC-SHA256 iterations by default, with a configurable work factor stored in the ZIP entry metadata.
The format derives independent encryption and authentication material using HKDF-SHA256, then stores a 16-byte GCM tag and a supplementary 4-byte FIN authentication value.
The most interesting implementation issue, however, was not AES-GCM itself. We already had that. It was how to handle authenticated ZIP extraction.
A ZIP reader can decompress data directly into a destination stream. But with GCM, the authentication tag is only available at the end of the encrypted entry. We must therefore avoid delivering unauthenticated plaintext to the caller.
For extraction into an arbitrary TStream, TZipRead now makes a preliminary authentication pass over file-backed encrypted entries, before delivering their contents.
We introduced a dedicated TAesGcmEngine.Auth() method for this purpose: it computes the authentication state over the ciphertext without performing the AES-CTR decryption. Then we can reset the GCM state while retaining the derived AES key, and perform the actual extraction. This requires reading the encrypted data twice, but avoids repeating the very expensive PBKDF2 derivation.
For extraction to a filename, we found a simpler solution.
TZipRead writes the extracted content to a temporary file with a .partial extension, located in the destination folder. After successful authentication and content validation, it closes the temporary file and renames it to the requested filename. If authentication fails, the temporary file is removed, and the existing destination remains unchanged.
This allows a single decryption pass even for very large files. It is not intended to provide atomic filesystem replacement, which TZipRead never promised, but prevents a failed authentication from publishing the output under its final name.
Another optimization was made to TestAll(), which no longer needs an unnecessary preliminary pass when validating encrypted entries into a discard stream.
These changes benefit the overall ZIP encryption architecture, not just AE-3.
Testing beyond simple round trips
We considerably extended the regression tests during the implementation.
In addition to generating encrypted archives with TZipWrite and reading them back with TZipRead, we use external reference vectors, including independently generated AE-3 archives.
We test stored, deflated, and empty entries, different password work factors, wrong passwords, corrupted ciphertext, invalid GCM tags, modified FIN fields, malformed metadata, and ZIP entry copying or compaction. We also explicitly test both memory-backed and file-backed readers.
The final regression tests therefore exercise both successful and unsuccessful extraction, including the two-pass authentication and reset sequence.
This work illustrates why cryptographic code needs independent test vectors and careful review, even when built on top of an existing, well-tested cipher engine.
And what about AE-4?
There is still one limitation in the new AE-3 format that we find particularly interesting.
The password derivation is performed independently for every encrypted ZIP entry.
This is a reasonable security choice, but it has a noticeable performance consequence: when an archive contains thousands of small files encrypted with the same password, the repeated 600,000 PBKDF2 iterations can easily dominate the total processing time.
AES-NI optimizations won't help much here, because the bottleneck is no longer AES encryption.
A possible future AE-4 specification could address this by deriving an archive-level master key from the password only once, then deriving independent encryption keys for each ZIP entry using a safe HKDF like HMAC. This would preserve separate keys for individual files, while greatly reducing the latency of archives containing many small entries protected by the same password.
Of course, reusing key hashing would still be an implementation detail: each file could have its own password, but adding a HMAC step could be enough to reuse a single expensive password hash for several files, without reducing the actual security.
It is only a proposal at this stage, not an existing format. We would welcome discussion with other ZIP implementers before going any further. It is worth asking how WinZip actually validated the proper use of their algorithm. Also why they reverted to AE-2 in WinZip 11 to keep a quick crc32 check of the encrypted content, but they force 0 in their AE-3...
Conclusion
ZIP encryption is now fully integrated into mORMot 2, through a small optional unit and the same familiar TZipRead / TZipWrite APIs.
We support the historical ZipCrypto format, the widely used WinZip AE-1 / AE-2 formats, and the very recent AES-256-GCM AE-3 format.
Beyond the new encryption algorithms, this development also brought improved AES-NI assembly routines, safer authenticated extraction, more efficient streaming, and extensive regression testing.
And it started with an AI-assisted pull request.
The experience was positive, but also instructive. AI helped initiate the work, while the detailed human review led to a substantially different and more complete implementation.
This combination of assisted development, careful architecture, independent testing and low-level optimization is probably where such tools bring the most value to a mature library like mORMot.
The full source is available in our mORMot 2 repository, especially mormot.crypt.zip.pas and mormot.core.zip.pas.
As always, feedback and interoperability tests with other ZIP implementations are welcome!








