Synopse

To content | To menu | To search

Tag - 64bit

Entries feed

2017, Thursday August 10

Faster and cross-platform SynLZ

You probably know about our SynLZ compression unit, in pascal and x86 asm, which is very fast for compression with a good compression ratio, and proudly compete with LZ4 or Snappy.
It is used in our framework everywhere, e.g. for WebSockets communication, for ECC encrypted file content, or to compress executable resources. 

Two news to share:

1. I've added SynLZ support for the NextGen compiler, now available in a new unit of the "CrossPlatform" sub-folder.
Feeback is welcome, since we don't use Delphi for iOS and Android with Delphi, and prefer FPC for Linux!

2. I've also written a new x64 asm optimized version of SynLZ, and profiled the existing x86 asm to be even faster than previously.
For a 100MB text log file, SynLZ is faster than Snappy, and compresses better (93% instead of 84%).
For other kind of files, Snappy is slightly faster at decompression, but SynLZ compresses better, and most of the time faster.
When used on a REST server solution, as with mORMot, compression speed does matter more than decompression.

For Win32:

Win32 Processing DragonFly-devpcm.log = 98.7 MB for 1 times
Snappy compress in 125.07ms, ratio=84%, 789.3 MB/s
Snappy uncompress in 70.35ms, 1.3 GB/s
SynLZ compress in 103.61ms, ratio=93%, 952.8 MB/s
SynLZ uncompress in 68.71ms, 1.4 GB/s

For Win64:

Win64 Processing DragonFly-devpcm.log = 98.7 MB for 1 times
Snappy compress in 107.13ms, ratio=84%, 921.5 MB/s
Snappy uncompress in 61.06ms, 1.5 GB/s
SynLZ compress in 97.25ms, ratio=93%, 1015.1 MB/s
SynLZ uncompress in 61.27ms, 1.5 GB/s

Of course, we didn't change the SynLZ binary format, so it is just perfectly backward compatible with any existing program.
Anyway, from my point of view, the main benefit of SynLZ is that it was designed in plain pascal, so it is clearly cross-platform and well integrated with Delphi/FPC (no external .obj/.o/.dll required).

Feedback is welcome in our forum, as usual!

2017, Wednesday March 22

Delphi 10.2 Tokyo Compatibility: DCC64 broken

We are proud to announce compatibility of our mORMot Open Source framework with the latest Delphi 10.2 Tokyo compiler...
At least for Win32.

For Win64, the compiler was stuck at the end of the compilation, burning 100% of one CPU core...

A bit disappointing, isn't it?

Continue reading...

2016, Monday February 8

Linux support for Delphi to be available end of 2016

Marco Cantu, product manager of Delphi/RAD Studio, did publish the official RAD Studio 2016 Product Approach and Roadmap.
The upcoming release has a codename known as "BigBen", and should be called Delphi 10.1 Berlin, as far as I understand.

After this summer, another release, which codename is "Godzilla", will support Linux as a compiler target, in its Delphi 10.2 Tokyo release.
This is a very good news, and some details are given.
I've included those official names to mORMot's internal compiler version detection.
Thanks Marco for the information, and pushing in this direction!

My only concern is that it would be "ARC-enabled"...

Continue reading...

2015, Monday October 5

Delphi 10 Seattle Win64 compiler Heisenbug: unusable target

Andy reported that he was not able to validate its IDE fix pack for Delphi 10 Seattle, due to its Win64 compiler not being deterministic anymore.
The generated code did vary, from one build to other.

Sadly, on our side, we identified that the code generated by the Win64 compiler of Delphi 10 Seattle is broken.
We have observed some weird code generation with the Win64 platform as a target. Some unexpected exception do occur (like a EPrivilege or EAccessViolation exception).
But it is a random issue, very difficult to reproduce.  After a recompile, no problem any more. Or a problem at another place... A typical Heisenbug...
And, to be clear, no such problem when using an older version of Delphi...

We hope that the corresponding QC entry would be quickly fixed.
So we will stay away from Delphi 10's Win64 compiler, and use Delphi XE8 instead, in the meanwhile.

Update: Issue fixed!
Allen Bauer recognized that "It was an uninitialized memory allocation" in the QC, and that he is pushing to include the fix into the upcoming Seattle 10 Update 1.
Nice seeing such a quick reaction. Delphi is not dead, even if Embarcadero was just acquired! :)

2015, Saturday February 21

SynCrypto: SSE4 x64 optimized asm for SHA-256

We have just included some optimized x64 assembler to our Open Source SynCrypto.pas unit so that SHA-256 hashing will perform at best speed.
It is an adaptation from tuned Intel's assembly macros, which makes use of the SSE4 instruction set, if available.

Continue reading...

2014, Friday November 14

BREAKING CHANGE - TSQLRecord.ID primary key changed to TID: Int64

Up to now, the TSQLRecord.ID property was defined in mORMot.pas as a plain PtrInt/NativeInt (i.e. Integer under Win32), since it was type-cast as pointer for TSQLRecord published properties.
We introduced a new TID type, so that the ORM primary key would now be defined as Int64.

All the framework ORM process relies on the TSQLRecord class.
This abstract TSQLRecord class features a lot of built-in methods, convenient to do most of the ORM process in a generic way, at record level.

It first defines a primary key field, defined as ID: TID, i.e. as Int64 in mORMot.pas:

type
  TID = type Int64;
  ...
  TSQLRecord = class(TObject)
  ...
    property ID: TID read GetID write fID;
  ...

In fact, our ORM relies now on a Int64 primary key, matching the SQLite3 ID/RowID primary key.
This primary key will be used as RESTful resource identifier, for all CRUD operations.

Continue reading...

2014, Monday June 30

Sub-optimized Win64 Delphi compiler: missing branch table for case of

As we already stated here, the Delphi compiler for the Win64 target performs well, as soon as you by-pass the RTL and its sub-optimized implementation - as we do for mORMot.
In fact, our huge set of regression tests perform only 10% slower on Win64, when compared to Win32.
But we got access to much more memory - which is not a huge gain for a mORMot server, which uses very little of RAM - so may be useful in some cases, when you need a lot of structures to be loaded in your RAM.

Slowdown on Win64 is mostly due to biggest pointer size, which will use twice the memory, hence may generate a larger number of cache misses (failed attempts to read or write a piece of data in the cache, which results in a main memory access with much longer latency).
But in Delphi, apart from the RTL which may need more tuning about performance (but seems not to be a priority on Embarcadero side), is also sometimes less efficient when generating the code.
For instance, sounds like if case ... of ... end statements do not generated branch table instructions on Win64, whereas it does for Win32 - and FPC does for any x64 platform it supports.

Continue reading...

2014, Monday June 9

Performance comparison from Delphi 6, 7, 2007, XE4 and XE6

Since there was recently some articles about performance comparison between several versions of the Delphi compiler, we had to react, and gives our personal point of view.

IMHO there won't be any definitive statement about this.
I'm always doubtful about any conclusion which may be achieved with such kind of benchmarks.
Asking "which compiler is better?" is IMHO a wrong question.
As if there was some "compiler magic": the new compiler will be just like a new laundry detergent - it will be cleaner and whiter...

Performance is not about marketing.
Performance is an iterative process, always a matter of circumstances, and implementation.

Circumstances of the benchmark itself.
Each benchmark will report only information about the process it measured.
What you compare is a limited set of features, running most of the time an idealized and simplified pattern, which shares nothing with real-world process.

Implementation is what gives performance.
Changing a compiler will only gives you some percents of time change.
Identifying the true bottlenecks of an application via a profiler, then changing the implementation of the identified bottlenecks may give order of magnitudes of speed improvement.
For instance, multi-threading abilities can be achieved by following some simple rules.

With our huge set of regression tests, we have at hand more than 16,500,000 individual checks, covering low-level features (like numerical and text marshaling), or high-level process (like concurrent client/server and database multi-threaded process).

You will find here some benchmarks run with Delphi 6, 7, 2007, XE4 and XE6 under Win32, and XE4 and XE6 under Win64.
In short, all compilers performs more or less at the same speed.
Win64 is a little slower than Win32, and the fastest appears to be Delphi 7, using our enhanced and optimized RTL.

Continue reading...

2014, Sunday May 25

New crc32c() function using optimized asm and SSE 4.2 instruction

Cyclic Redundancy Check (CRC) codes are widely used for integrity checking of data in fields such as storage and networking.
There is an ever-increasing need for very high-speed CRC computations on processors for end-to-end integrity checks.

We just introduced to mORMot's core unit (SynCommons.pas) a fast and efficient crc32c() function.

It will use either:

  • Optimized x86 asm code, with unrolled loops;
  • SSE 4.2 hardware crc32 instruction, if available.

Resulting speed is very good.
This is for sure the fastest CRC function available in Delphi.
Note that there is a version dedicated to each Win32 and Win64 platform - both performs at the same speed!

In fact, most popular file formats and protocols (Ethernet, MPEG-2, ZIP, RAR, 7-Zip, GZip, and PNG) use the polynomial $04C11DB7, while Intel's hardware implementation is based on another polynomial, $1EDC6F41 (used in iSCSI and Btrfs).
So you would not use this new crc32c() function to replace the zlib's crc32() function, but as a convenient very fast hashing function at application level.
For instance, our TDynArray wrapper will use it for fast items hashing.

Continue reading...

2013, Saturday May 11

Delphi XE4 NextGen compiler: using byte instead of ansichar?

When I first read the technical white paper covering all of the language changes in XE4 for mobile development (tied to the new ARM LLVM-based Delphi compiler), I have to confess I was pretty much confused.

Two great mORMot users just asked for XE4/iOS support of mORMot.

Win32/Win64 support for XE4 will be done as soon as we got a copy of it.
I suspect the code already works, since it was working as expected with XE3, and we rely on our own set of low-level functions for most internal work.

But iOS-targetting is more complex, due to the NextGen compiler, mainly.

Continue reading...

2013, Saturday March 23

Download latest version of sqlite3.dll for Windows 64 bit

Update: We now build the amalgamation file with mingw and release the latest version of SQLite3, from this direct SQLite3-64.7z link, as soon as it is published on the SQLite3 site.

Up to now, there is no official Win64 version of the SQlite3 library released in http://sqlite.org..
It is in fact very difficult to find a ready-to-use and up-to-date SQLite3-64.dll from Internet, for Win64.

You can find the latest version of the SQlite3 external library, to be used in 64 bit mode, to be downloaded from SQLite3-64.7z.

It includes FTS3/FTS4/FTS5 virtual tables, and was compiled in release mode.

This is the version we use when our mORMot framework targets Win64, using latest versions of the Delphi compiler.

2013, Wednesday March 13

x64 optimized asm of FillChar() and Move() for Win64

We have included x64 optimized asm of FillChar() and Move() for Win64 - for corresponding compiler targets, i.e. Delphi XE2 and XE3.
It will handle properly cache prefetch and appropriate SSE2 move instructions

The System.pas unit of Delphi RTL will be patched at startup, unless the NOX64PATCHRTL conditional is defined.
Therefore, whole application may benefit for this optimized version.

Performance improvement is noticeable, when compared with the original pascal-based version included in System.pas.

By the way, the Delphi x64 built-in assembler does not recognize the movnti opcode... so we had to inline it as plain db hexadecimal values.
A bit disappointing. Until now, we did not suffer from anything in regard to the x64 compatibility at Delphi level.

No stand-alone unit available yet, since it is included in our SynCommons.pas shared unit, starting with the 1.18 revision of mORMot.

Feedback is welcome, as usual!

2013, Thursday March 7

64 bit compatibility of mORMot units

I'm happy to announce that mORMot units are now compiling and working great in 64 bit mode, under Windows.
Need a Delphi XE2/XE3 compiler, of course!

ORM and services are now available in Win64, on both client and server sides.
Low-level x64 assembler stubs have been created, tested and optimized.
UI part is also available... that is grid display, reporting (with pdf export and display anti-aliasing), ribbon auto-generation, SynTaskDialog, i18n... the main SynFile demo just works great!

Overall impression is very positive, and speed is comparable to 32 bit version (only 10-15% slower).

Speed decrease seems to be mostly due to doubled pointer size, and some less optimized part of the official Delphi RTL.
But since mORMot core uses its own set of functions (e.g. for JSON serialization, RTTI support or interface calls or stubbing), we were able to release the whole 64 bit power of your hardware.

Delphi 64 bit compiler sounds stable and efficient. Even when working at low level, with assembler stubs.
Generated code sounds more optimized than the one emitted by FreePascalCompiler - and RTL is very close to 32 bit mode.
Overall, VCL conversion worked as easily than a simple re-build.
Embarcadero's people did a great job for VCL Win64 support, here!

Continue reading...

2013, Monday February 25

Using external MinGW/VisualC++ sqlite3.dll - including benchmark

With upcoming revision 1.18 of the framework, our SynSQlite3.pas unit is able to access the SQLite3 engine in two ways:

  • Either statically linked within the project executable;
  • Or from an external sqlite3.dll library file.

The SQLite3 APIs and constants are defined in SynSQlite3.pas, and accessible via a TSQLite3Library class definition. It defines a global sqlite3 variable as such:

var
  sqlite3: TSQLite3Library;

To use the SQLite3 engine, an instance of TSQLite3Library class shall be assigned to this global variable. Then all mORMot's calls will be made through it, calling e.g. sqlite3.open() instead of sqlite3_open().

There are two implementation classes:

Class Unit Purpose
TSQLite3LibraryStatic SynSQLite3Static.pas Statically linked engine (.obj within the .exe)
TSQLite3LibraryDynamic SynSQLite3.pas Instantiate an external sqlite3.dll instance

Referring to SynSQLite3Static.pas in the uses clause of your project is enough to link the .obj engine into your executable.

Warning - breaking change: before version 1.18 of the framework, link of static .obj was forced - so you must add a reference to SynSQLite3Static in your project uses clause to work as expected.

In order to use an external sqlite3.dll library, you have to set the global sqlite3 variable as such:

 FreeAndNil(sqlite3); // release any previous instance (e.g. static)
 sqlite3 := TSQLite3LibraryDynamic.Create;

Of course, FreeAndNil(sqlite3) is not mandatory, and should be necessary only to avoid any memory leak if another SQLite3 engine instance was allocated (may be the case if SynSQLite3Static is referred somewhere in your project's units).

Here are some benchmarks, compiled with Delphi XE3, run in a 32 bit project, using either the static bcc-compiled engine, or an external sqlite3.dll, compiled via MinGW or Microsoft Visual C++.

Continue reading...

2011, Sunday December 4

Total Commander 64 bit is using... Lazarus and FPC

I'm a long-time registered user of Total Commander.

This tool is my daily file manager. I never use Windows Explorer, since Total Commander is just faster, more easy to use (especially with the keyboard), has a lot of plug-ins. I even created my own plug-ins to access some custom file formats, and navigate into them just like with any folder. And it includes a lot of well written commands for FTP access or file comparison, which made other tools (like WinMerge) unnecessary.

There is a new beta version of Total Commander available, which targets Windows 64 bit. I just thought: 'Whoo, this is a real-world Delphi XE2 64 application'. I downloaded and tried it. Worked as expected, and integrates seamlessly with Windows Seven (for the shell extensions). Then I took a look at the executable... and discovered it was not compiled with Delphi XE2... but with FPC !

Continue reading...

2011, Tuesday November 8

Currency is your friend

The currency type is the standard Delphi type to be used when storing and handling monetary values. It will avoid any rounding problems, with 4 decimals precision. It is able to safely store numbers in the range -922337203685477.5808 .. 922337203685477.5807. Should be enough for your pocket change.

As stated by the official Delphi documentation:

Currency is a fixed-point data type that minimizes rounding errors in monetary calculations. On the Win32 platform, it is stored as a scaled 64-bit integer with the four least significant digits implicitly representing decimal places. When mixed with other real types in assignments and expressions, Currency values are automatically divided or multiplied by 10000.

In fact, this type matches the corresponding OLE and .Net implementation of currency, and the one used by most database providers (when it comes to money, a dedicated type is worth the cost in a "rich man's world"). It is still implemented the same in the Win64 platform (since XE 2). The Int64 binary representation of the currency type (i.e. value*10000 as accessible via PInt64(aCurrencyValue)^) is a safe and fast implementation pattern.

In our framework, we tried to avoid any unnecessary conversion to float values when dealing with currency values. Some dedicated functions have been implemented for fast and secure access to currency published properties via RTTI, especially when converting values to or from JSON text. Using the Int64 binary representation can be not only faster, but also safer: you will avoid any rounding problem which may be introduced by the conversion to a float type. Rounding issues are a nightmare to track - it sounds safe to have a framework handling natively a currency type from the ground up.

Continue reading...

2011, Monday September 12

Using Extended in Delphi XE2 64 bit

Unfortunately, Delphi's 64-bit compiler (dcc64) and RTL do not support 80-bit extended floating point values on Win64, but silently alias Extended = Double on Win64.

There are situations, however, where this is clearly undesirable, e.g. if the additional precision gained from Extended is required.

The Open-source uTExtendedX87 unit provides a replacement FPU-backed 80-bit Extended floating point type (TExtendedX87) for Win64.

Continue reading...

2011, Monday August 8

Our mORMot won't hibernate this winter, thanks to FireMonkey

Everybody is buzzing about FireMonkey...

Our little mORMot will like FireMonkey!
Here is why...

Continue reading...

2011, Thursday June 16

Which Delphi compiler produces faster code?

After a question on StackOverflow, I wanted to comment about the speed of generated code by diverse Delphi compiler versions.

Since performance matters when we write general purpose libraries like ours, we have some feedback to propose:

Continue reading...

2011, Thursday May 26

Calling a 64 bit library from a Delphi 32 bit process

Since we are still waiting for a Delphi 64 bit compiler, the only available solution to access a 64 bit library from an application written in Object pascal, is to use the 64 bit version of the FreePascal Compiler.

But you just can not recompile your VCL/GUI based Delphi application with FPC:

  • Some low-level part of your code may not be directly compatible with a 64 bit process (e.g. since the pointer size changed);
  • The GUI part of the application can not be ported directly with FPC - the Lazarus project try to be as close as possible to VCL, but it can be a very difficult, either impossible if you use some third-party components.
I just found out a solution from CodeCentral, allowing to call any 64 bit dll from a Delphi 32 bit process.

Continue reading...

- page 1 of 2