Tag - Source

Entries feed - Comments feed

2013-02-17

Interface-based service sample: remote SQL access

You will find in the SQLite3\Sample\16 - Execute SQL via services folder of mORMot source code a Client-Server sample able to access any external database via JSON and HTTP.
It is a good demonstration of how to use an interface-based service between a client and a server.
It will also show how our SynDB classes have a quite abstract design, and are easy to work with, whatever database provider you need to use.

The corresponding service contract has been defined:

  TRemoteSQLEngine = (rseOleDB, rseODBC, rseOracle, rseSQlite3, rseJet, rseMSSQL);

IRemoteSQL = interface(IInvokable) ['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}'] procedure Connect(aEngine: TRemoteSQLEngine; const aServerName, aDatabaseName, aUserID, aPassWord: RawUTF8); function GetTableNames: TRawUTF8DynArray; function Execute(const aSQL: RawUTF8; aExpectResults, aExpanded: Boolean): RawJSON; end;

Purpose of this service is:
- To Connect() to external databases, given the parameters of a standard TSQLDBConnectionProperties. Create() constructor;
- Retrieve all table names of this external database as a list;
- Execute any SQL statement, returning the content as JSON array, ready to be consumed by AJAX applications (if aExpanded is true), or a Delphi client (e.g. via a TSQLTableJSON and the mORMotUI unit).

Of course, this service will be define as sicClientDriven mode, that is, the framework will be able to manage a client-driven TSQLDBProperties instance life time.

Benefit of this service is that no database connection is required on the client side: a regular HTTP connection is enough.
No need to install nor configure any database provider, and full SQL access to the remote databases.

Due to our optimized JSON serialization, it will probably be faster to work with such plain HTTP / JSON services, instead of a database connection through a VPN. In fact, database connections are made to work on a local network, and do not like high-latency connections, which are typical on the Internet.

Continue reading

2013-02-12

Introducing ZEOS, UniDAC, NexusDB, BDE, any TDataset to SynDB and mORMot's ORM

Up to now, our SynDB database classes were handling ODBC, OleDB providers and direct Oracle or SQLite3 connection.

We have added a DB.pas based layer, ready to be used with UniDAC, NexusDB, or the BDE.
Any other TDataset based component is ready to be interfaced, including UIB, AnyDAC or DBExpress.

The ZEOS library (in its latest 7.0.3 stable version, which works from Delphi 7 up to XE3) has also been interfaced, but without the TDataset/DB.pas layer: our SynDBZEOS.pas unit calls the ZDBC layer, which is not tied to DB.pas nor its RAD components, and is therefore faster. By the way, it will work also with the Starter edition of Delphi (which does not include the DB components) - just like the other "regular" SynDB classes.

This is a work in progress, any testing and feedback is welcome!
We had to circumvent some particularities of the libraries, but I guess we have something interesting.

A dedicated "SynDBDataset" sub-folder has been created in the repository, to contain all SynDBDataset.pas-based database providers.
SynDBNexusDB.pas unit has been moved within this sub-folder, as SynDBUniDAC.pas + SynDBBDE.pas units have been added.
SynDBZeos.pas has a direct access to the ZDBC layer, so is not part of the "SynDBDataset" sub-folder.

Here is some benchmark, mainly about Oracle and SQlite3 database access.
Of course, our direct SynDBOracle / SynDBSQLite3 layers are the fastest around, and we can see that ZDBC layer is sometimes more efficient than the TDataset components.

Continue reading

2013-02-03

Log to the console

Our framework features an integrated logging class, ready to be enabled for support and statistics.

For debugging purposes, it could be very handy to output the logging content to a console window.
It enables interactive debugging of a Client-Server process, for instance: you can interact with the Client, then look in real time at the server console window, and inspect which requests are processed, without the need to open the log file.

Depending on the events, colors will be used to write the corresponding information. Errors will be displayed as light red, for instance.

Continue reading

2013-01-28

External database speed improvements

Some major speed improvements have been made to our SynDB* units, and how they are used within the mORMot persistence layer.
It results in an amazing speed increase, in some cases.

Here are some of the optimizations how took place in the source code trunk:

Overall, I observed from x2 to x10 performance boost with simple Add() operations, using ODBC, OleDB and direct Oracle access, when compare to previous benchmarks (which were already impressive).
BATCH mode performance is less impacted, since it by-passed some of those limitations, but even in this operation mode, there is some benefits (especially with ODBC and OleDB).

Here are some results, directly generated by the supplied "15 - External DB performance" sample.

Continue reading

2013-01-18

Register any class for proper TObjectList serialization

Up to now, the only way of directly serializing a list of class instances as a JSON array was to use a TCollection.

In fact, objects are not alone, just like mORMots tend to have a nice family:

You have either to let your collection class inherit from the new TInterfaceCollection type, either call the TJSONSerializer.RegisterCollectionForJSON() method.
Could sounds a bit over-sized for just a list of objects.

We have just added a new feature, adding a new "ClassName":"TMyObject" field in the JSON object serialization, and allowing it to create the proper class instance on both transmission sides, therefore able to properly let TObjectList be serialized and un-serialized.

Continue reading

Register any TCollection type for proper JSON serialization

Due to the current implementation pattern of the TCollection type in Delphi, it was not possible to implement directly this kind of parameter.

In fact, the TCollection constructor is defined as such:

 constructor Create(ItemClass: TCollectionItemClass);

And, on the server side, we do not know which kind of TCollectionItemClass is to be passed. Therefore, the TServiceFactoryServer is unable to properly instantiate the object instances, supplying the expected item class.

The framework propose two potential solutions:

  • You can let your collection class inherit from the new TInterfaceCollection type;
  • You can call the TJSONSerializer.RegisterCollectionForJSON() method to register the collection type and its associated item class.

First solution has already been detailed in this blog.
We will now describe the second (and new) way.

Continue reading

2013-01-05

Domain-Driven-Design and mORMot

Implementing Domain-Driven-Design (DDD) is one goal of our mORMot framework.

We already presented this particular n-Tier architecture.

It is now time to enter deeper into the material, provide some definition and reference.
You can also search the web for reference, or look at the official web site.
A general presentation of the corresponding concepts, in the .NET world, was used as reference of this blog entry.

Stay tuned, and ride the mORMot!

Continue reading

2012-12-20

How to make it fast?

On our forum, a clever question was posted about publishing some enhanced RTL functions for newer versions of Delphi - as we did for Delphi 7 and 2007.

I was looking for a faster IntToStr implementation and discovered SynCommons.pas.
(....)
That's really too bad, SynCommons.pas really does contain some seriously fast stuff, people would greatly benefit from it if it was made general-purpose.

In fact, it would not be enough to change the RTL function implementations.
IMHO, to write something scalable, you need to get rid of such functions.

Continue reading

2012-11-28

Synopse PDF Engine 1.18

Our SynPdf library was released as part of every mORMot version. But the stand-alone .zip was still in 1.15 revision. I have updated it, and now it reflects the latest version from our source code repository (i.e. 1.18). Open Source, working from Delphi 5 up to XE3. Thanks to nice proposals - like  […]

Continue reading

Breaking change in mORmot: SQLite3*.pas units renamed mORMot*.pas

All former SQLite3\SQLite3*.pas units have been renamed to SQLite3\mORMot*.pas to match the database-agnostic scheme of the mORMot framework. This is a major break change, so all your "uses" clauses in your code is to be change to follow the new naming. See this commit, which includes  […]

Continue reading

2012-10-28

SynDBOracle: Open Source native Oracle access

(this is an update of the article published in 2011/07)

For our mORMot framework, and in completion to our SynOleDB unit, we added a new Open Source unit, named SynDBOracle. It allows direct access to any remote Oracle server, using the Oracle Call Interface.

Oracle Call Interface (OCI) is the most comprehensive, high performance, native unmanaged interface to the Oracle Database that exposes the full power of the Oracle Database. We wrote a direct call of the oci.dll library, using our DB abstraction classes introduced for SynOleDB.

We tried to implement all best-practice patterns detailed in the official Building High Performance Drivers for Oracle document

Resulting speed is quite impressive: for all requests, SynDBOracle is 3 to 5 times faster than a SynOleDB connection using the native OleDB Provider supplied by Oracle. We noted also that our implementation is 10 times faster than the one provided with ZEOS/ZDBC, which is far from optimized.

You can use the latest version of the Oracle Instant Client provided by Oracle - see this link - which allows you to run your applications without installing the standard (huge) Oracle client or having an ORACLE_HOME. Just deliver the dll files in the same directory than your application, and it will work.

Continue reading

2012-10-18

Interfaces are not evil; or are Delphi-ers the new Vampires?

A very interesting comment by mpv in our forum highlighted some points about potential interface (ab)use:

IMHO: Idea is good, but "the devil is in the details". To use mocking I must use interfaces. When I use interfaces I lost control on code, because I don't see implementation. Debugging an optimization became very hard. Especially if a beginner developer read something like GOF (Gang Of Four) and wherever necessary and where not use design templates like Visitor, Decorator and so on, and in debugging I don't understand at all what class actually implement passed interface. As for me, this is a biggest problem for .NET framework - developers use interfaces, don't look on implementation (and often don't have it in sources at all), do not learn by reading someone else's code and therefore produce monkey-code. This is only IMHO...

Could sounds rude, and like a trolling subject, but I perfectly understand this point of view.
Introducing stubs and mocks in mORMot was not the open door to all problems.. but,on the contrary, to help write robust, efficient, and maintainable code.
It does not mean that using interfaces and C#/Java is the root of all evils and code inefficiency, but that it may lead into problems.

Continue reading

2012-10-14

Advanced mocks and stubs

Let's see some advanced topics about mORMot's mocking and stubbing features:

  • How to handle complex values in parameters / arguments or results, like record;
  • Stubbing via a custom delegate or callback;
  • Calls tracing.

Continue reading

Interfaces in practice: dependency injection, stubs and mocks

In order to fulfill the SOLID principles, two features are to be available when handling interfaces:

  • Dependency injection; 
  • Stubbing and mocking of interfaces for proper testing.

We will show now how mORMot provides all needed features for such patterns, testing a simple "forgot my password" scenario: a password shall be computed for a given user name, then transmitted via SMS, and its record shall be updated in the database.

Continue reading

Stubs and Mocks for Delphi with mORMot

Our mORMot framework is now able to stub or mock any Delphi interface.

As usual, the best way to explain what a library does is to look at the code using it.
Here is an example (similar to the one shipped with RhinoMocks) of verifying that when we execute the "forgot my password" scenario, we remembered to call the Save() method properly:

procedure TMyTest.ForgotMyPassword;
var SmsSender: ISmsSender;
    UserRepository: IUserRepository;
begin
  TInterfaceStub.Create(TypeInfo(ISmsSender),SmsSender).
    Returns('Send',[true]);
  TInterfaceMock.Create(TypeInfo(IUserRepository),UserRepository,self).
    ExpectsCount('Save',qoEqualTo,1);
  with TLoginController.Create(UserRepository,SmsSender) do
  try
    ForgotMyPassword('toto');
  finally
    Free;
  end;
end;

And... that's all, since the verification will take place when IUserRepository instance will be release.

If you want to follow the "test spy" pattern (i.e. no expectation defined a priori, but manual check after the execution), you can use:

procedure TMyTest.ForgotMyPassword;
var SmsSender: ISmsSender;
    UserRepository: IUserRepository;
    Spy: TInterfaceMockSpy;
begin
  TInterfaceStub.Create(TypeInfo(ISmsSender),SmsSender).
    Returns('Send',[true]);
  Spy := TInterfaceMockSpy.Create(TypeInfo(IUserRepository),UserRepository,self);
  with TLoginController.Create(UserRepository,SmsSender) do
  try
    ForgotMyPassword('toto');
  finally
    Free;
  end;
  Spy.Verify('Save');
end;

This is something unique with our library: you can decide if you want to use the classic "expect-run-verify" pattern, or the somewhat more direct "run-verify" / "test spy" pattern.
With mORMot, you pick up your mocking class (either TInterfaceMock or TInterfaceMockSpy), then use it as intended. You can even mix the two aspects in the same instance! It is just a matter of taste and opportunity for you to use the right pattern.

Continue reading

2012-10-06

Delphi XE3 is preparing (weak) reference counting for class instances

In Delphi, you have several ways of handling data life time, therefore several ways of handling memory:

  • For simple value objects (e.g.  byte integer double shortstring and fixed size arrays or record containing only such types), the value is copied in fixed-size buffers;
  • For more complex value objets (e.g. string and dynamic arrays or record containing such types), there is a reference counter handled by each instance, with copy-on-write feature and compiler-generated reference counting at code scope level (with hidden try..finally blocks);
  • For most class instances (e.g. deriving from TObject), you have to Create then Free each instance, and manage its life time by hand - with explicit try..finally blocks;
  • For class deriving from TInterfacedObject, you have a RefCount property, with _AddRef _Release methods (this is the reference-counted COM model), and you can use Delphi interface to work with such instances - see this blog article.
With Delphi XE3, we were told that some automatic memory handling at class level are about to be introduced at the compiler and RTL level.
Even if this feature is not finished, and disabled, there are a lot of changes in the Delphi XE3 Run Time Library which sounds like a preparation of such a new feature.

Continue reading

2012-10-03

Today's sugar: "stored AS_UNIQUE" syntax

The limited RTTI available in earlier versions of Delphi we want to support (starting with Delphi 6/7) lacks of attributes.
Even the attribute feature of newer Delphi version is not compatible with the one exposed by the FreePascalCompiler, we also want to support.

Therefore, our ORM expects unique columns in a TSQLRecord published property to be defined as stored false.
It could be misleading at first, as reported by several users.
In order to avoid any confusion, we just added a new constant named AS_UNIQUE.

Continue reading

2012-09-14

Updated mORMot benchmarks on another HW configuration

With a refreshed hardware, and the latest code modifications of the framework code, I run again the benchmark sample.

The new PC has an Intel Core i7-2600 CPU, running on Windows Seven 64-bit, with anti-virus (Norton) fully enabled.
Oracle 11g database is remotely accessed over a corporate network, so latency and bandwidth is not optimal.
Still no SSD, but a standard 7200 rpm hard drive of 500 GB.

The results are impressive, when compared to the previous run using a Core 2 Duo CPU - mORMot's optimized code achieves amazing speed on this platform.
And I guess the 8MB of L3 cache of the Core i7 does wonders with our code.

Continue reading

2012-09-10

Don't be confused by our little mORMot !

We got some interesting feedback in reddit.

I looked at your website and it is a bit confusing to be honest. After browsing for five minutes I still can't figure out what it is that this framework is supposed to do. It seems like a strange mish mash of different unrelated libraries. You've got some client-server stuff. Some SQLite stuff, which doesn't really fit with client server stuff as it is not a server database. Then there is some PDF stuff.
What is the big picture? What does this actually do?

Such confusion does make sense. Our web site is split into forum, blog, source-code repository and tickets, some wiki pages.

Here are some points of orientation.

Continue reading

2012-09-09

Synopse mORMot framework 1.17

Our Open Source mORMot framework is now available in revision 1.17.

The main new features are the following:

We have some very exciting features on the road-map for the next 1.18 release, like direct Event/CallBacks handling.
Stay tuned!

Continue reading

- page 4 of 7 -