Synopse

To content | To menu | To search

Tag - Documentation

Entries feed

2016, Tuesday October 18

Cheat Mode for Private Keys

In order to follow best practice, our .private key files are always protected by a password.  A random value with enough length and entropy is always proposed by the ECC tool when a key pair is generated, and could be used directly.
It is always preferred to trust a computer to create true randomness (and SynCrypto.pas's secure TAESPRNG was designed to be the best possible seed, using hardware entropy if available), than using our human brain, which could be defeated by dictionary-based password attacks.
Brute force cracking would be almost impossible, since PBKDF2_HMAC_SHA256 Password-Based Key Derivation Function with 60,000 rounds is used, so rainbow tables (i.e. pre-computed passwords list) will be inoperative, and each password trial would take more time than with a regular Key Derivation Function.

The issue with strong passwords is that they are difficult to remember. If you use not pure random passwords, but some easier to remember values with good entropy, you may try some tools like https://xkpasswd.net/s which returns values like $$19*wrong*DRIVE*read*61$$.
But even then, you will be able to remember only a dozen of such passwords. In a typical public key infrastructure, you may create hundredths of keys, so remembering all passwords is no option for an average human being as (you and) me.

At the end, you end up with using a tool to store all your passwords (last trend is to use an online service with browser integration), or - admit it - store them in an Excel document protected by a password. Most IT people - and even security specialists - end with using such a mean of storage, just because they need it.
The weaknesses of such solutions can be listed:

  • How could we trust closed source software and third-party online services?
  • Even open source like http://keepass.info/help/base/security.html may appear weak (no PBKDF, no AFSplit, managed C#, SHA as PRNG);
  • The storage is as safe as the "master password" is safe;
  • If the "master password" is compromised, all your passwords are published;
  • You need to know the master password to add a new item to the store.

The ECC tool is able to work in "cheat mode", storing all .private key files generated passwords in an associated .cheat local file, encrypted using a cheat.public key.

As a result:

  • Each key pair will have its own associated .cheat file, so you only unleash one key at a time;
  • The .cheat file content is meaningless without the cheat.private key and its master password, so you can manage and store them together with your .private files;
  • Only the cheat.public key is needed when creating a key pair, so you won't leak your master password, and even could generate keys in an automated way, on a distant server;
  • The cheat.private key will be safely stored in a separated place, only needed when you need to recover a password;
  • It uses strong File Encryption, with proven PBKDF, AFSplit, AES-PRNG, and ECDH/ECIES algorithms.

Continue reading...

2016, Saturday September 24

Public-key Asymmetric Cryptography via SynECC

After weeks of implementation and testing, we introduce today a new feature of our mORMot Open-Source Framework.

Asymmetric encryption, also known as public-key cryptography, uses pairs of keys:

  • Public keys that may be disseminated widely;
  • Paired with private keys which are known only to the owner.

The framework SynEcc unit features a full asymmetric encryption system, based on Elliptic curve cryptography (ECC), which may be used at application level (i.e. to protect your application data, by signing or encrypting it), or at transmission level (to enhance communication safety).
A full set of high-level features, including certificates and command line tool, offers a stand-alone but complete public-key infrastructure (PKI).

Continue reading...

2016, Friday April 22

Support of Delphi 10.1 Berlin

You should have noticed that Delphi 10.1 Berlin has been released.

Our Open Source projects, including mORMot and SynPDF and their associated documentation have been updated to support this new revision.
Any additional feedback is welcome, as usual!

2015, Friday December 11

Audit Trail for Services

We have seen previously how the ORM part of the framework is able to provide an Audit Trail for change tracking.
It is a very convenient way of storing the change of state of the data.

On the other side, in any modern SOA solution, data is not at the center any more, but services.
Sometimes, the data is not stored within your server, but in a third-party Service-Oriented Architecture (SOA).
Being able to monitor the service execution of the whole system becomes sooner or later mandatory.

Our framework allows to create an Audit Trail of any incoming or outgoing service operation, in a secure, efficient and automated way.

Continue reading...

2015, Tuesday November 17

Benefits of interface callbacks instead of class messages

If you compare with existing client/server SOA solutions (in Delphi, Java, C# or even in Go or other frameworks), mORMot's interface-based callback mechanism sounds pretty unique and easy to work with.

Most Events Oriented solutions do use a set of dedicated messages to propagate the events, with a centralized Message Bus (like MSMQ or JMS), or a P2P/decentralized approach (see e.g. ZeroMQ or NanoMsg). In practice, you are expected to define one class per message, the class fields being the message values. You would define e.g. one class to notify a successful process, and another class to notify an error. SOA services would eventually tend to be defined by a huge number of individual classes, with the temptation of re-using existing classes in several contexts.

Our interface-based approach allows to gather all events:

  • In a single interface type per notification, i.e. probably per service operation;
  • With one method per event;
  • Using method parameters defining the event values.

Since asynchronous notifications are needed most of the time, method parameters would be one-way, i.e. defined only as const - in such case, an evolved algorithm would transparently gather those outgoing messages, to enhance scalability when processing such asynchronous events. Blocking request may also be defined as var/out, as we will see below, inWorkflow adaptation.

Behind the scene, the framework would still transmit raw messages over IP sockets (currently over a WebSockets connection), like other systems, but events notification would benefit from using interfaces, on both server and client sides.
We will now see how...

Continue reading...

2015, Tuesday July 14

New blog about mORMot

An enthusiastic mORMot user, named willo in the forum, just started a blog about his experiments with our framework.

The information there is clear, simple, and right to the point.
If you are a little lost in our huge documentation, it is a good place to start!

Continue reading...

2015, Tuesday June 16

Handling Cross-Platform Time Zones

One common problem when handling dates and times, is that time is shown and entered as local, whereas the computer should better use non-geographic information - especially on a Client-Server architecture, where both ends may not be on the same physical region.

A time zone is a region that observes a uniform standard time for legal, commercial, and social purposes.
Time zones tend to follow the boundaries of countries and their subdivisions because it is convenient for areas in close commercial or other communication to keep the same time.
Most of the time zones on land are offset from Coordinated Universal Time (UTC) by a whole number of hours, or minutes.
Even worse, some countries use daylight saving time for part of the year, typically by changing clocks by an hour, twice every year.

The main rule is that any date and time stored should be stored in UTC, or with an explicit Zone identifier (i.e. an explicit offset to the UTC value).
Our framework expects this behavior: every date/time value stored and handled by the ORM, SOA, or any other part of it, is expected to be UTC-encoded.
At presentation layer (e.g. the User Interface), conversion to/from local times should take place, so that the end-user is provided with friendly clock-wall compatible timing.

As you may guess, handling time zones is a complex task, which should be managed by the Operating System itself.
Since this cultural material is constantly involving, it is updated as part of the OS.

In practice, current local time could be converted from UTC from the current system-wide time zone. One of the only parameters you have to set when installing an Operating System is to pickup the keyboard layout... and the current time zone to be used. But in a client-server environment, you may have to manage several time zones on the server side: so you can't rely on this global setting.

One sad - but predictable - news is that there is no common way of encoding time zone information.
Under Windows, the registry contains a list of time zones, and the associated time bias data. Most POSIX systems (including Linux and Mac OSX) do rely on the IANA database, also called tzdata - you may have noticed that this particular package is often updated with your system.
Both zone identifiers do not map, so our framework needed something to be shared on all systems.

Continue reading...

2015, Monday June 1

Updated Slides about ORM SOA MVC SOLID DDD

One year ago, we published a set of slides about the main concepts implemented by our framework.
Mainly about ORM (and ODM), NoSQL, JSON, SOA, MVC (and MVVM), SOLID, DDD, CQRS and some patterns like Stubs, Mocks, Factory, Repository, Unit-Of-Work.
Worth a look, if you want to find out the benefits of the latest software development techniques.
They try to open the landscape of any Delphi developer (probably with a mostly RAD and OOP background) to some new areas.

I just updated the slides from our public GoogleDrive folder.
They now reflect the latest state of the framework (e.g. ORM real-time synchronization, asynchronous callbacks, DDD CQRS services...).
They have also been polished after several public presentations, since I used them as base for trainings I made for some European companies.

If you want to go further, or have some more enlightenment, ensure you took a look at our framework Documentation, which would detail all those patterns, and how mORMot may help implementing them for your projects!

Feedback is welcome on our forum, as usual!

2015, Monday May 18

CQRS Persistence Service of any DDD object with mORMot

We introduced DDD concepts some time ago, in a series of articles in this blog.
At that time, we proposed a simple way of using mORMot types to implement DDD in your applications.
But all Domain Entitities being tied to the framework TSQLRecord class did appear as a limitation, breaking the Persistence Ignorance principle, since it couples the DDD objects to the framework implementation details.

We introduced a new mORMotDDD.pas unit, which is able to easily create CQRS Persistence services for any plain Delphi class (the famous PODOs - Plain Old Delphi Objects).
No need to inherit from TSQLRecord, or pollute your class definition with attributes!

For instance, a TUser class may be persisted via such a service:

type
  IDomUserCommand = interface(IDomUserQuery)
    ['{D345854F-7337-4006-B324-5D635FBED312}']
    function Add(const aAggregate: TUser): TCQRSResult;
    function Update(const aUpdatedAggregate: TUser): TCQRSResult;
    function Delete: TCQRSResult;
    function Commit: TCQRSResult;
  end;

Here, the write operations are defined in a IDomUserCommand service, which is separated (but inherits) from IDomUserQuery, which is used for read operations.
Separating reads and writes is a powerful pattern also known as CQRS, i.e. Command Query Responsibility Segregation, which we followed when defining our persistence services.
The framework make it pretty easy to create such services for storing any kind of class type in any SQL or NoSQL engine, with almost no code to write.
Last but not least, using such interface-based services for data persistence will allow to stub or mock the data access layer, making unit testing straightforward: you would not fear to write TDD code any more!

Please refer to our updated documentation for this unique and powerful feature.
You may take a look at the corresponding dddDomUserTypes.pas, dddDomUserCQRS.pas, and dddInfraRepoUser.pas units, detailed as sample reference.
Feedback is welcome in our forum, as usual!

2015, Sunday May 3

SOLID Design Principles

I've just updated the documentation part about the SOLID Design Principles.
The former blog article (almost 4 years old!) sounds like a bit deprecated now...
This is why I would extract here an updated version of this material.

Ensure you checked the corresponding part of the mORMot documentation, which is the updated reference, and probably the easiest to read - including links to all the other documentation.

The acronym SOLID is derived from the following OOP principles (quoted from the corresponding Wikipedia article):

  • Single responsibility principle: the notion that an object should have only a single responsibility;
  • Open/closed principle: the notion that "software entities ... should be open for extension, but closed for modification";
  • Liskov substitution principle: the notion that "objects in a program should be replaceable with instances of their subtypes without altering the correctness of that program” - also named as "design by contract";
  • Interface segregation principle: the notion that "many client specific interfaces are better than one general purpose interface.";
  • Dependency inversion principle: the notion that one should "Depend upon Abstractions. Do not depend upon concretions.". Dependency injection is one method of following this principle, which is also called Inversion Of Control (aka IoC).

If you have some programming skills, those principles are general statements you may already found out by yourself. If you start doing serious object-oriented coding, those principles are best-practice guidelines you would gain following.

They certainly help to fight the three main code weaknesses:

  • Rigidity: Hard to change something because every change affects too many other parts of the system;
  • Fragility: When you make a change, unexpected parts of the system break;
  • Immobility: Hard to reuse in another application because it cannot be disentangled from the current application.

Continue reading...

2015, Sunday April 12

Why Transmitting Exceptions in SOA services is not a good idea

Usually, in Delphi application (like in most high-level languages), errors are handled via exceptions. By default, any Exception raised on the server side, within an interface-based service method, will be intercepted, and transmitted as an error to the client side, then a safe but somewhat obfuscated EInterfaceFactoryException will be raised on the client side, containing additional information serialized as JSON.

You may wonder why exceptions are not transmitted and raised directly on the client side, with our mORMot framework interface-based services, as if they were executed locally.

We will now detail some arguments, and patterns to be followed.

Continue reading...

2015, Monday April 6

Asynchronous Service - WebSockets, Callbacks and Publish-Subscribe

When publishing SOA services, most of them are defined as stateless, in a typical query/answer pattern - see Service-Oriented Architecture (SOA).
This fits exactly with the RESTful approach of Client-Server services via interfaces, as proposed by the framework.

But it may happen that a client application (or service) needs to know the state of a given service. In a pure stateless implementation, it will have to query the server for any state change, i.e. for any pending notification - this is called polling.

Polling may take place for instance:

  • When a time consuming work is to be processed on the server side. In this case, the client could not wait for it to be finished, without raising a timeout on the HTTP connection: as a workaround, the client may start the work, then ask for its progress status regularly using a timer and a dedicated method call;
  • When an unpredictable event is to be notified from the server side. In this case, the client should ask regularly (using a timer, e.g. every second), for any pending event, then react on purpose.

It may therefore sounds preferred, and in some case necessary, to have the ability to let the server notify one or several clients without any prior query, nor having the requirement of a client-side timer:

  • Polling may be pretty resource consuming on both client and server sides, and add some unwanted latency;
  • If immediate notification is needed, some kind of "long polling" algorithm may take place, i.e. the server will wait for a long time before returning the notification state if no event did happen: in this case, a dedicated connection is required, in addition to the REST one;
  • In an event-driven systems, a lot of messages are sent to the clients: a proper publish/subscribe mechanism is preferred, otherwise the complexity of polling methods may increase and become inefficient and unmaintainable;
  • Explicit push notifications may be necessary, e.g. when a lot of potential events, associated with a complex set of parameters, are likely to be sent by the client.

Our mORMot framework is therefore able to easily implement asynchronous callbacks over WebSockets, defining the callbacks as interface parameters in service method definitions - see Available types for methods parameters.

Continue reading...

Real-Time ORM Master/Slave Replication via WebSockets

In a previous article, we presented how Master/Slave replication may be easily implemented in mORMot's RESTful ORM.
Do not forget to visit the corresponding paragraphs of our online documentation, which has been updated, and is more accurate!

Sometimes, the on-demand synchronization is not enough.
So we have just introduced real-time replication via WebSockets.
For instance, you may need to:

  • Synchronize a short list of always evolving items which should be reflected as soon as possible;
  • Involve some kind of ACID-like behavior (e.g. handle money!) in your replicated data;
  • Replicate not from a GUI application, but from a service, so use of a TTimer is not an option;
  • Combine REST requests (for ORM or services) and master/slave ORM replication on the same wire, e.g. in a multi-threaded application.

In this case, the framework is able to use WebSockets and asynchronous callbacks to let the master/slave replication - see Asynchronous callbacks - take place without the need to ask explicitly for pending data.
You would need to use TSQLRestServer.RecordVersionSynchronizeMasterStart, TSQLRestServer.RecordVersionSynchronizeSlaveStart and TSQLRestServer.RecordVersionSynchronizeSlaveStop methods over the proper kind of bidirectional connection.

Continue reading...

2015, Tuesday March 31

ORM Master/Slave Replication

As stated during TSQLRecord fields definition, the ORM is able to maintain a revision number for any TSQLRecord table, so that it the table may be easily synchronized remotely by another TSQLRestServer instance.
If you define a TRecordVersion published property, the ORM core will fill this field just before any write with a monotonically increasing revision number, and will take care of any deletion, so that those modifications may be replayed later on any other database.

This synchronization will work as a strict master/slave replication scheme, as a one-way on demand refresh of a replicated table.
Each write operation on the master database on a given table may be easily reflected on one or several slave databases, with almost no speed nor storage size penalty.

Continue reading...

2015, Tuesday March 17

Framework Documentation Enhanced By Links

The mORMot framework documentation, in its HTML online form, has been enhanced to include links to almost of the code symbols.

In fact, the latest version of our SynProject tool will search for code symbols (types, methods, constants, functions): 

Some minor cosmetic changes did also occur, especially in the API Reference.
We hope it would help you discover and work with out little mORMot!

2015, Saturday January 10

mORMot under Linux thanks to FPC

You can use the FreePascal Compiler (FPC) to compile the mORMot framework source code, targetting Windows and Linux.

Linux is a premium target for cheap and efficient server Hosting. Since mORMot has no dependency, installing a new mORMot server is as easy as copying its executable on a blank Linux host, then run it. No need to install any framework nor runtime. You could even use diverse operating systems (several Linux or Windows Server versions) in your mORMot servers farm, with minimal system requirements, and updates.

We will now see how to write your software with Linux-compiling in mind, and also give some notes about how to install a Linux Virtual Machine with Lazarus on your Windows computer, compiling both FPC and Lazarus from their SVN latest sources!

Continue reading...

2014, Sunday November 23

Breaking change: New SynLog and SynTests unit extracted from SynCommons.pas

In order to enhance code modularity, we extracted logging and testing features from SynCommons.pas.
Discover the new SynLog.pas and SynTests.pas units!
Documentation has been updated to reflect the changes.

This is a breaking change...
Ensure you add SynLog and/or SynTests to your uses clauses, just after SynCommons, if you use those logging or testing features in your programs.

The corresponding commit is this one.

Next code refactoring would probably be about extracting some features from mORMot.pas.
Stay tuned!

Feedback is welcome on our forum, as usual.

2014, Thursday November 20

BeDelphi 2014 Slides

We just finished our Be-Delphi 2014 sessions and drank our last beers, so here we are.

I published some slides for this great event.

Continue reading...

2014, Friday November 14

BREAKING CHANGE - TSQLRecord.ID primary key changed to TID: Int64

Up to now, the TSQLRecord.ID property was defined in mORMot.pas as a plain PtrInt/NativeInt (i.e. Integer under Win32), since it was type-cast as pointer for TSQLRecord published properties.
We introduced a new TID type, so that the ORM primary key would now be defined as Int64.

All the framework ORM process relies on the TSQLRecord class.
This abstract TSQLRecord class features a lot of built-in methods, convenient to do most of the ORM process in a generic way, at record level.

It first defines a primary key field, defined as ID: TID, i.e. as Int64 in mORMot.pas:

type
  TID = type Int64;
  ...
  TSQLRecord = class(TObject)
  ...
    property ID: TID read GetID write fID;
  ...

In fact, our ORM relies now on a Int64 primary key, matching the SQLite3 ID/RowID primary key.
This primary key will be used as RESTful resource identifier, for all CRUD operations.

Continue reading...

Automatic TSQLRecord memory handling

Working with objects is pretty powerful, but requires to handle manually the created instances life time, via try .. finally blocks. Most of the time, the TSQLRecord life time would be very short: we allocate one instance on a local variable, then release it when it goes out of scope.

If we take again the TSQLBaby sample, we may write:

function NewMaleBaby(Client: TSQLRest; const Name,Address: RawUTF8): TID;
var Baby: TSQLBaby;   // store a record
begin
  Baby := TSQLBaby.Create;
  try
    Baby.Name := Name;
    Baby.Address := Address;
    Baby.BirthDate := Date;
    Baby.Sex := sMale;
    result := Client.Add(Baby);
  finally
    Baby.Free;
  end;
end;

To ease this pretty usual pattern, the framework offers some kind of automatic memory management at TSQLRecord level:

function NewMaleBaby(Client: TSQLRest; const Name,Address: RawUTF8): TID;
var Baby: TSQLBaby;   // store a record
begin
  TSQLBaby.AutoFree(Baby);  // no try..finally needed!
  Baby.Name := Name;
  Baby.Address := Address;
  Baby.BirthDate := Date;
  Baby.Sex := sMale;
  result := Client.Add(Baby);
end; // local Baby instance will be released here

Continue reading...

- page 1 of 4