February 2014 (3)

2014-02-28

Are NoSQL databases ACID?

One of the main features you may miss when discovering NoSQL ("Not-Only SQL"?) databases, coming from a RDBMS background, is ACID.

ACID (Atomicity, Consistency, Isolation, Durability) is a set of properties that guarantee that database transactions are processed reliably. In the context of databases, a single logical operation on the data is called a transaction. For example, a transfer of funds from one bank account to another, even involving multiple changes such as debiting one account and crediting another, is a single transaction. (Wikipedia)

But are there any ACID NoSQL database?

Please ensure you read the Martin Fowler introduction about NoSQL databases.
And the corresponding video.

First of all, we can distinguish two types of NoSQL databases:

  1. Aggregate-oriented databases;
  2. Graph-oriented databases (e.g. Neo4J).

By design, most Graph-oriented databases are ACID!
This is a first good point.

Then, what about the other type?
In Aggregate-oriented databases, we can identify three sub-types:

  • Document-based NoSQL databases (e.g. MongoDB, CouchDB);
  • Key/Value NoSQL databases (e.g. Redis);
  • Column family NoSQL databases (e.g. Cassandra).
Whatever document/key/column oriented they are, they all use some kind of document storage.
It may be schema-less, blob-stored, column-driven, but it is always some set of values bound together to be persisted.
This set of values define a particular state of one entity, in a given model.
Which we may call Aggregate.

Continue reading

Think free as in free speech, not free beer

After more than 5 years of opening some huge part of Delphi code base, just my two cents.

Free software means free as a bird.

In practice, most Open Source "consumers" focus on free as a free beer...
This is a reality, especially for "niche" projects like developing libraries for Delphi.

Here are some thoughts from my little experiment with mORMot.
If you ask what Open Source for libraries mean, it may help you!

Continue reading

2014-02-25

TDocVariant custom variant type

With revision 1.18 of the framework, we just introduced two new custom types of variants:

  • TDocVariant kind of variant;
  • TBSONVariant kind of variant.

The second custom type (which handles MongoDB-specific extensions - like ObjectID or other specific types like dates or binary) will be presented later, when dealing with MongoDB support in mORMot, together with the BSON kind of content. BSON / MongoDB support is implemented in the SynMongoDB.pas unit.

We will now focus on TDocVariant itself, which is a generic container of JSON-like objects or arrays.
This custom variant type is implemented in SynCommons.pas unit, so is ready to be used everywhere in your code, even without any link to the mORMot ORM kernel, or MongoDB.

TDocVariant documents

TDocVariant implements a custom variant type which can be used to store any JSON/BSON document-based content, i.e. either:

  • Name/value pairs, for object-oriented documents;
  • An array of values (including nested documents), for array-oriented documents;
  • Any combination of the two, by nesting TDocVariant instances.

Here are the main features of this custom variant type:

  • DOM approach of any object or array documents;
  • Perfect storage for dynamic value-objects content, with a schema-less approach (as you may be used to in scripting languages like Python or JavaScript);
  • Allow nested documents, with no depth limitation but the available memory;
  • Assignment can be either per-value (default, safest but slower when containing a lot of nested data), or per-reference (immediate reference-counted assignment);
  • Very fast JSON serialization / un-serialization with support of MongoDB-like extended syntax;
  • Access to properties in code, via late-binding (including almost no speed penalty due to our VCL hack as already detailed);
  • Direct access to the internal variant names and values arrays from code, by trans-typing into a TDocVariantData record;
  • Instance life-time is managed by the compiler (like any other variant type), without the need to use interfaces or explicit try..finally blocks;
  • Optimized to use as little memory and CPU resource as possible (in contrast to most other libraries, it does not allocate one class instance per node, but rely on pre-allocated arrays);
  • Opened to extension of any content storage - for instance, it will perfectly integrate with BSON serialization and custom MongoDB types (ObjectID, RegEx...), to be used in conjunction with MongoDB servers;
  • Perfectly integrated with our Dynamic array wrapper and its JSON serialization as with the record serialization;
  • Designed to work with our mORMot ORM: any TSQLRecord instance containing such variant custom types as published properties will be recognized by the ORM core, and work as expected with any database back-end (storing the content as JSON in a TEXT column);
  • Designed to work with our mORMot SOA: any interface-based service is able to consume or publish such kind of content, as variant kind of parameters;
  • Fully integrated with the Delphi IDE: any variant instance will be displayed as JSON in the IDE debugger, making it very convenient to work with.

To create instances of such variant, you can use some easy-to-remember functions:

  • _Obj() _ObjFast() global functions to create a variant object document;
  • _Arr() _ArrFast() global functions to create a variant array document;
  • _Json() _JsonFast() _JsonFmt() _JsonFastFmt() global functions to create any variant object or array document from JSON, supplied either with standard or MongoDB-extended syntax.

Continue reading