"Sharding" or "Share nothing" architecture
By A.Bouchez on 2011, Sunday July 3, 20:12 - SQLite3 Framework - Permalink
Here is what wikipedia states at http://en.wikipedia.org/wiki/Shared_nothing_architecture:
A shared nothing architecture (SN) is a distributed computing architecture in which each node is independent and self-sufficient, and there is no single point of contention across the system. People typically contrast SN with systems that keep a large amount of centrally-stored state information, whether in a database, an application server, or any other similar single point of contention.
This is just one approach of "sharding". Sharding is indeed related to a
shared nothing architecture - once sharded, each shard can live in a totally
separate logical schema instance.
"I sharded, therefore it scales"...
You can do this in Delphi... and opens a new world of scaling opportunities... Just as Google, Facebook, or eBay do...
As you already know, in our ORM, high-level types like dynamic
arrays or TPersistent / TCollection properties
are perfectly handled, and stored as BLOB or TEXT inside the main data
row.
There is no external linked table, no Master/Detail to maintain. In
fact, each TSQLRecord instance content could be made
self-contained in our ORM.
Of course, you have the choice to implement the data in the class master/detail scheme. You have direct OOP implementation of one-to-one, one-to-many, or many-to-many relationship.
But when the server starts to have an increasing number of clients, such a "shared nothing" data layout could be a major benefit. In fact, the so-called sharding, or horizontal partitioning of data, is a proven solution for web-scale databases, such as those in use by social networking sites. How does eBay or Facebook scale with so many users? Just by sharding.
A simple but very efficient sharding mechanism could therefore be implemented with our ORM. In-memory databases, or our BigTable component are good candidate for light speed data process. Even SQLite could scale very well in some cases.
Storing detailed data in BLOB or in TEXT as JSON could first sounds a wrong idea. It does break one widely accepted principle of the RDBMS architecture. But even Google had to break this dogma. And when MySQL or such tries to implement sharding, it does need a lot of effort. Others, like the NoSQL MongoDB, are better candidates: they are not tight to the SQL flat scheme.
Therefore, on second thought, having at hand a shared nothing architecture
could be a great advantage. Our ORM is already ready to break the
table-oriented scheme of SQL.
Probably another benefit of the RESTful approach!
Just my Sunday's evening thoughts...
Any
feedback?