How would be 2015 like for our little rodents?
Due to popular request of several users of mORMot, we identified and
designed some feature requests dedicated to BigData process.
In fact, your data is the new value, especially if you propose
SaaS (Software As A Service)
hosting to your customers, with a farm of mORMot
servers.
Recent Linux support for mORMot servers, together with the
high performance and installation ease of our executable, open the gate to
cheap cloud-based hosting.
As a consequence, a lot of information would certainly be gathered by
your mORMot servers, and a single monolithic database
is not an option any more.
For mORMot solutions hosted in cloud, a lot of data may be generated. The default SQLite3 storage engine may be less convenient, once it reaches some GB of file content. Backup becomes to be slow and inefficient, and hosting this oldest data in the main DB, probably stored on an expensive SSD, may be a lost of resource. Vertical scaling is limited by hardware and price factors.
This is were data sharding comes into scene.
Note that sharding is not replication/backup, nor clustering, nor just
spreading. We are speaking about application-level data splitting, to ease
maintenance and horizontal scalability of mORMot servers.
Data sharding could already be implemented with mORMot servers,
thanks to TSQLRestStorage:
- Using
TSQLRestStorageExternal: any table may have its own external SQL database engine, may be in its separated DB server; - Using
TSQLRestStorageMongoDB: any table may be stored on a MongoDB cluster, with its own sharding abilities; - Using
TSQLRestStorageRemote: each table may have its own remote ORM/REST server.
But when data stored in a single table tends to grow without limit, this
feature is not enough.
Let's see how the close future of mORMot looks like.
