DataSnap-like Client-Server JSON RESTful Services in Delphi 6-XE3
By A.Bouchez on 2012, Wednesday December 5, 09:56 - SQLite3 Framework - Permalink
Article update:
The server side call back signature changed since this article was first
published in 2010. Please refer to the documentation or this forum article and
associated commit.
The article was totally rewritten to reflect the enhancements.
And do not forget to see mORMot's interface-based
services!
You certainly knows about the new DataSnap Client-Server features, based on
JSON, introduced in Delphi 2010.
http://docwiki.embarcadero.com/RADStudi
… plications
We added such communication in our mORmot Framework, in a KISS (i.e. simple) way: no expert, no new unit or new class. Just add a published method Server-side, then use easy functions about JSON or URL-parameters to get the request encoded and decoded as expected, on Client-side.
To implement a service in the Synopse mORMot framework, the first method is to define published method Server-side, then use easy functions about JSON or URL-parameters to get the request encoded and decoded as expected, on Client-side.
We'll implement the same example as in the official Embarcadero docwiki page above. Add two numbers. Very useful service, isn't it?
Publishing a service on the server
On the server side, we need to customize the standard
TSQLRestServer class definition (more precisely a
TSQLRestServerDB class which includes a SQlite3 engine,
or a lighter TSQLRestServerFullMemory kind of server, which is
enough for our purpose), by adding a new published method:
type TSQLRestServerTest = class(TSQLRestServerFullMemory) (...) published procedure Sum(var Ctxt: TSQLRestServerCallBackParams); end;
The method name ("Sum") will be used for the URI encoding, and will be
called remotely from ModelRoot/Sum URL.
The ModelRoot is the one defined in the Root parameter of
the model used by the application.
This method, like all Server-side methods, MUST have the same exact
parameter definition as in the TSQLRestServerCallBack prototype,
i.e. only one Ctxt parameter, which refers to the whole
execution context:
type TSQLRestServerCallBack = procedure(var Ctxt: TSQLRestServerCallBackParams) of object;
Then we implement this method:
procedure TSQLRestServerTest.Sum(var Ctxt: TSQLRestServerCallBackParams);
begin
with Ctxt do
Results([Input['a']+Input['b']]);
end;
The Ctxt variable publish some properties named
InputInt[] InputDouble[] InputUTF8[] and Input[] able
to retrieve directly a parameter value from its name, respectively as
Integer/Int64, double, RawUTF8 or
variant.
Therefore, the code above using Input[] will introduce a
conversion via a variant, which may be a bit slower, and in case
of string content, may loose some for older non Unicode versions
of Delphi. So it is a good idea to use the exact expected Input*[]
property corresponding to your value type. It does make sense even more when
handling text, i.e. InputUTF8[] is to be used in such case. For
our floating-point computation method, we may have coded it as such:
procedure TSQLRestServerTest.Sum(var Ctxt: TSQLRestServerCallBackParams);
begin
with Ctxt do
Results([InputDouble['a']+InputDouble['b']]);
end;
The Ctxt.Results([]) method is used to return the service value
as one JSON object with one "Result" member, with default
mime-type JSON_CONTENT_TYPE.
For instance, the following request URI:
GET /root/Sum?a=3.12&b=4.2
will let our server method return the following JSON object:
{"Result":7.32}
That is, a perfectly AJAX-friendly request.
Note that all parameters are expected to be plain case-insensitive
'A'..'Z','0'..'9' characters.
An important point is to remember that the implementation of the
callback method must be thread-safe.
In fact, the TSQLRestServer.URI method expects such callbacks to
handle the thread-safety on their side.
It's perhaps some more work to handle a critical section in the implementation,
but, in practice, it's the best way to achieve performance and scalability: the
resource locking can be made at the tiniest code level.
Defining the client
The client-side is implemented by calling some dedicated methods, and
providing the service name ('sum') and its associated
parameters:
function Sum(aClient: TSQLRestClientURI; a, b: double): double;
var err: integer;
begin
val(aClient.CallBackGetResult('sum',['a',a,'b',b]),Result,err);
end;
You could even implement this method in a dedicated client method - which make sense:
type
TMyClient = class(TSQLHttpClient) // could be TSQLRestClientURINamedPipe
(...)
function Sum(a, b: double): double;
(...)
function TMyClient.Sum(a, b: double): double;
var err: integer;
begin
val(CallBackGetResult('sum',['a',a,'b',b]),Result,err);
end;
This later implementation is to be preferred on real applications.
You have to create the server instance, and the corresponding
TSQLRestClientURI (or TMyClient), with the same
database model, just as usual...
On the Client side, you can use the CallBackGetResult method to
call the service from its name and its expected parameters, or create your own
caller using the UrlEncode() function. Note that you can specify
most class instance into its JSON representation by using some
TObject into the method arguments:
function TMyClient.SumMyObject(a, b: TMyObject): double;
var err: integer;
begin
val(CallBackGetResult('summyobject',['a',a,'b',b]),Result,err);
end;
This Client-Server protocol uses JSON here, as encoded server-side via
Ctxt.Results() method, but you can serve any kind of data, binary,
HTML, whatever... just by overriding the content type on the server with
Ctxt.Returns().
Direct parameter marshalling on server side
We have used above the Ctxt.Input*[] properties to retrieve the
input parameters.
This is pretty easy to use and powerful, but the supplied Ctxt
gives full access to the input and output context.
Here is how we may implement the fastest possible parameters parsing:
procedure TSQLRestServerTest.Sum(var Ctxt: TSQLRestServerCallBackParams);
var a,b: Extended;
if UrlDecodeNeedParameters(Ctxt.Parameters,'A,B') then begin
while Ctxt.Parameters<>nil do begin
UrlDecodeExtended(Ctxt.Parameters,'A=',a);
UrlDecodeExtended(Ctxt.Parameters,'B=',b,@Ctxt.Parameters);
end;
Ctxt.Results([a+b]);
end else
Ctxt.Error('Missing Parameter');
end;
The only not obvious part of this code is the parameters marshaling, i.e.
how the values are retrieved from the incoming Ctxt.Parameters
text buffer, then converted into native local variables.
On the Server side, typical implementation steps are therefore:
- Use the UrlDecodeNeedParameters function to check that all
expected parameters were supplied by the caller in
Ctxt.Parameters;
- Call UrlDecodeInteger / UrlDecodeInt64 / UrlDecodeExtended /
UrlDecodeValue / UrlDecodeObject functions (all defined in
SynCommons.pas) to retrieve each individual parameter from
standard JSON content;
- Implement the service (here it is just the a+b
expression);
- Then return the result calling Ctxt.Results() method or
Ctxt.Error() in case of any error.
The powerful UrlDecodeObject function (defined in
mORMot.pas) can be used to un-serialize most class instance from
its textual JSON representation (TPersistent, TSQLRecord,
TStringList...).
Using Ctxt.Results() will encode the specified values as a JSON
object with one "Result" member, with default mime-type
JSON_CONTENT_TYPE:
{"Result":"OneValue"}
or a JSON object containing an array:
{"Result":["One","two"]}
Using Ctxt.Returns() will let the method return the content in
any format, e.g. as a JSON object (via the overloaded
Ctxt.Returns([]) method expecting field name/value pairs), or any
content, since the returned mime-type can be defined as a parameter to
Ctxt.Returns() - it may be useful to specify another mime-type
than the default constant JSON_CONTENT_TYPE, i.e.
'application/json; charset=UTF-8', and returns plain text, HTML or
binary. For instance, you can return directly a value as plain text as
such:
procedure TSQLRestServer.TimeStamp(var Ctxt: TSQLRestServerCallBackParams); begin Ctxt.Returns(Int64ToUtf8(ServerTimeStamp),HTML_SUCCESS,TEXT_CONTENT_TYPE_HEADER); end;
So you can consume these services, implemented Server-Side in fast Delphi code, with any AJAX application on the client side (if you use HTTP as communication protocol).
Advanced process on server side
On server side, method definition has only one Ctxt parameter,
which has several members at calling time, and publish all service calling
features, including RESTful URI routing, session handling or low-level
HTTP headers (if any).
At first, Ctxt may indicate the expected
TSQLRecord ID and TSQLRecord class, as decoded from
RESTful URI. It means that a service can be related to any table/class
of our ORM framework, so you would be able to create easily any RESTful
compatible requests on URI like ModelRoot/TableName/ID/MethodName.
The ID of the corresponding record is decoded from its RESTful scheme
into Ctxt.ID, and the table is available in
Ctxt.Table or Ctxt.TableIndex (if you need its index
in the associated server Model).
For example, here we return a BLOB field content as hexadecimal, according
to its TableName/Id:
procedure TSQLRestServerTest.DataAsHex(var Ctxt: TSQLRestServerCallBackParams);
var aData: TSQLRawBlob;
begin
if (self=nil) or (Ctxt.Table<>TSQLRecordPeople) or (Ctxt.ID<0) then
Ctxt.Error('Need a valid record and its ID') else
if RetrieveBlob(TSQLRecordPeople,Ctxt.ID,'Data',aData) then
Ctxt.Results([SynCommons.BinToHex(aData)]) else
Ctxt.Error('Impossible to retrieve the Data BLOB field');
end;
A corresponding client method may be:
function TSQLRecordPeople.DataAsHex(aClient: TSQLRestClientURI): RawUTF8;
begin
Result := aClient.CallBackGetResult('DataAsHex',[],RecordClass,fID);
end;
If authentication
is used, the current session, user and group IDs are available in Session
/ SessionUser / SessionGroup fields. If authentication is not available,
those fields are meaningless: in fact, Ctxt.Context.Session will
contain either 0 (CONST_AUTHENTICATION_SESSION_NOT_STARTED) if any
session is not yet started, or 1 (CONST_AUTHENTICATION_NOT_USED)
if authentication mode is not active. Server-side implementation can use the
TSQLRestServer.SessionGetUser method to retrieve the corresponding
user details (note that when using this method, the returned
TSQLAuthUser instance is a local thread-safe copy which shall be
freed when done).
In Ctxt.Call^ member, you can access low-level communication
content, i.e. all incoming and outgoing values, including headers and message
body. Depending on the transmission protocol used, you can retrieve e.g. HTTP
header information. For instance, here is how you can access the caller remote
IP address and client application user agent:
aRemoteIP := FindIniNameValue(pointer(Ctxt.Call.InHead),'REMOTEIP: '); aUserAgent := FindIniNameValue(pointer(Ctxt.Call.InHead),'USER-AGENT: ');
Handling errors
When using Ctxt.Input*[] properties, any missing parameter will
raise an EParsingException.
It will therefore be intercepted by the server process (as any other
exception), and returned to the client with an error message containing the
Exception class name and its associated message.
But you can have full access to the error workflow, if needed.
In fact, calling either Ctxt.Results(),
Ctxt.Returns(), Ctxt.Success() or
Ctxt.Error() will specify the HTTP status code (e.g. 200 / "OK"
for Results() and Success() methods by default, or
400 / "Bad Request" for Error()) as an integer
value.
For instance, here is how a service not returning any content can handle those
status/error codes:
procedure TSQLRestServer.Batch(var Ctxt: TSQLRestServerCallBackParams);
begin
if (Ctxt.Method=mPUT) and RunBatch(nil,nil,Ctxt) then
Ctxt.Success else
Ctxt.Error;
end;
In case of an error on the server side, you may call
Ctxt.Error() method (only the two valid status codes are
200 and 201).
The Ctxt.Error() method has an optional parameter to specify a
custom error message in plain English, which will be returned to the client in
case of an invalid status code. If no custom text is specified, the framework
will return the corresponding generic HTTP status text (e.g. "Bad
Request" for default status code HTML_BADREQUEST =
400).
In this case, the client will receive a corresponding serialized JSON error
object, e.g. for Ctxt.Error('Missing
Parameter',HTML_NOTFOUND):
{
"ErrorCode":404,
"ErrorText":"Missing Parameter"
}
If called from an AJAX client, or a browser, this content should be easy to interpret.
Note that the framework core will catch any exception during the method
execution, and will return a "Internal Server Error" /
HTML_SERVERERROR = 500 error code with the associated textual exception
details.
Benefits and limitations of this implementation
Method-based services allow fast and direct access to all
mORMot Client-Server RESTful features, over all usual
protocols of our framework: HTTP/1.1, Named Pipe, Windows GDI messages, direct
in-memory/in-process access.
The mORMot implementation of method-based services gives full
access to the lowest-level of the framework core, so it has some
advantages:
- It can be tuned to fit any purpose (such as retrieving or returning some HTML
or binary data, or modifying the HTTP headers on the fly);
- It is integrated into the RESTful URI model, so it can be related to any
table/class of our ORM framework (like DataAsHex service above),
or it can handle any remote query (e.g. any AJAX or SOAP requests);
- It has a very low performance overhead, so can be used to reduce server
workload for some common tasks - it is faster
than any other frameworks, including DataSnap.
Note that due to this implementation pattern, the mORMot service implementation is very fast, and not sensitive to the "Hash collision attack" security issue, as reported with Apache - see http://blog.synopse.info/post/2011/12/30/Hash-collision-attack for details.
Interface based services
In real world, especially when your application relies heavily on services,
the method-based implementation pattern has some drawbacks:
- Most content marshaling is to be done by hand, so may introduce
implementation issues;
- Client and server side code does not have the same implementation pattern, so
you will have to code explicitly data marshaling twice, for both client and
server (DataSnap and WCF both suffer from a similar issue, by which
client classes shall be coded separately, most time generated by a
Wizard);
- The services do not have any hierarchy, and are listed as a plain list, which
is not very convenient;
- It is difficult to synchronize several service calls within a single context,
e.g. when a workflow is to be handled during the application process (you have
to code some kind of state machine on both sides, and use all session handling
by hand);
- Security is handled globally for the user, or should be checked by hand in
the implementation method (using the Ctxt members).
You can get rid of those limitations with the interface-based service implementation of mORMot. For a detailed introduction and best practice guide to SOA, you can consult this classic article: http://www.ibm.com/developerworks/webservices/library/ws-soa-design1
According to this document, all expected SOA features are now available in the current implementation of the mORMot framework (including service catalog aka "broker").
See mORMot's interface-based services, which are even more user-friendly and easy to work with than those method-based services.
Full source code is available in our
Source Code Repository.
It should work from Delphi 6 to Delphi XE3.
Feedback is welcome on our forum, as usual.