Interface based services - Implementation details

Low level client implementation

For technical information about how interfaces are called in Delphi, see this nice article and the FakeCall method implementation.

Here is the core of this client-side implementation of the "call stubs":

  for i := 0 to fMethodsCount-1 do begin
    fFakeVTable[i+RESERVED_VTABLE_SLOTS] := P;
    P^ := $68ec8b55; inc(P);                 // push ebp; mov ebp,esp
    P^ := i; inc(P);                         // push {MethodIndex}
    P^ := $e2895251; inc(P);                 // push ecx; push edx; mov edx,esp
    PByte(P)^ := $e8; inc(PByte(P));         // call FakeCall
    P^ := PtrUInt(@TInterfacedObjectFake.FakeCall)-PtrUInt(P)-4; inc(P);
    P^ := $c25dec89; inc(P);                 // mov esp,ebp; pop ebp
    P^ := fMethods[i].ArgsSizeInStack or $900000;  // ret {StackSize}; nop
    inc(PByte(P),3);
  end;

Just for fun... :)
I could not resist posting this code here; if you are curious, take a look at the "official" RTTI.pas or RIO.pas units as provided by Embarcadero, and you will probably find out that the mORMot implementation is much easier to follow, and also faster (it does not recreate all the stubs or virtual tables at each call, for instance).

Security

As stated above, in the features grid, a complete security pattern is available when using client-server services. In a 17, securing messages between clients and services is essential to protecting data.

Security is implemented at several levels:
- For communication stream - e.g. when using HTTPS protocol, or a custom cypher within HTTP content-encoding;
- At RESTful / URI authentication level - see 18; introducing Group and User notions;
- At interface or method (service/operation) level - we'll discuss this part now.

By default, all services and operations (i.e. all interfaces and methods) are allowed to execution.

Then, on the server side (it's an implementation detail), the TServiceFactoryServer instance (available from TSQLRestServer.Services property) provides the following methods to change the security policy for each interface:
- AllowAll() and Allow() to enable methods execution globally;
- DenyAll() and Deny() to disable methods execution globally;
- AllowAllByID() and AllowByID() to enable methods execution by Group IDs;
- DenyAllByID() and DenyByID() to disable methods execution by Group IDs;
- AllowAllByName() and AllowByName() to enable methods execution by Group names;
- DenyAllByName() and DenyByName() to disable methods execution by Group names.

The first four methods will affect everybody. The next *ByID() four methods accept a list of authentication Group IDs (i.e. TSQLAuthGroup.ID values), where as the *ByName() methods will handle TSQLAuthGroup.Ident property values.

In fact, the execution can be authorized for a particular group of authenticated users.
Your service can therefore provide some basic features, and then enables advanced features for administrators or supervisors only.
Since the User / Group policy is fully customizable in our RESTful authentication scheme, mORMot provides a versatile and inter-operable security pattern.

Here is some extract of the supplied regression tests:

 (...)
  S := fClient.Server.Services['Calculator'] as TServiceFactoryServer;
  Test([1,2,3,4,5],'by default, all methods are allowed');
  S.AllowAll;
  Test([1,2,3,4,5],'AllowAll should change nothing');
  S.DenyAll;
  Test([],'DenyAll will reset all settings');
  S.AllowAll;
  Test([1,2,3,4,5],'back to full acccess for everybody');
  S.DenyAllByID([GroupID]);
  Test([],'our current user shall be denied');
  S.AllowAll;
  Test([1,2,3,4,5],'restore allowed for everybody');
  S.DenyAllByID([GroupID+1]);
  Test([1,2,3,4,5],'this group ID won''t affect the current user');
  S.DenyByID(['Add'],GroupID);
  Test([2,3,4,5],'exclude a specific method for the current user');
  S.DenyByID(['totext'],GroupID);
  Test([2,3,5],'exclude another method for the current user');
 (...)

The Test() procedure is used to validate the corresponding methods of ICalculator (1=Add, 2=Multiply, 3=Subtract, 4=ToText...).

In this above code, the GroupID value is retrieved as such:

  GroupID := fClient.MainFieldID(TSQLAuthGroup,'User');

And the current authenticated user is member of the 'User' group:

  fClient.SetUser('User','synopse'); // default user for Security tests

Since TSQLRestServer.ServiceRegister method returns the first created TServiceFactoryServer instance, and since all Allow* / AllowAll* / Deny* / DenyAll* methods return also a TServiceFactoryServer instance, you can use some kind of "fluent interface" in your code to set the security policy, as such:

  Server.ServiceRegister(TServiceCalculator,[TypeInfo(ICalculator)],sicShared).
    DenyAll.AllowAllByName(['Supervisor']);

This will allow access to the ICalculator methods only for the Supervisor group of users.

Transmission content

All data is transmitted as JSON arrays or objects, according to the requested URI.

We'll discuss how data is expected to be transmitted, at the application level.

Request format

As stated above, there are two mode of routing, defined by TServiceRoutingMode. The routing to be used is defined globally in the TSQLRest.Routing property.

`rmREST`	`rmJSON_RPC`
Description	URI-based layout	JSON-RPC mode
Default	Yes	No
URI scheme	/Model/Interface.Method[/ClientDrivenID]	/Model/Interface
Body content	JSON array of parameters	`"method":"MethodName","params":[...][,"id":ClientDrivenID]`
Security	RESTful authentication for each method	RESTful authentication for the whole service (interface)
Speed	10% faster	10% slower

In the default rmREST mode, both service and operation (i.e. interface and method) are identified within the URI. And the message body is a standard JSON array of the supplied parameters (i.e. all const and var parameters).

Here is a typical request for ICalculator.Add:

 POST /root/Calculator.Add
(...)
[1,2]

For a sicClientDriven mode service, the needed instance ID is appended to the URI:

 POST /root/ComplexNumber.Add/1234
(...)
[20,30]

Here, 1234 is the identifier of the server-side instance ID, which is used to track the instance life-time, in sicClientDriven mode.

One benefit of using URI is that it will be more secure in our RESTful authentication scheme - see 18: each method (and even any client driven session ID) will be signed properly.

In this rmREST mode, the server is also able to retrieve the parameters from the URI, if the message body is left void. This is not used from a Delphi client (since it will be more complex and therefore slower), but it can be used for a client, if needed:

 POST root/Calculator.Add?+%5B+1%2C2+%5D

In the above line, +%5B+1%2C2+%5D will be decoded as [1,2] on the server side.

If rmJSON_RPC mode is used, the URI will define the interface, and then the method name will be inlined with parameters, e.g.

 POST /root/Calculator
(...)
{"method":"Add","params":[1,2],"id":0}

Here, the "id" field can be not set (and even not existing), since it has no purpose in sicShared mode.

For a sicClientDriven mode service:

 POST /root/ComplexNumber
(...)
{"method":"Add","params":[20,30],"id":1234}

This mode will be a little bit slower, but will probably be more AJAX ready. It's up to you to select the right routing scheme to be used.

Response format

The framework will always return the data in the same format, whatever the routing mode used.

Basically, this is a JSON object, with one nested "result": property, and the client driven "id": value (e.g. always 0 in sicShared mode):

 POST /root/Calculator.Add
(...)
[1,2]

will be answered as such:

 {"result":[3],"id":0}

The result array contains all var and out parameters values (in their declaration order), and then the method main result.

For instance, here is a transmission stream for a ICalculator.ComplexCall request in rmREST mode:

 POST root/Calculator.ComplexCall
(...)
[[288722014,1231886296], ["one","two","three"], ["ABC","DEF","GHIJK"], "ïƒ¿BgAAAAAAAAAAAAAAAAAAACNEOlxEZXZcbGliXFNRTGl0ZTNcZXhlXFRlc3RTUUwzLmV4ZQ==", "ïƒ¿Xow1EdkXbUkDYWJj"]

will be answered as such:

 '{"result":[ ["ABC","DEF","GHIJK","one,two,three"], "ïƒ¿X4w1EdgXbUkUMjg4NzIyMDE0LDEyMzE4ODYyOTY=", "ïƒ¿Xow1EdkXbUkjRDpcRGV2XGxpYlxTUUxpdGUzXGV4ZVxUZXN0U1FMMy5leGU="], "id":0}'

It matches the var / const / out parameters declaration of the method:

 function ComplexCall(const Ints: TIntegerDynArray; Strs1: TRawUTF8DynArray;
   var Str2: TWideStringDynArray; const Rec1: TVirtualTableModuleProperties;
   var Rec2: TSQLRestCacheEntryValue): TSQLRestCacheEntryValue;

And its implementation:

function TServiceCalculator.ComplexCall(const Ints: TIntegerDynArray;
  Strs1: TRawUTF8DynArray; var Str2: TWideStringDynArray; const Rec1: TVirtualTableModuleProperties;
  var Rec2: TSQLRestCacheEntryValue): TSQLRestCacheEntryValue;
var i: integer;
begin
  result := Rec2;
  result.JSON := StringToUTF8(Rec1.FileExtension);
  i := length(Str2);
  SetLength(Str2,i+1);
  Str2[i] := UTF8ToWideString(RawUTF8ArrayToCSV(Strs1));
  inc(Rec2.ID);
  dec(Rec2.TimeStamp);
  Rec2.JSON := IntegerDynArrayToCSV(Ints,length(Ints));
end;

Note that TIntegerDynArray, TRawUTF8DynArray and TWideStringDynArray values were marshaled as JSON arrays, whereas complex records (like TSQLRestCacheEntryValue) have been Base-64 encoded.

The framework is able to handle class instances as parameters, for instance with the following interface, using a TPersistent child class with published properties (it would be the same for TSQLRecord ORM instances):

type
  TComplexNumber = class(TPersistent)
  private
    fReal: Double;
    fImaginary: Double;
  public
    constructor Create(aReal, aImaginary: double); reintroduce;
  published
    property Real: Double read fReal write fReal;
    property Imaginary: Double read fImaginary write fImaginary;
  end;

  IComplexCalculator = interface(ICalculator)
    ['{8D0F3839-056B-4488-A616-986CF8D4DEB7}']
    /// purpose of this unique method is to substract two complex numbers
    // - using class instances as parameters
    procedure Substract(n1,n2: TComplexNumber; out Result: TComplexNumber);
  end;
As stated above, it is not possible to return a class as a result of a function (who will be responsible of handling its life-time?). So in this method declaration, the result is declared as out parameter.

During the transmission, published properties of TComplexNumber parameters will be serialized as standard JSON objects:

 POST root/ComplexCalculator.Substract
(...)
[{"Real":2,"Imaginary":3},{"Real":20,"Imaginary":30}]

will be answered as such:

 {"result":[{"Real":-18,"Imaginary":-27}],"id":0}

Those content have perfectly standard JSON declarations, so can be generated and consumed directly in any AJAX client.

In case of an error, the standard message object will be returned:

{
"ErrorCode":400,
"ErrorText":"Error description"
}

The following error descriptions may be returned by the service implementation from the server side:

`ErrorText`	Description
Method name required	`rmJSON_RPC` call without `"method":` field
Unknown method	`rmJSON_RPC` call with invalid method name (in `rmRest` mode, there is no specific message, since it may be a valid request)
Parameters required	The server expect at least a void JSON array (aka `[]`) as parameters
Unauthorized method	This method is not allowed with the current authenticated user group - see Security above
... instance id:? not found or deprecated	The supplied `"id":` parameter points to a wrong instance (in `sicPerSession / sicPerUser / sicPerGroup` mode)
ExceptionClass: Exception Message (with `500 Internal Server Error`)	An exception was raised during method execution

Continue reading...

This article is part of a list of other blog articles, extracted from the official Synopse mORMot framework documentation:

Feedback and questions are welcome on our forum, just as usual.

Synopse Open Source