Synopse Open Source

Authentication and Authorization

Here we cover the authentication part of User security: i.e. the proof of "who" a user is.
Authorization is another part - not detailed here - which is "what" a user can do (defining e.g. groups and execution policies, mainly at SOA interface level for mORMot).

Here, we assume we are on a safe client/server connection, i.e. that HTTPS is used over the wire. This is a mandatory requirement for any secure authentication.

Modular Crypt Hashes

Since mORMot 1, the user passwords were stored in the TAuthUser.PasswordHashHexa field as hexadecimal SHA-256 of 'salt' + password. Yes, I know, plain SHA-256 with a fixed salt - not very safe.
On demand, it could be stored as the hexadecimal result of PBKDF2-SHA256 with a specific salt and number of rounds, to be supplied when hashing the password on the client side - a bit safer, but not very convenient, because the client code should explicitly know those parameters.

Enter MCF. The Modular Crypt Format (MCF) is a standard for encoding password hash strings, following a simple $identifier$params$salt$checksum text format.

• $identifier indicates the algorithm to be used,
• then $params contains the associated parameters (e.g. number of rounds, various processing sizes),
• $salt contains a random salt (as expected for safe password storage at rest),
• and finally $checksum contain the actual hash value, in base-64 encoding (standard or with some variants).

In practice, those layouts came from the Unix crypt hashing utility, which is used to compute and store user password on most UNIX systems in a system text file. They were later adopted under the name "Modular Crypt" in most libraries, e.g. the great Python passlib module, which is a reference in the cryptographic world.

Typical values are:

$1$3azHgidD$SrJPt7B.9rekpmwJwtON31
$2a$06$m0CrhHm10qJ3lXRY.5zDGO3rS2KdeeWLuGmsfGlMfOxih58VYVfxe
$5$rounds=12345$q3hvJE5mn5jKRsW.$BbbYTFiaImz9rTy03GGi.Jf9YY5bmxN0LU3p3uI1iUB
$6$rounds=1400$anotherlongsalts$POfYwTEok97VWcjxIiSOjiykti.o/pQs..
$pbkdf2$1212$OB.dtnSEXZK8U5cgxU/GYQ$y5LKPOplRmok7CZp/aqVDVg8zGI
$pbkdf2-sha256$1212$4vjV83LKPjQzk31VI4E0Vw$hsYF68OiOUPdDZ1Fg.fJPeq1h...
$pbkdf2-sha512$1212$RHY0Fr3IDMSVO/RSZyb5ow$eNLfBK.eVozomMr.1gYa...
$pbkdf2-sha3$1000$G85lPNdJLXoDVzhbCmsBCA$T6UjUUihUTmnYpwiRbhH8yi...
$scrypt$ln=4,r=8,p=1$QNx4N454ppMeKmDjxyrhsh7Q/PYBQw$zeGG+tsAueRzkvXfE1/F58KOKFEFfI0KpBYwE/3ZUWg
$bcrypt-sha256$v=2,t=2b,r=12$n79VH.0Q2TMWmt3Oqt9uku$Kq4Noyk3094Y2QlB8NdRT8SvGiI4ft2

Under the hood, the patterns may be:

• $1${salt}${checksum} for md5crypt
• $2a${rounds}${salt}{checksum} for BCrypt
• $bcrypt-sha256$v=2,t=2b,r={rounds}${checksum} for BCrypt with a SHA-256 pre-hash
• $5$rounds={rounds}${salt}${checksum} for sha256crypt
• $6$rounds={rounds}${salt}${checksum} for sha512crypt
• $scrypt$ln={log2(N)},r={blocksize},p={parallelism}${salt}${checksum} for SCrypt
• $pbkdf2{-digest}${rounds}${salt}${checksum} for all PBKDF2 variants

Our REST client/server stack can now use this MCF format for storing its password. When a client connects, it will receive the expected layout, i.e. $identifier$params$salt$ without the checksum part, so that it could compute the checksum value from the password supplied by the end user. With strong algorithms like bcrypt or scrypt (more on them later), it could take half a second to compute the hash on the client side. So brute force attacks are more difficult. And the random $salt makes pre-computed "rainbow tables" attacks ineffective.

For instance, a client could ask the authentication nonce as such, to retrieve the server nonce in "result" and the Modular Crypt expected parameters in "mcf", for the given "User" logon name:

GET /root/auth?username=User&mcf=1
{"result":"daee2e059d9159857eb72ca4264c57f86b043a6b7c0971e6acaf1a3609858b88","mcf":"$1$3azHgidD$"}}

Now the client will hash the supplied plain password using $1$3azHgidD$ parameters, and send the expected authentication challenge to the server.

As direct consequence, if you store such values in the TAuthUser.PasswordHashHexa field instead of a "naive" SHA-256 hexa, you will benefit from all those new strong algorithms, with no specific code needed on the client side. Just use the latest version of mORMot, and its client side will adapt to the expected algorithm and parametrers. You may use a strong algorithm for an admin user, or a weaker and fast algorithm for a guest user. SCrypt could consume e.g. up to 1GB of RAM during its process, and take several seconds, if needed, to compute its digest.

Note that the default parameters of mORMot are tuned to be safe. So if you pickup any of our MCF type of hashing, you will have a safe enough process, expecting around 100-200ms of process time needed on modern hardware. Only mcfMd5Crypt is not safe, because it has no parameter - it could be used to obfuscate a guest or service user, for instance.

We may wonder if providing the username could not be a way of quickly identify actual logon names, via fuzzying. To mitigate this, we also added a new rsoNoUnknownUserResponse option in TRestServer.Options. When enabled, the client can't tell if the authentication failed because of a wrong password, or an unknown user name. And in our context, the server will create a fake but consistent "mcf" value - via ModularCryptFakeInfo() for any requested user, as response to any GET /root/auth?username=.. request.

New Strong Algorithms

As we have seen above, we supported PBKDF2 over SHA1/SHA2/SHA3 since years. But those kind of hashes are computed too quickly on modern hardware, therefore a huge number of rounds should be applied (200,000 rounds for instance) to make the process slow enough. But even slower, such algorithms are easy to implemented on GPU or dedicated FPGA. It was clear to us, that some new algorithms should be introduced.

Our toolbox now supports BCrypt and SCrypt, which are the most known and proven password hashing algorithm around. We considered Argon2, which is newer, but it is more complex, and in practice, with the same time of computation, even plain old BCrypt has the same level of complexity. And it is pretty easy to make Argon2 weak, with some non proper parameters. So we did not include it. If you want something really secure, use e.g. SCrypt with LogN=20, R=8, P=1 to consume 1GB of RAM during 1300ms on client side.
Note that the server side is not affected by the hashing algorithm used on client side. The slowest and more secure algorithm on the client side could take seconds to compute. But on the server side, it will take as few microseconds as it does with any other algorithm, since only the same few SHA-256 operations are involved - even with SCRAM. This the beauty of those challenges.

With the upcoming mORMot 2.4, we will introduce some new algorithms:

• $5$ and $6$ as sha256crypt and sha512crypt are just extensions of the good old $1$ md5crypt algorithm, with proper parameters.
• $2 as "BCrypt" is an historical algorithm. It was defined last century, in 1999, and requires a lot of expensive calculations to compute its hashed result. It is a blowfish algorithm on steroids, to burn as many CPU as requests by its "cost" parameter.
• $scrypt$ as "SCrypt" is another password key derivation function, created in 2009. In respect to BCrypt, it was designed to consume as much memory as defined, during its process. By default, mORMot will use 64MB for the process, but you could tune its parameters to consume e.g. 1GB of RAM. This makes hardware implementation much harder, even impossible on a FPGA. For its actual basic algorithm, it uses a 8-round version of the Salsa20 cipher, from well known security expert Daniel Bernstein.

About performance, we can note that OpenSSL does not implement BCrypt, and that our own SSE2 assembly version of SCrypt is 30 to 40% faster than the one offered by OpenSSL.

Now mORMot 2 offers all those algorithms, most in mormot.crypt.core.pas and mormot.crypt.secure.pas - and in the less common mormot.crypt.other.pas unit for BCrypt and SCrypt. We started from scratch, and did not rely on existing (much slower) pascal implementations.

Regular mORMot Authentication Challenge

Since mORMot 1, the server side used the following conditional expression to check the client challenge:

 result := PropNameEquals(ClientProof, Sha256U([Server.Model.Root, ServerNonce,
             ClientNonce, User.LogonName, User.PasswordHashHexa]));

One problem with this simple pattern is that, if the server database passwords are leaked (TAuthUser.PasswordHashHexa values), then an attacker could use them to log into the server, with a custom client class. The attacker could not know the password, but he/she would know the hash of the password used to compute the SHA-256 proof needed to authenticate.
This is one of the issues SCRAM was designed to fix.

SCRAM Authentication

Salted Challenge Response Authentication Mechanism (SCRAM) is a family of modern, password-based challenge–response authentication mechanisms. As part of the SASL standard, most sensitive services, like LDAP, IMAP, MongoDB or PostgreSQL, rely on it for their authentication.

As defined by RFC 5802, the following computations are involved:

     SaltedPassword  := Hi(Normalize(password), salt, i)
     ClientKey       := HMAC(SaltedPassword, "Client Key")
     StoredKey       := H(ClientKey)
     AuthMessage     := client-first-message-bare + "," +
                        server-first-message + "," +
                        client-final-message-without-proof
     ClientSignature := HMAC(StoredKey, AuthMessage)
     ClientProof     := ClientKey XOR ClientSignature
     ServerKey       := HMAC(SaltedPassword, "Server Key")
     ServerSignature := HMAC(ServerKey, AuthMessage)
     ServerProof     := ServerKey XOR ClientSignature

Here, Hi() is a PBKDF2 key derivation function, and H() HMAC() use SHA-256.

In respect to our "naive" mORMot 1 server-side challenge, it features:

• Irreversible password storage on server;
• Mutual authentication.

For the first new security feature - safe password hash persistence on the server, the trick is in StoredKey := H(ClientKey).
It means that the client side will compute a proof not directly from StoredKey, but from this transient ClientKey, which is not known by the server. As a consequence, if StoredKey is leaked, there is mathematically no way of computing this ClientProof value back, because there is no way of knowing ClientKey from StoredKey := H(ClientKey).

Mutual authentication indicates that both the client and server sides do prove that they are what they are supposed to be.
The server will check that H(ClientProof XOR ClientSignature) = StoredKey, as we have seen.
But the client will also check that ServerProof XOR ClientSignature = ServerKey, so that it could trust this server.

mORMot 2.4: Introducing SCRAM-MCF

In respect to the standard SCRAM version, we used an extended version, with two major improvements:

• We don't only use PBKDF2 as Hi() KDF, but any "Modular Crypt" hashes, e.g. more secure BCrypt or SCrypt;
• and ClientKey and ServerKey are computed as HMAC(SaltedPassword, User + "Client Key") and HMAC(SaltedPassword, User + "Server Key"), meaning their value would be tied to the User logon name. As a consequence, they can't be swapped on server side.

Fun fact: during our exploration of this SCRAM scheme, we identified that some blog articles of so-called security experts did propagate wrong information. Some are clearly generated by a weak AI engine. Others are just confused by the algorithm itself. Even the RFC is not very clear, e.g. not stating clearly how the ServerProof should be checked. So don't try to re-implement SCRAM by yourself. As far as I can tell, mORMot is the only REST library for Delphi and FPC featuring a SCRAM authentication scheme. And I was not able to find any other REST library in any language or runtime featuring SCRAM-MCF, i.e. combining SCRAM with hash algorithms other than PBKDF2-SHA, like BCrypt or SCrypt. Once again, object pascal is ahead of its competitors! :-)

Last but not least, mORMot allows channel binding during its handshake, as the most secure SCRAM implementations offer.
In practice, the server nonce could be tied to the actual TCP connection used by the client. If rsoPerConnectionNonce is defined in TRestServer.Options, then the raw connection ID will be used to make the server nonce unique per connection/client.

About server side storage, we just use the MCF prefix ($identifier$params$salt$), then append both 256-bit StoredKey and ServerKey, into a single 512-bit base-64 encoded blob, which replaces the regular MCF checksum. The only change is that the MCF identifier starts with a # and not a $ character, to indicate it is not a true MCF, but a SCRAM-MCF encoded password.

Typical values stored in TAuthUser.PasswordHashHexa could be:

#1$W$VYaNCqPtxbEzfA1X7yd_lraYtYusuve5XeFGNm3YbMldOIcLvrV2aCpgoT8NQ-O9...
#2b$06$1HUcnAPbb1HYgAGcnxMTYuAHLRNjFXQ__Lfipm-UiY9UiWD6qwCIoi...
#scrypt$ln=5,r=8,p=5$Oj1je16b6EweZ2EqnZRNiQ$y3CUmau_esKFWatYcAq...

Note the initial # before each value.

Over the wire, the only change is that the "mcf" value transmitted back with the server nonce will also start with a #:

GET /root/auth?username=User&mcf=1
{"result":"daee2e059d9159857eb72ca4264c57f86b043a6b7c0971e6acaf1a3609858b88","mcf":"#1$3azHgidD$"}}

Then the client will silently adapt its authentication challenge algorithm to follow SCRAM-MCF expectations.
No change in end-user code, all is transparently handled by mORMot.

Conclusion

Thanks to these SCRAM-MCF new algorithms, we are confident that our little mORMot toolbox could be safely used for the next years to come.
To use this feature, just re-hash your passwords using ScramPersistedKey() from mormot.crypt.secure.pas and you are done. :-)

You could look at our algorithm and technical discussion in the corresponding mORMot 2 issue.
Nothing fancy was used during the process. So you could implement a client side in most languages, using some proper cryptographic library, and basic HTTP + JSON encoding.

Any feedback is welcome on our forum, as usual!