Weaknesses in the Java JCEKS Keystore

Graham Steel
June 9, 2016

When strong cryptography was introduced into Java, the legacy JKS keystore with its "SHA-1 and XOR" encryption method was replaced by JCEKS, which uses Triple-DES (3DES) encryption to protect serialized keys when they are written to disk.There is a lot of JCEKS still around. So how exactly does the encryption work?

Like JKS, JECKS uses password-based encryption. Before creating a key, the JCEKS key derivation routine creates a random 64-bit salt. If the first and second half of the salt are the same, the first half is inverted.This is already pretty odd, but will be explained (to an extent) when we look at how the two halves of the salt are used. For now let's just look at the code that does the inversion of the halves:

// if the 2 salt halves are the same, invert one of them
        int i;
        for (i=0; i<4; i++) {
            if (salt[i] != salt[i+4])
                break;
        }
        if (i==4) { // same, invert 1st half
            for (i=0; i<2; i++) {
                byte tmp = salt[i];
                salt[i] = salt[3-i];
                salt[3-1] = tmp;
            }
        }
        


See anything strange? Yes 3-1, not 3-i. What's the effect of this typo? It means that halves aren't really inverted. The salt 12341234 becomes 41241234 instead of 43211234. This bug is present in Oracle Java and OpenJDK.

Weird bug, but it doesn't matter: the purpose is to make the two halves different. This is because of what comes next: the first half is attached to the password and hashed (using MD5) c times to produce the first 128 bits of the 3DES key. The second half is then attached to the same password and hashed c times to produce the last 64 bits of the 3DES key and the 64 bit IV to be used to encrypt the key using 3DES in CBC mode.

Why did the designers want to prevent the two salt halves from being the same? Remember that the salt will be stored in plaintext, so a would-be attacker can easily access it. Suppose he realises that the salt has two identical halves. Then he knows that the first and third 64-bits of the 3DES key are equal, and the IV is equal to the second 64-bits of the key.

Knowing these things are the same reduces the search space, but in practice it is very unlikely that this will make brute-forcing the key easier than a dictionary attack on the password. What is more interesting is that in the Oracle Java JCEKS, that constant c for the number of iterations is just 20. Note also that in the JCEKS scheme, the two calculations for the two key halves can be carried out in parallel, since the result of one does not depend on the result of the other.

While the "right" number of iterations is highly application-dependent, as a guide, most recommendations propose about 10000 iterations. Additionally, to counteract parallelized and specialist hardware-based attacks, it is considered prudent to use a hash function that requires considerable memory for each derivation. MD5 is certainly not such a function. Better alternatives are either PBKDF2-HMAC-SHA512, if you are restricted to NIST-approved schemes, or something like scrypt if not.

Conclusions

JCEKS is an improvement over the original JKS, but it cannot be considered secure: 20 iterations of MD5 is not considered a secure password-based key derivation method against today's attackers. Find out more about alternatives in our Java crypto security whitepaper.