Key generation – who’s really responsible?

When we think about cryptographic keys we tend to think about closely guarded secrets. Keys are the only thing that separates the attacker from your encrypted data. Some keys really are treated with the appropriate level of respect. Those of you in the payments industry or those that have deployed a PKI, know all too well about the importance of auditing key management processes – some cases of which are no less than full-blown ceremonies.

But I’m focused here on all of the other keys: the billions of keys that are created on the fly, automatically, every second. The ones used in SSL, SSH, file and disk encryption and a thousand other applications. How are they created and who is responsible for making sure that they are good enough to do their job? How do we make sure that the generation of these keys isn’t taken for granted?

When I talk about keys being ‘good enough’, what I mean is, are they truly random? When keys are less than perfectly random they start to become predictable, and predictability is the enemy of all cryptography – it makes the attacker’s job a lot easier.

So, where do these random numbers come from? In almost all cases they are software-generated. The trouble is that software only does what it’s programmed to do; it doesn’t do random things.

Ironically, erratic behavior is normally called a bug. To trigger behavior that is actually random, the software normally scavenges randomness (more properly called entropy) from wherever it can, ideally by sampling some aspect of its physical environment. Entropy can come from many sources, some better (more random) than others. Everything from user mouse clicks, to video signals to timing jitter in the hardware can all yield entropy.

The trouble is that capturing entropy and converting it into statistically random numbers (equal numbers of independent ones and zeros) is not easy. For this reason, few software developers write their random number generators. Instead they use shared services provided by the operating system – for example, one of the most widely used is dev/urandom in Linux. Of course this now means that all applications on the same host compete for the single supply of shared randomness. It now becomes the operating system’s job to gather sufficient entropy to meet that needs.

What becomes clear is that random number generation spans the entire stack from hardware to OS to application, and very often with a virtualization layer spliced in between. The various layers are often ‘owned’ by different people or teams. They are designed and often deployed independently, which raises the question, “Who owns the job of making sure that random numbers are done right?”

The hardware guys have no idea what applications will run on any given box or how much entropy each will require. The OS has no idea how many random numbers will be required or how to prioritize individual applications (you’d like to believe the crypto apps get the best random numbers). And the applications have no idea if they are getting what they asked for or have the ability to raise alarms if they don’t.

The reality is that at each successive layer in the stack makes the assumption is that everything below (the hardware, the OS etc.) is doing its job in creating, capturing and processing entropy. Worse still, the measurements for assessing the quality of entropy and randomness are notoriously unreliable and so in practice there’s no easy way to find out if the various other layers are doing their job. The end result is that the application makes keys but no one can attest to their quality – either in real-time or retrospectively.

It would be nice to think that the security team will save the day. After all, it is their job to take a holistic view. But is that realistic? How many security teams know the specifics of how individual applications are designed and what randomness services are employed? How can they possibly know how commercial software or security appliances work at that level of detail? Could a CISO ever answer the question of how many VMs are running at any point in time, never mind what proportion of them are satisfying the entropy demands of their crypto apps? How many organizations have a policy about such apparently mundane tasks as generating random numbers?

Actually, some really do. They might require product security certifications such as FIPS 140, which includes RNG requirements, and a subset of these invest in dedicated devices such as hardware security modules (HSMs). But now we are in the territory of those special, regulated applications I mentioned at the beginning.

If we return to the mainstream – the millions of SSL stacks whirring away across the datacenter, the SSH keys generated on almost every system, the corporate web of VPNs – we need a generic solution, a solution that deals with random number generation and entropy on a grand scale. It will soon be hard to find an application that doesn’t need random numbers and most will need crypto strength randomness. Entropy sourcing and random number generation shouldn’t be left to individual boxes and VMs to do the best they can. It should be independent of the platform and environment.

Poor random number generation is a basic hygiene issue and it should be addressed through a utility, as a standard of due care.

Crypto – good, bad or something in-between?

When we think about cryptography, we tend to think in black and white. Cryptography feels definitive, binary, clear-cut. Data is either encrypted or it isn’t, readable or not – useful or useless. Digital signatures either validate or they don’t – messages are authentic or not. And digital certificates can be authenticated or they can’t. You get my point, there’s no middle ground – data isn’t half encrypted – there’s no shades of grey.

Regulators jumped on this apparent clarity. Things that are true or false can be easily written into requirements and laws and are easily audited. Think about data breach disclosure laws – the obligation to tell people when you lose their data. In almost all cases only one exemption is allowed – if the data in encrypted then you are off the hook. The presumption is that even though the data was lost its useless to anyone who finds it or stole it, so there’s no problem.

But few things in life are so cut and dry. Other than life and death, most things hover somewhere between good and bad, or working and broken. In the world of IT security, we see shades of grey everywhere; Malware might be spotted, firewalls might stop attacks, logs might get examined and alarms might be responded to. Almost all security tools are situational, far too mushy to be embodied in laws and regulation – instead they land in the vague world of best practice. The bad news is that crypto is no different.

But hang-on – there really aren’t that many choices with crypto, right? We all know the standard list of algorithms (AES, RSA, EEC etc.). It’s a pretty short list, who can name more than 5? Anyone that strays from the list is asking for trouble and it only changes once every 10 years or so – it really isn’t that hard to track. Then there’s key size, but again there’s not much room for choice. They normally come in convenient powers of 2 and there’s only a few viable options when you factor in performance impact and the guidance from bodies such as NIST. So where’s the shades of grey? It’s all about the keys.

In reality there is far more scope for disastrous decisions when it comes to key management. This is when crypto actually does become binary. If the attacker gets the key, the game is up. Security evaporates in an instant – no half measures. Sure, some keys might only yield a single message, but some are used for years, protect enormous volumes of data or guard intellectual property that can bring down a whole company it ever exposed.

Key management is more about processes and people than algorithms and standards and this is where the shades of grey creep in. Attackers can steal keys, control keys, guess keys or calculate keys. Each of these threats deserves a blog all to itself but at a high level I’ll make a grand statement: stealing and controlling keys gets harder over time while guessing and calculating keys gets easier over time. Let me explain. We ought to be able to agree that our enormous investment in security tools and education every year really should be making it harder to steal keys (if not, why are we bothering?). But, on the other hand, calculating keys must inevitably get easier over time if only because computers get faster and attackers get smarter – that’s why we periodically increase key length. This is also the reason for the paranoia about quantum computers, which, it turns out are likely to be very good at calculating keys.

All of these threats to keys reduce the effective security of crypto systems. Even though you may be using AES with keys that each have 256 bit doesn’t mean you have 256 bits of “effective security.” If those keys aren’t perfectly random or if they have been exposed then your effective security could be way lower and, what’s even more important, you would never know. Just because your car has four wheels doesn’t make it safe to drive!

The current debate about the use of encryption by terrorists is another good example of how law enforcement agencies would really like the ability to reduce the effective security of encryption – without changing the algorithm or key length. The challenge of course is to guarantee that the backdoor is perfectly discriminating when it comes to who can exploit the weaker effective security.

Backdoors or not, we’re already starting to see a steady trickle of attacks against keys and it can only get worse. For years, there has been plenty of unencrypted data for attackers to steal, so why bother trying to crack the encrypted stuff. Going forward, even the low hanging fruit will be encrypted. Sometime soon attackers will be faced with a simple decision: go and get a proper job or try to figure out how to crack crypto – my guess is they will chose the latter. The big question is whether they will be successful? The days of taking crypto for granted are over. Key management is an essential organizational process and the tools to generate and protect keys represent critical security infrastructure and should be treated as such.

Whitewood is a subsidiary of Allied Minds Federal Innovations, the division of Allied Minds dedicated to commercializing U.S.federal intellectual property. Allied Minds is an innovative U.S. science and technology development and commercialization company. Operating since 2006, Allied Minds forms, funds, manages and builds products and businesses based on innovative technologies developed at leading U.S. universities and federal research institutions. Allied Minds serves as a diversified holding company that supports its businesses and product development with capital, central management and shared services. More information about the Boston-based company can be found at