HSM in the cloud: opportunity or contradiction?

As many other companies, my employer (Worldline) is busy building cloud applications, and moving current applications to the cloud. Of course, cryptography is used in many ways in these applications. Many of these applications use HSMs (Host Security Modules) to improve the security of the cryptographic keys. This is what an HSM looks like:

An HSM: a vault for computations with cryptographic keys.(As you can see if you look closely, this one actually is built by one of Worldline’s companies, but that’s not important here.)

The big question is:

How do we move the way we handle keys with HSMs into the cloud?

The cloud vendors tell you this is easy: just shove in your keys in their HSMs, move your application to use these instead of your own, and off you go!

(Spoiler: this is not the solution, and even worse: I don’t know what is the solution. But at least I can try to explain the problem.)

Of course, it isn’t that easy. I actually prepared a presentation for my colleagues titled “HSM in the cloud”. It was a well supported story explaining that it is impossible to move HSM based applications to the cloud. Fortunately, a few days before giving that presentation, I got Covid, so I couldn’t give it. That was fortunate, because it would have cost me my reputation as a useful consultant.

The reason for my conclusion is that the way HSMs protect keys does not work in the cloud. An HSM is part of key management, and everything about them is strictly regulated. Many of the requirements around HSMs were written in a time of big host computers. Of course, times have changed since then, but the requirements around key management are lagging behind, as you can check for yourself reading the literature (I mean the most common standard documents on physical protection of keys: FIPS PUB 140–2, ISO 13942–2, PCI PIN) Bolting down your HSM in the data center is not going to solve your security problems.

In my quest for the proper way to use HSMs in the cloud, I found that it is actually the wrong question. It is not about HSMs, it is about keys. Let’s start with the “old model”.

The keys in an HSM are managed by dedicated colleagues, called key custodians. Their responsibility is to make sure the keys you need are available to you, and nobody else. In order to do this they have procedures involving secure rooms, tamper proof envelopes, split knowledge, audit trails and whatnot. The key custodians never see the actual value of the keys, and there is no way they can be forced to leak key values. The HSMs contain these keys, and these boxes are then connected via a physically secure connection to an application (remember, in the old days an application ran on a physically separate computer). The application sends commands to the HSM (like “check PIN”) to perform the security relevant parts of the work. Keys are completely invisible to the application.

The way an HSM protects your keys (old view)

Separation of concerns

The point of this construction is one of the fundamental tools of software development: separation of concerns. There are three clearly separated domains in this story:

• handling keys by key custodians;
• use of keys by software in the HSM;
• the application only uses encrypted values.

While everybody is happily moving their applications to the cloud, it becomes clear that this way of separation does not work anymore in the cloud: both the application, the cryptographic calculations, and the keys are all in the domain of the cloud provider. All cloud providers that I have seen give you a “admin key” that allows you to do everything, and they leave it up to you to set up authorization domains that restrict the access of domains to certain people. And of course, they cannot prove that they cannot see your data. Even stronger, even if they have HSMs, they cannot prove to you that they actually use them: you don’t even know where there computers are. The HSM model clearly doesn’t work here.

The cloud provider’s view of an HSM: everything can be accessed using a single accountSecurity in the real world.

Another way to look at the problem is to check what kind of security breaches actually happen in practice, the story is completely different. What actually happens in practice (there are plenty of examples, and they are easy to find):

• employees just walk out of the building with the cryptographic keys;
• developers put passwords in their code, that are stolen;
• cryptographic solutions are used that aren’t secure.

Interestingly, using HSMs actually solves these problems. And that’s not because of the physical protection, but because of the separation of concerns.

Now what?

So it is clear that what we need is a way to separate key management again from cryptographic calculations, and these from the actual use of the outcomes. This method should work for encryption in software, in the cloud or not. It should be simple enough to do that developers are not tempted to cut corners at the cost of security, and it should be enforcable as much as possible. Currently, the “simplest” solution I can think of:

• The administrator of the cloud account makes three separate security domains for developers, key custodians, and HSM managers. The administrator never uses her account and stores the password on a piece of paper in a vault
• The key custodians generate the keys using secure procedures with split knowledge on a separate computer. After sending the keys to the cloud the computer must be completely erased or destroyed
• The HSM managers make sure the cloud HSMs are audited properly, verify the functionality of the HSMs, check that the access of the HSMs is restricted to the relevant applications, check the audit logs, and so on.
• The developers are explained that all the talk of the cloud provider about “easy key management” should not be believed, and that they have to use our procedures to obtain an HSM calculation with proper keys.

As you see, this works a lot against the way things are designed now. I am currently talking to network maintainers, key custodians, developers, HSM designers and everybody else I could find to make this work. If you think you have something to say about this, let me know!

A request to the cloud providers

Let me finish with a request for the cloud providers: please help us making the cloud more secure. The entire world relies on you.