How to Fail at Threat Modeling

How to Fail at Threat Modeling

What is threat modeling? Put simply, it’s fortune telling. It’s attempting to predict the future by reading the tea leaves of architecture, diagrams, data flow diagrams, technical specifications, and everything that goes with those. It’s about looking at the clues that exist and making educated guesses.

Most threat modeling methodologies involve looking at the design stage and putting a structure to the best guesses of what vulnerabilities and risks will exist in a finished product.

Is Threat Modeling Worth Doing?

The point of looking at the design stage is to address problems that can’t be fixed in a finished product. But the question is whether threat modeling is worth it. If you can do penetration testing, static code analysis, dynamic code analysis, automated pen testing, vulnerability scanning, secure development, and training, is there still a need to do threat modeling? Bore says without question, yes.

If you can catch something as early as possible in the design lifecycle, you solve it as early as possible. And, it is much, much easier to fix something before you’ve built it than after it’s finished.

But why doesn't everyone do it? The potential reasons are endless: lack of awareness, fear about friction in the design process, bad past experiences, bad relationships with security, budget, etc.

How Can Threat Modeling Go Wrong?

While Bore advocates for everyone to do threat modeling, it’s also vital to understand how it can go wrong.

1. Get Mired in Philosophy
2. Picking the Wrong Tool
3. Don’t Get Mazed

There’s a well-known threat modeling methodology developed by Microsoft called the STRIDE method.

It gives people a tool kit to look at a data flow diagram to decide where vulnerabilities might occur. You look at each of those data flows and try to determine what spoofing threats will exist, what temporary threats will exist, what repudiation, etc. This method is easy to apply, easy to pick up, intuitive, and it makes sense almost immediately to both technical and lay people.  

But under which STRIDE category does a man-in-the-middle attack fall? The answer is that it doesn’t matter! But too many people think it does.

Your threat modeling program may fail before it even starts when too much emphasis is put on where a particular category of attacks would fit. A man-in-the-middle attack could arguably go in either tampering or spoofing, or even information disclosure. The point is though, that it doesn’t matter where you put a potential attack vector—as long as you put it somewhere. If you want to have the existential discussion about whether man in the middle is temporary and spoofing, have that over a cocktail or two but don't bother with that discussion in the actual threat modeling workshop.

The next thing James talks about coming up a lot is picking the wrong tool. There are several different types of tools to help with threat modeling - from highly mathematical and scientific ones to others that basically boil down to an educated guessing. As James explains, because everyone who does threat modeling has a favorite method, the experience can go very wrong when people try to introduce their favorite tools to a new team; this is especially true when someone as part of the security function tries to bring it to those in the development function. James’ advice is not to try and bring your favorite tool with you. The wrong tool is the one people won’t use, and the right one, is the one they will. Figure out the tool that people are willing to implement and work based on what your organization needs - and that’s the one that’s going to work for your team.

You need to pick a model that allows your organization to move at the speed of development. If it takes you three weeks to build your threat model and your priority scoring, your threat model is useless because you've already finished two sprints by the time you're ready to fix something.

Your threat modeling process has to be light. And it has to be fast. And it has to use a method that makes sense and is usable by your development and design teams. It's not for the benefit of security.

What does don’t get mazed actually mean? In short - going into detail and trying to exhaust every single possibility. You’ll never succeed this way.

James highlights how detrimental this can be to your threat modeling program through the Rowhammer example. A sophisticated attack that is essentially exploiting the laws of physics, which fascinating to study, is highly unlikely to be the vector your organization is attacked by. Trying to go through all possible scenarios and following a maze that has you thinking about a Rowhammer attack, will almost always sink the success of your program.

If you are doing an exhaustive threat model, you are going to create friction with the development team, you are going to hit far too many walls, and you're not going to get something useful out of it fast enough for development teams and design teams to actually take that results and change them. Instead, you're going to get ignored.

Now, that's not to say you never want to consider these in certain environments, you probably do. But most of the time, look at realistic threats for your organization and your project. You don't need to consider threats, which are way above any capability you might have internally to defend against, look only at the ones that you can do something about and realistically threaten you.

So, What Should Your Threat Modeling Program Include?

Learning from mistakes is important, but so is understanding what works. James also outlines a few ways a program will succeed:

1. Action beats philosophy - something you can act upon is better than being absolutely 100% correct. Providing the design team with actionable information that they can address as part of design is far more important than being perfectly right.
2. Use what people will use -  try out different methods with teams and throw different methods at a problem. See which ones actually work for the development culture you have. You can build the most fantastic threat model and get absolutely nowhere because no one's actually going to use it.
3. Speed beats perfection - it is much, much better to get  20% of vulnerabilities during design and have them fixed, than to get 99% during design, but you've spent so long doing creating your model, that either the development project is massively delayed (which security gets blamed for) or, the project has moved on without, making your work useless.