Here you are. You’ve done your cultural assessment, you were able to identify the holes in the organizations security awareness efforts, you modified training and created a 12 month content plan to fix this. It’s time to sit back, and see some real user behavior change right?

Quick question: How do you know that your plan worked? Are users reporting more issues to the help desk? Are people more able to identify phishing emails? Are users retaining the information from annual training through the year? Basically, if your boss walked in and asked for proof that the budget was put to good use will you have anything to provide besides ‘trust me?’

Probably not and because of that you need to measure the behavior within your organization. Without measuring user behavior you have no way of knowing how successful, or unsuccessful, your security awareness architecture is. You are also left in the situation of ‘fire fighter’ in that you only know that a hole (fire) is present when that hole creates a big problem (i.e., a password attack causing a major data breech).

The Value of Baseline Measurements

There are two types of measurement that are going to be pivotal in showing you significant changes in behavior: baseline and continual. Baseline measurement shows you how users were performing before any changes were made thereby providing you with a point of comparison. Lets say that you started your intervention in June and you measured user behavior through September (see ‘No Baseline graph’). Did your intervention work? To be perfectly honest, this graph shows nothing impressive at all. As a matter of fact, it looks like nothing has happened. Money well spent for sure.

Now lets add a baseline measurement and see how that looks.

Much better! Now you can clearly see that (1) help desk calls have significantly increased, and (2) the number of successful phishing attacks have significantly decreased!

Furthermore, your new training/content plan seems to be producing long term behavior change over the following months. Great job.

This example really outlines the value of baseline measurement. Without it you really have no way of knowing if you made it better, worse, or broke even.

The Value of Continual Measurement

Once you have shown the effectiveness of your security awareness efforts, is their value in consistent measurement after? Of course. Constant measurement of user behavior allows you to see behavior trends and address issues before they become a problem. Lets go back to the help desk and phishing attack example. You continued to measure user behavior for several more months and suddenly you saw this.

What happened? Not only are your users not calling the help desk but they are also falling prey to more phishing attacks. They are performing similar to before your new training and content plan was implemented. Upon further investigation you find out that a new phishing method was just released and your users are having a hard time identifying it. This also leads to less calls to the help desk.

While initially this may seem like a giant leap in the wrong direction, it is exactly what behavior measurement is for. Security threats evolve and your security awareness architecture has to evolve with it. By measuring user behavior consistently you are able to see when patterns like this occur and develop an intervention (e.g., a news letter, quick email) that addresses this before it creates a big problem for your users and you.