New study reveals phishing simulations might not be effective in training users

When it comes to compromising a company’s network, the easiest way to start is usually to target the employees with phishing campaigns. They are the weakest part of your network environment.

Therefore, phishing simulations (aka phishing tests) have become increasingly common in corporations. Those simulations pretend to be real phishing email landing in the employees’ mailboxes, without any malicious payload. They show a realistic phishing page and collect statistics about who clicked with or without providing credentials, how many users reported it to the security staff, etc.

Companies can use professional phishing simulation services or even create their own simulation for free with tools like GoPhish.

No matter the method, the goal of phishing simulation stays the same: Get to know employees’ behaviors better within the company and raise awareness on that critical threat.

SEE: Fighting social media phishing attacks: 10 tips (free PDF) (TechRepublic)

A phishing simulation study at large scale over 15 months

A recent study published on the topic comes from the computer science department of ETH Zurich, a Swiss public university focused on science, technology and engineering. The study ran for 15 months in a large organization (more than 56,000 people employed, about 14,000 employees targeted by the study), making it the largest study both in terms of scale and length published to this day.

The method used consisted of sending either phishing emails leading to a phishing page, or emails containing a malicious file enticing the user to perform a dangerous action when launched, like providing credentials or enabling macros on an attachment.

The phishing emails could contain warnings, either short or more detailed (Figure A), while other emails did not contain any warning at all.

Figure A

The employee could also report the phishing attempts via a reporting button installed in their email client. The button was introduced prior to the study and advertised in the internal company news.

Once a user performed a dangerous action, the simulation could bring them to an educational page explaining what happened in detail, what they should have looked for to avoid the phishing, and tips for the future. An additional instructional video, further quizzes and learning material on phishing was also provided, but the user was not forced to watch or read it. Some users did not receive that educational page.

SEE: Digital natives more likely to fall for phishing attacks at work than their Gen X and Boomer colleagues (TechRepublic)

Which users were more prone to fall for phishing?

The study analyzed what kind of computer usage, gender and age range would perform the dangerous action (Figure B).

Figure B

Computer usage

Employees with a specialized usage of computers (e.g., branch workers who mostly use a single dedicated software) clicked on more phishing links and performed more dangerous actions than the other categories of users.

Age range

The youngest employees clicked more on dangerous links than the oldest ones. Employees in the 50-59 age range were also more prone to fall for phishing.

Gender

According to the study, the combination of gender and computer use was significant, but gender by itself was not.

SEE: Shadow IT policy (TechRepublic Premium)

Phishing at length

The study ran for 15 months and showed that a small number of employees will fall for phishing multiple times, especially the youngest employees.

It also revealed that many employees will eventually fall for phishing if continuously exposed to it. ETH researchers said that “a rather large fraction of the entire employee base will be vulnerable to phishing when exposed to phishing emails for a sufficiently long time.”

Warnings are helpful, educational pages are not

It appears that the warnings in the phishing emails significantly helped prevent the users from clicking on the links, but detailed warnings were not more effective than short ones.

More surprising, the users who did get the educational page after falling for a phishing ploy clicked more on later phishing pages. The researchers tempered this result with the fact that it could only be applied to this particular way of delivering voluntary training and that other methods might provide other results.

The researchers tried to find the cause for this significant finding in the post-experiment questionnaire filled out by the employees. One possible explanation is a false sense of security related to the deployed training method: 43% of the respondents selected the option “Seeing the training web page made me feel safe” and 40% selected “The company is protecting me from bad emails.” It remains an open question for future work to explore whether this is due to a misunderstanding of the training page (e.g., employees thought they were protected from a real phishing case) or due to an overconfidence in the company’s IT department.

SEE: Phishing attacks: A guide for IT pros (free PDF) (TechRepublic)

Employees are still an asset for fighting phishing

The study said that users kept reporting phishing emails over time and that there was no kind of “reporting fatigue” in the company. A significant number of users were active on reporting. The most active reporters were those who showed the best expected computer skills. Reporting users also felt encouraged when receiving positive feedback.

10% of the reports were sent by users within five minutes of receiving the email. The largest portion, between 30 and 40% of the reports, were sent within 30 minutes (Figure C).

Figure C

Yet for such crowd-sourcing to be effective, employees still need a convenient and easy way to report phishing cases. A button in their email client seems to be a good option.

Disclosure: I work for Trend Micro, but the views expressed in this article are mine.

Cybersecurity Insider Newsletter

Strengthen your organization’s IT security defenses by keeping abreast of the latest cybersecurity news, solutions, and best practices.
Delivered Tuesdays and Thursdays

Sign up today

Also see