Still using completion rates to measure the effectiveness of your compliance program?
If so, you’re not alone. According to a survey conducted by Deloitte and Compliance Week, many organizations still rely on completion rates to gauge success. Of those organizations, most considered training to be effective when between 90% and 95% of their learners finish it.1
Training professionals have long argued that completion doesn’t equal learning, let alone understanding the rules. It’s been hard for these ideas to get traction, given that auditors tend to scrutinize training completion rates to assess regulatory compliance. But finally, the standards for compliance training are changing.
This story about Morgan Stanley will explain why.
Completion Does Not Measure Learning
Morgan Stanley was one of those organizations that used completion rates to demonstrate their compliance training program’s effectiveness to regulators. They launched compliance training online to employees like Garth Peterson, who dialed into training teleconference calls or logged onto the learning management system to complete the training by the deadline.2
Morgan Stanley designed their training to ensure Peterson knew about important anti-corruption regulations and how to comply with them. They believed this is what was expected from them. If a regulation required organizations to train employees on X, Y and Z, and the organization designed, delivered and required employees to complete training that met the objectives of X, Y and Z, then the organization should be in compliance. Based on this logic, Morgan Stanley believed they complied with the letter of the law.
Unfortunately, complying with the letter of the law did not achieve the intent of the regulation. The intent of the regulation was to ensure employees knew the rules and how to follow them so they would not break the rules. After completing the training, at least one employee - in this case Garth Peterson - broke the rules.
So what went wrong? Morgan Stanley believed they ‘trained’ Peterson not to break the rules. But if you look at their training program, you’ll see this is not what it was designed to do.
Morgan Stanley’s compliance training program - just as many others in the financial services industry - was designed to deliver content that explained the regulations and how to comply with them. This design included establishing a process to gather evidence of training via attendance roster or learning management system completion report. All employees had to do was make sure they were marked complete within the system of record.
Morgan Stanley discovered what L&D professionals have always known, that completing a course does not mean that learning occurred. Despite training Peterson seven times on compliance policies and anti-corruption laws, this did not prevent him from violating the law. He eventually plead guilty to plotting to dodge the company’s internal accounting controls required by the Foreign Corrupt Practices Act (FCPA).
When asked about the teleconference training sessions he attended, Peterson said he’d either hang up or set aside his phone after the facilitator took attendance. “All you have to do is say, ‘Garth Peterson's on the phone,’ and they check the box that says, he's complied," he said.3
A federal court sentenced Peterson to serve nearly one year in prison.
Starting Over From Scratch
Regulators continue to crack down on employees who sidestep compliance training - and on organizations that fall short of complying with the intent of the law. Some no longer deem completion to be an appropriate measure of compliance training program effectiveness. Instead, they are asking organizations to produce evidence of compliant behavior.
This changes everything for compliance training programs. It means starting over from scratch to determine what evidence will satisfy regulators, and then figuring out how to gather that data, measure its effectiveness, and justify how it proves meaningful results.
If this sounds daunting, it should. It may be relatively easy to figure out the “what” - that is, what evidence regulators want to see. But figuring out the “how” is more complicated and a lot more work.
The good news is that this is the first in a series of posts that will help compliance leaders make the case for investing time and resources in policy and compliance training programs. The next post will take a look at what evidence of compliant behavior looks like, and how to go about measuring it.
Subscribe to our blog to be notified when Part 2 is published.
Already implementing a compliance program that produces evidenced of compliant behavior? We’d love to hear what evidence you’re providing regulators, or how you’re measuring effectiveness. Leave a comment below to share your thoughts with us.
1 Compliance Week | In Focus: 2016 Compliance Trends Survey. Copyright © 2017 Deloitte Development LLC. All rights reserved. https://www2.deloitte.com/content/dam/Deloitte/us/Documents/governance-risk-compliance/us-advisory-compliance-week-survey.pdf
2 DOJ’s criminal charges against Morgan Stanley employee Garth Peterson in 2012 https://www.justice.gov/opa/pr/former-morgan-stanley-managing-director-pleads-guilty-role-evading-internal-controls-required