<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=1373947175984693&amp;ev=PageView&amp;noscript=1">

Learning & Development Blog

Measuring Compliance Training Effectiveness

Compliance Training: Meaningful Evidence, Measures, and Results

Maybe your high-level leaders support your compliance training program, but do you have the information and resources you need to implement a meaningful program?

According to a survey conducted by Deloitte and Compliance Week, many companies have an opportunity to make better use of available data to measure the effectiveness of their compliance program.

This might not be news to you. In fact, you might be grappling with a bigger challenge, like figuring out what data - or evidence - will show that your compliance training program works.

This post takes a look at what meaningful evidence looks like, how to go about measuring it, and how to use it to inform your compliance training program.

Before we go there, our previous post in this series covered why training course completion isn’t enough to prove compliance. Completion as a metric reflects neither the quality of a training experience (how appropriate and valuable the content is) nor its effectiveness (how much employees actually learn and put into practice).

What Does Meaningful Evidence Look Like?

Some companies have hundreds - even thousands - of pages of policies and procedures. Regulators require some to put internal control programs in place to reduce risk and meet compliance objectives. These programs typically include mandatory training for employees.

Of these companies, some are also expected to test or conduct audits of those controls. If asked by a regulator, these companies must produce meaningful evidence proving that they tested those policies, procedures and controls. And this evidence must demonstrate that the safeguards work.     

But do training completion and controls testing constitute meaningful evidence of compliant behavior? Looking at this from a performance improvement perspective, the answer is no. Evidence is only meaningful when it corresponds to a specific outcome.

When employees complete training, it provides evidence of their ability to access an eLearning module and press play. Clicking through slides does not demonstrate compliant behavior.

When employees pass a quiz designed to assess how much of the eLearning content they recall, this measures their knowledge of a policy. While this may be a leading indicator of compliant behavior, it is not a meaningful measure of whether an employee’s behavior on the job is compliant.

When a compliance manager conducts controls testing, this produces evidence that when a particular process is followed, it generates the documentation associated with the control being tested. It does not produce evidence that employees are using those safeguards; that is, evidence of compliant behavior. 

Compliance Training Program Data

A Cautionary Tale

Remember the story of Garth Peterson? He’s the Morgan Stanley employee who plead guilty to plotting to dodge the company’s internal accounting controls required by the Foreign Corrupt Practices Act (FCPA).

Peterson completed training on compliance policies and anti-corruption laws seven times, yet he still violated the law. Clearly, Morgan Stanley’s evidence of an effective compliance program - its internal controls testing and training completion data - was not meaningful.

Although Morgan Stanley had complied with the letter of the law, it did not achieve the spirit of the law. The intent of the regulation was to ensure employees knew the rules and how to follow them so they would not break the rules. After completing the training, Garth Peterson, broke the rules.

What Happens When Regulators Ask to See the Data?

Maybe you work for a company that has an internal program to satisfy whistleblower protection or HIPAA rule requirements. That company probably presents auditors with documentation of their above-and-beyond program for complying with the rules. This is likely accompanied by elegantly prepared evidence of testing that program. It’s nicely branded and maybe even signed by a compliance officer.

What happens when a regulator asks to see the data that show whether - or how much - employees use the program? What do you show them?

Or, what if no one uses the program in a given reporting period? Does that mean everyone is complying with policy? How would you know for sure?

You wouldn’t. This would require a different set of metrics.

These are the metrics regulators are starting to ask for when they show up to audit.

What Does Compliant Behavior Look Like?

If a company’s objective is to identify whether employees are complying with a policy, they must first define what compliance looks like.

From a performance improvement perspective, that means identifying the actions and behaviors that constitute appropriate employee conduct. That is, compliant behavior. Once defined, the company would establish a way to monitor those behaviors and collect the associated data. Then, they would analyze and report that information on a scorecard.

Conversely, the company could identify leading indicators of employee misconduct, and then devise a plan to detect, collect, assess and report that data.

The company would use that scorecard to show regulators whether - or how much - employees are complying with policy. This would constitute meaningful evidence of compliant behavior. Without evidence like this, a company cannot measure compliance program effectiveness in a meaningful way.

Measuring Compliance Training Effectiveness

How To Leverage Meaningful Evidence for Compliance Training

A company can use evidence like this to take their training program to a higher level of effectiveness. They can use it to pivot from covering the what and why of a regulation to targeting the employee behaviors that drive the activity monitored, analyzed, and reported on the scorecard.

Training needs assessment

This starts with a training needs assessment. Training professionals use this process to identify the right employee actions to target, the right strategy to transfer learning, and the right instructional approaches to make the learning stick.

For example, a training needs assessment might identify that the best strategy for a particular audience is ongoing microlearning that incorporates spaced repetition.

Microlearning

Microlearning is an approach comprised of short, focused strategies designed for learning a specific, concrete skill delivered in small learning units. The units can be anything from a 30-second video to a 3-minute interactive module. These can even be infographics, articles, or a podcast.

Spaced Repetition

Spaced repetition means revisiting the learning units in gradually increasing intervals. This is similar to how you might have used flashcards to learn addition and subtraction in elementary school. Compliance trainers, like second grade math teachers, know that actions must be practiced so people remember them in the moment. This is how you get training to stick.

Imagine delivering a compliance microlearning program that revisits the skills critical to compliant behavior on a monthly or quarterly basis. And on top of that, being able to evaluate the success of that program using the metrics on your compliance scorecard.

Just think how different the results would be compared with a traditional one-and-done, annual compliance training. You would be able to show regulators proof of employee skill acquisition based on actual evidence that employees are using these skills on the job.

The Opportunity for Compliance Training Programs

The standards for compliance programs, including compliance training, are changing. Organizations that cannot produce meaningful measures of effectiveness will not withstand regulatory scrutiny.

Given the resources available to training professionals, we have an opportunity to deliver compliance training that produces meaningful results by using a variety of learning methodologies and instructional tactics.

By investing in compliance training, we can educate employees on actions that apply to their job responsibilities and equip them with the right skills to demonstrate compliant behaviors on the job.

New call-to-action

Are you investing in training that helps you produce measurable results? We’d love to hear what evidence you’re providing regulators, or how you’re measuring effectiveness.

Leave a comment below to share your thoughts with us.

 

Compliance Week | In Focus: 2016 Compliance Trends Survey. Copyright © 2017 Deloitte Development LLC. All rights reserved. https://www2.deloitte.com/content/dam/Deloitte/us/Documents/governance-risk-compliance/us-advisory-compliance-week-survey.pdf

Ex-MS Banker in China Bribery Case: My Side of Story https://www.cnbc.com/id/48693573

DOJ’s criminal charges against Morgan Stanley employee Garth Peterson in 2012 https://www.justice.gov/opa/pr/former-morgan-stanley-managing-director-pleads-guilty-role-evading-internal-controls-required

S. Department of Justice Criminal Division: Evaluation of Corporate Compliance Programs https://www.justice.gov/criminal-fraud/page/file/937501/download