Online assessments in compliance #2: How do you know what your workforce knows?

[JohnKleeman]

It's a critical requirement that your workforce know, understand and apply your policies and legal rules in modern business. But how do you know what your workforce knows?

In my first post on the business benefits of online assessments in regulatory compliance, I shared advice on how assessments could be used to save time and money by “testing out” of training.  In this post, I’ll cover how software like SAP Assessment Management by Questionmark is the most effective means to know what your workforce knows.

What regulators say

Almost all regulators are concerned to ensure that organizations have competent employees. Some require assessments to confirm this. For example, the US FDIC Compliance Manual states:

"Once personnel have been trained on a particular subject, a compliance officer should periodically assess employees on their knowledge and comprehension of the subject matter."

And the UK Financial Conduct Authority advise:

"Firms are required to assess the competence of their employees on a regular basis, as well as continue to assess employees’ training needs."

And in the health field, the World Health Organization (WHO) who influence many national regulators states in its good manufacturing practices for pharmaceutical products:

“Continuing training should also be given, and its practical effectiveness periodically assessed.”

Other regulators prefer to set principles, without detailed guidance. They focus on ensuring that your workforce is competent, often requiring documented training. For example, the US Food and Drug Administration states for medical device manufacturers:

"Each manufacturer shall have sufficient personnel with the necessary education, background, training, and experience ...  Each manufacturer shall  ... ensure that all personnel are trained to adequately perform their assigned responsibilities. Training shall be documented."

Advantages and disadvantages of assessments as a means of documenting training

If you are required to document training, the table below shows the three main routes:

Method	How to do	Advantages	Disadvantages
Take attendance	Group attendance list Photograph training class Record online attendance	Easy to do	Did they attend the whole session? Did they pay attention? Did they understand?
Get personnel to sign confirming attendance and understanding	Get employee to sign a form or confirm online that they attended and understood	Gets employee affirmation	Did they actually attend and understand, or did they just sign the form?
Give a test to employees to confirm understanding	Use SAP Assessment Management by Questionmark to deliver assessments, standalone or linked with SAP Learning Solution or SuccessFactors Learning.	Gives evidence that the employee genuinely understood the training Gives feedback to trainer and employee if there has been a misunderstanding	If an employee fails the test, you have to take remedial action

Just taking attendance doesn't provide much evidence if a regulator wants to check understanding. Getting employees to sign to confirm understanding can work if the organization culture is such that it is taken seriously. But the courts will not be impressed if employees sign the form without understanding the training. For example, the US Department of Justice fined an organization called Titan US$28 million for violations of the anti-bribery laws. The agreement between the US DOJ and Titan Corporation identified that simply getting employees to sign an anti-bribery policy was not in itself meaningful - it was required to inform and enforce the policy.

And a US Appeal Court in 2005 reviewed a case where serious injury was caused by some excavation work. According to the transcript, the company's defence of unpreventable employee misconduct was disallowed following a review of the training conducted, with the court saying:

"merely having an individual sign a form acknowledging his responsibility to read the safety manual is insufficient to insure that the detailed instructions contained therein have actually been communicated"

Many organizations realize that they need not just to get an employee's signature but genuine evidence of understanding from answering an assessment. To quote the UK FSA:

“… where staff understanding has not been tested, it is hard for firms to judge how well the relevant training has been absorbed.” 

How to assess employees

Here are some ways you can use SAP Assessment Management by Questionmark to assess employees within a compliance program:

Type of assessment	Business benefit
Internal exam	Check competence of your employees by assessing against the key skills and knowledge required for the job. Often done every 6 or 12 months for regulatory compliance purposes.
Knowledge check	Confirm learning and document understanding after training. Ensures training understood and reduces forgetting. Also called "level 2s".
Needs analysis or diagnostic tests	Measure current skills in topics, often as training needs analysis or pre-requisites for training. Used to allow "testing out" of training as described in my earlier blog post.
Observational assessments	Measure practical skills by having an observer monitor the employee to see if they are following correct procedures, often using a tablet or other mobile device. Checks competence in practical skills.
Course evaluation survey	Check employee reaction after training, to help measure training effectiveness. In compliance, allows gathering qualitative information from the field. Also called "level 1s" or "smile sheets".
Formative quiz during learning	Short quizzes during learning to confirm understanding, direct further learning and reduce forgetting. Makes learning more effective.
Job task analysis survey	Allows you to build evidence for the validity of your competency assessments by asking experts in a job role what tasks are most important. Helps the regulator trust your exam results.

Summary of business benefits

The key benefits for  using assessments to confirm understanding in a regulatory compliance setting are:

• Regulators are likely to explicitly or implicitly require it
• Gives evidence that the employee was trained and did understand the training
• If you are required to document training, it's the most reliable way of doing so
• Consistent and cost-effective means of checking your workforce know the law, your procedures and your products
• Reduces the cost, potential fines and inconvenience from mistakes caused by lack of understanding by your workforce

You can see the third post in this series, "Help your workforce retain critical information" here.