When defining the goals for a learning strategy most L&D teams would probably agree, that “high learner engagement” is among the key components to focus at. But what exactly is learner engagement? And how can you measure the “degree” of engagement in a training offer? In today’s blogpost we would like to shed some light on these questions. Additionally, we will provide you with a couple of metrics to look at in terms of learner engagement.
Sounds interesting? Keep on reading.
What makes things a little bit complicated is, that learner engagement actually has no clear, all-encompassing definition. The term has been used in various ways over the years. Fifteen authors would probably provide you with fifteen (slightly) different definitions, differencing between 2, 5 or even 7 types of engagement indicators. What definition should we go with then?
When opening the Wikipedia entry about learner engagement, the following quote appears:
”[Engaged] students make a psychological investment in learning. They try hard to learn what school offers. They take pride not simply in earning the formal indicators of success (grades), but in understanding the material and incorporating or internalizing it in their lives.” - Fred Newmann, 1992.
Although this definition is tailored to students in a school setting and was given quite a few years ago, it applies quite well to corporate learning today from our point of view. An engaged learner thus does not only learn due to the fact, that a certificate or something such as a promotion or prize is waiting at the end of a learning track. She learns, because she actually wants to understand and incorporate the information provided.
Measuring learner engagement equals measuring an employees’ participation and interaction with a certain training offer (either offline or online) in terms of quantity as well as quality. However, there is not THE ONE single metric to measure learner engagement. Rather, learner engagement can be seen as a goal that you are pursuing. And there are several metrics that can help to determine, whether you are on the right track / moving in the right direction in order to reach that goal.
What metrics should you look at then? Especially in an online setting, where possibilities of direct observations or assessments are limited?
Generally speaking, Data Analytics is key.
Given that you work with some kind of LMS system or learning platform, you are probably provided with lots of data about your employees. Data, that gives you the possibility to interpret and analyze the online learning behavior of your learners. If possible, make use of information such as attendance rate overviews, number of completed materials or hours learned in order to get a good understanding of your current “status quo” of learner engagement.
To become a bit more concrete, following are a couple of things we would advise you to especially look out for.
1. Sign-up Rates
Before you start to look at metrics that help you to measure the engagement with your learning offer during the actual learning activity, you could start by looking at your sign up rates in general. What percentage of your workforce signs up for learning courses? Do they sign up right after you announce a new course offering? Or does it take a couple of reminders from your L&D team, managers or marketing team?
Measuring sign up rates can be a great way to get a first understanding of the learning motivation within your workforce. Additionally, they can act as a great indicator to see how well your learning program is promoted and what internal marketing / promotion measures work best.
2. Completion & drop out rates
Next to a general overview of sign up and attendance rates, completion rates can be a great way of measuring your employees’ engagement level. In an offline-setting dropping out of a course (= leaving the room without explanation) is relatively difficult. In an online setting? Not so much. Because who is there to control or at least look disappointed?
So it could make sense to check completion rates for different courses on team, department and organization level on a regular basis. These rates can give you a direct indication whether the learning content was engaging enough or left the learner unmotivated and disengaged, resulting in a drop out.
3. Weekly and monthly active user
Besides regularly checking the completion rates of your training courses, you could keep an eye on your weekly and monthly active users (WAU / MAU). In case you are using an LMS system you probably have the opportunity to look up the necessary insights there or are even provided with a dashboard overview. But even if you are using an excel sheet to organize and track participation in your learning offers, you could build a simple dashboard yourself in order to regularly check learner activity. Why? First of all, it can be great in order to evaluate the success of internal learning promotion campaigns. How does activity change after a certain campaign? But more importantly, it provides you with a good understanding of employees with a “learning habit”. What percentage of your employees continue to learn on a regular basis? Especially MAU is a great indicator, as forming habits on a monthly basis is much harder compared to a weekly one.
4. Mandatory vs. voluntary courses
Remember the characteristics of an engaged learner mentioned above? An engaged learner actually wants to learn and is not working through materials because she has to. Why not try to measure this kind of “curiousness” or “hunger to learn”?
One possible way to do that is by having a look at the sign up and attendance rates for mandatory vs. voluntary courses. Are your team members only doing the training classes they have to? Or are they actively looking for additional material to improve their current skill-set? A high attendance rate in voluntary learning courses is usually a good indicator for learner engagement.
5. Learning time spent
Learning time spent is another great thing to look at in terms of learner engagement. Are your employees learning at all? Are they learning as much as you would like them to do? Are they maybe even spending more time on learning than you planned or anticipated initially?
--> Bonus tip:
You could use this information not only for your L&D or management team, but provide your employees with an overview as well. E.g. by establishing a leaderboard highlighting the top learners of the week / month / year based on learning time spent. This would not only be a great gamification feature to foster internal competition but could also result in a boost of learner motivation and ultimately engagement. In case learning time spent doesn’t work for you, you could also think about implementing some kind of “learning point” system, e.g. based on difficulty of certain materials and compare employees based on the learning points gained.
In general, every company has to determine what amount of time they want their employees to spend on development themselves. But from our experience, in case your employees spend 5-10% of their work-time on learning, you are definitely on the right track. 😉
6. Community interaction
What usually is missing in an online-learning environment is the possibility to interact with a trainer or fellow learners during a course. If your LMS / learning platform (in case you are using one) does not offer chat features, you could try to create such opportunities yourself by installing learning “channel” in tools that you are already using (e.g. Microsoft Teams or Slack). Both collaboration tools offer analytical insights. Among them number of direct messages sent, days active or also the amount of likes and comments given by a person.
An engaged learner would be much more likely to pose questions, take part in discussions and interact with learners and trainers, right? Measuring the amount of comments and reactions on individual learner, team and ultimately organization level thus states another possibility to infer a level of learner engagement.
7. Feedback surveys
Even if it may seem old-school, one way to measure engagement that shouldn’t be left out is the usage of feedback surveys. Actively ask your learners for feedback. Be it ratings (on a scale from 1-5) on materials and whole courses or written feedback via an online or offline evaluation form. Active or passive participation in feedback surveys is not only a great way to measure the engagement level of your employees but also provides you with great insights in order to improve your learning offer for the future.
8. Percentage of employees using their new skills
One of the most important goals any learning strategy has, is that employees are able to actually use the new skills they gained during the training. Trying to measure this “post-learning engagement” is one aspect you shouldn’t leave out of consideration. Ways to do that are e.g. 360-degree feedback, self-assessment questionnaires or ultimately looking at specific job performance KPIs. Compare KPIs on individual, team and organization level before a learning course offer with the results after completing this learning offer. Can you infer an increased productivity or quality of work?
Learner engagement is one of the most important aspects to continuously track, as it provides you with a good indication of how well your learning & development strategy is being executed. And while there are several things to assess in order to measure this engagement, one thing should be kept in mind: Data is key. So you should try to gather as much of it as possible.
What’s next? We are always looking for feedback or additional thoughts! So feel free to reach out any time via firstname.lastname@example.org.