Monitoring the success of eLearning
How to evaluate your online courses
When exactly can an online course be said to have been successful? The answer to this question is essential for assessing whether you are getting a return on your investment—after all, eLearning must first and foremost be cost-effective and move your company forward, regardless of any idealistic motives. This article tells you about the different levels you can use to analyze the success of your online courses, and how to obtain and analyze specific data.
4 levels of analysis for evaluating learning outcomes
Whenever a number of stakeholder groups are involved in something, all perspectives must be taken into account. This also applies to eLearning. In the long run, it is no use if test scores are high but employee satisfaction with the learning process is low, or vice versa. The renowned evaluation model developed by US scientist Dr. Donald Kirkpatrick ensures that all key viewpoints are taken into consideration. Donald “Don” L. Kirkpatrick (15 March 1924–9 May 2014) was a US economist and Professor at the University of Wisconsin. In the 1950s, he researched the evaluation of educational processes as part of his PhD dissertation. Later, he used this as the basis for his 4-stage model of evaluation—the foundation of contemporary training monitoring. His 1975 book, Evaluating Training Programs earned Don Kirkpatrick worldwide recognition.
Methods of obtaining meaningful data
Surveys
The most effective way to measure learners’ satisfaction is to ask them. The same applies to the applicability of knowledge if neither indicators nor monitoring options are available. Digital surveys can help in this context, and can even be integrated into your learning management system depending on how professional your software is. Ideally, you should conduct two surveys for each course—one directly after it is completed and another one to two weeks later. This will enable you to hear participants’ views when they have completed the course, but also to learn whether and how the online course has had an impact on their day-to-day work.
Make sure not to overwhelm your employees with too many questions; instead, formulate clear questions about issues that are really relevant to you. You can expect to receive particularly honest–and therefore meaningful–answers if the survey is conducted anonymously.
Questions that may help you evaluate learner satisfaction
- How relevant was the course content for you?
- Are there any topics that you think the course should have covered but did not include?
- How do you rate of the amount of material covered?
- Did you enjoy the course?
- How challenging did you find the course (too easy / just right / too hard)?
- Was the content sufficiently explained?
- Did you find the course structure clear?
- How easy was it to use the course?
- Did you have any technical difficulties during the course?
Questions about the applicability of content
- How helpful were the examples in the course?
- How would you rate your competence in relation to the content covered?
- Have you been able to apply your new knowledge to your everyday work?
- Please describe a situation where you have applied your new knowledge.
- Name three key things/concepts that you have learned from this course.
- List three things that you will change about your approach to your day-to-day work as a result of the course.
Tests within online courses
It’s easy to find out how much of your online courses’ content your employees have learned—at least if you have the right authoring tool. This will make it easy to check learning by incorporating different types of tests into your online courses. Tests also help learners to embed knowledge they have just learned, to get a better idea of their progress, and to gain an overview of learning outcomes. Knowledgeworker Create, for example, allows you to integrate the following test options at any point in your courses.
multiple-choice tests
gapfill exercises
open questions
Learning analytics: Using performance indicators
Just as the performance of websites can be evaluated, eLearning includes what are known as learning analytics performance indicators that enable you to evaluate learning processes and outcomes. Course evaluation has numerous advantages: It helps you to optimize your company’s training setup where necessary, and to identify and address skills gaps; it also provides information about the return on your investment and where to deploy your resources to the greatest effect.
Learning analytics also provide basic data to enable you to personalize and customize learning processes. So instead of offering the same rigid learning paths to all your learners, you can use the data automatically—i.e., without manual intervention—to offer the precise learning path that will suit each individual employee, based on their user behavior. Where employees’ user data highlights gaps in their qualifications, for example, this enables you to offer them the opportunity to repeat courses. Conversely, employees who can demonstrate that they have sufficient knowledge can skip refresher courses. In this way, learners perceive content as being even more relevant and you save on resources.
The bottom line
Monitoring the success of eLearning, including evaluation, is as natural a part of the training process as the design and delivery of online courses. Without it, you will never find out how successful your digital training actually is: Skills gaps will go unnoticed and you will not know whether learning outcomes have been achieved.
However, the decisive argument for evaluating the success of courses is at a higher level: Every investment your company makes in training measures must be worthwhile and must contribute to the goals you have set. So you should consider monitoring and evaluating your training products as less of an obligation and more of an opportunity to deliver a measurable increase in the value of your business through successful knowledge transfer.
Image source: Rymden/shutterstock.com