Learning analytics based on SCORM, xAPI, and cmi5
How to measure learning outcomes in eLearning
SCORM, xAPI, and cmi5 make learning outcomes measurable and provide you, as a company, with important indicators to help you monitor whether your online courses are being taken up, passed, and completed—decisive criteria for the success of your company. We will introduce you to the three foremost eLearning standards, show you how they differ, and help you determine exactly which standard is most suitable for your eLearning project.
What are eLearning standards?
eLearning standards transmit data relating to the learning process between the learning content and learning management systems. They ensure that two different systems can communicate with each other and speak the same language. This allows online courses (created with a learning content management system) to send information to the learning management system (LMS) and receive information from it. Among other things, this includes information on progress and learning outcomes. They store learner’s current status making it easy for them to resume courses at any time.
eLearning standard: SCORM
Shareable Content Object Reference Model
SCORM is a collection of technical standards, dating from 2001, that defines how online courses are packaged as SCORM packages and how they communicate with learning management systems. SCORM is a static model depicting a learning status. When a learner does something, their status changes. And this is described and recorded. Here, the vocabulary is limited to “progress complete”/”progress incomplete”, for example.
SCORM describes the relationship between individual learners and learning packages. Two versions of the SCORM standard are commonly used. Still widely used today, SCORM 1.2 was the first version to see widespread use and be given technical support within many learning management systems. The more comprehensive version—SCORM 2004, which features a slightly more precise static model specification—was introduced a few years after the original standard and has also seen widespread use.
The following data can be collected with SCORM 2004:
- Learning time
- Test results for individual elements
- Completion status for the entire course
- Test result for the entire course
With the advancement of eLearning and the use of mobile devices, SCORM reached its limits. For example, SCORM can accommodate purely web-based courses, but not app-based or offline learning activities. However, SCORM is still a respected industry standard and is frequently used. SCORM-compliant content can be quickly downloaded and uploaded, and virtually all LMSs are SCORM-compliant. Whether SCORM is suitable for your own eLearning project depends entirely on what you want to evaluate afterwards and what you use the data for. In any event, you should take account of the SCORM versions supported by your LMS and content, because they must always be compatible with each other.
xAPI eLearning standard
Experience Application Programming Interface
xAPI is considered to be the successor to SCORM, and is also used for eLearning content. It allows learning-activity data to be exchanged between what is known as the “learning-record store” (LRS) and reporting tools or the learning management system, regardless of format. “xAPI” stands for “Experience Application Programming Interface” (previously Tin Can API) and is a format-independent standard. It describes the communication of content to a learning record store. A learning record store (LRS) is a central database for documenting or reporting on learning activities. They are stored there in the xAPI communication standard and can then be forwarded to a reporting tool or an LMS.
How xAPI and LRS work
Communication between learning mediums or applications and LRSs is based on experience statements (xAPI statements), which describe individually defined learning events on the basis of simple rules.
In so-called “experience statements”, or xAPI statements, a learning event is defined on the basis of three main features: actor, object, and verb. This principle determines how the learning tool and the LRS communicate about whether and how the event described took place. An experience statement describes a learning event as follows: “Who used what, how and when?”
- Learner or learning group
- Virtual agents: A virtual agent is a technical system that delivers learning-related activities and sends its events to an LRS in the form of statements. Examples:
- A learning space recognizes the presence of a learner.
- A virtual machine performs a specific action in a simulation.
- A virtual opponent in an educational game makes a decision.
- Interactive knowledge content of any kind
Used how (verb)
xAPI enables learning events to be easily recorded using defined statements and cached in an LRS. Other data on (learning) performance, which were not initially recorded via xAPI, can also be fed in.
In the LRS, the various data strands are stored centrally and independently of the system, merged for evaluation purposes (if required), and/or transmitted to external evaluation tools.
This allows for diverse, digital learning that can be measured both in combination with and independently of learning management systems.
What results do data from an LRS provide?
Individual learning outcomes at the micro-level:
- Completion of content
- Time required
- Pass or fail
- Reporting of single and multiple scores
- Evaluation of responses to individual questions
- Detailed test results from a wide range of learning scenarios (e.g., adaptive learning, social learning)
Learning behavior of specific user groups:
- Number of learners on a system
- Comparisons of multiple user groups regarding time required, pass/fail, etc.
- Total number of learners, sessions, etc.
Hypotheses and interdependencies:
- Number of active learners per learning topic (popularity)
- How many learners have gone through the module in period X? (Trends)
- Most popular learning content in period X (latest trends), etc.
Complex depiction of multi-channel learning environments:
- combination of all these points, taking into account other external factors where appropriate. This enables conclusions can be drawn with regard to the relevance or effectiveness of different learning methods/trends for a specific use case.
- Effectiveness of adaptive learning
- Efficacy of social learning
- Benefits of interactive learning
- Added value of blended learning, etc.
On this basis, an overall impression of the learning process can be gained. In addition, it allows you to draw conclusions and come up with recommendations to improve learning processes.
What does reporting look like?
Learner activities can be evaluated by means of reporting tools. To do this, one or more LRSs send the desired learning activity data to the reporting tool in xAPI format. These are then visualized in configurable reports in line with your specific requirements. Specific hypotheses about learning activities can thus be easily tested.
Example of a report from country-specific eTraining sessions in wholesale trade
LRS Learning Locker dashboards are a well-known reporting tool that is also used by a global wholesaler to evaluate the learning activities of its employees and customers. An individual reporting system for a learner group (country level) might look like this:
Example of a report from a flashcard app
In contrast to learning in closed learning management systems, the content of whose communications is exclusively SCORM-based, xAPI and LRS also allow for external, digital content to be evaluated, and can be used in mobile apps. An example from a typical eLearning scenario shows the results of the Knowledgeworker Cards app, which employees of a company in the medical industry use to refresh their knowledge.
Benefits of xAPI and LRS
In, with, or without an LMS
Variety of digital content formats
Whether it’s offline or online, almost any learning format can communicate with an LRS via xAPI statements in connection with a digital activity and thus evaluated. The format of the learning content is therefore pretty much irrelevant. However, digital formats that allow specific learning events to be automatically measured are particularly suitable.
Targeted controllable data flow
Specific learning events can be defined at the micro-level for each individual learner and measured live. It is even possible to analyze learner-specific activities for ratios such as required learning time as a function of time already spent on active learning. Defined endpoints enable data to be exchanged specifically, quickly, and directly between learning tools and LRS.
Detailed reporting for each learner
Specific learning events can be defined at the micro-level for each individual learner and measured live. It is even possible to analyze learner-specific activities for ratios such as required learning time as a function of time already spent on active learning.
Micro-level tracking learning activities
Thanks to xAPI and LRS, it is possible not only to capture the completion of a learning activity, but also to collect other data and provide detailed evaluations in individual reports. Example from wholesale: Product cards in the “Apples” section provide valuable knowledge about individual apple varieties. xAPI and LRS make it possible to evaluate which types are called up most frequently.
LRS and xAPI enable you to take measurement of learning outcomes to a whole new level
Nearly all online and offline digital formats have measurable learning experiences. These are recorded and quantified by xAPI, stored in real-time in the learning record store, and further processed for reporting purposes. Surveys of learning activities can be undertaken in a wide range of online and offline digital formats. The use of xAPI and LRS enables new, more complex recording and evaluation of digital learning processes and promotes deeper understanding.
cmi5 eLearning standard
As the latest LMS standard, cmi5 combines the benefits of SCORM and xAPI. cmi5 is therefore also often referred to as “xAPI with rules”. Having created a standard with xAPI that is general enough to map as many use cases as possible, cmi5 streamlines the application through “constraining rules”. These now define how learning content is imported, launched, and tracked when LRS information is used and updated with xAPI. However, the range of functions remains almost the identical to xAPI.
The bottom line
Depending on which data you want to measure, evaluate, and use for further optimization, you should consider the options available to you when choosing your eLearning tools. Only xAPI and cmi5 enable seamless tracking of learning activities taking place outside of web browsers (e.g., on mobile devices, in apps, or even offline).
You may also be interested in the following articles
Scenario-based learning: Successful skills training
Arguments for eLearning: How to convince management and works councils
The secret to the perfect online course
Image source: Wright Studio/shutterstock.com