LMS Standards SCORM xAPI cmi5

Learning analytics based on SCORM, xAPI, and cmi5: How to measure learning success in eLearning

Magda Lehnert eLearning basics, eLearning software

Reading time: 7 minutes
SCORM, xAPI, and cmi5 make learning success measurable and provide you, as a company, with important indicators to help you monitor whether your continuing education opportunities in the form of online courses are successfully accepted, passed, and completed—criteria decisive for the success of your company. We will introduce you to the three most important eLearning standards, show you how they differ, and help you determine exactly which standard is most suitable for your eLearning project.

In this Blog Article, you can read about:

Learning analytics based on SCORM, xAPI and cmi5: How to measure learning success in eLearning

What are eLearning standards?

eLearning standards convey data about the learning process between learning content and learning management system. They ensure that two different systems can communicate with each other and speak the same language. This allows the online course (created with a learning-content-management system) to send and receive information to the learning-management system (LMS). Among other things, this includes information on learning progress or learning outcomes. By storing the current status, they enable the learner to resume courses easily at any time.

eLearning standard: SCORM

Shareable Content Object Reference Model

SCORM is a collection of technical standards, dating from 2001, that defines how online courses are packaged as SCORM packages, as well as how they communicate with learning-management systems. SCORM describes a static model that depicts a learning status. When a learner does something, the status changes. This is exactly what is described and recorded. Here, the vocabulary is limited to “progress complete”/”progress incomplete”, for example.

SCORM describes the relationship between exactly one learner and one learning package. The SCORM standard is commonly used in two different versions. Still widely used today, SCORM 1.2 was the first version to see widespread use and technical support in many learning-management systems. The more comprehensive version—SCORM 2004, which features a somewhat more precisely specified static model—was introduced a few years after the original standard and has seen similarly widespread use.

Data collection

The following data can be collected with SCORM 2004:

  • Course progress
  • Learning time
  • Test results for individual elements
  • Completion status for the entire course
  • Test result for the entire course

With the advancement of eLearning and the use of mobile devices, SCORM reached its limits. For example, SCORM can accommodate purely web-based courses, but not learning activities in apps or even offline activities. However, SCORM is still a respected industry standard and is frequently used. SCORM-compliant content can be quickly transferred by download and upload, and virtually any LMS is SCORM-compliant. Whether SCORM is suitable for your own eLearning project depends entirely on what exactly you want to evaluate afterwards and what you use the data for. In any case, you should pay attention to the supported SCORM versions of your LMS and content, because they must always be intercompatible.

eLearning standard xAPI

Experience Application Programming Interface

xAPI is considered to be the successor to SCORM, and is also used for eLearning content. It allows learning-activity data to be exchanged between the so-called “learning-record store” (LRS) and reporting tools or the learning-management system, regardless of format. “xAPI” stands for “Experience Application Programming Interface” (previously Tin Can API) and is a format-independent standard. It describes the communication of learning content with a learning-record store. An LRS is a central database for documenting or reporting learning activities, which are stored in the xAPI communication standard. From there, they can be forwarded to a reporting tool or an LMS.

1) The functioning of xAPI and LRS

Communication between the learning medium or application and the LRS is based on experience statements (xAPI statements), which describe individually defined learning events according to simple rules.
Image
In so-called “experience statements”, or xAPI statements, a learning event is defined on the basis of three main features: actor, object, and verb. This principle is used to communicate between the learning tool and the LRS whether and how the described event took place. An experience statement describes the following about a learning event: “Who used what, and how and when?”
Who (actor):
  • Learner or learning group
  • Virtual agents: A virtual agent is a technical system that itself performs learning-related activities and sends its own events to an LRS in the form of statements. Examples:
    • A learning space recognizes the learner present.
    • A virtual machine in a simulation performs a specific action.
    • A virtual opponent in an educational game makes a decision.
When:
  • Time
  • Context
What (object):
  • Courses
  • Assessments
  • Pages
  • Questions
  • Images
  • Video
  • Sound
  • Interactive knowledge content of any kind
How (verb):
  • Seen
  • Edited
  • Scored
  • Started/finished

2) Reporting

By using xAPI, learning events can be easily recorded using defined statements and cached in an LRS. In addition, other data on (learning) performance, which were not initially recorded via xAPI, can be loaded into this.
Image
Transmission of learning activities to a learning-data repository (LRS) and the evaluation of these

In the LRS, the various data strands are stored centrally and system-independently, merged for evaluation purposes (if required), and/or transmitted to external evaluation tools. 

This permits diverse, digital learning that can be measured both in combination with and independently of learning management systems.

3) What results do data from an LRS provide?

Individual learning outcomes at the micro-level:
  • Completion of learning content
  • Time required
  • Pass or fail
  • Reporting of single and multiple scores
  • Evaluation at the question level
  • Detailed test results from a wide range of learning scenarios (e.g., adaptive learning, social learning)
Learning behavior of specific user groups:
  • Number of learners of a system
  • Comparisons of multiple user groups on time required, pass/fail, etc.
  • Total number of all learners or sessions,
  • etc.
Hypotheses and interdependencies:
  • Number of active learners depending on the learning topic (popularity)
  • How many learners have gone through the module in period X? (Trends)
  • Most popular learning content in period X (latest trends)
  • etc.
Complex depiction of a multi-channel learning situation:
  • Combination of all these points, taking into account other external factors where appropriate. From this, conclusions can be drawn with respect to the relevance or effectiveness of different learning methods/trends for a specific use case.
    • Effectiveness of adaptive learning
    • Success of social learning
    • Benefits of interactive learning
    • Added value of blended learning
    • etc.
Based on this, an overall impression of the learning process can be gained. In addition, conclusions and recommendations for action for an improved learning process can now be derived.

4) What can reporting look like?

Reporting tools can be used to evaluate learner activities. To do this, one or more LRSs send the desired learning-activity data to the reporting tool in xAPI format. These are then visualized in configurable reports according to individual requirements. Specific hypotheses about learning activities can thus be easily tested.

a) Example of a report from country-specific e-training in wholesale trade

Dashboards of the LRS Learning Locker are a well-known reporting tool, which is also used to evaluate the learning activities of employees or customers of a global wholesaler. An individual reporting system for a learner group (country level) could look like this:
Report Lernergruppe
Individual reporting on learning activities around the product range of a globally operating wholesaler

b) Example of a report from a flashcard app

In contrast to learning in closed learning-management systems, which communicate exclusively with SCORM-based content, xAPI and LRS also allow the evaluation of system-independent, digital content, and can be used in mobile apps. An example from a typical eLearning scenario shows the results of the Knowledgeworker Cards learning-card app, which employees of a company in the medical industry use to refresh their knowledge.
Report Knowledgeworker Cards

5) The advantages of xAPI and LRS

In, with, or without LMS
Thanks to xAPI standard, learning is possible independently of an LMS, but can also take place within an LMS. Here, the final format of the learning content is not limited to HTML/JavaScript packages, as is the case with SCORM.
Diverse, digital content formats
Whether offline or online—almost any learning format can be communicated to an LRS via xAPI statements in conjunction with a digital activity and thus evaluated. The format of the learning content is therefore virtually irrelevant. However, digital formats against which specific learning events can be measured in an automated way are particularly suitable.
Targeted controllable data flow
Specific learning events can be defined at the micro-level and measured live for each learner individually. The analysis of learner-specific learning activities becomes possible even for contexts such as the required learning time as a function of the already-active learning time. Based on defined endpoints, data can be exchanged specifically, quickly, and directly between learning tools and LRS.
Detailed reporting for each learner
Specific learning events can be defined at the micro-level and measured live for each learner individually. The analysis of learner-specific learning activities becomes possible even for contexts such as the required learning time as a function of the already-active learning time.
Tracking of learning activities at the micro-level
Thanks to xAPI and LRS, it is possible not only to measure the completion of a learning activity, but also to collect other data and provide detailed evaluations in individual reports. Example from wholesale: Product cards in the “Apples” section provide valuable knowledge about individual apple varieties. Thanks to xAPI and LRS, it is possible to evaluate which types are called up most frequently.
LRS and xAPI permit a new level of learning-success measurement
Nearly all online and offline digital formats have measurable learning experiences. These are recorded and quantified by xAPI, stored in real-time in the learning-record store, and further processed for reporting purposes. A survey of learning activities can take place in numerous online and offline digital formats. The use of xAPI and LRS lead to new, more complex acquisition and evaluation as well as deeper understanding of digital learning processes.

eLearning standard cmi5

As the latest LMS standard, cmi5 combines the advantages of SCORM and xAPI. cmi5 is therefore also often referred to as “xAPI with rules”. Having created a standard with xAPI that is general enough to map as many use cases as possible, cmi5 streamlines the application through “constraining rules”. These now define how learning content is imported, launched, and tracked when LRS information is used and updated with xAPI. However, the range of functions remains almost the same as with xAPI.

Conclusion and recommendation

Depending on which data you want to measure, evaluate, and use for further optimization, you should pay attention to the options available to you when choosing your eLearning tools. Only xAPI and cmi5 enable the seamless tracking of learning activities that take place outside the web browser (e.g., on mobile devices, in apps, or even offline).

Sustainably shape the digital future of your organisation

We will be happy to advise you and work with you to ensure your sustainable continuing education success through the innovative design of digital learning processes.
Arrange a consultation now
header image source: Wright Studio/shutterstock.com