%20(1).svg)
Learning analytics, from the buyer's side
This page is about the decision, not the product. Where learning data actually lives, what each system will and will not tell you, how to tell which stage your organisation is at, and the cases where buying a learning analytics platform is the wrong answer. Written for people who have to defend a number to an auditor, a board or a renewal committee.
If you want the definitional version first, start with our guide to what learning analytics is and come back here when you are choosing between options.
What the category covers, and what it does not
Four things get called learning analytics in vendor conversations, and they are not interchangeable. Knowing which one you are being sold saves a procurement cycle.
LMS reporting
What the learning platform tells you about itself. Enrolments, completions, attempt scores, activity logs. Scoped to one system, usually scoped to one course at a time, and bounded by whatever the platform retains. Every LMS has it. It is not the thing being sold when someone says analytics.
A learning record store
A place to put statements about learning that happened anywhere, in xAPI format. An LRS is a warehouse, not a reporting tool. It answers "did we capture it", not "what does it mean". Buying one without a reporting layer is a common and expensive half-step.
A general BI tool
Power BI, Tableau, Looker and their peers will model learning data perfectly well. What they do not have is any knowledge of what a completion, a certification window or a recertification lapse means. You supply that, and you maintain it, forever.
A learning analytics platform
A reporting layer that already knows the shape of learning data, reads more than one source, and keeps history past what the source systems hold. The category this page is about. Its value is the modelling you do not have to build, which is also why it is worth checking how much of that modelling actually exists.
A useful test when a demo starts: ask which of these four the product is. If the answer takes more than a sentence, it is probably two of them stitched together and priced as one.
Where learning data actually lives
Most reporting frustration is not a reporting problem. It is that the answer needs three systems and nobody has drawn the map. This is the map for a typical association, healthcare or training-company estate.
| System | What it holds | What it will not tell you | Retention behaviour |
|---|---|---|---|
| LMS (Moodle, Totara) | Enrolment, progress, completion, grades, activity logs | Anything that happened outside a course, and anything about money | Detailed logs are usually pruned on a schedule to keep the database manageable |
| Credential or certification registry | Certification state, expiry dates, recertification windows | Why someone lapsed, or what they did instead | Usually current state only, with limited history of prior cycles |
| Commerce or membership system | Transactions, member pricing, renewals, non-dues revenue | Whether the thing purchased was ever completed | Kept for financial retention periods, which rarely match learning ones |
| Learning record store | xAPI statements from simulations, coaching, on-the-job assessment | Anything nobody instrumented, which is most things at first | Retained indefinitely, which is both the point and the cost |
| HRIS | Role, department, manager, start and leave dates | Anything about learning; it is the dimension table, not the fact table | Current state, often with poor history of role changes |
| Survey and evaluation tools | Reaction scores, self-reported confidence, free text | Whether any of it predicted behaviour | Frequently orphaned when the tool is replaced |
The retention column is the one people discover late. A year-over-year comparison is only possible back to the earliest date the underlying system still holds, and no platform can reconstruct data that was pruned before it was connected. If long-range trend reporting matters to you, the date you connect a platform is the date your history starts.
Completion is not evidence
A completion record says a person reached the end of a sequence of screens. It does not say they can do the thing, that they will still be able to do it in eleven months, or that the organisation is now safer. Every learning analytics conversation that goes badly goes badly at this point, because the report says 98% and the room does not believe it.
The gap is not closed by a better chart. It is closed by deciding, before you buy anything, what would count as evidence for the specific claim you have to make. For a compliance lead that is usually currency: how many people hold a valid credential today, not how many finished a course at some point. For an education director it is usually behaviour change or renewal. For a training company it is usually whether the people who bought the thing came back.
Write that sentence down first. It determines which systems you need connected, which is the only thing that determines what a platform is worth to you.
The four stages, and the question each one answers
Maturity models are usually vendor theatre. This one is useful only because it tells you what to stop trying to do. Find the stage you are actually at, then look at the next row, not the last one.
Stage one · Extract
Someone exports a CSV when asked. The question they can answer is "how many people completed this course". It arrives days later and is out of date on arrival. The bottleneck is a person, and every new question costs that person an afternoon.
Move on when: the same export is requested more than monthly.Stage two · Scheduled
The same reports run themselves and land in the right inboxes. The question is unchanged, but nobody is waiting. This stage is badly underrated. For a lot of organisations it is the whole return, and it can often be reached inside the LMS you already own.
Move on when: the answer needs a system the LMS cannot see.Stage three · Joined
Learning data sits alongside credential state and money. Now you can answer "which programmes hold their members" or "who is certified and current", which are questions about the organisation rather than about a course. This is where a learning analytics platform earns its price, and the earliest point at which it does.
Move on when: people trust the joined numbers enough to argue about them.Stage four · Anticipatory
You act before the outcome. Flagging a cohort drifting toward a lapse in sixty days, or a programme whose renewals are softening. Worth saying plainly: this requires clean history at stage three for long enough to see a pattern, and most organisations shopping for it are actually at stage one or two.
There is no stage five. Anyone selling one is selling a roadmap.Pick metrics by who is accountable
Most metric lists are organised by what is easy to collect. That is how dashboards fill up with numbers nobody is judged on. Organise by the person who has to answer for it instead, and the list gets short.
| Who | Judged on | The metric that matters | The trap |
|---|---|---|---|
| Compliance lead | Passing the audit | Share of the required population currently holding a valid credential, with the expiry date visible | Reporting cumulative completions, which counts people who lapsed two years ago |
| Association education director | Member renewal and engagement | Whether members who took education renew at a different rate than members who did not | Course satisfaction scores, which correlate with almost nothing that matters |
| Training company owner | Revenue and repeat purchase | Repeat purchase rate by course and by cohort, joined to completion | Enrolment counts, which look best in the quarter before refunds land |
| Clinical or safety educator | Practice change and incident rate | Time from assignment to competency sign-off, by unit | Assigning more training as the response to every finding |
| Finance | Cost and forecast | Cost per current credential holder, not cost per seat sold | Treating the LMS licence as the cost, and ignoring the administrative hours |
| Executive or board | Risk and defensibility | One number, with the record behind it available on request | A dashboard with eleven tiles and no argument |
If a metric cannot be traced to a row in a system when someone challenges it, it will not survive the meeting where it matters. Traceability is a requirement, not a nice-to-have, and it is worth testing in a demo with your own awkward question rather than the vendor's example.
Why these projects stall
Six patterns account for most of it. None of them are technical.
No named question
The project was approved as "better visibility". Nobody wrote down the sentence the organisation needs to be able to say, so there is no test for done and the scope grows until the budget runs out.
The data was never going to support it
The question needed three years of history and the source system holds nine months. Discovered after purchase, usually around month four, and there is no remedy.
One person holds the model
The reports work because someone understands the joins. They leave, or move team, and the reports become uneditable rather than wrong, which is harder to notice.
Numbers nobody owns
The dashboard is built and shared and then not looked at, because no individual is accountable for any figure on it. Reporting nobody is judged on stops being maintained within two quarters.
The first number was wrong in public
An early report disagreed with a figure a senior person already believed, the discrepancy was never chased down, and confidence never recovered. Reconcile against a known-good number before anyone outside the project sees anything.
Bought at the wrong stage
A platform bought to solve a stage-one problem that scheduled reporting inside the existing LMS would have solved. The tool works and the value is still disappointing, because the constraint was never the tooling.
Native reporting, a BI tool, or a platform
Three real options, plus the one most vendor pages leave out. The honest version of this section includes the cases where the answer is to buy nothing.
| Option | Right when | Wrong when | Real cost |
|---|---|---|---|
| Native LMS reporting | Your questions are about courses, one system holds the answer, and scheduling would remove the bottleneck | The answer needs credential state or money, or you need history past the platform's retention | Configuration time, and the ceiling you will eventually hit |
| A general BI tool | You already own one, you have analyst capacity, and learning is one of several domains it will serve | There is no analyst, or the learning model would be maintained by whoever has spare time | Licences are the small part. The data model and its ongoing maintenance are the real cost |
| A learning analytics platform | You are at stage three, the question crosses systems, and the people who need answers are not analysts | You are at stage one, or the question has not been written down and agreed | Subscription, integration effort, and the internal owner it still requires |
| Do nothing yet | The named question is not agreed, or the underlying data does not exist to support it | The reporting bottleneck is visibly costing named people days each month | Nothing, which is why it is worth considering seriously |
If your LMS is Moodle, our guide to Moodle analytics and reporting shows how far native reporting goes before anything else is worth buying.
Six questions to ask any vendor
Including us. These are the ones where answers actually differ between products, which is why they rarely come up in a scripted demo.
Does it read the source, or a copy?
Many tools import a nightly extract. Ask what the latency is, and what happens to a field the extract does not carry. Neither answer is disqualifying, but only one supports "as of this morning".
Who builds the second report?
The first report is always built for you. Ask to watch someone who is not an analyst build a new one during the demo, with a filter you invent on the spot.
Does it hold history the source discards?
If the platform reads from logs the LMS prunes, your trend line has a floor. Ask where history is kept and for how long, in writing.
Who owns the data if you leave?
Ask for the export format and the retention term before signing, not after. A platform that cannot hand back a usable extract is a platform you cannot leave.
Can it trace a number to a record?
Take one figure on the demo dashboard and ask to see the underlying rows. If that path does not exist, the number will not survive its first challenge.
Where is it hosted?
Canadian and EU data residency is a procurement gate, not a preference. Confirm the region before the demo rather than during contracting, and ask whether support staff access data from elsewhere.
A first ninety days that does not collapse
The common failure is building everything at once and launching to an audience that did not ask for it. This sequence is deliberately narrow.
Days 1 to 15 · Write the sentence
One question, one named owner, one deadline it has to be answered by. If more than three people need to agree on the wording, that disagreement is the actual project and it is better to have it now.
Days 15 to 40 · Connect only what that question needs
Resist connecting every system because the integration is available. Each source added before it is needed is scope, latency, and another thing to reconcile when a number looks wrong.
Days 40 to 60 · Reconcile in private
Produce the number and check it against something already trusted, such as last quarter's manual count. Chase every discrepancy to its cause before anyone outside the project sees a figure. This step is where credibility is won or lost.
Days 60 to 90 · Ship to one audience, on a schedule
One report, one recipient group, arriving without anyone requesting it. Add the second question only after the first has survived a month of real use and someone has complained about something specific.
Access, residency, and what procurement asks
Learning data is people data. In healthcare and member organisations it routinely includes information about individuals who never agreed to be analysed at the level a dashboard makes possible. Four things worth settling early, because they are slow to retrofit.
Who can see an individual
Decide whether managers see named learners or only aggregates, and at what group size an aggregate stops protecting anyone. A cohort of four is not anonymous. Role-based access is standard in this category, so ask how granular it goes rather than whether it exists.
Where the data rests, and who reaches it
Hosting region is the question procurement asks. The one it forgets is whether support and engineering staff access production data from other jurisdictions, which is a separate answer and often a different one.
How long you keep it
Analytics pushes toward keeping everything and privacy practice pushes toward keeping less. The two are genuinely in tension. Set a retention period deliberately rather than inheriting whatever the platform defaults to.
What happens on exit
Export format, the window you have to retrieve it, and deletion evidence afterwards. Ask for all three as contract terms. A vendor that answers these easily has been asked before, which is itself informative.
Terms that get used loosely
| Term | What it actually means |
|---|---|
| xAPI | A specification for recording statements about learning experiences in an actor, verb, object form, including experiences outside a course. Replaces SCORM's assumption that learning happens inside a package. |
| SCORM | An older packaging and tracking standard. Still ubiquitous, still fine for course content, and blind to anything that happens outside the course window. |
| Learning record store | The database xAPI statements are written to. Storage, not reporting. |
| cmi5 | A profile that lets xAPI content be launched and tracked from an LMS, closing part of the gap between the two standards above. |
| Descriptive, diagnostic, predictive, prescriptive | The standard four-type framing of analytics: what happened, why, what will happen, what to do. Useful as a ladder, frequently used as a sales device. |
| Learner analytics | Usually a synonym for learning analytics. Occasionally used to mean individual-level reporting specifically, so it is worth asking which is meant. |
| Currency | Whether a credential is valid right now, as opposed to whether it was ever earned. The distinction most compliance reporting gets wrong. |
| Non-dues revenue | Association income from sources other than membership fees, with education and certification prominent among them. |
Frequently asked, answered plainly
What is learning analytics?
The measurement, collection and analysis of data about learners and their context, in order to understand and improve learning. In corporate and association settings it usually means joining activity data from the LMS to credential state and revenue, so the organisation can answer what its training actually produced. Fuller treatment: our guide to learning analytics.
What is the difference between LMS reporting and learning analytics?
LMS reporting describes what happened inside the LMS. Learning analytics puts that alongside data the LMS does not hold, keeps history the LMS discards, and supports questions the original report was not designed for. Reporting answers a question you already asked. Analytics lets you ask a new one without a developer.
Does Moodle have built-in analytics?
Moodle ships course reports, logs and a configurable report builder, and they are genuinely capable at course level. The limits appear across the estate: detailed log retention is bounded, cross-course reporting is awkward, and nothing joins to commerce or credential data. More on working within and past those limits: enhance learning with LMS data analytics.
Do I need xAPI?
Only if meaningful learning happens outside your courses and you intend to instrument it: simulations, coaching, on-the-job sign-off. If everything happens inside the LMS, xAPI adds a component without adding an answer. Background: what xAPI and a learning record store are.
How is learning analytics used in higher education?
Mainly for retention and early intervention, and for evaluating programme effectiveness across cohorts. The techniques carry over to corporate and association training, though the outcome being measured shifts from persistence to compliance or renewal. More: five types of analytics used in higher education.
How long before it is worth anything?
Scheduled reporting on data you already hold can pay off inside a quarter. Anything depending on trend history pays off no sooner than the history allows, which means the honest answer to "when will we see a year-over-year view" is one year after connection, unless the source system already held that history.
Can we build this ourselves?
Yes, and organisations with analyst capacity and an existing BI stack sometimes should. The cost people underestimate is not the build. It is maintaining the learning data model through every LMS upgrade, course restructure and reorganisation, indefinitely, usually by one person.
What does it cost?
Platform subscriptions in this category are typically priced on learner volume or number of data sources, and they are not the whole cost. Budget for integration effort and for an internal owner, because a platform without a named owner reverts to stage one within a year regardless of what it can do.
Every learning analytics guide in one place
Choosing
Measuring
Dashboards
Data and standards
Read further, or bring us a hard question
If you are working through the decision, the guide to what learning analytics is covers the definitional ground, and the piece on LMS data analytics goes deeper on working with Moodle data specifically.
Lambda Learning Inc. has built learning platforms since 2002, and Zoola Analytics is our answer in this category. If you want to test it against the six questions above, or work out with us whether you are at the stage where buying anything makes sense, that is a useful conversation either way.
.svg)
%20(1).svg)