Luna LXD
Home / Flight Log / Beyond the smile sheet: measuring …

Most learning gets measured twice: did they finish, and did they like it. Both numbers are easy to collect and neither answers the question that funded the project — can people now do the thing?

Start from the decision, not the content

Kirkpatrick's levels are old but the logic holds: reaction, learning, behaviour, results. The trick is designing measurement into the course rather than bolting a survey on after. If the course exists so people make a better decision — escalate earlier, sequence a process correctly, choose the right option — then the assessment should be that decision, simulated. A scenario question with realistic distractors is a measurement instrument; a recall quiz is a memory test.

Cheap signals that beat surveys

Three data sources most teams already have and rarely read: wrong-answer patterns (which distractor is winning tells you exactly what's misunderstood), drop-off points (where attention dies is where the design failed, not the learner), and question logs from any chat or support channel (the questions people still ask after training are your gap analysis, written for you).

Behaviour needs a baseline

The strongest evidence is a before/after you planned in advance. It doesn't have to be a controlled trial — "how many minutes of a first consultation go to re-explaining basics" or "how many submissions bounce for the same documentation error" are countable this month and countable again next quarter. One real operational number, honestly sourced, is worth a page of five-star reactions.

None of this requires a research department. It requires deciding, at the design stage, what changed behaviour would look like — then building the course, and the measurement, backward from it.

The practice

Got a course that has to work?