What actually counts as impact: Kirkpatrick Levels 3 and 4 in practice

Completion rates and happy-sheets are not impact. A 98% completion rate and a room full of "very satisfied" survey responses tell you that people sat through the training and did not hate it. They do not tell you anyone does their job differently, and they certainly do not tell you the organisation is any safer, faster, or more compliant as a result. Impact, in the sense that actually matters, is Kirkpatrick Level 3: behaviour changed on the job. And it is Level 4: the result that behaviour change was supposed to produce.

I design learning for regulated and professional audiences, which means I get asked "did the training work?" more often than most, and I have learned to distrust any answer that stops at attendance figures. This piece sets out what the four levels actually are, and then spends most of its time on the part that is genuinely hard: how you measure Level 3 and Level 4 without kidding yourself.

What are the four Kirkpatrick levels?

The Kirkpatrick Model, developed by Donald Kirkpatrick and maintained today by Kirkpatrick Partners, is the most widely used framework for evaluating training, precisely because it separates four things that get conflated constantly.

• Level 1: Reaction. Did learners find the training engaging and relevant? This is the happy-sheet. It is worth collecting, because a training experience people actively dislike is unlikely to get taken seriously, but it is the weakest signal in the model.
• Level 2: Learning. Did learners gain the knowledge, skill, or confidence the training set out to build? An end-of-module quiz or scenario assessment sits here. It confirms someone can answer correctly in a controlled setting, not that they will act correctly under real conditions.
• Level 3: Behaviour. Are learners actually doing their job differently, back at their desk or on the ward or on the phone, weeks after the training rather than during it?
• Level 4: Results. Has that behaviour change produced the outcome the organisation actually needed, whether that is fewer safety incidents, better compliance, faster resolution times, or something else specific to the business?

Levels 1 and 2 are necessary. They are not sufficient. An organisation that only reports on them is reporting on effort, not on outcome.

What is Kirkpatrick Level 3?

Level 3, Behaviour, is the point where evaluation leaves the classroom and goes looking for evidence in the real work environment. It asks a specific, uncomfortable question: is this person doing anything differently now, compared with before the training, in the actual job?

That is a harder thing to measure than a quiz score, for a simple reason: behaviour change does not usually show up immediately, and it does not show up inside the training itself. Someone can demonstrate a skill perfectly in a simulated scenario and then not apply it under time pressure a month later. Level 3 has to be measured out in the field, over time, which means it cannot be bolted on as an afterthought. It has to be designed for from the start.

How do you measure training impact?

In practice, measuring Level 3 and Level 4 well comes down to a handful of disciplines, most of which have to be in place before the training is even built.

• Define the target behaviour before you design. Not "understand the escalation policy" but "escalates a red-flag case within the defined window, every time." A vague learning objective produces a vague, unmeasurable behaviour. A specific one gives you something to observe.
• Agree the metric that would move if the behaviour changed. Before training starts, decide what evidence would actually confirm the change: a call-handling metric, an audit pass rate, a specific field in a case management system. If you cannot name that metric in advance, you will not be able to attribute a change to it afterwards.
• Use on-the-job observation. Structured observation, shadowing, or spot-checks against a checklist derived directly from the target behaviour, not a general impression of "seems more confident."
• Bring in manager input. Line managers see the behaviour in situ far more often than a learning team does. A short structured check-in with managers, asking about the specific behaviour rather than general satisfaction, is one of the most practical Level 3 tools available.
• Read workflow and system data. Where the job runs through a system, whether that is a case management tool, a procurement platform, or an EHR, the system often already logs the behaviour you care about. Time-to-escalation, correct field completion, adherence to a required sequence of steps.
• Track error and incident rates. In regulated and safety-critical environments, error rates and incident reports are often the most honest Level 3 and Level 4 signal available, because they are recorded independently of the training programme itself.

The leading indicators are the bridge between the two levels. A Level 3 behaviour, such as consistently following a new escalation procedure, is the leading indicator for a Level 4 result, such as fewer missed safety events. If you cannot draw that line from behaviour to result before you start, the evaluation plan is not finished yet.

Why aren't completion rates enough?

Because completion is a Level 1 or Level 2 proxy, and it is entirely possible to hit 100% completion with zero behaviour change. Someone can click through every screen, pass the assessment, and change nothing about how they actually work, either because the training did not connect to their real conditions, because a system or process barrier got in the way, or because nobody ever checked. A completion dashboard that stops at "finished" and "passed" is measuring participation, not impact, and reporting it as if it were the same thing is the single most common evaluation mistake I see.

Being honest about attribution

None of this makes attribution easy, and I would rather say that plainly than pretend otherwise. Results at Level 4 are shaped by far more than one training intervention: staffing changes, new equipment, seasonal variation, a different manager, a policy change happening at the same time. Isolating the training's specific contribution to a Level 4 result is genuinely difficult, and any evaluation that claims perfect attribution should be treated with scepticism.

What you can do honestly is build the chain of evidence: a defined target behaviour, evidence that the behaviour changed after training and not before, and a plausible, documented link between that behaviour and the result you were tracking. That is not the same as proof of causation, but it is a defensible, honest account of impact, which is worth more than a confident-sounding number with no chain behind it.

Return on Expectations and required drivers

The New World Kirkpatrick Model, the current version of the framework from Kirkpatrick Partners, puts particular weight on working backwards from Level 4 rather than forwards from Level 1. It introduces the idea of Return on Expectations: the training should be judged against what stakeholders actually needed it to achieve, defined up front, rather than against generic satisfaction or completion metrics decided after the fact.

It also emphasises required drivers, the reinforcement, encouragement, and accountability structures in the workplace that determine whether a newly learned behaviour actually sticks once someone is back at their desk. A well-designed module can teach the right behaviour perfectly and still see it evaporate within a fortnight if there is no manager reinforcement, no system that supports the new way of working, and no accountability for reverting to the old one. Level 3 measurement, in the New World model, is as much about checking whether those drivers are in place as it is about checking the learner.

How this shapes what I build

I design for behaviour change deliberately, which means the Level 3 and Level 4 questions get asked at the start of a project, not retrofitted at the end. What is the specific on-the-job behaviour this training needs to produce. What evidence, already sitting in a system or a manager's normal routine, would show that behaviour happening. What result is that behaviour meant to protect or improve. Building those questions into the brief changes what the module actually needs to do, because a module built to change behaviour looks different from one built to transfer information and hope for the best.

Frequently asked questions