Tag Archives: Robert Brinkerhoff
More on Re-evaluating Evaluation – Jack Phillips and ROI

More on Re-evaluating Evaluation – Jack Phillips and ROI

by on August 19, 2011 in Evaluation

I have been blogging a lot about Training Evaluation this year—mostly Kirkpatrick, but also Brinkerhoff and Scriven.  I just realized that I haven’t included a single word about Jack Phillips, who introduced Return on Investment (ROI) as Level 5 to Kirkpatrick’s Four Levels of Evaluation.

My first exposure to Phillips’ ROI—although I didn’t realize it at the time—was through a colleague who introduced me to Kirkpatrick’s Four Levels.  Her four levels were Level 1: Reaction, Level 2: Learning, Level 3: Behavior, and Level 4: ROI.  And for a long time, that is what I thought the four levels were.  Years later, I realized that she had skipped Kirkpatrick’s Level 4: Results and substituted Phillips’ ROI.

So what exactly is ROI and, as Level 5, what does it measure?  First of all, it builds on Level 4: Results, which are the business results linked …

Read full story • Comments { 1 }
Kirkpatrick Revisited

Kirkpatrick Revisited

by on April 18, 2011 in Evaluation, Instructional Design

After I finished my post a few weeks ago on Reevaluating Evaluation, I found out that Donald Kirkpatrick, the granddaddy of the Four Levels of Evaluation, was taking a farewell tour before his retirement and would be presenting a workshop at the American Society of Training and Development (ASTD) in Minneapolis.

It was an opportunity I couldn’t miss.  I have included Kirkpatrick’s Four Levels of Evaluation in every proposal I have ever written, and I wanted to hear from Kirkpatrick himself regarding his take on the current state of evaluation and whether his four levels are still viable.

So I went to the workshop.  I wanted to get a more in-depth knowledge of the four levels and where the next generation, Kirkpatrick’s son James, was taking them.

I said in my last post that Kirkpatrick’s four levels were all about the training itself rather than how the training …

Read full story • Comments { 2 }
Re-evaluating Evaluation

Re-evaluating Evaluation

by on March 16, 2011 in Evaluation

For years, I have dutifully included a description of Kirkpatrick’s Four Levels of Learning Evaluation in every proposal for every company I have worked with.  And every company has agreed to use “Level 1: Reaction,” or―as it has come to be known—the “Smile Sheet.”  Some companies will use “Level 2: Learning” to measure whether the learners have mastered the training course content.  Hardly ever do they use “Level 3:  Behavior,” and they never use “Level 4: Results.”

I have found this to be extremely frustrating.  And as time has gone by, I have started to wonder about the validity of Kirkpatrick in today’s world.  The focus is on the training event itself and the follow-up to that event.  What is measured doesn’t seem to be what companies are interested in.  Company executives are typically interested in the bottom line, not how well their employees apply the learning from a training …

Read full story • Comments { 2 }