Better Trainer: Mr. Miyagi or Coach Norman Dale?

Better Educator - Both

Two of the best movies of all time, “The Karate Kid” and “Hoosiers“, featured plot lines involving strong leaders and the incredible transformation of their pupils. Mr. Miyagi, the aging, apartment complex handyman and Coach Norman Dale, the former NCAA Division III national champion basketball coach, had very different styles. From a learning and development perspective, which one of these immortal teachers was a more effective educator? Continue reading

4 Ways to Better Measure Corporate Training Results

I think results come out in lots of different ways, and some of them you measure, and some of them you feel.

In the January issue of TD magazine, SAP CEO Bill McDermott makes the point that training results aren’t always numbers-driven. I’ve seen this first-hand.

An India-based colleague has spent the last several years holding monthly training sessions focused on our company values and discussing “soft” topics such as teamwork and collaboration. When I dropped by one of these training sessions last month, one of her trainees commented: “In other organizations people try to pull other people down. Our organization is unique in that everybody tries to help each other and boost each other’s performance.”

Sometimes you can feel the results of a training program. But as I mentioned in Monday’s post, companies around the world spend over $75 billion (with a b!) annually and have no idea whether or not their training efforts have produced any results. This isn’t good.

If you happen to be interested in the ability to show other people (your boss, for example) that your training efforts don’t just feel good, but have made a measurable difference, here are four ways to do that:

1. Make sure you ask what should be different as a result of the training.

This one may sound like a no-brainer, but you’d be surprised at how many times training is planned and executed without specifically identifying what should be done new or differently or better as a result.

2. Pay some attention to Kirkpatrick’s Four Levels of Evaluation…

About 60 years ago, Donald Kirkpatrick espoused four “levels” of evaluation to assist training practitioners begin to quantify their results. First come post-training evaluation scores (“smile sheets”), then learning (most of the time through pre/post testing), then skill transfer on the job (maybe a self-reported survey, or a survey from a trainee’s supervisor) and finally impact (did sales increase? did on-the-job safety accidents decrease?). Levels 1 and 2 are most common, but trainers and organizations can certainly strengthen their Level 3 and 4 efforts.

3. …and then go beyond Kirkpatrick.

According to a research paper entitled The Science of Training and Development in Organizations, Kirkpatrick’s Four Levels is a model that can be helpful, but there is data to suggest it is not the be-all-and-end-all that training professionals have pinned their evaluation hopes on. The authors of this paper offer the following example as a specific way to customize the measurement of a training program’s success or failure:

“If, as an example, the training is related to product features of cell phones for call center representatives, the intended outcome and hence the actual measure should look different depending on whether the goal of the training is to have trainees list features by phone or have a ‘mental model’ that allows them to generate recommendations for phones given customers’ statements of what they need in a new phone. It is likely that a generic evaluation (e.g., a multiple-choice test) will not show change due to training, whereas a more precise evaluation measure, tailored to the training content, might.”

4. Continue to boost retention while collecting knowledge and performance data.

Cognitive scientist Art Kohn offers a model he calls 2/2/2. This is a strategy to boost learner retention of content following a training program. Two days after a training program, send a few questions about the content to the learners (this can give data on how much they still remember days after having left your training program). Two weeks later, send a few short answer questions (again, this helps keep your content fresh in their minds and it gives you a data point on how much they’ve been able to retain). Finally, two months after the training program, ask a few questions about how your content has been applied on the job (which offers data on the training’s impact).

If companies as spending billions of dollars on training, never to know whether or not those efforts were effective, there’s a problem. Spending a few hours thinking through your evaluation strategy prior to deploying your next training program can make your efforts literally worth your time.

 

Transfer of Training: A Case Study

I had an opportunity to attend a day-long training session called Hiring Winners which was delivered by a facilitator from Washington Employers. And over the next month, an amazing thing happened. I found that I was immediately using concepts and skills developed during this session. Following is a brief description of how this course seems to have hit upon the Holy Grail of training and development: actual skills transfer from the training room to everyday practice.

The Situation

Working for a rapidly growing organization, our HR team offered the opportunity for hiring managers to spend a day focused on our recruiting and hiring skills. This presented two immediate benefits:

1)      general professional development on an immediate need, and

2)      development of a common, organization-wide experience and language when it comes to recruiting and hiring as we go forward

The Training

Every participant was given a manual as the day started and we spent the day working our way through the manual. The course design included lecture, small group activities, large group discussions and role-plays.

The Transfer of the Training

Probably the most essential element of the session was a series of small group activities in which we were asked to develop a hiring plan and to develop behavior-based interview questions for an actual job for which we would soon be recruiting and hiring. The real-life nature of this activity and opportunity to leave the workshop with an actual plan that could be implemented right away led me to use these tools the following week.

A colleague of mine from India happened to be in town when this course was offered and brought some of her key learnings back to share with our co-workers in New Delhi. Immediately upon her return, she led a 60-minute session in the Delhi office to share highlights and to begin finding ways to transfer the lessons for our India-based context. The team then decided on and implemented several improvements to our ongoing hiring process in India. The team will have a longer session to discuss key concepts from this course after the new year.

What made this Course Sticky?

When so many other courses and workshop manuals simply gather dust on someone’s desk or bookshelf, what made this course achieve transfer of training?

There were several key factors that led to the immediate application and transfer of skills from our training room to our day-to-day routine.

Supervisor Support

My supervisor also attended this session and asked our team to begin using these new skills. He also set the expectation that lessons learned would be shared with other team members – both in the US and in India – who were unable to attend this session.

Immediacy

In espousing his theory on how adults learn best, Malcolm Knowles insisted that adult learners thrive when the education they receive can solve an immediate problem. As our organization (and more specifically, as my team) grows, we’re using hiring skills every day. This course allowed us an opportunity to re-visit our current process and make improvements in the moment during the actual training session. And with interviews already on my Outlook calendar, this was the perfect time to develop more effective interview questions.

Facilitation and Course Design

The facilitator was prepared, had obviously given this presentation in the past, provided a smooth delivery, and offered plenty of time for small and large group discussions. She delivered a course that was designed to offer three things that were of immense value to us: new content, a forum for staff from across the entire organization to discuss issues and align on processes, and an opportunity to either revise or create new ways to recruit and hire top-notch candidates.

Will our organization realize the return on the investment we were seeking when we brought Washington Employers in to facilitate this workshop? It’s too early to tell. But if an early indicator is whether or not people actually use the skills they learned in the training room, then we seem to be off to an unusually good start.

Know someone who could use some help designing learning experiences that will transfer of training from the training room into the real world? Get in touch with us.