Program Outcomes: A Closer Look
Summative evaluation, which focuses on program outcomes, is a major component of many evaluation efforts. In a practical sense, the creators of programs must demonstrate at least short-terms outcomes attributable to the program to maintain funding and/or satisfy themselves that some level of change has been achieved. It is unlikely that a program will produce long-term outcomes if short-term outcomes have not been documented.
Kirkpatrick has written extensively about the evaluation of training programs with a focus on program outcomes (Kirkpatrick and Kirkpatrick, 2006). According to Kirkpatrick, program outcomes can be classified into four different types:
- Participants' reaction to the program;
- Learning as a result of program participation, including attitudes, knowledge and skills;
- Changes in behavior as a result of the program;
- Real world results of the program.
Kirkpatrick's Model of Program Outcomes
Reaction
This is the frequently encountered measure of consumer satisfaction. Favorable reaction to a program is important for the on-going success of the program. Program participants who react positively to a program are more likely to have been engaged by the program, are more likely to make positive comments to others about the program, and are more likely to return for future programs.
Learning
All programs are intended to produce a change in attitudes, knowledge and/or skills, and learning outcomes are a reflection of change in these areas attributable to program participation. Evaluation focuses on achieving specific learning objectives, as defined by the program objectives.
Behavior
This area of evaluation focuses on the extent to which changes achieved during the training program are extended beyond the program to the workplace. A program can achieve learning outcomes without achieving behavioral outcomes if there are insufficient rewards or work climate to maintain or enhance the changes achieved during the program.
Results
We all want results! In the case of summative evaluation, this represents the real world changes expected as a consequence of the program, such as improved quality, more efficient procedures, better trained students and faculty and fewer dismissals.