Abstract

The report, Who Needs Goals?:An Analog Experiment Comparing Goal-Based Evaluation and Goal-Free Evaluation Utility describes the outcomes of a study on the perceived utility of goal-based evaluation (GBE) and goal-free evaluation (GFE). GBE is an evaluation in which evaluators measure a program’s attainment of its goals and objectives. By contrast, GFE is an evaluation in which the evaluators purposefully avoid the program’s goals or objectives.

The experimenters randomly assigned two groups of evaluators which were comprised of Master’s level evaluation students at a Master’s large university, to conduct either a GBE or GFE. The subject of the evaluation was a training program provided to a camp for individuals with ability needs and was facilitated by 30 third semester occupational therapy (OT) Master’s students enrolled in an OT practice course The evaluations were distributed to the evaluation users, i.e., the 30 OT students. After reading each of the evaluation reports, the evaluation users completed a semantic differential questionnaire to assess their attitude toward the utility of each report and then participated in a follow-up focus group. The findings of the surveys indicated that there is a marginally significant difference in utility between the GBE and GFE reports. In other words, the evaluation users who reviewed the GFE were more likely to rate the report in a more positive manner.

The findings of the focus group somewhat contradicted the findings of the questionnaire as 14 of the 26 respondents preferred the GBE report. Many of the evaluation users’ preferences stemmed from the readability of the reports and specific recommendations. The conclusion is that there is no significant difference in perceived utility between GBE and GFE according to the evaluation users.