Evaluation Capacity and Nonprofit Organizations: Is the Glass Half-Empty or Half-Full?

Carman, J. G., Fredericks, K. A. — Thu, 12 Nov 2009 16:54:01 PST

In this article, we explore the evaluation capacity of today’s nonprofit organizations. We report the findings of a cluster analysis that suggest that when it comes to evaluation, there are three types of nonprofit organizations. The first type of nonprofit organization is one that, by most accounts, is satisfied with their evaluation efforts. Although these organizations report that they struggle with not having as much time as they would like to devote to evaluation, they are fairly satisfied with their levels of evaluation expertise and report having few problems with the implementation of evaluation systems. The second type of organization has some struggles with evaluation. These organizations report having internal support for evaluation from management, the board and staff, and some capacity to implement an evaluation system, yet they struggle with evaluation design issues, data collection, and resources for evaluation. The third type of organization is one that is struggling across the board. These organizations report having substantial implementation challenges, in terms of lacking basic resources (i.e., staff, funding, time), lacking evaluation expertise, and they report having little support for evaluation from funders, the board, management, and staff. We conclude by exploring the implications of these findings.

Improving Data Quality in Large-Scale, Performance-Based Program Evaluations

Aldridge, M. L., Kramer, K. D., Aldridge, A., Goldstein, A. O. — Thu, 17 Sep 2009 15:11:38 PDT

This article examines a systematic education and skill development strategy used for improving data quality in a large-scale, statewide program evaluation. This approach includes streamlining data collection procedures, operationalizing measures, developing and disseminating codebook information, and training those responsible for data entry. This data quality improvement strategy was examined in a process and outcome evaluation of North Carolina’s statewide comprehensive tobacco control program. We calculated error rates in data (i.e., data did not match code book definition or construct) submitted by program coordinators before and after implementation, and we observed a 62% decrease in the total number of errors after the intervention was implemented (p = .014; 95% confidence interval [CI]: 17.6, 144.5). This article demonstrates that the intervention was effective in improving data quality, so that data entered matched operational definitions, thereby improving confidence in and validity of evaluation results.

American Journal of Evaluation RSS feed -- OnlineFirst Articles

Evaluation Capacity and Nonprofit Organizations: Is the Glass Half-Empty or Half-Full?

Improving Data Quality in Large-Scale, Performance-Based Program Evaluations