This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (12) Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Weitzman, B. C. Articles by Dillman, K.-N. Search for Related Content Social Bookmarking                         What's this?

Integrating a Comparison Group Design into a Theory of Change Evaluation: The Case of the Urban Health Initiative

Graduate School of Public Service, New York University, 40 West 4th Street, Tisch Hall, 6th Floor, New York, NY 10012, USA, beth.weitzman{at}nyu.edu

Diana Silver

Keri-Nicole Dillman

This paper describes how we strengthened the theory of change approach to evaluating a complex social initiative by integrating it with a quasi-experimental, comparison group design. We also demonstrate the plausibility of selecting a credible comparison group through the use of cluster analysis, and describe our work in validating that analysis with additional measures. The integrated evaluation design relies on two points of comparison: (1) program theory to program experience; and (2) program cities to comparison cities. We describe how we are using this integrated design to evaluate the Robert Wood Johnson Foundation’s Urban Health Initiative, an effort that aims to improve health and safety outcomes for children and youth in five distressed urban areas through a process of citywide, multi-sector planning and changed public and private systems. We also discuss how the use of two research frameworks and multiple methods can enrich our ability to test underlying assumptions and evaluate overall program effects. Using this integrated approach has provided evidence that the earliest phases of this initiative are unfolding as the theory would predict, and that the comparison cities are not undergoing a similar experience to those in UHI. Despite many remaining limitations, this integrated evaluation can provide greater confidence in assessing whether future changes in health and safety outcomes may have resulted from the Urban Health Initiative (UHI).

American Journal of Evaluation, Vol. 23, No. 4, 371-385 (2002)
DOI: 10.1177/109821400202300402


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				B. C. Weitzman, T. Mijanovich, D. Silver, and C. Brecher Finding the Impact in a Messy Intervention: Using an Integrated Design to Evaluate a Comprehensive Citywide Health Initiative American Journal of Evaluation, December 1, 2009; 30(4): 495 - 514. [Abstract] [PDF]

				C. Brecher, C. Brazill, B. C. Weitzman, and D. Silver Understanding the Political Context of "New" Policy Issues: The Use of the Advocacy Coalition Framework in the Case of Expanded After-School Programs J. Public Adm. Res. Theory., May 30, 2009; (2009) mup008v1. [Abstract] [Full Text] [PDF]

				A. Blamey and M. Mackenzie Theories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges? Evaluation, October 1, 2007; 13(4): 439 - 455. [Abstract] [PDF]

				E. M. Howell and A. Yemane An Assessment of Evaluation Designs: Case Studies of 12 Large Federal Evaluations American Journal of Evaluation, June 1, 2006; 27(2): 219 - 236. [Abstract] [PDF]

				H. Sullivan and M. Stewart Who Owns the Theory of Change? Evaluation, April 1, 2006; 12(2): 179 - 199. [Abstract] [PDF]

				W. L. Beery, S. Senter, A. Cheadle, H. P. Greenwald, D. Pearson, R. Brousseau, and G. D. Nelson Evaluating the Legacy of Community Health Initiatives: A Conceptual Framework and Example From the California Wellness Foundation's Health Improvement Initiative American Journal of Evaluation, June 1, 2005; 26(2): 150 - 165. [Abstract] [PDF]

				C. Brecher, D. Silver, C. Searcy, and B. C. Weitzman Following the Money: Using Expenditure Analysis as an Evaluation Tool American Journal of Evaluation, June 1, 2005; 26(2): 166 - 188. [Abstract] [PDF]

				J. A. Supovitz and B. S. Taylor Systemic Education Evaluation: Evaluating the Impact of Systemwide Reform in Education American Journal of Evaluation, June 1, 2005; 26(2): 204 - 230. [Abstract] [PDF]

				M. Mackenzie and A. Blamey The Practice and the Theory: Lessons from the Application of a Theories of Change Approach Evaluation, April 1, 2005; 11(2): 151 - 168. [Abstract] [PDF]

				M. M. Mark From the Editor American Journal of Evaluation, June 1, 2004; 25(2): 137 - 140. [PDF]

				M. Bamberger, J. Rugh, M. Church, and L. Fort Shoestring Evaluation: Designing Impact Evaluations under Budget, Time and Data Constraints American Journal of Evaluation, March 1, 2004; 25(1): 5 - 37. [Abstract] [PDF]

				R. E. Crew Jr. and M. R. Anderson Accountability and Performance in Charter Schools in Florida: A Theory-Based Evaluation American Journal of Evaluation, June 1, 2003; 24(2): 189 - 212. [Abstract] [PDF]