This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Web of Science (3) Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Johnson, R. L. Articles by Willeke, M. J. Search for Related Content Social Bookmarking                         What's this?

Using Portfolios in Program Evaluation: An Investigation of Interrater Reliability

College of Education, Department of Educational Psychology and Center for Excellence in the Assessment of Student Learning, University of South Carolina at Columbia, Columbia, SC 29208, johnsrl{at}vm.sc.edu

Fred McDaniel, II

Marjorie J. Willeke

Portfolios and other open-ended assessments are increasingly incorporated into evaluations and testing programs. However, questions about the reliability of such assessments continue to be raised. After reviewing forces that may be leading to increased interest in and use of portfolio assessment, we investigate the interrater reliability of a portfolio assessment used in a small-scale program evaluation. Three types of portfolio scores were investigated—analytic, combined analytic (formed by summing across analytic scores), and holistic. The interrater reliability coefficient was highest for summed analytic scores (r 5 .86). Results indicate that at least three raters are required to obtain acceptable levels of reliability for holistic and individual analytic scores.

American Journal of Evaluation, Vol. 21, No. 1, 65-80 (2000)
DOI: 10.1177/109821400002100106


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?