Corpus Approaches To Evaluation PDF Download

Linguists use the concept of discourse community (DC) to describe the speech of groups of people associated by some extrinsic purpose or interest (Swales, 1990). One defining feature of a DC is a shared lexicon. Scriven (1994), in his discussion of evaluation as a discipline, addresses the need for evaluation to have a "core subject" which would then allow, among other things, the development of "concepts and language to deal with core problems," while noting that there exists a definite lack of clarity of language among evaluators (p. 147). Christie and Rose (2003) suggest that the lack of clarification around definitions among evaluators is not simply a concern for linguists, raising fears that without this clarity, there could arise "a range of folk theories of evaluation that drive considerable real-world practice" (p. 42). They call for "a focused study of the language of evaluation [that] might help us to answer these lines of inquiry" (p. 42). To date, such a "focused study" of evaluation language has not occurred. The question as to whether the evaluation community in general and the community of evaluation scholars more specifically, constitutes a DC has never before been considered and it has implications for evaluation's standing as a field. The purpose of this study is to examine the issue of whether the evaluation community indeed possesses its own lexicon to the extent that would lend credence to its status as a DC. This study explores the lexicon and linguistic patterns of evaluation scholars using a set of established methods from corpus linguistics. CEJA2019, an approximately four-million-word corpus of scholarly evaluation literature, is analyzed using the Sketch Engine corpus linguistic platform. A word list and keyword list are generated. Other exploratory linguistic analyses are conducted. The results of this study indicate a low level of specialized vocabulary. What specialized vocabulary does exist mainly relates to taxonomical distinctions between different theories, models and approaches of evaluations. The findings also suggest more precision of use among evaluation scholars surrounding central evaluation vocabulary including "evaluation" and "assessment." This study lends credence to Shadish's (1998) assertion that "evaluation theory is who we are" and may allay the concerns of those who fear a lack of precision in evaluation definitions.