Evaluating Scholarship: Digital and Traditional

Grant-funded research represents work that has received approval from experts across the field through a rigorous form of “pre-publication” blind peer review for the variety of expressions of digital humanities projects. Grant proposals, particularly ones that receive federal funding from agencies like the National Endowment for the Humanities, are more heavily scrutinized and evaluated than a book prospectus, article abstract, or manuscript draft.

You are under review. Driven by fear, ignorance, and suspicion, your personnel committee is hostile to digital work and the plunging descent of scholarly standards that it represents. Where in God’s name is the peer-review? Who can keep up with publication formats whose shelf life is like unto that of soup? Who can even understand this stuff? How many journal articles is it equivalent to? Your review is doomed unless you can explain your work in 250 words. Go.

Semantic-Web technologies and Linked-Open-Data (LOD) have considerably affected the way data are published on the web. The adoption of LOD has important consequences also on the question of how digital scholarship is used and evaluated. By aligning data on common vocabularies and ontologies, the LOD paradigm introduces a form of standardization between projects, thus allowing for a more informed comparison. Moreover, interoperability makes projects easier to update and more discoverable, and thus more open to be used and debated by a larger community.

The first question is not how to evaluate digital scholarship but to articulate the contribution that scholarship on the Greco-Roman world makes to society as a whole. In practice, most departmental reviews of scholarship assume that the field deserves to exist and the paid professionals within have full autonomy to decide how to assess what does and does not matter.

This paper will discuss how a multidisciplinary department made up of classicists and other humanities scholars at a large public research university developed and implemented a policy for evaluating digital scholarship. The challenges will be familiar to most faculty members, regardless of the nature or size of their departments. In this instance, the majority of faculty members were not engaged in digital scholarship of any kind. Some were firmly committed to the book as the gold standard of scholarly productivity.