Quality Review
The Quality Review Team decides how to evaluate content before it is accepted into the repository. Ideas about quality review vary widely among repositories. Some repositories have strict guidelines and review policies for content quality, while others have few or none. For example, Rice University Connexions allows anyone to contribute resources with no review imposed. The Connexions philosophy is to make the resources available, but not offer an endorsement as to the quality of the items. Quality is determined by the users. However, if users consistently find resources that lack accuracy or educational value, repository usage will decrease over time.
In contrast, other repositories mandate a defined level of quality. An example of this approach is Intute, a UK-based repository of web resources which has a network of subject specialists who select and evaluate the websites in their database.
There are a variety of models for repository quality review. We will discuss two basic strategies, a centralized model and a distributed model. The ability of repository software to attach a review process to resources enables the review process to be performed asynchronously by one or more individuals. In the centralized model, repository administrative staff performs the reviews. This centralized control offers the opportunity to maintain process integrity by controlling those who review the materials and the review criteria. Even with specified criteria, when differences in interpretation may arise, a centralized model can provide the opportunity for immediate discussion. In a distributed model, the reviews are dispersed, perhaps among institutions or state agencies. While this reduces the burden on repository staff, it also offers opportunities to deviate from the process and requires more active communication and training for the reviewers.
When setting up your quality review process, we recommend that you institute at least a minimal review of contributed resources which will establish a review process as part of the repository culture.
The Orange Grove mandates a quality review process to ensure resource accuracy, instructional and editorial quality, and adherence to the repository technical and metadata standards. For each defined resource type in the repository, there is a virtual quality review process that requires from one to five steps. Items may be reviewed by:
- Subject Matter Expert (SME) to ensure content is accurate. Usually performed by faculty (other than the resource author) with subject area expertise
- Instructional Designer to ensure that the content aligns with the resource’s objectives and that it contains appropriate events of instruction. This review is not required for resources that do not contain instructional components or do not have an instructional purpose: e.g., syllabi, lesson plans, pictures or charts, and presentations
- Editor to ensure correct grammar and spelling
- Technical and Accessibility Validator to ensure that IMS/SCORM packages are conformant and that content displays correctly in a learning management system. If ADA requirements are to be met, this can take place here.
- Metadata Reviewer to review metadata for sufficiency and accuracy.
Resources
