RESPONDING TO THE CHALLENGES OF COMPLEX ONLINE ASSESSMENT.

Powerful Assessment Tools To Streamline Complexity

The University of Edinburgh has been using PebblePad for over ten years, which is now widely embedded in practice. The university has a complex double-blind marking workflow requirement for dissertation marking. The entire assessment workflow is managed in PebblePad’s Assessment Space, ATLAS, since it is the only system available within the university that is capable of managing this complex assessment form online. This process is currently used in three Schools - Law, Business and Divinity – but is being rolled out across the university.

The Double-Blind Marking Ambition 

In this workflow, moderation is a process that sits separately to the marking of assessments to reconcile grades. Double-Blind Marking is a two-stage process. In the first stage, two markers are required to assess a paper unaware of the other’s feedback and grade. Any difference between the grades and feedback can be acknowledged and addressed as part of the second stage in the DBM process. In the second stage, the two markers are able to see each other’s recorded comments and reconcile differences and agree a final mark. In PebblePad, this process is seamlessly supported from the perspective of the students, the markers, and the admin staff.

The key objectives were to:

• Provide the students with an easy way of submitting work and retrieving feedback.

• Create a simple-to-use workflow with detailed documentation for the admin and academic colleagues as well as for the students.

• Make sure that the new tool eased the administrative burden of the Student Services staff, by providing them with online tools for managing assignments, the markers, and tracking changes.

ATLAS has a lot of functionality to support a lot of different assessment activity and administration associated with it. In a process such as double-blind marking, most of these are in use and it all comes beautifully together.

Robert Chmielewski, Online Assessment and Feedback Adviser

The Successful Workflow

In PebblePad’s assessment engine, ATLAS, they found a robust system to support the process that did not compromise the quality and value of the assessment and feedback nor created additional or unnecessary work for the admin team.

Students receive a link to a dedicated PebblePad ATLAS submission page and submit their dissertations directly to this. They receive an automated email confirming that their submission has been successful. It’s beautifully simple.

Markers also receive direct links to the assessments they need to mark. Markers only see the dissertations assigned to them, thanks to the use of PebblePad Sets (a means of easily and securely allocating assessors to specific subsets of students). Marking is completed using custom PebblePad templates to ensure consistency with departmental grading. The dissertations are also run through Turnitin, and the Originality Scores are available to view in the ATLAS submission view. Finally, some markers require paper copies of the dissertations. PebblePad’s batch download feature enables print needs to be handled centrally, making life easier for the students.

Admin users are allocated appropriate permissions to allow them to scrutinise the progress of the marking process. Admin staff can also easily see approval indicators to check when first and second markers have provided their feedback. However, every element of feedback from the entire process is saved securely against the student record for future scrutiny and audit.tutors.

The above shows an example PebblePad workbook and PebblePad's Assessment Space, ATLAS.