Reconstructor for Exams

 

A project by Dr. Andrew Fluck at the University of Tasmania

With assistance from Joshua Hesketh of Open Technology Solutions

 

 

Basic concept

The Australian Government has decided all students in Years 9-12 will have a computer at school all day from 2010. Teaching and assessment should jointly prepare for this new environment. Therefore we may need a method for conducting external assessments in a fair way with this equipment.

 

Putting exams onto computers

Reconstructor for Exams is currently in second generation pilot stage. For this system the examiner provides:

  1. a digital folder of materials for candidates
  2. a unique picture for a background
  3. a list of functions to be disabled or enabled, such as networking, wireless connections, local hard drive access, sound output etc.

The Reconstructor for Exams system then generates a ‘live’ Ubuntu CD which boots on PCs or Intel-based Macs with the digital folder of materials on the desktop and the unique picture in the desktop background. Students use the CD on a computer in replacement of the examination paper, answer booklets and information sheet.

 

Examples

Indicative samples of new digital examinations are provided on a sample disk as follows:

Ancient Civilisations: answers are typed into four answer booklet documents.

Biology: lavishly coloured photographs are included for students to comment upon.

Chemistry: answers are placed into text form boxes on a single examination paper file.

Music: Music files can be played through headphones and commented upon in the typed answer paper.

Sport: Student observe sporting incident videos and respond orally using a microphone.

Health: an interactive simulation is provided, and students answer questions based upon deep understanding verified by the settings achieved on the simulator.

Information Technology & Systems: Uses a Wiki for recording student responses.

 

Advantages

Reconstructor for Exams has several important advantages:

§  Portable – can be set up using almost any available computers (PCs and Intel-based Apple Macs)

§  Equitable – accessible to a wide range to students

§  Familiar – students can have the opportunity to practice essential skills in this environment, because the basic system is free of restrictive copyrights

§  Technical capacity – it does not limit students’ creativity or expression

§  Archival – the environment produces material which will be accessible in future years

§  Inviolate – students cannot alter the environment to gain an unfair advantage.   (Fluck, 2004b)

 

Pilot Outcomes

Third Year University students who have trialled the system responded to an evaluation survey with comments such as:

“This was great! I can type much faster than I can write and no hand cramps!! J

“I believe that this testing way was better than hand writing for the reasons such as I am able to type faster than handwrite and your hands didn’t cramp up.”

“I think it is better on cXXXX paper as yXX you can writer faster.” [Alterations in handwriting].

82% of the students had used the practice CD before coming to the examination, and 61% said that, on balance, it was better to have formal tests conducted using computers instead of handwriting on paper. They also found the noise of computer keyboards a distraction during the eExam. The new system allowed eExaminations to be supervised by invigilators without specialist information technology skills.

 

Future pathways

Computer-based exams can have many new assessment dimensions:

Computer Science: the materials could include a compiler for students to run code and capture output for their script.

Physics: we could place a University-class projectile motion simulation into the materials folder (using Java virtual machine) to probe understanding.

Ecology: existing online learning objects could be used

 

The current draft disk requires development work (update from Ubuntu 7.04 to 7.10) to work with Intel-based Macs. Other aspects will be determined from field trials.

 

We hope to transfer the system to cheap USB drives. This will simplify the manual return of scripts (the student records results onto the same USB drive, which is subsequently collected). Alternatively we can network-boot Ubuntu from established Windows workstations within a school to provide a similar environment. This would simplify script collection and timing control. The portability, zero-vendor lock in and flexibility of the system make for a very cheap system. Building on top of open source tools (such as Reconstructor) has also allowed development costs to also be incredibly low. The current disk is just a start – more development and trialling will be needed.