Posted by Paul Nisbet on the 10th June, 2024
Just over a year ago we formed two working groups to investigate practical options for students to use Chromebooks and iPads for Digital Assessment Arrangements in SQA examinations. In this article we'll update on progress in 2023-2024.
Find out more by attending the Assistive Technology and Digital Exams session at our ASL & Technology Conference on Thursday.
Last year I wrote:
Learners in secondary schools in many parts of Scotland have been provided with personal Chromebooks or iPads, but our research has found that students are not generally able to use these devices in examinations. We have been asked by SQA to establish working groups to find solutions.
The purpose of SQA assessment arrangements is to
provide disabled learners and/or those with additional support needs with an equal opportunity to demonstrate their attainment in an assessment.
Digital assessment arrangements involves using a digital question paper and/or assistive technology to access question papers and create a response. Current practice in most schools and centres is for students to use Windows PCs or laptops for digital assessment arrangements. In most centres, technical colleagues create 'Exam profiles' on the PCs, which provide students with the software they need - typically, Adobe Reader to access the SQA digital question papers and answer booklets, sometimes with Microsoft Word for answer booklets in Word format. The Exam Profiles restrict access to the internet and to any tools or supports that are not be permitted in the examination, such as spellcheck.
Research by CALL Scotland [1, 2, 3, 4] and SQA Policy, Analysis and Standards staff [4, 5] identified challenges for staff and students when attempting to use Chromebooks and iPads for digital assessments arrangements:
Feedback from staff, pupils and parents/carers indicates that this situation is unacceptable because pupils in Senior Phase in many areas of Scotland have been provided with personal Chromebooks or iPads and have developed effective skills with their devices to support and access learning in class and internal assessments.
However, they are currently unable to utilise this ‘normal way of working’ in an examination. Instead, pupils are using Windows laptops and desktops for digital assessment arrangements, that have different apps, user interfaces and support tools to those with which they are familiar.
I have been working with a group of S1 pupils, teaching them to use Chromebook text to speech and dictation. Our pupils are capable but struggle to get ideas from their head onto paper. All have dyslexic traits to varying degrees as well as other barriers to learning.
Use of assistive technology on Chromebooks has vastly improved their confidence in, and quality of, writing.
Although this has been a very successful exercise, we are now thinking of how they can access such software for formal assessments and exams. Chromebooks are not suitable for this as they cannot function offline. Pupils cannot have internet connectivity while completing formal assessments.
Staff observed that there is an increasing demand for digital assessment arrangements, but that access to sufficient Windows laptops is problematic because stocks have been replaced by Chromebooks or iPads.
There is therefore an urgent requirement for digital assessment arrangement systems that are:
Students can access SQA digital question papers using Chromebooks (using for example, OrbitNote or Kami) and iPads (with e.g. PDF Expert, ClaroPDF or the OneDrive app) but teachers and technical colleagues report that it is a challenge to configure devices to be secure and practical for examinations.
We researched and tested some secure examination systems for possible use for digital assessment arrangements. The research identified systems that can be categorised into two distinct groups:
We explored COOL Check, Digiexam, exam.net, Examonline, Surpass and Trelson Assessment and of these, identified exam.net and Trelson as systems that might meet Scottish schools' and centres' requirements. The others were rejected because it was not clear that they provide pupils with a means of accessing SQA digital question papers and/or because they lack essential accessibility or assessment tools. Surpass already has a relationship with SQA through SOLAR and also have the current contract to run the National Standardised Assessments for Scotland.
These secure online assessment systems are of wider application and interest if and when SQA moves towards universal online assessment, but for this project we were looking to find a solution that meets the current context based on provision of question papers in PDF.
A recent exciting development is that Pearson (which provides GCSE examinations in England) is offering onscreen assessment for GCSE English Language and Literature examinations in 2025. (GCSE's are equivalent to National 5's.) The Pearson system looks very similar to exam.net, Trelson and the other online assessment systems - in the screen shot below, there is the text for the English assessment on the left, and the questions on the right.
We looked at DocsPlus, OrbitNote, WordQ and Read&Write Chrome and Read&Write for Windows. OrbitNote appeared to offer an option for Chromebooks. .
Colleagues across Scotland, and representatives from Apple and Google attended a number of meetings where the software providers demonstrated their systems and colleagues piloted them with some students.
Colleagues in the four local authorities with 1:1 iPads were keen to trial exam.net and teachers did explore it for internal assessments. One advantage of systems like exam.net and Trelson is that pupils can use any device and simply need to enter a passcode to access the examination. However, to make the examination secure, pupils must have the exam.net app on their iPad or Chromebook, or the 'Safe Exam Bowser' on their Windows device. It is necessary to carry out a Data Protection Impact Assessment before rolling out apps, and this took some months to arrange, by which time the window for piloting the system in 2023-2024 had passed. Teachers and staff were also uncertain about investing time when it was not known whether exam.net would actually be permitted in the 2024 examinations.
Colleagues in several local authorities and schools with 1:1 Chromebooks tested OrbitNote and identified a significant limitation, which was that spellchecking was not provided. Spellchecking is an essential feature: in 2023, SQA received 17,130 requests for pupils to use ICT with spellcheck compared to only 9,325 without. We understand that the new version of OrbitNote which will be available next term will have a spellchecker.
Trelson Assessment is similar to exam.net. It integrates into and requires Google Workspace which is used in some but not all local authorities in Scotland. Trelson requires staff to be able to access Google Admin Console which is not always available for school and centre staff. There was considerable interest in Trelson but no volunteers came forward to pilot the system.
There is clearly an appetite for finding secure, accessible digital assessment arrangement systems that provide a consistent and effective experience for staff and pupils, regardless of whether learners use Chomebooks, iPads or Windows PCs in class.
Exam.net and Trelson (and other similar products) are secure online systems designed for large-scale, high-stakes assessment. They claim to save teacher and administrative time and offer facilities that staff may value for internal assessments, such as auto-marking. They provide a consistent interface for staff and pupils regardless of whether the student is using a Chromebook, iPad or Windows device. Pupils can access SQA digital question papers but they cannot type into or annotate the question-and-answer papers. The accessibility tools are limited - for example, the text readers do not have the option of Scottish or Gaelic voices.
OrbitNote is an accessible PDF reader to which a secure 'exam mode' has been added. It is specifically for Chromebooks.
Centre and local authority staff were keen to trial the products this year but were constrained by:
We submitted a report on the work to SQA in May 2024 [7]. The experience over the last year suggests that staff and pupils require some assurance that these systems are acceptable for use in examinations and so we hope that the report will help inform decisions within SQA.
Ultimately, my own view is that SQA has to provide secure, online examinations that can be accessed by pupils using any device. Tools such as Digiexam, exam.net, Trelson and the system that has been developed by Pearson are really the only way to provide efficient and consistent assessments. There are too many barriers around security, usability, accessibility and practicality with the current digital question papers in PDF and the current processes for administering digital assessment arrangements in schools. OrbitNote and other PDF readers may provide a solution in the short term, but really we need online assessments that are designed for the modern digital learning age.
I think that SQA should draw up a specification for an online assessment for Scottish National Qualifications, detailing the necessary features and functions that are required, and put the project out to tender. Pearson have already created a system for England; there are plenty of firms that offer solutions; and we already have online National Standardised Assessments for Scotland. We need the same for National Qualifications.
Until then though, we'll keep trying to find workable solutions for the 2024-2025 academic session.
[1] Nisbet, P. (2023) SQA Assessment Arrangements and Assistive Technologies in 2022. Research Report commissioned and submitted to Scottish Qualifications Authority. January 2023.
[2] Nisbet, P. (2021) Assistive Technology and Assessment Arrangements following Covid-19. Published Report, commissioned by Scottish Qualifications Authority. October 2021.
[3] Nisbet, P., Lawson, S., Mill, C., Stewart, R. (2020) Chromebooks and SQA Assessment Arrangements: The use of Chromebooks by candidates who require Assessment Arrangements in Scottish Qualification Authority external examinations. March 2020.
[4] SQA (2022) SQA Survey Summary – use of ICT as an assessment arrangement.
[5] SQA (2023) Digital and Adapted Papers Survey 2022 and 2023.
[6] Nisbet, P. (2024) SQA Digital Assessment Arrtangements: an investigation into secure online assessment systems. Research Report commissioned and submitted to Scottish Qualifications Authority. May 2024.
Newsletter: join thousands of other peopleOnce a month we'll send you an email with news, research and thoughts, as well as training courses and free webinars you may wish to attend.
Our social media sites - YouTube, Twitter and Facebook