You are in:

Technology-based Assessment Arrangements - during Lockdown at Home

by Paul Nisbet

on Wed Jan 06, 2021

Share this blog

Share on:

  • Twitter share
  • Facebook share
  • linkedin share
  • Google+ share

At present there are:

training courseNewsletter

Get news, articles, advice and tips.

Sign me up!

On Monday the First Minister announced that schools would only be open for children of key workers and vulnerable children for January. In many schools, January is 'prelim month', so how can prelims or assessments be undertaken when most learners are at home?

Summary

The main points put forward in this blog are:

  • with the closure of schools for most learners, there is an urgent need to devise practical approaches to organising assessments for National Qualifications for students at home;
  • students with disabilities or additional support needs can and should be able to use Assessment Arrangements for assessments, whether in school or at home;
  • modern digital devices have good accessibility tools and options that can enable students with additional support needs to access digital assessments at home;
  • centres can use the internet and cloud-based systems such as Glow, Teams and Google Classroom to administer assessments;
  • there is scope to allow students to use tools such as spellcheck, calculator and even the internet during assessments.

However, allowing students to use spellcheck or the internet is a radical departure from previous SQA policies around the use of technology in assessment and there is a need for greater clarity, and for solutions to the practical and technological challenges facing teachers and learners.

Assessing National Qualifications in 2021

As we know, Covid-19 has already had significant impact on teaching and learning and assessment for National Qualifications: the 2020 examination diet was cancelled [1]; the 2021 National 5 external examinations were called off in October [2]; and in December we heard that examinations for Highers and Advanced Highers in 2021 are also cancelled [3].

Awards in 2020-21 will be “based on teacher and lecturer judgement supported by quality assurance[4], the evidence for which might be drawn from:

  • Prelims or mock exams
  • SQA specimen question papers or past papers
  • Commercially-produced question papers/assessments
  • Class tests
  • Classwork
  • Completed or partially-completed coursework
  • Performance evidence” [5]

To support centres, SQA have provided “National 5 question papers and marking instructions that teachers and lecturers can use when gathering evidence for producing estimates in session 2020-21[6] and question papers and marking instructions are also to be provided for Highers and Advanced Highers now that external examinations are not happening [7].  

What about Assessment Arrangements?

Teachers are using a variety of internal assessments to gather evidence of attainment. Students with disabilities or additional support needs who require support should be provided with suitable Assessment Arrangements [8] and SQA advise that centres should “continue to use the same assessment arrangements processes already in place in your centre to support candidates requiring an assessment arrangement for an internal assessment [9]. Note that "Centres do not need to submit assessment arrangement requests to SQA for National Qualifications in 2020-21".

Assessments at home

In many schools, January is when prelims are held to gauge students' progress, provide students with an opportunity to experience examination conditions, and to confirm the efficacy of Assessment Arrangements for students with disabilities or additional support needs. While SQA state that there is no expectation that schools and colleges hold a formal diet of prelims for National 5 this year [6], it seems that many centres timetabled prelims or assessments this month. How can these assessments be administered when most students are learning at home?

SQA have published guidance on remote assessment although my reading of it is that it has been written for a context where a small number of learners are at home taking part in an assessment, rather than the current situation where most learners are at home:

The following guidance is for centres who are using, or considering using, technology to enable candidates to complete an assessment in a different location from the invigilator, where standard invigilation procedures cannot be applied.

Let's take a look at some key points.

Assistive Technologies

The document with Advice on centres on using technology to support remote assessment [10] reminds us that:

Candidates with assessment arrangements must be able to access these in a remote assessment scenario. Centres should consider to what extent the existing assessment arrangements can be implemented remotely and whether additional or alternative arrangements are required to allow the candidate equal and fair access to the assessment. If it is not possible to provide assessment arrangements remotely, this may not be a suitable arrangement.

Students who use assistive technologies in class as their normal way of working should be able to use the same supports in internal assessments, whether in school or at home.

Students who need technology-based assessment arrangements such as a computer reader, spellcheck or computer dictation clearly require a device with the appropriate apps or software installed. The good news is that accessibility tools in modern devices are getting better all the time so in many cases the student can tackle assessments using a standard laptop or tablet. The table below summarises the main tools: for more detailed information refer to the CALL Scotland web site.

 

Built in Assistive Technology Windows iPad Chromebook  
Text Reader

Narrator

Learning Tools

Speak Selection

Speak Screen

Select and Speak
Screen reader Narrator VoiceOver ChromeVox
Large Text

Make Text / Everything Bigger.

Learning Tools

Bold / Large Text no
Zoom / magnify Magnifier Magnifier Magnifier
Colour adjustment

High contrast, colour filters.

Learning Tools

High contrast, smart invert colour filters High contrast
Computer Dictation Dictation Siri Dictation Google Voice typing / dictation
Spellcheck yes yes yes
Predictive text yes yes yes
On-screen keyboard yes yes yes
Keyboard shortcuts / navigation yes yes yes
Mouse / pointing device access yes yes yes
Eye-gaze access yes no no
Switch access yes yes no

 

Assessment delivery

Traditionally, students are not permitted to access the internet during examinations and that would make life pretty difficult for learners who are tackling an assessment at home, so it's helpful to have clarity that we can use cloud storage to administer assessments for learning at home:

 

Can I use GLOW or another environment such as Google Classrooms to make an assessment available to candidates?

We recognise that academic year 2020–21 is taking place in very challenging circumstances. Most digital learning and assessment environments are technically very secure and can be used to make assessment instruments available to candidates, and to store evidence. However, you will need to consider how you will limit visibility to other candidates and minimise the risk of exposure of any SQA-supplied secure assessment material through that environment.

SQA (2020) Using technology to support assessment remotely: questions and answers, p. 1 [10]

 

The question is how to do it securely. SQA [10] advises that:

As every centre has different IT set-ups and support, beyond the digital assessments that SQA offers on our SOLAR platform, we are unable to give specific advice. However, many of the currently available tools to support video conferencing and remote working (for example Microsoft Teams) can be used to contribute towards a suitable model to support controlled assessment remotely.

How will you use the software to uphold SQA’s conditions of assessment?

How are you ensuring that the software is secure? For example:

  • is it regularly updated?
  • is usage password protected?
  • are accounts locked after a specified number of failed access attempts?
  • if it is web-based does it use https to secure all pages?
  • can you monitor users and usage?

So it's helpful to know that systems that are currently being used in school such as Microsoft Teams or Google Classroom can also be used for remote assessment. However, from the conversations that I have had with teachers, there is a lot of uncertainty about to make the assessment secure and meet the SQA conditions of assessment.

Assessments are available via the SQA SOLAR system, but there are limitations [12]:

"Delivery on Chromebooks or mobile phones is not currently possible" which isn't much help if you have been given one of the 25,000 (50,000?) Chromebooks funded by Scottish Government or are in a local authority such as Aberdeen City, Highland, Stirling or West Dunbartonshire where Chromebooks are the norm.

"To deliver a scheduled test on i-pads or tablets requires a download of the Surpass App from the Apple and Google Stores" but I'm told that this app is not available on the iPads that have been provided to learners in Glasgow and Scottish Borders.

On a Windows device, you need to have the Surpass Viewer installed because the SOLAR assessments use Flash, which is now obsolete.

Conditions of Assessment

The document [10] recognises that the home may not provide suitable conditions for assessment and that learners may not have access to technology, and suggests that "it is worth considering whether remote assessment is appropriate and whether an alternative (such as postponing the assessment) may be a better option" but this may be difficult to achieve given where we are in the school year, and where most learners are at home.

SQA recommends issuing "a checklist for candidates to complete prior to undertaking a remotely invigilated assessment. This will ask for basic details such as the availability of a quiet space that can be cleared of prohibited items, and access to equipment and internet connectivity. The checklist should also explain the protocol surrounding the conduct of a remotely invigilated assessment, such as the initial environmental sweep, and how to minimise behaviour that could appear suspicious. You should also consider whether to include a trial run prior to the assessment or build in additional time on the day to allow for technical checks" [11].

Security and Invigilation

The Advice document [11] is mostly concerned with security and invigilation to ensure confidentiality of assessments and to authentic a student's work as their own. Where examinations are conducted in school, the papers are kept secure until the assessment starts, candidates do not have access to technology, books or other sources of help, the identity of candidates is reasonably easy to check, and there are invigilators who are physically present in the room.

For remote assessment, the guidance advises considering:

How you will ensure the security of the assessment, and suggests:

  • an assessment system that can be controlled to go 'live' by the class teacher at a set time (e.g. this can be done in Teams or Google Classroom);
  • a system such as SOLAR which can block access to other applications on the student's device (note however that the SOLAR SecureClient only works on Windows and not on iPads or Chromebooks);
  • using a camera to provide the remote invigilator with a view of the student at home (I'm not sure how realistic this is if the majority of learners in a class are to be assessed at home and there must be privacy considerations?).

How you will authenticate candidates with for example a Glow login  to ensure that the student taking the assessment is that person and not their pal / mother / uncle etc.

How you will uphold assessment conditions: SQA suggest use of locked down devices and a remote camera viewing on the students. Hmmm...

How you will ensure equity so that learners have practical and technical arrangements to access the assessment.

In practice…

The teachers I have spoken with have questions about how these measures can actually be implemented, especially now that most learners are going to be at home during January. The last page of the SQA document [11] has some pragmatic comments and suggestions recognising that "some temporary adjustments to assessment practices may be required for the academic year 2020–21".

The guidance notes that "for the duration of the academic year 2020–-21, for certain qualifications you are able to carry out assessments under more flexible open book conditions" although I could not find any links to information saying which qualifications or what 'open book conditions' means.

The guidance also notes that "In addition to relevant books or resources, you may also wish to allow access to software tools such as calculators, spellcheckers or internet-based resources" and this could make life much easier when using technology-based assessments because it may be difficult or impossible to disable spellcheckers or other literacy support tools or to prevent access to the internet on devices at home.

However, permitting access to the internet, calculators or spellcheck is a radical departure from decades of SQA policy and I have concern that this guidance is not well known or understood and that we do not know how to implement such remote assessment on a large scale with consistency across the range of devices and platforms in use in Scottish schools.

From discussions with teachers, I feel there is a good chance that assessment practices are varying across the country, especially now that schools are closed to most students, and so there is an urgent need to provide clarity and practical, accessible solutions.

 

[1] Scottish Government (2020) Update on school closures and exams 19/3/20.

[2] Scottish Government (2020) SQA Awards 2021. 7/10/20

[3] Scottish Government (2020) Coronavirus (COVID-19) - education sector: Deputy First Minister statement 8 December 2020.

[4] SQA (2020) Delivering National 5 results in 2020-21 8/12/20

[5] SQA (2020) National Courses: guidance on gathering evidence and producing estimates. October 2020, Publication Code BBB8283.

[6] SQA (2020) Assessment resources: National 5 question papers.

[7] SQA (2020) Update from the National Qualifications 2021 Group: Support and guidance for Higher and Advanced Higher in session 2020–21. 16/12/20.

[8] SQA (2019) Assessment Arrangements Explained: Information for centres

[9] SQA (2020) About assessment arrangements: Assessment Arrangements for 2020-21.

[10] SQA (2020) Using technology to support assessment remotely: questions and answers.

[11] SQA (2020) Advice for centres in using technology to support assessment remotely.

[12] SQA SOLAR About SOLAR

 

Tags: exams

Share this blog

Share on:

  • Twitter share
  • Facebook share
  • linkedin share
  • Google+ share

At present there are:

Conversations