So, after three days of insistence that their algorithm was appropriate, and following its enthusiastic endorsement by Michael Gove, the Department for Education capitulated and agreed to let Centre Assessed Grades (CAGs) stand as A-level and GCSE students’ final grades. However, many questions are left to ponder. The shadow attorney general maintains that in allocating these algorithm-derived grades to students, there have been at least three breaches of the law under the Apprenticeships, Skills, Children and Learning Act 2009, under which OFQUAL was established.
Ultimately, statistics applied to human behaviour are prone to generate problems. For this reason, those who engage in this process need to be acutely aware of the difficulties, and of the highly intricate ways that the numbers they are generating will interface and sometimes transact with the human beings to whom they are being applied. While numbers are sequential and logical, human life is fluid and unpredictable. This is an issue the schools minister doesn’t seem to have taken on board.
https://twitter.com/Channel4News/status/1296524144079785984?s=20
Meanwhile, as the ‘numberwang’ problem rumbles on, another data issue is developing in the wings, this time involving much younger children.
From September 2021, the government intends to apply a ‘baseline’ test to every child in their first half term at school. The test requires children to answer numeracy and literacy questions posed to them in a one-to-one session with a teacher. There has been no consideration given to children who might be shy, or to the fact that the Reception year starts at 48 months for children born on 31 August and at 60 months for the children born on 1 September in the relevant school year. Each child will be asked the same questions, in the same way, regardless of calendar age, or whether they speak English as a first language or not.
The aim of the test is to allocate a single numerical grade to each child that reflects their ‘school readiness’. This number will then be plotted on a graph, where it will remain ‘blackboxed’ until the child has progressed through the system to the Year 6 national curriculum tests (often referred to as ‘SATs’), taken in the spring of the final year at primary school. A trajectory will then be plotted from the child’s baseline score to their performance on the Year 6 SATs. The government’s intention is to use cohort collations of the baseline to Y6 SAT trajectory as an ‘accountability’ measure, to compare the effectiveness of individual teachers and schools.
In order to collate the data effectively for this process, each child’s grade has to be recorded against their name and date of birth. Children may relocate during the time they are at primary school, so there must be sufficient information recorded to allow their baseline score to follow them from school to school. However, the grade will not be shared with parents or with schools. They will instead be provided with a short paragraph of ‘narrative feedback’.
The nation’s premier experts on early years assessment expressed their strong concerns about baseline testing in the British Educational Research Association report A baseline without basis. The report makes the point that it is, in essence ‘an untried experiment’, given that no other nation in the world applies such a test to four-year-olds. It concludes that in the opinion of the authors, baseline testing will have no predictive value, principally due to the inherent difficulties in making a valid assessment of such very young children.
Additionally, given that this is an experiment, the ethical code for both educational and psychological research requires that formal permission for participation should be requested from the parents or guardians of children under 18, and that any data generated from the experimental tasks should be made available to them at the end of the process.
The fact that the government intends to use all the four-year-olds in England in this experiment, and does not intend to comply with either of these ethical requirements has been raised as a significant problem in Baseline assessment: why it doesn’t add up, a report produced by a coalition of professional early years organisations. Baseline testing has also been challenged by the data rights campaign Defend Digital Me, not only with respect to ethics, but also to GDPR requirements.
When a question was raised about these concerns in parliament, the government continued to insist that that the baseline testing would be going ahead from September 2020; though following the outbreak of Covid-19 it was rescheduled to September 2021. However, when this announcement was made in June, the schools minister Nick Gibb urged schools to become ‘early adopters’, for whom the materials would be made available in September this year.
Results from a trial run of the materials with a small sample of children in September 2019 were very mixed, as indicated by this heavily redacted report issued in response to a freedom of information request submitted by education author Sue Cowley.
The baseline debate shares many similarities with the 2020 exams debacle. It is a statistical exercise in which ministers have refused to listen to experts and have demonstrated a similar set of simplistic misunderstandings relating to the interface between human beings and statistical prediction. Ministers have also communicated a dogged determination to use the nation’s children in these data experiments without sufficient attention paid to their wellbeing.
In a week in which the nation has responded with disbelief at statistics-obsessed ministers’ behaviour, the fact that this is also happening at the other end of education needs to be brought to the attention of the public. The government’s recent attempt to subject children to an algorithm has been an unmitigated disaster. Perhaps then, parents of three-year-olds might consider whether they want the government to subject their children to a similar experiment in Autumn 2021?
By Dr Pam Jarvis
Yorkshire Bylines 21st August 2020