AI assessment systems could provide a fairer eleven plus selection, it could also start to address the vexed question of assessing potential rather than just current ability. We know that well designed AI systems that assess learning, are accurate in their assessment. AI assessment can tackle more than subject specific knowledge and reasoning, it can also evaluate skills such as planning and knowing what we know. AI assessment would also provide a fairer assessment system that would evaluate students across a longer period of time and from an evidence-based, value added perspective. We also know how to prevent people from gaming AI assessments, in addition to which AI Assessment systems would also offer tutoring for everyone and support and formative feedback to help students learn and improve. If there is to be a revamp of the grammar school system then we must explore these possibilities.
Theresa May’s plans for new or expanded grammar schools in England have brought a torrent of comment, debate, criticism and rhetoric since these plans were inadvertently revealed last week. Most of the discussions seem to have focused on whether or not grammar schools are the right mechanism to aid social mobility. This is an extremely important issue, but let’s put the rights and wrongs of selection and grammar schools to one side for a moment and look at the eleven-plus examination itself.
The eleven-plus examination is the key to the door of one of the 164 grammar schools in England, or one of the 69 grammar schools in Northern Ireland. The examination is sat by children in their last year of primary school and it varies depending upon where in the country it is taken. In fact, the situation is very complicated with a wide range of approaches even within the same county. For example, in Yorkshire there are three Local Authorities with Grammar Schools: Calderdale has 2, Kirklees has 1 and North Yorkshire has 3. The 2 grammar schools in Calderdale use Verbal Reasoning tests, and Maths and English examinations using GL Assessment, University of Edinburgh and the school themselves as their examiners. However, the 1 school in Kirklees uses tests in Verbal Reasoning and Non-Verbal Reasoning, plus an English examination and a Numerical Reasoning test. These are all examined by University of Durham. The situation in North Yorkshire is different yet again, with 2 schools using Verbal Reasoning and Non-verbal Reasoning tests examined by NFER and the 1 remaining school administering and examining its own selection tests.
The complexity in the selection process is not helpful to poorer parents, who do not have the time, and possibly not the capability, to navigate the process. In addition to which the examination approach is traditional and outdated. The need to look deeper than the selection process to the eleven plus examination itself was highlighted in an interesting discussion on the Radio 4 Today programme last week. The discussion was between Laura McInerney, the editor of Schools Week, and Sean Worth, from Policy Exchange. Sean pointed out that the current mechanism for selecting children for grammar schools can be gamed and that we therefore need to change the examination if we are to ensure that the poorest children are not disadvantaged. Laura McInerney also pointed out the major problem for poorer children accessing grammar schools is that we “put a test in the way”, especially divisive when the parents of poorer children can’t pay for tutoring to get their offspring through the eleven plus examination.
The Guardian published a depressing article on the problems inherent in the eleven plus test ‘‘Tutor-proof’ 11-plus professor admits grammar school test doesn’t work’. The article reports the failure of a ‘coaching resistant’ test developed by CEM at the University of Durham for use in Buckinghamshire. CEM has now withdrawn the claim that the test could assess “natural” ability. Prof Coe director of CEM is reported as saying: “Whatever system you use it is imprecise, there are false positives and negatives and probably more of those than people realise.” He goes on to reflect that whilst he does not agree with creating if we are to have more then we need to try and make the system fairer. I couldn’t agree more – and the need for a radical rethink is echoed in what the IOE’s Tina Isaacs says about the problems of coming up with any test that can assess future potential.
So, let’s take the test away and develop a radically different, socially equal eleven plus. We are lucky enough to be in a very different situation today from that which existed when the original eleven plus was introduced in 1944. There is now a realistic and economically attractive alternative at our fingertips. We have the Artificial Intelligence (AI) technology to build a superior assessment system should the proposed reforms become a reality. AI provides a powerful tool to open up the ‘black box of learning,’ to provide a deep, fine-grained understanding of when and how learning actually happens. Intelligent algorithms can process information about each learner and reach a view about their progress, knowledge and understanding of a subject or skill over a ‘period of time’. Unlike the eleven plus examination, this ‘period of time’ could be a whole school semester, a year, several years and beyond.
Of course there are serious ethical questions around AI being used in education and these must be explored. But the over-riding and uncontested fact in this debate is that education is the key to changing people’s lives. We trust AI with our personal, medical and financial data without a thought, so let’s trust it with the assessment of our children’s knowledge and understanding. Let’s open our minds and explore the challenges to build a new generation of eleven plus assessment that genuinely irons out the inequalities and gives all children a chance to shine.
[3] Hill, P. & Barber, M. (2014). Preparing for a renaissance in assessment. London: Pearson., DiCerbo, K. E. & Behrens, J. T. (2014). Impacts of the digital ocean on education. London: Pearson.
To appear on the IOE blog
The Knowledge Illusion 