We can radically redesign the 11 plus exam to make it fairer, so what is stopping us?

AI assessment systems could provide a fairer eleven plus selection, it could also start to address the vexed question of assessing potential rather than just current ability. We know that well designed AI systems that assess learning, are accurate in their assessment. AI assessment can tackle more than subject specific knowledge and reasoning, it can also evaluate skills such as planning and knowing what we know. AI assessment would also provide a fairer assessment system that would evaluate students across a longer period of time and from an evidence-based, value added perspective. We also know how to prevent people from gaming AI assessments, in addition to which AI Assessment systems would also offer tutoring for everyone and support and formative feedback to help students learn and improve. If there is to be a revamp of the grammar school system then we must explore these possibilities.

Theresa May’s plans for new or expanded grammar schools in England have brought a torrent of comment, debate, criticism and rhetoric since these plans were inadvertently revealed last week. Most of the discussions seem to have focused on whether or not grammar schools are the right mechanism to aid social mobility. This is an extremely important issue, but let’s put the rights and wrongs of selection and grammar schools to one side for a moment and look at the eleven-plus examination itself.


The eleven-plus examination is the key to the door of one of the 164 grammar schools in England, or one of the 69 grammar schools in Northern Ireland. The examination is sat by children in their last year of primary school and it varies depending upon where in the country it is taken. In fact, the situation is very complicated with a wide range of approaches even within the same county.  For example, in Yorkshire there are three Local Authorities with Grammar Schools: Calderdale has 2, Kirklees has 1 and North Yorkshire has 3. The 2 grammar schools in Calderdale use Verbal Reasoning tests, and Maths and English examinations using GL Assessment, University of Edinburgh and the school themselves as their examiners. However, the 1 school in Kirklees uses tests in Verbal Reasoning and Non-Verbal Reasoning, plus an English examination and a Numerical Reasoning test. These are all examined by University of Durham. The situation in North Yorkshire is different yet again, with 2 schools using Verbal Reasoning and Non-verbal Reasoning tests examined by NFER and the 1 remaining school administering and examining its own selection tests.


The complexity in the selection process is not helpful to poorer parents, who do not have the time, and possibly not the capability, to navigate the process. In addition to which the examination approach is traditional and outdated. The need to look deeper than the selection process to the eleven plus examination itself was highlighted in an interesting discussion on the Radio 4 Today programme last week. The discussion was between Laura McInerney, the editor of Schools Week, and Sean Worth, from Policy Exchange. Sean pointed out that the current mechanism for selecting children for grammar schools can be gamed and that we therefore need to change the examination if we are to ensure that the poorest children are not disadvantaged. Laura McInerney also pointed out the major problem for poorer children accessing grammar schools is that we “put a test in the way”, especially divisive when the parents of poorer children can’t pay for tutoring to get their offspring through the eleven plus examination.


The Guardian published a depressing article on the problems inherent in the eleven plus test ‘‘Tutor-proof’ 11-plus professor admits grammar school test doesn’t work’. The article reports the failure of a ‘coaching resistant’ test developed by CEM at the University of Durham for use in Buckinghamshire. CEM has now withdrawn the claim that the test could assess “natural” ability. Prof Coe director of CEM is reported as saying: “Whatever system you use it is imprecise, there are false positives and negatives and probably more of those than people realise.” He goes on to reflect that whilst he does not agree with creating if we are to have more then we need to try and make the system fairer. I couldn’t agree more – and the need for a radical rethink is echoed in what the IOE’s Tina Isaacs says about the problems of coming up with any test that can assess future potential.


So, let’s take the test away and develop a radically different, socially equal eleven plus. We are lucky enough to be in a very different situation today from that which existed when the original eleven plus was introduced in 1944. There is now a realistic and economically attractive alternative at our fingertips. We have the Artificial Intelligence (AI) technology to build a superior assessment system should the proposed reforms become a reality. AI provides a powerful tool to open up the ‘black box of learning,’ to provide a deep, fine-grained understanding of when and how learning actually happens. Intelligent algorithms can process information about each learner and reach a view about their progress, knowledge and understanding of a subject or skill over a ‘period of time’. Unlike the eleven plus examination, this ‘period of time’ could be a whole school semester, a year, several years and beyond.

Of course there are serious ethical questions around AI being used in education and these must be explored. But the over-riding and uncontested fact in this debate is that education is the key to changing people’s lives. We trust AI with our personal, medical and financial data without a thought, so let’s trust it with the assessment of our children’s knowledge and understanding. Let’s open our minds and explore the challenges to build a new generation of eleven plus assessment that genuinely irons out the inequalities and gives all children a chance to shine.

[3] Hill, P. & Barber, M. (2014). Preparing for a renaissance in assessment. London:  Pearson., DiCerbo, K. E. & Behrens, J. T. (2014). Impacts of the digital  ocean  on  education.  London: Pearson.

To appear on the IOE blog


Calling education: wake up and smell the coffAI, don’t miss a great opportunity to drive prosperity for all

A recent article in the THES got me thinking. David Matthews reported under the title: The robots are coming for the professionals, and asked if universities need to rethink what they do and how they do it now that artificial intelligence is beginning to take over graduate-level roles? This motivated me to write a blog post for THES that was published on 9 August: Four ways that artificial intelligence can benefit universities, in which I suggested that HE needs to embrace the positives of AI, not just look at the negatives.



These issues are not limited to HE, in fact this is a wake up call for all of Education. We must engage with these technologies and those who are developing them NOW in order to ensure that the AI that we end up with in classrooms, homes and the workplace is informed by what we know about learning and NOT what we know about what the technology can do.

7fba0586036d8b0f2cdf47df1d037557There is a huge and growing interest among those who invest in new technology ventures, specifically Artificial Intelligence (AI) techniques and methods. For example, between 2011 and 16 May 2016 Sentient Technologies received over 143 million USD in funding (Data from CB Insights) Much of the excitement about AI has focused on general purpose AI i.e. intelligence that is applicable across a variety of industries and activities. This is being promoted for technology businesses as a force for good. For example, Antoine Blondeau, the CEO of Sentient, has stated that: “From healthcare to finance to e-commerce, we’re focused on changing people’s lives.” Sentient is reported to be working on financial platforms and on an AI nurse to diagnose patients with sepsis.It is a business that like many who are adopting AI methods has no problem in attracting funding.

However, the same is not yet true of organisations who are adopting AI for education. Yes, there are things like Udacity, that claims it will change HE, and Knewton whose CEO Jose Ferreira, really does believe that his technology will replace human teachers. Such an outcome would make ‘driverless classrooms’ into a science reality. These commercial AI in Education ventures are well funded. BUT it is hard to find mass investment in the application of AI to education, despite the fact that the Educational Technology sector is predicted to grow from £45bn to £129bn by 2020. And to my mind much more significantly, despite the fact that education is the real key to changing people’s lives.maxresdefault.jpg

We need to take a fresh look at education if we are to ensure that the global population is able to reap the potential of the AI revolution that is sweeping across the workplace. AI is both a cause of the radical changes to the workplace that prompted David Matthews to write his piece in the THES and a provider of an answer to the problem of how we make the most of the workplace automation that AI is enabling. The purpose, methods and outcomes of education need re-thinking and AI can help us to tackle the challenge of this re-thinking if we invest in its development and build on the thirty years plus of research in AI for Education.

The importance of the social and economic significance of the developments in autonomous systems and AI was reflected at the annual meeting of the World Economic Forum 2016 in Davos, where the focus was on ‘The Fourth Industrial Revolution. This revolution “is characterized by a range of new technologies that are fusing the physical, digital and biological worlds, impacting all disciplines, economies and industries, and even challenging ideas about what it means to be human.” These radical changes do not however seem to manifest themselves in a concerted effort to use AI to revolutionize education. This oversight is shortsighted to say the least. The few exceptions that one can find where AI is being applied to education at some scale have a very narrow perspective and are a long way from changing people’s lives in the positive way that we want and need. For example: Knewton, is just one a a host of companies who believe that Subject Knowledge is the key to unlocking education for all. Through, for example, making artificially knowledgeable adaptive tutors who can personalize their content to meet an individual learner’s needs. This is all very well, but there is so much more to education than subject knowledge and so much more to AI than adaptive educational content

So what are the key attributes of AI for Education that will enable it to start attracting the sort of investment that Horizons Ventures and Tata Communications have made in Sentient Technologies? What are the attributes of AI that will persuade research funders that AI for education is a subject they must prioritize and that it must be a truly interdisciplinary enterprise that is not driven purely by technologist’s dreams. For a change let’s focus on disadvantaged learners’ dreams and see if we can work with technology to turn these dreams int o reality.

One key attribute of AI for Education is the ability that Educationally driven AI techniques and algorithms bring to the analysis of the vast amounts of data about learners that is routinely harvested by the increasing amount of technologies in the world around us from CCTV, to smartphones, wearable technologies and online courses, such as MOOCs. For example, we can

  • Conduct fine-grained analysis of learners’ skills and capabilities so that their development can be tracked at the student/employee, workplace, school, area, and country level;
  • Enable the collation of a dynamic catalogue of the best training and teaching practices across a range of environments and as a result enable us to educate and train the future workforce in an economically productive manner.

A second key attribute of Educationally driven AI is that it can help us to tackle the toughest educational challenges, including learner achievement gaps, teacher skill shortages, continuous professional development for educators. If we think about the business of education for a moment, imagine the AI teaching assistant that can be used to stretch the brightest pupils, while the human teacher devotes their expertise to giving the less able learners the sensitive human support that they need in order to progress. The teacher would train their personal assistant to work in the way that the teacher and their students need and would demand that the AI assistant explain the decisions it has made about students and the educational opportunities the assistant has provided.

But perhaps what we need to focus on first is using AI systems that go beyond the machine learning and neural network techniques that dominate the work of the main AI protagonists within and beyond education, from Knewton to Google DeepMind. The types shutterstock_260422808.jpgof AI we need within education is the AI that enables the technology it powers to explain its reasoning, to justify its decisions and to negotiate with its users. This is the sort of AI technology that could help us address one of the toughest challenges within the current workplace:  The lack of understanding about how humans can best work with AI systems so that the result is AI augmented human intelligence that is greater than the sum of its parts. We need workers who understand how to make the best use of the power that AI automation can bring to industry and commerce. Workers who understand enough about AI to know where and how human intelligence can work with AI to achieve a blended intelligence that can increase productivity. And what is beautiful about all this is that the appropriate type of AI can help us educate and train people to understand enough about their AI colleagues to work alongside them effectively.



Here is what ‘smart’ looks like in an AI tutor


So, what would a piece of education technology driven by AIEd look like? Here is a simplified picture of a typical model-based adaptive tutor.



It is based on the three core models as described above: the learner model (knowledge of the individual learner), the pedagogy model (knowledge of teaching), and the domain model (knowledge of the subject being learned and the relationships between the different parts of that subject matter). AIEd algorithms (implemented in the system’s computer code) process that knowledge to select the most appropriate content to be delivered to the learner, according to their individual capabilities and needs.

While this content (which might take the form of text, sound, activity, video, or animation) is being delivered to the learner, continuous analysis of the learner’s interactions (for example, their current actions and answers, their past achievements, and their current affective state) informs the delivery of feedback (for example, hints and guidance), to help them progress through the content they are learning. Deep analysis of the student’s interactions is also used to update the learner model; more accurate estimates of the student’s current state (their understanding and motivation, for example) ensures that each student’s learning experience is tailored to their capabilities and needs, and effectively supports their learning.

Some systems include so-called Open Learner Models, which present the outcomes of the analysis back to the learners and teachers. These outcomes might include valuable information about the learner’s achievements, their affective state, or any misconceptions that they held. This can help teachers understand their students’ approach to learning, and allows them to shape future learning experiences appropriately. For the learners, Open Learner Models can help motivate them by enabling them to track their own progress, and can also encourage them to reflect on their learning.

One of the advantages of adaptive AIEd systems is that they typically gather large amounts of data, which, in a virtuous circle, can then be computed to dynamically improve the pedagogy and domain models. This process helps inform new ways to provide more efficient, personalised, and contextualised support, while also testing and refining our understanding of the processes of teaching and learning.

In addition to the learner, pedagogical, and domain models, AIEd researchers have also developed models that represent the social, emotional, and meta-cognitive aspects of learning. This allows AIEd systems to accommodate the full range of factors that influence learning. Taken together, this set of increasingly rich AIEd models might become the field’s greatest contribution to learning.



This post is an adapted extract from Intelligence Unleashed published by Pearson.

A bit of Yackety Yak about the Hackety Hack

I am increasingly excited about Re-Designing our education and I have a song in my heart…

Get out those mobiles and that scratch
Or you won’t get to code and mash
If you don’t drum on that LKL door
You ain’t gonna teach and learn no more.
Hackety Hak (you won’t look back)

Re-Designing our Education – Education Hack – 7 days and counting

Re-Designing our Education – Education Hack 16-17 November, 2102 at The London Knowledge Lab

What a great weekend!  A new initiative from Nesta:  Digital Making initiative and the Mozilla Festival too. But even more exciting than all this is the final countdown to our Education Hack Event, which starts a week today where learners will be working with teachers, researchers and volunteers from the SME and StartUp community to ‘Hack their Education’ and show us all how technology really could make a difference to teaching and learning.

The Education Hack Event involves 50 students from 6 schools over 2 days at London Knowledge Lab working with volunteers to develop the future of education . It will enable secondary students to put into action their ideas about how technology could revolutionise their learning. Teachers get hands-on too, working with their students and experts from the LKL, industry and the media.

From e-diary systems to help them track their learning and homework to cyber bullying help networks, the students have the ideas and we will help them put them into action. This is ‘The real McCoy’ – kids hacking for good.

The technologies that are being used include MIT apps inventor, Java, HTML, digital film, web building using Thimble, Arduino kits, content mashups and  much, much more

Members of the public can also take part on the Saturday afternoon, see what the students have built and try their hand at a range of activities from making jewellery tagged with radio frequency detection devices to writing code.

Teaching children computer science, including programming, is at the heart of the policy shift in the current teaching of IT. Programming helps build understanding of and access to formal systems of thought that are essential in helping people to express their ideas about the world, and make sense of it. That’s why programming is important: not just to increase the supply of programmers (important) or to introduce to everyone what is under the bonnet of the systems that power our society (essential), but to introduce the power of computational thinking. Events like Education Hackday allow young people to develop their computational thinking and hack new ideas to revolutionise their learning environment, working with their peers, teachers and technology professionals.

As one of our participating teacher says:

“Education Hackday will provide our gifted and talented computing and IT students with a challenging “real life” opportunity. They will learn how to use new computing and programming tools, experience different design environments, and work with a range of other peers, agencies and experts. The benefits of such an experience have a long term impact on the pupils themselves, as well on as the staff of the school.”

Chris Betts, Blatchington Mill School:

iPads for learning: over sold, under valued or just plain unknown?

As microsoft launched its new Surface tablet the London Knowledge Lab started its Autumn series of ‘What the Research Says Events’ with a session about iPads and Education. The following text was used as a provocation for a lively and well-informed discussion that will be used to produce a full report in January.

Any report about iPads and Education must surely be falling into the trap of seeing learning and teaching with technology in terms of the type of technology being employed? In this short briefing paper we are certainly concerned with a type of technology: touch screen, portable, wireless devices, such as the iPad. However, the emphasis is on exploring how effective learning activities can be supported by iPad devices.  The aim of this short paper is to stimulate discussion: discussion that will feed into a larger event and report in January 2013.

An article in Time (tech) earlier this year sums up the situation about iPads and education rather nicely: we think they are good for learning, but we don’t have the evidence. The article reported that a study in Auburn, Maine found that young learners using the iPad 2 scored higher on literacy tests and were more enthused about learning. However, it also reported the powerful views of Larry Cuban, a professor emeritus of education at Stanford University, who urged caution, saying that  “There is very little evidence that kids learn more, faster or better by using these machines,” …“iPads are marvelous tools to engage kids, but then the novelty wears off and you get into hard-core issues of teaching and learning”.

We therefore focus on thoughts about learning with iPads based upon what we already know about learning and technology – hence the emphasis is on evidence based potential rather than hard evidence of performance.

12 Thoughts about iPads for Education

  1. Sharing Learning

We know that human knowledge arises from social interaction. Social interactions can take the form of traditional teacher and learner interactions, peer-centred interactions, or participation in activities with large groups or communities. iPad technologies can support easy sharing between 2 or 3 people sitting together and using a single device in the same location. They can also be effective at putting people in touch with each other when they are in distant physical locations, through wi-fi or phone network connections and using software applications such as skype. The quality of these shared experiences is not guaranteed by the iPad of course, but is there potential to develop high quality shared learning through with the iPad as a platform?

  1. 2.     Personalising learning

All learners are different and require teaching and learning interactions that acknowledge these differences and provide suitable support. A technology, such as an iPad, that is owned by an individual learner and populated with material and applications that are particularly suitable for their needs could be a powerful, portable, personal learning partner.

  1. 1 to 1 AND 1 to Many Learning

Benjamin Bloom from the University of Chicago wrote a seminal paper in 1984, which illustrated that learners who were individually tutored performed significantly better than those who received conventional classroom instruction. Do iPad type technologies offer the potential to work in combination with individualised tutoring systems and provide communication sessions with human tutors to provide such individual tutoring? iPad technologies can also be used as a receiver for broadcast media, such as lectures and videos given by individual experts and received by many individuals. Thus could learners be ‘scaffolded’ to success by others who are more knowledgeable and who may be teachers, experts, peers or the technology itself acting in a tutorial capacity?

  1. 4.     BIG Data

A small, portable device that is personal to a learner and through which they conduct learning activities is not only a window for learning, but also a window through which invaluable data about the learner and their interactions can be captured, stored and analysed. The potential for the widespread use of tools that can capture the progress of a learning episode, individual or joint is particularly powerful. The visualisation of this analysed data on the iPad also offers important support for assessment and personal reflection. The storage capacity of the iPad device is limited, but access to cloud based storage might alleviate this constraint, provided network access is available, but what are the social costs and implications here?

  1. 5.     Knowing what Learners Know and Need

Being able to understand what a learner knows and understands is key to teaching and learning. Learners need to be completing tasks that are appropriately challenging, without being so difficult that learners are bound to fail. In combination with the capacity for the collection and collation of data about learning activity as highlighted above, the analysis and representation of this learning data is vital. Could iPads can be used to access such representations and act as repositories for personal models of the learner and their progress? Could these models then be used by the learner to reflect on their performance and can be shared with teachers and parents?

  1. 6.     Making, Hacking and Mashing Learning

Making and sharing are important learning activities through which learners must construct their own understanding and share the fruits of their construction with others. Do iPads offer an effective platform through which tools such as scratch can be used, through which blogs can be written and viewed, and through which apps can be distribute? What are the limitations e.g. constraints from the AppStore and the need for the Apple development tools?

  1. 7.     Playing and Learning

Playing and exploring can be great fun and very motivating. However the games being played need to be well designed and offer real learning opportunities. Learners may well need support, at least initially, to develop the metacognitive and self-regulation skills to get the most from such ‘free-form’ self-guided activities. Could the iPad be a useful medium through which these activities can be effectively acted out, provided the activities themselves are driven by well–design software and timely human support interventions?

  1. 8.     Apps for Learning

Applications can be used to support many effective learning practices, depending upon the design and quality of the app. For example, we know that drill and practice activities have a role to play in learning. They can for example, be very effective at preparing learners for the examinations they need to pass and for achieving fluency in particular skills. Many Apps offer good opportunities to practice anything from language learning and maths to cooking skills. Other Apps can offer opportunities to track ones performance in sport or measure distances between points on a map, as well as a host of other possibilities, some more learning effective than others. Does the iPad provide the platform for these Apps and enables them to be used in multiple locations within and beyond formal educational institutions? Teachers have been quick to take up iPad type applications and there are many examples of their use to be found in practitioner blogs and discussion forums. This presents a challenge as well as an opportunity: authors rarely discuss how the applications they have used were integrated into practice effectively and it can be hard for teachers to know which of the many options available would be best for them and their learners.

  1. 9.     Untethered Learning

We know that the circumstances of learning make a great difference to the efficacy of learning.  The people, places, and things with which learners interact are crucial elements in their developing understanding of knowledge and skills. Does the mobility of the iPad offer great opportunities for learners and their teachers to build meanings and experiences across different locations? Could the portability of devices like the iPad also help to develop the coherence of the learner’s experience, which can in turn reinforce prior learning and create deeper understandings?

  1. 10.  A Platform for Future Technology

The constant updating of devices, such as the iPad is both a constraint and an enabler of their learning potential. It is clearly a constraint, because of the cost implications of constantly upgrading. Could it also be an enabler, because frequent upgrading provides the possibility to use the latest technology hardware and software features, such as voice input and embedded cameras?

  1. 11.   Too New for us to Know?

Launched in 2010 with 7 million sales units, the iPad is predicted to increase to 60 million sales units in 2012, uptake of this device has grown very, very fast. In the US the combination of iPad and Android tablets is likely to outstrip PC sales soon. However, as highlighted at the start of this report, the evidence about learning with iPads is still variable in quality and not well integrated between research and practice.

  1. 12.  Another thorn in the side of Network Managers?

The level of resourcing for technical support in schools is often low and the prospect of multiple devices moving in and out of a school or college and then re-connecting to a network can be challenging. This can be made worse if the devices are not provided by the school or college and belong to learners, are of different makes and types and use different operating systems. Could iPads alleviate some of the difficulties here, because of the limited channel they provide between device and network? Or does the limited platform afforded by the iPad ‘dumb down’ learning possibilities?

What Next?

Significant decreases in cost may make iPads increasingly attractive to educational purchasers with limited funds. However, the rapid evolution of technologies, such as the iPad, make it difficult to predict their future impact on learning. Learning impact will depend upon the quality of the Apps teachers and learners use. It will also depend upon the extent to which the resources that accompany the iPad in supporting learning and teaching are taken into account when planning their introduction and recording their impact. Resources such as:

  • The people who support learning: teachers, peers, classroom assistants and technicians.
  • The places and the constraints associated with them. For example, the rules of the school and home constrain the manner in which the device can be used;
  • The other kit that is required, such as a wi-fi connection, or access to a printer.

Geek is chic, but engineers are awesome

I went to a fascinating roundtable event hosted by the Guardian. It was designed to fly the flag for ICT and Computer Science and to ensure that young people enjoy the benefits of excellent teaching and the improved job prospects that come with being tech savvy and able to create and produce digital content and applications as well as consume them. There will be a piece in the Educational Guardian about our discussions on 10 July, so I don’t want to jump the gun on that, but I do want to talk about an issue that intrigued me: the poor popularity ratings of engineers…

Read more on the Institute of Education Blog