Learning machines

Ben Williamson

When educators talk about theories of learning they are normally referring to psychological conceptions of human cognition and thinking. Current trends in machine learning, data analytics, deep learning, and artificial intelligence, however, complicate human-centred psychological accounts about learning. Today’s most influential theories of learning are those that apply to how computers ‘learn’ from ‘experience,’ how algorithms are ‘trained’ on selections of data, and how engineers ‘teach’ their machines to ‘behave’ through specific ‘instructions.’

It is important for education research to engage with how some of its central concerns—learning, training, experience, behaviour, curriculum selection, teaching, instruction and pedagogy—are being reworked and applied within the tech sector. In some ways, we might say that engineers, data scientists, programmers and algorithm designers are becoming today’s most powerful teachers, since they are enabling machines to learn to do things that are radically changing our everyday lives.

Can the field of social scientific educational research yet account for how its core concerns have escaped the classroom and entered the programming lab—and, recursively, how technical ‘learning machines’ are re-entering classrooms and other digitized learning environments?

Non-human machine learning processes, and their effects in the world, ought to be the object of scrutiny if the field of education research is to have a voice with which to intervene in the data revolution. While educational research from different disciplinary perspectives has long fought over the ways that ‘learning’ is conceptualized and understood as a human process, we also need to understand better the nonhuman learning that occurs in machines. This is especially important as machines that have been designed to learn are performing a kind of ‘public pedagogy’ role in contemporary societies, and are also being pushed in commercial and political efforts to reform education systems at large scale.

Algorithmic autodidacts
One of the big tech stories of recent months concerns DeepMind, the Google-owned AI company pioneering next-generation machine learning and deep learning techniques. Machine learning is often divided into two categories. ‘Supervised learning’ involves algorithms being ‘trained’ on a selected dataset in order to spot patterns in other data later encountered ‘in the wild.’ ‘Unsupervised learning,’ by contrast, refers to systems that can learn ‘from scratch’ through immersion in data.

In 2016 DeepMind demonstrated AlphaGo, a Go-playing AI system that learned in a supervised way from a training dataset of thousands of games played by professionals and accomplished amateurs. Its improved 2017 version, AlphaGo Zero, however, is able to learn without any human supervision or assistance other than being taught the rules of the game. It simply plays the game millions of times over at rapid speed to work out winning strategies.

In essence, AlphaGo Zero is a self-teaching autodidactic algorithmic system.

‘It’s more powerful than previous approaches because by not using human data, or human expertise in any fashion, we’ve removed the constraints of human knowledge and it is able to create knowledge itself,’ said AlphaGo’s lead researcher in The Guardian.

At the core of AlphaGo Zero is a training technique that will sound familiar to any education researchers who have encountered the psychological learning theory of ‘behaviourism’—the theory that learning is an observable change in behaviours that can be influenced and conditioned through reinforcements and rewards.

Alongside neural network architecture, a cutting-edge ‘self-play reinforcement learning algorithm’ is AlphaGo Zero’s primary technical innovation, It is ‘trained solely by self-play reinforcement learning, starting from random play, without any supervision or use of human data,’ as its science team described it in Nature. Its ‘reinforcement learning systems are trained from their own experience, in principle allowing them to exceed human capabilities, and to operate in domains where human expertise is lacking.’ As the reinforcement algorithm processes its own experiences in the game, it is ‘rewarded’ and ‘reinforced’ by the wins it achieves, in order ‘to train to superhuman level.’

Beyond being a superhuman learning machine in itself, however, AlphaGo may also be used ‘to help teach human AlphaGo players about additional, “alien” moves and stratagems that they can study to improve their own play,’ according to DeepMind’s CEO and co-founder Demis Hassabis. During testing, AlphaGo Zero was able not just to recover past human knowledge about Go, but also to produce new knowledge based on a constant process of self-play reinforcement.

The implication, in other words, is that powerful learning algorithms could be put to the task of training better humans, or even of outperforming humans to solve real-world problems.

The computing company IBM, which has also piled huge effort and resources into ‘cognitive computing’ in the shape of IBM Watson, has applied similar claims in relation to the optimization of human cognition. Its own cognitive systems, it claims, are based on neuroscientific insights into the structure and functioning of the human brain—as Jessica Pykett, Selena Nemorin and I have documented.

‘It’s true that cognitive systems are machines that are inspired by the human brain,’ IBM’s senior vice-president of research and solutions has argued in a recent paper. ‘But it’s also true that these machines will inspire the human brain, increase our capacity for reason and rewire the ways in which we learn.’

DeepMind and IBM Watson are both based on scientific theories of learning—psychological behaviourism and cognitive neuroscience—which are being utilized to create ‘superhuman’ algorithmic systems of learning and knowledge creation. They translate the underlying theories of behaviourist psychology and cognitive neuroscience into code and algorithms which can be trained, reinforced and rewarded, and even become auodidactic self-reinforcing machines that can exceed human expertise.

For educators and researchers of education this should raise pressing questions. In particular, it challenges us to rethink how well we are able to comprehend processes normally considered part of our domain as they are now being refigured computationally. What does it mean to talk about theories of learning when the learning in question takes place in neural network algorithms?

‘Machine behaviourism’ of the kind developed at DeepMind may be one of today’s most significant theories of learning. But because the processes it explains occur in computers rather than in humans, education research has little to say about it or its implications.

Developments in machine learning, autodidactic algorithms and self-reinforcement processes might enlarge the scope for educational studies. Cognitive science and neuroscience already embrace computational methods to understand learning processes—in ways which sometimes appear to reduce the human mind to algorithmic processes and the brain to software. IBM’s engineers for cognitive computing in education, for example, believe their technical developments will inspire new understandings of human cognition.

A social scientific approach to these computational theories of learning will be essential, as we seek to understand better how a population of nonhuman systems is being trained to learn from experience and thereby learning to interact with human learning processes. In this sense, the models of learning that are encoded in machine learning systems may have significant social consequences. They need to be examined as closely as previous sociological studies have examined the expertise of the ‘psy-sciences’ in contemporary expressions of authority and management over human beings.

Public hypernudge pedagogy
The social implications of machine learning can be approached in two ways requiring further educational examination. The first relates to how behavioural psychology has become a source of inspiration for social media platform designers, and how social media platforms are taking on a distinctively pedagogic role.

Most modern social media platforms are based on insights from behaviour change science, or related variants of behavioural economics. They make use of extensive data about users to produce recommendations and prompts which might shape users’ subsequent experiences. Machine learning processes are utilized to mine user data for patterns of behaviours, preferences and sentiments, compare those data and results with vast databases of other users’ activities, and then filter, recommend or suggest what the user sees or experiences on the platform.

Machine learning-based data analytics processes have, of course, become controversial following  news about psychological profiling and microtargeting via social media during elections—otherwise described as ‘public opinion manipulation’ and ‘computational propaganda.’ The field of education needs to be involved in this debate because the machine learning conducted on social media performs the role of a kind of ‘public pedagogy’—that is, the lessons taught outside of formal educational institutions by popular culture, informal institutions, public spaces, dominant cultural discourses, and both the traditional and social media.

The public pedagogies of social media are significant not just because they are led by machine learning, though. They are also deeply informed by psychology, and specifically by behavioural psychology. The behavioural psy-sciences are today deeply involved in defining the nature of human behaviours through their disciplinary explanations, and in informing strategic commercial and governmental aspirations.

In Neuroliberalism, Mark Whitehead and coauthors suggest that big data software is being regarded as spelling a ‘golden age’ for behavioural science, since data will be used not just to reflect the user’s behaviour but to determine it as well. At the core of the social media and behavioural science connection are the psychological ideas that people’s attention can be ‘hooked’ through simple psychological tricks, and then that their subsequent behaviours and persistent habits can be ‘triggered’ through ‘persuasive computing’ and ‘behavioural design.’

Silicon Valley’s social media designers know how to shape behaviour through technical design since, according to Jacob Weisberg, ‘the disciplines that prepare you for such a career are software architecture, applied psychology, and behavioral economics—using what we know about human vulnerabilities in order to engineer compulsion.’ Weisberg highlights how many of Silicon Valley’s engineers are graduates of the Persuasive Computing Lab at Stanford University, which uses ‘methods from experimental psychology to demonstrate that computers can change people’s thoughts and behaviors in predictable ways.’

Behaviourist rewards—or reinforcement learning—is important in the field of persuasive computing since it compels people to keep coming back to the platform. In so doing, they generate more data about themselves, their preferences and behaviours, which can then be processed to make the platform experience more rewarding. These techniques are, in turn, interesting to behaviour change scientists and policymakers because they offer ways of triggering certain behaviours or ‘nudging’ people to make decisions within the ‘choice architecture’ offered by the environment.

Karen Yeung describes the application of psychological data about people to predict, target and change their emotions and behaviours as ‘hypernudging.’ Hypernudging techniques make use of both persuasive computing techniques of hooking users and of behavioural change science insights into how to trigger particular actions and responses.

‘These techniques are being used to shape the informational choice context in which individual decision-making occurs,’ argues Yeung, ‘with the aim of channelling attention and decision-making in directions preferred by the “choice architect”.’

Through the design of psychological nudging strategies, digital media organizations are beginning to play a powerful role in shaping and governing behaviours and sentiments.

Some Silicon Valley engineers have begun to worry about the negative psychological and neurological consequences of social media’s ‘psychological tricks’ on people’s attention and cognition. Silicon Valley has become a ‘global behaviour-modification empire,’ claims Jaron Lanier. Likewise, AI critics are concerned that increasingly sophisticated algorithms will nudge and cajole people to act in ways which have been deemed most appropriate—or optimally rewarding—by their underlying algorithms, with significant potential social implications.

Underpinning all of this is a particular behaviourist view of learning which holds that people’s behaviours can be manipulated and conditioned through the design of digital architectures. Audrey Watters has suggested that behaviourism is already re-emerging in the field of ed-tech, through apps and platforms that emphasize ‘continuous automatic reinforcement’ of ‘correct behaviours’ as defined by software engineers. In both the public pedagogies of social media and the pedagogies of the tech-enhanced classroom, a digital re-boot of behaviourist learning theory is being put into practice.

Behavioural nudging through algorithmic machine learning is now becoming integral to the public hypernudge pedagogies of social media. It is part of the instructional architecture of the digital environment that people inhabit in their everyday lives, constantly seeking to hook, trigger and nudge people towards particular persistent routines and to condition ‘correct’ behavioural habits that have been defined by platform designers as preferable in some way. Educational research should engage closely with the public hypernudge pedagogies that occur when the behavioural sciences combine with the behaviourism of algorithmic machine learning, and look more closely at the underlying behavioural science theories of learning on which they are based and the behaviours they are designed to condition.

Big Dewey
The second major set of implications of machine learning relates to the uptake of data-driven technologies within education specifically. Although the concept of ‘personalized learning’ has many different faces, its dominant contemporary framing is through the logic of big data analytics. Personalized learning has become a powerful idea for the ed-tech sector, which is increasingly influential in envisioning large-scale educational reform through its adaptive platforms.

Personalized learning platforms usually consist of some combination of data-mining, learning analytics, and adaptive software. Student data are collected by such systems, then compared with an ideal model of student performance, in order to generate predictions of likely future progress and outcomes, or adapt responsively to meet individual students’ needs as deemed appropriate by the analysis.

In short, personalized learning depends on autodidactic machine learning algorithms being put to work to mine, extract and process student data in an automated fashion.

The discourse surrounding personalized learning frames it as a new mode of ‘progressive’ education, with conscious echoes of John Dewey’s student-centred pedagogies and associated models of project-based, experiential and inquiry-based learning. Dewey’s work has proven to be one of the most influential and durable philosophical theories in education, often used in conjunction with more overtly psychological accounts of the role that experience plays in learning.

With its combination of big data analytics and machine learning with progressivism, we could call the learning theory behind personalization ‘Big Dewey.’

Mark Zuckerberg’s philanthropic Chan-Zuckerberg Initiative is typical of the application of Big Dewey to education. CZI aims ‘to support the development and broad adoption of powerful personalized learning solutions. … Many philanthropic organizations give away money, but the Chan Zuckerberg Initiative is uniquely positioned to design, build and scale software systems … to help teachers bring personalized learning tools into hundreds of schools.’

To test out this model of learning in practice, new startup ‘lab schools’ have been established by Silicon Valley entrepreneurs. Many act as experimental beta-testing sites for personalized learning platforms–using students as guinea pigs–that might then be sold to other schools. As Benjamin Doxtdator has documented, these new lab school models of ‘hyperpersonalization’ utilize digital data technologies to ‘extract’ the ‘mental work’ of students from the learning environment in order to tune and optimize their platforms prior to marketing to other institutions.

Larry Cuban, however, has detailed the variety of ways that personalized learning has been taken up in schools in Silicon Valley, and himself sees strong traces of progressivism in their practices.

However, Cuban also notes that many employ methods more similar to the kind of ‘administrative progressivism’ associated with the psychologist EL Thorndike than Dewey. Thorndike was interested in identifying the ‘laws of learning’ through statistical analysis, which might then be used to inform the design of interventions to improve ‘human resources.’ Measurement of learning could thereby contribute to the optimization of ‘industrial management’ techniques both within the school and the workplace. Administrative progressivism was concerned with measurement, standardization and scientific management of schools rather than the student-centred pedagogies of Dewey.

‘What exists now is a re-emergence of the efficiency-minded “administrative progressives” from a century ago,’ argues Cuban, ‘who now, as entrepreneurs and practical reformers want public schools to be more market-like where supply and demand reign, and more realistic in preparing students for a competitive job market.’

With machine learning as its basis, personalization is a twenty-first century algorithmic spin on administrative progressivism. The ‘laws of learning’ are becoming visible to those organizations with the technical capacity to mine and analyse student data, who can then use this knowledge to derive new theoretical explanations of learning processes and produce personalized learning software solutions. As an emerging form of algorithmic progressivism, personalization combines the appeal of Dewey with the scientific promise of big data and autodidactic machine learning.

Ultimately, with the Big Dewey model, the logics of machine learning are being applied to the personalization of the learning experiences to be had by human learners. With this new model of education being supported with massive financial power and influence by Bill Gates, Mark Zuckerberg, and other edtech entrepreneurs, philanthropists and investors, Big Dewey is being forwarded as the philosophy and learning theory for the technological reform of education.

Machine learning escapes the lab
The machine behaviourism of autodidactic algorithm systems, public hypernudge pedagogies and personalized learning have become three of the most significant educational developments of recent years. All are challenging to educational research in related ways.

Machine behaviourism requires educational researchers to move their focus on to the kinds of reinforcement learning that occurs in automated nonhuman systems, and on how computational systems are being taught and trained by programmers, algorithm designers and engineers to learn from experience in an increasingly autodidactic way.

It’s not a sufficient response to claim that companies like DeepMind, IBM and so on take a reductionist view of what learning is—DeepMind’s Nature paper reveals an incredibly sophisticated learning model as pertains to neural networks software, while IBM has built its cognitive systems on the basis of established neuroscience knowledge about the human brain.

These systems can learn, but are not the same forms of learning known to most education researchers. As technical innovation proceeds, more and more learning is going to be happening inside computers. Just as educators hope to cultivate young minds to become lifelong independent learners, the tech sector is super-powering learning processes to create increasingly automated nonhuman machine learning agents to share the world with humans. What’s to say that educational researchers should not seek to develop their expertise in understanding nonhuman machine learning?

Theories of nonhuman learning are also becoming increasingly influential since machine learning processes underpin both the public hypernudge pedagogies of social media and personalized learning platforms I’ve outlined. The new behaviourist public hypernudge pedagogies, inspired both by behavioural science and behaviour design, are occurring at great scale among different publics, often according to political and commercial objectives, yet education research is oddly silent in this area.

While much has been written about big data and personalization, we’ve also still to fully explore how the tech sector philosophy of Big Dewey might affect and influence schools, teachers and students as adaptive learning platforms escape from the beta-testing lab and begin to colonize state education. Future studies of personalized learning could examine the forms of autodidactic machine learning occurring in the computer as well as the educational effects and outcomes produced in the classroom.

Image by Terry Kimura