Performing data

‘Performance information’ in the Scottish Government national improvement plan for education

Ben Williamson

At the end of June 2016 the Scottish Government published a major national delivery plan for improving Scottish education over the next few years. Drafted in response to a recent independent review of Scottish education carried out by the OECD, the delivery plan is part of a National Improvement Framework with ambitious plans to raise attainment and achieve equity.

It is the relentless focus of the delivery plan on the use of performance measurement, metrics and evidence gathering to drive forwards these improvements that is especially arresting. In a striking line from the introduction it is stated that:

As the OECD review highlighted, current … arrangements do not provide sufficiently robust information across the system to support policy and improvement. We must move from a culture of judgement to a system of judgement.

A ‘system of judgment’: right from the start, it is clear that the delivery plan is based on the understanding—imported from the OECD via its recommendation that new ‘metrics’ be devised to measure Scottish education—that data can be used to drive forward performance improvement and for the purposes of disciplining under-performance.

Productive measurement
In a series of articles, the sociologist David Beer has been writing about the socially productive power of metrics in a variety of sectors and institutions of society:

We often think of measurement as in some way capturing the properties of the world we live in. This might be the case, but we can also suggest that the way that we are measured produces certain outcomes. We adapt to the systems of measurement that we are living within.

Metrics and measurements are not simply descriptive of the world, then, but play a part in reshaping it in particular ways, affecting how people behave and understand things and act to do things differently. As Beer elaborates:

The measurements themselves matter, but it is knowing or expecting how we will be measured that is really powerful. Systems of measurement then have productive powers in our lives, both in terms of how we respond to them and how they inform the judgments and decisions that impact upon us.

Performance measurement techniques, of the kind to be implemented through the Scottish Government’s proposed ‘system of judgement’, can similarly be understood as productive measures that will be used to attach evaluative numbers to practices and institutions in ways that are intended to change how the system performs overall. This is likely to affect how school teachers, leaders, and maybe even pupils themselves and their parents act and perform their roles, as they expect to be measured, judged, and acted upon as a result.

‘Performance information’ is one of the key ‘drivers of improvement’ listed in the plan, and clearly shows how a range of ‘measures’ are to be collected:

We will pull together all the information and data we need to support improvement.  Evidence suggests … we must ensure we build a sound understanding of the range of factors that contribute to a successful education system. This is supported by international evidence which confirms that there is no specific measure that will provide a picture of performance. We want to use a balanced range of measures to evaluate Scottish education and take action to improve further.

Scanning through the plan and the improvement framework, it becomes clear just how extensive this new focus on performance measurement will become. The plan emphasizes:

• the use of standardized assessment to gather attainment data
• the gathering of diverse data about the academic progress and well-being of pupils at all stages
• pre-inspection questionnaires, school inspection and local authority self-evaluation reports
• the production of key performance indicators on employability skills
• greater performance measurement and measurement of schools
• new standards and evaluation frameworks for schools
• information on teacher induction, teacher views, and opportunities for professional learning
• evidence on the impact of parents in helping schools to improve
• regular publication of individual school data
• the use of visual data dashboards to make school data transparent
• training for ‘data literacy’ among teachers
• comparison with international evidence

All of this is in addition to system-wide national benchmarking, international comparisons, defining and monitoring standards, quality assurance, and is all to be overseen by an international council of expert business and reformatory advisors to guide and evaluate its implementation.

Performative numbers
The delivery plan makes for quite a cascade of new and productive measures–an ‘avalanche of numbers‘ –though Scottish schools are unlikely to be terribly surprised by the emphasis in the delivery plan on performance information, targets, performance indicators and timelines. (In England the emphasis on performance data has been even more pronounced, with Paul Morris claiming ‘the purposes of schooling and what it means to be educated are effectively being redefined by the metrics by which we evaluate schools and pupils.’)

Since 2014, all Scottish schools have been encouraged by the Scottish Government to make use of Insight, an online benchmarking tool ‘designed for use by secondary schools and local authorities to identify success and areas where improvements can be made, with the ultimate aim of making a positive difference for pupils’. It provides data on ‘four national measures, including post-school destinations and attainment in literacy and numeracy as well as information on a number of local measures designed to help users take a closer look at their curriculum, subjects and courses’. It features data dashboards that allow schools to view an overall picture of the data from their school and compare it with the national measures presented on the national dashboard.

A notable feature of Insight is the ‘Virtual Comparator’ which allows users to see how the performance of their pupils compares to a similar group of pupils from across Scotland. The Virtual Comparator feature takes the characteristics of pupils in a school and matches them to similar pupils from across Scotland to create a ‘virtual school’ against which a ‘real’ school may benchmark its progress.

The relentless focus by the Scottish Government on performance information, inspection, comparison, measurement and evidence is demonstrative of how education systems, organizations and individuals are now the subjects of increasing demands of producing data.

As the concept of ‘productive measures’ reminds us, though, performance measurement is not simply descriptive. It also brings the matter it describes into being. Captured in the term ‘performativity,’ it has become apparent that education systems and institutions, and even individuals themselves, are changing their practices to ensure the best possible measures of performance. Closely linked to this is the notion of accountability, that is, the production of evidence that proves the effectiveness—in terms of measurable results—of whatever has been performed in the name of improvement and enhancement. As Stephen Ball phrases it:

Performativity is … a regime of accountability that employs judgements, comparisons and displays as a means of control, attrition and change. The performance of individuals and organizations serve as measures of productivity or output … [and] stand for, encapsulate or represent the worth, quality or value of an individual or organization within a field of judgement.

In other words, performativity makes the question of what counts as worthwhile activity in education into the question of what can be counted and of what account can be given for it. It reorients institutions and individuals to focus on those things that can be counted and accounted for with evidence of their delivery, results and positive outcomes, and de-emphasises any activities that cannot be easily and effectively measured.

In practical terms, performativity depends on databases, audits, inspections, reviews, reports, and the regular publication of results, and tends to prioritize the practices and judgements of accountants, lawyers and managers who subject practitioners to constant processes of target-setting, measurement, comparison and evaluation. The appointment of an international council of experts to oversee the collection and analysis of all the performance information required by the improvement and delivery plans is ample illustration of how Scottish education will be subject to a system of expert techniques and judgement.

Political analytics
It is hard, then, to see the Scottish Government delivery plan as anything other than a series of policy instruments that via specific data-driven techniques and particular technical tools will reinforce performativity and accountability, all under the aspiration of closing attainment gaps and achieving equity.

Although no explicit mention is made of the technologies required to enact this system of judgement, it is clear that a complex data infrastructure of technologies and technical experts will also be needed to collect, store, clean, filter, analyse, visualize and communicate the vast masses of performance information. Insight and other dashboards already employed in Scottish education are existing products that doubtless anticipate a much more system-wide digital datafication of the sector. Data processing technologies are making the performance of education systems and institutions into enumerated timestreams of data by which they might be measured, evaluated and assessed, held up to both political and public scrutiny, and then made to account for their actions and decisions, and either rewarded or disciplined accordingly. A new kind of political analytics that prioritizes digitized forms of data collection and analysis is likely to play a powerful role in the governance of Scottish education in coming years.

Data technologies of various kinds are the enablers of performativity and accountability, and translate the numerical logics of the technologies into the material and practical realities of professional life. As a data-driven ‘system of judgement’, Scotland’s delivery plan for education will, in other words, usher in more and more ‘productive measures’ into Scottish education, reconfiguring it and those who work and learn in it in ways that will need to be studied closely for many years to come.