Opinion Government must learn the lessons of the Leaving Cert calculated grades debacle

THE USE OF technology in public administration is part of the wider digitalisation of the administrative state. Processes of the state are moving online and into code. This is not a new phenomenon.

However, the coding errors in the Leaving Certificate calculated grades process reinforce the importance of ensuring transparency and openness are incorporated into the use of technology employed by the government.

Despite the growing role of technology, Government is still very much a human endeavour. The human consequences of the coding errors are being felt by Leaving Certificate students and their support networks.

There are also consequences across the educational sector including for third-level institution resourcing and the points system operated by the Central Applications Office.
If there is to trust in public administration, our politicians and civil servants need to be frank regarding the role of automated processes.

There can be no question of hiding behind machines or outsourcing agreements. In moving government processes online or into code, there needs to be a commitment to transparency and openness in the relationship between individuals and the state.

Mistakes happen but when they do there needs to be a culture of clear and swift communication of what went wrong, how the matter is going to be resolved and steps being taken to ensure similar mistakes do not happen again.

What is coding?

As noted above, Government is a human endeavour and so is coding. Code is how a programmer tells a computer what to do, much the same as we use speech or text to communicate with each other.

There are many programming languages that a programmer can use, much like there are many languages spoken across the globe. Regardless of which programming language is being used, code comes down to the same thing: a series of commands that the computer follows to implement a process.

If those commands are incorrect, the computer doesn’t know that; it just implements a process incorrectly. This happens frequently, and programming largely consists of tracking down these errors and fixing them. Occasionally one will slip through.

The consequences of such an error could be as minor as a broken link on a website, but as code becomes increasingly integral to every facet of public administration, an error can have a significant negative impact on a large number of individuals. That is exactly what happened in implementing the Leaving Certificate calculated grades process.

What went wrong?

The Leaving Certificate calculated grades process was designed to ensure that students are graded fairly throughout the country. The question could be raised as to why this isn’t left to the teachers and schools; the people best placed to give accurate estimates of a student’s likely performance in an exam.

The answer is that teachers are extremely good at estimating the relative ability of the students in their class, but a report by the National Standardisation Group notes that “it is more difficult for teachers to align their estimates accurately with the judgements of other teachers in other schools and with an external national standard”.

In order to account for this factor, the Leaving Certificate calculated grades process made use of Junior Certificate results data to standardise the Leaving Certificate results nationally.

At the time of writing, three errors have been found in the code written to implement the Leaving Certificate calculated grades process. The first error, which was reported by the contractor, caused students’ Junior Certificate results in the core subjects (English, Irish and Maths), along with their weakest two other subjects, to be included in the data used by the process.

However, it was intended that their core subject results would be used together with their strongest two other subjects. The second error was that students’ results in CSPE were not removed from the data used by the model when they should have been excluded.

The third and most recently discovered error was that cases, where a student had no result in one of their core subjects, were not treated in the correct manner. These errors should have been noticed during code review, a quality assurance practice where the code is examined by a programmer who was not involved in writing it.

Read Next

Related Reads

Leaving Cert students affected by calculated grades error to be included in Round 4 CAO offers

6,100 students to get text message this evening upgrading their LC result

Students mistakenly awarded higher grades may have 'squeezed out' others from college, Minister concedes

What can we learn from the coding errors?

There is a need for transparency in the automated processes of government. There is significant frustration with the Department’s week-long delay in announcing the coding errors. The Department had the opportunity to develop a more open and transparent relationship with individuals but may have instead contributed to a further erosion of trust between individuals and the state.

Education Testing Services (ETS) was contracted by the Department “to provide an independent expert opinion on the adequacy of the coding”. The review was limited to “an audit of a sampling of the coding rather than a full audit of the entire coding”. To ensure there is full transparency and trust in the process, we recommend a complete review of the code.

We call on the Minister for Education Norma Foley to publish correspondence between the Department and the original contractor in relation to code review to ensure there is transparency in relation to the roles played by different actors involved in the calculated grades process.

Using code to automate tasks on behalf of the state is a practice that will only grow more prevalent. We need now to learn from the mistakes of the Leaving Certificate calculated grades process and work to develop an open and transparent digital administrative state that people understand and trust.

Aisling Ryan is a PhD student in administrative law at the School of Law, UCC. Pierce Ryan is a PhD student in applied mathematics at the School of Mathematical Sciences, UCC.