Skip to content
This site uses cookies. By continuing to browse, you agree to the use of cookies. You can change your settings or learn more here.
OK
#Open journalism No news is bad news

Your contributions will help us continue to deliver the stories that are important to you

Support The Journal

Irish Water website crashed after boil water notice when just 5,000 people tried to log on

Irish Water said its site had “previously handled this volume of traffic during heavy public awareness campaigns”.
Nov 22nd 2019, 10:20 PM 17,340 34

water 00_90584331 Source: Sam Boal/Rollingnews.ie

IRISH WATER WAS concerned about the reputational risks it faced when it issued the first boil water notice last month, including how its website crashed almost as soon as the notice was announced even though it “previously handled this volume of traffic during heavy public awareness campaigns”.

On two occasions in the past five weeks, Irish Water has issued boil water notices for 600,000 people across parts of Dublin, Kildare and Meath.

The first came on the evening of 22 October as a precautionary measure following issues with the treatment process at the Leixlip water treatment plant.

Documents released to TheJournal.ie under the Freedom of Information Act have revealed the timeline behind the initial identification of the problem leading up to the issuing of the boil water notice, as well as the website woes which meant people couldn’t access the Irish Water website when the announcement was made. 

In a mail sent by a senior engineer at Fingal County Council to a number of stakeholders on the day the boil water notice was issued, he highlighted how an initial blockage identified at 3pm the day before escalated over the course of the day. 

boil water 1

In an email to senior figures in Irish Water, the EPA and the HSE after 2pm on the afternoon of the boil water notice, a Department of Health public health specialist said they must be on standby to issue the notice.

“We have discussed this incident internally in HSE and our concern is whether or not a boil water notice is required for ongoing protection of human health in the Leixlip distribution system as a consequence of this incident,” she said. “Please be prepared to issue a BWN [boil water notice] promptly after our teleconference if that is our conclusion.”

That was indeed the conclusion, and Irish Water immediately sought to take action. It had six strategic aims, including protecting public health, minimising the impact on customers, eventually lifting the ban and also managing Irish Water’s reputation risk.

Website down

One problem with this plan of communicating effectively with the public about the boil water notice came almost immediately when the announcement was made just before 6pm on the evening of 22 October.

The Irish Water website almost immediately stopped working, meaning people were unable to see if their area was affected by the notice.

This inevitably led to criticism on social media.

The following morning, Irish Water’s head of IT provided an update of what had happened.

He said: “At 18.05 yesterday evening the Irish Water website had 5,000 simultaneous hits following the media release of boil water notices (we had ~300 connections prior to this public announcement.

The website has previously handled this volume of traffic during heavy public awareness campaigns; why it cannot currently do so is still being investigated.
As an interim measure, the web-server capacity has been doubled is available since late yesterday evening and a separate contingency stream has been activated to migrate the Water.ie website to an alternative system.

A further update from the head of IT later that morning said the website was currently having 600 users at any one time. “Monitoring and performance will be managed in conjunction with media updates and lunch time periods,” he said.

A later update that day said that the “root cause not yet confirmed however testing ongoing”. 

It was identified that this problem with the website could affect Irish Water’s aim of managing its reputational risk through the crisis.

An Irish Water spokesperson told TheJournal.ie this week: “As a result of difficulties encountered by customers accessing the Irish Water website on the 22 October we mobilised a dedicated support team to resolve the issues and to restore access to the website for customers as quickly as possible. Measures were put in place to prevent similar issues occurring in the future. 

Irish Water used multiple channels to inform and update customers including social media, press releases, interviews on TV and radio at national and local levels. We also made direct contact with vulnerable customers and water dependent businesses. Irish Water would like to apologise to customers for any inconvenience caused and to thank them for their patience during the outage. 

The website didn’t experience a similar issue when the second boil water notice was issued earlier this month. And, even before that notice was issued, an investigation had identified the process that led to the problems in the first instance.

As highlighted by the Environmental Protection Agency, a blockage at the Leixlip plant led to the issuing of the largest-ever boil water notice in Ireland.

The EPA confirmed that a blockage leading to elevated turbidity levels in treated water was the main cause of the incident which led to the notice.

The blockage was made up of small black particles and debris, which appeared to emerge from two alum storage tanks.

The EPA noted that the two tanks are approximately 25 years old, and that it appeared the their internal lining material deteriorated, creating small flakes which later caused a blockage.

“This compromised the integrity of the treatment barriers for the removal of  Cryptosporidium/Giardia [parasites] creating a significant risk to the safety of the water supply,” the EPA said.

However, the agency also found that Irish Water subsequently failed to respond to process alarms which activated at the plant as a result of the elevated turbidity levels.

After the most recent boil water notice was lifted on 12 November, Irish Water said it would work with the EPA and the council to minimise the risk of having to issue another one. 

With reporting from Stephen McDermott

Send a tip to the author

Sean Murray

COMMENTS (34)

This is YOUR comments community. Stay civil, stay constructive, stay on topic. Please familiarise yourself with our comments policy here before taking part.
write a comment

    Leave a comment

     
    cancel reply
    Back to top