With most Puerto Ricans going into month three without power, and recent reports about Russian and Chinese cyber activities in North America, let us then pause for a moment and talk about collapse. It is on everyone's minds even if we're all not actually talking about it. While the role of energy CIO has a lot to do with new gadgets for customers and smooth operating automated distribution equipment and analytics tools, there is ultimately—for me—a much more compelling public compact: CIOs are ultimately leading the information technology evolution to keep the power on at all times.
"We’re going to solve these problems by setting up the humans that run the grid to be successful"
Embedded in signing up as energy CIO, you are signing up for committing to ensure public health, welfare, and safety through delivery of electricity—a critical public good and service. So how do you address outages, cyber attacks, natural disasters? These are the moments when there is only a thin veneer of civilization. When the lights go out, the “Fear of Missing Out” often becomes purely fear for your customers, and stress for your operators. The only thing ensuring electricity quality and reliability is control room and field and system operators maintaining information technology on the grid. “Society’s uptime”, if you will. The weakness is when your control centers lose connection to the precious IT assets they maintain.
Control Rooms have largely been configured to Eisenhower Administration era technology, while the rest of the world has gone digital. Of the thousands of control rooms that are impacting and impacted by the grid today, the majority of them have one-lines and wholesale markets, SCADA data, and Operations and fleet information. But the truth is, that is a silo - the grid touches everything - everything. Our gas infrastructure, our water infrastructure, our highways, our homes, our hospitals, and even our phones. In fact, our very way of life as Americans. The current control room setup is designed to see 0-1% and 99-100% of the grid, but no low-, mid-, and high-level data and collaboration across those operators. Speaking for the IT part of this equation: This simply does not set the operator up to succeed.
One of the greatest simple problems to solve in our industry is also a wildly political and behavioral one: namely, the human factors of technology have not been designed to the benefit of control room operators to be able to see what’s happening on the grid. That leads to collapse.
The most devastating intersection of challenges come when technology overloads across organizations and their people. Operators falling or having things they used to control automated, so that operators are “out of the loop”. This is the impact that rapidly changing technology is having on the role and responsibilities of Energy CIO today—IT makes it evermore paramount to keep the human at the center of the grid. IT will not get your grid back up, but humans collaborating together and set up to succeed will.
Unless you lived through it, you may not vividly recall August 4th, 2003: The Big East Coast Blackout. This particular system failure was rooted not in technology nor operations— it was caused by human factors. First, a lack of situational awareness left operators unaware of the need to re-distribute power after the overloaded lines triggered a manageable blackout. Second, lack of collaboration with neighboring grid operators led to whole neighboring ISO systems shutting down as well.
For this period of time, operators in control rooms worked like mad—but mostly on the process of sharing their data with other control rooms and doing so without their regular tools, just in order to start addressing the most pressing grid-centric problems. For days, there was complete loss of situational awareness and collaboration across emergency response systems, utilities, and safety and public services. The human toll was real and it led to hundreds of harrowing situations with lives significantly impacted or lost.
Our industry has come a long way with new technology since 2003. But not all the way. Credibly, collapse can be on the horizon from non-natural events – say, Cybersecurity collapse. (e.g. – Ukraine, 2016). Let's talk about security. Everyone has been dealing with a plethora of attacks, and most of them have been detected and prevented. But as the reality sets in that Russia and China have been actively attacking our grid for over a half-decade, if they wanted to hack, they will have succeeded, and you won’t necessarily have a true control of your information technology (e.g. Stuxnet). As recently as last week the New York Times reported how the NSA has been hung out to dry. Two years ago it was found that China has all of our OPM files, and last year we realized Russia has access to all of our emails. Attacks on different parts of the North American system—an annual occurrence with darker undertones.
Energy CIOs can go far in addressing these problems simply by setting up the humans that run the grid to be successful. This mid-November, 2017, NERC conducted it’s GridExIV exercise, ensuring that many (e.g. thousands) organizations and different parts of those organizations are ready to work together effectively through many different situations. It is well recognized that the more people can collaborate, share information, and maintain situational awareness, the more resilient the grid becomes. The concept of Resilience Engineering (Hollnagel, Woods & Leveson) focuses specifically on this point—the primary sources of reliability and robustness may be in our technology, but our primary source of resiliency is solidly human, and everything we do to set our people up to be successful lets them set up our grids to be successful.
GridEX IV provided a rich example of using situational awareness and collaboration to our advantage when the grid is under threat. Today’s Energy CIO has a front row seat to the most significant transformative moment in our industry. What we learn from GridEX and successful responses to grid situations like Hurricanes Harvey and Irma is that we do best when we set up people with situational awareness and collaboration all the way through from day to day activity and across crises.
Energy CIOs prevent collapse by investing in people. The CIO’s role is to live the mission of technology and harness a love of technology to make the world a better place and empowers the grid to be all the things it can be. When the next big challenge, conflict, or crisis occurs on our home turf, the story of success will ultimately be around empowering humans to grow situational awareness and collaboration across the grid.