Smart inverters expand opportunities for more distributed resources on the grid. But the internet-enabled communications that allow smart inverters to work with the grid also open the door for something sinister.

Cybersecurity issues increase as inverters get smarter, and the threat level will only rise as solar makes up more and more of the energy mix — which will accelerate even more rapidly with California’s 2020 mandate of solar on all new homes.

Scientists at Lawrence Berkeley National Lab have been working on solutions to combat cybersecurity threats on smart inverters since 2016. They’ve found some promising solutions to inevitable hacks, but also some serious challenges that come with the growing proliferation of smart inverters.

The threats that come with smart inverters

Any device connected to the internet could be in danger of being hacked, even smart inverters. They communicate with the grid to perform voltage management functions autonomously, using internet-connected software. When used for good, this means smart inverters can regulate the voltage of power feeding into the grid in such a way that no damaging fluctuations occur. But if hackers gain control of smart inverters, they could potentially feed bad settings into the software and throw the voltage out of control, leading to brownouts or blackouts in extreme cases, according to Dan Arnold, research scientist at Berkeley Lab and one of the leads on the inverter cybersecurity project.

If only a few inverters have bad voltage settings programmed, the grid likely wouldn’t feel much of an impact. But when bad software is controlling a large aggregation of smart inverters and moving their voltages simultaneously in the wrong direction, it could cause the grid to collapse.

Inverter manufacturers can take precautions to ensure cybersecurity on their end, but when the internet is involved and there aren’t communication standards in place from Day 1, hacks become inevitable.

“How I frame our research is, we’re asking the question, ‘What if those mechanisms fail and somebody does gain access to large populations of inverters? What damage could they do?'” Arnold said.

Depending on who you talk to, that damage could range from moderate to “the sky is falling,” he said. Damage to the grid would vary based on the sophistication of the attack. At the local level, a population of inverters introducing voltage oscillations to the system could bring voltage down and damage sensitive loads on the distribution grid. At a wider scale, a well-coordinated attack over large areas of the grid could harm the sub-transmission and transmission systems, leading to widespread blackouts.

Arnold said these attacks could come from either individual hackers or unfriendly foreign governments looking to disrupt America’s grid. Utility officials he’s spoken with are most concerned about cyber threats following natural disasters. Utility resources are always overburdened in the aftermath, prioritizing fixing downed powerlines and important regulation equipment. Distributed energy resources like solar might be called on to fill the energy gaps in the meantime, so the stakes are high for them to operate correctly. If inverters were compromised in this scenario and low voltage caused them to trip and disconnect from the grid, the whole grid could go down and stay down.

If the grid does go down, it could take time for utilities to determine when it’s safe to reconnect the compromised parts.

“Utilities are very hesitant to reconnect parts of the system if they are just going to go down again if a cyberattack is still there,” Arnold said. “We really want to avoid the collapse of the system and facilitate speedy reconnection.”

The Berkeley Lab solution

Arnold and his co-project lead and staff scientist at the lab, Sean Peisert, have discovered a relatively simple and effective solution to fighting cyber threats to smart inverters — using other, uncompromised smart inverters to neutralize cyberattacks.

Two smart inverters connected at the same point in the grid look like one big inverter from the grid’s perspective. This allows them to work in tandem to regulate voltage.

“If somebody rolls out bad settings to one of those inverters, then the slopes of that particular inverter’s control curve are going to increase. But at the same time, we could reduce the slopes of the other inverter (its neighbor), and then the overall curve would be unchanged, even though one inverter has been hacked,” Arnold said. “And we’ve designed an approach that allows these inverters to individually make that decision — ‘Should I reduce my slopes?’ — without having to talk to each other or talk to some centralized entity.

“Our research has shown that even a very small reduction in the slopes of some of these good inverters could mitigate a wide variety of cyberattacks,” Arnold added.

This solution’s greatest strength is that it doesn’t come with much overhead to implement. The software uses the functions already programmed in smart inverters to take note of when voltages are oscillating and respond with voltage settings that are less aggressive. The Berkeley Lab scientists’ software solution can be housed inside individual smart inverters, on a utility’s back-end system or inverter manufacturer’s back-end system.

“You don’t need a sophisticated communications network that has to activate when an attack takes place. You don’t need a large coordinated response. The individual inverters respond on their own, and it’s the aggregate response of the system that mitigates the attack,” Arnold said.

The neutralizing approach works, but the challenge lies in rolling it out to the industry. Arnold said he would be interested in exploring the possibility of adding this software as a requirement to the IEEE 1547 smart inverter standard and Rule 21 interconnection mandate.

“If that is not enough of an incentive to get inverter manufacturers to behave correctly, then I would like to see PUCs or utilities come down and impose some steeper regulations,” Arnold said.

The manufacturer’s responsibility

Arnold said he realizes inverter manufacturers are in a tough spot because they face increasing pressure to open their systems to be controlled remotely. But they need to be cognizant that doing so comes with cybersecurity risks.

“Manufacturers should just follow best practices for ensuring communication and they should particularly pay attention to how secure their back-end systems are,” Arnold said.

These back-end systems, used for aggregating their inverters and selling them to electricity bidding markets, can be very tempting targets for attackers, since it’s one location to target and bring down a large swath of inverters. He said those systems in particular need to be extremely secure.

Morningstar Corporation manufactures inverters mostly for the off-grid market, but these inverters are still connected to cellular networks, so cybersecurity precautions are necessary, said Brad Berwald, product director for the company. Morningstar plans to make its inverters more grid-interactive and software-defined in the near future, too.

“We’re seeing so much more cellular and even satellite connectivity. Just because we’re off-grid doesn’t mean we’re disconnected,” Berwald said. “We fully intend to provide just as much of a robust security platform for off-grid as we would for grid-connected.”

The company prioritizes cybersecurity and follows ISO information technology standards in every step of its supply chain, from the firmware to the cloud. In pre-manufacturing, the firmware is pre-encrypted and programmed to the chip in an independent, secure facility. The chip can detect tampering in the future, too. Morningstar Corporation’s inverters come with a hardware lockout set as the default, so even though authorized entities can upgrade the firmware remotely, unknown sources can read the firmware but cannot make changes unless the lock is taken off manually.

“If someone did attempt to try to get through this secure boot and write something to it, which they would not be able to do, it will actually detect that and flag that an unauthorized attempt was made at trying to put the chip into a boot-load mode,” Berwald said. “So, we not only prevent it, but we also are notified if that occurs.”

Morningstar is also developing cloud-based software called Envision to enable grid communication. The cloud service would be hosted on a private server and encrypted.

Berwald thinks most sophisticated inverter manufacturers are taking similar security steps, but that smaller off-grid companies may be less equipped to add this amount of protection. He said many of the security steps Morningstar has in place is handled by the data partners it chooses.

“In the era of cloud connectivity and data partnerships with a lot of these hosted virtual services, a lot of this is part and parcel of our product, but it is a carry-through from the partners that we choose,” Berwald said.

Like Morningstar, Fronius also follows ISO standards for IT security. It employs security experts continuously improving security measures to meet changing requirements.

“We conduct internal security testing as well as penetration testing with external partners to identify potential vulnerabilities,” said a Fronius spokesperson. “Based on these test results, we issue software updates or improve other security measures. Furthermore, customer feedback or insights shared by other third parties are analyzed as well and trigger software updates and other security measures.”

The spokesperson said Fronius supports the SunSpec Alliance’s efforts to establish industry-wide standards for inverter cybersecurity, and that a standard will be even more critical when the next phases of California’s Rule 21 go into effect, requiring more remote-control inverter functionalities.

“In order to ensure highest security levels for our customers, we take a holistic approach and not only look at our devices and servers but also communications between them. In addition, we provide guidelines to customers on how they can secure their networks, e.g. by using safe passwords, firewalls and router settings,” the spokesperson said.

A Yaskawa Solectria spokesperson said the company also uses authentication and encryption in its inverters and performs penetration testing to verify its cyberattack mitigation efforts are sound.

Enphase Energy has always viewed its microinverters as connected, IoT devices, according to a spokesperson, so the company has prioritized cybersecurity and devised stringent procedures for development, testing and operations. Its engineering team includes security architects and engineers who work on hardening existing products against cyber threats. Each inverter contains cryptography blocks and intrusion mitigation features as well as the ability to send encrypted software updates if necessary. It also provides end-to-end encryption of all information sent between microinverters in the field and Enphase’s back-end system, and the company’s security team conducts continuous cycles of iterative penetration testing on all devices.

“Beyond testing, Enphase invests in the personnel and tools designed to ensure that the company’s security posture remains strong and that we can continue our endeavor to deliver secure energy products and services,” said a spokesperson.

The challenges that remain

Legislators have shown concern for inverter cybersecurity by moving to ban Huawei from selling inverters in the United States. In February, 11 U.S. senators wrote a letter to Department of Energy Secretary Rick Perry and former Department of Homeland Security Secretary Kirstjen Nielsen asking for a ban on the sale of Huawei solar products in the United States “to protect U.S. electrical systems and infrastructure” due to “concerns with the company’s links to China’s intelligence services.”

“For a very long time now, there have been suggestions that there have been many manufacturers out there that have been tempted by governments to put some sort of backdoor in their software or hardware, whatever it happens to be,” said Berkeley Labs’ Peisert.

Arnold said it’d be great if legislative solutions could prevent inverters from having back channels to allow bad settings, but it’s not entirely necessary to get congresspeople involved. Even if the entire Huawei inverter fleet in the United States was compromised, the Berkeley Lab solution could still help minimize the danger.

However, that approach only works if there are trustworthy, uncompromised inverters able to mitigate harmful voltage oscillations in others.

Even though inverter companies appear to be doing their due diligence to prevent supply chain vulnerabilities from infiltrating their products and minimize other bugs that could be exploited, it’s still impossible to know which companies are the most trustworthy, or how secure and robust each of those products is.

“I’m not seeing any groundbreaking work in the power industry on this,” Peisert said. “I wish I could say otherwise.”

Arnold said the scariest aspect of the modern age is exposing devices to the internet and allowing remote firmware updates to take place. He said a device could be in security compliance with correctly installed software, and hackers could still break into the channels meant for rolling out normal, safe updates and install bad software instead.

“This is just a darn hard problem because even if you take out the fact that there might be a particular vendor of a system that’s legit, it doesn’t mean that somewhere else along the supply chain, there isn’t a vendor that’s malicious as well,” Peisert said. “It could be upstream, for the people who are doing the packaging. It could be downstream, for the people who are building the sub-components. The whole issue of supply chain security is a problem far and wide.”

Peisert said there’s no good solution to that large-scale issue, but the work Berkeley Lab is doing on neutralizing threats can address some elements.

He draws a parallel between equipment operating the power grid and the evolution of Microsoft Windows computers. When computers first began to be plugged into the internet in the mid-90s, they were constantly being hacked. Windows had so many vulnerabilities that virus scanners didn’t stand a chance. In 2002, Bill Gates famously wrote a memo to the Windows team, basically telling them, “Fix this.” And they did.

“There has been no equivalent of the Gates memo for many other kinds of devices that are plugged into the internet,” Peisert said.

Manufacturers are constantly developing internet-enabled technologies meant to improve lives — from smart thermostats to smart doorbells, smart speakers and more. But sometimes these improvements are done without the appropriate protections being put in place first.

“We’ve developed all of these things in a well-intentioned way without thinking about security at all,” Peisert said. “The ship has sailed on being able to do that a long time ago. Developing systems without security front and center that get plugged into the internet is a bit like driving a car without a seatbelt now. It’s just ridiculous and you shouldn’t be doing it.”

Peisert said he hopes that moving forward, manufacturers will prioritize better development practices when building components that will interact with the electric grid, although there will still be a need for technologies that enable automated resilience and fault tolerance, like what Berkeley Lab has developed, because there’s no way to eliminate all vulnerabilities.

“With the human out of the loop, things can get out of control very quickly when you have large aggregations of devices that are making decisions on their own,” Arnold said.