Applying blockchain, Kafka and MongoDB in green energy trading

Energy trading platform Powerledger has leveraged the power of blockchain, Kafka and MongoDB to make its mark in renewable energy trading in Australia

Stephen Withers

Published: 16 Nov 2022 8:00

Energy trading platform Powerledger is among a growing number of blockchain technology providers that have been riding on the renewable energy wave.

The Western Australian company provides various services that are built on its blockchain-based energy trading platform, facilitating peer-to-peer (P2P) trading of excess electricity from rooftop solar power installations, the assembly of multiple household batteries into virtual powerplants and the purchase of renewable electricity from a particular provider.

It even allows a participant to give their excess solar electricity to a customer in another country on a mutual basis between related parties in opposite hemispheres, or as a gift to family members living in a country where electricity prices are high relative to incomes.

As with any blockchain platform, it provides traceability with non-fungible tokens (NFTs) representing renewable energy certificates (RECs), so participants can be assured that the electricity they are buying is indeed from a renewable source.

The company works in cooperation with electricity retailers but provides a decentralised rather than the conventional unidirectional market. The benefits include reduced customer acquisition costs, increased customer satisfaction, better prices for buyers and sellers (compared with feed-in and supply tariffs), and provision for cross-retailer trading.

Vivek Bhandari, Powerledger’s chief technology officer, said although there are existing markets for RECs, they are opaque with prices set by brokers. An organisation might be committed to buying 100% of its electricity from renewable sources, but there may be times when it has no choice other than conventionally generated electricity. When that happens, it will need to buy RECs to compensate.

Powerledger’s market is transparent, and is much faster than transactions conducted through brokers, Bhandari said. Furthermore, it can cope with situations where supply data is subsequently updated, and offsets need to be bought or sold to balance the customer’s consumption.

To date, Powerledger has been operating on a relatively small scale, but there are billions of users in its addressable market, so the company had to identify technology that would be reliable at hyperscale.

It was already using Apache Kafka as a core piece of infrastructure, specifically to ingest data from smart electricity meters and feed it into the trading system. To cope with the anticipated scale, and to help with the adoption of an event-driven architecture, it turned to Kafka specialist Confluent.

Confluent was founded by the creators of Kafka and is used as the basis of scalable real-time applications at companies such as Netflix and around 80% of the Fortune 100, said Confluent’s vice-president in Australia and New Zealand, Gavin Jones.

Examples of such applications include the detection of possibly fraudulent credit card transactions, automatically blocking access to systems when a user’s behaviour resembles that of a breach in progress and determining that a piece of equipment requires preventative maintenance.

Groundswell in P2P solar energy trading

Powerledger’s 2019 Fremantle trial of P2P solar electricity trading among 48 households showed the use of a blockchain for this purpose was technically feasible and had the support of participants.

The trial allowed households to buy and sell excess rooftop solar energy in near real-time, with participants able to view electricity usage in 30-minute intervals.

The report on the trial, prepared by Powerledger, Curtin University and Murdoch University, pointed out that prevailing tariff structures meant some participants’ financial outcomes were largely dependent on their daily energy consumption rather than trading volume. It recommended the combination of P2P trading and dynamic feed-in tariffs as a way to deal with excess solar during the day without requiring government subsidies.

Combining P2P with virtual power plants (VPPs) comprising battery-equipped premises in the same area would allow those households “to monetise their excess solar at all times of the day, without any subsidy, and also provide services to the grid”.

According to report co-author and Powerledger chairman Jemma Green, “for markets that have or are retiring their feed-in tariffs (FiTs), P2P and VPP trading market mechanisms can replace FiT income for households and at the same time facilitate a more stable grid, dealing with the grid problems that renewables can cause”.

In 2021, Carlton & United Breweries worked with energy retailer Diamond Energy and Powerledger to introduce Solar Exchange, a scheme that allowed householders to exchange the credits earned from excess solar generation for beer.

Participants receive a slab of Victoria Bitter for every A$30 in credits – a good deal, given that a slab usually sells for more than A$50.

“Powerledger is delighted that our technology is being used to keep participants informed of their progress in the programme,” said Green. “We are a global pioneer of P2P energy trading, and in a country that thrives on the beer economy, we are excited to be part of Australia’s first P2P energy exchange scheme.”

Confluent offers the most complete edition of Kafka, Jones said, with the scalability, security and high availability needed for important applications. Furthermore, it can be used on-premise, in the cloud, or as a fully managed cloud service on Amazon Web Services, Microsoft Azure, or Google Cloud.

The managed service means customers spend less time on the backend systems, so they can more quickly gain insights into their data.

What Powerledger is doing “is not trivial” due to the combination of internet of things (IoT), network monitoring, blockchain and more, said Jones, suggesting that there are few energy providers that could achieve something similar.

MongoDB is another important part of Powerledger’s infrastructure. The company provided Powerledger with a proof-of-concept showing that the combination of MongoDB and Confluent could handle 100 million electricity meters with the ability to scale even further, said MongoDB regional vice-president Anoop Dhankhar.

Bhandari claimed that the system has been resilient so far, with no data lost when it was tested to the point of failover. He also described the combination of the two products as “very powerful” when it comes to security. And with the need to store data onshore to meet regulations such as the General Data Protection Regulation (GDPR), “the integrity of the data becomes very important”, he added.

Both Confluent and MongoDB are certified to a level that allows their use by security-conscious government agencies, Jones said, so Powerledger did not need to worry about that, and instead focus on its business model and applications.

Dhankhar picked up this theme, noting that even the GDPR’s “right to be forgotten” provision is accommodated by MongoDB, so Powerledger and other users can be confident that they can comply with the rules and regulations applicable in various countries.

The relationship between Confluent and MongoDB extends to reference architectures addressing a range of situations including cloud migration, application modernisation and IoT, said Jones, and the integration provided by the two companies “is a huge benefit” to customers, allowing them to leverage the power of real-time technology.

In Powerledger’s case, it provided not only the scalability required, but also simplified the integration of multiple applications, reducing both time-to-market and operational overheads