P2P Energy Trading through Prospect Theory, Differential Evolution, and Reinforcement Learning

Ashutosh Timilsina, Simone Silvestri

Research output: Contribution to journalArticlepeer-review

Abstract

Peer-to-peer (P2P) energy trading is a decentralized energy market where local energy prosumers act as peers, trading energy among each other. Existing works in this area largely overlook the importance of user behavioral modeling and assume users' sustained active participation and full compliance in the decision-making process. To overcome these unrealistic assumptions, and their deleterious consequences, in this article, we propose an automated P2P energy-trading framework that specifically considers the users' perception by exploiting prospect theory. We formalize an optimization problem that maximizes the buyers' perceived utility while matching energy production and demand. We prove that the problem is NP-hard and we propose a Differential Evolution-based Algorithm for Trading Energy (DEbATE) heuristic. Additionally, we propose two automated pricing solutions to improve the sellers' profit based on reinforcement learning. The first solution, named Pricing mechanism with Q-learning and Risk-sensitivity (PQR), is based on Q-learning. Additionally, given the scalability issues of PQR, we propose a Deep Q-Network-based algorithm called ProDQN that exploits deep learning and a novel loss function rooted in prospect theory. Results based on real traces of energy consumption and production, as well as realistic prospect theory functions, show that our approaches achieve 26% higher perceived value for buyers and generate 7% more reward for sellers, compared to recent state-of-the-art approaches.

Original languageEnglish
Article number3603148
JournalACM Transactions on Evolutionary Learning and Optimization
Volume3
Issue number3
DOIs
StatePublished - Sep 20 2023

Bibliographical note

Publisher Copyright:
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.

Keywords

  • Deep Q-Network
  • Peer-to-peer energy trading
  • Q-learning
  • differential evolution
  • dynamic pricing
  • non-linear optimization
  • prospect theory

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science (miscellaneous)
  • Computer Science Applications
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'P2P Energy Trading through Prospect Theory, Differential Evolution, and Reinforcement Learning'. Together they form a unique fingerprint.

Cite this