Skip to content
2000
Volume 18, Issue 8
  • ISSN: 2352-0965
  • E-ISSN: 2352-0973

Abstract

Background

With the increasing integration of renewable energies into smart microgrids, the parameter space of the microgrid system model has increased dramatically, leading to critical computational challenges in achieving the dispatch strategy.

Objective

Additionally, in remote areas, connecting renewable energy resources to the national grid is unavailable due to the high costs. Consequently, we have introduced microgrids as a viable alternative solution. Microgrids are small-scale distributed power grids that rely on renewable energy sources and generators to balance the load demand. The aim of this paper is to improve the reliability of microgrids by integrating machine learning techniques into their control. This technology ensures that all loads in the microgrid are met through energy trading.

Methods

This algorithm utilizes reinforcement learning as the control mechanism for the trading process. We have established a set of trading rules that facilitate energy trades among three microgrids. We designate one of these three microgrids as the primary microgrid, while the other two function as trading microgrids. Firstly, a deep reinforcement learning algorithm (FH-TD3) is developed as a solution. Subsequently, the performances obtained in the three regions are compared with those derived from traditional algorithms. Finally, the effectiveness of the proposed energy management strategy was validated through simulations.

Results

FH-TD3 has shown superior performance to DDPG in three different microgrid experiments, with the FH-TD3 algorithm consistently exhibiting shorter iteration times compared to the DDPG algorithm.

Conclusion

The research shows that the algorithm realizes the balance of the power without the loss of the power grid, and realizes the profit in the transaction process.

© 2025 Bentham Science Publishers
Loading

Article metrics loading...

/content/journals/raeeng/10.2174/0123520965304961240522075448
2024-06-05
2025-11-15

  • From This Site

    /content/journals/raeeng/10.2174/0123520965304961240522075448
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. VerginadisD KarlisA. Design of a Management Algorithm for Energy Trading in Microgrids.Recent Advan. Electric. Electro. Eng.202013710281040
     [Google Scholar]
  2. ShiT SiX LiZ Research on flexibility evaluation of microgrid system with energy storage.Recent Advan. Electric. Electro. Eng.2021145525534
     [Google Scholar]
  3. SaeedM.H. FangzongW. KalwarB.A. IqbalS. A Review on Microgrids’ Challenges & Perspectives.IEEE Access2021916650216651710.1109/ACCESS.2021.3135083
     [Google Scholar]
  4. IsmailA.F.S. DC Microgrid Planning, Operation, and Control: A Comprehensive Review.IEEE Access20219361543617210.1109/ACCESS.2021.3062840
     [Google Scholar]
  5. EspinaE. LlanosJ. MelladoB.C. DobsonC.R. GómezM.M. SáezD. Distributed Control Strategies for Microgrids: An Overview.IEEE Access2020819341219344810.1109/ACCESS.2020.3032378
     [Google Scholar]
  6. NejabatkhahF. LiY.W. Overview of Power Management Strategies of Hybrid AC/DC Microgrid.IEEE Trans. Power Electron.201530127072708910.1109/TPEL.2014.2384999
     [Google Scholar]
  7. AkhtarI. KirmaniS. JameelM. Reliability Assessment of Power System Considering the Impact of Renewable Energy Sources Integration Into Grid With Advanced Intelligent Strategies.IEEE Access20219324853249710.1109/ACCESS.2021.3060892
     [Google Scholar]
  8. AlamM.S. IsmailA.F.S. SalemA. AbidoM.A. High-Level Penetration of Renewable Energy Sources Into Grid Utility: Challenges and Solutions.IEEE Access2020819027719029910.1109/ACCESS.2020.3031481
     [Google Scholar]
  9. TorresG.F. BordonsC. TobajasJ. CalvoR.R. SantiagoI. GrieuS. Stochastic Optimization of Microgrids With Hybrid Energy Storage Systems for Grid Flexibility Services Considering Energy Forecast Uncertainties.IEEE Trans. Power Syst.20213665537554710.1109/TPWRS.2021.3071867
     [Google Scholar]
  10. ShanY. HuJ. LiZ. GuerreroJ.M. A Model Predictive Control for Renewable Energy Based AC Microgrids Without Any PID Regulators.IEEE Trans. Power Electron.201833119122912610.1109/TPEL.2018.2822314
     [Google Scholar]
  11. LiuZ. WangL. MaL. A Transactive Energy Framework for Coordinated Energy Management of Networked Microgrids With Distributionally Robust Optimization.IEEE Trans. Power Syst.202035139540410.1109/TPWRS.2019.2933180
     [Google Scholar]
  12. HuangY. WangL. GuoW. KangQ. WuQ. Chance Constrained Optimization in a Home Energy Management System.IEEE Trans. Smart Grid20189125226010.1109/TSG.2016.2550031
     [Google Scholar]
  13. CobanH.H. RehmanA. MousaM. Load Frequency Control of Microgrid System by Battery and Pumped-Hydro Energy Storage.Water20221411181810.3390/w14111818
     [Google Scholar]
  14. HosseiniS.M. CarliR. DotoliM. Robust Optimal Energy Management of a Residential Microgrid Under Uncertainties on Demand and Renewable Power Generation.IEEE Trans. Autom. Sci. Eng.202118261863710.1109/TASE.2020.2986269
     [Google Scholar]
  15. FatimaI. AlqahtaniJ. HabibR. AkramM. NazT. AlqahtaniA. AtifM. AlyamiS.S. Enhancing Grid-Connected Microgrid Power Dispatch Efficiency Through Bio-Inspired Optimization Algorithms.IEEE Access202412235782359410.1109/ACCESS.2024.3360340
     [Google Scholar]
  16. RaghavL.P. KumarR.S. RajuD.K. SinghA.R. Optimal Energy Management of Microgrids Using Quantum Teaching Learning Based Algorithm.IEEE Trans. Smart Grid20211264834484210.1109/TSG.2021.3092283
     [Google Scholar]
  17. HuangQ. HuangR. HaoW. TanJ. FanR. HuangZ. Adaptive Power System Emergency Control Using Deep Reinforcement Learning.IEEE Trans. Smart Grid20201121171118210.1109/TSG.2019.2933191
     [Google Scholar]
  18. MocanuE. MocanuD.C. NguyenP.H. LiottaA. WebberM.E. GibescuM. SlootwegJ.G. On-Line Building Energy Optimization Using Deep Reinforcement Learning.IEEE Trans. Smart Grid20191043698370810.1109/TSG.2018.2834219
     [Google Scholar]
  19. ZhouH. AralA. BrandićI. KantarciE.M. Multiagent Bayesian Deep Reinforcement Learning for Microgrid Energy Management Under Communication Failures.IEEE Int. Things J.2022914116851169810.1109/JIOT.2021.3131719
     [Google Scholar]
  20. YanR. WangY. XuY. DaiJ. A Multiagent Quantum Deep Reinforcement Learning Method for Distributed Frequency Control of Islanded Microgrids.IEEE Trans. Control Netw. Syst.2022941622163210.1109/TCNS.2022.3140702
     [Google Scholar]
  21. KhalidJ. RamliM.A.M. KhanM.S. HidayatT. Efficient Load Frequency Control of Renewable Integrated Power System: A Twin Delayed DDPG-Based Deep Reinforcement Learning Approach.IEEE Access202210515615157410.1109/ACCESS.2022.3174625
     [Google Scholar]
  22. ZhangQ. LinM. YangL. T. ChenZ. KhanS. U. LiP. A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling.IEEE Trans. Serv. Comp.201912573974910.1109/TSC.2018.2867482
     [Google Scholar]
  23. BuiY.H. HussainA. KimH.M. Double Deep $Q$ -Learning-Based Distributed Operation of Battery Energy Storage System Considering Uncertainties.IEEE Trans. Smart Grid202011145746910.1109/TSG.2019.2924025
     [Google Scholar]
  24. LvP. WangX. ChengY. DuanZ. Stochastic Double Deep Q-Network.IEEE Access20197794467945410.1109/ACCESS.2019.2922706
     [Google Scholar]
  25. ArwaE.O. FollyK.A. Reinforcement Learning Techniques for Optimal Power Control in Grid-Connected Microgrids: A Comprehensive Review.IEEE Access2020820899220900710.1109/ACCESS.2020.3038735
     [Google Scholar]
  26. WooJ.H. WuL. ParkJ.B. RohJ.H. Real-Time Optimal Power Flow Using Twin Delayed Deep Deterministic Policy Gradient Algorithm.IEEE Access2020821361121361810.1109/ACCESS.2020.3041007
     [Google Scholar]
  27. LiangY. GuoC. DingZ. HuaH. Agent-Based Modeling in Electricity Market Using Deep Deterministic Policy Gradient Algorithm.IEEE Trans. Power Syst.20203564180419210.1109/TPWRS.2020.2999536
     [Google Scholar]
  28. HeT. WuX. DongH. GuoF. YuW. Distributed Optimal Power Scheduling for Microgrid System via Deep Reinforcement Learning with Privacy PreservingIEEE 17th International Conference on Control & Automation (ICCA) Naples, Italy, 2022, pp. 820-825.10.1109/ICCA54724.2022.9831947
     [Google Scholar]
/content/journals/raeeng/10.2174/0123520965304961240522075448
Loading
Multi-microgrid Energy Management Based on Finite Horizon Twin Delayed Deep Deterministic Policy Gradient
Recent Advances in Electrical & Electronic Engineering 18, 1274 (2025); https://doi.org/10.2174/0123520965304961240522075448
/content/journals/raeeng/10.2174/0123520965304961240522075448
/content/journals/raeeng/10.2174/0123520965304961240522075448
Loading

Data & Media loading...


  • Article Type:     Research Article
Keyword(s): energy management; finite horizon twin delayed DDPG; multi-microgrids; Reinforcement learning; scheduling algorithm; system scheduling

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test