load forecasting, big data, artificial intelligence, large-scale, large volume, energy supplier, smart grid
Abstract :
[en] Large-scale short-term load forecasting involves predicting energy consumption across geographic areas or large sets of users. This practice is crucial in power systems, particularly for energy suppliers. The widespread installation of smart metering technology has facilitated the collection of extensive data on user load profiles. By incorporating such granular data, large-scale load forecasting becomes more accurate and reliable, capturing the variability and trends across different consumer segments. However, transforming smart-meter data into effective load forecasting models faces significant challenges. For instance, smart-meter data present issues related to its high volume, variety, and fine-grained temporal resolution. Consequently, different techniques can be considered to mitigate these issues before applying forecasting models to the data. In this paper, we conduct a two-step literature review to provide insights into the data, forecasting approaches, and model evaluation used in large-scale, short-term load forecasting from smart-meter data. We propose classifying the different forecasting approaches into three strategies: integrated, residential-based, and cluster-based. We additionally draw insights on the effect of data volume on forecasting models, performance comparison between forecasting approaches, and the tendency of model complexity in this research domain.
Research center :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > FINATRAX - Digital Financial Services and Cross-organizational Digital Transformations
Disciplines :
Computer science
Author, co-author :
NGUYEN, Quoc Viet ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX
POTENCIANO MENCI, Sergio ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX
DELGADO FERNANDEZ, Joaquin ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX
External co-authors :
no
Language :
English
Title :
Literature review for large-scale load forecasting with large volume of smart-meter data
Petropoulos F, Apiletti D, Assimakopoulos V, Babai MZ, Barrow DK, Taieb SB, Bergmeir C, Bessa RJ, Bijak J, Boylan JE, et al. Forecasting: theory and practice. International Journal of Forecasting 2022; 38:705–871
Kamran M. Chapter 8 - Energy statistics and forecasting for smart grids. Fundamentals of Smart Grid Systems. Ed. by Kamran M. Academic Press, 2023:365–92. DOI: https://doi.org/10.1016/B978-0-323-99560-3.00007-7. Available from: https://www.sciencedirect.com/science/article/pii/B9780323995603000077
Hernandez L, Baladron C, Aguiar JM, Carro B, SanchezEsguevillas AJ, Lloret J, and Massana J. A Survey on Electric Power Demand Forecasting: Future Trends in Smart Grids, Microgrids and Smart Buildings. IEEE Communications Surveys & Tutorials 2014; 16:1460–95. DOI: 10.1109/SURV.2014.032014.00094
Boicea V. Energy Management: Big Data in Power Load Forecasting. 2021 Jun. DOI: 10.1201/9781003147213
Zhang Y, Huang T, and Bompard E. Big data analytics in smart grids: a review. Energy Informatics 2018 Aug; 1. DOI: 10.1186/s42162-018-0007-5
Anand H, Nateghi R, and Alemazkoor N. Bottom-up forecasting: Applications and limitations in load forecasting using smart-meter data. Data-Centric Engineering 2023; 4:e14. DOI: 10.1017/dce.2023.10
Quilumba FL, Lee W, Huang H, Wang DY, and Szabados RL. Using Smart Meter Data to Improve the Accuracy of Intraday Load Forecasting Considering Customer Behavior Similarities. IEEE Transactions on Smart Grid 2015; 6:911–8. Available from: https:// api. semanticscholar. org/CorpusID:440102
Wijaya TK, Vasirani M, Humeau S, and Aberer K. Cluster-based aggregate forecasting for residential electricity demand using smart meter data. 2015 IEEE International Conference on Big Data (Big Data). 2015:879– 87. DOI: 10.1109/BigData.2015.7363836
Nti I, Adekoya A, Nyarko-Boateng O, and Teimah M. Electricity load forecasting: a systematic review. Journal of Electrical Systems and Information Technology 2020 Sep; 7. DOI: 10.1186/s43067-020-00021-8
Mujeeb S, Javaid N, Javaid S, Rafique A, and Ilahi Tamimy M. Big Data Analytics for Load Forecasting in Smart Grids: A Survey. 2019 Feb
Rowley J and Slack F. Conducting a literature review. Management research news 2004; 27:31–9
Paré G, Trudel MC, Jaana M, and Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & management 2015; 52:183– 99
Snyder H. Literature review as a research methodology: An overview and guidelines. Journal of Business Research 2019; 104:333–9. DOI: https:// doi. org/10. 1016/j. jbusres. 2019. 07. 039. Available from: https:// www. sciencedirect. com/science/article/pii/S0148296319304564
Paré G, Trudel MC, Jaana M, and Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management 2015; 52:183– 99. DOI: https:// doi. org/10. 1016/j. im. 2014. 08. 008. Available from: https://www.sciencedirect.com/science/article/pii/S0378720614001116
Aseeri A. Effective RNN-Based Forecasting Methodology Design for Improving Short-Term Power Load Forecasts: Application to Large-Scale Power-Grid Time Series. Journal of Computational Science 2023 Mar; 68:101984. DOI: 10.1016/j.jocs.2023.101984
Mujeeb S, Javaid N, Ilahi M, Wadud Z, Ishmanov F, and Afzal MK. Deep Long Short-Term Memory: A New Price and Load Forecasting Scheme for Big Data in Smart Cities. Sustainability 2019; 11. DOI: 10.3390/su11040987. Available from: https://www.mdpi.com/2071-1050/11/4/987
Harzing AW. Publish or Perish. https://harzing.com/ resources/publish-or-perish. Accessed: 2024-05-20. 2007
Litmap. https://app.litmaps.com/
Zhao H, Li X, and Ren Y. A Big Data Framework for Short-Term Load Forecasting Using Heterogenous Data. 2020 International Conference on Virtual Reality and Intelligent Systems (ICVRIS). 2020:490–4. DOI: 10. 1109/ICVRIS51417.2020.00123
Ali El-Sayed Ali H, Alham MH, and Ibrahim DK. Big data resolving using Apache Spark for load forecasting and demand response in smart grid: a case study of Low Carbon London Project. Journal of Big Data 2024 Apr; 11:59. DOI: 10.1186/s40537-024-00909-6. Available from: https: //doi.org/10.1186/s40537-024-00909-6
Zhang P, wu X, Wang X, and Bi S. Short-Term load forecasting based on big data technologies. CSEE Journal of Power and Energy Systems 2015 Mar; 1:59–67. DOI: 10.17775/CSEEJPES.2015.00036
Qi J, Wang J, Liang X, and Xu Z. User-Level Large Scale Load Forecasting Based on GPU. 2023 9th Annual International Conference on Network and Information Systems for Computers (ICNISC). 2023:282–6. DOI: 10. 1109/ICNISC60562.2023.00017
Alemazkoor N, Tootkaboni M, Nateghi R, and Louhghalam A. Smart-Meter Big Data for Load Forecasting: An Alternative Approach to Clustering. IEEE Access 2022; 10:8377–87. DOI: 10.1109/ACCESS.2022.3142680
Oprea SV and Bâra A. Machine Learning Algorithms for Short-Term Load Forecast in Residential Buildings Using Smart Meters, Sensors and Big Data Solutions. IEEE Access 2019; 7:177874–89. DOI: 10. 1109/ACCESS. 2019. 2958383
Shahzadeh A, Khosravi A, and Nahavandi S. Improving load forecast accuracy by clustering consumers using smart meter data. 2015 International Joint Conference on Neural Networks (IJCNN). 2015:1–7. DOI: 10.1109/IJCNN. 2015.7280393
Humeau S, Wijaya TK, Vasirani M, and Aberer K. Electricity load forecasting for residential customers: Exploiting aggregation and correlation between households. 2013 Sustainable Internet and ICT for Sustainability (SustainIT). 2013:1–6. DOI: 10.1109/SustainIT.2013.6685208
Kong W, Dong ZY, Jia Y, Hill DJ, Xu Y, and Zhang Y. Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network. IEEE Transactions on Smart Grid 2019; 10:841–51. DOI: 10.1109/TSG.2017.2753802
Shi H, Xu M, and Li R. Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN. IEEE Trans actions on Smart Grid 2018; 9:5271–80. DOI: 10. 1109/TSG.2017.2686012
Wang P, Liu B, and Hong T. Electric load forecasting with recency effect: A big data approach. International Journal of Forecasting 2016; 32:585–97. DOI: https://doi. org /10.1016 /j.ijforecast.2015.09.006. Available from: https:// www. sciencedirect. com/science/article/pii/S0169207015001557
Safhi HM, Frikh B, and Ouhbi B. Energy load forecasting in big data context. 2020 5th International Conference on Renewable Energies for Developing Countries (REDEC). 2020:1–6. DOI: 10.1109/REDEC49234.2020.9163901
Irish smart meter data trial CER. https://www.ucd.ie/issda/data/commissionforenergyregulationcer/. 2007
Low Carbon London. https:// data. london. gov. uk/dataset/smartmeter - energy - use - data - in - london - households. 2014
Wen M, Yu Z, Li W, Luo S, Zhong Y, and Changqing C. Short-term load forecasting based on feature mining and deep learning of big data of user electricity consumption. AIP Advances 2023 Dec; 13. DOI: 10.1063/5.0176239
Emami P, Sahu A, and Graf P. BuildingsBench: A LargeScale Dataset of 900K Buildings and Benchmark for ShortTerm Load Forecasting. 2024. arXiv: 2307.00142 [cs.LG]
Bessani M, Massignan J, Santos T, London Jr J, and Maciel C. Multiple households very short-term load forecasting using bayesian networks . Electric Power Systems Research 2020 Aug; 189. DOI: 10. 1016/j. epsr. 2020. 106733
Zhou K, Fu C, and Yang S. Big data driven smart energy management: From big data to big insights. Renewable and Sustainable Energy Reviews 2016; 56:215–25. DOI: https://doi.org/10.1016/j.rser.2015.11.050. Available from: https:// www. sciencedirect. com/science/article/pii/S1364032115013179
Ansari MH, Tabatab Vakili V, and Bahrak B. Evaluation of big data frameworks for analysis of smart grids. Journal of Big Data 2019 Dec; 6:109. DOI: 10.1186/s40537-019- 0270-8. Available from: https://doi.org/10.1186/s40537- 019-0270-8
Syed D, Refaat SS, and Abu-Rub H. Performance Evaluation of Distributed Machine Learning for Load Forecasting in Smart Grids. 2020 Cybernetics & Informatics (K&I). 2020:1–6. DOI: 10.1109/KI48306.2020.9039797
Lei J, Jin T, Hao J, and Li F. Short-term load forecasting with clustering–regression model in distributed cluster. Cluster Computing 2019 Jul; 22. DOI: 10.1007/s10586 - 017-1198-4
Saber AY and Alam AR. Short term load forecasting using multiple linear regression for big data. 2017 IEEE symposium series on computational intelligence (SSCI). IEEE. 2017:1–6
Zainab A, Ghrayeb A, Houchati M, Refaat SS, and Abu-Rub H. Performance Evaluation of Tree-based Models for Big Data Load Forecasting using Randomized Hyperparameter Tuning. 2020 IEEE International Conference on Big Data (Big Data). 2020:5332–9. DOI: 10. 1109/BigData50022.2020.9378423
Tian Y, Sehovac L, and Grolinger K. Similarity-Based Chained Transfer Learning for Energy Forecasting With Big Data. IEEE Access 2019; 7:139895–908. DOI: 10. 1109/ACCESS.2019.2943752
Alhussein M, Khursheed K, and Haider S. Hybrid CNNLSTM Model for Short-Term Individual Household Load Forecasting. IEEE Access 2020 Jan; 8:180544–57. DOI: 10. 1109/ACCESS.2020.3028281
Hertel M, Ott S, Neumann O, Schäfer B, Mikut R, and Hagenmeyer V. Transformer Neural Networks for Building Load Forecasting. 2022 Dec
Lin W, Wu D, and Boulet B. Spatial-Temporal Residential Short-Term Load Forecasting via Graph Neural Networks. IEEE Transactions on Smart Grid 2021; 12:5373– 84. DOI: 10.1109/TSG.2021.3093515
Lin W and Wu D. Residential Electric Load Forecasting via Attentive Transfer of Graph Neural Networks. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. Ed. by Zhou ZH. Main Track. International Joint Conferences on Artificial Intelligence Organization, 2021 Aug:2716–22. DOI: 10.24963/ ijcai.2021/374. Available from: https://doi.org/10.24963/ ijcai.2021/374
Fernandez JD, Menci SP, and Pavic I. Towards a peerto-peer residential short-term load forecasting with federated learning. 2023 IEEE Belgrade PowerTech. 2023:1–6. DOI: 10.1109/PowerTech55446.2023.10202782
Kim HS, Park S, and Kim S. Time-series clustering and forecasting household electricity demand using smart meter data. Energy Reports 2023. Available from: https:// api.semanticscholar.org/CorpusID:257546287
Ming Y, Zhu E, Wang M, Liu Q, Liu X, and Yin J. Scalable k-means for large-scale clustering. Intelligent Data Analysis 2019; 23:825–38
Syed D, Abu-Rub H, Ghrayeb A, Refaat SS, Houchati M, Bouhali O, and Bañales S. Deep Learning-Based ShortTerm Load Forecasting Approach in Smart Grid With Clustering and Consumption Pattern Recognition. IEEE Access 2021; 9:54992–5008. DOI: 10.1109/ACCESS.2021. 3071654
Xu D and Tian Y. A Comprehensive Survey of Clustering Algorithms. Annals of Data Science 2015 Jun; 2:165– 93. DOI: 10. 1007/s40745 - 015 - 0040 - 1. Available from: https://doi.org/10.1007/s40745-015-0040-1
Marinescu A, Harris C, Dusparic I, Clarke S, and Cahill V. Residential electrical demand forecasting in very small scale: An evaluation of forecasting methods. 2013 2nd International Workshop on Software Engineering Challenges for the Smart Grid (SE4SG). 2013:25–32. DOI: 10. 1109/SE4SG.2013.6596108
Makridakis S, Spiliotis E, and Assimakopoulos V. The M4 Competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting 2020; 36. M4 Competition:54–74. DOI: https:// doi. org/10. 1016/j. ijforecast. 2019. 04. 014. Available from: https:// www. sciencedirect. com/science/article/pii/S0169207019301128
Chadoulos S, Koutsopoulos I, and Polyzos GC. One model fits all: Individualized household energy demand forecasting with a single deep learning model. Proceedings of the Twelfth ACM International Conference on Future Energy Systems. e-Energy’21. Virtual Event, Italy: Association for Computing Machinery, 2021:466–74. DOI: 10.1145/3447555.3466587. Available from: https://doi. org/10.1145/3447555.3466587
He Y, Luo F, and Ranzi G. Transferrable ModelAgnostic Meta-learning for Short-Term Household Load Forecasting With Limited Training Data. IEEE Transactions on Power Systems 2022; 37:3177–80. DOI: 10. 1109/TPWRS.2022.3169389
Cini A, Lukovic S, and Alippi C. Cluster-based Aggregate Load Forecasting with Deep Neural Networks. 2020 International Joint Conference on Neural Networks (IJCNN). 2020:1–8. DOI: 10. 1109/IJCNN48605. 2020. 9207503
