Forecasting different dimensions of liquidity in the intraday electricity markets: A review

Abbreviations: ANN: Artificial Neural Network; AQL: Adjusted Quantile Lines; ARIMAX: Autoregressive Integrated Moving Average model with Exogenous inputs; ARMA: Autoregressive Moving Average; ARX: Autoregressive with eXternal Model Input; BRP: Balance responsible party; BSP: Balance service providers; CET: Central European time; CSP: Congestion service providers; CO2: Carbon dioxide; DCGAN: Deep convolutional generative adversarial networks; EPEX: European power exchange; EPIAS: Energy exchange Istanbul; ESN: Echo state network; EXAA: Energy exchange Austria; FANN: Feed-forward artificial neural network; GAM: Generalized additive model; GAMLSS: Generalized additive models for location scale and shape; GARCH: Generalized autoregressive conditional heteroskedasticity; GIR: Generalized impulse response; GRU: Gated recurrent units; HJB: Hamilton-Jacobi-Bellman; LASSO: Least absolute shrinkage and selection operator; LOB: Limit order book; LSTM: Long-short term memory; LQC: LASSO, quantile regression and a copula-modeled temporal structure; MAPE: Mean absolute percentage error; MDP: Markov decision process; MIBEL: Iberian electricity market; MLP: Multi-layer perceptron; NEM: National electricity market; NUTS: No-U-turn sampler; OMIE: OMI, Polo Espanol S.A.; OLS: Ordinary ˜ least squares; PCA: Principal component analysis; PDF: Probability distribution function; PJM: Pennsylvania, New Jersey, and Maryland; RES: Renewable energy sources; RMSE: Root mean square error; RNN: Recurrent neural network; SVM: Support vector machine; SVR: Support vector regression; TGE: Polish power exchange; TSO: Transmission system operators; VAR: Vector autoregressive process; VWAP: Volume-weighted average price; WLS: Weighted least squares; XBID: Cross border intraday project; XG-Boost: Extreme gradient boosting

1.   Introduction

Electricity is an important commodity, but storing it in large quantities at grid level is costly. Generation and consumption must always be balanced to ensure a stable operation of the energy systems, avoid outages, manage supply and demand fluctuations, and other issues^{[1,2,3,4,5,6]}. This balance between supply and demand is ensured through a chain of electricity trading markets ^[5]. Traditionally, electricity markets were controlled by government agencies. Liberalization of electricity markets in Europe commenced in the early 1990s^[3]. Electricity markets are specific to individual sovereign states, with each country regulating its own market. However, in some regions, neighboring countries may share and integrate their electricity markets through cross-border trading and interconnections, and by promoting regional energy cooperation. Liberalization attracted more participants such as producers, traders, and transmission system operators (TSOs) to the markets, leading to competition in the generation, supply, transmission, and distribution of electric energy ^[4]. Initially, electric energy production was mostly dependent on core sources like natural gas, oil, nuclear energy, and coal among others. In recent years, non-programmable renewable energy sources (RESs) have been utilized by many energy producers due to the rising price of fossil fuels, which have helped them to reduce the overall energy production cost as well as carbon emissions ^[7]. The prioritization of renewables over conventional power plants, driven by their minimal marginal cost, has resulted in participants adapting their business models accordingly^[8]. This shift has led to the emergence of new electricity trading markets. Wind and solar energy are the predominant RES sought after in electricity markets. Their zero marginal cost, combined with subsidies and incentives, has made producers actively participate in these markets, aiming to maximize their profit^[7,8,9,10]. Renewable energy participants must behave like any other market participant, involving a commitment to a specific production level within the designated settlement period just as with conventional energy producers ^[11].

The introduction of distributed renewable generation caused an increase in the uncertainty of future supply, and prices. Electricity demand is influenced by weather conditions such as temperature, peak activity in commercial buildings, and fluctuation in the daily activities of the population. This creates the price dynamics. To ensure the balance between generation and consumption, participants forecast the production and demand before and/or during the day of settlement by using forecasting methods ^[11,12]. This forecast results in increased uncertainty in electricity prices due to the growing share of renewable resources in markets, as well as variations in weather conditions and people's behavior. Consequently, it calls for the development of new prediction strategies. A review of techniques such as statistical, machine learning, and ensemble/hybrid methods has been carried out for different intraday electricity markets. This review primarily focuses on the European electricity markets, with some inclusion of markets from other countries.

1.1.   Background of electricity market

Electricity markets involve a wide array of participants, as shown in Figure 1, who play crucial roles in the functioning and dynamics of the market. The roles of these participants fall into the physical, administrative or market domain. The physical domain deals with the electricity production, consumption, transmission and security of the system. Electricity producers, consumers (small and large) and TSOs are the main actors in the physical domain. Producers generate electrical energy from wind, solar power, water, etc., and to a notable extent from natural gas and coal. The high-voltage electricity grid is operated by a TSO. A TSO is a regulated entity responsible for the operation, control, and maintenance of the high-voltage transmission network. The TSO ensures the reliable and secure transmission of electricity across the grid. It also includes manages and monitors the balance of supply and demand in the grid. The administrative domain includes actors who manage the relationships between customers and the market or grid operators. For example, they monitor actual consumption and production and arrange the invoicing for their customers. There are different parties involved in this, such as electricity suppliers, traders, balancing responsible party (BRP), balancing service providers (BSPs) and congestion service providers (CSPs). Electricity suppliers conclude delivery contracts with consumers and invoice them for all costs related to their electricity bills. To supply their consumers, they purchase electricity on the wholesale market (forward futures, day-ahead and intraday).

BRPs are market participants, such as electricity suppliers or large consumers, who have contractual responsibilities to maintain the balance between their contracted electricity positions and actual consumption or generation. They are accountable for ensuring that their injections or withdrawals of electricity align with their contractual obligations. BSPs are entities that offer services to support BRPs in balancing their positions effectively. A BSP provides tools, technologies, and expertise to help BRPs manage their imbalances and maintain compliance with their contractual obligations. A BSP may offer services like load forecasting, portfolio optimization, real-time monitoring, and energy trading platforms. CSPs are market participants that focus on managing congestion within the electricity transmission network. Congestion occurs when there is limited transmission capacity to accommodate the flow of electricity between different areas or zones. A CSP provides services to alleviate congestion, optimize the utilization of transmission infrastructure, and ensure the efficient and secure transfer of electricity. They may offer services such as congestion management, transmission capacity allocation, congestion forecasting, or transmission system optimization. Together, the BRP, BSP and CSP collaborate to ensure the stability of the electricity market, manage imbalances, and optimize the utilization of transmission infrastructure, ultimately contributing to the reliable and secure delivery of electricity to consumers, and the TSO oversees the overall operation and coordination of market participants to ensure reliable grid operation and facilitate efficient electricity trading. The market domain lists the different market platforms needed for all these transactions. It includes retail market, wholesale market, balancing market, imbalance settlement, and congestion management services. Different marketplaces are available for electricity trading. Mainly, the electricity delivery time frame and form of transaction characterize these marketplaces.

The retail market is the part of the electricity market where electricity is sold directly to end consumers. It involves the interaction between electricity retailers (suppliers) and individual households, businesses or other electricity consumers. Retail market participants offer various pricing plans and services to meet consumer demand. The wholesale market is where electricity is bought and sold in bulk between electricity generators (producers) and electricity retailers, as well as other market participants such as industrial consumers or electricity traders. It operates on a larger scale than the retail market and involves the trading of electricity in larger quantities. The balancing market is responsible for maintaining the balance between electricity supply and demand in real time. It deals with any deviations or imbalances that occur between scheduled or contracted electricity supply and actual consumption. Market participants, such as grid operators or BSPs, offer balancing services to adjust supply or demand in response to fluctuations and ensure system stability. The spot market, also known as the electricity spot market or day-ahead market, is where electricity is traded for immediate or near-future delivery. It involves the buying and selling of electricity for specific time periods, usually ranging from a few hours to a day ahead. Spot market prices are typically determined through market mechanisms such as auctions or continuous trading, based on supply and demand dynamics. The connection between these markets lies in the overall process of electricity generation, transmission, and consumption. Electricity is generated in the wholesale market, where producers sell it to retailers and other market participants. The retailers, in turn, sell electricity to end consumers in the retail market. The balancing market ensures that supply and demand are balanced in real time, while the spot market facilitates the trading of electricity for immediate delivery or near-future time periods. Electricity trading is regulated through long-term agreements and short-term trading for future energy requirements. Long-term agreements involve over-the-counter trading and power purchase agreements. In over-the-counter trading, the electricity is traded by direct negotiation between two parties considering the required prices and volumes, whereas in power purchase agreements, the participants, i.e., producers and consumers, sign an agreement to earn long-term income for long-term electricity provision.

Short-term electricity spot markets exhibit variations in transparency across different regions and jurisdictions. This includes the day-ahead market and intraday market which trade for either the next day or the current day, respectively. Market operators typically provide publicly available information on market prices, volumes, and other relevant market data. This allows market participants and stakeholders to access and analyze market information, assisting utility companies or BRPs in making timely electricity transactions to meet demand and address forecast deviations in production and consumption. Thus, the short-term electricity markets minimize the risks to participants via cost-effective electricity production. These markets are organized sequentially, encompassing day-ahead, intraday, and balancing markets, each serving distinct objectives and operations to satisfy the demand and supply at each time.

1.2.   Day-ahead market

The day-ahead market is responsible for electricity trading based on the settlement period to be delivered after a day. Electricity for the next day is traded on day-ahead auctions for a dedicated hour or quarter-hour interval as blocks. Figure 2 shows the time framework to forecast for the day-ahead market that has been used to plan the market price forecast in the day-ahead energy markets. In this, BRP submits the expected production or consumption in each hour for the following day to the TSO ^[13].

The transactions in the day-ahead market occur on a daily basis; it permits the market participants to buy and sell the energy for the next day and to form a balance between the production and/or the consumption needs and their contract-based commitments a day before. The day-ahead market works in such a way that the demand and supply of electricity on an hourly basis are initially ensured by sorting all of the available offers and bids, forming the supply and demand curves respectively. Based on that, an average consumption amount is decided that is required to be produced by the generating companies in a given time frame. Any deviations from the projected production are handled by the intraday and balancing markets ^[11,18]. Usually, the day-ahead market gets closed at noon each day.

1.3.   Intraday market

To self-balance the deviations that occurred in the day-ahead electricity production, market participants depend on the intraday electricity markets. The intraday markets involve activities required to be performed on the day of delivery by continuing the trading until the moment of delivery ^[19]. The main objective of the intraday market is to help market participants balance the contract-based electricity commitments to avoid unnecessary charges. Trading in the intraday market is useful because balancing services provided by the TSO are more expensive than the use of a self-balancing mechanism through the intraday market. In addition to that, the TSO can penalize the market participants for frequent imbalances in energy production as well as lead to a breach of contract ^[19]. This market does not only create energy trading opportunities for market participants but it also provides a balanced system before handling the net balancing by the TSO to reduce the energy imbalance ^[20]. Considering the case of the German market, the intraday trading of hourly products starts at 3 PM every day i.e. three hours after the closure of day-ahead trading, and continues up to 5 minutes prior to delivery of each product^[19,21,57].

Even though the intraday markets hold less importance than the day-ahead markets due to lower trading and hedging activities, they still plays a vital role in ensuring system stability. Even a small contribution from the intraday market has the potential to reduce the need for the activation of electricity reserves and control energy by the TSO, reinforcing grid stability. Imbalances in the electricity market lead to charges imposed on individual market participants. As a result, market participants strive to minimize the use of control energy for two key reasons. First, utilizing balancing services is more costly than self-balancing through the intraday market. Second, the TSO may apply sanctions to market participants who consistently generate imbalances, potentially jeopardizing their balancing contracts^[5]. Post-closure of the intraday market balancing energy is used to fill the gap between the generation and consumption. The intraday electricity market is beneficial to market participants as it provides flexibility in portfolio management, risk mitigation, and the ability to manage imbalances. It enables participants to optimize their positions, respond to real-time market conditions, and capitalize on favorable pricing opportunities, ultimately enhancing profitability and supporting the integration of renewable energy^[5,19].

1.4.   Balancing market

This market balances the scheduled deviations or imbalance volume in the electricity market. The scheduled deviation is the difference between the planned net electrical energy exchange with the grid and the actual exchange measure in real time ^[14,15]. It carries out three important activities: balance planning, balancing service provision and imbalance settlement. For balance planning, the BRP submits day-ahead energy schedules to the TSO. However, the stochastic nature of RES production and random electricity demand/consumption is dependent on people's behavior, which leads to deviations in scheduled generation and to system imbalance, which is the net sum of the imbalance volumes of all BRPs. A positive imbalance volume means that the system is short where the actual electricity supply falls short of the scheduled or contracted supply relative to the demand, and a negative imbalance means that the system is long where the actual electricity supply exceeds the scheduled or contracted supply relative to the demand ^[13]. In response to that, balancing service provision, which is normally activated by the TSO, helps to obtain the required amount of electricity in real time to balance the system ^[22,23]. Finally, an imbalance settlement is carried out. An imbalance settlement is the financial settlement of the scheduled deviations or imbalance volume, whereby the BRP needs to pay the imbalance price to the TSO ^[14,15]. The TSO applies some pre-defined rules to calculate the imbalance price by using the total balancing cost and total system imbalance for a specific unit of time like a 15-minute period. There are two imbalance price mechanisms; one is applied for energy surplus/oversupply and the other for energy shortage/undersupply. A BRP with a surplus receives the long imbalance price the reduce the system imbalance from the TSO and with a shortage needs to pay the short imbalance price to the TSO ^{[13,14,15,23]}. For example, with a perfect forecast, when all volumes are perfectly sold on the day-ahead market, the earnings would be 27.51 €/MWh. But, the imbalances reduce the earnings by 2.48 to 25.03 €/MWh ^[13]. Balancing the market and especially the imbalanced prices is of interest for spotting market participants.

Some of the specific intraday market studies that have been conducted on a global level include the German and Nord Pool markets. Table 1 and Figure 3 show different intraday markets. The differing day-ahead closure times and intraday open times in electricity markets across countries or regions arise from a combination of factors, including market design, demand patterns, regulatory frameworks, and the integration of renewable energy. These variations are strategically set to align with local demand fluctuations, ensure efficient trading, and adhere to regulatory guidelines. The increasing incorporation of renewables introduces an additional layer of complexity, necessitating flexible scheduling mechanisms to accommodate intermittent generation. Ultimately, the diverse factors at play drive the customization of market timings to optimize energy supply, trading, and grid stability.

1.5.   Importance of intraday market forecasting

Forecasting plays an important role in energy markets to maximize profit and optimize decision-making ^[4,17,24]. In order to make the decision for energy companies at the corporate level, these forecasts become the fundamental inputs. In the short-term electricity market, portfolio managers use price forecasts with a reasonable level of accuracy to adjust the bidding strategy reduce the risk and increase profit by managing their production and consumption. Participants have to decide how much energy is required for a bid. The decision for this amount is made based on imperfect knowledge of the generation and consumption of energy in the future, in the given amount of time before the actual delivery of the energy, by using the forecasting strategies based on available past data of energy production ^[25]. Production forecasts have become a dominant part of renewable energy production over the last two decades. During the initial days, these forecasts were used to plan production but, with market liberalization, they are also increasingly being used when the energy is being sold to avoid imbalance prices ^[2].

Intraday demand forecasting helps in the efficient dispatch of generation, renewable integration, and frequency control. An accurate short-term demand forecast minimizes the generation cost by communicating dispatch instructions to utilities. It also prevents over- and under-production and hence minimizes the control frequency rate. Traders use this forecast to calculate future electricity prices, and on that basis, a profitable generator can be selected. As the forecast is dependent on different factors such as weather conditions on that particular day, people's behavior, and unavoidable events like plant outages, it leads to errors in the actual and the forecasted energy production and demand.

To balance the gap in production and consumption, minimize the imbalance in prices, and gain high profits, BRPs and traders depend on intraday forecasting ^[21]. As the lead-time of the forecast determines the efficiency of the forecast, intraday forecasting is more accurate than day-ahead as it considers the updates that are available in time closer to the actual delivery of the energy. The difference between the previously forecasted day-ahead profile and the more precise intraday forecast is called the forecast error. Errors in RES forecasting have become an important source of liquidity which influences the prices. Wind forecasting errors have been found to have a quantified impact of 2–3 €/MWh per GWh of error, indicating a significant influence on electricity prices. On the other hand, solar forecasting errors exhibit a varying impact depending on the direction of the error. Positive forecasting errors result in a quantified impact of 2 €/MWh per GWh, while negative forecasting errors have a smaller impact at less than 1 €/MWh per GWh. These quantifications highlight the importance of accurate wind and solar forecasts in managing electricity pricing and optimizing renewable energy integration^[8]. When the forecast changes from a day-ahead forecast to an intraday forecast, the error gets reduced significantly ^[18,25,26]. Market participants can trade the quantity difference between day-ahead and intraday forecasts to self-balance this gap ^[5]. Along with the improvement of the forecasting, the deviation and demand for the balancing energy also go on decreasing, and hence it has resulted in the establishment of intraday markets in European countries to allow adjustment ^[18]. The forecasting methods still have limited accuracy and it depends on the accuracy of the prediction tool and previous data.

2.   Materials and method

2.1.   Intraday market forecasting techniques

The primary objective of this paper is to provide an overview of different techniques used in the literature for forecasting intraday electricity prices and volume, i.e., the number of trades, and price volatility in intraday electricity markets. The forecasting methodologies are categorized into three main models: statistical/econometric, machine learning, and ensemble/hybrid methods, based on various forecasting parameters. This section will provide detailed explanations of all of the methodologies employed for intraday electricity forecasting.

Overview of modeling approaches:

● Statistical or econometric approaches, which involve direct applications of statistical techniques based on the available historical data to predict future values.

● Machine learning techniques, which combine elements of learning, evolution, and fuzziness to create approaches that are capable of adapting to complex dynamic systems ^[27,28].

● Ensemble techniques, which involve creating and then combining multiple models to obtain improved results and hybrid methods, which combine different approaches or techniques to create a more comprehensive and effective model.

Firstly, the literature review on statistical/econometric methods is presented, followed by machine learning and ensemble/hybrid techniques in the second and third sections, respectively. Table 2 presents an overview of the various forecasting techniques employed in the context of intraday electricity forecasting.

2.2.   Statistical/Econometric approaches

Traditionally, statistical approaches have been applied for the forecasting of electricity. Such models are useful for forecasting different parameters or their volatility based on linear and stationary datasets ^[65]. A statistical or econometric model uses a mathematical combination of past values of prices, and/or past or present values of exogenous variables such as weather forecasts, consumption and production figures, renewable energy forecasts, etc. Typically, they work on approaches that include the addition or multiplication of the previous data. If the predicted variable is the sum of several components, the approach can be considered as additive and if the approach contains multiplication of several factors, then it is multiplicative. The most popular among them is the additive approach. Statistical models are attractive because they can be argued to be based on physical models and thus provide insight into their behavior which is helpful for engineers and TSOs in the development of new models. However, they are often criticized for their limited ability to model the nonlinear behavior of parameters and related fundamental variables ^[28]. These models are not able to capture the non-linearity and complexity present in them. It results in the accuracy of such models being lower than those of the machine learning models and ensemble/hybrid models ^[65].

2.2.1.   Prices

An ARIMAX-based experiment was carried out on the European Power Exchange (EPEX) continuous intraday market ^[29]. In this, continuous intraday prices were taken as endogenous parameters, and 15-minute auction prices, the actual total load, and the onshore wind in-feed were included in the exogenous parameters list to observe their effects on the price level and its variability. The model framework has significantly facilitated understanding of the behavior of driving factors of the intraday market prices. In the future, more exogenous variables can be employed to understand the correlation of the model and variables along with its tuning of the internal parameters of the model.

Kulakov and Ziel ^[30] designed novel auction-curves-based econometric models such as a simple linear naïve model, a non-linear model, and a combination model to forecast the intraday electricity prices to examine the effects of errors in solar and wind energy forecasts on the intraday prices. The error in renewable forecasts was taken as the magnitude to shift the day-ahead supply curve to obtain an approximation of the intraday supply curve and the intraday price was calculated from the intersection of the approximated supply curve and demand curve as shown in Figure 4. Assuming the forecast error of 2500 MW, even though the shapes and the distances between the day-ahead and intraday curves were identical on both sides, the only difference between them was the realized demand size i.e. low demand in Figure 4(a) and high demand in Figure 4(b). The results of this study indicated that the impact of forecast error and its volatility on intraday prices are nonlinear. Furthermore, the approximated demand curve can be employed to check for model performance, and the use of intraday prices a few hours before delivery can provide better results.

An econometric model was developed to predict the price of electricity in 15-minute contracts over the course of a day, using the high-frequency intraday data, the fundamental supply and demand data, and the forecasts of solar and wind power generation ^[32]. The model was then refined to include the slope of the merit order curve, the changes in neighboring 15-minute contracts and the 15-minute intraday auction price. As shown in Figure 5a, the merit order is a ranking of available sources of energy in ascending order of their short-run marginal costs of production ^[66]. The merit order curve with renewable power in-feed shifts the curve to the right; as a consequence, if demand is low, the electricity price decreases by a small amount; however, if demand is high, the electricity price decreases as well by a much larger amount. The results demonstrated a statistically significant and consistently negative trend in autoregressive price changes, regardless of the time of day. Moreover, changes in neighboring contracts have a positive impact on the intraday prices for a given contract. A threshold regression model was used for calibration to study the dependence of the merit order curve slope on intraday prices. It was concluded that negative and positive renewable forecasts affect intraday prices asymmetrically i.e. prices are more affected in high-demand regimes as shown in Figure 5b.

Glas et al. ^[33] modeled an optimal hourly trading and energy generation strategy by using a second-order Hamilton-Jacobi-Bellman equation and found the approximate solution for the German-Austrian continuous intraday market. The model has incorporated wind energy, conventional units, prices, pay off and value functions like the half-spread as explanatory parameters. Results indicated that the absolute trading rate value increases to nearly 15 hours as half-spread and immediate price impact are minimal in that area; the results also indicated that, when the execution price goes below the cheapest power plant marginal cost, it is better for the agent to buy electricity and reduce production.

Hu et al. ^[34] developed a multivariate and dynamic model known as a vector autoregression (VAR) model to analyze the response of fundamental variables and investigate the imbalance caused by prices in the Swedish intraday market. The important variable was intraday price premia (i.e., the difference between the intraday electricity price and the day-ahead price), whereas intraday flow, transmission congestion, forced outages of nuclear power plants, load forecast error, error in the wind forecast, and non-wind forecast error were used as the exogenous variables. The findings of this study indicated that wind power forecast errors, non-wind power forecast errors and load forecast errors were significant to explain the spread in day-ahead and intraday prices. The addition of the congestion variable resulted in robust forecast performance for the model consisting of intraday flow and non-wind power generation and consumption as variables. Moreover, the wind forecast errors have shown a negative impact on intraday prices, and no influence was shown by unplanned (forced) outages. Further work needs to be done to understand intraday market functioning through microstructure analysis.

A Markov Decision Process (MDP) was used to formulate the bidding problem for the Spanish intraday market MIBEL by considering the random electricity prices and uncertain wind production forecasts ^[35]. The problem has been optimized with a multi-stage stochastic dual dynamic programming algorithm. It has been observed that the model beats the conservative spot-only trading strategy, a deterministic planner, and an industry-developed solution. Modeling complex bidding functions and improved wind production forecasts increased the value of intraday trading. In the future, the model can be improved with the inclusion of more frequent data on wind production forecasts.

A principal component analysis (PCA) method to average point forecasts was designed to automatically aggregate the prediction information of models with different calibration windows ^[50]. To test the proposed methodology, the authors utilized multiple datasets obtained from the German market. The day-ahead and intraday prices were predicted based on the inputs such as the day-ahead and volume-weighted average price (VWAP) of the last 15 minutes before forecasting, whereas day-ahead prognosis, wind and solar generation forecasts were included as exogenous variables. Results indicated that the use of the best set of calibration windows has outperformed the averaged windows and weighted averaged windows averaging strategies.

Uniejewski et al. ^[49] presented a method to select explanatory variables and forecast very short prices in the German intraday market. This method used the LASSO to select the important variables to forecast intraday prices. It was found that the recent intraday price and day-ahead price that corresponds to the same hour were the best explanatory variables. The LASSO-estimated model with specific tuning parameters outperformed the naïve benchmark model. The study revealed that parsimonious autoregressive–exogenous-type models can perform well.

A linear regression model with LASSO estimation was employed to quantify the effects of wind and solar generation forecast errors on the hourly electricity prices ^[61]. The study utilized the LASSO estimation technique and the Hannan-Quinn criterion to determine the number of non-zero parameters for estimation. The methodology was applied to the German intraday market and encompasses the interdependence between the German-Austrian EPEX and the Energy Exchange Austria (EXAA) day-ahead market. The model explained the seasonal and autoregressive effects of the day-ahead prices on the intraday prices. The investigation revealed that the model exhibited superior performance when analyzing the time-varying and asymmetric renewable forecast errors. Additionally, it was discovered that the intraday prices remained unaffected by positive or negative forecasts of renewable generation. A linear regression model employing LASSO regularization was introduced to forecast the intraday electricity prices.

A linear regression model using LASSO regularization was presented to forecast the intraday electricity price in the German intraday market^[63]. The VWAP index prediction is made four hours before delivery by using various LASSO-estimated models that are developed by considering variables such as past intraday prices, day-ahead prices, partial intraday prices and exogenous variables (historical and predicted demand, generation, and weather). Finally, forecast averaging of LASSO and naïve forecast models is carried out to obtain more precise and robust predictions. The numerical results showed high accuracy compared to naïve benchmark models in terms of point forecasting, still, further research can be made in the area o forecasting the full descriptive trajectories along with using more machine learning-based algorithms.

Narajewski and Ziel ^[62] performed probabilistic forecasting of hourly prices by simulating the trajectories for each trading window in the intraday market to receive a realistic ensemble to allow for more efficient intraday trading. Assuming that the price difference in the German intraday market follows the mixture of various distributions such as Dirac and the Student's t-distributions, a generalized additive model (GAM) was used to fit them. Logistic regression was used to estimate the mixing term. The value and volatility were modeled by using the autoregressive and no-trade effects of load and forecast of wind and solar production, and by accounting for the non-linearity in e.g. time to maturity. The comparisons with the benchmark models showed that the different versions of probabilistic forecasting models performed better for the last 3 hours of trading. The study has shown that the inclusion of the cross-border intraday project (XBID) can reduce market volatility. Future research should concentrate on adding the traded volume or price of nearby hours as repressors in the forecasting model.

Serafin et al. ^[57] introduced a profitable trading strategy for generating the short-term path forecast of the German intraday electricity prices. A novel forecasting framework was developed to determine the prediction bands by using a large number of path forecasts or by using probabilistic price forecasts to find approximate bands. A time-dependent price threshold on time was used for trading activities, and when exceeded it showed a strong opportunity to sell electricity. In addition to that, six different methods to generate prediction bands were analyzed, namely the LQC, adjusted quantile lines (AQL), student's t-distribution, similar-day method, LASSO bootstrap, and LASSO point. A case study on the German intraday market revealed that in terms of energy score the path forecasts showed better performance than the two benchmark models i.e., the similar-day method and student's t-distribution method. The result of the study revealed that the proposed LQC approach with increased computational burden ensured higher trading profits, while the less complex AQL method based on a probabilistic forecast offered a reasonable trade-off but generated less profit than LQC, also, LASSO bootstrap was the best performer. In the future, path forecasts can be improved by using a more realistic temporal dependence structure.

2.2.2.   Volume

Martin and Otterson ^[41] presented a limit order book (LOB) approach to model the German intraday market through the simulation of historic data for optimal electricity trading for any moment in the trading window for any delivery hour. The number of transactions, VWAP, and traded volume data from the German intraday market were considered as the information to build the simulation model. The results showed a substantial improvement in performance relative to those obtained based on the historic transactions data via the mean and median error method.

Favetto ^[44] shed light on the presence of a self-excitement phenomenon in the European intraday electricity market, raising the question of understanding its underlying causes. Possible explanations may involve economic factors, such as price convergence due to adjustments in supply and demand at the end of the trading period, or technical implications related to fine adjustments in electricity production. Additionally, the research primarily focused on analyzing trade dates, especially under the condition of considering self-excitation as a potential source of endogeneity. A future perspective involves examining the pricing process in the European market while considering exogenous variables and seasonality.

Kath and Ziel ^[38] derived a model for electricity trading to forecast optimal order execution in the German intraday market. The variables such as order book depth, time to delivery and different trading regimes like XBID trading were considered for optimization purposes. Two GAM models were modeled to fit the minute-based data to evaluate the impact of the temporary and permanent markets. The evidence from this study suggested that the relationships between spreads and lead time, trading volumes, and times of trading can be effectively modeled by penalized splines. It has also been found that bid-ask spread levels as shown in Figure 6 notably increase one hour before delivery when the XBID orders stop. A comparison of a GAM with the benchmark models explained that complex computation serves better to lower spread but only for trading nearly 90 minutes before delivery; however, a large trade in that phase causes more GAM calculation which results in non-optimal trading. The authors stated that consideration of only the VWAP can be problematic and that optical execution should hence be considered for the study. Quarter-hourly trading can be analyzed and modeled to get further insights into the intraday market.

Narajewski and Ziel ^[39] presented a method utilizing a rolling window forecasting strategy to estimate and simulate transaction arrivals in the German continuous intraday market. Assuming different distributions of inter-arrival times from 3 hours to 30 minutes before the delivery such as exponential, gamma, generalized gamma, and generalized F-distributions maximum likelihood estimation of the distributions was carried out by using some of the time-dependent coefficients. Using the estimation, new trajectories were simulated and their performance was compared with the benchmark models. The results showed that in the central part of the distribution of the transaction arrivals exponential and gamma distributions with exponential rate function performed better forecasting, while best forecasts were obtained from the Generalized Gamma model with quadratic rate function and exponential shape function. Future research can incorporate more complex probability distribution parameter functions for inter-arrival times.

Narajewski ^[39] utilized a simple intensity estimation and forecasting method to predict the hourly transaction arrival during the last 4 hours of trading for the German intraday electricity market. The intensity function was a maximum likelihood function. Four trajectories of simulation for four different delivery times were compared with the actual observation of transactions arrival, and the evidence from this study suggested that these arrivals follow the exponential distribution function. The study revealed that forecasting performance was superior for products having a high transaction count. Other intensity functions, as well as distribution, can serve as a base for future studies.

Coskun and Korn ^[40] utilized a Jacobi process to model the electricity demand in the German intraday market. A study was performed to compare the performance of the Jacobi process to model the demand with the Ornstein-Uhlenbeck process and the Cox-Ingersoll-Ross process. Simulation results for all three models have shown that the Jacobi process model outperformed the Ornstein-Uhlenbeck and Cox-Ingersoll-Ross models as the stationary distribution of the Jacobi process models the following beta distribution. Jacobi-I and Jacobi-II models were evaluated with a different upper bound and lower bounds; thus, the bounding setting for demand has become an attractive feature for the model, but it led to challenges in the forecasting and estimating of the parameters of the Jacobi process. Further studies may include price-sensitive customers as a parameter for supply-demand variation forecasting, and also for forecasting energy production and prices.

Various point process models have been explored to quantify the influence of self-excited jumps and the exogenous factors on the intensity of order arrival during the last 3 hours in a continuous intraday market ^[42]. An exogenous model was formed which considered only external factor processes such as the generation of solar and wind power, and the activated volumes in the balancing market; on the other hand, a pure self-exciting model was formed by considering the fundamental factors only, and a combination of both models resulted in the formation of a full model. The performance of the full model was evaluated with each of the individual models, and it was concluded that the exogenous model was not sufficient to capture the dynamics in the intraday markets; hence, the inclusion of the self-exciting factors is necessary.

Ordinary least squares (OLS) regression, quantile regression, and the autoregressive moving average were employed to model the supply-demand imbalance and the market prices in the intraday market using quarter-hourly data ^[20]. The study has demonstrated that wind and solar energy forecast error increases the supply and demand imbalance, and that large imbalances affect the prices. Furthermore, the prices were more affected by wind forecast errors than by photovoltaic forecast ones. The regulatory intervention to provide timely wind and solar forecasts can improve the accuracy of the forecast.

2.2.3.   Price volatility

Pape et al. ^[5] proposed a model to estimate the hourly electricity price based on aggregated supply and demand information for day-ahead and intraday markets. The forecasting models utilized data from the German electricity market, such as the load, demand, spot prices, coal prices, carbon dioxide (CO$_2$), gas prices, wind, and solar feed-in, cross-border flow, etc. A detailed discussion of fundamental and regression models was held. The results suggest that the model performs better when market peculiarities are considered. The intraday price forecast can be improved with the inclusion of day-ahead price information. In addition, the authors found that the start-up cost of power plants leads to the increase of day-ahead and intraday prices. The market state also affects the prices, as an increase in supply will result in a price drop, and a lack of supply will lead to a substantial price increase. Finally, other promising areas of research would be finding advanced approaches to deal with inefficiencies and improve the modeling of intraday market prices.

A Nash equilibrium model was developed for a finite number of agents as well as in the asymptotic framework of mean-field games to predict market prices and identify the effects of the optimal activity of market participants on prices concerning demand and generation forecasts ^[36]. To determine the price, the authors utilized the LOB data sourced from the German intraday market EPEX. The empirical results revealed that the volatility in price increases near delivery time, and that market price exhibits a negative correlation with renewable forecasts. Moreover, the model showed that the price volatility decreases with market depth and increases for low trade cost, and that huge competition limits the profit.

Maciejowska et al. ^[31] experimented with an ARX and probit model to forecast the price spread between day-ahead and intraday prices to maximize the economic benefits for German and Polish markets. For the Polish market, the endogenous parameters were the day-ahead prices and the balancing prices, while the exogenous ones were the forecasted demand, the forecasted wind generation, and the forecasted reserves. Day-ahead prices and intraday prices were considered as endogenous for the German market and load forecast, and the forecast of wind generation was exogenous. The linear ARX model was useful to link the past endogenous and exogenous variables with current values. On the other hand, the probit model effectively describes the probability distribution of the binary variable. Numerous model specifications were employed to analyze the accuracy, such as the data aggregation level, calibration window length, lag structure, etc., and it has been found that more information does not increase accuracy and profits. Weekly seasonality consideration was reasonable, and a short calibration window was beneficial to capture the non-linear behavior of variables. The results showed that day-ahead and intraday/balancing prices depend mostly on the fundamental variables.

Gürtler and Paulsen ^[8] presented a regression model for day-ahead and intraday price forecasting under the influence of the wind and solar generation forecasts by using the day-ahead and intraday data. The model was designed to simulate a variable that relates electricity generation technology with price determination and to capture the non-linear behavior of price change following demand; also, different fuel types like gas, coal, etc. were also considered. The study indicated that even though the demand for renewables increased in the electricity market, their price-dampening effect had been reduced due to the drop in fuel prices, while the reduction in the forecasting errors in the wind and solar forecast resulted in minimized price volatility.

Martin ^[43] developed a simple historical market simulator model based on the LOB data of the German intraday electricity market to reconstruct the complete state of the market at any time. The model was able to capture the effect of trade size and trade time on intraday prices. A stochastic market model was then created from order flow data. The simulation study was performed on data collected over a month for the last two hours of trading and it has been found that the market was able to represent the behavior of markets such as the order arrival rate, the order prices, the mid-price, the price at the best ask and the price of the best bid. Still, further modifications are required to make it useful for a short time intraday market.

Karanfil and Li ^[45] applied a methodology consisting of a VAR framework and generalized impulse response simulations to test the causality of the intraday market fundamentals on the difference between the intraday prices and the day-ahead values in the Nordic intraday electricity market. Deviations between wind generation, conventional generation, total demand levels, and cross-border electricity trades were the market fundamentals considered to study the relationships. The result of the study revealed that wind and conventional generation forecast errors were the most important factors creating deviation in intraday and day-ahead prices, and that a decrease in the level of wind forecast errors resulted in a decrease in the price deviations. Moreover, it was found that the difference caused due to the impact of wind deviation on the intraday market does not fade away quickly, as it occurs through the generator's continuous adjustments using real wind generation for the next hours. The wind forecast errors adversely impacted cross-border exchange, leading to a damping out of these errors due to the causal relationship between price differences and cross-border trading associated with wind forecast errors. Additionally, the wind forecast errors showed a negative influence on conventional forecast errors and a positive impact on load forecast errors.

A regression analysis was applied to the time series data derived from the difference between day-ahead and intraday prices within the German intraday market. This analysis aimed to illuminate the mechanisms of price formation and the influential factors impacting prices ^[26]. The day-ahead prices, intraday prices, trading volumes, and errors of wind and solar energy production forecasts were utilized as the dependent variables for forecasting whereas, power plant outages, load forecast errors, foreign demand and supply, the merit-order curve, market behavior, and ramping costs were used as independent parameters. This research has indicated that errors in solar and wind forecasting as well as outages highly affect intraday prices. The merit-order curve, ramping costs, and market behavior have not shown any remarkable influence on the prices. Additional studies to understand price formation might include an analysis of unobserved determinants. Table 3 provides an overview of the statistical/econometric forecasting methods.

2.3.   Machine learning models

Machine learning techniques allow systems to be trained rather than explicitly programmed; they learn to recognize statistical structures in the examples they are shown. Such techniques have become very popular in recent years due to the increase in computational power and big data, as they excel on complex problems with comprehensive datasets on which many traditional statistical methods would become impractical ^[46]. Several authors have reported their excellent performance over traditional prediction tools because of their accuracy and ability to handle nonlinear, non-stationary, and complex data structures. In literature, numerous machine learning models such as artificial neural networks (ANNs), support vector machines (SVMs), and recurrent neural networks (RNNs) were unquestionably preferred over conventional techniques to forecast intraday electricity. However, the forecasting accuracy of such models mainly depends on the parameter selection and data used to fit the model ^[65]. In the following sections, the machine learning methods are discussed according to the forecasting parameters, starting with prices, and followed by volume and volatility.

2.3.1.   Prices

Kolberg and Waage ^[46] developed numerous types of neural networks-based models to predict hourly prices in continuous electricity intraday markets by using data from the Nord Pool's intraday market Elbas and weather forecast data. A virtual wheeling platform (VWP) of trades for a given hour of electricity delivery was used as the output of the prediction over the six hours remaining until delivery. Pre-processing of market data and weather forecast data was performed to map the price to each set of forecasts. At first, a deep multi-layer perceptron was designed; it uses a sliding window of input data to capture the temporal dimension; later, through experimentation two more architectures ResNet1 and ResNet2 were designed with some modifications to the original. In addition to that, two long short-term memory (LSTM) recurrent networks were designed, i.e., normal LSTM and bidirectional LSTM. All of these models used only the market data for prediction. Finally, with the inclusion of market and weather data, a multi-input network known as multi-LSTM ResNet was developed. The performance of these models was compared with that of different heuristic models along with statistical and simpler machine learning models such as gradient boosting, multivariate adaptive regression spline feature selection, LASSO, ridge regression, principal component regression models; the proposed model outperformed the heuristic and benchmark models.

A comparison study of the neural network-based models for weighted-average intraday price prediction was performed for the Turkish intraday market ^[47]. Exogenous variables such as the day-ahead price, balancing market price, renewables forecast and demand/supply, and trade value were employed as independent variables. The RNN shown in Figure 7 has outperformed the LASSO and linear regression models. The numerical results illustrated that gated recurrent units (GRUs) performed better than the classical models. Even though day-ahead prices were used as an independent variable for forecasting, the spread between forecasted intraday and day-ahead prices was lower and can hence be considered as one of the important variables. Important feature selection can be a topic of discussion for future work as most studies of the paper were focused on forecasting only.

An LSTM neural network-based model consisting of two layers was designed to forecast the two-hour intraday prices by using a multi-step prediction approach ^[51]. LSTM cells have input, output, and forget gates, which allow the model to maintain information over time and regulate the flow of information. This approach involves building separate models for each prediction step, using only past observations. The model incorporated 16 variables, including average prices and lagged average prices, and it considered month and hour data to address seasonality effects. Additional variables like production, consumption, and imbalance cost were added to enhance performance. Error metrics were used to assess the model's accuracy, and the lowest error was achieved when considering all variables. The model also performed better for certain variables. Future work might involve the inclusion of exogenous variables with the application of direct multi-step-ahead prediction.

Klein et al. ^[7] proposed two deep time series models based on the RNN variant called an echo state network (ESN) to enable probabilistic point forecasting of the price. In the first model, the ESN was modified with the introduction of random disturbances and shrinkage before additional regularization. The second method used a Gaussian probabilistic ESN with the application of the deep distributional regression method. Bayesian Markov chain Monte Carlo methods were used to estimate both models and compute forecasts. These models have captured all three features, including nonlinear serial dependence, extreme levels of asymmetry, and strong time variation in the distribution, which are essential for the accurate modeling and forecasting of complex time series. The results of the study showed that the models were accurate when applied to the Australian National Electricity Market, and they also showed that including demand forecasts in the model improves the accuracy of the forecasting. Future work might involve using various methods to estimate time series speeding up computations as well as using deep time-series models for economic and financial applications.

Kath and Ziel ^[54] proposed a multivariate elastic net regression model to forecast the quarter-hourly electricity prices for the German intraday market. The authors have studied the impact of day-ahead EXAA prices on the intraday continuous and intraday call-auction prices. The results of the elastic net based forecast model were compared with those of classical linear regression models; the model showed considerable accuracy and outperformed benchmark models. The addition of EXAA prices as an explanatory variable resulted in a further increase in the accuracy, but it mattered only to a small extent. On the other hand, the authors have also discussed making trading profitable decisions based on the forecasting model. Further research includes the study of the effect of adding more vendor data on forecasting accuracy, the use of non-linear prediction models like random forests, and the study of directional forecasting approaches.

Narajewski and Ziel ^[55] developed a model using the LASSO and elastic net techniques to forecast the VWAPs of hourly and quarter-hourly products in the German continuous intraday electricity market. The performance of the models was compared with that of seven different benchmark models. The results from hourly products indicated that transaction data were not useful in providing enough market information even after considering the recent prices; thus, most of the models did not perform better than naïve models. On the other hand, the quarter-hourly products did not give satisfactory results for the most recent value due to the lower number of transactions compared to hourly products. The elastic net with the standard penalty and correctly back-transformed performed best for the information model using variables such as intraday auction price, the most recent value of the corresponding product, and the closest hourly product. The authors further stated that the correct backward transformation makes the forecast performance better. Future studies can involve using different estimation methods, including more fundamental variables, or employing the probabilistic forecasting technique.

Mohammadi and Hesamzadeh ^[56] built different econometric frameworks which include LSTM, deep convolutional generative adversarial network (DCGAN) and No-U-Turn Sampler (NUTS) algorithms to model the intraday prices using inadequate historical data. Day-ahead and intraday market data were used to study the effects of time, area, bidding zone, and volume on the behavior of these markets. At first, an LSTM-based algorithm was applied to the intraday prices to capture the temporal trends; this resulted in the formation of time series similar to the actual data. Later, a DCGAN-based approach was employed to model intraday prices which were considered as unknown functions without using probabilistic features; it generated prices that are more adaptive to market changes. Finally by considering intraday prices as random numbers, and fitting the probability distribution function (PDF) to the actual data, a NUTS-based algorithm was applied to generate the prices with a similar PDF. The results of this study have shown that DCGAN-based and NUTS-based models were more accurate than the LSTM. LSTM, as applied with enough data was the best choice for the short-term studies of power systems, whereas the DCGAN-based model was best for the long-term studies and the NUTS model was the best choice in the case of limited availability of data. Further research can include building a combination of all of these methods for better forecasting.

Lehna et al. ^[58] formulated the intraday prices as an MDP to apply deep reinforcement learning (DRL) in the form of a proximal policy optimization algorithm for automatic trading in German intraday markets. A simulation framework was employed that enabled the trading of the continuous intraday price in one-minute steps; it was tested from the perspective of a wind farm operator. In order to reduce the complexity and capture the essential information of the LOB, a restrictive environment for DRL agents was constructed, so that adequate simulation of the trades could be realized. The framework used two external factors; the first one was the wind forecast, which was used to indicate expected production capacity in the intraday market and price forecasts to learn about price change and trading trends. Even with high variance in prices, the framework captured the relevant patterns effectively and formed a good trading strategy; thus, the results showed better performance of DRL agents over multiple baseline models with 45.24% improvement. As the evaluation was carried out on the data over one time period of one week, the framework might not be able to specify long-term influence; hence large datasets should be considered. In the future, a more complex rule-based approach can be developed to compete with the framework by considering more explanatory variables.

2.3.2.   Volume

Janke and Steinke ^[68] developed various linear regression and neural network models to predict the quantiles of the price distribution for the last 3 hours before delivery, using trading and fundamental data like the load forecast, the forecasts of wind and solar power, etc. A multi-output neural network model is visualized in Figure 8; this model uses an architecture that accounts for the structure of the inputs and limits the number of parameters in the hidden layers. The study found that the exogenous variables did not improve the accuracy of the forecast, unlike the time-series information from neighboring products and quantiles. The comparison of the proposed models with the naïve and statistical models showed that the proposed models are superior, and that the LASSO regularized linear regression model is the best performer. Future work should focus on the results of quarter-hour predictions and any effect this has on the accuracy of the hour predictions and the forecasting of prices and volumes for short-term trading and risk management.

Pozzetti and Cartlidge ^[60] experimented to forecast the electricity transmission system demand in the British intraday continuous electricity market. A feed-forward ANN (FANN) was trained to predict one-hour electricity demand and this demand prediction was an input to the second FANN to predict the net imbalance volume in the electricity market. The model showed high precision accuracy during the live testing, where net imbalance volume prediction was used to decide on the buying and selling of 30-minute electricity contracts. Finally, another promising line of research would be the optimization of the model through the use of different periods, time horizons and deep learning methods, as well as exploring more sophisticated algorithmic trading strategies, including risk hedging and portfolio optimization.

2.3.3.   Price volatility

Cramer et al. ^[37] employed multivariate probabilistic forecasting approaches to predict the intraday electricity prices for the German EPEX spot market. As the intraday market is used to adjust the day-ahead deviations, it represents the characteristics of the intraday prices. Here, the prediction was performed by considering the day-ahead prices as fixed, and forecasting the difference between the day-ahead and the intraday prices. In this study, the 15-minute intervals per hour of the day-ahead price interval were considered as a four-dimensional joint distribution to capture the hourly fluctuation patterns. A deep generative model known as a normalizing flow model was used to study the multivariate price difference distribution which integrates the multivariate density estimation and probabilistic regression. Comparison of the model with the historical data and two other models namely, Gaussian copula and multivariate Gaussian regression has proved that the normalizing flow model works better as a tool to yield an accurate forecast with narrow prediction intervals. Moreover, the impact of external factors was studied to determine whether their effects were negligible on the model performance; however, the prior realizations of the price difference and all of the input factors could lead to improved performance. In the future, more meaningful impact factors can be employed to increase the accuracy of the model. Table 4 provides an overview of the machine learning forecasting methods.

2.4.   Ensemble and hybrid methods

To date, researchers have developed ensemble and hybrid methods to increase the accuracy level of the forecasts and create more comprehensive and effective models. Considering the ensemble methods, these methods work to enhance the strengths and accuracy of the individual methods. There are three different ways to integrate the methods, i.e., linear, nonlinear and a combination of both. Statistical and machine learning methods entail the use of ensemble models to obtain aggregated decisions by using multiple predictors. Moreover, competitive and cooperative ensemble forecasting are the two categories of ensemble methods. In competitive ensemble methods, a forecasting task is divided into different sub-tasks and solved individually to enhance the performance of the model. The prediction values get added to obtain the final forecasting results. The competitive ensemble forecasting models use multiple predictors with different parameters to build individual forecast models and form an ensemble forecast model. The results of the forecasting simulation from the selected models are generally aggregated by averaging ^[69]. Hybrid methods, on the other hand, combine different types of models or techniques to create a single unified model. This could involve combining statistical models with machine learning algorithms or using different types of data preprocessing and feature selection techniques. Hybrid methods aim to leverage the advantages of different approaches to create a more robust and accurate model.

2.4.1.   Prices

A method of statistical point and probabilistic forecasting was proposed for day-ahead and intraday forecasts for the Iberian electricity market ^[48]. Gradient boosting trees and linear quantile regression models were utilized for forecasting along with a careful selection of the multiple explanatory variables like future contract trading and a forecast of the daily average spot price. The prices from the previous sessions were useful to get high-quality probabilistic forecasts for the intraday market. Figure 9 depicts an example of day-ahead and intraday hourly prices, and it can be noticed that day-ahead prices strongly influence the intraday sessions. Moreover, each intraday session is highly correlated with the previous one. Future work should include forecasting operating reserve prices, and mid-term probabilistic prices and employing models such as autoregressive logit models or deep learning to extract important features.

Kath ^[52] proposed a regression-based model to understand the effect of the XBID variable on the price, volatility and volume of the German intraday market. Fundamental variables included in the study were day-ahead prices from EPEX and foreign markets, EPEX intraday prices, flow, load, wind, and photovoltaics production forecast and the future prices of coal, and gas. Linear regression analysis was used in the paper to answer the research question of how coupling affects electricity prices. The random forest approach was employed to compute variable importance. This metric measures the change in prediction error when a single input variable is altered randomly while keeping all other factors constant. The variance-stabilizing transformation and the Newey-West estimator, which assumes heteroskedasticit, were applied to model the electricity time series. Initially, three hypotheses were made on the effects of the XBID on the intraday market. The study showed that the XBID does not impact intraday prices. Moreover, it has an impact on cross-border trades that is dependent on the regulator's wish to increase the market liquidity. Finally, volatility is not affected by the XBID.

Multiple linear regression methods like OLS and Weighted Least Squares (WLS), a GARCH, and an SVR model were investigated to forecast the per-minute price of the last trading hour for the German intraday market ^[27]. The analysis and experiment indicated that all the models outperformed the naïve and benchmark models in terms of minimizing prediction errors. The best performance was shown by the GARCH and SVM with less deviation in prices than benchmark models. Linear regression and SVM stood second and third respectively according to the absolute error. The study has found that the price does not behave completely randomly but changes in relation to endogenous information.

Hamilton et al. ^[64] designed a hybrid SVR and FANN-based model to forecast very short-term electricity prices of the National Electricity Market in Australia. The six variables used in this forecasting model were date/time, historical price, historical demand, rain, temperature, and natural gas prices. At first, multiple SVRs were used to preprocess the selection of pairs of strongly correlated variables and filter out the extreme outliers that skew the forecast. A sigmoid kernel was used for SVM training. Later, data that exhibited weak or no correlations were fed directly and independently into the FANN to determine its relevance to the forecast price. Results have shown that the hybrid model performed better than the individual SVR and FANN models in terms of overall accuracy and precision. The model built by the averaging of individual SVR and SVR-FANN predictions exhibited the best performance in terms of predicting the magnitude and duration of price spikes, as well as rapidly changing trends. Future research could include improvement in the accuracy of price spike forecasting through the use of neural computation and iterative methods, with the inclusion of more variables such as bidding and rebidding strategies and generator availability.

Scholz et al. ^[53] presented shallow and deep learning methods to predict the 15 min electricity prices in the German intraday market. The extreme gradient boosting (XG-Boost) and random forests were the two types of shallow learning-based models employed in the study, and the other was the LSTM-based deep learning model. A rolling window was created to model the VWP during intraday data. The comparison of these machine learning models with the state-of-the-art baseline models showed that the latter type was outperformed by the former. The LSTM model performed the best followed by the XG-Boost model. A multi-step time series prediction was also carried out to forecast the prices for the next 4 hours, but a study showed that the accuracy of prediction decreases with the time horizon. Future work should contribute to enhancing the performance of the algorithm by integrating more influencing factors of the intraday market, an extension of the forecast period, possible use of a convolutional neural network, etc.

2.4.2.   Volume

Demirtas ^[59] proposed 15 different types of machine learning models to predict the energy imbalance or net loading in the Turkish intraday and balance the energy market for the future 32 hours up to the delivery time. Prediction of net loading can be useful for making profitable trades in the intraday market. The ensemble model that consisted of linear regression, LASSO, elastic net and gradient boosting models outperformed the naïve model for the study of $T+1$ to $T+32$ hours. In addition to that, techniques such as stacking and elastic net ensemble models also showed better performance. Additional studies include understanding how the models work on the prices for the intraday market. Table 5 provides an overview of ensemble and hybrid forecasting methods.

3.   Discussion

In this review paper, we comprehensively examined various intraday forecasting techniques applied in electricity markets. Our analysis aimed to provide a comprehensive overview of the diverse methodologies employed to forecast intraday electricity prices, volume and price volatility. The synthesis of these techniques has illuminated key trends, challenges and areas of potential improvement in the field of intraday market forecasting. One prominent observation from our analysis is the wide array of statistical, machine learning and ensemble techniques that have been utilized in intraday forecasting. This diversity underscores the complexity of the intraday electricity market, necessitating the adoption of a variety of approaches to capture its nuances accurately. Statistical and econometric models play a pivotal role in understanding and quantifying the intricate relationships, dependencies and patterns within complex systems. In the context of intraday electricity markets, these models prove to be indispensable tools for forecasting electricity prices, volumes, and volatility. They accomplish this by incorporating a multitude of factors, including supply and demand dynamics, market fundamentals and exogenous variables. Statistical and econometric models serve as essential tools for understanding and quantifying the relationships, dependencies, and patterns within complex systems. In the context of intraday electricity markets, statistical and econometric models play a pivotal role in forecasting electricity prices, volumes and volatility by incorporating various factors such as supply and demand dynamics, market fundamentals and exogenous variables. The study encompasses a comprehensive spectrum of distinct statistical and econometric models, covering regression-based methodologies, LASSO-estimated models, time series-based approaches, auction curve-based techniques, GAM, point forecast models, historical market simulators and MDP, PCA and LOB models. Each of these analytical paradigms contributes a distinct facet to the intricate tapestry of intraday electricity market forecasting. Regression-based models offer structured insights into variable relationships, while LASSO estimation aids in variable selection and regularization. Time series models capture temporal dynamics, auction curve-based techniques incorporate market mechanisms and GAM introduces flexibility for nonlinearity. Point forecast models provide succinct predictions, historical market simulators replicate past behaviors, the MDP contributes decision-making insights, PCA extracts essential features, and LOB models offer granular insights into market orders.

Machine learning techniques, such as ANN, LSTM, ESN, GRU and FANN techniques have gained traction due to their ability to capture complex nonlinear relationships and adapt to changing market conditions. They share the commonality of being neural network-based approaches that can capture intricate patterns and relationships in data. ANNs excel in nonlinear mapping, while LSTM and GRU models efficiently manage sequential data with memory and gating mechanisms. ESNs leverage reservoir computing and FANNs provide simplicity and computational efficiency. The integration of historical data, market fundamentals, and exogenous variables has improved the predictive accuracy of these models. The choice of model depends on data characteristics and forecasting requirements, offering researchers a spectrum of options to optimize predictive performance and potentially explore hybrid strategies for further advancement.

Ensemble and hybrid methods, leveraging the strengths of multiple techniques, have showcased heightened robustness and reliability, thereby contributing to improved forecasting performance. Notably, the implementation of gradient boosting, random forest approaches, tree-based models, SVR, and SVR-FANN has demonstrated a remarkable leap in predictive performance. The concept of gradient boosting, characterized by its sequential improvement of weak learners, offers a dynamic mechanism to aggregate predictive power, resulting in a robust overall model. In a similar vein, the random forest approach capitalizes on the aggregation of decision trees, fortifying the model against the pitfalls of overfitting and noise inherent in complex datasets. The efficacy of tree-based models, encompassing both decision trees and their ensemble counterparts, lies in their adeptness at capturing intricate relationships within the data through hierarchical structures. On the other hand, the application of SVR introduces a nonlinear dimension to forecasting, as it skillfully transforms data into higher-dimensional spaces to unveil underlying patterns. Furthermore, the fusion of SVR with a FANN, aptly termed SVR-FANN, exemplifies a synthesis of techniques that transcend the capabilities of individual components. This hybrid approach allows for the nuanced capture of complex data patterns, enabling enhanced accuracy in terms of predicting electricity prices. In the realm of electricity price forecasting, the utilization of ensemble and hybrid methods has emerged as a promising avenue, showcasing their distinct advantages over traditional machine learning and statistical techniques.

Throughout our analysis, the influence of renewable energy sources on intraday forecasting emerged as a key focus area. Wind and solar power generation pose challenges due to their intermittent nature, requiring specialized techniques to account for their impact on price formation and demand patterns. The inclusion of renewable forecasts has shown potential for enhancing accuracy, though challenges related to their volatility and uncertainty warrant further investigation. Furthermore, our examination revealed the significance of incorporating exogenous variables such as weather forecasts, supply-demand imbalances, and market state indicators. These variables contribute valuable information for the modeling of intraday dynamics, enhancing the precision of forecasts and capturing the effects of external influences. While this review provides a comprehensive understanding of intraday forecasting techniques, several avenues for future research have emerged. The development of hybrid models that leverage the strengths of different techniques holds promise for further improving accuracy and robustness. Exploring the incorporation of advanced data sources, such as real-time data, could lead to deeper insights and more accurate forecasts.

4.   Conclusions

This review paper contributes a synthesized overview of intraday forecasting techniques, shedding light on the evolution, challenges and potential advancements in this dynamic field. The diversity of approaches and the incorporation of emerging technologies offer a promising path toward more accurate and reliable intraday electricity market forecasts. As the energy landscape continues to evolve, continuous research and innovation will play a pivotal role in refining and advancing intraday forecasting methodologies. The accessibility of data furnishes researchers with valuable insights for the analysis and refinement of intraday electricity forecasting models, ultimately contributing to enhanced accuracy.

Use of AI tools declaration

The authors declare that the research that was conducted and presented in this article did not entail the use of AI tools at any stage of the research process.

Conflict of interest

Andres Elias Feijoo-Lorenzo is an editorial board member for AIMS Energy and was not involved in the editorial review or the decision to publish this article. All authors declare that there are no competing interests.