Forecasting power systems
Electricity producers rely on forecasts of electricity prices for bidding in the markets and power plant scheduling. Markets are changing: A much tighter integration between European markets and a rise in unregulated renewable energy production, especially wind and photo-voltaic, call for joint probabilistic forecasts. Incorporating the transient interplay between productions from renewable sources is critical to power production and financial operations. Multivariate probabilistic forecasts of electricity prices in the short horizon are required.
Appropriately characterising multivariate uncertainty will enable more effective operational decisions to be made.
Conventional power grids add extra generation and distribution capacity. Smart grids actively match energy supply and demand and combine the needs of the markets with the limitations of the grid infrastructure. With the implementation of smart meters and grid sensors, enormous amounts of time series data are generated, with seconds resolution. Our objective is to develop new methods that extract the right information from data to optimise grid control and for real time operation.
Error dressing spot price forecasts
Electricity markets "spike" and "crash" when volumes are respectively slightly higher or lower than typical and these extreme price swings make uncertainty quantification a critical part of forecasts. However, the limited degree to which these extremes are observed makes such constructions difficult. We have developed a system using published bid/ask curves that determine the final price, to construct realistic distributional price forecasts that embed this extreme behaviour. We employ the concept of "error dressing" by using the curves to translate residual behaviour of market volume forecasts into price uncertainties.
This system has proven highly useful for our industrial partner Hydro and is now already embedded in their nightly production system. The output of this model informs operational considerations both on the production and the trading sides of NHY’s operations. An academic article outlining the approach and highlighting some of the results has been drafted and is nearly ready for submission to an applied statistics journal.
High-Dimensional demand prediction via principal component analysis
The approach that underlies our error dressing methodology depends on point forecasts of demand and supply and improves as the quality of the underlying models increase. Some of the underlying models that are currently used in practice are decades old. We therefore took this occasion to revisit these models and investigate whether they too could use a reconsideration.
In particular, we found that the model for predicting electricity demand, which is currently based on neural networks, was in need of improvement. Electricity demand in the Nordic countries is primarily forecasted by projected weather conditions, which are a highly correlated, both across the Nordic region and across time. We conducted a preliminary investigation and learned that models based on principal component analysis (PCA) yield a 30% improvement in demand forecasts over the current model in use at Hydro.
The size of this improvement has led us to move quickly towards implementing the new model for Hydro, in tandem with writing an academic paper that explores the use of PCA to filter weather forecasts in the demand prediction problem. Our early 2018 plans will include a roll out of this methodology for Hydro and a draft paper that discusses the results of this investigation.
Research into Vintage Adjustment of Forecast Trajectories
Our work in the power system domain is high-dimensional both in the input space (factors used to make predictions) as well as the output space (the number of quantities simultaneously forecasted is also large). It is frequently the case that a set of forecasts, all issued at the same time, have different timeframes for resolution. Likewise, each observed outcome can be associated with a large number of different forecasts, each issued on a different date.
We call the set of forecasts issued on the same date a vintage. When the members of a vintage have different resolution dates, it was our belief that performance early in the vintage could help adjust later forecasts. Preliminary research into this topic, conducted in 2017, has proven highly promising. In particular, we found that taking a two-day ahead forecast of electricity demand and adjusting for the performance of the one-day ahead forecast from the same vintage outperforms a newly issued one-day ahead forecast for the same quantity by a considerable degree.
The potential for use of vintage adjustment is vast and seemingly relatively unexplored. We plan on continuing the research into this general methodology and creating a high quality general purpose implementation of the method in R. While there are a great number of applications of vintage adjustment in power systems, the scope of this methodology is far reaching and we plan on expanding our considerations to applications in meteorology, finance and natural language processing.
Optimisation based curve disaggregation methods for regional bid/ask curve construction
At present, the bid/ask curves published by Nordpool are aggregated across regions. While this is helpful in modeling the overall system price of the Nordpool region, operations are ultimately conducted on the regional level. However, regional level bid/ask curves are not published. Thus, a methodology that was capable of using market level
bid/ask curves to construct their regional level equivalents would be highly valuable. This methodology corresponds to an assignment problem where each point on the market-level bid/ask curve is assigned to a given region.
Fortunately, regional price and volume information is published and it is possible to verify whether a given regional assignment yields clearing prices and volumes that roughly match the observed regional price and volume distribution. Thus, we have begun constructing a constrained optimization system that performs assignments of points in market-level bid/ask curves to the regional level.
This is a massive-dimensional optimization problem with an extremely large number of potential solutions. We have begun researching various methods that couple integer-programming routines with market simulators and statistical estimation procedures. A successful methodology here will have a substantial impact on Hydro’s ability to conduct fine-tuned and localized price forecasts.