Figure 1. Number of weeks before the CRPSS goes below 0.1 for all the points in Europe (left) and for all major river points (area >=2000km2). Reference forecast was persistence. Blue colour indicates a good performance, and red colour indicates a less good performance.
Figure 1. Number of weeks before the CRPSS goes below 0.1 for all the points in Europe (left) and for all major river points (area >=2000km2). Reference forecast was persistence. Blue colour indicates a good performance, and red colour indicates a less good performance.

The extended EFAS domain was put in operations on the 16 May 2018, and it is now time to make an assessment over the performance over this first year. The assessment of skill scores where done over the period 1 June 2018-31 July 2019, well over a year. The skill was computed using the continuous ranked probability skill score (CRPSS). The proxy for observations were the simulations forced with observations. For this validation, the catchments in Africa were omitted.

EFAS headline score CRPS Skill score over the entire period for all grid points which have an upstream area of >=2000 km2 as a function of lead time
Figure 2. EFAS headline score CRPS Skill score over the entire period for all grid points which have an upstream area of >=2000 km2 as a function of lead time. Persistence (blue line) uses the forecast from the previous day as the reference, which is a very difficult forecast to beat. The black line represents CRPSS against climatology, which is comprised of simulated hydrological time serios for previous years. The shaded area denotes the 10th and 90th percentile of the scores respectively

Current skill of the forecasts

The CRPSS for the first 15 days of forecasts shows a good skill against climatology (Figure 2), whereas using the previous day’s forecast as your reference forecast shows a more rapidly decreasing skill. Any skill above 0 is considered better than the reference forecast. The performance shows a strong spatial variability, where the model performs generally better in the northern part of Europe than in the south. It can be noted that the larger rivers perform on average better than smaller rivers (top figure).

The skill also varies greatly over the year, which can be seen in Figure 3. The skill in winter is normally higher than in summer, because most rivers are at their lowest flow during that season. The variability is small, which makes it easier to predict. It is yet not enough data to say something more substantial about the long-term score development of EFAS.

Monthly means of CRPSS for lead time 5 days for all the major river points in Europe over the test period. Reference forecast was climatology.
Figure 3. Monthly means of CRPSS for lead time 5 days for all the major river points in Europe over the test period. Reference forecast was climatology. The skill is largest during the winter months, when there is less variation in the flow in large parts of Europe.

Plans for verification

The plans for EFAS verification will follow several development paths. Firstly, the points where we have observational data in near-real time will be used for an assessment of the skill of the forecasts against observations. This will give a better understanding of the real value of the system in those points. Secondly, hindcast experiments will be able to give an idea of the development of the scores over time. Such experiments will take time to run and will be presented at a later stage. Thirdly, verification of the issued notifications is a priority. Currently, there is not enough feedback to properly assess this, but the new web interface will enable a more systematic assessment of this skill.