by Marc Girons Lopez and Diana Fuentes Andino, EFAS Dissemination Centre

In January 2021 the EFAS partners were invited to answer the yearly survey regarding the satisfaction with the EFAS performance, services and products. A link for the web-based survey was made available to all EFAS partners and a total of 54 responses were received, compared to 36 in 2019 and 43 in 2018. The survey was, as in previous years, anonymous.

The survey responses are summarised below according to the following categories:
•    Overall satisfaction
•    Skill, model performance and trust
•    EFAS products
•    Feedback and contributions
•    Concluding remarks

Overall satisfaction

No major changes in the overall satisfaction were reported in 2020 with respect to the previous three years. The satisfaction in the overall performance of EFAS, however, increased slightly with respect to 2019. Even so, there was still a small percentage of participants who rated the EFAS performance as being low.
 

According to most participants, the main benefit of being an EFAS member were the forecasts. Other important benefits according to participants were receiving flood notifications, the learning practices during annual meetings, and getting access to observed and forecasted precipitation.

The Operational EFAS consists of four different centres: the computational centre, dissemination centre, hydrological data collection centre, and meteorological data collection centre. The satisfaction of the participants with each of the EFAS centres in 2020 decreased slightly when compared to that of 2019. Nevertheless, it was still very good and, in general, higher than that of 2018.

Skill, model performance and trust

The use of model skill scores provided in the EFAS bulletin increased for the year 2020. Most participants used the EFAS model skill scores provided both in the bulletin and in the skill layers in the EFAS-IS. Conversely, the skill scores provided on the wiki page were only used by a small percentage of participants.

In general, there were no major differences in the perceived added value of the three different types of EFAS notifications (i.e. Formal Flood Notifications, Informal Flood Notifications, and Flash Flood Notifications) with respect to previous years. Nevertheless, a slight increase could be observed in the added value of Informal Flood Notifications and Flash Flood Notifications with respect to 2019.

Focusing on the flash flood forecast products, more than half of the participants used EFAS flash flood forecasts on a daily/weekly basis while about 30% of the participants used them on a monthly basis. Around 10% of participants reported that they never used them.

Participants tended to use flash flood forecast products when significant rainfall was to be expected and to get alerts from likely events. Nevertheless, several participants commented on the large number of flash flood notifications they received and their uncertainties, and some of them did not find these products useful. In general, there was a demand for further development and refinement of flash flood forecast products, such as increased coverage, and presenting more information about precipitation, among others.

Regarding the use of alternative sources of information, 61% of participants compared EFAS forecasts with other providers, while 13% of them did not. The remaining percent did not answer this question. Most of the participants who answered positively reported that they compared EFAS with national and/or regional forecasts and some of the participants also compared them directly against outputs from other meteorological/hydrological models.

EFAS training and resources

EFAS provides training to partners on a variety of topics and in different formats. When asked about which topics they would like to receive training about in the future, most participants responded EFAS skill and EFAS layers and products, followed by flash flood forecasts and notifications. Other popular topics where the models used by EFAS (e.g. LISFLOOD), new products, and uncertainty and its communication.

Regarding the preferred training format, participants favoured having regular webinars (42 votes), followed by short online tutorials (36). Online documentation and workshops during the annual meeting were the least preferred methods, with 26 and 23 votes respectively.

Relevant information on the EFAS system is provided online in different formats such as the bulletin, wiki page, annual hydrological report, and annual performance report. Out of these sources of information, most participants reported that they read the bulletin, annual hydrological report, and annual performance report, at least sometimes. The EFAS wiki page was the least used of these sources of information. In fact, the EFAS wiki, together with the annual performance report, where the least known documents among participants.

In addition to the aforementioned information sources about EFAS, partners have the opportunity to use a number of resources from EFAS. However, the participants’ awareness of the existence of the different EFAS resources differed significantly. On the one side, the possibility to activate EMS Mapping for satellite imagery, the fact LISFLOOD model is now open source, and the availability of training material on the EFAS-IS platform was well-known among participants. On the other side, the participants’ awareness on the possibility to request training from the EFAS Dissemination Centre and of downloading EFAS hydrological forecasts from the Climate Data Store was somewhat limited. The most unknown EFAS resource were the Jupyter notebooks.

EFAS products

In general, participants were positive about the usability and added value of all EFAS products. The highest rated product was the 6-hourly flood forecast information while the lowest rated ones were the risk assessment layer and the sub-seasonal forecasts.

Only a minor percentage of participants stated that their organisations were using some of the EFAS data (runoff, snow and soil moisture) available through the Sensor Observation Service (SOS) and The Climate Data Store (CDS) web services, and about 20% of them stated that their organisations were planning on using these data in the future. What’s more, nearly half of the participants reported that their organisations did not use these data and another 25% of the participants were not sure about it. The least used variable was simulated snow and the most used was runoff.

Regarding data quality and standardization, most participants stated that the would like to receive both quality flags for their meteorological and/or hydrological data, and their data in a standard format (i.e. harmonised time step and units, and quality controlled).

Feedback and contributions

Most participants stated that providing feedback to EFAS Formal Notifications with the built-in feedback form on EFAS-IS was easier than with the old system, following the same trend as in previous years. Nevertheless, some partners pointed out the existence of bugs in the system that hindered the reporting of feedback and the lack of resources available to provide feedback to all notifications.

Even so, more than half of the participants were willing to provide feedback also for Informal Flood Notifications and Flash Flood Notifications. Most participants were however not sure about writing short articles to be published on the EFAS website or conducting verifications analyses.

Concluding remarks

Overall, partners were satisfied with the performance and added value of EFAS service as well as in its products and resources. However, a number of remarks were also provided on aspects of the system that need be further improved, such as on the performance of flash flood forecasts and notifications or the need to promote and provide training for some of the EFAS products and resources (e.g. EFAS wiki, Jupyter notebooks). Additionally, other suggestions include the expansion of the EFAS domain, and the increased resolution of certain variables (initial conditions, forecasted accumulated precipitation).