For the second consecutive year, Milieu Ltd, as part of the COWI consortium, has been entrusted by DG SANCO with the task of assessing the quality of the monitoring activities undertaken by the members of the European Alcohol and Health Forum (EAHF). The results of this assessment are presented in this report, which may be of interest to officials in the EU institutions who deal with alcohol and health policy; to Forum members; and to a wider audience of policy-makers and researchers.
The Third Monitoring Progress Report suggests a clear improvement in the quality of information provided in the Forum members’ annual monitoring reports. As set out in the Second Monitoring Progress Report, the benefits of this improved performance in members’ monitoring practices are twofold. First, the transparency and accountability of the performance of the members increases, which might contribute to building trust amongst Forum members. Second, improvement of the monitoring reports, especially their clarity, is crucial to the Forum, as the reports communicate to the general public on the efforts carried out to reduce alcohol-related harm.
This year’s evaluation results signal that many members of the Forum have succeeded in providing clear and useful information with regard to their actions to reduce alcohol-related harm. They also suggest that the recommendations issued in previous editions of this report have been taken into account, although the uptake level varies significantly across Forum members. These results must however be considered along with a number of statistical caveats: a one-third drop in the total number of submissions is the most prominent amongst them. The lower share of final reports in the total number of submitted reports in 2011 (40.91%) compared to 2010 (48.86%) should also be taken into account, since sections nine and ten are only mandatory in final reports and the evaluation results for these two sections may appear artificially high due to statistical effects.
Notwithstanding the overall improvement observed in the 2011 evaluation, most of the critical remarks from the 2009 and 2010 evaluators remain relevant. These remarks relate to lack of understandable or sufficient information in some sections, deficient distinction between output and outcome, and persisting difficulties to find a middle ground between providing excessively detailed (and sometimes superfluous information) on the one hand, and lacking sufficient relevant information (particularly with regard to the commitment’s timeline) on the other hand.
Particularly noteworthy is the fact that the number of monitoring reports that provide little or no information concerning the commitment’s outcome and impact remains significantly high. Although the provision of this information is beyond the Forum’s minimum monitoring requirements, it is critical for the effectiveness of commitments to be appropriately understood and further reporting efforts are therefore required in this area.
Furthermore, linkages between the different aspects of the commitment, as presented in the various report sections, still need to be more clearly identified.
Read Full Report (PDF)
