 Internship of SPO staff at UVER, 5-16 November 2007
Participants
Volkan Idris Sari (SPO - Monitoring and Evaluation Department)
Zeyneb Ersayin (SPO - Monitoring and Evaluation Department)
Tutor
Carlo Amati (UVER - Monitoring and Statistics Area)
Introduction
The internship project was carried out in the framework of the bilateral cooperation agreement between Italy (DPS) and Turkey (SPO - State Planning Organization) signed in November 2006 and aimed at promoting a regional development approach in Turkey.
The internship was based on the most general vision of a monitoring and evaluation system for public investment projects and how the different components of such a system are implemented in DPS.
The activities carried out during the internship were organised in a programme of half- or full-day meetings on specific issues related to the system. All meetings were accompanied by supporting documents listed in this page.
The outcome of the internship is a project work, presented in Rome at the end of the internship, which outlines various topics of common interest for eventual future collaboration between Italy and Turkey.
Day 1
Welcome meeting with Head of UVER and Area Coordinators (Antonio Caponetto, Ivo Bianchini, Francisco Barbaro)
Data sources and their integration: public investment projects and other data (Fabio De Angelis, Carlo Amati)
Public investment data can be found in different repositories, in general not directly linked to one another.
The main monitoring systems on investments databases were presented, together with public contracts and calls for tender data sources. Data-quality and data-integration issues that arise when dealing with multiple repositories were examined and the methodological approach and software tools set to solve them were discussed.
 Main repositories on public investments (ppt, 2.8 MB)
 Data integration with textual information (doc, 59 KB)
Day 2
Introductory presentations (Antonio Caponetto, Volkan Idris Sari, Zeyneb Ersayin, Margherita Onofrio, Carlo Amati)
 UVER Presentation (ppt, 739 KB)
 SPO Presentation (ppt, 1,4 MB)
 Bilateral cooperation project presentation (ppt, 187 KB)
 Internship presentation (ppt, 791 KB)
A framework for a monitoring and evaluation system for public investments (Francisco
Barbaro, Carlo Amati)
A monitoring and evaluation system on public investments supports the operational and political decision-making process and acts on the three following levels: integration, analysis and interpretation. This vision was presented with a focus on how this system is implemented at UVER.
 Framework for a monitoring system (ppt, 134 KB)
 Monitoring the project life-cycle (doc, 153 KB)
 A poster on tools for public investment analysis (pdf, 2.9 MB)
Day 3
Monitoring reports: the case of the Framework Programme Agreements (Mario Vella, Saverio Massari)
The meaning and aim as well as the procedural and technical aspects of monitoring public investments was analysed, with a focus to the "Progetto Monitoraggio", the initiative worth 100 million euros launched in 2003 by DPS in order to reinforce monitoring systems both at central and regional government level. The discussion was then focused on how to analyse and report monitoring data to several bodies and stakeholders. The procedures and the reports made available to central and regional governments for performance assessment were browsed in real time on the web-based system.
 Report on planning and implementation of Agreements (doc, 180 KB)
The forecasting system of the expenditure for public investments (Carlo Amati, Valentina Bravi)
The need for an ever more efficient allocation of the available resources and their optimal utilisation, along with the poor reliability of estimates made by the
local implementing bodies, prompted the development of a tool to forecast yearly expenditure on public investments in a systematic manner. The forecasting system for the expenditure of public investments was presented as a tool that increases the information made available to decision-makers.
 The forecasting system and its application to the Agreements (pdf, 533 KB)
Day 4
Measuring the performances of public investments and public administrations: benchmarking and ranking (Carla Carlucci, Francesca Spagnolo)
The performances of local implementing bodies can be analysed in terms of reliability and effectiveness in relation to the duration of the various implementation
stages of their investment projects. An analysis of the factors that influence this duration, at the level of public body, showed the importance of project features, the socio-economic context and the internal organisation of the local body itself.
 A model for performance analysis of local bodies (ppt, 681 KB)
 Ranking of government levels (doc, 145 KB)
Further analyses on the outcome of the forecasting system of public expenditure (Carlo Amati, Valentina Bravi)
The availability of a forecasting system for public investments opens the possibility of a wide range of analyses on the factors that influence the duration of the
various implementation stages of a project and on the need for specific actions aimed at enhancing the time efficiency of investment programmes.
 The importance of factor levels in the forecasting system for the expenditure of public investments (pdf, 421 KB)
Day 5
The evaluation of the effectiveness of public investments: an individual approach (Luigi Guerci, Tiziana Tamborrini)
The methodology for the evaluation of effectiveness of individual project was presented starting from the legal framework and then discussing the guidelines prepared by UVER with a focus on evaluation objectives and evaluation steps. Also the data collection format and questionnaire was analysed, as well as the indicators used in order to assess the projects. At the end of the day,
some relevant documents were examined in relation to the mission to Benevento (see Day 9).
 Effectiveness guidelines (doc, 355 KB)
 Individual approach to the evaluation of effectiveness (ppt, 826 KB)
 Governance questionnaire (doc, 62 KB)
 Preliminary information on Sant'Agostino complex (ppt, 614 KB)
Day 6
The evaluation of the effectiveness of public investments: a global approach (Carla Carlucci)
The global effectiveness of public infrastructure investments can be measured by their contribution to local growth. This can be analysed at a territorial level involving the community wellbeing firms and families through a spatial economic equilibrium model taking into account the socioeconomic relations existing among several geographic areas. The analyses carried out both at a national level and at a regional level were presented.
The effectiveness of public investments at a territorial level: a spatial economic equilibrium approach (doc, 237 KB)
Territorial analyses on public investments (Fabio De Angelis, Carlo Amati)
The integration of public investment data offers the possibility of sophisticated analysis, whose results can be presented in appealing formats. Map representations and
various forms of charts and graphics were presented as well as territorial analysis techniques and main indicators used to show spatial distribution of public investment expenditure.
 Territorial analyses (ppt, 15.3 MB)
 Analyses on public investments (ppt, 1.5 MB)
Day 7
The verification of public investments: theory (Luigi Guerci, Paola Ibba, Fabio Di Matteo)
The methodology for controls and, in particular, second-level controls on projects implemented with EU funds was presented. The discussion was then focused on the guidelines prepared by UVER for checks on the 2000-2006 Operational Programme "Technical Assistance and System Actions" and, more in general, on the
concept of audit applied to operations (projects) and to management and control systems.
 The verification of public investments (ppt, 653 KB)
 Guidelines for second-level controls in the 2000-2006 Operational Programme "Technical Assistance and System Actions" (doc, 817 KB)
 Risk analysis - First level (ppt, 752 KB)
 Risk analysis - Second level (ppt, 721 KB)
The verification of public investments: simulation of a second-level control (Massimo Patassini, Paola Ibba,
Fabio Di Matteo)
The simulation involved a replay of the activities carried out for second-level controls on projects implemented under the aforementioned Operational Programme. The project concerned an "Example of audit report on the procurement of services" and the simulation involved
representatives of the Structural Funds Service acting as implementing body.
 Form for second-level controls in the 2000-2006 Operational Programme "Technical Assistance and System Actions" (doc, 150 KB)
Day 8
The full story of the "Completamenti" investment programme (Carlo Amati, Umberto Cafiero)
The "Completamenti" investment programme, launched in 1998, has been managed by DPS since the initial allocation of resources to the completion of the projects. UVER has been in charge of all the monitoring and control activities, including the evaluation of the effectiveness of the completed projects. All the activities were presented with a specific focus on monitoring procedures, tools, outcomes and reports.
 The Completamenti programme (ppt, 6.6 MB)
 Monitoring questionnaire form (doc, 212 KB)
 Monitoring questionnaire instructions (doc, 95 KB)
 Monitoring data controls (doc, 29 KB)
Day 9
The evaluation of effectiveness of an investment project in practice: a mission to Benevento (Luigi Guerci, Carlo Amati)
The mission was aimed at assessing the effectiveness of the "Complete restoration of the Sant'Agostino monastic complex". The on-site visit encompassed the check of administrative papers, output and outcome indicators, services provided and visual examination of the completed structure.
 Preliminary information on Sant'Agostino complex (ppt, 614 KB)
 Effectiveness evaluation report (doc, 42 KB)
Day 10
Final presentation of the project work (Volkan Idris Sari, Zeyneb Ersayin)
 Presentation (ppt, 987 KB)
 Project work (doc, 172 KB)
|
