The case study focuses on the approach adopted by the EU Public Administration to efficiently manage the realization of new projects, develop software products and supply services, according to specific Service Level Agreements (SLAs), especially when this activity is farmed out to IT enterprises. Nowadays this approach is more and more adopted, fostering the improvement of the process quality as well as the transparency of the communication among the actors involved in the development process. According to this, the involved actors can jointly verify the quality of the development performances, through data analysis based on pre-defined SLAs that are built on well-identified metrics. This allows to monitor software quality improvement over time. This approach enables Public Administrations to purchase quality from services providers in order to provide quality services to citizens, enterprises and to their employees. This study reports the experience of the Veneto Region, a Regional Department of the Italian Public Administration, which has adopted the open source Spago4Q platform (www.spago4q.org), within their Project Management Service. The project developed for the Veneto Region is subsequent to the projects carried out for other European Public Administrations. In particular, the Veneto Region has integrated Spago4Q platform into the IT governance system managing the application and information services supplied to the government bodies, citizens and enterprises, located on the regional territory. Spago4Q is an effective and entirely open source solution allowing to measure and monitor quality improvement progress during the project realization, the software project development and the ICT services supply, in addition to facilitating the project management over time.
| Quick Facts | |
| Project Name | Veneto Region maintenance and development of the Information System |
| Sector | eGovernment |
| Start Date | 2009 |
| End Date | 2016 |
| Objectives | Specific objective related to the case study: development of the dashboard to monitor SLA of the business services. |
| Target group | Public sector |
| Scope | National, Regional |
| Budget | N/A |
| Funding | Internal funding |
| Achivements | Successful introduction of a SLA dashboard and process improvement procedures. |
Introduction
The ICT supplies create a customer-supplier chain where the Public Administration (PA) acts both as a customer, in its relation towards the ICT services supplying companies, and as a business services supplier, for citizens, enterprises and its employees.
Usually, a business service delivered by a PA is the result of a mixture of different software and hardware components and services, taken by different suppliers that have to integrate and cooperate in order to maintain the high level of quality expected by a public administration.
In many cases, a discrepancy may occur between the realisation phase of an application and the successive production phase, inducing a lower quality of the final business service.
Members of the organization dealing with the software constituents and the ones dealing with the service management tend to have conflicting objectives:
- The development teams are usually focused on the single service components, while the management structure is focalised on the delivery of the services as a whole;
- The developments teams usually work on specific projects, while the management structure handles recurring processes and activities.
| Aspect | Project development | Process management |
|---|---|---|
| Time scale | Limited duration | Continuous |
| Process type | Unique | Repetitive |
| Initiative | Scheduled | Event Driven |
| Level of analysis | Clear and well-defined objectives | Overall objectives |
It may happen that, separately considered, the project development and the management process both comply with the pre-defined SLAs, but that the consequent business service provided does not meet the end user needs.
In order to prevent this, it is important that:
- the overall business services goals, and
- the design and implementation of the service supply
link the life-cycle of each application with the life-cycle of the complete service.
The objective can be achieved by working on two levels:
- At the organizational level, a specific team is dedicated to the process of "transformation" of the software product to a customer service. The team must be able to ensure the integration between the service design phase and the service delivery phase, also ensuring a strong focus on the testing and validation phases of the service (Service Transition).
- In terms of approach, defining the expected quality of the service. The quality of the service should:
- be correlated to the goals of the delivery process;
- allow to formalize the requested service level; and
- allow to verify the service level attained in a totally transparent way for all the actors involved.
In particular, the Italian Public Administration, which follows the above-mentioned pragmatic approach, has defined some guidelines (www.digitpa.gov.it) including specific service quality levels as well as ICT governance indicators. This fosters a comprehensive, integrated and thoroughly transparent perspective that involves the whole delivery process: quality policies to be applied within institutional missions, ICT sourcing strategies, public procurement, deal negotiation and operations management.
The service levels are based on the various actors’ points of view, on the activities performed during the whole services lifecycle and on operational requirements. This aims to foster the projects and ICT services success, and to consequently protect the investments over time, provided that they are supported by effective solutions.
Service levels and quality indicators, allow to:
- provide operational goals with measurable results;
- monitor the supply lifecycle and all ICT services activities;
- monitor overall quality over time.
The Spago4Q platform (www.spago4q.org) is an effective solution allowing to measure and monitor the quality improvement over time in project realization, software project development, ICT services supply and facility management. In the case of the Veneto region, Spago4Q is a central dashboard platform collecting information from all services tools and providing reporting and presentation facilities to monitor and control the SLA.
In particular, in the Veneto Region, a Regional Department of the Italian Public Administration, Spago4Q platform has been integrated into the IT governance system managing the application and information services supplied to the local government bodies, citizens and enterprises located on the regional territory. The services delivered by the suppliers (service desk, change management, application lifecycle management, asset and configuration management, network and hardware management) as well as the overall service provided to end-users are evaluated according to Service Level Agreements and Key Performance Indicators (KPIs), based on ITIL v3 and CMMI standard processes.
Organisation and background
The Veneto Region invests in the development of its information system (SIRV) to keep it aligned with the constantly evolving technology in order to increase the efficiency, the transparency and the effectiveness of its administrative processes but in particular to ensure support for innovative projects of e-government, e-democracy, net-economy and net-welfare that help to create a competitive system for all entities in the territory (local authorities, citizens, enterprises).
The objective of the Veneto Region is to create a system able to support a growing digital cooperation between all involved actors.
To this end, the Veneto Region offers:
- an infrastructure, the Information System of the Veneto Region (SIRV), which delivers more than 300 business services both to internal and external users, like local bodies, citizens, enterprises operating on the territory of the region;
- on-line marketplace services for the reuse of experiences, information products and eGovernment projects;
- results and knowledge derived from participation, with other European regions, in EU research projects.
In order to provide services that are increasingly facing out towards businesses and citizens, the most important aspect is not the individual service or a single technological component, but the entire chain of technological elements that contribute to the delivery in parallel of multiple business services.
This means that it makes no sense to measure performance of a single component; the business service has to be evaluated as a whole, in order to ensure accessibility, availability, performance and quality from the end users’ perspective.
Within the SIRV project, the Operative Governance System (SGO) has been created, which allows the Veneto Region to:
- monitor the suppliers’ performance through the measurement and monitoring of the SLA related to the business services, with maximum transparency;
- commit to the satisfaction of users (both internal and external) and improve the quality of the delivered services;
- constantly align and adapt the provided services to the needs of the Administration and the regional territory;
- standardize the IT management and improve the productivity.
Within SGO, Spago4Q monitors the SLAs of the business services, as well as incident, change and release management services.
Budget and funding/Financial issues
The project carried out for the Veneto Region is divided into three main phases: takeover, operational and handover.
Within the takeover phase, the contractor sets up the organisation, the resources and the necessary infrastructure for the provision of the services in the operational and handover phases.
Among the various infrastructure components, the project focuses on the tools to monitor the services and project management quality (audits, surveys, control board, indicators statistics) and the tools to provide reports such as Monthly Progress Report.
In the operational phase the contractor is fully responsible for delivering the required services, developing information systems, correcting defects, monitoring progress and in guaranteeing their quality referring to the defined service levels.
The implementation of the dashboard to monitor the services provided for the Veneto Region has no specific budget but it is part of the set-up of the infrastructure for the governance of the services developed during the takeover.
The project is internally funded.
Technical issues
What is Spago4Q
Spago4Q (www.spago4q.org) is an OSS multi-process and multi-project monitoring platform, released under GNU LGPL license. It allows measuring and monitoring of the quality improvement during the project lifetime, software projects development, ICT services supply and facility management, according to specific service level based on the various actors’ points of view.
Spago4Q architecture, obtained as a specialization of SpagoBI (the Open Source Business Intelligence suite www.spagobi.org), is designed in order to be easily adapted to complex organizational contexts. It integrates an advanced meta-model which makes Spago4Q fully independent from the adopted software development processes, infrastructure tools, measurement and assessment frameworks.
As shows in the previous figure, the ETL (Extract Transform Load) procedures extract data from the set of tools used by the organization and load them into the DWH-Spago4Q data warehouse module, based on the meta-model definition. Spago4Q analyzes data and represents metrics, and the Configuration & Administration module allows system configuration. A set of predefined plug-ins and extractors allows the extraction of data from the most common tools used to support project and service management.
As shown in the following figure, in Veneto Region Spago4Q collects data from the tools used to support the services supplied and provides dashboards for the continuous and proactive monitoring of the SLAs as well as periodic reports on the status of the SLAs and on the result of the undertaken improvement actions, by ensuring transparency in the sharing and dissemination of information through the publication in the information portal of the specific organization.
The most important features follow.
Main Dashboard
The picture below shows the dashboard designed to show SLA/KPI value (also with gauge-bar when required). For each SLA/KPI, where applicable, one or more icons show the presence of an exception or a correction report (or both). The other icons show the historical trend of the SLA value and allow drill down to visualize the items responsible for the excess of the SLA threshold.
Corrections Management
This optional feature grants the possibility to apply corrections to the Liquidated Damage Unit (LDU) already calculated, however the SAL/KPI calculation would remain unchanged:
- the KPI non-compliance will still be visible in the Report;
- the correction description will be visualized in the single impacted KPI detail Report as well as in the global detail Report;
- the LDU calculation will be diminished/augmented taking into account the KPI delta value included in the correction;
This possibility of correction is required because in some cases the SLA non-compliance is not accountable to the contractor (Service Supplier). Such cases, due to the nature of the obstacle impeding the SLA/KPI fulfilment wouldn’t require an explicit and official exception but need to be officially tracked and seen by the Customer in order to enable it to request the annulment of this correction if necessary.
Exceptions Management
This feature gives the Customer the possibility of applying exceptions to the SLA/KPI non-compliances that are deemed to be independent from Service Supplier will and commitment.
Exceptions must be explicit, official and linked to a punctual non-compliance, there is no such thing as general exceptions.
When an exception is granted:
- the SLA/KPI non-compliance will still be visible in the Report;
- the exception description will be visualized in the single impacted KPI detail Report as well as in the global detail Report;
- the LDU calculation will be diminished taking into account the KPI delta value included in the exception;
Comments Management
This is an online functionality that allows inserting comments into the detail Report, to share these comments with the other users and eventually import them in the Monthly Progress Report.
Import data logger and quality control
This is the data upload functionality that performs, via semantic and chronological checks, a quality and consistency control over the uploaded data.
The logger is accessible on the online platform and records the uploads and checks history.
Licensing issues
The Spago4Q platform is licensed under the GNU Lesser General Public License (GNU LGPL).
Cooperation with other public bodies
Spago4Q project is hosted by OW2 forge (www.ow2.org), a global, independent and non-profit open source community, ensuring transparency, openness, sustainability and open source availability over time.
Moreover, it takes part in the durable and collaborative development of the OW2 long-lasting ecosystem, where each player can define his/her own open source strategy. In this way everyone contributes to the global sustainability of a wide stack of open source solutions at enterprise level, which no enterprise could ever develop on its own.
Spago4Q and the other projects within the SpagoWorld initiative belong to a new generation of open source projects, characterized by:
- holistic approach and planned development process;
- complex design in vertical domains according to project business requirements;
- mature process development;
- involvement of a wide community of users, companies and organizations, open source specialists, consultants and developers;
- guaranteed support service.
The communities born around these projects and supported by the active governance role of Engineering Group are as follows:
- A free/open source community has been initially growing thanks to the OW2 community; now it’s growing around Engineering Group’s projects and cooperates through the tools provided by the initiative itself.
- The Open Source Competence Centres, both Italian ones (such as the Italian Open Source Competence Centre www.ccos.it) and international ones (e.g. QualiPSo Competence Centres Network and the FLOSS CC initiative www.flosscc.org); they are based on the results coming from the European research project QualiPSo (www.qualipso.org). The labelling infrastructure of the QualiPSo Competence Centres is based on the Spago4Q platform, which integrates the different tools supporting the assessment of products, processes and services, in compliance with QualiPSo MOSST (Model of Open Source Software Trustworthiness) and OMM (Open Maturity Model) models.
- A commercial community is based on a wide network of technology and integrator partners, which jointly work to support and develop the SpagoWorld projects, providing a wider stack of open source solutions with an integrated offering of support services.
Evaluation, including lessons learned, and future plans
Evaluation
In the project, the pre-defined objectives have been achieved. The implemented solution meets the following goals:
- it provides both global and detailed view of the SLAs values on the monitored services;
- it supports the reporting process (Monthly Progress Report, SLA reports);
- it helps in LDU calculation;
- it shares and stores feedbacks, comments and exceptions;
It also ensures:
- Transparency: all actors visualize the SLAs according to their role
- Uniformity: rules are defined in a single repository;
- Consistency: well defined data concept;
- Univocity: the information is collected once;
- Archiving: data are consolidated at Service level.
Benefits of this approach follows:
- From the Public Administration/customer point of view:
The IT services managers of the Public Administration can constantly monitor the quality level of the provided services, according to real and updated data. In this way, the customer pays for the services that are actually supplied, instead of paying for the professional work of the development team. - From the systems integrator/supplier point of view:
The supplier can improve the efficiency of its organization, by identifying the existing systems inefficiencies and some areas of improvement, making it benefit from the consequent reduction of the organizational costs
Lessons learned
The project provided evidence on clear benefits on the measurement process. The Public Administration pays particular attention to the low-cost tools (OSS) for a better monitoring and control both at the project and organizational level. The right combination should include both methods and tools.
Sometimes it could be difficult to obtain a good quality of data extracted from the tools that support the ICT service and perform the actions for process improvements. Time pressure, workload and urgent activities may encumber the team to provide sufficient effort to the Software Process Improvement activities, so continuous commitment is needed.
Future Plans
Spago4Q is a platform developed and managed over time both by a company and a community, so that its development is continuous and progressive. In addition, its flexibility and easy adaptability to the requirements of users make it particularly suitable to projects such as those described here, that naturally evolve over time both in terms of expansion of the operational and application infrastructures to be monitored, and of typical change over time in terms of technical and functional characteristics.
Acknowledgments
The study was based on information presented by Davide Dalle Carbonare during the “Open Source: Its place in cross-border environment” workshop organised by the European Commission in Brussels, and on extensive material received from Davide Dalle Carbonare, Sergio Oltolina, and Gabriele Ruffatti of Engineering, the company that supports Spago4Q and the rest of the SpagoWorld Open Source initiative. Spago4Q is hosted by the OW2 Consortium.
Links
- www.spago4q.org
- www.spagoworld.org
- www.ow2.org
- www.qualipso.org
- www.ccos.it
- www.digitpa.gov.it
- www.regione.veneto.it
