The amount of data generated worldwide increases at an astounding pace - by 40% annually and will increase 30 fold between 2010-2020. Due to the segmentation of public services and due to using non-interoperable means to describe data, most of the data generated in the public sector is not available for re-use.
This action, titled 'Big Data for Public Administrations', aims at addressing the use of big data within public administrations' as a means for enabling wiser decision making. With recent technologies such as big data, data mining, social media, cloud etc. organisations have greater potentials in generating, creating and storing data, information and knowledge thus providing greater opportunities for wisdom.
The previous phases of this action, started in 2015 aimed at:
- Carrying out a landscape analysis in order to identify the requirements and challenges of public administrations in Europe and the Commission in the context of big data and data analytics, as well as the on-going initiatives and best practices in these areas.
- Launching a number of pilot projects on big data and data analytics both with European Commission services and Member States public administrations. The output of these pilots (documentation, source code) are published under this collection.
- Identifying the requirements and designing the architecture of a big data test infrastructure. The big data test infrastructure will be an operational infrastructure with analytics services and supporting tools, which will allow Member States public administrations to experiment with big data and implement their own big Data pilots. The infrastructure has been designed in close cooperation with a working group composed (so far) of 8 MSs. The results were published in Q4 2017. The actual implementation of the infrastructure has been proposed to be financed through the CEF programme, as part of the 2018 Work Programme. The proposal has been put forward as a joint effort with the European Data Portal project, already financed through the CEF programme. The European Data Portal will be one of the main sources of data to be processed and analysed on the infrastructure.
This action will continue to build upon the results of the previous phases. Specifically:
- Track 1: further identify big data and data analytics needs through cooperation with the Member States, with the aim of understanding the needed tools and services Member States' public administrations need in this domain. An additional objective is to enlarge the current working group of 8 Member States in 2018, in order to include more Member States interested in the action.
- Track 2: develop re-usable tools, or generalise existing tools already in use in the EC services or in Member States, for big data processing and text/data analytics (e.g. analysis of open public consultations), to be used by Member States in different policy domains. The tools are meant to be shared on Joinup and re-used directly by public administrations, and/or deployed on the big data test infrastructure mentioned above.
- Track 3: implement pilot projects, in cooperation with EC services or Member States public administrations, re-using the tools developed in track 2.