Over the last year, Bulgaria has published 80 datasets on its open data portal, and with more to come. In January, the Council of Ministers accepted a list of 119 prioritised datasets that are important to open up first. We have set four deadlines for this year, when the datasets should be publicly available on the open data portal, all depending, of course, on the importance of the specific dataset and how ready the responsible agency is in technical terms.
The European PSI Directive is currently being transposed into Bulgarian legislation. Specific to the Bulgarian transposition of the PSI Directive will be the obligation for public administrations to prioritise data for publishing each year. So this government-driven prioritisation and publication of datasets will be an ongoing process.
Policy Context
Pilot project
The Bulgarian open data portal was announced in September 2014. It was originally launched by the Obshtestvo.bg Foundation, a non-governmental organisation whose members call themselves independent creators of electronic tools for fast and easy connecting government and citizen
.
The portal was born as a pilot project that we did last autumn for the interim government of Bulgaria,
explains Dimitar Dimitrov, a Bulgarian software developer who has been involved in civil activities for quite some time. For this project we had to:
- examine the existing open data platforms and pick one;
- find a few existing published datasets we could download and import into the portal, preferably from different agencies; and
- create a small demo application that uses these datasets, preferably combining two or more to show some added value.
It was a pro-bono initiative with the purpose of demonstrating what open data can do in return for a relatively small effort.
The portal was initially available at the address data.obshtestvo.bg,
Dimitrov explains. The Interim Deputy Minister at the Ministry of Transportation, Telecommunication and Information Technologies wanted to do this project, but since he was a member of an interim government there was little time and budget. So out of necessity it was purely for demonstration purposes when we started.
Priority list
The open data initiative is under the Deputy Prime Minister for Coalition Policy and Public Administration and the Directorate for Modernisation of the Administration under the Deputy Prime Minister,
says Kalina Cherneva, Adviser to the Cabinet of the Deputy Prime Minister. We have a team here at the Council of Ministers that is very IT oriented. There is also an expert at the directorate who communicates with the public agencies on a daily basis with regard to technical support.
In January, the Council of Ministers accepted a list of 119 prioritised datasets that are important to open up first. We have set four deadlines for this year — in March, June, September and December — when the datasets should be publicly available on the open data portal, all depending, of course, on the importance of the specific dataset and how ready the responsible agency is in technical terms.
We currently have about eighty datasets published on the portal. We work together with the public agencies on the quality of the data they make available. Interestingly, some agencies have even published datasets that were not on the list, on their own initiative,
Cherneva says.
PSI Directive
The European PSI Directive is currently being transposed into Bulgarian legislation,
Cherneva continues. This amended 'Access to Public Information Act' is now before the National Assembly.
This year, the Council of Ministers has set the priority list of 119 datasets to be published, in an open machine-readable format, together with their metadata. Specific to the Bulgarian transposition of the PSI Directive will be the obligation for public administration to prioritise data for publishing each year. So this government-driven prioritisation and publication of datasets will be an ongoing process. Furthermore, starting October 1st, public agencies will have to keep their information in an electronic format.
Description of the way to implement the initiative
New government
After the elections last autumn, the new government has proven to be very keen on open data. According to Dimitrov, Anton Gerunov, the Head of the Political Cabinet of Deputy Prime Minister Rumyana Bachvarova, has placed the topic very high on his agenda. The Council of Ministers has established a small team working to overcome resistance within the agencies and help them to extract and cleanse the data from their databases.
The new government wanted to make the existing open data portal the official one, so we migrated what we had done for the demo project to a virtualised government infrastructure and made it available at the opendata.government.bg address. Gerunov and his team did not want to wait several years for a full-blown government tender to get a contractor to set up a new open data portal. The political situation in Bulgaria is volatile, so there is a chance that the current government and this team will no longer be in office at the end of the year. So they want tangible results before then.
So this now has become the official open data portal for Bulgaria. Although it is owned by the government, i.e. the Council of Ministers, we are still doing the maintenance of the system. Most likely, we will also be involved in the further evolution of the portal.
Supreme partnership
We have the honour to have a very active civil society and NGO sector,
says Cherneva. Some of the organisations are very active in open data specifically. So I would rate this partnership as supreme. The open data portal was originally created in October 2014 as a demonstration platform. Currently, the national platform opendata.government.bg has eighty datasets published. The Obshtestvo.bg Foundation also helped us immensely to troubleshoot initial problems.
We have two IT advisers at the Council of Ministers who are helping the public agencies with more complex queries to extract data from their databases. The agencies can often do this themselves, since they are obviously managing their own databases. Sometimes they might need some technical help — specifically with converting the data to an open, machine-readable format, for that is a specific field of expertise.
Technology solution
Free software
The Bulgarian portal is built on the popular open data platform CKAN, available as open source software under the Affero GPL license. It uses the Nginx web server, a PostgreSQL database and the Java-based Apache Solr full-text search engine, all running on a Debian-based Linux server.
We are using a plain vanilla CKAN installation,
Dimitrov notes, with the DataStore, the DataPusher and some other extensions activated. Last fall, we translated the software into Bulgarian, so that is now part of the package. And we created a custom visual theme for the portal, which is available for re-use by others.
Open source
At first, the government was pretty skeptical about open sourcing their stuff,
says Dimitrov, but that is the way we want to operate here at Obshtestvo.bg. So we fought for that. We have actually made a lot of progress on that front, but we have a lot more to go before they start asking this from contractors and before open source will become a standard practice.
Main results, benefits and impacts
Published datasets
At the moment of writing, 80 datasets from 33 public agencies have been published on the Bulgarian open data portal. The organisations involved were told to do so by the Council of Ministers.
Some of the public agencies that have contributed to the portal are:
- Geodesy, Cartography and Cadastre Agency (AGCC), part of the Ministry of Regional Development and Public Works,
- Bulgarian Institute of Metrology,
- Bulgarian Posts,
- State Agency for Bulgarians Abroad,
- Executive Environment Agency (EEA), part of the Ministry of Environment and Water,
- Ministry of Economy,
- Council of Ministers,
- Bulgarian National Statistical Institute, and
- National Revenue Agency, part of the Ministry of Finance.
As far as Dimitrov knows, no new projects were set up to make this possible. What has been done so far was within the current budgets and capacities of the participating agencies,
he says.
Resistance
The ambition is to publish another few dozen datasets before the end of this year,
says Dimitrov. These datasets were selected as being the most useful with regard to the public interest and the easiest to get, meaning that the public agency already has the data available in a decent database and that it can easily be exported.
This will require a lot of work from the public agencies involved, in administrative efforts as well as in technical capabilities. Agencies often resist the process, complaining that they do not have the capacity or technical ability to extract and publish the data, or to filter out personal information before publishing. These obstacles and objections are being addressed by the open data team, answering questions and sometimes even visiting an IT department to help people there with their SQL queries.
We will probably create a software tool to act as a temporary data pusher between internal software systems of the public agencies and the open data portal. The agencies can use that to automatically publish their datasets until their own software systems provide an API (Application Programming Interface) or something like that. Out-of-the-box support for open data will be part of the requirements for any new government-commissioned software projects.
Return on investment
Volunteer-run
Until now, the Obshtestvo.bg Foundation has invested about two person-months — maybe a little more — of pro-bono work into this open data portal,
says Dimitrov. That comprises the time to research the available portal software — both paid and free — to get it up and running, customise the visual theme, upgrade it to the latest version of CKAN, migrate it to the government-operated infrastructure, fix errors and so on. We had not planned this, and since we are a volunteer-run organisation and we all have other full-time jobs, it was quite a bit of work for us. We plan to move the server to a new government-operated private cloud by the end of this year.
Future plans
Obviously, Gerunov's team has a list of features they would like to see implemented in the site,
Dimitrov continues, such as basic CMS (Content Management System) functionality and some changes to the portal itself. The CMS part will probably be done as an integration with Drupal, as the data.gov.uk team has done. But these changes are planned for a future date when there will be people officially commissioned to work on this platform, since our pro-bono work is not a sustainable solution.
Except for the visual theme, we have not made any major changes to the CKAN software, except some small URL tweaks to avoid HTTPS warnings. There are a number of issues with the portal, however, specifically related to malformed data handling and Cyrillic encoding. These need to be tackled if we — or other people — find the time or are paid to do so.
Datasets, visualisations and applications
We are planning two things we want to add to the open data portal,
says Cherneva. We want to upgrade it with an automatic mechanism for the publication of datasets. That way, public agencies will be able to publish data directly on the due day, for example, bi-monthly or weekly, depending on the dataset.
And we envision a section where citizens, NGOs and the private sector can upload visualisations, applications, and updated datasets they have been working on. Almost any large European open data portal I know has such a section.
Of course, an important part of our open data policy is the education and training of public agencies, for they are the main actors in this initiative. So we will do our best to provide that in due time.
Track record of sharing
In Bulgaria we have a great civil society and NGO sector,
says Cherneva. They are actively involved in the open data process and have built several visualisations. They are already using the data two days after it has been published, and signalling any problems that might be in the files.
This partnership with citizens, businesses and NGOs is essential to the process. Data provision and transparency are only the first steps; they lead to data analysis on the part of both public administration and society. And that leads to better-informed decision-making and better policies,
Cherneva says.
In addition to this essential control and monitoring, there is also the economic impact,
she continues. At the open data event we organised last month, for example, we also had a business panel. Their discussion showed that people are actually seeing what the economic impact of open data could be.
And, of course, the same is true for the political cabinet. Innovative services and apps — my personal favourites — are adding a lot of value. That is one of the reasons we are undertaking this initiative. And don't forget that better policies also have an important positive impact on economic growth.