Three French government ministries have pooled resources to build a digital archiving software to secure and maintain digital documents. The software, called Vitam, is an open back-end which could be used by any administration. It makes use of new technologies to promote re-use. Agility is a priority in keeping the project aligned with business needs and ensuring that it remains economically viable.
The origins of Vitam
The French government has put in place an ambitious e-government policy under which more and more administrative procedures will be digitalised. French administrations need to manage a growing volume of native digital documents that now have a legal value in their own right. They must be preserved accordingly, and their evidential value maintained.
In 2011 the French Ministries of Foreign Affairs and International Development (MAEDI), Culture and Communication (MCC), and Defence began discussions on the development of a common archiving solution. These administrations have important needs for long-term and permanent archiving. The Ministries of Defence and Foreign Affairs have to maintain their own critical archiving processes and sensitive data (active, semi active records and archives). The Ministry of Culture, meanwhile, manages inactive records with permanent value for France’s other ministries.
“On their own, none of the ministries could finance a permanent archive on this scale,” says Jacques Marzin, the former government CIO, on the SGMAP website. “This explains why we managed to build such a collective project that is so important for the nation.”
Twelve market solutions were tested in an effort to find the right solution. However, none of them met all the needs defined by the Vitam project teams, a presentation reveals (PDF).
Vitam : five projects
Vitam is managed by DINSIC and the Interministerial Committee for Archives (CIAF). Vitam is composed of five distinct projects:
• Develop a digital archiving free software capable of supporting large volumes of documents;
• Deploy the solution in the three ministries and adapt it to their respective archiving processes and rules. The three projects are:
• Promote the re-use of the solution by other administrations through a project called AD-ESSOR. This is supported by the Ministry of Culture and the Interministerial Committee for Archives. The goal is to make installation as easy as possible and encourage its adoption by other administrations. In due course, some new software components will be added and detailed documentation is already published, says Jean-Séverin Lair, the program’s director.
Vitam : the features
Vitam is intended to be a technological back-end that provides all the essential functions for archiving. Administrations who use Vitam are responsible for their own front-ends and must therefore use Vitam with their own archiving rules, explains Jean-Séverin Lair. Administrations are also responsible for the IT infrastructure needed to support Vitam. A development platform with 400 virtual machines and more than 100 TB of storage is currently hosted by the Ministry of Agriculture.
Vitam provides functions to manage the entire lifecycle of every document (active, semi-active and inactive stages from short-term to long-term archiving), as required to meet all the use cases identified by administrations in France. The type of archiving chosen for a particular document depends, for example, on how frequently it is accessed and the acceptable delay, Lair explains. Active and semi-active stages suits documents that require regular access and therefore a rapid response from the archive system. Permanent archiving is for documents that are accessed more rarely, and hence do not require high speed. Vitam must also guarantee that the evidential value of every archived document is preserved.
The basic functions of Vitam span the ingestion of a document, its conservation (or destruction), and its maintenance. These functions are:
“Vitam has been developed to support large volumes of documents, so it can address both large administrations and groups of smaller communities,” says Lair. The platform will be able to manage petabytes of data.
Vitam has been designed to be used via web services. Document ingestion, metadata searching and document access are accessible via RESTful APIs. Administrations can integrate these into their own applications, such as messaging or human resources (HR) systems.
Administrations can participate in:
Agility and open source
“Vitam is entirely composed of open source software,” says Lair. 14 modules have been developed by Vitam’s development teams, and 70,000 lines of Java code have been produced. According to Lair, the application relies on open source libraries and uses only two packaged applications: MongoDB, a NoSQL database which stores the metadata, and ElasticSearch, used to search metadata.
The source code was published on GitHub on January 16, 2017 under a CeCILL licence, a French and standard licence for free software created by the three research organisations CEA, CNRS and INRIA . “We are committed to updating it,” Lair says.
According to Lair, free software was a “natural” foundation for this project, and Vitam “would not have been possible without the use of free software”. Openness encourages the reuse of the solution and its adoption by other administrations. It also makes customisation easier.
“Open source was the best way to share the developed software, and thus to make the financial effort of the French government profitable. Open source offers a wide range of innovative technologies useful for developing the solution. Finally, it is also the best way to extend the user community and diversify the needs and the functionalities. It also facilitates the maintenance and durability of the software solution”, Lair adds.
Lair explains that the program makes extensive use of agile techniques to respond quickly to the needs of administrations. Being agile helps to prevent delays in the progress of the project and ensures that “the deliverables remain aligned with the initial needs”, explains the SGMAP website.
The resources contributed by the three ministries to the Vitam development team include IT personnel, archivists and other business experts. The interministerial team currently has 14 members, and will eventually grow to 60 people.
Agile methods allow:
16 million euros were allocated to the development of the Vitam software. In total, the project will cost 56 million euros (resources and all ministerial projects included), Lair explains.
The tender was divided into several lots, separating the development and integration processes to optimize profitability, according to the SGMAP site. This separation makes it possible to include a broad scope of commercial suppliers, “especially SMEs”.
Development was awarded to HIT-Part, Linagora and Smile, and integration and software quality to Thalès. Smile was also included in the tender for some development expertise
Vitam’s roadmap is: