Why interoperability testing?
The continuous growth of information exchanges between public administrations (across borders and sectors) gives a high need for interoperability in Europe. The ISA² interoperability test bed (Action 2016.25) provides generic testing facilities to initiatives and public administrations that create interoperability solutions in a cross-border context or linked to European Legislation. Achieving interoperability is a challenging task, considering the range of different technologies that are used, the diverse organisational structures they support and the varying semantics that may be applied to the exchanged information.
This testing challenge is in fact twofold, on the one hand on the system under development that needs to run interoperability tests as early as possible in its elaboration phase, and on the other hand on service providers or peer systems that need a means to expose IT services for testing without impacting their production system operations.
What is the ISA² interoperability test bed?
The ISA² interoperability test bed is a set of hardware and software resources offered as a central, standards-based reusable service, to facilitate interoperability and conformance testing. Examples of its supported use cases would be:
- Simulation of a web service for clients to test against.
- Validation of content sent through various channels.
- Conformance testing against a message exchange protocol.
- Testing of an entire message exchange choreography.
The test bed allows users and systems to connect for the execution of test cases against simulators or reference implementations of specifications that are transparently hosted on its infrastructure. In addition it offers a test registry and repository (TRR) to store test artefacts (assertions, test cases, validation schemas etc.) and federate test services (validation services, simulator services etc.). The centralised test bed service is cloud-based but separate instances can also be easily ran on premise if such a requirement exists.
What should it not be used for?
Given that the test bed’s focus is interoperability and conformance testing there are some cases that it is less suited for:
- Functional or regression testing: The test bed's focus is on interoperability, not finding internal bugs.
- Performance testing: The test bed aims for configurability and traceability of test sessions, not the automated generation of traffic for stress testing.
- Penetration testing: To enhance connectivity and ease of use, security is relaxed and exchanged data is exposed.
How do I use the test bed?
Using the testbed can be summarised in three steps:
(1) A user logs onto the test bed platform and selects one from a set of reusable test cases.
(2) The test case executes, testing the user’s system against simulators and validators. Tests can range from the verification of complex message exchanges as complete conversations, to validation of content, received through a variety of communication channels.
(3) During execution and upon completion, exchanged messages and validation results are displayed and compiled in a test report that is stored for later reference.
The picture below shows a screenshot of an executed test case. It visualizes the actions done by the user and the Test engine, and it gives for each step a report of the results.
The main users of the test bed are initiatives such as public administrations, projects or other, which have relevance to interoperability. They can make use of the test bed by having their reference implementation and test suites installed on it, so it is ready for their community to be used. Anyone who has a relation with this community can then test their software against these test suites.
Anyone, including someone not eligible to directly use the ISA2 test bed service, is free to download the test bed software and install a local instance in their own premises.
Where to find the test bed:
In order to start using the test bed there are some options:
There is an online test bed accessible via the link below. It is free for everyone to register on the test bed and make use of its services.
-> The Interoperability Test Bed (GITB)
You can install a version of the test bed on your own premises. To do so it is recommended to follow the installation steps explained in the document “Introduction to the ISA test bed”. This document can be found under the tab “Download releases” of this action.
For more information or customized versions, contact us via this e-mail address (with mentioning of "ISA test bed").
Presentations, leaflets and extra information:
The GITB Test Description Language (TDL) Controlled Vocabulary