Why interoperability testing?
The continuous growth of information exchanges between public administrations (across borders and sectors) gives a high need for interoperability in Europe. The ISA² interoperability test bed (Action 2016.25) provides generic testing facilities to initiatives and public administrations that create interoperability solutions in a cross-border context or linked to European Legislation. Achieving interoperability is a challenging task, considering the range of different technologies that are used, the diverse organisational structures they support and the varying semantics that may be applied to the exchanged information.
This testing challenge is in fact twofold, on the one hand on the system under development that needs to run interoperability tests as early as possible in its elaboration phase, and on the other hand on service providers or peer systems that need a means to expose IT services for testing without impacting their production system operations.
What is the ISA² interoperability test bed?
The ISA² interoperability test bed is a set of hardware and software resources offered as a central, standards-based reusable service, to facilitate interoperability and conformance testing. Examples of its supported use cases would be:
- Simulation of a web service for clients to test against.
- Validation of content sent through various channels.
- Conformance testing against a message exchange protocol.
- Testing of an entire message exchange choreography.
The test bed allows users and systems to connect for the execution of test cases against simulators or reference implementations of specifications that are transparently hosted on its infrastructure. In addition it offers a test registry and repository (TRR) to store test artefacts (assertions, test cases, validation schemas etc.) and federate test services (validation services, simulator services etc.). The centralised test bed service is cloud-based but separate instances can also be easily ran on-premise if such a requirement exists.
What should it not be used for?
Given that the test bed’s focus is interoperability and conformance testing there are some cases that it is less suited for:
- Functional or regression testing: The test bed's focus is on interoperability, not finding internal bugs.
- Performance testing: The test bed aims for configurability and traceability of test sessions, not the automated generation of traffic for stress testing.
- Penetration testing: To enhance connectivity and ease of use, security is relaxed and exchanged data is exposed.
How do I use the test bed?
Using the test bed can be summarised in three steps:
- A user logs onto the test bed platform and selects one from a set of reusable test cases.
- The test case executes, testing the user’s system against simulators and validators. Tests can range from the verification of complex message exchanges as complete conversations, to validation of content, received through a variety of communication channels.
- During execution and upon completion, exchanged messages and validation results are displayed and compiled in a test report that is stored for later reference.
The picture below shows a screenshot of an executed test case. It visualises the actions done by the user and the test engine, and it gives for each step a report of the results.
The main users of the test bed are initiatives such as public administrations, initiatives or projects linked to the delivery of cross-border public services. They can use the test bed to define a conformance testing campaign involving one or more specifications, each linked to test suites and supporting services (e.g. data validators). Once in place, members of such user communities can, in a self-service manner, proceed to test their software against the defined test scenarios, supported by rich monitoring and reporting capabilities.
Cross-border projects may use a shared instance of the test bed powered by ISA². Alternatively, the test bed is also freely available for anyone to download and use either for experimentation or to run as a separate on-premise instance.
Where to find the test bed
The following options are available to start using the test bed:
- Follow our online guides to get more information on what the test bed is, how it works and what it can do for you. Alternatively you may read the “Introduction to the ISA test bed” for a summary introduction.
- Use the ISA² test bed instance accessible at https://www.itb.ec.europa.eu/itb. This is a shared instance operated by ISA² that anyone from the interoperability testing community can use (through configured user communities).
- Install the test bed yourself either to try out or to operate as an on-premise instance. The latest installation guide for the test bed is available here.
- Requests for more information and potential collaborations can be sent to DIGIT-ITB@ec.europa.eu.
Provide us your feedback
The ISA² Interoperability Test Bed Action is constantly trying to improve the Interoperability Test Bed and its supporting services to best match the needs of the interoperability testing community. If you have a few minutes to spare it would be highly appreciated if you would consider completing our feedback survey. Please don't hesitate if you are not already familiar with the Test Bed; all input helps and it could be that you also discover information that is useful to your project.
The survey is accessible at https://ec.europa.eu/eusurvey/runner/itb.
Presentations, leaflets and extra information:
Resources on the GITB Test Description Language (TDL), test services and online guides:
- The GITB TDL Controlled Vocabulary (SKOS).
- The online GITB TDL documentation.
- The online GITB test services documentation.
- The test bed's online guides and tutorials.
The Interoperability Test Bed user guides:
- For organisation users (i.e. basic test bed users).
- For organisation administrators (i.e. administrators of organisations registered for testing).
- For community administrators (i.e. administrators for sets of organisations corresponding to a project or user community).
- For test bed administrators (i.e. overall system administrators).
Resources on the GITB test bed software