DT2: Service-based data access

One additional Indikation: When we transfer the HTTP/KVP service description into an external File, it becomes more complicated for the provider as he has to create and process this file on a web-accessible endpoint. This would speak for a description within the DCAT-AP representation. But then we need to develop (or use -if somewhere already existing) such an RDF-based representation.

During the "Smart Descriptions and Smarter Vocabularies" (SDSVoc) workshop, we had an ad hoc session on service / API-based data access.

I'm now going to include here the summary of that session, and some off-list email exchange following it.

Chair: Andrea Perego

Participants:

• Makx Dekkers
• Matthias Palmer
• Christian Mader
• Simon Dutkowski
• Uwe Voges
• Phil Archer (for some time)

Andrea: [Summarising the problem, starting from what proposed in the DCAT-AP IGs - see https://www.w3.org/2016/11/sdsvoc/SDSVoc16_paper_27#modelling-service-api-based-data-access]

Makx: Recap: we have files and endpoints/API - so this is about the distribution type. The type may be more than one.

All: [Discussion on the type of distributions from the OP's MDR and how to use them: http://publications.europa.eu/mdr/authority/distribution-type/ ]

Uwe: It may be not so straightforward whether it's a file or a service

Andrea: The distinction can simply be based on what we get back, a file or a query interface

All: [Discussing again about the distribution types from the OP's MDR: fine with downloadable file, fine with visualisation]

Matthias: It may be good to provide information on the "type of service", e.g., REST or SOAP - which is a different thing from the specific service interface - e.g., SPARQL, WMS, WPS

Makx: An option is to use the format, maybe

Simon: What we are still missing is how to query the service, and the relevant dataset

Makx: We have 4 levels: distribution type, service type, link to a service description, and a description of how to instantiate the service.

Simon + Matthias: [discussion about the need of being able to parametrise the call to the service to get the relevant dataset]

Matthias: I would also care having a textual, human-readable description.

All: [No problem to have it, but we have to decide whether to have also a machine readable description]

Phil: I think this came up frequently, and the issue is that there's no standard way of specifying the relevant data subset.

All: [Discussion on the possibility of using URL templates to specify parameters as a general approach - see https://tools.ietf.org/html/rfc6570]

Andrea: Trying to make the point:

1. We agree to have the distribution type (dct:type) - see MDR NAL: http://publications.europa.eu/mdr/resource/authority/distribution-type/
2. We agree to have a free text description to inform humans this is not a file and, possibly, how to use the service / API. A possibility is to use dct:description
3. We agree to have the "service macro-type" (we can also call it "service category" or "service protocol") - SOAP, REST, etc. - but we have to decide how to model this.
4. We agree to have the "service type" (dct:conformsTo) - WMS, SPARQL - see, e.g., https://github.com/OSGeo/Cat-Interop/blob/master/LinkPropertyLookupTable.csv
5. We agree to have the template URL but (a) we need a specific property for this and (b) this does not address POST requests
6. We agree to have the information used to instantiate the template URL - to be investigated how this information can be found / derived

Barcamp session closes. Results reported to the plenary by Matthias.

Mail from Uwe Voges (6 Dec 2016):

The only point I have is, how can a client detect if the response of a request send to the URL includes already the data or just the service description (an indirection to the data): e.g. a SOAP-based GetProduct request already provides the data while a SOAP-based GetCapabilities requests returns a service description including a link to a WSDL doc - both of the same service type "OGC EO Order 1.0") ? 

Mail from Matthias Palmér (6 Dec 2016):

I think that a distribution provides a way to get to the data of a dataset. If it is something that is downloadable, the distribution tells you about the format of the data and how to get to it (the downloadURL). In the same sense, for a service the distribution should tell you about the format of the data AND how to get to it. The latter includes the service endpoint as well as some information that helps you to use it (invoke it). The focus should never be solely on the service, the intent is to provide a way to get to the data.

So I think we better examplify, I am a bit unsure, but I guess a GIS dataset could be provided both via a VMS and a WFS service. If so I would argue they correspond to two distributions of the same dataset, e.g. getting the information in a structured format vs getting the information as images. 

(I am a bit new to the geo domain, so you have to pardon my ignorance here.) 

Assuming this is correct, an example of a WFS service is outlined:

Distribution type: http://publications.europa.eu/resource/authority/distribution-type/WEB_…

Service type: SOAP  (we have to introduce a vocabulary here)

Access URL:  ?   (*)

Download URL: -

Format: application/json,text/csv,application/gml+xml  (**)

URITemplate: http://example.com/wfs?service=wfs&version=2.0.0&request=GetFeature&typ…{featuretype}&outputFormat={format}

Invocation information: http://example.com/wfs?service=wfs&version=1.1.0&request=GetCapabilities

Conforms to: http://www.opengis.net/def/serviceType/ogc/wfs/

(*) Provide only if there is a nice webpage describing how this dataset is accessed via the WFS service.

(**) Assuming the requirement on a single format has been lifted

After writing this, I am starting to doubt the need for a "invocation information" property, maybe we can use the access URL for this anyway.

The logic would be quite clear:

• If the distribution is downloadable, use downloadURL.
• If the distribution is provided through a service, use the URITemplate.
• In both cases accessURL can be used to provide more assistance for accessing the data.

Mail from Uwe Voges (7 Dec 2016):

From a formal point of view I guess there is no difference between a view service and a webService (as we try to differentiate): a getMap-request (a view-service) just returns another format (e.g. geoTiff) as getFeature-request (e.g. GML in some flavor).

The question for me is to which endpoints shall the distribution point: the (data)service-URL (e.g. the GetMap-request of a OGC WMS) AND/OR the serviceDescription-URL (e.g. the GetCapabilities-request) OR to something we specified on the SEMIC meeting in Rome: the URL of an OpenSearch-Description docment (OSDD) embedding one or both of the URLs described before). In any case we need to define what the response format is (e.g. geoTiff (for GetMap), Capabilities-Doc (for the GetCapabilities) or application/opensearchdescription+xml (for the OSDD)*). So we would maximally need two format: that of the data (returned after invoking the service) and that of the service-URL (which may in some case be the same (when it is already the link to GetMap)

* in OSDD we could provide more than one URL with different formats. But the problem with OSDD is that for every service a separate OSDD must be created, which may not be preferred by people…

I propose to provide one or more service endpoints having a different response formats.

Example:

• GetMap / geoTiff (correct mimeType to use here)
• GetCapabilities / OGCCapabilities (correct mimeType to use here)