OpenSearch API
Terrascope offers an OpenSearch API that enables users to search for and access the EO data products available on the platform. Based on the OpenSearch standard, this API provides a straightforward and flexible method for querying the catalogue for collections and products. Accessible via HTTP GET requests, the API returns search results in GeoJSON format. It can be utilized by various clients, from basic web browsers to more sophisticated portal implementations.
The OpenSearch Description Document
The OSDD (OpenSearch Description Document) outlines the web interface of the Terrascope OpenSearch. It provides necessary information about allowed values, maximum and minimum values, and patterns for search parameters. This helps in creating a compatible interface for client applications without requiring prior knowledge of the interface.
The Terrascope Catalogue OpenSearch employs a two-step approach: the initial search is conducted at the collection level, followed by a detailed search at the product (granule) level.
The OSDD response includes:
- Template for the Collection Search Request with all available collection search parameters.
- The allowed values for the collection search parameters.
Collection specific OSDD
The collection-specific OSDD, for example, this link provides a link/template to the product level search. Where collectionID is parentIdentifier (URN) for the VITO EO data collections.
The collection-specific OSDD response includes the following:
- Template for the Product Search Request with all available search parameters.
- The allowed values for the product search parameters.
Collection Level Search
With this search, the client can request a list of available collections according to the search criteria. The search request can be made by issuing an HTTP GET call on the Collections endpoint, which is available in the following OpenSearch URL template:
https://services.terrascope.be/catalogue/collections.geojson?httpAccept={sru:httpAccept?}&recordSchema={sru:recordSchema?}&count={os:count?}&startIndex={os:startIndex?}&startPage={os:startPage?}&title={eo:title?}&denominator={eo:denominator?}&distanceValue={eo:distanceValue?}&distanceUOM={eo:distanceUOM?}&resolution={eo:resolution?}&specificationTitle={eo:specificationTitle?}&specificationDate={eo:specificationDate?}&specificationdateType={eo:specificationdateType?}°ree={eo:degree?}&topicCategory={eo:topicCategory?}&keyword={eo:keyword?}&abstract={eo:abstract?}&organisationName={eo:organisationName?}&organisationRole={eo:organisationRole?}&lineage={eo:lineage?}&useLimitation={eo:useLimitation?}&accessConstraint={eo:accessConstraint?}&otherConstraint={eo:otherConstraint?}&classification={eo:classification?}&start={time:start?}&end={time:end?}&trelation={time:relation?}&uid={geo:uid?}&productType={eo:productType?}&platform={eo:platform?}&platformSerialIdentifier={eo:platformSerialIdentifier?}&instrument={eo:instrument?}&sensorType={eo:sensorType?}&compositeType={eo:compositeType?}&processingLevel={eo:processingLevel?}&lat={geo:lat?}&lon={geo:lon?}&radius={geo:radius?}&name={geo:name?}&bbox={geo:box?}&geometry={geo:geometry?}&grelation={geo:relation?}&sortKeys={sru:sortKeys?}&q={os:searchTerms?}&clientId={referrer:source?}&accessedFrom={referrer:accessedFrom?}
Example:
GET https://services.terrascope.be/catalogue/collections?resolution=10When no parameters are included in the request, the complete list of accessible collections is returned. The parameters and possible values are detailed in the OSDD response and depend on the user’s authorization. Only collections and corresponding parameters accessible to the user will be included in the search results.
The collection-level search response can be provided in various formats, such as GeoJSON (standard), Atom, etc., and includes:
- General info, e.g. total results, startIndex, itemsPerPage, ….
- Information for each entry: * General info, e.g. title, id, summary, etc. * Link to the VITO collection with relation ‘alternate’ and type ‘text/html’ * Link to the collection INSPIRE metadata
Product Level Search
With this search, the client can request a list of accessible products within a collection according to the search criteria.
The search request can be done by issuing an HTTP GET call on the Products endpoint, which is available according to the following OpenSearch URL template:
https://services.terrascope.be/catalogue/products.geojson?collection={collection}&...
The exact URL template varies based on the collection definition passed to the parentIdentifier. This definition can be found in the collection-specific OSDD for the given collection.
The product level search response can be in different formats, e.g. GeoJSON (standard), atom, etc. and includes:
- General info, e.g. total results, startIndex, itemsPerPage, etc.
- Information for each entry:
- General info, e.g. title, id, summary, etc.
- Link to the product INSPIRE metadata
- Data access URLs
- Product metadata
Cancelled Products
Products can be deleted from online storage to comply with the Terrascope rolling archive policy for Sentinel-derived products. However, product metadata remains available in the catalogue. It can still be discovered using the OpenSearch API by filtering the productionStatus parameter. Deleted products can be found by using productionStatus=CANCELLED.
Example: .../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_TOC_V2&productionStatus=CANCELLED
Product Download
Downloading the products from the Terrascope platform is not necessary. The data is available locally on the user’s VM and Notebooks.
The product download documentation contains more information on how to download data when not working on the Terrascope platform.
Catalogue Polling / Subscription
Web clients can implement a polling mechanism using the Catalogue OpenSearch API to identify newly ingested products for a specific collection. However, the catalogue does not support a subscription mechanism that was available in the previous PROBA-V Product Distribution Portal. Therefore, web clients should use the OpenSearch API to search for newly available products periodically.
Two search parameters used for the polling are:
- modificationDate: this is a dateTime linked to the date and time of a significant update of the metadata resource.
- publicationDate: this is a dateTime linked to the date and time of the first availability of the metadata resource.
Examples of polling request:
https://services.terrascope.be/catalogue/products?collection=urn%3Aeop%3AVITO%3ATERRASCOPE_S2_CCC_V2&modificationDate=%5B2020-05-01T00%3A00%3A00Z%2C2020-05-11T00%3A00%3A00Z%5B ==> response lists all Sentinel-2 CCC V2 products modified between 01/05/2020 at 00:00 UTC (dateTime included) and 11/05/2020 at 00:00 UTC (dateTime excluded).
https://services.terrascope.be/catalogue/products?collection=urn%3Aeop%3AVITO%3ATERRASCOPE_S2_CCC_V2&publicationDate=%5B2020-05-01T00%3A00%3A00Z ==> response lists all S2 CCC V2 products modified after 01/05/2020 at 00:00 UTC (dateTime included).
Applicable standards
The TerraCatalogue OpenSearch implementation applies to the following standards:
- OpenSearch
- OGC- 10-032r8 OGC OpenSearch Geo and Time Extensions
- OGC 13-026r9 OGC OpenSearch Extension for Earth Observation
The OpenSearch query parameters are aligned with the Earth Observations Metadata profile of Observations and Measurements (O&M EOP): OGC 10-157r4 Interface development is based on the CEOS OpenSearch Best Practice Document.
The GeoJSON implementation is based on the following standards:
- eo-geojson - OGC EO Dataset Metadata GeoJSON(-LD) Encoding Standard - Overview
- OGC-17-003r1 – OGC EO Dataset Metadata GeoJSON(-LD) Encoding Standard
- OGC-17-047-D8 - OpenSearch-EO GeoJSON-LD Response Encoding Standard
- OGC-17-084-v0.9.0-D2 - GeoJSON EO Collection Metadata Encoding Standard
OpenSearch API in practice
Table 1 provides examples to help users get started with the Terrascope Catalogue OpenSearch API. Clients can use services ranging from simple web browsers to complex aggregated portal implementations. The standard response format for OpenSearch request examples is GeoJSON:
| OpenSearch Description Document | OpenSearch Description Document |
|---|---|
| OSDD with atom response | .../catalogue/description.xml |
| OSDD for SENTINEL-2 TOC V2 data | .../catalogue/description?collection=urn:eop:VITO:TERRASCOPE_S2_TOC_V2 |
| Collections Search | Collections Search |
| Discover all collections (first 100) | .../catalogue/collections |
| Discover all collections (next 100) | .../catalogue/collections?startIndex=101 |
| Discover all collections with atom response | .../catalogue/collections.atom |
| Discover all collections with e.g. resolution 10 m | .../catalogue/collections?resolution=10 |
| Discover all SENTINEL-2 collections | .../catalogue/collections?platform=Sentinel-2 |
| Products Search | Products Search |
| Discover all SENTINEL-2 FAPAR Products V2 | .../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_FAPAR_V2 |
| Discover all SENTINEL-2 FAPAR Products V2 with atom response | .../catalogue/products.atom?collection=urn:eop:VITO:TERRASCOPE_S2_FAPAR_V2 |
| Discover all SENTINEL-2 LAI products V2, between 05/01/2020 and 15/01/2020 for the bounding box -180, 0,180, 90 (=Northern Hemisphere) | .../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_LAI_V2&start=2020-01-05T00:00:00&end=2020-01-15T00:00:00&bbox=-180.00,0.00,180.00,90.00 |
| Discover all SENTINEL-2 TOC products V2, for a specific polygon | .../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_TOC_V2&geometry=POLYGON((0 50,40 50,40 20,0 20,0 50)) |
| Discover all SENTINEL-2 TOC products V2, for a specific tile | .../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_TOC_V2&tileId=32TPN |
| Discover all SENTINEL-2 CCC products V2, between 01/01/2020 and 31/01/2020 with cloudcover between 6% and 17% (both included) | .../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_CCC_V2&start=2020-01-01T00:00:00&end=2020-01-31T23:59:59&cloudCover=[6,17] |
| Discover all SENTINEL-2 TOC products V2, between 01/01/2020 and 31/01/2020 over Europe | .../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_TOC_V2&start=2020-01-01T00:00:00&end=2020-01-31T23:59:59&name=Europe |
| Filter only the 10M resolution Products from SENTINEL-2 FAPAR V2 Important remark: filter on resolution is not possible for Sentinel-2 TOC products, as one TOC product combines files with different resolutions (10 m, 20 m and 60 m) |
.../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_FAPAR_V2&resolution=10 |
| Direct Product Search | Direct Product Search |
| A direct product search can be performed using the unique identifier of the product, together with the collection parent identifier |
.../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_CCC_V2&uid=urn:eop:VITO:TERRASCOPE_S2_CCC_V2:S2A_20221201T082311_34HCH_CCC_20M_V210 |
| Cancelled Products Search | Cancelled Products Search |
| Discover all cancelled SENTINEL-2 TOC products V2 Cancelled products are no longer available online, but the metadata is still available in the catalogue and can be discovered |
.../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_TOC_V2&productionStatus=CANCELLED |
| Products Search by Nominal Date | Products Search by Nominal Date |
| Using start and end parameters will search on beginningDateTime and endingDateTime. If you want to select on nominal date, you can use the Lucene query parameter q. For example: q=properties.date:2020-03-25 or q=properties.date:[2020-03-25 TO 2020-03-30] |
.../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S1_SLC_COHERENCE_V1&q=properties.date:2020-03-25 |
| Product Search with direct access links in the response | Product Search with direct access links in the response |
| An application that runs on the Terrascope MEP, can access the products directly and request for direct access links instead of download URLs | .../catalogue/products?collection=urn:eop:VITO:TERRASCOPE_S2_TOC_V2&start=2020-01-01T00:00:00&end=2020-01-31T23:59:59&accessedFrom=MEP |