C-READ Management System, an SDI for climate change monitoring
The C-READ system has been developed by GeoSolutions for the Caribbean Community Climate Change Centre (CCCCC or 5Cs) leveraging on several Open Source components to provide SDI functionalities for climate change monitoring, allowing scientists and users in the Caribbean area to upload and fuse historical as well as near real-time data in a single infrastructure.
The complete architecture envisages a presentation node (hub) that is the front-end for the final external users; it provides both a search interface for searching into the catalog, and a map viewer which will display the desired geographic data. The data node is the main node which contains all the data added to the system. It provides the entry point both for data publishers, where new data can be added using a GUI, as well as for external processes that can send data to the system for automatic publishing. Processing nodes are external nodes that can process data; Human attended and unattended nodes are available and are implemented with different software. The whole system takes also advantage of other external nodes providing data for publication; such data is harvested directly into the presentation node, an example is represented by the CCCCC Clearinghouse which is a custom software developed by 5Cs for sharing documents and articles with the external users. An high-level diagram of the infrastructure is presented here below.
Before describing more in detail the infrastructure developed it is worth to point out that all the components used are Open Source and that extensive training and webinars have been provided to the customer. Here below a more detailed diagram of the C-READ system along with the used products.
Presentation node (hub)
The hub gathers all the metadata distributed in the various C-READ nodes and present them to the users as a single catalog view. The hub offers the user a front-end toward the geographic data and documents that have been uploaded or exposed via harvesting to the infrastructure. It is implemented leveraging on the CKAN Open Source data portal software. It is worth to point out that this is the main component that external users (or better, users which are only data consumers) need to have access to; this node is only an aggregation node: no data will ever be uploaded by users on this node. All of the presented data and metadata are harvested from other nodes.
The main node that is harvested is of course the C-READ data node (see below), but other sites can be harvested as well, provided they expose an entry point using a known protocol like OGC CSW or the CKAN API for instance. Nodes supporting the OGC CSW protocol can be harvested using the CSW harvester provided by the official spatial extension. Harvesters for other resources can be developed using the CKAN API likewise it has been done harvesting reports and other documents from the CCCCC Clearinghouse.
A processing node creates resources as the result of processing and sends them to the data node for publishing. A processing node only runs processing services; it will not run services to disseminate data and metadata to the public. Once the product is ready, it will be transferred to the data node as part of the processing script, where it will be published.
There are two different kinds of processing nodes: automated and attended.
Automated processing node
Automated processing nodes are implemented using the Jenkins Open Source software. Jenkins is a web based tool for running and monitoring the execution of custom scripts. Jenkins executes scripts which extract data from known sources, transforms them in data publishable by the CREAD system and sends the data to the data node.
Script execution can be triggered on a time basis, hence only administrators need to access the Jenkins site to install new scripts and monitor the processes, since processed data can be viewed by the end users on the hub once it’s ready.
Attended workflow node
Attended processing is implemented leveraging on QGIS, a desktop GIS application. By the use of a simplified interface for running workflows, users can be taught the basics of GIS processing. The implemented workflows also enables users to create new layers in the data node.
The data node is the core of the whole C-READ system as it contains all the data and related metadata of the published resources and provides OGC services to retrieve the data using standard protocols; this node is implemented using the Open Source component called GeoNode which internally uses the well-known Open Source component called GeoServer.
GeoNode provides some user friendly functionalities around GeoServer features, serving also as a local catalog and allowing users to upload non geographic data. GeoServer is the “geospatial engine” behind GeoNode, offering a wide set of OGC services.
At the time of writing this post (early December 2015) the infrastructure has become operational and the last finishing touches are being made.
Data is ingested in near real-time from external sources via the automated processing nodes (e.g. sea surface temperature models from JPL lab are pulled and ingested daily) but we expect that more data will be ingested over time using the attended processing (i.e. QGIS).