GeoSolutions » Blog https://www.geosolutionsgroup.com Your one-stop-shop for geospatial open source software Wed, 08 Jul 2020 11:00:07 +0000 en-US hourly 1 https://wordpress.org/?v=4.2.28 GeoServer Bug Stump Code Sprint July 2-3 2020 https://www.geosolutionsgroup.com/news/geoserver-codepsrint-202006/ https://www.geosolutionsgroup.com/news/geoserver-codepsrint-202006/#comments Wed, 01 Jul 2020 18:56:36 +0000 http://www.geosolutionsgroup.com/?p=5366 GeoServer

Dear Reader,

GeoSolutions, along with the GeoServer community, is participating in a two days long “GeoServer Bug Fix Code Sprint”, on July the 2nd and the 3rd. The sprint also happens to overlap with the Bolsena Online Code Sprint 2020, so we’ll end up sharing some of the infrastructure and enjoying the good company of other sprinters! This sprint revives and seeks to improve the old monthly bug fix code sprint, that we used to do before. From those sprints, we learned that bugs that can be assigned to less experienced developers, because they seem like a one day fix, are hard to find. That was often coupled with lower participation of experienced developers, who need to review the pull requests for the fixes, and sit on the side (now virtually) of other developers to provide guidance.

So this time the sprint is two days long. We hope to get more tickets looked at, while allowing participant developers to tackle harder to fix problems.

Please join us on the GeoServer gitter channel, and don’t forget to go say hi to the other Bolsena sprinters on the OSGeo/Sprint channel!

Cordially,

Luis 320x100_eng]]>
GeoServer

Dear Reader,

GeoSolutions, along with the GeoServer community, is participating in a two days long “GeoServer Bug Fix Code Sprint”, on July the 2nd and the 3rd. The sprint also happens to overlap with the Bolsena Online Code Sprint 2020, so we’ll end up sharing some of the infrastructure and enjoying the good company of other sprinters! This sprint revives and seeks to improve the old monthly bug fix code sprint, that we used to do before. From those sprints, we learned that bugs that can be assigned to less experienced developers, because they seem like a one day fix, are hard to find. That was often coupled with lower participation of experienced developers, who need to review the pull requests for the fixes, and sit on the side (now virtually) of other developers to provide guidance.

So this time the sprint is two days long. We hope to get more tickets looked at, while allowing participant developers to tackle harder to fix problems.

Please join us on the GeoServer gitter channel, and don’t forget to go say hi to the other Bolsena sprinters on the OSGeo/Sprint channel!

Cordially,

Luis 320x100_eng]]>
https://www.geosolutionsgroup.com/news/geoserver-codepsrint-202006/feed/ 0
State of GeoNode 3.0 Free Webinar https://www.geosolutionsgroup.com/news/state-geonode-3-webinar/ https://www.geosolutionsgroup.com/news/state-geonode-3-webinar/#comments Tue, 23 Jun 2020 13:16:29 +0000 http://www.geosolutionsgroup.com/?p=5333 GeoNode pic

Dear Reader,

One of the core technologies in a Spatial Data Infrastructure (SDI) is a catalog. It’s like a town center where food providers and food consumers meet. Shoppers walk around and visually or by smelling they discover the food they want. They engage in a small chat and if they agree with the price they buy it. In the case of an SDI, a catalog is the virtual place where data providers and data consumers meet.  Data consumers discover resources and via rich descriptions (metadata) and previews,  they make a decision about using or not a particular data or service. However, not all organizations have implemented a catalog  and in the case of a disaster, discovery of data is still a big issue as highlighted in the recently published OGC Development of Disaster Spatial Data Infrastructures for Disaster Resilience Engineering Report.

Discovery of data is only part of the challenge. Once you find the service or dataset, you want to access it and use it. If the resource is not available via open standards, like the ones promoted by OGC and ISO, then it's like buying a piece of bread that you can only cut with one knife. It becomes frustrating and expensive to buy this kind of bread!

What if you are a data provider and in minutes you can register your data or service on the web, add metadata to it, and immediately have it available via open standards? This is what GeoNode does pretty well. It’s a content management system for geospatial, build on open standards and open source. It powers lots of Spatial Data Infrastructures in the world. For example GeoSolutions has helped install GeoNode in various countries (Malaway, Afghanistan, Mozambique, Ghana, Uganda, Nepal, and Madagascar).

The World Bank, in particular, has greatly supported its development and even published a report about the return over investment they encountered when investing in the development of GeoNode. A great blog about it is available at the World Bank web site. So, with all these said, GeoNode 3.0 was released about a month ago.  See GeoNode blog for complete details. There is usually a presentation GeoSolutions does at  the FOSS4G meeting and at  the GeoNode summit about the latest release, but unfortunately the two meetings were cancelled, so we are doing  a free webinar on July 7th at  11:00 Eastern Time. Some highlights of what’s new:

- GeoNode components have been updated to new versions: Python upgrade 3.7, Django upgrade 2.2.12, GeoServer upgrade 2.16.2.

MapStore is now the default client greatly enriching the experience for viewing the data.

- Greatly improved analytics and getting information about who is visiting the website and their actions. For example, you can see unique visitors who trigger a specific type of event.

- Improved full GeoNode backup & restore.

- Extended GeoNode shared permission via GeoLimits to restrict users or groups to a specific geographical area (see below).

[caption id="attachment_5335" align="aligncenter" width="799"]geolimits GeoLimits at work for GeoNode 3[/caption]  

I just highlighted my favorite enhancements. To learn more about GeoNode 3,0 and enjoy a 20 minutes of Q&A time with our lead GeoNode Lead Developer Alessio Fabiani, Francesco Bartoli, as well as, Giovanni Allegri and Florian Hoedt also part of the GeoNode Project Steering Committee. I cordially invite you to a free webinar on July 7 at 11:00 Eastern Time by registering at the link below!

register
Hope to see you virtually on July 7th, meanwhile stay safe and keep strong! Cordially, Luis 320x100_eng]]>
GeoNode pic

Dear Reader,

One of the core technologies in a Spatial Data Infrastructure (SDI) is a catalog. It’s like a town center where food providers and food consumers meet. Shoppers walk around and visually or by smelling they discover the food they want. They engage in a small chat and if they agree with the price they buy it. In the case of an SDI, a catalog is the virtual place where data providers and data consumers meet.  Data consumers discover resources and via rich descriptions (metadata) and previews,  they make a decision about using or not a particular data or service. However, not all organizations have implemented a catalog  and in the case of a disaster, discovery of data is still a big issue as highlighted in the recently published OGC Development of Disaster Spatial Data Infrastructures for Disaster Resilience Engineering Report.

Discovery of data is only part of the challenge. Once you find the service or dataset, you want to access it and use it. If the resource is not available via open standards, like the ones promoted by OGC and ISO, then it's like buying a piece of bread that you can only cut with one knife. It becomes frustrating and expensive to buy this kind of bread!

What if you are a data provider and in minutes you can register your data or service on the web, add metadata to it, and immediately have it available via open standards? This is what GeoNode does pretty well. It’s a content management system for geospatial, build on open standards and open source. It powers lots of Spatial Data Infrastructures in the world. For example GeoSolutions has helped install GeoNode in various countries (Malaway, Afghanistan, Mozambique, Ghana, Uganda, Nepal, and Madagascar).

The World Bank, in particular, has greatly supported its development and even published a report about the return over investment they encountered when investing in the development of GeoNode. A great blog about it is available at the World Bank web site. So, with all these said, GeoNode 3.0 was released about a month ago.  See GeoNode blog for complete details. There is usually a presentation GeoSolutions does at  the FOSS4G meeting and at  the GeoNode summit about the latest release, but unfortunately the two meetings were cancelled, so we are doing  a free webinar on July 7th at  11:00 Eastern Time. Some highlights of what’s new:

- GeoNode components have been updated to new versions: Python upgrade 3.7, Django upgrade 2.2.12, GeoServer upgrade 2.16.2.

MapStore is now the default client greatly enriching the experience for viewing the data.

- Greatly improved analytics and getting information about who is visiting the website and their actions. For example, you can see unique visitors who trigger a specific type of event.

- Improved full GeoNode backup & restore.

- Extended GeoNode shared permission via GeoLimits to restrict users or groups to a specific geographical area (see below).

[caption id="attachment_5335" align="aligncenter" width="799"]geolimits GeoLimits at work for GeoNode 3[/caption]  

I just highlighted my favorite enhancements. To learn more about GeoNode 3,0 and enjoy a 20 minutes of Q&A time with our lead GeoNode Lead Developer Alessio Fabiani, Francesco Bartoli, as well as, Giovanni Allegri and Florian Hoedt also part of the GeoNode Project Steering Committee. I cordially invite you to a free webinar on July 7 at 11:00 Eastern Time by registering at the link below!

register
Hope to see you virtually on July 7th, meanwhile stay safe and keep strong! Cordially, Luis 320x100_eng]]>
https://www.geosolutionsgroup.com/news/state-geonode-3-webinar/feed/ 0
Upcoming Webinar: Best Practices for Optimizing Performance with GeoServer https://www.geosolutionsgroup.com/news/geoserver-performance-2020-webinar/ https://www.geosolutionsgroup.com/news/geoserver-performance-2020-webinar/#comments Thu, 28 May 2020 15:59:17 +0000 http://www.geosolutionsgroup.com/?p=5293 GeoServer

Dear Reader,

One of the main goals of GeoSolutions customers is to improve performance of their geospatial server. We have been improving open source tools (e.g. GeoServer and GeoWebCache) to allow for proper dissemination of large geospatial datasets in private and public cloud environments. In this post, I highlight some tricks and tips to improve performance and I also cordially invite you to a webinar on June 10th 11:00-12:00 EDT / 15:00 GMT, led by our GeoServer lead developer Andrea Aime. Check other time zones here

We want to enable a client to play with maps backed by millions of geometries or terabytes of images. Proper configuration of GeoServer and the data, in particular for large datasets, will keep us in the happy zone! GeoServer has been improving over the years and more options are available to the administrators to tweak its configuration. A blog posted two years ago, provides some tweaking details. Lots of things have changed since then. For example, back then the Marlin renderer was not part of the JAVA JDK. Now it is part of JDK-9 onwards (See a JaveOne presentation from Bourges about this topic). You don’t need to do this extra configuration. If you are running on JDK-8 then follow the recommendations on the blog to improve the performance when rendering images. So, what do you do if you have a ½ terabyte of OSM data and want to use it as a basemap, or you want to show on a map data from areas of interest that are tessellated in grid cells with very high resolution, or you want to cache layer groups containing thousands of GeoTIFFs (e.g. see thread on gis.stackexchange)? The strategies and tricks revolve on minimizing the work to be performed by GeoServer and the database once a request is being made. This requires preparing as much as possible data in advance, selecting best formats, caching, and other strategies.  I will provide key strategies in this post, as an introduction to the topic.
[caption id="attachment_5304" align="aligncenter" width="936"]Server performance Server performance[/caption]

Can I portray millions of geometries in my web client?

Let’s say you have an area composed of 1 million geometries. You are not always going to show everything. You pick and choose what you want to present depending on the zoom levels. This can be tricky because you also need to process the attribute data from the contained geometries (e.g. adding, averaging). There is this kind of illusion that the user is getting all the data all the time. The magic is done via a smart setup on the backend to minimize the features being retrieved.  Several approaches to tackle this issue:

1) Reduce the number of features being returned. You want super fast performance, present maximum 1000 features.

Pre-configure and select what to show at different levels (e.g. via style optimizations). As the user zooms-in, the data presented changes. New features and new labels might show-up, while others disappear.

data-simplification Create bigger geometries that are composed of smaller geometries. The geometries returned depend on the zoom level of the request or the information of the request. When done properly discrete grids are an incredible mechanism to compute fast analytics. A good read about this topic is the  OGC Discrete Global Grid System (DGGS). There is also related work going on, which we are helping to advance, as part of the  OGC Testbed 16.

2) Simplify the vertices of the polygon. For example, if a polygon has 500,000 vertices at a higher zoom level the same polygon can be represented in hundreds of vertices and the end user should not notice the difference. The tutorial for using GeoTools Feature-pregenralized module explains this in detail.

Can I serve petabytes of raster data with GeoServer?

The answer is yes, but you need to properly configure them. Regarding formats GeoTIFF is the champion. It is very flexible, can be tiled and can be fined tuned for performance.  

How do you structure a GeoTIFF? There are several structures such as structuring the data in a Single GeoTiff, in a Mosaic or in a Pyramid.

  [caption id="attachment_5302" align="aligncenter" width="936"]GeoTIFF Structures GeoTIFF Structures[/caption]

The structure strategy basically depends on the size of the data, and the dimensions. 

  • If single granules are < 20 Gb, using Single GeoTiffs are a good choice.
  • If files are > 20 Gb or you have too many dimensions (e.g. numerical models might have multiple times, elevation and others), then use ImageMosaics.
  • If the dataset if tremendously large and the data needs to be served at different resolutions, then ImagePyramid is the best option.
With the above approaches we have served satellite imagery covering the entire planet, as well as long time series of meteorological models, and the like, up to several petabytes of backend data.

As said before, I have only provided some tips and tricks in this post. Don’t miss our webinar and register, so you will have the opportunity to hear more about this topic and ask anything you want to Andrea.

register

Hope to see you virtually on June 10th, meanwhile stay safe and keep strong!

Cordially,

Luis 320x100_eng]]>
GeoServer

Dear Reader,

One of the main goals of GeoSolutions customers is to improve performance of their geospatial server. We have been improving open source tools (e.g. GeoServer and GeoWebCache) to allow for proper dissemination of large geospatial datasets in private and public cloud environments. In this post, I highlight some tricks and tips to improve performance and I also cordially invite you to a webinar on June 10th 11:00-12:00 EDT / 15:00 GMT, led by our GeoServer lead developer Andrea Aime. Check other time zones here

We want to enable a client to play with maps backed by millions of geometries or terabytes of images. Proper configuration of GeoServer and the data, in particular for large datasets, will keep us in the happy zone! GeoServer has been improving over the years and more options are available to the administrators to tweak its configuration. A blog posted two years ago, provides some tweaking details. Lots of things have changed since then. For example, back then the Marlin renderer was not part of the JAVA JDK. Now it is part of JDK-9 onwards (See a JaveOne presentation from Bourges about this topic). You don’t need to do this extra configuration. If you are running on JDK-8 then follow the recommendations on the blog to improve the performance when rendering images. So, what do you do if you have a ½ terabyte of OSM data and want to use it as a basemap, or you want to show on a map data from areas of interest that are tessellated in grid cells with very high resolution, or you want to cache layer groups containing thousands of GeoTIFFs (e.g. see thread on gis.stackexchange)? The strategies and tricks revolve on minimizing the work to be performed by GeoServer and the database once a request is being made. This requires preparing as much as possible data in advance, selecting best formats, caching, and other strategies.  I will provide key strategies in this post, as an introduction to the topic.
[caption id="attachment_5304" align="aligncenter" width="936"]Server performance Server performance[/caption]

Can I portray millions of geometries in my web client?

Let’s say you have an area composed of 1 million geometries. You are not always going to show everything. You pick and choose what you want to present depending on the zoom levels. This can be tricky because you also need to process the attribute data from the contained geometries (e.g. adding, averaging). There is this kind of illusion that the user is getting all the data all the time. The magic is done via a smart setup on the backend to minimize the features being retrieved.  Several approaches to tackle this issue:

1) Reduce the number of features being returned. You want super fast performance, present maximum 1000 features.

Pre-configure and select what to show at different levels (e.g. via style optimizations). As the user zooms-in, the data presented changes. New features and new labels might show-up, while others disappear.

data-simplification Create bigger geometries that are composed of smaller geometries. The geometries returned depend on the zoom level of the request or the information of the request. When done properly discrete grids are an incredible mechanism to compute fast analytics. A good read about this topic is the  OGC Discrete Global Grid System (DGGS). There is also related work going on, which we are helping to advance, as part of the  OGC Testbed 16.

2) Simplify the vertices of the polygon. For example, if a polygon has 500,000 vertices at a higher zoom level the same polygon can be represented in hundreds of vertices and the end user should not notice the difference. The tutorial for using GeoTools Feature-pregenralized module explains this in detail.

Can I serve petabytes of raster data with GeoServer?

The answer is yes, but you need to properly configure them. Regarding formats GeoTIFF is the champion. It is very flexible, can be tiled and can be fined tuned for performance.  

How do you structure a GeoTIFF? There are several structures such as structuring the data in a Single GeoTiff, in a Mosaic or in a Pyramid.

  [caption id="attachment_5302" align="aligncenter" width="936"]GeoTIFF Structures GeoTIFF Structures[/caption]

The structure strategy basically depends on the size of the data, and the dimensions. 

  • If single granules are < 20 Gb, using Single GeoTiffs are a good choice.
  • If files are > 20 Gb or you have too many dimensions (e.g. numerical models might have multiple times, elevation and others), then use ImageMosaics.
  • If the dataset if tremendously large and the data needs to be served at different resolutions, then ImagePyramid is the best option.
With the above approaches we have served satellite imagery covering the entire planet, as well as long time series of meteorological models, and the like, up to several petabytes of backend data.

As said before, I have only provided some tips and tricks in this post. Don’t miss our webinar and register, so you will have the opportunity to hear more about this topic and ask anything you want to Andrea.

register

Hope to see you virtually on June 10th, meanwhile stay safe and keep strong!

Cordially,

Luis 320x100_eng]]>
https://www.geosolutionsgroup.com/news/geoserver-performance-2020-webinar/feed/ 6
Upcoming Free Webinar: What’s new in GeoServer 2.17? https://www.geosolutionsgroup.com/news/geoserver-2-17-webinar/ https://www.geosolutionsgroup.com/news/geoserver-2-17-webinar/#comments Sun, 03 May 2020 14:06:15 +0000 http://www.geosolutionsgroup.com/?p=5251 styleGroup - MBTiles

Dear Reader,

As most of you readers are aware, GeoServer is a very  popular open source tool to publish geospatial data. We, at GeoSolutions, are very fortunate to have in our team key code contributors, as well as, customers that support the advancement of GeoServer for the greater benefit of the geospatial community. Version 2.17 was released a week ago. A blog with detailed information is available at the GeoServer blog. In this post, I highlight some new functionality and invite you to a webinar on May 11th to learn more about the release from our GeoServer lead developer, Andrea Aime,

The GeoServer 2.17 release comes with many interesting new functionalities, let us briefly introduce some of them.

Faster, due to improvements in GeoWebCache

You will experience a much better startup performance because of the improvement to load the tiles layers configuration. Now, you can have  better control over failed seeding operations, by configuring error tolerance on single threads and across seed jobs (enabling seeding threads to continue if they encounter failures, like it occurred in the past).

Ability to render vector tiles with Mapbox Styles

You can now configure GeoServer to serve layers as vector tiles which can be used as sources for Mapbox styles rendered by web applications such as OpenLayers.

Improved the ability to read and write MBTiles files in GeoServer

Now you can read and write MBTiles files in GeoServer. You can use MBTiles as a raster input datastore as well as an WMS GetMap output format.

[caption id="" align="alignnone" width="767"] MBTiles[/caption]  

And more functionality like:

  • Added support for FlatGeoBuf (incredible binary encoding for fast transfers)
  • Added functionality to view and manage resources in the data directory (e.g. configuration files, icons and fonts)
  • Better manage security for layers when looking at layer
  • Ability to view when a layer was created and last modified
  • Improved the rendering of icons and maps
  • Enable changing background in the SLD
  • Enable ability to enable/advertise layer groups
  • Ability to add custom dimension in vector layers (similar to time and elevation, dimensions in WMS)
  • Added more capabilities to WMS Cascading (e.g. specify format, min/max scale denominators for cascaded layers)
  • Added support for JSON-LD
  • Added support for OpenAPI
  • And much more..

We cordially invite you to join the webinar on Monday May 18th at 11:00 EDT / 3:00 pm GMT (check more time zones), to learn more about this release from our GeoServer Technical Lead, Andrea Aime. We are planning to do 30 minutes presentation and 30 minutes for questions.  You can register by clicking on the button below.

register

Hope to see you virtually on May 18th, meanwhile stay safe and keep strong!

Cordially,

Luis 320x100_eng]]>
styleGroup - MBTiles

Dear Reader,

As most of you readers are aware, GeoServer is a very  popular open source tool to publish geospatial data. We, at GeoSolutions, are very fortunate to have in our team key code contributors, as well as, customers that support the advancement of GeoServer for the greater benefit of the geospatial community. Version 2.17 was released a week ago. A blog with detailed information is available at the GeoServer blog. In this post, I highlight some new functionality and invite you to a webinar on May 11th to learn more about the release from our GeoServer lead developer, Andrea Aime,

The GeoServer 2.17 release comes with many interesting new functionalities, let us briefly introduce some of them.

Faster, due to improvements in GeoWebCache

You will experience a much better startup performance because of the improvement to load the tiles layers configuration. Now, you can have  better control over failed seeding operations, by configuring error tolerance on single threads and across seed jobs (enabling seeding threads to continue if they encounter failures, like it occurred in the past).

Ability to render vector tiles with Mapbox Styles

You can now configure GeoServer to serve layers as vector tiles which can be used as sources for Mapbox styles rendered by web applications such as OpenLayers.

Improved the ability to read and write MBTiles files in GeoServer

Now you can read and write MBTiles files in GeoServer. You can use MBTiles as a raster input datastore as well as an WMS GetMap output format.

[caption id="" align="alignnone" width="767"] MBTiles[/caption]  

And more functionality like:

  • Added support for FlatGeoBuf (incredible binary encoding for fast transfers)
  • Added functionality to view and manage resources in the data directory (e.g. configuration files, icons and fonts)
  • Better manage security for layers when looking at layer
  • Ability to view when a layer was created and last modified
  • Improved the rendering of icons and maps
  • Enable changing background in the SLD
  • Enable ability to enable/advertise layer groups
  • Ability to add custom dimension in vector layers (similar to time and elevation, dimensions in WMS)
  • Added more capabilities to WMS Cascading (e.g. specify format, min/max scale denominators for cascaded layers)
  • Added support for JSON-LD
  • Added support for OpenAPI
  • And much more..

We cordially invite you to join the webinar on Monday May 18th at 11:00 EDT / 3:00 pm GMT (check more time zones), to learn more about this release from our GeoServer Technical Lead, Andrea Aime. We are planning to do 30 minutes presentation and 30 minutes for questions.  You can register by clicking on the button below.

register

Hope to see you virtually on May 18th, meanwhile stay safe and keep strong!

Cordially,

Luis 320x100_eng]]>
https://www.geosolutionsgroup.com/news/geoserver-2-17-webinar/feed/ 0
Health SDI, Simple APIs and Web Map Clients for COVID Tracking https://www.geosolutionsgroup.com/blog/health-sdi-covid-map/ https://www.geosolutionsgroup.com/blog/health-sdi-covid-map/#comments Thu, 23 Apr 2020 17:08:18 +0000 http://www.geosolutionsgroup.com/?p=5213 covid

Dear Reader,

Sharing of health data has become one of the most important data sharing activities in the last months. The general population and authorities need to know the current situation not only about reported cases, deaths and recovered, but other data (e.g. hospital beds, ventilators, etc.) that supports properly responding to the COVID-19 pandemic. In addition to this health data, there are other non-health datasets (e.g. boundaries, socio-economic data) that provide context, complement and help decision makers better understand the possible actions to take. So, in today’s health emergency situation, an information sharing infrastructure with modern web visualization tools is critical to guide the way we share health and complementary data. 

In this blog I am going to 1) highlight the importance of a Health Spatial Data Infrastructure (SDI) documented in a recent paper published by OGC, which I co-authored, 2) make the case that sharing of data should be easy, and 3) provide an example of a dashboard tracking COVID-19 data based on MapStore.

The Open Geospatial Consortium (OGC) published some weeks ago the white paper: Health Spatial Data Infrastructure: Application Areas, Recommendations, and Architecture. It provides reference to initiatives (e.g. EO4Health, Eo2Heaven, INSPIRE Human Health and Safety Specifications,)  that have occurred in the last years related to building an infrastructure that enables the collection, exchange, integration, analysis, and visualization of health and non-health data to help identify and address health issues at global and local level. The sections I consider worth reading are: Section 4.7. Pandemic Response Section 5 Data Considerations and Related Recommendations, and  Section 6 - Health SDI Architecture Framework.

Section 4.7 discusses the development and use of different indexes (e.g. Transmission Risk Index) that are only possible to develop if there is enough information available. This is the reason why a Spatial Data Infrastructure is critical. This is the reason why we need to publish the data on the Web using open standards, following recommendations from Section 5 that takes into account security, anonymization and suggested formats and interfaces (e.g. WFS or WMS). 

Section 6 provides a discussion on the architecture and also a workflow about what is needed when creating indexes that involve different types of data (See Figure 1). What is critical from the workflow is the catalog, which is the heart of an SDI. If you don't know what data is available you can’t put it together in a fast manner and it requires phone calls, emails, searching, scraping websites, etc., wasting precious time when building a common operational picture.

[caption id="attachment_5221" align="aligncenter" width="1024"]Health SDI Workflow Figure 1 - Health SDI Workflow[/caption] Worth to mention, that the prototype of a Health SDI was advanced in a recent OGC initiative, well documented in an OGC video. The tool used was GeoNode which is an-easy-to-setup tool for SDIs that allows the data to be easily cataloged, updated and shared via open standards.

Organizations harvesting information from different sources to create dashboards mostly rely on personal communications and getting the data from official web reports available at government/intergovernmental websites. Then, they create “machine readable formats” such as JSON or CSV that are ingested in the web clients. The “manual process” of getting the data requires a lot of human power, and fortunately for this crisis there are a lot of people willing to help. This is not the ideal. Government and other official sources should be making data available via open standards following the recommendation in the report.

At GeoSolutions, we were investigating how COVID-19 data was being published. We found an initiative, the COVID Tracking Project that actually does a great job on aggregating data for the US states. They perform quality control, publish the provenance, and make the data available via a simple Data API. If organizations around the world can follow simple APIs like the one mentioned, it will be easier to aggregate data in our current and other health emergency events. Also, fortunately, OGC is working on simple APIs that will enable government, agencies and projects (similar to the COVID Tracking Project), to share data via an open-standard simple API

Based on the previous mentioned project, and the simplicity of their API, we wanted to develop a simple sleek dashboard for tracking COVID-19 cases in the US. We wanted to reuse the best tools that we have in house at GeoSolutions, so we reused MapStore to build such a dashboard; you can reach it live here.

[caption id="attachment_5240" align="aligncenter" width="1024"]COVID Tracking Map Figure 2 - US COVID Tracking Dashboard with MapStore[/caption]

The client interacts directly with the API. There is no database on the backend and this is the reason it can be installed in a S3 bucket (which is a cheaper option than using a virtual machine under EC2). The user interface allows to view more than one variable at a time by enabling the user to select from the left side and displaying bubble plots located at the centroids of each state. The user can also view and order the statistics on the right column enabling easy comparison among the states. Hovering over a state shows all the data for that particular state, as it is shown for New York . Also, in our spirit of open source advocates, we have released the code at GitHub under a BSD License.

Sharing data via open standards is important, as well as, simple APIs that can enable web clients to provide dashboards and critical information to end users. I hope you find this blog useful and can inspire some ideas about sharing and visualizing pandemic tracking.  If you would like GeoSolutions to help with your health data publication, cataloging or visualization needs we are here to help!

Please don’t hesitate to contact us. We would love to hear how we can help you!

Cordially,

Luis Bermudez CEO GeoSolutions USA 320x100_eng]]>
covid

Dear Reader,

Sharing of health data has become one of the most important data sharing activities in the last months. The general population and authorities need to know the current situation not only about reported cases, deaths and recovered, but other data (e.g. hospital beds, ventilators, etc.) that supports properly responding to the COVID-19 pandemic. In addition to this health data, there are other non-health datasets (e.g. boundaries, socio-economic data) that provide context, complement and help decision makers better understand the possible actions to take. So, in today’s health emergency situation, an information sharing infrastructure with modern web visualization tools is critical to guide the way we share health and complementary data. 

In this blog I am going to 1) highlight the importance of a Health Spatial Data Infrastructure (SDI) documented in a recent paper published by OGC, which I co-authored, 2) make the case that sharing of data should be easy, and 3) provide an example of a dashboard tracking COVID-19 data based on MapStore.

The Open Geospatial Consortium (OGC) published some weeks ago the white paper: Health Spatial Data Infrastructure: Application Areas, Recommendations, and Architecture. It provides reference to initiatives (e.g. EO4Health, Eo2Heaven, INSPIRE Human Health and Safety Specifications,)  that have occurred in the last years related to building an infrastructure that enables the collection, exchange, integration, analysis, and visualization of health and non-health data to help identify and address health issues at global and local level. The sections I consider worth reading are: Section 4.7. Pandemic Response Section 5 Data Considerations and Related Recommendations, and  Section 6 - Health SDI Architecture Framework.

Section 4.7 discusses the development and use of different indexes (e.g. Transmission Risk Index) that are only possible to develop if there is enough information available. This is the reason why a Spatial Data Infrastructure is critical. This is the reason why we need to publish the data on the Web using open standards, following recommendations from Section 5 that takes into account security, anonymization and suggested formats and interfaces (e.g. WFS or WMS). 

Section 6 provides a discussion on the architecture and also a workflow about what is needed when creating indexes that involve different types of data (See Figure 1). What is critical from the workflow is the catalog, which is the heart of an SDI. If you don't know what data is available you can’t put it together in a fast manner and it requires phone calls, emails, searching, scraping websites, etc., wasting precious time when building a common operational picture.

[caption id="attachment_5221" align="aligncenter" width="1024"]Health SDI Workflow Figure 1 - Health SDI Workflow[/caption] Worth to mention, that the prototype of a Health SDI was advanced in a recent OGC initiative, well documented in an OGC video. The tool used was GeoNode which is an-easy-to-setup tool for SDIs that allows the data to be easily cataloged, updated and shared via open standards.

Organizations harvesting information from different sources to create dashboards mostly rely on personal communications and getting the data from official web reports available at government/intergovernmental websites. Then, they create “machine readable formats” such as JSON or CSV that are ingested in the web clients. The “manual process” of getting the data requires a lot of human power, and fortunately for this crisis there are a lot of people willing to help. This is not the ideal. Government and other official sources should be making data available via open standards following the recommendation in the report.

At GeoSolutions, we were investigating how COVID-19 data was being published. We found an initiative, the COVID Tracking Project that actually does a great job on aggregating data for the US states. They perform quality control, publish the provenance, and make the data available via a simple Data API. If organizations around the world can follow simple APIs like the one mentioned, it will be easier to aggregate data in our current and other health emergency events. Also, fortunately, OGC is working on simple APIs that will enable government, agencies and projects (similar to the COVID Tracking Project), to share data via an open-standard simple API

Based on the previous mentioned project, and the simplicity of their API, we wanted to develop a simple sleek dashboard for tracking COVID-19 cases in the US. We wanted to reuse the best tools that we have in house at GeoSolutions, so we reused MapStore to build such a dashboard; you can reach it live here.

[caption id="attachment_5240" align="aligncenter" width="1024"]COVID Tracking Map Figure 2 - US COVID Tracking Dashboard with MapStore[/caption]

The client interacts directly with the API. There is no database on the backend and this is the reason it can be installed in a S3 bucket (which is a cheaper option than using a virtual machine under EC2). The user interface allows to view more than one variable at a time by enabling the user to select from the left side and displaying bubble plots located at the centroids of each state. The user can also view and order the statistics on the right column enabling easy comparison among the states. Hovering over a state shows all the data for that particular state, as it is shown for New York . Also, in our spirit of open source advocates, we have released the code at GitHub under a BSD License.

Sharing data via open standards is important, as well as, simple APIs that can enable web clients to provide dashboards and critical information to end users. I hope you find this blog useful and can inspire some ideas about sharing and visualizing pandemic tracking.  If you would like GeoSolutions to help with your health data publication, cataloging or visualization needs we are here to help!

Please don’t hesitate to contact us. We would love to hear how we can help you!

Cordially,

Luis Bermudez CEO GeoSolutions USA 320x100_eng]]>
https://www.geosolutionsgroup.com/blog/health-sdi-covid-map/feed/ 0