So this time the sprint is two days long. We hope to get more tickets looked at, while allowing participant developers to tackle harder to fix problems.

Please join us on the GeoServer gitter channel, and don’t forget to go say hi to the other Bolsena sprinters on the OSGeo/Sprint channel!

Cordially,

Can I portray millions of geometries in my web client?

Let’s say you have an area composed of 1 million geometries. You are not always going to show everything. You pick and choose what you want to present depending on the zoom levels. This can be tricky because you also need to process the attribute data from the contained geometries (e.g. adding, averaging). There is this kind of illusion that the user is getting all the data all the time. The magic is done via a smart setup on the backend to minimize the features being retrieved.  Several approaches to tackle this issue:

1) Reduce the number of features being returned. You want super fast performance, present maximum 1000 features.

Pre-configure and select what to show at different levels (e.g. via style optimizations). As the user zooms-in, the data presented changes. New features and new labels might show-up, while others disappear.

Create bigger geometries that are composed of smaller geometries. The geometries returned depend on the zoom level of the request or the information of the request. When done properly discrete grids are an incredible mechanism to compute fast analytics. A good read about this topic is the  OGC Discrete Global Grid System (DGGS). There is also related work going on, which we are helping to advance, as part of the  OGC Testbed 16.

2) Simplify the vertices of the polygon. For example, if a polygon has 500,000 vertices at a higher zoom level the same polygon can be represented in hundreds of vertices and the end user should not notice the difference. The tutorial for using GeoTools Feature-pregenralized module explains this in detail.

Can I serve petabytes of raster data with GeoServer?

The answer is yes, but you need to properly configure them. Regarding formats GeoTIFF is the champion. It is very flexible, can be tiled and can be fined tuned for performance.  

How do you structure a GeoTIFF? There are several structures such as structuring the data in a Single GeoTiff, in a Mosaic or in a Pyramid.

  [caption id="attachment_5302" align="aligncenter" width="936"] GeoTIFF Structures[/caption]

The structure strategy basically depends on the size of the data, and the dimensions. 

• If single granules are < 20 Gb, using Single GeoTiffs are a good choice.
• If files are > 20 Gb or you have too many dimensions (e.g. numerical models might have multiple times, elevation and others), then use ImageMosaics.
• If the dataset if tremendously large and the data needs to be served at different resolutions, then ImagePyramid is the best option.

With the above approaches we have served satellite imagery covering the entire planet, as well as long time series of meteorological models, and the like, up to several petabytes of backend data.

As said before, I have only provided some tips and tricks in this post. Don’t miss our webinar and register, so you will have the opportunity to hear more about this topic and ask anything you want to Andrea.

Hope to see you virtually on June 10th, meanwhile stay safe and keep strong!

Cordially,

As most of you readers are aware, GeoServer is a very  popular open source tool to publish geospatial data. We, at GeoSolutions, are very fortunate to have in our team key code contributors, as well as, customers that support the advancement of GeoServer for the greater benefit of the geospatial community. Version 2.17 was released a week ago. A blog with detailed information is available at the GeoServer blog. In this post, I highlight some new functionality and invite you to a webinar on May 11th to learn more about the release from our GeoServer lead developer, Andrea Aime,

The GeoServer 2.17 release comes with many interesting new functionalities, let us briefly introduce some of them.

Faster, due to improvements in GeoWebCache

You will experience a much better startup performance because of the improvement to load the tiles layers configuration. Now, you can have  better control over failed seeding operations, by configuring error tolerance on single threads and across seed jobs (enabling seeding threads to continue if they encounter failures, like it occurred in the past).

Ability to render vector tiles with Mapbox Styles

You can now configure GeoServer to serve layers as vector tiles which can be used as sources for Mapbox styles rendered by web applications such as OpenLayers.

Improved the ability to read and write MBTiles files in GeoServer

Now you can read and write MBTiles files in GeoServer. You can use MBTiles as a raster input datastore as well as an WMS GetMap output format.

And more functionality like:

• Added support for FlatGeoBuf (incredible binary encoding for fast transfers)
• Added functionality to view and manage resources in the data directory (e.g. configuration files, icons and fonts)
• Better manage security for layers when looking at layer
• Ability to view when a layer was created and last modified
• Improved the rendering of icons and maps
• Enable changing background in the SLD
• Enable ability to enable/advertise layer groups
• Ability to add custom dimension in vector layers (similar to time and elevation, dimensions in WMS)
• Added more capabilities to WMS Cascading (e.g. specify format, min/max scale denominators for cascaded layers)
• Added support for JSON-LD
• Added support for OpenAPI
• And much more..

We cordially invite you to join the webinar on Monday May 18th at 11:00 EDT / 3:00 pm GMT (check more time zones), to learn more about this release from our GeoServer Technical Lead, Andrea Aime. We are planning to do 30 minutes presentation and 30 minutes for questions.  You can register by clicking on the button below.

Hope to see you virtually on May 18th, meanwhile stay safe and keep strong!

Cordially,

Sharing of health data has become one of the most important data sharing activities in the last months. The general population and authorities need to know the current situation not only about reported cases, deaths and recovered, but other data (e.g. hospital beds, ventilators, etc.) that supports properly responding to the COVID-19 pandemic. In addition to this health data, there are other non-health datasets (e.g. boundaries, socio-economic data) that provide context, complement and help decision makers better understand the possible actions to take. So, in today’s health emergency situation, an information sharing infrastructure with modern web visualization tools is critical to guide the way we share health and complementary data. 

In this blog I am going to 1) highlight the importance of a Health Spatial Data Infrastructure (SDI) documented in a recent paper published by OGC, which I co-authored, 2) make the case that sharing of data should be easy, and 3) provide an example of a dashboard tracking COVID-19 data based on MapStore.

The Open Geospatial Consortium (OGC) published some weeks ago the white paper: Health Spatial Data Infrastructure: Application Areas, Recommendations, and Architecture. It provides reference to initiatives (e.g. EO4Health, Eo2Heaven, INSPIRE Human Health and Safety Specifications,)  that have occurred in the last years related to building an infrastructure that enables the collection, exchange, integration, analysis, and visualization of health and non-health data to help identify and address health issues at global and local level. The sections I consider worth reading are: Section 4.7. Pandemic Response,  Section 5 Data Considerations and Related Recommendations, and  Section 6 - Health SDI Architecture Framework.

Section 4.7 discusses the development and use of different indexes (e.g. Transmission Risk Index) that are only possible to develop if there is enough information available. This is the reason why a Spatial Data Infrastructure is critical. This is the reason why we need to publish the data on the Web using open standards, following recommendations from Section 5 that takes into account security, anonymization and suggested formats and interfaces (e.g. WFS or WMS). 

Section 6 provides a discussion on the architecture and also a workflow about what is needed when creating indexes that involve different types of data (See Figure 1). What is critical from the workflow is the catalog, which is the heart of an SDI. If you don't know what data is available you can’t put it together in a fast manner and it requires phone calls, emails, searching, scraping websites, etc., wasting precious time when building a common operational picture.

Organizations harvesting information from different sources to create dashboards mostly rely on personal communications and getting the data from official web reports available at government/intergovernmental websites. Then, they create “machine readable formats” such as JSON or CSV that are ingested in the web clients. The “manual process” of getting the data requires a lot of human power, and fortunately for this crisis there are a lot of people willing to help. This is not the ideal. Government and other official sources should be making data available via open standards following the recommendation in the report.

At GeoSolutions, we were investigating how COVID-19 data was being published. We found an initiative, the COVID Tracking Project that actually does a great job on aggregating data for the US states. They perform quality control, publish the provenance, and make the data available via a simple Data API. If organizations around the world can follow simple APIs like the one mentioned, it will be easier to aggregate data in our current and other health emergency events. Also, fortunately, OGC is working on simple APIs that will enable government, agencies and projects (similar to the COVID Tracking Project), to share data via an open-standard simple API

Based on the previous mentioned project, and the simplicity of their API, we wanted to develop a simple sleek dashboard for tracking COVID-19 cases in the US. We wanted to reuse the best tools that we have in house at GeoSolutions, so we reused MapStore to build such a dashboard; you can reach it live here.

The client interacts directly with the API. There is no database on the backend and this is the reason it can be installed in a S3 bucket (which is a cheaper option than using a virtual machine under EC2). The user interface allows to view more than one variable at a time by enabling the user to select from the left side and displaying bubble plots located at the centroids of each state. The user can also view and order the statistics on the right column enabling easy comparison among the states. Hovering over a state shows all the data for that particular state, as it is shown for New York . Also, in our spirit of open source advocates, we have released the code at GitHub under a BSD License.

Sharing data via open standards is important, as well as, simple APIs that can enable web clients to provide dashboards and critical information to end users. I hope you find this blog useful and can inspire some ideas about sharing and visualizing pandemic tracking.  If you would like GeoSolutions to help with your health data publication, cataloging or visualization needs we are here to help!

Please don’t hesitate to contact us. We would love to hear how we can help you!

Cordially,