Skip to main content

Working with Multiple Data Sources in your Environment of Choice

Boson makes bringing disparate datasets together very easy. In this demo, we will bring together data from flat files, STAC collections and ArcGIS online. Once Boson pulls the datasets together in a uniform way, we can serve them out in a variety of ways, depending on our needs.

Setup​

First, let's setup the environment and create a project, if you don't already have one set up.

Load the datasets​

In other tutorials, we have shown how to load a variety of datasets. Here, we will make use of a few of them. In particular, we will load one dataset from a STAC collection, one from a cloud bucket, and one from an ArcGIS online service.

Load a dataset from a STAC collection​

First, let's load the STAC Collection. This is Sentinel 2 L2A raster data. You can examine the tutorial in more detail in the Adding a STAC Collection tutorial, but we really only need a few lines of code:

>>> stac_link = "https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a-cogs"
>>> ds = geodesic.Dataset.from_stac_collection('sentinel-2',stac_link)
>>> ds.save()

Load a dataset from a cloud bucket​

Next, let's load a dataset from a cloud bucket. Again, you can examine the tutorial in more detail in the Adding a GeoJSON Dataset tutorial, but we can load the dataset like so:

..code-block:: python

ds = geodesic.Dataset.from_tabular_data( name='uscb-pop-centers', url='gs://geodesic-public-data/CenPop2020_Mean_CO.geojson', index_data=True, crs='EPSG:4326', ) ds.save()

Load a dataset from an ArcGIS online service​

Finally, let's load a dataset from an ArcGIS online service. Again, you can examine the tutorial in more detail in the Adding an ArcGIS Service Dataset tutorial. Here is the relevant code:

>>> url = "https://geodata.epa.gov/arcgis/rest/services/OEI/FRS_INTERESTS/MapServer/7"
>>> ds = geodesic.Dataset.from_arcgis_layer(name = "epa-frs-locations", url=url)
>>> ds.save()

Bringing the datasets together​

Even though these datasets come from different sources, Boson makes it easy to bring them together.
Now that they are saved in this project, we can serve them out in a variety of ways.

First, let's retrieve the datasets we just saved. We can do this by using the geodesic.entanglement.get_objects method. We can use the object_class parameter to filter the results to only include datasets. Additionally, we can use the search parameter to query for the particular datasets we want.

>>> from geodesic.entanglement import get_objects

>>> datasets = get_objects(object_class='Dataset')
[dataset:*:*:*:sentinel-2,
dataset:*:*:*:uscb-pop-centers,
dataset:*:*:*:epa-frs-locations]

>>> sentinel2_data = get_objects(object_class='Dataset', search='sentinel')[0]
>>> uscb_data = get_objects(search='uscb-pop-centers')[0]
>>> epa_data = get_objects(search='epa')[0]

Now, we can create share links to serve the data.

Serving the Datasets to a Web Map​

We can serve the datasets out in a variety of ways. Let's start by simply creating feature and tile services for each dataset, and then we will serve them to an ipyleaflet map. Because Sentinel 2 data is large and contains 10 bands, we will first have to create a view of the dataset that is appropriate for serving as an image service. Views allow us to apply a persistent filter to a dataset. Here, we will create a view that only includes the red band and a small bounding box around Philadelphia, PA, USA for a 20-day period in January 2020.

>>> import datetime

>>> bbox = [-75.207767, 39.934486, -75.155411, 39.970806]
>>> start_time = datetime.datetime(2020, 1, 1)
>>> end_time = start_time + datetime.timedelta(days=20)

>>> asset_bands = [{'asset': 'B04', 'bands': [0]}]

>>> sentinel2_view = sentinel2_data.view(
        name='sentinel2_philadelphia',
        bbox=bbox,
        datetime=[start_time, end_time],
        asset_bands=asset_bands,
    )

>>> sentinel2_view.save()

Now, we can create share links to serve the data. First, we create a share token.

>>> sentinel2_token = sentinel2_view.share_as_ogc_tiles_service()

Then, we can retrieve the URL for the service. Here, we will use the get_ogc_raster_tile_url method to get the URL for the raster tile service.

>>> sentinel2_url = sentinel2_token.get_ogc_raster_tile_url()

Next, we will create vector tile feature services for the other two datasets.

>>> uscb_token = uscb_data.share_as_ogc_vector_tile_service()
>>> uscb_url = uscb_token.get_ogc_vector_tile_url()

>>> epa_token = epa_data.share_as_ogc_vector_tile_service()
>>> epa_url = epa_token.get_ogc_vector_tile_url()

Finally, we can serve the datasets to an ipyleaflet map.

>>> from geodesic import mapping
>>> from ipyleaflet import TileLayer, VectorTileLayer

>>> sentinel2_layer = TileLayer(name='sentinel2-philadelphia', url=sentinel2_url)
>>> epa_layer = VectorTileLayer(name='epa-frs-sites', url=epa_link)
>>> census_layer = VectorTileLayer(name='uscb-pop-centers', url=census_link)

>>> m = mapping.Map()
>>> m.add_layer(sentinel2_layer)
>>> m.add_layer(epa)
>>> m.add_layer(census)
>>> m