Working with Multiple Data Sources in your Environment of Choice - ArcGIS Online

Boson makes bringing disparate datasets together very easy. In this demo, we will bring together data from flat files, STAC collections and ArcGIS online. Once Boson pulls the datasets together in a uniform way, we can serve them out in a variety of ways, depending on our needs.

Setup

import geodesic
geodesic.set_active_project('docs-example')

Load the datasets

In othertutorials, we have shown how to load a variety of datasets. Here, we will make use of a few of them. In particular, we will load one dataset from a STAC collection, one from a cloud bucket, and one from an ArcGIS online service.

Load a dataset from a STAC collection

First, let's load the STAC Collection. This is Sentinel 2 L2A raster data. You can examine the tutorial in more detail in the Adding a STAC Collectiontutorial, but we really only need a few lines of code:

stac_link = "https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a-cogs"
ds = geodesic.Dataset.from_stac_collection('sentinel-2',stac_link)
ds.save()

Load a dataset from a cloud bucket

Next, let's load a dataset from a cloud bucket. Again, you can examine thetutorial in more detail in the Adding a GeoJSON Dataset tutorial, but we can load the dataset like so:

ds = geodesic.Dataset.from_tabular_data(
    name='uscb-pop-centers',
    url='gs://geodesic-public-data/CenPop2020_Mean_CO.geojson',
    index_data=True,
    crs='EPSG:4326',
    )
ds.save()

Load a dataset from an ArcGIS online service

Finally, let's load a dataset from an ArcGIS online service. Again, you can examine thetutorial in more detail in the Adding an ArcGIS polygon_features Layer Dataset tutorial. Here is the relevant code:

url = "https://geodata.epa.gov/arcgis/rest/services/OEI/FRS_INTERESTS/MapServer/7"
ds = geodesic.Dataset.from_arcgis_layer(name = "epa-frs-locations", url=url)
ds.save()

It is worth noting that we don't strictly need to pull this dataset into Boson if our intention is to add it to ArcGIS online, since it is already an ArcGIS online service. However, the point of this tutorial is to show that, once we have connected to a dataset with Boson, it does not matter what the source was. We now have a uniform way to interact with the data, and we can serve it out in a variety of ways, including but not limited to ArcGIS Online.

Bringing the datasets together

Now that our disparate datasets are saved in this project in a uniform representation, we can serve them out in a variety of ways. Here, we will serve them out to ArcGIS online.

First, let's retrieve the datasets we just saved. We can do this by using the geodesic.boson.dataset.get_datasets method. We can use the search parameter to query for the particular datasets we want. If we know the name of the dataset, we can use the geodesic.boson.dataset.get_dataset method.

from geodesic.boson.dataset import get_datasets, get_dataset

datasets = get_datasets()
datasets

sentinel2_data = get_dataset('sentinel-2')
uscb_data = get_dataset('uscb-pop-centers')
epa_data = get_dataset('epa-frs-locations')

Now, we can create share links to serve the data.

Serving the Datasets to ArcGIS Online

We can serve the datasets out in a variety of ways. Let's start by simply creating vector tile services for each dataset, and then we will serve them to an ArcGIS Online map. We will serve the Sentinel 2 data as a vector tile service, which will display the footprints of the imagery. Because Sentinel 2 data is large and contains 10 bands, we will first have to create a view of the dataset that is appropriate for serving as an feature service. Views allow us to apply a persistent filter to a dataset. Here, we will create a view that only includes the red band and a bounding box around Pennsylvania, USA for a 20-day period in January 2020.

import datetime

bbox = [-80.463867, 39.707187, -75.102539, 41.983994]
start_time = datetime.datetime(2020, 1, 1)
end_time = start_time + datetime.timedelta(days=20)

asset_bands = [{'asset': 'B04', 'bands': [0]}]

sentinel2_view = sentinel2_data.view(
    name='sentinel2-pa',
    bbox=bbox,
    datetime=[start_time, end_time],
    asset_bands=asset_bands,
)

sentinel2_view.save()

Now, we can create share links to serve the data. First, we create a share token. You can set an expiration time, in seconds, for the token. Here, we will set the expiration time to 3600 seconds (1 hour). Leaving the expiration time blank will create a token that never expires.

sentinel2_token = sentinel2_view.share_as_arcgis_service(3600)

Then, we can retrieve the URL for the service. Here, we will use the get_vector_tile_service_url method to get the URL for the vector tile service.

Alternatively, you can share features in ArcGIS with the methods

:meth:get_feature_layer_url, and :meth:get_feature_service_url.

Furthermore, Geodesic supports serving the data as an image service with the :meth:get_image_service_url method. This will be covered in anothertutorial as it requires additional steps.

sentinel2_url = sentinel2_token.get_vector_tile_service_url()

Next, we will create vector tile feature services for the other two datasets.

uscb_token = uscb_data.share_as_arcgis_service()
uscb_url = uscb_token.get_vector_tile_service_url()

epa_token = epa_data.share_as_arcgis_service()
epa_url = epa_token.get_vector_tile_service_url()

Finally, we can serve the datasets to an ArcGIS Online map.

Adding the Datasets to an ArcGIS Online Map

You can add the vector tile services to your ArcGIS Online account by using the urls we just created. First, add a new item.

Then, select "URL".

Paste the URL for the vector tile service into the URL field.

print(sentinel2_url)

# Example output:
https://api.geodesic.seerai.space/ted/api/v1/share/1b0a6922868f493139106baf7cb2c85ec3b5e2343f9bb3798a25dbc5874c4730/rest/services/sentinel2-pa/VectorTileServer

Edit the title, folder and other options if desired, and click "Save".

Now your vector tile service is available in your ArcGIS Online account as a tile layer.

You can add the other datasets in the same way, or you can open the Map Viewer and add them directly to a map. Here, we will open the Map Viewer to view the Sentinel 2 imagery footprints. By default, the Sentinel 2 imagery footprints will be opaque, as shown below.

You can edit the layer using ArcGIS online tools, which will make the footprints visible.

And we will add the other two datasets using the "Add layer from URL" option in the "Layers" pane.

And there you have it. Using Geodesic, we've easily assembled three datasets from three different sources and served them out to ArcGIS Online in a uniform way.