What is Middleware?

When the Geodesic Platform accesses data, it does so in a decentralized manner. This means that it reaches out to where the the data is instead of copying the data to a local database or bucket. This allows your data catalogue to grow well beyond your cloud storage budget. When Geodesic serves data to you, the requested data first flows into the platform in order to be translated into the format that whatever client you are using is requesting.

Normal flow of data through the platform using Snowflake and Esri as an example. When a user requests data, Geodesic knows how to make the request to Snowflake (or any other data source). Once Snowflake returns the data, Geodesic translates it into the desired output format - in this case, Esri Geoservices. If the same request has been made before, Geodesic's smart cache is used to return the results even more quickly.

While the data is flowing through the platform, we can perform operations on it. This is where Middlware comes in.

Middleware allows you to modify the data, perform calculations on the data, or otherwise change the data into what you want to see in your application. This all happens on-the-fly and only on the data that flows through our platform as requests are made. No more copying gigabytes of feature data over to your own database just to rename a column so that it works with your downstream system.

Here, we see the same workflow as before, except Middleware is operating on the data as it flows through the platform. Middleware can perform lightweight transformations on the data so that it works better with your downstream applications.

What Kinds of Things Can Middleware Do?

The types of data that flow through the platform can be grouped into two main categories: raster/image/array data and tabular/feature data. With raster data we can perform operations like scaling or shifting values, mathematically combining different arrays or setting map defaults. Feature data can have its columns renamed, its geometry transformed or new columns calculated. For a more complete list of available operations, see the Middleware Types page.

What Kinds of Things Can Middleware NOT Do?

Middleware is not designed to do extrememly heavy weight computing like deep learning or large matrix operations. It is designed to be fast and lightweight so that it can operate on data as it flows through the platform. If you need to do heavy weight computing you want to use the Tesseract part of the Geodesic Platform.

To get started building Middleware, see the Middleware Builder Intro.