Introduction

In the aerospace industry, there are two types of data: (1) synthetic data (e.g., models and simulations) or (2) experiments. Each has their own associated costs; while modeling and simulation require high-performance computers, graphical processing units (GPUs) and specialized software, empirical experiments require time, money, hardware, and people. As we’ve discussed, synthetically created scenes can be very cheap to build but they can also be very inaccurate compared to the real world. The truth is that both methods are needed truly validate the other and progress our scientific understanding of the world around us.

For a Data Scientist like me, experiments like the one discussed below offered a rare opportunity to collect the data I’ve spent years processing. The goal of the experiment — studying the ionosphere — was simple, but the execution took years to plan and bring together.

Why? because the behavior of the Earth’s ionosphere is tricky to study. Consider for a moment that you live in Miami, FL and you want to have a friend living in Haiti. If you wanted to send them an electromagnetic birthday greeting you could broadcast a signal from your back yard directly to your friend! But to do so, you would have to reflect your signal off of the Earth’s ionosphere. So, you point your Miami antenna just East of Cuba, transmit your signal, and your friend receives the signal on the other end.

Based on the strength of the transmitted signal and the strength of the received signal, your friend can calculate the strength of the ionosphere’s reflectance as a function of frequency, but they can only calculate the reflectance at one point. If you asked your friend to then send you a map of the spatial reflectance, their plot would only contain a single point instead of the heatmap you were hoping for. To get more than two points, you would have to launch your signal at different angles, and your friend would have to likewise collect them at different locations!

A multi-regime campaign

This, in essence, is what Space Dynamics Laboratory (SDL). In a coordinated campaign, SDL transmitted RF signals from different locations, and collected the reflected signals at different locations! In the experiment, transceivers were placed on land, in the water, and in the air, all location- and time-coordinated. Locations of three of the sensors are plotted below.

Each land-, sea-, and air-based transceiver was further equipped with cameras, video recorders, and time-locked GPS units so that data from each location could be compared at the same instances in time! At the end of the experiment, there were petabytes of data that needed to be ingested, fused, processed, and interpreted. Two images similar to those taken during the campaign are illustrated below.

Although not taken at the same time, the image on the left is one of the images captured of the RF-reflective plume. The image on the right is an ionogram of one of the signals used during the campaign. Although axes have been removed from the ionogram, these data typically tell us the altitude, spectral reflectance, and “shape” of each ionospheric layer.

But how do you process each data type much less fuse the data together?

Data fusion

I won’t get into the details here, instead I’ll go straight for the spoiler. Videos can be sliced into individual images, and image processing can be used on each image to (a) isolate, (b) threshold, (c) fit, and (d) plot the time evolution of the cloud’s shape. These steps are illustrated in the image below.

So now we’re able to plot the time evolution of the major and minor axes of the plume, but what does that buy us? From the point of view for the current data camera, we can begin to back out different parameters such as:

• Plume shape over time
• Plume density over time
• Plume chemistry over time (assuming data is hyperspectral)
• Prevailing wind direction at the plume altitude
• Reflectance as a function of density

We can then begin to fuse the image-based metrics with data extracted from the ionograms. Recall that the images and ionogram data are all being taken lock-step in time. This means that any disruption to the ionosphere caused by the plume might show up in the ionogram — and if that’s true, then we can correlate the relative change in strength of the ionosphere layer with the size, density, composition, and shape of the plume at the same moment in time! This data can be further coupled with weather data recorded on each transceiver’s platform to account for atmospheric visibility, absorption, and other factors.

Conclusion

Campaigns of this scale occur infrequently; when they do occur, the sponsoring companies make sure to record all the data they can. By front-loading the project with data collection, the company is then able to spend the following months and years processing the petabytes of data!

In the case of SDL, the event occurred in a single evening, but the campaign could have been extended significantly. As it is, the project was an incredible success! The birth, life, and death of the plume was recorded on 10+ sensors, and the final report was highly insightful, detailed, and well received by the funding agency. It was a great experience for me, as well!