This post was first published by the Center for Climate and Life, a research initiative based at Lamont-Doherty Earth Observatory.
Earth scientists collect vast amounts of information from the myriad instruments they deploy in the field — drones, sensors, satellites — but computers and storage infrastructure are often unable to keep up with the scale and speed of this data, resulting in serious science bottlenecks.
Real-Time Earth (RTE), a Lamont-Doherty Earth Observatory strategic initiative, was created to address this problem.
The initiative is the brainchild of Lamont scientists Ryan Abernathey, a physical oceanographer; Tim Crone, a marine geophysicist; and Chris Zappa, an oceanographer. Each researcher brings unique expertise to the endeavor: Crone is an instrument designer, Abernathey is focused on big data, and Zappa specializes in unmanned aerial systems or drones.
Lamont scientists are trailblazers in observing Earth in real time. They place scientific platforms and sensors under the sea, on the ground, and in the air — all of which result in volumes of data. For example, the Cabled Array, part of the National Science Foundation Ocean Observatories Initiative, is an undersea volcanic observatory in the northeast Pacific Ocean that uses seafloor and water column instruments to relay information about underwater earthquakes and volcanic eruptions. Another project, the Lamont Air-Sea Interaction Laboratory, deploys unmanned aerial vehicles to observe the ocean surface, which helps scientists develop better models of air-sea interactions.
RTE supports Lamont scientists and the Observatory’s other initiatives, including the Center for Climate and Life, by increasing their technological capabilities. It’s made up of two centers within Lamont.
The Technology Innovation Center helps researchers realize their ideas by engineering innovative technologies and devices and connecting them to the internet. It provides resources and produces the next-generation equipment needed to help researchers design global Earth observation experiments.
The Data Science Center enables scientists to visualize and analyze huge, complex volumes of data more easily.
“Now we’re able to iterate much more quickly and make new discoveries. And there are a lot of new discoveries that are coming out through the use of these tools.”
In the past, scientists would need to find data, download it to their computers, and then write code to analyze it. The desire to make this process more efficient was a motivator for Abernathey.
“I was observing my students and post docs spending all their time on this really menial work of transferring data, moving data around, and waiting for really slow calculations to happen,” he said.
Abernathey is now developing the computational infrastructure required to understand how to work with big data via cloud computing. This involves renting computers from Google, Amazon or Microsoft; using their platforms to hold the data; analyze it; and processing it with machine learning — a type of artificial intelligence — that can discern patterns and relationships in data.
Abernathey said the advantage of using many computers working in parallel in a network is that they “accelerate by orders of magnitude the rate at which we can do calculations.” This is crucial because science is a process of trial and error: Scientists have to test hypotheses, see if they work, and repeat this process numerous times.
RTE also cultivates interdisciplinary science on the Lamont campus and further afield. A benefit of working in the cloud is that anyone can access the data and code and reproduce the methods the scientists are using. “This means it’s a lot easier to share things with colleagues and have them actually pick up and reproduce your work… and put all of our scientific processes out in the open,” Abernathey said.
A key component of RTE is the Computational Technology Intern Program, which connects undergrad and graduate computer and data science students with Lamont scientists. The program was established with a gift from Todd Sandoz, managing director of Barclays Investment Bank, who was brought in by the Center for Climate and Life.
The intern program provides a unique opportunity for the interns to gain hands-on experience with challenging scientific research, working in collaboration with Lamont scientists to develop solutions to big data problems through cloud computing.
The 2018 RTE interns worked on three ongoing research projects. Crone leads a project that uses seafloor sensors to study the evolution of bacterial colonies under the influence of geological and geophysical processes. Abernathey leads a project that’s creating a catalog of datasets that can be efficiently searchable through Pangeo, which is focused on developing software and infrastructure to deal with big data. Zappa heads a project that uses machine learning to develop models that show interactions between the ocean and atmosphere to better predict hurricanes. Three new interns will be paired with these projects in 2019.
The donor support for RTE is a prime example of what Peter de Menocal, director of the Center for Climate and Life, sees as a new model for funding scientific research. At the moment, Lamont research is funded primarily through federal support.
“It’s a model that’s currently under threat, so we’re working to diversify the research funding portfolio,” de Menocal said. “Philanthropic support allows us to pursue urgent, mission-critical science.”
The Center for Climate and Life does this by finding donor support for solution-oriented projects, such as RTE. This model benefits scientists who pursue “high risk, high-reward” research because they sometimes have trouble obtaining federal funding if their research is considered too unconventional for standard research grants. Yet these types of projects can also lead to transformational breakthroughs with real-world implications, across scientific disciplines, as is the case with RTE.
“Now we’re able to iterate much more quickly and make new discoveries. And there are a lot of new discoveries that are coming out through the use of these tools,” Abernathey said.
Crone also stressed that initiatives like RTE can facilitate scientific advances. “Scientists have a lot of really great ideas that can be put into motion if they are given the kinds of tools that the Real-Time Earth initiative seeks to put in place,” he said.
“The support from Todd Sandoz kick-started our move into the cloud, and it’s a huge thing. It’s a real revolution in terms of how we’re going to do Earth science going forward.”
