February 13, 2017

Sea to Space Trek: Oceans, The Final Frontier

Posted by larryohanlon

This is the latest in a series of dispatches from scientists and education officers aboard the Schmidt Ocean Institute’s R/V Falkor. Read more posts here, and track the Falkor’s progress here.

By Benjamin Knorlein

Benjamin Knorlein is a Computer Scientist from the Center for Computation and Visualization from Brown University, he is working on the design of software that will enable scientists to study plankton through virtual reality. Image credit: SOI/ Mónika Naranjo González

Every great ship needs a Holodeck. Not for entertainment, but for science. Star Trek‘s fictional Holodeck can create matter virtually (in our case, it would be scientific data) which can be seen, touched and interacted with. Unfortunately, real technology is not as advanced as we know it from TV, so for now we skip the touch. But there is already a lot we can do within Virtual Reality to immerse ourselves in the data we collect by seeing and interacting with it in a virtual world.

What do we need to get the Falkor ready for exploration?

  • Get a commercially available Virtual Reality system and install it in the science control room – Check.
  • Acquire data. Lots of data. And then even more data. – Check. Check. Check.
  • Step into the virtual wonderland of science – Beam me up, Scotty!

Become a Phytoplankton

On board we have holographic microscope. (Yes, holographic!) In contrast to a normal microscope, the recorded holograms can refocus the microscopic image at different distances to the camera. So instead of a normal image, we acquire three-dimensional information about the location of phytoplankton in the water column. By using the the high-performance computers onboard Falkor, combined with algorithms I wrote, we can identify the kind of phytoplankton we are looking at and know its size and its location. When we plug the data in Virtual Reality, we can wander in the phytoplankton universe and see how the little organism organize themselves in the water. Scientists – who usually stare at a screen looking at phytoplankton for hours – can suddenly become part of the world they study so hard, thereby developing new perspectives, including the perspective of a phytoplankton.

This is a holographic picture of a Diatom Chain as it looks when it is captured by the camera that descends together with the CTD Rosette. Image credit: SOI/ Ben Knorlein

This image merges all of the plankton organisms found in a drop of water the size of a needle’s head. The image is now ready to be processed to become part of a virtual reality environment. Image credit: SOI/ Ben Knorlein

Take a Hike on the Seafloor or Enter the Twilight Zone

The R/V Falkor has two high resolution multibeam echo sounders. The sonars use sound to identify seafloor features in 3D. By plugging the data into Virtual Reality we can walk on the seafloor and become geologists of the ocean.

A lot of our equipment on board samples and records data in the twilight zone of the ocean – the region below the sunlit surface. The ‘wire walker’ is an instrument currently being used by the experts here – it moves up and down on a wire 150 times a day, measuring data such as temperature, salinity, pressure, oxygen, fluorescence of the phytoplankton, sunlight, and turbidity. Quite a busy explorer, this wirewalker. Taking this data combined with its GPS reporting and plugged into the Virtual Reality we can explore the depths of the ocean where no one has gone before.

As the Sea to Space expedition progresses, on board personnel on board enjoy immersing themselves in the plankton communities collected through holographic cameras through virtual reality. Image credit: SOI/ Melissa Omand.

Virtual Reality and New Scientific Possibilities

As you can see, Virtual Reality offers new possibilities to interact with scientific data. The experience is often more personal and helps to develop a different relation with the data as we coexist with it in the same world. We are suddenly not separated from it by a computer screen, but can enter its universe. We can shrink to the size of phytoplankton or we can walk deep in the oceans. We can create and modify matter and can interact with the target of our research not by using a mouse or keyboard, but by using natural interactions, e.g. we simply move our head or walk around to see the data from different perspectives.

We might also play a round of minigolf or draw phytoplankton in 3D. All, of course, for pure science.

This post originally appeared on the Schmidt Ocean Institute blog.