Tracking whales: AI to help researchers make more rendezvous

For several years now, researchers have been using machine learning and artificial intelligence to find out how the highly developed sperm whales in the oceans communicate with each other using complex click sounds. There are also "moonshot" projects such as Project CETI, which hopes to achieve "advances and breakthroughs in interspecies communication". The name, which stands for "Cetacean Translation Initiative", is not coincidentally based on the SETI project, which has been trying to find signals from extraterrestrial life forms since 1999. Scientists from Harvard University involved in Project CETI have now set up a new reinforcement learning network to solve a central problem in the research: The discovery of sperm whale meeting points in the sea in order to be able to adequately eavesdrop on the animals in the first place. The data required for this comes from robotic probes and drones.

Where will a whale appear next?

The novel framework described in the journal Science Robotics is called AVATARS, which stands for "Autonomous Vehicles for whAle Tracking And Rendezvous by remote Sensing", i.e. autonomous driving and flying vehicles that can locate whales and their rendezvous points using remote sensors. The study led by Stephanie Gil, Assistant Professor of Computer Science at the Harvard John A Paulson School of Engineering and Applied Sciences (SEAS), is intended to help "understand the complex communication and behavior of these creatures". This will make it possible to study sperm whales in their natural environment. The algorithms are based on methods that are more familiar from the city: the positioning of rideshare providers was the inspiration for this, with which vehicles are brought to places where potential passengers are located in real time.

The whales themselves are fitted with ultra-short wave transmitters in the form of small tags that are used to locate them. But even with these, the all-important surfacing of the animals, which signals an encounter, is often overlooked. The drones that are part of Project CETI capture the VHF signals and use them together with the movement of the drone to form an "antenna array in the air". This allows the direction of the received "pings" to be determined. If the algorithm works as desired, it is possible to use this various sensor data plus prediction models for diving behavior to determine when and where a whale is most likely to surface.

Accuracy of more than 80 percent at 500 meters

"Our autonomy module delivers an overall success rate of 81.31 percent for a rendezvous distance of 500 meters with three autonomous systems. A second class of experiments, in which only acoustic bearing measurements were taken to three unmarked sperm whales, yielded an overall success rate of 68.68 percent for a rendezvous distance of 1000 meters using two robots," the study states. However, the data had to be processed retrospectively.

The question remains as to when we will really understand the sperm whales. The astonishingly loud signals with over 200 decibels of sound pressure are even said to have vowels and diphthongs – albeit clicky. "In the first phase, a unique, large-scale set of acoustic and behavioral data will be generated to train CETI's technology to view whale communication in context and ultimately translate whale language," reads the Project CETI mission description. All of this is reminiscent of long-forgotten ancient languages whose script can no longer be identified today. In the case of the sperm whales, the first step is to collect sufficient data to record behavioral information in addition to the actual communication to build a form of language model. "This interdisciplinary work, which combines wireless sensor technology, artificial intelligence and marine biology, is a prime example of how robotics can be part of the solution to further decipher the social behavior of sperm whales," says first author and doctoral student Ninad Jadhav.

(dahe)