A SmallSat AUV Network

There’s more to a global network than satellite links, and yet sat-linked remote operations experts can heap huge scientific and technological gains on the world. That’s what’s happening in the background — a smallish network of the best scientists from around the world are orchestrating the communications and robotics that’ll yield groundbreaking achievement in the networked use of autonomous underwater vehicles, or AUVs.

If you contact Professor Kanna Rajan, as we did, he’ll tell you he’s not the world’s foremost “drone” researcher, yet he’s programmed at least two Mars missions and advises a transatlantic network of scientists on the use of AUVs and satellites. Prof. Rajan suggests AUV-network control guru, Professor João Tasso De Figueiredo Borges De Sousa of the University of Porto, is the go-to guy if you’re contemplating what’s possible in the world of AUVs. Dr. Rajan is affiliated with Porto in Portugal, and he’s quoted at length (about 50 times) in a vision statement written by the Norwegian university NTNU on their planned use of small satellites, or SmallSats, to control AUVs. NTNU post-docs, Ph.Ds and masters students are nearing their SmallSat launch day, after which North Sea AUVs will have taken a step toward being able to relay data to scientists onshore for long periods. Both SmallSats and AUVs will have hyperspectral cameras able to detect chemicals, metals, plankton or do survey work.

The California-based Prof. Rajan is a senior scientist known for planning, executing and creating autonomous systems and robotics at NASA’s Ames Research Center. Prof. De Sousa leads an international network of AUV-keen scientists akin to NTNU’s. Both have highly evolved AUV labs. The three research communities have combined to make lasting AUV operations controlled or augmented by Micro or Nano satellites — SmallSats of 10-100 kilograms or 1-10 kg — a reality. Apart from the command and control of networked AUVS (and other drones) en masse, which is Prof. De Sousa’s specialty, it is hoped that SmallSats will do for ocean understanding and marine operations what the Internet did for marketing and instant knowledge.

AUV control
“Most of our work is joint, and strategically we all (NTNU, Porto, NASA) are moving in the same direction,” Dr. Rajan says, adding, “So this is both institutional and collective.” Rajan admits switching from Mars rovers to “piloting” AUVs at sea was harder than expected. “The infrastructure came with the tasks associated with commanding the Rover in 2004 and the Deep Space 1 spacecraft in 1999. But the nitty gritty of getting to a coastal zone on a reasonable vessel in weather that can be harsh and not trivial to work with, is something most people don’t understand … So yes, both the Rover and an AUV are an extension of the human senses per se, but there’s a lot more work involved in getting to sea and making things work.”

That’s why Prof. Rajan, Prof. De Sousa (who we failed to contact) and their NTNU colleagues are so looking forward to launching SmallSats: detailed, long-standing ocean examinations. That focus will revolutionize ocean research. In the networks of this heavyweight science network, the limitations of AUVs are acutely known, and researchers from Hawaii to Hong Kong will attest, that it is harder to communicate with AUVs than it is with space probes and their “infrastructure of Deep Space Network for spacecraft.” While “power is an important shortcoming”, too, it’s the communications problem that keeps this triad of researchers awake at nights.

AUV networker: Professor Joao Tasso de Sousa aboard the Schmidt Ocean Institute vessel, Falkor. Photo: Schmidt Ocean Institute

Portuguese explorer
Rajan is right about Prof. Tasso: “The superpower in AUVs and marine robotics is our close friend and collaborator, Joao Tasso (De Sousa).”
In fact, Portugal is a powerhouse of AUV development and research. Apart from hosting the biennial IEEE OES Autonomous Underwater Vehicle Symposium, the country boasts both Tasso’s Underwater Systems & Technology Laboratory, or LSTS, and the researchers of INESCTEC. Prof. De Sousa recently led a Schmidt Ocean Institute expedition aboard their Falkor research vessel that explored ocean salinity fronts using multiple areal and underwater robots. He’s also been known to lecture NATO navies on combined AUV operations.

De Sousa is a pioneer of and continues to work on the managing of multiple autonomous vehicles simultaneously. He has helped develop commercial and researcher software which incorporates mobile devices for easy control of networked AUVs. While applications might one day include mass ocean surveillance, mass inspection, mass survey or massive attack, the oceanic study involved AUVs (and aerial drones) in contact with each other via a ship-based ocean controller. SmallSats could provide these expeditions with the researchers own’ dedicated bandwidth (understood to only be NTNU, for now).

Networking AUVs need open-source control software like that developed by the U. of Porto’s LSTS. Several AUVs communicating via their own satellite can, with their onboard sniffing payloads, relay in real-time an accurate picture of the dramatic changes developing across a vast ocean area: or they can confirm by sensing, sampling or photographing what the SmallSats sense or see. The Schmidt Institute chronicled Prof. De Sousa teams deploying “several autonomous vehicles to find, track, and sample various physical, chemical, and biological features of the ocean, on time-space scales not previously possible by ship or aircraft studies alone”. Given their own spectral cameras, the SmallSats would then add another layer of real-time data.

During the Schmidt journey, “onboard AI,” Rajan’s shared area of interest with Prof. De Sousa, was used to coordinate AUVs, unmanned aerial vehicles and autonomous surface vessels doing “complex sampling tasks”. Multiple vehicles “at times” were said to have worked in “combined effort”, monitored and controlled by software dubbed Ripples and Neptus. To do that over the long haul, you need dedicated bandwidth for the expected torrent of ocean data. To get that, your AUVs need their own SmallSats in space.

Adding depth: the full range of ocean views. Illustrations courtesy Professor Kanna Rajan

NASA-quality supervisor
Enter the Norwegians. Funding that’s hard to fathom, for some, and experience storing and processing vast volumes of data (like the results of seismic surveys) are part of Norway’s contribution to AUV and AUV-based research.

Add to that a growing SmallSat faculty, two AIS satellites and a handful of “space” outfits, including some with military ties. While the researcher haul of all three “schools” — Porto, NASA and NTNU — is large and international, it’s the Norwegians and their invited talent that’ll provide the SmallSats triad’s space-side boost. There’s also NTNU students with satellite launch experience, and then there’s the supervision of Prof. Rajan with his combined space and robotics experience. Rajan is listed as an NTNU employee, a NASA employee and an associate of the U. of Porto. He executed NASA’s 1999 New Millennium Deep Space One Remote Agent Experiment (RAX) and was principal investigator of the 2003 Mars Exploration Rovers MAPGEN controller for the twin rovers — “the longest running AI system anywhere and still in-command of the Opportunity rover on Mars”, according to an NTNU text. He was also principal researcher for autonomy at the Monterey Aquarium Research Institute, where he designed, built and deployed the AI-based embedded T-REX autonomous controller on AUVs, as well as a shore-based support system!

Cost-efficient comms

While the Norwegian AUV lab at NTNU is poised to log the great achievement of a SmallSat launch at sea, the funding of the Norwegians cannot be overstated. The finance outmatches EU funds, and it does for AUV and oceanographic research in Norway what the Schmidt Institute did for Porto’s ocean fronts study and the work of other oceanographic researchers.

“Funding is a serious issue in doing marine robotics and/or ocean science,” Prof. Rajan writes. “In the US, we have multiple agencies and stakeholders who believe they need to do something about the ocean. This includes NSF, NOAA, DARPA, ONR all of whom have interests which intersect and compete. In Norway, this is less so, but (the Research Council Norway) has done a good job in streamlining the funding process. Not so in the US. Contrast this with going to space — there is really one entry point. And that is NASA. So the process of sending up a spacecraft is a lot better understood, clearer and more reasonable (via the US National Academies) than going to sea, quite often.”

Prof. Rajan says space agencies also tend to engineer the hell out of their vehicles while spending “enormous sums of money” on duplication and robustness. That’s not quite the Norwegian SmallSats approach. In their SmallSats whitepaper, NTNU acknowledges a need to focus on small dedicated satellites with cameras and coms rather than asking for periodic bandwidth on a more robust but costly commercial satellite. However giving to research, NTNU knows that the state’s finance isn’t always forthcoming. So, they’ve focused on a SmallSat program that’ll cost between USD 100,000 and USD 7 million.

Cash is king, even for this network of giants. However, AUVs make ocean research of scale affordable. In a mission brief for the Schmidt Ocean Institute, Prof. Joao Borges de Sousa wrote the following: “We need to develop sustained ocean studies with unprecedented spatial and temporal resolution. Ships alone cannot help to achieve these levels of resolution, scale, and economic feasibility.” He might already have been thinking “SmallSats”, and NASA’s Prof. Rajan is affiliated with both NTNU’s Center for Autonomous Marine Operations and Systems (AMOS) and Porto’s Underwater Systems and Technology Laboratory.
Sometime between this spring and 2020, a Norwegian launcher will send up a SmallSat, and then there use is expected to expand in tandem with 

the proliferation of AUVs. A Norwegian Space Centre already supervises satellite launches in Norway, including those by Norwegian students who design their own micro satellites (under 1 kg). NTNU’s stated vision is to be a SmallSats leader, starting with the hyperspectral camera they built using off-the-shelf parts and parts they 3D-printed themselves.
Apart from NTNU, Norway’s near-arctic, island Space Centre is getting ready to expand its SmallSat launch program. Defence outfit, Norske Nammo, is reportedly readying an eco-friendly micro-rocket to carry SmallSats into pre-defined orbits.

Aerial and underwater: a range of AUVs and aerial drones were networked during a Schmidt Ocean Institute survey of ocean fronts. Photo: Schmidt Ocean Institute