Meet NOUS: An Underwater ‘Artificial Mind’
The establishment of Maritime Protective Areas (MPA) and marine archaeological sites has been an objective of the Ministries of Culture and Environment in Greece for a long time. One of the main problems that delayed the implementation specifically of the "underwater museums", was the way they were protected. Usually in ancient shipwrecks, only their cargo is preserved, which is mostly amphorae, that can be easily stolen.Therefore it was both a necessity and a requirement for a system which should be self-powered (since in most of the places where wrecks are located there is no electrical power supply), able to connect to the internet with all available technologies for communication and remote control, have sensors which can monitor the site of interest continuously and alert appropriately in case of violations, tampering or other events.Such a system could have many useful applications, such as monitoring and surveillance of marine protected parks, diving parks, sea farms; make real-time scientific observations throughout the day and over long periods of time and monitoring of biodiversity and climate change.At the beginning of 2019, the design of the Underwater Visual Observation System "NOUS" (uNdersea visiOn sUrveillance System, which in Greek means “mind”) was launched, which was successfully completed a year later."NOUS" allows continuous (24/7) monitoring and protection of an underwater site using machine learning and image and sound processing algorithms.Briefly, it is a system consisting of an underwater array of self-cleaning cameras and microphones, which are either connected by optical fiber and electrical power conductor to the nearest shore, or to a floating platform on the sea surface. On the shore or on the platform there is a photovoltaic panel power supply and communication devices for Internet connection.On the occasion of the successful completion of the initial operation phase of the "NOUS" system in Alonissos, more detailed information and perspectives are presented in this article.The system was installed in February 2020, at the ancient shipwreck of Peristera, at Alonissos island, Northern Sporades, Greece, providing for the first time worldwide the capability for continuous monitoring of an underwater museum. The wreck is located 200 merres from the rocky islet of Peristera and at a depth of 21 to 33 meters. It is a merchant ship loaded with thousands of amphorae from Mendi (the ancient city of Halkidiki) and Peparithos (today's Skopelos), areas known in antiquity for their wine. The pile of amphorae, which extends over a length of 25 meters, gives a sense of the contour and large dimensions of the ancient merchant ship. The ship is described as one of the largest of the classical period in ancient Greece. It is estimated that it could carry about 4,000 amphorae and had a displacement of 120 tons. The pointed bottom (oxypythmenos in Greek) amphorae are the most common commercial transport vessels of antiquity, weighing about 8-10 kg and having a capacity of 15-25 liters. They are perhaps one of the most ingeniously invented and practical vessels of antiquity, having two handles to aid transportation, a narrow high neck and a sharp end to allow them to be safely stowed in the hold. The spectacular shipwreck proved that large merchant ships of over 100 tons were already sailing in the Mediterranean in the 5th century BC.The prototype system in Peristera started its pilot operation in March 2020. The project was commissioned and funded by the Ephorate of Underwater Antiquities / Ministry of Culture & Sports under the European program "Bluemed MED 2014-2020".After the opening on August 3, 2020, divers from Greece and around the world can now dive in Greece's first underwater museum in Alonissos.The system consists of an onshore section at Peristera and an underwater section at the wreck site. Between the two sections there is an interconnection consisting of an optical fiber and a copper power supply.All images courtesy Dr. George Papalambrou, Assistant Professor, School of Naval Architecture and Marine Engineering, National Technical University of AthensOnshore sectionThe electricity of the system is provided by a fully autonomous system of photovoltaic panels and batteries, installed in Peristera. The sizing of the PV system was based on load requirement estimates, meteorological data for the area and the requirement for electrical autonomy for at least up to 7 days.In terms of communications, WiFi at a frequency of 5 GHz is currently the main form of interface with the remote control station and is carried out via a point-to-point link between Peristera and Steni Vala.At the land station, visual monitoring of both the site and the sea surface of the wreck is carried out by means of a remote-controlled infrared camera. In addition, there is a meteorological station for measuring atmospheric parameters such as air temperature, barometric pressure, relative humidity, wind speed and direction.There is also an appropriate electrical power distribution and protection circuit and devices for measuring the voltage and current of the power lines, all monitored by the local computer.Underwater sectionThe underwater image transmission system consists of five cameras and one network switch. Each camera is easily adjusted in height and bearing so that it covers ("sees") the area of interest during the underwater installation. All cameras send video (stream) in real time over the network for processing to a remote station via the internet.The underwater cameras are digital, color, state-of-the-art, with high-resolution technology and have a wide-angle lens. Each camera is connected to a local computer, which uses appropriate software to monitor its operating parameters. It is also possible to install additional instruments such as measuring sensors for the abiotic environment, underwater lights for night observations, lasers, infra-red cameras, etc.The computer communicates via an Ethernet network with the other devices in the system, for transferring images and measurements (ie. electrical quality, speed and network status, etc.). It has a Linux operating system which allow to add applications, change settings remotely and secure access via ssh protocol. The computer has a custom electronic circuit designed for voltage and current measurements, both on the power line and on other consumers (wiper, lights, etc.).The camera, lens and electronic parts are mounted in a watertight housing (camera housing). The housing is made of thermoplastic, with a removable front part made of transparent crystal and an O-ring seal. The ethernet switch and the fiber-to-copper converter are also located in a sealed hub housing.The removal of all individual parts is by design easy, allowing for on-site maintenance or replacement of parts of the system without having to lift out of the sea the entire installation.Wet plugs suitable for the marine environment have been fitted, allowing the cables to be connected and disconnected in the water. The two enclosures were designed entirely for the needs of the project, taking into account parameters such as depth, mounting, oxidation, cathodic protection and resistance to ultraviolet (UV) radiation.Particular attention was paid to the ability of disconnection - cut-off of the system. So in case that a fishing gear gets entangled in one of the cameras (housing), or "catches" a cable branch, then by the tensile force exerted, this particular branch can be disconnected from the rest of the overall installation without pulling apart and eventually losing the whole system.An automatic cleaning function of the crystal has been provisioned in the camera housing that removes marine deposits that develop over time and reduce the effectiveness of the optical system. Thus, a wiper system made for the needs of this particular housing was installed, consisting of a sealed servo motor and a cleaning arm with a rubber cleaning material. The cleaning system is fully and autonomously controlled (via image processing algorithms and optical system quality diagnostics) by the camera computer. During the design phase, the practice of 3D-printer manufacturing was followed for all components to ensure full size functionality. All images courtesy Dr. George Papalambrou, Assistant Professor, School of Naval Architecture and Marine Engineering, National Technical University of AthensSoftware applicationThe "NOUS" system has a wealth of automatic monitoring and protection capabilities through machine learning algorithms and image and sound processing. It is thus possible to detect any presence in the area of interest, such as divers, remote-controlled robots-ROVs, as well as the slightest light source in the dark. In particular, any alteration of the area of interest can be identified. In addition, an appropriate algorithm counts and detects the slightest change in the image on the camera housing crystal and, if necessary, immediately activates the wiper.In all the above cases, an appropriate message is sent to selected recipients to evaluate the change in the area and carry out further investigation. The machine learning algorithms used allow both the recognition of the object and the determination of its position in the image (object detection). They can also identify and classify their findings into specific categories (classification).The Artificial Intelligence (AI) system on the "NOUS" uses neural networks that during the "training" phase were adapted to the data of the environment of the wreck monitored in Peristera, the land area of the equipment installation and the marine area on the surface of the wreck. Different operational scenarios in the wreck area were tested and video and image footage was collected in different phases, such as in calm conditions, with sea turbulence, during daytime and night diving, with use of lighting, ROV passage, different water column turbidities, with creation of debris on camera lenses, the passage of boats and other floating objects. PerspectivesAfter the inauguration (August 2020) there has been intense interest from Greek and international stakeholders, both from the diving public that visited the marine museum and from scientists about the potential of "smart" monitoring of marine life which can be continuous and in real time. The topic was covered by the world's major news agencies and media [1], [2], [3].After about one year of pilot operation, the system is gradually moving into the second phase of its development, during which it will be enriched with new classes: fish fauna and protected species living in the National Marine Park of Alonissos Northern Sporades (N.M.P.A.N.S.), with the prospect of extending the EEAX Peristera at Stratoni, Halkidiki. In this direction, image processing and classification tests for the fish fauna observed at the wreck site have already started.In April 2021, Microsoft has chosen to support the biodiversity conservation effort through "NOUS" by providing technical support on AI issues and access to its computing infrastructure in the form of a grant, supporting the proposal "Machine Learning and Real Time Monitoring for the conservation of marine life with emphasis on seahorses".More information can be found on the NOUS website at https://nous.com.grPublicationsL. Papadimas, V. Triantafyllou, Ancient 'Acropolis of the sea' opens to divers, guarded by high tech, Reuters, 30/7/2020, https://www.usnews.com/news/world/articles/2020-07-30/ancient-acropolis-of-the-sea-opens-to-divers-guarded-by-high-techM. Atmatzidou, Diving into Greece’s first underwater museum, National Geographic, 2/12/2020, https://www.nationalgeographic.com/travel/article/diving-into-the-first-underwater-museum-in-greece?cmpid=org=ngp::mc=social::src=facebook::cmp=editorial::add=fb20201202travel-greeceunderwatermuseum::rid=&sf240674775=1&fbclid=IwAR2PjCSz2WoJF0V1pgg1YsWl9ILtjy3M4r916F_rH_IPw0p-KVikgQINNuUΥ. Elafros, Science Culture, “Shipwreck surveillance gets Microsoft funding”, Kathimerini, 27/4/2021, https://www.ekathimerini.com/culture/1160037/shipwreck-surveillance-gets-microsoft-funding/?fbclid=IwAR27xKef-CL6jDZa8m--j1yBVhBfR0CNOY3tgwG_Fc-5uJ3Q0iV0UOFOtwQ