Proceedings of the 10th Mediterranean Conference on Control and Automation - MED2002 Lisbon, Portugal, July 9-12, 2002.

INTERNET-BASED SATELLITE TELEOPERATION OF THE ROMEO ROV IN ANTARCTICA

Ga. Bruzzone*, R. Bono*, Gi. Bruzzone*, M. Caccia*, M. Cini, P. Coletta*, M. Maggiore, E. Spirandelli*, G. Veruggio*
* Consiglio Nazionale delle Ricerche Istituto Automazione Navale Via de Marini, 6 16149 Genova Italy fax: +39-0106475600 e-mail: {gabry, ric, gio, max, paolo, edo, gian}@ian.ge.cnr.it Consiglio Nazionale delle Ricerche Servizio Reti e Telecomunicazioni P.le Aldo Moro, 7 00185 Roma Italy fax: +39-0649933159 e-mail: {m.cini, m.maggiore}@src.cnr.it Keywords: Underwater Robotics, Teleoperation, led to the birth and the development of the robotic teleoperation. Generally speaking, teleoperation is Internet. defined as the extension in a remote place of sensing and manipulation capabilities of a person Abstract [14]. Actually, robotic teleoperation is a growing In this paper the Internet-based satellite and highly-promising field of scientific research teleoperation system of the Romeo ROV (Remotely from the practical applications point of view. Operated Vehicle), will be described. Romeo is the Possible applications are innumerable and go from latest ROV prototype developed by Robotlab, the remote manipulation in dangerous or unreachable Robotics Department of the Istituto Automazione places for human beings (nuclear plants, oceans and Navale of Consiglio Nazionale delle Ricerche space explorations, etc.), to remote education, to (CNR-IAN) for scientific applications and robotics virtual visits to places of attraction (museums, research. Some teleoperation tests were carried out parks, etc.), to remote control of personal robots in on the period December 2001 - January 2002 in the the office or at home. The high level of maturity course of the XVII Italian Expedition to Antarctica. achieved by teleoperation in space robotics During the experiments, manifold users connected applications [2], together with the resolution of to the World Wide Web had the possibility to basic problems in Unmanned Underwater Vehicles remotely operate Romeo immerse in the Antarctic navigation, guidance and control [8, 9, 12], has sea nearby the Terra Nova Bay Italian Base. These suggested its extension to the field of underwater experiments are to authors knowledge the first robotics in order to allow a scientific end-user to example of satellite teleoperation of a ROV directly program, execute, and follow the state of progress of robotic vehicle missions at sea from the comfort usable by users surfing on the web. of his/her laboratory. A pioneer work in this field 1 Introduction was the experiment carried out by the NASA Ames Ever increasing human beings needs of remotely Research Center in 1993 austral spring. A underwater vehicle operating in unreachable or dangerous places have telepresence-controlled,

(TROV) was remotely operated via satellite from USA Ames labs and used to study sea floor ecology in Antarctica [16]. Moreover, recently, thanks to the rapid growth and ever-increasing diffusion of Internet, manifold experiments have been carried out trying to use Internet as a support for teleoperation. The first experiment in which a robot was connected to Internet was performed by Ken Goldberg at the University of California at Berkeley in 1994 [10]. In that case Internet was used as an infrastructure for teleoperating a SCARA type robot by means of a web browser. This allowed, for the first time, to focus attention on and start studying problems connected to the remote interaction with robotic structures by means of Internet. Among the various international Internet robotic teleoperation projects which followed, we can also remember the Australia s Telerobot of the University of Western Australia which could be used, by anybody having a web connection, to teleoperate an ASEA IRb-6 robotic arm manipulating small wooden blocks [17]; the Web Interface for Telescience developed by the Jet Propulsion Laboratory accessible by the public for the planning, simulation and execution of the Mars Pathfinder missions on Mars [2]; the experiment s carried out by CNR-IAN in collaboration with ISTISR of Lisbon where the underwater robotic vehicle Romeo performed a mission in a pool in Genoa guided via Internet by an operator in Lisbon [5]. The above-mentioned research aims at developing reliable teleoperation of remote robotic systems over communication channels characterised by nondeterministic and time-varying delays and at finding methodologies for easy sharing of robotic resources using the World Wide Web. Actually, this brings closer to end-users, such as oceanographers, biologists and geologists, in the case of marine applications, the perspective of controlling from their labs the execution of scientific missions carried out by robots operating in remote and harsh environments. In order to evaluate the possibility of executing reliable Internet teleoperation of remote robotic systems in actual operating conditions, some preliminary experiments of Internet-based satellite teleoperation of the Romeo ROV in Antarctica were performed. Given the preliminary phase of the research, the

experiments focused on verifying the possibility of piloting the remote ROV from a generic Internet interface by non-specialised end-users rather than on the execution of an Internet-controlled scientific mission. Roughly speaking, the basic requirement for remote missions in unknown unstructured environment was mainly evaluated, leaving to further experiments and research the development of tools required by more specific scientific applications. An overview of system requirements and architecture is given in section 2, where a particular attention will be paid to describe the main modules of a robotic teleoperated system. In section 3 the experimental set-up and a report of carried out experiments are presented. Finally, some final conclusive remarks and discussion of current and future research, given in section 4, will conclude the paper.

System requirements architecture

and

The basic aim of an Internet robotic teleoperation system is to furnish the possibility, for a common user, having a standard connection to the World Wide Web, of easily teleoperating a robot located anywhere in the world from the comfort of his/her home or lab. In order to allow Internet users to effectively interact with robots operating in far away places, mobile robots on the Web have to satisfy some basic specifications [15]: the robot system must have a high degree of autonomy to face any large time delay; a minimal data transfer should always indicate the instant status, events and robot position; the control strategy of the robot should be as intuitive as possible, and the update rate of the transmitted video images should be as high as possible to provide a good feeling of reality. Moreover, in the design of the overall system a particular attention should be paid to using commercial hardware and software components to keep costs low and to guarantee easy availability to the public. Analysing the typical architecture of a robotic teleoperated system we can individuate three main modules: the robot to teleoperate, the communication infrastructure and the user Human Computer Interface (HCI). In the following a detailed description of these modules referring to

Romeo Internet-based satellite teleoperation in and manages communications from the human s Antarctica will be given. interfaces and the robot control system on-board the vehicle, dispatching the telemetry data and collecting, by solving conflicts, the user commands. 2.1 The robot: Romeo ROV Romeo is the latest ROV (Remotely Operated Vehicle) prototype developed by CNR-IAN for scientific applications and robotics research [6].

Figure 1: Romeo ROV ready for immersion in the Antarctic sea. Its architecture basically consists of three Ethernet LANs (surface, on-board and lab LAN), which can be connected to the World Wide Web for scientific co-operations and teleoperation [3]. In particular, the connection of robotics, scientific and multidisciplinary payload is possible [3,6]. The surface LAN connects a net manager computer (IPER) and a multi-machine distributed Human Computer Interface (HCI), which allows a number of different operators to interact with the robot at various levels of the control system. The conventional HCI for scientific applications consists of three interfaces for the pilot, who teleoperates the vehicle, the supervisor, who supervises the plant behaviour and resources allocation, and the marine scientist, who examines real-time images and sensors data to detect areas of interest. The IPER machine acquires all the vehicle surface sensors, as, for instance, acoustic positioning system, ship GPS and gyro-compass,

Figure 2: Romeo control and software-networked s architecture. The surface LAN can also support the connection of an external supervisory/mission control module or of a remote human computer interface from which a user can perform a vehicle mission, as in the case of the teleoperation experiments described in this article. In addition to the computer running the vehicle control system, the on-board LAN can also connect advanced end-user devices carried by the vehicle. Furthermore, the lab LAN supports the

Underwater Virtual World simulation facilities (6 d.o.f. vehicle dynamics, environment and sensors) and graphics interfaces [4]. As far as the Romeo s control system is concerned, it is based on a hierarchical dual-loop architecture [7] providing the human operator/mission controller with a set of basic guidance task functions performing autoheading, auto-depth, auto-altitude, which are usually used for direct teleoperation, and automatic manoeuvring on the horizontal plane, which allow a high degree of autonomy and are usually used for way point navigation in the case of mission control. Moreover, Romeo control system makes it s available a telemetry data packet at a 10 Hz frequency, which contains the current status of the system. Finally, it is worth noting that, in the case of the Internet-based satellite teleoperation experiment, where the communication reliability was quite low, a security time out mechanism was added: if for more than ten seconds there weren t commands arriving to Romeo control system, all s speed references were zeroed. 2.2 Communications To allow a remote Internet user access to Romeo operating in Antarctica, its control system was integrated with a satellite communication system. This system was furnished, installed and made it to function by Servizio Reti e Telecomunicazioni of Consiglio Nazionale delle Ricerche (CNR-SRT). In particular, it consisted of three Inmarsat channels at 64 Kbps transparently seen by the user as three ISDN lines. Two lines were used to set up a standard H320 videoconference system that allowed communicating with the team operating in Antarctica during the experiment. The third line was used, by means of an ISDN modem, to perform a Remote Access from Antarctica to a Remote Access Server running at CNR-IAN lab in Genoa or to an identical one at CNR-SRT lab in Rome. By operating in this way, it was easily possible to connect to the World Wide Web the Romeo s control system computers located in Antarctica. Moreover, suitable software applications were developed on one hand to effectively manage communications through the narrow and unreliable satellite channel and on the other to allow Internet

users to easily access Romeo control system. To s collect and dispatch commands, data and video through the satellite channel and to handle multiple accesses of web users to Romeo control system, s two ad hoc software applications, respectively called RServer (Remote Server) and LServer (Local Server), were designed and developed. Furthermore, to allow users located anywhere and connected to the World Wide Web to teleoperate Romeo, also a Human Computer Interface written in Java, runnable from a common web client, was designed and developed. It is worth noting that in the design of the overall system a particular attention was paid to using commercial and low cost hardware and software components. In particular, standard Inmarsat channels were utilised for satellite communications and the Java programming language was used to develop the Human Computer Interface. In the following, a more detailed description of the developed software applications will be given.

Figure 3: Logical communication software architecture. 2.2.1 RServer The RServer aim is to handle communications s between the Antarctic end of the satellite channel and Romeo control system. RServer is a software s application subdivided in three different threads: telemetry, command and video thread. The telemetry thread receives the telemetry data coming from Romeo control system and sends them s through the satellite channel. The command thread receives remote user commands from the satellite channel and sends them to Romeo control system. s Finally, the video thread acquires, by means of a frame grabber, the video-images coming from Romeo video camera and compresses them using s a JPEG algorithm before sending them through the satellite channel. RServer is entirely written in C++

and works on a standard IBM compatible PC with an Intel Pentium III processor and running Microsoft Windows 98 OS. The PC where RServer runs mounts a Leutron PicPort Colour frame grabber that is used to acquire video images. 2.2.2 LServer The LServer aim is to handle communications s between the Italian end of the satellite channel and the HCIs used by web users. LServer is a software application subdivided in four different threads: telemetry, command, video and subscription thread. The telemetry thread receives the telemetry data coming from the satellite channel and sends them to the HCIs. The command thread receives remote user commands from HCIs and sends them through the satellite channel. The video thread receives the images coming from the satellite channel and sends them to the HCIs. Finally, the subscription thread manages web users accesses to Romeo control s system guaranteeing on the one hand that not more than one pilot at a time is enabled to send commands to Romeo and on the other not allowing more than a prefixed maximum number of observers is connected at a time (too many users could cause a congestion of the Ethernet channel). LServer is entirely written in C++ and works on a standard IBM compatible PC with an Intel Pentium III processor and running Microsoft Windows 98 OS. 2.2.3 Web Server On the same PC running LServer the SimpleServer web server [1] is also running, and its function is to serve requests coming from web users who want to connect to Romeo control system. In particular, s the web server main function is to allow the s downloading of the HTML page containing the Java applet used as HCI by web users.

the HCI is a Java applet, i.e. a particular program written in Java language which can be launched from a standard HTML page and then easily accessed by a common web browser that runs on all major computer platforms and operating systems. The HCI can work in two different modes: observer and pilot. When a user runs the HCI in observer mode he/she can only observe Romeo telemetry s and video images coming from Romeo video s camera. On the contrary, when a user runs the HCI in pilot mode, he/she can also send to Romeo s control system reference values of surge, heading, depth or altitude. As already said, LServer application guarantees that not more than one user at a time is able to run the HCI in pilot mode.

Figure 4: Human Computer Interface.

Experimental set-up and results

On December 2001, Romeo participated to the XVII Italian Expedition to Antarctica and had the opportunity to operate in the Marine Antarctic Specially Protected Area established nearby the Italian Terra Nova Bay Station. During this period manifold experiments of Internet-based satellite teleoperation of the Romeo ROV were carried out. The basic idea was to demonstrate the possibility, 2.3 Human Computer Interface for a common user, located in any corner of the The main goal we intended to reach designing and world and having a standard Internet connection, of developing the HCI used to teleoperate Romeo via easily teleoperating via satellite an ROV in Internet, was to create one that was not limited to a Antarctica. particular or a few computers or sites, but which was available to all users on any computer platform 3.1 Ice pack field operational logistic at any site. This capability was provided by writing the HCI in the Java language [11]. More precisely, The sea near Terra Nova Bay Italian Antarctic Base is covered by ice for most part of the year and the

boat can safely operate from the wharf just from mid January to mid February. Thus, the experiments had to be done working through an artificial hole in the pack, settling a number of ice camps to support a safe and efficient activity. A trestle with an electrical winch was needed to launch and recovery the vehicle. All the equipment was host in a suitable tent, provided with a wooden floor and heated by a couple of electrical heaters. The electrical equipment was powered by a 7.5 kW diesel generator, running 24 hours continuously to avoid freezing of the equipment and reduce thermal chocks at the power on, followed by an Uninterruptible Power System to reduce disturbances and avoid black-outs during the dives. The three satellite antennas were mounted on a container on the ice pack near the tent.

Figure 5: Ice pack operational logistic.

Figure 6: Camp on the ice pack.

3.2 Experiments and results In a first phase of the period November 2001 January 2002, some days were spent building and preparing the camp on the ice pack. Then, a number of preliminary tests carried out by teleoperating Romeo from CNR-IAN lab in Genoa were performed. On December 17th the first successful mission was carried out. The day after, there was a public demonstration in Rome to the presence of the press, of the CNR president and other authorities, and a large audience of undergraduate students. The main aims of that event (organised by CNR-IAN in collaboration with the science foundation School of Robotics) were on the on hand to demonstrate the possibility, for a common user, located in any corner of the world and having a standard Internet connection, of easily teleoperating via satellite an ROV in Antarctica; on the other to promote the knowledge of Robotics among students and the large public. The characteristic quality of the environment in which the experiment took place (the extreme surroundings of Antarctica) made easy to transmit and understand the concept of "intelligence" as the global ability to recognise and solve new problems in an unknown environment. Students had a first hand demonstration of the multiple applications of the Telerobotics and of the Internet Robotics, to satisfy user's needs in many fields of activity and research, from the exploration to the monitoring and investigations of extreme environments, to the remote control of intelligent robotic systems engaged in complex operations. Actually, during the experiment, which lasted more than one hour and half, about 40 people had the possibility to use the HCI in pilot mode and remotely operate Romeo immerse in the Antarctic sea and to explore the Antarctic sea-bed rich of benthic life. Many people using the HCI in observer mode, connected to the World Wide Web and located anywhere had the possibility to follow the experiment. Even if the video images received by the HCI had a low refresh frequency (approximately 1.5 Hz) because of the narrow band of the satellite communication channel, a quite good teleoperation of Romeo was still possible. Finally, on 15th January, another experiment was carried out in Arezzo during the prize giving of a competition for undergraduate

students. The competition aim (organised by s CNR-IAN in collaboration with the European Foundation for Space) was to develop a scientific instrument to mount on Romeo and which could be teleoperated in Antarctica via Internet. Actually, during the experiment, the students had the possibility to remotely actuate their scientific payload (which was an electronic camera with the possibility of inserting an infrared lens filter) and take some photographs of the Antarctic seabed.

the real system (Romeo) with a real-time simulator (already existent) able of locally predicting, in realtime, and as much precisely as possible, the robot behaviour and its interactions with the environment in which it operates. The simulator will work in parallel to the actual robot and it will receive the same commands and its telemetry. Every time a telemetry packet will arrive from the remote system, the simulator will be re-synchronised with the robot, using predicting techniques. Therefore, operating in such a way, a high and constant frequency real-time virtual telemetry provided by the simulator will be always available for the operator. This telemetry will be used to build and show a virtual reality visualisation of the robot that the operator will be able to use, together with the actual images at lower frequency refresh coming from the actual system, to perform a more effective teleoperation. In addition, the possibility to using augmented reality and synthetic fixturing techniques to further improving teleoperation is also being considered.

Figure 7: Antarctic Benthic life.

Acknowledgements
This work was partially funded by the Italian PNRA (Programma Nationale di Ricerche in Antartide) in the framework of the ABS (Antarctic Benthic Shuttle) project and by CNR-SRT (Consiglio Nazionale delle Ricerche - Servizio Reti e Telecomunicazioni) in the framework of the project Internet telerobotics in extreme environment with satellite support. The authors wish to thank all the PNRA staff at Terra Nova Bay for their precious logistic support and all the staff of CNR-SRT in Rome for their invaluable technical support in satellite communications and cooperation in experiment execution.

Conclusions

In this paper, the Internet-based satellite teleoperation experiments of the Romeo ROV in Antarctica, performed on December 2001 January 2002 have been described. These experiments are to authors knowledge the first example of satellite teleoperation of a ROV directly usable by users surfing on the web. The obtained results are quite positive and encouraging, and demonstrate that the current state-of-the-art permits the development of Internet-based teleoperated robotics systems allowing users to remotely operate robots in unknown unstructured environments by means of a simple connection to the World Wide Web. References However, the problems arising from the limited satellite communication channel affected by time- [1] http://www.analogx.com varying and non-deterministic delays, which can [2] P.G. Backes, K.S. Tso and G.K. Tharp Mars considerably compromise the effectiveness of the Pathfinder mission Internet-based operations robot control, are still open. The current research using WITS, Proc. of IEEE ICRA 98, pp. work at CNR-IAN lab is focused on using virtual s 284-291, Leuven, Belgium, (1998). reality techniques to reduce or possibly eliminate some of these problems. The basic idea is to couple

[3] R. Bono, G. Bruzzone, M. Caccia and G. Veruggio, A flexible networked architecture for scientific applications and robotics research, Proc. of 11th International Symposium on Unmanned Untethered Submersible Technology, Durham, USA, (1999). [4] G. Bruzzone, R. Bono, M. Caccia, G. Veruggio: "A Simulation Environment for Unmanned Underwater Vehicles Development", MTS/IEEE Oceans 2001, Honolulu, Havaii, USA, (2001). [5] G. Bruzzone, M. Caccia, G. Veruggio, C. Ferreira, C. Silvestre, P. Oliveira and A. Pascoal, "Internet mission control of the ROMEO Unmanned Undewater Vehicle using the CORAL Mission Controller", Oceans '99, Seattle, Vol. 3, pp. 1081-1087, (1999.) [6] M. Caccia, R. Bono, G. Bruzzone. G. Veruggio, "Unmanned Underwater Vehicles for scientific applications and robotics research: the ROMEO Project", Marine Technology Society Journal, Vol. 34, No. 2, pp. 3-17, (2000). [7] M. Caccia, G. Bruzzone and G. Veruggio, "Hovering and altitude control for open-frame UUVs", ICRA '99, International Conference on Robotics and Automation, Detroit, USA, CD ROM, p. 72-77, (1999). [8] T.I. Fossen, Guidance and Control of Ocean Vehicles, John Wiley & Sons, England, (1994). [9] D. Fryxell, P. Oliveira, A. Pascoal, C. Silvestre and I. Kaminer, Navigation, guidance and control of AUVs: an application to the MARIUS vehicle, IFAC Control Engineering Practice, Vol. 4, No. 3, pp. 401-409, Elsevier Science Ltd., England, (1996). [10] K. Goldberg, M. Mascha, S. Gentner, N. Rothenberg, C. Sutter, and J. Wiegley, Robot teleoperation via WWW, In Proceedings IEEE International Conference on Robotics and Automation, (1995).

[11] M.A. Hamilton, Java and the shift to netcentric computing, IEEE Computer, 29(8), pp. 31-39, (1996). [12] A.J. Healey and D. Lienard, Multivariable sliding-mode control for autonomous diving and steering of unmanned underwater vehicles, IEEE Journal of Oceanic Engineering, Vol. 18, No. 3, pp. 327-339, (1993). [13] E. Paulos and K. Goldberg, Current challenges in Internet robotics, Workshop IEEE ICRA 1999. Detroit, USA, (1999). [14] T.B. Sheridan, Telerobotics, Automatica, 25, pp. 487-507, (1989). [15] R. Siegwart and P. Saucy, Interacting mobile robots on the Web, Proc. of Workshop IEEE ICRA 1999, Detroit, USA, (1999). [16] C. R. Stoker, D. R. Barch, B. P. Hine III. and J. Barry, Antarctic undersea exploration using a robotic submarine with a telepresence user interface, IEEE Expert, 10(6), 14-23, (1995). [17] K. Taylor, J. Trevelyan, A Telerobot on the World Wide Web, National Conference of the Australian Robot Association, Melbourne, (1995). [18] L. Whitcomb, D. Yoerger Advances in Doppler-based underwater robotic vehicles, ICRA 1999, pp. 399-406, (1999). and H. Singh, navigation of Proc. of IEEE Detroit, USA,