SERIOUS LECTURES FROM FUNNY COMPUTER GAMES

Bruno Vigrio, FEUP / Real Games, Porto, Portugal mai04009@fe.up.pt, bruno.vigario@realgames.pt

Miguel Pereira, Real Games, Porto, Portugal miguel.pereira@realgames.pt
Lus Correia, Real Games, Porto, Portugal luis.correia@realgames.pt
Antnio Pessoa de Magalhes, IDMEC Plo FEUP, Porto, Portugal a.p.magalhaes@fe.up.pt

ABSTRACT: A rewarding but not trivial goal and a highly interactive and realistic virtual environment
are central to a successful and addictive computer game. These can also be the key ingredients for
modern hands-on and risk free lectures in industrial control, making easier to teach, assess and enforce
good thinking and improved practices. The paper describes the development of an educational virtual
training system simulating five industrial plants that features cutting-edge 3D real-time graphics, physics
and sound. The goal is to make plants work by developing the proper controlling software. The bonus is a
funny but very realistic and interactive virtual environment from where trainees get a real and serious
sense of how their programming actually works or not.
Keywords: Virtual environments, computer games technologies, interactive training systems, control
education and training.

INTRODUCTION
The simulation of real world phenomena is a wide
and very interesting application area of computer
sciences and technologies. Traditionally, it is used
in forecasting; more recently, in training.
A flight simulator is a classical example of a
computer based training system. Here, a trainee
pilot using a real cockpit interface interacts with a
computer based system representing a real world
scenario [1]. Until recently, man in the loop
simulation systems [2], as flight simulators, were
only common in training scenarios where risk and
cost are major concerns, such as nuclear plants,
aerospace and military [3-4]. Yet, low cost and
great advances in computing technologies are
making computer simulation a common training
tool [5-8]. For instance, light cars, trucks and bus
simulators are being increasingly used for training
in driving schools all over the world.
The ever increasing demand for more realistic
video games is being for decades the main
research motivation in computer simulation and
related technologies [9]. As a result, modern
computer games allow for high-performance
visualization of complex systems reacting in realtime to external or internal events according to
physics laws or man-made rules. Play stations are
becoming amusing training stations, where
common people can feel the power of driving an
F1 or a world championship rally car or,
alternatively, can try air or rail traffic control
While addictive video games are a major concern
for parents and educators, they can play a major

role in serious education [10-13]. Modern video

games technologies enable the development of
computer applications than can dramatically
facilitate
the
understanding
of
physics
phenomena, scheduling and planning strategies,
systems control, or machinery operation and
maintenance, just to mention a few potential
areas.
This paper is about a software package aimed at
control systems education and training. It consists
of five highly interactive virtual plants (described in
the paper) and it is based on modern computer
games technologies, featuring cutting-edge 3D
real-time graphics, physics and sound. The goal is
to make the five virtual plants work correctly by
using an external (and real) controller running the
proper software, which the trainee must develop.
The subject matter is logic control specification
and programming; the target trainees are industrial
engineering students. As such, the development is
focused in five typical industrial applications of
programmable logic controllers (PLC); namely,
sorting, batching, palletizing, pick and place and
automated storage and retrieval systems. The
idea is to let students see a PLC in its typical
industrial environment and fully understand and
program common control tasks. This is impossible
to get if a PLC is simply wired to a few lamps and
switches as commonly found in school labs due to
cost, space and safety constraints. The paper is
organized as follows:
The next section surveys virtual environments in
logic control education, contextualising the
developed system and its functional requirements.
Then, the major guidelines in system development
are presented and justified. Next, a short

description of the final product is made and the

five virtual plants are presented. Feedback from
users trainers and trainees comes before a
short discussion about the work in progress. A
summary of the most important conclusions ends
the paper.

RELATED WORK AND SYSTEM

SPECIFICATION
Computer based simulators for control training and
education do exist for a considerable time.
Reference [14] includes a good survey on the
subject.
Authors are developing and using virtual systems
for PLC training since the middle 90s. Their first
virtual systems were just 2D representations of
common real world machines and systems. Fig. 1
shows a filling/draining virtual plant; painting,
drilling, and pick and place are other examples of
the more than a dozen virtual training scenarios
incorporating the simulation pack. These virtual
plants are still being used for challenging trainees
to develop and test proper control algorithms for
different Programmable Logic Controllers.

application. This means that they no longer had to

guess the meaning of a lamp or a switch or even
to imagine a conveyer moving, a tank filling, a part
being drilled, a door closing or a robot arm picking
and placing parts to develop and test their
programs. Additionally, trainees could control the
plant manually to get more familiar with it. Last but
not least, the mixing of synthetic environments
simulating interesting practical plants with real
controlling devices lead students very naturally to
distributed and supervisory control applications.
Teaching PLC and industrial programming
became easier and motivating than ever!
When compared to those educational, reduced
scale, physical training plants, virtual systems are
low cost, space saving and maintenance free
solutions that do not present any risk of injury to
humans or damage to machines. Yet, virtual
systems have to be very realistic to properly rival
the main advantages of these physical and more
complex systems: global and detailed observation
from any angle, total interaction with the trainee
leading to a complete understanding of sensors,
actuators and mechanical parts and even
unexpected faults and malfunctions!
A realistic 3D representation is a natural step to
bridge this gap. This was performed in 2005 by
taking an automated warehouse as the target
system Fig 2. This awarded application [15] was
a detailed and highly realist representation of a
real system existing in the automation lab of an
engineering school. In this new virtual system the
user could freely navigate in a virtual 3D world
appreciating with a great level of detail all the parts
of the warehouse. Due to the large number of I/O
points involved, the data exchange with the PLC
was accomplished through the OPC server/client
technology [16] instead of a physical computer
board.

Fig. 1.

A 2D virtual system.

The secret of these and other fairly simple

simulation packages are pre-defined animations.
These are executed when the trainee uses manual
control to get acquainted with the plant or, in PLC
mode, as virtual actuators are turned on and off,
according to the output signals of the PLC. These
reach the computer through a digital I/O card.
Virtual sensors are also included in the
simulations. Virtual sensors are a special class of
variables whose values are passed to PLC inputs
through the same digital I/O interface card.
While very simple, this 2D virtual solution showed
a great advantage over the traditional switches &
lamps panels: trainees were able to visualize
plant reactions to PLC control in the context of the

Fig. 2.

A 3D virtual system.

From this application it became obvious that

detailed 3D visualization is essential in complex

systems representation. Yet, in contrast to the

benefits of the realistic visualization arising from
detailed textures and high polygon models, there
were some system limitations introduced by the
predefined animation patterns used in the project.
The next step was to simulate a real and huge
industrial automated warehouse developed by a
Portuguese automation enterprise. In here, the
traditional animation method approach was not
desirable (and even not possible) due to the very
complex and hard to predict behaviour of the
system. As such, a real-time physics engine was
included at the kernel of the application. The result
was a virtual but highly interactive scenario whose
behaviour proved to be very similar to that of the
real system; boxes falling when not correctly
placed and conveyors jammed if improperly
controlled are some examples of such realism. To
accomplish the perfect simulation, real-time 3D
sound was implemented making possible to enjoy
a convincing 3D world.
The lessons learned from the stated research are
the point of the departure for the work described in
this paper: the development of a high quality
system for logic control training and education.
High quality means, on one hand, realistic
graphics, physics and sound, immersing users in
a convincing 3D world and, on the other hand,
emulation of attractive and very common plants
coping everyday problems and challenges found
in most industrial environments.

SYSTEM DEVELOPMENT
The technology underneath this software solution
can be divided in three major groups: a graphics
engine, a physics engine and an original, specially
designed, instrumentation engine [17]. The
graphics engine is responsible for all the visual
and sound presentation; the physics engine
computes the motions of the objects and their
interactions; the instrumentation engine emulates
sensors and actuators and manages the
communication between the software and an I/O
DAQ board on which is based all the data
exchange with the controlling PLC.
The 3D graphics engine used by this software is
proprietary. It was developed on top of the XNA
Framework v1.0 Refresh and .NET Framework 2.0
targeting the vertex/pixel shader 1.1 compatible
boards, making it thus available to a wide group of
users. The main features of this technology are
batched rendering, material system, scene graph
optimization, post-process bloom effect, efficient
translucent materials, GUI system and custom
content pipeline processors for models, GUI and
collision shapes.

Newton Game Dynamics physics engine is the

second kernel nuclear resource of the application.
This is a fast and widely featured physics engine
that proved to be stable enough to represent
complex industrial machinery. It was integrated
with the 3D engine scene graph through the XNA
content pipeline using a custom collision shape
processor. Fig 3 shows some examples of physics
collision shapes used in the application. These are
represented by the red lines.

Fig. 3.

Physics collision shapes.

The instrumentation engine consists of a sensing

layer and a communication layer. The sensing
layer offers several generic models of sensors and
actuators. The communication layer establishes a
data flow between the virtual sensors and
actuators and the I/O DAQ board. Fig 4 shows a
simple model for an optical sensor; the sensor is
represented graphically by the graphics engine
using a ray-cast to detect rigid-bodies simulated
and handled by the physics engine.

Fig. 4.

Example of a ray-cast sensor.

Available Systems
The developed training package offers five target
systems based on common real world industrial
plants. Each system addresses standard problems
requiring PLC control and proper programming.

The systems are organized according their

expected difficulty level: Sorting, Batching,
Palletizer, Pick & Place and Automatic
Warehouse. Figs. 5 to 9 show a view of each.
The goal of the sorting system is to transport
cases from an entry bay to two exit elevators while
sorting them by height. The batching emulates a
paint mixing process in order to get a desired
colour. The palletizer challenges the user to group
and palletize cases up to three layers. The goal of
the Pick & Place application is to place parts
inside boxes using a three axes incremental
manipulator. In the automated warehouse the
objective is to transport, store and retrieve boxes
from a rack.

Fig. 8.

A view of the Pick & Place application

Fig. 9.
Fig. 5.

A view of the Automatic Warehouse

application

A view of the Sorting application

User Interaction
All the systems can be manually controlled by the
user or in automatic mode by the PLC. In manual
mode the user acts as the system controller, so
that he can get familiar with the working process.
When in automatic mode the PLC is the only
system controller. Fig 10 shows the systems user
interface. Here, the user can observe the state of
virtual sensors and actuators, chose manual or
automatic mode and press virtual switches and
watch virtual lamps similar to those found in
industrial machines panels, including an
emergency switch!
Fig. 6.

A view of the Batching application

Fig. 10. Human machine Interface

Fig. 7.

A view of the Palletizer application

Worth to say is that system allows user to stuck

sensors and actuators, simulating failures. These
failures can be in open-circuit or short circuit see
Fig 11. Using this feature, the user is able to inject
faults in the system, presenting new challenges to
programmers and increasing the realism of the
simulation.

surprising number of industrial enterprises is

adopting the product for internal training,
especially in the UK. Product documentation and
user interface are now available in eight
languages. Several industry and automation
oriented magazines and newsletters surveyed the
product in the last months. Users and other people
interested in the product can contribute to a forum.
This information and a demo version of the
product is available at http://www.realgames.pt/.

Fig. 11. Fault Injection panel

Another main feature of the system is its ability to

interact with movable objects in real-time. This is
allowed at any moment and just by clicking,
dragging and dropping. This feature allows the
user to:
Interact with the system as in a real plant.
Add and remove objects from the production
circuit.
Introduce plant error jams.
Manually control several parts of the system.

Hardware/Configuration
The information exchange between the PLC and
the software is supported by an USB DAQ board
with 32 isolated I/O channels. The USB interface
makes possible to host the simulation software in
any kind of PC (workstations, laptops and even
netbooks) and to wire the system to any type of
PLC of any brand having at least 16 input and 10
output points for a complete data interchange with
the virtual sensors and actuators.
The minimum computer requirements for running
the developed software are a Pentium IV or AMD
K8 at 1GHz, 256MB of RAM memory, 200MB of
available disk space, a graphics card compatible
with DirectX 9.0, two USB ports and Windows XP
with Service Pack 2 or Windows Vista.
The installation process and configuration is
straightforward and a standard computer can run
the application using less than 20% of system
resources.

FEEDBACK FROM USERS AND WORK IN

PROGRESS
The developed software package became a
commercial product in 2008 under the name ITS
PLC [18] and is now being used worldwide,
mostly in colleges and universities. Yet, a

Feedback from users is being collected and

carefully analyzed. At the time of the writing of
this publication the amount of collected data is not
very extensive but it suffices to get a qualitative
judgment of the product from those who have
been using it more intensely.
The first very positive note goes to the realism of
the application, often defined as impressive.
Machinery seems so real that users are commonly
tempted to observe it from all possible angles. To
those who are familiar with computer games it is
very easy to take an imaginary trip by driving the
camera using the computer mouse. Not so
enthusiastic players can enjoy a guided tour by
taking different views of the plants from several
pre-defined cameras. Zooming to observe
sensors, actuators and mechanical parts is
another feature that users find very interesting.
The interface of the application is generally found
simple and effective. User interaction in manual
control is obvious and gives users a total
immersion into the system; especially when they
start interacting with moving parts, such as
jamming boxes or dropping parts on conveyors.
Wiring a PLC to the interface board is trivial for
most users. Yet, it may be a little tricky for PLCs
having solid state inputs and outputs. User manual
gives the required information for the job but for
some users this is definitely not a simple task
exactly as it happens when the problem is to wire
a PLC to real sensors and actuators!
When it comes to make each plant work by
developing the proper controlling software, users
opinions disagree considerably about the difficulty
of the tasks, according to their previous practice in
PLC programming. Senior programmers have not
many difficulties, but beginners find the problems
quite more complex than the expected.
Some researchers are finding interesting usages
of the presented system; for instance, its
application in supervisory and fault tolerant
control. These are important research areas that
benefit a lot from the low cost and risk-free usage
of virtual systems.
Users feedback is guiding some of the work in
progress around this product. For instance, an
exercises book with dozens of solved problems is

actually in preparation. It is expected to be an

important guide for trainers and trainees, as it
provides step by step control solutions for each
virtual plant. The book also includes a set of more
challenging exercises for advanced programmers
and automation professionals. These concern fault
tolerance, systems integration and supervisory
control.
Another line of progress is the usage of the OPC
technology to interface the system to controlling
devices. A prototype version of the product using
this feature is already developed, enabling or
facilitating the usage of soft controllers, distributed
control solutions and integration with higher level
applications.
Further steps are still open, from where the
development of new virtual scenarios is a natural
course.

CONCLUSIONS
The paper presented a software package
emulating five real-world-based industrial plants.
Its purpose is to have a synthetic environment
similar to those found in modern computers
games, from where trainees can validate their
PLCs control programs. This is a real-time, low
cost and interactive solution without any risk of
injury to staff or damage to machinery.
Each plant includes proper virtual sensors and
actuators exchanging information with the PLC
through an USB interface data acquisition board.
This makes possible to control the virtual plants
with any PLC from any manufacturer.
Preliminary feedback from trainers and trainees
sustain that the concept is interesting and
valuable, predicting an increasing number of users
in the near future. Moreover, its original purpose of
serving just basic logic control education is being
surpassed, as the synthetic plants are being used
for testing and training supervisory and
dependable control strategies in advanced
educational and research activities. The ongoing
integration of an OPC client allowing different and
distributed control equipment will certainly broad
the spectrum of the application in future releases.

References
[1] Menendez, R and Bernard J. (2001) Flight
Simulation in Synthetic Environments. IEEE AESS
Magazine, p. 19-23, September 2001.
[2] Innocenti, M. Pollini, L (1999). A Synthetic
Environment for Simulation and Visualization of
Dynamic Systems. Proc American Control Conf.
1999 p. 1769-1774.
[3] Paquette, D.L.; Gouveia, L.M. (2000) Virtual training
for live fire naval gunfire support OCEANS 2000
MTS/IEEE Conf. and Exhib. Vol 2, p. 733 737.

[4] Tazama U. St. Julien, Chris D. Shaw (2003)

Firefighter command training virtual environment
TAPIA '03: Proc. of the 2003 Conf. Diversity in
computing.
[5] Chawla, R. and Banerjee, A. (2901) A Virtual
Environment
for
Simulating
Manufacturing
Operations in 3D. Proc of the ACM 2001 Winter
Simulation Conference, p. 991-997.
[6] Jonsson, A., Bathelt, J. and Broman, G. (2003)
Interacting with Real Time Simulations Virtual
Reality in Industry Applications. Proc. Int.
Immersive Projection Technologies Workshop, Pp
305-306, Eurographics Association.
[7] Houston, M. (2004) Visualizing Dynamic
Architectural Environments. Comm. ACM Vol. 47,8
P. 54-59, Aug 2004.
[8] Fast, K.; Gifford, T.; Yancey, R. (2004) Virtual
training for welding. Third IEEE and ACM Int.
Symp. on Mixed and Augmented Reality P 298 299 Nov. 2004.
[9] Yoav Shoham (2008) Computer science and game
theory Comm. ACM, Vol 51, 8
[10] Peter M. Border (2004) Teaching physics by
designing games SIGGRAPH '04 Educators
program.
[11] Vinod Srinivasan (2008) Using video games to
enhance learning in digital systems Future Play '08:
Proc. 2008 Conference on Future Play. November
2008
[12] Tzafestas, C.S.; Palaiologou, N.; Alifragis, M.
(2006) Virtual and remote robotic laboratory:
comparative experimental evaluation IEEE Trans
on Education Vol 49, 3, P. 360 - 369
[13] Kaveh Kardan (2006) Computer role-playing games
as a vehicle for teaching history, culture, and
language. Sandbox '06: Proc. of the 2006 ACM
SIGGRAPH symposium on Videogames.
[14] B. Riera, B. Vigario, J.-P. Chemla, L. Correia, F.
Gellot (2008) 10 ans de Maquettes Virtuelles pour
l'enseignement des automatisms J3eA 8 HORS
Srie 1 (2009) 1004.
[15] Freitas, F. Magalhes, A. and Vigrio B (2005)
Armazm Automtico Virtual Controlado por PLC
1.o Prmio ANIMEE de Inovao e Criatividade
2005. Revista ANIMEE Vol2.p.30 2005.
[16] Juergen Lange and Frank Iwanitz (2008) OPC
Fundamentals 3 ed. the OPC Foundation,
http://www.opcfoundation.org/.
[17] Bruno T. Vigrio, Antnio Pessoa de Magalhes
and Francisco T. Freitas (2006) Modern computer
games technology in systems and control education
Controlo 2006 7th Portuguese Conference on
Automatic Control, Lisboa, Portugal, Sep 2006..
[18] Real Games (2008) ITS PLC Professional Edition
Users Manual. Available at www.realgames.pt.