2015 IEEE 39th Annual International Computers, Software & Applications Conference
Agent-Oriented Cooperation of IoT Devices towards
Advanced Logistics
Takumi Kato†,‡ , Ryo Chiba†,‡ , Hideyuki Takahashi†,‡ , Tetsuo Kinoshita†,‡
† Graduate School of Information Sciences, Tohoku University
‡ Research Institute of Electrical Communication, Tohoku University
{takumi, r-chiba}@k.riec.tohoku.ac.jp,
{hideyuki, kino}@riec.tohoku.ac.jp
Abstract—There are several researches, products and stan-
dards for smart devices to realize Internet of Things (IoT)
applications. In order to take full advantage of IoT devices,
this paper proposes an agent-oriented cooperation infrastructure.
As the core functionalities of the infrastructure, we propose
agent-based device management function, self-state management
function, and cooperation function among IoT agents. We have
implemented 2 experimental systems to confirm the effectiveness
of the proposed functionalities of the infrastructure.
I. I NTRODUCTION
There are several investigations about the utilization of
smart appliances, smartphones, sensors, network robots, etc.
Mostly, the existing approaches focus on the cooperation based
on fixed scenarios with a central control server. In recent years,
there is demanding needs for these network-enabled devices
to provide more flexible and better life-aid services for the
users. In order to realize flexible and more comfortable life-aid
services, this paper proposes the cooperation function among
Internet of Things (IoT) devices. IoT devices are capable of
managing physical sensing or actuation, which have access to
other device or server mostly through TCP/IP networks.
The objective of our research is to realize an agent-oriented
infrastructure, which enables flexible cooperation among IoT
devices, such as smart appliances, smartphones, sensors and
robots. Our approach is to introduce intelligent agents which
manage various devices. In this paper, we describe main
functions of the infrastructure for various kinds of devices to
cooperate each other in advanced logistics applications. We
also describe the implemented experimental system to confirm
the validity of our function designs.
II. R ELATED W ORK
There are several investigations about cooperation between
sensors and devices. PUCC (P2P Universal Computing Con-
sortium) enables sensors and smart home appliances control
using a smartphone [1]. It is based on the central control by a
web server, and the smartphone is able to access and control
sensors and smart home appliances through the web server.
In order to utilize multiple networked robots by considering
the users’ situations, the network robots control system with
Field Data Markup Language (FDML) is proposed in a existing
research [2].
There is also a research on common protocol for communi-
cations among different types of devices [3]. They also employ
central control scheme using a web server. The proposed
0730-3157/15 $31.00 © 2015 IEEE
DOI 10.1109/COMPSAC.2015.237
Converged Device Management (CDM) server which manages
smartphones, PCs, and other mobile devices. CDM controls
fixed devices through a control interface through mobile de-
vices.
In order to manage network-enabled devices smarter, there
is an investigation on multiagent-based approach for device
discovery and orchestration in home and building automation
[4], [5]. They utilize single knowledge base and a control
server, and the cooperation among devices is organized in
one server. Each agent manages assigned appliance, and they
exchange messages with each other. In the system, services
are provided based on the cooperation of agents based on a
single user’s request.
The advances in networked robot technologies enable au-
tonomous operation of flying and wheeled robots. The research
[6] has developed the device cooperation mechanism in a broad
area, not only with in a single household, but in a broad
area including household, shopping mall, etc. Another research
[7] demonstrates the indoor flight based on image processing
which works as an eye of flying device.
Despite of the number of investigations on smart device
utilizations, the existing approaches only enable static coop-
eration among devices, which only allows fixed scenario in a
fixed area. In the existing investigations, it is highly difficult
for static devices and dynamic devices to cooperate with each
other, e.g. sharing sensor information and web cam’s image
with flying drone in a room. The existing works on utilizing
network-enabled robots are only working on individual robots’
ability, but the utilization of sensor and actuator nearby. It is
not flexible enough to add more devices with new devices
which supports different kind of standards upon the change of
system requirements and users’ request.
The lack of flexibility causes some difficulties, e.g. difficult
for new device to join an existing group of devices while they
are in operation. Since the existing approaches employ single
control server architecture, it is difficult for individual devices
to perform extempore cooperation with local adjustment, task
sharing, etc. Also, the single control server architecture is not
scalable to the large number of controlled devices.
In order to provide situation aware, and user-oriented ser-
vices, it is necessary for IoT devices to orchestrate dynamically
and cooperate each other by considering the function and
restriction of each device.
We propose core functionalities of agent-oriented infras-
tructure for the autonomous cooperation of heterogeneous
223
 Fig. 2. Architecture of IoT Agent and Managed Device

Fig. 1. Overview of Agent-Oriented Cooperation Infrastructure for IoT

Devices

devices, e.g. various sensors, smart appliances and robots. The

infrastructure mainly provides 3 functions, namely (1) agent-
based device management function, (2) self-state management
function, and (3) cooperation function among IoT agents. In
our previous paper [8], we have presented the basic design
of the agent-oriented infrastructure. In this paper, we present
the core functionalities of the infrastructure, and the prototype
applications in the advanced logistics domain. In the exper-
iments of the applications, we demonstrate the cooperation
of heterogeneous devices, and the addition and removal of
components in runtime.
III. AGENT-O RIENTED C OOPERATION I NFRASTRUCTURE
FOR I OT D EVICES
Figure 1 shows the overview of the services and the
proposed agent-oriented infrastructure. Based on the proposed
infrastructure, various sensors, smart appliances, and robots
cooperate autonomously to provide services, such as, elderly
people support, life-aid services, monitoring, logistics and
so on. The infrastructure enables the services to take into
account the users’ situation and environment by utilizing
several sensors installed in the distributed environment. We
implement the proposed infrastructure as multiagent system
to realize flexible cooperation and service composition. The
agents manage software, device and network to composite
services based on users’ preference and situation. In order to
realize extempore cooperation, each device needs to monitor its
condition, find other devices to perform task together, choosing
communication protocol, agent organization information, task
conflict adjustment and so on. Our proposed infrastructure pro-
vides common agent-development templates for sensors and
devices to bridge the gap between different device standards
and sensor types.
We propose 3 core functionalities of IoT agent which
manages IoT device to realize flexible cooperation among
such devices. The core functionalities are, namely, (1) agent-
based device management function, (2) self-state management
function, and (3) cooperation function among IoT agents.
Figure 2 shows the architecture of IoT agents and relationship
among the functions.
Fig. 3. Overview of IoT Agent Template
The following subsections explain each function and its
characteristics.
A. Agent-Based Device Management Function
Agent-based device management function provides an in-
telligent agent with the access to the managing IoT device.
Having access to a physical actuator and sensor, IoT agents can
manage themselves by utilizing knowledge about the device
and task. We provide a common agent-development template
to establish the common sense among IoT agents to manage
and cooperate among the devices.
There are 2 kinds of templates, which are sensor template
and actuator template. Figure 3 shows the overview of IoT
agent templates. Developers can use both templates if the
IoT device has both sensor and actuator as components. For
example, the templates for sensors define the data format, the
templates for devices (actuators) defines the types of controls,
communication protocols, etc. These templates provide the set
of common sense which wraps various standards and protocols,
in order to provide common interface for agents’ cooperation.
B. Self-State Management Function
The state of managed IoT device is represented and man-
aged in the self-state management function. IoT agents have
knowledge and self-state about managed IoT device. The IoT
agent updates its self-state based on the situation derived
from environmental data. The common self-state consists of
following 3 parts:

224
 • Sensor State
• Actuator State
• Self-Location
Sensor state and actuator State are provided through Agent-
Based Device Management Function, and the IoT agent up-
dates its working memory. Agents monitor sensor’s current
value all the time, and it can be utilized by the agent itself,
or the agent can send sensor data to other agent upon request.
Actuator’s move is also monitored all the time, and as soon
as an error or exception found in the actuator’s move, agents
can know the trouble. About self-location state, IoT agents
always monitor their location in the system. The representation
of location varies in the application. The representation can be
string information like “Point A, Point B, Point C” or it can be
linear number like Global Positioning System. The developer
needs to set what kind of value should be used in the system.
IoT agents can have more states, but all the IoT agents
share these 3 states above so that they can understand other
agent’s situation. Other possible self-states are current pro-
cessing load, communication condition, task progress, etc. For
example, if the assigned task takes too long time for one agent
to finish in time, the agent asks other agents to share the
process to accomplish the task. When the agent asks for help,
it considers the appropriate communication option to negotiate
and share the task, i.e. through Wi-Fi, Bluetooth, ZigBee, etc.
Other example is the sampling rate adjustment in sensor agents
to meet the power consumption requirement of the battery, etc.
C. Cooperation Function among IoT Agents
In order to provide appropriate services, it is necessary
to organize agents based on users’ request, situation and
agents’ environment. The fundamental action of the proposed
cooperation function is a discovery of agents which are capable
of performing the request task. Individual agents know their
ability and limitation, and the information can be shared among
other agents to organize appropriate set of agents, i.e. set of
sensed information and actions by actuators.
In our proposal, the cooperation among IoT agents start
with talking to each other, e.g. asking other agent’s self-state,
asking other agent to perform specific action, etc. Since agents
always talk to each other to accomplish individual assigned
tasks, it is easy for them to adjust to the change, i.e. addition
of device, environmental (sensor data’s) change, etc. The
agent’s ability of knowing one’s self-state, and communication
to know other agent’s state, help the agents to persistently
perform task with heterogeneous devices.
For example, if a new sensor becomes available during
a task, and the agent requesting the task decided that the
new sensor is more appropriate than currently used sensor,
the sensor can be replaced and the task continues. When there
are several possible IoT devices to cooperate, agents simply
ask the first agent which replied the “accepted” message to
the requesting agent.
IV. A PPLICATIONS OF A DVANCED L OGISTICS
We have designed and implemented 2 experimental sys-
tems to demonstrate our proposal through logistics domain.
225
Fig. 4. Agent Placement in Experiment 1
First one is the cooperation between wheeled robots and drone.
Second one is the dynamical cooperation among wheeled
robots which number of robots changes during the operation.
We have used Interactive Design Environment for Agent
system (IDEA) [9] to develop Dash [10] agents and the
experiment.
Experiment 1: Cooperation between Wheeled Robot and Drone
The purpose of experiment is to evaluate the coopera-
tion of heterogeneous IoT devices based on our proposed
infrastructure. Figure 4 shows the overview of experimental
system developed based on the proposed infrastructure. In
the experimental system, a wheeled robot with a sensor and
flying robot (quad copter) with a sensor cooperate together.
The objective of the experimental system is to carry packages
from Place 1 to Place 4. Robot 1, Robot 2, Drone 1 are
all managed by IoT agent developed based on the proposed
infrastructure. Robots and Drone are managed by IoT agent
which has 3 key functions proposed earlier in this paper. Touch
sensors, and motors are also controlled by agents which always
communicate with Robot agent.
First, Robot 1 picks up a package and carries to Place 2
by using motors built in, and there is an obstacle after Place
2. Robot 1 senses the obstacle by touch sensor, and it knows
it can not go over the obstacle because Robot can not fly.
Robot 1 then broadcasts a request message to request other
agents to carry the package. Drone 1 knows its location is
Place 2, so that it replies to Robot 1 to accept the request. After
that, Robot 1 hands over a package to Drone 1, and Drone 1
starts to go to Place 3. Drone 1 tries not to fly long distance
because it knows the battery is very limited, so when Drone 1
arrives at Place 3, it broadcasts the request message to carry
the package. Robot 2 knows its location is at Place 3, and it
replies to accept the request from Drone 1. Drone 1 hands over
the package to Robot 2, and comes back to Place 2 to pick up
more package. And Robot 2 goes to Place 4, puts package at
the goal, and comes back to Place 3 to pick up more package.
Simultaneously, right after handing over package to Drone 1,
Robot 1 comes back to starting point to pick up more package.
In each area, Robots and Drone work independently, and ask
for help when it’s necessary. The hand over of packages is
not implemented physically, because it makes the robots and
drone highly complicated and difficult not to have error in the
Fig. 5. Implementation of Drone and Wheeled Robot in Experiment 1
Fig. 6. Overview of Agent Settings in Experiment 1
action. We implemented the negotiation of agents in software
level.
The physical implementation of robot, motor and sensor is
shown in Figure 5. To implement the wheeled robot, we used
LEGO Mindstorms Intelligent Brick NXT [11]. The NXT has
2 interactive servo motor which work as 2 wheels, and a touch
sensor is attached in the front. We used Parrot AR. Drone 2.0
[12] to implement the flying robot.
The correspondence between physical devices and agents
are shown in Figure 6. There is one agent workplace, and all
the agents are working in the same workplace. The connection
between agent and drone is Wi-Fi. Robots are communicat-
ing via Bluetooth. Agents communicate with each other by
exchanging messages via TCP/IP network. These agents can
work together if they are in separated computers if the TCP/IP
communication is permitted.
Experiment 2: Cooperation among Wheeled Robots with a New
Robot Joins and Leaves the System
The purpose of experiment is to evaluate the flexibility of
agent-oriented cooperation among IoT devices based on our
proposed infrastructure. In experiment 2, we use LED light in
addition to the robot settings in the experiment 1. The snapshot
of wheeled robot in experiment 2 is shown in Figure 7. The
LED light represents whether or not the robot is carrying a
package. The basic settings and goal of the system is the same
as experiment 1. The difference is that there are only wheeled
robots, and there is addition and removal of a wheeled robot
during operation. The overview of agent settings in experiment
2 is shown in Figure 8. Robot 4 and the corresponding
226
Fig. 7. Wheeled Robot with SunSPOT (LED Light) in Experiment 2
Fig. 8. Overview of Agent Settings in Experiment 2
agents are added during the operation. Also after a while,
the added Robot 4 is removed and corresponding agents are
terminated. We can check the flow of package by the signal
of LED light representing the package. The communications
between NXT and agents in PC are via Bluetooth, and the
communications between SunSPOT and the SunSPOT agents
in PC are via Zigbee. Since state of individual devices are
managed by agents and information is exchanged through
agent communication, robots can act like motors, touch sensors
and SunSPOT like built-in devices.
Similar to experiment 1, the hand over of packages is not
implemented physically, because it makes the robots and drone
highly complicated and difficult not to have error in the action.
We implemented the negotiation of agents in software level,
and the carried package is represented as the light of SunSPOT
on each robot.
V. E XPERIMENT
In the experiment 1, the flow of package has been demon-
strated as shown in Figure 9. The experimental setting is as
shown in Figure 4 and Figure 6. In (A), drone carries object
and fly over to the Place 3. The drone talks with Robot 2,
and hands over the object to Robot 2 in (B). In (C), drone
goes back to Place 2 to fetch the next object, and Robot
2 goes to Place 4 to carry the object to the destination.
It is confirmed that, based on our proposed agent-oriented
infrastructure, heterogeneous devices such as wheeled robots
and drone can cooperate together by exchanging common
information and requesting tasks on another. The common

Fig. 9. Result of Experiment 1
Fig. 10. Result of Experiment 2
sense of IoT agents and the cooperation function enabled
this heterogeneous device’s cooperation as demonstrated in
experiment 1. In the experiment 1, we have confirmed that
heterogeneous devices can cooperate together in the proposed
infrastructure.
Figure 10 shows the snapshots of experiment 2. In the
experiment, at (A), there are Robot 1, Robot 2, and Robot
3 carrying objects. In (B), Robot 4 has been added to the
area where Robot 2 is working, and the corresponding agents
for managing robot 4 is deployed, and set the self-location
manually. After the set up, robot 4 start working together with
Robot 2 in the same area. There were cases where Robot 2 and
Robot 4 are at the same place, and these 2 robots broadcasts the
same request message to Robot 3 at almost the same timing.
However, in (C), because of the proposed conflict resolving
method, Robot 4 hands over package to Robot 3 because the
message from Robot 4 arrived Robot 3 earlier, and Robot
3 takes the request from the earlier requester. As shown in
(D), after the removal of Robot 4, the operation comes back
to exactly like before Robot 4 was added. This flexibility is
ensured based on the core functionality of our proposal. As the
result, the flow of package has been properly demonstrated,
and the removal and addition of new wheeled robot worked
successfully. In the experiment 2, we have confirmed that
addition and removal of IoT devices is possible in the proposed
infrastructure.
VI. C ONCLUSION
In this paper, we proposed an agent-oriented infrastructure
for IoT device cooperation. The overall design of the proposed
infrastructure and the experiment using experimental system is
227
explained, and we have confirmed that the proposed infrastruc-
ture enabled heterogeneous device cooperation, addition and
removal of components in the advanced logistics scenarios.
Future work is the refinement of cooperation mechanism and
its evaluation in various domains.
ACKNOWLEDGMENT
This work was partially supported by JSPS KAKENHI
Grant Number 25730052.
R EFERENCES
[1] T. Tomita, K. Ushiki, Y. Kawakatsu, N. Fujino, H. Mineno, 窶抖 ask-
Driven Device Ensemble System Supporting Seamless Execution of
User Tasks Despite Multiplexed Interruptions, 窶 International Journal
of Informatics Society, vol.5, no.1, pp.49-58, Apr. 2013.
[2] H. Kawata, Y. Takano, Y. Iwata, N. Kanamaru, K. Shimokura, Y. Fujita,
”Field Trial of Asynchronous Communication Using Network-Based
Interactive Child Watch System for the Participation of Parents in
Day-Care Activities,” Proc. of the IEEE International Conference on
Robotics and Automation 2008, pp.2558-2563, Pasadena, USA, May
2008.
[3] G. Xing, R. Lan-lan, G. Shao-Yong, Q. Xue-Song, ”The Converged
Protocol for Management of Heterogeneous Devices” Proc. of the 6th
International Conference on Wireless Communication Networking and
Mobile Computing 2010, pp,1-5, Chengdu, China, Sept. 2010.
[4] M. Ruta, F. Scioscia, E. D. Sciascio, G. Loseto, 窶彜 emantic-Based
Enhancement of ISO/IEC 14543-3 EIB/KNX Standard for Building
Automation, 窶 IEEE Trans. on Industrial Informatics, vol.7, no.4,
pp.731-739, Nov. 2011.
[5] M. Ruta, F. Scioscia, G. Loseto, E. D. Sciascio, 窶彜 emantic-Based Re-
source Discovery and Orchestration in Home and Building Automation:
A Multi-Agent Approach, 窶 IEEE Trans. on Industrial Informatics,
vol.10, no.1, pp.730-741, Feb. 2014.
[6] M. Sato, K. Kamei, S. Nishio, N. Hagita, ”The Ubiquitous Network
Robot Platform: Common Platform for Continuous Daily Robotic
Services,” Proc. of the IEEE/SICE International Symposium on System
Integration 2011, pp.318-323, Kyoto, Japan Dec. 2011.
[7] C. Bills, J.Chen, A.Saxena, ”Autonomous MAV flight in indoor environ-
ments using single image perspective cues,” Proc. of 2011 IEEE Interna-
tional Conference on Robotics and Automation (ICRA), pp.5776-5783,
Shanghai, China, May 2011.
[8] T. Kato, R. Chiba, H. Takahashi, K. Sasai, G. Kitagata, T. Ki-
noshita, ”Multiagent-Based Cooperation Infrastructure for IoT Devices,”
Proc. of the 3rd IEEE Global Conference on Consumer Electronics
(GCCE2014), pp.571-572, Chiba, Japan, Oct. 2014.
[9] T. Uchiya,T. Maemura, H. Hara, K. Sugawara, T. Kinoshita, ”Interactive
Design Method of Agent System for Symbiotic Computing,” Interna-
tional Journal of Cognitive Informatics and Natural Intelligence, vol.3,
no.1, pp.57-74, Aug. 2009.
[10] IDEA/DASH Tutorial. Available online:
http://www.k.riec.tohoku.ac.jp/idea/index.html
[11] LEGO Mindstorms.
http://mindstorms.lego.com/en-us/Default.aspx
[12] Parrot AR. Drone 2.0
http://www.parrot.com/jp/products/