Feasibility of Social Interfaces based on Tactile Senses

for Caring Communication

Shigueo Nomura, Keng S. Teh, Hooman A. Samani,
Isuru Godage, Michelle Narangoda, and Adrian D. Cheok
Mixed Reality Lab, National University of Singapore
21 Lower Kent Ridge Road, Singapore 119077
{shigueo|james.tks|hooman|
isuru|michelle|adriancheok}@mixedrealitylab.org
Osamu Katai
Dept. of Systems Science, Grad. School of Informatics, Kyoto University
Yoshida Honmachi, Sakyo, Kyoto, Japan 606-8501
katai@i.kyoto-u.ac.jp
Abstract
In this paper, we investigate the feasibility of social interfaces as a new form of media to
connect humans in the physical world by means of the virtual world by providing caring communication. We propose hugging and loving communication interfaces as well as conceptual
frameworks for reinforcing advantages of the tactile sense of the skin to exchange feelings in a
remote caring communication. The hugging communication interface provides physical interaction in a remote communication between humans by enabling to hug one another through
a hugging input device and a wearable hug reproducing jacket. In this interface, we have
examined the human brain activity when using computer mediated touch devices compared
with real physical touch. The objective is to prove that a computer mediated touch can produce a similar affective response on humans relative to a real physical touch. The loving
communication interface aims to develop the functionalities yielding a new form of intimate
relationship with robots and virtual entities exploring the concept and possibilities of human
love. Furthermore, a conceptual framework for a more accurate tactile feeling interaction and
a platform for kissing interaction are proposed as well.
Keywords: Caring communication, social interface, tactile sense

Introduction

Lately, there has been an increasing interest on touch and feeling communication with humans. Researchers around the world have been examining the possibility of embedding haptics to everyday human
communication since by nature, humans are physical creatures that long for physical touch [1, 4, 6, 24].
Recent advances in media provide audio and visual communication over long distances in many ways
(Internet, mobile phones, broadband). However, we have a lack of understanding of real feelings between
the sender and receiver [7]. The desktop metaphor of todays workstation is instead an airplane-seat
metaphor. One can see only a few things at once. This kind of metaphor limits the ability for humans to
communicate effectively and affectively [39].
Analogical communication instead of digital communication involves representing something with a
likeness and includes nonverbal communication. For instance, the set of words I love you is the digital
form of communication, while physical attentiveness, embracing, kissing and long soulful looks would be

the analogical communication of love. In the healthcare field much of the communication that could be
called caring communication is analogical in nature [25].
In the area of haptics, researchers have proposed novel applications of mediated touch that can be
used to improve the fidelity in multimodal or multisensory communication. In a review of current research
and future directions of mediated social touch, Haans provided a comprehensive overview of various
physiological, psychological, and technological aspects of mediated social touch [15]. Early works such as
HandJive [10] and InTouch [5] focused on providing a sense of connectedness in a remote communication
through the sense of touch.
Other works [3, 40] included studies that focused on the medium or methods of haptic communication.
These works are more focused on the context of providing a haptic channel for communication.

1.1

Rationale

Researchers have concluded that the metaphor of digitally communicating, that is interacting through
displays and speakers using symbols (words), does not provide the complete sense of immersion and
effective communication for humans.
There is a great need of creating new forms of media to take advantages of caring communication to
increase the sense of telepresence using all the senses [7] which also includes nonverbal communication.
Humans use senses such as touch, smell and taste to deeply and emotionally communicate with others.
Among these, touch can better transmit the intended mind and provide intrinsic information in analogical
communication.
Social interface theory [9, 20, 27] is attracting considerable interest within the field of human-computer
interaction. Researchers in this field have argued that people treat computer interfaces as though they
were social actors rather than inanimate tools [8]. Additional evidence comes from a study by Walker and
his colleagues [37]. The study consisted of questionnaires to people using either a text display or one of
two talking-face displays to ask the questions. People interacting with a talking-face display spent more
time, made fewer mistakes, and wrote more comments than those interacting with the text display.
Social interface researchers have used a range of humanizing cues to foster a sense of virtual presence
[35]. These humanizing cues are to confer human characteristics on the interface.
In contrast to the existing social interfaces, our proposed design is not restricted to provide computer
interfaces working as social actors rather than inanimate tools. It is a paradigm shift to design novel social
interfaces for people making meaningful exchanges using media in the virtual world. In the physical world,
we often communicate using sight and auditory senses, but also through touch, taste and smell senses for
meaningful exchanges, that is, we perform multisensory communication. We believe that multisensory
communication based on Nursing Theory [29] represents a relevant component for caring communication.
Our social interfaces can satisfy users who want to touch their targets such as persons, materials,
objects, environments located remotely. Human beings as smart perceivers are able to conceptualize
haptic feeling information using their tactile senses. They can have tactile senses on texture, softness,
roughness or other features regarding several kinds of targets. We hope that they will have opportunities
to recover and enjoy the freedom to conceptualize themselves and their target without depending on
limited and unsatisfactory interpretations from an artificial expert system perspective.
There is a need for new forms of media to connect humans in the physical world and through the
virtual world, using meaning and feeling communication. Our proposed social interface designs (hugging
and loving communication interfaces) aim to satisfy this necessity, providing users with the possibility to
exchange tactile feelings represented by a hug and kiss to communicate remotely.
Our basic thesis is the feasibility of social interfaces to convey such relevant tactile feelings such as
hug and kiss, increasing the sense of telepresence as well as enhancing analogical caring communication.
In todays context, the possibility of tactile feeling, instead of only seeing or hearing remotely, can
be very useful when loved ones are far apart. For instance, it can be useful when parents are distant
from their children and they can not only see or hear them but also feel them. Also, when boyfriends are
distant from their girlfriends they can hug and kiss their loved ones. In this sense, parents and loved ones
can build and carry out a stronger caring communication even if they are not close to each other.
Section 2 presents nursing theories as a basis for multisensory communication leading to the overall
motivation of this work. Our proposed designs such as the hugging and loving communication interfaces

are described in Section 3. Section 4 presents a conclusion of this work.

Overall Motivation

The activities of care and caring have long traditions and histories with humans that may be of significant
comparativeness with the historic human being. So it may be said that focusing on the world of care
and caring elucidate the essential aspects of human relationships and human-human interactions and
communication in the deep sense. Also, it should be noted that there are many theories on nursing
proposed so far. This is due to the fact that nursing is tightly related to the conceptualization of what
a human being is, that is, the understanding and interpretation of the whole aspects of a human being.
Thus, considering about caring and nursing seems to be one of the crucial initiating points in exploring
new means of human interfaces and interactions.

2.1

Nursing Theories

Nursing theories are concerned with inventing and discovering various focal concepts and methods of
nursing by examining various events and their relevance in nursing interventions with the purpose of
describing, explaining and prescribing caring processes. They provide us with basic frameworks and
aspects for organizing various pieces of knowledge, information and events in nursing processes. The
monumental book Notes on Nursing: What It Is and What Is It not [29] by Florence Nightingale
had not been noticed for many years. After 1950, various theories motivated by the Notes have been
developed. These theories can be categorized into four groups: Environmental Theories, Need-Oriented
Theories, System Theories, and Interaction-Oriented Theories [14].
Besides the above established theories, there are a group of theories that may be categorized as Postmodern Approaches to nursing. It seems natural to start with these approaches to consider the effect of
tactile senses such as touching and hugging between human beings.
First of all, we will focus on the Theory of Human Becoming introduced by Rosemarie R. Parse
[30]. At first, it was called Man-Living-Health Theory. This theory is based on the philosophies of
Heidegger, Sartre, and Merleau-Ponty. Also, the theory is based on the grand theory of nursing, the
Unitary Human-being Theory introduced by Martha Rogers [33].
The theory focuses on the following three basic themes:
(1) Meaning
(2) Phythmicity
(3) Transcedence
The first theme is to structurize multidimensionality of meaning through imaging, valuing and verbalizing. The second theme is to co-create rhythmical patterns of relating through constituting paradoxical
unity of revealing-concealing, enabling-limiting, and connecting-separating. The unity of life encompasses
apparent opposites in rhythmic patterns of relating. The third theme is to move beyond the meaning
of (now) moment with what is not-yet. Moving beyond is propelling with envisioned possibility of
transforming.
Also, we should mention the Theory of Health as Expanding Consciousness introduced by Margaret
Newman based on Martha Rogers grand theory [28]. Rogers theory regards a human being as a unitary
existence that cannot be reduced to parts but be a field of energy which can be characterized as patterns.
Newman regarded that consciousness is a manifestation of an evolving pattern of human-environment
interactions. Consciousness is related not only to brain activities such as emotions and cognitions but also
the interconnectedness of the entire living system as endocrine, immune, and gene systems. This pattern
(information) is a part of a larger and undivided patterns of an expanded universe.
Her theory is also affected by David Bohms [2] theory of Undivided Wholeness. This leads to the
idea that diseases and illness are not negative situations but are the steps towards forming or moving
towards a higher level of consciousness. Thus illness and health can be regarded as a unitary process of
growth.

This process-oriented aspects of health and the theory of nursing are also affected by the famous system
theory of Dissipative Structures by Prigogine [31] which is regarded as one of the origins of complex
systems theories and clarifies how a piece of micro-scaled fluctuations lead to a macro-scaled order of
formation and structure.
Thus, nursing is not regarded as treatment of separated symptoms such as is done in the usual medical
treatment but as assistance of clients to search for the essential meaning of the whole patterns of their lives
through recognizing and understanding the evolution of patterns. Hence, a nurse and a client constitute
an expanding process of consciousness on their own living patterns.

2.2

Caring Communication

In discussing caring communication, we should mention the famous thoughts on caring developed by
Milton Mayeroff. In his book, On Caring [23], he defined caring as follows: To care for another person,
in the most significant sense, is to help him grow and actualize himself. For pursuing this objective, he
discussed on crucial factors such as knowledge, honesty, trust, hope, courage, etc. Also, he thought that
through caring, a man lives up to the meaning of his own life. In this sense, he can be said to be at home
in the world, that is, he is in place through caring.
For elucidating a more concrete image of caring, we will introduce the Human Caring Theory [38]
of Jean Watson as one of the typical and radical characterization of Caring Communication.
Jean Watsons theory was developed on the ground of phenomenological philosophy, existential psychology, and humanistic-altruistic values which is also characterized as Transpersonal Caring.
This caring process and communication is based on the seven assumptions as follows:
(1) Caring can be effectively demonstrated and practiced only interpersonally.
(2) Caring consists of carative factors that result in the satisfaction of certain human needs.
(3) Effective caring promotes health and individual or family growth.
(4) Caring responses accept a person not only as he or she is now but as what he or she may be.
(5) A caring environment is one that offers the development of potential while allowing the person to
choose the best action for himself or herself at a given point in time.
(6) Caring is more healthogenic than is curing. A science of caring is complementary to the science of
curing.
(7) The practice of caring is central to nursing.
Also, the caring communication processes focus on the following ten carative factors:
Humanistic-altruistic value system
Faith-hope
Sensitivity to own-self and others
Helping-trust relationship
Expression of positive and negative feelings
Use of scientific problem-solving methods
Interpersonal teaching-learning
Supportive, protective and/or corrective mental, physical, socio-cultural and spiritual environment
Gratification of human needs
Existential-phenomenological forces

Actuation

Loving Communication

Ultrasound

Human x Robot

Mechanical

Interface

Hugging Communication
Human x Human

Interface

t1

t2

Time

Figure 1: Proposed social interfaces categorized by actuation and interaction types

Proposed Social Interface Designs

In this work, we have designed two kinds of social interfaces to increase the sense of telepresence and
provide haptic feeling exchanges of a hug and kiss. We have directed our research towards providing a
realistic feel of a hug and kiss for caring communication. Figure 1 shows a diagram categorizing these
interfaces in terms of actuation and interaction types. For the exchange of hugs, we have designed a
hugging communication interface that stimulates users through mechanical (air pressure) actuation. For
the exchange of kisses, a loving communication interface has been designed. In this loving communication
interface, an ultimate actuation based on ultrasound signals is used to stimulate lips and exchange kisses.
Although the hugging communication interface is to provide a remote interaction between humans, the
loving communication interface is used to provide future kissing interactions with robots and virtual
entities.

3.1

Technological Innovations for Tactile Feeling Interaction

The proposed framework is to provide users with the possibility of touching and feeling targets (objects
or persons) which are remotely located from them. Then, we hope that tactile feelings can be conveyed
toward a richer caring communication. Particularly, this kind of interface could be useful also for visually
impaired users who want to know or recognize the objects or persons surrounding them. Furthermore, it
could be useful for children who like to touch everything reachable to them. It could work as a protection
way for these children against dangerous or poisonous targets.
Figure 2 shows a schematic overview of the proposed framework. In this overview we can basically
verify a signal sensor media module to capture the features from the target and an actuator module
to stimulate the tactile senses of the users skin. To provide an accurate non-contact tactile sensing
mechanism, we intend to test focused ultrasound [13] for nerve stimulation of the skin. The interaction
between the user and target is to be performed by the following modules:
A coder to convert the signals from the capture module to remote communication signals.
A decoder to reconvert the remote communication signals to the captured signals.
We can consider the following stages to achieve this goal:
First stage: For physical distance interaction between users and targets. This physical distance
should be in a range that is reachable by the sensor media to capture the feature information from
the target.

USER
CONTROLLER

6
ACTUATOR

5
CODER

DECODER

3
Types of signal:
1. Emitted signal
2. Reflected signal
3. Captured signal
4. Digital signal
5. Captured signal
6. Stimulation signal

SIGNAL
SENSOR
1

2
TARGET

Figure 2: Schematic overview of the proposed tactile feeling framework

Second stage: For long distance interaction, that is, for long distances via remote communication
(Internet, for example).

3.2

A Hugging Communication Interface: Huggy Pajama

Nowadays, hugging communication is one of the most fundamental needs and desires of children. The
convergence of the Internet, broadband and mobile devices make remote communication easily available,
even while on the move. However parents lack the understanding of the real feeling of children by simple
video, sound or text. Huggy Pajama seeks to explore caring communication methods that allow for
affective, emotional and individual expressions. More specifically, parents and children would be able to
express love and the kind of delicate human communication through a wearable, soft and ambient media
interface. In this work, we have sought to identify and implement some aspects of hug that will benefit
remote parent and child communication. We have developed a soft, hugging communication interface
connected to an ambient and soft wearable interface that allows a parent and child to communicate with
each other remotely.

3.2.1

Importance of Hugging Communication

In todays modern urban lifestyle, it is an undeniable fact that working parents and their children are
constantly kept apart from one another by work, business trips and social events. With the advent of
technology, many new methods of communication are devised. However, one major draw back is these
methods only focus on the visual and audible communication between the two parties. In our daily lives,
our interactions with others involve more than just talking. Through a hug, a handshake or a pat, people
interact with each other tangibly.
One of the more important acts of showing affection to young children is hugging. Hugging and
touching is a vital part of the human brain communication essential for the mental development of young
children [11]. Young children may not fully understand words, but a hug is a natural and intuitive
way to communicate feelings of care to them. Research suggests that infants who are unable to speak,
communicate their affective states through touch [26]. Through hugging and touching, we can transcend
spoken language to the language of wider expression and inner feeling. Hugging is therefore an important
interaction between parent and child.
Although never intended to replace real physical hugging, we believe this system would be of great
benefit for times when the parent and child cannot be at the same physical location. Our system offers a

supplement to normal physical contact, encouraging new interactions while separated physically.

3.2.2

System Description

We have designed and implemented the Huggy Pajama system with a number of features that we believe
will make the hugging communication between parents and children more meaningful. The system is able
to detect the different hug areas and the force of the hug as input, and transmitting this data via the
Internet reproducing the force at the receiving end. The system consists of key innovations as follows:
Input hug sensing: It corresponds to the signal sensor module of the framework presented in Figure 2.
Our input hugging device acts as a caring communication interface that allows parents and child
to hug each other. It can sense varying levels of soft touch and squeeze force. The force variation
is sensed by this module, digitized, and transmitted via the Internet to the output module. The
device is shaped like a human body, and the force sensors are mapped on a one-to-one basis to the
actuators on the wearable hug actuating jacket. This design simplifies the interface and enables
users to form a correct and semantically meaningful mental representation of the system model.
The cute and small design promotes empathy and evokes feelings in a caring communication.
Output hug actuation: It corresponds to the actuation module of the framework presented in Figure 2. Unlike haptic or tactile systems which uses vibrotactile actuators, our hug sensation is
produced in a calm and relaxing way through air. This ambient and calm feeling helps to invoke
emotional responses and feelings in users for caring communication. The output device is also a
wearable jacket which is soft and cute, making it easy to be adorned by the users.
Real time bidirectional hug interaction: Our system enables two remote people to send and receive
hugs in real time. The remote connection is made using the Internet as the second stage of the
goal that was presented in Section 3.1. Just as remote people can communicate with voice using
mobile phones, they can communicate with hugs using the Huggy Pajama. For example, the parent
(sender) sends a hug to their child (receiver) and if parents are at work or in a business meeting,
it might not be suitable for them to put on the pajama. They can easily hug their child by using
only the input device. However in the case of being in an office, hotel, or airport, the parent could
wear the pajama and have a two-way hugging communication with the child. When coupled with
voice communication, it is possible to increase the capability to express and invoke deeper emotional
feelings in users.
Color changing clothes: On the pajama jacket, we implemented color changing patterns. One
example is that of a flower pattern. Initially, the flower pattern is not visible. As the parent
communicates feelings to the child using the input device, the flower pattern gradually appears.
This allows users to communicate emotional and mood expressions.
The overall Huggy Pajama system is a wearable hugging jacket which includes the two modules mentioned above. An overall block diagram of the system is shown in Figure 3. Apart from hug sensing,
warmth is also essential for accurate reproduction of a human hug. The overall system consists of a digitally controlled and monitored thermal unit to reproduce the warmth. In addition, thermally controlled
color changing fabric displays are attached to enable expressive communication. The overall system is
connected via the Internet to reproduce a real-time hug sensation and affective or caring communication.
The overall remote hugging jacket consists of twelve individual air compartments which corresponds
to each of the twelve sensors on the input doll as shown in Figure 4. We used micro air pumps to
control the input and output of air from the air pockets in the jacket. These pumps are controlled by our
microcontroller circuit which constantly monitors and maintains the set air pressure in the air pockets.
These air pockets exert pressure at a particular area on the body of the wearer of the jacket.

3.2.3

Psychological Evaluation

Despite several works and developed prototypes, researchers are not clear about the effect of a remotely
mediated touch in relation to an actual human touch. The distance between two interacting persons would
be critical for a remotely mediated touch system. Since a touch naturally happens only when two persons

Touch Sensing
User
Actions

Air Pockets

Haptic
Reproduction

Warmth

Heat
Reproduction

Thermo-chromic
ink fabric flower

Emotional
Transfer

Force Sensing

SendEmoticons
Emoticons

Force level
Indicator

Input Doll

Wearable output
jacket

Figure 3: Overall block diagram showing different modules of system

Figure 4: Mapping of input sensors to output actuation modules

are close by, this would imply that for humans to benefit from a hug exchange, physical closeness is a
necessary condition. Our hypothesis is that physical distance may be irrelevant for a touch to be beneficial.
Thus, touch would influence the recipient in a bottom-up manner through a somatosensory experience.
The implication is that, rather than closeness being a necessary condition, a remotely mediated touch
would help to create closeness to a distant person.
The evaluation results for Huggy Pajama in a remotely mediated touch interaction were promising
since we might be able to reproduce a physical touch remotely. And we might be able to elicit feelings in
humans comparable to when they are physically touching each other. We cannot make concrete claims
that a remotely mediated touch can be a direct substitute for a real physical touch but it gives scientifically
positive indication for its use.

3.2.4

Possible Application Scenarios

Direct case: Parents who are away can communicate openly with the child. At night, parents can
tuck-in their child to bed by giving their child a hug. Parents can also send a sweet dreams
message by activating a smiling moon-shaped pattern on the pajama.
Indirect case: It is the first day of school for the child. The child feels insecure and parents worry
about their child. In order not to interrupt class and to reassure the child at the same time, parents
give the child a soft, calm and quiet hug. The child can see this coming from the change of color on

Figure 5: Lovotics schematic

the badge accessory on the pajama.
In the following section, a loving communication interface is proposed and designed to exchange kisses
between humans and virtual entities through high accurate non-contact tactile sensing.

3.3

A Loving Communication Interface: Lovotics

There has been a few attempts to communicate intimacy among humans [36]. Yet not much attention has
been given to the area of social/emotional interaction with robotic entities through loving communication.
Our aim is to address that vacuum and give humans and robots a new dimension to express themselves
better emotionaly through a loving communication interface.
We have investigated and proposed a novel social interface for providing the intimate relationship with
robots called Lovotics in loving communication. Love is any of a number of emotions and experiences
related to a sense of strong affection and the goal of Lovotics is to develop such association between a
human to robot and even a robot to robot.
Considering love as the critical issue for the human being constantly and with recent advances in
the field of the robotics the question rises that Can humans love robots?!. Furthermore, Can robots
love humans?! Obviously, the answer to these two questions depends on the definition of love in this
case. Lovotics as the science of studying love and robots requires to employ available methodologies for
love in human beings firstly, in order to establish the platform by considering requirements, limitations,
possibilities and circumstances for further development in this field.
Lovotics is a multidisciplinary research field that involves philosophy, psychology, biology, anthropology, neuroscience, social science, computer science, robotics, engineering and artificial intelligence.
Specifically, it is very close to the scientific study of emotion or affect, including mood and affect display
and their underlying physiology which is known as affective science.
The first step in Lovotics is to develop a deep understanding of the human affection system in order
to model this in the robot. Even though various fields have proposed ideas about the role and function
of love, the current understanding about love is still quite limited. Furthermore, developing an affection
system similar to that of the human being presents considerable technological challenges. The robot
needs to be an active participant in the communication process depending on inputs and feedback from
the human and should be able to generate suitable states and behaviors.
Our approach aims to provide a system, allowing the development of functionalities yielding a new form
of intimate relationships essential for caring communication. These relationships offer new possibilities
for exploring the concept and possibilities of human love. Developing the feeling of love for a robot is not
possible without considering the overall emotional module that can handle robots affective states. So we
have mathematically modeled the emotional properties of robot which controls the affective states. In
this work we describe the mechanism for transition in emotional system.

3.3.1

Lovotics Description

The robot requires data acquisition from a human and the processing of this data. Based on such
information the internal affective states of the robot would change. In order to create the love ability of
a robot, we propose a model for handling the affective states of a robot. Such an emotional engine can
manage the overall affective properties of the robot. A robot with this ability can have several states.
The robot with such emotional input sensory and affective state processor can be equipped with relevant
actions and behaviors like a hug and kiss. Figure 5 shows the overall structure of Lovotics.

Lovotics Affective Processor

During interaction, different emotions could be transferred from other robots or humans to robot. Such
transition can be modeled by considering each emotion as a combination of 6 basic emotional values over
time: happiness, sadness, disgust, surprise, anger, and fear. Furthermore, the robot itself is in one of
the internal states over time. The affective state of the robot at any time depends on its initial state
plus the interaction result caused by the interaction input and that result should be affected by the extra
parameters which push the resulted state toward the certain point in the affective space. To model the
system to link interaction and affective state, the transition in the affective state space can be formulated
as follows:
~t
~th +
~ +
~
S
=S
(1)
Act,M ot,Sub
~t is the affective state of the agent in the affective space at time t and S
~th is the affective state
where S
in time t h, where h is the processing time gap as a discrete system;
is the adjusting parameter (which can be a number) for a convergent vector field to emulate the
same functionality as desires for the human being;
~ is the vector field over the states which converges to the certain point in our affective state

coordinate system. This convergent vector field varies by time in a similar manner as it changes in the
human being, at least based on daily needs. Obviously a similar inducement can cause different reactions
from the human in different times. That fact also helps that affective state vector doesnt move far away
from affective states areas;
is the affective state coefficient which represents the personality of the agent that controls the rate
of change in the mood. Larger means that the state would change faster which makes the agent more
moody;
is the learning rate. The change in state is different when an agent has more interaction and this
parameter helps to have more realistic changes in the affective state;
~ =
~ Act,M ot,Sub is the 3D normal vector to transfer the state over time in an affective state space

based on the emotional input according to the interactions. The first two components are in the Activation
- Motivation plane which are driven from the emotional input:
~ Act,M ot =

6
X
m=1

eM otm M~ot +

6
X

~
eActm Act

(2)

m=1

~ are representation of the Motivation and Activation axes in x and y directions

where M~ot and Act
in the Cartesian coordinate system and vectors e are corresponding value for 6 basic emotions as
happiness, sadness, disgust, surprise, anger, and f ear in the Activation and Motivation directions.
~ represents the movement in the sub-state direction is which obtained from
The third component of
the rate of the first two components:
~
~ Act,M ot |Sub
~ Sub = | d

dt

(3)

~ is the representation of sub-states axes in the z direction in the Cartesian coordinate system.
where Sub
~ finds its direction to reach the next affective state.
In this way the vector
The proposed system can handle affective processes of the robot which can lead to different emotional
values, for example when the robot is happy if can smile or when it feels love it can show romantic
behaviors. If the robot can have a realistic affection system, realistic romantic behaviors from the robot
could be expected. Kissing can be one of those most pertinent behaviors of Lovotics that we describe in
detail through the next section.

Figure 6: System overview of the proposed kiss platform

Figure 7: Prototype for kissing interaction platform

Lovotics Behaviors: Platform for Kissing Interaction

Our proposed interactive kiss platform aims to provide a new hi-fidelity physical interface for transferring
a kiss between humans and robots in real-time or humans and virtual entities through digital media.
When a real kiss is given to the system, it detects the physical qualities of the kiss quantitatively and
encodes it into digital information. Since there are several means to realize the capture of the kiss action,
different data transformation methods are required. For instance, kiss actuation could employ several
classical and novel stimulation technologies such as mechanical, pneumatic, ultrasound, and electrical.
Under mechanical actuation, electrical linear actuators, small pneumatic drives can be considered. In this
case, the pressure/force parametric kiss information is transformed into mechanical parameters of these
actuation techniques (electric power, pressure) in real-time. Ultrasound radiated pressure may be utilized
in the same manner [12, 17, 18]. Vibrotactile stimulation is another mechanical technique which we can
implement for the kiss actuation by means of voice coils and micro-pin actuators. Mechanoreceptors in
humans can also be excited by electrical stimulation [19]. In this scenario, electric currents in electrodes
embedded to the kiss platform excite appropriate cutaneous receptors to provoke tactile feelings.
The proposed platform aims to elicit a feeling of being kissed physically and at the same time transfer
the physical details to the other device to which it is attached to. An overall system illustration is given
in Figure 6 and a prototype for this system is shown in Figure 7.
In the kiss actuation module of the proposed system, ultrasound direct nerve stimulation is preferred.
In this case, modulated acoustic waves propagate inside the human tissues to excite and activate the
relevant mechanoreceptors to elicit tactile feelings. By varying the signal power, modulating frequency,
and the pulse width of the modulating signal, different tactile sensations can be invoked [16]. Successful
tuning of such a system allows to electrically transfer the physical kiss properties without mechanical
interferences as in mechanical solutions.
Kiss sensing can be divided into two main categories: contact and non-contact type sensing. In contact
type sensing, all mechanical derived pressure/force measuring like capacitive, resistive, and piezoelectric
methods could be used. The transformation involved may be as simple as the difference in the measured
force/pressure. In the proposed kiss feedback module, mechanical pressure sensing is employed. The
heart-shaped pad in the prototype is mounted on several of high sensitive capacitive pressure sensors.
Pressure is only one parameter of the kissing and gives a very coarse approximation of a kiss. To capture
the other qualities like the lip shape and temperature, other non-contact sensing techniques need to be
considered. Optical sensing may be used for a high resolution kiss approximation. Its non interfering
operation is very suitable in a sensitive application like this. To eliminate the lighting related issues on

the kissing interaction, infrared cameras may be used. The advancement of the vision processing field
makes this realization straightforward.
The acquired and received data will be manipulated to create and recreate the kiss action physically
by the main lip control module of the proposed platform. Further, the data will be encoded in digital
format that enables the communication with artificial entities.
Finally, the communication module ensures the full duplex fast communication between the real and
the artificial worlds.

3.3.2

Possible Applications

Once the kiss platform is realized, loving communication applications can be extended to enable at least
three modes of interactions as follows:
Human to robot kiss: This enables an intimate relationship with a robot, such technology provides a
new facility for closer and a more realistic interactions between humans and robots. In this scenario,
one set of artificial lips is integrated in a humanoid robot.
Human to virtual entity physical/virtual kiss: This scenario provides a link between the virtual and
real worlds. Here, humans can kiss virtual entities while playing games and receiving physical kisses
from their favorite virtual entities.
Human to human tele-kiss through the kiss platform: It bridges the physical gap between two
intimately connected individuals. This platform plays the mediating role in the kissing interaction
by imitating and recreating the lip movement of both users in real time using two digitally connected
artificial lips.
Here, the objective of the proposed platform could be extended to facilitate all the scenarios. However,
there lies a few challenges to deal with in order to enable interaction with robotic entities. Firstly, it is
possible to integrate an artificial intelligence based trainable system to identify the kiss action and respond
with an appropriate kiss. For a more closer emotional interaction, the robot needs to have some way of an
understanding of intimacy and affection. Thus, it needs to be pre-programmed to react to various colours
of intimacy as in these lip interactions, giving a sense of understanding and reality. Artificial intelligence
can come into the picture once a rigorous quantified idea of kissing is established. In order to achieve
this, extensive studies need to be carried out to generalize the intimacy in machine readable form. There
have been some attempts to cross this border to bring the man-machine relationship into an intimate
level [21, 34].
Unlike humans, robotic entities will be having limited sense of ethical and societal values. Humans
may exploit this limitation to satisfy their natural physiological and psychological needs. With no rules
being defined this may also lead to abuse. In a futuristic scenario, this platform can be used to transfer
special affection to, specially another human being, whom may get attached to the robot, which may give
rise to psychological problems due to the lack of human contact [32]. The ethical aspects of robotics have
recently gained sufficient importance to be debated in international forums [22].
As this platform could be used as a long distance kiss transfer device, people who are geographically
dispersed may tend to express their feelings to each other through this kiss interface. This may create a
negative influence on both parties since they may not experience the actual physical affection. According
to research findings this may even cause social exclusion as well.
Also this system can be used to express love and devotion towards another person, it can have a
negative impact on the society, especially by creating a illusion of life where humans may fail to express
true physical affection towards another human. People may find it as an easy mechanism to care for
another person, for instance in the act of caring for elders. Moreover, the recreation of the loving tender
kiss of this system may create a major concern on the society based on human-robot intimacy related
issues. The rising question here is, whether it is really ethical to treat a robot in the same manner as
treating a human being especially with expressing natural feelings.

Conclusion

In this paper, we investigated the feasibility of social interfaces for exchanging hugs and kisses in a remote
caring communication.
For remote interaction between humans, we designed Huggy Pajama as a hugging communication
interface composed by a mobile doll hug input device and a wearable hug reproducing pajama. The
input device sends hug signals to a haptic jacket that stimulates the feeling of being hugged to the wearer.
Psychological evaluation using Huggy Pajama showed that computer mediated touch can produce a similar
affective response from humans related to a real physical touch.
For remote interaction between humans and robots (or virtual entities), we designed Lovotics as a loving
communication interface supported by a kiss platform. We presented a general overview of the designed
system showing the feasibility of this kind of social interface. It provides a hi-fidelity physical interface
enabling kiss exchanges and facilitating caring communication as well as intimate human telepresence
with both the real and virtual worlds.
Using these investigation results, in the future we plan to explore other aspects of touch such as
texture, temperature, softness, hardness enhancing remote caring communication.

References
[1] Bailenson, J. N., Yee, N., Brave, S., Merget, D., and Koslow, D. (2007). Virtual interpersonal touch:
Expressing and recognizing emotions through haptic devices. Human-Computer Interaction, 22(3):325
353.
[2] Bohm, D. (2002). Wholeness and the implicate order. Routledge.
[3] Bonanni, L., Vaucelle, C., Lieberman, J., and Zuckerman, O. (2006). Taptap: a haptic wearable for
asynchronous distributed touch therapy. In CHI 06: CHI 06 extended abstracts on Human factors in
computing systems, pages 580585, New York, NY, USA. ACM.
[4] Brave, S. and Dahley, A. (1997). inTouch: a medium for haptic interpersonal communication. In
Conference on Human Factors in Computing Systems, pages 363364. ACM New York, NY, USA.
[5] Brave, S., Ishii, H., and Dahley, A. (1998). Tangible interfaces for remote collaboration and communication. In CSCW 98: Proceedings of the 1998 ACM conference on Computer supported cooperative
work, pages 169178, New York, NY, USA. ACM.
[6] Chang, A., OModhrain, S., Jacob, R., Gunther, E., and Ishii, H. (2002). ComTouch: design of
a vibrotactile communication device. In Proceedings of the 4th conference on Designing interactive
systems: processes, practices, methods, and techniques, pages 312320. ACM New York, NY, USA.
[7] Cheok, A. D. (2009). Feeling communication: Social and physical interactive communication and
entertainment. Bussei Kenkyu, 91(4):416418.
[8] Couper, M. P. (1998). Review of Byron Reeves and Clifford Nass, The media equation: how people
treat computers, television, and new media like real people and places. Journal of Official Statistics,
13(4):441443.
[9] Dryer, D. C. (1999). Getting personal with computers: How to design personalities for agents. Applied
Artificial Intelligence, 13(3):273295.
[10] Fogg, B. J., Cuttler, L. D., Arnold, P., and Eisbach, C. (1998). Handjive: a device for interpersonal haptic entertainment. In CHI 98: Proceedings of the SIGCHI conference on Human factors in
computing systems, pages 5764, New York, NY, USA. ACM Press/Addison-Wesley Publishing Co.
[11] Fosshage, J. L. (2000). The meanings of touch in psychoanalysis: A time for reassessment. Psychoanalytic Inquiry, 20(1):2043.

[12] Gavrilov, L. R. (2008). The possibility of generating focal regions of complex configurations in application to the problems of stimulation of human receptor structures by focused ultrasound. Acoustical
Physics, 54(2):269278.
[13] Gavrilov, L. R., Tsirulnikov, E. M., and Davies, I. A. (1996). Application of focused ultrasound for
the stimulation of neural structures. Ultrasound in medicine & biology, 22(2):179192.
[14] George, J. B. (1985). Nursing theories: The base for professional nursing practice. Appleton &
Lange, Stamford, CT.
[15] Haans, A. and IJsselsteijn, W. (2006). Mediated social touch: A review of current research and future
directions. Virtual Reality, 9(2):149159.
[16] Iwamoto, T., Maeda, T., and Shinoda, H. (2001). Focused ultrasound for tactile feeling display. In
Proc. of the Eleventh International Conference on Artificial Reality and Telexistence, pages 121126.
[17] Iwamoto, T. and Shinoda, H. (2006). Two dimensional stress reproduction using ultrasound tactile
display. In SICE-ICASE, 2006. International Joint Conference, pages 48184821.
[18] Iwamoto, T., Tatezono, M., and Shinoda, H. (2008). Non-contact method for producing tactile
sensation using airborne ultrasound. In EuroHaptics 08: Proceedings of the 6th international conference
on Haptics, pages 504513, Berlin, Heidelberg. Springer-Verlag.
[19] Kajimoto, H., Kawakami, N., Maeda, T., and Tachi, S. (1999). Tactile feeling display using functional
electrical stimulation. In Proc. of the 9th Int. Conf. on Artificial Reality and Telexistence, pages 107
114.
[20] Kiesler, S. and Sproull, L. (1997). Social human-computer interaction. In Friedman, B., editor,
Human values and the design of computer technology, pages 191199. Center for the Study of Language
and Information Stanford, CA, USA.
[21] Levy, D. (2007). Love + Sex with Robots: The Evolution of Human-robot Relations. Harper Collins,
New York.
[22] Levy, D. (2008). The ethical treatment of artificially conscious robot. In International conference on
Human-robot personal relationship, Maastricht, Netherlands.
[23] Mayeroff, M. (1972). On caring. Harper-Collins Publishers.
[24] McLaughlin, M., Jung, Y., Peng, W., Jin, S. A., and Zhu, W. (2008). Touch in computer-mediated
communication. Mediated Interpersonal Communication, page 158.
[25] Montgomery, C. L. (1993). Healing Through Communication: The practice of Caring. SAGE.
[26] Moszkowski, R. J. and Stack, D. M. (2007). Infant touching behaviour during mother-infant face-toface interactions. Infant and Child Development, 16(3):307319.
[27] Nass, C. and Reeves, B. (1998). The media equation: How people treat computers, television, and
new media like real people and places. Cambridge University Press Cambridge, UK.
[28] Newman, M. A. (2000). Health as expanding consciousness. Jones & Bartlett Publishers.
[29] Nightingale, F. N. (1895). Note on nursing: What it is and what it is not (Com. Ed.). Lippincott,
Philadelphia.
[30] Parse, R. R. (1993). Theory of Health as Human Becoming. Sage Publications Inc.
[31] Prigogine, I. and Nicolis, G. (1977). Self-organization in nonequilibrium systems: from dissipative
structures to order through fluctuations. Wiley.
[32] Roboethics (2008). Roboethics website. http://www.roboethics.org/.

[33] Rogers, M. E. (1970). Introduction to the Theoretical Basis of Nursing. F. A. Davis, Philadelphia.
[34] Samani, H. A., Ge, S. S., and Hang, C. C. (2008). LOVOTIC: LOVE and ROBOTIC, Affective
States based on Visual Perceptions. In International conference on Human-robot personal relationship,
Maastricht, Netherlands.
[35] Tourangeau, R., Couper, M. P., and Steiger, D. M. (2003). Humanizing self-administered surveys:
Experiments on social presence in Web and IVR surveys. Computers in Human Behavior, 19(1):124.
[36] Vetere, F., Gibbs, M. R., Kjeldskov, J., Howard, S., Mueller, F. F., Pedell, S., Mecoles, K., and
Bunyan, M. (2005). Mediating intimacy: designing technologies to support strong-tie relationships.
In CHI 05: Proceedings of the SIGCHI conference on Human factors in computing systems, pages
471480, New York, NY, USA. ACM.
[37] Walker, J. H., Sproull, L., and Subramani, R. (1994). Using a human face in an interface. In Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence,
pages 8591. ACM New York, NY, USA.
[38] Watson, J. (1999). Nursing human science and human care: A theory of nursing. Jones & Bartlett
Publishers.
[39] Welch, G., Fuchs, H., Raskar, R., Towles, H., and Brown, M. S. (2000). Projected imagery in your
office of the future. IEEE Computer Graphics and Applications, 20(4):6267.
[40] Yohanan, S. and MacLean, K. E., editors (2008). The Haptic Creature Project: Social Human-Robot
Interaction through Affective Touch. AISB.