Humanoides - Osaka University

CNR101x Week2 Handout
Android Science
Hiroshi
ISHIGURO
Distinguished Professor of Osaka University

Visiting Director & Fellow of ATR Hiroshi Ishiguro Laboratories
www.is.sys.es.osaka-u.ac.jp, www.geminoid.jp
Robot Society
2016 Osaka University. All Rights Reserved.

1
Android
Android
Human
Robot
Society

2
Why do we use the

humanoid?
Humans have brains that
recognize humans
The ideal interface for
humans are humans.
Studies on humans and
humanoids are studies on
principle of interface.

3
Near future society

supported by humanoids and android
Humanoids that interact
with humans by using
multiple modalities
Elderly care
Public places
Learning
Department stores
Stations

4
Research topics in the past and future

Neuroscience
Robotics
Pattern
Recognition
Information Society
1990
Cerebellum
Mobile robot
Kansei
Model-based
Parametric
Web service
Sharing information
2000
Cerebrum
Active vision
Behaviorism
Learning
Learning XXX
Social Network
Society on the net
2010
BMI
Interaction
Development
Deep Learning
Smart Phone
Real time small society
2020
Frontal
cortex
Intention
Language
Intention
Consciousness
Society
Language
Non-parametric
Big data
Language
Personal Agent
Language processing
10
Android Science

5
11
Robotics for understanding humans
Bio-mimetic mechanisms
Learning and Development of the software
Conversation
Perception
Movement
Appearance
Human
12
Studies on humanlikeness

6
Development of interactive robots
13
Studies on Human-Robot Interaction

300 action patterns and 700 transition
rules
Robots working in a shopping mall
14
Combination of robots and sensor networks

Semi-autonomous robots

7
Development of child androids
15
Very humanlike appearance

But not humanlike movements
16
Android:
very humanlike humanoid robot

8
Subconscious and reactive movements
17
Reactive movement
Subconscious movement
The subconscious movement gives human likeness to the android. However,

professionals find the difference.
Reactive movements of a human is more complicated since it has mental states.
Studies on cognitive and brain science improve the android.
Familiarity
Overcoming the uncanny valley

Human
Uncanny
Adult
valley
Toy robot
android
18
Neuroscience
study on
uncanny valley
Doll
Similarity
Child android
100%
Moving corpse
The female android is not uncanny anymore.

Cognitive science and neuroscience with androids
Android Science: Integration of human studies and robotics

9
Android Science
Robotics and Cognitive Science
Development of
mechanical humans
Engineering
Science
Understanding on
Human behaviors
Mechanical
Engineering
Systems
Engineering
Robotics
Pattern
Recognition
Brain
science
Hypothesis
and
verification
Cognitive
science
Psychology
Scientific issue
Human likeness (appearance , movement, perception)
Engineering issue
Simple and interactive communication tasks
20
Practical application of
interactive robots

10
Human archiving
21
Representation of human presence

Eternal life as a national treasure
22
What is the human presence?
Android vs. the person

Which presence is more strong?

11
Story-telling android
The original show window
23
24
Emotional states and 65 behaviors

Reactions to visitors
Practical role of the android

12
25
The control system

Kinect
Camera
Sensor info
procession
How
many people
How many people
look at F
Behavior model
Gazing someone
Position of people
Looking around
Gesture of People
Emotion
control
Output
Sensor
Basic motion
Neutral
(Waiting)
Looking around
Emotion
Arousal
Russels Model
1
Excitation
Tension
-1
Unhappy
Peace
Depression
Normal
Happy
Happy
Sad

13
Idol singer
27
Ideally beautiful singer

Many fans
28
What is beauty?
Which is more beautiful, humans or androids?

What is the perfect idol singer?

14
Sales clerk of the department store
29
Dialogue system using tablet computers (without voice

recognition).
It handles more customers than human clerks.
Verbal conversation
30
Voice recognition system and Cleverbot (Winner of

Loebner Prize in 2005 and 2006)
One question and one answer

15
Conversation among robots
31
They seem conversational.

Feeling of intelligence from the robot conversation.
32
What is intelligence? What is thinking?
What is intelligence?
Is a human more intelligent than computers?
How can we define to think?

16
Intelligence in the cloud
33
Emotion is the fastest way to connect among humans

and robots.
Knowledge on the net for more intelligent conversation.
Facial
expression
Gaze
control
Representation
of emotion
Conversational pattern
Selection of
related topics by
Google search
Understanding
concepts with
Wikipedia
34
Humanlike conversation
Practical teleoperation
and data collection

17
35
Geminoid
Internet
Teleoperated android of an existing person
Operator
Meeting with Geminoid
36

18
Comparison with video and speaker
The Geminoid has stronger presence than

others.
But it is more uncanny.
Geminoid
Video
Speaker
38
What is identity?
Who has the identity?

Can the human have identity?
Is there the peak for the
identity?
The identity is relative and it is
in the society.

19
39
Adaptation to
the different body
40
Internet
Virtual tactile sensation
Operator

20
Body Ownership Model [Tsakiris M. et al 2005]

Arm
displacement
Motor
plan
Motor
command
Arm
displacement
Efference copy
Forward model
Self-body
Recognition
Comparator
Proprioceptive
afference
Visual
afference
Body Ownership Model

during Geminoid Opereation
Arm
displacement
Motor
plan
Motor
command
Arm
displacement
Efference copy
Forward model
Body
Ownership
Illusion
Comparator
Proprioceptive
afference
Visual
afference
Geminoid

21

Arm
displacement
Motor
plan
Motor
command
Arm
displacement
Efference copy
Forward model
Body
Ownership
Illusion
Comparator
Does the body ownership transfer

occur in the absence of
proprioceptive feedback?
Proprioceptive
afference
Visual
afference
Geminoid
BCI Configuration
The subject imagines a left or right hand movement
Geminoids hands move according to the subjects
brain activity
gTEC BCI system
Geminoid system
Visual
Feedback
Amplifier
Raw
data
Signal
processing
classifier
Geminoid
server
Motion command

22
Experimental setup
The Procedure
Training
Task
Stimulus
Questionnaire
Q: When the robots hand was injected, did you feel as if

your own hand was injected?
I didnt feel at all 1
7 I felt very strongly
12

23
Evaluation method
Questionnaire
Participants' score to the question in each condition were averaged and
compared
Skin Conductance Response (SCR)

SCR peak value after stimulus in each condition
were averaged and compared
Results with 19 subjects

7
6
Questionnaire
*
*
SCR
2.5
Peak Value (S)
Score
5
4
3
2
2
1.5
1
0.5
Still
Match
Raw
Significant difference in
questionnaire seen as
Match > {Still, Raw}
* ( p < 0.05)
Still
Match
Raw
Significant difference for SCR

seen as
Match > Still
* ( p < 0.05)

24

Motor
Motor
Body ownership
plan
command
transfer
in the absence of
Efference copy
proprioception
Arm
displacement
Arm
displacement
Forward model
Self-body
Recognition
Comparator
Proprioceptive
afference
Visual
afference
Geminoid
Do we know ourselves?
50
We do not know ourselves as much as we know others.

25
51
New media
that transfer our presence
to distant places
Conference with Geminoid
52

26
53
What is experience?
Experience is in our brain.

Experience is in others brain.
We, myself and the Geminoid can share the
experience.
54
Humanlike emotion
and heart/mind

27
Emotional facial expression
55
Is it richer than human facial expression?
56
What is emotion?
Which is more emotional and more

sympathetic?
Can the android have human emotion?

28
57
Android theater
Moving and impressive theater with
androids.
It is world famous
with Hirata
58
Robot Theater
Representation of humanlike heart
with Hirata and Kuroki

29
Robot Theater
59
with Hirata and Kuroki
60
What is heart/mind?
Can the android have

humanlike heart/mind?
Which is more strong?

30
61
Development of
fully autonomous robot
JST ERATO ISHIGURO
Symbiotic Human Robot
Interaction
Project
2014-2020
62
Development of autonomous robots
Interlocutor
Teleoperated
Android
(Geminoid)
Internet
The teleoperated androids and robots
Operator
I have developed the Geminoid originally

for gathering data to make it autonomous.
Autonomous
Android
Autonomous androids and robots

31
63
Enhancement of the internal humanlikeness

Big data approach for developing autonomous robots
Collect a great deal

of conversational
patterns
Development of an acoustic model for

effective speech recognition by
Deep Learning
Speech and gesture
recognition
including movements and

emotional expressions
Interlocutor
Teleoperated
Android
(Geminoid)
Internet
Behavior
Operator
Classification of
the patterns
Intention
Desire
Hierarchical
models of
utterance and
behavior
generation
Robot
Role assignment
in particular situations
Utterance and
behavior generation
64
Hierarchical models of
utterance and behavior generation
Sensing
Real
world
Behavior
Action
Intention
Desire
Sensing
Internal
(Prediction) representation
Action
(Revision)
Robot
Hierarchical models studied for decision-making in mobile

robotics.
The models are based on the idea accepted in cognitive science
and artificial intelligence.
The desires generate intentions, and intention generate
behaviors and utterances.

32
65
Acquisition of the Hierarchical Models

Behavior
Behavior
Behavior
Behavior
Behavior
Behavior
Behavior
Behavior
Intention
Behavior
Behavior
Behavior
Intention
Intention
Intention
Desire
Desire
Behavior
Behavior
Behavior
Behavior
Behavior
Behavior
Behavior
Intention
Intention
Desire
Intention controls activation or selection of behavior.

Desire controls activation or selection of intention.
Machine learning techniques classify collected data with
intention flags given by the operator and find paths for satisfying
intentions.
66
Inference of peoples desires and

intentions
based on the hierarchical models
Robots need to infer others intention and desire in social situations for having
proper conversations.
The robots assume the hierarchical models not only for other robots but also
for humans, and infer humans intentions and desires.
What is mind? What is consciousness?
Speech and gesture
recognition
Behavior
Intention
Desire
Speech and gesture

recognition
Hierarchical
models of
utterance and
behavior
generation
Behavior
Geminoid
Geminoid
Robot
Utterance and
behavior generation
Intention
Human
Desire
Hierarchical
models of
utterance and
behavior
generation
Human
Utterance and
behavior generation

33
67
The minimal human
How do we recognize
humans and geminoids
68

34
How do we recognize
the human presence with Telenoid?
69
70
Human has imagination
Humans interact with

others by using the
imagination.
The minimum design
maximizes the imagination.

35
71
Telenoid
Internet
The imagined face is mentally projected onto the neutral

appearance.
Communication service for elderlies
72

36
Field tests with Telenoids
73
Practical service for elderly care.

In Denmark and Japan.

37
Portable media
for transmitting human presence
75
New information media after smartphone

From voice and visual to human presence
76
The 3G prototype

38
77
The minimum condition

to feel human presence
78
It requires two modalities.

Hugvie: the minimum media.
Talk by phones
Smartphone holder

39
Relax effect of Hugvie
79
Cortisol decrease by using Hugvie

Talking with unknown person in 15min.
Phone
Blood
test
Blood
test
5
min.
Hugvie
p<0.05
Hugvie
Phone
80
Children in the kinder garden

40
81
With hugvies
What do we need for feeling human

presence?
More than two modalities

Voice + Tactile sensation
Appearance + Tactile sen.
Smell + Tactile sensation
Voice + Smell
82
Strength
of the feeling
0 1 2 3 4 5
Number of modalities

41
83
Humanlike Mechanism
Engineering explanation
of Yuragi formula
Current d
x f ( x)
control dt
Control model
Yuragi
formula
d
x f ( x) activity
dt
Control model with attractors
Optimization measure
Yuragi

42
Engineering explanation
of Yuragi formula
Current d
x f ( x)
control dt
Control model
Yuragi
formula
85
d
x f ( x) activity
dt
Control model with attractors
Optimization measure
Yuragi
When activity is small, Yuragi becomes relatively large, and it searches semioptimal solution based on Yuragi. Then, activity becomes large and it stays there.
It can be applied for complicated cases where proper models do not exist.
It does not compute all combinations, but it rather searches by Yuragi and
drastically reduce the computational cost/energy.
Potential
u(x)
f ( x)
State valuable
u(x) does not have to be precise or

accurate
du( x )
dx
activity
u(x)activity is minimized by the

yuragi formula
u( x ) activity
Attractor
86
Control of complicated robots based

on the biological principle
Bacteria robot with one motor

and one sensor
Robotic arm that has humanlike bone structure

and muscle arrangement

43
87
Simulation of
humanlike development
with Asada (Osaka Univ.)
Humanlike developmental mechanism
88
with Asada (Osaka Univ.)
Variations
of the
movements
Combinations of the
elemental behavior
General
movements
Identification of the
elemental behaviors
Age
U-shape development of infant
Yuragi simulates the infant

developmental process.

44
89
Difference between humans and robot?
Can we distinguish them with

the fresh body?
Why do humans want to be
humans?
Someday, the boundary may
disappear. Is the boundary
necessary?
90
What is the definition of Human?
It is changing.
Are humans always more valuable than robots?
Do humans have the unlimited value to survive?

45
91
Robot society
Android
Human
Android
Systems
Engineering
Standardization
User
Interface
Ubiquitous
systems
Material
Engineering
Pattern
Recognition
Artificial Life
Complex systems
Chaotic systems
Economy
Marketing
Media & Art
Sensor
Actuator
Field test
Robotics
Biology
Neuroscience
Philosophy
Cognitive science
Psychology
Sociology

46

Humanoides - Osaka University

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Humanoides - Osaka University

Hochgeladen von

Copyright:

Verfügbare Formate

CNR101x Week2 Handout

Distinguished Professor of Osaka University

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Why do we use the

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Near future society

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Research topics in the past and future

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Robotics for understanding humans

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Development of interactive robots

Studies on Human-Robot Interaction

Robots working in a shopping mall

Combination of robots and sensor networks

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Development of child androids

Very humanlike appearance

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Subconscious and reactive movements

The subconscious movement gives human likeness to the android. However,

Overcoming the uncanny valley

The female android is not uncanny anymore.

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Representation of human presence

What is the human presence?

Android vs. the person

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

The original show window

Emotional states and 65 behaviors

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

The control system

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Ideally beautiful singer

Which is more beautiful, humans or androids?

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Sales clerk of the department store

Dialogue system using tablet computers (without voice

Voice recognition system and Cleverbot (Winner of

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Conversation among robots

They seem conversational.

What is intelligence? What is thinking?

2016 Osaka University. All Rights Reserved.

CNR101x Week2 Handout

Intelligence in the cloud