Chapter 9: The Human Eye

Introduction
• Half the human cerebral cortex is involved with vision
• Distance from refractive surface to the point where
parallel light rays converge is called the focal point
• Vision is 20/20 when you can recognize a letter that
subtends an angle of 0.083º = 5 minutes of arc
– newborn's visual acuity is approximately 20/400
– 20/120 @ one month, 20/60 @ four, 20/30 @ eight
• Rods are 1000 times more sensitive to light than cones
• The central fovea is unique in that it contains only cones
• The receptive field of a cell is the area of retina that, when
stimulated when light, changes the cell’s potential
Introduction (continued)
• The center-surround organization of retinal ganglion cell
receptive fields emphasizes contrast at light-dark edges
• There are three types of ganglion cells in mammals:
– M-type (magno) respond with burst, constitute 5% of
the total and have relatively large receptive fields
– P-type (parvo) respond with a sustained activation,
constitute 90% of total, have smaller receptive fields
– nonM-nonP cells constitute the remaining 5% and
along with the P-type cells tend to be color sensitive
• There are approximately 125 million photoreceptors in
the retinal that are funnelled into 1 million ganglion cells
• Clearly there is compression of the incoming information
Properties of Light
• Light
– Electromagnetic radiation
– Wavelength, frequency, amplitude
Properties of Light
• Light
– Energy is proportional to frequency
– e.g., gamma radiation and cool colors — high
energy
– e.g., radio waves and hot colors — low energy
Properties of Light
• Optics
– Study of light rays and their
interactions
• Reflection
• Bouncing of light rays off a
surface
• Absorption
• Transfer of light energy to a
particle or surface
• Refraction
• Bending of light rays from
one medium to another
The Structure of the Eye
• Gross Anatomy of the Eye
– Pupil: Opening where light
enters the eye
– Sclera: White of the eye
– Iris: Gives color to eyes
– Cornea: Glassy transparent
external surface of the eye
– Optic nerve: Bundle of
axons from the retina
The Structure of the Eye
• Ophthalmoscopic Appearance of the Eye
The Structure of the Eye
• Cross-Sectional Anatomy of the Eye
Image Formation by the Eye
• Refraction of light by the cornea
– Eye collects light, focuses on retina, forms images
Image Formation by the Eye
• Accommodation by the Lens
– Changing shape of lens allows extra focusing power
Image Formation by the Eye
• The Pupillary Light Reflex
– Connections between retina and brain stem neurons
that control muscle around pupil
– Continuously adjusting to different ambient light
levels
– Consensual
– Pupil similar to the aperture of a camera
Image Formation by the Eye
• The Visual Field
– Amount of space viewed by the
retina when the eye is fixated
straight ahead
Image Formation by the Eye
• Visual Acuity
– Ability to distinguish two nearby
points
– Visual angle: distances across
the retina described in degrees
Microscopic Anatomy of the Retina

• Direct (vertical) pathway:

– Ganglion cells

↑
– Bipolar cells

↑
– Photoreceptors
Microscopic Anatomy of the Retina
• Retinal processing also influenced
lateral connections:
– Horizontal cells
• Receive input from
photoreceptors and project
to other photoreceptors and
bipolar cells
– Amacrine cells
• Receive input from bipolar
cells and project to ganglion
cells, bipolar cells, and other
amacrine cells
Microscopic Anatomy of the Retina
• The Laminar Organization
– Inside-out
– Light passes through ganglion and bipolar cells before
reaching photoreceptors
Microscopic Anatomy of the
Retina
• Photoreceptor Structure
– Converts electromagnetic radiation
to neural signals
– Four main regions
• Outer segment
• Inner segment
• Cell body
• Synaptic terminal
– Types of photoreceptors
• Rods and cones
Microscopic Anatomy of the
Retina
• Regional Differences in Retinal
Structure
– Varies from fovea to retinal
periphery
– Peripheral retina
• Higher ratio of rods to
cones
• Higher ratio of
photoreceptors to
ganglion cells
• More sensitive to light
Microscopic Anatomy of the Retina
• Regional Differences in Retinal Structure (Cont’d)
– Cross-section of fovea: pit in retina where outer
layers are pushed aside
• Maximizes visual acuity
– Central fovea: All cones (no rods)
• 1:1 ratio with ganglion cells
• Area of highest visual acuity
Phototransduction
• Phototransduction in Rods
– Light energy interacts with photopigment to produce a change in
membrane potential
– Analogous to activity at G-protein coupled neurotransmitter
receptor — but causes a decrease in second messenger
Phototransduction
• Phototransduction in Rods
– Dark current: rod outer segments are depolarized in the dark
because of steady influx of Na+
– Photoreceptors hyperpolarize in response to light
Phototransduction
• Phototransduction in Rods
– Light activated biochemical cascade in a photoreceptor
– The consequence of this biochemical cascade is signal
amplification
Phototransduction
• Dark and Light Adaptation
20–25 minutes

All-cone daytime vision All-rod nighttime vision

– Dark adaptation—factors
• Dilation of pupils
• Regeneration of unbleached rhodopsin
• Adjustment of functional circuitry
 Phototransduction
• Phototransduction in Cones
– Similar to rod
phototransduction
– Different opsins
• Red, green, blue
• Color detection
– Contributions of blue, green,
and red cones to retinal signal
– Spectral sensitivity
– Young-Helmholtz trichromacy
theory of color vision
Unlike the RGB and CMYK color models, Lab color — 1976 CIE LAB also known as CIELAB — is designed to approximate
human vision. It aspires to perceptual uniformity, and its L component closely matches human perception of lightness.
Phototransduction
• Dark and Light Adaptation
– Calcium’s Role in Light Adaptation
• Calcium concentration changes in photoreceptors
• Indirectly regulates levels of cGMP  channels
Retinal Processing
• Research in ganglion cell output by
– Keffer Hartline, Stephen Kuffler, and Horace Barlow
– Only ganglion cells produce action potentials
• Research in how ganglion cell properties are generated by
synaptic interactions in the retina
– John Dowling and Frank Werblin
– Other retinal neurons produce graded changes in
membrane potential
Retinal Processing
• Transformations in the Outer Plexiform Layer
– Photoreceptors release less neurotransmitter when
stimulated by light
– Influence horizontal cells and bipolar cells
Retinal Processing
• Receptive Field: “On” and “Off” Bipolar Cells
– Receptive field: Stimulation in a small part of the visual
field changes a cell’s membrane potential
– Antagonistic center-surround receptive fields
Retinal Processing
• On-center Bipolar Cell
– Light on (less glutamate); Light off -> more
glutamate
Retinal Output
• Ganglion Cell Receptive Fields
– On-Center and Off-Center ganglion cells
– Responsive to differences in illumination
Retinal Output
• Types of Ganglion Cells
– Appearance, connectivity, and electrophysiological
properties
• M-type (Magno) and P-type (Parvo)ganglion cells in
monkey and human retina
Retinal Output
• Color-Opponent Ganglion Cells
Retinal Output
• Color-Opponent Ganglion Cells
Retinal Output
• Parallel Processing
– Simultaneous input from two eyes
• Information from compared in cortex
• Depth and the distance of object
– Information about light and dark: ON-center and
OFF-center ganglion cells
– Different receptive fields and response properties of
retinal ganglion cells: M- and P- cells, and nonM-
nonP cells
Concluding Remarks
• Light emitted by or reflected off objects in space 
imaged onto the retina
• Transduction
– Light energy converted into membrane
potentials
– Phototransduction parallels olfactory
transduction
• Electrical-to-chemical-electrical signal
• Mapping of visual space onto retina cells not uniform
 Chapter 10: Central Visual System

Introduction
• Lateral geniculate receptive fields identical to retina
• 10% to superior colliculus; 80% from striate cortex
• LGN is more than simple relay but not well understood
• Striate cortex is responsive to orientation, motion, color
• Striate cortex has complex cells implementing invariants
• Dorsal stream analyzes motion and visual-control action
• Ventral stream facilitates shape and color perception
• Inferior temporal lobe analyzes complex shapes objects
• Complex stimuli are represented by a distributed code
The Retinofugal Projection
• The Optic Nerve, Optic Chiasm, and Optic Tract
The Retinofugal Projection
• Right and Left Visual Hemifields
– Left hemifield projects to right side of brain
– Ganglion cell axons from nasal retina cross, temporal retinal
axons stay ipsilateral
The Retinofugal Projection
• Visual deficits from lesions in the retinofugal projection
The Retinofugal Projection
• Nonthalamic Targets of the Optic Tract:
– Hypothalamus: biological rhythms, including
sleep and wakefulness

– Pretectum: size of the pupil; certain types of

eye movement

– Superior colliculus: orients the eyes in response

to new stimuli
The Lateral
Geniculate
Nucleus
(LGN)
The Lateral Geniculate Nucleus (LGN)

• Inputs Segregated by Eye and Ganglion Cell Type

The Lateral Geniculate Nucleus (LGN)

• Receptive Fields
– Receptive fields of LGN neurons: identical to the
ganglion cells that feed them

– Magnocellular LGN neurons: large, monocular

receptive fields with transient response

– Parvocellular LGN cells: small, monocular

receptive fields with sustained response
The Lateral Geniculate Nucleus (LGN)
• Nonretinal Inputs to the LGN
– Primary visual cortex provides 80% of the synaptic
input to the LGN

– Brain stem neurons provide modulatory influence on

neuronal activity
Anatomy
of the
Striate
Cortex
Anatomy of the Striate Cortex
• Retinotopy
– Map of the visual field onto a target structure (retina, LGN,
superior colliculus, striate cortex)

– Central visual field overrepresented

– Discrete point of light: Activates many cells in the target

structure due to overlapping receptive fields

– Perception: Based on the brain’s interpretation of

distributed patterns of activity
Anatomy of the Striate Cortex
• Retinotopy
Anatomy of the Striate Cortex
• Lamination of the Striate Cortex
– Layers I - VI
– Spiny stellate cells: spine-
covered dendrites; layer IVC
– Pyramidal cells: spines; thick
apical dendrite;
layers III, IVβ, V, VI
– Inhibitory neurons: lack
spines; all cortical layers;
form local connections
Anatomy of the Striate Cortex
• Inputs to the Striate Cortex
– Magnocellular LGN neurons: project to layer IVCα

– Parvocellular LGN neurons: project to layer IVCβ

– Koniocellular LGN axons: bypasses layer IV to make

synapses in layers II and III
Anatomy of the Striate Cortex
• Ocular Dominance Columns
– Studied with transneuronal autoradiography from retina, to LGN,
to striate cortex.
Anatomy of the Striate Cortex
• Ocular Dominance Columns
– Present in layer IV of macaque monkeys - alternating inputs
from two eyes
Anatomy of the Striate Cortex
• Inputs to the Striate Cortex
– First binocular neurons found in striate cortex — most
layer III neurons are binocular (but not layer IV)
Anatomy of the Striate Cortex

• Outputs of the Striate Cortex:

– Layers II, III, and IVB:
projects to other cortical areas
– Layer V: projects to the
superior colliculus and pons
– Layer VI: projects back to the
LGN
Anatomy of the Striate Cortex
• Cytochrome Oxidase Blobs
– Cytochrome oxidase: mitochondrial
enzyme used for cell metabolism
– Blobs: cytochrome oxidase staining
in striate cortex
– Each blob centered on an ocular
dominance column in layer IV
– Receive koniocellular inputs from
LGN
Physiology of the Striate Cortex
• Monocular Receptive Fields
– Layer IVC: similar to LGN cells
– Layer IVCα: insensitive to the wavelength
– Layer IVCβ: center-surround color opponency
• Binocular Receptive Fields
– Layers superficial to IVC: first binocular receptive
fields in the visual pathway
– Two receptive fields — one for each eye
Physiology of the Striate Cortex
• Cortical Receptive Fields
– Orientation Selectivity
Physiology of the Striate Cortex

• Cortical Receptive Fields

– Direction Selectivity
• Neuron fires action potentials in response to
moving bar of light
Physiology of the Striate Cortex
• Cortical Receptive Fields
– Simple cells: binocular; orientation-selective;
elongated on-off region with antagonistic flanks
responds to optimally oriented bar of light
– Possibly composed of three LGN cell axons with
center-surround receptive fields
Physiology of the Striate Cortex
• Cortical Receptive Fields
– Complex cells: binocular; orientation-selective; ON
and OFF responses to the bar of light but unlike
simple cells, no distinct on-off regions
Physiology of the Striate Cortex
• Cortical Receptive Fields
– Blob Receptive Fields:
• Circular
• Monocular
• No orientation or direction selectivity
• Majority of color-sensitive neurons outside
layer IVC
• Specialized for analysis of object color
Physiology of the Striate Cortex
• Parallel Pathways: Magnocellular; Koniocellular;
Parvocellular
Physiology of the Striate Cortex
• Cortical Module
– Each module capable of analyzing every
aspect of a portion of the visual field
Beyond Striate Cortex
• Dorsal stream
– Analysis of visual motion and
the visual control of action
• Ventral stream
– Perception of the visual world
and the recognition of objects
Beyond Striate Cortex
• The Dorsal Stream (V1, V2, V3, MT, MST, other dorsal
areas)
– Area MT (temporal lobe)
• Most cells: direction-selective; respond more to
the motion of objects than their shape
– Beyond area MT — three roles of cells in area MST
(parietal lobe)
• Navigation
• Directing eye movements
• Motion perception
Beyond Striate Cortex
• The Ventral Stream (V1, V2, V3, V4, IT, other ventral
areas)
– Area V4
• Achromatopsia: clinical syndrome in humans-
caused by damage to area V4; partial or complete
loss of color vision
– Area IT
• Major output of V4
• Receptive fields respond to a wide variety of colors
and abstract shapes
From Single Neurons to Perception
• Visual perception
– Identifying & assigning meaning to objects
• Hierarchy of complex receptive fields
– Retinal ganglion cells: center-surround
structure, sensitive to contrast, and wavelength
of light
– Striate cortex: orientation selectivity, direction
selectivity, and binocularity
– Extrastriate cortical areas: selective responsive
to complex shapes; e.g., faces
From Single Neurons to Perception
• From Photoreceptors to Grandmother Cells
– Grandmother cells: face-selective neurons in area IT?
– Probably not: perception is not based on the activity
of individual, higher order cells
• Parallel Processing and Perception
– Groups of cortical areas contribute to the perception
of color, motion, and identifying object meaning
Concluding Remarks
• Vision
– Perception combines individually identified properties
of visual objects
– Achieved by simultaneous, parallel processing of
several visual pathways
• Parallel processing
– Like the sound produced by an orchestra of visual
areas rather than the end product of an assembly
line
Computational Models

Complete Bipartite Hierarchical Structure

Spatial Structure Temporal Structure

Computational Models

Simple V1-like Model

 Computational Models

Simple V1-like Model

• Resize the input to 150 pixels preserving the aspect ratio
• Grayscale plus normalize to zero-mean and unit-variance
• Local normalization: pixel intensity / norm of 3 x 3 patch

Pinto, N., Cox, D., DiCarlo J. Why is real-world visual object recognition hard? PLoS Computational Biology 4(1): e27, 2008.
 Simple V1-like Model
• Resize the input to 150 pixels preserving the aspect ratio
• Grayscale plus normalize to zero-mean and unit-variance
• Local normalization: pixel intensity / norm of 3 x 3 patch
Local contrast normalization by divisive normalization

M. J. Wainwright, O. Schwartz, and E. P. Simoncelli. Natural image statistics and divisive normalization: Modeling nonlinearity
and adaptation in cortical neurons. Probabilistic Models of the Brain: Perception and Neural Function, pages 203–222. 2002.
 Simple V1-like Model
• Resize the input to 150 pixels preserving the aspect ratio
• Grayscale plus normalize to zero-mean and unit-variance
• Local normalization: pixel intensity / norm of 3 x 3 patch
• Orientation filters: apply 96 43 x 43 Gabor wavelets

Spanning sixteen orientations and six spatial scales

Pinto, N., Cox, D., DiCarlo J. Why is real-world visual object recognition hard? PLoS Computational Biology 4(1): e27, 2008.
 Simple V1-like Model
• Resize the input to 150 pixels preserving the aspect ratio
• Grayscale plus normalize to zero-mean and unit-variance
• Local normalization: pixel intensity / norm of 3 x 3 patch
• Orientation filters: apply 96 43 x 43 Gabor wavelets
• Normalize output: response / norm of 3 x 3 x 96 patch
• Threshold output: clip values < 0 to 0 and clip > 1 to 1

Corresponds to half-wave rectification and is omnipresent

Pinto, N., Cox, D., DiCarlo J. Why is real-world visual object recognition hard? PLoS Computational Biology 4(1): e27, 2008.
 Simple V1-like Model
• Resize the input to 150 pixels preserving the aspect ratio
• Grayscale plus normalize to zero-mean and unit-variance
• Local normalization: pixel intensity / norm of 3 x 3 patch
• Orientation filters: apply 96 43 x 43 Gabor wavelets
• Normalize output: response / norm of 3 x 3 x 96 patch
• Threshold output: clip values < 0 to 0 and clip > 1 to 1
• Classification: multiple kernel support vector machines

Pinto, N., Cox, D., DiCarlo J. Why is real-world visual object recognition hard? PLoS Computational Biology 4(1): e27, 2008.
Experimental Datasets
 Simple V1-like Model

Pinto, N., Cox, D., DiCarlo J. Why is real-world visual object recognition hard? PLoS Computational Biology 4(1): e27, 2008.
Experimental Datasets
 Simple V1-like Model

Pinto, N., Cox, D., DiCarlo J. Why is real-world visual object recognition hard? PLoS Computational Biology 4(1): e27, 2008.
Ventral Pathway Model
Recording and Stimulating Cortical Tissue

• Positron Emission Tomography (PET)

• Functional Magnetic Resonance Imagery
• Single Neuron Spike Recordings
• Multielectrode Implantable Arrays
• Two-Photon Microscope Imaging
• Magnetoelectroencephalography (MEG)
• Transcranial magnetic stimulation (TMS)
In Vivo Two-Photon Microscopy
 Inferotemporal area TE

Y. Yamane, K. Tsunoda, M. Matsumoto, N. A. Phillips, and M. Tanifuji. Representation of the spatial relationship among object
parts by neurons in macaque inferotemporal cortex. Journal Neurophysiology, 96:3147-3156, 2006.
K. Tsunoda, Y. Yamane, M. Nishizaki, and M. Tanifuji. Complex objects are represented in macaque inferotemporal cortex by
the combination of feature columns. Nature Neuroscience, 4:832-838, 2001.
Y. Yamane, K. Tsunoda, M. Matsumoto, N. A. Phillips, and M. Tanifuji. Representation of the spatial relationship among object
parts by neurons in macaque inferotemporal cortex. Journal Neurophysiology, 96:3147-3156, 2006.
K. Tsunoda, Y. Yamane, M. Nishizaki, and M. Tanifuji. Complex objects are represented in macaque inferotemporal cortex by
the combination of feature columns. Nature Neuroscience, 4:832-838, 2001.
Nicolas Pinto, David Doukhan, James DiCarlo, and David Cox. A high-throughput screening approach to discovering good
forms of biologically inspired visual representation. PLoS Computational Biology, 5(11):e1000579, November 2009.
 Top Performing Models in the Long Tail

Nicolas Pinto, David Doukhan, James DiCarlo, and David Cox. A high-throughput screening approach to discovering good
forms of biologically inspired visual representation. PLoS Computational Biology, 5(11):e1000579, November 2009.
 1

SIFT: Scale Invariant Feature Transform [Lowe, 2004]

GB: Geometric Blur [Berg & Malik, 2001]
PHOG: Pyramidal Histogram of Gradients [Lazebnik et al, 2006]
PHOW: Pyramidal Histogram of Words [Bosch et al, 2007]
SLF: Sparse Localized Features [Mutch & Lowe, 2008]
1. Face Multiple Pose Illumination Expression Dataset