Pitzer Presentation

Automatic 3D Reconstruction and Modeling
Title Slide

Nice Picture of 3D Models

Nice Picture of Segway
Automatic 3D Reconstruction
and Modeling
Research and Technology Center North America
Benjamin Pitzer | 10/18/2010 | 2010 Robert Bosch LLC and affiliates. All rights reserved.
Spot the difference!
3D Model inspection
Goal and Motivation

Growing demand for digitizing indoor and outdoor environments
Applications
3D content generation for
mapping service applications, e.g. google maps, bing maps
video games based on real locations
interactive museum exhibits
Pre-visualization and virtual sets for the film industry
Virtual Reality Systems
Realistic simulators for robotic algorithms (e.g. ROS/gazebo)
Goal: autonomous system for efficient creation of photo

realistic 3D models of indoor and outdoor environments
Overview
Data Acquisition
Registration
Surface
Reconstruction
Texture
Reconstruction
3D Object Representations
Raw Data

Point Cloud
Rang Image
Polygon Soup
Surfaces

Meshes
Parametric
Implicit
Solids

Voxels
BSP Trees
CSG
High Level Structures

Scene Graphs
Skeleton
etc.
Data Acquisition (1)

Robot hardware including range and imaging sensors

Frontier based exploration for active environment exploration
2D mapping and localization (Grisetti et. al, 2005)
5MP Camera
Scanning
Lidar
prev. 3D scans
next scan
trajectory
Stereo Cameras
SLR Camera
Current
location
Tilting Lidar
Pan-tilt unit
frontier
Exploration
Lidar
Base Lidar
Boschs 3D Mapping Robot

Willow Garages PR2
Pairwise Registration

Iterative Closest Point (Besl & McKay 92)

Optimization loop
Finding point correspondences
(distance, normal, color)
Minimizing error metric

Apply transform and iterate
Fast convergence
Easy to implement
Errors accumulate for multiple scans
10
Global Registration
SLAM: estimating the robots pose and

a map at the same time.
Full SLAM posterior:
Map Prior
Measurement links (scan matching)
Motion Model
Measurement Model
Finding the most probable solution:
Motion links (wheel odometry)
A vector of features representing the environment.

The robots position and orientation.
Maximum a-posteriori estimation!
A feature observation.
The control vector applied a time t-1.
11
Global Registration Example
2D Floorplan
12
3D Map
Global Registration with Map Prior

Full SLAM posterior:
p(m, x | u, z ) = p( x0 ) p( m) p( xi | xi 1 , ui ) p( zi | xi , mi )
Map Prior
Motion Model
Sensor Model
Use a local surface model to

approximate global map prior:
Intuition: observation of surfaces belong to smooth manifolds
Result: Probabilistic Non-rigid Registration (Pitzer et al. ICRA 2010)
13
Global Registration with Map Prior Example
Registration without map priors
14
Registration with map priors
Surface Reconstruction
15
Volumetric Surface Reconstruction

Surface reconstruction using a indicator function
oriented points
indicator function
iso-surface
Gradient of the smoothed indicator function is equal to the

smoothed surface normal field (Kazhdan et al., 2006)
Point Cloud
Formulation as a Poisson problem:

indicator gradient
Bilateral filter as smoothing function.
Surface Model
16
Appearance Reconstruction

Reconstruction of material properties (e.g. color, specularity, temperature)

Inference of material types (e.g. wood, plastic, vegetation)
Enhancing visual impression by adding more realism
Surface Model
17
Textured Surface Model
Color Reconstruction (1)

1. Surface partitioning

Surface segmentation into nearly planar regions

Merging of similar regions using adjacency graphs
2. Unfolding

Mapping of surface into texture domain

Finding a conformal mapping which
preserves the angular distortion

Minimizing the complex conformal
energy (Levy et al., 2002):
3. Color Reconstruction

Unprojection of 3D points into camera images
18
Color Reconstruction (2)

Global optimization of texture color to remove

discontinuities.
Multi-view blending algorithm:
before blending
averaging
Poisson formulation with Dirichlet boundary

conditions (Perez et al., 2003)
19
our algorithm
20
Handling Large 3D Models (1)

Office model (50m x 140m)

28M Vertices, 55M Faces
772 MByte Geometry
600 MByte Texture
Generation of scene graphs by

iteratively dividing 3D model
Model simplification of scene graph
nodes to limit number of elements
per node
View-dependent culling and loading
21
22
23
24
Failure cases:
Reconstruction of fine structures

(plants, furniture, etc.)
Texture reconstruction failures due

to registration errors
Future work:

Overcome limitation to static environments

Online integration of new data vs. batch processing
25
Thank you for your attention!

http://bosch-ros-pkg.sourceforge.net/
26

Pitzer Presentation

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Pitzer Presentation

Hochgeladen von

Copyright:

Verfügbare Formate

Automatic 3D Reconstruction and Modeling

Nice Picture of 3D Models

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Spot the difference!

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Goal and Motivation

Growing demand for digitizing indoor and outdoor environments

Goal: autonomous system for efficient creation of photo

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Research and Technology Center North America

High Level Structures

Automatic 3D Reconstruction and Modeling

Data Acquisition (1)

Robot hardware including range and imaging sensors

Boschs 3D Mapping Robot

Willow Garages PR2

Automatic 3D Reconstruction and Modeling

Data Acquisition (2)

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Data Acquisition (3)

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Iterative Closest Point (Besl & McKay 92)

Apply transform and iterate

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

SLAM: estimating the robots pose and

Measurement links (scan matching)

Finding the most probable solution:

Motion links (wheel odometry)

A vector of features representing the environment.

Maximum a-posteriori estimation!

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Global Registration Example

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Global Registration with Map Prior

Full SLAM posterior:

Use a local surface model to

Intuition: observation of surfaces belong to smooth manifolds

Result: Probabilistic Non-rigid Registration (Pitzer et al. ICRA 2010)

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Global Registration with Map Prior Example

Registration without map priors

Research and Technology Center North America

Registration with map priors

Automatic 3D Reconstruction and Modeling

Research and Technology Center North America

Automatic 3D Reconstruction and Modeling

Volumetric Surface Reconstruction

Surface reconstruction using a indicator function

Gradient of the smoothed indicator function is equal to the

Formulation as a Poisson problem:

Bilateral filter as smoothing function.