Available online at www.sciencedirect.

com
Available online at www.sciencedirect.com
ScienceDirect
Available ScienceDirect
online
Procedia
Available atonline
CIRPwww.sciencedirect.com
00 (2019) 000–000
at www.sciencedirect.com
www.elsevier.com/locate/procedia
ScienceDirect
Procedia CIRP 00 (2019) 000–000
ScienceDirect www.elsevier.com/locate/procedia

Procedia CIRP 00 (2017)

Procedia 000–000
CIRP 81 (2019) 346–351
www.elsevier.com/locate/procedia
52nd CIRP Conference on Manufacturing Systems
52nd CIRP Conference on Manufacturing Systems
An Application of 3D Model Reconstruction and Augmented Reality for
An Application of 3D
Real-Time
28th CIRP Model
Monitoring Reconstruction
Design Conference, May 2018,and
of Additive Augmented
Manufacturing
Nantes, France Reality for
Real-Time Monitoring of Additive Manufacturing
A newAmmar
methodology
Malika*, Hugo to Lhachemi
analyzea,the Joernfunctional
Ploennigsb, Amadou and physical architecture
Bab, Robert Shortena of
existing products
Ammar for an
Malika*,Electrical
Hugo and assembly
Lhachemi
Electronic
a
a
, Joernoriented
Engineering, Ploennigs
University product
b
College ,Dublin,
Amadou family
Dublin, Ba b
identification
, Robert
Ireland Shortena
b
Ireland Research Lab, IBM, Dublin, Ireland
Electrical and Electronic Engineering, University College Dublin, Dublin, Ireland
a

Paul Stief *, Jean-Yves Dantan, Alain Etienne, Ali Siadat

* Corresponding author. E-mail address: ammar.malik@ucdconnect.ie
b
Ireland Research Lab, IBM, Dublin, Ireland

* Corresponding author. E-mail address: ammar.malik@ucdconnect.ie

École Nationale Supérieure d’Arts et Métiers, Arts et Métiers ParisTech, LCFC EA 4495, 4 Rue Augustin Fresnel, Metz 57078, France

Abstract
* Corresponding author. Tel.: +33 3 87 37 54 30; E-mail address: paul.stief@ensam.eu
Abstract
This paper presents a novel scan-based method for the real-time monitoring of additive manufacturing processes. Most traditional scanning
techniques used for generating 3D models capture the only outer shape of the object after completion of the printing process. The method proposed
This paper presents a novel scan-based method for the real-time monitoring of additive manufacturing processes. Most traditional scanning
in this paper differs as it relies on a layer-by-layer scanning of the 3D object directly during the printing process. This strategy has been
Abstract
techniques used for generating 3D models capture the only outer shape of the object after completion of the printing process. The method proposed
successfully implemented with a fused filament 3D printer (PRUSA i3 MK3). Furthermore, in order to offer an increased interaction between the
in this paper differs as it relies on a layer-by-layer scanning of the 3D object directly during the printing process. This strategy has been
Inobtained
today’s 3D
successfully
modelenvironment,
business and the user,the
implemented
a virtual
with a fused trend environment
towards
filament more has
3D printer
beenvariety
product
(PRUSA
developed for the augmented
and customization
i3 MK3). Furthermore, inisorder
reality glasses
unbroken. Due
to offer toHoloLens. The novelty
this development,
an increased
of this
the need
interaction between of
the
method
agile and lies in the layer-by-layer
reconfigurable production3Dsystems
model reconstruction
emerged to of with
cope both various
the outerproducts
shape and and the inner layers
product of the
families. To printedand
design part.optimize
It enables the user,
production
obtained 3D model and the user, a virtual environment has been developed for the augmented reality glasses HoloLens. The novelty of this
directly as
systems during the printing
chooseprocess, to view and detect potential defects, not only at theare surface but Indeed,
also in the inner layers of themethods
printed aim
object.
method lieswell as to
in the the optimal
layer-by-layer 3D model product matches,
reconstruction product
of analysis
both the outer methods
shape and theneeded.
inner layers of most of the
the printed known
part. It enables the to
user,
Therefore,
analyze it can provide detailed information about the build quality and can be used as the basis of a decision-making tool.
directlyaduring
productthe orprinting
one product family
process, on theand
to view physical
detect level. Different
potential defects,product
not onlyfamilies, however,
at the surface butmay
alsodiffer
in thelargely in terms
inner layers of of
thethe number
printed and
object.
nature of components.
Therefore, it can provide This fact impedes
detailed informationan efficient
about thecomparison
build quality andandchoice
can beofused
appropriate product
as the basis family combinations
of a decision-making tool. for the production
© 2019AThe
system. new Authors. Published
methodology by Elsevier
is proposed Ltd. This
to analyze is an open
existing access
products in article
view ofunder
theirthe CC BY-NC-ND
functional license
and physical architecture. The aim is to cluster
© 2019 The Authors. Published by Elsevier Ltd.
(http://creativecommons.org/licenses/by-nc-nd/3.0/)
these
© products
2019 The in new assembly
Authors. Published oriented product
by Elsevier Ltd.families foropen
This is license
an the optimization
access articleofunder
existing
the assembly
CC BY-NC-NDlines and the creation of future reconfigurable
license
This is an
Peer-review open access
under article under
responsibility the scientific
of Flow
the CC BY-NC-ND committee (http://creativecommons.org/licenses/by-nc-nd/3.0/)
ofstructure
the 52ndofCIRP Conference on Manufacturing Systems.
assembly systems. Based on Datum Chain,
(http://creativecommons.org/licenses/by-nc-nd/3.0/) the physical the products is analyzed.
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference on Manufacturing Systems. Functional subassemblies are identified, and
aPeer-review
functional analysis is performed.
under responsibility ofMoreover, a hybrid
the scientific committeefunctional and physical
of the 52nd architecture
CIRP Conference on graph (HyFPAG)
Manufacturing is the output which depicts the
Systems.
Keywords: Additive Manufacturing; Augmented Reality; Real-Time 3D Model Reconstruction; Real-Time Monitoring.
similarity between product families by providing design support to both, production system planners and product designers. An illustrative
example
Keywords:of Additive
a nail-clipper is used toAugmented
Manufacturing; explain the proposed
Reality; methodology.
Real-Time 3D Model An industrial case
Reconstruction; study Monitoring.
Real-Time on two product families of steering columns of
thyssenkrupp Presta France is then carried out to give a first industrial evaluation of the proposed approach.
© 2017 The Authors. Published by Elsevier B.V.
1. Introduction parameters, such as process parameters (heat source, trajectory
Peer-review under responsibility of the scientific committee of the 28th CIRP Design Conference 2018.
1. Introduction generation,
parameters, such etc.), ambientparameters
as process parameters (heat (temperature
source, trajectory or
Additive manufacturing (AM), also known
Keywords: Assembly; Design method; Family identification as 3D printing, humidity), and intrinsic properties of
generation, etc.), ambient parameters (temperature or the employed materials,
refers to the manufacturing
Additive various processes (AM),of adding
also knowntogether materials
as 3D to
printing, have a strong
humidity), andimpact on the
intrinsic quality of
properties ofthethefinal productmaterials,
employed [7]. Any
create three-dimensional objects based on 3D model
refers to the various processes of adding together materials to files under uncertainty in even one of these parameters
have a strong impact on the quality of the final product [7]. Any might result in
computer control. Originally used for fast prototyping
create three-dimensional objects based on 3D model files under [1-2], defects. For instance, due to the off-line
uncertainty in even one of these parameters might result in generation and
1.AM
Introduction
has emerged of the product range and characteristics manufactured and/or
computer control. as a disruptive
Originally usedtechnology poised to deeply
for fast prototyping [1-2], optimization
defects. For of the printing
instance, due trajectories,
to the off-line the accuracy
generationof andthe
transform manufacturing [3]. However, despite its huge assembled
geometry in
maythis not
system.
reachIn this
the context,
required thelevel
main for
challenge
a in
given
AM has emerged as a disruptive technology poised to deeply optimization of the printing trajectories, the accuracy of the
Due toAMthe
potential, still fast
does development
not meet the in theof conventional
standards domain of modelling
application, andand analysis
because is ofnowthe not only to cope
layer-based withprocess,
printing single
transform manufacturing [3]. However, despite its huge geometry may not reach the required level for a given
communication
manufacturing. andparticular,
In an ongoing trendfrom
it suffers of digitization
low productivityand products, a limited can
inhomogeneities product range yielding
appear, or existingproblems
product families,
in the
potential, AM still does not meet the standards of conventional application, and because of the layer-based printing process,
digitalization,
rates, poor manufacturing
quality and enterprises
repeatability of are facing important
manufacture, uncertain but also to beproperties
mechanical able to analyze
[8]. and to compare products to define
manufacturing. In particular, it suffers from low productivity inhomogeneities can appear, yielding problems in the
challenges
properties inthetoday’s market parts,
environments: a continuing newItproduct families. It can that
be observed that classical existing
rates, poorofquality manufactured
and repeatability etc.manufacture,
of [4]. These limitations
uncertain is widely
mechanical recognized
properties [8]. the development of adequate
tendency
are some towards
of the reduction
reasons that of product
prevent the development
widespread times and
adoption of product
monitoringfamilies
methodsare regrouped
combined inwith
function of clients
efficient or features.
decision-making
properties of the manufactured parts, etc. [4]. These limitations It is widely recognized that the development of adequate
shortened
AM product lifecycles.
technologies with stringentIn addition,
process there is an increasing
requirements (such ofas However,
tools is aassembly
key steporiented
toward product
thewith families are
widespread hardly of
to find.
are some of the reasons that prevent the widespread adoption monitoring methods combined efficient adoption
decision-making AM
demand
aerospace of customization,
[5] and being industries
biomedical at the same timeThe
[6]). in aessential
global On the product
technologies in family
industriallevel,sectors
productspresenting
differ mainly in two
stringent
AM technologies with stringent process requirements (such as tools is a key step toward the widespread adoption of AM
competition
cause of these withproblems
competitors all inherent
is the over the world. This trend, main characteristics: (i) the number of components and (ii) the
aerospace [5] and biomedical industries difficulty
[6]). Thetoessential
model, requirements
technologies [9]. in Recent years
industrial have seen
sectors many applications
presenting stringentin
which
monitor, is and
inducing
control thethedevelopment
underlying fromprocess
AM macro [4]. to micro
Many type
which of components
augmented (e.g.
realitymechanical,
is used electrical,
to enhance electronical).
3D printing
cause of these problems is the inherent difficulty to model, requirements [9]. Recent years have seen many applications in
markets, results in diminished lot sizes due to augmenting Classical methodologies considering mainly single products
monitor, and control the underlying AM process [4]. Many which augmented reality is used to enhance 3D printing
product varieties (high-volume to low-volume production) [1]. or solitary, already existing product families analyze the
2212-8271 © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
To cope with this augmenting variety as well as to be able to
(http://creativecommons.org/licenses/by-nc-nd/3.0/)
product structure on a physical level (components level) which
identify
2212-8271 possible
© 2019 The optimization
Authors. Published potentials
by Elsevier in
Ltd. the
This is existing
an open access
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference causes on difficulties
article under the CC BY-NC-ND
Manufacturing regarding
Systems. license an efficient definition and
(http://creativecommons.org/licenses/by-nc-nd/3.0/)
production system, it is important to have a precise knowledge comparison of different product families. Addressing this
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference on Manufacturing Systems.
2212-8271 © 2019 The Authors. Published by Elsevier Ltd.
This is an©open
2212-8271 2017access article Published
The Authors. under theby CC BY-NC-ND
Elsevier B.V. license (http://creativecommons.org/licenses/by-nc-nd/3.0/)
Peer-review
Peer-review under
under responsibility
responsibility of scientific
of the the scientific committee
committee of the of theCIRP
28th 52ndDesign
CIRPConference
Conference2018.
on Manufacturing Systems.
10.1016/j.procir.2019.03.060
 Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351
2 Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000
technology. This paper presents a novel scan-based method for
the real-time monitoring of AM processes using 3D model
reconstruction and Augmented Reality (AR). The principal
difference between our work and that of other authors is that
image processing, and component models are used to realize a
layer-by-layer reconstruction of the printed part. It consists of
a 3D model reconstruction of the printed part based on images
taken layer-by-layer during the printing process. The resulting
3D model can be used either for the monitoring of the printing
process or the detection of internal defects of the final printed
object. In order to improve the interaction between the user and
the resulting 3D model, an AR environment has been
developed for the mixed-reality glasses HoloLens*. Note that
in this paper, AR is used as a technology brick to allow the user
to interact with the 3D model, either to visualize the full part or
to perform layer-by-layer interrogation of the final printed part.
The remainder of the paper is organized as follows. Related
works are presented in Section 2. An overview of our
experimental equipment is given in Section 3. The proposed
methodology is detailed in Section 4 and obtained results for a
case study are related in Section 5. Finally, concluding remarks
are provided in Section 6.
2. Related works
Geometry reconstruction is a well-studied area of research
in computer graphics and vision. In the past, the 3D geometries
have been reconstructed using passive cameras [10,11], active
sensors [12], online images [13] or from unordered 3D points
to generate watertight surfaces [14]. Traditional 3D model
reconstruction techniques can be generally classified as
photogrammetry and 3D scanning. In Photogrammetry (or
Structure from Motion [15]), the three-dimensional coordinates
of surface points of a 3D printed object are estimated using
pictures taken from different angles. In non-contact active 3D
scanning, the scanner estimates the surface points of the object
by detecting the reflection of radiation or light emitted by the
scanner towards the object. The emitted radiations could be
infra-red, ultrasound or X-ray. This scanning technique either
use time of flight [16] or triangulation method [17] to estimate
surface points of the object. Both photogrammetry and 3D
scanning either require multiple cameras/scanners to be
installed or a single camera/scanner to be moved around the
object to acquire images/data from different angles. The 3D
surface points generated from these images/data also contain
surface points of the environment around the object, and hence
a manual removal (3D cropping) of these surface points is
required to retain surface points of the object only. These
techniques capture only the surface information of the object
and do not provide any information about the internal condition
of the product.
Augmented Reality (AR) is a human-computer
interaction tool that augments computer-generated perceptual
information on the real-world environment. Extensive research
is being carried out to design and implement integrated AR-
*
https://www.microsoft.com/en-us/hololens
347
assisted manufacturing systems to achieve reduced cost and
improved quality. For example, in the automotive industry, the
3D models of car interiors have been overlaid on real body
mock-up cars during the initial phases of development using
AR [18]. Similarly, a gesture-based AR design environment,
GARDE, has been developed [19]. In which, using gestures,
the designer can visualize, design and make modifications in a
3D model in an AR environment. AR in manufacturing has
been mainly focused on the design phase of the manufacturing
process [20,21,22]. However, AR can also play a significant
role in the improvement of process monitoring. For instance, it
was proposed in [23] to bridge the gap between digital and
physical works by projecting the 3D model directly on top of
the currently printed part in the printer workspace. In this
context, the user can control both the displayed 3D model and
the printer actuations toward an integrated design. The idea of
superimposing the 3D model to the currently printed part was
also developed in [24] as a comparison tool for fast
prototyping. An automatic image comparison procedure was
developed for the detection of the printing failures by
comparing a picture of the part being printed and the original
3D model.
Proposed system: The 3D reconstruction technique
proposed in this paper requires only one camera mounted on
the top of the build plate; thus, avoiding the need for multiple
static cameras or moving a single camera around the object.
Also, the proposed scanning technique suggests an automated
cropping of the background from the acquired images, thus
generating a 3D model of the object only. Since the proposed
technique reconstructs the 3D model layer by layer, it can be
used for real-time monitoring of the build to detect the defects
on the surface as well as within the object. Finally, we use
AR to monitor the process of 3D printing using the
reconstructed 3D model of a part.
3. System overview
The setup of the system used to test the proposed technique
is shown in Fig. 1. It includes a Fused Filament Fabrication
(FFF) technology based, Prusa i3 MK3, 3D printer with 13
Mega Pixels camera mounted at its gantry and HoloLens. The
camera lens has been oriented parallel to the printer’s print bed
to acquire top view image of each printed layer. An app has
also been developed in Unity Platform which enables the user
to interact with the digital model in a mixed reality environment
using HoloLens.
a
Camera
Filament
Extruder
Print Bed
b
Fig. 1. (a) Camera Mounted on 3D Printer; (b) HoloLens.
348 Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351
Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000 3

4. Methodology SPECT. In image registration, one of the images (referred to as

This section presents the methodology for layer-by-layer
reconstruction of the 3D model of the printed product and
development of the app to interact with the digital model for
real-time monitoring of the build process using HoloLens.
4.1. Image acquisition and processing
To acquire images after the deposition of each layer, the
extruder and print bed must be positioned in such a way that
the work is in front of the camera. Therefore, the G-code has
been modified by adding the following command before the
commands for the next layer (i.e. before the change in z-
coordinate).
G1 X0 Y200;
This returns the extruder to the left and pushes the build plate
forward so that top view is clearly visible by the camera after
deposition of each layer. The camera-acquired images are then
processed in MATLAB to generate the 3D model. Since the
image acquired after deposition of each layer also contains
information about the previous layers (and/or about the print
bed) therefore, the image must be processed to extract the
information concerning the current layer.
4.1.1. Segregation of the Current Layer
During the acquisition of the top view image, depth
information is lost. Therefore, it is difficult to differentiate
between the current layer-area and the previous layer-area. One
possible solution to this problem is to use color-based image
segmentation. Since the majority of the layers in a 3D printed
product are of the same color therefore, we need some other
strategy for segregation. One such strategy, as proposed in this
paper, is to use the information about the current layer through
the G-code of the 3D model being used for printing. The idea
is to use the image obtained by the G-code simulator of the
current layer as a mask, and then to keep the information from
the camera acquired image which lies under the area delineated
by this mask. Then, we assign the color black to the area which
does not lie under the mask. This image, containing
information about the current layer, is termed as the processed
image in the remainder of the paper, and we refer to the overall
technique as automated cropping. Since the camera-acquired
image and the image acquired from G-Code simulator are from
different sources, we need to align both images before using
the image acquired from the G-Code simulator as a mask.
the moving or source image) is spatially transformed to align
with the other image (referred to as the target, fixed or sensed
image). In our particular case, we have set mask as a moving
image, and the camera-acquired image as a target image. While
image registration algorithms can be intensity-based or feature-
based, we have used MATLAB’s inbuilt function ‘imregister’
for intensity-based automatic image registration to align the
images. In intensity-based registration, the misaligned image
(moving image) is spatially transformed to align with the target
image by comparing intensity patterns in images [25].
4.2. 3D Model Reconstruction
The reconstruction of the 3D model involves converting the
processed image to a 3D file format. A 3D file format is used
for storing information concerning the 3D models. The
common 3D file formats used for 3D printing are STL, OBJ,
PLY, AMF, 3MF, and FBX. Since HoloLens supports OBJ and
FBX 3D model file formats, therefore, we have used OBJ file
format to estimate the geometry of the 3D model from
processed images.
4.2.1. 3D model surface encoding
The processed image is converted into a binary image. The
area of the pixels of the previous layers is assigned a black
color, whereas the current layer area is assigned a gray color.
These pixel values are then used as depth values to convert 2D
processed image to 3D image. Fig. 2 (a) shows an example of
a down-sampled triangular estimation of a processed 2D image
and Fig. 2 (b) shows 3D geometry estimation from the
processed image. The processed binary image is then converted
to OBJ file to encode the geometry of the current layer based
on the processed image. In this encoding technique, the binary
processed image is covered with the mesh of non-overlapping
triangles. The pixels of the binary processed image are assigned
as vertices of these triangles. The row number and column
number of the image pixels are assigned as x and y coordinates
whereas pixel values are assigned as z-coordinates of the
vertices of these triangles. Since we do not want the triangles
lying on the previous layer to be encoded in the OBJ file of the
current layer therefore, we use a condition to encode only the
triangles having non-zero z-coordinates.
a

4.1.2. Image Registration

b
To use the image acquired from G-Code simulator as mask, the
mask must be aligned with the camera-acquired image to
overcome issues such as image rotation, scale, and skew. This
problem of aligning multiple scenes into a single integrated
image is solved by image registration. Image registration is an
image processing technique usually used to align satellite Fig. 2. (a) Triangle mesh over Processed 2D image; (b)
images or medical images captured with different imaging Estimated 3D Geometry from Processed Image.
sources, such as Magnetic resonance imaging (MRI) and
 Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351 349
4 Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000

4.3. Interaction with the reconstructed model via HoloLens 5.1. Current Layer area Segregation

The Microsoft HoloLens is a holographic computer,
enabling the user to interact with holograms in a mixed reality
environment. An application has been developed in Unity
Platform which allows the user to interact with 3D models
using HoloLens. The user can move and rotate to inspect the
reconstructed model of each layer using hand gestures. Two
voice commands, expand model and reset model, have also
been added. The expand model command shows the digital
model layer-by-layer and let the user inspect each layer using
hand gestures whereas reset model command reconstructs and
joins all the layers back together.
Fig. 4 (b) shows the camera-acquired image after deposition of
layer 2. The white area in Fig. 4 (b) is layer 1, whereas the blue
area is layer 2. The mask (image obtained from G-Code
Simulator) to segregate layer 2 area from layer 1 area in Fig. 4
(b) is shown in Fig. 5. The misalignment between the camera
acquired image (Fig. 4 (b)) and G-Code simulator acquired
image (Fig. 5) is shown in Fig. 6. To use the layer 2 image from
G-code simulator as a mask, we need to align Fig. 5 with Fig.
4 (b). Fig. 7 (a) shows the result of alignment using Image
registration. The segregated area of the layer 2 obtained after
applying the aligned mask (Fig. 7 (b)) is shown in Fig. 8.

5. Results

The methodology described in Section 4 has been tested on

a 3D printed object shown in Fig. 3. Although the printed object
consists of 35 layers, for illustration purposes, we assume this
object consists only the of 4 layers shown in Fig. 4. The 3D
model of this product has been designed to contain a hidden I-
Form logo in the second layer and an intentional defect in the
third layer.
Fig.5. Layer 2 Image acquired from G-Code Simulator.

Fig. 3. 3D Printed Object.

a
Fig. 6. Misalignment between camera acquired and G-Code Simulator
acquired images.

c
Defect
b

Fig.4. Camera acquired images after deposition of (a) Layer 1; (b) Layer Fig. 7. (a) Aligned Images; (b) Mask Image after alignment.
2; (c) Layer 3; (d) Layer 4.
350 Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351
Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000
Fig. 8. Segregated Layer 2 (Processed) Image.
5.2. 3D Model Reconstruction
After extraction of the layer 2 area from camera acquired
image (Fig. 4 (b)), the current layer (Layer 2) geometry is
encoded in the OBJ file. The generated OBJ file for the current
layer viewed in 3D builder app is shown in Fig. 9. This process
is repeated after deposition of each layer to generate OBJ file
for each layer and these files are then merged in the developed
app.
5.3. Interaction with the reconstructed 3D model
Fig. 10 shows the reconstructed 3D Model of the four layers
viewed through HoloLens. The defect occurred during the
deposition of the third layer can be seen in Fig. 10. Hence,
using the proposed scanning technique, the user can visualize
the hidden I-Form logo in the second layer as well as the defect
in the third layer.
Fig. 9. Generated OBJ file for layer 2.
Layer 1 Layer 2
Layer 3 Layer 4
Defect
Fig. 10. Reconstructed 3D Model of the layers viewed through HoloLens.
5
6. Concluding remarks
In this research, a novel scan-based method to reconstruct
the 3D model of the product during the manufacturing process
has been proposed and tested. The proposed method updates
the 3D model of the work in progress, layer after layer, hence
enabling real-time monitoring of the process. The proposed
scanning technique also allows the user to inspect the build
quality of the product by visualizing the reconstructed 3D
model in a mixed reality environment using HoloLens. The app
developed for HoloLens also allows the user to interact with
the digital model using hand gestures and voice commands.
Thus, the proposed scanning technique not only speeds up the
iterative process development by enabling real-time
monitoring but also provides detailed information about the
build quality. The proposed scanning technique has the
potential to facilitate the development of a decision support
system using techniques from machine learning. This decision
support system will not only be responsible for real-time defect
detection but will also provide support for the operator’s
decision-making capabilities in order to mitigate the occurred
defects. This approach will be the direction of future work.
Acknowledgments
This publication has emanated from research supported in
part by a research grant from Science Foundation Ireland (SFI)
under grant number 16/RC/3872 and is co-funded under the
European Regional Development Fund and by I-Form industry
partners.
References
[1] Ramakers R, Anderson F, Grossman T, Fitzmaurice G. Retrofab: A design
tool for retrofitting physical interfaces using actuators, sensors and 3d
printing. In Proceedings of the 2016 CHI Conference on Human Factors in
Computing Systems 2016 May 7 (pp. 409-419). ACM.
[2] Mueller S, Im S, Gurevich S, Teibrich A, Pfisterer L, Guimbretière F,
Baudisch P. WirePrint: 3D printed previews for fast prototyping.
InProceedings of the 27th annual ACM symposium on User interface
software and technology 2014 Oct 5 (pp. 273-280). ACM.
[3] Cotteleer M, Joyce J. 3D opportunity: Additive manufacturing paths to
performance, innovation, and growth. Deloitte Review. 2014 Jan 17;14:5-
19.
[4] Tapia G, Elwany A. A review on process monitoring and control in metal-
based additive manufacturing. Journal of Manufacturing Science and
Engineering. 2014 Dec 1;136(6):060801.
[5] Uriondo A, Esperon-Miguez M, Perinpanayagam S. The present and future
of additive manufacturing in the aerospace sector: A review of important
aspects. Proceedings of the Institution of Mechanical Engineers, Part G:
Journal of Aerospace Engineering. 2015 Sep;229(11):2132-47.
[6] Gu DD, Meiners W, Wissenbach K, Poprawe R. Laser additive
manufacturing of metallic components: materials, processes and
mechanisms. International materials reviews. 2012 May 1;57(3):133-64.
[7] Zeng K, Pal D, Stucker B. A review of thermal analysis methods in laser
sintering and selective laser melting. In Proceedings of Solid Freeform
Fabrication Symposium Austin, TX 2012 Aug 6 (Vol. 60, pp. 796-814).
[8] Abeykoon C, Martin PJ, Li K, Kelly AL. Dynamic modelling of die melt
temperature profile in polymer extrusion: Effects of process settings, screw
geometry and material. Applied Mathematical Modelling. 2014 Feb
15;38(4):1224-36.
[9] Oropallo W, Piegl LA. Ten challenges in 3D printing. Engineering with
Computers. 2016 Jan 1;32(1):135-48.
[10] Hartley, Richard, and Andrew Zisserman. "Multiple view geometry in
computer vision." Robotica 23.2 (2005): 271-272.
 Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351 351
6 Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000

[11] Merrell P, Akbarzadeh A, Wang L, Mordohai P, Frahm JM, Yang R,

Nistér D, Pollefeys M. Real-time visibility-based fusion of depth maps.
In2007 IEEE 11th International Conference on Computer Vision 2007 Oct
14 (pp. 1-8). IEEE.
[12] Levoy M, Pulli K, Curless B, Rusinkiewicz S, Koller D, Pereira L, Ginzton
M, Anderson S, Davis J, Ginsberg J, Shade J. The digital Michelangelo
project: 3D scanning of large statues. InProceedings of the 27th annual
conference on Computer graphics and interactive techniques 2000 Jul 1
(pp. 131-144). ACM Press/Addison-Wesley Publishing Co..
[13] Frahm JM, Fite-Georgel P, Gallup D, Johnson T, Raguram R, Wu C, Jen
YH, Dunn E, Clipp B, Lazebnik S, Pollefeys M. Building rome on a
cloudless day. In European Conference on Computer Vision 2010 Sep 5
(pp. 368-381). Springer, Berlin, Heidelberg.
[14] Kazhdan M, Hoppe H. Screened poisson surface reconstruction. ACM
Transactions on Graphics (ToG). 2013 Jun 1;32(3):29.
[15]Schonberger JL, Frahm JM. Structure-from-motion revisited.
InProceedings of the IEEE Conference on Computer Vision and Pattern
Recognition 2016 (pp. 4104-4113).
[16] Cui Y, Schuon S, Chan D, Thrun S, Theobalt C. 3D shape scanning with
a time-of-flight camera. In s2010 IEEE Computer Society Conference on
Computer Vision and Pattern Recognition 2010 Jun 13 (pp. 1173-1180).
IEEE.
[17] Chow J, Ang K, Lichti D, Teskey W. Performance analysis of a low-cost
triangulation-based 3D camera: Microsoft Kinect system. InInt. Soc. for
Photogrammetry and Remote Sensing Congress (ISPRS) 2012 Aug 25
(Vol. 39, p. B5).
[18] Fründ J, Gausemeier J, Matysczok C, Radkowski R. Using augmented
reality technology to support the automobile development. InInternational
Conference on Computer Supported Cooperative Work in Design 2004
May 26 (pp. 289-298). Springer, Berlin, Heidelberg.
[19] Ng LX, Oon SW, Ong SK, Nee AY. GARDE: a gesture-based augmented
reality design evaluation system. International Journal on Interactive
Design and Manufacturing (IJIDeM). 2011 Jun 1;5(2):85.
[20] Ong S, Pang Y, Nee AY. Augmented reality aided assembly design and
planning. CIRP annals. 2007 Jan 1;56(1):49-52.
[21] Pang Y, Nee AY, Khim Ong S, Yuan M, Youcef-Toumi K. Assembly
feature design in an augmented reality environment. Assembly
Automation. 2006 Jan 1;26(1):34-43.
[22] Shen Y, Ong SK, Nee AY. Augmented reality for collaborative product
design and development. Design Studies. 2010 Mar 1;31(2):118-45.
[23] Yamaoka J, Kakehi Y. MiragePrinter: interactive fabrication on a 3D
printer with a mid-air display. In ACM SIGGRAPH 2016 Studio 2016 Jul
24 (p. 6). ACM.
[24] Ceruti A, Liverani A, Bombardi T. Augmented vision and interactive
monitoring in 3D printing process. International Journal on Interactive
Design and Manufacturing (IJIDeM). 2017 May 1;11(2):385-95.
[25] Oliveira FP, Tavares JM. Medical image registration: a review. Computer
methods in biomechanics and biomedical engineering. 2014 Jan
25;17(2):73-93.