See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/326822437

Automated Process Monitoring in 3D Printing Using Supervised Machine

Learning

Article · August 2018

DOI: 10.1016/j.promfg.2018.07.111

CITATIONS READS

6 201

2 authors:

Ugandhar Delli Shing I. Chang

Kansas State University Kansas State University
2 PUBLICATIONS   7 CITATIONS    54 PUBLICATIONS   867 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Bulk Dispatch Optimization View project

All content following this page was uploaded by Ugandhar Delli on 04 September 2018.

The user has requested enhancement of the downloaded file.

 Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
ScienceDirect
ScienceDirect
Procedia Manufacturing
Available 00 (2018) 000–000
Availableonline
onlineatatwww.sciencedirect.com
www.sciencedirect.com
Procedia Manufacturing 00 (2018) 000–000 www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia
ScienceDirect
ScienceDirect 
Procedia Manufacturing 26 (2018) 865–870
Procedia Manufacturing 00 (2017) 000–000
www.elsevier.com/locate/procedia
46th SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA
46th SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA
Automated Process Monitoring in 3D Printing Using Supervised
Automated Process Monitoring in 3D Printing Using Supervised
Machine Learning
Machine
Manufacturing Engineering Society Learning
International Conference 2017, MESIC 2017, 28-30 June
2017, Vigo (Pontevedra), Spain

Costing models for capacity optimization in Industry 4.0: Trade-off

Ugandhar Delli* and Shing Chang
between used capacity
Ugandhar and
Delli* andoperational
Shing Chang efficiency
Department of Industrial and Manufactruing Systems Engineering, Kansas State University, 2061 Rathbone Hall, 1701B Platt St, Manhattan KS
Department of Industrial and Manufactruing Systems a Engineering, Kansas
66502, United State
States University,
of America b 2061 Rathbone Hall,
b 1701B Platt St, Manhattan KS
A. Santana , P. Afonso , A. Zanin , R. Wernke
a,*
66502, United States of America
* Corresponding author. Tel.: +1-785-851-9541; fax: +0-000-000-0000 .
a
University of Minho, 4800-058 Guimarães, Portugal
E-mail address:author.
* Corresponding Tel.: +1-785-851-9541; bfax: +0-000-000-0000 .
ugandhardelli@ksu.edu
Unochapecó, 89809-000 Chapecó, SC, Brazil
E-mail address: ugandhardelli@ksu.edu

Abstract
Abstract
Abstract
Quality monitoring is still a big challenge in additive manufacturing, popularly known as 3D printing. Detection of
Quality
defectsthe
Under monitoring
during is
ofstill
the printing
concept aprocess
big challenge
"Industry will help
4.0", ineliminate
additiveprocesses
production manufacturing,
waste ofwill bepopularly
material and time.
pushed known
to Defect asdetection
3D printing.
be increasingly Detection
during of
the initial
interconnected,
defects
stages during
of the
printing printing
may process
generate an will
alert help
to eliminate
either pause waste
or stopof material
the printing and time.
process Defect
so that detection
corrective
information based on a real time basis and, necessarily, much more efficient. In this context, capacity optimization during
measuresthe initial
can be
stages ofprevent
takenbeyond
to printingthe may generate
need toaim an alert
reprint to either
the parts. pause
This paperorproposes
stop the printing
a method process so that corrective
to organization’s
automatically assess measures
the quality can be
of 3D
goes the traditional of capacity maximization, contributing also for profitability and value.
taken
printedtoparts
prevent
with thetheneed to reprintofthe
integration a parts. This
camera, paper
image proposes and
processing, a method to automatically
supervised machine assess the
learning. quality
Images of of 3D
semi-
Indeed, lean management and continuous improvement approaches suggest capacity optimization instead of
printed
finishedparts
partswith
are the integration
taken atofseveral ofcritical
a camera,
stagesimage
ofand processing,
the and supervised
printingmodels
process machine learning. Images machine
of semi-
maximization.
finished
Thetaken
studyat capacity optimization costing isaccording
an importantto the part
research geometry.
topic thatA deserves
learning parts are support
method, severalmachine
vector critical (SVM),
stages ofisthe printingtoprocess
proposed classifyaccording
the parts to theeither
into part geometry.
‘good’ or A machine
‘defective’
contributions
learning fromsupport
both thevector
practical and theoretical isperspectives. This paper presents andeither
discusses
‘good’a mathematical
orA‘defective’
category.method,
Parts using ABS and PLA machine (SVM),
materials were proposed
printed totodemonstrate
classify thethe
parts into
proposed framework. numerical
model for
category. capacity
Parts usingmanagement
ABS and based
PLA on different
materials werecosting
printed models
to (ABC
demonstrate and TDABC).
the proposed A generic
framework.model has been
A numerical
example is provided to demonstrate how the proposed method works.
developed and it was used to analyze idle capacity and
example is provided to demonstrate how the proposed method works. to design strategies towards the maximization of organization’s
value.
© 2018TheThe trade-off capacitybymaximization
Authors. Published Elsevier B.V. vs operational efficiency is highlighted and it is shown that capacity
© 2018 The Authors.
optimization might Published
hide by Elsevier
operational B.V.
inefficiency.
© 2018 The under
Peer-review Authors. Published by
responsibility of Elsevier B.V. committee of NAMRI/SME.
the scientific
Peer-review
© under responsibility
2017 The Authors. of Elsevier
Published by the scientific
B.V.committee of the 4th International Conference on System-Integrated Intelligence.
Peer-review under responsibility of the scientific committee of NAMRI/SME.
Peer-review
Keywords: 3Dunder responsibility
printing, of theSupervised
Image processing, scientific machine
committee of theSVM
learning, Manufacturing Engineering Society International Conference
2017.
Keywords: 3D printing, Image processing, Supervised machine learning, SVM

1. Introduction
Keywords: Cost Models; ABC; TDABC; Capacity Management; Idle Capacity;as
Operational Efficiency
3D printing allows for more manufacturing options
1. Introduction as 3D printing
because allows
it has theforability
more manufacturing options
to create complex
Additive manufacturing (AM) has become a viable because it shapes
geometrical has thewithability
the helptoof create complex
design software.
1. Introduction
Additive
option formanufacturing (AM)years.
production in recent has become
AM oftena knownviable geometrical shapesofwith
Other advantages AMthe help ofrapid
include design software.
prototyping,
option for production in recent years. AM often known Other advantages of AM include rapid prototyping,
The cost of idle capacity is a fundamental information for companies and their management of extreme importance
in modern©production
2351-9789 systems.
2018 The Authors. In general,
Published it isB.V.
by Elsevier defined as unused capacity or production potential and can be measured
in several©under
2351-9789
Peer-review ways: tons of production,
2018responsibility
The Authors. Published
of available
by Elsevier
the scientific B.V.hours
committee of manufacturing, etc. The management of the idle capacity
of NAMRI/SME.
Peer-review underTel.:
* Paulo Afonso. responsibility
+351 253 of the761;
510 scientific committee
fax: +351 253 604of741
NAMRI/SME.
E-mail address: psafonso@dps.uminho.pt

2351-9789 © 2017 The Authors. Published by Elsevier B.V.

Peer-review under responsibility of the scientific committee of the Manufacturing Engineering Society International Conference 2017.
2351-9789 © 2018 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the 4th International Conference on System-Integrated Intelligence.
10.1016/j.promfg.2018.07.111
866 Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870
2 Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000
inventory reduction, and decentralized production. 3D
printing has already contributed significantly to
several technological advances in medicine and is
emerging in many other fields like fashion design, and
architecture and even Nike recently launched the first
athletic shoe including 3D printed components [1].
Many 3D printing machines do not have a
designated system to track and monitor the progress of
printing process. 3D printers may continue to print the
part until all the layers have been completed even if
the filament ran out or there are any potential defects
in the print. Detecting defects by performing quality
check at various (critical) stages of the printing
process not only help in assuring corrective measures
but also eliminate waste of printing bad parts.
Automatic process monitoring is especially important
for production scale 3D printing operation in which
hundreds of 3D printing machines are used in mass
production of the same part.
A method has been proposed to monitor the 3D
printing process by pausing the printer at various
checkpoints to detect any possible defects and take
proper corrective actions such as stopping the print if
a potential defect is detected.
2. Literature review
Given the fact that 3D printing is still a relatively
modern technology in the manufacturing field, the
amount of literature available addressing the quality
aspects in 3D printing is limited. Straub [2] proposed
a method to detect completion failure defects during
the 3D printing process. A multi-camera system
comprised of five camera units (each unit consisted of
a Raspberry Pi and a Raspberry Pi camera) and image
processing software created using C# and Dot Net
framework was used to assess the quality of the parts
produced. Images (of the in-process object) were taken
at eight positions by each of five cameras placed at five
different angles (to make a total of 40 images) and they
were compared against a final object image. The
comparison was done on a pixel-by-pixel basis. The
author affirmed that the comparison in his method
could detect two types of errors: when a part has been
stopped during the progress of printing which results
in an incomplete part and when there are issues with
the printer which results in a failure of injection of
filament into the nozzle. However, this method cannot
detect geometrical or structural defects which are
common in any manufacturing process.
Wuest et al. [3] proposed a method to deal with the
large amounts of complex and high-dimensional data
in conventional manufacturing. First, they described
the concept of individual product states along the
entire manufacturing line including all sorts of
information which could be utilized during the
process. Then, application of a combination of Cluster
Analysis and SVM on product state data has been
presented as a potential way for improved quality
monitoring. Fastowicz and Okarma [4] proposed a
method for live monitoring of the progress of 3D
printing process based on texture analysis using the
Gray-Level Co-occurrence Matrix (GLCM) and
chosen Haralick features. Images obtained by
scanning the 3D printed plates have been used to
verify the approach. Also, taking the influence of
lighting conditions into consideration, several
experiments have been performed by taking pictures
with the help of a camera mounted on a 3D printer.
Due to the influence of lighting conditions on the
images obtained when compared to the scanned
images, several modifications have been made to the
proposed method to achieve better results.
Vaezi and Chua [5] studied the effects of two
parameters of layer thickness and binder saturation
level on the mechanical strength, surface quality and
dimensional accuracy in the 3D printing process.
Experimental results show that under the same layer
thickness, increasing the binder saturation level from
90% to 125% would result in an increase of tensile and
flexural strengths of the specimens and decrease of
dimensional accuracy and surface uniformity of the
specimens. On the other hand, under the same binder
saturation level, increase in layer thickness from 0.087
to 0.1 mm would decrease tensile strength and increase
flexural strength. Also, it yields better surface
uniformity. Other work related to 3D printing quality
can be found in [7] and [8]. Besides the efficient use
of supervised machine learning to monitor quality in
3D printing which is first of its kind, our research
focuses on some of the key issues in 3D printing that
were not addressed in the previous research like
detection of structural defects.
3. Implementation strategy
Conventionally, the quality of 3D printed parts is
being checked after the printing is done. Detecting
defects during the printing process not only help to
eliminate waste of material and time but also prevent
 Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870 867
Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000 3

the need to reprint the whole part. In this study, quality

is being checked at various checkpoints during the
printing process with the help of images taken by a
camera. A checkpoint is defined as an
important/critical stage in the printing process where
LED light
there is a significant change in the geometry of the
part. For example, consider a complex part which
involves different stages of printing like skirt/base,
body and the top. These stages could be considered as
the desired checkpoints to inspect quality at. The
following steps implement the proposed quality
monitoring of a 3D printed part during production.

Step 1. Identify proper check points for 3D printing

part according to its geometry.
Step 2. Take images of the semi-finished part at each
check point.
Step 3. Perform image processing and analysis. Camera

Conventionally, quality is represented in terms of an

integer or a numerical value. But here, an image is
used to represent quality of the 3D printed material. A
python code has been developed to automate the
process of taking pictures at checkpoints, then image
processing and finally classifying the parts as either
‘good’ or ‘bad’ using SVM. The proposed method is
designed to detect both completion failure defects as
well as geometrical or structural defects.
4. Experimental setup and research methods
The 3D printer used for this research is Lulzbot Mini
and the 3D printing process applied is Fused
Deposition Modeling (FDM). It should be noted that,
at this point, our proposed method is specific to the
mentioned printer and printing process only. The parts
being 3D printed are monitored by taking pictures at
various checkpoints with the help of a camera and the
setup is shown in Figure 1. The printing materials used
during this research are ABS and PLA. Both materials
have different mechanical properties which have a
significant impact on the quality of the printed
products using these two materials. Programming
language python has been used to build a code to
automate real-time monitoring of the printing process.
We maintained the same 3D printer configuration
throughout the duration of this research.
Bed
Figure 1. Experimental setup
Considering the effect of lighting conditions, several
experiments have been performed initially by taking
images using the camera for different lighting
conditions. Changes in the lighting conditions have
significantly changed the results of pixel analysis. So,
an assumption has been made to maintain constant
lighting (using the LED light shown in Figure 1)
throughout the course of experiments to facilitate
image comparison.
At first, the part desired to be printed is designed by
a design software such as SolidWorks. After that, the
STL file of the designed model is obtained. Then,
checkpoints are identified depending on the geometry
of the part. The next step is to start the printing process
and it continues until it reaches a specified checkpoint
for the given print. When it reaches a checkpoint, the
printing process is paused for a while and the nozzle is
raised up to the specified height. Then the bed moves
all the way to the front end of the printer where camera
is positioned to take pictures. Then the camera takes a
picture after a small pause. After that, the bed and the
nozzle move back to their original position (where
they were before the printer paused) and starts printing
again.
868 Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870
4 Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000

4.1 Image processing of parts is SVM classification to identify the parts as

The dimensions of the images taken at each
checkpoint is 640x480. A sample image taken at a
checkpoint (of the part printed with PLA material) is
shown in Figure 2.
either ‘good’ or ‘defective’. Detection of defective
parts may need the printing process to be stopped to
take corrective measures. If both the SVM results and
the results of initial quality check using experimental
error data go hand in hand, it indicates that the
proposed method is working as expected.

4.2 Supervised machine learning

The machine learning method used in this research is

Support Vector Machine (SVM). Applying SVM
classification is the final stage in the quality assurance
process. Before going into the research methodology
applied, let us take a brief look into SVM. Support
Vector Machine is a supervised machine learning
model which analyses data used for both regression
Figure 2. Image taken at a checkpoint analysis/classification. Given a set of training models,
each of them belonging to one or the other category,
To facilitate image analysis, every image taken at the the SVM training algorithm builds a model that
checkpoints is divided into 16 identical sections in a assigns any new models to one or the other category.
4x4 matrix. For each section, average of the RGB A simple example of SVM is shown in Figure 3.
values of all pixels is calculated. That means, for an Several applications of SVM are found in
image taken at a checkpoint, 16 average (RGB) values bioinformatics, image and text recognition etc.
have been calculated for all the 16 sections. Let si be
the average (RGB) values for sections where i = 1 to
16. The same procedure is applied for all the images
taken at various checkpoints during the printing
process. For proposed method to work, an ideal print
should be available at each check point. Any new
image at a checkpoint is then compared to the ideal
print image. At each checkpoint, experimental error Ei
(i = 1 to 16) is calculated for each section’s average
RGB values. Experimental error is calculated using
the formula:

Ei = ((Ai – Ii)/Ii) * 100 (1)

where Ei = experimental error

Ai = actual value
Ii = ideal value
i = 1 to 16.

One way to initially detect potential errors or defects

is by generating an alert if the experimental error value
of at least 4 sections are > 10% (metric chosen after
performing several experiments taking environmental Figure 3. Basic concept of SVM
factors into consideration) as it signals a significant
variation from the ideal print. After the initial quality
check, the main method applied to assess the quality
 Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870
Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000
SVM classification can be either linear or non-linear.
As it is not possible to generate linear fits for most of
the real word problems, non-linear classifiers are
widely used for pattern analysis. The most commonly
used approach to create a non-linear classifier is the
kernel trick. Kernel methods are fundamentally based
on calculating the similarity between the unlabeled
current inputs and the training inputs. Prediction for
the unlabeled inputs, i.e. for those which are not in the
training datasets, is done by the application of a
similarity function k, called a kernel, between the
unlabeled current input x’ and each of the training
inputs xi. For example, a binary classifier using kernel
trick typically calculates a weighted sum of
similarities which is given by:
ŷ = sgn ∑𝑖𝑖=1,𝑛𝑛 wiyik(xi,x’) (2)
where: 𝑛𝑛
• ŷ ∈ {-1, +1} is the predicted label of
kernelized binary classifier for the unlabeled
input x’ whose hidden true label y is to be
found
• k is the kernel function that measures
similarity between any pairs of inputs xi, x’
• yi ∈ {-1, +1} is the true label for the given
training set of inputs xi
• wi ∈ R are the weights for the training
examples determined by the learning
algorithm
• sgn is the sign function that determines
whether the predicted classification ŷ results
as positive or negative [6].
Now coming to this research, the section averages
(of RGB values) of the images calculated at
checkpoints are loaded as input to the vectors of
training models. Two categories of training models
namely ‘good’ (identical to the ideal parts) and ‘bad’
(defective parts) have been loaded into the system.
Then the SVM training algorithm has built a model
which classifies any new models loaded into the
system as either ‘good’ or ‘bad’ print. If a part is
identified as ‘bad’ during the initial stages of printing,
then a corrective measure to stop the printing process
will be taken to prevent further wastage of material
and time and then reprint the part again. The SVM
used for this research is a 5 or 6 stage process
depending upon how many checkpoints (5 or 6) are
defined for a specific print.
869
5
5. A numerical example
A 3D model of a DNA has been used to print parts for
this example. Five checkpoints are defined according
to the geometry of the DNA structure to perform
quality check at five critical stages of the printing
process. ABS material is used to print the parts. Once
the printing process starts, it continues until it reaches
a specific checkpoint. After reaching a checkpoint, the
printing process pauses for a while and the camera
mounted on the printer takes an image of the part. The
same procedure goes on until pictures are taken at
every checkpoint. A training set containing both
‘good’ and ‘defective’ parts should be used to train the
proposed SVM model to classify any new incoming
inputs. A python package scikit-learn is used to both
train and execute the proposed SVM algorithm.
Then the model is validated by another dataset which
is not used in the training dataset. The outcome of the
trained SVM model will predict the quality of each
part as either ‘good’ or ‘defective’. A sample image of
a part taken at a checkpoint is shown in Figure 4.
Figure 4. Image at a checkpoint for an unlabeled
current input
The average RGB values of the 16 sections of the
image at the (first) checkpoint are as follows: (148.18,
120.62, 106.28, ..., 87.60), which is a vector having 16
average elements. It is then fed into the SVM shown
in equation (2). Then the kernelized binary classifier
labels the input into either ‘good’ or ‘defective’
category. In this case, the result is that it is a ‘good
part’. This numerical example demonstrates how the
870 Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870
6 Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000

proposed method can be applied to perform real-time

monitoring of a 3D printing process.

6. Results and conclusions

The proposed method which integrates the efficacy of

image processing and supervised machine learning
successfully serves the purpose of real-time
monitoring of a 3D printing process. The method is
capable of detecting both completion failure defects
such as filament running out or printing stopped in the
mid-progress and structural or geometrical defects.
The main drawback of the proposed method is that the
printing process needs to be paused while the images
of a semi-finished part are taken. Another drawback is
that since only top view images are taken, the proposed
method might not be able to detect the defects on the
vertical plane which cannot be seen in the top view
image. This gives us a direction for future research to
incorporate cameras on the sides of the printer as well
to detect defects on both the horizontal and vertical
planes. A possible improvement can be made by in situ
camera mounting on print head. Future work will also
focus on the study of impacting factors for choosing
proper check points.

References

[1] G. White, Industry analysis: The pros and cons of 3D printing

(2015). Retrieved from http://www.manufacturingglobal.com/
[2] J. Straub, Initial work on the characterization of additive
manufacturing (3D printing) using software image analysis.
Journal of Machines, 3 (2015) 55-71.
[3] T. Wuest, C. Irgens, K. D. Thoben, An approach to monitoring
quality in manufacturing using supervised machine learning on
product state data. Journal of Intelligent Manufacturing, 25
(2014) 1167-1180.
[4] J. Fastowicz, K. Okarma, Texture based quality assessment of
3D prints for different lighting conditions. In Proceedings of the
International Conference on Computer Vision and Graphics,
ICCVG (2016) 17-28.
[5] M. Vaezi, C. K. Chua, Effects of layer thickness and binder
saturation level parameters on 3D printing process.
International Journal of Advanced Manufacturing Technology,
53 (2011) 275-284.
[6] J. S. Taylor, N. Cristianni, Kernel methods for pattern analysis.
Cambridge University Press, 2004.
[7] D.A. Roberson, D. Espalin, R.B. Wicker, 3D printer selection:
A decision-making evaluation and ranking model. Virtual and
Physical Prototyping, 2015, 8:3, 201-212.
[8] W. M. Wang, C. Zanni, L. Kobbelt, Improved surface quality in
3D printing by optimizing the printing direction. Eurographics,
2016.

View publication stats