Application of Support Vector Machines in Predicting

Employee Turnover Based on Job Performance

Wei-Chiang Hong1, Ping-Feng Pai2,*, Yu-Ying Huang3, and Shun-Lin Yang3

1 School of Management, Da-Yeh University,

112 Shan-Jiau Rd., Da-Tusen, Chang-hua, 51505, Taiwan

d9230006@mail.dyu.edu.tw
2 Department of Information Management, National Chi Nan University,

1 University Rd. Puli, Nantou, 545, Taiwan

paipf@ncnu.edu.tw
3 Department of Industrial Engineering and Technology Management, Da-Yeh University,

112 Shan-Jiau Rd., Da-Tusen, Chang-hua, 51505, Taiwan

{r9315001, r9315018}@mail.dyu.edu.tw

Abstract. Accurate employee turnover prediction plays an important role in

providing early information for unanticipated turnover. A novel classification
technique, support vector machines (SVMs), has been successfully employed in
many fields to deal with classification problems. However, the application of
SVMs for employee voluntary turnover prediction has not been widely ex-
plored. Therefore, this investigation attempts to examine the feasibility of
SVMs in predicting employee turnover. Besides, two other tradition regression
models, Logistic and Probability models are used to compare the prediction ac-
curacy with the SVM model. Subsequently, a numerical example of employee
voluntary turnover data from a middle motor marketing enterprise in central
Taiwan is used to compare the performance of three models. Empirical results
reveal that the SVM model outperforms the logit and probit models in predict-
ing the employee turnover based on job performance. Consequently, the SVM
model is a promising alternative for predicting employee turnover in human re-
source management.

1 Introduction
In general, employee turnover can be divided into two types, namely involuntary turn-
over and voluntary turnover. Involuntary turnover is often defined as the movements
across the membership boundary among an organization, over which the employee only
conducts slight affections. On the other hand, voluntary turnover is defined as the
movements across the membership boundary among an organization, over which the
employee conducts heavy affections [1]. Mobley [2] first proposed the structure models
regarding employee turnover. Based on the developed voluntary turnover structure, job
satisfaction links to the initiate actual voluntary turnover indirectly [3]. In addition, the
organizational commitment is treated as intervening variables to explain the stay inten-
tions or employee turnover [4]. Recently, job performance has become one of the most
*
Corresponding author.

L. Wang, K. Chen, and Y.S. Ong (Eds.): ICNC 2005, LNCS 3610, pp. 668 – 674, 2005.
© Springer-Verlag Berlin Heidelberg 2005
 Application of Support Vector Machines in Predicting Employee Turnover 669

important factors which influence the decision of employee turnover. The relationships
between performance and turnover can be divided into four categorizations, a positive
relationship, a negative relationship, no relationship and non-linear relationship [5].
Schwab [6] indicated that high performance employees are more likely to leave. Jackof-
sky [7] showed that low job performance appears high voluntary turnover. Therefore,
the turnover displays a non-linear relationship to job performance. Trevor et al. [8]
found that high performance employees would be less likely to leave than lower per-
formance ones. Vecchio & Norris [9] concluded the correlations between turnover and
job performance are negative. Morrow et al. [1] proved that the negative relationship
between the turnover and job performance is statistical significant. Williams & Living-
stone [10] showed that the poor performance employees in the marketing department of
any organization tend to leave easily.
The voluntary turnover prediction problems can be treated as discrete choice prob-
lems. The logistic regression model presented by McFadden [11] is one of the most
popular discrete choice models in practical application [12]. Besides, logit models
have also been employed in commercial affairs forecasting [13]. However, independ-
ence of irrelevant alternatives (IIA) property limits the application of logit models
[14]. To relax IIA restriction of the logit model, multinomial probability regression
model, namely probit model, allows a free correlation structure among each discrete
choice alternative. Dow & Endersby [15] compared the performance of the multino-
mial logit model and the multinomial probit model in the voting analysis.
Recently, based on statistical learning algorithms, an emerging technique called
support vector machines (SVMs) [16] has been widely employed for pattern classifi-
cation and regression problems. However, the application of SVMs for employee
voluntary turnover prediction has not been widely explored.
In this investigation, SVM model, logit model and probit models are employed to
compare the prediction performance of employ turnover. The rest of this article is
organized as follows. Section 2 briefs three prediction models. Section 3 addresses a
numerical example taken from a motor marketing enterprise in Taiwan to compare
prediction results of three models. Finally, conclusions are made in section 4.

2 Prediction Models

2.1 Support Vector Machines in Classification

SVMs derive a class decision by determining the separate boundary with maximum
distance to the closest points, namely support vectors (SVs), of the training data set.
By minimizing structural risk rather than empirical risk, SVMs could efficiently avoid
a potential misclassification for testing data. Therefore, SVM classifier has superior
generalization performance over that of other conventional classifiers.
Given a training data set D = {xi , yi }iN=1 , where xi ∈ ℜ n is the i-th input vector with
known binary output label yi ∈ {−1,+1} . Then, the classification function is given by
yi = f ( xi ) = wT ϕ ( xi ) + b . (1)
where ϕ : ℜ → ℜ is the feature mapping the input space to a high dimensional fea-
n m

ture space. The data points become linearly separable by a hyperplane defined by the
670 W.-C. Hong et al.

pair ( w ∈ ℜ m , b ∈ ℜ ) [16]. The optimal hyperplane that separates the data is formu-
lated as Eq. (2).
2
Minimize Φ( w) = w 2
.
[ ]
(2)
Subject to yi w ϕ ( xi ) + b ≥ 1
T
i = 1,..., N
where w is the norm of a normal weights vector of hyperplane. This constrained
optimization problem is solved using the following primal Lagrangian form:
1
[ ]
N
L ( w, b , α ) = w − ∑ α i yi ( wT ϕ ( xi ) + b) − 1 .
2
(3)
2 i =1

where α i are the Lagrange multipliers. Applying the Karush-Kuhn-Tucker conditions,

the solutions of the dual Lagrangian problem , α i0 , then determine the parameters
w0 and b0 of the optimal hyperplane. Then, the decision function is given by Eq. (4):
( )
d ( xi ) = sgn w0T ϕ ( xi ) + b0 = sgn⎛⎜ ∑ α i0 yi K ( x, xi ) + b0 ⎞⎟,
N
i = 1,..., N . (4)
⎝ i =1 ⎠
Here, K(x,xi) is called the kernel function and should satisfies Mercer’s condition [16].
In addition, its value is equal to the inner product of two vectors x and xi in the feature
space ϕ ( x) and ϕ ( xi ) , i.e., K ( x, xi ) = ϕ ( x ) * ϕ ( xi ) . In this investigation, the Gaussian
radical basis function, exp⎛⎜ − xi − x j 2σ 2 ⎞⎟ , is used in the SVMs classifier model.
2

⎝ ⎠
To deal with overlapping classes, the concept of a soft margin is applied for the
SVM classifier. The width of the soft margin is controlled by a penalty parameter C
that determines the trade-off between maximizing the margin and minimizing the
training error. Small values of C result in insufficient stress on fitting the training
data. On the other hand, too large C leads to the over-fitting of the training data.
Therefore, the selection of two positive parameters, σ and C, of a SVM model is
important to the classification accuracy. The procedure for selecting two parameters is
conducted as follows. Step 1. Set a fixed value of the parameter C. Then, adjust the
value of σ till a maximum of prediction accuracy is achieved. The finalized σ value
is denoted as σ ′ .Step 2. The value of σ is set at σ ′ . Then, adjust the value of C to
achieve a maximum prediction accuracy. The finalized C is defined as C′ . Finally, the
suitable values of parameters σ and C are determined as σ ′ and C′ .

2.2 Logit and Probit Models

Logit and probit models are used to predict two discrete alternatives, for example, fail
or non-fail. Without the preliminary normality assumption of all explanatory variables
and with the capability of incorporating nonlinear factors, both logit and probit mod-
els are popular in the social science area [17,18]. The following is a brief of logit and
probit models.
Assume that the state Si for each observation appears absolute certainty in the dis-
crete choice models. Thus, the Si is equal to one when an alternative is selected. On
the other hand, the Si equals to zero if an alternative is not chosen. The decision vari-
able S is a dependent variable in logistic function, represented as Eqs. (5) and (6),
respectively,
P (Si = 1, non - fail) = exp( βxi ) (1 + exp( βxi )) = 1 (1 + exp(- βxi )) (5)
 Application of Support Vector Machines in Predicting Employee Turnover 671

P (S i = 0 , fail ) = 1 (1 + exp( βxi )) (6)

where xi is the explanatory variable for decision makers and βi is the coefficients of
the logit model.
Eqs. (5) and (6) are then estimated by maximum likelihood estimation (MLE). The
values of variables coefficients, βi , are represented as the log odds ratio and obtained
by estimated probability. Finally, the prediction values of decision variable Si is ei-
ther non-fail (Si=1) or fail (Si=0). Instead of using the logit function, a cumulative
standard normal distribution functional form is employed in probit model. Therefore,
when the alternative is selected, the Si is equal to one. On the contrary, the Si equals
to zero if the alternative is not chosen. The probit model can be expressed as Eq. (7) ,
P (Si ) = Φ(βxi ) (7)
where Φ(⋅) is the cumulative standard normal distribution function; xi is the explana-
tory variable, and βi is the coefficient of the probit model. The remaining procedure
for estimating βi is the same as that of logit model.

3 A Numerical Example

3.1 Data Set and the Measurement of the Prediction Performance

An empirical data regarding employee turnover and job-performance is given by Hui-

Lien Motor marketing Co. The Hui-Lien Motor marketing Co. is located in central
Taiwan with 300 million NTD annual business volumes and more than 200 marketing
specialists since 1992. The data contains totally 132 marketing specialists. The job-
performance was evaluated by motor marketing volumes and the voluntary turnover
statuses in 2003. For the measurement of prediction accuracy, some indices, such as
Cox & Snell R2 [17], Nagelkerke R2 [18], McFadden R2 [11], classification table, and
model Chi-square test [19] are often used. However, the core of this study is to predict
employees’ stay or leave in an organization by the job-performance. Therefore, the
measurement of prediction accuracy is the most important. The classification table is
employed to compare the total prediction accuracy of three models.

3.2 Experimental Results

The data set is divided into two parts, namely the modeling data set (from 1st em-
ployee to 100th employee) and the testing data set (from 101st employee to 132nd em-
ployee). The modeling data set is used to train models. The testing data set is applied
to estimating model performance for future unseen data. The Maximum likelihood
estimation procedure is employed to determine the free parameters for both logit and
probit models. Table 1 shows these coefficients βi for logit and probit models. For
both models, the explanatory capability is over 50% level. In addition, the two esti-
mated coefficients β0 and β1 are statistical significant.
For the SVM model, set the value of the parameter C at 1. Then, adjust the value
of σ till a maximum prediction accuracy is obtained, the finalized σ value equaled
to 0.001, therefore, set the σ value as 0.001. Secondly, adjust the value of C. The
672 W.-C. Hong et al.

finalized C value is 2.0 and the maximum prediction accuracy is achieved. Thus, the
finalized C value is 2.0. Finally, the predicting accuracy is 84.38% while the values of
σ and C are 0.001 and 2.0, respectively.

Table 1. Coefficients of explanatory variables for logit and probit models

Logit model Probit model

Explanatory Estimated Standard Explanatory Estimated Standard
p-value p-value
variables coefficients error variables coefficients error
Constant -0.206 0.041 0.000** Constant -0.112 0.020 0.000**
Job- Job-
2.343 0.437 0.000** 1.361 0.228 0.000**
performance performance
Model Chi-square p-value Model Chi-square p-value
0.000** 0.000**
(d.f.=1) (d.f.=1)
Cox & Snell R2 0.387 Cox & Snell R2 0.382
Nagelkerke R2 0.529 Nagelkerke R2 0.521
McFadden R2 0.372 McFadden R2 0.365

Table 2. The prediction accuracy in training stage for logit, probit and SVM models

Table 3. The prediction accuracy in testing stage for logit, probit and SVM models

*
: Si=1 implies actual turnover. **: Si=0 implies actual non-turnover.
***
: Pi=1 implies predicting as turnover. ****: Pi=0 implies predicting as non-turnover.

Table 2 shows the prediction accuracy of three models in training stages. In the
training stage, the total prediction accuracy are 74%, 72%, and 72% for logit model,
probit model and SVM model respectively. Table 3 lists the prediction performance
of three models in testing stages. It is indicated that the SVM model has higher total
prediction accuracy (84.38%) than logit model (71.9%) and probit model (71.9%).
Therefore, the SVM model has better generalization ability than the logit and probit
models in predicting the employee turn-over.
 Application of Support Vector Machines in Predicting Employee Turnover 673

4 Conclusions
The accurate employee turnover prediction plays an important role in early detection
of unanticipated turnover of an organization. Therefore, a suitable model for predict-
ing turnover is vital. In this investigation, the SVM classifier is used to examine the
feasibility in predicting the employee turnover. Two other discrete choice models,
namely logit model and probit model, are employed to compare the prediction accu-
racy. A numerical data set of employee turnover is used for the numerical experiment.
The simulation results reveal that SVM model outperform the logit model and probit
model. Therefore, the SVM model is a valid alternative in dealing with employee
turnover prediction problems. In the future, some other factors, such as job-
satisfaction, organization commitment, and abnormal absenteeism of employee can be
included in the SVM model to predict the employee turnover. In addition, developing
a structured way in determining free parameters of SVM model could be another
direction for future research.

References
1. Morrow, P.C., McElroy, J.C., Laczniak, K.S.: Using Absenteeism and Performance to Pre-
dict Employee Turnover: Early Detection Through Company Records. Journal of Voca-
tional Behavior. 55 (1999) 358-374.
2. Mobley, W.H.: Intermediate Linkages in The Relationship Between Job Satisfaction and
Employee Turnover. Journal of Applied Psychology. 63 (1977) 237-240.
3. van Breukelen, W., van der Vlist, R., Steensma, H.: Voluntary Employee Turnover: Combin-
ing Variables From the 'Traditional' Turnover Literature with The Theory of Planned Behav-
ior. Journal of Organizational Behavior. 25 (2004) 893-915.
4. Inverson, R.D.: Employee Intent to Stay: An Empirical Test of A Revision of The Price and
Muller model. The University of Iowa (unpublished doctoral dissertation), Iowa City, IA
(1992).
5. Jackofsky, E.F., Ferris, K.R., Breckenridge, B.G.: Evidence for A Curvilinear Relationship
Between Job Performance and Turnover. Journal of Management. 12 (1986) 105-111.
6. Schwab, D.: Contextual Variables in Employee Performance-Turnover Relationships.
Academy of Management Journal. 34 (1991) 966-975.
7. Jackofsky, E.: Turnover and Job Performance: An Integrated Process Model. Academy of
Management Review. 9 (1984) 74-83.
8. Trevor, C., Gerhart, B., Boundreau, J.: Voluntary Turnover and Job Performance: Curvi-
linearity and The Moderating Influences of Salary Growth and Promotions. Journal of ap-
plied Psychology. 82 (1997) 44-61.
9. Vecchio, R., Norris, W.: Voluntary Turnover and Job Performance, Satisfaction, and
Leader-Member Exchange. Journal of Business and Psychology. 11 (1996) 113-125.
10. Williams, C.R., Livingstone, L.P.: Another Look at The Relationship Between Perform-
ance and Voluntary Turnover. Academy of Management Journal. 37 (1994) 269-298.
11. McFadden, D.: Conditional Logit Analysis of Qualitative Choice Behavior. In: Zorem-
mbka, P. (eds.): Frontiers in Econometrics, Academic Press, New York (1973).
12. Tseng, F.M., Yu, C.Y.: Partitioned Fuzzy Integral Multinomial Logit Model for Taiwan’s
Internet Telephony Market. Omega. 33 (2005) 267-276.
674 W.-C. Hong et al.

13. Tseng, F.M., Lin, L.: A Quadratic Interval Logit Model for Forecasting Bankruptcy.
Omega. 33 (2005) 85-91.
14. Stopher, P.R., Meyburg, A.H., Brg, W.(ed.): New Horizons in Travel Behavior Research.
Lexington Books, Lexington, M.A. (1981).
15. Dow, J.K., Endersby, J.W.: Multinomial Probit and Multinomial Logit: A Comparison of
Choice Models for Voting Research. Electoral Studies. 23 (2004) 107-122.
16. Vapnik, V. (ed.): The Nature of Statistical Learning Theory. Springer-Verlag, New York
(1995).
17. Cox, D.R., Snell, E.J. (ed.): The Analysis of Binary Data. Champman and Hall, London (1989).
18. Nagelkerke, N.J.D.: A Note on A General Definition of The Coefficient of Determination.
Biometrica. 7 (1991) 691-692.
19. Greene, W.H. (ed.): Econometric Analysis. Prentice Hall, New Jersey (2003).