SOA grey box testing a constraint-based approach

Seema Jehan, Ingo Pill and Franz Wotawa

Institute for Software Technology
Graz University of Technology, Austria
Email: {sjehan,ipill,wotawa}@ist.tugraz.at
AbstractService-Oriented Architectures (SOAs) offer attrac-
tive advantages in respect of reusability, interoperability and
dynamics, and are nowadays widely accepted in industry. Achiev-
ing established software quality levels also with SOAs, while
mandatory, is challenging, as, for instance, a SOAs dynamics
and heterogeneity exacerbate verication issues like observability,
controllability, and distribution. Regarding verication, we thus
have to evolve available technology in order to enable the
assessment of essential functional and non-functional system
properties, including correctness, performance, stability, robust-
ness and scalability. Adopting a model-based grey box testing
approach that can exploit mixed description levels for individual
(possibly 3
rd
party) services promises the required exibility
for successful development workows. In this paper, we propose
such a testing approach that, considering a SOA model, denes
constraint satisfaction problems for the test case generation step.
First empirical results for our approach are promising.
I. INTRODUCTION
Service-Oriented Architectures, SOAs for short, offer at-
tractive advantages in respect of interoperability, reusability,
dynamics and compatibility. Their essentially loose (i.e. dy-
namic) client and server coupling principle leveraged new
R&D in challenging elds such as autonomic computing [1],
adaptive systems [2], self-healing distributed systems [3] and
cloud computing [4] that is considered an evolution of the
SOA idea. Unsurprisingly, SOAs thus have been gaining in
attention by software-related communities and are nowadays
widely accepted in industry [5].
A crucial factor for the attractiveness and success of any
software is its quality. Achieving the same, established quality
levels as for non-SOA software is a mandatory business
requirement, but a severe challenge in practice. In the context
of testing, SOAs exacerbated issues [6] like observability,
controllability and distribution [7], [8], [9] due to inherent
traits like heterogeneity and dynamics. Traditional white or
black box testing approaches also do not take full advantage
of all available data: For SOAs, service data are offered at
different levels for individual services, due to, for instance,
Intellectual Property encapsulation. While a service provider
might not disclose a services actual implementation, it has to
offer some abstract model depicting its interfaces as well as
pre and post conditions at least at that abstract level required
for system integration purposes. For our work, we thus adopt a
grey box strategy using a model-based testing [10] approach.
One advantage of developing a model-based approach is
that we can effectively accomodate varying description levels
for the individual services in an actual scenario. Adopting
an established approach for the SOA domain also shortens
the time needed to provide the essential technology for a
corresponding example test and diagnosis [11] workow like
the one we envisioned for SOAs in [12], so that our focus is
solely on scientic questions arising from the SOA domain.
Targeting grey box support, we show in this paper our
model-based testing approach for SOAs that derives and solves
a sequence of constraint satisfaction problems in order to de-
rive a sensible test suite. In particular, we focus on SOA testing
based on BPEL processes [13], where the underlying idea is
to extract paths from these processes and check their feasibil-
ity via underlying path constraints obtained from the BPEL
process elements, i.e., its activities. BPEL is an XML based
business process executable language, whose primary scope is
the orchestration of multiple heterogeneous web services for
the fulllment of a business process. BPEL processes can be
abstract or executable, where in our work, we are interested in
synchronous executable BPEL processes. BPEL activities can
be categorized into basic and structured ones. Our approach
handles most of the basic activities, including receive, reply,
assign and invoke. Among the structured activities, we support
if, while and sequence. The ow activity, which is used for
concurrent execution of process parts, will be handled in future
work. Here it is worth noting that when assuming that there
is no (hidden) interaction between concurrent sub-processes,
they can be easily handled in a sequential way.
This paper is structured as follows: We discuss the char-
acteristics of our work in the context of related research in
Section II, with Section III offering the specic details and
features of our approach. Following a report of empirical
results in Section IV, we conclude the paper in Section V
discussing also perspectives for future work.
II. RELATED WORK
Exploiting constraints for testing has already been consid-
ered in the literature. Gotlieb and colleagues [14] presented
an approach for extracting test cases from programs using a
constraint representation of source code. Our work is quite
close to Gotlieb et al.s seminal paper on constraint based
test generation. The papers differ, however, in the application
domain and in the extraction of constraints. In our case we
rely also on a services pre and post conditions, because, as
a matter of fact, the implementation of a service is hardly
available in the context of SOAs.
In the context of web service testing, researchers have
reported on the use of mainly three model based testing
techniques in literature, i.e., symbolic execution, petri nets,
2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops
978-0-7695-4993-4/13 $26.00 2013 IEEE
DOI 10.1109/ICSTW.2013.35
232
and model checking [15]. Yuan et al. [16] presented a graph
search based test case generation of BPEL processes that
makes use of matrix transformations of control ow graphs,
path coverage, and the classication of nodes in the graph
depending on incoming and outgoing edges. Yuan et al.s
approach is close to ours in two ways. First, they suggest to
transform a BPEL program to a control ow graph. Second,
they generate the test data using the Lp constraint solver [17],
which is later on combined with the test paths to generate the
abstract test cases. However, our approach differs from Yuans
work. Although we use the path coverage criterion for test path
generation, we further add pre and post condition contracts to
the test paths, in order to handle the test oracle problem. In
addition, we use the Minion constraint solver [18] to generate
test data instead of relying on Lp.
Another closely related work is from Yan et al. [19], which
relies on an extended Control Flow Graph (XCFG). The idea
behind their work is to extract all sequential paths from
the XCFG, and to combine them into concurrent test paths.
From the concurrent test paths they collect constraints using
backward substitution. In our approach, we transform each
sequential path directly into a set of constraints, which is
checked for satisability directly, using the Minion constraint
solver. If the constraints are not satisable, we discard the
corresponding path. Otherwise, the constraint solver returns
values for all variables used to execute the path, which we
directly register as test case in the test suite under construction.
For the execution of generated test cases, we use the work of
Mayer [20], where we convert the abstract test cases manually
to executable ones accepted by the BPELUnit tool [21]. This
tool supports both, simulated as well as real-life testing.
Another advantage of using this tool is that it supports many
BPEL engines like Active VOS [22], Oracle BPEL Process
Manager[23], and Apache ODE [24]. For simulated testing,
a BPEL process is not deployed, rather the intended engine
is called through a debug API. In real-life testing mode, a
business process is actually deployed onto the selected engine
and the partner web services are tested using mocks.
The survey [25] by Zakaria et al. gives a very good
comparison of different unit testing approaches applied to
BPEL processes. A key issue they pointed out is the lack of
an empirical evaluation. Surprisingly, only one out of 27 con-
sidered studies provides results on real-life BPEL processes.
Another widely discussed model based technique exploited
in the context of web service testing is model checking
[15]. The general idea behind its application is to convert
BPEL specications into a formal modeling language like
PROMELA [26] and test criteria into a formal property lan-
guage like LTL [27]. Specications and properties then serve
as input to, e.g. the SPIN model checker. The actual test cases
then are derived from the counter examples provided by the
model checker. Zhen et al. [28] applied the same idea to web
services and BPEL processes. There is a more enhanced work
by the same authors, in which Zhou et al.[29] also address the
state space explosion problem inherent with model checking.
Moreover, they also developed a tool for the generation of
JUnit test cases for automated test execution.
Model based testing techniques using Petri Nets have also
been explored extensively. Petri Nets are used for modeling
concurrent processes, and can be categorized into Plain Petri
Nets [30], Colored Petri Nets [31] and High-level Petri Nets.
Dong [32] developed a tool for test case generation of BPEL
processes using High-level Petri Nets. The basic approach is
to build a reachability graph from which test cases can be
extracted. The approach has a very high space complexity.
Contract based testing techniques are also relevant to our
work. This idea [33] has been applied in WSDL Web Service
Technology [34], where the authors argue that the contracts
applied at the model level are useful in the automated genera-
tion of test oracles, but can be very costly to implement. Also,
the assertions are easier to apply in OWL WS technology but
are difcult to implement in a WSDL based process model
[15]. Dai et al. [35] combine this approach with Petri Nets.
They specify contracts using a OWL-S model and transform
them into Petri Nets. The test cases are generated based on a
Petri Net behavioral analysis. In contrast, we combine contract
based testing with symbolic execution in our approach.
While our work is based on classical symbolic execution,
the usage of dynamic symbolic execution for test input gen-
eration has been explored extensively. Corresponding tools
include DART [36], Pex [37], CUTE and JCUTE [38]. DART
was the rst attempt for dynamic symbolic execution of
C programs. CUTE is an extension of DART for handling
concurrent programs, where its java version is named JCUTE.
Similarly Pex is a white box test generator for .NET [37].
The feasibility of execution paths is determined using the
constraint solver Z3 [39].
III. TESTING BPEL PROCESSES USING CONSTRAINTS
Web Services are considered the life blood of a Service Ori-
ented Architecture (SOA). The internal structure of services is
usually not available, which distinguishes SOAs from ordinary
software architectures. Therefore, these highly distributed,
autonomous web services can only be tested as a black box.
On the other hand, when considering service compositions
that are usually dened using BPEL processes, a white box
testing approach can be performed considering each service
as a black box again. However, because BPEL processes are
highly dynamic, i.e., actual partner services are not known
prior to workow execution time, there is still a need to tackle
the issue of trust for the involved partner web services. Hence,
BPEL process testing can be characterized as grey box testing.
In order to automate test case generation, we have to introduce
additional knowledge that captures the behavior of the black
boxes, i.e., the BPEL services. It is worth noting that this
augmentation knowledge has only to capture those parts of
the behavior that are relevant for testing.
For example, have a look at the Bank Loan example
[40] depicted in Figure 1. The business process starts upon
receiving a loan request from a client. Low risk clients, who
request loans up to 10.000 credits are issued an immedi-
ate approval. Requests from high risk clients or with loan
233
Fig. 1. BPEL Business Process from [40]
amounts starting at 10.000 credits are decided upon only after
thorough assessment. The BPEL services calculateRisk
and thoroughAssessment take information like the loan
amount and the client ID in order to compute an output. While
the actual implementation of these services is unknown, their
behavior has to be known to some extent. In this example,
we assume that calculateRisk returns low if the amount
is less than 1000 credits or the client has an excellent bank
record. Such knowledge can be easily formalized and added
to the process components using the concept of pre and post
conditions.
Our contribution is based on model based testing of BPEL
processes using symbolic execution. For our test case gener-
ation, we adopt a graph based search method that relies on a
combination of path extraction from control ow graphs and
constraint solving. Both, control ow graph and constraints,
can be extracted from BPEL source code automatically. We
aspire to address three issues in this paper: space complex-
ity of test case generation, the test oracle problem and an
empirical evaluation. We focus on the functional aspects of
BPEL executable processes. While this area has been explored
previously [25], there is still a lack of empirical studies.
A. Basic denitions
Before discussing the test case generation algorithm in more
detail, we briey introduce the underlying basic denitions.
We start with the denition of BPEL Flow Graphs, which
are used as an intermediate representation of BPEL processes.
As we consider only assign, receive, reply, invoke, while, if,
and sequence activities in a BPEL process, its corresponding
ow graph is very similar to the graphical representation of the
original process. However, in case of ow constraints, the ow
graph might be different, e.g., when representing concurrency
in form of sequential computation (if possible).
Denition 1 (BPEL Flow Graph): A BPEL Flow Graph G
is a tuple (V, E, s, F,
A
,
C
), where V are the vertices
representing the activities of a BPEL process, E are the edges
that correspond to the connections between BPEL activities,
s V is the start vertex, F V are the graphs end vertices
and
A
as well as
C
are functions that map vertices to activity
assignments and conditions respectively.
A path in a BPEL Flow Graph is a sequence of vertices,
leading from the start vertex to an end vertex. Formally, we
dene a path as follows:
Denition 2 (Path): Given a BPEL Flow Graph G, a se-
quence of vertices < v
1
, . . . , v
n
> is a path iff (1) for all
i {1, . . . , n} the vertex v
i
is an element of V , (2) v
1
is the
start vertex s, (3) v
n
F is an end vertex, and (4) for all
i {1, . . . , n1} we have that v
i
and v
i+1
are connected via
some edge e E .
In the following, we use paths to dene a test case. In
particular, we are interested in value assignments to variables
in the BPEL process that cause the execution of a specic path.
Note that it might be the case that such a value assignment
does not exist. Hence, we have to check whether a path can
be executed by setting the variables to certain values. For
checking purposes, we now dene path conditions comprising
the assignments and conditionals of the vertices in a path.
Denition 3 (Path condition): Given a path p =<
v
1
, . . . , v
n
> of a BPEL Flow Graph G, ps path condition
is a sequence of assignments and conditionals of its vertices,
obtained from <
A
(v
1
)
C
(v
1
), . . . ,
A
(v
n
)
C
(v
n
) >,
where variables are replaced by variables with indices in
order to ensure static single assignment form [41].
Using a static single assignment form (SSA) [41] ensures
that every variable is dened only once in a program. In
our case, we make use of indices for variables (i.e. temporal
variable instances clocked by assignment events). At the
beginning, every variable has index 0. If a variable is dened,
i.e., occurs at the right side of an assignment, its index is
increased by one. Every reference to this variable afterwards
is using this new index, unless a redenition of the variable
occurs again. In this case, a new index is computed and used
afterwards, and so on. Representing the path condition in SSA
form, we can easily check feasibility using any constraint
solver. For this purpose, the assignments have to be converted
to conditions, i.e., by replacing the assignment operator with
the sign of the equality operator and the conditions can be left
as they are.
Denition 4 (Feasible path): A path p of a BPEL Flow
Graph G is feasible if its path condition is satisable.
Using the previously stated denitions, we can easily dene
a test case.
Denition 5 (Test Case): A test case of a BPEL Flow
Graph G is a variable assignment that makes a path p feasible.
Having only one test case is usually not sufcient for testing
BPEL processes. Hence, we further dene test suites.
Denition 6 (Test Suite): A test suite TS for a BPEL Flow
Graph G is a set of test cases.
Computing a test suite for a BPEL process can be easily
done by computing feasible paths. We discuss this process in
more detail in the following subsection.
B. BPEL Test Case Generation
Figure 2 comprises our test case generation algorithm. The
algorithm takes the BPEL Flow Graph G and the maximum
path length MaxLen as inputs and computes feasible test cases.
Note that MaxLen has to be equal or larger than the length
of the smallest path in G from the start to an end vertex. The
algorithm is search based and traverses the ow graph using
a depth-rst search strategy for extracting paths. Afterwards,
the path conditions are computed and checked for consistency.
If the path condition is feasible, the variable assignments that
234
1: procedure TEST CASE GENERATION(G,MaxLen)
2: Compute all possible paths P up to length MaxLen.
3: for Each path p P do
4: Compute the path condition of a path p
5: Check this path condition for consistency (using a
constraint solver)
6: if the constraints are satised then
7: Save the solution, i.e., the variable assignments
coming from the constraint solver, as a feasible test case.
8: end if
9: end for
10: Return all stored solutions.
11: end procedure
Fig. 2. BPEL Test Case Generation algorithm
result from such a check are saved as test case. Finally, the
algorithm returns a test suite.
In the following, we discuss the different steps of the
algorithm and those activities that have to be carried out in
more detail. In particular, we focus on the representation of
the path conditions as constraint satisfaction problem (CSP).
Note that the approach is similar to symbolic execution, which
is very well known in testing [42]. Similar to symbolic exe-
cution, conditions are computed which belong to a particular
execution path. In our case, we convert each path condition
of a path into a constraint satisfaction problem. Unlike earlier
work, e.g., [43] , the conversion algorithm works as follows:
SSA conversion: The static single assignment (SSA) form
is an intermediate representation of a program such that
no two left-side variables share the same name. This
intermediate representation enables an easier conversion
into a CSP. The basic rules used for the conversion of a
BPEL path into its SSA representation are listed below:
We convert an Assign activity by adding an index to
a To variable each time the variable is dened, i.e.
declared as the TO variable. If a variable is redened,
the index is incremented so as to satisfy the SSA
property. The index of a From variable i.e. referenced
variable is equal to the last denition of the variable.
We convert Receive and Reply activities into assign-
ments.
Invoke is easily converted into assignments, where
the right hand side variable is the input variable
and the left side variable is the output variable.
We convert the structured activity If in two steps:
1) the condition is saved in an auxiliary variable. 2)
each assign or invoke activity is converted according
to the above rule. 3) The condition variable is set to
true in case of an if or else if branch, but to
false in case of an else branch.
The while structure is converted similarly to if, with
the exception that the condition is always set to true.
The loop is repeated up till the maximum length
specied by the tester.
The following example illustrates our conversion process.
For the Bank loan example described earlier (see Fig-
ure 1), the SSA form of that path with high loan amounts
is as follows:
1: input_0 > 0
2: loan_1 = input_0
3: loan_1 > 10000
4: AssessRiskPLRequest_1 = loan_1
5: output_1 = AssessRiskPLResponse_0
The statement in Line 1 is the pre-condition dened on
the loan request variable in the contract specication step.
Line 2 is simply an assignment activity in the BPEL pro-
cess, while the third line denotes the representation of the
if condition. The AssessRiskPLRequest is the BPEL
variable responsible for passing the loan amount to the
Assessment web service. The response from the partner
web service is received by the AssessRiskPLResponse
variable. The last statement shows the SSA representation
of the Reply BPEL activity.
Constraint conversion: The second step involves the
conversion of SSA statements into their corresponding
constraints. The representation of the conditions and
assignments resulting from the SSA conversion as con-
straints is simple and requires basically nothing else than
a direct mapping from variables to constraint variables
and from the conditions to assignments to their respective
representation. The concrete representation of constraints
depends on the used constraint solver. In our implemen-
tation we make use of MINION [18]. MINION is an
out of the box, open source constraint solver. Its syntax
requires a little more effort on modeling the constraints
than other constraint solvers, e.g., it does not support
different operators to be used within one constraint. In
order to illustrate the constraint conversion we show the
constraint representation for the path from the Bank Loan
example using MINION constraints.
ineq(0, input_0, -1)
eq(loan_1, input_0)
ineq(10000, loan_1, -1)
eq(AssessRiskPLRequest_1, loan_1)
eq(output_1, AssessRiskPLResponse_0)
The constraint ineq(x, y, k) ensures that x y+k,
and eq states that both variables used as parameters have
to have the same value.
IV. EMPIRICAL EVALUATION
For rst empirical results, we implemented the proposed
approach making use of the MINION constraint solver [18],
and BPELUnit [21] for unit test execution, where the idea is
to automatically generate tests using the proposed approach.
Furthermore, our implementation relies on the Eclipse BPEL
Designer
1
and the Apache ODE deployment engine [24]. We
considered three examples in our tests, namely LoanApproval
1
http://www.eclipse.org/bpel/
235
TABLE I
EXAMPLES
Prog BPEL Activites Used
Loan Receive, Reply, Assign, If, Else, Invoke, Sequence
Atm Receive, Reply, Assign, If, Else if, While, Invoke, Sequence
While Receive, Reply, Assign, While, Sequence
as described earlier, ATM
2
, and a simple hand-crafted example
comprising a while loop. The rst two examples were tailored
in order to remove ow and fault handling BPEL constructs,
which are currently not supported by our implementation.
Table I gives a brief description of activities used in these
examples.
In the current implementation, the test cases are transformed
manually to BPELUnit test cases. We are currently working on
a prototype where BPELUnit tests are automatically computed.
We carried out the experiments using a MacBook Pro 2.4 GHz
Intel Core i5, 4 GB 1333 MHz DDR3, under OSX 10.7.2
operating system. The obtained gures are depicted in Table II
. In this table we state the following results: the number of
BPEL activities in any process n, the number of paths p which
is in our case equal to the number of derived test cases, the
maximum path length maxLen varying from 10 to 50 (in case
of examples with while loops), the minimum path length of the
BPEL process minP, the maximum path length of the BPEL
Process maxP, the minimum number of Minion constraints
minC, the maximum number of Minion constraints maxC.
totalT represents the total time in milliseconds taken to run
an experiment. The time for constraint solving used to check
feasibility of paths was always very small ranging from 11 to
26 milliseconds and is therefore omitted in the table.
From the results we see that even for a vast amount of paths,
computation time never exceeds half an hour, which seems
to be acceptable from a practical point of view. However,
the underlying examples are rather small and improvements
regarding the selection of paths are necessary. One idea here
is to make use of mutation scores and heuristics in order to
generate a test suite. Note that the current approach ensures
activity and path coverage with respect to the pre-specied
maximum path length.
V. CONCLUSIONS
In this paper, we discussed the use of constraints for test
case generation in the context of SOA and in particular BPEL
processes. We proposed a test case generation approach that
focuses on functional testing and combines constraints that are
extracted from the BPEL process activities directly with pre
and post conditions for called services. The underlying idea of
the approach is to extract paths from the process and to check
feasibility using the corresponding path constraints. First em-
pirical results show that constraint solving can be effectively
used for SOA test case generation. However, computing all
paths might be infeasible for larger BPEL processes, and thus
2
http://docs.jboss.com/jbpm/bpel/v1.1/userguide/tutorial.atm.html
optional means for terminating the test case generation on the
y are necessary. In particular, using mutation score or source
code based metrics other than path and activity coverage might
help in this respect.
In the near future, we will improve our test case generation
using mutation scores and coverage. Moreover, in the context
of our envisioned test and diagnosis workow for SOAs [12],
we plan to combine testing with fault localization similar
to [44]. Here the question is how to adapt the concept of
distinguishing test cases, which heavily relies on mutations in
the SOA domain. Another research area of interest is the use
of constraints for verication purposes (see for example [45]).
Aside functional testing, non-functional testing is aimed at,
which is of specic interest for SOAs. Checking the fulllment
of service level agreements (SLAs) like the maximum time
for responses or checking the automated adaptation of the
underlying SOA structure in case of faults seems to be a
promising area. To the best of our knowledge, the question
about the potential of using constraints for this purpose has
not been sufciently answered.
ACKNOWLEDGMENT
We would like to thank Schahram Dustdar, Philipp Leitner,
and Stefan Schulte for fruitful discussions on the research
presented in this paper. This work is partially supported by the
Austrian Science Fund (FWF): P23313-N23, P22959-N23.
REFERENCES
[1] K. Verma and A. P. Sheth, Autonomic web processes, in International
Conference on Service-Oriented Computing (ICSOC), 2005, pp. 111.
[2] G. Denaro, M. Pezz` e, D. Tosi, and D. Schilling, Towards self-adaptive
service-oriented architectures, in Workshop on Testing, analysis, and
verication of web services and applications, 2006, pp. 1016.
[3] R. B. Halima, K. Drira, and M. Jmaiel, A qos-oriented recongurable
middleware for self-healing web services, in IEEE International Con-
ference on Web Services (ICWS), 2008, pp. 104111.
[4] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, Cloud
computing and emerging it platforms: Vision, hype, and reality for
delivering computing as the 5th utility, Future Generation Computer
Systems, vol. 25, no. 6, pp. 599616, 2009.
[5] M. P. Papazoglou, P. Traverso, S. Dustdar, and F. Leymann, Service-
oriented computing: a research roadmap, International Journal of
Cooperative Information Systems, vol. 17, no. 2, pp. 223255, 2008.
[6] G. Canfora and M. D. Penta, Testing services and service-centric
systems: Challenges and opportunities, IT Professional, vol. 8, no. 2,
pp. 1017, 2006.
[7] L. Cacciari and O. Raq, Controllability and observability in distributed
testing, Information & Software Technology, vol. 41, no. 11-12, pp.
767780, 1999.
[8] R. Hierons and H. Ural, Overcoming controllability problems with
fewest channels between testers, Computer Networks, vol. 53, no. 5,
pp. 680690, 2009.
[9] R. M. Hierons and H. Ural, The effect of the distributed test architecture
on the power of testing, The Computer Journal, vol. 51, no. 4, pp. 497
510, 2008.
[10] A. C. Dias Neto, R. Subramanyan, M. Vieira, and G. H. Travassos, A
survey on model-based testing approaches: a systematic review, in ACM
international workshop on Empirical assessment of software engineering
languages and technologies (WEASELtech), 2007, pp. 3136.
[11] R. Reiter, A theory of diagnosis from rst principles, Articial
Intelligence, vol. 32, no. 1, pp. 5795, 1987.
[12] F. Wotawa, M. Schulz, I. Pill, S. Jehan, P. Leitner, W. Hummer,
S. Schulte, P. Hoenisch, and S. Dustdar, Fifty shades of grey in SOA
testing, in 9th Workshop on Advances in Model Based Testing, 2013,
to appear.
236
TABLE II
EMPIRICAL RESULTS OBTAINED
Prog n maxLen p minP maxP minC maxC totalT
Loan 16 10 3 8 10 8 11 160
Atm 27 10 1 9 9 10 10 57
20 17 9 20 10 22 1,312
30 132 9 30 10 33 9,150
40 1,367 9 40 10 46 108,983
50 12,950 9 50 10 57 1,372,059
While 8 10 3 8 10 13 16 169
20 28 8 20 13 28 2,028
30 78 8 30 13 45 6,194
40 153 8 40 13 62 13,703
50 253 8 50 13 75 23,849
[13] Web services business process execution language 2.0, 2007. [Online].
Available: http://docs.oasis-open.org/wsbpel/2.0/wsbpel-v2.0.pdf
[14] A. Gotlieb, B. Botella, and M. Rueher, Automatic test data genera-
tion using constraint solving techniques, in Proceedings of the ACM
SIGSOFT International Symposium on Software Testing and Analysis
(ISSTA), Mar. 1998, pp. 5362.
[15] M. H. Mustafa Bozkurt and Y. Hassoun, Testing web services: A
survey, Department of Computer Science, Kings College London,
Tech. Rep. TR-10-01, January 2010.
[16] Z. L. Yuan Yuan and W. Sun, A graph-search based approach to
bpel4ws test generation, in Software Engineering Advances, Interna-
tional Conference on, oct. 2006, p. 14.
[17] M. Berkelaar, lp solve, a public domain mixed integer linear program
solver, available at http://groups.yahoo.com/group/lp solve/.
[18] I. P. Gent, C. Jefferson, and I. Miguel, Minion: A fast scalable constraint
solver, in In: Proceedings of ECAI 2006, 2006, pp. 98102.
[19] J. Yan, Z. Li, Y. Yuan, W. Sun, and J. Zhang, Bpel4ws unit testing: Test
case generation using a concurrent path analysis approach. in Software
Reliability Engineering, 2006. ISSRE 06. 17th International Symposium
on. IEEE Computer Society, 2006, pp. 7584.
[20] P. Mayer and D. L ubke, Towards a bpel unit testing framework, in
Proceedings of the 2006 workshop on Testing, analysis, and verication
of web services and applications, ser. TAV-WEB 06. New York, NY,
USA: ACM, 2006, pp. 3342.
[21] D. L ubke, Bpel unit, 2006. [Online]. Available:
http://bpelunit.github.com
[22] Active vos engine. [Online]. Available: http://www.activevos.com
[23] Oracle bpel process manager. [Online]. Available:
http://www.oracle.com/technetwork/middleware/bpel
[24] Apache ode. [Online]. Available: http://ode.apache.org/
[25] Z. Zakaria, R. Atan, A. A. A. Ghani, and N. F. M. Sani, Unit testing
approaches for bpel: A systematic review, in Proceedings of the 2009
16th Asia-Pacic Software Engineering Conference, ser. APSEC 09.
Washington, DC, USA: IEEE Computer Society, 2009, pp. 316322.
[Online]. Available: http://dx.doi.org/10.1109/APSEC.2009.72
[26] J. Garcia-fanjul, J. Tuya, and C. D. L. Riva, Gen-
erating test cases specications for bpel compositions
of web services using spin, 2006. [Online]. Available:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.60.9287
[27] A. Pnueli, The temporal logic of programs, in 18th Annual Symposium
on Foundations of Computer Science (FOCS), 1977, pp. 4657.
[28] Y. Zheng, J. Zhou, and P. Krause, A model checking based test case
generation framework forweb services, Information Technology: New
Generations, Third Int. Conference on, vol. 0, pp. 715722, 2007.
[29] J. Zhou, Y. Zheng, and P. Krause, An automatic test case generation
framework for web services, Journal of Software, vol. 2, no. 3, 2007.
[30] C. Ouyang, E. Verbeek, W. M. P. van der Aalst, S. Breutel, M. Dumas,
and A. H. M. ter Hofstede, Formal semantics and analysis of control
ow in ws-bpel, Sci. Comput. Program., vol. 67, no. 2-3, pp. 162198,
2007.
[31] Y. Yang, Q. Tan, and Y. Xiao, Verifying web services composition
based on hierarchical colored petri nets, in Proceedings of the rst
international workshop on Interoperability of heterogeneous information
systems, ser. IHIS 05. New York, NY, USA: ACM, 2005, pp. 4754.
[32] W. Dong, Test case generation method for bpel-based testing, in
Computational Intelligence and Natural Computing, 2009. CINC 09.
International Conference on, vol. 2, june 2009, pp. 467 470.
[33] B. Meyer, Applying design by contract, Computer, vol. 25, no. 10,
pp. 4051, Oct. 1992.
[34] R. Heckel and M. Lohmann, Towards contract-based testing of web
services, Electron. Notes Theor. Comput. Sci., vol. 116, pp. 145156,
Jan. 2005.
[35] G. Dai, X. Bai, Y. Wang, and F. Dai, Contract-based testing for web
services, in Computer Software and Applications Conference, 2007.
31st Annual International, vol. 1, july 2007, pp. 517 526.
[36] P. Godefroid, N. Klarlund, and K. Sen, Dart: Directed automated ran-
dom testing, in In Programming Language Design and Implementation
(PLDI, 2005.
[37] N. Tillmann and J. de Halleux, Pex - white box test generation for
.net, in Proceedings of Tests and Proofs (TAP08), ser. LNCS, vol.
4966. Prato, Italy: Springer Verlag, 2008, pp. 134153.
[38] K. Sen and G. Agha, Cute and jcute: Concolic unit testing and explicit
path model-checking tools, in In CAV. Springer, 2006, pp. 419423.
[39] L. De Moura and N. Bjrner, Z3: an efcient smt solver, in Proceed-
ings of TACAS08. Berlin, Heidelberg: Springer-Verlag, pp. 337340.
[Online]. Available: http://dl.acm.org/citation.cfm?id=1792734.1792766
[40] Eclipse bpel designer tutorial. [Online]. Available:
http://servicetechnologies.wordpress.com/exercises/
[41] M. M. Brandis and H. M ossenb ock, Single-pass generation of static
assignment form for structured languages, ACM TOPLAS, vol. 16(6),
pp. 16841698, 1994.
[42] J. C. King, Symbolic execution and program testing, Commun.
ACM, vol. 19, no. 7, pp. 385394, Jul. 1976. [Online]. Available:
http://doi.acm.org/10.1145/360248.360252
[43] N. Mihai, N. Simona, and W. Franz, Does testing help to reduce
the number of potentially faulty statements in debugging? in Testing
AI Practice and Research Techniques, ser. Lecture Notes in Computer
Science. Springer Berlin Heidelberg, 2010, vol. 6303, pp. 88103.
[44] F. Wotawa, M. Nica, and B. K. Aichernig, Generating distinguishing
tests using the MINION constraint solver, in Proc. of 2nd Workshop
on Constraints for Testing, Verication and Analysis. IEEE, 2010.
[45] H. Collavizza and M. Rueher, Exploring different constraint-based
modelings for program verication, in In Principles and Practice of
Constraint Programming (CP 2007), Providence, RI, USA, September
2007, pp. 4963.
237