Have a Free Demo with Us through Online

Quontra Solutions
Best Online IT Training Institute in United States
Topics
0 Automated software testing. What? Why?
0 Automated vs. Manual
0 Common types of automated testing methods
0 Automated software testing framework
0 Summary
2
What is automated software
testing?
A process which covers:
0 the use of software to control the execution of
tests
0 the comparison of actual outcomes to
predicted outcomes
0 the setting up of test preconditions
0 other test control and test reporting functions.


3
Why testing is critical?
4
Why automated testing?
0 A time savings translates directly into cost savings
0 Improves testing productivity
0 Improves accuracy
0 Increases test coverage
0 Does what manual testing cannot do


5
Effort and Cost
0 Lets assume 6 test cases
0 Effort required to run all 6 manually => 10 min
0 Effort required to write unit tests for all 6 cases => 10 min
0 Effort required to run unit tests for all 6 cases => < 1 min
0 Number of testing iterations => 5
0 Total manual testing time => 50 min
0 Total unit testing time => 10 min

6
Release Manual Test Auto Test
Manual Test
Cumulative
1 10 10 10
2 10 0 20
3 10 0 30
4 10 0 40
5 10 0 50
Automated vs. Manual testing

0 Not all tests can be automated and most
times is difficult to decide what to automate
and what to manually test.

7
Advantages
8
Automated Testing Manual Testing
Need to run a set of tests
repeatedly
Test cases have to be run a
small number of times
Helps performing "compatibility
testing" (on different
configurations and platforms)
Allows the tester to perform
more specific tests
Long term costs are reduced Short term testing costs are
reduced
Possible to run regressions on a
code that is continuously
changing and in shorter time
The more time tester spends
testing a module the grater
chance to find real bugs
Disadvantages & other factors
9
Automated Testing Manual Testing
It's more expensive to
automate (bigger initial
investments)
Manual tests can be very
time consuming
You cannot automate
everything, some tests still
have to be done manually
For every release you must
rerun the same set of tests
which can be tiresome
Other Factors
The performance of your test tools
The knowledge level of your testing team
The continuous growth of software to be tested
Common types of automated
testing methods
0 Monkey testing
0 Capture / playback
0 Code-based (Unit) testing
0 Intelligent test automation
10
Monkey testing
0 Randomly selecting inputs from a large range of
values and monitoring if exceptions are thrown.
0 For example, a monkey test can enter random strings
into text boxes to ensure handling of all possible user
input.
0 It applies not only for GUI or WEB testing, but also for
Unit testing.
11
Tools example
Monkey Android testing tools.
The simplest way to use the monkey is with
the following command, which will launch the
application and send 500 pseudo-random
events to it.

$ adb shell monkey -v -p your.package.name 500
12
Capture / playback
0 Its a set of software programs that capture user
inputs and stores it into a script for later replay.
+ Repeated testing can be performed quickly.
+ Does not require programming skills.
- When the GUI changes, input sequences
previously recorded may no longer be valid.
- Difficult to determine location of bugs.
13
Tools example
Autohotkey - Free keyboard macro program.
Supports hotkeys for keyboard, mouse.

14
Code-based (Unit) testing
0 Individual units of source code are tested to
determine if they fit for use.
0 Ideally, each test case is independent from the
others
+ Bugs can be found in early development stage
+ Easy to test boundary cases
- Not effective for the integrated system testing
15
Unit test-case generation method
0 Random

0 Genetic

0 Other
16
Random
0 Input values generated randomly
0 Run with generated inputs
0 User checks output values (pass/fail) and
saves them for later automatic checking
(Oracle).


17
Genetic
This methods approach is to combine JML (JAVA
model language) and genetic algorithms. JUnit is
used as a test execution platform.
18
Tools examples
Parasoft Jtest is an
automated Java testing
and static code analysis
product

19
TestGen is a collection of open-source tools
that automatically generates unit test cases.

Intelligent test automation
0 A model-based method for generating tests from a
description of an applications behavior (UML)
+ Models are simple to modify, so model-based tests
are economical to maintain over the life of a project.
- Most of these methods are researched only
theoretically.


20
OCL + UML
21
Inputs
generator
UML +
OCL
Tested software
Oracle
Report
Inputs Constraints
Constraints Outputs
pass / fail
Automated testing framework
0 A framework is an integrated system that sets
the rules of automation of a specific product.
0 This system integrates the function libraries,
test data sources, object details and various
reusable modules.
0 The framework provides the basis of test
automation and simplifies the automation
effort.
22
Common types of frameworks
0 Data-driven testing

0 Keyword-driven testing

0 Hybrid testing
23
Data-driven testing
0 Data-focused automation
0 User defines just data sets to run tests with
0 Data is defined in external data source and de-
coupled from script
0 Good fit for a single application testing, featuring
multi-environment, big datasets, and rarely
changing Test Cases
24
+ -
Good usability and re-
usability
Quality and coverage strongly depends on the
automation skills of implementing person
Good test flow coverage Continuous maintenance issues due to a big amount
of the code
Multi-environment support Manual validation is required
Example
25
Data set
Test script
Keyword-driven testing
0 Good fit for a multi-application testing featuring
multi-environment, big datasets, and a big number
of short and straight Test Cases
0 Users de-compose test cases into reusable action
keywords
0 Non-coders are able to author automated test
cases with action keywords
26
+ -
Test development doesnt require
programming skills
Due to keywords limit doesnt allow covering
very complicated Test Cases
Good test flow coverage Meta-language specific training is required for
staff
Compact code Manual validation is required
Example
27
Hybrid testing
0 Combines the best of data-driven and keyword-driven
testing:
0 User defines data sets to run tests with
0 User also defines flow control through action keywords
0 Data is separated from script and stored in external data
source with action keywords
28
+ -
Test Plan / Test Scenario creation doesnt require
programming or training on the Testing Tool
Requires initial investment for
design and implementation of
the framework
Compact code and scalable architecture
Built-in strict validation
Framework components
0 Test generation (data and script generators).Tools might create
specialized data such as randomized email messages, or populate
databases etc.
0 System configuration. Tools might preserve or reproduce system
parameters.
0 Test execution. Tools might operate the software itself, either simulating
a user working through the GUI and using an alternative testable
interface.
0 Oracles. An oracle is any mechanism by which we detect failure or
success. Tools might automatically detect certain kinds of error
conditions in a product.
0 Activity recording & coverage analysis. Tools might watch testing as it
happens and retrospectively report what was and was not tested.
0 Test management. Tools might record test results, organize test ideas or
metrics.
29
General framework structure
30
Common functional automated test tools
Tool + -
IBM /
Rational
Functional
Tester (RFT)
Built as Eclipse plug-in with full IDE,
Java support and source management
Supports Web-based applications,
.Net, Java, Siebel, SAP, PowerBuilder,
AJAX, Adobe Flex, Adobe PDF documents
Full GUI Object Map repository
License cost
HP / Mercury
Quick Test
Pro (QTP)
Supports Web 2.0, GWT, Dojo, YUI,
Silverlight, Flex, Ajax, JAVA, .NET
applications
Full GUI Object Map repository
Seamless integration with
QualityCenter
No IDE (changing in
release 10)
License cost
VisualBasic
scripting is limited
Selenium RC
& IDE
Good browser support
Good language support (JAVA, Ruby,
C#)
Can be easily extended as JUnit suite
Open-source (no license cost)
No GUI Object
repository
Only web-based
application support
Tool support is
limited
31
Tools examples
32
Summary
0 Automated testing improves productivity, accuracy,
increases test coverage, saves time and cost
0 Need to decide what to automate and what to
manually test
0 The framework provides the basis of test
automation and simplifies the automation effort.


33
More information
0 http://www.slideshare.net/lfingerman/test-automation-best-prcatices-
with-soa-test-approach
0 http://opuntia.cs.utep.edu/utjml/utest.html
0 http://www.slideshare.net/yogindernath/understanding-of-
automation-framework-presentation
0 http://www.ibm.com/developerworks/rational/library/591.html
0 http://www.satisfice.com/presentations/agileauto.pdf
0 http://automation-beyond.com/2009/05/30/test-automation-
practices-recordplayback/
0 http://en.wikipedia.org/wiki/Test_automation
0 http://www.slideshare.net/directi/automated-testing-vs-manual-
testing-391121


34
Thank you!
35
Question ???
0 What is the benefit of automated testing?
0 What are the common types of automated
testing methods?
0 What are the common types of frameworks?
0 What components come to framework?
0 What are the examples of automated testing
tools?
36