http://www.kualitatem.

com

Thinking ahead: Defining a testing strategy for a Practical Priorities in System

Testing

Software Engineering I Research Paper

Abstract
.
This paper is about the methods of System Testing, which will involves the design
and engineering of the system to provide a functional, performance, security, and
usability testing services. Also, this paper discusses how these tools are in the
software environment.

Table of Contents
1. Introduction 2
1.1 Information Goal 2
1.2 Purpose 3
2. Method of Approach 5
3. Results and Conclusions 5
3.1 Results 5
3.2 Conclusions 5
4. References and Bibliography 9
4.1 References 9
4.2 Bibliography 10

Introduction
System testing accounts for 30 to 50 percent of software development costs and the
percentage is increasing. Organizations are seeing the costs explode due to large
testing staffs, long schedules, and most damaging of all, missing opportunities.
The costs of not testing are also exploding. Users will not tolerate down time,
processing errors, bad performance, and difficult to use software. In spite of
these escalating costs, software is still routinely fielded with defects that
range in severity from mere annoyances to the user to total disasters. The
results often show up headlines and can cause embarrassment and ruin to companies
who have produced the faulty software. It seems that companies are spending a lot
on software testing without a clear return on their investment. This
contradiction is rooted in two clear facts about software testing.
• First, testing software enormously difficult and increasing complexity for
software designs will get more so.
• Seconding, testing is typically done without a clear methodology and without
the required automation support.
In addition many test organizations that have been using methodology for mainframe
development now find that it is obsolete for new software environment. [IDOL02]

o Information Goal
System testing faces new challenges with the changing technology used for
developing and delivering software. System help solve complex testing issue
related to interaction, concurrency, and time/fault tolerance by addressing the
functional, robustness, load, performance and regression testing phases from
small, single threads or tasks up to very large, distributed system [HELM02]. In
other words, system testing verifies that all elements mesh properly and that
overall system functions / performance is achieved.
Project management veteran Tom Mochal is director of internal development at a
software company in Atlanta and author of “System test: Can your program take a
lickin’ and keep on tickin”? says that there is no specific “system test”. Mr
Mocal says but there is a variety of test fall under the general system test
umbrella. Some of these tests can be difficult to set up and execute because
they require you to simulate the production environment. If a company has the
hardware and software that required simulating the live environment, it can be
difficult to stage the proper set of people, transaction, and circumstance you see
in production [MOCH01]. For instance, although a web application works fine and
meet the business requirements can it resist the attack of a hacker who wants to
read your database? That event will be hard to test for.

Companies that have made the press because of major system failures within the
system, typically are not brought on by code errors or logic problems. The
problems are caused by poor response times, weak security, lost data, screens
people cannot understand, and so forth. These are all aspects that the system may
address [PRESS97]. There are several types of system test and test strategy.
This paper will address all aspect of system testing and the use of test strategy
on a project to avoid processing errors, bad performance, and difficult to use
software.

o Purpose
The purpose is to addressed the aspects of system testing and to define testing
strategy (overall context for the entire test process) too avoid a major errors,
bad performance and difficult to use software application. System testing is a
collection of tests designed to verify that a program or system of programs is
ready for production. There is a need for system testing to determine whether the
application is at the proper level of industrial strength and ready for the prime-
time production environment. There are several types of system tests:
• Performance: Test the application with transaction volume that is within
and at top of the range expected from the business requirement. Make sure that
the resulting system behavior is within expectations or any formal service level
agreement.
• Stress: Expose the system to transaction volume substantially higher than
what would normally be expected and over a concentrated time period.
• Security: Ensure that people have the access level that is required and
deny access to the system to unauthorized people.
• Requirements: Track each business requirement through the development
process and make sure that it is include in the final system.
• Usability: Determine that people can use the system easily and without
frustration.
• Documentation: Check that hard copy and online documentation are
understandable and accurate.
• Training: Ensure that online or in person training is effective and meets
the training requirements.
• Interface: Test your application interfaces with external databases or
third-party companies.
• Disaster recovery: See whether you can recover the system from a simulated
disaster.
• Multiple locations: Verify your system can function between multiple
locations, if necessary.
Not all of types of system tests are necessary or related to an IT system. It is
important to know that all of the system tests start with assumption that the
system is complete and stable. It is difficult to do a system testing if the
person who is running into a large number of program bugs as well. [MOCH01].

Method of Approach
Of all the many aspects of System testing: performance testing, stress testing,
recovery testing, and security testing are four (4) areas that most developers
think about when they plan the system test. [MOCT02]
1. Performance testing. The development team must understand the level of
transaction volume the system needs to handle. For real-time and embedded
systems, functional requirements may be satisfied but performance problems make
the unacceptable. Performance testing checks the run-time performance
in the context of the integrated system. [SOBE97]. The areas that you look for
include:
• Online response time for screen navigation.
• Online response time for processing data against databases.
• The time needed to run batch processes.
• Delivery time for hard-copy reports (e.g., are they ready by 8:00 A.M.?).
• Printer capacity to handle requests.
• Downtime for batch processing or periodic maintenance,
• File sizes and storage capacity.

2. Stress testing. Stress testing is designed to test the software with

abnormal situations. Stress testing attempts to find the limits at which the
system will fail through abnormal quantity or frequency of inputs. [SOBE97] For
example:
• High rate of interrupts
• Data rates an order of magnitude above ‘normal’
• Test cases that require maximum memory or other resources.
• Test case that cause ‘thrashing’ in a virtual operating system.
• Test case that cause excessive ‘hunting’ for data on the disk systems.
Can also attempt sensitivity testing to determine if particular combination of
otherwise normal inputs can cause improper processing.
3. Recovery Testing. Many computer-based systems must recover from faults and
resume processing within a prespecified time. In some cases, a system must be
fault tolerant; that is, processing faults must not cause overall system function
to cease. In other case, a system failure must be corrected with a specific
period of time or severe economic damage will occur. [PRES97]
4. Security Testing. Systems with sensitive information or which have
potential to harm individuals can be a target for improper or illegal use. This
include:
• Attempted penetration of the system by ‘outside’ individuals for fun or
personal gain.
• Disgruntled or dishonest employee.
During security testing the tester plays a role of the individual trying to
penetrate the system. [SOBE97] Large range of methods:
• Attempt to acquire passwords through external clerical means
• Use custom software to attack the system
• Overwhelm the system with respects
• Cause the system errors and attempt to penetrate the system during recovery
• Browse through insecure data.

Results and Conclusion

o Result
Most developer think about the four (4) important areas that developers of System
testing, which include performance testing, stress testing, recovery testing, and
security testing for they plan the system test. [MOCT02] Most developer has been
on a
development project where the testing process was not considered until the
programming was already in progress. With this in mind, the developer end up
trying to plan the test process while they are in a testing process. The test
environments are probably not set up properly and it’s unclear whom responsible.
This situation can be avoided by thinking ahead and determining your testing
strategy before the coding process begins.

When focusing on a test process and its life cycle, it focused mainly on the basic
principles that needed to plan the test process early in the development life
cycle. The testing process starts in the analysis phase, with the formulation of
the Testing Strategy, which is later translated into the lower-level Test Plan for
a larger project. During the analysis phase, you gather and validate the business
requirements for the solution. It makes sense that the Test Strategy is completed
during this phase as well. In a sense, you are defining the overall testing
requirements. The purpose of the Testing Strategy is define the overall context
for the entire test process. The process is different depending on the specific
characteristic of your solution. The characteristic is the most important part of
the test process, since all future testing decisions will be made within the
context for the strategy. [MOCH01]. Here are the basic parts of the testing
strategy:
• Project Overview: You can copy this from the Project Definition.
• Business Risks: These are high-level risks of the project that will affect
the overall testing. The risk can be classified as high, medium, and low,
depending on the nature and impact of the problem. For instance, the risk of
doing business on the Internet may drive the need for rigorous system tests of
firewalls, technical architecture, and security.
• Test Milestones: This section gives the reader a preliminary overview of
the testing timelines.
• Test Approach: This describes the test process at a high level, including
how you will conduct uniting test, integration testing, system testing, and
acceptance testing. (If the project is large enough, each of these might be its
own section.) This is where you make fundamental decisions regarding the type of
testing that makes sense for your project:
1. If you are implementing a packaged solution, the approach may start in
system test, with vendor providing close support.
2. If you are doing iterative development cycles, the testing approach will
reflect thus overall development life cycle.
3. For system testing, define the major testing events, such as stress testing,
security testing, disaster recovery testing, usability testing, and response time
testing.
• Testing Environment: Think through the technologies and facilities needed
for the testing process. If the overall testing environment needs are understood
up front, it will be easier to break out the specific activities required putting
the environment in place. In addition, plan for and acquire some parts of the
environment well advance.
There may be other high-level section, depending on the project, such as testing
objectives, test assumptions, testing organization, and testing tools, along with
effort and cost estimates. [MOCH01]

o Conclusion

When thinking ahead by preparing a Testing Strategy remembers that it is a

foundation of successfully testing. When preparing a Testing Strategy it is
important to keep these three points in mind:
• Formulate an overall Testing Strategy during the analysis phase. An overall
approach to testing and describes how the test process will ensure that the
solution has the appropriate level of quality and reliability.
• Provides the overall guidelines from which all future testing decisions are
made. It allows the rest of the testing process to be defined more effectively.
• The Testing Strategy needs to be understood and approved by the sponsor. If
the strategy is accepted, there is a much better likelihood that the final
solution will meet the customer’s expectations.
Companies that have made the press because of major system failures within the
system typically are not brought on by code errors or logic problems. If a test
strategy were conducted properly problems that caused by poor response times, weak
security, lost data, screens people cannot understand, and so forth would have
been prevented. The results can be successful and not cause embarrassment and
ruin companies who produced the fault software application.
References and Bibliography

o References

[HELM02] Helm, James C., System Testing with Rational Test RealTime
Comprehensive Test Automation of Message-Based Systems. November 2002. Internet
URL: http://www.rational.com/products/testrt/sys_test.jsp?
[IDOL02] Idol, Chuck, Automated Testing Technology. November 2002. Internet
URL: http://www.longislandbuilders.com/ATT.htm
[PRES97] Pressman, Roger S., Software Engineering: A Practitioner Approach. 4th
ed., New York: McGraw-Hill Companies, Inc., 1997.
[MOCH02] Mochal, Tom, System testing: Can your program take a lickin’ and keep
on tickin’? September 18, 2001. Internet URL: http://www.builder.com
[MOCT02] Mochal, Tom, System testing to check security and validate system
requirements. September 25, 2001. Internet URL: http://www.builder.com
[SOBE97] Sobey, A. J., Software Engineering 1A/1B/1M. June 8, 1997. Internet
URL: http://www.louisa.level.unisa.edu.au/se1/testing-notes/test02_5.htm
[MOCH01] Mochal, Tom, Defining your testing strategy Aug. 25, 2001. Internet
URL: http://www.builder.com

o Bibliography

James C. Helm, System Testing with Rational Test RealTime Comprehensive Test
Automation of Message-Based Systems. November 2002. Internet URL:
http://www.rational.com/products/testrt/sys_test.jsp?
Chuck Idol Automated Testing Technology. November 2002. Internet URL:
http://www.longislandbuilders.com/ATT.htm
Roger S. Pressman, Ph.D., Software Engineering: A Practitioner Approach. 4th ed.,
New York: McGraw-Hill Companies, Inc., 1997.
Tom Mochal, System testing: Can your program take a lickin’ and keep on tickin’?
September 18, 2001. Internet URL: http://www.builder.com
Tom Mochal, System testing to check security and validate system requirements.
September 25, 2001. Internet URL: http://www.builder.com
Dr. A. J. Sobey, Software Engineering 1A/1B/1M. June 8, 1997. Internet URL:
http://www.louisa.level.unisa.edu.au/se1/testing-notes/test02_5.htm
Tom Mochal, Defining your testing strategy Aug. 25, 2001. Internet URL:
http://www.builder.com